diff --git a/latex/main.pdf b/latex/main.pdf index fa3289c8e5ac81791e77033ea35352ea4bd25600..38af26525bf70aafea30ab8a0de62d44fe98b56b 100644 --- a/latex/main.pdf +++ b/latex/main.pdf @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:8b2c75b96d21689e0a0f512735774dc857295beaad9a4ad3c65e147e3d1cf2e4 -size 349512 +oid sha256:24b8b698b150b0890698755e3f381d86663e3e6dbc6f90756ea2eea72bd15fec +size 352512 diff --git a/latex/main.tex b/latex/main.tex index c3d645d8189061be3e7cca291657ae222da61460..f445ab31c2013417081411900e9445fff33b5485 100644 --- a/latex/main.tex +++ b/latex/main.tex @@ -462,6 +462,24 @@ points. The selector failure remains: selected success is 24.15\%, below the deterministic visual statistics help proposal support, but they do not solve deployment-clean action selection. +\begin{table}[t] +\centering +\caption{Measured validation rollout for the deterministic RGB object-layout +chart token, across three train seeds, K=8. The proxy gate passes by mean +positive distance, but measured rollout does not improve over the RGB-stat +validation row.} +\label{tab:ctt-base-context-obj-val-rollout} +\scriptsize +\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obj_val_rollout_comparison/table}} +\end{table} + +Table~\ref{tab:ctt-base-context-obj-val-rollout} closes the loop on the +object-layout proxy diagnostic. Despite passing the proxy gate, the measured +proposal oracle is only 38.16\%, below the RGB-stat row's 40.58\%, while +\outcomeptr{}@8 remains 50.24\% and selected success drops to 20.29\%. This is +negative representation evidence: foreground component layout alone is not the +missing visual-language or object-centric causal chart token. + \begin{table}[t] \centering \caption{Measured residual \ctt{} rollout on 48 test positive-support charts diff --git a/outputs/hpc/logs/ctt_ms_rollout_15116890_0.err b/outputs/hpc/logs/ctt_ms_rollout_15116890_0.err new file mode 100644 index 0000000000000000000000000000000000000000..81c13e9da1fb46a21b7f2fdc3853d2d0809678a8 --- /dev/null +++ b/outputs/hpc/logs/ctt_ms_rollout_15116890_0.err @@ -0,0 +1,34 @@ +2026-07-03 13:15:47,083 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:47,982 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:48,876 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:49,844 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:50,749 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:51,684 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:52,562 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:53,365 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:54,180 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:54,995 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:55,941 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:56,886 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:57,810 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:58,659 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:59,522 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:00,467 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:01,406 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:02,253 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:03,090 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:03,949 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:04,883 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:05,825 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:06,666 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:07,528 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:08,381 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:09,304 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:10,202 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:11,030 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:11,854 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:12,689 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:13,584 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:14,503 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:15,398 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:16,280 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. diff --git a/outputs/hpc/logs/ctt_ms_rollout_15116890_0.out b/outputs/hpc/logs/ctt_ms_rollout_15116890_0.out new file mode 100644 index 0000000000000000000000000000000000000000..6d9800b9361bd86c057232a7758a3710f2d77798 --- /dev/null +++ b/outputs/hpc/logs/ctt_ms_rollout_15116890_0.out @@ -0,0 +1,9 @@ +{ + "out_dir": "runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics", + "num_rows": 69 +} +{ + "out_dir": "runs/ctt_residual_base_context_obj_rollout_val69_seed0", + "num_rows": 69, + "measured_candidates": "runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_candidates.json" +} diff --git a/outputs/hpc/logs/ctt_ms_rollout_15116891_0.err b/outputs/hpc/logs/ctt_ms_rollout_15116891_0.err new file mode 100644 index 0000000000000000000000000000000000000000..6448f55bf5d03c12086e5467f8fa0ad3bec608f0 --- /dev/null +++ b/outputs/hpc/logs/ctt_ms_rollout_15116891_0.err @@ -0,0 +1,34 @@ +2026-07-03 13:15:46,763 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:47,710 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:48,594 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:49,543 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:50,440 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:51,318 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:52,250 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:53,056 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:53,837 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:54,602 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:55,462 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:56,380 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:57,261 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:58,095 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:58,933 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:59,774 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:00,712 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:01,629 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:02,468 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:03,298 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:04,134 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:05,060 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:05,980 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:06,828 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:07,660 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:08,490 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:09,398 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:10,306 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:11,118 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:11,929 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:12,759 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:13,664 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:14,561 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:15,376 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. diff --git a/outputs/hpc/logs/ctt_ms_rollout_15116891_0.out b/outputs/hpc/logs/ctt_ms_rollout_15116891_0.out new file mode 100644 index 0000000000000000000000000000000000000000..469df4baf07120249cc30cfb36e284f4c7d67c87 --- /dev/null +++ b/outputs/hpc/logs/ctt_ms_rollout_15116891_0.out @@ -0,0 +1,9 @@ +{ + "out_dir": "runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics", + "num_rows": 69 +} +{ + "out_dir": "runs/ctt_residual_base_context_obj_rollout_val69_seed1", + "num_rows": 69, + "measured_candidates": "runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_candidates.json" +} diff --git a/outputs/hpc/logs/ctt_ms_rollout_15116892_0.err b/outputs/hpc/logs/ctt_ms_rollout_15116892_0.err new file mode 100644 index 0000000000000000000000000000000000000000..37529c9f071b1d5d943d657d86d1fe2ca6dd5f28 --- /dev/null +++ b/outputs/hpc/logs/ctt_ms_rollout_15116892_0.err @@ -0,0 +1,34 @@ +2026-07-03 13:15:45,526 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:46,301 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:47,025 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:47,741 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:48,499 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:49,219 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:49,933 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:50,650 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:51,406 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:52,158 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:52,909 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:53,693 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:54,405 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:55,155 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:55,863 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:56,631 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:57,429 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:58,144 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:58,860 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:15:59,603 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:00,346 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:01,087 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:01,825 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:02,573 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:03,328 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:04,071 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:04,807 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:05,551 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:06,288 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:07,024 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:07,739 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:08,448 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:09,164 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. +2026-07-03 13:16:09,898 - mani_skill - WARNING - No initial pose set for actor builder of goal_region, setting to default pose q=[1,0,0,0], p=[0,0,0]. Not setting reasonable initial poses may slow down simulation, see https://github.com/haosulab/ManiSkill/issues/421. diff --git a/outputs/hpc/logs/ctt_ms_rollout_15116892_0.out b/outputs/hpc/logs/ctt_ms_rollout_15116892_0.out new file mode 100644 index 0000000000000000000000000000000000000000..3b1e3487e32a1a0b1982f6cc970ed115d45bcaca --- /dev/null +++ b/outputs/hpc/logs/ctt_ms_rollout_15116892_0.out @@ -0,0 +1,9 @@ +{ + "out_dir": "runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics", + "num_rows": 69 +} +{ + "out_dir": "runs/ctt_residual_base_context_obj_rollout_val69_seed2", + "num_rows": 69, + "measured_candidates": "runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_candidates.json" +} diff --git a/reports/2026-07-03.md b/reports/2026-07-03.md index 4725ad11348e675943ab90ea6be93b59f7bd821a..601c65e2bf23c06a410fe3e1c7cfbf4b294414c5 100644 --- a/reports/2026-07-03.md +++ b/reports/2026-07-03.md @@ -96,7 +96,7 @@ - `15116890`: `base_context_obj` seed 0, validation 69 positive-support charts, K=8 - `15116891`: `base_context_obj` seed 1, validation 69 positive-support charts, K=8 - `15116892`: `base_context_obj` seed 2, validation 69 positive-support charts, K=8 - - current scheduler state: pending on priority. + - final scheduler state: all three jobs completed with exit code `0:0`. - purpose: object-layout proxy passed the support/safety gate by mean positive distance, so the next evidence must be measured OutcomePTR rather than another proxy claim. ## Jobs Completed @@ -230,6 +230,13 @@ - `runs/ctt_residual_base_context_obj_seed{0,1,2}` and `_val_proxy`: PPTC@0.20 `0.2271`, PPTC@0.40 `0.6425`, NegativeNear@0.20 `0.0380`, mean positive distance `0.4340`, diversity `0.2417`, proxy gate `pass`. - `runs/ctt_residual_base_context_obs_obj_seed{0,1,2}` and `_val_proxy`: PPTC@0.20 `0.2077`, PPTC@0.40 `0.6425`, NegativeNear@0.20 `0.0285`, mean positive distance `0.4429`, diversity `0.2448`, proxy gate `pass`. - interpretation: object-layout-only slightly improves mean positive distance over the RGB-stat row (`0.4340` vs `0.4347`) while staying inside the NegativeNear safety slack, but it does not beat RGB-stat on PPTC. This is enough to submit measured validation rollout, not enough for an OutcomePTR claim. +- Measured object-layout CTT validation rollout jobs `15116890`, `15116891`, and `15116892` completed with exit code `0:0`. + - seed artifacts: `runs/ctt_residual_base_context_obj_rollout_val69_seed{0,1,2}` + - aggregate artifact: `runs/ctt_base_context_obj_val_rollout_comparison` + - rows: 207 validation measured rows = 69 charts x 3 train seeds, K=8 + - selected success `0.2029`, base success `0.2754`, proposal oracle `0.3816`, hidden oracle `0.6667` + - OutcomePTR@8 `0.5024`, success support gap `0.2947`, success selector gap `0.1787` + - interpretation: proxy gate pass did not translate into better measured support. Object-layout-only is worse than RGB-stat on proposal oracle (`0.3816` vs `0.4058`) and selected success (`0.2029` vs `0.2415`), so it is a negative representation diagnostic. - Base-context-observation CTT proxy diagnostics completed for three seeds: - train artifacts: `runs/ctt_residual_base_context_obs_seed{0,1,2}` - proxy artifacts: `runs/ctt_residual_base_context_obs_seed{0,1,2}_val_proxy` @@ -353,6 +360,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- | | validation | 3 | 207 | 0.2850 | 0.2415 | 0.3768 | 0.6667 | 0.4589 | 0.2947 | 0.1353 | fail: selected/oracle/support/selector | | validation + base-context-observation | 3 | 207 | 0.2754 | 0.2415 | 0.4058 | 0.6667 | 0.5024 | 0.2754 | 0.1643 | fail: selected/oracle/support/selector | +| validation + base-context-object-layout | 3 | 207 | 0.2754 | 0.2029 | 0.3816 | 0.6667 | 0.5024 | 0.2947 | 0.1787 | fail: selected/oracle/support/selector | | test | 3 | 144 | 0.2847 | 0.2222 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2847 | fail: selected/support/selector | | test + base-context-observation | 3 | 144 | 0.2917 | 0.2708 | 0.5139 | 0.7292 | 0.5347 | 0.2639 | 0.2431 | fail: selected/support/selector | | test + dominance auto | 3 | 144 | 0.2847 | 0.2500 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2917 | fail: selected/support/selector | @@ -369,7 +377,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s | test + learned dominance | 3 | 144 | 0.2847 | 0.3056 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2569 | fail: selected/support/selector | | test + base-context-observation learned context dominance | 3 | 144 | 0.2917 | 0.3264 | 0.5139 | 0.7292 | 0.5347 | 0.2639 | 0.2431 | fail: selected/support/selector | -Interpretation: the test proposal oracle clears the internal 50% support target, and visual-stat CTT plus validation-calibrated learned context dominance gives the best current held-out selected success (`0.3264`). The cleaner train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. All rows remain far below the selected-success gate and leave large selector gaps. Standalone train-only utility-energy checkpoints do not transfer well enough across three seeds. The next paper-critical work is richer visual/object-centric chart representation plus a stronger train-only utility/dominance model, not another proxy-only generator result. +Interpretation: the test proposal oracle clears the internal 50% support target, and visual-stat CTT plus validation-calibrated learned context dominance gives the best current held-out selected success (`0.3264`). The cleaner train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. The object-layout proxy pass does not survive measured validation rollout and is worse than the RGB-stat row on proposal oracle/selected success. All rows remain far below the selected-success gate and leave large selector gaps. Standalone train-only utility-energy checkpoints do not transfer well enough across three seeds. The next paper-critical work is richer learned visual-language/object-centric chart representation plus a stronger train-only utility/dominance model, not another deterministic proxy descriptor. ## Leakage Audit Status @@ -390,7 +398,7 @@ Interpretation: the test proposal oracle clears the internal 50% support target, - Split hash remains `aaa857b92cbe5d455009f3ca3380e8b124908aa6c4c99fc1da408f738ed2dcf6`. - `runs/chart_feature_audit` documents the original `data/cil_charts` inputs: instruction and scene ids exist, but observation embeddings/raw refs do not. - `runs/chart_feature_audit_rgb_refs` documents the new non-destructive RGB-ref export: all splits have observation refs and deterministic 32D RGB-stat embeddings. This supports visual-stat diagnostics only; it does not yet support a learned visual-language/object-centric claim. -- `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. This is a leakage-clean representation diagnostic; full proxy rows are pending and should not be promoted into main results until `15116833`/`15116834` complete. +- `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. The full proxy rows and measured validation rollout are complete; the measured result is negative relative to RGB-stat support. ## Paper Changes @@ -409,7 +417,8 @@ Interpretation: the test proposal oracle clears the internal 50% support target, - Added RGB-ref/embedding audit evidence, the three-seed `base_context_obs` proxy row, measured validation/test rollout aggregates, and visual-stat dominance selector diagnostics. The draft describes this as a deterministic visual-stat support plus selector diagnostic that improves held-out selected success, but still not object-centric, learned visual-language, or enough for selected-action deployment evidence. - Added `cil/chart_features.py`, `scripts/audit_chart_feature_sources.py`, `scripts/export_chart_observation_embeddings.py`, and `scripts/slurm/train_ctt_feature_proxy.sbatch` to the reproducibility artifact list. - Added `scripts/export_chart_object_embeddings.py` to the reproducibility artifact list as a deterministic object-layout chart-token exporter. -- Updated the validation proxy gate discussion to account for object-layout rows now present in `runs/ctt_val_proxy_comparison/table`: RGB-stat remains best on PPTC@0.20, object-layout-only is slightly best on mean positive distance, and both remain proxy-only until rollout jobs `15116890`--`15116892` finish. +- Updated the validation proxy gate discussion to account for object-layout rows now present in `runs/ctt_val_proxy_comparison/table`: RGB-stat remains best on PPTC@0.20, object-layout-only is slightly best on mean positive distance, and measured rollout shows the object-layout row is negative relative to RGB-stat support. +- Added the measured object-layout validation rollout table via `\input{../runs/ctt_base_context_obj_val_rollout_comparison/table}` and described it as a negative representation diagnostic. - Reframed the title/abstract around `Counterfactual Action Atlas`; CTT is now the current transport generator inside the Atlas, not the whole novelty. - Tightened the abstract to avoid overclaiming a successful learned generator: the paper now says the Atlas measures local do-action geometry and audits whether deployment-clean generators reach positive support. - Added the train-calibration leakage rule to the split contract: when target charts are from train, retrieval excludes same chart id and same restored-state hash. @@ -423,8 +432,8 @@ Interpretation: the test proposal oracle clears the internal 50% support target, ## Next Actions -1. Monitor Slurm arrays `15116833` and `15116834`; aggregate proxy comparison and summary as soon as object-layout feature-mode runs finish. -2. If object-layout proxy passes the local-atlas gate, submit measured validation rollouts before making any OutcomePTR claim. -3. If object-layout proxy fails, diagnose whether the foreground layout token hurts source retrieval or transport collapse, and keep it as a negative representation diagnostic. -4. Replace the current weak train-only utility/dominance model with richer visual-language/object-centric chart tokens; the best clean train-calibrated selector reaches only `0.3125` selected success. -5. Add unsafe execution rate and calibrated fallback rate to measured rollout artifacts. +1. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat. +2. Replace the deterministic RGB/object descriptors with learned visual-language or task/object tokens that can distinguish target, distractor, gripper/contact region, and task stage. +3. Train utility/dominance on richer outcome-vector features; the best clean train-calibrated selector still reaches only `0.3125` selected success. +4. Add unsafe execution rate and calibrated fallback rate to measured rollout artifacts. +5. Rerun validation/test measured rollouts only after a representation or utility model change, not another shallow selector/proxy-only change. diff --git a/reports/draft_audit_ctt.md b/reports/draft_audit_ctt.md index a9e83d421c72114c71854439f9fbfc9d0d49a45d..7284ac42bd2f1838cb1c9bc9719da020e183ad91 100644 --- a/reports/draft_audit_ctt.md +++ b/reports/draft_audit_ctt.md @@ -116,7 +116,12 @@ positive distance over the RGB-stat row (`0.4340` vs `0.4347`) while keeping NegativeNear@0.20 within the safety slack, but it does not beat RGB-stat on PPTC. Therefore measured validation rollout jobs `15116890`--`15116892` were submitted for `base_context_obj`; the main paper should add OutcomePTR rows -only after those jobs finish. +only after those jobs finish. Those rollout jobs now finish as +`runs/ctt_base_context_obj_val_rollout_comparison`: selected success = 0.2029, +proposal-oracle success = 0.3816, and OutcomePTR@8 = 0.5024. This is negative +relative to the RGB-stat validation row (selected 0.2415, proposal oracle +0.4058), so deterministic foreground object-layout should remain a negative +representation diagnostic, not a promoted method improvement. Current artifact evidence: @@ -159,8 +164,9 @@ Current artifact evidence: `runs/chart_feature_audit_rgb_refs_object` contain the new deterministic RGB object-layout chart-token export and audit; Slurm arrays `15116833` and `15116834` contain the completed full proxy jobs for the corresponding CTT - feature modes, and `15116890`--`15116892` are the submitted measured - validation rollout jobs for the best proxy row. + feature modes; `runs/ctt_base_context_obj_val_rollout_comparison` contains + the completed measured validation rollout for the best proxy row, which is + negative relative to RGB-stat support. - `runs/ctt_learned_dominance_val_to_test` contains the first learned validation-calibrated dominance selector diagnostic. - `runs/ctt_learned_dominance_{success,success_weighted,margin_ext,tangent}_val_to_test` diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/combined_measured_candidates.json b/runs/ctt_base_context_obj_val_rollout_comparison/combined_measured_candidates.json new file mode 100644 index 0000000000000000000000000000000000000000..3d0efb18bc0a76eb8a1eb68031e90df0d24ba7cb --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/combined_measured_candidates.json @@ -0,0 +1,133484 @@ +{ + "data_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "k": 8, + "num_rows": 207, + "report_type": "ctt_measured_rollout_comparison", + "rows": [ + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7270061373710632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7270061373710632 + }, + "base_progress": 0.7270061373710632, + "base_success": false, + "base_utility": 0.7270061373710632, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7229657769203186, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7229657769203186 + }, + { + "contact_quality": null, + "progress": 0.7291424870491028, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7291424870491028 + }, + { + "contact_quality": null, + "progress": 0.7071980834007263, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7071980834007263 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7273857593536377, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7273857593536377 + }, + { + "contact_quality": null, + "progress": 0.7259960770606995, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7259960770606995 + }, + { + "contact_quality": null, + "progress": 0.727085530757904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.727085530757904 + }, + { + "contact_quality": null, + "progress": 0.7209001183509827, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7209001183509827 + } + ], + "candidate_progress": [ + 0.7229657769203186, + 0.7291424870491028, + 0.7071980834007263, + 1.0, + 0.7273857593536377, + 0.7259960770606995, + 0.727085530757904, + 0.7209001183509827 + ], + "candidate_source_chart_ids": [ + "ms-efbde8652e75274fbd69", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-df961fe24a43e56d1716", + "ms-9ced92cccacfe4d157b0", + "ms-efbde8652e75274fbd69", + "ms-dd32ce9859c365c535b4", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-484ea653bbca2de3a4a2", + "generated_tangents": [ + [ + -0.07231272757053375, + -0.05361426621675491, + 0.20084986090660095, + 0.2549743354320526, + 1.2756636142730713, + 0.26493221521377563, + -0.2770671248435974, + 0.1440885066986084, + -0.12119890749454498, + 0.18552835285663605, + 0.1064738929271698, + 0.9770814776420593, + 0.11324295401573181, + 0.2581104040145874, + 0.37570127844810486, + 0.14162081480026245, + 0.015085473656654358, + -0.25780385732650757, + 1.0848740339279175, + -0.438931405544281, + -0.16297724843025208 + ], + [ + -0.058384791016578674, + -0.007196493446826935, + 0.2863960862159729, + 0.4616982042789459, + 0.8279897570610046, + 0.026855826377868652, + -0.2453579306602478, + 0.108298659324646, + -0.010689212009310722, + 0.24004188179969788, + -0.11884095519781113, + 1.309671401977539, + 0.21858593821525574, + 0.2140667587518692, + 0.10542087256908417, + 0.04384743049740791, + -0.2249954491853714, + 0.033864542841911316, + 1.0120162963867188, + -0.18448886275291443, + -0.001940220594406128 + ], + [ + 0.0398905873298645, + 0.03704178333282471, + 0.46594569087028503, + 0.5894558429718018, + 0.9248438477516174, + 0.15800584852695465, + -0.2163742184638977, + -0.031178757548332214, + -0.059490740299224854, + 0.3896148204803467, + 0.03863520175218582, + 1.3416554927825928, + 0.23786357045173645, + 0.2686057686805725, + 0.308365136384964, + 0.09169572591781616, + -0.10002933442592621, + -0.12801562249660492, + 1.0708489418029785, + -0.11242121458053589, + -0.06980758905410767 + ], + [ + 0.04414123296737671, + 0.05196486413478851, + 0.31238871812820435, + 0.5328384637832642, + 1.097949504852295, + -0.24277085065841675, + -0.4461241364479065, + 0.03629066050052643, + 0.051987677812576294, + 0.116251140832901, + -0.11456320434808731, + 1.1187355518341064, + 0.14071431756019592, + -0.06319041550159454, + 0.18655209243297577, + 0.11288036406040192, + -0.1382099837064743, + -0.07332096993923187, + 1.2357680797576904, + -0.11672963201999664, + -0.23744389414787292 + ], + [ + -0.04112638533115387, + 0.045152448117733, + 0.25177115201950073, + 0.41037455201148987, + 0.9143605828285217, + 0.062146469950675964, + -0.29822736978530884, + 0.003190666437149048, + -0.017644867300987244, + 0.17557278275489807, + 0.030187007039785385, + 1.3407450914382935, + 0.12058094143867493, + 0.18465670943260193, + 0.14753586053848267, + 0.02446366474032402, + -0.15492106974124908, + 0.05071836709976196, + 1.0075147151947021, + -0.0926724225282669, + -0.005339428782463074 + ], + [ + -0.07725875079631805, + -0.03352033719420433, + 0.3147411346435547, + 0.3762994110584259, + 0.9385537505149841, + 0.1675524264574051, + -0.6023285984992981, + -0.0003843158483505249, + -0.04391191899776459, + 0.2910858988761902, + 0.17489781975746155, + 1.2902905941009521, + 0.03016170859336853, + 0.02391386777162552, + 0.19830992817878723, + 0.04218254238367081, + -0.17453138530254364, + 0.14636392891407013, + 1.2725138664245605, + -0.411970317363739, + 0.10102862864732742 + ], + [ + 0.0015261024236679077, + 0.04789748787879944, + 0.2689938545227051, + 0.38422873616218567, + 0.9711422324180603, + 0.04219160974025726, + -0.2555607557296753, + -0.0010332614183425903, + -0.018974101170897484, + 0.21855208277702332, + 0.002623647451400757, + 1.3563590049743652, + 0.016860365867614746, + 0.10275750607252121, + 0.2021571546792984, + 0.05061839148402214, + -0.09986038506031036, + 0.09534766525030136, + 1.0070284605026245, + -0.07725197076797485, + -0.054655373096466064 + ], + [ + -0.19963373243808746, + 0.025442376732826233, + 0.1647450029850006, + 0.462211936712265, + 0.8526129126548767, + -0.09105783700942993, + -0.21945291757583618, + 0.11289386451244354, + -0.03213582932949066, + 0.1903569996356964, + -0.06764494627714157, + 1.3399022817611694, + -0.006895154714584351, + 0.030383944511413574, + 0.21221129596233368, + 0.08718833327293396, + -0.1539117842912674, + 0.04677437245845795, + 0.9803423881530762, + -0.09331116080284119, + 0.07617859542369843 + ] + ], + "generated_utilities": [ + 0.7229657769203186, + 0.7291424870491028, + 0.7071980834007263, + 2.0, + 0.7273857593536377, + 0.7259960770606995, + 0.727085530757904, + 0.7209001183509827 + ], + "group_id": "ms-484ea653bbca2de3a4a2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7242962718009949, + 0.7209752798080444, + 2.0, + 0.19684189558029175, + 0.21780464053153992, + 0.19326990842819214, + 0.22509369254112244, + 0.20856516063213348, + 0.21041584014892578, + 0.19898515939712524, + 0.20721960067749023, + 0.20830264687538147, + 0.19554394483566284, + 0.22549311816692352, + 0.19481703639030457, + 0.2172507494688034 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.4380870461463928, + 0.4186934530735016, + 0.6294705867767334, + -0.08003801107406616, + 0.5882754325866699, + 0.006411761045455933, + 0.0, + 0.27439412474632263, + -0.43702489137649536, + -0.26345914602279663, + -0.3102925717830658, + 1.1419475078582764, + -0.05133828520774841, + 0.0, + 0.11964668333530426, + -0.04283810034394264, + 0.021698474884033203, + -0.3213534653186798, + 1.444929838180542, + 0.058529436588287354, + 0.0 + ], + [ + 0.18277202546596527, + -0.31313952803611755, + 1.5374140739440918, + 0.9199619889259338, + 1.58827543258667, + -0.42588773369789124, + -1.307411789894104, + -0.1648276150226593, + 0.17999683320522308, + 0.7365408539772034, + -0.2825929522514343, + 2.1419475078582764, + -0.44205278158187866, + 0.3557547330856323, + 0.11787930130958557, + 0.04347073659300804, + 0.5306029915809631, + -0.6201959848403931, + 2.444929838180542, + -0.7078790068626404, + 0.7410526275634766 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.614823579788208, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7115094661712646, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4821052551269531 + ], + [ + 0.2415536344051361, + -0.2574644386768341, + 1.3600168228149414, + 0.8914169669151306, + 1.311052918434143, + -0.13483268022537231, + -1.4109176397323608, + -0.007070749998092651, + 0.30199557542800903, + 0.867316484451294, + -0.0111103355884552, + 2.228789806365967, + -0.26305240392684937, + 0.4046628475189209, + 0.25534987449645996, + 0.29946526885032654, + 0.777583122253418, + -0.535997748374939, + 2.2955121994018555, + -0.6670190095901489, + 0.8460665941238403 + ], + [ + -0.212661013007164, + -0.6653347015380859, + 1.8750245571136475, + 0.7373600006103516, + 1.2695744037628174, + -0.7584145069122314, + -1.695502758026123, + 0.05721297860145569, + -0.04310476779937744, + 0.4696756601333618, + -0.2954420745372772, + 1.9897286891937256, + -0.6554491519927979, + 0.5805841088294983, + 0.17026324570178986, + 0.3433569669723511, + 0.3734014630317688, + -0.8102566003799438, + 2.792973279953003, + -0.8463170528411865, + 1.0759955644607544 + ], + [ + -0.039009034633636475, + -0.7773740291595459, + 1.064732313156128, + 0.712108850479126, + 1.270837426185608, + -0.5491334199905396, + -1.7919384241104126, + -0.5782418251037598, + 0.4058297872543335, + 0.5231752395629883, + -0.521438479423523, + 2.2066850662231445, + -0.5258626341819763, + -0.09100466966629028, + 0.24944378435611725, + -0.2271452248096466, + 0.9009795784950256, + -0.8140718340873718, + 2.6494152545928955, + -0.5153427124023438, + 0.671539306640625 + ], + [ + -0.3804699182510376, + -0.5808351039886475, + 1.8389031887054443, + 0.7141245603561401, + 1.0997881889343262, + -0.1678517758846283, + -1.884540319442749, + -0.3411717414855957, + 0.4830858111381531, + 0.8356180787086487, + -0.8238832950592041, + 1.743321418762207, + -0.3751307427883148, + 0.5275082588195801, + -0.08622422814369202, + 0.23666565120220184, + 0.802552342414856, + -0.8389871120452881, + 1.9817423820495605, + -0.4874984323978424, + 0.4390762150287628 + ], + [ + 0.4497724771499634, + -0.7880887389183044, + 1.4436085224151611, + 1.083756446838379, + 2.2298290729522705, + -0.03008332848548889, + -1.0250177383422852, + -0.6266056299209595, + 0.3751387298107147, + 1.1566712856292725, + -0.6914000511169434, + 1.953734278678894, + 0.08129626512527466, + 0.9812837839126587, + -0.4981238543987274, + 0.6697847247123718, + -0.05150604248046875, + -0.06941354274749756, + 1.9621039628982544, + -1.345557451248169, + 0.43397778272628784 + ], + [ + 0.03950639069080353, + -1.037135124206543, + 1.1041697263717651, + 1.109439730644226, + 1.144017219543457, + -0.07355684041976929, + -0.8942177295684814, + -0.16419151425361633, + -0.12125559151172638, + 0.36935141682624817, + -0.7298048734664917, + 1.4954023361206055, + -0.018926143646240234, + 0.4787553548812866, + -0.3127076327800751, + 0.6351315379142761, + 0.5993886590003967, + -0.5143972635269165, + 2.429173469543457, + -0.6674217581748962, + 0.5048977732658386 + ], + [ + 0.9763806462287903, + -1.1549197435379028, + 2.0378684997558594, + 1.2418556213378906, + 2.427785634994507, + -1.1326193809509277, + -1.5491721630096436, + -0.8937685489654541, + 0.2583051919937134, + -0.10687285661697388, + 0.2628750801086426, + 1.6517728567123413, + -0.21639958024024963, + 0.7581590414047241, + -0.29044294357299805, + -0.08927425742149353, + 1.3009071350097656, + -0.9239004850387573, + 2.5070056915283203, + -1.3246500492095947, + 0.24751555919647217 + ], + [ + 0.9781984090805054, + -0.13759100437164307, + 1.8698725700378418, + 1.4942535161972046, + 2.5587658882141113, + -1.1335766315460205, + -0.6910277605056763, + -0.1434302181005478, + 0.4021325707435608, + -0.12275522947311401, + 0.6571680307388306, + 1.9964357614517212, + -0.1442476212978363, + 1.3201656341552734, + 0.2586786448955536, + -0.38222962617874146, + 0.9950553774833679, + -0.8371140956878662, + 1.8977141380310059, + -0.795723021030426, + 1.505234956741333 + ], + [ + -0.6250126361846924, + 0.4231366813182831, + 1.701646089553833, + 1.6233667135238647, + 1.0861127376556396, + -0.3387698233127594, + -0.9825164079666138, + -1.1511213779449463, + 0.11523068696260452, + 1.5823121070861816, + 0.5840664505958557, + 2.145885705947876, + -1.2145874500274658, + 1.1558246612548828, + -0.5131542682647705, + -0.4715161919593811, + 1.0824741125106812, + -0.7667124271392822, + 1.8457963466644287, + -1.228658676147461, + 1.2112116813659668 + ], + [ + 1.1640653610229492, + -1.1697030067443848, + 1.6035369634628296, + 0.2161661982536316, + 1.4819366931915283, + 0.2569611370563507, + -1.3529595136642456, + 0.41208329796791077, + 0.03121529519557953, + 1.731370210647583, + -0.8131548166275024, + 2.996675729751587, + -1.1173620223999023, + 0.7474988102912903, + 0.21459203958511353, + -0.2133086621761322, + -0.22307389974594116, + -0.6306187510490417, + 2.0490896701812744, + -0.49808698892593384, + 0.8432145714759827 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.014628380537033081, + -0.18207009136676788, + 0.5374140739440918, + -0.04374772310256958, + 0.5882754325866699, + 0.25251278281211853, + 0.0, + -0.23516394197940826, + 0.12562739849090576, + 0.3381730020046234, + 0.033050984144210815, + 1.1419475078582764, + -0.3290551006793976, + 0.0, + 0.04625481367111206, + -0.0026901662349700928, + 0.0754578709602356, + -0.1481514573097229, + 1.444929838180542, + 0.049897611141204834, + 0.0 + ] + ], + "predicted_scores": [ + 8.008492469787598, + 7.5172505378723145, + 7.382919788360596, + 7.347182750701904, + 7.306814670562744, + 7.287628650665283, + 7.275352954864502, + 7.248529434204102 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1231780997/train0", + "selected_index": 0, + "state_hash": "771fb752a186163f05a49a9b404284bde55d89838769a764fca1d6d1a9419652", + "stored_base_utility": 0.7242962718009949, + "task_id": "LiftPegUpright-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.528373658657074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.528373658657074 + }, + "base_progress": 0.528373658657074, + "base_success": false, + "base_utility": 0.528373658657074, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7183936238288879, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7183936238288879 + }, + { + "contact_quality": null, + "progress": 0.40290379524230957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.40290379524230957 + }, + { + "contact_quality": null, + "progress": 0.70465487241745, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.70465487241745 + }, + { + "contact_quality": null, + "progress": 0.21204102039337158, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21204102039337158 + }, + { + "contact_quality": null, + "progress": 0.7051377892494202, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7051377892494202 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.5513363480567932, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5513363480567932 + }, + { + "contact_quality": null, + "progress": 0.6998324394226074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6998324394226074 + } + ], + "candidate_progress": [ + 0.7183936238288879, + 0.40290379524230957, + 0.70465487241745, + 0.21204102039337158, + 0.7051377892494202, + 1.0, + 0.5513363480567932, + 0.6998324394226074 + ], + "candidate_source_chart_ids": [ + "ms-0660d5f65df5fc4b6132", + "ms-c090bce47703a4c4bb0b", + "ms-13d8bae55d3c3710138f", + "ms-e1b6bcb41532a800d3cb", + "ms-df961fe24a43e56d1716", + "ms-e1b6bcb41532a800d3cb", + "ms-8f3d4e1d07bb5dcc7c57", + "ms-502dba1003c09ec60ba0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank6" + ], + "candidates_evaluated": true, + "chart_id": "ms-4b4cd12d70c0615e0930", + "generated_tangents": [ + [ + -0.08855167031288147, + -0.01947638764977455, + 0.8525065183639526, + 0.7275866270065308, + -0.02662205696105957, + -0.24836260080337524, + 0.06696748733520508, + 0.12216038256883621, + 0.01841357909142971, + -0.20160210132598877, + -0.02135186642408371, + 1.4949530363082886, + -0.39647719264030457, + -0.21711276471614838, + 0.25206345319747925, + -0.0022895345464348793, + -0.1862376183271408, + -0.2952665686607361, + 1.3065345287322998, + -0.4216145873069763, + -0.30772337317466736 + ], + [ + -0.2863571047782898, + -0.015488618984818459, + 0.7250853776931763, + 0.6399716138839722, + 0.04953143000602722, + -0.3244479298591614, + -0.16923648118972778, + 0.1838855892419815, + 0.011750123463571072, + -0.20457276701927185, + 0.18300533294677734, + 1.4162662029266357, + -0.42234745621681213, + -0.1418713480234146, + 0.3295390009880066, + -0.06663493067026138, + -0.1425269991159439, + -0.1821397989988327, + 1.3395352363586426, + -0.3949070870876312, + -0.30942055583000183 + ], + [ + 0.013350412249565125, + -0.023595791310071945, + 0.8012206554412842, + 0.7013665437698364, + -0.060644835233688354, + -0.07592709362506866, + 0.015901386737823486, + 0.09888153523206711, + 0.05620475858449936, + -0.11867684870958328, + 0.0716228112578392, + 1.5111439228057861, + -0.2909482717514038, + -0.14769954979419708, + 0.14648599922657013, + -0.0014249207451939583, + -0.14039750397205353, + -0.24590559303760529, + 1.3765792846679688, + -0.50099778175354, + -0.3407852351665497 + ], + [ + -0.003335699439048767, + -0.10702508687973022, + 1.04110586643219, + 0.7626993656158447, + -0.07068541646003723, + -0.0499863475561142, + -0.03780931234359741, + 0.08456481993198395, + 0.10477641224861145, + -0.08769210427999496, + 0.0016717985272407532, + 1.5022451877593994, + -0.3843003809452057, + 0.15376827120780945, + 0.17219732701778412, + 0.022844955325126648, + -0.21136970818042755, + -0.2430574744939804, + 1.319983959197998, + -0.46171414852142334, + 0.028106391429901123 + ], + [ + -0.10632289946079254, + -0.0019309855997562408, + 0.8705449104309082, + 0.5842269659042358, + -0.2546175718307495, + -0.026848211884498596, + -0.247003436088562, + 0.1544450968503952, + 0.017757084220647812, + -0.2705927789211273, + 0.09732816368341446, + 1.4460221529006958, + -0.2708267271518707, + -0.2734214663505554, + 0.2506394684314728, + 0.0013196561485528946, + -0.14811690151691437, + -0.2630149722099304, + 1.469775915145874, + -0.46460628509521484, + -0.08990907669067383 + ], + [ + -0.07995054125785828, + -0.04254212602972984, + 0.9654157161712646, + 0.7605307102203369, + -0.2271745800971985, + -0.2645329535007477, + -0.1342710256576538, + 0.23652783036231995, + 0.1368769258260727, + -0.16730135679244995, + 0.1140194833278656, + 1.2983299493789673, + -0.17730361223220825, + -0.1553676277399063, + 0.2944020628929138, + 0.0955798327922821, + -0.015879377722740173, + -0.3163635730743408, + 1.5000483989715576, + -0.12904201447963715, + -0.20126837491989136 + ], + [ + -0.006187573075294495, + -0.006004199385643005, + 0.6518256068229675, + 0.7949789762496948, + -0.162237286567688, + 0.09742814302444458, + -0.24809986352920532, + 0.004151567816734314, + 0.052042752504348755, + -0.004879884421825409, + 0.3146045506000519, + 1.6640344858169556, + -0.2553577423095703, + -0.1807306855916977, + 0.18132221698760986, + -0.12746618688106537, + -0.09342710673809052, + -0.18865381181240082, + 1.5411925315856934, + -0.37166672945022583, + -0.12060332298278809 + ], + [ + -0.02540813386440277, + 0.031248025596141815, + 0.5566630363464355, + 0.8019369840621948, + -0.2968810200691223, + 0.25957709550857544, + -0.4576603174209595, + 0.07570387423038483, + -0.055670879781246185, + -0.03128836303949356, + 0.28585225343704224, + 1.640716552734375, + -0.3310239315032959, + 0.05364423990249634, + 0.15118665993213654, + -0.08055293560028076, + -0.011531352996826172, + -0.21709395945072174, + 1.5603160858154297, + -0.372607558965683, + 0.26071345806121826 + ] + ], + "generated_utilities": [ + 0.7183936238288879, + 0.40290379524230957, + 0.70465487241745, + 0.21204102039337158, + 0.7051377892494202, + 2.0, + 0.5513363480567932, + 0.6998324394226074 + ], + "group_id": "ms-4b4cd12d70c0615e0930", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7225138545036316, + 2.0, + 0.6951258182525635, + 0.18915215134620667, + 0.20867900550365448, + 0.20041285455226898, + 0.20335882902145386, + 0.20815956592559814, + 0.2024221420288086, + 0.1940193623304367, + 0.20689067244529724, + 0.19995209574699402, + 0.21901433169841766, + 0.19443893432617188, + 0.20502614974975586, + 0.1959807276725769 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.09018868207931519, + 0.3819495439529419, + 0.9586174488067627, + 1.0247726440429688, + -0.179018035531044, + 0.125870943069458, + 0.0, + -0.1166897565126419, + 0.07711493968963623, + -0.24461698532104492, + -0.005436837673187256, + 1.737370491027832, + -0.4425608515739441, + 0.0, + 0.24725346267223358, + -0.03395397961139679, + 0.643236517906189, + 0.05422210693359375, + 1.2738127708435059, + 0.18164724111557007, + 0.0 + ], + [ + 0.6095315217971802, + 0.35729360580444336, + 1.9586174488067627, + 2.0247726440429688, + -0.2820541560649872, + 1.044631004333496, + -2.239265203475952, + -0.14250506460666656, + -0.4363628029823303, + -1.244616985321045, + -0.32413747906684875, + 2.737370491027832, + 0.24581307172775269, + -0.053983256220817566, + 0.2877395749092102, + 0.18538890779018402, + 0.45247530937194824, + -0.9457778930664062, + 2.273812770843506, + -0.8183527588844299, + 0.5866778492927551 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -4.478530406951904, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.10796651244163513, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1733556985855103 + ], + [ + 0.8415398597717285, + 0.17323988676071167, + 1.9669679403305054, + 2.188286304473877, + -0.386857271194458, + 1.3415166139602661, + -2.06968355178833, + -0.38555508852005005, + -0.6971946954727173, + -1.0110756158828735, + -0.19943451881408691, + 3.0229673385620117, + 0.03575621545314789, + -0.26961493492126465, + 0.5202459692955017, + 0.0921676978468895, + 0.7439858913421631, + -0.724668562412262, + 2.116668939590454, + -1.0585715770721436, + 0.7851569652557373 + ], + [ + 0.6909375190734863, + -0.018118590116500854, + 1.677350401878357, + 1.9505902528762817, + 0.11725521087646484, + 1.2839276790618896, + -2.0543243885040283, + -0.38843441009521484, + -0.6865814328193665, + -1.5103397369384766, + -0.2952713072299957, + 2.384809970855713, + -0.10558199882507324, + 0.29018354415893555, + 0.33942633867263794, + -0.19196103513240814, + 0.35257238149642944, + -0.8657878637313843, + 2.5445473194122314, + -0.706183910369873, + 0.7214455008506775 + ], + [ + 0.8977934122085571, + 0.7311913371086121, + 2.203964948654175, + 1.686698079109192, + -0.338996022939682, + 0.631161630153656, + -2.3877103328704834, + 0.07195566594600677, + -0.6892462968826294, + -0.9634823799133301, + -0.5621621012687683, + 2.479537010192871, + 0.675940752029419, + -0.047649919986724854, + 0.14082925021648407, + 0.19517391920089722, + 0.15626585483551025, + -1.3974462747573853, + 2.759931802749634, + -1.166107416152954, + 0.18018442392349243 + ], + [ + 0.7796773910522461, + 0.06440207362174988, + 1.5159449577331543, + 1.6007665395736694, + -0.12459242343902588, + 1.5781673192977905, + -2.5707991123199463, + 0.034798189997673035, + -0.44772639870643616, + -1.3837177753448486, + -0.5199699401855469, + 3.1566030979156494, + -0.22805768251419067, + -0.5881848931312561, + 0.49021050333976746, + -0.08376060426235199, + 0.5395568609237671, + -1.3038995265960693, + 2.37483811378479, + -0.615712583065033, + 0.7884383201599121 + ], + [ + 0.4830939471721649, + -0.12824741005897522, + 2.133068561553955, + 2.42085337638855, + -0.1641906201839447, + 1.6598048210144043, + -2.5325045585632324, + 0.06623834371566772, + 0.14292877912521362, + -1.85994553565979, + 0.01158285140991211, + 3.2622194290161133, + -0.017848491668701172, + -0.08653604984283447, + 0.2199593186378479, + -0.0463244765996933, + -0.04547742009162903, + -0.5160569548606873, + 2.1955723762512207, + -0.364714115858078, + 1.0323584079742432 + ], + [ + 0.7637265920639038, + -0.4020084738731384, + 2.4714059829711914, + 1.33060622215271, + 0.1464400589466095, + 0.7656213045120239, + -1.9972842931747437, + 0.42333751916885376, + -1.118815302848816, + -1.5639967918395996, + -0.32944348454475403, + 2.0334420204162598, + 0.37165334820747375, + -0.2861292064189911, + 0.6376298666000366, + 0.8521975874900818, + 0.6845490336418152, + -1.515796422958374, + 1.5925071239471436, + -0.4112323224544525, + 1.1011840105056763 + ], + [ + 1.456933617591858, + 1.0030901432037354, + 1.1997551918029785, + 1.7652757167816162, + 0.03292572498321533, + 1.302201271057129, + -2.748408317565918, + 0.7148768305778503, + -0.5089412331581116, + -0.5701506733894348, + -0.40410247445106506, + 3.586395502090454, + -0.37424683570861816, + -0.6850852966308594, + 0.9945629835128784, + 0.5821471810340881, + 0.8666700124740601, + -1.1607441902160645, + 2.1460094451904297, + -0.7912135720252991, + 0.6301633715629578 + ], + [ + -0.09475094079971313, + -0.28330767154693604, + 1.8353716135025024, + 2.3373594284057617, + 0.7110240459442139, + 1.9665697813034058, + -1.8502212762832642, + -0.40577346086502075, + -0.03506562113761902, + -0.7167142033576965, + -0.39153769612312317, + 2.3472883701324463, + 0.08497899770736694, + -0.9910531640052795, + -0.32254326343536377, + -0.11388416588306427, + -0.4873260259628296, + -0.8018304109573364, + 2.295731782913208, + -1.325164794921875, + 1.2513983249664307 + ], + [ + -0.32657402753829956, + 0.8198520541191101, + 1.211392879486084, + 1.974273443222046, + -0.11595214903354645, + 1.954041838645935, + -1.5547080039978027, + 0.06433123350143433, + -0.079216867685318, + -0.5887003540992737, + -0.6015424728393555, + 2.756274938583374, + 0.641853928565979, + -0.8642687201499939, + 0.07965350151062012, + 0.11391039192676544, + 1.10663902759552, + -1.506749153137207, + 3.222418785095215, + -1.0799858570098877, + -0.12158209085464478 + ], + [ + 0.9412136077880859, + -0.5255860090255737, + 1.9096710681915283, + 2.113077402114868, + -1.0082788467407227, + 0.1505374312400818, + -3.0413341522216797, + -0.24721917510032654, + -0.20225940644741058, + -1.6247544288635254, + -0.33320704102516174, + 1.8443769216537476, + 0.7569764256477356, + -0.5270920395851135, + 1.1813421249389648, + 1.0764858722686768, + 0.01533237099647522, + -1.4125463962554932, + 1.82404363155365, + 0.054037630558013916, + 0.45250433683395386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.056152522563934326, + 0.005679905414581299, + 0.9586174488067627, + 1.0247726440429688, + -0.054595887660980225, + 0.044631004333496094, + 0.0, + -0.09190673381090164, + 0.09592300653457642, + -0.24461698532104492, + -0.007907181978225708, + 1.737370491027832, + -0.029382914304733276, + 0.0, + 0.06305502355098724, + 0.03764393925666809, + 0.0785532295703888, + 0.005711078643798828, + 1.2738127708435059, + -0.04058480262756348, + 0.0 + ] + ], + "predicted_scores": [ + 8.606647491455078, + 8.40678596496582, + 8.361595153808594, + 8.183815002441406, + 8.076087951660156, + 7.391698837280273, + 6.736213684082031, + 6.477442264556885 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1025740212/train0", + "selected_index": 0, + "state_hash": "5b394fca1712c0667d126198527a4f0ba81893d536a5fcbf7b90f5b13fa703dc", + "stored_base_utility": 0.7225138545036316, + "task_id": "LiftPegUpright-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7146047949790955, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7146047949790955 + }, + { + "contact_quality": null, + "progress": 0.6888599991798401, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6888599991798401 + }, + { + "contact_quality": null, + "progress": 0.20082925260066986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20082925260066986 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.4248526990413666, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4248526990413666 + } + ], + "candidate_progress": [ + 0.7146047949790955, + 0.6888599991798401, + 0.20082925260066986, + 1.0, + 1.0, + 1.0, + 1.0, + 0.4248526990413666 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dfc38c43e9a87f99ef7f", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + true, + true, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6e268d1e2915740816a1", + "generated_tangents": [ + [ + 0.05382830649614334, + -0.11331828683614731, + 0.24708518385887146, + 0.2622750401496887, + 1.526719570159912, + 0.1897905468940735, + 0.06158256530761719, + 0.012340977787971497, + -0.06714219599962234, + 0.2525322735309601, + 0.18585392832756042, + 1.2123268842697144, + 0.10725459456443787, + 0.2626100778579712, + 0.20533233880996704, + 0.10720622539520264, + 0.10231655836105347, + -0.23213641345500946, + 0.8290656805038452, + -0.16684581339359283, + 0.006923690438270569 + ], + [ + -0.20426492393016815, + -0.0987476259469986, + -0.040768831968307495, + 0.24197199940681458, + 1.2935476303100586, + -0.34130075573921204, + 0.22074918448925018, + 0.1377646028995514, + -0.07693739235401154, + 0.2259937971830368, + 0.41069290041923523, + 1.3074543476104736, + -0.01693519949913025, + 0.33140867948532104, + 0.2790125012397766, + 0.057907070964574814, + 0.3529486656188965, + 0.031113892793655396, + 1.1380252838134766, + -0.3711043894290924, + 0.3389366567134857 + ], + [ + 0.021486014127731323, + 0.031080424785614014, + -0.009821504354476929, + 0.17265313863754272, + 1.284395694732666, + -0.1241624653339386, + 0.21560007333755493, + 0.08408765494823456, + -0.020719414576888084, + -0.051108650863170624, + -0.03287827968597412, + 0.9338744878768921, + 0.000622481107711792, + 0.016557902097702026, + 0.09422267973423004, + 0.0624970979988575, + -0.04071719944477081, + -0.21155504882335663, + 0.6939430236816406, + -0.09980952739715576, + -5.316734313964844e-05 + ], + [ + 0.06665319204330444, + -0.01946987211704254, + -0.016627758741378784, + 0.00996515154838562, + 1.5666582584381104, + -0.030515894293785095, + 0.039327532052993774, + 0.008302077651023865, + -0.026523491367697716, + 0.0132223442196846, + 0.05194508284330368, + 1.0613441467285156, + -0.14745602011680603, + 0.040985725820064545, + 0.00977376475930214, + 0.0787392109632492, + 0.04062876105308533, + -0.07444895803928375, + 0.7646706104278564, + -0.11978107690811157, + 0.04210434854030609 + ], + [ + 0.09597141295671463, + -0.05036396533250809, + -0.01115688681602478, + 0.19940954446792603, + 1.1502326726913452, + -0.20421072840690613, + 0.0500774085521698, + -0.059695541858673096, + -0.03573247790336609, + 0.12813347578048706, + 0.1269264817237854, + 1.308370590209961, + -0.0026264190673828125, + 0.29808908700942993, + 0.1656384915113449, + 0.026575077325105667, + 0.1393168419599533, + -0.07339702546596527, + 1.13913893699646, + -0.23282338678836823, + 0.20437416434288025 + ], + [ + -0.001097455620765686, + -0.030729271471500397, + -0.031170636415481567, + 0.047467440366744995, + 1.507990837097168, + -0.00832030177116394, + -0.05092889070510864, + -0.009620800614356995, + -0.02732613869011402, + 0.013773016631603241, + 0.0974811390042305, + 1.0921239852905273, + -0.03999364376068115, + 0.10736331343650818, + -0.041851554065942764, + 0.057456668466329575, + 0.005798563361167908, + -0.09395335614681244, + 0.8034914135932922, + -0.11294423043727875, + 0.06748195737600327 + ], + [ + 0.011999949812889099, + -0.0961015596985817, + 0.012530922889709473, + 0.0672605037689209, + 1.4114291667938232, + 0.0021196305751800537, + 0.06693747639656067, + 0.05514101684093475, + 0.009224820882081985, + 0.07333384454250336, + 0.021902766078710556, + 1.0683848857879639, + 0.07747206091880798, + 0.11161360144615173, + -0.08942191302776337, + 0.08119836449623108, + -0.04012192785739899, + -0.026301532983779907, + 0.791610836982727, + -0.1961107701063156, + 0.13662290573120117 + ], + [ + -0.25270652770996094, + -0.13033172488212585, + 0.03448626399040222, + -0.04329735040664673, + 1.2990400791168213, + -0.2328546941280365, + 0.02604636549949646, + -0.17190861701965332, + -0.08365319669246674, + -0.012697093188762665, + 0.27850812673568726, + 1.6041350364685059, + 0.07613518834114075, + 0.6118152737617493, + 0.07233655452728271, + -0.06824109703302383, + 0.12395923584699631, + -0.07512913644313812, + 1.2368290424346924, + -0.3936001658439636, + 0.5614538192749023 + ] + ], + "generated_utilities": [ + 0.7146047949790955, + 0.6888599991798401, + 0.20082925260066986, + 2.0, + 2.0, + 2.0, + 2.0, + 0.4248526990413666 + ], + "group_id": "ms-6e268d1e2915740816a1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7257392406463623, + 0.7165254950523376, + 2.0, + 0.72465980052948, + 0.7087006568908691, + 0.2306610643863678, + 0.7143489122390747, + 0.7045385837554932, + 0.5905889272689819, + 0.1919487714767456, + 0.1914421170949936, + 0.2230607271194458, + 0.20312398672103882, + 0.19757536053657532, + 0.19724789261817932, + 0.19653666019439697 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.44810131192207336, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0264720916748047, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.9882198572158813 + ], + [ + -0.4707597494125366, + 0.09937268495559692, + 0.6395220160484314, + -0.40935325622558594, + 2.4514670372009277, + -0.02397046983242035, + -0.21874001622200012, + 0.16799744963645935, + 0.0360843725502491, + -0.09719255566596985, + -0.8619975447654724, + 2.2499821186065674, + -0.4933375120162964, + 0.35259634256362915, + -0.34735018014907837, + -0.517992377281189, + 0.6015943288803101, + -0.7677568793296814, + 1.5267367362976074, + -0.16888000071048737, + 0.3771851658821106 + ], + [ + -0.8530820608139038, + -0.6707327961921692, + 0.7970488667488098, + 0.34902632236480713, + 2.1393990516662598, + 0.5651285648345947, + 0.113044872879982, + 0.2987540364265442, + 0.4139185845851898, + 0.41691792011260986, + -0.4906405806541443, + 1.5687955617904663, + -0.8899085521697998, + -0.01883232593536377, + 0.24473188817501068, + 0.554391622543335, + 0.23424841463565826, + -0.10042615234851837, + 2.1214542388916016, + -0.15780393779277802, + 0.5807479023933411 + ], + [ + -0.6562768220901489, + -0.059059903025627136, + 0.24766850471496582, + 0.06101328134536743, + 2.5052053928375244, + 0.5204365849494934, + 0.1595638245344162, + 0.47176462411880493, + 0.3036114573478699, + 0.13197286427021027, + -0.2551935017108917, + 2.185537099838257, + -1.0984127521514893, + 0.3213496804237366, + 0.49875718355178833, + 0.1223822832107544, + 0.2575722932815552, + -0.7017092704772949, + 1.7495591640472412, + -0.19501343369483948, + 1.1214004755020142 + ], + [ + -0.45427173376083374, + -0.041353583335876465, + 0.8143320083618164, + -0.39680930972099304, + 2.4542014598846436, + 0.382559597492218, + -0.9993512630462646, + 0.5202391147613525, + 0.5282149314880371, + 0.4165073335170746, + 0.43153735995292664, + 2.5625123977661133, + -0.721943199634552, + 0.6765530109405518, + -0.6697012186050415, + -0.0637422502040863, + 0.6919896602630615, + -1.033813714981079, + 2.317282199859619, + -0.13503669202327728, + -0.07009828090667725 + ], + [ + -0.548885703086853, + -0.609938383102417, + 0.7181543707847595, + 0.11203774809837341, + 2.478057861328125, + -0.6170704960823059, + -1.0383654832839966, + 0.3524467349052429, + 0.5748242139816284, + 0.42987847328186035, + 0.013119548559188843, + 1.3954555988311768, + -0.6403790712356567, + 0.025414258241653442, + -0.7780460119247437, + -0.23124903440475464, + 0.48347827792167664, + 0.05266502499580383, + 1.692553997039795, + 0.362108051776886, + 0.6330842971801758 + ], + [ + -1.3070502281188965, + -0.947432816028595, + 0.4198402762413025, + -1.1768442392349243, + 1.7843327522277832, + 0.7587301731109619, + 0.7447960376739502, + 0.9411643147468567, + 0.3013244867324829, + -0.3862171769142151, + -1.0263384580612183, + 2.97957181930542, + 0.1849566102027893, + 0.08799406886100769, + 0.1984272301197052, + 0.5277765393257141, + 0.2931878864765167, + -1.1385071277618408, + 1.884846568107605, + 0.23083388805389404, + 0.6359583139419556 + ], + [ + -0.5375385880470276, + 0.16159841418266296, + 1.260543942451477, + -0.6744770407676697, + 2.1646244525909424, + 0.6190755367279053, + -0.746461808681488, + -0.38674360513687134, + 0.8897944092750549, + -0.594306468963623, + -0.05647379159927368, + 1.7390328645706177, + -0.037059009075164795, + 0.470071941614151, + 0.010003047063946724, + 0.08421775698661804, + -0.4613104462623596, + -0.22854508459568024, + 1.772307276725769, + -0.29165884852409363, + 0.6179300546646118 + ], + [ + -0.6489957571029663, + 0.4526708126068115, + 0.5115790367126465, + 0.31459903717041016, + 2.9655239582061768, + 0.07586031407117844, + -1.1667817831039429, + -0.7146965861320496, + 1.0295621156692505, + 0.1932714432477951, + -0.2703063488006592, + 2.9046523571014404, + -1.5882594585418701, + 0.3030351996421814, + 0.6543201208114624, + -0.448961466550827, + 0.4115087389945984, + -0.4087216854095459, + 2.6432881355285645, + -0.6797424554824829, + 0.2669999599456787 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.22510427236557007, + -0.26737168431282043, + 0.10375632345676422, + 0.12666502594947815, + 1.379244327545166, + -0.01088210940361023, + 0.0, + 0.23721180856227875, + -0.10981421172618866, + -0.021816372871398926, + -0.071512371301651, + 1.0360438823699951, + 0.09019589424133301, + 0.0, + 0.03181244805455208, + -0.06466590613126755, + 0.2068846970796585, + 0.14229199290275574, + 0.7610219717025757, + -0.03211131691932678, + 0.0 + ] + ], + "predicted_scores": [ + 8.914471626281738, + 8.706720352172852, + 8.530256271362305, + 8.064072608947754, + 7.93787145614624, + 7.906517505645752, + 7.663044452667236, + 6.954155921936035 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1475440973/train0", + "selected_index": 0, + "state_hash": "38ff5a264c3c39ff1ec8756d84e622963a11ff1ac61568ba57fbea0c114c3140", + "stored_base_utility": 0.7257392406463623, + "task_id": "LiftPegUpright-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7239725589752197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7239725589752197 + }, + "base_progress": 0.7239725589752197, + "base_success": false, + "base_utility": 0.7239725589752197, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7199956774711609, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7199956774711609 + }, + { + "contact_quality": null, + "progress": 0.7059324383735657, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7059324383735657 + }, + { + "contact_quality": null, + "progress": 0.7095306515693665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7095306515693665 + }, + { + "contact_quality": null, + "progress": 0.7091171145439148, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7091171145439148 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7119956612586975, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7119956612586975 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7287879586219788, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7287879586219788 + } + ], + "candidate_progress": [ + 0.7199956774711609, + 0.7059324383735657, + 0.7095306515693665, + 0.7091171145439148, + 1.0, + 0.7119956612586975, + 1.0, + 0.7287879586219788 + ], + "candidate_source_chart_ids": [ + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + true, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8864796e31bbe93d7c5d", + "generated_tangents": [ + [ + -0.24948729574680328, + -0.10154072195291519, + 0.18723857402801514, + 0.04775512218475342, + 1.304319143295288, + -0.6922988891601562, + 0.09477657079696655, + 0.2288207709789276, + -0.19800102710723877, + 0.09405873715877533, + 0.33343005180358887, + 1.656205654144287, + 0.13534113764762878, + 0.5131504535675049, + 0.2962728440761566, + 0.026287253946065903, + 0.38559427857398987, + -0.07534115016460419, + 1.5071253776550293, + -0.5716783404350281, + 0.48471352458000183 + ], + [ + -0.051453426480293274, + -0.20163246989250183, + 0.09931141138076782, + -0.28834080696105957, + 1.3524904251098633, + -0.6391878128051758, + 0.03127303719520569, + 0.20816756784915924, + -0.1451248824596405, + -0.04293700307607651, + -0.0452580600976944, + 1.5044714212417603, + 0.10335925221443176, + 0.3435777425765991, + 0.19738075137138367, + -0.001113283447921276, + 0.00808076560497284, + -0.3290145993232727, + 1.2862768173217773, + -0.49025091528892517, + 0.16652131080627441 + ], + [ + -0.05811721086502075, + -0.1174122765660286, + 0.07382097840309143, + -0.27939334511756897, + 1.4479877948760986, + -0.5250906944274902, + 0.023662209510803223, + 0.12426549196243286, + -0.13466525077819824, + -0.07124709337949753, + 0.036029260605573654, + 1.5989625453948975, + -0.13268673419952393, + 0.21213477849960327, + 0.29586273431777954, + 0.03312433138489723, + 0.08937002718448639, + -0.30316615104675293, + 1.2356297969818115, + -0.28782886266708374, + 0.12316034734249115 + ], + [ + -0.07423686981201172, + -0.11393339186906815, + 0.05387812852859497, + -0.2970837950706482, + 1.4041261672973633, + -0.5529136657714844, + -0.010064423084259033, + 0.12289397418498993, + -0.12582704424858093, + -0.10877882689237595, + 0.0802784413099289, + 1.5274192094802856, + -0.05774989724159241, + 0.2809354364871979, + 0.24325363337993622, + 0.008402958512306213, + 0.0143260657787323, + -0.2726101875305176, + 1.2107031345367432, + -0.3309725224971771, + 0.17144231498241425 + ], + [ + -0.1317937821149826, + -0.042197875678539276, + 0.0962325930595398, + -0.07744404673576355, + 1.1796233654022217, + -0.5120006799697876, + 0.20142537355422974, + -0.01813267171382904, + -0.13524067401885986, + 0.16094869375228882, + 0.12391580641269684, + 1.3178013563156128, + -0.09236329793930054, + 0.26182490587234497, + 0.24179939925670624, + -0.05918389558792114, + 0.18928943574428558, + -0.299518346786499, + 1.5256214141845703, + -0.4084209203720093, + 0.23592253029346466 + ], + [ + -0.16534243524074554, + -0.1010637879371643, + 0.1295507848262787, + -0.16955994069576263, + 1.2102713584899902, + -0.4370811879634857, + 0.3040257692337036, + 0.15275846421718597, + -0.13499030470848083, + -0.1539989411830902, + -0.0651867613196373, + 1.4934074878692627, + -0.1477290689945221, + 0.1023603081703186, + 0.1667497307062149, + -0.018498580902814865, + 0.019455790519714355, + -0.4738162159919739, + 1.3704724311828613, + -0.3399258852005005, + 0.2120511531829834 + ], + [ + 0.0197494775056839, + -0.09809909760951996, + 0.10868239402770996, + 0.004440367221832275, + 1.1123642921447754, + -0.536103367805481, + 0.0898403525352478, + -0.012524142861366272, + -0.14132468402385712, + 0.1149870827794075, + 0.07034085690975189, + 1.5279051065444946, + 0.0010775923728942871, + 0.503241777420044, + 0.17414912581443787, + -0.01966852694749832, + 0.21650609374046326, + -0.28958094120025635, + 1.553039312362671, + -0.3947843909263611, + 0.1815936416387558 + ], + [ + -0.30940502882003784, + -0.20311163365840912, + -0.015687376260757446, + -0.24670451879501343, + 1.1989598274230957, + -0.7257016897201538, + -0.05698734521865845, + -0.1187492311000824, + -0.13922591507434845, + -0.039650849997997284, + 0.20960965752601624, + 1.7681872844696045, + 0.11312928795814514, + 0.6690563559532166, + 0.21477000415325165, + -0.10057109594345093, + 0.19097304344177246, + -0.2768039107322693, + 1.617624044418335, + -0.6416243314743042, + 0.503345251083374 + ] + ], + "generated_utilities": [ + 0.7199956774711609, + 0.7059324383735657, + 0.7095306515693665, + 0.7091171145439148, + 2.0, + 0.7119956612586975, + 2.0, + 0.7287879586219788 + ], + "group_id": "ms-8864796e31bbe93d7c5d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.728268027305603, + 0.7257949113845825, + 2.0, + 0.7287554740905762, + 0.72548508644104, + 0.7011849880218506, + 0.7089968919754028, + 0.689879298210144, + 0.722068727016449, + 0.7046384811401367, + 0.1933039277791977, + 0.629345178604126, + 0.18965840339660645, + 0.18740439414978027, + 0.2032846063375473, + 0.20379847288131714 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.3859826922416687, + 0.1119421124458313, + 0.6443181037902832, + 0.11836245656013489, + 1.863237738609314, + -1.166780948638916, + 0.8025261163711548, + -0.5342166423797607, + -0.2057022899389267, + 0.8987585306167603, + -0.525823712348938, + 1.6212486028671265, + 0.25674968957901, + 0.4037417471408844, + -0.4561460018157959, + -0.18429945409297943, + -0.1778680682182312, + -0.22491087019443512, + 2.095893144607544, + -0.3484030067920685, + 0.47628670930862427 + ], + [ + -0.446367084980011, + -0.19229409098625183, + -0.03271818161010742, + 0.39123907685279846, + 1.1786221265792847, + -0.16883885860443115, + -0.3337137699127197, + 0.7789723873138428, + -0.12953178584575653, + -0.028340458869934082, + -0.871502161026001, + 0.7702060341835022, + -0.645424485206604, + 1.0858956575393677, + -0.05413459241390228, + -0.14592209458351135, + -0.03685358166694641, + -0.686826229095459, + 2.084512710571289, + 0.11898508667945862, + 0.20845119655132294 + ], + [ + 0.09377387166023254, + -0.03508002683520317, + -0.29283326864242554, + -0.9559528827667236, + 2.0994086265563965, + 0.02743762731552124, + 0.6819101572036743, + 0.607413113117218, + 0.4378519356250763, + 0.6497513055801392, + -0.5086554288864136, + 0.9556822180747986, + -0.8518581986427307, + 0.31636691093444824, + -0.4196639060974121, + 0.6866374015808105, + 0.5175505876541138, + -0.572692334651947, + 1.651871681213379, + -0.7227057814598083, + -0.33861052989959717 + ], + [ + -0.42610836029052734, + 0.10144555568695068, + 0.7105271220207214, + -1.291037678718567, + 2.2307684421539307, + 0.17725014686584473, + -0.08436906337738037, + 0.9681323766708374, + 0.5482062697410583, + 1.368379831314087, + -1.021532416343689, + 2.5197110176086426, + -0.3508763909339905, + 0.7584415078163147, + 1.0978648662567139, + 0.2626119554042816, + 0.6115584373474121, + -0.7777479290962219, + 1.7278802394866943, + -0.15240566432476044, + 0.4239119291305542 + ], + [ + 1.0207308530807495, + 0.8557561039924622, + 0.4489574134349823, + -1.0117379426956177, + 0.8262714147567749, + -0.630092442035675, + 0.09812727570533752, + 0.21663640439510345, + 0.23771141469478607, + -0.47456812858581543, + -1.3189021348953247, + 2.269665002822876, + -0.7066116333007812, + 1.1826255321502686, + 1.0058563947677612, + 0.1694127470254898, + -0.5649446845054626, + -0.32043421268463135, + 0.46698564291000366, + -0.9052557945251465, + -0.12436902523040771 + ], + [ + -0.5202209949493408, + 0.30583277344703674, + 1.2163622379302979, + 0.4305659234523773, + 1.7276779413223267, + -0.1535550355911255, + -0.5289345383644104, + -0.632349967956543, + 0.9393473267555237, + 0.38391292095184326, + -1.1998018026351929, + 1.9739618301391602, + -1.3405330181121826, + 0.048939138650894165, + 0.6369897127151489, + 0.629721999168396, + 1.388192892074585, + -1.065996766090393, + 1.9119879007339478, + -1.0432264804840088, + 0.9627590179443359 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.13860583305358887, + -0.06957411766052246, + 0.1504676342010498, + 0.1671028435230255, + 0.5675687789916992, + 0.16587841510772705, + 0.0, + -0.21137550473213196, + -0.21542736887931824, + 0.2565074861049652, + -0.00923699140548706, + 0.5251069068908691, + -0.06964048743247986, + 0.0, + -0.19547803699970245, + 0.10278619825839996, + -0.19206219911575317, + -0.1487000733613968, + 0.44575393199920654, + 0.0725468099117279, + 0.0 + ] + ], + "predicted_scores": [ + 9.099912643432617, + 8.674956321716309, + 8.618236541748047, + 8.410686492919922, + 8.1096830368042, + 7.981508731842041, + 7.959592342376709, + 6.850729465484619 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "551504635/train0", + "selected_index": 0, + "state_hash": "ab6587e0b2a3b5f21f23c56daf8894249774912a3b20794bd49dc1b78ec736a7", + "stored_base_utility": 0.728268027305603, + "task_id": "LiftPegUpright-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.42231252789497375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.42231252789497375 + }, + "base_progress": 0.42231252789497375, + "base_success": false, + "base_utility": 0.42231252789497375, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.44024136662483215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.44024136662483215 + }, + { + "contact_quality": null, + "progress": 0.3434007465839386, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.3434007465839386 + }, + { + "contact_quality": null, + "progress": 0.2951448857784271, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2951448857784271 + }, + { + "contact_quality": null, + "progress": 0.30091413855552673, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.30091413855552673 + }, + { + "contact_quality": null, + "progress": 0.21940119564533234, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21940119564533234 + }, + { + "contact_quality": null, + "progress": 0.46552690863609314, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.46552690863609314 + }, + { + "contact_quality": null, + "progress": 0.3839980363845825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.3839980363845825 + }, + { + "contact_quality": null, + "progress": 0.5588552355766296, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5588552355766296 + } + ], + "candidate_progress": [ + 0.44024136662483215, + 0.3434007465839386, + 0.2951448857784271, + 0.30091413855552673, + 0.21940119564533234, + 0.46552690863609314, + 0.3839980363845825, + 0.5588552355766296 + ], + "candidate_source_chart_ids": [ + "ms-e1b6bcb41532a800d3cb", + "ms-0660d5f65df5fc4b6132", + "ms-df961fe24a43e56d1716", + "ms-c090bce47703a4c4bb0b", + "ms-8bc7aa419d17368d81b5", + "ms-8bc7aa419d17368d81b5", + "ms-13d8bae55d3c3710138f", + "ms-e1b6bcb41532a800d3cb" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b04b489f4fbad93b65ca", + "generated_tangents": [ + [ + -0.09565496444702148, + -0.13263177871704102, + 0.6378915309906006, + 0.6397560834884644, + 0.19761790335178375, + -0.47166624665260315, + -0.24765628576278687, + 0.12111947685480118, + 0.2253347933292389, + -0.019841589033603668, + 0.003625147044658661, + 1.1748700141906738, + -0.07906889915466309, + -0.18107132613658905, + 0.39330577850341797, + 0.15019100904464722, + -0.11007456481456757, + -0.17186932265758514, + 1.6526024341583252, + -0.09600405395030975, + -0.1769442856311798 + ], + [ + -0.10873229801654816, + -0.1422015279531479, + 0.545904815196991, + 0.7196091413497925, + 0.4515576958656311, + -0.5527061223983765, + -0.19033455848693848, + 0.042164817452430725, + 0.1231064647436142, + -0.04291204363107681, + -0.04714444279670715, + 1.3284800052642822, + -0.32049399614334106, + -0.24472157657146454, + 0.40207263827323914, + 0.038899146020412445, + -0.2567472457885742, + -0.26330113410949707, + 1.5134329795837402, + -0.5167853236198425, + -0.31856706738471985 + ], + [ + -0.13590170443058014, + -0.11261137574911118, + 0.43233710527420044, + 0.6035963296890259, + 0.43890368938446045, + -0.3353692591190338, + -0.4279569387435913, + 0.07998514175415039, + 0.10984005779027939, + -0.11447293311357498, + 0.0253337100148201, + 1.31846284866333, + -0.23573988676071167, + -0.30657511949539185, + 0.3927244544029236, + 0.03903277590870857, + -0.29468464851379395, + -0.21908621490001678, + 1.6010043621063232, + -0.48466551303863525, + -0.29354098439216614 + ], + [ + -0.2986154556274414, + -0.1027606651186943, + 0.2969779968261719, + 0.5915371179580688, + 0.4253876209259033, + -0.5407867431640625, + -0.24466967582702637, + 0.09115974605083466, + 0.12147343903779984, + -0.12931665778160095, + 0.04664324223995209, + 1.3543797731399536, + -0.5234538316726685, + -0.24561326205730438, + 0.4230315685272217, + -0.0007578907534480095, + -0.23283682763576508, + -0.2190060168504715, + 1.5609934329986572, + -0.4632875919342041, + -0.4699924886226654 + ], + [ + -0.39335954189300537, + -0.2351718693971634, + 0.549156129360199, + 0.9912879467010498, + 0.6062486171722412, + 0.06946003437042236, + -0.6914623379707336, + 0.2573244571685791, + 0.0023165950551629066, + -0.07012902945280075, + 0.2582142651081085, + 1.4619543552398682, + 0.18062707781791687, + -0.20514728128910065, + 0.37644141912460327, + 0.097018763422966, + -0.3484461307525635, + -0.03962719440460205, + 1.5388891696929932, + -0.38853707909584045, + 0.2924542725086212 + ], + [ + -0.17095820605754852, + -0.16430068016052246, + 0.40511637926101685, + 0.8542691469192505, + 0.044514626264572144, + 0.3263012766838074, + -0.7373185753822327, + 0.12780943512916565, + 0.0031920671463012695, + 0.06597979366779327, + 0.3259868621826172, + 1.5937522649765015, + -0.053085148334503174, + 0.025669053196907043, + 0.327912300825119, + -0.008785219863057137, + -0.177174374461174, + -0.20275263488292694, + 1.6584527492523193, + -0.24582432210445404, + -0.01783326268196106 + ], + [ + 0.03756118565797806, + -0.13389122486114502, + 0.41211017966270447, + 0.7176592350006104, + 0.4354831278324127, + -0.40109220147132874, + -0.2534736394882202, + 0.018677935004234314, + 0.1709856390953064, + 0.0442802831530571, + 0.04523874446749687, + 1.3362438678741455, + -0.28743869066238403, + -0.23050443828105927, + 0.27787747979164124, + 0.032572779804468155, + -0.23757053911685944, + -0.1685778945684433, + 1.602299451828003, + -0.500733494758606, + -0.3583735525608063 + ], + [ + -1.6555190086364746e-05, + -0.12617585062980652, + 0.6307705044746399, + 0.7687804698944092, + 0.34772413969039917, + -0.4515925943851471, + -0.24265265464782715, + -0.013622060418128967, + 0.18106137216091156, + 0.02512286603450775, + -0.0892622247338295, + 1.3640633821487427, + -0.4228178858757019, + -0.02813822031021118, + 0.24052219092845917, + 0.07421454787254333, + -0.19758112728595734, + -0.12108777463436127, + 1.4839928150177002, + -0.5218106508255005, + -0.02068018913269043 + ] + ], + "generated_utilities": [ + 0.44024136662483215, + 0.3434007465839386, + 0.2951448857784271, + 0.30091413855552673, + 0.21940119564533234, + 0.46552690863609314, + 0.3839980363845825, + 0.5588552355766296 + ], + "group_id": "ms-b04b489f4fbad93b65ca", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7198216915130615, + 0.7247374057769775, + 2.0, + 0.22730140388011932, + 0.21130704879760742, + 0.2076549530029297, + 0.2006116509437561, + 0.21285679936408997, + 0.19614991545677185, + 0.20440611243247986, + 0.2015489637851715, + 0.19511334598064423, + 0.20091164112091064, + 0.19681888818740845, + 0.19675713777542114, + 0.20853900909423828 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.30706143379211426, + 0.2797066569328308, + 0.4565920829772949, + 0.7342244386672974, + -0.28428468108177185, + 0.012420237064361572, + 0.0, + 0.19500797986984253, + -0.1289721131324768, + -0.3524337112903595, + 0.07460063695907593, + 1.7684156894683838, + 0.11335399746894836, + 0.0, + -0.5014548301696777, + 0.4537479281425476, + 0.1963198333978653, + -0.025568604469299316, + 1.6286182403564453, + -0.0963512659072876, + 0.0 + ], + [ + 1.3070614337921143, + -0.7202933430671692, + 1.456592082977295, + 1.7342244386672974, + -0.15130555629730225, + 0.9519147872924805, + -1.4541058540344238, + -0.6432010531425476, + -0.26959508657455444, + -0.3406839668750763, + -0.4419664740562439, + 2.768415689468384, + -0.3271861970424652, + -0.13685251772403717, + 0.23842881619930267, + 0.20406432449817657, + 0.3414191007614136, + -1.0255686044692993, + 2.6286182403564453, + -0.9051820039749146, + 0.6850019693374634 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9082117080688477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.27370503544807434, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.3700039386749268 + ], + [ + 1.0634682178497314, + -0.8590207099914551, + 1.6810569763183594, + 1.8525642156600952, + 0.06716938316822052, + 1.0774534940719604, + -1.177553415298462, + -0.7412493228912354, + -0.3193308115005493, + -0.4253617525100708, + -0.4890587329864502, + 2.624048948287964, + -0.109145388007164, + -0.10380380600690842, + 0.20137116312980652, + 0.20110605657100677, + 0.06929725408554077, + -0.9908366203308105, + 2.5783159732818604, + -0.7248227000236511, + 0.919882595539093 + ], + [ + 1.2083125114440918, + -0.9112091064453125, + 1.1255995035171509, + 1.9678500890731812, + -0.3424606919288635, + 1.350451946258545, + -1.5799542665481567, + -1.014689326286316, + -0.3427739441394806, + -0.24094250798225403, + -0.4992668032646179, + 2.403038740158081, + -0.29080134630203247, + -0.1569681167602539, + 0.4877905249595642, + -0.16464389860630035, + 0.41957730054855347, + -0.6997033357620239, + 2.80957293510437, + -1.0509651899337769, + 0.8937826156616211 + ], + [ + 1.2133954763412476, + -1.0171141624450684, + 1.1572445631027222, + 1.8050156831741333, + -0.29570114612579346, + 0.7672240734100342, + -0.976697564125061, + -0.8219587802886963, + -0.746895432472229, + -0.346658855676651, + -0.7942779064178467, + 3.0403196811676025, + -0.6172577738761902, + 0.3615778088569641, + 0.546725332736969, + -0.06415437161922455, + 0.43834370374679565, + -1.332592248916626, + 3.029940605163574, + -0.5720348358154297, + 1.1660562753677368 + ], + [ + 0.7937494516372681, + -0.45201122760772705, + 2.0367026329040527, + 2.1590728759765625, + 0.38935917615890503, + 0.5424834489822388, + -1.391097903251648, + -1.0657228231430054, + -0.44411367177963257, + -0.3797546923160553, + -0.03612968325614929, + 2.867069721221924, + -0.26322364807128906, + -0.32477179169654846, + 0.007963955402374268, + 0.053216055035591125, + 0.36466139554977417, + -0.7477234601974487, + 2.170649766921997, + -0.46266868710517883, + 0.7669195532798767 + ], + [ + 1.0920023918151855, + -1.0678586959838867, + 1.4391454458236694, + 2.155820608139038, + -0.7347199320793152, + 0.6208673715591431, + -1.6380563974380493, + -0.6589961051940918, + -0.8267081379890442, + 0.011062979698181152, + -0.1169244647026062, + 2.743579626083374, + -0.41342735290527344, + -0.5980290770530701, + 0.49618369340896606, + -0.09863747656345367, + 0.2179805338382721, + -0.6006474494934082, + 2.2965025901794434, + -0.97348952293396, + 1.2447597980499268 + ], + [ + 1.8286218643188477, + -1.486896276473999, + 1.8610713481903076, + 1.792364478111267, + -0.06209900975227356, + 0.7138283252716064, + -1.0023046731948853, + -0.8588749170303345, + -0.30936315655708313, + 0.33444294333457947, + 0.2771298885345459, + 3.0452492237091064, + -0.6618144512176514, + -0.747363805770874, + 0.19540274143218994, + -0.05051566660404205, + 0.5057645440101624, + -0.3770519495010376, + 3.028423309326172, + -0.19366812705993652, + 1.1766406297683716 + ], + [ + 1.057653784751892, + -0.15160071849822998, + 1.1646175384521484, + 2.517653226852417, + 0.037323400378227234, + 1.4844036102294922, + -1.2659661769866943, + -0.9694168567657471, + -0.792635440826416, + 0.15484732389450073, + -0.6592323780059814, + 2.426063060760498, + -0.8063763976097107, + 0.37280529737472534, + -0.1984816938638687, + 0.3488052487373352, + 0.36773377656936646, + -0.9600323438644409, + 1.801730990409851, + -1.1618295907974243, + 0.526383101940155 + ], + [ + 0.8403785228729248, + -1.7131881713867188, + 1.7484346628189087, + 1.3915244340896606, + 0.038706690073013306, + 1.7859117984771729, + -0.6996253132820129, + -0.12596702575683594, + -0.07181461155414581, + -0.8556877374649048, + -0.6567700505256653, + 2.935109853744507, + 0.3808797299861908, + 0.1733608990907669, + 0.9097857475280762, + 0.7001722455024719, + 0.14087507128715515, + -0.32659411430358887, + 2.278188943862915, + -1.87721586227417, + 1.438266634941101 + ], + [ + 1.259635329246521, + -0.5244767665863037, + 2.201894998550415, + 2.2337045669555664, + -0.333431601524353, + 0.7420244216918945, + -2.0896377563476562, + 0.00931471586227417, + 0.6261725425720215, + -0.49785688519477844, + -0.9404669404029846, + 3.756554365158081, + 0.5252982378005981, + -0.3158212900161743, + 0.4053819179534912, + 0.8204222917556763, + 0.5041900277137756, + -0.43809741735458374, + 3.519717216491699, + -1.0316743850708008, + 0.3069072663784027 + ], + [ + 0.9530733823776245, + -0.011007547378540039, + 2.396453380584717, + 2.6382436752319336, + 0.29263433814048767, + 1.6715352535247803, + -0.819398045539856, + -0.5916767120361328, + 0.006437480449676514, + -0.3942071795463562, + -0.5187541842460632, + 3.677276849746704, + -0.4376804232597351, + 0.3331298828125, + -0.3693196773529053, + -0.3432707190513611, + 1.104022741317749, + -1.0780245065689087, + 3.0859882831573486, + -0.26956111192703247, + 0.8377041816711426 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.30706143379211426, + 0.024911105632781982, + 0.4565920829772949, + 0.7342244386672974, + 0.01698923110961914, + -0.04808521270751953, + 0.0, + 0.10375595092773438, + 0.23247671127319336, + -0.12021404504776001, + -0.19280612468719482, + 1.7684156894683838, + -0.23127278685569763, + 0.0, + -0.1010780781507492, + 0.00952194631099701, + -0.16307592391967773, + -0.14496707916259766, + 1.6286182403564453, + -0.28656232357025146, + 0.0 + ] + ], + "predicted_scores": [ + 6.63995361328125, + 6.638190746307373, + 6.4909772872924805, + 6.348557949066162, + 6.141026020050049, + 5.725180625915527, + 5.716888427734375, + 5.514687538146973 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113268940/train0", + "selected_index": 0, + "state_hash": "76e9a962970055fa7565c007db1b35e003cd352908be4cf573066d8b58e2d443", + "stored_base_utility": 0.7198216915130615, + "task_id": "LiftPegUpright-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7235286831855774, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7235286831855774 + }, + { + "contact_quality": null, + "progress": 0.7263574600219727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7263574600219727 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7248995304107666, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7248995304107666 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7258209586143494, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7258209586143494 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.7235286831855774, + 0.7263574600219727, + 1.0, + 0.7248995304107666, + 1.0, + 0.7258209586143494, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-efbde8652e75274fbd69", + "ms-b483cac145e16ccdf42a", + "ms-efbde8652e75274fbd69", + "ms-062addb9da32a0c2ea97", + "ms-baa36ee75ec2de550b9b", + "ms-b483cac145e16ccdf42a", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + false, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-c663f7e73e7e8376a05c", + "generated_tangents": [ + [ + 0.11944491416215897, + 0.031826525926589966, + -0.33816173672676086, + 0.029209166765213013, + 1.7001824378967285, + 0.2834256887435913, + 0.1120532751083374, + 0.13245084881782532, + -0.11714623868465424, + 0.06882806867361069, + -0.08093375712633133, + 0.7392621636390686, + 0.10305061936378479, + -0.002112284302711487, + 0.19480200111865997, + 0.14463014900684357, + 0.0032646209001541138, + -0.31107252836227417, + 0.9268627166748047, + -0.20665471255779266, + -0.23001036047935486 + ], + [ + 0.08528398722410202, + 0.0176863893866539, + -0.38725516200065613, + 0.15819844603538513, + 1.5603866577148438, + 0.01915358006954193, + -0.05864226818084717, + 0.021228879690170288, + -0.03750317543745041, + 0.14633160829544067, + -0.018779106438159943, + 0.9331331849098206, + 0.14669212698936462, + 0.11888884752988815, + 0.1058644950389862, + 0.05184245482087135, + -0.10833440721035004, + -0.0656033605337143, + 1.0013415813446045, + -0.06810598075389862, + -0.027165308594703674 + ], + [ + 0.05952262133359909, + -0.013512890785932541, + -0.3565492331981659, + 0.16039448976516724, + 1.6706795692443848, + 0.020124569535255432, + -0.22814667224884033, + -0.025210320949554443, + -0.06667977571487427, + 0.17737813293933868, + 0.043758049607276917, + 0.9179978370666504, + 0.07474192976951599, + 0.09498706459999084, + 0.09281895309686661, + 0.03722773492336273, + -0.17794738709926605, + 0.022280260920524597, + 0.9304789304733276, + -0.0854514092206955, + -0.01712864637374878 + ], + [ + 0.09514965116977692, + -0.0005120635032653809, + -0.37256380915641785, + 0.15310999751091003, + 1.5711615085601807, + 0.013629376888275146, + -0.0076225996017456055, + 0.03125244379043579, + -0.04634927213191986, + 0.09063060581684113, + -0.050507135689258575, + 1.019006371498108, + 0.12158665060997009, + 0.07587262988090515, + 0.13865254819393158, + 0.03530486300587654, + -0.06182853877544403, + -0.08614863455295563, + 0.9436187744140625, + -0.07899908721446991, + 0.005535438656806946 + ], + [ + 0.07381761819124222, + 0.04191262274980545, + -0.36779484152793884, + 0.14787515997886658, + 1.6664400100708008, + -0.10343590378761292, + -0.03951066732406616, + -0.0016361922025680542, + -0.015768131241202354, + 0.12629695236682892, + -0.0589388832449913, + 0.9795559644699097, + 0.1183227002620697, + 0.03436177968978882, + 0.12721997499465942, + 0.04293711483478546, + -0.10107798874378204, + -0.02046598494052887, + 0.844606876373291, + -0.005056291818618774, + -0.05871495604515076 + ], + [ + 0.20549790561199188, + 0.06547529250383377, + -0.3145895302295685, + 0.036741018295288086, + 1.6055974960327148, + -0.26423338055610657, + -0.0729488730430603, + 0.034103766083717346, + 0.026352986693382263, + 0.12014830112457275, + 0.04075225442647934, + 0.8376508355140686, + 0.22271093726158142, + 0.052514754235744476, + 0.16109326481819153, + 0.061621490865945816, + 0.001104786992073059, + -0.05881549417972565, + 1.1016579866409302, + 0.029139339923858643, + 0.0015471577644348145 + ], + [ + 0.07018432766199112, + 0.06617987155914307, + -0.270182341337204, + 0.2668461501598358, + 1.4160938262939453, + -0.21099865436553955, + 0.026192456483840942, + 0.05559873580932617, + -0.033873192965984344, + 0.09704998880624771, + -0.11236021667718887, + 0.9734514951705933, + -0.11001318693161011, + 0.1813044548034668, + 0.172516331076622, + 0.04784749075770378, + -0.16102685034275055, + -0.04399195313453674, + 0.7409241795539856, + 0.07622691988945007, + -0.12989309430122375 + ], + [ + 0.09469524025917053, + 0.051173701882362366, + -0.27179792523384094, + 0.250503808259964, + 1.4205782413482666, + -0.22312268614768982, + 0.07304289937019348, + 0.08305234462022781, + -0.04132872074842453, + 0.07534755766391754, + -0.12687236070632935, + 0.9700124859809875, + -0.05576625466346741, + 0.16492176055908203, + 0.14910683035850525, + 0.05023576319217682, + -0.14562980830669403, + -0.07328973710536957, + 0.7361587285995483, + 0.0778588354587555, + -0.012201830744743347 + ] + ], + "generated_utilities": [ + 2.0, + 0.7235286831855774, + 0.7263574600219727, + 2.0, + 0.7248995304107666, + 2.0, + 0.7258209586143494, + 2.0 + ], + "group_id": "ms-c663f7e73e7e8376a05c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7233090400695801, + 0.7198824882507324, + 2.0, + 0.7120869159698486, + 0.27145418524742126, + 0.23090600967407227, + 0.2526220679283142, + 0.2656436562538147, + 0.264312207698822, + 0.27572086453437805, + 0.296668142080307, + 0.2128777951002121, + 0.22965002059936523, + 0.2196153998374939, + 0.22027263045310974, + 0.19939398765563965 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.038699887692928314, + -0.3685438632965088, + -1.3628361225128174, + 0.0675731897354126, + 2.4906270503997803, + 0.8971554040908813, + 0.06863358616828918, + 0.14689946174621582, + 0.0500560887157917, + 0.5852231383323669, + -0.5345680713653564, + 1.7433128356933594, + -0.5520187020301819, + 0.36285334825515747, + 0.06837204098701477, + 0.04091557487845421, + 0.26646411418914795, + -0.3870337903499603, + 1.7901486158370972, + -0.46956348419189453, + 0.5412080883979797 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.13726717233657837, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7257066965103149, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0824161767959595 + ], + [ + -0.09956050664186478, + -0.6439464092254639, + -1.5331499576568604, + -0.2275703251361847, + 2.2570624351501465, + 1.145668387413025, + -0.19933581352233887, + 0.15889883041381836, + -0.07524599134922028, + 0.6580387353897095, + -0.5754004716873169, + 1.6105921268463135, + -0.25637149810791016, + 0.37256744503974915, + 0.17931225895881653, + 0.286465048789978, + 0.07884949445724487, + -0.35266172885894775, + 1.548264980316162, + -0.2917717695236206, + 0.3307930827140808 + ], + [ + 0.13072630763053894, + -0.5927069783210754, + -1.4328954219818115, + 0.1252535879611969, + 2.5439467430114746, + 1.119347095489502, + -0.12134809792041779, + -0.09485669434070587, + -0.040146272629499435, + 0.4683093726634979, + -0.6791773438453674, + 2.1314187049865723, + -0.4368498623371124, + 0.5876930952072144, + 0.27027207612991333, + -0.1808907389640808, + 0.037005022168159485, + -0.6130290627479553, + 1.60811185836792, + -0.8427046537399292, + 0.5648807883262634 + ], + [ + -0.019807856529951096, + -0.40178078413009644, + -1.5710430145263672, + 0.3568149507045746, + 2.8693275451660156, + 1.2080440521240234, + 0.45903119444847107, + -0.07479061186313629, + 0.2587376832962036, + 0.37667450308799744, + -0.5607666373252869, + 1.9825769662857056, + -0.7874732613563538, + 0.48894837498664856, + -0.04161303490400314, + 0.2237108200788498, + 0.12043289840221405, + -0.8541218638420105, + 1.3131444454193115, + -0.8318716287612915, + 0.7653946876525879 + ], + [ + -0.039324238896369934, + 0.18316024541854858, + -1.2052042484283447, + -0.020045489072799683, + 2.441659927368164, + 1.3116326332092285, + 0.04364071041345596, + 0.617156982421875, + -0.5366188287734985, + 0.5837841629981995, + -0.6157102584838867, + 1.946446180343628, + -0.44197604060173035, + -0.12030342221260071, + 0.2635759711265564, + -0.34203875064849854, + 0.3788418769836426, + -0.01673591136932373, + 1.7612254619598389, + -0.8283271193504333, + 0.44542378187179565 + ], + [ + 0.020479684695601463, + -0.3728349208831787, + -1.2566280364990234, + -0.4903675317764282, + 2.7149407863616943, + 1.1410136222839355, + 0.18438707292079926, + 0.15751159191131592, + 0.5994289517402649, + 0.7441410422325134, + -0.5757432579994202, + 2.025078058242798, + -1.1044142246246338, + -0.24153441190719604, + 0.7623711824417114, + -0.07102327048778534, + 0.7491155862808228, + -0.853058397769928, + 1.1346031427383423, + -0.4604499638080597, + 0.3380756974220276 + ], + [ + -0.0438276082277298, + 0.21775567531585693, + -0.9180121421813965, + 0.2164689004421234, + 1.7669851779937744, + 0.7701845765113831, + -0.4171431064605713, + -0.44690096378326416, + -0.6473215818405151, + 0.5592721700668335, + 0.1537061333656311, + 1.2201651334762573, + -0.02862715721130371, + 0.759445309638977, + -0.08159622550010681, + -0.038715001195669174, + 0.8136870265007019, + -1.129607081413269, + 1.5099153518676758, + -1.2117259502410889, + 1.2496943473815918 + ], + [ + -0.6884663105010986, + -0.7670764923095703, + -1.3375780582427979, + 0.42161574959754944, + 2.4925031661987305, + 0.18111640214920044, + -0.4373469054698944, + 0.6228322386741638, + -0.3739543557167053, + 1.3811917304992676, + 0.05884122848510742, + 1.0582032203674316, + -0.38174134492874146, + 0.7282494902610779, + 0.9550073146820068, + -0.6478474736213684, + -0.467768132686615, + -0.22179017961025238, + 2.1456754207611084, + -0.39938730001449585, + 1.08339524269104 + ], + [ + 0.7042056322097778, + -0.4828507900238037, + -1.914047122001648, + -0.7084186673164368, + 1.87782621383667, + 0.42005008459091187, + 0.511604905128479, + -0.2043081521987915, + 0.9001545906066895, + 1.083567500114441, + -0.9796027541160583, + 1.3277345895767212, + -0.5564201474189758, + 0.6932777166366577, + -0.5934457778930664, + -0.296670138835907, + 0.14519014954566956, + -0.8612968325614929, + 0.8968887329101562, + -0.5584146976470947, + -0.434501588344574 + ], + [ + 0.19210004806518555, + 0.5024477243423462, + -0.4993329644203186, + -0.826042115688324, + 1.6430644989013672, + 0.9877098798751831, + 0.49057987332344055, + -0.13611945509910583, + 0.7228546738624573, + 1.4937405586242676, + 0.18012672662734985, + 2.188062906265259, + -0.6084107160568237, + 1.0874407291412354, + 0.5744171142578125, + 0.9375161528587341, + -0.43115919828414917, + 0.19787153601646423, + 2.652085781097412, + -0.529538631439209, + 0.03626328706741333 + ], + [ + -0.014053341001272202, + -1.351341724395752, + -0.7293765544891357, + 0.9410406947135925, + 3.139275074005127, + 1.3463419675827026, + -0.728021502494812, + -0.6353646516799927, + -0.19685329496860504, + 1.5315749645233154, + -1.2550475597381592, + 1.7411813735961914, + 0.14418578147888184, + 0.7038239240646362, + 0.1004495620727539, + 0.7590892910957336, + 0.9287852048873901, + -0.23726919293403625, + 0.8381044864654541, + -1.2906644344329834, + 0.9584261178970337 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03644278272986412, + -0.12137487530708313, + -0.3628361225128174, + -0.027813076972961426, + 1.4906270503997803, + -0.10284459590911865, + 0.0, + 0.15174832940101624, + 0.015595663338899612, + -0.10680150985717773, + -0.14723241329193115, + 0.7433128356933594, + 0.25961190462112427, + 0.0, + -0.23801246285438538, + -0.10784421861171722, + 0.09500086307525635, + -0.06586915254592896, + 0.7901486158370972, + -0.10978007316589355, + 0.0 + ] + ], + "predicted_scores": [ + 6.860178470611572, + 6.529614448547363, + 6.503523826599121, + 6.478070259094238, + 6.436008453369141, + 6.433952808380127, + 6.285012722015381, + 6.118747234344482 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1528740417/train0", + "selected_index": 0, + "state_hash": "c02fdfdd2339e4215d60b2221be925e066a7763f9e3bba3ff33a15f20e286f5a", + "stored_base_utility": 0.7233090400695801, + "task_id": "LiftPegUpright-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15342965722084045, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15342965722084045 + }, + "base_progress": 0.15342965722084045, + "base_success": false, + "base_utility": 0.15342965722084045, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.045715175569057465, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.045715175569057465 + }, + { + "contact_quality": null, + "progress": 0.05852393060922623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05852393060922623 + }, + { + "contact_quality": null, + "progress": 0.04664717987179756, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04664717987179756 + }, + { + "contact_quality": null, + "progress": 0.03780760616064072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03780760616064072 + }, + { + "contact_quality": null, + "progress": 0.4659159183502197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4659159183502197 + }, + { + "contact_quality": null, + "progress": 0.037175167351961136, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037175167351961136 + }, + { + "contact_quality": null, + "progress": 0.03381476551294327, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03381476551294327 + }, + { + "contact_quality": null, + "progress": 0.030572224408388138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030572224408388138 + } + ], + "candidate_progress": [ + 0.045715175569057465, + 0.05852393060922623, + 0.04664717987179756, + 0.03780760616064072, + 0.4659159183502197, + 0.037175167351961136, + 0.03381476551294327, + 0.030572224408388138 + ], + "candidate_source_chart_ids": [ + "ms-a509a572b3871e591d26", + "ms-b7e7fe66a6c86898eec2", + "ms-b7e7fe66a6c86898eec2", + "ms-4015c296239564108da2", + "ms-0541a144afe57b5a7ae6", + "ms-23a355317920e192147f", + "ms-307f9067d5b66f55b3a6", + "ms-5df54a8baaceb8c73c8a" + ], + "candidate_source_task_ids": [ + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "generated_tangents": [ + [ + -0.09744466841220856, + 0.1615401655435562, + 0.9181876182556152, + 0.28216755390167236, + -0.1380867063999176, + 0.039674438536167145, + 0.2712133526802063, + 0.0829060822725296, + -0.09420070052146912, + 0.19250546395778656, + 0.3357296586036682, + 0.1237170398235321, + 0.3213690221309662, + 1.6887140274047852, + -0.16327401995658875, + -0.020726287737488747, + 0.18956893682479858, + -0.3159496784210205, + -0.06023472547531128, + 0.05663737654685974, + 1.5617576837539673 + ], + [ + -0.38515937328338623, + 0.2430860847234726, + 0.5041823387145996, + 0.06428265571594238, + -0.2101939618587494, + -0.4898875653743744, + 0.08606752753257751, + 0.12442474067211151, + -0.08885777741670609, + 0.05913913995027542, + 0.6192091703414917, + 0.26889485120773315, + 0.7972333431243896, + 1.415450096130371, + 0.13711148500442505, + 0.030369684100151062, + 0.20247438549995422, + -0.16939528286457062, + 0.00013431906700134277, + 0.08190920948982239, + 1.030199646949768 + ], + [ + -0.4897669553756714, + 0.2485741823911667, + 0.7056241035461426, + 0.06932103633880615, + -0.1584480106830597, + -0.4563254415988922, + 0.1472950577735901, + 0.12362554669380188, + -0.07727830857038498, + -0.13273054361343384, + 0.633302628993988, + 0.3257061243057251, + 0.8085421323776245, + 1.2082523107528687, + 0.10343445837497711, + 0.1136644035577774, + 0.2867079973220825, + -0.35651665925979614, + 0.0778837502002716, + 0.22079920768737793, + 1.1263872385025024 + ], + [ + -0.3609389662742615, + -0.11747902631759644, + 1.1539137363433838, + -0.038696080446243286, + -0.13913333415985107, + 0.057155631482601166, + -0.043315112590789795, + 0.027488023042678833, + -0.12646590173244476, + 0.13201157748699188, + 0.3032550513744354, + 0.06060728430747986, + 0.6882154941558838, + 1.3331786394119263, + 0.08885516226291656, + -0.07271460443735123, + -0.029094234108924866, + -0.2840335965156555, + -0.14702251553535461, + 0.25668275356292725, + 1.2032619714736938 + ], + [ + -0.12941421568393707, + -0.04688846319913864, + 0.8783526420593262, + 0.02738828957080841, + -0.018859386444091797, + 0.4211578369140625, + -0.09172028303146362, + 0.1925777941942215, + -0.18124040961265564, + 0.15872693061828613, + 0.5672739744186401, + 0.41217777132987976, + 0.6639254093170166, + 1.3441522121429443, + 0.06883271038532257, + -0.0027641523629426956, + -0.07542546093463898, + -0.3838971257209778, + -0.18540611863136292, + 0.10263770818710327, + 1.2445440292358398 + ], + [ + -0.336281418800354, + -0.06685802340507507, + 1.120572805404663, + -0.13340243697166443, + -0.2491285502910614, + 0.11167621612548828, + 0.23930606245994568, + 0.05797678232192993, + -0.17424003779888153, + 0.1641796976327896, + 0.2597396671772003, + 0.1383146047592163, + 0.8167866468429565, + 1.2172259092330933, + 0.029573053121566772, + -0.06002848967909813, + -0.00965861976146698, + -0.29838794469833374, + -0.22452014684677124, + 0.2784825563430786, + 1.1556915044784546 + ], + [ + -0.40881097316741943, + -0.015014121308922768, + 1.1260370016098022, + -0.05635581910610199, + -0.17688068747520447, + 0.13964474201202393, + 0.06273573637008667, + 0.15282008051872253, + -0.12866325676441193, + 0.15834760665893555, + 0.42304787039756775, + 0.16547508537769318, + 0.7014168500900269, + 1.255672574043274, + 0.018889322876930237, + -0.04071212559938431, + 0.0684061199426651, + -0.2770296335220337, + -0.2071492075920105, + 0.1232648491859436, + 1.2043311595916748 + ], + [ + -0.03652302920818329, + -0.013153169304132462, + 0.9712647199630737, + 0.05203777551651001, + -0.2379392385482788, + 0.33988142013549805, + -0.003175705671310425, + 0.2110832929611206, + -0.18604235351085663, + 0.17053517699241638, + 0.5016647577285767, + 0.2015095353126526, + 0.6059832572937012, + 1.3119239807128906, + 0.06690624356269836, + 0.014003142714500427, + -0.04031161963939667, + -0.37628525495529175, + -0.19961538910865784, + 0.2973998785018921, + 1.0768033266067505 + ] + ], + "generated_utilities": [ + 0.045715175569057465, + 0.05852393060922623, + 0.04664717987179756, + 0.03780760616064072, + 0.4659159183502197, + 0.037175167351961136, + 0.03381476551294327, + 0.030572224408388138 + ], + "group_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.6040759086608887, + 2.0, + 0.14582239091396332, + 0.5723002552986145, + 0.10770338028669357, + 0.10128920525312424, + 0.08499570190906525, + 0.14736615121364594, + 0.07181884348392487, + 0.11773071438074112, + 0.06549974530935287, + 0.06568789482116699, + 0.10681070387363434, + 0.05432777479290962, + 0.0625491738319397, + 0.06991603225469589 + ], + "instruction": "Pick up the cube and move it to the goal position.", + "negative_tangents": [ + [ + -0.072060227394104, + 0.2043294906616211, + 0.8692872524261475, + 0.020746350288391113, + -0.004742175340652466, + 0.05779099464416504, + 0.0, + -0.0498790442943573, + -0.20218299329280853, + 0.41556206345558167, + -0.2306707501411438, + 0.32090064883232117, + -0.2089032530784607, + 0.0, + 0.05318737402558327, + -0.04637153446674347, + 0.04412045329809189, + 0.08670947700738907, + -0.10918038338422775, + -0.059010982513427734, + 0.0 + ], + [ + -0.5282122492790222, + 1.204329490661621, + 1.8692872524261475, + -0.8456907868385315, + 0.42858073115348816, + -0.3134727478027344, + -2.5308754444122314, + 0.1655428111553192, + -0.12024828046560287, + 0.0010203085839748383, + 0.2592625319957733, + 0.2679504156112671, + 0.25922709703445435, + 2.851924419403076, + 0.0039860643446445465, + -0.014717570506036282, + 0.01173020526766777, + 0.10578173398971558, + 0.05138798803091049, + 0.3216937780380249, + 2.9243662357330322 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.061750888824463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.703848838806152, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.8487324714660645 + ], + [ + -0.6558206081390381, + 1.3553937673568726, + 2.057647466659546, + -1.0405915975570679, + 0.2629950940608978, + -0.28233209252357483, + -2.589219093322754, + -0.06961745023727417, + 0.02406928688287735, + 0.2341437190771103, + 0.49766814708709717, + 0.4930538535118103, + 0.370309442281723, + 3.111422061920166, + 0.06151486933231354, + -0.2028706818819046, + 0.12007936835289001, + 0.2164750099182129, + 0.16723006963729858, + 0.26430171728134155, + 3.0643396377563477 + ], + [ + -0.18174728751182556, + 0.8890945315361023, + 1.5388343334197998, + -0.4532086253166199, + 0.3864310085773468, + -0.45410382747650146, + -2.623199462890625, + 0.5360933542251587, + -0.04706605523824692, + 0.11160072684288025, + 0.5460608005523682, + 0.39992716908454895, + 0.17028246819972992, + 2.8760411739349365, + 0.29450932145118713, + -0.3028962016105652, + 0.06744809448719025, + 0.1687500774860382, + 0.42097336053848267, + 0.3965923488140106, + 3.28550386428833 + ], + [ + -0.4313088357448578, + 0.9752410054206848, + 2.0068275928497314, + -0.5783756375312805, + 0.5761726498603821, + -0.5289677381515503, + -2.321643590927124, + 0.0023015737533569336, + -0.07372284680604935, + 0.05664224550127983, + 0.03747282922267914, + 0.4900178611278534, + 0.31588616967201233, + 3.018609046936035, + 0.12529893219470978, + -0.2872275412082672, + -0.2500917911529541, + 0.0006694421172142029, + -0.39133700728416443, + 0.6382719874382019, + 2.810041904449463 + ], + [ + -0.5560051202774048, + 1.4042110443115234, + 1.662226915359497, + -1.4059122800827026, + 0.8242433071136475, + -0.10998553037643433, + -2.7556281089782715, + -0.26875895261764526, + -0.4201813042163849, + -0.371376633644104, + 0.7718684673309326, + 0.6452833414077759, + 0.027421817183494568, + 2.3879575729370117, + -0.47588983178138733, + -0.045049820095300674, + 0.0838315486907959, + 0.10257662832736969, + 0.4181908369064331, + 0.2650004029273987, + 3.006909132003784 + ], + [ + -0.4896228015422821, + 1.0570447444915771, + 1.408416509628296, + -0.7269805669784546, + 1.023213267326355, + -0.22378163039684296, + -2.1348509788513184, + -0.4423682987689972, + 0.26834720373153687, + -0.5835619568824768, + 0.3675343692302704, + 0.3307996988296509, + -0.01537257432937622, + 3.4422543048858643, + 0.3622148334980011, + -0.26806333661079407, + 0.4238347113132477, + 0.2583734393119812, + 0.10431534051895142, + 0.2077663242816925, + 2.446356773376465 + ], + [ + -1.1077957153320312, + 0.4289678931236267, + 1.9882663488388062, + -0.3021383285522461, + -0.006195276975631714, + -0.22295325994491577, + -1.8697566986083984, + 0.9590625762939453, + -0.0024877265095710754, + -0.3088582456111908, + 0.6517632007598877, + 0.4271944761276245, + 0.34532302618026733, + 2.704721450805664, + 0.7557594776153564, + -0.8073300719261169, + -0.7479258179664612, + -0.19068273901939392, + 0.14240311086177826, + 0.2233867347240448, + 2.1774275302886963 + ], + [ + -0.9354162812232971, + 1.5971527099609375, + 1.0168101787567139, + -0.7164653539657593, + 0.519247829914093, + -0.7477964162826538, + -3.338376045227051, + 0.7212944030761719, + -0.6477807760238647, + 0.38193029165267944, + -0.1568622589111328, + 0.47457069158554077, + -0.30742186307907104, + 2.6848907470703125, + -0.4251534044742584, + 0.6623334884643555, + 0.03348249942064285, + -0.2996619939804077, + -0.8319172263145447, + 0.7883892059326172, + 3.5870752334594727 + ], + [ + -0.44698429107666016, + 0.3201255202293396, + 1.5405346155166626, + -0.9956957101821899, + 0.8944320678710938, + -0.4196317791938782, + -2.010188102722168, + 0.7199084758758545, + -0.49276289343833923, + 0.4512734115123749, + 0.48276829719543457, + 0.22657281160354614, + 0.8205416798591614, + 3.6653342247009277, + 0.6818231344223022, + 0.4660317897796631, + -0.14916014671325684, + 0.5328372716903687, + -0.5979922413825989, + 0.3124805986881256, + 3.110726833343506 + ], + [ + -0.5630171298980713, + 1.934722900390625, + 1.4954721927642822, + -1.684873342514038, + 0.133146733045578, + -0.4359630346298218, + -2.681238889694214, + 0.8904907703399658, + -0.48221057653427124, + -0.7635119557380676, + -0.23804441094398499, + 0.5691602230072021, + -0.23042798042297363, + 3.760842800140381, + 0.3684825003147125, + -0.20475278794765472, + 0.04838913306593895, + 0.17439094185829163, + 0.07042267918586731, + -0.1653938889503479, + 3.844882011413574 + ], + [ + -0.3705383241176605, + 1.5082495212554932, + 0.9002141356468201, + -1.3291171789169312, + 0.15676641464233398, + -1.2286558151245117, + -1.7006758451461792, + 0.34438908100128174, + 0.32962289452552795, + -0.6556114554405212, + 0.3502740263938904, + 0.7518025636672974, + -0.49080890417099, + 2.10355806350708, + 0.36426910758018494, + -0.4822288453578949, + 0.7496572732925415, + 0.43558254837989807, + 0.34650692343711853, + 0.9977220296859741, + 1.9853770732879639 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.005563676357269287, + 0.2043294906616211, + 0.8692872524261475, + 0.08163481950759888, + 0.05995750427246094, + 0.012857973575592041, + 0.0, + 0.01583634316921234, + 0.04724548012018204, + 0.018815450370311737, + 0.02742627263069153, + -0.03381502628326416, + 0.007930099964141846, + 0.0, + 0.01774722896516323, + 0.08064591884613037, + -0.06814983487129211, + -0.002621002495288849, + 0.05759798362851143, + 0.05164051055908203, + 0.0 + ] + ], + "predicted_scores": [ + 3.5102407932281494, + 3.406304121017456, + 2.9733526706695557, + 2.7440531253814697, + 2.741786003112793, + 2.675002098083496, + 2.5794975757598877, + 2.283445358276367 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1273247072/train0", + "selected_index": 0, + "state_hash": "237fafab6bfdb561d66c862026f803fbd19d4007f6313bdb3d79b6dd74f69327", + "stored_base_utility": 0.6040759086608887, + "task_id": "PickCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.050487834960222244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.050487834960222244 + }, + "base_progress": 0.050487834960222244, + "base_success": false, + "base_utility": 0.050487834960222244, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0001621246337890625, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0001621246337890625 + }, + { + "contact_quality": null, + "progress": 6.29425048828125e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 6.29425048828125e-05 + }, + { + "contact_quality": null, + "progress": 0.10238292068243027, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10238292068243027 + }, + { + "contact_quality": null, + "progress": 0.019885918125510216, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019885918125510216 + }, + { + "contact_quality": null, + "progress": 0.0074851117096841335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0074851117096841335 + }, + { + "contact_quality": null, + "progress": 0.0004499355854932219, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0004499355854932219 + }, + { + "contact_quality": null, + "progress": 0.0001078049317584373, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0001078049317584373 + }, + { + "contact_quality": null, + "progress": 0.1601850390434265, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1601850390434265 + } + ], + "candidate_progress": [ + 0.0001621246337890625, + 6.29425048828125e-05, + 0.10238292068243027, + 0.019885918125510216, + 0.0074851117096841335, + 0.0004499355854932219, + 0.0001078049317584373, + 0.1601850390434265 + ], + "candidate_source_chart_ids": [ + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0345d8ac530f88cefd61", + "generated_tangents": [ + [ + 0.477573037147522, + -0.08181564509868622, + -0.2302217185497284, + 0.2649448812007904, + 0.41885775327682495, + 0.2656289339065552, + 0.6136137247085571, + 0.04889358580112457, + 0.3357800543308258, + 0.522717297077179, + -0.8441930413246155, + -0.06290313601493835, + -0.47927236557006836, + -0.05249950289726257, + -0.43283581733703613, + 0.1604544073343277, + 0.19137689471244812, + -0.25409168004989624, + -1.2012383937835693, + 0.6015713810920715, + 0.19387075304985046 + ], + [ + 0.546461284160614, + -0.1376083493232727, + -0.19663384556770325, + 0.12129625678062439, + 0.6050100326538086, + 0.44333183765411377, + 0.3731582760810852, + 0.0072501301765441895, + 0.3092706501483917, + 0.6923475861549377, + -0.7566579580307007, + -0.2266210913658142, + -0.4289129078388214, + -0.2520459294319153, + -0.5079135894775391, + 0.3172563314437866, + 0.28310734033584595, + -0.18701614439487457, + -1.145477056503296, + 0.4701024889945984, + 0.06293369829654694 + ], + [ + 0.2231089174747467, + -0.053958710283041, + -0.13760390877723694, + -0.0734052062034607, + 0.2534986138343811, + 0.212349072098732, + 0.2112533450126648, + 0.20914889872074127, + 0.3499075174331665, + 0.6650373935699463, + -0.7289736270904541, + -0.5831161141395569, + -0.6725144386291504, + -0.4102931022644043, + -0.525953471660614, + 0.22492794692516327, + 0.29086461663246155, + -0.22250987589359283, + -1.1890788078308105, + 0.7573410868644714, + -0.09697198867797852 + ], + [ + 0.4393550753593445, + -0.09524684399366379, + -0.21939662098884583, + 0.29503098130226135, + 0.26314741373062134, + 0.1890282928943634, + 0.6083811521530151, + 0.03921709954738617, + 0.2885315418243408, + 0.44384944438934326, + -0.8744981288909912, + -0.04786786437034607, + -0.24640974402427673, + -0.032241448760032654, + -0.4413410425186157, + 0.21452073752880096, + 0.24054858088493347, + -0.11529876291751862, + -1.2987656593322754, + 0.48039329051971436, + 0.1959494650363922 + ], + [ + 0.3388286828994751, + 0.0007252991199493408, + 0.025149255990982056, + 0.20357713103294373, + 0.326875776052475, + 0.32344821095466614, + 0.050565630197525024, + -0.12077262997627258, + 0.42205339670181274, + 0.5342826843261719, + -0.8682466745376587, + -0.059065788984298706, + 0.040771037340164185, + -0.3112889528274536, + -0.37888073921203613, + 0.17731750011444092, + -0.2201598435640335, + -0.07172225415706635, + -1.107750415802002, + 0.061236560344696045, + -0.14528051018714905 + ], + [ + 0.47605276107788086, + -0.05462750792503357, + -0.3020217716693878, + 0.39696410298347473, + 0.5762397050857544, + 0.39844024181365967, + 0.731613278388977, + -0.08021789789199829, + 0.29469701647758484, + 0.6287778615951538, + -1.0296783447265625, + 0.015520885586738586, + -0.45798254013061523, + 0.346148818731308, + -0.4485040307044983, + 0.14196041226387024, + 0.12641339004039764, + -0.2951399087905884, + -1.1279394626617432, + 0.3661571741104126, + 0.20907360315322876 + ], + [ + 0.39464622735977173, + -0.020881781354546547, + 0.015634357929229736, + 0.3644595444202423, + 0.3951271176338196, + 0.0026221275329589844, + 0.37702199816703796, + -0.153532475233078, + 0.2507711350917816, + 0.4081285297870636, + -1.0615930557250977, + -0.07481244206428528, + -0.24236957728862762, + 0.34386101365089417, + -0.4009522497653961, + 0.17217549681663513, + 0.013853684067726135, + -0.14843608438968658, + -1.3126630783081055, + 0.2959710955619812, + 0.19673213362693787 + ], + [ + 0.44646137952804565, + -0.12067033350467682, + 0.08336853981018066, + 0.5232672691345215, + 0.3084275722503662, + 0.2186690866947174, + 0.5163891315460205, + -0.17833521962165833, + 0.25861865282058716, + 0.39861077070236206, + -1.1110869646072388, + -0.07697618007659912, + -0.2048243284225464, + 0.24569550156593323, + -0.435116171836853, + 0.05934559181332588, + -0.07593880593776703, + -0.29134851694107056, + -1.3166427612304688, + 0.047914475202560425, + 0.12264642119407654 + ] + ], + "generated_utilities": [ + 0.0001621246337890625, + 6.29425048828125e-05, + 0.10238292068243027, + 0.019885918125510216, + 0.0074851117096841335, + 0.0004499355854932219, + 0.0001078049317584373, + 0.1601850390434265 + ], + "group_id": "ms-0345d8ac530f88cefd61", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.007898053154349327, + 0.0058197579346597195, + 0.052657388150691986, + 0.016109248623251915, + 0.1698339879512787, + 0.008177022449672222, + 0.12333428859710693, + 0.1150994747877121, + 0.15141737461090088, + 0.08515926450490952, + 0.1523766964673996, + 0.04073335975408554, + 0.027175527065992355, + 0.1798098385334015, + 0.08658158779144287, + 0.0618208646774292 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2244938164949417, + -0.708204448223114, + 1.4058912992477417, + 0.19480958580970764, + 1.1751772165298462, + -1.2265020608901978, + -0.8636159896850586, + -0.07021944224834442, + 1.2708500623703003, + 1.0246398448944092, + -1.7615737915039062, + -0.44495686888694763, + -1.2927910089492798, + 0.27074551582336426, + -1.6349679231643677, + 0.9748592376708984, + 1.0210484266281128, + 0.36906859278678894, + -2.528050661087036, + 0.7326674461364746, + 1.553327202796936 + ], + [ + 0.35595428943634033, + -0.8224175572395325, + 1.1192705631256104, + 0.4673205614089966, + 1.1115962266921997, + -1.4676826000213623, + -1.057816505432129, + -0.008461203426122665, + 0.9729399085044861, + 0.8019976615905762, + -1.4944581985473633, + -0.21817980706691742, + -1.3722625970840454, + 0.3557305932044983, + -1.8202308416366577, + 1.2552388906478882, + 0.7855625748634338, + 0.5709130167961121, + -2.4140331745147705, + 0.9063631296157837, + 1.282220721244812 + ], + [ + 0.19927366077899933, + -0.7607958912849426, + 1.4928451776504517, + 0.38265493512153625, + 1.2970150709152222, + -1.2964622974395752, + -0.5547049045562744, + -0.2585906684398651, + 1.3020356893539429, + 1.0737547874450684, + -1.5593467950820923, + -0.1617565155029297, + -1.4868489503860474, + 0.15243440866470337, + -1.4100472927093506, + 0.7991768717765808, + 1.0339381694793701, + 0.6228882074356079, + -2.8132376670837402, + 0.7685560584068298, + 1.8755284547805786 + ], + [ + 0.6393168568611145, + -0.2793915271759033, + 1.4690577983856201, + 0.4745514690876007, + 0.9989874362945557, + -0.7359552979469299, + -1.2171094417572021, + -0.39386802911758423, + 1.1646839380264282, + 0.6661151051521301, + -1.9162347316741943, + -0.43809595704078674, + -1.3973355293273926, + 0.7664861679077148, + -1.553026795387268, + 1.1995127201080322, + 1.285676121711731, + 0.822955846786499, + -2.574894666671753, + 0.3542059361934662, + 1.5986168384552002 + ], + [ + 0.26509445905685425, + -0.5443360805511475, + 1.2502005100250244, + 0.6112543940544128, + 0.924185037612915, + -1.0554168224334717, + -0.7927505373954773, + -0.07152043282985687, + 0.9228929281234741, + 1.545881986618042, + -1.6973015069961548, + -0.496639221906662, + -0.8727940320968628, + 0.5596445202827454, + -2.1939897537231445, + 1.239556074142456, + 1.4699029922485352, + -0.182618647813797, + -2.7263355255126953, + 0.976334810256958, + 1.0402796268463135 + ], + [ + 0.1353418529033661, + -1.1921336650848389, + 1.6913124322891235, + -0.43279919028282166, + 1.1331026554107666, + -1.8503069877624512, + -0.46699997782707214, + 0.5785945653915405, + 1.5516390800476074, + 0.38185298442840576, + -2.192314386367798, + -0.6304612159729004, + -1.6237419843673706, + -0.06383657455444336, + -1.2674099206924438, + 0.7967972755432129, + 1.2453874349594116, + 0.01532512903213501, + -2.6341590881347656, + 0.8366512656211853, + 1.9056174755096436 + ], + [ + 1.0916199684143066, + 0.23665857315063477, + 1.7737994194030762, + -0.8046588897705078, + 1.944507360458374, + -1.3606563806533813, + -0.022103488445281982, + 0.5769228935241699, + 1.0605480670928955, + 1.3100652694702148, + -1.9243974685668945, + 0.43040379881858826, + -1.9046732187271118, + -0.49132758378982544, + -1.9095335006713867, + 1.8850300312042236, + 1.366606593132019, + 0.37651723623275757, + -2.86037540435791, + 0.7753443717956543, + 2.072996139526367 + ], + [ + 0.7277938723564148, + 0.23822063207626343, + 0.4208102226257324, + -0.18717050552368164, + 1.1951282024383545, + -1.1622966527938843, + -0.6617333292961121, + -0.6339871883392334, + 0.8901646137237549, + 0.9977425932884216, + -1.8568902015686035, + 0.3291373550891876, + -0.9377886056900024, + 1.0705609321594238, + -0.90934818983078, + 1.7013394832611084, + 1.5952999591827393, + -0.05327540636062622, + -2.546459197998047, + 0.15792781114578247, + 1.2277774810791016 + ], + [ + 0.9283657073974609, + -1.421413779258728, + 1.6497737169265747, + -0.1984737515449524, + 0.9705085158348083, + -0.8917888402938843, + 0.026184558868408203, + 0.09181642532348633, + 0.6335604786872864, + 1.6531281471252441, + -1.4918432235717773, + -0.01735416054725647, + -0.5425058603286743, + 0.7866935133934021, + -0.7011347413063049, + 0.9797741770744324, + 1.6596866846084595, + -0.13051137328147888, + -2.692556858062744, + -0.05617690086364746, + 0.9771407842636108 + ] + ], + "predicted_scores": [ + 8.901857376098633, + 8.735929489135742, + 8.728458404541016, + 8.656224250793457, + 7.982328414916992, + 7.478723049163818, + 6.688599109649658, + 6.352318286895752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train0", + "selected_index": 0, + "state_hash": "1ca7c651b2dc4638488078318fe8aa293aa6f6da78672d440752daa9f2d4e8a3", + "stored_base_utility": 0.007898053154349327, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07743936777114868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07743936777114868 + }, + "base_progress": 0.07743936777114868, + "base_success": false, + "base_utility": 0.07743936777114868, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008875548839569092, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008875548839569092 + }, + { + "contact_quality": null, + "progress": 0.009828686714172363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009828686714172363 + }, + { + "contact_quality": null, + "progress": 0.0055730342864990234, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0055730342864990234 + }, + { + "contact_quality": null, + "progress": 0.008734583854675293, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008734583854675293 + }, + { + "contact_quality": null, + "progress": 0.008340597152709961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008340597152709961 + }, + { + "contact_quality": null, + "progress": 0.006452083587646484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006452083587646484 + }, + { + "contact_quality": null, + "progress": 0.007816612720489502, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007816612720489502 + }, + { + "contact_quality": null, + "progress": 0.007453759666532278, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007453759666532278 + } + ], + "candidate_progress": [ + 0.008875548839569092, + 0.009828686714172363, + 0.0055730342864990234, + 0.008734583854675293, + 0.008340597152709961, + 0.006452083587646484, + 0.007816612720489502, + 0.007453759666532278 + ], + "candidate_source_chart_ids": [ + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-02311c7783bc67e188b4", + "ms-02311c7783bc67e188b4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-0819eb89e627595e7e1b", + "generated_tangents": [ + [ + -0.17152051627635956, + -0.038122884929180145, + 1.340081810951233, + 0.47131815552711487, + 0.9815719127655029, + 1.6158288717269897, + 1.3484032154083252, + -0.5256087779998779, + -0.18259528279304504, + 1.419359803199768, + 0.17800158262252808, + 0.598058819770813, + 2.6912803649902344, + 1.049239158630371, + -0.5156311392784119, + -0.058238785713911057, + 0.9955028891563416, + 0.6501286029815674, + 0.5224418640136719, + 1.1980977058410645, + 0.792129635810852 + ], + [ + -0.18255217373371124, + -0.05061272904276848, + 1.2194602489471436, + 0.4751015603542328, + 0.9917681217193604, + 1.566107153892517, + 1.3416451215744019, + -0.4890477955341339, + -0.17969372868537903, + 1.2580264806747437, + 0.10395483672618866, + 0.5031521916389465, + 2.7228047847747803, + 0.972133457660675, + -0.5049965381622314, + -0.07441367208957672, + 0.9628568291664124, + 0.6687726974487305, + 0.40544599294662476, + 1.200563669204712, + 0.7176809310913086 + ], + [ + -0.45789438486099243, + -0.09919900447130203, + 0.6399000883102417, + 0.706047773361206, + 0.4280887842178345, + 1.7801594734191895, + 1.6320436000823975, + -0.38427188992500305, + -0.41299012303352356, + 1.3940125703811646, + 0.6254867911338806, + 0.8550894856452942, + 3.0050437450408936, + 2.065139055252075, + -0.4582861065864563, + -0.22844375669956207, + 0.9927664399147034, + 0.47996097803115845, + 0.6306631565093994, + 1.4493701457977295, + 0.7380000352859497 + ], + [ + -0.35566914081573486, + 0.007236521691083908, + 1.0994839668273926, + 0.7574254274368286, + 1.263378381729126, + 1.4886584281921387, + 1.5016376972198486, + -0.5011583566665649, + -0.3243262767791748, + 1.4724236726760864, + 0.18251696228981018, + 0.6980839967727661, + 2.178234338760376, + 1.044251799583435, + -0.28006064891815186, + -0.06012527272105217, + 0.7044966816902161, + 0.8577057719230652, + 0.32886219024658203, + 0.8952638506889343, + 0.3278191089630127 + ], + [ + -0.3057391047477722, + 0.004349537193775177, + 1.0532543659210205, + 0.7294998168945312, + 1.3418724536895752, + 1.5626895427703857, + 1.5369725227355957, + -0.6195144653320312, + -0.22753973305225372, + 1.4388295412063599, + 0.13561950623989105, + 0.769778311252594, + 2.3072755336761475, + 1.0169832706451416, + -0.24158892035484314, + -0.18682266771793365, + 0.6608469486236572, + 0.7943220734596252, + 0.401975154876709, + 0.9778363108634949, + 0.5728996992111206 + ], + [ + -0.26647835969924927, + -0.11081273853778839, + 0.4576881229877472, + 0.583040714263916, + 1.4537286758422852, + 1.577552080154419, + 1.454306960105896, + -0.5423310995101929, + -0.32421642541885376, + 1.5886645317077637, + -0.06395097821950912, + 1.2373117208480835, + 2.02956223487854, + 0.7510688304901123, + -0.2381059229373932, + -0.17396648228168488, + 0.7958939075469971, + 0.6796910762786865, + 0.395303875207901, + 1.1233546733856201, + 0.04672019183635712 + ], + [ + -0.20528943836688995, + 0.10780379176139832, + 1.2915796041488647, + 0.19357836246490479, + 0.6346437931060791, + 1.8733876943588257, + 0.9660515785217285, + -0.44898679852485657, + -0.28768423199653625, + 1.721643090248108, + 0.4985387921333313, + 0.7298404574394226, + 2.0187413692474365, + 0.9921810030937195, + 0.08743637055158615, + -0.28891173005104065, + 0.9795303344726562, + 0.334699809551239, + 0.758746862411499, + 0.8109932541847229, + 0.305303156375885 + ], + [ + -0.3197581171989441, + 0.12134270370006561, + 1.3069734573364258, + 0.22892767190933228, + 0.7294712662696838, + 1.7926363945007324, + 0.8312811255455017, + -0.5016231536865234, + -0.31257396936416626, + 1.6537065505981445, + 0.5000704526901245, + 0.818077802658081, + 1.9235714673995972, + 1.0741196870803833, + 0.11200298368930817, + -0.30520081520080566, + 0.9781718850135803, + 0.3617241382598877, + 0.7511255741119385, + 0.8701092600822449, + 0.3813852071762085 + ] + ], + "generated_utilities": [ + 0.008875548839569092, + 0.009828686714172363, + 0.0055730342864990234, + 0.008734583854675293, + 0.008340597152709961, + 0.006452083587646484, + 0.007816612720489502, + 0.007453759666532278 + ], + "group_id": "ms-0819eb89e627595e7e1b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07638250291347504, + 0.11176031827926636, + 0.139996737241745, + 0.06282445043325424, + 0.08466696739196777, + 0.0763862133026123, + 0.05953681468963623, + 0.06854937970638275, + 0.06834080070257187, + 0.09309522807598114, + 0.14572584629058838, + 0.07182137668132782, + 0.020047685131430626, + 0.05106111615896225, + 0.03756682202219963, + 0.061605654656887054 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.04522436857223511, + -0.3834587037563324, + 2.32004976272583, + 1.6386330127716064, + 1.508955717086792, + 1.681371808052063, + 2.9060468673706055, + -0.5984516143798828, + 0.36428770422935486, + 0.7442611455917358, + 1.163742184638977, + 0.4921322464942932, + 1.9586588144302368, + 3.335318088531494, + -0.02929070219397545, + 0.5442408919334412, + 2.4529502391815186, + 0.17471881210803986, + 1.804263710975647, + 1.5498225688934326, + 1.9392809867858887 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.19627264142036438, + 0.015629824250936508, + 0.723259687423706, + 0.0313839316368103, + 0.02149224281311035, + 0.7142634391784668, + 0.0, + -0.29767167568206787, + -0.10304851830005646, + 0.6168417930603027, + 0.0271989107131958, + 0.369077205657959, + 1.5924487113952637, + 0.0, + -0.11617732048034668, + -0.1419914811849594, + 0.9400725364685059, + -0.04139697551727295, + 0.7166774272918701, + 1.092864990234375, + 0.0 + ], + [ + -0.18197131156921387, + 0.30914804339408875, + 1.4557228088378906, + 1.4375734329223633, + 0.9600651264190674, + 1.7448713779449463, + 3.031123399734497, + -0.6073551177978516, + -0.4749099016189575, + 1.3016932010650635, + 0.32392996549606323, + 0.6961254477500916, + 2.1961381435394287, + 3.730100154876709, + -0.6154190897941589, + -0.3908892273902893, + 2.0994153022766113, + 0.18937554955482483, + 1.5752670764923096, + 2.2457275390625, + 1.5629570484161377 + ] + ], + "predicted_scores": [ + 7.262422561645508, + 7.140332221984863, + 6.653880596160889, + 6.33031702041626, + 5.919139862060547, + 5.892663955688477, + 4.470665454864502, + 4.338894367218018 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train0", + "selected_index": 0, + "state_hash": "582daf047d78a1b01018d54f5798143c4be6129fd7b6cbc025fc85818e519f3c", + "stored_base_utility": 0.07638250291347504, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19400137662887573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19400137662887573 + }, + "base_progress": 0.19400137662887573, + "base_success": false, + "base_utility": 0.19400137662887573, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03749382495880127, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03749382495880127 + }, + { + "contact_quality": null, + "progress": 0.04604196548461914, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04604196548461914 + }, + { + "contact_quality": null, + "progress": 0.03564453125, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03564453125 + }, + { + "contact_quality": null, + "progress": 0.0471922941505909, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0471922941505909 + }, + { + "contact_quality": null, + "progress": 0.033758122473955154, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.033758122473955154 + }, + { + "contact_quality": null, + "progress": 0.04913504794239998, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04913504794239998 + }, + { + "contact_quality": null, + "progress": 0.04759601876139641, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04759601876139641 + }, + { + "contact_quality": null, + "progress": 0.0932278037071228, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0932278037071228 + } + ], + "candidate_progress": [ + 0.03749382495880127, + 0.04604196548461914, + 0.03564453125, + 0.0471922941505909, + 0.033758122473955154, + 0.04913504794239998, + 0.04759601876139641, + 0.0932278037071228 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "generated_tangents": [ + [ + 1.8905562162399292, + 0.3944854736328125, + -1.7180052995681763, + -0.1349945217370987, + -0.5599319934844971, + -4.130958080291748, + -2.674062490463257, + -1.320008635520935, + 0.0017402367666363716, + 1.5784907341003418, + 1.1209584474563599, + -0.13940736651420593, + 0.5197409391403198, + -0.3171689510345459, + -0.6695423722267151, + -0.19212783873081207, + 0.45296603441238403, + 1.6345059871673584, + 0.11391966044902802, + 0.6570355892181396, + 0.7228900194168091 + ], + [ + 1.8707126379013062, + 0.41409242153167725, + -1.839680790901184, + 0.03306730091571808, + -0.2336004674434662, + -4.06272554397583, + -2.651236057281494, + -1.1379244327545166, + -0.06307099014520645, + 1.6282907724380493, + 1.0120092630386353, + 0.0491805374622345, + 0.14716574549674988, + -0.8551839590072632, + -0.5584245324134827, + -0.11800333857536316, + 0.24090628325939178, + 1.3936920166015625, + 0.4446020722389221, + 0.44069159030914307, + 0.3575472831726074 + ], + [ + 2.0141654014587402, + 0.5335431098937988, + -1.2890888452529907, + -0.20137447118759155, + -0.5691367983818054, + -4.134076118469238, + -3.181198835372925, + -1.1823093891143799, + 0.06391821801662445, + 1.7136962413787842, + 1.2516483068466187, + 0.49389082193374634, + 0.5434608459472656, + -0.962183952331543, + -0.812558650970459, + -0.17930343747138977, + 0.3569537401199341, + 1.2693164348602295, + 0.5134433507919312, + 1.1229405403137207, + 0.39776551723480225 + ], + [ + 1.9669517278671265, + 0.518299400806427, + -1.7549928426742554, + -0.16333818435668945, + -0.5614626407623291, + -3.7372679710388184, + -3.6689260005950928, + -1.1913681030273438, + 0.04750648885965347, + 1.492919683456421, + 1.0730613470077515, + 0.5416345596313477, + 0.21158185601234436, + -0.44773614406585693, + -0.7684616446495056, + -0.14286024868488312, + 0.25122955441474915, + 1.5649583339691162, + 0.3667457103729248, + 0.665382444858551, + 0.47018304467201233 + ], + [ + 1.797508955001831, + 0.5287662148475647, + -1.608214259147644, + -0.18494614958763123, + -0.8209599852561951, + -3.972693920135498, + -3.305769205093384, + -1.0902042388916016, + -0.05148264020681381, + 1.5328097343444824, + 1.3380097150802612, + 0.44910597801208496, + 0.16804906725883484, + -0.22458527982234955, + -0.8841680288314819, + -0.2944192886352539, + 0.6147395372390747, + 1.554239273071289, + 0.5591936707496643, + 0.4726758599281311, + 0.06745950877666473 + ], + [ + 1.8798527717590332, + 0.47933393716812134, + -1.692487120628357, + -0.22272565960884094, + -0.6054636240005493, + -3.8631744384765625, + -3.3329451084136963, + -1.049681544303894, + 0.03719450533390045, + 1.4770680665969849, + 1.1387784481048584, + 0.5392275452613831, + 0.28753021359443665, + -0.5957773923873901, + -0.7329325079917908, + -0.2532622516155243, + 0.3489782512187958, + 1.4910532236099243, + 0.4014784097671509, + 0.6709848642349243, + 0.2531224489212036 + ], + [ + 1.7510665655136108, + 0.48223114013671875, + -1.705487847328186, + -0.32500597834587097, + -0.6420989036560059, + -3.8617329597473145, + -3.319277048110962, + -1.0553152561187744, + 0.03045062907040119, + 1.4631993770599365, + 1.137038230895996, + 0.5703677535057068, + 0.2730879485607147, + -0.5021155476570129, + -0.7537959814071655, + -0.2491789013147354, + 0.31574124097824097, + 1.545933485031128, + 0.5048801302909851, + 0.7393592000007629, + 0.5303146839141846 + ], + [ + 1.8198521137237549, + 0.3624112010002136, + -1.5549098253250122, + -0.5984248518943787, + -0.322936475276947, + -3.43082857131958, + -3.303149700164795, + -1.0558255910873413, + 0.124747633934021, + 1.0588371753692627, + 1.1908138990402222, + 0.7646531462669373, + 0.4460482895374298, + -0.20368830859661102, + -0.6139333248138428, + -0.40287846326828003, + 0.13066786527633667, + 1.6527705192565918, + 0.13463273644447327, + 0.49561959505081177, + 0.5370721817016602 + ] + ], + "generated_utilities": [ + 0.03749382495880127, + 0.04604196548461914, + 0.03564453125, + 0.0471922941505909, + 0.033758122473955154, + 0.04913504794239998, + 0.04759601876139641, + 0.0932278037071228 + ], + "group_id": "ms-0dee962a5901fb8b5bbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23161475360393524, + 0.23611454665660858, + 0.1380448043346405, + 2.0, + 0.25444933772087097, + 0.25831863284111023, + 0.22753827273845673, + 0.2483263909816742, + 0.21622240543365479, + 0.17534737288951874, + 0.23816657066345215, + 0.09714888036251068, + 0.05844185873866081, + 0.12679120898246765, + 0.0472835898399353, + 0.03277532383799553 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7612967491149902, + 0.192471444606781, + -0.2699308395385742, + 0.017076104879379272, + -0.14400714635849, + -1.9032442569732666, + 0.0, + -0.28746497631073, + -0.2857179641723633, + -0.09024286270141602, + -0.03538453206419945, + 0.06432074308395386, + -0.2535901665687561, + 0.0, + 0.06180626153945923, + 0.16256888210773468, + -0.031329914927482605, + 1.563896656036377, + 0.04795600473880768, + 0.02485179901123047, + 0.0 + ], + [ + 1.2132649421691895, + 0.536715567111969, + -1.4273607730865479, + -1.0538015365600586, + -0.5170869827270508, + -3.4203505516052246, + -2.3889083862304688, + -0.7717514634132385, + -1.1465628147125244, + 0.8393621444702148, + -0.4419952630996704, + -0.23069629073143005, + -0.0637764036655426, + -1.5193647146224976, + -1.0273863077163696, + 0.39812424778938293, + -0.02874515950679779, + 2.7962026596069336, + 0.2370568960905075, + 1.5884850025177002, + 0.9987253546714783 + ], + [ + 2.1451404094696045, + 0.09629477560520172, + -1.8500688076019287, + 0.012462526559829712, + 0.13087880611419678, + -2.4062256813049316, + -3.303264856338501, + -1.3855981826782227, + -0.0292893648147583, + -0.17000514268875122, + 0.0005041323602199554, + -0.7487195134162903, + 0.06868582963943481, + -2.107095718383789, + -0.5890907645225525, + 0.1777246743440628, + -0.5576009750366211, + 2.767899990081787, + -0.3420543074607849, + 1.1357430219650269, + 1.8840694427490234 + ], + [ + 2.032343626022339, + 0.4649246335029602, + -0.5597869753837585, + 0.37935349345207214, + 0.14823919534683228, + -3.389478921890259, + -1.972822904586792, + -0.0014220476150512695, + -0.6147069334983826, + 0.9483420252799988, + 0.641323983669281, + -0.7033213973045349, + 0.4629128575325012, + -1.6774024963378906, + -1.1420938968658447, + 0.5891244411468506, + -0.712251603603363, + 2.1924612522125244, + 0.18694229423999786, + 0.2795591354370117, + 2.4723148345947266 + ], + [ + 2.299381732940674, + -0.6742787957191467, + -0.886760950088501, + -0.1637115478515625, + 0.32663029432296753, + -1.9267126321792603, + -1.9424545764923096, + -0.3586595058441162, + -1.1000827550888062, + 0.45532336831092834, + -0.4314403235912323, + -0.12869995832443237, + -0.032513588666915894, + -2.046614170074463, + -1.2942235469818115, + 0.3962012231349945, + 0.797533392906189, + 2.202794313430786, + 0.4938191771507263, + 0.15863806009292603, + 1.745660662651062 + ], + [ + 2.107815742492676, + 0.963295042514801, + -1.2979395389556885, + -1.0134333372116089, + -1.319197177886963, + -3.176518440246582, + -2.4782180786132812, + -1.3465216159820557, + -1.0473096370697021, + -0.22631770372390747, + 0.7881518006324768, + -1.4033093452453613, + 0.3855947256088257, + -1.3816941976547241, + -1.1257284879684448, + -0.7653472423553467, + 0.6845007538795471, + 2.9796371459960938, + 0.1967305988073349, + 1.4200057983398438, + 1.2843091487884521 + ], + [ + 1.7303649187088013, + 1.0728741884231567, + -1.1676844358444214, + -0.20355769991874695, + -0.8773125410079956, + -3.2701919078826904, + -2.1479928493499756, + -0.044036924839019775, + -1.252698302268982, + -0.01393243670463562, + 0.6447213292121887, + -0.7088868618011475, + 0.11592492461204529, + -2.1431162357330322, + -0.8284967541694641, + 0.4642922580242157, + -0.5207862854003906, + 1.8997660875320435, + -0.6910213828086853, + 0.2906065583229065, + 1.4081878662109375 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7612967491149902, + -0.5331565737724304, + -0.2699308395385742, + -0.20421931147575378, + -0.22613269090652466, + -1.9032442569732666, + 0.0, + 0.09110367298126221, + 0.009229481220245361, + 0.004811525344848633, + 0.656868577003479, + -0.28485387563705444, + -0.31901538372039795, + 0.0, + 0.08918565511703491, + 0.3355104923248291, + 0.026827186346054077, + 1.563896656036377, + 0.13480885326862335, + 0.02485179901123047, + 0.0 + ] + ], + "predicted_scores": [ + 9.025599479675293, + 8.796256065368652, + 8.275415420532227, + 8.15134048461914, + 7.997213840484619, + 7.892197608947754, + 7.640031814575195, + 7.517190456390381 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train0", + "selected_index": 0, + "state_hash": "73f2826502af62b561f13c32614947588918c3deb631165739e66d787008fa98", + "stored_base_utility": 0.23161475360393524, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.04583283141255379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04583283141255379 + }, + "base_progress": 0.04583283141255379, + "base_success": false, + "base_utility": 0.04583283141255379, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05473164841532707, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05473164841532707 + }, + { + "contact_quality": null, + "progress": 0.0898062214255333, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0898062214255333 + }, + { + "contact_quality": null, + "progress": 0.09243235737085342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09243235737085342 + }, + { + "contact_quality": null, + "progress": 0.04779624938964844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04779624938964844 + }, + { + "contact_quality": null, + "progress": 0.1050744280219078, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1050744280219078 + }, + { + "contact_quality": null, + "progress": 0.08136358112096786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08136358112096786 + }, + { + "contact_quality": null, + "progress": 0.10417115688323975, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10417115688323975 + }, + { + "contact_quality": null, + "progress": 0.05421211197972298, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05421211197972298 + } + ], + "candidate_progress": [ + 0.05473164841532707, + 0.0898062214255333, + 0.09243235737085342, + 0.04779624938964844, + 0.1050744280219078, + 0.08136358112096786, + 0.10417115688323975, + 0.05421211197972298 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-7a044cb6c7bb03f3ea10", + "ms-38dff128761905bb1507" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-126eb8b36769f9ba4671", + "generated_tangents": [ + [ + 1.6391998529434204, + -0.10492207109928131, + -1.2173104286193848, + 0.06312951445579529, + 0.034627094864845276, + -3.809929609298706, + -2.655278205871582, + 1.55155611038208, + -0.049403004348278046, + 0.14256125688552856, + 0.07096967101097107, + 1.5670653581619263, + -0.8856337070465088, + -0.3646290898323059, + -0.3188478946685791, + 0.23871849477291107, + 0.5090757608413696, + 0.578620970249176, + 0.6506721377372742, + 1.3572771549224854, + 1.5281459093093872 + ], + [ + 1.632312297821045, + -0.073216512799263, + -1.279280185699463, + 0.04200807213783264, + 0.13810274004936218, + -3.7365379333496094, + -2.5355067253112793, + 1.5218451023101807, + -0.11356581747531891, + 0.07684358209371567, + 0.02117886021733284, + 1.4602035284042358, + -0.9178345203399658, + -0.33722400665283203, + -0.41049888730049133, + 0.2655564248561859, + 0.510861873626709, + 0.48537325859069824, + 0.6985797882080078, + 1.3627231121063232, + 1.4742491245269775 + ], + [ + 1.3266669511795044, + -0.21584570407867432, + -1.0917428731918335, + 0.5911462306976318, + -0.091316819190979, + -3.1719870567321777, + -2.0689079761505127, + 1.423898458480835, + -0.051188841462135315, + 0.0882360190153122, + 0.01707397773861885, + 1.9493176937103271, + -0.2595904469490051, + -0.4032917022705078, + -0.5435537099838257, + 0.18033447861671448, + 0.6712300777435303, + 0.5846545696258545, + 0.4349537491798401, + 1.231971025466919, + 0.9769790172576904 + ], + [ + 1.688482642173767, + -0.14923816919326782, + -1.429479718208313, + 0.03564286231994629, + 0.049825385212898254, + -3.9505109786987305, + -2.792681932449341, + 1.6349338293075562, + -0.0660700872540474, + 0.06310125440359116, + 0.04371916875243187, + 1.4827817678451538, + -0.9865245819091797, + -0.6827672719955444, + -0.2880074679851532, + 0.3062531352043152, + 0.49833357334136963, + 0.47552281618118286, + 0.5298202037811279, + 1.146613597869873, + 1.6703341007232666 + ], + [ + 1.4574108123779297, + 0.0944143533706665, + -0.8780643939971924, + 0.31943944096565247, + 0.07821029424667358, + -3.033165454864502, + -1.998957633972168, + 1.0657954216003418, + -0.2797210216522217, + -0.008155487477779388, + -0.14730030298233032, + 1.6937888860702515, + -0.2869578003883362, + -0.6444816589355469, + -0.5542390942573547, + 0.1780693531036377, + 0.6108785271644592, + -0.19697339832782745, + 0.4930592179298401, + 0.9760110974311829, + 1.555848240852356 + ], + [ + 1.4546087980270386, + 0.03867151588201523, + -0.8583587408065796, + 0.4685031473636627, + 0.006796449422836304, + -3.1022281646728516, + -1.964062213897705, + 1.0652985572814941, + -0.2551897168159485, + -0.042359210550785065, + -0.25985828042030334, + 1.5183671712875366, + -0.3688545525074005, + -0.6041058897972107, + -0.6142784357070923, + 0.20553471148014069, + 0.5365957617759705, + -0.24196593463420868, + 0.5062716007232666, + 0.869109570980072, + 1.450219988822937 + ], + [ + 1.4247417449951172, + -0.21659106016159058, + -1.1823608875274658, + 0.8322272300720215, + -0.0005451738834381104, + -3.0540988445281982, + -2.1361265182495117, + 1.3151968717575073, + -0.03238707035779953, + 0.10789983719587326, + 0.1868794858455658, + 2.102938175201416, + -0.4955365061759949, + -0.32517802715301514, + -0.41326475143432617, + 0.08100670576095581, + 0.5753326416015625, + 0.6134408712387085, + 0.4230189919471741, + 1.4242136478424072, + 0.9708864688873291 + ], + [ + 1.3358324766159058, + -0.11553329974412918, + -1.3399983644485474, + 0.6663671731948853, + 0.23060394823551178, + -2.7645184993743896, + -2.443643808364868, + 2.059368848800659, + -0.024196317419409752, + -0.31755587458610535, + 0.39442718029022217, + 2.483910083770752, + -1.1303515434265137, + -0.4766498804092407, + -0.5132682919502258, + 0.19200073182582855, + 0.46852320432662964, + 0.2817460298538208, + 0.6593990325927734, + 1.3848536014556885, + 0.8981823921203613 + ] + ], + "generated_utilities": [ + 0.05473164841532707, + 0.0898062214255333, + 0.09243235737085342, + 0.04779624938964844, + 0.1050744280219078, + 0.08136358112096786, + 0.10417115688323975, + 0.05421211197972298 + ], + "group_id": "ms-126eb8b36769f9ba4671", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05304628610610962, + 0.14190539717674255, + 0.1985912322998047, + 0.1210482120513916, + 0.18570420145988464, + 0.05259696766734123, + 0.1827520728111267, + 0.15291261672973633, + 0.07772710174322128, + 0.15551692247390747, + 0.08802877366542816, + 0.07544025033712387, + 0.08058679103851318, + 0.10843702405691147, + 0.023751219734549522, + 0.00727109145373106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7799564599990845, + -0.0863567665219307, + -0.04370701313018799, + -0.04603701829910278, + -0.003975361585617065, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.02617853879928589, + 0.009054642170667648, + -0.050068795680999756, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + -0.007249712944030762, + 0.0760425329208374, + 1.0404269695281982, + -0.04351449012756348, + -0.010379016399383545, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + 0.005863763391971588, + -0.04370701313018799, + 0.010433673858642578, + -0.05825299024581909, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.22243881225585938, + -0.007246788591146469, + 0.19961372017860413, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.11177664995193481, + -0.1665668487548828, + 1.0404269695281982, + -0.09226274490356445, + -0.18212640285491943, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + -0.7482962012290955, + -0.04370701313018799, + -0.2370874285697937, + 0.3763168752193451, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + -0.441777765750885, + -0.44057023525238037, + -0.6443575024604797, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.008161306381225586, + 0.22021520137786865, + 1.0404269695281982, + -0.09226274490356445, + -0.35497862100601196, + 1.0522968769073486, + 0.0 + ], + [ + 1.7799564599990845, + 0.04087676852941513, + -1.043707013130188, + -0.8482787609100342, + 0.3640388548374176, + -3.3430333137512207, + -2.8715221881866455, + 1.8326367139816284, + -0.4197058081626892, + -0.040397901087999344, + -0.37471577525138855, + 1.7451434135437012, + -1.6306462287902832, + -0.7785218954086304, + -0.5121169090270996, + -0.7797847986221313, + 2.0404269695281982, + 0.9077372550964355, + 0.645021378993988, + 2.0522968769073486, + 1.2749258279800415 + ], + [ + 1.6519862413406372, + 0.1747584044933319, + -0.9129671454429626, + -0.6869257092475891, + 0.12357090413570404, + -3.2319328784942627, + -2.704270839691162, + 1.7347972393035889, + -0.40612348914146423, + 0.23568139970302582, + -0.3912501931190491, + 1.8630532026290894, + -1.8548930883407593, + -0.9794526100158691, + -0.25177696347236633, + -0.6884964108467102, + 2.237109422683716, + 1.1246912479400635, + 0.7490227222442627, + 2.320941686630249, + 1.2039268016815186 + ], + [ + 1.7142105102539062, + 0.3941279351711273, + -1.021533489227295, + -1.1548089981079102, + 0.5052449107170105, + -3.714951992034912, + -3.04028058052063, + 1.529634952545166, + -0.22215746343135834, + 0.19690367579460144, + -0.12262976169586182, + 1.4847477674484253, + -1.490212082862854, + -0.7427945137023926, + -0.7506002187728882, + -0.9221256375312805, + 1.9211885929107666, + 0.5114957094192505, + 0.30887648463249207, + 2.4437777996063232, + 1.3625030517578125 + ], + [ + 1.3140592575073242, + -0.1129835918545723, + -0.9130957722663879, + -1.1089756488800049, + 0.7560886144638062, + -3.3889732360839844, + -3.1182656288146973, + 1.7436503171920776, + -0.9962512254714966, + 0.43109408020973206, + -0.8453629016876221, + 1.2057833671569824, + -2.194112777709961, + -0.22479140758514404, + -0.7258806824684143, + -0.9702881574630737, + 1.8160195350646973, + 0.649415135383606, + 0.6846981048583984, + 2.1997463703155518, + 1.7655411958694458 + ], + [ + 2.6430320739746094, + -0.7696453332901001, + -1.765415906906128, + -0.8281408548355103, + -0.1440155804157257, + -3.3335886001586914, + -3.398141860961914, + 2.4138498306274414, + 0.47537559270858765, + -0.9514593482017517, + 0.2674855887889862, + 2.6307411193847656, + -0.7876039743423462, + -0.35013434290885925, + -0.7511636018753052, + -0.04397457838058472, + 2.123702049255371, + 1.1604151725769043, + 0.7323322892189026, + 1.88338303565979, + 1.5078264474868774 + ] + ], + "predicted_scores": [ + 6.030942440032959, + 5.877603054046631, + 5.745731830596924, + 5.655466556549072, + 5.458425521850586, + 5.389834403991699, + 5.023163318634033, + 4.013148307800293 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train0", + "selected_index": 0, + "state_hash": "2390fc23ce8b50bdcdcbb1ab5f0583d2dad2b94fb871db73b1a8401265f59987", + "stored_base_utility": 0.05304628610610962, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.027269184589385986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027269184589385986 + }, + { + "contact_quality": null, + "progress": 0.04800540208816528, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04800540208816528 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.030054310336709023, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030054310336709023 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.027269184589385986, + 0.04800540208816528, + 1.0, + 1.0, + 1.0, + 0.030054310336709023, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-1f19fd22562f87bdb776", + "ms-c1620854ba52f09e24d5", + "ms-067cac2e4dbeddb12239", + "ms-067cac2e4dbeddb12239", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-cad6645f8175e38ad155", + "ms-babaeb20129c46551281" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-16661b2fd2c85f57a66c", + "generated_tangents": [ + [ + 0.07182317972183228, + -0.25195980072021484, + 0.8839491009712219, + -0.1986687183380127, + 0.3855177164077759, + 0.746080756187439, + 2.4408936500549316, + 1.2690387964248657, + -0.36878761649131775, + -0.40345215797424316, + 0.591017484664917, + -0.168995201587677, + 1.0220140218734741, + 1.8888201713562012, + 0.7933868169784546, + -0.24187451601028442, + -1.1002981662750244, + -0.26305556297302246, + -0.1259298026561737, + -1.0836772918701172, + -2.1015565395355225 + ], + [ + 0.10315769910812378, + -0.20259320735931396, + 0.34571170806884766, + -0.8588720560073853, + 0.36758148670196533, + 1.6789014339447021, + 2.649001121520996, + 1.201626181602478, + -0.19718889892101288, + -0.2105022370815277, + 0.14505070447921753, + 0.018547385931015015, + 1.5460213422775269, + 1.587507724761963, + 0.8956091403961182, + -0.18251945078372955, + -0.8965319991111755, + -0.22472302615642548, + 0.241672083735466, + -1.4455063343048096, + -1.3432704210281372 + ], + [ + 0.10031376034021378, + -0.11849141120910645, + 0.7268736958503723, + -0.7146854400634766, + 0.5790175199508667, + 0.8996216058731079, + 2.2201876640319824, + 1.3257726430892944, + -0.1206386536359787, + -0.713047206401825, + 0.5406262874603271, + -0.8258600234985352, + 1.211934208869934, + 1.4730538129806519, + 0.9376559257507324, + -0.0715533196926117, + -1.210080862045288, + -0.12432421743869781, + -0.035953670740127563, + -0.8264876008033752, + -1.9952523708343506 + ], + [ + 0.24770130217075348, + -0.0796661376953125, + 0.21971911191940308, + -1.67794930934906, + -0.008098766207695007, + 1.618289828300476, + 2.0531442165374756, + 1.5219308137893677, + -0.09871240705251694, + -0.47520357370376587, + 0.4357303977012634, + -0.5788902044296265, + 1.6422539949417114, + 1.752223253250122, + 0.9891992211341858, + -0.20013757050037384, + -1.398559808731079, + -0.343084454536438, + 0.010496914386749268, + -1.246861457824707, + -2.188300371170044 + ], + [ + -0.07818293571472168, + -0.19883371889591217, + 0.436402291059494, + -0.05419538915157318, + 0.4581429958343506, + 1.3210529088974, + 2.8119122982025146, + 1.2695547342300415, + -0.18630556762218475, + -0.5067341327667236, + 0.2933301031589508, + -0.00015692412853240967, + 1.197396993637085, + 1.4097285270690918, + 1.2938770055770874, + -0.1203649491071701, + -0.7299903035163879, + -0.07005701959133148, + 0.002072364091873169, + -1.2534825801849365, + -1.885866641998291 + ], + [ + -0.08446246385574341, + -0.21925637125968933, + 0.6055530905723572, + -0.021726801991462708, + 0.5052362084388733, + 1.4888149499893188, + 2.800708055496216, + 1.181645154953003, + -0.1087164506316185, + -0.4942442774772644, + 0.2532975375652313, + 0.01925988495349884, + 1.042837381362915, + 1.2664003372192383, + 1.3407131433486938, + -0.10998737812042236, + -0.6873182654380798, + -0.14442871510982513, + -0.03526574373245239, + -1.0848643779754639, + -1.9860200881958008 + ], + [ + 0.08374780416488647, + -0.22458630800247192, + 0.4597207307815552, + -0.24619752168655396, + 0.026582494378089905, + 1.2279653549194336, + 2.9190549850463867, + 1.5459320545196533, + -0.16730205714702606, + -0.5966821908950806, + 0.24363699555397034, + -0.008007138967514038, + 1.217457890510559, + 1.8039884567260742, + 1.1827164888381958, + -0.12527476251125336, + -0.7073969841003418, + 0.08288136124610901, + -0.025503098964691162, + -1.0367083549499512, + -2.144688129425049 + ], + [ + -0.0046021342277526855, + -0.0868464931845665, + 0.8236863613128662, + -0.37719231843948364, + 0.7548291087150574, + 1.6137198209762573, + 2.9738709926605225, + 1.1011310815811157, + -0.1957755833864212, + -0.6226890087127686, + 0.012403644621372223, + 0.06340701878070831, + 1.4640778303146362, + 1.4870837926864624, + 1.3600130081176758, + 0.010157547891139984, + -1.0214133262634277, + -0.11970458924770355, + -0.16698560118675232, + -1.4716496467590332, + -2.4429054260253906 + ] + ], + "generated_utilities": [ + 2.0, + 0.027269184589385986, + 0.04800540208816528, + 2.0, + 2.0, + 2.0, + 0.030054310336709023, + 2.0 + ], + "group_id": "ms-16661b2fd2c85f57a66c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23635749518871307, + 0.1864171028137207, + 0.18280422687530518, + 0.16576561331748962, + 0.2967458665370941, + 0.2833873927593231, + 0.2587948143482208, + 0.2695174813270569, + 0.2239263951778412, + 0.15686756372451782, + 0.2505500912666321, + 0.12250886857509613, + 0.16453635692596436, + 2.0, + 0.13450095057487488, + 0.11914549767971039 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.12099093198776245, + 0.06886856257915497, + 0.2952747344970703, + -0.1764048933982849, + -0.12411242723464966, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + -0.046127498149871826, + 0.12581980228424072, + 0.042072445154190063, + -0.06942430138587952, + 0.7517287731170654, + 0.0, + 0.690804123878479, + -0.06023237854242325, + 0.049633681774139404, + 0.01943105459213257, + 0.013327360153198242, + -0.8525059223175049, + 0.0 + ], + [ + 0.3599843978881836, + -0.34296315908432007, + 0.2952747344970703, + -0.3797188997268677, + -0.41771262884140015, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + 0.06634362041950226, + 0.138022780418396, + -0.2342309057712555, + -0.36017337441444397, + 0.7517287731170654, + 0.0, + 0.690804123878479, + 0.10550708323717117, + -0.35431206226348877, + 0.16382712125778198, + 0.06674730777740479, + -0.8525059223175049, + 0.0 + ], + [ + 0.567882239818573, + -0.4729353189468384, + 1.6256722211837769, + -1.5372763872146606, + 1.0487556457519531, + 1.5327277183532715, + 2.2847697734832764, + 2.6703484058380127, + 0.4013517200946808, + -1.3629915714263916, + 0.6385155916213989, + 0.2664203345775604, + 1.2691267728805542, + 1.2942520380020142, + 2.2084813117980957, + 0.6982719302177429, + -0.8422143459320068, + 1.070263385772705, + -0.9558742046356201, + -1.3996182680130005, + -1.4242117404937744 + ], + [ + 0.6830785274505615, + -0.5584439635276794, + 0.841848611831665, + -0.5293771624565125, + -0.14354348182678223, + 2.097932815551758, + 2.9733188152313232, + 2.603226661682129, + -0.1639159619808197, + -0.918163537979126, + 1.002030849456787, + 0.40132570266723633, + 1.1751198768615723, + 1.455662727355957, + 2.3202755451202393, + -0.18584474921226501, + -1.3483099937438965, + 0.0660635232925415, + 0.4330971837043762, + -1.1989169120788574, + -2.0896859169006348 + ], + [ + 0.44418102502822876, + 0.07469291985034943, + 1.6960139274597168, + -1.329486608505249, + 1.3067576885223389, + 2.2694292068481445, + 2.0299551486968994, + 1.6317954063415527, + 0.7472026348114014, + -0.12518703937530518, + 0.3035789430141449, + -0.5889836549758911, + 2.345867395401001, + 1.9470158815383911, + 0.8398547172546387, + -0.6090301871299744, + -1.5060018301010132, + 0.5695335865020752, + -0.6025367975234985, + -1.8703374862670898, + -2.728482246398926 + ], + [ + 0.31904691457748413, + 0.04995492100715637, + 1.3823881149291992, + -0.5106721520423889, + -0.2236303687095642, + 0.9921484589576721, + 1.5933295488357544, + 2.741320848464966, + -0.21808990836143494, + -0.7775652408599854, + 1.224307894706726, + -0.08168366551399231, + 2.692474842071533, + 2.269430160522461, + 2.3270838260650635, + -0.2901480495929718, + -1.2098140716552734, + -0.07570821046829224, + -1.1340655088424683, + -1.9999847412109375, + -1.6802332401275635 + ], + [ + -0.6257733106613159, + 0.3840647339820862, + 0.9012271165847778, + -1.248098611831665, + -0.1650761365890503, + 2.7645747661590576, + 2.7588071823120117, + 3.0172548294067383, + 0.4120163321495056, + -1.6110656261444092, + -0.10423526167869568, + 0.5334746241569519, + 1.7809263467788696, + 0.5690644383430481, + 1.5881801843643188, + 0.8860509991645813, + -1.1502506732940674, + 0.7821243405342102, + -0.549727201461792, + -1.326378345489502, + -1.5100535154342651 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2773973345756531, + -0.13796104490756989, + 1.2952747344970703, + -1.0323810577392578, + 0.5605664849281311, + 1.8606208562850952, + 2.439758777618408, + 2.353257894515991, + 0.24642756581306458, + -0.861977219581604, + 0.4799116551876068, + 0.264199823141098, + 1.7517287731170654, + 1.4282559156417847, + 1.690804123878479, + 0.1215878501534462, + -0.9322434067726135, + 0.8536378145217896, + -0.35758620500564575, + -1.8525059223175049, + -1.910990595817566 + ], + [ + -0.4916617274284363, + -0.5934253931045532, + 2.074575424194336, + -0.9512085318565369, + 0.3705800175666809, + 1.691048264503479, + 2.2907681465148926, + 1.532329797744751, + 0.8991785049438477, + -0.7733803987503052, + 1.3577321767807007, + 0.9844847917556763, + 1.7379451990127563, + 2.050845146179199, + 0.7804086804389954, + -0.8400509357452393, + -1.6339564323425293, + 0.9922879934310913, + 0.6006971597671509, + -1.0612030029296875, + -1.3134464025497437 + ] + ], + "predicted_scores": [ + 7.757726192474365, + 6.17840051651001, + 5.942082405090332, + 5.575725078582764, + 5.345473766326904, + 4.9476094245910645, + 4.759594440460205, + 4.737806797027588 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train0", + "selected_index": 0, + "state_hash": "74b116fd9ceddd5b2261edd96a14934a4b4a78a40191a654d6743661b819f407", + "stored_base_utility": 0.23635749518871307, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0007540385122410953, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007540385122410953 + }, + { + "contact_quality": null, + "progress": 0.0013887485256418586, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013887485256418586 + }, + { + "contact_quality": null, + "progress": 0.0012941956520080566, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0012941956520080566 + }, + { + "contact_quality": null, + "progress": 0.0005388259887695312, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0005388259887695312 + }, + { + "contact_quality": null, + "progress": 0.0009148319368250668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009148319368250668 + }, + { + "contact_quality": null, + "progress": 0.0015756288776174188, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0015756288776174188 + }, + { + "contact_quality": null, + "progress": 0.0008049209718592465, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008049209718592465 + }, + { + "contact_quality": null, + "progress": 0.0010410547256469727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0010410547256469727 + } + ], + "candidate_progress": [ + 0.0007540385122410953, + 0.0013887485256418586, + 0.0012941956520080566, + 0.0005388259887695312, + 0.0009148319368250668, + 0.0015756288776174188, + 0.0008049209718592465, + 0.0010410547256469727 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-8e12f7ad446482627dd3", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-20e1efbb745fd02abc95", + "generated_tangents": [ + [ + 0.06185590475797653, + -0.04598106071352959, + 1.7466535568237305, + -0.7043129205703735, + 0.90647292137146, + 2.1465933322906494, + 1.7426588535308838, + -0.2489314079284668, + -0.25054457783699036, + 1.6796447038650513, + 0.8467791676521301, + 0.7874410152435303, + 2.371890068054199, + 2.07037615776062, + -1.2214183807373047, + -0.4488571584224701, + 2.096590280532837, + 0.8521057963371277, + 0.6450879573822021, + 2.701753854751587, + 2.2453272342681885 + ], + [ + 0.17104472219944, + -0.05031730607151985, + 1.577580213546753, + -0.3243550658226013, + 1.1014646291732788, + 1.9892752170562744, + 1.8683958053588867, + -0.10321146249771118, + -0.12209179997444153, + 1.6653223037719727, + 0.5897968411445618, + 0.64275062084198, + 2.425130605697632, + 1.7522447109222412, + -1.1056369543075562, + -0.2550979554653168, + 1.9297103881835938, + 0.4884070158004761, + 0.8509853482246399, + 2.699551820755005, + 1.9898948669433594 + ], + [ + 0.17789174616336823, + 0.03985103964805603, + 1.661165475845337, + -0.8181127309799194, + 1.0224950313568115, + 2.2704620361328125, + 2.1650853157043457, + -0.26494675874710083, + 0.035614825785160065, + 1.4089508056640625, + 0.49506404995918274, + 1.2877193689346313, + 2.4452085494995117, + 2.5399973392486572, + -1.1478750705718994, + -0.31928303837776184, + 2.1709225177764893, + 0.49177998304367065, + 0.7644610404968262, + 3.309995412826538, + 2.784111261367798 + ], + [ + 0.17100991308689117, + -0.07769951224327087, + 1.689501166343689, + -0.553665041923523, + 0.8678314089775085, + 2.2808072566986084, + 2.4742748737335205, + -0.5780261754989624, + -0.1790362298488617, + 1.832779049873352, + 0.9880365133285522, + 0.8067461252212524, + 2.8785808086395264, + 2.3283064365386963, + -1.0634372234344482, + -0.3440331816673279, + 2.48679780960083, + 0.5058993101119995, + 0.7634018063545227, + 2.664320468902588, + 2.1168289184570312 + ], + [ + -0.012141048908233643, + 0.14199289679527283, + 1.4682300090789795, + -0.6070286631584167, + 1.2793245315551758, + 2.4998717308044434, + 1.9116482734680176, + -0.3240754306316376, + -0.16107217967510223, + 1.746077537536621, + 0.7436660528182983, + 0.9374769330024719, + 2.0339906215667725, + 2.403305768966675, + -1.0634084939956665, + -0.41096529364585876, + 2.193126916885376, + 0.6560408473014832, + 0.9145709872245789, + 2.9004108905792236, + 2.422254800796509 + ], + [ + -0.13088126480579376, + -0.16616514325141907, + 1.3518954515457153, + -0.06645262241363525, + 0.8044505715370178, + 2.339695692062378, + 2.1612751483917236, + -0.21809199452400208, + -0.21274007856845856, + 1.3352855443954468, + 0.20809608697891235, + 0.7528249621391296, + 1.6302942037582397, + 3.068659543991089, + -0.8531628847122192, + -0.3069373071193695, + 1.8509960174560547, + 0.5853466987609863, + 0.6205123662948608, + 2.9361536502838135, + 2.5021796226501465 + ], + [ + 0.10634994506835938, + -0.06957791745662689, + 1.5537424087524414, + -0.7663921117782593, + 1.0942169427871704, + 2.330355405807495, + 2.1368660926818848, + -0.3144095838069916, + -0.1361977607011795, + 1.6854244470596313, + 0.7905531525611877, + 0.9736430048942566, + 2.2341573238372803, + 2.2617242336273193, + -1.1853855848312378, + -0.3789823651313782, + 2.3444769382476807, + 0.656904935836792, + 0.6562376022338867, + 3.087292194366455, + 2.7287371158599854 + ], + [ + 0.03744655102491379, + -0.05327264592051506, + 1.592834711074829, + -0.5894061326980591, + 1.4490220546722412, + 2.5584278106689453, + 2.2257516384124756, + -0.5216046571731567, + -0.07101935148239136, + 1.5448328256607056, + 0.4931126832962036, + 0.8571008443832397, + 2.0394937992095947, + 2.248305320739746, + -0.9961563944816589, + -0.3045949339866638, + 1.9616707563400269, + 0.6377894878387451, + 0.9306171536445618, + 3.1057169437408447, + 2.726101875305176 + ] + ], + "generated_utilities": [ + 0.0007540385122410953, + 0.0013887485256418586, + 0.0012941956520080566, + 0.0005388259887695312, + 0.0009148319368250668, + 0.0015756288776174188, + 0.0008049209718592465, + 0.0010410547256469727 + ], + "group_id": "ms-20e1efbb745fd02abc95", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13665315508842468, + 0.1696699559688568, + 0.049180448055267334, + 0.05942712351679802, + 0.22736775875091553, + 0.0799722671508789, + 0.17261533439159393, + 0.21202586591243744, + 0.18980810046195984, + 0.24345096945762634, + 0.08349347114562988, + 0.11624568700790405, + 0.17245106399059296, + 0.07060472667217255, + 0.041495781391859055, + 0.028585990890860558 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.08645056188106537, + -0.060760222375392914, + 0.6493666172027588, + -0.016124725341796875, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + -0.2971414625644684, + 0.023367643356323242, + 0.6914786100387573, + -0.1027369350194931, + 0.1733318567276001, + 1.2948026657104492, + 0.0, + -0.30476057529449463, + -0.10394972562789917, + 0.9822354316711426, + 0.055961012840270996, + 0.004969358444213867, + 2.122485637664795, + 0.0 + ], + [ + 0.046016186475753784, + -0.24098780751228333, + 0.6493666172027588, + 0.05737954378128052, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + 0.22233793139457703, + 0.3896067142486572, + 0.6914786100387573, + -0.061455681920051575, + 0.17170286178588867, + 1.2948026657104492, + 0.0, + -0.4717581272125244, + -0.5722612738609314, + 0.9822354316711426, + 0.3690842390060425, + 0.33333829045295715, + 2.122485637664795, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.590132236480713, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.670657634735107, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.27303409576416 + ], + [ + -0.4654892086982727, + -0.6659128069877625, + 2.053840160369873, + -0.17420679330825806, + 1.1238669157028198, + 2.773953914642334, + 2.9706993103027344, + -0.9652835726737976, + -0.6603021621704102, + 1.957632303237915, + 0.30636119842529297, + 1.0932379961013794, + 2.4988603591918945, + 2.1735024452209473, + -0.7229697108268738, + -0.6565687656402588, + 2.1097376346588135, + 0.79427170753479, + 1.1421128511428833, + 3.200641393661499, + 3.3294520378112793 + ], + [ + 0.8783296346664429, + -0.6790766716003418, + 1.3789844512939453, + -1.578566551208496, + 1.2084856033325195, + 1.906846046447754, + 2.7278928756713867, + -0.8570022583007812, + -0.3114888072013855, + 2.660332202911377, + 0.4615887999534607, + -0.16199523210525513, + 1.9830265045166016, + 2.4224376678466797, + -0.9260885715484619, + 0.5555675029754639, + 2.5474588871002197, + 0.0070996880531311035, + 1.627825379371643, + 3.9487287998199463, + 3.03330135345459 + ], + [ + -0.1274794638156891, + -0.26015421748161316, + 2.5414626598358154, + -1.0220305919647217, + 1.4708960056304932, + 2.0982167720794678, + 3.1457836627960205, + -1.0992621183395386, + -0.9075721502304077, + 0.8330968022346497, + 0.15631377696990967, + 1.3095502853393555, + 1.8582203388214111, + 2.101317882537842, + -0.9184049367904663, + 0.4676415026187897, + 2.784534454345703, + 0.4529607892036438, + 0.6403366327285767, + 3.1566896438598633, + 2.5268735885620117 + ], + [ + 0.9289348125457764, + 0.1545984447002411, + 1.6792887449264526, + -0.6400681138038635, + 0.5956984758377075, + 2.5183756351470947, + 1.6261346340179443, + -0.019961506128311157, + -0.2638699412345886, + 1.310987114906311, + 0.4996453523635864, + -0.06606829166412354, + 1.9830868244171143, + 2.392476797103882, + -0.3871336579322815, + -0.7797489166259766, + 1.927225947380066, + 0.4438522458076477, + 1.0257132053375244, + 3.211881160736084, + 3.374328851699829 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05862678587436676, + -0.042836256325244904, + 1.6493666172027588, + -0.7465556859970093, + 1.1865841150283813, + 2.873413562774658, + 2.2950661182403564, + -0.48127153515815735, + -0.11381363868713379, + 1.6914786100387573, + 0.06965510547161102, + 0.8288453221321106, + 2.294802665710449, + 2.3353288173675537, + -1.3047605752944946, + -0.39009055495262146, + 1.9822354316711426, + 0.9993376731872559, + 0.81731116771698, + 3.122485637664795, + 2.63651704788208 + ], + [ + -0.045653510838747025, + 0.00397084653377533, + 1.3591042757034302, + -0.6414353847503662, + 0.8377937078475952, + 2.8277816772460938, + 2.203674554824829, + -0.1951642632484436, + 0.12272481620311737, + 1.7411555051803589, + -0.004818864166736603, + 0.8769108057022095, + 2.243906021118164, + 2.4328396320343018, + -1.6330946683883667, + -0.4807296395301819, + 2.2988247871398926, + 0.8021057844161987, + 0.6970767378807068, + 2.8728103637695312, + 2.288278102874756 + ], + [ + -0.4273069500923157, + 0.16743427515029907, + 1.538269281387329, + -0.7002415657043457, + 0.7622614502906799, + 2.805328369140625, + 2.267277479171753, + -0.6541028022766113, + -0.2980780303478241, + 1.456940770149231, + 0.5105806589126587, + 1.040317177772522, + 2.4321720600128174, + 2.696331024169922, + -1.1555191278457642, + -0.025772064924240112, + 1.6384800672531128, + 1.4557092189788818, + 0.6594157814979553, + 3.1446406841278076, + 2.249678134918213 + ], + [ + -0.6431475877761841, + -0.24224388599395752, + 1.3122833967208862, + -0.16711556911468506, + 0.9232780337333679, + 2.850522994995117, + 2.645284414291382, + -0.379825621843338, + -0.6750738620758057, + 1.1236441135406494, + 0.4938291311264038, + 1.1075668334960938, + 2.203859806060791, + 2.799032688140869, + -1.359815239906311, + -0.5104091763496399, + 2.0970702171325684, + 1.057171106338501, + 1.1167995929718018, + 2.8190994262695312, + 2.2184524536132812 + ] + ], + "predicted_scores": [ + 8.597485542297363, + 8.596193313598633, + 8.534893035888672, + 8.443713188171387, + 8.348173141479492, + 8.27610969543457, + 8.26349925994873, + 8.097298622131348 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1375492786/train0", + "selected_index": 0, + "state_hash": "56acb365a02ebac0ac4893531e168df3c430b486416156feeea40eeb178bda3e", + "stored_base_utility": 0.13665315508842468, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-2dc5138251afd26a040d", + "generated_tangents": [ + [ + 2.229734420776367, + 0.07782519608736038, + -1.911167025566101, + -0.29360246658325195, + 0.03879709541797638, + -0.060477957129478455, + -1.1201860904693604, + 0.7442946434020996, + 0.0975741595029831, + -0.6522977352142334, + 0.061974864453077316, + 0.5730195045471191, + -2.8622052669525146, + -3.0645809173583984, + 0.8219016194343567, + 0.2688923478126526, + 0.6182039976119995, + -1.8234680891036987, + 0.9948716759681702, + -2.5656540393829346, + -1.6501619815826416 + ], + [ + 2.120349884033203, + 0.07830886542797089, + -1.718955636024475, + -0.5642669200897217, + -0.06578114628791809, + 0.1020921990275383, + -1.2869176864624023, + 0.9663091897964478, + 0.017531784251332283, + -0.5773118734359741, + -0.30360743403434753, + 0.4511907994747162, + -3.376307487487793, + -3.2012767791748047, + 0.9626085758209229, + 0.39912110567092896, + 0.6345493197441101, + -1.8809996843338013, + 0.9704519510269165, + -2.526883125305176, + -2.046485424041748 + ], + [ + 2.1305999755859375, + 0.07375235855579376, + -1.5923258066177368, + -0.04012773931026459, + -0.32000255584716797, + -0.30043599009513855, + -0.6495680212974548, + 0.5238656401634216, + 0.3066084682941437, + -0.3996679186820984, + -0.16557320952415466, + 0.20371773838996887, + -3.1474225521087646, + -2.543048620223999, + 0.7771709561347961, + 0.11174799501895905, + 0.6515008211135864, + -1.7105644941329956, + 0.8088074922561646, + -2.6919498443603516, + -1.465714454650879 + ], + [ + 2.3269028663635254, + -0.24510174989700317, + -1.5488661527633667, + 0.07778221368789673, + -0.4977880120277405, + -0.5684682130813599, + -0.7216100096702576, + 0.6022121906280518, + 0.2655333876609802, + -0.38561928272247314, + -0.35218241810798645, + 0.4150071144104004, + -3.3866188526153564, + -2.8475182056427, + 0.894115149974823, + 0.22672773897647858, + 0.2564984858036041, + -1.740992784500122, + 1.0877726078033447, + -2.64404559135437, + -2.1503942012786865 + ], + [ + 2.1344754695892334, + 0.0311201810836792, + -1.6761525869369507, + -0.45513665676116943, + -0.02005326747894287, + -0.06954048573970795, + -1.3852035999298096, + 0.8574519157409668, + 0.012595122680068016, + -0.597812831401825, + -0.12699559330940247, + 0.3659045100212097, + -3.115100622177124, + -3.1035373210906982, + 0.8429934978485107, + 0.3820308446884155, + 0.6470757126808167, + -1.9184659719467163, + 0.9036051034927368, + -2.6084611415863037, + -1.7757612466812134 + ], + [ + 2.328768014907837, + -0.07092507183551788, + -1.7646962404251099, + -0.5830026268959045, + -0.03958779573440552, + 0.29470857977867126, + -1.1875858306884766, + 0.997925877571106, + 0.0465409979224205, + -0.5099322199821472, + -0.14093852043151855, + 0.44292837381362915, + -3.160677671432495, + -3.035654306411743, + 0.8148537278175354, + 0.3543292284011841, + 0.6301447153091431, + -2.0400350093841553, + 0.8489795327186584, + -3.1253316402435303, + -1.534785270690918 + ], + [ + 2.012805700302124, + -0.30843496322631836, + -1.6121374368667603, + 0.1371026337146759, + 0.11304327845573425, + -0.41266801953315735, + -1.0327105522155762, + 0.8589532375335693, + 0.09012092649936676, + -0.31097614765167236, + -0.16042950749397278, + 0.6064033508300781, + -3.2289366722106934, + -2.1153907775878906, + 0.7666473388671875, + 0.3098098039627075, + 0.8585698008537292, + -1.6206648349761963, + 1.3534557819366455, + -2.3459465503692627, + -1.2446222305297852 + ], + [ + 2.11568021774292, + -0.004121083766222, + -1.6251686811447144, + -0.4688686430454254, + 0.09955087304115295, + -0.3957272469997406, + -1.2614753246307373, + 1.0185052156448364, + -0.04080372303724289, + -0.5344919562339783, + -0.24649104475975037, + 0.1748218536376953, + -3.2859137058258057, + -2.9421751499176025, + 0.7877935767173767, + 0.46348288655281067, + 0.7609414458274841, + -1.991679310798645, + 0.8857476115226746, + -2.7842953205108643, + -1.7650737762451172 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-2dc5138251afd26a040d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1373138427734375, + 0.1398601531982422, + 0.11154793202877045, + 0.07755142450332642, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7500379085540771, + -0.32870030403137207, + -0.3281364440917969, + 0.020404458045959473, + 0.03884005546569824, + 0.26651138067245483, + 0.0, + 0.2614295184612274, + 0.02006034180521965, + 0.6256309151649475, + 0.10119366645812988, + 0.0365104079246521, + -1.9807136058807373, + 0.0, + -0.4040032625198364, + -0.3390212059020996, + 0.444367915391922, + -1.3744113445281982, + 0.3090496063232422, + -1.6372854709625244, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7500379085540771, + 0.1347455233335495, + -1.3281364440917969, + -0.3984496593475342, + -0.6640693545341492, + -0.7334886193275452, + -1.63351309299469, + 0.4696621894836426, + -0.03721961751580238, + 0.0896599143743515, + -0.8988063335418701, + -0.047963276505470276, + -2.9807136058807373, + -2.542311906814575, + 0.5875393152236938, + 0.41857796907424927, + 0.32660895586013794, + -2.3744113445281982, + 0.9575246572494507, + -2.6372854709625244, + -1.9586280584335327 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.26702618598938, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.08462381362915, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.9172561168670654 + ], + [ + 1.7928142547607422, + -0.046942055225372314, + -1.3733142614364624, + -0.4212188422679901, + -0.5566414594650269, + -0.9966021180152893, + -1.7926092147827148, + 0.37284746766090393, + 0.18181614577770233, + -0.056899696588516235, + -0.8793746829032898, + -0.06206277757883072, + -3.2539288997650146, + -2.3651227951049805, + 0.543649435043335, + 0.4969414472579956, + 0.3953157365322113, + -2.6153955459594727, + 1.239845633506775, + -2.9289329051971436, + -2.1138057708740234 + ], + [ + 1.4313925504684448, + 0.3287010192871094, + -1.4217545986175537, + -0.11546039581298828, + -0.38684383034706116, + -0.3866080045700073, + -1.4945378303527832, + 0.4180656969547272, + 0.08673545718193054, + -0.30675017833709717, + -0.8370212316513062, + -0.3124745488166809, + -3.2861690521240234, + -2.5144338607788086, + 0.7133253216743469, + 0.5272455215454102, + 0.6288780570030212, + -2.7024707794189453, + 1.0730262994766235, + -2.603350877761841, + -1.8104162216186523 + ], + [ + 2.0841901302337646, + 0.18348857760429382, + -1.4159170389175415, + -0.5868262052536011, + -0.3290694057941437, + -0.8189557790756226, + -1.811798334121704, + 0.8927561044692993, + -0.4510294198989868, + -0.1373635083436966, + -0.46978816390037537, + -0.2017078548669815, + -3.1228857040405273, + -2.619961977005005, + 0.7154320478439331, + 0.4623824954032898, + 0.8127838373184204, + -2.5683043003082275, + 0.8921399116516113, + -2.799823522567749, + -1.6220401525497437 + ], + [ + 1.5681514739990234, + -0.11813957989215851, + -1.066882848739624, + -0.4835556447505951, + -0.4833664894104004, + -0.1345365047454834, + -1.872452735900879, + 1.059093952178955, + -0.3537854254245758, + 0.5130321979522705, + -0.7769420146942139, + -0.4462882876396179, + -3.249842882156372, + -2.2144932746887207, + 0.6907999515533447, + 0.14545467495918274, + 0.8816275596618652, + -2.4357430934906006, + 1.424092411994934, + -2.656961441040039, + -2.3529839515686035 + ], + [ + 2.0935373306274414, + 0.20671290159225464, + -1.0638595819473267, + -0.37751641869544983, + -0.5913090705871582, + -0.3702518939971924, + -1.3611384630203247, + 0.8432458639144897, + 0.4395832121372223, + -0.19578908383846283, + -1.2028591632843018, + -0.07535962760448456, + -3.2909016609191895, + -2.9216225147247314, + 1.1581575870513916, + 0.9284459352493286, + -0.3347962498664856, + -1.9277539253234863, + 0.8971017003059387, + -3.245394229888916, + -2.1499102115631104 + ], + [ + 2.5108895301818848, + -0.2977973222732544, + -1.458537220954895, + -0.5705423355102539, + -1.4453105926513672, + -0.4854125380516052, + -2.383378028869629, + 0.39735814929008484, + -0.3523370325565338, + 0.4586251974105835, + -1.2388068437576294, + 0.1082226037979126, + -3.5858566761016846, + -1.8809643983840942, + -0.18688076734542847, + 0.0871247947216034, + 0.0062812864780426025, + -3.012958526611328, + 1.0266033411026, + -3.1649136543273926, + -2.6460556983947754 + ], + [ + 1.2948591709136963, + 0.21422013640403748, + -1.3904463052749634, + -0.13111025094985962, + -0.1661812961101532, + -0.4983506500720978, + -2.0006139278411865, + -0.4048727750778198, + 0.10130663216114044, + 0.1955089420080185, + -0.27839744091033936, + 0.8026529550552368, + -3.2792744636535645, + -1.6839708089828491, + 0.6468846201896667, + -0.2772523760795593, + 0.45975640416145325, + -2.612403631210327, + 1.6571701765060425, + -1.8971503973007202, + -1.2593979835510254 + ], + [ + 0.9154124855995178, + -0.2073056846857071, + -2.1218466758728027, + -0.4230979084968567, + -1.2478010654449463, + -0.5439814925193787, + -1.185287594795227, + -0.06685006618499756, + -0.7776275873184204, + -0.015365853905677795, + -1.3256038427352905, + 0.46163398027420044, + -2.8853814601898193, + -2.387786865234375, + 1.2300693988800049, + 0.9541484117507935, + 0.29864931106567383, + -2.931433916091919, + 1.3643410205841064, + -2.001633882522583, + -2.6548502445220947 + ], + [ + 2.035454750061035, + 0.03935282677412033, + -1.9080866575241089, + -0.17657285928726196, + -1.4843900203704834, + -0.4207392632961273, + -1.554489016532898, + 0.1909814178943634, + -0.5061874389648438, + -0.420978844165802, + -0.7634562849998474, + -0.7818551063537598, + -3.374903678894043, + -2.5324442386627197, + 1.2213574647903442, + 0.9676649570465088, + 0.5052346587181091, + -2.0974624156951904, + 0.9588823914527893, + -3.6367201805114746, + -1.953636646270752 + ], + [ + 1.4856661558151245, + 0.3259274363517761, + -2.1635096073150635, + -0.8793277740478516, + 0.2686290740966797, + -0.340938925743103, + -0.8516338467597961, + -0.3830271363258362, + 0.5376158356666565, + -0.2721732258796692, + -1.2276334762573242, + -0.38624608516693115, + -3.346294403076172, + -3.093522071838379, + 0.6735774278640747, + 0.5564011931419373, + -0.5450864434242249, + -1.5815953016281128, + 1.4068235158920288, + -2.7531042098999023, + -2.515698194503784 + ] + ], + "predicted_scores": [ + 9.033143997192383, + 8.816910743713379, + 8.61434555053711, + 8.342315673828125, + 8.196283340454102, + 7.688064098358154, + 7.435515880584717, + 7.300971031188965 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565621826/train0", + "selected_index": 0, + "state_hash": "f6edd849b0b51788d7be25e402fb514cb777ccf220797faaf769000814df02cb", + "stored_base_utility": 0.1373138427734375, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08252689987421036, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08252689987421036 + }, + "base_progress": 0.08252689987421036, + "base_success": false, + "base_utility": 0.08252689987421036, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09974726289510727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09974726289510727 + }, + { + "contact_quality": null, + "progress": 0.06912392377853394, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06912392377853394 + }, + { + "contact_quality": null, + "progress": 0.04023398831486702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04023398831486702 + }, + { + "contact_quality": null, + "progress": 0.07608189433813095, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07608189433813095 + }, + { + "contact_quality": null, + "progress": 0.10834071785211563, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10834071785211563 + }, + { + "contact_quality": null, + "progress": 0.059768736362457275, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.059768736362457275 + }, + { + "contact_quality": null, + "progress": 0.06140579655766487, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06140579655766487 + }, + { + "contact_quality": null, + "progress": 0.06661012023687363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06661012023687363 + } + ], + "candidate_progress": [ + 0.09974726289510727, + 0.06912392377853394, + 0.04023398831486702, + 0.07608189433813095, + 0.10834071785211563, + 0.059768736362457275, + 0.06140579655766487, + 0.06661012023687363 + ], + "candidate_source_chart_ids": [ + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-206099bab6bfe7c84fce", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-387e6771a55be98f1a59", + "generated_tangents": [ + [ + 0.31287968158721924, + -0.04098396748304367, + 0.6680215001106262, + 0.6064445972442627, + 1.366408109664917, + -1.0435786247253418, + -0.5852083563804626, + -0.37832167744636536, + 0.049432963132858276, + -0.18769046664237976, + 0.7405737042427063, + 0.09098008275032043, + 0.30926015973091125, + -0.6268219947814941, + 0.13203081488609314, + 0.1299392133951187, + 0.13280877470970154, + -0.4919320344924927, + -0.3845571279525757, + -0.08100757002830505, + -0.7122464179992676 + ], + [ + 0.053921133279800415, + -0.11941242218017578, + 0.07218372821807861, + 0.366675466299057, + 1.1164110898971558, + -0.686102032661438, + -0.5308035016059875, + -0.21884965896606445, + 0.00896742008626461, + -0.19040140509605408, + 0.6187528371810913, + 0.06797173619270325, + -0.15170753002166748, + -0.44486385583877563, + 0.18360966444015503, + 0.04875386506319046, + 0.027315065264701843, + -0.30808204412460327, + -0.34212684631347656, + -0.3659634590148926, + -0.08947956562042236 + ], + [ + 0.009828701615333557, + -0.06292109191417694, + 0.21970674395561218, + 0.32178470492362976, + 1.2790746688842773, + -0.6256959438323975, + -0.1923474669456482, + -0.31916186213493347, + 0.08678598701953888, + 0.012465961277484894, + 0.4704698622226715, + 0.1250826120376587, + -0.3155871629714966, + -0.24395690858364105, + 0.22821149230003357, + 0.05037960037589073, + 0.056975677609443665, + -0.43167275190353394, + -0.4937831163406372, + -0.20799744129180908, + -0.05310693383216858 + ], + [ + -0.0005264431238174438, + -0.04468899592757225, + 0.024656325578689575, + 0.31170758605003357, + 1.4589426517486572, + 0.13515056669712067, + -0.47113358974456787, + -0.059382736682891846, + 0.02040584199130535, + -0.14974522590637207, + 0.5188410878181458, + 0.2623038589954376, + -0.12810590863227844, + -0.5048943758010864, + 0.00869831070303917, + 0.1569286435842514, + 0.10765936225652695, + -0.27352994680404663, + -0.46486592292785645, + 0.20116907358169556, + 0.2253880500793457 + ], + [ + 0.08948926627635956, + -0.24587205052375793, + 0.561453104019165, + -0.09249259531497955, + 0.8570199012756348, + -0.9929342269897461, + -0.33456337451934814, + 0.22664164006710052, + 0.05681823939085007, + 0.11038579791784286, + 0.7123820781707764, + -0.02159436047077179, + -0.24912689626216888, + -0.9577621221542358, + 0.5930779576301575, + -0.02983243763446808, + -0.29516899585723877, + -0.28703922033309937, + -0.3999032974243164, + -0.13385555148124695, + -0.524699330329895 + ], + [ + 0.09799520671367645, + -0.22077828645706177, + 0.20052370429039001, + 0.021120905876159668, + 0.22559405863285065, + -1.1440191268920898, + -0.09133774042129517, + 0.21286557614803314, + -0.010845828801393509, + -0.026369161903858185, + 0.7046958208084106, + -0.4672173857688904, + -0.5708131194114685, + -0.10577349364757538, + 0.16442500054836273, + 0.022730842232704163, + 0.1690899133682251, + -0.20347245037555695, + -0.3841298222541809, + 0.2875376343727112, + 0.3768569529056549 + ], + [ + 0.12132717669010162, + -0.24124819040298462, + 0.19241198897361755, + 0.03107506036758423, + 0.39432281255722046, + -1.046919345855713, + -0.16484200954437256, + 0.23667003214359283, + -0.018077827990055084, + -0.016010873019695282, + 0.7737922072410583, + -0.4034934639930725, + -0.6087322235107422, + -0.25388848781585693, + 0.163319393992424, + -0.011344634927809238, + 0.0313936322927475, + -0.08741562068462372, + -0.3662693500518799, + 0.2606653571128845, + 0.39731377363204956 + ], + [ + 0.09076237678527832, + -0.22154167294502258, + 0.11351713538169861, + 0.11486765742301941, + 0.2227010577917099, + -1.207038402557373, + -0.16412675380706787, + 0.250341534614563, + 0.08164624869823456, + 0.048383429646492004, + 0.8020948171615601, + -0.34508824348449707, + -0.6843000650405884, + -0.226591095328331, + 0.1049489825963974, + 0.028447404503822327, + 0.11986105144023895, + -0.16639776527881622, + -0.350197434425354, + 0.3151845932006836, + 0.3616069257259369 + ] + ], + "generated_utilities": [ + 0.09974726289510727, + 0.06912392377853394, + 0.04023398831486702, + 0.07608189433813095, + 0.10834071785211563, + 0.059768736362457275, + 0.06140579655766487, + 0.06661012023687363 + ], + "group_id": "ms-387e6771a55be98f1a59", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.03822460025548935, + 0.18199902772903442, + 0.20744429528713226, + 0.05679210275411606, + 0.07646497339010239, + 0.038853906095027924, + 0.0824488028883934, + 0.05751838535070419, + 0.02475176379084587, + 0.040490686893463135, + 0.027167975902557373, + 0.06515108048915863, + 0.04604649543762207, + 0.014139334671199322, + 0.0258652176707983, + 0.034187138080596924 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0643576979637146, + 0.02293895184993744, + 0.03272289037704468, + 0.02799379825592041, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + 0.05611860752105713, + -0.010578691959381104, + 0.03396539017558098, + 0.754948616027832, + -0.12030506134033203, + 0.05740344524383545, + 0.0, + 0.01674991101026535, + 0.0017660856246948242, + 0.1110122799873352, + -0.29033517837524414, + -0.015073239803314209, + -0.062492311000823975, + 0.0 + ], + [ + 0.03256732225418091, + 0.029803231358528137, + -0.11524808406829834, + -0.03975573182106018, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + -0.11635321378707886, + -0.12916040420532227, + -0.10934318602085114, + 0.754948616027832, + -0.10127544403076172, + 0.10553282499313354, + 0.0, + 0.2124219685792923, + -0.13985675573349, + 0.20122885704040527, + -0.29033517837524414, + -0.09104716777801514, + -0.33053046464920044, + 0.0 + ] + ], + "predicted_scores": [ + 11.25488567352295, + 8.949102401733398, + 8.357065200805664, + 7.5187668800354, + 7.139684200286865, + 6.768582820892334, + 6.739411354064941, + 6.115901947021484 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train0", + "selected_index": 0, + "state_hash": "6b753a027f9378ff308946331f6bb658d452d8e77220ab0cd6fc813be0ac75cc", + "stored_base_utility": 0.03822460025548935, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03917040303349495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03917040303349495 + }, + "base_progress": 0.03917040303349495, + "base_success": false, + "base_utility": 0.03917040303349495, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12086125463247299, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12086125463247299 + }, + { + "contact_quality": null, + "progress": 0.0998828187584877, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0998828187584877 + }, + { + "contact_quality": null, + "progress": 0.03178328275680542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03178328275680542 + }, + { + "contact_quality": null, + "progress": 0.05774128437042236, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05774128437042236 + }, + { + "contact_quality": null, + "progress": 0.09802126884460449, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09802126884460449 + }, + { + "contact_quality": null, + "progress": 0.0678229108452797, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0678229108452797 + }, + { + "contact_quality": null, + "progress": 0.04722684621810913, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04722684621810913 + }, + { + "contact_quality": null, + "progress": 0.0725700631737709, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0725700631737709 + } + ], + "candidate_progress": [ + 0.12086125463247299, + 0.0998828187584877, + 0.03178328275680542, + 0.05774128437042236, + 0.09802126884460449, + 0.0678229108452797, + 0.04722684621810913, + 0.0725700631737709 + ], + "candidate_source_chart_ids": [ + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-7dedf60ad41ea2d92ee5", + "ms-98eb81debb5186ebbe38", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-be219977a9db8dbb1e58" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-38f7298184a2c5005e94", + "generated_tangents": [ + [ + 0.5549951791763306, + -0.011571042239665985, + -0.2968709170818329, + -0.005292847752571106, + 0.23439687490463257, + -2.1923131942749023, + 0.06370526552200317, + 0.4421412944793701, + -0.026232538744807243, + 0.12058635801076889, + -0.8667294383049011, + 1.0839509963989258, + -2.001406192779541, + -0.4074585437774658, + 0.7603763341903687, + 0.44663211703300476, + -0.5391051769256592, + 0.19690975546836853, + 0.6973133087158203, + -1.2703843116760254, + -1.0618451833724976 + ], + [ + 0.39722830057144165, + -0.1376485377550125, + -0.1670621931552887, + -0.06577108800411224, + 0.7057873010635376, + -1.2404489517211914, + 0.1869962215423584, + 0.3036459982395172, + 0.05380883812904358, + 0.5606752634048462, + -0.44692346453666687, + 1.2536674737930298, + -2.245656967163086, + -0.0013831406831741333, + 0.8167431950569153, + 0.4715690314769745, + -0.32873111963272095, + 0.38793396949768066, + 0.5243843793869019, + -1.7466576099395752, + -0.9168370962142944 + ], + [ + -0.08484864234924316, + -0.34610819816589355, + -0.5074306726455688, + 0.17315590381622314, + 0.8245617747306824, + -0.8755991458892822, + 0.5550235509872437, + 0.3240965008735657, + -0.08381851017475128, + 0.44868627190589905, + -0.3857881724834442, + 1.4537556171417236, + -0.32488834857940674, + 0.812469482421875, + 0.5597079992294312, + 0.34746792912483215, + -0.11295543611049652, + 0.10157401114702225, + 0.5262054204940796, + -2.596778631210327, + -0.43700775504112244 + ], + [ + 0.289387583732605, + -0.23139259219169617, + 0.6175347566604614, + -0.005333542823791504, + 0.717567503452301, + -1.312751293182373, + 0.08327877521514893, + 0.3430246114730835, + 0.019609035924077034, + 0.5139692425727844, + -0.43625301122665405, + 1.0173360109329224, + -1.7371784448623657, + -0.0010738074779510498, + 0.3718268871307373, + 0.4355454444885254, + -0.1140870600938797, + 0.027144625782966614, + 0.4506351351737976, + -2.010155439376831, + -1.7357187271118164 + ], + [ + 0.6307879686355591, + -0.02272743359208107, + -0.17862418293952942, + 0.12746986746788025, + 0.792668342590332, + -1.7350504398345947, + 0.04134935140609741, + 0.1101926937699318, + 0.06904052942991257, + 0.26937684416770935, + -0.5020616054534912, + 1.1872003078460693, + -1.7481523752212524, + -0.26263177394866943, + 0.5534759163856506, + 0.44994625449180603, + -0.40802204608917236, + 0.36935731768608093, + 0.6418775320053101, + -1.568533182144165, + -1.0431292057037354 + ], + [ + 0.5947062373161316, + -0.04412045329809189, + -0.1653849184513092, + 0.14315366744995117, + 0.8885510563850403, + -1.604752540588379, + 0.026499778032302856, + 0.02529023587703705, + 0.07011409848928452, + 0.2430332601070404, + -0.46852627396583557, + 1.2573213577270508, + -1.7811534404754639, + -0.24917252361774445, + 0.4690021574497223, + 0.40950876474380493, + -0.45773476362228394, + 0.2865621745586395, + 0.7381479740142822, + -1.572784662246704, + -0.9362776279449463 + ], + [ + 0.29183512926101685, + -0.1997244954109192, + 0.10648375749588013, + -0.16376827657222748, + 0.8216965198516846, + -1.3862097263336182, + 0.352537602186203, + 0.21297241747379303, + 0.059821225702762604, + 0.3932165801525116, + -0.3990858495235443, + 1.1140875816345215, + -1.1900757551193237, + -0.06085972487926483, + 0.05613749101758003, + 0.48587721586227417, + 0.07300464808940887, + 0.05014313757419586, + 0.40683120489120483, + -2.082291841506958, + -1.4638994932174683 + ], + [ + 0.5243908762931824, + 0.08422103524208069, + 0.1465301513671875, + -0.20865270495414734, + 0.6889839172363281, + -1.152407169342041, + 0.11039990186691284, + 0.22387364506721497, + 0.020331615582108498, + 0.8254491686820984, + -0.5028030872344971, + 0.660714864730835, + -0.8017787337303162, + 0.3961716294288635, + 0.4127199649810791, + 0.31803300976753235, + 0.273790568113327, + -0.3687719702720642, + 0.6621459722518921, + -1.9532408714294434, + -0.9297870397567749 + ] + ], + "generated_utilities": [ + 0.12086125463247299, + 0.0998828187584877, + 0.03178328275680542, + 0.05774128437042236, + 0.09802126884460449, + 0.0678229108452797, + 0.04722684621810913, + 0.0725700631737709 + ], + "group_id": "ms-38f7298184a2c5005e94", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1194220781326294, + 0.1767096221446991, + 0.0505807027220726, + 0.026383202522993088, + 0.09035396575927734, + 0.11899666488170624, + 0.059947751462459564, + 0.08427222818136215, + 0.041255753487348557, + 0.05034176632761955, + 0.02779809758067131, + 0.011037330143153667, + 0.020422041416168213, + 0.049474477767944336, + 0.03517866134643555, + 0.02796018123626709 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.08923250436782837, + 0.05675226449966431, + -0.1250607967376709, + -0.07566308975219727, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.001595139503479004, + 0.026048749685287476, + 0.1681501865386963, + -0.11568224430084229, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.18507611751556396, + -0.15127414464950562, + 0.07637521624565125, + -0.15284772217273712, + 0.0320390909910202, + -1.8774652481079102, + 0.0 + ], + [ + -0.2749255299568176, + 0.2578675150871277, + -0.46759605407714844, + -0.5041444897651672, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.13614141941070557, + 0.11120480298995972, + 0.15478569269180298, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.01827770471572876, + -0.38639169931411743, + -0.18168723583221436, + 0.3532869219779968, + -0.5437159538269043, + -1.8774652481079102, + 0.0 + ], + [ + 0.4466920495033264, + 0.07532927393913269, + 0.8145812749862671, + -0.963495135307312, + 1.3846542835235596, + -3.0133423805236816, + -1.0476078987121582, + 0.6759306788444519, + 0.5485053658485413, + 0.634345293045044, + -0.7682492733001709, + 2.2281322479248047, + -2.7100830078125, + -0.8660445213317871, + 0.7648183703422546, + 0.607989490032196, + -0.6848516464233398, + -0.06598936021327972, + 0.2030458152294159, + -2.9672060012817383, + -1.4293488264083862 + ], + [ + 1.0058056116104126, + 0.3426892161369324, + 0.6375484466552734, + -0.5828341841697693, + 1.7963712215423584, + -3.3760194778442383, + -0.9086123704910278, + 0.20509907603263855, + 0.10472571849822998, + 1.1158980131149292, + -1.0710231065750122, + 2.203950881958008, + -2.4466300010681152, + -0.3184358477592468, + 0.43084120750427246, + 0.32133060693740845, + -0.6005205512046814, + 0.07880236953496933, + 0.52874755859375, + -2.7805917263031006, + -1.6972103118896484 + ], + [ + 0.48786461353302, + -0.007949680089950562, + 0.41629794239997864, + -0.2277727723121643, + 1.0953291654586792, + -2.7825636863708496, + -1.7178279161453247, + 0.8412191271781921, + 0.6389099359512329, + 0.7100303769111633, + -1.1701807975769043, + 2.8546230792999268, + -2.570579767227173, + -0.2383948564529419, + 0.4282584488391876, + 0.27555936574935913, + -0.4073549509048462, + 0.49247825145721436, + 0.6483681797981262, + -3.3716540336608887, + -1.891394853591919 + ], + [ + 0.17139381170272827, + 0.831203818321228, + 0.1966383457183838, + -1.5011540651321411, + 1.811257004737854, + -2.6416001319885254, + -1.8791863918304443, + 0.03218984603881836, + 1.0182924270629883, + 1.5643268823623657, + -1.2954871654510498, + 2.249680757522583, + -1.9383416175842285, + -1.4303685426712036, + 0.48937559127807617, + 0.2848057746887207, + -0.6113857626914978, + 0.6453920006752014, + -0.13643476366996765, + -2.514732599258423, + -1.8046306371688843 + ], + [ + 0.1380026936531067, + 0.5165722370147705, + 0.9639596939086914, + -0.3682268261909485, + 2.2811193466186523, + -3.5902557373046875, + -1.047336220741272, + 1.3064022064208984, + 0.7264143228530884, + 1.3356540203094482, + -0.41242116689682007, + 1.8184869289398193, + -3.016441822052002, + -0.8501555919647217, + 1.0147737264633179, + 1.0417438745498657, + -0.24422766268253326, + 0.3441629707813263, + 1.075059175491333, + -3.178764820098877, + -1.7388081550598145 + ], + [ + 1.0805304050445557, + -0.5906903743743896, + -0.32329434156417847, + -1.7024246454238892, + 1.3012806177139282, + -3.3655173778533936, + -2.0100932121276855, + 0.28595322370529175, + -0.42309290170669556, + 0.9014679193496704, + -0.11985284090042114, + 3.033172607421875, + -1.890408992767334, + -0.049424707889556885, + 0.8836197853088379, + 0.07151538133621216, + -0.6898007392883301, + 0.6614744663238525, + 0.7956362962722778, + -3.344180107116699, + -1.4918344020843506 + ], + [ + 1.4192681312561035, + -0.5825785994529724, + 0.11329725384712219, + -0.08465814590454102, + 1.0504729747772217, + -2.438842535018921, + -0.9858216047286987, + -0.35852980613708496, + 1.4301284551620483, + 0.37676453590393066, + -1.3725404739379883, + 3.114079475402832, + -2.0786290168762207, + -0.5835205912590027, + 0.1985606551170349, + 0.260464072227478, + 0.41686397790908813, + -0.104665607213974, + 0.6382055282592773, + -2.7474825382232666, + -1.663126826286316 + ], + [ + 1.0832164287567139, + 0.11555392295122147, + 1.2108526229858398, + -1.5446858406066895, + 0.9112894535064697, + -3.3027408123016357, + -1.2419135570526123, + -0.057880282402038574, + -0.07349961996078491, + 0.026286423206329346, + -0.897603452205658, + 3.2857306003570557, + -2.232046365737915, + -1.2509676218032837, + 1.1900405883789062, + 0.8446143269538879, + -1.339659333229065, + -0.07882180064916611, + 0.1166829764842987, + -2.662961721420288, + -1.0079293251037598 + ], + [ + 0.43863287568092346, + -0.2024492472410202, + 0.0567094087600708, + -0.3029903173446655, + 1.612892746925354, + -3.26739764213562, + -1.7898839712142944, + -0.1649491786956787, + 0.2927369475364685, + 1.020188808441162, + -0.10171866416931152, + 2.3230679035186768, + -2.615206003189087, + -0.08625847101211548, + 0.9057100415229797, + 1.4201092720031738, + -0.3173455595970154, + 0.09508261829614639, + 0.9565507173538208, + -2.2954390048980713, + -1.2229193449020386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.03360384702682495, + -0.028338372707366943, + -0.12341004610061646, + -0.04655510187149048, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + 0.032517433166503906, + -0.02922782301902771, + -0.033792078495025635, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + -0.06465893983840942, + -0.026740312576293945, + 0.046701133251190186, + -0.06953387707471848, + 0.02321198582649231, + -1.8774652481079102, + 0.0 + ] + ], + "predicted_scores": [ + 10.316132545471191, + 7.8692193031311035, + 7.442691326141357, + 7.317057132720947, + 7.183282375335693, + 7.134940147399902, + 5.849112510681152, + 4.547755718231201 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train0", + "selected_index": 0, + "state_hash": "dee046e5a3c9b5e78be27f1df1f4f235344ec818a1a7439afe5f4cb30139cf34", + "stored_base_utility": 0.1194220781326294, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.009124159812927246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009124159812927246 + }, + "base_progress": 0.009124159812927246, + "base_success": false, + "base_utility": 0.009124159812927246, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.00878363847732544, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00878363847732544 + }, + { + "contact_quality": null, + "progress": 0.005450725555419922, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005450725555419922 + }, + { + "contact_quality": null, + "progress": 0.008500377647578716, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008500377647578716 + }, + { + "contact_quality": null, + "progress": 0.011097331531345844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011097331531345844 + }, + { + "contact_quality": null, + "progress": 0.0051685175858438015, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0051685175858438015 + }, + { + "contact_quality": null, + "progress": 0.0060965814627707005, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0060965814627707005 + }, + { + "contact_quality": null, + "progress": 0.007386326789855957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007386326789855957 + }, + { + "contact_quality": null, + "progress": 0.0030460357666015625, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0030460357666015625 + } + ], + "candidate_progress": [ + 0.00878363847732544, + 0.005450725555419922, + 0.008500377647578716, + 0.011097331531345844, + 0.0051685175858438015, + 0.0060965814627707005, + 0.007386326789855957, + 0.0030460357666015625 + ], + "candidate_source_chart_ids": [ + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-397e502908d16b7804b0", + "ms-6758deb06644a14aa722", + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-6758deb06644a14aa722" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-482c5c8d208b5907a2c2", + "generated_tangents": [ + [ + -0.049815550446510315, + -0.04656240716576576, + 0.705135703086853, + -0.0781581848859787, + 2.057030200958252, + 0.6591335535049438, + 0.360661119222641, + 0.22798390686511993, + 0.03546068072319031, + 1.0741779804229736, + -0.45267316699028015, + 1.567286729812622, + 1.4588344097137451, + 0.44813960790634155, + 0.10778860002756119, + -0.0587795116007328, + 0.148574560880661, + 0.2691079378128052, + 0.3142581582069397, + 0.273465096950531, + -1.0474905967712402 + ], + [ + -0.23655660450458527, + -0.041145652532577515, + 0.6016224026679993, + 0.012054666876792908, + 2.1290836334228516, + 0.7756989002227783, + 0.4296463429927826, + 0.1317601501941681, + 0.051725782454013824, + 1.3776813745498657, + -0.4998546540737152, + 1.4107518196105957, + 1.6778227090835571, + 0.28998851776123047, + 0.08754293620586395, + -0.10128724575042725, + 0.36676013469696045, + 0.2597346305847168, + 0.49081090092658997, + 0.30386966466903687, + -0.8922455310821533 + ], + [ + -0.5773165822029114, + 0.25543367862701416, + 0.9771597385406494, + -0.27743569016456604, + 1.397900104522705, + 0.6213514804840088, + 0.20530939102172852, + 0.18989403545856476, + 0.10289112478494644, + 1.0490386486053467, + -0.5493795871734619, + 1.824069857597351, + 1.4883545637130737, + 0.08942125737667084, + 0.04887757450342178, + -0.17101269960403442, + -0.18515239655971527, + 0.5316981077194214, + 0.5965712070465088, + -0.3456134796142578, + -0.9148825407028198 + ], + [ + -0.06939999759197235, + -0.048543453216552734, + 0.7926576137542725, + 0.0693444311618805, + 1.4298961162567139, + 0.38798975944519043, + 0.5715478658676147, + 0.19409309327602386, + 0.18216733634471893, + 1.0031983852386475, + -0.7306927442550659, + 1.1852830648422241, + 0.12281766533851624, + 0.47779446840286255, + -0.027130860835313797, + -0.14800427854061127, + -0.04451192915439606, + 0.6661083698272705, + 0.2828511893749237, + 0.4495900273323059, + -0.5427080392837524 + ], + [ + -0.29909998178482056, + 0.14783556759357452, + 0.8953452110290527, + 0.042054951190948486, + 2.0431716442108154, + 0.6359003782272339, + 0.04129081964492798, + 0.04279421269893646, + 0.13073933124542236, + 1.0399121046066284, + -0.43709564208984375, + 1.3877813816070557, + 1.1673877239227295, + 0.07621181011199951, + 0.08724887669086456, + -0.23650838434696198, + 0.2798236310482025, + 0.48398661613464355, + 0.5118979215621948, + 0.1778949499130249, + -0.7096110582351685 + ], + [ + -0.31867051124572754, + 0.160978764295578, + 0.771454930305481, + -0.025081157684326172, + 1.8512072563171387, + 0.6680611371994019, + -0.13499385118484497, + 0.07672888785600662, + 0.0857548788189888, + 1.0777310132980347, + -0.5266196727752686, + 1.4448391199111938, + 0.9337642192840576, + 0.005745984613895416, + 0.07911935448646545, + -0.2396399974822998, + 0.23839005827903748, + 0.5045702457427979, + 0.43266230821609497, + 0.1670060157775879, + -0.67417311668396 + ], + [ + -0.16751603782176971, + -0.0066785626113414764, + 0.857994556427002, + -0.015456080436706543, + 2.1056907176971436, + 0.6720499992370605, + 0.4198402464389801, + 0.139546737074852, + 0.14446470141410828, + 1.180782675743103, + -0.46742770075798035, + 1.5358421802520752, + 1.3812111616134644, + 0.4399641156196594, + 0.1323907971382141, + -0.026886098086833954, + 0.13452789187431335, + 0.1520424336194992, + 0.42508020997047424, + 0.2228962779045105, + -1.1466140747070312 + ], + [ + -0.06298300623893738, + -0.12911100685596466, + 0.8522658348083496, + 0.73267662525177, + 1.7848231792449951, + 0.3261961340904236, + 0.15888312458992004, + -0.08184540271759033, + 0.037027791142463684, + 1.0633642673492432, + -0.5972768068313599, + 1.5755444765090942, + 1.2024102210998535, + 0.6883900165557861, + 0.26560884714126587, + -0.06333936750888824, + 0.11718880385160446, + 0.2505548894405365, + 0.793315052986145, + 0.9461155533790588, + -0.7721607685089111 + ] + ], + "generated_utilities": [ + 0.00878363847732544, + 0.005450725555419922, + 0.008500377647578716, + 0.011097331531345844, + 0.0051685175858438015, + 0.0060965814627707005, + 0.007386326789855957, + 0.0030460357666015625 + ], + "group_id": "ms-482c5c8d208b5907a2c2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0359329991042614, + 0.1834145486354828, + 0.04271410033106804, + 0.03202331066131592, + 0.12089761346578598, + 0.03575414419174194, + 0.07773216813802719, + 0.1062050461769104, + 0.10849495977163315, + 0.08369352668523788, + 0.14167599380016327, + 0.008830646984279156, + 0.014355997554957867, + 0.02518286369740963, + 0.07873731851577759, + 0.015250584110617638 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.007817894220352173, + 0.01831631362438202, + 0.14601469039916992, + 0.10826835036277771, + 1.6591682434082031, + -0.005238652229309082, + 0.0, + -0.02753499150276184, + -0.009624123573303223, + 0.5762790441513062, + -0.3171614408493042, + 1.4374253749847412, + 0.7921857833862305, + 0.0, + -0.037753358483314514, + -0.025671005249023438, + 0.027724146842956543, + 0.4680846929550171, + -0.08415359258651733, + 0.033902376890182495, + 0.0 + ], + [ + 0.4543180465698242, + 0.20962828397750854, + 1.14601469039917, + -0.33832865953445435, + 2.659168243408203, + -0.7141264081001282, + 0.3707960247993469, + 0.21857939660549164, + 0.6796972155570984, + 1.5762790441513062, + -1.3171614408493042, + 2.437425374984741, + 1.7921857833862305, + 1.7551628351211548, + -0.19566796720027924, + 0.6842412948608398, + 0.11985036730766296, + 1.468084692955017, + 0.29799214005470276, + 0.3715361952781677, + 0.2624160647392273 + ], + [ + 0.2115812450647354, + -0.09577503800392151, + 0.7509078979492188, + -0.6239187121391296, + 2.4163239002227783, + -0.5409671664237976, + 0.16154196858406067, + 0.5615253448486328, + 1.0765573978424072, + 1.4198161363601685, + -1.430187463760376, + 2.5212466716766357, + 1.9029841423034668, + 1.5550287961959839, + -0.5099657773971558, + 1.0103299617767334, + -0.21028786897659302, + 1.4441221952438354, + 0.5605546236038208, + 0.6799722909927368, + 0.5554409027099609 + ], + [ + 0.46792101860046387, + 0.08014003932476044, + 1.5437835454940796, + 0.08789932727813721, + 2.3046298027038574, + -1.1965231895446777, + 0.8527317047119141, + 0.22510167956352234, + 1.0473172664642334, + 1.3720369338989258, + -1.2376489639282227, + 2.066061019897461, + 2.033992052078247, + 2.166865825653076, + -0.46199846267700195, + 1.1675353050231934, + -0.22657811641693115, + 1.7576167583465576, + 0.766140878200531, + 0.1856600046157837, + 0.008163660764694214 + ], + [ + 0.9431964755058289, + 0.6587716341018677, + 0.8381389379501343, + -0.8407833576202393, + 2.3666741847991943, + -1.05813729763031, + 0.1429125964641571, + 0.10640819370746613, + 1.0277029275894165, + 1.193630576133728, + -0.7695048451423645, + 2.9484994411468506, + 1.1866973638534546, + 1.5341145992279053, + -0.1225738674402237, + 0.9121334552764893, + 0.7116963863372803, + 0.8198207020759583, + 0.6562460660934448, + 0.1118304431438446, + -0.2320157289505005 + ] + ], + "predicted_scores": [ + 9.43040657043457, + 9.323161125183105, + 8.993653297424316, + 8.677950859069824, + 8.614483833312988, + 8.41909408569336, + 8.413841247558594, + 6.8669514656066895 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train0", + "selected_index": 0, + "state_hash": "903f78d5baa541d73612cac5e063bb553fba566c0d4adcb9603aa055d710fb15", + "stored_base_utility": 0.0359329991042614, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14663122594356537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14663122594356537 + }, + "base_progress": 0.14663122594356537, + "base_success": false, + "base_utility": 0.14663122594356537, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10735879093408585, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10735879093408585 + }, + { + "contact_quality": null, + "progress": 0.08596404641866684, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08596404641866684 + }, + { + "contact_quality": null, + "progress": 0.1273784190416336, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1273784190416336 + }, + { + "contact_quality": null, + "progress": 0.10580883175134659, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10580883175134659 + }, + { + "contact_quality": null, + "progress": 0.10845587402582169, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10845587402582169 + }, + { + "contact_quality": null, + "progress": 0.1056336760520935, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1056336760520935 + }, + { + "contact_quality": null, + "progress": 0.09012359380722046, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09012359380722046 + }, + { + "contact_quality": null, + "progress": 0.06587376445531845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06587376445531845 + } + ], + "candidate_progress": [ + 0.10735879093408585, + 0.08596404641866684, + 0.1273784190416336, + 0.10580883175134659, + 0.10845587402582169, + 0.1056336760520935, + 0.09012359380722046, + 0.06587376445531845 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-19cb10d10516bd0216a6", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-4da03300d92420e80ed0", + "generated_tangents": [ + [ + 2.634641170501709, + -0.3039514422416687, + -0.7532751560211182, + 0.6672413349151611, + -0.5675072073936462, + 1.9859477281570435, + 1.4871575832366943, + 2.147381067276001, + 0.20933063328266144, + -1.4684480428695679, + -0.5884499549865723, + -0.9568467140197754, + -3.2579658031463623, + -3.3139076232910156, + 1.7043448686599731, + -0.08272968232631683, + -1.1290931701660156, + 0.9141604900360107, + -0.08389303088188171, + -0.5761035680770874, + -0.5719590187072754 + ], + [ + 2.464007616043091, + -0.18512862920761108, + -0.8321398496627808, + 0.5767568349838257, + -0.30320900678634644, + 1.859495997428894, + 0.9342386722564697, + 1.911637306213379, + 0.153948575258255, + -1.4111170768737793, + -0.4420132339000702, + -0.8328026533126831, + -2.989961862564087, + -2.6137354373931885, + 1.9184253215789795, + -0.2443116456270218, + -1.2287453413009644, + 0.5091448426246643, + 0.23109352588653564, + -0.7274672985076904, + -0.08955064415931702 + ], + [ + 2.5132815837860107, + -0.15570500493049622, + -0.9391363859176636, + 0.5854440927505493, + -0.6245869398117065, + 1.7375870943069458, + 1.9102296829223633, + 2.0940334796905518, + 0.2903728485107422, + -1.451993703842163, + -0.693550169467926, + -0.9917333126068115, + -3.2797203063964844, + -3.627514123916626, + 2.05561900138855, + -0.2094503939151764, + -1.0858231782913208, + 0.8155262470245361, + -0.11378800868988037, + -0.6239851117134094, + -0.5379022359848022 + ], + [ + 2.5011184215545654, + -0.1403145045042038, + -0.8085476160049438, + 0.4468512237071991, + -0.660551905632019, + 1.8433722257614136, + 1.7551867961883545, + 2.2682418823242188, + 0.2240917980670929, + -1.4834834337234497, + -0.6019911766052246, + -1.0474611520767212, + -3.151282787322998, + -3.324010133743286, + 1.8510380983352661, + -0.21391025185585022, + -0.9325986504554749, + 1.0144681930541992, + -0.12915343046188354, + -0.7882421016693115, + -0.492827445268631 + ], + [ + 2.663836717605591, + -0.25584420561790466, + -1.0287872552871704, + 0.4594871699810028, + -0.5309082865715027, + 1.634170651435852, + 1.7936627864837646, + 1.9982304573059082, + 0.27105197310447693, + -1.551898717880249, + -0.5929011106491089, + -1.1161805391311646, + -3.272610664367676, + -3.361637830734253, + 1.9080802202224731, + -0.2649810016155243, + -1.1685264110565186, + 0.7430374026298523, + -0.1481437385082245, + -0.6536601781845093, + -0.8733140230178833 + ], + [ + 2.6133060455322266, + -0.1696094274520874, + -0.9842472076416016, + 0.4757023751735687, + -0.5343642830848694, + 1.8002469539642334, + 1.8543155193328857, + 2.150381326675415, + 0.24019616842269897, + -1.530577301979065, + -0.6877753138542175, + -0.9864175319671631, + -3.030122756958008, + -3.4371330738067627, + 2.08855938911438, + -0.29169246554374695, + -1.1221535205841064, + 0.8243769407272339, + -0.1287095844745636, + -0.7423315048217773, + -0.5376543998718262 + ], + [ + 2.6695475578308105, + -0.1514185220003128, + -0.8053789138793945, + 0.07773524522781372, + -0.48225104808807373, + 1.7529746294021606, + 1.6568171977996826, + 1.9272019863128662, + 0.28966644406318665, + -1.7189970016479492, + -0.49786052107810974, + -0.9074803590774536, + -2.829857587814331, + -3.054576873779297, + 1.805141568183899, + -0.4113864004611969, + -1.086623191833496, + 0.9642729163169861, + 0.018310487270355225, + -0.8757671117782593, + -0.7149254083633423 + ], + [ + 2.584070920944214, + -0.17364898324012756, + -0.796271562576294, + 0.599783182144165, + -0.2156740427017212, + 1.8570375442504883, + 0.8505010604858398, + 2.0467300415039062, + 0.22562220692634583, + -1.358185887336731, + -0.1811409592628479, + -0.8399637937545776, + -2.6548526287078857, + -2.437251329421997, + 1.983381986618042, + -0.22362051904201508, + -0.9771478772163391, + 0.6202883720397949, + 0.6414510011672974, + -0.07619726657867432, + -1.3737967014312744 + ] + ], + "generated_utilities": [ + 0.10735879093408585, + 0.08596404641866684, + 0.1273784190416336, + 0.10580883175134659, + 0.10845587402582169, + 0.1056336760520935, + 0.09012359380722046, + 0.06587376445531845 + ], + "group_id": "ms-4da03300d92420e80ed0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18184101581573486, + 0.1803790181875229, + 0.09926676750183105, + 0.20791122317314148, + 0.26338279247283936, + 0.2025860995054245, + 0.2575649917125702, + 0.23894403874874115, + 0.2174234390258789, + 2.0, + 0.1918075978755951, + 0.10365168750286102, + 2.0, + 2.0, + 2.0, + 0.15933696925640106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.1168947219848633, + 0.13667210936546326, + -0.051466986536979675, + 0.03360617160797119, + -0.19173425436019897, + 1.7646396160125732, + 0.0, + 0.9886866807937622, + 0.19114632904529572, + -0.3927978277206421, + -0.21108940243721008, + -0.031822919845581055, + -1.6683034896850586, + 0.0, + 0.9908627271652222, + -0.05301591753959656, + 0.08875280618667603, + 0.04224884510040283, + -0.03718748688697815, + -0.06952071189880371, + 0.0 + ], + [ + 2.1713812351226807, + -0.41443270444869995, + -0.23275701701641083, + 0.6524178981781006, + -0.25828492641448975, + 3.331055164337158, + 2.47529673576355, + 1.4984867572784424, + -0.396874338388443, + -0.725897490978241, + 0.1870076060295105, + -0.4014764428138733, + -2.239036798477173, + -2.218061923980713, + 1.6540865898132324, + 0.013689309358596802, + -0.22782719135284424, + 1.2475574016571045, + 0.7231295704841614, + -0.281164288520813, + -1.4593302011489868 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 2.1168947219848633, + 0.045049574226140976, + -0.25911271572113037, + 0.6112266778945923, + -0.5530642867088318, + 2.7646396160125732, + 1.9622430801391602, + 1.9886866807937622, + 0.28067120909690857, + -1.392797827720642, + -0.5793421268463135, + -1.031822919845581, + -2.6683034896850586, + -2.812993049621582, + 1.9908627271652222, + -0.28380459547042847, + -0.7305920720100403, + 0.7978808879852295, + 0.1574108749628067, + -1.0695207118988037, + -0.9408642053604126 + ], + [ + 2.228377103805542, + -0.19656506180763245, + -0.23844876885414124, + 0.6676552295684814, + -0.7639804482460022, + 2.937652111053467, + 1.882341980934143, + 1.8207013607025146, + 0.27702635526657104, + -1.549763798713684, + -0.5955427289009094, + -1.185685157775879, + -2.8421716690063477, + -3.054234266281128, + 1.691805362701416, + -0.37051287293434143, + -0.43653741478919983, + 0.514695405960083, + -0.0132378488779068, + -0.899186372756958, + -0.8349848985671997 + ], + [ + 2.425011157989502, + -0.03216857090592384, + -0.010275468230247498, + 0.4265761971473694, + -0.7737585306167603, + 3.1123812198638916, + 1.5686779022216797, + 2.112421989440918, + 0.3475198745727539, + -1.3452436923980713, + -0.6067132949829102, + -1.1150503158569336, + -2.461210250854492, + -2.8643558025360107, + 2.185579299926758, + -0.6223857402801514, + -0.8183724880218506, + 0.8074819445610046, + 0.536148726940155, + -0.7826522588729858, + -0.6682058572769165 + ], + [ + 1.6831411123275757, + -0.19312052428722382, + 0.010244876146316528, + 1.1032377481460571, + -0.4994945824146271, + 2.9000508785247803, + 2.187715768814087, + 2.200220823287964, + 0.4744641184806824, + -1.6715843677520752, + -0.493664413690567, + -1.459894061088562, + -2.90458345413208, + -2.3254852294921875, + 1.4726771116256714, + -0.21338894963264465, + -1.293912649154663, + 1.3146823644638062, + -0.09375743567943573, + -0.5040683150291443, + -0.9596792459487915 + ], + [ + 1.6738543510437012, + 0.6350066661834717, + 0.19669359922409058, + -0.1396545171737671, + -0.9625160694122314, + 2.207883596420288, + 1.7678128480911255, + 1.319273829460144, + -0.267609566450119, + -0.5747214555740356, + 0.19376236200332642, + -1.4490230083465576, + -2.6397883892059326, + -3.2567262649536133, + 1.2646806240081787, + -0.14523190259933472, + -0.050948500633239746, + 0.6943628787994385, + -0.1503342241048813, + -0.7313232421875, + -1.7930638790130615 + ], + [ + 2.139432668685913, + 0.34125545620918274, + 0.6370852589607239, + 0.8591548204421997, + 0.14890235662460327, + 2.6835381984710693, + 2.324920892715454, + 2.0091233253479004, + 0.13480046391487122, + -0.47407835721969604, + -1.102479100227356, + -1.0710523128509521, + -2.3226847648620605, + -3.6534740924835205, + 1.8767409324645996, + -0.01776304841041565, + -1.167153000831604, + 0.693484902381897, + 0.563054084777832, + -0.17544519901275635, + -0.8315125703811646 + ], + [ + 1.1254388093948364, + -0.03027276322245598, + -0.8753920793533325, + 0.3578681945800781, + -0.4256642460823059, + 1.7686536312103271, + 2.253190517425537, + 2.8274970054626465, + 0.3938461244106293, + -1.4084759950637817, + -0.7252857685089111, + -0.07288885116577148, + -1.7749395370483398, + -2.1030378341674805, + 1.92497980594635, + 0.20398634672164917, + -0.03521168231964111, + 1.4702520370483398, + -0.36331063508987427, + -1.5291211605072021, + -0.8369957804679871 + ] + ], + "predicted_scores": [ + 7.433258056640625, + 6.800236225128174, + 6.594235420227051, + 6.578792095184326, + 6.326344966888428, + 6.24533224105835, + 5.853445529937744, + 4.849090099334717 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1747178030/train0", + "selected_index": 0, + "state_hash": "87c74ddf97c53ddeeaae98467b4a13b1e0e499bd94eddb617759d39164cab542", + "stored_base_utility": 0.18184101581573486, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.021834811195731163, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021834811195731163 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.019551733508706093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019551733508706093 + }, + { + "contact_quality": null, + "progress": 0.024202564731240273, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024202564731240273 + }, + { + "contact_quality": null, + "progress": 0.02782593108713627, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02782593108713627 + }, + { + "contact_quality": null, + "progress": 0.005327919963747263, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005327919963747263 + }, + { + "contact_quality": null, + "progress": 0.00631558895111084, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00631558895111084 + } + ], + "candidate_progress": [ + 1.0, + 0.021834811195731163, + 1.0, + 0.019551733508706093, + 0.024202564731240273, + 0.02782593108713627, + 0.005327919963747263, + 0.00631558895111084 + ], + "candidate_source_chart_ids": [ + "ms-6a3add21affa4796f9be", + "ms-dc26b448cce4164a23e6", + "ms-0e592951fa32139b6dc9", + "ms-e84163d9ef2e7014daca", + "ms-f85a653af630821f2572", + "ms-6a3add21affa4796f9be", + "ms-e84163d9ef2e7014daca", + "ms-7eaea428dfdef7255a66" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + false, + true, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank3", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-57be9ae181c077e7821f", + "generated_tangents": [ + [ + -0.2797073721885681, + 0.13484108448028564, + 0.6566584706306458, + -0.4192536473274231, + -0.010051384568214417, + 1.6424551010131836, + 0.4706100821495056, + -0.018184825778007507, + -0.07831493020057678, + 0.726234495639801, + 0.32589784264564514, + 0.461823970079422, + 1.713438868522644, + 0.613983690738678, + 0.8218561410903931, + -0.05769515037536621, + -0.039725109934806824, + -0.0921669751405716, + -0.014564752578735352, + 0.924396812915802, + 0.1459539532661438 + ], + [ + -0.309101939201355, + 0.009493179619312286, + 0.9876956939697266, + -0.30612802505493164, + 0.5839683413505554, + 1.243936538696289, + 1.1610774993896484, + 0.04859870672225952, + -0.13851861655712128, + 0.8417831063270569, + -0.06353219598531723, + 0.38287553191185, + 1.6576346158981323, + 1.1466187238693237, + 0.6121523976325989, + -0.11832182109355927, + 0.20552831888198853, + 0.08858387917280197, + 0.24941027164459229, + 0.9831594824790955, + 0.9868859052658081 + ], + [ + -0.19697992503643036, + 0.09599083662033081, + 0.9275882244110107, + -0.28163638710975647, + 0.3541561961174011, + 1.3389673233032227, + 1.0469138622283936, + -0.22703340649604797, + -0.09654742479324341, + 1.0090891122817993, + 0.039891310036182404, + 0.791820228099823, + 1.5588668584823608, + 0.6951661109924316, + 0.7266017198562622, + -0.14599080383777618, + -0.07227890193462372, + -0.008913800120353699, + 0.1740609109401703, + 1.2036690711975098, + 0.5470603108406067 + ], + [ + -0.34583592414855957, + 0.03694339841604233, + 1.0050984621047974, + -0.4203651249408722, + 0.4847651720046997, + 1.7126538753509521, + 0.7576368451118469, + 0.04789559543132782, + -0.19675417244434357, + 0.7332325577735901, + -0.050296053290367126, + 0.5248405933380127, + 1.6081269979476929, + 1.103381872177124, + 0.6356422305107117, + -0.12914817035198212, + 0.1542941927909851, + 0.08850230276584625, + 0.04196876287460327, + 1.2002403736114502, + 0.8037097454071045 + ], + [ + -0.42469507455825806, + -0.04173311963677406, + 0.7264179587364197, + 0.06337344646453857, + -0.3319808840751648, + 1.5662562847137451, + 1.011260986328125, + 0.10568541288375854, + -0.10509131103754044, + 0.9041814208030701, + -0.06494040042161942, + 0.21946558356285095, + 1.254018783569336, + 1.038018822669983, + 0.5477806925773621, + -0.043720562011003494, + 0.12014467269182205, + 0.09301504492759705, + 0.11733841896057129, + 1.2624869346618652, + 0.3243844509124756 + ], + [ + -0.33015257120132446, + 0.029943212866783142, + 0.9176104068756104, + -0.43329212069511414, + 0.2955731749534607, + 1.3744759559631348, + 0.6465149521827698, + 0.1046447902917862, + -0.18162041902542114, + 0.6467805504798889, + 0.006262581795454025, + 0.562364935874939, + 1.4995815753936768, + 0.8783308267593384, + 0.7870968580245972, + -0.08096165955066681, + 0.16088604927062988, + -0.006748855113983154, + 0.23232635855674744, + 1.063927173614502, + 0.4299710988998413 + ], + [ + -0.4944083094596863, + -0.08790145069360733, + 1.29244863986969, + -0.3121066093444824, + 0.26694127917289734, + 1.671442985534668, + 0.7521165609359741, + -0.0812029242515564, + -0.29114994406700134, + 0.8142672181129456, + -0.13427919149398804, + 0.5399618148803711, + 1.3985484838485718, + 1.0996084213256836, + 0.47243985533714294, + -0.12033167481422424, + 0.17510870099067688, + 0.14865869283676147, + -0.12706416845321655, + 1.2753329277038574, + 0.5404187440872192 + ], + [ + -0.37128323316574097, + -0.014755124226212502, + 1.141469955444336, + -0.08029656112194061, + 0.2596544623374939, + 1.8179931640625, + 0.9341777563095093, + 0.044848546385765076, + -0.2405034601688385, + 0.8614826202392578, + 0.06876950711011887, + 0.22597625851631165, + 1.7494922876358032, + 1.202664852142334, + 0.2998929023742676, + -0.22034409642219543, + 0.2735995948314667, + 0.3597015142440796, + 0.49169713258743286, + 1.3961365222930908, + 0.8222110271453857 + ] + ], + "generated_utilities": [ + 2.0, + 0.021834811195731163, + 2.0, + 0.019551733508706093, + 0.024202564731240273, + 0.02782593108713627, + 0.005327919963747263, + 0.00631558895111084 + ], + "group_id": "ms-57be9ae181c077e7821f", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.28031715750694275, + 0.19804009795188904, + 0.15024888515472412, + 0.23104076087474823, + 0.21580509841442108, + 0.013450047001242638, + 0.20318475365638733, + 0.22562944889068604, + 0.14178292453289032, + 0.15763869881629944, + 0.27735981345176697, + 0.1909358948469162, + 0.06666654348373413, + 2.0, + 0.060366373509168625, + 0.05085722729563713 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.06739383935928345, + 0.06319573521614075, + 0.8779754638671875, + -0.03682136535644531, + -0.07093041390180588, + 1.3382306098937988, + 0.0, + 0.07168813049793243, + 0.01971740648150444, + 0.4866671562194824, + 0.07402461767196655, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + 0.03574584424495697, + 0.021435141563415527, + 0.0021790266036987305, + -0.06565424799919128, + 1.1393508911132812, + 0.0 + ], + [ + -0.17397063970565796, + -0.08753994852304459, + 0.8779754638671875, + -0.09028738737106323, + -0.17196941375732422, + 1.3382306098937988, + 0.0, + 0.022144019603729248, + 0.1154828891158104, + 0.4866671562194824, + -0.09212043881416321, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + -0.2021142691373825, + -0.022606968879699707, + 0.0021790266036987305, + 0.03272634744644165, + 1.1393508911132812, + 0.0 + ], + [ + -1.0144902467727661, + -0.023572498932480812, + 1.8779754638671875, + -1.0368213653564453, + -0.03799626976251602, + 2.338230609893799, + 2.239948272705078, + 0.16168542206287384, + 0.07221447676420212, + 1.4866671562194824, + -0.3557073175907135, + 1.3622210025787354, + 1.896626591682434, + 2.0382778644561768, + 1.4886415004730225, + 0.1997920423746109, + -0.4386618137359619, + 1.0021790266036987, + -0.3174663782119751, + 2.1393508911132812, + 1.9118322134017944 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.479896545410156, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0765557289123535, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.823664426803589 + ], + [ + -1.0723698139190674, + -0.05779706686735153, + 2.0670831203460693, + -0.830497682094574, + -0.17302533984184265, + 2.2277159690856934, + 2.0435242652893066, + 0.08191673457622528, + 0.08711377531290054, + 1.416495680809021, + -0.5633984804153442, + 1.3216067552566528, + 1.8660295009613037, + 2.2198705673217773, + 1.6887608766555786, + 0.06658947467803955, + -0.4010601043701172, + 1.1803714036941528, + -0.15144284069538116, + 2.308220386505127, + 1.8460347652435303 + ], + [ + -0.8901733756065369, + 0.3378032147884369, + 2.2360360622406006, + -1.207388162612915, + -0.34048303961753845, + 2.460798978805542, + 2.100795030593872, + -0.1847335249185562, + -0.06362440437078476, + 1.2987924814224243, + -0.11972835659980774, + 1.170393943786621, + 2.0620977878570557, + 2.4285032749176025, + 1.3666276931762695, + -0.01825574040412903, + -0.4620283842086792, + 1.1138218641281128, + -0.5787107944488525, + 1.78761887550354, + 1.740113615989685 + ], + [ + -1.1796774864196777, + -0.12369386106729507, + 1.546891689300537, + -1.0500564575195312, + -0.026454759761691093, + 2.177122116088867, + 2.0003931522369385, + 0.13697585463523865, + 0.37715619802474976, + 1.9734747409820557, + -0.16338464617729187, + 1.3775254487991333, + 2.3054146766662598, + 1.5546050071716309, + 1.2645983695983887, + 0.3812609910964966, + 0.02191823720932007, + 1.0944219827651978, + 0.09844094514846802, + 1.8145865201950073, + 2.0624771118164062 + ], + [ + -0.5514534711837769, + -0.4441075921058655, + 1.8971614837646484, + -1.6112515926361084, + 0.1306135058403015, + 2.736365556716919, + 2.270737886428833, + -0.35948747396469116, + 0.03968408331274986, + 1.061530351638794, + -0.29366230964660645, + 1.5066968202590942, + 2.34443998336792, + 2.3445637226104736, + 1.2286460399627686, + 0.13919776678085327, + -0.21014492213726044, + 0.9723342061042786, + -0.5822539329528809, + 1.8932968378067017, + 2.159864664077759 + ], + [ + -0.9470903873443604, + -0.7433359622955322, + 1.798203945159912, + -0.5850675106048584, + -0.7611241340637207, + 2.2761964797973633, + 2.0114798545837402, + 0.7724847793579102, + -0.23453289270401, + 0.7384847402572632, + -1.022956132888794, + 1.0059813261032104, + 1.652310848236084, + 2.6376113891601562, + 2.148331642150879, + 0.9844839572906494, + 0.18531352281570435, + 1.6280735731124878, + -0.06812705099582672, + 2.2684829235076904, + 1.1641287803649902 + ], + [ + -1.729402780532837, + 0.15382106602191925, + 1.0736894607543945, + -0.5141006708145142, + -0.6476962566375732, + 2.0120980739593506, + 2.8703343868255615, + 0.5578619837760925, + -0.8154823184013367, + 1.4325131177902222, + -1.0468122959136963, + 0.9670607447624207, + 1.973939061164856, + 2.7364890575408936, + 1.8303502798080444, + 0.8089333772659302, + -0.41138362884521484, + 1.8146085739135742, + -0.11676108837127686, + 2.3541221618652344, + 2.3875224590301514 + ], + [ + -1.8409323692321777, + -0.2118261456489563, + 1.5354409217834473, + -0.3873547911643982, + 0.7043789029121399, + 2.2068612575531006, + 2.5864486694335938, + 0.8632195591926575, + -0.10360705107450485, + 1.6340630054473877, + -0.017105937004089355, + 1.90720796585083, + 2.608898639678955, + 2.3387808799743652, + 2.3744726181030273, + 0.13692857325077057, + -0.7716406583786011, + 1.3477450609207153, + -0.9811362028121948, + 2.15514874458313, + 2.298004150390625 + ], + [ + -1.2484530210494995, + 0.5977066159248352, + 2.8256869316101074, + -0.3418053388595581, + -1.031638503074646, + 1.7023751735687256, + 1.4487309455871582, + -0.3013373017311096, + -0.3682725131511688, + 2.4404053688049316, + -0.8263542652130127, + 2.1105899810791016, + 2.5769248008728027, + 1.6751525402069092, + 0.8367876410484314, + -0.1554209142923355, + 0.17985862493515015, + 1.0017495155334473, + -0.3176954984664917, + 2.6737403869628906, + 1.7722500562667847 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -1.0015538930892944, + 0.43126925826072693, + 1.2786648273468018, + -0.7182899713516235, + 0.06109866499900818, + 2.323270797729492, + 1.3882524967193604, + -0.2783380150794983, + 0.35085979104042053, + 0.5870303511619568, + -1.0134913921356201, + 0.4675567150115967, + 0.9185858964920044, + 1.9441449642181396, + 1.3693488836288452, + 0.05759717524051666, + 0.19058758020401, + 1.9360744953155518, + -1.0018632411956787, + 2.043489456176758, + 1.1226990222930908 + ] + ], + "predicted_scores": [ + 6.654320240020752, + 6.55549430847168, + 6.52157735824585, + 6.20645809173584, + 5.943201065063477, + 5.8155646324157715, + 5.276596546173096, + 5.020887851715088 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train0", + "selected_index": 0, + "state_hash": "9c4de49d50d74c8c571620759fff1d43be23093a5f66e946f7ae6d75cb26ade5", + "stored_base_utility": 0.28031715750694275, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1544450968503952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1544450968503952 + }, + "base_progress": 0.1544450968503952, + "base_success": false, + "base_utility": 0.1544450968503952, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06245501711964607, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06245501711964607 + }, + { + "contact_quality": null, + "progress": 0.057290513068437576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057290513068437576 + }, + { + "contact_quality": null, + "progress": 0.05731761455535889, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05731761455535889 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06114928051829338, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06114928051829338 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.056892890483140945, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.056892890483140945 + }, + { + "contact_quality": null, + "progress": 0.052340131253004074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.052340131253004074 + } + ], + "candidate_progress": [ + 0.06245501711964607, + 0.057290513068437576, + 0.05731761455535889, + 1.0, + 0.06114928051829338, + 1.0, + 0.056892890483140945, + 0.052340131253004074 + ], + "candidate_source_chart_ids": [ + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-6388b5d2356c556c3e55", + "generated_tangents": [ + [ + 0.14085455238819122, + 0.06595045328140259, + 1.8042998313903809, + -0.10692273825407028, + 0.7195324301719666, + 2.251777410507202, + 2.3358867168426514, + 2.566293716430664, + -0.19326137006282806, + -0.9937949180603027, + 2.420006036758423, + -0.030497834086418152, + 1.7948921918869019, + 1.7487417459487915, + -0.09461890161037445, + -0.10772505402565002, + 1.1486867666244507, + -0.1017664223909378, + 1.311736822128296, + 3.0273165702819824, + 3.01809024810791 + ], + [ + 0.16229595243930817, + 0.08912916481494904, + 1.6193690299987793, + -0.1410122513771057, + 0.8347203135490417, + 2.2059223651885986, + 2.0994341373443604, + 2.6510233879089355, + -0.291916161775589, + -0.9467397928237915, + 2.579068183898926, + -0.11217635869979858, + 1.8465524911880493, + 1.8256187438964844, + -0.054164987057447433, + -0.18806178867816925, + 1.0668989419937134, + -0.00449015200138092, + 1.264209508895874, + 2.879173755645752, + 3.214197874069214 + ], + [ + 0.03580384701490402, + 0.22809405624866486, + 2.0692615509033203, + -0.4423205256462097, + 0.892505943775177, + 2.2922213077545166, + 2.545778274536133, + 2.5572688579559326, + -0.26238247752189636, + -0.6881310939788818, + 2.270177125930786, + 0.20732051134109497, + 2.214596748352051, + 2.104609489440918, + 0.03223583102226257, + 0.19487497210502625, + 0.9772140383720398, + 0.009927839040756226, + 1.5587480068206787, + 2.3638627529144287, + 3.3562300205230713 + ], + [ + 0.0926257073879242, + 0.04837973415851593, + 1.2499054670333862, + -0.3065544068813324, + 0.8986547589302063, + 2.4435501098632812, + 2.1432111263275146, + 2.4063634872436523, + -0.3533810079097748, + -0.6136487722396851, + 2.9670586585998535, + -0.06668078899383545, + 2.0088655948638916, + 2.0828006267547607, + -0.3177120089530945, + -0.014724710024893284, + 0.974626362323761, + -0.026060760021209717, + 1.354006052017212, + 2.240135669708252, + 2.896519660949707 + ], + [ + 0.07820472866296768, + 0.25062239170074463, + 1.8994182348251343, + -0.5110874176025391, + 0.8189709782600403, + 2.1727852821350098, + 2.6295180320739746, + 2.643580198287964, + -0.29673078656196594, + -0.6997730731964111, + 2.2748427391052246, + 0.0454077273607254, + 2.230524778366089, + 2.1575260162353516, + 0.14411985874176025, + 0.2165374904870987, + 0.8992494940757751, + -0.006085202097892761, + 1.5451223850250244, + 2.374079704284668, + 2.979363441467285 + ], + [ + 0.3065474331378937, + -0.1955948770046234, + 1.202223539352417, + 0.05112254619598389, + 0.8425878882408142, + 2.120248317718506, + 1.3745603561401367, + 2.951903820037842, + -0.5075027942657471, + -0.490806519985199, + 2.6812856197357178, + 0.1408054232597351, + 1.8529664278030396, + 2.296215295791626, + 0.020097725093364716, + -0.019534600898623466, + 1.205973744392395, + 0.013922587037086487, + 1.3845617771148682, + 2.048712968826294, + 3.2170848846435547 + ], + [ + 0.1498587429523468, + 0.24786995351314545, + 1.7572414875030518, + -0.5540152788162231, + 0.8134774565696716, + 1.9461852312088013, + 2.737715721130371, + 2.755290985107422, + -0.3167344927787781, + -0.7857706546783447, + 2.2916619777679443, + 0.014517530798912048, + 2.2775135040283203, + 2.3802807331085205, + 0.15740102529525757, + 0.17527072131633759, + 0.9125971794128418, + 0.030883222818374634, + 1.5133967399597168, + 2.335270404815674, + 2.915964365005493 + ], + [ + 0.5088602900505066, + 0.10750138759613037, + 1.8345903158187866, + -0.16440215706825256, + 0.8437427878379822, + 2.507436752319336, + 1.8617327213287354, + 2.5663468837738037, + -0.08433163166046143, + -0.8705015778541565, + 2.830353260040283, + 0.49730610847473145, + 1.8448563814163208, + 1.9258184432983398, + 0.015072338283061981, + -0.058465342968702316, + 1.3149197101593018, + 0.10777977854013443, + 1.3854131698608398, + 3.1543467044830322, + 3.2663450241088867 + ] + ], + "generated_utilities": [ + 0.06245501711964607, + 0.057290513068437576, + 0.05731761455535889, + 2.0, + 0.06114928051829338, + 2.0, + 0.056892890483140945, + 0.052340131253004074 + ], + "group_id": "ms-6388b5d2356c556c3e55", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08186312764883041, + 0.11667267978191376, + 0.09751813113689423, + 2.0, + 0.27431026101112366, + 0.11760994046926498, + 0.20386962592601776, + 0.19832774996757507, + 0.22735612094402313, + 0.1356993168592453, + 0.1713169515132904, + 0.1792232096195221, + 0.06848804652690887, + 0.1094626635313034, + 0.11555596441030502, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0018978863954544067, + -0.4448504149913788, + 0.7108399868011475, + -0.33157044649124146, + -0.062200844287872314, + 1.4758033752441406, + 0.0, + 1.4153923988342285, + 0.14116239547729492, + 0.00777280330657959, + 0.5541017055511475, + 0.07029426097869873, + 0.26837193965911865, + 0.0, + -0.015542566776275635, + 0.3573532998561859, + 0.6168417930603027, + 0.3612897992134094, + 0.369077205657959, + 1.5924487113952637, + 0.0 + ], + [ + -0.09549225866794586, + -0.20593886077404022, + 1.7108399868011475, + -0.1554051786661148, + 0.9377991557121277, + 2.4758033752441406, + 2.4156031608581543, + 2.4153923988342285, + -0.2459496259689331, + -0.9922271966934204, + 1.5541017055511475, + 0.4461512267589569, + 1.2501436471939087, + 1.7566912174224854, + -0.8014360666275024, + 0.04551727697253227, + 1.6168417930603027, + 0.5393297672271729, + 1.369077205657959, + 2.5924487113952637, + 3.1126015186309814 + ], + [ + -0.11610143631696701, + -0.3961976170539856, + 1.9081053733825684, + -0.14575888216495514, + 1.1381605863571167, + 2.7434439659118652, + 2.5136890411376953, + 2.6401467323303223, + -0.45453745126724243, + -0.9589858651161194, + 1.4083189964294434, + 0.5800129175186157, + 0.9940754175186157, + 1.7450767755508423, + -1.0145976543426514, + 0.3312625586986542, + 1.4729199409484863, + 0.2520747482776642, + 1.3623151779174805, + 2.5670394897460938, + 3.3589630126953125 + ], + [ + -0.2318952977657318, + 0.024853751063346863, + 1.829875111579895, + 0.09877045452594757, + 1.1515556573867798, + 2.2239503860473633, + 2.153250217437744, + 2.8057901859283447, + -0.1800437569618225, + -1.2225371599197388, + 1.421208143234253, + 0.6627710461616516, + 1.3895055055618286, + 1.856442928314209, + -0.5928603410720825, + 0.4364123046398163, + 1.7852745056152344, + 0.3345714807510376, + 1.1167365312576294, + 2.7676005363464355, + 3.041402578353882 + ], + [ + -0.2227625697851181, + -0.012646213173866272, + 1.7988159656524658, + 0.21457992494106293, + 0.8877373933792114, + 2.9257700443267822, + 2.2101094722747803, + 2.2874886989593506, + -0.09066365659236908, + -0.49909964203834534, + 2.042389392852783, + 0.6287553906440735, + 1.074569582939148, + 1.7805378437042236, + -1.1840007305145264, + 0.3728531002998352, + 1.165173053741455, + 0.3054528832435608, + 1.4924153089523315, + 2.5540966987609863, + 2.664964199066162 + ], + [ + -0.6864494681358337, + 0.119636669754982, + 1.2460956573486328, + -0.6367122530937195, + 1.4080986976623535, + 2.5262017250061035, + 2.508194923400879, + 2.1323020458221436, + -0.43302130699157715, + -1.2678899765014648, + 1.0870052576065063, + 0.46296897530555725, + 0.765267550945282, + 2.3093044757843018, + -0.46595194935798645, + -0.43870145082473755, + 2.028986930847168, + 0.26284897327423096, + 1.5542359352111816, + 2.530392646789551, + 2.7435970306396484 + ], + [ + 0.18175412714481354, + -0.478390634059906, + 1.286731481552124, + -0.8066962361335754, + 1.4787592887878418, + 2.1033878326416016, + 1.750453233718872, + 2.047861099243164, + 0.3163147568702698, + -0.30588728189468384, + 1.0766479969024658, + 0.5822455286979675, + 1.673675775527954, + 2.039386510848999, + -0.62891685962677, + 0.615638792514801, + 2.018416404724121, + 0.2530737817287445, + 1.34544837474823, + 3.2287912368774414, + 2.9459850788116455 + ], + [ + 0.5059412121772766, + 0.17952178418636322, + 1.3319568634033203, + -0.8850483894348145, + 0.6336387395858765, + 1.775376796722412, + 2.3280270099639893, + 1.7305657863616943, + -0.02617873251438141, + -0.7963138818740845, + 1.9142115116119385, + 0.6635866165161133, + 1.3493314981460571, + 1.9258644580841064, + -0.3795176148414612, + 0.7861210107803345, + 1.0032029151916504, + 0.8491684794425964, + 0.5918128490447998, + 2.399027109146118, + 2.3932981491088867 + ], + [ + -0.34448128938674927, + -0.2996852695941925, + 0.9233196377754211, + -0.22890916466712952, + 1.6932814121246338, + 3.465762138366699, + 2.2005038261413574, + 2.6472361087799072, + -0.8981531858444214, + -0.2658765912055969, + 1.8755900859832764, + 0.2962568402290344, + 1.8239326477050781, + 1.0176585912704468, + -0.7912051677703857, + 0.6727061867713928, + 2.199978828430176, + 1.015731692314148, + 0.43087130784988403, + 3.5876548290252686, + 3.893789052963257 + ] + ], + "predicted_scores": [ + 3.226083755493164, + 2.7768771648406982, + 2.7206974029541016, + 2.6579699516296387, + 1.9610471725463867, + 1.883847713470459, + 1.7879574298858643, + 1.453969955444336 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train0", + "selected_index": 0, + "state_hash": "38d539e46140304e3f157cbc207aa0d9489e5470633c216a0379906af445a1ec", + "stored_base_utility": 0.08186312764883041, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06173701956868172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06173701956868172 + }, + "base_progress": 0.06173701956868172, + "base_success": false, + "base_utility": 0.06173701956868172, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10160752385854721, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10160752385854721 + }, + { + "contact_quality": null, + "progress": 0.01461742352694273, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01461742352694273 + }, + { + "contact_quality": null, + "progress": 0.046429235488176346, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.046429235488176346 + }, + { + "contact_quality": null, + "progress": 0.029767373576760292, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029767373576760292 + }, + { + "contact_quality": null, + "progress": 0.10250582545995712, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10250582545995712 + }, + { + "contact_quality": null, + "progress": 0.15528590977191925, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15528590977191925 + }, + { + "contact_quality": null, + "progress": 0.03336955979466438, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03336955979466438 + }, + { + "contact_quality": null, + "progress": 0.06291665881872177, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06291665881872177 + } + ], + "candidate_progress": [ + 0.10160752385854721, + 0.01461742352694273, + 0.046429235488176346, + 0.029767373576760292, + 0.10250582545995712, + 0.15528590977191925, + 0.03336955979466438, + 0.06291665881872177 + ], + "candidate_source_chart_ids": [ + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-d535b38d768633f6d922", + "ms-a2518826fa0a816cc46f", + "ms-3a4e2d080c9c4b279895", + "ms-7e88428497a6b13936c4", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-638d2ba1917eff4d09e0", + "generated_tangents": [ + [ + 0.20012439787387848, + -0.3785173296928406, + 0.28743481636047363, + 0.057181090116500854, + 1.5917069911956787, + -0.11141654849052429, + 0.5379866361618042, + 0.11633799970149994, + -0.04841218888759613, + 0.24334482848644257, + 0.07627934217453003, + 0.6672567129135132, + 0.03587189316749573, + 0.08851487934589386, + 0.7830518484115601, + 0.1023007184267044, + -0.3814665675163269, + -0.2820456027984619, + 0.9330448508262634, + -1.4249541759490967, + -0.9199477434158325 + ], + [ + 0.03092768043279648, + -0.4322233200073242, + 0.2368573248386383, + 0.17836228013038635, + 1.8263330459594727, + 0.304324746131897, + 0.5671171545982361, + -0.05702170729637146, + -0.057950958609580994, + 0.2906959056854248, + 0.27702105045318604, + 0.8885995149612427, + 0.09528079628944397, + -0.31255626678466797, + 0.4552042782306671, + -0.030130892992019653, + -0.19965483248233795, + -0.4369654059410095, + 0.798297643661499, + -1.5233819484710693, + -0.9924530982971191 + ], + [ + 0.13268131017684937, + -0.38998734951019287, + 0.4595482051372528, + 0.13375499844551086, + 1.5339815616607666, + 0.18641261756420135, + 0.593211829662323, + 0.1785918027162552, + -0.05231153219938278, + 0.09126681834459305, + 0.027456795796751976, + 0.6521029472351074, + -0.07697564363479614, + -0.16427795588970184, + 0.6907652020454407, + 0.1520497351884842, + -0.4432736039161682, + -0.23638613522052765, + 0.711456298828125, + -1.4344983100891113, + -0.8440983295440674 + ], + [ + 0.01169373095035553, + -0.46332138776779175, + 0.2159072756767273, + -0.0831880271434784, + 1.592785120010376, + 0.417820006608963, + 1.3587965965270996, + 0.2749141454696655, + -0.261660635471344, + 0.23481294512748718, + -0.15995407104492188, + 0.545703649520874, + 0.33227142691612244, + 0.0875142440199852, + 0.6457905769348145, + 0.003443394787609577, + -0.46850788593292236, + -0.08103956282138824, + 0.658976137638092, + -0.8811950087547302, + -0.45714953541755676 + ], + [ + 0.12743258476257324, + -0.4842522144317627, + 0.1821211576461792, + -0.11842761188745499, + 1.7030308246612549, + 0.09627389162778854, + 0.49623584747314453, + 0.20606066286563873, + -0.14327944815158844, + 0.20020434260368347, + 0.1686018705368042, + 0.9053546190261841, + 0.15055426955223083, + -0.40241777896881104, + 0.6978731155395508, + 0.048628415912389755, + -0.3393581509590149, + -0.20582269132137299, + 0.8972986936569214, + -1.3415148258209229, + -0.7074575424194336 + ], + [ + 0.3362257480621338, + -0.2868293821811676, + 0.11786612868309021, + 0.3090389668941498, + 1.3401665687561035, + 0.017566069960594177, + 0.4646468758583069, + 0.39307647943496704, + 0.0022079935297369957, + 0.13397003710269928, + 0.20744341611862183, + 0.8147593140602112, + 0.8614964485168457, + 0.2619123160839081, + 0.529552161693573, + 0.09387147426605225, + -0.2555035948753357, + -0.018945425748825073, + 0.5246317386627197, + -0.565754771232605, + -0.4137805998325348 + ], + [ + -0.029415994882583618, + -0.34208476543426514, + 0.0194321870803833, + -0.3278515338897705, + 1.7799267768859863, + 0.023271575570106506, + 0.6620983481407166, + 0.10545758903026581, + -0.175527885556221, + 0.08659183979034424, + 0.2616179585456848, + 0.9107527732849121, + 0.19200876355171204, + -0.20335085690021515, + 0.7550808191299438, + 0.04693594202399254, + -0.3627760410308838, + -0.36395782232284546, + 0.8388381004333496, + -1.1179988384246826, + -0.8995155096054077 + ], + [ + 0.20678108930587769, + -0.5316930413246155, + 0.5591402649879456, + -0.1520969569683075, + 1.5175111293792725, + 0.5175721645355225, + 0.4183806777000427, + 0.3324818015098572, + -0.12351128458976746, + 0.34945082664489746, + 0.09516577422618866, + 1.0129802227020264, + 0.2763718068599701, + 0.2244502753019333, + 0.7656605839729309, + 0.03992048278450966, + -0.4802477955818176, + -0.2962467670440674, + 0.9774383306503296, + -1.3668091297149658, + -0.4972158968448639 + ] + ], + "generated_utilities": [ + 0.10160752385854721, + 0.01461742352694273, + 0.046429235488176346, + 0.029767373576760292, + 0.10250582545995712, + 0.15528590977191925, + 0.03336955979466438, + 0.06291665881872177 + ], + "group_id": "ms-638d2ba1917eff4d09e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.028925320133566856, + 0.11176212877035141, + 0.057868462055921555, + 0.09960909932851791, + 0.07349967956542969, + 0.029011588543653488, + 0.06443297863006592, + 0.05166193097829819, + 0.025834282860159874, + 0.056667208671569824, + 0.04477008432149887, + 0.019685089588165283, + 0.018628716468811035, + 0.08587563037872314, + 0.02232462167739868, + 0.010265171527862549 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04120570421218872, + 0.038339823484420776, + 0.5979268550872803, + -0.05306033045053482, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.3407015800476074, + -0.21580028533935547, + -0.036036908626556396, + -0.004389047622680664, + 0.5364620685577393, + 0.026709645986557007, + 0.0, + 1.0126078128814697, + -0.10034149885177612, + -0.16565871238708496, + 0.07125610113143921, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + -0.5276790261268616, + -0.19392354786396027, + 0.5979268550872803, + 0.05016906559467316, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.6852816939353943, + -0.21580028533935547, + 0.04531368613243103, + -0.09276384115219116, + 0.5364620685577393, + 0.021915271878242493, + 0.0, + 1.0126078128814697, + 0.29255303740501404, + -0.16565871238708496, + 0.14600259065628052, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + 0.5910290479660034, + -0.038985610008239746, + 1.5555552244186401, + -1.028314232826233, + 2.3103959560394287, + 0.8163888454437256, + 2.064336061477661, + 1.168483018875122, + -0.21718227863311768, + -0.19770365953445435, + 0.11310696601867676, + 0.6850457191467285, + -0.06804794818162918, + 0.33076921105384827, + 1.7529149055480957, + -0.562712550163269, + -1.6352436542510986, + 0.1224215030670166, + 3.354771852493286, + -1.1317338943481445, + -1.7348618507385254 + ] + ], + "predicted_scores": [ + 9.201016426086426, + 9.197566986083984, + 9.097320556640625, + 8.973604202270508, + 8.596654891967773, + 8.47115421295166, + 8.250267028808594, + 7.663734436035156 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train0", + "selected_index": 0, + "state_hash": "5d6f4227fab3b4bb36756c9a15c32b6fe6c9a0ae4b07b3f20a2bd65c2594df13", + "stored_base_utility": 0.028925320133566856, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.016599038615822792, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016599038615822792 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.005813241004943848, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005813241004943848 + }, + { + "contact_quality": null, + "progress": 0.011173486709594727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011173486709594727 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.016599038615822792, + 1.0, + 0.005813241004943848, + 0.011173486709594727, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-a6a6de64c39abad45f86", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-652ad552ba98d2736620", + "generated_tangents": [ + [ + 0.7725136876106262, + -0.03431672602891922, + 0.8048340678215027, + -0.6374152898788452, + 1.5430598258972168, + 1.4735641479492188, + 0.8320761919021606, + 0.06031659245491028, + -0.11114482581615448, + 0.6126077771186829, + 0.6522234678268433, + 1.0537309646606445, + 0.13850536942481995, + -0.1393839567899704, + -0.18330305814743042, + -0.2873992323875427, + 1.2469110488891602, + -0.3705272078514099, + 1.253023386001587, + 2.37744402885437, + 1.3933371305465698 + ], + [ + 0.3059130311012268, + 0.1567693054676056, + 1.0042239427566528, + -0.9915871620178223, + 2.010279417037964, + 1.189953327178955, + 1.4461641311645508, + 0.05451717972755432, + -0.032694846391677856, + 0.6890350580215454, + 0.657055675983429, + 1.0749082565307617, + 0.8170125484466553, + 0.39462989568710327, + -0.14280761778354645, + -0.170671746134758, + 1.371072769165039, + -0.31456679105758667, + 1.4619224071502686, + 2.193998098373413, + 1.4680534601211548 + ], + [ + 0.21431779861450195, + 0.112717404961586, + 1.1256744861602783, + -1.0663964748382568, + 1.601625680923462, + 1.118420958518982, + 0.9243471622467041, + -0.02183324098587036, + -0.036620594561100006, + 0.9164050221443176, + 0.8578189015388489, + 1.2168374061584473, + 1.1195783615112305, + 0.2532307505607605, + 0.044171541929244995, + -0.03379511088132858, + 1.4963239431381226, + -0.1239987462759018, + 1.2028415203094482, + 1.8915939331054688, + 2.001439332962036 + ], + [ + 0.3998737335205078, + 0.1277637630701065, + 1.0006946325302124, + -0.688167929649353, + 1.4692552089691162, + 0.8067119121551514, + 1.0645239353179932, + -0.04656617343425751, + -0.11209604144096375, + 0.7425906658172607, + 0.7275300025939941, + 1.2272968292236328, + 0.40083661675453186, + 0.3038826584815979, + -0.1416773945093155, + -0.09133654832839966, + 1.551317811012268, + -0.325903058052063, + 1.350942850112915, + 2.2170968055725098, + 1.2916264533996582 + ], + [ + 0.3216557204723358, + 0.20781610906124115, + 0.8242918252944946, + -1.1267178058624268, + 2.052201509475708, + 1.4753129482269287, + 1.241973638534546, + 0.19720833003520966, + -0.13343657553195953, + 0.603374183177948, + 0.6644425392150879, + 1.220246434211731, + 0.5619497299194336, + 0.6616710424423218, + -0.11985982954502106, + -0.12091799080371857, + 1.4248565435409546, + -0.46000170707702637, + 1.15903639793396, + 1.783252477645874, + 1.544547438621521 + ], + [ + 0.2983326315879822, + 0.18560101091861725, + 0.9183748960494995, + -0.9748181104660034, + 1.922440767288208, + 1.3594715595245361, + 1.3505611419677734, + 0.055494919419288635, + -0.06414700299501419, + 0.6860648989677429, + 0.6104075312614441, + 1.1296989917755127, + 0.5256576538085938, + 0.320169061422348, + -0.09568566083908081, + -0.15992745757102966, + 1.4739841222763062, + -0.22081340849399567, + 1.463674545288086, + 2.2050154209136963, + 1.76393723487854 + ], + [ + 0.3417985439300537, + 0.002032928168773651, + 0.7763372659683228, + -1.1412901878356934, + 2.0572874546051025, + 2.0098888874053955, + 1.3594467639923096, + 0.29879310727119446, + -0.24184048175811768, + 0.858045756816864, + 0.8992239832878113, + 0.896336555480957, + 0.3268645107746124, + 0.7245426774024963, + -0.16418415307998657, + -0.1624162793159485, + 1.325331211090088, + -0.4718431830406189, + 1.2613296508789062, + 2.447932243347168, + 2.0713791847229004 + ], + [ + 0.21297046542167664, + 0.046242982149124146, + 1.0241634845733643, + -0.5505623817443848, + 1.6444051265716553, + 1.4813787937164307, + 0.8535062074661255, + 0.1877230852842331, + -0.09278794378042221, + 0.409674733877182, + 0.397246390581131, + 0.8508703112602234, + 0.20856890082359314, + 0.3168705701828003, + -0.07951639592647552, + -0.22509704530239105, + 1.1620877981185913, + -0.0015839487314224243, + 0.9055641889572144, + 2.1325931549072266, + 1.0961734056472778 + ] + ], + "generated_utilities": [ + 0.016599038615822792, + 2.0, + 0.005813241004943848, + 0.011173486709594727, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-652ad552ba98d2736620", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08378833532333374, + 0.10458396375179291, + 0.19086095690727234, + 0.04039820283651352, + 0.15980230271816254, + 0.0428888201713562, + 0.2367449700832367, + 0.18185830116271973, + 0.17593276500701904, + 0.16103637218475342, + 0.15250858664512634, + 0.05226989835500717, + 0.1701633185148239, + 0.11590366065502167, + 0.05217729136347771, + 0.03575938940048218 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.09009888768196106, + 0.08798432350158691, + 0.718157172203064, + -0.6351244449615479, + 1.405301570892334, + 0.6596384048461914, + 0.0, + 0.060225605964660645, + -0.0894029438495636, + -0.10323715209960938, + 0.052752673625946045, + 0.5354657173156738, + -0.013395614922046661, + 0.0, + -0.3450838029384613, + -0.11172826588153839, + 0.7912200689315796, + 0.33702391386032104, + 0.5586241483688354, + 1.8093180656433105, + 0.0 + ], + [ + 0.557296633720398, + 1.087984323501587, + 1.718157172203064, + -1.6351244449615479, + 2.405301570892334, + 1.6596384048461914, + 1.7430553436279297, + 0.3360595703125, + -0.2521040737628937, + 0.8967628479003906, + 0.26587578654289246, + 1.5354657173156738, + -0.12294083833694458, + -0.022678710520267487, + -0.416314959526062, + -0.26016199588775635, + 1.7912200689315796, + -0.16825227439403534, + 1.5586241483688354, + 2.8093180656433105, + 2.2589924335479736 + ], + [ + 0.2635665237903595, + 1.0005124807357788, + 1.973160743713379, + -1.7548720836639404, + 2.176497459411621, + 1.5416926145553589, + 1.9556171894073486, + 0.2215275913476944, + -0.5432531833648682, + 0.8434366583824158, + 0.3559061288833618, + 1.6657240390777588, + -0.3025258779525757, + 0.04401832073926926, + -0.17062894999980927, + -0.1509227454662323, + 1.7901219129562378, + 0.0010261386632919312, + 1.726505994796753, + 2.9778714179992676, + 2.0530292987823486 + ], + [ + 0.23987123370170593, + 0.8904718160629272, + 1.467047929763794, + -1.990342378616333, + 2.7685346603393555, + 2.0486786365509033, + 2.013002395629883, + 0.006436288356781006, + -0.25436922907829285, + 1.1101897954940796, + 0.4448293447494507, + 1.3105798959732056, + 0.07367905974388123, + 0.23779401183128357, + -0.34580594301223755, + -0.35948312282562256, + 1.4435756206512451, + -0.35472720861434937, + 1.801603078842163, + 3.0683224201202393, + 2.2330009937286377 + ], + [ + 0.6738064289093018, + 1.3973212242126465, + 2.0960373878479004, + -1.735988736152649, + 2.5216798782348633, + 1.1975955963134766, + 2.234865665435791, + 0.4602587819099426, + -0.7169049382209778, + 0.9412583708763123, + 0.7081141471862793, + 1.1730585098266602, + -0.016052812337875366, + -0.10474976152181625, + -0.03458032011985779, + 0.044317394495010376, + 1.6339548826217651, + -0.5292693376541138, + 1.5412750244140625, + 3.018833875656128, + 2.6051149368286133 + ], + [ + 0.6305910348892212, + 1.540043592453003, + 1.626009464263916, + -1.8823078870773315, + 2.2431485652923584, + 2.1866872310638428, + 2.3003249168395996, + 0.25838330388069153, + -0.2936764061450958, + 1.2413102388381958, + -0.12014040350914001, + 1.743309497833252, + 0.3618801534175873, + -0.37178292870521545, + 0.11553955078125, + 0.04856392741203308, + 1.2029554843902588, + 0.04960852861404419, + 2.0190422534942627, + 2.5883281230926514, + 2.7636377811431885 + ], + [ + 0.5004369616508484, + 1.4795310497283936, + 2.325191020965576, + -1.7464677095413208, + 1.9271103143692017, + 1.2483245134353638, + 1.9852193593978882, + 0.24291110038757324, + 0.4463852345943451, + 0.5267151594161987, + 0.32179588079452515, + 1.2356117963790894, + 0.03201058506965637, + 0.2881847023963928, + -1.0345205068588257, + -0.4171831011772156, + 1.5080864429473877, + 0.11479203402996063, + 1.9481834173202515, + 2.840816020965576, + 2.427428960800171 + ], + [ + -0.3133848309516907, + 0.43174320459365845, + 2.305823802947998, + -1.657350778579712, + 2.010749578475952, + 1.9222235679626465, + 1.3648147583007812, + 0.1938892900943756, + -0.6084526777267456, + 0.7855725288391113, + -0.07715931534767151, + 0.9140809178352356, + -0.9242425560951233, + -0.4681842029094696, + -0.07827043533325195, + -0.23733587563037872, + 0.9328996539115906, + -0.6088451147079468, + 0.8040639162063599, + 2.620466947555542, + 2.5148611068725586 + ] + ], + "predicted_scores": [ + 6.483285427093506, + 6.3353800773620605, + 6.225844383239746, + 6.091470718383789, + 5.966473579406738, + 5.65541410446167, + 5.353701114654541, + 5.223989009857178 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train0", + "selected_index": 0, + "state_hash": "8c1ab306d1e82f81085196c60b3c846d2e2eb20ebc538597af359a6252623e7e", + "stored_base_utility": 0.08378833532333374, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03784912824630737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03784912824630737 + }, + "base_progress": 0.03784912824630737, + "base_success": false, + "base_utility": 0.03784912824630737, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10381317138671875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10381317138671875 + }, + { + "contact_quality": null, + "progress": 0.034510593861341476, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.034510593861341476 + }, + { + "contact_quality": null, + "progress": 0.03771030902862549, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03771030902862549 + }, + { + "contact_quality": null, + "progress": 0.04082806780934334, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04082806780934334 + }, + { + "contact_quality": null, + "progress": 0.051552534103393555, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.051552534103393555 + }, + { + "contact_quality": null, + "progress": 0.04706305265426636, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04706305265426636 + }, + { + "contact_quality": null, + "progress": 0.033499401062726974, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.033499401062726974 + }, + { + "contact_quality": null, + "progress": 0.036169927567243576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.036169927567243576 + } + ], + "candidate_progress": [ + 0.10381317138671875, + 0.034510593861341476, + 0.03771030902862549, + 0.04082806780934334, + 0.051552534103393555, + 0.04706305265426636, + 0.033499401062726974, + 0.036169927567243576 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-48178f52a96de3d1ead0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-674fad0e7c042d23629e", + "generated_tangents": [ + [ + 1.2150087356567383, + 0.09120199084281921, + -1.1203824281692505, + -0.20854628086090088, + -0.32002097368240356, + -1.7052526473999023, + -0.3327398896217346, + 0.4944519102573395, + 0.21956883370876312, + -0.5045254826545715, + -0.34264978766441345, + -0.2553563714027405, + -0.9397702217102051, + -0.7462403178215027, + -0.001640181988477707, + 0.1612023413181305, + -0.07497359812259674, + -1.3687654733657837, + -0.3035293221473694, + -0.4836989641189575, + -0.18741783499717712 + ], + [ + 0.9599210619926453, + 0.05319547653198242, + -0.886374831199646, + 0.03741931915283203, + -0.28284740447998047, + -2.136850357055664, + -0.46446549892425537, + 0.5796945095062256, + 0.1916380375623703, + -0.5130059123039246, + -0.7689229846000671, + -0.499808132648468, + -1.1670085191726685, + -0.8926345109939575, + 0.008976638317108154, + 0.13501359522342682, + 0.09401464462280273, + -1.3207203149795532, + -0.32666802406311035, + -0.2381674349308014, + 0.0005655288696289062 + ], + [ + 1.250658631324768, + 0.06865726411342621, + -0.7989288568496704, + -0.05752471089363098, + -0.21533417701721191, + -2.410933017730713, + -0.4800800085067749, + 0.6189042329788208, + 0.19408512115478516, + -0.551322877407074, + -0.679336667060852, + -0.30612629652023315, + -0.9502570629119873, + -1.1006320714950562, + 0.22709600627422333, + 0.11564396321773529, + 0.013436108827590942, + -1.2888011932373047, + -0.2702549695968628, + -0.49332308769226074, + -0.01012738049030304 + ], + [ + 1.2414168119430542, + -0.02945125102996826, + -0.6219155788421631, + 0.16242310404777527, + -0.009782344102859497, + -1.6693488359451294, + -0.7143471837043762, + 0.3483055531978607, + 0.19092519581317902, + -0.39566612243652344, + -0.4636634290218353, + -0.19324156641960144, + -1.0706819295883179, + -0.8795170783996582, + 0.15468624234199524, + -0.02661825716495514, + 0.33147311210632324, + -1.3698220252990723, + -0.3245302438735962, + -0.5012667179107666, + -0.028592616319656372 + ], + [ + 1.181836724281311, + -0.022661203518509865, + -0.7083592414855957, + 0.07817289233207703, + -0.1884171962738037, + -1.9427238702774048, + -0.8347828984260559, + 0.35041749477386475, + 0.1952485591173172, + -0.45024192333221436, + -0.49669674038887024, + -0.16414275765419006, + -0.7385827302932739, + -0.7952350378036499, + 0.11043927073478699, + -0.08943100273609161, + 0.10465411096811295, + -1.1381027698516846, + -0.5002793669700623, + -0.4082665741443634, + 0.019734010100364685 + ], + [ + 1.087027907371521, + 0.09609384834766388, + -0.7468534708023071, + 0.030354589223861694, + -0.18484348058700562, + -2.2053050994873047, + -0.5580365061759949, + 0.5209965705871582, + 0.21578319370746613, + -0.513771116733551, + -0.7245553731918335, + -0.3617209196090698, + -1.0672869682312012, + -0.8714480400085449, + 0.07010158151388168, + 0.12505894899368286, + 0.07052204012870789, + -1.2415176630020142, + -0.3515787124633789, + -0.26459744572639465, + -0.14684909582138062 + ], + [ + 1.285050392150879, + -0.05704961344599724, + -0.6177409887313843, + 0.1098192036151886, + 0.018284335732460022, + -1.8350772857666016, + -0.923826277256012, + 0.40002524852752686, + 0.12818598747253418, + -0.36871203780174255, + -0.510184109210968, + -0.18622660636901855, + -0.7467116117477417, + -0.9038758277893066, + 0.07761846482753754, + 0.01538304053246975, + 0.2775520086288452, + -1.2692476511001587, + -0.38015347719192505, + -0.4504619836807251, + 0.0455203652381897 + ], + [ + 1.344892144203186, + 0.05414992570877075, + -0.5603504180908203, + -0.09251777082681656, + -0.17817610502243042, + -2.0477712154388428, + -0.0023575127124786377, + 0.6702426075935364, + 0.2437015026807785, + -0.49742692708969116, + -0.8353731036186218, + 0.2615560293197632, + -1.0196740627288818, + -1.1268343925476074, + 0.15198864042758942, + 0.14432001113891602, + 0.1254526972770691, + -1.67019522190094, + -0.4600372314453125, + -0.6491325497627258, + -0.47356680035591125 + ] + ], + "generated_utilities": [ + 0.10381317138671875, + 0.034510593861341476, + 0.03771030902862549, + 0.04082806780934334, + 0.051552534103393555, + 0.04706305265426636, + 0.033499401062726974, + 0.036169927567243576 + ], + "group_id": "ms-674fad0e7c042d23629e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07829316705465317, + 0.080319844186306, + 0.11335399001836777, + 0.15264075994491577, + 0.14950451254844666, + 0.07767044007778168, + 0.12988285720348358, + 0.10878288745880127, + 0.15573063492774963, + 0.10951457917690277, + 0.06582474708557129, + 0.02477661892771721, + 0.07745532691478729, + 0.03348777815699577, + 0.10050909221172333, + 0.033770203590393066 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.5533217191696167, + 0.5758762359619141, + -0.6480545997619629, + -0.10892429947853088, + 0.7943102717399597, + -2.7399702072143555, + -1.8395006656646729, + 1.447904109954834, + 1.4495785236358643, + -0.055675506591796875, + -1.3537942171096802, + 0.38944748044013977, + -1.6196211576461792, + -1.2710431814193726, + -0.9744676947593689, + 0.057668983936309814, + 1.8350472450256348, + -1.2618876695632935, + -1.2385435104370117, + -1.7976703643798828, + -1.9491409063339233 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9072847366333008, + 0.06923278421163559, + -0.14729690551757812, + 0.15558937191963196, + 0.40412187576293945, + -2.342223644256592, + 0.0, + 0.2589508295059204, + 0.35182610154151917, + 0.10176032781600952, + -0.3489869236946106, + 0.13150231540203094, + -1.3123579025268555, + 0.0, + 0.10376164317131042, + 0.0909266471862793, + 0.1919306516647339, + 0.28928786516189575, + -0.4625590741634369, + -0.30475670099258423, + 0.0 + ], + [ + 1.9072847366333008, + 0.05373973771929741, + -1.1472969055175781, + -0.47107377648353577, + 0.3914180397987366, + -3.342223644256592, + -2.4980990886688232, + 1.2589508295059204, + 0.8455543518066406, + -0.7174267172813416, + -1.0694091320037842, + 0.33346641063690186, + -2.3123579025268555, + -1.1108808517456055, + -0.33852219581604004, + -0.7232105731964111, + 1.145590901374817, + -0.7107121348381042, + -0.5049349069595337, + -1.112295150756836, + -1.244770884513855 + ], + [ + 1.808228611946106, + -0.18174676597118378, + -1.2057973146438599, + -0.6115378737449646, + 0.11874309182167053, + -3.3499772548675537, + -2.3861043453216553, + 1.2540870904922485, + 1.1210845708847046, + -0.953934371471405, + -1.3015953302383423, + 0.6089286804199219, + -2.1655566692352295, + -0.8375923037528992, + -0.26793593168258667, + -0.998324990272522, + 1.2481704950332642, + -0.8320856690406799, + -0.376935750246048, + -1.0882012844085693, + -0.9857641458511353 + ], + [ + 2.291844606399536, + 0.05574151128530502, + -0.9760321974754333, + -0.5710926055908203, + 0.8478215336799622, + -3.22356915473938, + -2.591245651245117, + 1.2184754610061646, + 0.855300784111023, + -1.0267595052719116, + -1.3318730592727661, + 0.4197496771812439, + -2.0368778705596924, + -1.3685163259506226, + -0.582176685333252, + -0.8082455992698669, + 1.3435251712799072, + -0.44022125005722046, + -0.06775334477424622, + -0.7692400217056274, + -0.8863754272460938 + ] + ], + "predicted_scores": [ + 7.748844146728516, + 6.1985087394714355, + 6.143899440765381, + 6.135042667388916, + 5.7854323387146, + 5.741323471069336, + 5.381580829620361, + 5.288109302520752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train0", + "selected_index": 0, + "state_hash": "6802b2d1fb8a5afa54b7321e2bb938176de48d74f28e9b2a9c741c23547f0ef1", + "stored_base_utility": 0.07829316705465317, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.12791956961154938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12791956961154938 + }, + "base_progress": 0.12791956961154938, + "base_success": false, + "base_utility": 0.12791956961154938, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.13394887745380402, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13394887745380402 + }, + { + "contact_quality": null, + "progress": 0.06245779991149902, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06245779991149902 + }, + { + "contact_quality": null, + "progress": 0.06475379317998886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06475379317998886 + }, + { + "contact_quality": null, + "progress": 0.07846011966466904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07846011966466904 + }, + { + "contact_quality": null, + "progress": 0.06508652120828629, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06508652120828629 + }, + { + "contact_quality": null, + "progress": 0.051506221294403076, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.051506221294403076 + }, + { + "contact_quality": null, + "progress": 0.07821440696716309, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07821440696716309 + }, + { + "contact_quality": null, + "progress": 0.08463313430547714, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08463313430547714 + } + ], + "candidate_progress": [ + 0.13394887745380402, + 0.06245779991149902, + 0.06475379317998886, + 0.07846011966466904, + 0.06508652120828629, + 0.051506221294403076, + 0.07821440696716309, + 0.08463313430547714 + ], + "candidate_source_chart_ids": [ + "ms-1ba37e30b90a3b28e62f", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-728b865677134d3788e3", + "generated_tangents": [ + [ + -0.5147793292999268, + 0.01635870337486267, + 0.5936979055404663, + 0.22692140936851501, + 0.806601345539093, + -0.31535428762435913, + 0.9833135008811951, + 0.49836429953575134, + 0.06978798657655716, + -0.24665281176567078, + -0.4312334358692169, + 0.40476545691490173, + -0.7772910594940186, + -0.9794929027557373, + -1.0157660245895386, + 0.5901522040367126, + 0.9773255586624146, + -0.36914050579071045, + -0.2945755124092102, + -1.3986175060272217, + -0.3224315345287323 + ], + [ + -0.4392404556274414, + -0.28786003589630127, + 0.3327646851539612, + 0.6239948272705078, + 1.2620210647583008, + -0.15100327134132385, + -0.5116145014762878, + 0.5213421583175659, + -0.06871151924133301, + 0.1391371637582779, + -0.2599793076515198, + 0.5955996513366699, + -0.9769980907440186, + -0.04995851218700409, + -0.9729883074760437, + 0.6893180012702942, + 0.6582030057907104, + -0.4735340476036072, + -0.9789217114448547, + -0.4554268717765808, + -0.7393642663955688 + ], + [ + -0.346935510635376, + -0.2151426374912262, + 0.2607693076133728, + 0.4880162179470062, + 0.9172625541687012, + 0.4352869391441345, + -0.16314536333084106, + 0.7385636568069458, + 0.04108767956495285, + 0.35693731904029846, + -0.46944478154182434, + 0.1872413158416748, + -1.5915976762771606, + -0.5122110843658447, + -1.081807017326355, + 0.6284721493721008, + 0.7538945078849792, + -0.45732825994491577, + -1.1423842906951904, + -0.5460995435714722, + -0.994105339050293 + ], + [ + -0.47438687086105347, + -0.2941962480545044, + 0.08930006623268127, + 0.47087201476097107, + 1.0732719898223877, + 0.7297742366790771, + -0.37707096338272095, + 0.6198458671569824, + -0.01236293837428093, + 0.19176959991455078, + -0.3815520703792572, + 0.4481086730957031, + -1.3105427026748657, + -0.6763743758201599, + -0.928192138671875, + 0.6959629654884338, + 0.8909534811973572, + -0.2225792557001114, + -1.0983164310455322, + 0.0018378496170043945, + -0.6349530220031738 + ], + [ + -0.29603272676467896, + -0.21692737936973572, + 0.34010499715805054, + 0.5396671295166016, + 1.0621541738510132, + 0.47140786051750183, + -0.16609090566635132, + 0.6417175531387329, + 0.02341892570257187, + 0.2948392331600189, + -0.5026776790618896, + 0.1336493194103241, + -1.3191566467285156, + -0.436301052570343, + -1.0181485414505005, + 0.6648247838020325, + 0.8199318051338196, + -0.44266223907470703, + -1.0431838035583496, + -0.4796689748764038, + -0.9188610315322876 + ], + [ + 0.00443209707736969, + -0.2032320648431778, + 0.6682220697402954, + 0.32137826085090637, + 0.6583718061447144, + 0.18535318970680237, + 0.11783808469772339, + 0.5542389154434204, + -0.036951079964637756, + 0.23047593235969543, + -0.48136118054389954, + -0.26839587092399597, + -1.2748585939407349, + -0.48537975549697876, + -0.9735562205314636, + 0.598412275314331, + 1.0493358373641968, + -0.3935312032699585, + -1.1261112689971924, + -0.5450727939605713, + -0.6077508926391602 + ], + [ + -0.3254464864730835, + -0.2069281041622162, + 0.5057176351547241, + 0.6721740961074829, + 0.6974209547042847, + 0.6202588081359863, + 0.06076979637145996, + 0.4048321545124054, + 0.07321679592132568, + 0.29294872283935547, + -0.5859494209289551, + 0.32686787843704224, + -1.3482577800750732, + -0.1401742547750473, + -1.0666649341583252, + 0.7469267249107361, + 0.6608222723007202, + -0.6406729221343994, + -1.144120454788208, + -0.49726879596710205, + -1.376233458518982 + ], + [ + -0.1317814141511917, + -0.08274295926094055, + 0.14742302894592285, + 0.12061131000518799, + 0.7764797806739807, + 0.5823787450790405, + -0.5051177144050598, + 0.5014482736587524, + 0.04729006439447403, + 0.3133741617202759, + -0.712706446647644, + 0.03383323550224304, + -1.275682806968689, + -0.030220016837120056, + -0.9907340407371521, + 0.704752504825592, + 0.8170828223228455, + -0.7511892318725586, + -1.035818338394165, + -0.5890134572982788, + -1.2898623943328857 + ] + ], + "generated_utilities": [ + 0.13394887745380402, + 0.06245779991149902, + 0.06475379317998886, + 0.07846011966466904, + 0.06508652120828629, + 0.051506221294403076, + 0.07821440696716309, + 0.08463313430547714 + ], + "group_id": "ms-728b865677134d3788e3", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05257302522659302, + 0.05559448525309563, + 0.11835493892431259, + 0.04238405078649521, + 0.16897141933441162, + 0.02001136541366577, + 0.1609041690826416, + 0.14248299598693848, + 0.14091376960277557, + 0.17822234332561493, + 0.22658847272396088, + 0.1736571490764618, + 0.18659374117851257, + 0.12753915786743164, + 0.07164359092712402, + 0.0063618822023272514 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04065108299255371, + 0.05943702906370163, + 0.0039327144622802734, + 0.10760968923568726, + -0.3465921878814697, + -0.6843758821487427, + 0.0, + 0.3034271001815796, + -0.38217779994010925, + 0.019724853336811066, + -1.1125431060791016, + -0.016437113285064697, + 0.09826856851577759, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + 0.055904269218444824, + -0.2721228003501892, + 0.07830148935317993, + 0.0 + ], + [ + -0.9075424671173096, + 0.06243014335632324, + 1.0039327144622803, + -0.6845037937164307, + 0.050797268748283386, + -1.6843758821487427, + -1.3340954780578613, + 1.3034271001815796, + -0.29618290066719055, + -0.035879287868738174, + -2.1125431060791016, + 0.9835628867149353, + -0.9017314314842224, + -1.01132333278656, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233 + ], + [ + -1.117501139640808, + 0.23503510653972626, + 0.9669153094291687, + -0.8317459225654602, + 0.06331060081720352, + -1.5474717617034912, + -1.3840656280517578, + 1.240980863571167, + -0.053158730268478394, + 0.10333722829818726, + -2.2084054946899414, + 1.1802411079406738, + -0.8987206816673279, + -1.136216402053833, + -1.9180536270141602, + 1.4563629627227783, + 2.625094413757324, + -0.1296394169330597, + 0.3340006470680237, + -0.7212131023406982, + 0.9384214878082275 + ], + [ + -1.2640053033828735, + 0.2415706366300583, + 0.9424604177474976, + -0.8472410440444946, + 0.35185110569000244, + -1.8022713661193848, + -1.2391949892044067, + 0.9333417415618896, + 0.03309231996536255, + 0.22933605313301086, + -1.7521884441375732, + 1.0360857248306274, + -0.6365935802459717, + -0.7614871859550476, + -2.103987216949463, + 1.5381183624267578, + 2.11069917678833, + -0.5904175639152527, + -0.07874824106693268, + -0.3884509205818176, + 1.0602728128433228 + ], + [ + -0.7705078721046448, + 0.18350359797477722, + 0.579454779624939, + -0.5434049963951111, + 0.2274666726589203, + -1.7208446264266968, + -1.5971848964691162, + 1.6145182847976685, + -0.6915329694747925, + 0.20122811198234558, + -1.8584421873092651, + 0.9026989340782166, + -0.5741626024246216, + -0.5299967527389526, + -1.544856309890747, + 1.7236231565475464, + 2.1288580894470215, + -0.7592856287956238, + 0.3358893394470215, + -0.2929669916629791, + 1.67848801612854 + ], + [ + -0.3686429262161255, + 0.3043383061885834, + 0.7294905781745911, + -0.8736650347709656, + 0.3122289776802063, + -1.5051836967468262, + -1.8593363761901855, + 0.8389579653739929, + -0.8785254955291748, + -0.5520821213722229, + -1.8517990112304688, + 1.3417646884918213, + -0.32390326261520386, + -0.42145586013793945, + -1.6530441045761108, + 1.7202519178390503, + 2.180105209350586, + -0.4262692928314209, + -0.1161729246377945, + -0.08550512790679932, + 1.1130309104919434 + ], + [ + -0.9971213936805725, + -0.4085577130317688, + 0.9404422044754028, + -0.9977414011955261, + 0.4963066577911377, + -1.4445728063583374, + -1.8557989597320557, + 1.3813822269439697, + -0.9350934028625488, + 0.6622016429901123, + -2.0982701778411865, + 0.5712680220603943, + -1.3476169109344482, + -0.8056140542030334, + -2.2595674991607666, + 1.7601569890975952, + 2.5158329010009766, + -0.8607662916183472, + -0.010314732789993286, + -0.9802933931350708, + 0.587103545665741 + ], + [ + -0.538765549659729, + -0.2632336914539337, + 0.574018120765686, + -1.3315669298171997, + 0.23096206784248352, + -1.4029772281646729, + -1.2746275663375854, + 1.674896478652954, + -0.6276504993438721, + -0.06763898581266403, + -1.44939124584198, + 1.1256837844848633, + -0.6540690064430237, + -0.7850072979927063, + -2.472240686416626, + 1.201231837272644, + 2.056978702545166, + 0.4725829064846039, + 0.7320148944854736, + -0.47291314601898193, + 1.2244172096252441 + ], + [ + -0.5474913120269775, + 0.3537777066230774, + 0.1717698574066162, + -0.9464929103851318, + -0.24061335623264313, + -2.5145106315612793, + -1.9605224132537842, + 2.171875476837158, + 0.5462924242019653, + 0.2324821650981903, + -2.4616305828094482, + 1.3455290794372559, + -0.703314483165741, + -0.6289631128311157, + -1.848944902420044, + 1.755210518836975, + 2.649005174636841, + -0.931267499923706, + -0.3109248876571655, + 0.21283608675003052, + 0.6994417309761047 + ], + [ + -1.4273285865783691, + 0.48591291904449463, + 1.4724129438400269, + -1.1116547584533691, + -0.04224221408367157, + -1.1567587852478027, + -1.0686161518096924, + 1.7429871559143066, + -1.1814582347869873, + 0.07730802893638611, + -2.1004433631896973, + 0.45253968238830566, + -0.7293854355812073, + -0.4477439522743225, + -1.986203908920288, + 1.1158549785614014, + 2.0316240787506104, + 0.5763667821884155, + -0.48857587575912476, + -1.07709801197052, + 0.5611822605133057 + ] + ], + "predicted_scores": [ + 6.5777764320373535, + 5.295867443084717, + 5.214710712432861, + 5.058055400848389, + 4.8819661140441895, + 4.6562299728393555, + 4.570862293243408, + 4.170901298522949 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296/train0", + "selected_index": 0, + "state_hash": "62e00728e448102460275bc2c35150dbf429ab0d5670a6706fa41256273bac3c", + "stored_base_utility": 0.05257302522659302, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20529556274414062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20529556274414062 + }, + "base_progress": 0.20529556274414062, + "base_success": false, + "base_utility": 0.20529556274414062, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10225218534469604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10225218534469604 + }, + { + "contact_quality": null, + "progress": 0.20188765227794647, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20188765227794647 + }, + { + "contact_quality": null, + "progress": 0.08769619464874268, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08769619464874268 + }, + { + "contact_quality": null, + "progress": 0.08251267671585083, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08251267671585083 + }, + { + "contact_quality": null, + "progress": 0.11142279952764511, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11142279952764511 + }, + { + "contact_quality": null, + "progress": 0.11381959915161133, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11381959915161133 + }, + { + "contact_quality": null, + "progress": 0.10222844034433365, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10222844034433365 + }, + { + "contact_quality": null, + "progress": 0.06529740244150162, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06529740244150162 + } + ], + "candidate_progress": [ + 0.10225218534469604, + 0.20188765227794647, + 0.08769619464874268, + 0.08251267671585083, + 0.11142279952764511, + 0.11381959915161133, + 0.10222844034433365, + 0.06529740244150162 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-6ddecc7f074b9e267409", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "generated_tangents": [ + [ + 0.6351838707923889, + 0.0554143562912941, + -0.6360185146331787, + 0.26119446754455566, + 0.40959879755973816, + -3.0019710063934326, + -1.2204558849334717, + -0.9697473049163818, + -0.07487937062978745, + 1.4689207077026367, + 0.8908620476722717, + 0.1178353801369667, + 1.4230598211288452, + 0.7022956013679504, + 0.10874947905540466, + 0.0357222706079483, + -0.4160614609718323, + 0.6562888622283936, + 0.3235812187194824, + -0.5519699454307556, + -0.43149104714393616 + ], + [ + 0.7582695484161377, + 0.02489960938692093, + -0.4969500005245209, + -0.16363608837127686, + 0.28399404883384705, + -3.015408754348755, + -2.1042070388793945, + -1.028049349784851, + 0.1981009840965271, + 0.9594929814338684, + 0.9217855334281921, + 0.3362846374511719, + 1.3983911275863647, + 1.0891989469528198, + 0.1956273913383484, + -0.2859035134315491, + -0.5257833003997803, + 0.8497936725616455, + -0.03918731212615967, + -0.18944242596626282, + 0.14280645549297333 + ], + [ + 0.8326467871665955, + 0.0583222359418869, + -0.6408113241195679, + 0.2002996802330017, + -0.02010539174079895, + -2.884190082550049, + -1.7671189308166504, + -1.2717655897140503, + -0.002875702455639839, + 1.2529200315475464, + 0.8150051236152649, + 0.26369452476501465, + 1.3973784446716309, + 0.9947073459625244, + 0.09533832967281342, + -0.07589471340179443, + -0.36194247007369995, + 0.7034773826599121, + 0.39536020159721375, + -0.37164270877838135, + -0.13375824689865112 + ], + [ + 0.8077759146690369, + 0.042051851749420166, + -0.0677703320980072, + 0.09784567356109619, + 0.2418215423822403, + -3.357654094696045, + -1.4908311367034912, + -1.162305235862732, + 0.06381805986166, + 1.28940749168396, + 1.0562357902526855, + 0.1521969735622406, + 1.536420226097107, + 0.3797754645347595, + -0.06844878196716309, + -0.1321399062871933, + -0.40969645977020264, + 0.5953527688980103, + 0.522000789642334, + -0.021678835153579712, + -0.348192423582077 + ], + [ + 0.7868461608886719, + -0.029668498784303665, + -0.7807492017745972, + -0.25344526767730713, + -0.43694227933883667, + -3.8253989219665527, + -2.1909995079040527, + -0.9644989967346191, + 0.08103717863559723, + 1.0811434984207153, + 1.4036166667938232, + 0.7007911205291748, + 1.2752180099487305, + 0.7197125554084778, + -0.0727132111787796, + -0.04798858240246773, + -0.02189616858959198, + 0.4078947603702545, + 0.31187865138053894, + 0.25585711002349854, + -0.39843496680259705 + ], + [ + 0.6747683882713318, + 0.009402386844158173, + -0.4406537711620331, + 0.04984545707702637, + 0.15408065915107727, + -3.1775104999542236, + -1.4603171348571777, + -1.0818332433700562, + 0.08093071728944778, + 1.2560621500015259, + 0.8789746165275574, + 0.21762767434120178, + 1.328750491142273, + 1.098083734512329, + 0.11343250423669815, + -0.1459447741508484, + -0.3442935347557068, + 0.622115433216095, + 0.3984382152557373, + -0.4171231985092163, + -0.29798778891563416 + ], + [ + 0.5690445303916931, + 0.00414787232875824, + -0.5417822599411011, + -0.05685913562774658, + 0.09590134024620056, + -3.223590135574341, + -1.3231456279754639, + -1.07421875, + 0.07517006993293762, + 1.3157005310058594, + 0.9222729206085205, + 0.3335229158401489, + 1.3532094955444336, + 1.1141421794891357, + 0.05757361277937889, + -0.15622928738594055, + -0.37984025478363037, + 0.6170745491981506, + 0.4994942247867584, + -0.30145910382270813, + -0.11340078711509705 + ], + [ + 0.5306358933448792, + 0.07138641923666, + -0.3610648810863495, + 0.08101078867912292, + -0.20862016081809998, + -3.3606131076812744, + -1.4476242065429688, + -1.1502933502197266, + -0.018826277926564217, + 1.3719052076339722, + 1.0567415952682495, + 0.14625899493694305, + 1.2521876096725464, + 1.2068270444869995, + -0.07887710630893707, + -0.20102551579475403, + -0.17374534904956818, + 0.6486114263534546, + 0.6353914737701416, + -0.6090413331985474, + -0.6406717300415039 + ] + ], + "generated_utilities": [ + 0.10225218534469604, + 0.20188765227794647, + 0.08769619464874268, + 0.08251267671585083, + 0.11142279952764511, + 0.11381959915161133, + 0.10222844034433365, + 0.06529740244150162 + ], + "group_id": "ms-7ea98b8f4d2220347ee0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18600693345069885, + 0.07475636899471283, + 0.27452313899993896, + 0.20920448005199432, + 0.18411576747894287, + 0.27989473938941956, + 0.19416475296020508, + 0.10948207229375839, + 0.123555026948452, + 0.1468026340007782, + 0.12693151831626892, + 0.11466284841299057, + 0.07585643231868744, + 0.12908481061458588, + 0.22501108050346375, + 0.06431269645690918 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.06974434852600098, + -0.02351950854063034, + 0.04567508399486542, + -0.07042121887207031, + -0.038941264152526855, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.08617860078811646, + 0.9112892150878906, + 0.7374391555786133, + 0.06539928913116455, + 0.03787863254547119, + 0.0, + -0.021528124809265137, + -0.04566385596990585, + -0.03331831097602844, + 1.6466763019561768, + 0.04444807767868042, + -0.27494049072265625, + 0.0 + ], + [ + 1.1160619258880615, + -0.10018755495548248, + 0.19327251613140106, + -1.3990073204040527, + 0.672177255153656, + -3.303851842880249, + -2.832740545272827, + -1.902563214302063, + 0.39503830671310425, + 1.542332410812378, + 1.6850805282592773, + 0.6842766404151917, + 1.1400834321975708, + 1.670598030090332, + 0.631821870803833, + -0.26601675152778625, + -0.2725638151168823, + 3.003178119659424, + 0.23009130358695984, + -1.0388810634613037, + -0.19119825959205627 + ], + [ + 1.031497836112976, + -0.2943698763847351, + -0.11846622079610825, + -1.3361037969589233, + 0.6225074529647827, + -3.545761823654175, + -2.5078811645507812, + -1.4024183750152588, + 0.6736462116241455, + 2.093475580215454, + 1.9168713092803955, + 0.6377919316291809, + 0.5105260014533997, + 1.8172619342803955, + 1.1300766468048096, + 0.1616329848766327, + 0.03408849239349365, + 2.2435190677642822, + 0.735376238822937, + -1.5760226249694824, + -0.7514586448669434 + ], + [ + 1.7542359828948975, + -0.7351009249687195, + -0.854544997215271, + -1.223644495010376, + 0.07327532768249512, + -3.150663137435913, + -2.5486767292022705, + -2.110039472579956, + 1.0206904411315918, + 2.1492340564727783, + 2.187993288040161, + 1.2547765970230103, + 0.8607165217399597, + 1.1150517463684082, + 0.9828771948814392, + 0.5030426383018494, + -0.4691219627857208, + 2.2485060691833496, + -0.14985057711601257, + -1.9480454921722412, + -0.05157876014709473 + ], + [ + 0.6040669679641724, + 0.5665496587753296, + 0.24913550913333893, + -1.6600215435028076, + -0.10813313722610474, + -3.8031911849975586, + -3.226392984390259, + -1.5098689794540405, + 1.0684192180633545, + 2.1557064056396484, + 1.545849084854126, + 1.011389970779419, + 1.113816499710083, + 0.9937302470207214, + 1.372757911682129, + -0.4055754244327545, + -0.23403912782669067, + 3.0650274753570557, + 0.1330145299434662, + -0.7907523512840271, + -0.6428619623184204 + ], + [ + 0.30788952112197876, + 0.6807127594947815, + -0.035938769578933716, + -0.8788743615150452, + -0.30191969871520996, + -4.111589431762695, + -2.432647705078125, + -1.7479791641235352, + -0.321926474571228, + 1.9823087453842163, + 1.6617395877838135, + 1.3319180011749268, + 1.5937504768371582, + 2.0502309799194336, + 0.63725745677948, + 0.7232187390327454, + -0.23885302245616913, + 1.8472312688827515, + 0.3300834596157074, + -0.43588966131210327, + -1.2869892120361328 + ], + [ + 0.3636443018913269, + -0.4284004867076874, + 0.2675324082374573, + -0.4861376881599426, + 0.2239738255739212, + -2.813422918319702, + -2.4891929626464844, + -1.165071725845337, + 0.6442201733589172, + 1.1830427646636963, + 1.177597999572754, + 0.33577069640159607, + 0.6362329721450806, + 1.5409071445465088, + 0.29718804359436035, + -0.5603961944580078, + -0.8089720010757446, + 2.6271419525146484, + -0.5456377267837524, + -0.2824941873550415, + -0.7960987091064453 + ], + [ + 1.1001086235046387, + -0.9762114882469177, + -0.00277864933013916, + -1.704659104347229, + 1.2373628616333008, + -3.385840654373169, + -1.6910454034805298, + -1.470743179321289, + 0.9354362487792969, + 1.9378679990768433, + 1.515415906906128, + -0.3002503514289856, + 0.21796274185180664, + 1.542279839515686, + 1.7353785037994385, + 0.23501873016357422, + -0.16601301729679108, + 2.1989498138427734, + 0.8671687841415405, + -1.6252647638320923, + -1.2141560316085815 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06974434852600098, + -0.15391214191913605, + -0.22159501910209656, + -0.07042121887207031, + -0.09706336259841919, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.1400253176689148, + 0.9112892150878906, + 0.7374391555786133, + 0.13415807485580444, + -0.10778343677520752, + 0.0, + -0.08470147848129272, + 0.1231045201420784, + 0.08352774381637573, + 1.6466763019561768, + -0.005736887454986572, + -0.27494049072265625, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.125826835632324, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.152101516723633, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.8942345976829529 + ] + ], + "predicted_scores": [ + 6.957677364349365, + 6.9082770347595215, + 6.669839859008789, + 6.668172359466553, + 6.608238697052002, + 6.531609058380127, + 5.921152114868164, + 5.370590686798096 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train0", + "selected_index": 0, + "state_hash": "a2b9f3b3ea6458c6734081cea861cf54026796a0a594bf9be3657d28f12e9b3c", + "stored_base_utility": 0.18600693345069885, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18213583528995514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18213583528995514 + }, + "base_progress": 0.18213583528995514, + "base_success": false, + "base_utility": 0.18213583528995514, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.004793623927980661, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004793623927980661 + }, + { + "contact_quality": null, + "progress": 0.0028257768135517836, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0028257768135517836 + }, + { + "contact_quality": null, + "progress": 0.0050675272941589355, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0050675272941589355 + }, + { + "contact_quality": null, + "progress": 0.0031685233116149902, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0031685233116149902 + }, + { + "contact_quality": null, + "progress": 0.0052528977394104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0052528977394104 + }, + { + "contact_quality": null, + "progress": 0.0036421220283955336, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0036421220283955336 + }, + { + "contact_quality": null, + "progress": 0.0021342237014323473, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0021342237014323473 + }, + { + "contact_quality": null, + "progress": 0.0036080479621887207, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0036080479621887207 + } + ], + "candidate_progress": [ + 0.004793623927980661, + 0.0028257768135517836, + 0.0050675272941589355, + 0.0031685233116149902, + 0.0052528977394104, + 0.0036421220283955336, + 0.0021342237014323473, + 0.0036080479621887207 + ], + "candidate_source_chart_ids": [ + "ms-8e12f7ad446482627dd3", + "ms-02311c7783bc67e188b4", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7f57045c114ddad153b5", + "generated_tangents": [ + [ + 0.19242781400680542, + -0.23486019670963287, + 1.0551730394363403, + -0.21431955695152283, + 1.8361363410949707, + 1.890921711921692, + 1.3092901706695557, + 0.3047295808792114, + -0.029983485117554665, + 0.7014550566673279, + 0.7062481045722961, + 1.2387549877166748, + 0.944677472114563, + 2.0297346115112305, + -0.12552250921726227, + -0.09496954083442688, + 1.6345525979995728, + -0.637722373008728, + 1.8834655284881592, + 1.8923728466033936, + 1.549077033996582 + ], + [ + 0.16655893623828888, + -0.2928306460380554, + 1.2990028858184814, + -0.5302565097808838, + 1.5151221752166748, + 1.952709674835205, + 1.515507698059082, + 0.39710211753845215, + -0.132785364985466, + 0.9167823195457458, + 0.584041953086853, + 1.1585752964019775, + 1.3343281745910645, + 1.6751090288162231, + -0.021682340651750565, + -0.006324931979179382, + 1.7948086261749268, + -0.5942758321762085, + 1.473301887512207, + 1.2825021743774414, + 1.409175992012024 + ], + [ + 0.1552114635705948, + -0.22567999362945557, + 1.1855015754699707, + -0.5588181614875793, + 1.2776870727539062, + 2.3382818698883057, + 1.4986867904663086, + 0.4236941337585449, + -0.12306109070777893, + 0.7310000061988831, + 0.6485676765441895, + 1.2998638153076172, + 1.7941218614578247, + 1.2643907070159912, + 0.12577852606773376, + 0.020068857818841934, + 1.6570929288864136, + -0.4710620641708374, + 1.4371609687805176, + 1.2294025421142578, + 1.42363703250885 + ], + [ + 0.2973508834838867, + -0.2790142893791199, + 1.1926442384719849, + -0.5703457593917847, + 1.4720335006713867, + 1.7663577795028687, + 1.6142139434814453, + 0.4225847125053406, + -0.10414119064807892, + 0.7925950288772583, + 0.8482772707939148, + 1.0342340469360352, + 1.5343438386917114, + 1.5455867052078247, + -0.214212566614151, + -0.04831160604953766, + 1.9689501523971558, + -0.5318179130554199, + 1.4848265647888184, + 1.660297155380249, + 1.5958086252212524 + ], + [ + 0.21230991184711456, + -0.18754854798316956, + 1.5562338829040527, + -0.1213027760386467, + 1.1427690982818604, + 1.6968116760253906, + 0.987946093082428, + 0.5736876726150513, + -0.13050565123558044, + 0.7572095990180969, + 0.550136387348175, + 0.46257203817367554, + 1.6667393445968628, + 1.0320311784744263, + -0.0623105950653553, + 0.08181780576705933, + 1.3406203985214233, + -0.35682857036590576, + 1.4835853576660156, + 1.5075814723968506, + 1.0305763483047485 + ], + [ + 0.22347886860370636, + -0.2839810252189636, + 1.3782404661178589, + -0.5439313054084778, + 1.3498327732086182, + 2.2578561305999756, + 1.5557303428649902, + 0.5230444669723511, + -0.17672592401504517, + 0.7562742233276367, + 0.5604004859924316, + 1.095818042755127, + 1.5878697633743286, + 1.8663661479949951, + -0.08795695006847382, + 0.01589171588420868, + 1.7301238775253296, + -0.5995655655860901, + 1.4616014957427979, + 1.4391319751739502, + 1.274280309677124 + ], + [ + 0.04314441233873367, + -0.31074202060699463, + 1.23866605758667, + -0.3395167291164398, + 1.4657118320465088, + 1.9613734483718872, + 1.8112022876739502, + 0.4000732898712158, + -0.178915336728096, + 0.9016785025596619, + 0.8109214901924133, + 0.8285111784934998, + 1.2619868516921997, + 1.8626943826675415, + -0.071990966796875, + -0.10383464395999908, + 2.030254364013672, + -0.37101006507873535, + 1.51361083984375, + 1.507206678390503, + 1.6656185388565063 + ], + [ + 0.15120479464530945, + -0.2534239888191223, + 1.0571345090866089, + -0.6449457406997681, + 1.4762277603149414, + 2.024937152862549, + 1.728184461593628, + 0.5318474173545837, + -0.19190508127212524, + 0.733979344367981, + 0.8016237020492554, + 0.957295298576355, + 1.5044066905975342, + 1.654699683189392, + -0.20007109642028809, + -0.04618840292096138, + 1.9570878744125366, + -0.4365650415420532, + 1.5185191631317139, + 1.5623745918273926, + 1.5185866355895996 + ] + ], + "generated_utilities": [ + 0.004793623927980661, + 0.0028257768135517836, + 0.0050675272941589355, + 0.0031685233116149902, + 0.0052528977394104, + 0.0036421220283955336, + 0.0021342237014323473, + 0.0036080479621887207 + ], + "group_id": "ms-7f57045c114ddad153b5", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.11277659982442856, + 0.1479199379682541, + 0.07541074603796005, + 0.042439721524715424, + 0.15517297387123108, + 0.03584214299917221, + 0.1660110056400299, + 0.12549421191215515, + 0.227622389793396, + 0.09924932569265366, + 0.18198737502098083, + 0.11130931228399277, + 0.10854844748973846, + 0.01617596670985222, + 0.08437560498714447, + 0.04182587191462517 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.013166621327400208, + -0.12399804592132568, + 0.48011910915374756, + 0.4925293028354645, + 0.6256281137466431, + 2.213102102279663, + 0.0, + -0.4044480323791504, + 0.1722363382577896, + 0.3389366865158081, + -0.08761294186115265, + 0.21791136264801025, + 0.9396321773529053, + 0.0, + 0.06717276573181152, + -0.5456919074058533, + 0.722014307975769, + -0.051606684923172, + 0.7784175276756287, + 0.5338386297225952, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.9143741130828857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.455221176147461, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.263336420059204 + ], + [ + -0.47406601905822754, + -0.37954479455947876, + 1.123671293258667, + -0.8903511762619019, + 1.5515481233596802, + 3.7488584518432617, + 0.9699987173080444, + -0.23581115901470184, + -0.2149372398853302, + 0.8883978128433228, + 0.5803861021995544, + 1.7047908306121826, + 2.149672746658325, + 2.2741973400115967, + 0.2513730227947235, + 0.6982267498970032, + 2.1354174613952637, + -1.151021122932434, + 1.9550246000289917, + 0.5981679558753967, + 1.338004469871521 + ], + [ + -0.1836935132741928, + -0.42300617694854736, + 1.7664438486099243, + -0.5576877593994141, + 1.0626983642578125, + 2.5276265144348145, + 2.3329849243164062, + -0.4834023118019104, + -0.25722599029541016, + 0.5209662914276123, + 0.24240870773792267, + 0.8533134460449219, + 1.5531291961669922, + 2.304234504699707, + 0.7609332799911499, + 1.0344312191009521, + 2.270921230316162, + -0.926997184753418, + 0.9014018177986145, + 2.259265422821045, + 2.172292470932007 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3910996615886688, + -0.9467402100563049, + 1.4744372367858887, + -0.2748112082481384, + 1.753147840499878, + 3.3321292400360107, + 1.7170560359954834, + 0.09920699894428253, + -0.11250317096710205, + 1.5213767290115356, + 0.0891318991780281, + 1.0193004608154297, + 1.7396382093429565, + 2.202378988265991, + 0.1176290512084961, + 0.5433868765830994, + 1.6113250255584717, + -0.2701912522315979, + 1.7789868116378784, + 1.246854305267334, + 1.6763029098510742 + ], + [ + 0.3669140636920929, + -0.8853410482406616, + 1.1054010391235352, + -0.008406088687479496, + 1.683157205581665, + 2.8090970516204834, + 2.1524839401245117, + -0.1140800267457962, + -0.16068093478679657, + 1.3338439464569092, + -0.34659242630004883, + 1.4494925737380981, + 2.0485284328460693, + 2.244007110595703, + 0.4862309694290161, + 0.6211890578269958, + 1.5708913803100586, + -0.22552455961704254, + 1.7964838743209839, + 1.386763095855713, + 1.3543281555175781 + ], + [ + 0.7718826532363892, + -0.5710152983665466, + 2.151604175567627, + -0.5648711323738098, + 2.1273765563964844, + 3.657623291015625, + 1.9845168590545654, + -0.1599704772233963, + -0.6097310781478882, + 1.135812520980835, + -0.09587362408638, + 1.555975079536438, + 2.3309824466705322, + 1.9832960367202759, + 0.46482378244400024, + 0.6467846035957336, + 1.7182108163833618, + -0.0750286877155304, + 1.9485927820205688, + 0.959426760673523, + 0.9474185109138489 + ] + ], + "predicted_scores": [ + 6.556987762451172, + 6.543363094329834, + 6.536247253417969, + 6.463170051574707, + 6.38189697265625, + 6.139730453491211, + 6.116417407989502, + 5.926776885986328 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524/train0", + "selected_index": 0, + "state_hash": "4885f9acf2601b3d2c201bc8e00ef617845ad2a6d08f96d585249345f457be89", + "stored_base_utility": 0.11277659982442856, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0056487321853637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056487321853637695 + }, + "base_progress": 0.0056487321853637695, + "base_success": false, + "base_utility": 0.0056487321853637695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.011145631782710552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011145631782710552 + }, + { + "contact_quality": null, + "progress": 0.005635479930788279, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005635479930788279 + }, + { + "contact_quality": null, + "progress": 0.0070413947105407715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0070413947105407715 + }, + { + "contact_quality": null, + "progress": 0.007375796791166067, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007375796791166067 + }, + { + "contact_quality": null, + "progress": 0.014856874942779541, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014856874942779541 + }, + { + "contact_quality": null, + "progress": 0.01037945318967104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01037945318967104 + }, + { + "contact_quality": null, + "progress": 0.025841474533081055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025841474533081055 + }, + { + "contact_quality": null, + "progress": 0.012394865043461323, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012394865043461323 + } + ], + "candidate_progress": [ + 0.011145631782710552, + 0.005635479930788279, + 0.0070413947105407715, + 0.007375796791166067, + 0.014856874942779541, + 0.01037945318967104, + 0.025841474533081055, + 0.012394865043461323 + ], + "candidate_source_chart_ids": [ + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "generated_tangents": [ + [ + -0.2132212072610855, + -0.35612571239471436, + 0.3206631541252136, + -0.6110020875930786, + 0.6720107793807983, + 1.8810362815856934, + 3.4679126739501953, + 0.42357224225997925, + -0.43441298604011536, + 0.38837340474128723, + -0.8442613482475281, + 1.170778512954712, + 1.1464968919754028, + 1.0374091863632202, + 0.8390651345252991, + -0.85950767993927, + -1.3317396640777588, + 0.4298861622810364, + 0.6741939783096313, + -1.2672675848007202, + -1.9145469665527344 + ], + [ + -0.15743465721607208, + -0.3636484146118164, + 0.842019259929657, + -0.7178770303726196, + 0.8974410891532898, + 2.18573260307312, + 2.904831886291504, + 0.42274922132492065, + -0.4765997529029846, + -0.02085917443037033, + -1.1092324256896973, + 1.50281822681427, + 1.4443620443344116, + 1.0851465463638306, + 1.1450011730194092, + -0.9535958170890808, + -1.3039427995681763, + 0.2522483170032501, + 0.7284320592880249, + -1.1985808610916138, + -1.8808517456054688 + ], + [ + -0.16789843142032623, + -0.38044852018356323, + 0.34564560651779175, + -0.7870309352874756, + 1.1070396900177002, + 1.9073704481124878, + 3.3052501678466797, + 0.5817533731460571, + -0.5266355872154236, + 0.40520206093788147, + -0.8947142362594604, + 1.0438597202301025, + 1.6923879384994507, + 0.6507884860038757, + 1.108149766921997, + -0.8551848530769348, + -1.1152418851852417, + 0.6647673845291138, + 0.48249322175979614, + -1.077775478363037, + -2.070512056350708 + ], + [ + -0.26104307174682617, + -0.2625615894794464, + 0.2304205298423767, + -0.793540358543396, + 1.4105303287506104, + 1.854358434677124, + 3.293538808822632, + 0.5218722820281982, + -0.5116921067237854, + 0.3137792646884918, + -0.7852323651313782, + 1.2465404272079468, + 1.62022864818573, + 0.8447396159172058, + 1.2068796157836914, + -0.9205024838447571, + -1.1353341341018677, + 0.5821287035942078, + 0.2939908802509308, + -0.8216809630393982, + -2.2633392810821533 + ], + [ + -0.6575313806533813, + -0.23661266267299652, + 0.48808079957962036, + -0.8921552896499634, + 1.520573377609253, + 1.6683564186096191, + 2.1068243980407715, + 0.3933829069137573, + -0.33618858456611633, + 0.18094909191131592, + -1.3109817504882812, + 0.3120250701904297, + 1.5246118307113647, + 1.3523991107940674, + 0.9605004787445068, + -0.4774652421474457, + -1.7408987283706665, + 0.3444976806640625, + 0.009613960981369019, + -1.4281773567199707, + -1.9245011806488037 + ], + [ + -0.598523736000061, + -0.30479729175567627, + 0.5191957354545593, + -0.8284745216369629, + 1.6283166408538818, + 1.6675724983215332, + 2.249469518661499, + 0.5003149509429932, + -0.3607867360115051, + 0.1619831770658493, + -1.466862678527832, + 0.3162725269794464, + 1.5596030950546265, + 1.3193833827972412, + 1.085457682609558, + -0.3821507692337036, + -1.578502893447876, + 0.38166701793670654, + 0.4683684706687927, + -1.3051135540008545, + -1.7824459075927734 + ], + [ + -0.7587069869041443, + -0.46214473247528076, + 0.35767853260040283, + -0.9201654195785522, + 1.1327975988388062, + 1.5855541229248047, + 2.024463176727295, + 0.25509265065193176, + -0.22467383742332458, + 0.2761390209197998, + -1.2066025733947754, + 0.3324081301689148, + 1.442461609840393, + 1.7368476390838623, + 0.9266427159309387, + -0.4226803183555603, + -1.7133150100708008, + 0.2403825968503952, + 0.1079477071762085, + -1.9669630527496338, + -1.821255087852478 + ], + [ + -0.3811972737312317, + -0.5896664261817932, + 0.9327340126037598, + -1.2426563501358032, + 1.617842197418213, + 1.8171278238296509, + 2.357351064682007, + 0.24668310582637787, + -0.2678268849849701, + 0.17105263471603394, + -1.330452799797058, + 0.31128764152526855, + 0.7893147468566895, + 1.9798027276992798, + 0.8406444787979126, + -0.400086909532547, + -1.5437424182891846, + 0.1732153743505478, + 0.13039374351501465, + -1.1935398578643799, + -2.0147836208343506 + ] + ], + "generated_utilities": [ + 0.011145631782710552, + 0.005635479930788279, + 0.0070413947105407715, + 0.007375796791166067, + 0.014856874942779541, + 0.01037945318967104, + 0.025841474533081055, + 0.012394865043461323 + ], + "group_id": "ms-81f4e8858b2ba90aea2e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.01361372135579586, + 0.007307033054530621, + 0.021326780319213867, + 0.04093001410365105, + 0.15059307217597961, + 0.013598322868347168, + 0.09240063279867172, + 0.1631626933813095, + 0.05027441307902336, + 0.05712141841650009, + 0.1538398563861847, + 0.06023097038269043, + 0.020811915397644043, + 0.019836824387311935, + 0.1318017691373825, + 0.051341354846954346 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06500570476055145, + 0.046472612768411636, + 1.2439939975738525, + -1.446747064590454, + 1.7077040672302246, + 2.291205883026123, + 2.1016674041748047, + -0.6354803442955017, + -0.4465000331401825, + 0.9666666984558105, + -1.1514010429382324, + 0.45639365911483765, + 1.613925814628601, + 1.0568119287490845, + 1.3792463541030884, + -0.7852597832679749, + -1.576931118965149, + 1.1258286237716675, + 0.9614027738571167, + -0.9620904922485352, + -1.7528356313705444 + ], + [ + -0.23400194942951202, + -0.21453839540481567, + 1.458961844444275, + -1.701543927192688, + 1.4158955812454224, + 2.2648162841796875, + 2.2270467281341553, + -0.5239131450653076, + -0.44378146529197693, + 1.0904332399368286, + -1.223331093788147, + 0.45784294605255127, + 1.6057316064834595, + 0.9251953363418579, + 1.308751106262207, + -0.6351475715637207, + -1.5950077772140503, + 1.0428481101989746, + 0.6781296730041504, + -0.9007717370986938, + -1.880659818649292 + ], + [ + 0.2324782907962799, + 0.05129373073577881, + 1.1239540576934814, + -1.1853233575820923, + 1.3290393352508545, + 1.933064341545105, + 2.192976951599121, + -0.23718085885047913, + -0.279358446598053, + 1.3028839826583862, + -1.4676742553710938, + 0.16475489735603333, + 1.67984938621521, + 0.6600161790847778, + 1.5654886960983276, + -1.1545368432998657, + -1.5593464374542236, + 1.1976646184921265, + 0.5715436935424805, + -0.9047394394874573, + -1.9614591598510742 + ], + [ + 0.7452290058135986, + 0.614987313747406, + 0.7383754849433899, + -1.5856380462646484, + 1.264894723892212, + 1.7337512969970703, + 2.502504825592041, + 0.040616750717163086, + -1.1382783651351929, + 0.7095704674720764, + -1.014717698097229, + 0.5126742720603943, + 1.3401073217391968, + 1.4275505542755127, + 1.4739655256271362, + -0.7308655381202698, + -0.9416123032569885, + 0.661049485206604, + 0.9453439712524414, + -1.5968079566955566, + -1.715583324432373 + ], + [ + 0.15219944715499878, + 0.5165310502052307, + 1.0654370784759521, + -0.5756344199180603, + 1.2834129333496094, + 1.6370232105255127, + 2.414426326751709, + -1.1256691217422485, + -1.1171015501022339, + 0.5337101817131042, + -0.9850769639015198, + -0.35428309440612793, + 2.4350547790527344, + 1.8563504219055176, + 2.1929399967193604, + -0.1821107268333435, + -1.4600040912628174, + 1.5430631637573242, + 1.4937412738800049, + -0.8152608275413513, + -2.020042896270752 + ] + ], + "predicted_scores": [ + 13.856819152832031, + 12.800796508789062, + 12.187582015991211, + 12.16637897491455, + 11.499547004699707, + 11.33340072631836, + 10.854905128479004, + 10.459314346313477 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train0", + "selected_index": 0, + "state_hash": "562e8bc3804d1d310e0eaf612dbc110b8efc94514458dacf551ef33f4f0b8ee7", + "stored_base_utility": 0.01361372135579586, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.00698717450723052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00698717450723052 + }, + "base_progress": 0.00698717450723052, + "base_success": false, + "base_utility": 0.00698717450723052, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06672348827123642, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06672348827123642 + }, + { + "contact_quality": null, + "progress": 0.029613912105560303, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029613912105560303 + }, + { + "contact_quality": null, + "progress": 0.14832614362239838, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14832614362239838 + }, + { + "contact_quality": null, + "progress": 0.1094479188323021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1094479188323021 + }, + { + "contact_quality": null, + "progress": 0.10840656608343124, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10840656608343124 + }, + { + "contact_quality": null, + "progress": 0.09923364967107773, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09923364967107773 + }, + { + "contact_quality": null, + "progress": 0.055296678096055984, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.055296678096055984 + }, + { + "contact_quality": null, + "progress": 0.09003522247076035, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09003522247076035 + } + ], + "candidate_progress": [ + 0.06672348827123642, + 0.029613912105560303, + 0.14832614362239838, + 0.1094479188323021, + 0.10840656608343124, + 0.09923364967107773, + 0.055296678096055984, + 0.09003522247076035 + ], + "candidate_source_chart_ids": [ + "ms-45fbd749d5ed3fe37c13", + "ms-48178f52a96de3d1ead0", + "ms-25ab20bb3da8ba34be95", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-9229ce141dd41a11fac1", + "ms-9229ce141dd41a11fac1" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8a72b73ac452bc190aeb", + "generated_tangents": [ + [ + 0.5802809000015259, + -0.17232182621955872, + -0.13074412941932678, + -0.9402573108673096, + 0.24096477031707764, + -2.593315362930298, + -1.6888880729675293, + 0.38430607318878174, + -0.14453405141830444, + -0.09521184116601944, + 0.7020482420921326, + 0.7879524230957031, + 0.5401550531387329, + -0.31366825103759766, + 0.40505534410476685, + -0.15511159598827362, + -0.3023306727409363, + 0.30714017152786255, + 0.24821415543556213, + -0.5041369199752808, + -0.7858303785324097 + ], + [ + 0.585083544254303, + 0.010379355400800705, + -0.24248263239860535, + -0.8754599094390869, + 0.4871060848236084, + -3.1324737071990967, + -0.9886792302131653, + 0.08241399377584457, + -0.1512652486562729, + -0.04688064008951187, + 0.7735022306442261, + 0.4781321883201599, + 0.3961280286312103, + -0.13697953522205353, + 0.14559334516525269, + -0.17399317026138306, + -0.23298145830631256, + 0.3033895492553711, + 0.06670184433460236, + -0.3057488799095154, + -0.6404585838317871 + ], + [ + 0.41040998697280884, + -0.16142237186431885, + -0.21280953288078308, + -0.6524162292480469, + 0.4905272424221039, + -2.5773022174835205, + -1.0271387100219727, + 0.2494909167289734, + -0.1529950648546219, + -0.03776315599679947, + 0.550376296043396, + 0.3485774099826813, + 0.14609310030937195, + -0.1448907107114792, + -0.09965980052947998, + -0.11929327249526978, + 0.058319732546806335, + 0.5182379484176636, + 0.07967442274093628, + -0.2018723487854004, + -0.012257322669029236 + ], + [ + 0.544282078742981, + -0.1067652553319931, + -0.37422457337379456, + -0.8048790693283081, + 0.10579922795295715, + -2.541926622390747, + -1.207503080368042, + 0.35123443603515625, + -0.1142362654209137, + -0.0673704519867897, + 0.7051560282707214, + 0.46379974484443665, + -0.07719600200653076, + 0.3612074851989746, + -4.211440682411194e-05, + -0.1668364554643631, + 0.2595708966255188, + 0.6407525539398193, + 0.3123566806316376, + -0.25552186369895935, + -0.0617387592792511 + ], + [ + 0.4974521994590759, + -0.12853306531906128, + -0.4152775704860687, + -0.7610821723937988, + 0.13901180028915405, + -2.373908519744873, + -1.3535418510437012, + 0.3698514997959137, + -0.14082284271717072, + -0.04558420926332474, + 0.7580047845840454, + 0.5020407438278198, + -0.051934778690338135, + 0.4010813236236572, + 0.002136319875717163, + -0.17584328353405, + 0.19179822504520416, + 0.7005574107170105, + 0.3955182433128357, + -0.19895638525485992, + -0.027681156992912292 + ], + [ + 0.532270073890686, + -0.057570524513721466, + -0.3229869306087494, + -0.8626034259796143, + 0.03135751187801361, + -2.48866605758667, + -1.1516649723052979, + 0.3311689496040344, + -0.10964304953813553, + 0.00415164977312088, + 0.6936989426612854, + 0.5873826146125793, + -0.2168811857700348, + 0.25383713841438293, + -0.00653226301074028, + -0.16879411041736603, + 0.17690184712409973, + 0.6384727954864502, + 0.4651601016521454, + -0.18900656700134277, + -0.113162100315094 + ], + [ + 0.2758582532405853, + -0.11883850395679474, + -0.0904715359210968, + -0.8463842868804932, + 0.4136640429496765, + -2.2815024852752686, + -0.79213947057724, + 0.09477245807647705, + -0.22156566381454468, + -0.08652854710817337, + 0.6423437595367432, + 0.5776209831237793, + -0.06339520215988159, + 0.28895413875579834, + 0.08238177746534348, + -0.19808807969093323, + 0.13033677637577057, + 0.6563591957092285, + 0.33592918515205383, + 0.0385357141494751, + -0.17127865552902222 + ], + [ + 0.2607884705066681, + -0.12364064902067184, + -0.3031565845012665, + -0.7547322511672974, + 0.5611181259155273, + -2.331019163131714, + -0.8667411208152771, + 0.09982959181070328, + -0.23611301183700562, + -0.10993482917547226, + 0.659524142742157, + 0.5154794454574585, + -0.25269031524658203, + 0.3010188937187195, + 0.05662054196000099, + -0.20263463258743286, + 0.09702354669570923, + 0.7584511041641235, + 0.25624024868011475, + -0.11499883234500885, + -0.2106596827507019 + ] + ], + "generated_utilities": [ + 0.06672348827123642, + 0.029613912105560303, + 0.14832614362239838, + 0.1094479188323021, + 0.10840656608343124, + 0.09923364967107773, + 0.055296678096055984, + 0.09003522247076035 + ], + "group_id": "ms-8a72b73ac452bc190aeb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.02298194169998169, + 0.01763375662267208, + 0.009361605159938335, + 0.02526181936264038, + 0.09214292466640472, + 0.022978028282523155, + 0.07704488933086395, + 0.056830525398254395, + 0.045355141162872314, + 0.06095697730779648, + 0.10329639911651611, + 0.05954315513372421, + 0.03003152273595333, + 0.008136491291224957, + 0.05599620193243027, + 0.011116823181509972 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.0913797616958618, + 0.3305599093437195, + -0.0778055489063263, + -2.223818778991699, + 0.9291321635246277, + -2.905466318130493, + -2.4575929641723633, + 1.1732350587844849, + 0.22933529317378998, + -0.8654288053512573, + 2.1122353076934814, + 1.5987088680267334, + 1.144788146018982, + 0.40604642033576965, + 0.10343833267688751, + -0.7513935565948486, + -1.054006814956665, + 1.396103024482727, + -0.16642586886882782, + -0.04196041822433472, + -1.2630513906478882 + ], + [ + 1.3690472841262817, + 0.2738865315914154, + -0.27599766850471497, + -2.431976556777954, + 0.7025187015533447, + -3.155372142791748, + -2.6898910999298096, + 1.1788215637207031, + 0.19074484705924988, + -0.7378551959991455, + 2.3089826107025146, + 1.3478807210922241, + 1.3458092212677002, + 0.3417958617210388, + 0.24155153334140778, + -1.004831314086914, + -1.3499912023544312, + 1.311987042427063, + -0.25646817684173584, + -0.3180008828639984, + -1.3290650844573975 + ], + [ + 0.9032705426216125, + -0.3011789321899414, + -0.21911907196044922, + -2.884286403656006, + 0.319735586643219, + -2.613290548324585, + -2.175532817840576, + 1.5185720920562744, + -0.1741684526205063, + -0.4879068434238434, + 2.721827507019043, + 2.1753995418548584, + 0.8214534521102905, + 0.7874165177345276, + 0.013686276972293854, + -0.7485443949699402, + -0.764183521270752, + 0.9452257752418518, + 0.07210072875022888, + 0.2788098156452179, + -1.8349425792694092 + ] + ], + "predicted_scores": [ + 9.848700523376465, + 9.15388298034668, + 8.921833038330078, + 8.639586448669434, + 8.411186218261719, + 8.158957481384277, + 7.247722625732422, + 7.04017972946167 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train0", + "selected_index": 0, + "state_hash": "722240dc5e14c5a30badc5fbec9f15ecf388cf6402fb77013c8f9dbdcd100846", + "stored_base_utility": 0.02298194169998169, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08285123109817505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08285123109817505 + }, + "base_progress": 0.08285123109817505, + "base_success": false, + "base_utility": 0.08285123109817505, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.020249387249350548, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020249387249350548 + }, + { + "contact_quality": null, + "progress": 0.023661911487579346, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023661911487579346 + }, + { + "contact_quality": null, + "progress": 0.022281387820839882, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022281387820839882 + }, + { + "contact_quality": null, + "progress": 0.016953786835074425, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016953786835074425 + }, + { + "contact_quality": null, + "progress": 0.03129972890019417, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03129972890019417 + }, + { + "contact_quality": null, + "progress": 0.016959169879555702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016959169879555702 + }, + { + "contact_quality": null, + "progress": 0.03157500550150871, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03157500550150871 + }, + { + "contact_quality": null, + "progress": 0.018817603588104248, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018817603588104248 + } + ], + "candidate_progress": [ + 0.020249387249350548, + 0.023661911487579346, + 0.022281387820839882, + 0.016953786835074425, + 0.03129972890019417, + 0.016959169879555702, + 0.03157500550150871, + 0.018817603588104248 + ], + "candidate_source_chart_ids": [ + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-a96bca1ef0b74f686018", + "generated_tangents": [ + [ + 0.3499390184879303, + 0.028515934944152832, + 0.48948806524276733, + -0.19309116899967194, + 0.6901988983154297, + 0.7414987087249756, + 1.3991317749023438, + 0.3446234166622162, + -0.09190987050533295, + 0.4723714292049408, + -1.2144007682800293, + 0.027294978499412537, + 0.2527715861797333, + -0.9874541759490967, + 0.16269424557685852, + 0.018625933676958084, + 0.18118378520011902, + -0.828308641910553, + -0.5594342350959778, + -0.08648960292339325, + 0.18008112907409668 + ], + [ + -0.09269364178180695, + -0.1040714681148529, + 0.22524771094322205, + -0.3401437997817993, + 0.35260310769081116, + -0.31100767850875854, + 0.9163292646408081, + 0.5302736759185791, + -0.2567659616470337, + 0.14966259896755219, + -1.0596189498901367, + 0.646892786026001, + -0.529009222984314, + -0.9970532655715942, + -0.02623092755675316, + 0.17372286319732666, + 0.23263417184352875, + -0.3831707835197449, + -0.5076351165771484, + -0.2827019691467285, + -1.0047863721847534 + ], + [ + 0.27982860803604126, + 0.038833484053611755, + 0.5253910422325134, + -0.2953806221485138, + 0.7850337028503418, + 0.04751972109079361, + 0.6080531477928162, + 0.255418062210083, + -0.08356447517871857, + 0.4131060838699341, + -1.0247589349746704, + 0.26699909567832947, + -0.08355796337127686, + -0.8738237619400024, + 0.292209655046463, + 0.10542076826095581, + 0.22872647643089294, + -0.5168390870094299, + -0.3613443970680237, + -0.38312795758247375, + -0.16131848096847534 + ], + [ + 0.24409164488315582, + -0.07521600276231766, + 0.4027271270751953, + -0.21762821078300476, + 0.7221702337265015, + 0.15511786937713623, + 0.7601394653320312, + 0.31568828225135803, + -0.17028523981571198, + 0.398857057094574, + -1.0321813821792603, + 0.28403183817863464, + -0.012781113386154175, + -0.808844804763794, + 0.3304174244403839, + 0.10524564981460571, + 0.17134501039981842, + -0.5901852250099182, + -0.24532389640808105, + -0.3594452440738678, + -0.03938356041908264 + ], + [ + 0.221520334482193, + 0.13385054469108582, + 0.5074416399002075, + -0.5112059116363525, + 0.6898107528686523, + 0.50984787940979, + 0.28407323360443115, + 0.2582123875617981, + 0.007201352156698704, + 0.3026183843612671, + -0.8207933902740479, + 0.3518885374069214, + -0.16800397634506226, + 0.003198191523551941, + 0.29176315665245056, + 0.11074809730052948, + 0.12544113397598267, + -0.5459253191947937, + -0.35079002380371094, + -0.3331269919872284, + -0.412781685590744 + ], + [ + 0.3424416184425354, + -0.03370369225740433, + 0.48118382692337036, + -0.23526175320148468, + 0.7294629812240601, + 0.1193767637014389, + 0.536425769329071, + 0.2792583405971527, + -0.16809089481830597, + 0.3494786322116852, + -1.0050755739212036, + 0.27076736092567444, + 0.1457333266735077, + -0.9353020191192627, + 0.36139115691185, + 0.05636360123753548, + 0.17889519035816193, + -0.6878611445426941, + -0.2541653513908386, + -0.47968965768814087, + 0.10619336366653442 + ], + [ + 0.35013851523399353, + 0.19337916374206543, + 0.6391496658325195, + -0.44786036014556885, + 0.8085934519767761, + 0.7319002151489258, + 0.5004884004592896, + 0.22929589450359344, + -0.0033745383843779564, + 0.08201625198125839, + -0.8485991358757019, + 0.2280895859003067, + -0.3726976215839386, + -0.11025431752204895, + 0.4019944965839386, + 0.133483424782753, + 0.10760033875703812, + -0.5622149109840393, + -0.2597987651824951, + 0.11388179659843445, + -0.44191405177116394 + ], + [ + 0.2736077308654785, + -0.04517064243555069, + 0.520946204662323, + -0.3296894133090973, + 1.0444467067718506, + 0.8971071243286133, + 0.7990139722824097, + 0.08666207641363144, + -0.24233770370483398, + 0.5963983535766602, + -0.9442305564880371, + -0.13999488949775696, + -0.15416544675827026, + -0.3024454116821289, + 0.16026315093040466, + 0.14706969261169434, + 0.12341497838497162, + -0.4079155921936035, + -0.12926006317138672, + -0.03957882523536682, + 0.3990253806114197 + ] + ], + "generated_utilities": [ + 0.020249387249350548, + 0.023661911487579346, + 0.022281387820839882, + 0.016953786835074425, + 0.03129972890019417, + 0.016959169879555702, + 0.03157500550150871, + 0.018817603588104248 + ], + "group_id": "ms-a96bca1ef0b74f686018", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07923875749111176, + 0.1507580727338791, + 0.09708476066589355, + 0.1572066992521286, + 0.09626718610525131, + 0.0781460627913475, + 0.07252912223339081, + 0.04244875907897949, + 0.026137471199035645, + 0.03892441838979721, + 0.04886416718363762, + 0.10530619323253632, + 0.0477486252784729, + 0.0898323506116867, + 0.01968318223953247, + 0.0289560966193676 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.48996758460998535, + 0.9053895473480225, + 1.8985445499420166, + -1.3594893217086792, + 1.6097791194915771, + 1.132408618927002, + 1.8319320678710938, + 0.46463528275489807, + 0.13208985328674316, + 0.8792686462402344, + -1.868321418762207, + 1.1490275859832764, + -2.0193891525268555, + -2.0637927055358887, + 0.11448347568511963, + 1.2957178354263306, + 1.293219804763794, + -1.7789109945297241, + -0.4312591254711151, + -1.7441130876541138, + -0.06536102294921875 + ], + [ + 0.03654795140028, + 1.7676925659179688, + 1.048339605331421, + -1.366391658782959, + 2.5093846321105957, + 2.160305976867676, + 2.8675124645233154, + -0.309950590133667, + -0.5841768980026245, + 1.3030188083648682, + -1.2228018045425415, + 1.1373907327651978, + -2.831526041030884, + -2.2111129760742188, + -0.14044174551963806, + 1.660491704940796, + 1.5215414762496948, + -1.0137104988098145, + -0.40579986572265625, + -1.9571893215179443, + 1.241256833076477 + ], + [ + -0.2826732099056244, + 2.2520718574523926, + 2.418309450149536, + -1.7572941780090332, + 1.5832278728485107, + 0.6361246705055237, + 2.2009639739990234, + 0.9109495878219604, + 0.22027403116226196, + 0.7075455188751221, + -1.2762477397918701, + 0.8630691766738892, + -1.2986351251602173, + -2.567821979522705, + -0.4690782427787781, + 2.1363346576690674, + 0.1853562593460083, + -1.541505217552185, + -0.033681899309158325, + -1.0161254405975342, + 0.06877446174621582 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03534998744726181, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.3504420518875122, + 0.0, + -0.013733148574829102, + 0.021970629692077637, + 0.014389097690582275, + -0.7432765960693359, + 0.006671249866485596, + -0.9755387306213379, + 0.0, + -0.027152668684720993, + 0.2708500623703003, + 0.052558183670043945, + -0.7615737915039062, + 0.020878344774246216, + -0.2927910089492798, + 0.0 + ], + [ + -0.1400495320558548, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.5714048147201538, + 0.0, + -0.3271818161010742, + -0.04413123428821564, + 0.16598933935165405, + -0.7432765960693359, + 0.2282017469406128, + -0.9755387306213379, + 0.0, + -0.4382812976837158, + 0.2708500623703003, + 0.2094714641571045, + -0.7615737915039062, + 0.23577484488487244, + -0.2927910089492798, + 0.0 + ] + ], + "predicted_scores": [ + 8.650808334350586, + 8.515298843383789, + 8.338103294372559, + 7.722647190093994, + 7.502362251281738, + 7.491379261016846, + 7.030318260192871, + 6.6235456466674805 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train0", + "selected_index": 0, + "state_hash": "61714505b9ac6cbb1c6deff698bb3e96fd54c7cdb8f0d4307bfdd236634525c0", + "stored_base_utility": 0.07923875749111176, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.13897453248500824, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13897453248500824 + }, + { + "contact_quality": null, + "progress": 0.10920015722513199, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10920015722513199 + }, + { + "contact_quality": null, + "progress": 0.1372818946838379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1372818946838379 + }, + { + "contact_quality": null, + "progress": 0.11862999200820923, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11862999200820923 + }, + { + "contact_quality": null, + "progress": 0.11477562040090561, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11477562040090561 + }, + { + "contact_quality": null, + "progress": 0.13643063604831696, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13643063604831696 + }, + { + "contact_quality": null, + "progress": 0.128423273563385, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.128423273563385 + }, + { + "contact_quality": null, + "progress": 0.09418871253728867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09418871253728867 + } + ], + "candidate_progress": [ + 0.13897453248500824, + 0.10920015722513199, + 0.1372818946838379, + 0.11862999200820923, + 0.11477562040090561, + 0.13643063604831696, + 0.128423273563385, + 0.09418871253728867 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-ac4fb33f3ab4db853995", + "generated_tangents": [ + [ + 2.3757364749908447, + -0.27223077416419983, + -0.3891852796077728, + 0.7745223045349121, + -0.446671724319458, + 2.279881000518799, + 1.8678135871887207, + 2.2902791500091553, + 0.25990647077560425, + -1.510072112083435, + -0.7205798625946045, + -0.9597678184509277, + -2.9762816429138184, + -3.139012098312378, + 1.6004019975662231, + 0.10748055577278137, + -1.2820088863372803, + 0.7429741024971008, + -0.2779664397239685, + -0.9638658165931702, + -1.136738657951355 + ], + [ + 2.2156591415405273, + -0.1741964966058731, + -0.6328035593032837, + 0.6409015655517578, + -0.20161858201026917, + 2.043597459793091, + 1.1954050064086914, + 2.1038784980773926, + 0.17956028878688812, + -1.4242427349090576, + -0.5126789808273315, + -0.8243266344070435, + -2.6796391010284424, + -2.430363893508911, + 1.8180451393127441, + -0.07958535850048065, + -1.4335682392120361, + 0.4636613130569458, + 0.040396884083747864, + -1.315406322479248, + -0.4311096966266632 + ], + [ + 2.419240951538086, + -0.23622797429561615, + -0.6210044622421265, + 0.5424371957778931, + -0.3453226089477539, + 2.029294490814209, + 2.1774356365203857, + 2.1821088790893555, + 0.301272988319397, + -1.5728907585144043, + -0.7277917265892029, + -1.073206901550293, + -3.0157980918884277, + -3.2045888900756836, + 1.7884563207626343, + -0.04911618307232857, + -1.4222990274429321, + 0.6680842638015747, + -0.36462271213531494, + -1.1607216596603394, + -1.462888479232788 + ], + [ + 2.3462376594543457, + -0.21311748027801514, + -0.5850708484649658, + 0.58466637134552, + -0.3197994828224182, + 2.0855517387390137, + 2.11960768699646, + 2.3327271938323975, + 0.25269460678100586, + -1.5906559228897095, + -0.6323912739753723, + -0.9704151153564453, + -2.539240598678589, + -3.236276865005493, + 1.8979101181030273, + -0.004533700179308653, + -1.3840817213058472, + 0.7861436009407043, + -0.29279518127441406, + -1.1931861639022827, + -1.2252819538116455 + ], + [ + 2.338474750518799, + -0.13967642188072205, + -0.39760419726371765, + 0.5495784282684326, + -0.6413520574569702, + 2.1784238815307617, + 1.938929557800293, + 2.422785758972168, + 0.26703953742980957, + -1.5116997957229614, + -0.6223490834236145, + -1.0662301778793335, + -2.746263265609741, + -3.1816065311431885, + 1.769331932067871, + -0.002677437849342823, + -1.1240551471710205, + 0.8676563501358032, + -0.2712363004684448, + -1.125828742980957, + -1.083319902420044 + ], + [ + 2.255201578140259, + -0.12751361727714539, + -0.5567959547042847, + 0.6307381391525269, + -0.4799419045448303, + 2.024019479751587, + 2.1195852756500244, + 2.303117513656616, + 0.2746199369430542, + -1.4792096614837646, + -0.6680079102516174, + -1.0119272470474243, + -2.908134937286377, + -3.2704591751098633, + 1.9480485916137695, + -0.011402618139982224, + -1.2913742065429688, + 0.7679071426391602, + -0.230330228805542, + -1.0150169134140015, + -1.1405054330825806 + ], + [ + 2.3860251903533936, + -0.22043639421463013, + -0.4959961473941803, + 0.20974993705749512, + -0.1611173152923584, + 1.9938403367996216, + 2.0718514919281006, + 2.077143430709839, + 0.31236532330513, + -1.8010988235473633, + -0.5873530507087708, + -0.979231595993042, + -2.47955584526062, + -2.967294216156006, + 1.6589266061782837, + -0.19949805736541748, + -1.3049830198287964, + 0.8772557973861694, + -0.18165990710258484, + -1.5418906211853027, + -1.3559668064117432 + ], + [ + 2.299116373062134, + -0.1917165219783783, + -0.4935736358165741, + 0.6249113082885742, + -0.025500982999801636, + 1.851459264755249, + 1.139762043952942, + 2.1488285064697266, + 0.2613762617111206, + -1.504454255104065, + -0.25766506791114807, + -0.9192250967025757, + -2.4594264030456543, + -2.520716905593872, + 1.9626132249832153, + -0.038190580904483795, + -1.1951498985290527, + 0.6068487763404846, + 0.4906315207481384, + -0.607388973236084, + -1.844285488128662 + ] + ], + "generated_utilities": [ + 0.13897453248500824, + 0.10920015722513199, + 0.1372818946838379, + 0.11862999200820923, + 0.11477562040090561, + 0.13643063604831696, + 0.128423273563385, + 0.09418871253728867 + ], + "group_id": "ms-ac4fb33f3ab4db853995", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.2651830017566681, + 0.22676818072795868, + 0.22454051673412323, + 2.0, + 0.31214088201522827, + 0.21535459160804749, + 0.18962346017360687, + 2.0, + 0.1830430030822754, + 0.15030252933502197, + 0.12050267308950424, + 0.17022761702537537, + 0.11389490216970444, + 0.07058519124984741, + 0.07076358795166016, + 0.18658748269081116 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 2.0962133407592773, + 0.057441189885139465, + -0.2733909487724304, + 0.4170292317867279, + -0.7772165536880493, + 2.858414888381958, + 2.1098427772521973, + 1.8366810083389282, + 0.017507493495941162, + -1.6127345561981201, + -0.39152991771698, + -1.017486333847046, + -2.815659999847412, + -2.787520408630371, + 1.6748408079147339, + 0.2986273169517517, + -0.6001855134963989, + 0.5475705862045288, + -0.28012242913246155, + -1.5382447242736816, + -1.6652274131774902 + ], + [ + 2.1338062286376953, + -0.07656055688858032, + -0.7531881332397461, + 0.48830646276474, + -0.36628106236457825, + 2.381558418273926, + 1.7315943241119385, + 1.838928461074829, + -0.050897032022476196, + -1.171897292137146, + -0.8350631594657898, + -0.8261303901672363, + -2.2635834217071533, + -3.2190167903900146, + 1.3416705131530762, + 0.44279617071151733, + -0.7299901247024536, + 1.2929750680923462, + -0.0032969415187835693, + -1.5000369548797607, + -1.6060972213745117 + ], + [ + 1.5521352291107178, + 0.03659695386886597, + -0.44382959604263306, + 0.21355122327804565, + -1.0471938848495483, + 3.194620370864868, + 1.9892739057540894, + 2.5084128379821777, + 0.531404972076416, + -0.9168282747268677, + -0.49052953720092773, + -1.1588059663772583, + -3.156189441680908, + -3.278008460998535, + 0.921509325504303, + 0.49181532859802246, + -1.0228296518325806, + 0.5866082310676575, + -0.4882396459579468, + -0.7946261763572693, + -2.119061231613159 + ], + [ + 2.7138938903808594, + -0.3406146764755249, + -0.32174432277679443, + 0.9962238669395447, + -0.558576226234436, + 2.049264907836914, + 1.8703808784484863, + 2.113438367843628, + 0.9615883827209473, + -1.9615708589553833, + -0.0062177181243896484, + -0.5808464288711548, + -2.1758205890655518, + -2.219890594482422, + 1.022050142288208, + 0.7201904654502869, + -1.3177021741867065, + 0.4089192748069763, + -0.0315951406955719, + -0.7215378880500793, + -2.069262981414795 + ], + [ + 2.5701913833618164, + 0.3137266933917999, + -0.5592924356460571, + 0.0026398897171020508, + -0.19283491373062134, + 2.2903807163238525, + 2.637770652770996, + 2.5494186878204346, + 0.9258015155792236, + -0.7146614193916321, + -0.686233639717102, + -0.7985042929649353, + -3.431988000869751, + -3.2178471088409424, + 1.319100260734558, + -0.15109702944755554, + -0.2694774866104126, + 1.0912322998046875, + -0.6380462050437927, + -2.0780839920043945, + -2.1209774017333984 + ], + [ + 2.6195321083068848, + -0.2646539509296417, + 0.15159690380096436, + 0.8749752044677734, + -1.1848887205123901, + 2.8789353370666504, + 1.6037814617156982, + 1.3343384265899658, + -0.2254078984260559, + -2.041776657104492, + -0.9811336994171143, + -1.3441768884658813, + -2.2502636909484863, + -1.9091612100601196, + 0.9959704279899597, + 0.42473238706588745, + -0.5330828428268433, + 1.6945891380310059, + 0.4069111943244934, + -0.5568764805793762, + -1.472819447517395 + ], + [ + 2.9485361576080322, + -0.6907311677932739, + -0.057036399841308594, + 1.2457408905029297, + 0.09488356113433838, + 2.6509125232696533, + 2.0378005504608154, + 2.278454303741455, + 1.1734973192214966, + -1.853393316268921, + 0.09614759683609009, + -0.35674989223480225, + -2.1597909927368164, + -1.9672894477844238, + 1.9760816097259521, + -0.47744297981262207, + 0.021782994270324707, + 1.6328153610229492, + -0.8569514751434326, + -1.0814869403839111, + -1.7780723571777344 + ], + [ + 2.1897926330566406, + -0.3883178234100342, + -0.6492674946784973, + 1.3223958015441895, + 0.30954307317733765, + 2.031817674636841, + 1.878535270690918, + 2.5030624866485596, + 0.6644855737686157, + -1.7382179498672485, + -1.1862120628356934, + -0.7967990636825562, + -2.8425841331481934, + -2.3635308742523193, + 2.0061349868774414, + -0.6015456318855286, + -1.429253339767456, + 1.2773160934448242, + 0.04122643172740936, + -2.0497689247131348, + -1.1791472434997559 + ], + [ + 1.586853265762329, + 0.06395213305950165, + -0.8944701552391052, + 0.5118889808654785, + -1.1310638189315796, + 2.865907669067383, + 1.3295278549194336, + 1.0862370729446411, + 0.6303088665008545, + -0.3754369616508484, + -0.04129624366760254, + -0.7044278979301453, + -2.923245429992676, + -1.7837438583374023, + 1.5932514667510986, + 1.2467985153198242, + 0.13401156663894653, + 1.6706414222717285, + -0.816297709941864, + -1.2893165349960327, + -1.0985345840454102 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.1383795738220215, + -0.1483764797449112, + -0.27178800106048584, + 0.26738643646240234, + -0.12845781445503235, + 1.6495440006256104, + 0.0, + 0.9713280200958252, + -0.050046056509017944, + -0.36543774604797363, + -0.022124826908111572, + 0.08494043350219727, + -1.671510934829712, + 0.0, + 0.5271386504173279, + 0.3898613452911377, + 0.21280205249786377, + -0.2029280662536621, + -0.01802508533000946, + -0.35994601249694824, + 0.0 + ], + [ + 1.7570452690124512, + 0.07137972116470337, + 0.02318662405014038, + 0.2907020151615143, + -0.2308352291584015, + 2.89892578125, + 2.537220001220703, + 2.238605499267578, + 0.16207315027713776, + -1.6996564865112305, + -0.8050198554992676, + -1.2014062404632568, + -2.5764517784118652, + -2.393839120864868, + 1.1276613473892212, + 0.31928181648254395, + -1.036102533340454, + 1.1266506910324097, + -0.1944657564163208, + -1.5998979806900024, + -2.151465892791748 + ] + ], + "predicted_scores": [ + 7.231818675994873, + 6.486433506011963, + 6.4527907371521, + 6.395447731018066, + 6.298037052154541, + 6.243171215057373, + 5.761966705322266, + 4.903831958770752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train0", + "selected_index": 0, + "state_hash": "0b62c42c3beda0eb959692d55792514223681b522edd0a5e73247682cf0bfe95", + "stored_base_utility": 0.2651830017566681, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "generated_tangents": [ + [ + 1.9789299964904785, + 0.18482418358325958, + -1.461500644683838, + -0.10560663789510727, + -0.481938898563385, + -0.6769506931304932, + -0.8734853863716125, + 0.45116496086120605, + 0.2704058289527893, + -0.25655752420425415, + -0.3241361677646637, + 0.23260793089866638, + -3.059511661529541, + -2.303499698638916, + 0.5132331848144531, + 0.14198794960975647, + 0.6234413385391235, + -1.2580815553665161, + 0.6944096684455872, + -2.3318138122558594, + -1.224258542060852 + ], + [ + 2.0985231399536133, + -0.11934754997491837, + -1.388933777809143, + 0.053231775760650635, + -0.6105248928070068, + -0.7982304096221924, + -0.7380546927452087, + 0.559706449508667, + 0.21137775480747223, + -0.22078683972358704, + -0.41030898690223694, + 0.47239795327186584, + -3.256456136703491, + -2.5135691165924072, + 0.6467328071594238, + 0.25664636492729187, + 0.3035256266593933, + -1.3714854717254639, + 0.9082530736923218, + -2.3454537391662598, + -1.850858449935913 + ], + [ + 1.9810668230056763, + 0.16683797538280487, + -1.8061074018478394, + -0.3732788860797882, + -0.0841829776763916, + -0.29402515292167664, + -1.1140425205230713, + 0.6322353482246399, + -0.005932627245783806, + -0.46816450357437134, + -0.08492376655340195, + 0.6054893136024475, + -2.676499366760254, + -2.7999207973480225, + 0.6312494277954102, + 0.30878767371177673, + 0.614536464214325, + -1.368439793586731, + 0.8445847630500793, + -2.334414482116699, + -1.2858355045318604 + ], + [ + 1.9530315399169922, + 0.15125535428524017, + -1.6527477502822876, + -0.6363152861595154, + -0.15198585391044617, + -0.07052426040172577, + -1.2695701122283936, + 0.8097606897354126, + -0.09663358330726624, + -0.3970586061477661, + -0.47795388102531433, + 0.5740829706192017, + -3.153839349746704, + -2.915522813796997, + 0.7115260362625122, + 0.4020631015300751, + 0.5967467427253723, + -1.4221256971359253, + 0.8709081411361694, + -2.372349739074707, + -1.5803728103637695 + ], + [ + 1.9482799768447876, + 0.11157537996768951, + -1.57022225856781, + -0.5205692052841187, + -0.11835727095603943, + -0.28709328174591064, + -1.3624851703643799, + 0.7494869232177734, + -0.05077909678220749, + -0.41678833961486816, + -0.28302374482154846, + 0.44418662786483765, + -2.9330546855926514, + -2.786648750305176, + 0.6355831623077393, + 0.4074035882949829, + 0.6077445149421692, + -1.4902455806732178, + 0.7996195554733276, + -2.435551881790161, + -1.4435468912124634 + ], + [ + 1.7593673467636108, + -0.1820324957370758, + -1.3708407878875732, + -0.11217817664146423, + 0.006536126136779785, + -0.5965646505355835, + -1.046247959136963, + 0.7461029291152954, + 0.024881847202777863, + -0.20207908749580383, + -0.24686360359191895, + 0.6631032228469849, + -3.025989294052124, + -1.7664202451705933, + 0.5276257991790771, + 0.3083820939064026, + 0.9129725098609924, + -1.3154927492141724, + 1.2958452701568604, + -2.1096575260162354, + -0.78299880027771 + ], + [ + 2.1856632232666016, + 0.0016049854457378387, + -1.6249693632125854, + -0.5716937780380249, + -0.16711804270744324, + 0.08690698444843292, + -1.261176347732544, + 0.9117177724838257, + 0.002620358020067215, + -0.3168095648288727, + -0.3148176670074463, + 0.5934818983078003, + -2.957610845565796, + -2.7233242988586426, + 0.7135811448097229, + 0.35289466381073, + 0.5558337569236755, + -1.6773006916046143, + 0.8219923973083496, + -2.949918508529663, + -1.2962119579315186 + ], + [ + 1.944191575050354, + 0.09757903218269348, + -1.4973571300506592, + -0.5029289722442627, + -0.022460639476776123, + -0.5843757390975952, + -1.260545015335083, + 0.872179388999939, + -0.11007992923259735, + -0.3647612929344177, + -0.3585472106933594, + 0.22520257532596588, + -3.1110293865203857, + -2.633798599243164, + 0.6089034676551819, + 0.46097666025161743, + 0.7376998662948608, + -1.6006757020950317, + 0.7458087205886841, + -2.6057984828948975, + -1.4574354887008667 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.149173304438591, + 0.06221481412649155, + 0.1385849118232727, + 0.06656398624181747, + 2.0, + 0.10397814214229584, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7398372888565063, + -0.02836926281452179, + -0.4292973279953003, + 0.028730571269989014, + 0.009982943534851074, + -0.3464317321777344, + 0.0, + 0.009800434112548828, + -0.06816069781780243, + 0.030653484165668488, + -0.00554502010345459, + -0.06216755136847496, + -1.9802618026733398, + 0.0, + 0.01717478036880493, + -0.05601567029953003, + 0.010390102863311768, + -0.8072932958602905, + -0.05775028467178345, + -1.9342114925384521, + 0.0 + ], + [ + 0.7398372888565063, + -0.1961192637681961, + -0.4292973279953003, + -0.0017886161804199219, + 0.06461763381958008, + -0.3464317321777344, + 0.0, + 0.42972221970558167, + -0.5797654986381531, + 0.19436834752559662, + -0.23844504356384277, + 0.1733923852443695, + -1.9802618026733398, + 0.0, + -0.27269166707992554, + 0.04145494103431702, + 0.12319430708885193, + -0.8072932958602905, + 0.4949256181716919, + -1.9342114925384521, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7398372888565063, + 0.2242223471403122, + -1.4292973279953003, + -0.537040650844574, + -0.9353823661804199, + -1.3464317321777344, + -2.159137487411499, + 0.489885151386261, + -0.006211067084223032, + 0.09238272905349731, + -1.0055450201034546, + -0.04778099060058594, + -2.98026180267334, + -2.576570987701416, + 0.2992324233055115, + 0.34874510765075684, + 0.5165960192680359, + -1.8072932958602905, + 0.834364652633667, + -2.934211492538452, + -2.0790040493011475 + ], + [ + 1.9740948677062988, + 0.4187958538532257, + -1.2225801944732666, + -0.6112513542175293, + -1.0962133407592773, + -1.4875872135162354, + -2.1883857250213623, + 0.6603437662124634, + -0.20142723619937897, + 0.1697872281074524, + -0.8377580046653748, + -0.16444604098796844, + -2.87237286567688, + -2.4153482913970947, + 0.03898555040359497, + 0.5175161957740784, + 0.7254846096038818, + -1.9598647356033325, + 1.1053451299667358, + -3.172945976257324, + -1.9334450960159302 + ], + [ + 1.5938913822174072, + -0.018363401293754578, + -1.1851862668991089, + -0.20344951748847961, + -0.9226735234260559, + -1.3553601503372192, + -1.932102084159851, + 0.6446930170059204, + 0.30503973364830017, + 0.3522501289844513, + -0.9527953267097473, + -0.15980565547943115, + -2.9574379920959473, + -2.9056527614593506, + 0.6069583892822266, + 0.3722725808620453, + 0.7255810499191284, + -1.8503519296646118, + 0.5613387823104858, + -3.095244884490967, + -2.023747444152832 + ], + [ + 1.6626678705215454, + -0.02834661304950714, + -1.0125243663787842, + -0.5629516839981079, + -1.230878233909607, + -1.414373755455017, + -2.42856764793396, + 0.8520264625549316, + -0.1296539008617401, + -0.22410845756530762, + -1.0539721250534058, + -0.060277245938777924, + -2.740527391433716, + -2.1194698810577393, + 0.6826534867286682, + 0.7535779476165771, + 0.35573869943618774, + -1.9124289751052856, + 0.7101565003395081, + -3.326658010482788, + -1.8195855617523193 + ], + [ + 2.2881288528442383, + 0.6951426863670349, + -1.0978585481643677, + -0.008003771305084229, + -1.0754073858261108, + -1.7734596729278564, + -2.139570474624634, + 0.9205271601676941, + 0.29560548067092896, + 0.49158430099487305, + -1.2606573104858398, + 0.43956148624420166, + -2.831833600997925, + -2.960129976272583, + 0.6832801103591919, + -0.03894898295402527, + 1.036036491394043, + -2.127277135848999, + 0.6759494543075562, + -3.1690073013305664, + -1.9895020723342896 + ], + [ + 1.9020261764526367, + 0.6575945019721985, + -1.3994193077087402, + -0.5040701031684875, + -0.6857678890228271, + -0.7056208252906799, + -1.5344107151031494, + 0.7615087032318115, + 0.6105715036392212, + 0.19082021713256836, + -0.6303516626358032, + -0.4614318609237671, + -2.3375303745269775, + -2.188478469848633, + 0.6886799335479736, + -0.03237125277519226, + 0.483882874250412, + -1.3982734680175781, + 0.23996800184249878, + -3.3969180583953857, + -1.5116117000579834 + ], + [ + 1.666227102279663, + 0.6915197968482971, + -1.6392340660095215, + -0.9650112390518188, + -0.2591104507446289, + -1.5417896509170532, + -1.695828914642334, + 1.130251169204712, + -0.555443525314331, + -0.4872768521308899, + -1.0994479656219482, + 0.5934816002845764, + -3.5218958854675293, + -1.8239338397979736, + 0.7150859832763672, + 0.8003170490264893, + 1.0113730430603027, + -2.1905770301818848, + 0.8595032095909119, + -3.039506673812866, + -2.3675880432128906 + ], + [ + 1.59856116771698, + -0.036071136593818665, + -1.700652837753296, + 0.27602267265319824, + -1.608276128768921, + -0.875938355922699, + -1.6728798151016235, + 0.8851535320281982, + 0.7461313605308533, + -0.7497223019599915, + -1.0928030014038086, + -0.4735495150089264, + -2.223254680633545, + -2.5322465896606445, + 0.044906944036483765, + 0.6160990595817566, + 1.3697642087936401, + -1.6367510557174683, + 0.086475670337677, + -3.323115825653076, + -2.058189630508423 + ], + [ + 2.285855531692505, + 0.6314746141433716, + -1.2119637727737427, + 0.039707839488983154, + -0.9751565456390381, + -1.7796330451965332, + -1.5261995792388916, + 0.5450726747512817, + -0.8933275938034058, + 0.7903657555580139, + -1.5466060638427734, + 0.5994364023208618, + -3.370340585708618, + -3.1641077995300293, + 0.3225497603416443, + -0.39524197578430176, + 1.152822732925415, + -1.508494257926941, + 1.1221402883529663, + -3.361729860305786, + -1.9251339435577393 + ], + [ + 2.2599079608917236, + -0.005553290247917175, + -2.4053664207458496, + 0.39961689710617065, + -0.40980857610702515, + -0.9905421733856201, + -1.7244036197662354, + -0.06417369842529297, + 0.6376181840896606, + 0.5168956518173218, + -1.6740490198135376, + -0.1833474189043045, + -3.9368975162506104, + -1.7146861553192139, + 0.9366075992584229, + 0.701168417930603, + -0.23131346702575684, + -2.740941286087036, + 0.908729076385498, + -3.6231396198272705, + -1.3161351680755615 + ], + [ + 1.819408655166626, + -0.5504165291786194, + -1.302573800086975, + -0.6927691698074341, + -0.7548186779022217, + -2.1824703216552734, + -2.4921669960021973, + 0.6337187886238098, + 0.1691494882106781, + 0.7547447681427002, + -0.4371253252029419, + -0.6562469601631165, + -2.313655376434326, + -2.4974589347839355, + 0.05105690658092499, + 0.3473445773124695, + -0.4244192838668823, + -1.8233305215835571, + -0.1157217025756836, + -2.0408761501312256, + -2.2010600566864014 + ] + ], + "predicted_scores": [ + 7.137040615081787, + 6.98091983795166, + 6.582064628601074, + 5.941701412200928, + 5.787785053253174, + 5.627976894378662, + 5.44954252243042, + 5.1042938232421875 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2054899126/train0", + "selected_index": 0, + "state_hash": "b26c5f64bc7e877098323281f2b58e7850275b30405bc411ed7ffc7cf69f4e26", + "stored_base_utility": 0.149173304438591, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.037537217140197754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037537217140197754 + }, + "base_progress": 0.037537217140197754, + "base_success": false, + "base_utility": 0.037537217140197754, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.004155457019805908, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004155457019805908 + }, + { + "contact_quality": null, + "progress": 0.003685553791001439, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003685553791001439 + }, + { + "contact_quality": null, + "progress": 0.004423340316861868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004423340316861868 + }, + { + "contact_quality": null, + "progress": 0.005550503730773926, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005550503730773926 + }, + { + "contact_quality": null, + "progress": 0.005441188812255859, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005441188812255859 + }, + { + "contact_quality": null, + "progress": 0.00635818624868989, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00635818624868989 + }, + { + "contact_quality": null, + "progress": 0.007275819778442383, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007275819778442383 + }, + { + "contact_quality": null, + "progress": 0.004515886306762695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004515886306762695 + } + ], + "candidate_progress": [ + 0.004155457019805908, + 0.003685553791001439, + 0.004423340316861868, + 0.005550503730773926, + 0.005441188812255859, + 0.00635818624868989, + 0.007275819778442383, + 0.004515886306762695 + ], + "candidate_source_chart_ids": [ + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-6f26118aa94bacd82250" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c832439976a37f52d094", + "generated_tangents": [ + [ + -0.29347485303878784, + -0.007099565118551254, + 0.12988796830177307, + 0.19824010133743286, + -0.018882334232330322, + 0.2551068961620331, + -0.6858754754066467, + 1.5179896354675293, + -0.3161171078681946, + -0.8263975977897644, + -1.5724046230316162, + 1.4986058473587036, + -1.329538106918335, + -2.121022939682007, + 0.14946617186069489, + 0.22442542016506195, + 0.04264265298843384, + 0.0784381628036499, + 0.5420059561729431, + 2.1648428440093994, + 0.6930601596832275 + ], + [ + -0.0771825760602951, + 0.1640229970216751, + 0.20471450686454773, + 0.5147385597229004, + -0.1493327021598816, + -0.21187573671340942, + -0.5549275279045105, + 1.415863037109375, + -0.30152058601379395, + -0.8002134561538696, + -1.7488105297088623, + 1.2112573385238647, + -1.4759513139724731, + -1.9330061674118042, + 0.21658742427825928, + 0.1840268075466156, + -0.12504936754703522, + -0.024263814091682434, + 0.45800065994262695, + 1.1132078170776367, + 0.2909638583660126 + ], + [ + -0.12291647493839264, + 0.16723252832889557, + 0.301577091217041, + 0.6293084621429443, + 0.07272940874099731, + -0.11016964912414551, + -0.4484826326370239, + 1.4451688528060913, + -0.3433241546154022, + -0.7893127799034119, + -1.6048904657363892, + 1.4750607013702393, + -1.3179724216461182, + -1.5939264297485352, + 0.16794894635677338, + 0.19196021556854248, + -0.14914895594120026, + -0.0010900944471359253, + 0.3348276615142822, + 1.2292795181274414, + 0.34833234548568726 + ], + [ + -0.2835384011268616, + 0.05340753495693207, + -0.14205411076545715, + 0.43108847737312317, + 0.12497077882289886, + 0.537225604057312, + 0.06485310196876526, + 1.2107707262039185, + -0.4080606997013092, + -0.5053415894508362, + -1.3102506399154663, + 1.2501064538955688, + -1.1867755651474, + -1.7842912673950195, + 0.3090161085128784, + 0.10927611589431763, + 0.10385721921920776, + -0.042656123638153076, + 0.3441688120365143, + 1.5987038612365723, + 0.43927308917045593 + ], + [ + -0.46174389123916626, + 0.07585708796977997, + -0.09058967232704163, + 0.45497867465019226, + 0.07016071677207947, + 0.3548803925514221, + -0.4941389560699463, + 1.3059459924697876, + -0.500140368938446, + -0.5840087532997131, + -1.5642633438110352, + 1.3629822731018066, + -1.434400200843811, + -2.0762150287628174, + 0.2914178669452667, + 0.13851869106292725, + 0.0022342801094055176, + -0.01099364459514618, + 0.5860754251480103, + 1.1163792610168457, + 0.6413156986236572 + ], + [ + -0.4503415822982788, + 0.11110465228557587, + -0.03892597556114197, + 0.3356201946735382, + 0.12225022912025452, + 0.2972986698150635, + -0.30001384019851685, + 1.2160446643829346, + -0.461029589176178, + -0.46410125494003296, + -1.5512192249298096, + 1.3206120729446411, + -1.3483890295028687, + -1.9572304487228394, + 0.35477444529533386, + 0.1161455512046814, + 0.19328224658966064, + -0.057489290833473206, + 0.43606698513031006, + 1.4491631984710693, + 0.534327507019043 + ], + [ + -0.5631589889526367, + 0.02370622009038925, + 0.020105421543121338, + 0.538189172744751, + -0.03366956114768982, + 0.1482982486486435, + -0.5607455372810364, + 1.1600546836853027, + -0.3802933692932129, + -0.6177942156791687, + -1.509847640991211, + 1.4257028102874756, + -1.2602781057357788, + -1.923789620399475, + 0.2732735872268677, + 0.08432956039905548, + 0.00864499807357788, + -0.051631078124046326, + 0.490532249212265, + 1.2050237655639648, + 0.6564277410507202 + ], + [ + 0.09293289482593536, + 0.042575716972351074, + -0.11180070042610168, + 0.5683645009994507, + -0.07158035039901733, + -0.35383954644203186, + 0.19918754696846008, + 1.123085618019104, + -0.18470652401447296, + -0.3706522583961487, + -1.3804121017456055, + 0.8163624405860901, + -1.9296634197235107, + -1.5383012294769287, + 0.4167742431163788, + -0.036186885088682175, + -0.03164835274219513, + 0.019610613584518433, + 0.2125152349472046, + 1.6182937622070312, + 0.4322349429130554 + ] + ], + "generated_utilities": [ + 0.004155457019805908, + 0.003685553791001439, + 0.004423340316861868, + 0.005550503730773926, + 0.005441188812255859, + 0.00635818624868989, + 0.007275819778442383, + 0.004515886306762695 + ], + "group_id": "ms-c832439976a37f52d094", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0808674544095993, + 0.13129448890686035, + 0.11199682950973511, + 0.2594742774963379, + 0.17684690654277802, + 0.11511725187301636, + 0.17100229859352112, + 0.12906593084335327, + 0.12404869496822357, + 0.10653237998485565, + 0.15492436289787292, + 0.05800905078649521, + 0.04767048358917236, + 0.09533219039440155, + 0.03300365060567856, + 0.07704520225524902 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025296777486801147, + 0.11629299819469452, + -0.024434387683868408, + 1.0812468528747559, + 0.04022188484668732, + -0.01547396183013916, + 0.0, + 0.05167186260223389, + 0.07629208266735077, + -0.00870610773563385, + -0.5777420997619629, + 0.07687681913375854, + -0.6336902379989624, + 0.0, + -0.043449461460113525, + -0.029798656702041626, + 0.18944299221038818, + 0.03022998571395874, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + 0.05685463547706604, + -0.3914306163787842, + 0.4578048586845398, + 1.0812468528747559, + 0.10990457236766815, + -0.18159747123718262, + 0.0, + 0.05167186260223389, + -0.5002377033233643, + 0.04299694299697876, + -0.5777420997619629, + 0.02747189998626709, + -0.6336902379989624, + 0.0, + -0.08638033270835876, + 0.17561247944831848, + 0.37304359674453735, + 0.2538231611251831, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + -0.3902576267719269, + 0.03852551057934761, + 0.16114424169063568, + 2.081246852874756, + 0.22516529262065887, + 0.8184025287628174, + 0.3388257324695587, + 1.0516718626022339, + -0.061541248112916946, + 0.22279328107833862, + -1.577742099761963, + 1.027471899986267, + -1.6336902379989624, + -1.6284003257751465, + -0.24836044013500214, + -0.3427097499370575, + 1.1894429922103882, + -0.15580247342586517, + 1.7312084436416626, + 2.8623785972595215, + 1.0248357057571411 + ], + [ + -0.5903857946395874, + -0.10628272593021393, + -0.07594148814678192, + 2.194340944290161, + 0.11632551997900009, + 0.942036509513855, + 0.06351110339164734, + 0.7966359853744507, + 0.1702205240726471, + 0.1549067199230194, + -1.439386248588562, + 1.3076335191726685, + -1.3458164930343628, + -1.90281081199646, + -0.2967699468135834, + -0.27783676981925964, + 1.0556988716125488, + 0.06510581076145172, + 1.6170521974563599, + 3.027111530303955, + 1.154527187347412 + ], + [ + -0.7559880614280701, + 0.694898247718811, + -0.3551136255264282, + 1.8615511655807495, + 0.9249891042709351, + 0.969862699508667, + 0.4452719986438751, + 1.5801937580108643, + 0.13303470611572266, + -0.13816028833389282, + -1.2399333715438843, + 1.5769362449645996, + -1.5071980953216553, + -2.0578060150146484, + -0.8449342846870422, + 0.3327537477016449, + 1.6781622171401978, + 0.4016150236129761, + 1.8172147274017334, + 3.1609320640563965, + 1.4916398525238037 + ] + ], + "predicted_scores": [ + 7.05134916305542, + 6.958633899688721, + 6.76113748550415, + 6.500876426696777, + 6.25551700592041, + 6.220318794250488, + 6.049957752227783, + 5.959582805633545 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train0", + "selected_index": 0, + "state_hash": "0f2810f22383259329f6e94fdc8384176c664f2ed476924e59db7cda4f4a4863", + "stored_base_utility": 0.0808674544095993, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09138330072164536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09138330072164536 + }, + "base_progress": 0.09138330072164536, + "base_success": false, + "base_utility": 0.09138330072164536, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.012376229278743267, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012376229278743267 + }, + { + "contact_quality": null, + "progress": 0.006582856178283691, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006582856178283691 + }, + { + "contact_quality": null, + "progress": 0.005475322250276804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005475322250276804 + }, + { + "contact_quality": null, + "progress": 0.015102227218449116, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015102227218449116 + }, + { + "contact_quality": null, + "progress": 0.007560948375612497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007560948375612497 + }, + { + "contact_quality": null, + "progress": 0.010635594837367535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010635594837367535 + }, + { + "contact_quality": null, + "progress": 0.0225119199603796, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0225119199603796 + }, + { + "contact_quality": null, + "progress": 0.003913839813321829, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003913839813321829 + } + ], + "candidate_progress": [ + 0.012376229278743267, + 0.006582856178283691, + 0.005475322250276804, + 0.015102227218449116, + 0.007560948375612497, + 0.010635594837367535, + 0.0225119199603796, + 0.003913839813321829 + ], + "candidate_source_chart_ids": [ + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c96b415f083f60857334", + "generated_tangents": [ + [ + -0.2556810975074768, + 0.1454085111618042, + 0.9927430152893066, + 0.615959882736206, + 1.1354053020477295, + 1.9658490419387817, + 1.965430498123169, + -0.10682886838912964, + -0.3494552969932556, + 0.8687063455581665, + 0.34482988715171814, + 0.40220969915390015, + 1.392223834991455, + 1.1267098188400269, + -0.06866274774074554, + -0.06443829834461212, + 0.03723445534706116, + 1.259427785873413, + -0.4557071924209595, + 0.15480095148086548, + -0.4620818793773651 + ], + [ + -0.5475700497627258, + 0.12537738680839539, + 1.3051954507827759, + 0.2073497474193573, + 1.4401142597198486, + 1.6757891178131104, + 1.8697736263275146, + -0.32841405272483826, + -0.39485877752304077, + 0.9930245280265808, + -0.22751465439796448, + 0.3758634924888611, + 2.1365277767181396, + 1.4237288236618042, + -0.37164661288261414, + -0.445719838142395, + -0.14464490115642548, + 1.3406617641448975, + 0.010874032974243164, + 0.2400699257850647, + -0.7034790515899658 + ], + [ + -0.3907206058502197, + 0.0502595528960228, + 0.9871405959129333, + 0.31128743290901184, + 1.4287807941436768, + 1.6949974298477173, + 1.7582051753997803, + -0.25582271814346313, + -0.34462422132492065, + 1.0377522706985474, + -0.09380679577589035, + 0.47910740971565247, + 1.8251665830612183, + 1.2638260126113892, + -0.3799847364425659, + -0.4535689055919647, + 0.02676314115524292, + 1.3922412395477295, + 0.07411304116249084, + 0.33839088678359985, + -0.6817395687103271 + ], + [ + -0.24493928253650665, + 0.19693492352962494, + 0.9518611431121826, + 0.5513356924057007, + 1.1608537435531616, + 2.050548553466797, + 1.889594554901123, + -0.17655092477798462, + -0.27446961402893066, + 0.7984658479690552, + 0.33503103256225586, + 0.4188429117202759, + 1.4712934494018555, + 1.1284092664718628, + -0.00196128711104393, + -0.17179828882217407, + -0.049263134598731995, + 1.2499834299087524, + -0.42109644412994385, + 0.1783105432987213, + -0.18764570355415344 + ], + [ + -0.3054867386817932, + 0.13826729357242584, + 1.0732299089431763, + 0.330486923456192, + 1.3794970512390137, + 1.8489264249801636, + 1.951803207397461, + -0.13306602835655212, + -0.3248480558395386, + 1.088460922241211, + 0.05854111164808273, + 0.5211734771728516, + 1.756280541419983, + 1.2685545682907104, + -0.2445927858352661, + -0.32804498076438904, + 0.06367179751396179, + 1.267756700515747, + 0.09229117631912231, + 0.22304201126098633, + -0.7972757816314697 + ], + [ + -0.3916054368019104, + 0.13326752185821533, + 1.1780461072921753, + 0.06310266256332397, + 1.5708105564117432, + 1.890318512916565, + 2.242936849594116, + -0.04418173432350159, + -0.3064570426940918, + 1.042787790298462, + 0.12111161649227142, + 0.6410789489746094, + 1.734998106956482, + 1.1888160705566406, + -0.15793560445308685, + -0.39206135272979736, + -0.12200386822223663, + 1.140578269958496, + 0.08651520311832428, + 0.050670623779296875, + -0.5860601663589478 + ], + [ + -0.5884166359901428, + 0.06495284289121628, + 0.22037458419799805, + -0.3189915716648102, + 0.9872580766677856, + 1.8330312967300415, + 1.990811824798584, + 0.1546839475631714, + -0.24982738494873047, + 1.0657873153686523, + 0.23279926180839539, + 0.9398138523101807, + 1.384837031364441, + 1.351023554801941, + 0.02624328061938286, + -0.3241584300994873, + -0.5798515677452087, + 1.010702133178711, + 0.10710541903972626, + 0.38128888607025146, + -0.37607601284980774 + ], + [ + -0.749691903591156, + -0.2506883442401886, + 0.6185764074325562, + -0.19299718737602234, + 1.258925199508667, + 2.0286459922790527, + 1.6187307834625244, + -0.08473244309425354, + -0.4074743092060089, + 0.8556436896324158, + 0.5780456066131592, + 0.38366544246673584, + 1.8112229108810425, + 2.0794122219085693, + -0.5160185098648071, + -0.6219602227210999, + -0.17286916077136993, + 1.4655016660690308, + -0.12483066320419312, + 0.32146573066711426, + -0.16379857063293457 + ] + ], + "generated_utilities": [ + 0.012376229278743267, + 0.006582856178283691, + 0.005475322250276804, + 0.015102227218449116, + 0.007560948375612497, + 0.010635594837367535, + 0.0225119199603796, + 0.003913839813321829 + ], + "group_id": "ms-c96b415f083f60857334", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15946325659751892, + 0.16313008964061737, + 0.2226802557706833, + 0.18744397163391113, + 0.21545825898647308, + 0.15985511243343353, + 0.16218391060829163, + 0.26426148414611816, + 0.184181809425354, + 0.24352896213531494, + 0.14179718494415283, + 0.1026589497923851, + 0.1274908483028412, + 0.05275428295135498, + 0.08544860780239105, + 0.07143914699554443 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.16191565990448, + 0.4561867117881775, + 1.1241415739059448, + 0.08947351574897766, + 0.32140541076660156, + 3.6895532608032227, + 3.6284518241882324, + -1.8666775226593018, + -0.22328677773475647, + 1.48662531375885, + 1.4737720489501953, + 1.35750150680542, + 2.507124423980713, + 1.9315069913864136, + -0.8371121883392334, + 0.5339290499687195, + 1.0061794519424438, + 2.149812698364258, + 0.08911214768886566, + 0.17410686612129211, + 0.3549031913280487 + ], + [ + -0.14804881811141968, + -0.5333454608917236, + 0.7481307983398438, + 0.22411420941352844, + 1.7204210758209229, + 3.908766746520996, + 3.3698034286499023, + -1.6747719049453735, + 0.2769082486629486, + 2.042428970336914, + 0.44748249650001526, + 1.9492043256759644, + 2.918121576309204, + 3.2413060665130615, + -0.6994609832763672, + -0.3854917883872986, + -0.6702244281768799, + 2.408869504928589, + -0.5672509670257568, + 0.5941196084022522, + 0.46011871099472046 + ], + [ + -0.46204906702041626, + -0.4236203730106354, + 1.8582539558410645, + 0.8416532278060913, + 1.2386386394500732, + 3.7320151329040527, + 2.242549180984497, + -1.8652616739273071, + -1.0259326696395874, + 1.3383660316467285, + 1.7068204879760742, + 0.7941814661026001, + 3.3389508724212646, + 2.7797679901123047, + -0.9363516569137573, + 0.17812880873680115, + 0.6111975908279419, + 2.066016435623169, + -0.28941091895103455, + 1.501695990562439, + -0.35273995995521545 + ], + [ + -0.7471081614494324, + -0.029650092124938965, + 1.019482135772705, + -0.30564722418785095, + 1.380143642425537, + 3.247985601425171, + 3.0548973083496094, + -0.5582134127616882, + -0.920264482498169, + 2.3859832286834717, + 1.1080355644226074, + 1.5716153383255005, + 3.089156150817871, + 2.043484687805176, + 0.1913408637046814, + -0.7557333707809448, + -0.6297664642333984, + 1.7829738855361938, + -0.6967166662216187, + -0.10757219791412354, + 0.229112908244133 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027880847454071045, + -0.2692106068134308, + 0.12925803661346436, + -0.1745016872882843, + -0.1231425404548645, + 1.9967715740203857, + 0.0, + -0.1433020830154419, + 0.2667914927005768, + 0.8867976665496826, + -0.057856738567352295, + 0.5789351463317871, + 1.7831475734710693, + 0.0, + -0.35565105080604553, + 0.293251097202301, + 0.012226536870002747, + 1.121243953704834, + -0.15060651302337646, + 0.03409510850906372, + 0.0 + ], + [ + -0.7738120555877686, + 0.08056297898292542, + 1.1292580366134644, + 0.4734484851360321, + 0.8768574595451355, + 2.9967715740203857, + 3.0509116649627686, + -1.143302083015442, + -0.1310054361820221, + 1.8867976665496826, + 0.9421432614326477, + 1.578935146331787, + 2.7831475734710693, + 2.6037895679473877, + -0.5597631931304932, + 0.11406822502613068, + 0.2593899667263031, + 2.121243953704834, + -0.1791832000017166, + 0.5773167610168457, + 0.41381677985191345 + ], + [ + -0.6024693250656128, + 0.29007619619369507, + 1.3939473628997803, + 0.3474765419960022, + 1.1848478317260742, + 3.127882719039917, + 3.3329083919525146, + -1.2855982780456543, + 0.20972535014152527, + 1.8329110145568848, + 1.267270803451538, + 1.709198236465454, + 2.927372455596924, + 2.9659509658813477, + -0.7555317282676697, + 0.015305742621421814, + 0.6009905338287354, + 2.188507556915283, + 0.07961152493953705, + 0.4547080397605896, + 0.20378917455673218 + ], + [ + -1.2702982425689697, + 0.06162632256746292, + 0.9447290897369385, + 0.837199330329895, + 0.5848468542098999, + 3.4801836013793945, + 2.8138537406921387, + -1.5083554983139038, + -0.45768383145332336, + 2.08561110496521, + 1.4951050281524658, + 1.8484808206558228, + 2.5617129802703857, + 2.942486047744751, + -0.562404215335846, + 0.17441512644290924, + 0.7880480289459229, + 1.8807158470153809, + -0.043379828333854675, + 0.39090824127197266, + 0.43096962571144104 + ] + ], + "predicted_scores": [ + 9.991924285888672, + 9.819470405578613, + 9.650274276733398, + 9.505208969116211, + 9.463309288024902, + 9.436361312866211, + 7.990633487701416, + 7.58080530166626 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train0", + "selected_index": 0, + "state_hash": "bf24d506cf320effed2cd7faf94d8618b0626bf8d5c3abc1f837513597c18231", + "stored_base_utility": 0.15946325659751892, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05853639170527458, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05853639170527458 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06729111820459366, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06729111820459366 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.05853639170527458, + 1.0, + 1.0, + 0.06729111820459366, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ebacae0135af2832dfbb", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683", + "ms-3d090ac2459c4b891dd1", + "ms-003a2225c0892dcda683", + "ms-ebacae0135af2832dfbb", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "generated_tangents": [ + [ + 0.19821582734584808, + 0.11862018704414368, + 0.2940768003463745, + -0.2675343453884125, + 1.5529329776763916, + 0.03267388045787811, + -0.3338521122932434, + 0.20622263848781586, + -0.16410942375659943, + -0.0012958496809005737, + 0.2084309458732605, + 0.06816083192825317, + 0.009602665901184082, + -0.1524500995874405, + -0.5379070043563843, + -0.0378229059278965, + 0.7765757441520691, + 0.12415192276239395, + -0.17680013179779053, + 1.699528694152832, + -0.1770472228527069 + ], + [ + 0.030832655727863312, + 0.0030104704201221466, + 0.40686947107315063, + -0.09046083688735962, + 1.7328708171844482, + 0.3114061951637268, + -0.18760895729064941, + 0.008012473583221436, + -0.06498051434755325, + 0.23737448453903198, + 0.14674559235572815, + 0.2794705033302307, + 0.2682911455631256, + -0.07735355198383331, + -0.6674419641494751, + -0.00031372811645269394, + 0.8229855895042419, + 0.08752962201833725, + -0.184705913066864, + 1.746021032333374, + 0.003982633352279663 + ], + [ + 0.11717712879180908, + 0.018371693789958954, + 0.31803399324417114, + -0.17400242388248444, + 1.637786865234375, + 0.34610849618911743, + -0.2596966028213501, + -0.007424697279930115, + -0.16548901796340942, + 0.20763692259788513, + 0.25298720598220825, + 0.33889806270599365, + 0.35871079564094543, + 0.051082104444503784, + -0.6170861124992371, + -0.022225230932235718, + 0.9731844067573547, + -0.06910558044910431, + -0.19000715017318726, + 1.9738883972167969, + 0.01560114324092865 + ], + [ + -0.011447951197624207, + 0.018537312746047974, + 0.40751609206199646, + -0.21178218722343445, + 1.707357406616211, + 0.25973063707351685, + 0.005991578102111816, + 0.0486777126789093, + -0.12137611210346222, + 0.22774119675159454, + 0.11506661772727966, + 0.22778189182281494, + 0.2834572494029999, + -0.07247871160507202, + -0.5803357362747192, + -0.030795760452747345, + 0.7830451726913452, + -0.07372553646564484, + -0.2007940709590912, + 1.7946207523345947, + 0.06326296180486679 + ], + [ + 0.06426630169153214, + -0.01832268014550209, + 0.32810738682746887, + -0.19499707221984863, + 1.594602108001709, + 0.31487834453582764, + -0.17571765184402466, + 0.09489920735359192, + -0.11910350620746613, + 0.20192202925682068, + 0.24325668811798096, + 0.3401080071926117, + 0.34818747639656067, + -0.018125757575035095, + -0.6066502928733826, + 0.04364753141999245, + 0.9730539321899414, + -0.05469249188899994, + -0.10362094640731812, + 2.006929636001587, + 0.05612626671791077 + ], + [ + -0.039808452129364014, + -0.030687691643834114, + 0.3820580244064331, + -0.102969229221344, + 1.6634316444396973, + 0.43563738465309143, + -0.13620656728744507, + 0.06918269395828247, + -0.11970779299736023, + 0.2843729257583618, + 0.15523378551006317, + 0.2955305278301239, + 0.30110087990760803, + -0.13235266506671906, + -0.690915048122406, + 0.03766452521085739, + 0.9249383211135864, + -0.03850726783275604, + -0.30874335765838623, + 1.7154533863067627, + 0.04161655902862549 + ], + [ + -0.08949635922908783, + 0.003914676606655121, + 0.4041188359260559, + -0.09749069064855576, + 1.6887085437774658, + 0.41461846232414246, + -0.20469671487808228, + -0.08192518353462219, + -0.13876117765903473, + 0.1715545803308487, + 0.14327865839004517, + 0.33035677671432495, + 0.4296446144580841, + -0.12914808094501495, + -0.6596945524215698, + 0.04474303126335144, + 0.8887852430343628, + 0.03434893488883972, + -0.23880088329315186, + 1.7352321147918701, + 0.08018404245376587 + ], + [ + -0.06557545065879822, + 0.1724398285150528, + 0.388532429933548, + -0.22448638081550598, + 1.5847713947296143, + 0.27204227447509766, + -0.37074482440948486, + 0.08504746854305267, + -0.22743508219718933, + 0.23558951914310455, + 0.22090449929237366, + 0.41367703676223755, + 0.24430879950523376, + 0.18120113015174866, + -0.6481141448020935, + -0.07451341301202774, + 0.8715323805809021, + -0.0502660870552063, + -0.13620072603225708, + 1.8175432682037354, + 0.1621093600988388 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.05853639170527458, + 2.0, + 2.0, + 0.06729111820459366, + 2.0, + 2.0 + ], + "group_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20931994915008545, + 2.0, + 2.0, + 0.2008315920829773, + 0.09476469457149506, + 0.05857282876968384, + 0.10009106248617172, + 0.07648500055074692, + 0.05889680236577988, + 0.014406224712729454, + 0.09262995421886444, + 0.15997587144374847, + 0.04682888835668564, + 0.09751574695110321, + 0.06459556519985199, + 0.20335909724235535 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7537296414375305, + 0.4158197343349457, + 1.4326180219650269, + -1.4107877016067505, + 2.9579341411590576, + 0.9669660925865173, + 1.4339183568954468, + 0.5987398624420166, + -0.2018759399652481, + 0.7105501890182495, + -0.03802938759326935, + 1.3799947500228882, + -0.6568997502326965, + -0.37473973631858826, + -1.857657790184021, + 0.038770925253629684, + 1.931185007095337, + 0.6317195296287537, + 0.1380508691072464, + 2.954265832901001, + 2.9641003608703613 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8678367137908936, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.7494794726371765, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.928200721740723 + ], + [ + 0.6226351857185364, + 0.34334811568260193, + 1.492361068725586, + -1.5917733907699585, + 3.1334922313690186, + 1.0313001871109009, + 1.2075668573379517, + 0.3046378493309021, + -0.11550238728523254, + 0.4499495327472687, + 0.16646838188171387, + 1.5877203941345215, + -0.6488803625106812, + -0.5760968327522278, + -1.8349132537841797, + 0.14188142120838165, + 2.0770444869995117, + 0.5806879997253418, + 0.23913151025772095, + 2.8127381801605225, + 3.0649962425231934 + ], + [ + 0.5975998044013977, + 0.722756028175354, + 1.7548110485076904, + -1.166775107383728, + 2.6060047149658203, + 0.6675758361816406, + 1.125187635421753, + 0.3373170793056488, + -0.1842741072177887, + 0.8186168670654297, + -0.31481224298477173, + 1.2082113027572632, + -0.9804694652557373, + -0.14131735265254974, + -1.700500249862671, + 0.07075637578964233, + 1.5488730669021606, + 0.549576461315155, + 0.034750498831272125, + 3.227431297302246, + 2.9464566707611084 + ], + [ + 0.844982922077179, + 0.7132244110107422, + 1.4430484771728516, + -1.3331272602081299, + 3.1703803539276123, + 1.1851513385772705, + 1.355688452720642, + 0.6115275025367737, + 0.2616361379623413, + 1.2030918598175049, + 0.05625858157873154, + 1.6050300598144531, + -0.7155078053474426, + -0.021488159894943237, + -2.073806047439575, + 0.5387547016143799, + 1.6223118305206299, + 0.7140727043151855, + -0.06027664244174957, + 2.7503156661987305, + 2.6344075202941895 + ], + [ + 0.5821706056594849, + 0.78206866979599, + 1.8119354248046875, + -0.8252442479133606, + 3.038909435272217, + 1.3900940418243408, + 1.7462186813354492, + 0.005083680152893066, + -0.27262064814567566, + 0.867202877998352, + -0.5555376410484314, + 1.172202706336975, + -1.2016760110855103, + -0.06355220079421997, + -2.2925045490264893, + 0.6078628301620483, + 1.7895314693450928, + 0.8720078468322754, + 0.472573459148407, + 3.377012014389038, + 2.6039178371429443 + ], + [ + 0.3789674639701843, + 0.3270051181316376, + 1.056584358215332, + -2.0788497924804688, + 2.948819875717163, + 1.6022355556488037, + 0.8599810004234314, + 0.9570736885070801, + -0.6380544900894165, + 0.3103100061416626, + -0.42896610498428345, + 1.7150276899337769, + -0.8975871205329895, + -0.278381884098053, + -1.305256724357605, + 0.3637259304523468, + 1.8004443645477295, + 0.16446983814239502, + 0.19097819924354553, + 3.43778657913208, + 3.538135528564453 + ], + [ + 0.6903814673423767, + 1.1327197551727295, + 1.8523926734924316, + -2.0428731441497803, + 3.1870431900024414, + 0.5352929830551147, + 0.6004904508590698, + 1.2120530605316162, + 0.16280107200145721, + 1.119707465171814, + 0.7612010836601257, + 0.7890661954879761, + -1.3447637557983398, + -0.1703537553548813, + -1.1010353565216064, + 0.32831913232803345, + 1.5534476041793823, + -0.2119135856628418, + -0.562728762626648, + 2.4702467918395996, + 2.5528204441070557 + ], + [ + 0.9177048206329346, + -0.04751536250114441, + 1.4819984436035156, + -2.1542677879333496, + 3.522390842437744, + 1.8177759647369385, + 2.2213237285614014, + 0.8223017454147339, + -0.1973809152841568, + -0.053272128105163574, + -0.5862784385681152, + 0.6421516537666321, + -0.8713911771774292, + -0.9495017528533936, + -1.6454768180847168, + -0.38891172409057617, + 1.648432970046997, + 1.4951133728027344, + 0.5893608331680298, + 2.449934482574463, + 3.419659376144409 + ], + [ + 1.2956775426864624, + -0.3119873106479645, + 1.5930066108703613, + -1.7682520151138306, + 2.4385876655578613, + 1.4679458141326904, + 1.4536705017089844, + 0.31068816781044006, + 0.660710871219635, + 0.8104690909385681, + 0.483506441116333, + 2.1112046241760254, + -0.22927385568618774, + 0.17858651280403137, + -2.485126495361328, + -0.9187790155410767, + 1.2810213565826416, + 1.3727718591690063, + 0.44044917821884155, + 3.4018898010253906, + 3.39054799079895 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0012856721878051758, + 0.00795695185661316, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + 0.11055737733840942, + 0.0, + 0.02143383026123047, + 0.0025591105222702026, + -0.030314385890960693, + -0.0125969760119915, + 0.3799947500228882, + 0.0076233744621276855, + 0.0, + -0.857657790184021, + 0.013764806091785431, + 0.9311850070953369, + 0.07110422849655151, + 0.03423507511615753, + 1.954265832901001, + 0.0 + ], + [ + -0.24627035856246948, + -0.16095152497291565, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + -0.023127973079681396, + 0.0, + -0.24312204122543335, + 0.1389055997133255, + 0.11457735300064087, + 0.06618259102106094, + 0.3799947500228882, + 0.07616078853607178, + 0.0, + -0.857657790184021, + 0.09948430210351944, + 0.9311850070953369, + 0.006249547004699707, + -0.33439844846725464, + 1.954265832901001, + 0.0 + ] + ], + "predicted_scores": [ + 8.333680152893066, + 8.00536060333252, + 7.794225215911865, + 7.692945957183838, + 7.54518985748291, + 7.450100898742676, + 7.431529998779297, + 7.38453483581543 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "534717141/train0", + "selected_index": 0, + "state_hash": "5318995ba859471b8678d6594274c7c0565a1ccfd83ea302378cc2884dc5cc32", + "stored_base_utility": 0.20931994915008545, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14809735119342804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14809735119342804 + }, + "base_progress": 0.14809735119342804, + "base_success": false, + "base_utility": 0.14809735119342804, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0012347499141469598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0012347499141469598 + }, + { + "contact_quality": null, + "progress": 0.0006952881813049316, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006952881813049316 + }, + { + "contact_quality": null, + "progress": 0.0009832581272348762, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009832581272348762 + }, + { + "contact_quality": null, + "progress": 0.001222034334205091, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001222034334205091 + }, + { + "contact_quality": null, + "progress": 0.001184225082397461, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001184225082397461 + }, + { + "contact_quality": null, + "progress": 0.0017026265850290656, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0017026265850290656 + }, + { + "contact_quality": null, + "progress": 0.0010568499565124512, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0010568499565124512 + }, + { + "contact_quality": null, + "progress": 0.0020139615517109632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0020139615517109632 + } + ], + "candidate_progress": [ + 0.0012347499141469598, + 0.0006952881813049316, + 0.0009832581272348762, + 0.001222034334205091, + 0.001184225082397461, + 0.0017026265850290656, + 0.0010568499565124512, + 0.0020139615517109632 + ], + "candidate_source_chart_ids": [ + "ms-f2b2b2a1aa2863c3676f", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-da142025733b53fcf7eb", + "generated_tangents": [ + [ + -0.14792807400226593, + -0.12037685513496399, + 1.2677156925201416, + 0.22903186082839966, + 0.7840882539749146, + 2.7241084575653076, + 2.1215412616729736, + -0.26216986775398254, + -0.20857413113117218, + 1.357494592666626, + 0.36197763681411743, + 0.7453258633613586, + 1.7902954816818237, + 2.072998046875, + -1.1923516988754272, + -0.32508280873298645, + 1.9591964483261108, + 0.8809031248092651, + 0.35911834239959717, + 2.363125801086426, + 1.8979345560073853 + ], + [ + -0.008208125829696655, + -0.15411368012428284, + 1.5386511087417603, + -0.11599639058113098, + 0.3467293083667755, + 2.42981219291687, + 2.20508074760437, + -0.571887731552124, + -0.09749794006347656, + 1.5345336198806763, + 0.8130164742469788, + 0.654485285282135, + 2.4730300903320312, + 1.6559977531433105, + -1.1437333822250366, + -0.29220739006996155, + 2.4106664657592773, + 0.5348500609397888, + 0.42562544345855713, + 1.9994966983795166, + 1.5532678365707397 + ], + [ + -0.12392519414424896, + -0.10915147513151169, + 1.6301875114440918, + -0.11622365564107895, + 0.41216981410980225, + 2.134881019592285, + 1.5720794200897217, + -0.2339523434638977, + -0.1566099375486374, + 1.3242512941360474, + 0.5845993161201477, + 0.8133550882339478, + 1.8213011026382446, + 1.6081379652023315, + -1.277977705001831, + -0.3984118402004242, + 2.031397581100464, + 0.8786622285842896, + 0.3703434467315674, + 2.047316312789917, + 1.8929274082183838 + ], + [ + -0.25291842222213745, + -0.001621447503566742, + 1.231136679649353, + -0.13570603728294373, + 0.867662250995636, + 2.5823974609375, + 1.6352968215942383, + -0.27264800667762756, + -0.07834309339523315, + 1.3511162996292114, + 0.5456430315971375, + 0.8499860167503357, + 1.3987655639648438, + 1.8022953271865845, + -1.127785086631775, + -0.399728924036026, + 2.1605467796325684, + 0.685965359210968, + 0.6784347295761108, + 2.2681829929351807, + 1.9904145002365112 + ], + [ + -0.047950804233551025, + -0.09295839071273804, + 1.4757448434829712, + -0.2742197513580322, + 1.0798732042312622, + 2.7047958374023438, + 2.0619595050811768, + -0.4945589601993561, + -0.03208528459072113, + 1.337076187133789, + 0.49107930064201355, + 0.7432037591934204, + 1.6977874040603638, + 1.8670111894607544, + -1.0429173707962036, + -0.26384344696998596, + 1.962184190750122, + 0.5686109066009521, + 0.6349022388458252, + 2.673661470413208, + 2.3525197505950928 + ], + [ + -0.007411673665046692, + -0.08534751832485199, + 1.3961024284362793, + -0.2842656970024109, + 0.445515900850296, + 2.27243971824646, + 1.8400790691375732, + -0.19332990050315857, + 0.022110791876912117, + 1.1534346342086792, + 0.37642133235931396, + 1.1431910991668701, + 1.9179803133010864, + 2.2775497436523438, + -1.1335296630859375, + -0.35102662444114685, + 2.029925584793091, + 0.6594119668006897, + 0.5136226415634155, + 2.668724536895752, + 2.2608978748321533 + ], + [ + -0.11100061237812042, + -0.1824159175157547, + 1.3723037242889404, + -0.2519010305404663, + 0.6622046232223511, + 2.399218797683716, + 1.8872203826904297, + -0.22966846823692322, + -0.07654914259910583, + 1.373837947845459, + 0.5841163396835327, + 0.9163534045219421, + 1.6412230730056763, + 1.7161020040512085, + -1.2128006219863892, + -0.34733760356903076, + 2.2784645557403564, + 0.7162875533103943, + 0.46822768449783325, + 2.420989513397217, + 2.2971346378326416 + ], + [ + -0.3248453736305237, + -0.3080900311470032, + 1.159929871559143, + 0.49731698632240295, + 0.577373743057251, + 2.29689884185791, + 1.7428398132324219, + -0.19394347071647644, + -0.18259301781654358, + 1.1071598529815674, + 0.11351655423641205, + 0.7244806885719299, + 1.2099586725234985, + 2.5740606784820557, + -0.7791820168495178, + -0.2937033474445343, + 1.731724739074707, + 0.6920046806335449, + 0.42418861389160156, + 2.5110080242156982, + 1.9522656202316284 + ] + ], + "generated_utilities": [ + 0.0012347499141469598, + 0.0006952881813049316, + 0.0009832581272348762, + 0.001222034334205091, + 0.001184225082397461, + 0.0017026265850290656, + 0.0010568499565124512, + 0.0020139615517109632 + ], + "group_id": "ms-da142025733b53fcf7eb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.14843207597732544, + 0.1717694252729416, + 0.09356185048818588, + 0.23231086134910583, + 0.19341367483139038, + 0.04822012037038803, + 0.23024770617485046, + 0.16561973094940186, + 0.2643269896507263, + 0.12403462827205658, + 0.12689702212810516, + 0.16619956493377686, + 0.13481859862804413, + 0.07537531852722168, + 0.11867459863424301, + 0.04740651696920395 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.04848463833332062, + -0.18164527416229248, + 0.6558789014816284, + 0.038211286067962646, + 0.08492374420166016, + 2.265157699584961, + 0.0, + -0.09011825919151306, + 0.0018594712018966675, + 0.6884076595306396, + 0.027596401050686836, + 0.06117302179336548, + 1.278116226196289, + 0.0, + -0.3951960802078247, + -0.08061015605926514, + 1.07906174659729, + 0.3229694366455078, + -0.21076995134353638, + 1.8564467430114746, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0903544425964355, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.570276260375977, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.736253261566162 + ], + [ + 0.08095964789390564, + -0.7926032543182373, + 1.571103811264038, + 0.07894467562437057, + 0.2701656222343445, + 3.806621789932251, + 2.203669786453247, + -0.042464032769203186, + 0.7168105244636536, + 1.8671371936798096, + -0.30838707089424133, + 1.7242767810821533, + 1.8847527503967285, + 1.305934190750122, + -0.42665380239486694, + -0.2973874807357788, + 1.1881210803985596, + 1.6921244859695435, + 1.070060133934021, + 2.067784309387207, + 1.916501760482788 + ], + [ + 0.7620636224746704, + -2.1605899333953857, + 2.1443262100219727, + 0.17886832356452942, + 1.6259945631027222, + 3.787412166595459, + 1.3663169145584106, + 0.49799659848213196, + -0.6301277279853821, + 2.275850534439087, + -0.6735645532608032, + 1.7504758834838867, + 2.7512195110321045, + 1.7462939023971558, + -0.6835377216339111, + 0.36623528599739075, + 1.405979871749878, + 0.6472548842430115, + 1.4791045188903809, + 2.7838869094848633, + 3.0856707096099854 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.29792410135269165, + -0.30587875843048096, + 0.6558789014816284, + -0.4171115756034851, + -0.1733894944190979, + 2.265157699584961, + 0.0, + 0.14607194066047668, + 0.05651267617940903, + 0.6884076595306396, + -0.0433836504817009, + 0.07522618770599365, + 1.278116226196289, + 0.0, + -0.41462820768356323, + 0.26706579327583313, + 1.07906174659729, + 0.44956159591674805, + -0.37072819471359253, + 1.8564467430114746, + 0.0 + ], + [ + 0.18432100117206573, + -0.9473621249198914, + 1.5389783382415771, + -0.041053857654333115, + 0.6282865405082703, + 3.276425838470459, + 2.1839330196380615, + -0.3612370491027832, + 0.23081421852111816, + 1.8094429969787598, + 0.10581490397453308, + 0.8865737915039062, + 2.5146288871765137, + 2.2134878635406494, + -1.6751070022583008, + -0.08268362283706665, + 2.303063154220581, + 1.3061904907226562, + 0.4770132899284363, + 3.109205722808838, + 2.509927272796631 + ], + [ + -0.5288646817207336, + -1.5067847967147827, + 1.376499056816101, + 0.13231416046619415, + 1.048660159111023, + 3.5048489570617676, + 1.9678237438201904, + -0.14848726987838745, + 0.05475667119026184, + 1.5529061555862427, + 0.1597936749458313, + 0.7318928241729736, + 2.17749285697937, + 2.0782582759857178, + -1.5473355054855347, + -0.6715520620346069, + 1.7043644189834595, + 1.1190917491912842, + 0.4214729070663452, + 3.3400330543518066, + 2.3147284984588623 + ] + ], + "predicted_scores": [ + 9.4291353225708, + 9.417654037475586, + 8.99586009979248, + 8.808168411254883, + 8.785951614379883, + 8.686368942260742, + 8.476127624511719, + 7.9679741859436035 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train0", + "selected_index": 0, + "state_hash": "0026cc5b3762b520baadbda7bf138a2d24fffa5b2306f245ddbc1d1556cccc22", + "stored_base_utility": 0.14843207597732544, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1362423449754715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1362423449754715 + }, + "base_progress": 0.1362423449754715, + "base_success": false, + "base_utility": 0.1362423449754715, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03034065105021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03034065105021 + }, + { + "contact_quality": null, + "progress": 0.03361199423670769, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03361199423670769 + }, + { + "contact_quality": null, + "progress": 0.033086616545915604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.033086616545915604 + }, + { + "contact_quality": null, + "progress": 0.019350707530975342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019350707530975342 + }, + { + "contact_quality": null, + "progress": 0.0312865786254406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0312865786254406 + }, + { + "contact_quality": null, + "progress": 0.008349676616489887, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008349676616489887 + }, + { + "contact_quality": null, + "progress": 0.007631381507962942, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007631381507962942 + }, + { + "contact_quality": null, + "progress": 0.00866107176989317, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00866107176989317 + } + ], + "candidate_progress": [ + 0.03034065105021, + 0.03361199423670769, + 0.033086616545915604, + 0.019350707530975342, + 0.0312865786254406, + 0.008349676616489887, + 0.007631381507962942, + 0.00866107176989317 + ], + "candidate_source_chart_ids": [ + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-cf4b63d109a88a4c42bf", + "ms-7e88428497a6b13936c4", + "ms-76f27d9a935b25e15c38", + "ms-a2518826fa0a816cc46f", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-e2bad6e99e8a281207cc", + "generated_tangents": [ + [ + -0.16096971929073334, + -0.31177908182144165, + 1.854225516319275, + -0.2686333656311035, + 1.4799003601074219, + 2.5605580806732178, + 2.5912859439849854, + 1.0004174709320068, + -0.15160799026489258, + -0.3394080698490143, + 1.0288563966751099, + 1.3309565782546997, + 0.2021808922290802, + -0.01970010995864868, + 0.9773865938186646, + 0.24064843356609344, + -0.39815592765808105, + -0.9252974390983582, + 1.032639503479004, + -2.2695326805114746, + -1.7332607507705688 + ], + [ + -0.2856951951980591, + -0.32508283853530884, + 1.8239376544952393, + -0.4597397446632385, + 1.4449050426483154, + 2.6679069995880127, + 2.6012749671936035, + 0.9416310787200928, + -0.10728119313716888, + -0.2472185492515564, + 0.9183798432350159, + 1.1879003047943115, + 0.1981048882007599, + -0.10409484803676605, + 1.0273622274398804, + 0.24549944698810577, + -0.45119577646255493, + -0.8537749648094177, + 0.9723955988883972, + -2.472135543823242, + -1.6466033458709717 + ], + [ + -0.2856389880180359, + -0.3050289750099182, + 1.602755069732666, + -0.5033504366874695, + 1.32881498336792, + 2.7149498462677, + 2.5599634647369385, + 1.0255323648452759, + -0.12451672554016113, + -0.2931959927082062, + 0.8575699329376221, + 1.1229257583618164, + 0.0978078544139862, + -0.051552996039390564, + 1.0300428867340088, + 0.1655631810426712, + -0.6067094802856445, + -0.9393851161003113, + 0.9195767641067505, + -2.4180305004119873, + -1.631742238998413 + ], + [ + -1.2150877714157104, + -0.5264142155647278, + 2.4768450260162354, + -0.3646775186061859, + 1.4079499244689941, + 2.2131717205047607, + 2.17423152923584, + 1.2163509130477905, + -0.19565480947494507, + -0.2968308925628662, + 1.1425195932388306, + 1.0473694801330566, + -0.047198206186294556, + 0.05088570713996887, + 1.2641568183898926, + -0.25054869055747986, + -0.3730945587158203, + -1.5274989604949951, + 0.6246023178100586, + -3.6558520793914795, + -2.540768623352051 + ], + [ + -0.36524295806884766, + -0.2680980861186981, + 1.5086112022399902, + -0.3838243782520294, + 1.519239902496338, + 3.145357370376587, + 3.230030059814453, + 0.8950217962265015, + -0.11261885613203049, + -0.11335598677396774, + 0.5106876492500305, + 1.5056214332580566, + 0.21764037013053894, + 0.12477201968431473, + 1.3788115978240967, + 0.09976649284362793, + -0.7163516879081726, + -0.819511353969574, + 1.0259342193603516, + -2.596590518951416, + -1.5088063478469849 + ], + [ + -1.3984849452972412, + -0.4602620005607605, + 2.099067211151123, + -0.43182337284088135, + 2.5447330474853516, + 1.9244742393493652, + 2.7048635482788086, + 1.1009728908538818, + -0.5510379076004028, + 0.26409685611724854, + 1.122795820236206, + 0.7084417343139648, + -1.1655607223510742, + -0.22580210864543915, + 1.222090482711792, + -0.14433996379375458, + -0.9677732586860657, + -1.102054238319397, + 0.7394117712974548, + -2.6706671714782715, + -2.4400110244750977 + ], + [ + -0.7646927833557129, + -0.5394712090492249, + 2.0804500579833984, + -0.38889607787132263, + 2.2072486877441406, + 2.602515935897827, + 2.5818116664886475, + 1.3543554544448853, + -0.5414590835571289, + -0.05968841165304184, + 1.278780221939087, + 1.177679181098938, + -1.1409351825714111, + -0.7847484350204468, + 1.2067512273788452, + -0.13696825504302979, + -0.7112036943435669, + -0.5763885974884033, + 1.5668368339538574, + -2.2055201530456543, + -1.8194376230239868 + ], + [ + -1.420310616493225, + -0.4101560115814209, + 2.1167399883270264, + -0.42431551218032837, + 2.489821434020996, + 2.1338396072387695, + 2.5821707248687744, + 1.0836492776870728, + -0.551445484161377, + 0.18411391973495483, + 1.121557593345642, + 0.8325760364532471, + -1.1308687925338745, + -0.350838303565979, + 1.4645041227340698, + -0.18100634217262268, + -1.0054962635040283, + -1.1382744312286377, + 0.761302649974823, + -2.7134416103363037, + -2.494117498397827 + ] + ], + "generated_utilities": [ + 0.03034065105021, + 0.03361199423670769, + 0.033086616545915604, + 0.019350707530975342, + 0.0312865786254406, + 0.008349676616489887, + 0.007631381507962942, + 0.00866107176989317 + ], + "group_id": "ms-e2bad6e99e8a281207cc", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13993772864341736, + 0.16482847929000854, + 0.20825156569480896, + 0.16188287734985352, + 0.13454246520996094, + 0.14753298461437225, + 0.14504572749137878, + 0.06676968187093735, + 0.09772863239049911, + 0.0585557222366333, + 0.12863430380821228, + 0.10292482376098633, + 0.07993738353252411, + 0.03641533851623535, + 0.04985213279724121, + 0.1516878604888916 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.3249012231826782, + -0.9610954523086548, + 2.054325819015503, + 1.018629789352417, + 2.393625497817993, + 2.2159433364868164, + 2.098443031311035, + -0.49140021204948425, + 0.044472046196460724, + 0.8045110702514648, + 1.4511574506759644, + -0.3077920377254486, + -0.052587300539016724, + 0.036380887031555176, + 0.7875679731369019, + -0.3928021192550659, + -0.09456783533096313, + -1.4901847839355469, + 1.1711406707763672, + -2.1349995136260986, + -2.076193332672119 + ], + [ + -1.4271517992019653, + -1.114187479019165, + 1.618905782699585, + 1.0621737241744995, + 1.6005314588546753, + 3.017716407775879, + 2.3444156646728516, + -1.0368298292160034, + 0.6155713796615601, + 1.0049463510513306, + 1.3708372116088867, + -0.27396243810653687, + -0.19098177552223206, + -0.5735776424407959, + 0.863328218460083, + -0.4340485632419586, + 0.09502258896827698, + -1.8856427669525146, + 0.9514620304107666, + -2.3912711143493652, + -2.6571128368377686 + ], + [ + -1.0088777542114258, + -1.803802251815796, + 2.6037826538085938, + 0.763943612575531, + 1.7241847515106201, + 2.433204174041748, + 2.273603916168213, + 0.06359493732452393, + -0.17274422943592072, + -0.23299545049667358, + 1.7126038074493408, + -0.3797045052051544, + 0.7567765712738037, + -0.9981052875518799, + 1.0371832847595215, + -0.5120593309402466, + -0.059397101402282715, + -1.9757622480392456, + 0.7378945350646973, + -2.6195693016052246, + -2.178041458129883 + ], + [ + -0.09093856811523438, + -0.14735746383666992, + 1.350311517715454, + 1.477137565612793, + 2.134960174560547, + 1.7331886291503906, + 2.24615740776062, + -1.7148966789245605, + -0.3949584364891052, + 0.4433448612689972, + 0.9034961462020874, + -0.23445925116539001, + -0.6371403932571411, + -0.038229942321777344, + 2.010648250579834, + 0.14472149312496185, + -0.26303571462631226, + -2.3003270626068115, + 0.565727949142456, + -2.586710214614868, + -2.098479747772217 + ], + [ + -1.8506121635437012, + -1.4700038433074951, + 2.4313056468963623, + 1.6264371871948242, + 1.1548577547073364, + 2.9678268432617188, + 2.990251302719116, + -0.25485146045684814, + -0.36889538168907166, + -0.14847010374069214, + 0.6971698999404907, + -0.6752342581748962, + 0.44545215368270874, + 0.663374125957489, + 0.6998850107192993, + -0.6500043869018555, + -0.04876291751861572, + -2.446995258331299, + 0.7760334014892578, + -1.38918137550354, + -1.5372083187103271 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.053252458572387695, + -0.006605625152587891, + 1.0103552341461182, + 0.25736117362976074, + 1.010573148727417, + 1.4690883159637451, + 0.0, + -0.0840916633605957, + -0.24044206738471985, + -0.06801849603652954, + 0.061444878578186035, + 0.11697901785373688, + 0.03979811072349548, + 0.0, + 0.054485201835632324, + -0.11758939921855927, + 0.31161248683929443, + -0.602575421333313, + -0.021049559116363525, + -1.3257782459259033, + 0.0 + ] + ], + "predicted_scores": [ + 6.693980693817139, + 6.22277307510376, + 5.9340410232543945, + 5.360879898071289, + 5.2152886390686035, + 4.9879469871521, + 4.837738513946533, + 4.346799850463867 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train0", + "selected_index": 0, + "state_hash": "f054ce13f25dcda8808ebde2bee98ca84bfbdc3fc328a0eb831e21faada3fa4c", + "stored_base_utility": 0.13993772864341736, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09695577621459961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09695577621459961 + }, + "base_progress": 0.09695577621459961, + "base_success": false, + "base_utility": 0.09695577621459961, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03324349597096443, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03324349597096443 + }, + { + "contact_quality": null, + "progress": 0.06069042161107063, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06069042161107063 + }, + { + "contact_quality": null, + "progress": 0.05841739848256111, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05841739848256111 + }, + { + "contact_quality": null, + "progress": 0.05722795054316521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05722795054316521 + }, + { + "contact_quality": null, + "progress": 0.07222036272287369, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07222036272287369 + }, + { + "contact_quality": null, + "progress": 0.05981389805674553, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05981389805674553 + }, + { + "contact_quality": null, + "progress": 0.07067686319351196, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07067686319351196 + }, + { + "contact_quality": null, + "progress": 0.04897226765751839, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04897226765751839 + } + ], + "candidate_progress": [ + 0.03324349597096443, + 0.06069042161107063, + 0.05841739848256111, + 0.05722795054316521, + 0.07222036272287369, + 0.05981389805674553, + 0.07067686319351196, + 0.04897226765751839 + ], + "candidate_source_chart_ids": [ + "ms-6e27d292a9cbea108fc9", + "ms-206099bab6bfe7c84fce", + "ms-6e27d292a9cbea108fc9", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e30fd80adc089e9f697a", + "generated_tangents": [ + [ + 0.14059625566005707, + 0.28916049003601074, + -0.16963151097297668, + 1.314062237739563, + 0.7050162553787231, + -1.0451172590255737, + -1.3139665126800537, + -0.2140854001045227, + -0.011177374050021172, + -0.08635316044092178, + 0.8264796733856201, + 0.22506049275398254, + 0.16762247681617737, + -0.4966709017753601, + -0.38111889362335205, + 0.11698208749294281, + 0.43732184171676636, + -0.2739487290382385, + -0.5380127429962158, + 0.09203675389289856, + -0.018765494227409363 + ], + [ + 0.12499076873064041, + -0.05133494734764099, + -0.36809441447257996, + 1.2461128234863281, + 0.8098480105400085, + -0.6893341541290283, + -1.1495845317840576, + -0.07863315939903259, + -0.0462343692779541, + -0.10261634737253189, + 0.7539368271827698, + 0.17526593804359436, + 0.05268070101737976, + -0.49184125661849976, + -0.6165304183959961, + 0.31208768486976624, + 0.39507371187210083, + -0.14809341728687286, + -0.5398120880126953, + 0.4846477508544922, + 0.326396644115448 + ], + [ + 0.14577935636043549, + 0.1370626538991928, + -0.2611781656742096, + 0.999873161315918, + 0.6963958740234375, + -1.1857969760894775, + -1.25225830078125, + -0.125323086977005, + -0.03071071393787861, + 0.06095661222934723, + 0.7633594870567322, + 0.15987130999565125, + 0.07322022318840027, + -0.6642807722091675, + -0.5136773586273193, + 0.22974568605422974, + 0.375954806804657, + -0.2856103777885437, + -0.35792022943496704, + 0.3997807502746582, + 0.006541267037391663 + ], + [ + 0.17108535766601562, + -0.1494685709476471, + -0.061604052782058716, + 1.040161371231079, + 0.242130309343338, + -0.7885425090789795, + -1.5219964981079102, + 0.2713053226470947, + 0.18858802318572998, + 0.17530256509780884, + 1.0080199241638184, + 0.807894766330719, + -0.001270681619644165, + -0.5104037523269653, + -0.9724703431129456, + 0.2597637176513672, + 0.7744123339653015, + 0.22221793234348297, + -0.28697168827056885, + 0.4465206265449524, + -0.16993442177772522 + ], + [ + 0.13455934822559357, + -0.0938536748290062, + -0.007500439882278442, + 0.8863233327865601, + 0.19334590435028076, + -1.0735853910446167, + -1.4584605693817139, + 0.25584304332733154, + 0.12632793188095093, + 0.09572681039571762, + 1.2192953824996948, + 0.4742032289505005, + 0.02632009983062744, + -0.622262716293335, + -0.9880751967430115, + 0.2867356836795807, + 0.5601792931556702, + 0.06965525448322296, + -0.4905439019203186, + 0.6875616312026978, + -0.03526487946510315 + ], + [ + 0.24791967868804932, + -0.19294780492782593, + 0.014462172985076904, + 0.935484766960144, + 0.2310054898262024, + -0.996497392654419, + -1.5055625438690186, + 0.15239405632019043, + 0.21127238869667053, + 0.16762784123420715, + 0.9673843383789062, + 0.7749419212341309, + -0.05129283666610718, + -0.4622701406478882, + -0.9616836905479431, + 0.25189459323883057, + 0.6646955609321594, + 0.2064647078514099, + -0.24755346775054932, + 0.5304515957832336, + -0.11553612351417542 + ], + [ + 0.3456129729747772, + -0.24611318111419678, + 0.0590285062789917, + 0.9294030666351318, + 0.21586112678050995, + -0.8951681852340698, + -1.4217758178710938, + -0.010192766785621643, + 0.24937164783477783, + 0.1815493106842041, + 0.7579646110534668, + 0.8004189133644104, + 0.09421303868293762, + -0.26927292346954346, + -1.0741379261016846, + 0.19147054851055145, + 0.5296486616134644, + 0.02319309115409851, + -0.23959577083587646, + 0.5275599956512451, + 0.00764957070350647 + ], + [ + 0.32306885719299316, + -0.24005533754825592, + -0.03500804305076599, + 1.013843059539795, + 0.17810168862342834, + -1.1318376064300537, + -1.4078500270843506, + 0.17959968745708466, + 0.24683411419391632, + 0.1291821449995041, + 0.8833457231521606, + 0.9837989807128906, + 0.22038337588310242, + -0.5237548351287842, + -0.9737306237220764, + 0.28015780448913574, + 0.5724561214447021, + 0.2557889521121979, + -0.0892578661441803, + 0.626552164554596, + -0.23747605085372925 + ] + ], + "generated_utilities": [ + 0.03324349597096443, + 0.06069042161107063, + 0.05841739848256111, + 0.05722795054316521, + 0.07222036272287369, + 0.05981389805674553, + 0.07067686319351196, + 0.04897226765751839 + ], + "group_id": "ms-e30fd80adc089e9f697a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.12557005882263184, + 0.11941087245941162, + 0.10894306749105453, + 0.060310762375593185, + 0.18920521438121796, + 0.11678676307201385, + 0.14083251357078552, + 0.12988364696502686, + 0.18350017070770264, + 0.1286114901304245, + 0.06744732707738876, + 0.04428015276789665, + 0.08399941772222519, + 0.0742691159248352, + 0.05598282814025879, + 0.09703926742076874 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.044197678565979004, + -0.12298870831727982, + -0.37176433205604553, + 0.5503368377685547, + 0.5411782264709473, + 0.6186751127243042, + 0.0, + 0.28674131631851196, + 0.08053566515445709, + 0.3295105993747711, + 0.8157306909561157, + 0.1715114712715149, + -0.43090736865997314, + 0.0, + -0.5639103651046753, + -0.48903688788414, + 0.7485904693603516, + -0.4474331736564636, + -0.18787053227424622, + 1.2062294483184814, + 0.0 + ], + [ + 1.3153167963027954, + 0.7455203533172607, + -1.0113484859466553, + 2.097999334335327, + 0.7455009818077087, + 0.3843202590942383, + -0.16713692247867584, + -0.8400754928588867, + -0.3845864534378052, + 1.065396785736084, + 1.712794542312622, + 0.803345263004303, + 0.6529266238212585, + 0.6022698879241943, + -1.1222561597824097, + 0.06260442733764648, + 1.101211428642273, + -0.133493572473526, + -0.8626402616500854, + 1.8406728506088257, + 0.8011860847473145 + ], + [ + 0.23532289266586304, + -0.516789972782135, + -0.3119599223136902, + 1.501338005065918, + 1.4930639266967773, + -0.419913113117218, + -0.6910054683685303, + 0.30246037244796753, + 0.041916340589523315, + 1.2685117721557617, + 2.5912232398986816, + 1.2864888906478882, + 0.3821069598197937, + 0.4677737355232239, + -1.84663724899292, + -0.8953573703765869, + 2.3256053924560547, + 0.18948839604854584, + -0.5580936670303345, + 1.528401494026184, + 1.4914395809173584 + ], + [ + 0.6903802752494812, + 0.3962534964084625, + -0.8207939863204956, + 0.5973623991012573, + 1.310564398765564, + 0.6411572694778442, + -0.2032768577337265, + 0.25504088401794434, + 0.6581551432609558, + 0.3938753306865692, + 1.0127826929092407, + 1.6796927452087402, + 0.4867250621318817, + 0.05769670009613037, + -2.026149034500122, + -0.17722094058990479, + 0.9414948225021362, + 0.6044259667396545, + 0.03290170431137085, + 2.9423327445983887, + -0.060306549072265625 + ], + [ + 0.47530579566955566, + -0.6085019111633301, + 0.12691906094551086, + 0.5673632025718689, + 0.08076304197311401, + 0.8255819082260132, + -0.9781221151351929, + -0.796129584312439, + -0.6745100021362305, + 0.7784937024116516, + 1.7613102197647095, + 1.4703174829483032, + -0.36175650358200073, + 0.16939619183540344, + -0.9399533271789551, + -1.2696547508239746, + 1.3314080238342285, + -0.3690394163131714, + -1.0323668718338013, + 2.305772304534912, + 0.27392566204071045 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.6770331859588623, + 0.09077263623476028, + -0.3141186833381653, + 1.5503368377685547, + 1.0133107900619507, + 0.12400172650814056, + -0.2399108111858368, + -0.4914711117744446, + -0.11917559802532196, + 0.7716305255889893, + 1.8157306909561157, + 0.7167991995811462, + 0.5690926313400269, + 0.6154549717903137, + -1.5639103651046753, + -0.5166093111038208, + 1.7485904693603516, + -0.15158627927303314, + -0.46638885140419006, + 2.2062294483184814, + 0.7902159690856934 + ], + [ + 0.9456368684768677, + -0.3964601159095764, + -0.5845282077789307, + 1.2133667469024658, + 0.5843623876571655, + -0.02685314416885376, + -0.006928771734237671, + -0.2159697711467743, + -0.3017333149909973, + 0.40500110387802124, + 1.9734654426574707, + 0.46303731203079224, + 0.33380845189094543, + 0.7758987545967102, + -2.060479164123535, + -0.955610990524292, + 1.942940592765808, + -0.23636659979820251, + -0.10507094860076904, + 1.8367226123809814, + 1.2234702110290527 + ] + ], + "predicted_scores": [ + 5.362038612365723, + 4.596473217010498, + 4.504575252532959, + 4.01229190826416, + 3.903841018676758, + 3.577493667602539, + 3.082627534866333, + 2.9932708740234375 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train0", + "selected_index": 0, + "state_hash": "ca8873454d7cf9723b1e743355bf255813544ae3212d80abc954fd347d7adf27", + "stored_base_utility": 0.12557005882263184, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09445196390151978, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09445196390151978 + }, + { + "contact_quality": null, + "progress": 0.13109225034713745, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13109225034713745 + }, + { + "contact_quality": null, + "progress": 0.07720557600259781, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07720557600259781 + }, + { + "contact_quality": null, + "progress": 0.0657956600189209, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0657956600189209 + }, + { + "contact_quality": null, + "progress": 0.08409891277551651, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08409891277551651 + }, + { + "contact_quality": null, + "progress": 0.10617973655462265, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10617973655462265 + }, + { + "contact_quality": null, + "progress": 0.11467918008565903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11467918008565903 + }, + { + "contact_quality": null, + "progress": 0.07619664818048477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07619664818048477 + } + ], + "candidate_progress": [ + 0.09445196390151978, + 0.13109225034713745, + 0.07720557600259781, + 0.0657956600189209, + 0.08409891277551651, + 0.10617973655462265, + 0.11467918008565903, + 0.07619664818048477 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "generated_tangents": [ + [ + 1.8835853338241577, + -0.36436957120895386, + -0.36908021569252014, + 0.19512054324150085, + -0.06663826107978821, + 2.007988452911377, + 1.8292548656463623, + 2.070711374282837, + 0.27917805314064026, + -1.3850232362747192, + -0.705543577671051, + -0.9560496807098389, + -2.571524143218994, + -3.2129359245300293, + 1.9666616916656494, + -0.006010702811181545, + -1.6693131923675537, + -0.13773630559444427, + -0.44948458671569824, + -1.9241769313812256, + -1.8779592514038086 + ], + [ + 1.8852283954620361, + -0.376431941986084, + -0.11510768532752991, + 0.44733771681785583, + -0.21439245343208313, + 2.242299795150757, + 1.5470595359802246, + 2.1995959281921387, + 0.26516249775886536, + -1.354644536972046, + -0.7016180157661438, + -0.7497810125350952, + -2.5319321155548096, + -3.1401779651641846, + 1.7917345762252808, + 0.1281343400478363, + -1.514054298400879, + -0.0010046660900115967, + -0.3372483253479004, + -1.6821763515472412, + -1.5676567554473877 + ], + [ + 1.737519383430481, + -0.26957106590270996, + -0.2971702516078949, + 0.2566094398498535, + -0.15461108088493347, + 1.9364793300628662, + 1.6857423782348633, + 2.138697624206543, + 0.27545449137687683, + -1.2138079404830933, + -0.6317903995513916, + -0.8632838726043701, + -2.3770337104797363, + -3.1577882766723633, + 2.170635938644409, + 0.02896834909915924, + -1.6298086643218994, + 0.026990994811058044, + -0.24416643381118774, + -1.7977702617645264, + -1.639520287513733 + ], + [ + 1.613640546798706, + -0.29559779167175293, + -0.3531942665576935, + 0.19700193405151367, + 0.005303964018821716, + 2.067160129547119, + 1.7746155261993408, + 1.940363883972168, + 0.12679347395896912, + -1.5333608388900757, + -0.6188918352127075, + -0.9786030054092407, + -2.2097017765045166, + -3.2174675464630127, + 1.716732382774353, + -0.04426439106464386, + -1.3874294757843018, + -0.03497208654880524, + -0.45412445068359375, + -2.0235114097595215, + -2.2255663871765137 + ], + [ + 1.8246638774871826, + -0.31953758001327515, + -0.6649669408798218, + 0.42153432965278625, + 0.14494477212429047, + 1.8097628355026245, + 0.9051036834716797, + 2.0791258811950684, + 0.19699890911579132, + -1.3843746185302734, + -0.5807974338531494, + -0.6608120203018188, + -2.4870944023132324, + -2.3387491703033447, + 1.909354567527771, + -0.06960231810808182, + -1.6148953437805176, + -0.17316053807735443, + -0.06122326850891113, + -1.9254603385925293, + -0.9899587631225586 + ], + [ + 1.8433128595352173, + -0.33298271894454956, + -0.41172775626182556, + 0.10377022624015808, + -0.052587538957595825, + 2.0646111965179443, + 1.7742080688476562, + 2.2596335411071777, + 0.23512783646583557, + -1.2986104488372803, + -0.6844664812088013, + -0.8772629499435425, + -2.056953191757202, + -3.096824884414673, + 2.1217925548553467, + 0.004722908139228821, + -1.7100458145141602, + -0.0009026676416397095, + -0.29141080379486084, + -1.7385342121124268, + -1.6228255033493042 + ], + [ + 1.9682713747024536, + -0.3308408856391907, + -0.4873868525028229, + 0.002664342522621155, + 0.182377427816391, + 2.071789026260376, + 1.8561325073242188, + 2.1711413860321045, + 0.3807043731212616, + -1.516637921333313, + -0.4644528925418854, + -0.6258189082145691, + -2.9982683658599854, + -2.980710506439209, + 1.9127426147460938, + 0.03430766612291336, + -1.6087192296981812, + 0.046004101634025574, + -0.03771457076072693, + -1.7101502418518066, + -2.1152431964874268 + ], + [ + 1.793805480003357, + -0.188386932015419, + -0.12450191378593445, + 0.22051435708999634, + 0.3403468728065491, + 1.8198211193084717, + 0.9342815279960632, + 2.1495068073272705, + 0.3071441054344177, + -1.4927021265029907, + -0.364870548248291, + -0.696351170539856, + -2.0207245349884033, + -2.5684070587158203, + 2.123406171798706, + -0.06222609058022499, + -1.3718527555465698, + -0.1457296460866928, + 0.38743728399276733, + -1.2874568700790405, + -2.3363471031188965 + ] + ], + "generated_utilities": [ + 0.09445196390151978, + 0.13109225034713745, + 0.07720557600259781, + 0.0657956600189209, + 0.08409891277551651, + 0.10617973655462265, + 0.11467918008565903, + 0.07619664818048477 + ], + "group_id": "ms-e9ed5d488afdfd6449d1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.22621813416481018, + 0.26591962575912476, + 0.22478708624839783, + 0.074407659471035, + 0.3134628236293793, + 0.19173070788383484, + 0.2438776195049286, + 0.26689577102661133, + 0.24477621912956238, + 0.24051737785339355, + 0.2738616168498993, + 0.04947046563029289, + 2.0, + 0.04723116010427475, + 0.08354459702968597, + 0.06540697813034058 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.6524851322174072, + -0.023243099451065063, + -0.05794792249798775, + 0.1783304661512375, + -0.3767814338207245, + 1.4893875122070312, + 0.0, + 1.024657964706421, + 0.3163401186466217, + -0.49971193075180054, + 0.05114656686782837, + -0.13273626565933228, + -1.9796202182769775, + 0.0, + 0.818771481513977, + -0.13705189526081085, + -0.21289467811584473, + 0.3914462625980377, + -0.6477798819541931, + -1.0717217922210693, + 0.0 + ], + [ + 2.363477945327759, + -1.042197585105896, + -0.8369066119194031, + 0.17341986298561096, + 0.6040215492248535, + 1.9536304473876953, + 2.2788424491882324, + 1.5015904903411865, + -0.07916292548179626, + -0.6816228628158569, + -1.3893373012542725, + -0.2375274896621704, + -2.5662975311279297, + -2.7615673542022705, + 1.151218295097351, + 0.29418760538101196, + -0.4583754539489746, + -0.4555256962776184, + -0.6620294451713562, + -2.534846067428589, + -1.916935682296753 + ], + [ + 2.0104916095733643, + -0.539351224899292, + 0.13553066551685333, + 0.9507300853729248, + 0.6636125445365906, + 1.5649433135986328, + 2.0495975017547607, + 2.832026958465576, + 1.1830073595046997, + -1.6508865356445312, + -0.26773035526275635, + -0.3988863527774811, + -2.8235108852386475, + -2.2897520065307617, + 1.4448039531707764, + -0.5226297378540039, + -2.1615242958068848, + 0.8214395046234131, + -0.755327045917511, + -2.571093797683716, + -1.7457375526428223 + ], + [ + 2.166736602783203, + -0.3969564735889435, + 0.5171664357185364, + 0.12789201736450195, + -0.9176173806190491, + 1.8307013511657715, + 1.834255576133728, + 1.3594005107879639, + 0.13751044869422913, + -0.9923363327980042, + -1.136932611465454, + -1.2923126220703125, + -2.445897102355957, + -2.4779889583587646, + 2.6843042373657227, + 0.10015779733657837, + -2.0766196250915527, + 0.1269409954547882, + 0.16698843240737915, + -1.2709240913391113, + -2.7113101482391357 + ], + [ + 1.0476312637329102, + -1.2484432458877563, + -0.8652145862579346, + 0.8706620335578918, + 0.6091254353523254, + 1.9634168148040771, + 2.105522394180298, + 1.2906999588012695, + -0.3275601267814636, + -1.0110361576080322, + -1.6514918804168701, + -1.376278281211853, + -2.345550060272217, + -2.4931728839874268, + 1.00270676612854, + 0.04506785422563553, + -0.9420130252838135, + -0.6248484253883362, + -1.469360589981079, + -3.059006929397583, + -3.495709180831909 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.6524851322174072, + -0.3197403848171234, + -0.07008056342601776, + 0.38692954182624817, + -0.013657404109835625, + 2.4893875122070312, + 2.3340368270874023, + 2.024657964706421, + 0.35216206312179565, + -1.4125897884368896, + -0.8262880444526672, + -0.49936625361442566, + -2.9796202182769775, + -2.9234721660614014, + 1.818771481513977, + 0.09626653790473938, + -1.2128946781158447, + 0.10123217105865479, + -0.7020833492279053, + -2.0717217922210693, + -2.5944693088531494 + ], + [ + 0.9812780022621155, + 0.2612687647342682, + 0.8081930875778198, + 0.3689844310283661, + -0.16453352570533752, + 2.8411240577697754, + 2.2401933670043945, + 1.7904633283615112, + -0.04837304353713989, + -0.6139949560165405, + -0.8549875020980835, + 0.3574656546115875, + -2.8409454822540283, + -2.1011672019958496, + 1.382948875427246, + 0.6656897068023682, + -1.5090553760528564, + 0.4825518727302551, + -0.3345044255256653, + -2.26375150680542, + -2.561906576156616 + ] + ], + "predicted_scores": [ + 10.056475639343262, + 10.005084991455078, + 9.567598342895508, + 9.539158821105957, + 9.329384803771973, + 9.105973243713379, + 9.076784133911133, + 6.996925354003906 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train0", + "selected_index": 0, + "state_hash": "5608eea5b7e99b5fed6b316f742ecd34044df87286b2338705d8472043fc4375", + "stored_base_utility": 0.22621813416481018, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09514755010604858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09514755010604858 + }, + "base_progress": 0.09514755010604858, + "base_success": false, + "base_utility": 0.09514755010604858, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0540335588157177, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0540335588157177 + }, + { + "contact_quality": null, + "progress": 0.054324448108673096, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054324448108673096 + }, + { + "contact_quality": null, + "progress": 0.03629179671406746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03629179671406746 + }, + { + "contact_quality": null, + "progress": 0.05553414300084114, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05553414300084114 + }, + { + "contact_quality": null, + "progress": 0.05642610788345337, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05642610788345337 + }, + { + "contact_quality": null, + "progress": 0.02399025298655033, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02399025298655033 + }, + { + "contact_quality": null, + "progress": 0.08578010648488998, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08578010648488998 + }, + { + "contact_quality": null, + "progress": 0.07784923166036606, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07784923166036606 + } + ], + "candidate_progress": [ + 0.0540335588157177, + 0.054324448108673096, + 0.03629179671406746, + 0.05553414300084114, + 0.05642610788345337, + 0.02399025298655033, + 0.08578010648488998, + 0.07784923166036606 + ], + "candidate_source_chart_ids": [ + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-8c0266b35fc59963795c" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-ec013396e4aff6170ee1", + "generated_tangents": [ + [ + 0.07290751487016678, + 0.042255647480487823, + 0.1707490086555481, + 0.8329348564147949, + 0.44402647018432617, + -0.3983820378780365, + 0.10629785060882568, + -0.803307294845581, + 0.4631201922893524, + 1.6467640399932861, + -0.6992282271385193, + -0.4703778624534607, + 0.10239848494529724, + 0.43653029203414917, + -0.37150177359580994, + 0.6536993980407715, + 0.33850470185279846, + 0.36333948373794556, + -0.3031330704689026, + -1.2828834056854248, + -0.7913951873779297 + ], + [ + 0.03804558515548706, + -0.0011482909321784973, + 0.08555513620376587, + 0.824670672416687, + 0.472704142332077, + -0.2663976550102234, + 0.04659953713417053, + -0.7887725830078125, + 0.4096556603908539, + 1.7044203281402588, + -0.6283047199249268, + -0.45713889598846436, + 0.042866021394729614, + 0.6094751954078674, + -0.3879110813140869, + 0.6427829265594482, + 0.2444669008255005, + 0.48182326555252075, + -0.3695583939552307, + -1.1493947505950928, + -0.7599753141403198 + ], + [ + 0.1189035177230835, + 0.07704579830169678, + 0.19802147150039673, + 0.5909813642501831, + 0.32087916135787964, + -0.533021092414856, + -0.17082196474075317, + -0.7932863235473633, + 0.4692364037036896, + 1.8881500959396362, + -0.40812453627586365, + -0.1450972855091095, + -0.4662216901779175, + 0.2754039764404297, + -0.5360662341117859, + 0.6342980861663818, + 0.4219111204147339, + 0.6539356112480164, + -0.1759810447692871, + -1.173879861831665, + -0.9458827972412109 + ], + [ + -0.17840464413166046, + 0.0069093964993953705, + 0.010284781455993652, + 0.6991860866546631, + 0.7778811454772949, + -0.2496197521686554, + 0.17770043015480042, + -0.6878794431686401, + 0.43920040130615234, + 1.8311333656311035, + -0.6852989196777344, + -0.4559307098388672, + 0.09635785222053528, + 0.3610568642616272, + -0.2797762453556061, + 0.717698872089386, + 0.10359212756156921, + 0.29468992352485657, + -0.49036896228790283, + -1.2893831729888916, + -0.7048418521881104 + ], + [ + 0.026275962591171265, + 0.012835856527090073, + 0.0007028877735137939, + 0.6091057062149048, + 0.5208897590637207, + -0.22868439555168152, + 0.04974102973937988, + -0.7718795537948608, + 0.4654478132724762, + 1.7504278421401978, + -0.4801095426082611, + -0.44256842136383057, + 0.2232321798801422, + 0.30360251665115356, + -0.33298662304878235, + 0.6408830285072327, + 0.16710898280143738, + 0.42723631858825684, + -0.24493813514709473, + -1.30322265625, + -0.6780710220336914 + ], + [ + 0.023552551865577698, + -0.07760243117809296, + 0.7463976740837097, + 1.2157301902770996, + 0.1827164590358734, + -0.3384844660758972, + -0.036058902740478516, + -0.7126957178115845, + 0.3476243019104004, + 1.75017511844635, + -0.7299929857254028, + 0.0388372540473938, + -0.007054537534713745, + 1.4830682277679443, + -0.1733592450618744, + 0.5351212024688721, + 0.08154581487178802, + 0.39984363317489624, + -0.071033775806427, + -0.722948431968689, + -0.041042447090148926 + ], + [ + -0.12444590032100677, + 0.030472688376903534, + -0.06746813654899597, + 0.7309269905090332, + -0.10249662399291992, + 0.11368206143379211, + 0.2779630422592163, + -1.0385422706604004, + 0.4566822946071625, + 1.4930098056793213, + -0.2374260127544403, + 0.09677603840827942, + 0.3349209725856781, + 0.48507100343704224, + -0.8404542207717896, + 0.6343052387237549, + 0.17045344412326813, + 0.49400824308395386, + -0.5750795006752014, + -1.7462642192840576, + 0.06725037097930908 + ], + [ + -0.27741050720214844, + -0.14250724017620087, + -0.1027999222278595, + 0.7692158222198486, + -0.019599080085754395, + 0.22784535586833954, + 0.2978934049606323, + -0.9854612350463867, + 0.47614023089408875, + 1.4697589874267578, + -0.4946310222148895, + 0.36601242423057556, + 0.36599960923194885, + 0.4570406675338745, + -0.7772973775863647, + 0.4872051477432251, + 0.004006624221801758, + 0.7636597156524658, + -0.7082494497299194, + -1.9388642311096191, + 0.7660597562789917 + ] + ], + "generated_utilities": [ + 0.0540335588157177, + 0.054324448108673096, + 0.03629179671406746, + 0.05553414300084114, + 0.05642610788345337, + 0.02399025298655033, + 0.08578010648488998, + 0.07784923166036606 + ], + "group_id": "ms-ec013396e4aff6170ee1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.010581731796264648, + 0.049839019775390625, + 0.06402494758367538, + 0.012959301471710205, + 0.11614581197500229, + 0.012509982101619244, + 0.09607907384634018, + 0.1605721116065979, + 0.1708059310913086, + 0.055588722229003906, + 0.2052522599697113, + 0.0690404623746872, + 0.11059975624084473, + 0.03047112748026848, + 0.24502670764923096, + 0.22897732257843018 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.16510963439941406, + -0.011389851570129395, + -0.16074393689632416, + 0.8547754287719727, + -0.09704409539699554, + 0.09471678733825684, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + -0.29076793789863586, + 0.0122823566198349, + 0.059903502464294434, + 0.0, + 0.13894814252853394, + 0.2961620092391968, + 0.025543689727783203, + 0.01881265640258789, + 0.052319109439849854, + -1.1815040111541748, + 0.0 + ], + [ + -0.6906574964523315, + 0.5966935753822327, + 0.1873793751001358, + 1.8547754287719727, + 0.16492335498332977, + 0.4949548840522766, + 0.624785304069519, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233, + -0.44174641370773315, + 0.866999626159668, + 1.0255436897277832, + 1.018812656402588, + 0.6033775210380554, + -2.181504011154175, + -0.4328075349330902 + ], + [ + -0.68684321641922, + 0.35380983352661133, + 0.41641929745674133, + 2.0589241981506348, + 0.20610027015209198, + 0.2780776619911194, + 0.6000017523765564, + -1.8711607456207275, + 1.71079421043396, + 2.484286069869995, + -0.12494567036628723, + 0.3767848014831543, + -0.59480881690979, + 1.3215627670288086, + -0.18650269508361816, + 0.5971667766571045, + 1.00047767162323, + 1.2905888557434082, + 0.7682754993438721, + -2.280778408050537, + -0.26339399814605713 + ], + [ + -1.0669870376586914, + 0.5963312387466431, + 0.24800772964954376, + 1.5324974060058594, + 0.049914538860321045, + 0.33535104990005493, + 0.9774697422981262, + -2.318913698196411, + 1.14731764793396, + 2.1673643589019775, + -0.3369846045970917, + -0.13161931931972504, + -0.7527784705162048, + 1.2647147178649902, + -0.5229569673538208, + 0.6999921798706055, + 0.654599666595459, + 0.9745514988899231, + 0.5995839238166809, + -2.3844704627990723, + -0.7404599189758301 + ], + [ + -0.660759449005127, + 0.560627818107605, + -0.19398994743824005, + 1.7401045560836792, + 0.006708189845085144, + 0.2270365059375763, + 0.9201371073722839, + -1.6917380094528198, + 1.8382139205932617, + 2.184737205505371, + -0.07137502729892731, + 0.13004565238952637, + -0.8376798629760742, + 1.3597201108932495, + -0.8258764147758484, + 0.521450400352478, + 1.4799309968948364, + 0.7742871046066284, + 1.007819414138794, + -2.398472785949707, + -0.7120757102966309 + ], + [ + -0.3511212170124054, + 0.6072589755058289, + 0.6547060012817383, + 2.306784152984619, + 0.03337647020816803, + 1.1676279306411743, + 0.48420602083206177, + -1.7057392597198486, + 1.7822117805480957, + 2.741852283477783, + -0.970782995223999, + 0.3188708424568176, + -0.1370975375175476, + 1.740339994430542, + 0.18519049882888794, + 0.2771400809288025, + 0.8003557920455933, + 0.8813821077346802, + 0.9829010963439941, + -1.9104408025741577, + -0.7000454664230347 + ], + [ + -1.1815872192382812, + 1.0524637699127197, + 0.5610368251800537, + 2.407832622528076, + -0.2576238512992859, + 0.1467430293560028, + 0.27881303429603577, + -1.6810011863708496, + 1.5921515226364136, + 2.929525375366211, + 0.19559642672538757, + 0.6552202105522156, + -1.3889155387878418, + 1.1560360193252563, + -0.4104887843132019, + 0.8887493014335632, + 1.283969759941101, + 0.7811828851699829, + -0.10630232095718384, + -1.4329519271850586, + -0.5646982192993164 + ], + [ + -1.4926873445510864, + 0.11535564064979553, + -0.6240940690040588, + 1.631784200668335, + 0.7930782437324524, + -0.35493338108062744, + 0.20104485750198364, + -1.5626122951507568, + 1.3110854625701904, + 3.0686309337615967, + -0.8419785499572754, + -0.0010939836502075195, + -0.49860918521881104, + 1.5949748754501343, + -0.938766598701477, + 1.762406587600708, + 0.625543475151062, + 0.20789378881454468, + 1.1513795852661133, + -2.3980860710144043, + -0.36970049142837524 + ], + [ + -1.0271471738815308, + 0.6133835911750793, + -0.3870598077774048, + 1.9299802780151367, + 0.9606215357780457, + -0.19876420497894287, + 0.29684266448020935, + -1.6164112091064453, + 1.3925114870071411, + 1.9474635124206543, + -1.2669438123703003, + -0.6821421980857849, + -0.7869906425476074, + 1.3602250814437866, + -0.14778050780296326, + 0.9938437938690186, + 0.6618988513946533, + 0.1858283281326294, + 0.8312947750091553, + -1.7043554782867432, + -0.23048581182956696 + ], + [ + -0.409736305475235, + 1.5572996139526367, + 1.0024399757385254, + 1.6385810375213623, + 0.83731609582901, + -0.1629825234413147, + -0.05952054262161255, + -1.0968945026397705, + 0.728961706161499, + 3.1370975971221924, + -0.44591856002807617, + 1.149456262588501, + -1.3439809083938599, + 1.738822340965271, + 0.4063607454299927, + 0.4372418224811554, + 1.276847004890442, + 1.4612212181091309, + 0.6917726993560791, + -1.914740800857544, + 0.5415292978286743 + ] + ], + "predicted_scores": [ + 4.424034118652344, + 3.8042120933532715, + 3.8025412559509277, + 3.6810898780822754, + 3.4984779357910156, + 3.059438467025757, + 1.8871726989746094, + -0.2703617811203003 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296/train0", + "selected_index": 0, + "state_hash": "f5a19cf07e6a6cc71c95ce01e976d34d1eeae68c8a93ecc985fd0a78cb0c1a5e", + "stored_base_utility": 0.010581731796264648, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01964247226715088, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01964247226715088 + }, + { + "contact_quality": null, + "progress": 0.015831351280212402, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015831351280212402 + }, + { + "contact_quality": null, + "progress": 0.020106056705117226, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020106056705117226 + }, + { + "contact_quality": null, + "progress": 0.01859968900680542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01859968900680542 + }, + { + "contact_quality": null, + "progress": 0.014677624218165874, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014677624218165874 + }, + { + "contact_quality": null, + "progress": 0.013308187015354633, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013308187015354633 + }, + { + "contact_quality": null, + "progress": 0.024462858214974403, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024462858214974403 + }, + { + "contact_quality": null, + "progress": 0.017958322539925575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017958322539925575 + } + ], + "candidate_progress": [ + 0.01964247226715088, + 0.015831351280212402, + 0.020106056705117226, + 0.01859968900680542, + 0.014677624218165874, + 0.013308187015354633, + 0.024462858214974403, + 0.017958322539925575 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-003a2225c0892dcda683", + "ms-0285f5815c210ac00bec", + "ms-0285f5815c210ac00bec", + "ms-be219977a9db8dbb1e58", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-ee30ebaf98622759d888", + "generated_tangents": [ + [ + 0.3891523480415344, + -0.5476866364479065, + 0.6235164999961853, + -0.20499247312545776, + 2.1759257316589355, + 1.303849220275879, + 1.1280488967895508, + 0.45156192779541016, + -0.09827959537506104, + 0.2792431116104126, + 0.09855307638645172, + 0.8370361328125, + -0.6810653805732727, + -0.23990671336650848, + 0.3469979465007782, + 0.31861793994903564, + 0.7256045937538147, + -0.5493374466896057, + 1.4329445362091064, + -1.4406800270080566, + -0.9786754846572876 + ], + [ + 0.37029337882995605, + -0.5844452977180481, + 0.6414795517921448, + -0.19069625437259674, + 2.104177951812744, + 1.303071141242981, + 1.1138598918914795, + 0.34920334815979004, + -0.10691405832767487, + 0.3129993677139282, + 0.07368563115596771, + 0.7885751128196716, + -0.6418795585632324, + -0.11100578308105469, + 0.35918915271759033, + 0.2706502676010132, + 0.6997917294502258, + -0.6293377876281738, + 1.474531650543213, + -1.5463147163391113, + -0.9765516519546509 + ], + [ + 0.4721396565437317, + -0.5913329720497131, + 0.8011988401412964, + -0.13740664720535278, + 2.0072481632232666, + 1.1481791734695435, + 0.8831367492675781, + 0.5639805793762207, + -0.036464571952819824, + 0.3808593451976776, + 0.015503205358982086, + 0.7085211277008057, + -1.0928772687911987, + -0.18293340504169464, + 0.3239908814430237, + 0.3095884323120117, + 0.6343111395835876, + -0.6279930472373962, + 1.5116267204284668, + -1.6963310241699219, + -1.1867793798446655 + ], + [ + 0.46050024032592773, + -0.5885443091392517, + 0.8316137194633484, + -0.1667107492685318, + 2.0395278930664062, + 1.1687372922897339, + 0.8199740648269653, + 0.5970370769500732, + -0.054191239178180695, + 0.44421935081481934, + -0.016800306737422943, + 0.6808668971061707, + -1.1212115287780762, + -0.24237792193889618, + 0.3289034068584442, + 0.3153565526008606, + 0.6198508739471436, + -0.5979838371276855, + 1.4770114421844482, + -1.6674690246582031, + -1.1689096689224243 + ], + [ + 0.7132302522659302, + -0.17021259665489197, + 0.49647775292396545, + -0.16039976477622986, + 1.8878815174102783, + 1.0434235334396362, + 0.8708289265632629, + 0.3052845001220703, + -0.09951447695493698, + 0.7682023048400879, + -0.0017570219933986664, + 0.37346988916397095, + -0.7110070586204529, + 0.2721971869468689, + 0.21601971983909607, + 0.2050550878047943, + 0.8422085642814636, + -0.7900719046592712, + 1.510375738143921, + -1.7179946899414062, + -0.6792943477630615 + ], + [ + 0.3379596769809723, + -0.6248862743377686, + 0.14679178595542908, + 0.23979803919792175, + 1.7175281047821045, + 1.069279432296753, + 0.9246268272399902, + 0.3281165063381195, + -0.1648385375738144, + 0.5740443468093872, + 0.18547499179840088, + 0.8467764854431152, + -0.5968696475028992, + 0.3580775260925293, + 0.23001572489738464, + 0.3481334149837494, + 0.6544145345687866, + -0.5044842958450317, + 1.3106539249420166, + -2.1478288173675537, + -0.2725996971130371 + ], + [ + 0.5897027850151062, + -0.43777740001678467, + 0.2121739387512207, + -0.2832771837711334, + 1.2150468826293945, + 0.8416855335235596, + 1.674008846282959, + 0.576818585395813, + -0.07556045800447464, + 0.7981460690498352, + -0.025754310190677643, + 0.9362323880195618, + -0.21349698305130005, + -0.30837517976760864, + 0.49233758449554443, + 0.3927343189716339, + 0.4829665422439575, + -1.0015685558319092, + 1.53029465675354, + -1.8476758003234863, + -0.7793848514556885 + ], + [ + 0.6190139055252075, + -0.6827624440193176, + 0.6732935905456543, + -0.034232839941978455, + 1.2853405475616455, + 0.5646194219589233, + 0.8531442880630493, + 0.18110041320323944, + -0.19192703068256378, + 0.3013923168182373, + 0.270673930644989, + 0.7177053689956665, + 0.03218114376068115, + 0.22037963569164276, + 0.6184743046760559, + 0.307494580745697, + 0.5671831965446472, + -1.006978154182434, + 1.4893624782562256, + -1.7233738899230957, + -0.5448800325393677 + ] + ], + "generated_utilities": [ + 0.01964247226715088, + 0.015831351280212402, + 0.020106056705117226, + 0.01859968900680542, + 0.014677624218165874, + 0.013308187015354633, + 0.024462858214974403, + 0.017958322539925575 + ], + "group_id": "ms-ee30ebaf98622759d888", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08405601978302002, + 0.09846439212560654, + 0.04595305770635605, + 2.0, + 0.08787640184164047, + 0.032481253147125244, + 0.08755072206258774, + 0.058936577290296555, + 0.10809408128261566, + 0.06723630428314209, + 0.0625009536743164, + 0.1415095031261444, + 0.08326160907745361, + 0.10944461822509766, + 0.0673408955335617, + 0.033725105226039886 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.4581542015075684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9406367540359497, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.470216989517212 + ], + [ + 1.6340367794036865, + -0.30366095900535583, + 0.2830277681350708, + -1.1627482175827026, + 3.58767032623291, + 0.6565636396408081, + 1.4321975708007812, + 0.8704822659492493, + 0.14327874779701233, + 1.0190284252166748, + 0.3078446090221405, + 0.7927982807159424, + -1.9305998086929321, + -1.7108646631240845, + 0.10729888081550598, + 0.9427294731140137, + 0.41770225763320923, + -1.7673753499984741, + 1.3209179639816284, + -2.8053033351898193, + -1.5330681800842285 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.00628507137298584, + 0.0595628023147583, + 0.39481037855148315, + -0.006641983985900879, + 1.8433213233947754, + 0.14830267429351807, + 0.0, + -0.24425631761550903, + 0.42130333185195923, + 0.41399094462394714, + 0.478841632604599, + -0.007461488246917725, + -0.5437495708465576, + 0.0, + 0.255215585231781, + -0.07331699132919312, + -0.1736500859260559, + -0.285358190536499, + 0.567956805229187, + -1.7103407382965088, + 0.0 + ], + [ + 1.0371580123901367, + -0.33467406034469604, + 0.9327473640441895, + -0.5396467447280884, + 2.1487879753112793, + 0.6294846534729004, + 0.6252679228782654, + 0.5140941143035889, + -0.20645928382873535, + -0.04528486728668213, + 0.14780870079994202, + 0.430325984954834, + -0.9293508529663086, + -1.5147639513015747, + 1.1104007959365845, + -0.05476301908493042, + 0.1755150556564331, + -1.3470268249511719, + 1.675830364227295, + -3.2944674491882324, + -1.9277915954589844 + ] + ], + "predicted_scores": [ + 8.547076225280762, + 8.304204940795898, + 7.853271484375, + 7.627488136291504, + 7.04011344909668, + 6.585521221160889, + 6.538170337677002, + 5.8520026206970215 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "636680137/train0", + "selected_index": 0, + "state_hash": "2de607a9c69005130e51ff2b551c0eeedc85c58fef796682e6527bb5f719ff7b", + "stored_base_utility": 0.08405601978302002, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.026694953441619873, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026694953441619873 + }, + "base_progress": 0.026694953441619873, + "base_success": false, + "base_utility": 0.026694953441619873, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05164949223399162, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05164949223399162 + }, + { + "contact_quality": null, + "progress": 0.058844465762376785, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058844465762376785 + }, + { + "contact_quality": null, + "progress": 0.06657024472951889, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06657024472951889 + }, + { + "contact_quality": null, + "progress": 0.0681784376502037, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0681784376502037 + }, + { + "contact_quality": null, + "progress": 0.06253000348806381, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06253000348806381 + }, + { + "contact_quality": null, + "progress": 0.07150205224752426, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07150205224752426 + }, + { + "contact_quality": null, + "progress": 0.03520200774073601, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03520200774073601 + }, + { + "contact_quality": null, + "progress": 0.06939387321472168, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06939387321472168 + } + ], + "candidate_progress": [ + 0.05164949223399162, + 0.058844465762376785, + 0.06657024472951889, + 0.0681784376502037, + 0.06253000348806381, + 0.07150205224752426, + 0.03520200774073601, + 0.06939387321472168 + ], + "candidate_source_chart_ids": [ + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-53a86042af3c0887e556", + "ms-02fd0338e80ef19ceec1", + "ms-1a4979e969aa331fb126", + "ms-02fd0338e80ef19ceec1" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-f3f1d07cfd59b823da47", + "generated_tangents": [ + [ + 1.675511360168457, + 0.26294803619384766, + -1.4619871377944946, + 0.09748843312263489, + 0.3256249725818634, + -1.1040034294128418, + -0.6247721314430237, + 0.3977947533130646, + 0.4779442548751831, + -0.24419444799423218, + -1.0073573589324951, + 0.9466355443000793, + -0.7505614757537842, + 0.035916902124881744, + 0.5673705339431763, + -0.12787234783172607, + 0.2507711946964264, + -1.643969178199768, + 0.35970455408096313, + 0.5917714834213257, + 1.1986117362976074 + ], + [ + 1.560312032699585, + 0.2568896412849426, + -1.2574419975280762, + 0.07654470205307007, + 0.06049570441246033, + -1.1585447788238525, + -0.3848813772201538, + 0.48328763246536255, + 0.4009150564670563, + -0.10568415373563766, + -1.0240111351013184, + 0.8088661432266235, + -0.5029674172401428, + -0.18004967272281647, + 0.36323776841163635, + -0.11226554214954376, + 0.45954424142837524, + -1.7849628925323486, + 0.16309618949890137, + 0.4507681131362915, + 1.1250041723251343 + ], + [ + 1.6391892433166504, + 0.21951808035373688, + -1.2775074243545532, + 0.15611949563026428, + -0.13830643892288208, + -1.5244075059890747, + -0.721285879611969, + 0.47953909635543823, + 0.4167535603046417, + -0.0974070206284523, + -0.986589789390564, + 0.8505012392997742, + -0.30110645294189453, + -0.1401648372411728, + 0.38734349608421326, + -0.07386188209056854, + 0.30531561374664307, + -1.8018094301223755, + 0.0659090131521225, + 0.31123441457748413, + 1.0316061973571777 + ], + [ + 1.7891749143600464, + 0.1960882842540741, + -1.3303453922271729, + 0.2006845474243164, + -0.07155710458755493, + -1.420583724975586, + -0.4426800608634949, + 0.5741879940032959, + 0.346225768327713, + -0.13232919573783875, + -0.9523600339889526, + 0.5992389917373657, + -0.2720524072647095, + 0.03333738446235657, + 0.4209025204181671, + -0.07059133797883987, + 0.2101317048072815, + -1.7509722709655762, + 0.14263051748275757, + 0.2756538987159729, + 0.9637851715087891 + ], + [ + 1.5337421894073486, + 0.10512694716453552, + -1.608415961265564, + 0.09830436110496521, + -0.13416674733161926, + -1.8996893167495728, + -0.7178241610527039, + 0.5179807543754578, + 0.07927774637937546, + -0.0958782508969307, + -0.7517118453979492, + 0.5402031540870667, + -0.8901242017745972, + -0.42635107040405273, + 0.5358250737190247, + -0.18648803234100342, + 0.4157024919986725, + -1.4566999673843384, + 0.3473930060863495, + 0.42524397373199463, + 0.6094003915786743 + ], + [ + 1.5719108581542969, + 0.3079289197921753, + -1.0445280075073242, + 0.3755629360675812, + 0.2826114296913147, + -1.1220356225967407, + -0.13396495580673218, + 0.476203054189682, + 0.10016392916440964, + -0.13018935918807983, + -1.5082701444625854, + 0.618819534778595, + -0.4119489789009094, + -0.49988651275634766, + 0.2877805829048157, + -0.03060813993215561, + 0.5619557499885559, + -1.9755491018295288, + 0.0077082812786102295, + -0.4956511855125427, + 0.6007959246635437 + ], + [ + 1.664137840270996, + -0.013941869139671326, + -1.6251887083053589, + 0.08530622720718384, + -0.3102415204048157, + -1.9219149351119995, + -0.34388530254364014, + 0.5052124261856079, + 0.14826075732707977, + 0.05399750918149948, + -0.842513918876648, + 0.7535050511360168, + -0.6396822333335876, + -0.2530655264854431, + 0.3765254616737366, + -0.25213462114334106, + 0.14109429717063904, + -1.3795732259750366, + 0.3979029655456543, + 0.18450382351875305, + 0.26160329580307007 + ], + [ + 1.533952236175537, + 0.2404658943414688, + -0.9675089120864868, + 0.45229610800743103, + 0.14575034379959106, + -1.1170729398727417, + -0.10040819644927979, + 0.28900712728500366, + 0.07777386903762817, + -0.16432088613510132, + -1.539223074913025, + 0.5006495714187622, + -0.47292834520339966, + -0.4586854577064514, + 0.12763994932174683, + 0.029766645282506943, + 0.513634204864502, + -1.9874070882797241, + -0.11878997087478638, + -0.5150457620620728, + 0.5731385946273804 + ] + ], + "generated_utilities": [ + 0.05164949223399162, + 0.058844465762376785, + 0.06657024472951889, + 0.0681784376502037, + 0.06253000348806381, + 0.07150205224752426, + 0.03520200774073601, + 0.06939387321472168 + ], + "group_id": "ms-f3f1d07cfd59b823da47", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07767709344625473, + 0.07153411954641342, + 0.08480638265609741, + 0.06087514013051987, + 0.15252648293972015, + 0.0776997059583664, + 0.1476559340953827, + 0.17124073207378387, + 0.12327007949352264, + 0.12037092447280884, + 0.08562581241130829, + 0.09917601197957993, + 0.07137986272573471, + 0.039922356605529785, + 0.04496842622756958, + 0.05561637878417969 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.9664052724838257, + 0.20301294326782227, + -1.1196715831756592, + -0.17260856926441193, + -0.5341777801513672, + -2.678128242492676, + -2.431448459625244, + 1.5603530406951904, + 0.03585217520594597, + -0.3520800471305847, + -2.473386287689209, + 1.0162299871444702, + -0.7868502736091614, + -0.9102090001106262, + 0.9442894458770752, + -0.41473180055618286, + 0.7349075675010681, + -0.8863289952278137, + 0.8796874284744263, + 0.028114937245845795, + 0.24920660257339478 + ], + [ + 1.900488257408142, + -0.00838443636894226, + -1.2311820983886719, + -0.361179918050766, + -0.3987593352794647, + -2.551008462905884, + -2.701385021209717, + 1.7117550373077393, + -0.20423506200313568, + -0.3429180979728699, + -2.6584055423736572, + 0.8018259406089783, + -0.856230616569519, + -0.901038408279419, + 1.2131764888763428, + -0.6892215609550476, + 0.6726197600364685, + -0.7798839211463928, + 0.6868253946304321, + 0.07731680572032928, + 0.30107203125953674 + ], + [ + 1.6257587671279907, + 0.275219202041626, + -0.7838636636734009, + -0.5118123292922974, + -0.3429456353187561, + -2.342111349105835, + -2.7158243656158447, + 1.7038744688034058, + -0.1860528439283371, + -0.11617843806743622, + -2.1894750595092773, + 1.2306283712387085, + -0.8203369379043579, + -0.5157363414764404, + 0.7951346635818481, + -0.464976966381073, + 0.8823708295822144, + -0.7671404480934143, + 0.9700374007225037, + 0.21184596419334412, + -0.11244213581085205 + ] + ], + "predicted_scores": [ + 2.532451629638672, + 2.415501594543457, + 1.4083845615386963, + 0.6119946241378784, + 0.362302303314209, + -0.1249924898147583, + -0.4573264718055725, + -0.7109801173210144 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train0", + "selected_index": 0, + "state_hash": "3f477a29208b02558b99bd0f3305d0f1845c277d32e0b5afdafa71012673383b", + "stored_base_utility": 0.07767709344625473, + "task_id": "PullCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07080724835395813, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07080724835395813 + }, + "base_progress": 0.07080724835395813, + "base_success": false, + "base_utility": 0.07080724835395813, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07753174006938934, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07753174006938934 + }, + { + "contact_quality": null, + "progress": 0.08595690131187439, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08595690131187439 + }, + { + "contact_quality": null, + "progress": 0.07273110747337341, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07273110747337341 + }, + { + "contact_quality": null, + "progress": 0.05202282965183258, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05202282965183258 + }, + { + "contact_quality": null, + "progress": 0.07089017331600189, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07089017331600189 + }, + { + "contact_quality": null, + "progress": 0.07804974913597107, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07804974913597107 + }, + { + "contact_quality": null, + "progress": 0.05986259877681732, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05986259877681732 + }, + { + "contact_quality": null, + "progress": 0.07560504972934723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07560504972934723 + } + ], + "candidate_progress": [ + 0.07753174006938934, + 0.08595690131187439, + 0.07273110747337341, + 0.05202282965183258, + 0.07089017331600189, + 0.07804974913597107, + 0.05986259877681732, + 0.07560504972934723 + ], + "candidate_source_chart_ids": [ + "ms-bfe867e8d518b9f8333a", + "ms-bfe867e8d518b9f8333a", + "ms-8278abbc47a6efcb5b68", + "ms-dab80e1bb7987577e33d", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-081952a63087fc8f83cd", + "generated_tangents": [ + [ + -0.08413964509963989, + 0.03917499631643295, + 1.2771568298339844, + -0.5833404660224915, + -1.2979159355163574, + -0.041353851556777954, + -0.3111611008644104, + -0.05563884973526001, + -0.12758560478687286, + 0.067821204662323, + 0.06210175156593323, + -1.336356520652771, + 0.2173217236995697, + -0.1255672425031662, + -0.012791294604539871, + -0.04858876392245293, + -0.0979764312505722, + -0.056227996945381165, + -0.8472711443901062, + -0.14808081090450287, + -0.3216269612312317 + ], + [ + -0.07111789286136627, + 0.03896534442901611, + 1.2926208972930908, + -0.5241761207580566, + -1.3137810230255127, + -0.09370563924312592, + -0.23570245504379272, + -0.03557869791984558, + -0.12659049034118652, + 0.10744508355855942, + -0.004305452108383179, + -1.3119266033172607, + 0.11784610152244568, + -0.10802482068538666, + 0.010881159454584122, + -0.04123491793870926, + -0.06381000578403473, + 0.006138816475868225, + -0.8246253132820129, + -0.1516846865415573, + -0.24148374795913696 + ], + [ + -0.053984493017196655, + -0.00562024861574173, + 1.1701840162277222, + -0.7135963439941406, + -1.5532948970794678, + -0.23730385303497314, + -0.3853493332862854, + 0.05915568768978119, + -0.2242712825536728, + 0.07229231297969818, + -0.012854605913162231, + -1.1027201414108276, + 0.18021747469902039, + -0.16772504150867462, + 0.03970526158809662, + -0.08681954443454742, + -0.11705993115901947, + 0.08885939419269562, + -0.647961437702179, + -0.021091073751449585, + -0.2371947467327118 + ], + [ + -0.04518130421638489, + 0.0816914290189743, + 1.1488096714019775, + -0.7166935205459595, + -1.2783079147338867, + -0.350945383310318, + -0.22076040506362915, + 0.030124112963676453, + -0.17097710072994232, + 0.1264670193195343, + 0.0489087849855423, + -1.3729066848754883, + 0.2075645625591278, + -0.14103834331035614, + 0.06115962937474251, + -0.09586217999458313, + -0.17797888815402985, + -0.0452701598405838, + -0.8797468543052673, + -0.10645312070846558, + -0.3185126483440399 + ], + [ + 0.11026055365800858, + -0.0033897459506988525, + 1.1837600469589233, + -0.6096736192703247, + -1.4959893226623535, + -0.28977492451667786, + -0.4396909475326538, + 0.0747135728597641, + -0.1966264396905899, + 0.12885478138923645, + -0.011177584528923035, + -1.1425373554229736, + 0.4872763454914093, + -0.2498861402273178, + 0.08118508011102676, + -0.09010106325149536, + -0.13109330832958221, + 0.04437825083732605, + -0.6183353662490845, + -0.03854522109031677, + -0.34984156489372253 + ], + [ + -0.1432105451822281, + -0.02981628105044365, + 1.0101110935211182, + -0.5665549039840698, + -1.189751148223877, + -0.12700220942497253, + -0.34022319316864014, + 0.07882219552993774, + -0.311112642288208, + 0.10574111342430115, + 0.031146898865699768, + -1.334547996520996, + 0.5379418134689331, + -0.01745852828025818, + 0.055465541779994965, + -0.2213810831308365, + -0.4261760711669922, + -0.035221174359321594, + -0.9912137389183044, + -0.1770738661289215, + -0.30926287174224854 + ], + [ + 0.15350456535816193, + -0.10617871582508087, + 0.9539602994918823, + -0.47117069363594055, + -1.4828500747680664, + -0.4168761074542999, + -0.36822497844696045, + 0.04427656531333923, + -0.3682369291782379, + 0.10710634291172028, + 0.09130032360553741, + -1.3806426525115967, + 0.5532610416412354, + -0.3403542637825012, + 0.12044143676757812, + -0.17413827776908875, + -0.4426705837249756, + -0.04571153223514557, + -0.885425865650177, + -0.11885681748390198, + -0.21574485301971436 + ], + [ + 0.10382343828678131, + 0.007059935480356216, + 0.9621686935424805, + -0.6107028126716614, + -1.3704118728637695, + -0.447879821062088, + -0.5862154364585876, + 0.26559633016586304, + -0.3464195430278778, + 0.02941250056028366, + 0.20671603083610535, + -1.2919561862945557, + 0.472435861825943, + -0.25946474075317383, + 0.14030812680721283, + -0.051595766097307205, + -0.31563740968704224, + 0.01749001443386078, + -0.9021003842353821, + -0.042577385902404785, + -0.33679935336112976 + ] + ], + "generated_utilities": [ + 0.07753174006938934, + 0.08595690131187439, + 0.07273110747337341, + 0.05202282965183258, + 0.07089017331600189, + 0.07804974913597107, + 0.05986259877681732, + 0.07560504972934723 + ], + "group_id": "ms-081952a63087fc8f83cd", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0006054937839508057, + 2.0, + 2.0, + 2.0, + 0.09041815996170044, + 0.03755016624927521, + 0.10023769736289978, + 0.08501018583774567, + 0.10316208004951477, + 0.13556814193725586, + 0.0687420666217804, + 0.16420090198516846, + 0.07440806925296783, + 0.03177754580974579, + 0.05512912571430206, + 0.05822136998176575 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05530714988708496, + -0.0015276968479156494, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.028481781482696533, + 0.0, + 0.07634823024272919, + 0.10118435323238373, + 0.01472395658493042, + -0.021184463053941727, + -1.3624117374420166, + 0.06702810525894165, + 0.0, + 0.064424529671669, + -0.13110561668872833, + -0.04677841067314148, + -0.0666094422340393, + -0.8611308336257935, + -0.02982822060585022, + 0.0 + ], + [ + -0.13999104499816895, + -0.16139763593673706, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.17549771070480347, + 0.0, + -0.15019088983535767, + 0.2207401990890503, + -0.06357991695404053, + 0.10177183151245117, + -1.3624117374420166, + -0.05549207329750061, + 0.0, + -0.1271420419216156, + -0.057942673563957214, + -0.279840350151062, + -0.11312717199325562, + -0.8611308336257935, + 0.1586582064628601, + 0.0 + ], + [ + 0.2317524552345276, + -0.2474955916404724, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.2997032403945923, + 0.0, + -0.3180278241634369, + 0.1991673707962036, + 0.26833486557006836, + 0.005899060517549515, + -1.3624117374420166, + 0.032782405614852905, + 0.0, + 0.17653466761112213, + 0.5550464987754822, + 0.13015127182006836, + -0.11312717199325562, + -0.8611308336257935, + 0.020897403359413147, + 0.0 + ], + [ + -0.7016198635101318, + 0.312343955039978, + 2.582733154296875, + -1.5283455848693848, + -2.530975103378296, + 0.7002967596054077, + 2.0224196910858154, + -0.031223494559526443, + -0.07945594191551208, + 0.7471928000450134, + -0.05505431815981865, + -2.3624117374420166, + 0.4337144196033478, + 1.385794997215271, + -0.18244768679141998, + -0.22281835973262787, + -0.5003347992897034, + 0.8868728280067444, + -1.8611308336257935, + 0.19736243784427643, + 0.2646070420742035 + ], + [ + -0.41503679752349854, + 0.3199126422405243, + 2.3341803550720215, + -1.5432449579238892, + -2.6036009788513184, + 0.8186125755310059, + 1.889835238456726, + -0.20609386265277863, + 0.0827421247959137, + 0.8381537795066833, + -0.0944519191980362, + -2.4805798530578613, + 0.3916063606739044, + 1.2392640113830566, + -0.2266693413257599, + -0.33516526222229004, + -0.3536234498023987, + 0.9749420881271362, + -2.1439476013183594, + 0.3261662721633911, + 0.13869160413742065 + ], + [ + -0.6778035759925842, + 0.3674989640712738, + 2.3216195106506348, + -1.6129508018493652, + -2.3233790397644043, + 0.37060582637786865, + 2.3460428714752197, + 0.2269098311662674, + -0.269074022769928, + 0.9810001254081726, + 0.023842941969633102, + -2.478595733642578, + 0.46739575266838074, + 1.1671457290649414, + -0.41542381048202515, + 0.030661001801490784, + -0.48762595653533936, + 0.7803086638450623, + -1.7778995037078857, + 0.10219369828701019, + 0.08611755073070526 + ], + [ + -1.0292376279830933, + 0.05653715133666992, + 3.017642021179199, + -1.2965388298034668, + -2.1434991359710693, + 0.8112732172012329, + 2.3553943634033203, + 0.14745590090751648, + -0.42473679780960083, + 0.8983602523803711, + -0.5389888286590576, + -1.9595260620117188, + 0.8665074110031128, + 1.7057592868804932, + -0.36306142807006836, + -0.6694610714912415, + -0.3699086308479309, + 0.7089512944221497, + -2.2496888637542725, + 0.1759258657693863, + 0.13106243312358856 + ], + [ + -0.2738665044307709, + 0.13772070407867432, + 2.3146424293518066, + -1.9326081275939941, + -2.760838508605957, + 0.10491085052490234, + 1.9823331832885742, + 0.3494338393211365, + 0.3550114631652832, + 0.7471076846122742, + -0.06264704465866089, + -2.2210469245910645, + 0.5386381149291992, + 1.431454062461853, + 0.1620958298444748, + -0.00858105719089508, + -0.8883554935455322, + 1.1764272451400757, + -1.8280467987060547, + 0.30690017342567444, + -0.32479730248451233 + ], + [ + -0.8548018932342529, + -0.2078946828842163, + 2.740286111831665, + -1.6552648544311523, + -2.470123291015625, + 1.0607044696807861, + 2.4267592430114746, + -0.16535070538520813, + -0.2640931308269501, + 0.2447068691253662, + 0.5616287589073181, + -2.808340311050415, + 1.0820176601409912, + 1.2134929895401, + 0.2653927206993103, + -0.454708456993103, + 0.03163468837738037, + 0.8172736763954163, + -1.8861355781555176, + 0.45078086853027344, + 0.29641321301460266 + ], + [ + -0.7025277018547058, + 0.5556451678276062, + 2.3374006748199463, + -1.9256110191345215, + -1.7495018243789673, + 0.9721168279647827, + 1.8326293230056763, + 0.5355191826820374, + -0.5788277983665466, + 0.19061005115509033, + -0.6322742700576782, + -3.031383514404297, + -0.1651831567287445, + 0.8584551215171814, + 0.2280312329530716, + 0.27221786975860596, + -0.3728860020637512, + 1.102063775062561, + -2.5995376110076904, + 0.7578867673873901, + -0.3784477412700653 + ], + [ + -7.212162017822266e-06, + 0.552659273147583, + 2.0397679805755615, + -1.8501746654510498, + -2.5457160472869873, + 1.0514975786209106, + 2.5270917415618896, + -0.7367458343505859, + 0.7676748037338257, + 0.7428221702575684, + -0.538831353187561, + -2.7423243522644043, + 0.07969537377357483, + 1.0069079399108887, + -0.8100140690803528, + -0.9174093008041382, + -0.9411231875419617, + 1.0430617332458496, + -1.4683198928833008, + 1.0334573984146118, + 0.4721405506134033 + ], + [ + -0.41159626841545105, + 0.41878798604011536, + 1.6363914012908936, + -2.4960880279541016, + -3.049954891204834, + -0.09567022323608398, + 2.3269245624542236, + 0.11159682273864746, + -0.7885257005691528, + 0.44582536816596985, + -0.47802892327308655, + -1.5574463605880737, + 1.0153859853744507, + 2.258706569671631, + -0.36591836810112, + 0.49115514755249023, + -0.9523675441741943, + 0.5892280340194702, + -2.1512646675109863, + -0.7744256854057312, + 0.592117428779602 + ], + [ + 0.1343594193458557, + 0.45838433504104614, + 2.6735801696777344, + -2.005676746368408, + -3.340463161468506, + 1.0152086019515991, + 1.429718255996704, + 0.7929191589355469, + -0.2598758339881897, + 1.6483545303344727, + -0.7411167025566101, + -2.8978958129882812, + 1.3759692907333374, + 1.4385837316513062, + 0.2961248755455017, + -0.042098239064216614, + -0.6380948424339294, + 1.0523303747177124, + -2.1783251762390137, + -0.045405834913253784, + 0.7960561513900757 + ] + ], + "predicted_scores": [ + 9.857619285583496, + 9.777491569519043, + 9.474225997924805, + 9.17353343963623, + 9.032880783081055, + 8.81174087524414, + 8.667104721069336, + 8.220312118530273 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2141802816/train0", + "selected_index": 0, + "state_hash": "d1cf06bba4675ad6a6942dc4f6fbade890fbf21b5c3068f8bbe30cfc60c54d9b", + "stored_base_utility": 0.0006054937839508057, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-2ba02dcefa15a9371706" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-121bc2e6676c1ac10911", + "generated_tangents": [ + [ + -0.318436861038208, + -0.24442674219608307, + -0.6643575429916382, + -0.1913587898015976, + -1.2045369148254395, + -0.887955904006958, + 1.3046760559082031, + 0.26366913318634033, + 0.2270069718360901, + -0.6376742720603943, + 0.27735474705696106, + 0.12393875420093536, + -0.2746342122554779, + 0.2749885618686676, + 0.07337480783462524, + -0.08053277432918549, + -0.23778875172138214, + 0.6927629709243774, + -0.4625341296195984, + -0.5742101669311523, + 0.22084379196166992 + ], + [ + 0.13696816563606262, + -0.0004087015986442566, + -0.24295303225517273, + -0.5239930152893066, + -0.5989259481430054, + -0.3572954833507538, + 1.5271000862121582, + 0.2594227194786072, + 0.17775587737560272, + -0.2258957028388977, + 0.21136316657066345, + -0.05476680397987366, + -0.3642699420452118, + 1.1860625743865967, + 0.05777456611394882, + -0.024060523137450218, + -0.20670513808727264, + 0.5988383889198303, + -0.0570034384727478, + -0.08882161974906921, + 0.37320005893707275 + ], + [ + -0.25128161907196045, + -0.3132287859916687, + -0.6979422569274902, + -0.3270675241947174, + -1.327434778213501, + -0.15659767389297485, + 1.2446153163909912, + 0.5223498940467834, + 0.14893178641796112, + -0.5016502737998962, + 0.32958194613456726, + -0.14257335662841797, + -0.07503217458724976, + 0.3861290216445923, + -0.01677720621228218, + -0.13776861131191254, + -0.3512023687362671, + 0.7592557072639465, + 0.08203499019145966, + -0.45473146438598633, + -0.15495941042900085 + ], + [ + -0.2094193547964096, + -0.2954549193382263, + -0.8341259956359863, + -0.7916555404663086, + -0.8491094708442688, + -0.5231232643127441, + 1.3542044162750244, + 0.42853909730911255, + 0.12180132418870926, + -0.44825518131256104, + 0.2387983500957489, + -0.12857186794281006, + 0.1692427694797516, + 0.27743467688560486, + 0.019608058035373688, + 0.19956675171852112, + -0.10563810169696808, + 0.5816810727119446, + 0.1299280971288681, + -0.6717466711997986, + 1.3120108842849731 + ], + [ + -0.24599553644657135, + -0.23191988468170166, + -0.6010323762893677, + -0.30621418356895447, + -1.1158783435821533, + -0.5426607131958008, + 1.2219572067260742, + 0.40301400423049927, + 0.0556047260761261, + -0.5179594159126282, + 0.34780120849609375, + -0.04690465331077576, + -0.40648210048675537, + 0.38549935817718506, + -0.05463051423430443, + -0.006608236115425825, + -0.24081142246723175, + 0.6482144594192505, + 0.11671988666057587, + -0.17064699530601501, + 0.6720317006111145 + ], + [ + -0.31878381967544556, + -0.12913677096366882, + -0.6215265989303589, + -0.24297234416007996, + -0.9468169808387756, + -0.5165524482727051, + 1.264082431793213, + 0.34228554368019104, + 0.0017401427030563354, + -0.599934458732605, + 0.3516547977924347, + 0.22312335669994354, + -0.43553489446640015, + 0.4596540331840515, + -0.016192656010389328, + -0.06471820175647736, + -0.2596778869628906, + 0.7153933048248291, + -0.12586182355880737, + -0.1682382971048355, + 0.8540645837783813 + ], + [ + -0.21072883903980255, + -0.20984023809432983, + -0.7151176929473877, + -0.422762393951416, + -0.9866161942481995, + -0.8116936683654785, + 1.283542275428772, + 0.33878257870674133, + 0.17407064139842987, + -0.4530002474784851, + 0.3152390122413635, + 0.13520950078964233, + -0.5315513610839844, + 0.12147153913974762, + -0.008829396218061447, + -0.08355154097080231, + -0.3370814323425293, + 0.738246500492096, + 0.08903366327285767, + -0.27276211977005005, + 0.1756531000137329 + ], + [ + -0.1948457807302475, + -0.2748200595378876, + -0.7478799819946289, + -0.5438199043273926, + -1.2477014064788818, + -0.44240716099739075, + 1.1769423484802246, + 0.2894083261489868, + -0.015147244557738304, + -0.15536320209503174, + 0.3076595366001129, + -0.4715762138366699, + -0.42085719108581543, + 0.38623708486557007, + -0.3163565695285797, + -0.100701242685318, + -0.5614150166511536, + 0.5407496094703674, + 0.21292541921138763, + 0.2552551031112671, + 0.5606482028961182 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-121bc2e6676c1ac10911", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15086030960083008, + 0.08354458212852478, + 2.0, + 0.1453719437122345, + 0.19646552205085754, + 0.10343040525913239, + 0.1443832516670227, + 0.13976123929023743, + 0.17431864142417908, + 2.0, + 0.12949708104133606, + 2.0, + 2.0, + 0.11397694051265717, + 0.1271887570619583, + 0.025320395827293396 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.39665424823760986, + 0.08244594186544418, + -0.026901081204414368, + -0.959688663482666, + -1.2648048400878906, + 0.05568990111351013, + 0.0, + -0.048488348722457886, + 0.03576555848121643, + -0.012020409107208252, + 0.08576416969299316, + -0.25812816619873047, + 0.03589746356010437, + 0.0, + 0.0016258209943771362, + -0.04025750607252121, + 0.010670006275177002, + 0.5142596960067749, + 0.06220144033432007, + 0.07906517386436462, + 0.0 + ], + [ + -2.15720796585083, + 0.13025060296058655, + 0.41441529989242554, + -1.4498568773269653, + -2.6386165618896484, + 0.5572659969329834, + 0.7093658447265625, + -1.0835764408111572, + -0.896172285079956, + 0.07744038105010986, + 1.1224509477615356, + -1.6321502923965454, + 0.877888560295105, + 0.4978562295436859, + 0.5774499177932739, + 0.8093571662902832, + -0.8521628379821777, + 2.054164171218872, + -0.6021197438240051, + 1.0350128412246704, + 0.823830246925354 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.39665424823760986, + -0.2284793257713318, + -0.027931194752454758, + -0.959688663482666, + -1.2648048400878906, + -0.17197783291339874, + 0.0, + -0.06679429113864899, + 0.1646350920200348, + -0.11601293087005615, + 0.08576416969299316, + -0.25812816619873047, + 0.1479925513267517, + 0.0, + -0.04833713173866272, + 0.22201494872570038, + -0.16177666187286377, + 0.5142596960067749, + -0.10178101062774658, + 0.21631890535354614, + 0.0 + ], + [ + -0.8824495077133179, + 0.0034109503030776978, + -0.5904110074043274, + -1.400491714477539, + -2.4447474479675293, + -0.00789790228009224, + 1.6208159923553467, + 0.26819390058517456, + -0.23575741052627563, + -0.7184416651725769, + 0.9072303771972656, + -1.2077033519744873, + -0.21496516466140747, + 0.8258882761001587, + -0.4575563669204712, + -0.3655202388763428, + -0.5908474922180176, + 2.0210399627685547, + -1.2132787704467773, + 0.9719676971435547, + 0.24625355005264282 + ], + [ + -0.6953714489936829, + 0.4026735723018646, + 0.6648998856544495, + -1.356613278388977, + -2.953742742538452, + 0.037713564932346344, + 1.53464937210083, + 0.027994811534881592, + 0.03435474634170532, + -1.1939163208007812, + 0.9877840280532837, + -1.180513620376587, + 0.4064505994319916, + 0.5743880867958069, + 0.1558205783367157, + 0.3796072006225586, + -0.6676555275917053, + 1.4665511846542358, + 0.04170459508895874, + 0.17645099759101868, + 0.2388516366481781 + ], + [ + -1.4095232486724854, + -0.06358731538057327, + -0.7774888873100281, + -2.806776285171509, + -2.44041109085083, + -0.8258867859840393, + 2.383592128753662, + -0.093393474817276, + -0.24701429903507233, + -0.5286431312561035, + 0.7741218209266663, + -2.111828565597534, + -0.31807881593704224, + 1.3003538846969604, + -0.9306488037109375, + -0.8708829283714294, + -0.3694953918457031, + 2.169797420501709, + -0.7153271436691284, + 0.029519766569137573, + -0.5629553198814392 + ] + ], + "predicted_scores": [ + 4.049079895019531, + 3.6869189739227295, + 3.434694528579712, + 2.4521331787109375, + 2.2561216354370117, + 2.2208027839660645, + 2.131687641143799, + 1.2841386795043945 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264/train0", + "selected_index": 0, + "state_hash": "d1280a9e9362f8da3290de84fdd0f0b78475d470cd0230e590ccaf8f5428548a", + "stored_base_utility": 0.15086030960083008, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-22f153f0d4fa656018e6", + "generated_tangents": [ + [ + -0.5488069653511047, + -0.12914320826530457, + -0.6730979681015015, + -1.359136700630188, + -1.8728837966918945, + -0.8446304798126221, + -0.013707607984542847, + 0.10725326836109161, + -0.11694855988025665, + -0.07521618157625198, + 0.20123475790023804, + -1.3166704177856445, + -0.21968255937099457, + 0.005218483507633209, + -0.2585197985172272, + 0.1388433575630188, + -0.07499019801616669, + -0.01642654836177826, + -0.47000378370285034, + 0.2271711230278015, + 0.0815325528383255 + ], + [ + -0.5242918729782104, + -0.12676024436950684, + -0.3443840444087982, + -1.167953610420227, + -1.8117244243621826, + -0.3880108892917633, + -0.34209388494491577, + -0.05148926377296448, + -0.014074873179197311, + -0.0524558499455452, + 0.10325033217668533, + -1.1297495365142822, + 0.027836233377456665, + 0.0595313236117363, + -0.1211487203836441, + -0.029480289667844772, + -0.35129958391189575, + 0.005054488778114319, + -0.5940192341804504, + -0.3171018362045288, + -0.09436988830566406 + ], + [ + -0.6183275580406189, + -0.14431670308113098, + -0.2307959496974945, + -1.1094334125518799, + -1.853135108947754, + -0.6286224126815796, + -0.3655887842178345, + -0.057424843311309814, + -0.05028947442770004, + -0.09524308890104294, + 0.04739289730787277, + -1.1648361682891846, + -0.06320396065711975, + 0.024961866438388824, + 0.05146493762731552, + 0.038867440074682236, + -0.3965842127799988, + -0.01534724235534668, + -0.5904173254966736, + -0.24465589225292206, + -0.3948509991168976 + ], + [ + -0.4657665491104126, + -0.07714395225048065, + -0.24930688738822937, + -1.0198901891708374, + -1.9631257057189941, + -0.4246378242969513, + -0.30768728256225586, + 0.022464826703071594, + -0.05418386310338974, + -0.07536391168832779, + 0.06656061857938766, + -1.1389085054397583, + -0.043238699436187744, + -0.1611955612897873, + -0.07273463904857635, + 0.018092108890414238, + -0.3483579158782959, + -0.1254226416349411, + -0.5164749026298523, + -0.14282383024692535, + -0.2232438325881958 + ], + [ + -0.36254262924194336, + -0.07197297364473343, + -0.2658368647098541, + -1.0821423530578613, + -1.8539245128631592, + -0.5465748310089111, + -0.18021124601364136, + 0.070470429956913, + -0.06224729120731354, + -0.11129259318113327, + 0.1406990885734558, + -1.0396536588668823, + -0.13670280575752258, + -0.17410142719745636, + -0.1736098825931549, + 0.02032379060983658, + -0.2861585021018982, + 0.04271472990512848, + -0.49530476331710815, + 0.033352822065353394, + -0.09253549575805664 + ], + [ + -0.4510018229484558, + -0.0446300283074379, + -0.2147749960422516, + -1.1680705547332764, + -1.8639743328094482, + -0.5043494701385498, + -0.3151446580886841, + 0.017371058464050293, + -0.04363667964935303, + -0.15871462225914001, + 0.13338208198547363, + -0.9481333494186401, + 0.10634538531303406, + -0.1777450293302536, + 0.031976766884326935, + -0.009491289034485817, + -0.3257819414138794, + -0.037767812609672546, + -0.535429835319519, + -0.11300502717494965, + -0.41360774636268616 + ], + [ + -0.6611120104789734, + 0.1196136474609375, + 0.00514674186706543, + -1.0458996295928955, + -1.8806705474853516, + -0.6966314315795898, + -0.07055884599685669, + -0.022101834416389465, + -0.000422540120780468, + -0.00888977199792862, + 0.07196106761693954, + -0.9307974576950073, + 0.33626559376716614, + -0.3816136121749878, + 0.09100475162267685, + 0.09832784533500671, + -0.38475584983825684, + -0.11374054849147797, + -0.5637039542198181, + -0.1436951756477356, + 0.009382277727127075 + ], + [ + -0.2370283454656601, + 0.04556134343147278, + 0.2720962464809418, + -0.6675891876220703, + -1.6301751136779785, + -0.39291802048683167, + -0.0955129861831665, + 0.04320399463176727, + -0.013827163726091385, + 0.011234015226364136, + -0.015867426991462708, + -1.0916461944580078, + 0.17198309302330017, + 0.15019218623638153, + 0.005726959556341171, + 0.042269546538591385, + -0.19741766154766083, + -0.07113371789455414, + -0.46631568670272827, + 0.133294939994812, + -0.4270465672016144 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-22f153f0d4fa656018e6", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 2.60770320892334e-06, + 3.129243850708008e-07, + 6.109476089477539e-07, + 2.0, + 0.20781123638153076, + 2.0, + 0.16136802732944489, + 0.15920132398605347, + 0.16996727883815765, + 0.14405608177185059, + 0.15874098241329193, + 0.08526891469955444, + 0.0828208178281784, + 2.0, + 0.034265756607055664, + 2.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.3937166929244995, + 0.17073306441307068, + -0.5225132703781128, + -0.8479964733123779, + -1.713785171508789, + -0.19215916097164154, + 0.0, + 0.8004336953163147, + 0.4925512969493866, + 0.3161412179470062, + 0.10414126515388489, + -0.9857386350631714, + -0.4383167624473572, + 0.0, + 0.05412057042121887, + 0.021832358092069626, + 0.5262433290481567, + -0.24631118774414062, + -0.20008409023284912, + -0.04257836565375328, + 0.0 + ], + [ + -1.3937166929244995, + 0.2384568303823471, + 0.3933030962944031, + -1.847996473312378, + -2.713785171508789, + 0.20405007898807526, + 1.2454378604888916, + 0.0006654821336269379, + -0.07783810049295425, + -0.002305123955011368, + 0.18527865409851074, + -1.9857386350631714, + -0.009703370742499828, + 0.7078995108604431, + -0.08121174573898315, + -0.031056255102157593, + -0.40927961468696594, + 0.5954965949058533, + -0.8039777874946594, + 0.05187072232365608, + 0.2283477634191513 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.490875720977783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4157990217208862, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4566955268383026 + ], + [ + -1.5219146013259888, + 0.23501667380332947, + 0.5663670301437378, + -1.892876386642456, + -2.744781494140625, + -0.04844887554645538, + 1.3053655624389648, + -0.11823110282421112, + -0.02477371320128441, + -0.17061075568199158, + 0.22949494421482086, + -1.854048252105713, + -0.025548618286848068, + 0.7958164215087891, + 0.12320645153522491, + -0.23218455910682678, + -0.6995364427566528, + 0.8081986308097839, + -0.558222234249115, + -0.18823161721229553, + 0.04578563570976257 + ], + [ + -1.608424186706543, + 0.022089049220085144, + 0.15639334917068481, + -1.982436180114746, + -2.6509876251220703, + -0.01820056140422821, + 0.95000159740448, + -0.08617040514945984, + -0.46914470195770264, + 0.3453706204891205, + -0.020491018891334534, + -1.738605260848999, + -0.0859340950846672, + 0.443394273519516, + 0.09907735884189606, + -0.38721394538879395, + -0.3415156602859497, + 0.524202287197113, + -0.7343614101409912, + -0.22556106746196747, + 0.4535057842731476 + ], + [ + -1.7957028150558472, + 0.593864381313324, + 0.06846150755882263, + -2.0563552379608154, + -2.2595136165618896, + -0.12188448011875153, + 0.9899688959121704, + -0.14783401787281036, + -0.22327497601509094, + -0.32116562128067017, + -0.07331573963165283, + -1.910165786743164, + 0.264303058385849, + 1.0567166805267334, + 0.016188345849514008, + 0.159999281167984, + -0.22885318100452423, + 0.705758273601532, + -0.8778964877128601, + 0.4381847381591797, + 0.19897118210792542 + ], + [ + -1.499897837638855, + -0.2043466418981552, + 0.42094048857688904, + -1.430327296257019, + -3.1506710052490234, + -0.08664844930171967, + 1.4003938436508179, + 0.24064289033412933, + -0.35471564531326294, + -0.37839531898498535, + 0.47059693932533264, + -1.7703200578689575, + 0.101543128490448, + 0.8470064401626587, + 0.16171279549598694, + -0.0019259154796600342, + -0.6455138325691223, + 0.5694639682769775, + -0.9997703433036804, + 0.5366826057434082, + -0.11106891930103302 + ], + [ + -1.8847153186798096, + 0.21543154120445251, + 0.9309400320053101, + -2.1095237731933594, + -2.136512517929077, + 0.6948637366294861, + 1.233637809753418, + 0.577111542224884, + -0.5727867484092712, + -0.6055393815040588, + 0.8738593459129333, + -2.6654083728790283, + -0.171565979719162, + 0.8934992551803589, + 0.17394188046455383, + 0.6209473609924316, + -0.1149752140045166, + 0.6160256266593933, + -0.5519447326660156, + 0.7195507287979126, + 0.5175096392631531 + ], + [ + -2.138639450073242, + 0.36738795042037964, + -0.33481496572494507, + -1.9255597591400146, + -2.2420146465301514, + 0.31966179609298706, + 1.3172208070755005, + -0.48551350831985474, + 0.27687638998031616, + -0.41806119680404663, + 0.05362778902053833, + -2.3665032386779785, + -0.47635504603385925, + 0.081889808177948, + 0.542697548866272, + -0.7215265035629272, + -0.906980037689209, + 0.6977691054344177, + -1.4041197299957275, + -0.11211369931697845, + 0.5149455070495605 + ], + [ + -1.4869698286056519, + -0.30130720138549805, + 1.28191077709198, + -0.9839465618133545, + -2.2501890659332275, + -0.20459605753421783, + 1.3607591390609741, + 0.6349532008171082, + -0.8627415895462036, + -0.623334527015686, + 0.21189503371715546, + -2.2093186378479004, + -0.4655338525772095, + 0.7016672492027283, + 0.6916180849075317, + -0.0807163268327713, + 0.1465948522090912, + 0.05083376169204712, + -0.8381134271621704, + 0.015291888266801834, + 0.05565965175628662 + ], + [ + -2.365525245666504, + 0.45377761125564575, + 0.2984481453895569, + -1.3690096139907837, + -2.843494176864624, + 0.32462257146835327, + 0.5082458257675171, + 0.7033615112304688, + 0.31817564368247986, + -0.4890395700931549, + 0.09036337584257126, + -2.904804229736328, + 0.560937762260437, + 1.649811029434204, + -0.9795587062835693, + -0.46419358253479004, + -0.4739357829093933, + 0.34800490736961365, + -1.0712809562683105, + 0.9167320132255554, + -0.48252081871032715 + ], + [ + -1.2987136840820312, + 0.7370830774307251, + -0.49668365716934204, + -1.201794147491455, + -3.552318572998047, + -0.5406968593597412, + 2.1693286895751953, + 0.5925421714782715, + 0.7903083562850952, + 0.991620659828186, + 0.7936639785766602, + -2.3832802772521973, + -0.621701180934906, + 0.43542394042015076, + -0.14415577054023743, + 0.15921397507190704, + -0.395956814289093, + -0.2778439521789551, + -1.069305419921875, + -0.5347616076469421, + -0.26581722497940063 + ] + ], + "predicted_scores": [ + 5.7841668128967285, + 5.764790058135986, + 5.701019763946533, + 5.555482387542725, + 5.275122165679932, + 4.941834449768066, + 4.859184265136719, + 3.589162588119507 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007/train0", + "selected_index": 0, + "state_hash": "3139b3ac90f6fb958f14d875d082c95c8569f039086e0789e6c9e079d602b945", + "stored_base_utility": 2.60770320892334e-06, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0320269912481308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0320269912481308 + }, + "base_progress": 0.0320269912481308, + "base_success": false, + "base_utility": 0.0320269912481308, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.025111451745033264, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025111451745033264 + }, + { + "contact_quality": null, + "progress": 0.01827371120452881, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01827371120452881 + }, + { + "contact_quality": null, + "progress": 0.027498364448547363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027498364448547363 + }, + { + "contact_quality": null, + "progress": 0.026322290301322937, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026322290301322937 + }, + { + "contact_quality": null, + "progress": 0.020748361945152283, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020748361945152283 + }, + { + "contact_quality": null, + "progress": 0.02072523534297943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02072523534297943 + }, + { + "contact_quality": null, + "progress": 0.02157720923423767, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02157720923423767 + }, + { + "contact_quality": null, + "progress": 0.017048463225364685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017048463225364685 + } + ], + "candidate_progress": [ + 0.025111451745033264, + 0.01827371120452881, + 0.027498364448547363, + 0.026322290301322937, + 0.020748361945152283, + 0.02072523534297943, + 0.02157720923423767, + 0.017048463225364685 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-ad5107846b45eae5a0ab", + "ms-543339410c02ce2269d1", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-2423891634e7b469eed8", + "ms-940cce8479900a244d54" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "generated_tangents": [ + [ + 0.21391832828521729, + 0.27911072969436646, + 0.6514626741409302, + -1.236355185508728, + 0.026780158281326294, + 0.04066044092178345, + 0.1448931097984314, + -0.11620920896530151, + 0.0004434185102581978, + 0.02462116628885269, + -0.18335825204849243, + -0.42883485555648804, + 0.3130630552768707, + 0.34273093938827515, + 0.15518979728221893, + -0.06447125971317291, + 0.24042530357837677, + -0.5651291608810425, + -0.5243905782699585, + 0.4151548147201538, + 0.3093748390674591 + ], + [ + 0.031997427344322205, + 0.22568179666996002, + 0.7923535108566284, + -1.2483689785003662, + -0.08416551351547241, + -0.019817635416984558, + 0.09127455949783325, + -0.055856361985206604, + 0.009183604270219803, + 0.14164593815803528, + 0.02516769990324974, + -0.5547645092010498, + 0.5003151893615723, + 0.17507660388946533, + -0.05569937452673912, + -0.14774176478385925, + -0.0396450012922287, + -0.5300953984260559, + -0.5317016243934631, + 0.03582993149757385, + 0.18680451810359955 + ], + [ + 0.19381044805049896, + 0.19411520659923553, + 0.6819138526916504, + -1.2794206142425537, + -0.10723105072975159, + 0.007015615701675415, + 0.22282150387763977, + -0.058224886655807495, + 0.014266949146986008, + -0.022819705307483673, + -0.1932016909122467, + -0.6837509870529175, + 0.32226911187171936, + 0.16902875900268555, + -0.017442185431718826, + -0.15719394385814667, + 0.005850657820701599, + -0.49608033895492554, + -0.5501047372817993, + 0.3775603175163269, + 0.2457629293203354 + ], + [ + 0.18482279777526855, + 0.22926713526248932, + 0.7575491666793823, + -1.2488725185394287, + 0.06903427839279175, + 0.16657881438732147, + 0.21184700727462769, + -0.09736546874046326, + 0.0339469239115715, + 0.00885976105928421, + 0.04673461616039276, + -0.44935286045074463, + 0.20288899540901184, + 0.3733958303928375, + -0.0769776701927185, + -0.15453627705574036, + 0.03576599061489105, + -0.5208722352981567, + -0.46502888202667236, + 0.19200679659843445, + 0.49625730514526367 + ], + [ + 0.45117801427841187, + 0.21957920491695404, + 0.66351318359375, + -1.3271337747573853, + 0.09396044909954071, + -0.03816880285739899, + 0.38407742977142334, + -0.026697292923927307, + 0.06369594484567642, + -0.009219236671924591, + -0.3157995641231537, + -0.448066771030426, + 0.13001969456672668, + 0.3111880421638489, + -0.016908634454011917, + -0.12007813155651093, + -0.03340102732181549, + -0.6338077783584595, + -0.6817090511322021, + 0.06826531887054443, + 0.36431747674942017 + ], + [ + 0.23480692505836487, + 0.15920279920101166, + 0.6638433933258057, + -1.2215197086334229, + -0.009544029831886292, + 0.18046826124191284, + 0.3554733097553253, + 0.00799146294593811, + 0.03908376395702362, + 0.05289533734321594, + -0.08747530728578568, + -0.5114601254463196, + 0.37424519658088684, + 0.15484236180782318, + -0.010930489748716354, + -0.1264394074678421, + 0.05480580031871796, + -0.5123270750045776, + -0.5571259260177612, + 0.4353986978530884, + 0.24013832211494446 + ], + [ + 0.305020272731781, + 0.2628641128540039, + 0.6076853275299072, + -1.1824610233306885, + 0.018159329891204834, + 0.12448372691869736, + 0.07640895247459412, + 0.010020792484283447, + 0.07619402557611465, + 0.07756601274013519, + -0.05289169400930405, + -0.3794039487838745, + 0.886386513710022, + 0.27065667510032654, + 0.15791872143745422, + -0.06339281052350998, + 0.013592511415481567, + -0.4755313992500305, + -0.43516629934310913, + 0.4843834638595581, + 0.27058181166648865 + ], + [ + 0.2745092511177063, + 0.1170053482055664, + 0.6184108853340149, + -1.2551156282424927, + -0.1045483648777008, + 0.14739400148391724, + 0.24699805676937103, + 0.011442571878433228, + 0.0034699542447924614, + -0.038183487951755524, + -0.07970982044935226, + -0.551439106464386, + 0.3451023995876312, + 0.22736720740795135, + -0.10189378261566162, + -0.12885276973247528, + 0.02862030267715454, + -0.5456748604774475, + -0.5757142305374146, + 0.15437385439872742, + 0.2642448842525482 + ] + ], + "generated_utilities": [ + 0.025111451745033264, + 0.01827371120452881, + 0.027498364448547363, + 0.026322290301322937, + 0.020748361945152283, + 0.02072523534297943, + 0.02157720923423767, + 0.017048463225364685 + ], + "group_id": "ms-27c9f70e97b8850cc3ab", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.09440004825592041, + 0.14095592498779297, + 2.0, + 0.09219714999198914, + 0.05765572190284729, + 0.02530825138092041, + 0.07676126062870026, + 0.06643831729888916, + 0.047928065061569214, + 0.07107676565647125, + 0.05561619997024536, + 0.02124820649623871, + 0.08948668837547302, + 0.10203869640827179, + 0.06119762361049652, + 0.04547174274921417 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.984761953353882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.7230467796325684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.554086923599243 + ], + [ + 1.0343247652053833, + 1.287368655204773, + 1.867376446723938, + -2.4465248584747314, + 1.036996841430664, + -1.2664310932159424, + 1.4399157762527466, + -0.8265262246131897, + -0.28319495916366577, + 0.16331201791763306, + -1.5917816162109375, + -1.5816115140914917, + -0.1664504110813141, + 0.7195659279823303, + 0.29144588112831116, + -0.18517816066741943, + -0.08826595544815063, + -0.8266173601150513, + -0.7933371067047119, + -0.938305139541626, + 0.5359267592430115 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3968086242675781, + 0.5359851121902466, + 0.7176156044006348, + -1.52370023727417, + -0.1481943428516388, + 0.10887271165847778, + 0.0, + -0.12331700325012207, + 0.14385993778705597, + -0.041522324085235596, + 0.16089874505996704, + -0.6189663410186768, + -0.01699581742286682, + 0.0, + -0.07216246426105499, + -0.15073662996292114, + -0.134138286113739, + -0.06049776077270508, + -0.26776671409606934, + -0.019470810890197754, + 0.0 + ] + ], + "predicted_scores": [ + 4.651674270629883, + 4.646581649780273, + 4.335030555725098, + 4.235018253326416, + 4.019102096557617, + 4.008563041687012, + 3.998105525970459, + 3.9744935035705566 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1881223807/train0", + "selected_index": 0, + "state_hash": "0f58a11c36c331c9abab00795189dccbc7e0aebf985ca9df67bf9da756f154e3", + "stored_base_utility": 0.09440004825592041, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.10841864347457886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10841864347457886 + }, + "base_progress": 0.10841864347457886, + "base_success": false, + "base_utility": 0.10841864347457886, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.09370604157447815, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09370604157447815 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.09923991560935974, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09923991560935974 + }, + { + "contact_quality": null, + "progress": 0.08881917595863342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08881917595863342 + }, + { + "contact_quality": null, + "progress": 0.047322407364845276, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.047322407364845276 + }, + { + "contact_quality": null, + "progress": 0.0901307463645935, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0901307463645935 + } + ], + "candidate_progress": [ + 1.0, + 0.09370604157447815, + 1.0, + 1.0, + 0.09923991560935974, + 0.08881917595863342, + 0.047322407364845276, + 0.0901307463645935 + ], + "candidate_source_chart_ids": [ + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + true, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2867deaca7e98582620b", + "generated_tangents": [ + [ + -0.04763971269130707, + -0.0678432285785675, + 0.6449398398399353, + -0.1981954127550125, + -1.3741230964660645, + -0.33557868003845215, + -0.6917322278022766, + -0.09107854962348938, + -0.041970714926719666, + 0.18134674429893494, + 0.4880267083644867, + -1.5248818397521973, + 0.36955246329307556, + -0.1101178228855133, + 0.01678931713104248, + -0.13375253975391388, + 0.24641911685466766, + 0.37347614765167236, + -0.6676339507102966, + -0.26728612184524536, + -0.6003302335739136 + ], + [ + -0.09627243876457214, + -0.11876752227544785, + 0.6641285419464111, + -0.16966316103935242, + -1.4288198947906494, + -0.3930072486400604, + -0.6900435090065002, + -0.22106891870498657, + -0.09698735922574997, + 0.1734238713979721, + 0.4727734327316284, + -1.5060700178146362, + 0.40777507424354553, + -0.12340553104877472, + -0.036167170852422714, + -0.16675320267677307, + 0.3040286898612976, + 0.36621904373168945, + -0.6314959526062012, + -0.21443834900856018, + -0.4289957582950592 + ], + [ + -0.039548009634017944, + -0.022421829402446747, + 0.8354246616363525, + -0.4206569790840149, + -1.8109169006347656, + -0.6381659507751465, + -0.4683982729911804, + -0.12251394987106323, + -0.14813877642154694, + 0.02744618058204651, + 0.4390793442726135, + -1.274978756904602, + 0.11389526724815369, + -0.6276320219039917, + 0.017472438514232635, + -0.12346456944942474, + 0.14443257451057434, + 0.6089066863059998, + -0.755329430103302, + -0.400123655796051, + -0.5216860771179199 + ], + [ + -0.14656580984592438, + -0.01067439466714859, + 0.7990288734436035, + -0.5086795687675476, + -1.7642951011657715, + -0.646909236907959, + -0.4217536449432373, + -0.1634492576122284, + -0.09018047899007797, + 0.12375391274690628, + 0.3884236812591553, + -1.5418099164962769, + 0.0873769223690033, + -0.6337696313858032, + 0.018779784440994263, + -0.11734679341316223, + 0.20474588871002197, + 0.5171203017234802, + -0.7829434275627136, + -0.39981991052627563, + -0.6596001386642456 + ], + [ + -0.0746893435716629, + -0.045699138194322586, + 0.5160688757896423, + -0.42363274097442627, + -1.7745115756988525, + -0.2991913855075836, + -0.48841381072998047, + -0.1964462697505951, + -0.11526210606098175, + -0.048754192888736725, + 0.3773079514503479, + -1.0535447597503662, + 0.0609075129032135, + -0.16449986398220062, + -0.0647135078907013, + -0.07983292639255524, + 0.2888284921646118, + 0.6147249937057495, + -0.48144179582595825, + -0.773140013217926, + -0.5666354894638062 + ], + [ + -0.09176909923553467, + -0.03295668959617615, + 0.7303811311721802, + -0.6170456409454346, + -1.4957702159881592, + -0.1792428195476532, + -0.6505621075630188, + -0.24939551949501038, + -0.07375060021877289, + 0.23282092809677124, + 0.4616556167602539, + -1.710983157157898, + -0.19623568654060364, + -0.3865475654602051, + 0.05085122957825661, + -0.10729581117630005, + 0.29688090085983276, + 0.5926147103309631, + -0.7100726366043091, + -0.46041321754455566, + -0.404010146856308 + ], + [ + -0.10214368999004364, + -0.1263304054737091, + 0.9959338903427124, + -0.5758398771286011, + -1.518005132675171, + -0.41378673911094666, + -0.7111075520515442, + -0.5181982517242432, + -0.08770646899938583, + 0.15437161922454834, + 0.2556975781917572, + -1.6821696758270264, + 0.002704530954360962, + -0.261150598526001, + -0.14594191312789917, + -0.026637643575668335, + 0.29601413011550903, + 0.4495137929916382, + -0.9120617508888245, + -0.4980655908584595, + -0.44152769446372986 + ], + [ + -0.10134515166282654, + -0.14118391275405884, + 0.9698335528373718, + -0.6442772150039673, + -1.2969911098480225, + -0.9057308435440063, + -0.7732080817222595, + -0.26239848136901855, + -0.11714011430740356, + -0.04360116273164749, + 0.43906840682029724, + -1.540845274925232, + 0.11133697628974915, + -0.5487735271453857, + -0.061068709939718246, + 0.02590206265449524, + 0.2362658828496933, + 0.4810160994529724, + -0.7179185152053833, + -0.2162087857723236, + -0.5954456329345703 + ] + ], + "generated_utilities": [ + 2.0, + 0.09370604157447815, + 2.0, + 2.0, + 0.09923991560935974, + 0.08881917595863342, + 0.047322407364845276, + 0.0901307463645935 + ], + "group_id": "ms-2867deaca7e98582620b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0924256443977356, + 0.05646410584449768, + 2.0, + 2.0, + 0.183049738407135, + 2.0, + 0.16824382543563843, + 0.1424575299024582, + 0.08230158686637878, + 0.15871968865394592, + 0.16382521390914917, + 0.05336849391460419, + 0.07674244046211243, + 0.10727545619010925, + 0.06594061851501465, + 0.045542001724243164 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.44511327147483826, + 0.05827039107680321, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.2619132101535797, + 0.0, + -0.37528133392333984, + -0.07968637347221375, + -0.07514983415603638, + 0.19317352771759033, + 0.03868463635444641, + -0.06793206930160522, + 0.0, + 0.13187909126281738, + -0.20905672013759613, + 0.030073702335357666, + 0.21597933769226074, + -0.0761370062828064, + -0.11662125587463379, + 0.0 + ], + [ + -0.15486939251422882, + 0.24427275359630585, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.44790399074554443, + 0.0, + -0.37528133392333984, + -0.29508399963378906, + 0.7311322093009949, + 0.19317352771759033, + 0.09925723075866699, + 0.1767415851354599, + 0.0, + -0.5768744349479675, + 0.1300974041223526, + 0.21151533722877502, + 0.21597933769226074, + 0.04431194067001343, + -0.040631115436553955, + 0.0 + ], + [ + -0.29166290163993835, + 0.09681949019432068, + 2.2608249187469482, + -1.3577959537506104, + -2.195330858230591, + -0.13884210586547852, + 2.5595173835754395, + -1.3752813339233398, + 0.390796959400177, + 0.673005223274231, + 1.1931735277175903, + -0.41860872507095337, + -0.24474714696407318, + 0.3015621602535248, + -0.2400326132774353, + 0.14207373559474945, + -0.26512715220451355, + 1.2159793376922607, + -0.3237651586532593, + -0.7466862201690674, + 0.30001604557037354 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.119034767150879, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6031243205070496, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6000320911407471 + ], + [ + -0.3580354154109955, + 0.03320832550525665, + 2.5546932220458984, + -1.138798713684082, + -2.260084629058838, + -0.04964215308427811, + 2.8354909420013428, + -1.6732244491577148, + 0.21277914941310883, + 0.5097208023071289, + 1.387328028678894, + -0.691602349281311, + -0.28483790159225464, + 0.35576167702674866, + -0.0605899840593338, + 0.3764685094356537, + -0.28031954169273376, + 1.063202977180481, + -0.03841540217399597, + -0.5410391092300415, + 0.5580100417137146 + ], + [ + -0.5578174591064453, + 0.3727092146873474, + 2.1929497718811035, + -1.577579140663147, + -1.8962993621826172, + -0.2802738845348358, + 2.1670117378234863, + -1.0752147436141968, + 0.16372516751289368, + 0.6087679862976074, + 0.8854888081550598, + -0.18080148100852966, + -0.18821784853935242, + 0.010591715574264526, + -0.12560874223709106, + 0.15850795805454254, + -0.1898641288280487, + 1.1823316812515259, + 0.06655246019363403, + -0.4014361798763275, + 0.4376043379306793 + ], + [ + -0.43790310621261597, + -0.12778332829475403, + 2.6857643127441406, + -1.6293541193008423, + -1.9850949048995972, + -0.7167636752128601, + 2.465301513671875, + -1.4102312326431274, + 0.2071499079465866, + 0.5742072463035583, + 0.9525313377380371, + -0.34116798639297485, + -0.04760183393955231, + -0.05569759011268616, + -0.4777999520301819, + 0.17186680436134338, + 0.0841393768787384, + 0.7708102464675903, + -0.2784478962421417, + -1.2458651065826416, + 0.46140795946121216 + ], + [ + 0.06170770525932312, + 0.45555436611175537, + 1.6506444215774536, + -0.9394457340240479, + -1.9117670059204102, + -0.6910014152526855, + 2.40145206451416, + -1.7531975507736206, + 0.10217079520225525, + 1.2514894008636475, + 1.3236911296844482, + -0.6504120826721191, + -0.6943472027778625, + 0.7029284238815308, + -0.7387814521789551, + -0.33777421712875366, + -0.8187699317932129, + 1.8448829650878906, + 0.00849190354347229, + -1.394152045249939, + 0.918946385383606 + ] + ], + "predicted_scores": [ + 9.423457145690918, + 9.199984550476074, + 9.018994331359863, + 8.800616264343262, + 8.53002643585205, + 8.281440734863281, + 8.193784713745117, + 7.748565196990967 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1189392917/train0", + "selected_index": 0, + "state_hash": "e8cf9b95f09f32d8e7942b114792725bd89c392a97ff7e6468ebf09df1bb0d21", + "stored_base_utility": 0.0924256443977356, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.13927479088306427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13927479088306427 + }, + "base_progress": 0.13927479088306427, + "base_success": false, + "base_utility": 0.13927479088306427, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.054733291268348694, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054733291268348694 + }, + { + "contact_quality": null, + "progress": 0.130090594291687, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.130090594291687 + }, + { + "contact_quality": null, + "progress": 0.1239127367734909, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1239127367734909 + }, + { + "contact_quality": null, + "progress": 0.11601470410823822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11601470410823822 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1307433694601059, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1307433694601059 + } + ], + "candidate_progress": [ + 0.054733291268348694, + 0.130090594291687, + 0.1239127367734909, + 0.11601470410823822, + 1.0, + 1.0, + 1.0, + 0.1307433694601059 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + true, + true, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-365d8c6de6de16f66744", + "generated_tangents": [ + [ + -0.3216032385826111, + -0.09330794960260391, + 1.3372516632080078, + -0.4772129952907562, + -1.5004887580871582, + 0.016235902905464172, + 0.21235880255699158, + -0.044765010476112366, + -0.183041512966156, + 0.28999531269073486, + -0.13156822323799133, + -1.3798290491104126, + -0.18255624175071716, + 0.23481377959251404, + -0.014678005129098892, + -0.056500520557165146, + 0.03761991858482361, + -0.21641232073307037, + -0.2754674553871155, + -0.09078086912631989, + 0.05808679014444351 + ], + [ + -0.1144808977842331, + -0.05292811617255211, + 1.2491137981414795, + -0.39868515729904175, + -1.4802651405334473, + -0.3096119463443756, + -0.09811574220657349, + -0.005206048488616943, + -0.11761322617530823, + 0.13060137629508972, + 0.05133647844195366, + -1.5365304946899414, + 0.04028269648551941, + -0.007292404770851135, + 0.1082511842250824, + -0.07130888104438782, + 0.08780141174793243, + -0.014397680759429932, + -0.2765340805053711, + -0.244420126080513, + -0.20084506273269653 + ], + [ + -0.06980770826339722, + -0.10473823547363281, + 1.195068359375, + -0.4298571050167084, + -1.5672507286071777, + -0.20873722434043884, + -0.15355783700942993, + -0.05104941129684448, + -0.12815922498703003, + 0.15062212944030762, + -0.02957005798816681, + -1.336713194847107, + 0.015305101871490479, + -0.006348222494125366, + 0.13029927015304565, + -0.07177798449993134, + 0.04681609570980072, + 0.0243389755487442, + -0.32356107234954834, + -0.18493227660655975, + -0.22214433550834656 + ], + [ + -0.042171478271484375, + -0.12031754106283188, + 1.2885713577270508, + -0.46994948387145996, + -1.515977144241333, + -0.39441218972206116, + 0.03862711787223816, + -0.1264740526676178, + -0.11714942753314972, + 0.13802669942378998, + -0.046181827783584595, + -1.3958179950714111, + -0.008493602275848389, + 0.1297491043806076, + 0.07320552319288254, + -0.09979991614818573, + -0.034600868821144104, + 0.0075378865003585815, + -0.3996841311454773, + -0.1345905214548111, + -0.22771862149238586 + ], + [ + 0.011613085865974426, + -0.08167794346809387, + 1.2190431356430054, + -0.40916967391967773, + -1.4880385398864746, + -0.39875528216362, + -0.14453530311584473, + 0.0459790974855423, + -0.12014222145080566, + 0.18404307961463928, + 0.07235030829906464, + -1.5115779638290405, + 0.03647473454475403, + -0.054496511816978455, + 0.24218566715717316, + -0.04489048942923546, + 0.09471248090267181, + -0.05882169306278229, + -0.3326870799064636, + -0.15918555855751038, + -0.1406424641609192 + ], + [ + 0.01318301260471344, + -0.1407550722360611, + 1.2396866083145142, + -0.3165585398674011, + -1.4180755615234375, + -0.3175802528858185, + 0.043541669845581055, + 0.03394855558872223, + -0.1642056405544281, + 0.16651961207389832, + -0.27376940846443176, + -1.378136396408081, + 0.5216007232666016, + -0.04210418462753296, + 0.11567295342683792, + -0.030030319467186928, + -0.10715444386005402, + -0.13276152312755585, + -0.4976702928543091, + -0.30539098381996155, + -0.48064926266670227 + ], + [ + -0.0824265331029892, + -0.06618291139602661, + 1.276452660560608, + -0.3603707253932953, + -1.496096134185791, + -0.4063102900981903, + -0.4149729609489441, + -0.15529516339302063, + -0.08880845457315445, + 0.10781368613243103, + 0.021841958165168762, + -1.2306064367294312, + 0.34022465348243713, + -0.04823920130729675, + -0.19550997018814087, + -0.03683430328965187, + -0.20409326255321503, + -0.011105716228485107, + -0.28989434242248535, + -0.10742746293544769, + -0.3438708484172821 + ], + [ + -0.00818982720375061, + -0.11088667809963226, + 1.400978446006775, + -0.3424351215362549, + -1.455199956893921, + -0.3057328462600708, + -0.11656457185745239, + 0.08311361819505692, + -0.1791704148054123, + 0.1343432366847992, + -0.2693428099155426, + -1.2886607646942139, + 0.6036311388015747, + -0.4333891272544861, + -0.0030657686293125153, + 0.08980372548103333, + -0.06903629004955292, + -0.12000139057636261, + -0.4740470051765442, + -0.32972341775894165, + -0.5390377044677734 + ] + ], + "generated_utilities": [ + 0.054733291268348694, + 0.130090594291687, + 0.1239127367734909, + 0.11601470410823822, + 2.0, + 2.0, + 2.0, + 0.1307433694601059 + ], + "group_id": "ms-365d8c6de6de16f66744", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20231953263282776, + 0.12319359183311462, + 2.0, + 2.0, + 0.1590207815170288, + 0.019343823194503784, + 0.1322992593050003, + 0.09479346871376038, + 0.1768283247947693, + 0.1054161787033081, + 0.16278749704360962, + 0.08510464429855347, + 0.09564858675003052, + 0.07681193947792053, + 0.1334480196237564, + 0.08148902654647827 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.09058855473995209, + -0.022760659456253052, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.07837581634521484, + 0.0, + -0.05228453874588013, + -0.0679071694612503, + 0.026890814304351807, + 0.03908693790435791, + -1.4526653289794922, + 0.04786187410354614, + 0.0, + 0.019858457148075104, + -0.05534793436527252, + 0.17386651039123535, + -0.024165034294128418, + -0.05880159139633179, + 0.044484615325927734, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.063174724578857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6561365127563477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0078073740005493 + ], + [ + 0.05168074369430542, + 0.27186232805252075, + 2.0911388397216797, + -1.6812132596969604, + -2.3280625343322754, + 0.38023167848587036, + 2.5090978145599365, + -0.2604735791683197, + 0.04725922644138336, + 0.8121852874755859, + -0.6659450531005859, + -2.2462568283081055, + 0.49988994002342224, + 1.9786770343780518, + 0.02324119210243225, + -0.45050501823425293, + 1.1065142154693604, + 0.5724118947982788, + -0.6320602893829346, + -0.7826599478721619, + 0.47463753819465637 + ], + [ + 0.46382713317871094, + -0.06331050395965576, + 2.657292366027832, + -1.1560972929000854, + -2.765941619873047, + 0.7805119752883911, + 2.3117406368255615, + -0.3644295036792755, + -0.25628525018692017, + 0.7623568773269653, + -0.4176616668701172, + -2.646364688873291, + 0.40904325246810913, + 1.7654002904891968, + -0.23315654695034027, + 0.026929572224617004, + 0.8236149549484253, + 0.8322737812995911, + -0.7612741589546204, + -0.4725078046321869, + 0.5895571708679199 + ], + [ + -0.26226741075515747, + 0.3922410011291504, + 2.450082302093506, + -0.9020822048187256, + -3.028742551803589, + 1.0746917724609375, + 3.104015350341797, + 0.3752234876155853, + -0.009036362171173096, + 0.3652568459510803, + -0.7608495950698853, + -2.337242603302002, + -0.03944078087806702, + 1.972575306892395, + -0.32381975650787354, + -0.45855021476745605, + 1.6927499771118164, + 0.8358478546142578, + -0.8610806465148926, + -0.6741821765899658, + 0.4013170003890991 + ], + [ + 0.5790615677833557, + 0.8308188915252686, + 3.141986846923828, + -1.218785285949707, + -2.0300495624542236, + 0.4300342798233032, + 3.0737316608428955, + -0.09167198091745377, + 0.4237035810947418, + 0.7421274185180664, + -0.637946367263794, + -2.1245827674865723, + 0.4184945523738861, + 1.2377147674560547, + -0.4068492650985718, + -0.03615438938140869, + 1.5385099649429321, + 0.51850825548172, + -0.5737811923027039, + -1.472043752670288, + 0.45800793170928955 + ], + [ + 0.38714414834976196, + 0.25513744354248047, + 2.9400248527526855, + -1.8116681575775146, + -1.7500431537628174, + 1.403240442276001, + 1.9709672927856445, + 0.6755956411361694, + 0.49445489048957825, + 0.19490671157836914, + -1.1990764141082764, + -2.803361177444458, + 0.499578058719635, + 1.164682149887085, + 0.9588776230812073, + 0.5154257416725159, + 1.0328537225723267, + 1.1309876441955566, + -0.9365650415420532, + -0.9663279056549072, + -0.032733798027038574 + ], + [ + 0.7417005896568298, + -0.12406697869300842, + 3.0325844287872314, + -0.5656485557556152, + -2.5084800720214844, + 0.27374324202537537, + 2.2886011600494385, + 0.5997821092605591, + 0.9623891115188599, + 0.38451504707336426, + -1.4501185417175293, + -1.6908833980560303, + 0.6743149757385254, + 1.8593690395355225, + -0.018024832010269165, + -0.20444278419017792, + 1.364742636680603, + 0.34230074286460876, + -1.4427728652954102, + -1.5577378273010254, + 0.5589802265167236 + ], + [ + 0.3448556661605835, + -0.5947309732437134, + 2.3888885974884033, + -1.0376405715942383, + -2.218167781829834, + 1.568857192993164, + 3.0995821952819824, + -0.4682580530643463, + 0.3072889745235443, + 0.14532917737960815, + -0.567415177822113, + -3.085655689239502, + -0.6656016111373901, + 1.6447856426239014, + 0.8135664463043213, + 0.3758860230445862, + 1.399398922920227, + 0.11528515815734863, + -0.531513512134552, + -0.6104787588119507, + 1.4474445581436157 + ], + [ + -0.18966330587863922, + -0.7208725214004517, + 2.586766481399536, + -0.6973344087600708, + -3.2187445163726807, + 1.4117884635925293, + 3.0797414779663086, + -0.29149553179740906, + 0.09620378166437149, + 0.874613881111145, + -0.8004729151725769, + -2.0231733322143555, + -0.5948910713195801, + 1.5080900192260742, + -0.750367283821106, + 0.8155628442764282, + 1.6562600135803223, + 0.8091130256652832, + -1.1569771766662598, + -1.4121384620666504, + 0.3738836348056793 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06428352743387222, + -0.015524566173553467, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + -0.1607142686843872, + 0.0, + 0.27928173542022705, + 0.05664636939764023, + -0.03914147615432739, + 0.1700872778892517, + -1.4526653289794922, + 0.05555404722690582, + 0.0, + 0.18240894377231598, + -0.05239863693714142, + 0.17386651039123535, + -0.21921080350875854, + -0.3728579580783844, + 0.11250120401382446, + 0.0 + ], + [ + 0.008915487676858902, + 0.540733277797699, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.22902190685272217, + 0.0, + -0.3544532060623169, + -0.2348114550113678, + 0.04456782341003418, + 0.2907915711402893, + -1.4526653289794922, + -0.1788860559463501, + 0.0, + 0.07762694358825684, + -0.07040916383266449, + 0.17386651039123535, + 0.44059625267982483, + -0.19159233570098877, + 0.09149748086929321, + 0.0 + ] + ], + "predicted_scores": [ + 10.039008140563965, + 9.910670280456543, + 9.840947151184082, + 9.763011932373047, + 9.72486400604248, + 9.312314987182617, + 9.310540199279785, + 9.040696144104004 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1997090716/train0", + "selected_index": 0, + "state_hash": "98870eab9f394b355af9b8a6d689b3bcd574af94db5d1a9139d0aed0b633b959", + "stored_base_utility": 0.20231953263282776, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03280790150165558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03280790150165558 + }, + "base_progress": 0.03280790150165558, + "base_success": false, + "base_utility": 0.03280790150165558, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.02049049735069275, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02049049735069275 + }, + { + "contact_quality": null, + "progress": 0.019123628735542297, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019123628735542297 + }, + { + "contact_quality": null, + "progress": 0.02138802409172058, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02138802409172058 + }, + { + "contact_quality": null, + "progress": 0.021766602993011475, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021766602993011475 + }, + { + "contact_quality": null, + "progress": 0.020316720008850098, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020316720008850098 + }, + { + "contact_quality": null, + "progress": 0.020232409238815308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020232409238815308 + }, + { + "contact_quality": null, + "progress": 0.020674332976341248, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020674332976341248 + }, + { + "contact_quality": null, + "progress": 0.01947517693042755, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01947517693042755 + } + ], + "candidate_progress": [ + 0.02049049735069275, + 0.019123628735542297, + 0.02138802409172058, + 0.021766602993011475, + 0.020316720008850098, + 0.020232409238815308, + 0.020674332976341248, + 0.01947517693042755 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1", + "ms-fe38b164397dd669f391", + "ms-2423891634e7b469eed8", + "ms-2cbe83da02847008bf24", + "ms-2cbe83da02847008bf24", + "ms-4e67cfd30a257b4bad0c", + "ms-3f6cd47af5e9e6781b73" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3c326d05fec66e1d6bea", + "generated_tangents": [ + [ + 0.16476602852344513, + -0.7746200561523438, + 0.4733996093273163, + 0.6301577091217041, + 0.4205824136734009, + -0.45246633887290955, + 0.2243681252002716, + 0.14780843257904053, + 0.4026601314544678, + -0.151661217212677, + -0.2815152704715729, + -0.6137023568153381, + 0.01335868239402771, + -0.3578207492828369, + 0.22738154232501984, + 0.0726105272769928, + 0.0457034707069397, + -0.21299202740192413, + 0.11014659702777863, + -0.14157848060131073, + 0.07258753478527069 + ], + [ + 0.276533842086792, + -0.7341556549072266, + 0.526631772518158, + 0.6573271751403809, + 0.22801227867603302, + -0.5764880180358887, + -0.07588177919387817, + 0.1517592966556549, + 0.42064860463142395, + -0.07303262501955032, + -0.2933375835418701, + -0.4886290431022644, + -0.2643055319786072, + -0.2738226056098938, + -0.0169597826898098, + -0.11798045039176941, + -0.03258328139781952, + -0.17629940807819366, + 0.20040032267570496, + -0.1956155002117157, + 0.2750420570373535 + ], + [ + 0.31270211935043335, + -0.7637926340103149, + 0.6518735289573669, + 0.5016700029373169, + 0.306433767080307, + -0.3301326632499695, + -0.14478343725204468, + 0.2674139142036438, + 0.4751649796962738, + -0.22880062460899353, + -0.274884432554245, + -0.4472060203552246, + -0.07112160325050354, + -0.24198009073734283, + 0.20803046226501465, + -0.1601962298154831, + -0.10024450719356537, + -0.12129254639148712, + 0.16944289207458496, + -0.19254659116268158, + 0.4723520576953888 + ], + [ + 0.3514142334461212, + -0.7639657258987427, + 0.5289261937141418, + 0.4677295386791229, + 0.3823462724685669, + -0.20860344171524048, + -0.7424694895744324, + 0.30742573738098145, + 0.44662776589393616, + -0.07738064974546432, + -0.2882902920246124, + -0.007942810654640198, + 0.06950792670249939, + -0.36652618646621704, + 0.23742523789405823, + -0.18107864260673523, + -0.2582334876060486, + -0.3388598561286926, + 0.25595366954803467, + -0.12364216148853302, + 0.2660079598426819 + ], + [ + 0.3825204372406006, + -0.8296485543251038, + 0.69476717710495, + 0.4517246186733246, + 0.327655166387558, + -0.5421425104141235, + -0.344096839427948, + 0.1634248048067093, + 0.45052796602249146, + -0.201113760471344, + -0.31148332357406616, + -0.5693178772926331, + -0.15159597992897034, + -0.08663521707057953, + 0.10822102427482605, + -0.07047496736049652, + -0.11448802053928375, + -0.17071433365345, + 0.26362794637680054, + 0.14572039246559143, + 0.38151073455810547 + ], + [ + 0.38860803842544556, + -0.8245835304260254, + 0.6431694030761719, + 0.53558349609375, + 0.3753895163536072, + -0.5509328842163086, + -0.4268704056739807, + 0.12425629794597626, + 0.3904174566268921, + -0.17857274413108826, + -0.34679609537124634, + -0.5098536014556885, + -0.019853979349136353, + -0.16961641609668732, + -0.018545765429735184, + -0.1217147707939148, + -0.16610486805438995, + -0.20802204310894012, + 0.12054277956485748, + -0.0583534836769104, + 0.3944220542907715 + ], + [ + 0.09971822798252106, + -0.8076961040496826, + 0.6701511144638062, + 0.656691312789917, + 0.4004058241844177, + -0.4149051010608673, + -0.007472395896911621, + 0.16103240847587585, + 0.37219664454460144, + -0.1118859276175499, + -0.2798059582710266, + -0.6645269393920898, + 0.07491078972816467, + -0.3090340495109558, + -0.12858299911022186, + -0.06796237826347351, + -0.04526151716709137, + -0.18992768228054047, + 0.1250699907541275, + -0.4557264745235443, + 0.210085928440094 + ], + [ + 0.5095956325531006, + -0.5202342867851257, + 0.839331865310669, + 0.5186622142791748, + 0.4385504126548767, + -0.6590684652328491, + 0.07915988564491272, + 0.1053347960114479, + 0.5073033571243286, + -0.2813633978366852, + -0.19366136193275452, + -0.6410282850265503, + -0.19180193543434143, + -0.14046071469783783, + 0.2305084615945816, + -0.10779756307601929, + -0.37554121017456055, + -0.2032083123922348, + 0.08785460889339447, + -0.053603753447532654, + 0.3816084563732147 + ] + ], + "generated_utilities": [ + 0.02049049735069275, + 0.019123628735542297, + 0.02138802409172058, + 0.021766602993011475, + 0.020316720008850098, + 0.020232409238815308, + 0.020674332976341248, + 0.01947517693042755 + ], + "group_id": "ms-3c326d05fec66e1d6bea", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18246883153915405, + 2.0, + 2.0, + 0.11388173699378967, + 0.035728439688682556, + 0.05159728229045868, + 0.043176501989364624, + 0.04580867290496826, + 0.02321702241897583, + 0.040450215339660645, + 0.04055213928222656, + 0.024657413363456726, + 0.017660364508628845, + 0.044850513339042664, + 0.031234994530677795, + 0.06195336580276489 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.018136948347091675, + 0.10074883699417114, + 0.11574286222457886, + 0.0, + 0.1410461664199829, + 0.38714247941970825, + 0.2381301373243332, + 0.06927800178527832, + -0.6138286590576172, + -0.33484986424446106, + 0.0, + 0.012610852718353271, + 0.5407634973526001, + 0.07784794270992279, + 0.3537199795246124, + -0.1602279543876648, + 0.002093076705932617, + 0.0 + ], + [ + 1.9327397346496582, + -1.7429183721542358, + 1.8877019882202148, + -0.33001917600631714, + 0.49746543169021606, + 0.47390231490135193, + 1.7802670001983643, + -0.8589538335800171, + 0.37976425886154175, + -0.18104670941829681, + -0.8859321475028992, + -1.6138286590576172, + 0.34242650866508484, + 1.2023069858551025, + 0.555707573890686, + -0.35764527320861816, + -0.10027600824832916, + 0.4702257812023163, + -0.6294202208518982, + 0.1926015019416809, + 0.5916687846183777 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.5605340003967285, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.404613971710205, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1833375692367554 + ], + [ + 1.7389304637908936, + -1.4791877269744873, + 2.158487319946289, + -0.3266654312610626, + 0.5907376408576965, + 0.43260636925697327, + 1.9136874675750732, + -0.5760036110877991, + 0.3071907162666321, + -0.4073694348335266, + -0.8863769769668579, + -1.8163083791732788, + 0.5969879627227783, + 1.196010947227478, + 0.7674136161804199, + -0.2368052750825882, + -0.12123071402311325, + 0.36782756447792053, + -0.8187789916992188, + 0.15793970227241516, + 0.43327897787094116 + ], + [ + 1.9560860395431519, + -2.008063793182373, + 1.6910814046859741, + -0.2576735019683838, + 0.647503137588501, + 0.5322431921958923, + 1.9268598556518555, + -1.2501428127288818, + 0.775363564491272, + -0.0965004414319992, + -0.5331029891967773, + -1.639766812324524, + 0.44450435042381287, + 1.5980249643325806, + 0.9402694702148438, + -0.1454993337392807, + -0.2541998028755188, + 0.6709543466567993, + -0.44338259100914, + -0.043697476387023926, + 0.7131086587905884 + ], + [ + 1.4609386920928955, + -1.6896755695343018, + 2.174467086791992, + -0.34207087755203247, + 0.962078332901001, + 0.28904104232788086, + 2.0044631958007812, + -1.184704303741455, + 0.8424319624900818, + -0.6237084865570068, + -0.9171007871627808, + -1.5616872310638428, + 0.3556978702545166, + 1.5233795642852783, + 0.7210954427719116, + 0.028062552213668823, + 0.21759141981601715, + 0.2823620140552521, + -0.588395357131958, + -0.12581703066825867, + 0.7275858521461487 + ], + [ + 2.087728500366211, + -2.0203187465667725, + 2.4627456665039062, + 0.09322953224182129, + 0.4771752953529358, + 0.2439241260290146, + 2.3431313037872314, + -0.7176190614700317, + 0.8298519849777222, + -0.47738587856292725, + -1.4635096788406372, + -1.2598087787628174, + 0.008061617612838745, + 0.9835944175720215, + 0.10656380653381348, + -0.1548040360212326, + 0.20115382969379425, + 0.4581896960735321, + -1.197885274887085, + -0.28882738947868347, + 0.17229679226875305 + ], + [ + 2.535038471221924, + -2.2647459506988525, + 1.280318260192871, + -0.8243153095245361, + 0.36149507761001587, + 0.01678559184074402, + 1.259200096130371, + -0.5383518934249878, + 1.0661393404006958, + -0.1134992390871048, + -1.0009663105010986, + -1.8629896640777588, + 0.5329263806343079, + 1.1952301263809204, + 0.5380017757415771, + -0.21264730393886566, + 0.33232176303863525, + 0.9111320972442627, + -1.0945085287094116, + 0.7546109557151794, + 0.8692140579223633 + ], + [ + 2.3966352939605713, + -2.3338406085968018, + 2.0661728382110596, + -0.1174439787864685, + -0.28071242570877075, + 0.8010929822921753, + 2.485050678253174, + -0.5360689163208008, + 0.9712552428245544, + 0.5487139821052551, + -0.46108368039131165, + -2.0631186962127686, + 0.6046813726425171, + 0.9991111159324646, + 1.299267292022705, + 0.3742367625236511, + -0.7156072854995728, + 0.7805436849594116, + -1.3091955184936523, + -0.5538330674171448, + -0.20684683322906494 + ], + [ + 2.614837884902954, + -2.24919056892395, + 2.2911505699157715, + -0.2939787209033966, + 1.1876089572906494, + 0.15644106268882751, + 1.7348854541778564, + -0.7635170221328735, + 0.5141214728355408, + -0.17918013036251068, + -0.8421157598495483, + -1.5084272623062134, + -0.18822845816612244, + 1.8389434814453125, + 0.24652135372161865, + 0.38509100675582886, + 0.5761538147926331, + -0.15904095768928528, + 0.0013433098793029785, + 0.6120359897613525, + 1.3977106809616089 + ], + [ + 1.5386989116668701, + -2.0177698135375977, + 1.8581953048706055, + -1.2861346006393433, + 1.0945217609405518, + 0.8363389372825623, + 2.708517074584961, + -0.9201098084449768, + 0.8546208739280701, + -0.058566831052303314, + -1.5932974815368652, + -2.015472412109375, + 0.2565937638282776, + 1.988462209701538, + -0.04372602701187134, + -0.8754178881645203, + 0.27551382780075073, + -0.29693683981895447, + -0.2824576497077942, + 0.40926772356033325, + 0.7658611536026001 + ], + [ + 2.8735361099243164, + -2.675261974334717, + 1.459324836730957, + 0.380817174911499, + 0.6231281757354736, + 0.03267902135848999, + 2.296022415161133, + -1.587423324584961, + 0.7833051681518555, + 0.6618785262107849, + -1.7110093832015991, + -1.436572551727295, + -0.14784568548202515, + 1.525026559829712, + 1.4909214973449707, + -1.0601696968078613, + -0.5454426407814026, + -0.09859517216682434, + -1.4344520568847656, + -0.058256953954696655, + -0.3444262146949768 + ], + [ + 2.0854334831237793, + -1.3952012062072754, + 2.651411771774292, + -0.7663257122039795, + -0.05179548263549805, + 0.8797686100006104, + 2.0904030799865723, + -0.8943268656730652, + 1.0378066301345825, + 0.08841629326343536, + -0.36119532585144043, + -1.236596703529358, + 1.0752983093261719, + 0.349973201751709, + -0.19527214765548706, + -0.8867040872573853, + 0.18460334837436676, + 1.454195261001587, + -0.6220338940620422, + 0.9919598698616028, + 1.229588508605957 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.009577363729476929, + -0.06759434938430786, + 0.05406871438026428, + 0.0, + 0.003663182258605957, + 0.0492824912071228, + -0.038241639733314514, + -0.02580159902572632, + -0.6138286590576172, + -0.056222498416900635, + 0.0, + -0.026217222213745117, + -0.051223963499069214, + -0.031640589237213135, + -0.010870158672332764, + 0.08728909492492676, + -0.020528942346572876, + 0.0 + ], + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.06735062599182129, + -0.023807168006896973, + 0.1439460813999176, + 0.0, + -0.19209599494934082, + -0.14847975969314575, + -0.22449488937854767, + 0.09710657596588135, + -0.6138286590576172, + -0.1854294240474701, + 0.0, + -0.17072844505310059, + 0.09961000084877014, + -0.017846792936325073, + 0.22415585815906525, + -0.18760734796524048, + -0.060083091259002686, + 0.0 + ] + ], + "predicted_scores": [ + 8.38741397857666, + 8.032026290893555, + 7.941018581390381, + 7.737374305725098, + 7.561871528625488, + 7.263994216918945, + 7.038136959075928, + 6.148074626922607 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524/train0", + "selected_index": 0, + "state_hash": "6333fcf80f76890398820f27d1a99efe459cdb6ff795201d73e3bcedc966664f", + "stored_base_utility": 0.18246883153915405, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18095605075359344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18095605075359344 + }, + "base_progress": 0.18095605075359344, + "base_success": false, + "base_utility": 0.18095605075359344, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.061786800622940063, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061786800622940063 + }, + { + "contact_quality": null, + "progress": 0.07495661079883575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07495661079883575 + }, + { + "contact_quality": null, + "progress": 0.061137303709983826, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061137303709983826 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.061786800622940063, + 0.07495661079883575, + 0.061137303709983826, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "generated_tangents": [ + [ + -0.7440127730369568, + 0.014998644590377808, + -0.5712763071060181, + -1.8051837682724, + -0.9082604050636292, + -1.1683580875396729, + -0.6399858593940735, + -0.12223121523857117, + -0.2638425827026367, + -0.1335526704788208, + 0.1557423621416092, + 0.018410593271255493, + 0.4193209707736969, + -0.3744836449623108, + -0.024524342268705368, + -0.5091266632080078, + 0.1175609827041626, + 0.5742997527122498, + 0.20372354984283447, + -0.4122728705406189, + -0.4277847707271576 + ], + [ + -0.3522629737854004, + -0.06639464199542999, + -0.26701971888542175, + -2.1856768131256104, + -0.9222970604896545, + -0.8079099655151367, + -0.6796756386756897, + -0.2296118438243866, + -0.12299269437789917, + 0.09488151222467422, + 0.08475707471370697, + -0.17584702372550964, + 0.007751137018203735, + -0.2910386919975281, + -0.10719436407089233, + -0.4741843640804291, + 0.18213149905204773, + 0.27579787373542786, + 0.28895407915115356, + -0.6325862407684326, + -0.62404465675354 + ], + [ + -0.598300039768219, + 0.04896893352270126, + -0.20403441786766052, + -2.139519214630127, + -1.08447265625, + -0.4189446270465851, + -0.6346327662467957, + -0.14519792795181274, + -0.1999586522579193, + 0.005803830921649933, + 0.30325591564178467, + -0.2296145260334015, + -0.08870476484298706, + -0.24513129889965057, + -0.06902675330638885, + -0.4448564350605011, + 0.0938514918088913, + 0.297969788312912, + 0.36479681730270386, + -0.5445393919944763, + -0.7320643663406372 + ], + [ + -0.4095943570137024, + -0.0890439972281456, + -0.2352524697780609, + -2.2584950923919678, + -0.8821102976799011, + -0.8211241960525513, + -0.7584159970283508, + -0.21534979343414307, + -0.08541803807020187, + 0.11774807423353195, + 0.1859683394432068, + -0.11890029907226562, + 0.007693499326705933, + -0.2932388186454773, + -0.06781680881977081, + -0.5304293036460876, + 0.2986355423927307, + 0.17837464809417725, + 0.29122135043144226, + -0.6381885409355164, + -0.6429785490036011 + ], + [ + -0.6171810626983643, + -0.15433329343795776, + -0.5142687559127808, + -2.111546039581299, + -1.4152092933654785, + -0.9933813810348511, + -1.0423638820648193, + -0.05100378394126892, + -0.2275928407907486, + 0.14460963010787964, + 0.304403156042099, + -0.596270740032196, + 0.14442339539527893, + -0.6126604676246643, + 0.026041459292173386, + -0.433432936668396, + 0.12705498933792114, + 0.3901751637458801, + 0.37796664237976074, + -0.5136858224868774, + -0.45762547850608826 + ], + [ + -0.5448775291442871, + -0.18337921798229218, + -0.5216808319091797, + -1.9630666971206665, + -1.3005261421203613, + -0.9385174512863159, + -0.9818581938743591, + -0.07276719808578491, + -0.2240539789199829, + 0.14343151450157166, + 0.40550243854522705, + -0.5454055070877075, + 0.028278887271881104, + -0.5270435214042664, + 0.012806683778762817, + -0.42532992362976074, + 0.23145246505737305, + 0.3342704772949219, + 0.35551169514656067, + -0.5486299395561218, + -0.4504261910915375 + ], + [ + -0.6717805862426758, + -0.19726186990737915, + -0.6103730201721191, + -2.224104642868042, + -1.4604973793029785, + -0.4195825755596161, + -1.0958538055419922, + -0.11987295746803284, + -0.228374645113945, + 0.08262364566326141, + -0.03332985192537308, + -0.440670907497406, + -0.14603298902511597, + -0.7307149767875671, + -0.030686799436807632, + -0.4650437533855438, + 0.13531675934791565, + 0.2419317364692688, + 0.48018038272857666, + -0.787667989730835, + -0.016932129859924316 + ], + [ + -0.6525436043739319, + -0.12835577130317688, + -0.7270898818969727, + -1.7835642099380493, + -1.3232734203338623, + -1.0632609128952026, + -0.8576175570487976, + -0.10045084357261658, + -0.16924014687538147, + 0.00046965479850769043, + 0.3871758282184601, + -0.3993864059448242, + -0.0893002450466156, + -0.5050527453422546, + -0.16586345434188843, + -0.44161608815193176, + 0.38248682022094727, + 0.2313019037246704, + 0.18844608962535858, + -0.2711566388607025, + -0.5559765100479126 + ] + ], + "generated_utilities": [ + 2.0, + 0.061786800622940063, + 0.07495661079883575, + 0.061137303709983826, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.17512565851211548, + 2.0, + 2.0, + 2.0, + 0.10078734159469604, + 2.0, + 2.0, + 0.06518048048019409, + 2.0, + 2.0, + 0.05268773436546326, + 0.056524619460105896, + 0.07100173830986023, + 0.16304254531860352, + 2.0, + 0.07095202803611755 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -1.7685134410858154, + 0.4561518430709839, + 0.8554723262786865, + -2.9925076961517334, + -1.652122139930725, + -0.9696183204650879, + 1.440989375114441, + -0.1663060486316681, + 0.005464707035571337, + -0.08078178763389587, + 0.5969918370246887, + -1.7046172618865967, + 0.3472420275211334, + 0.7052499651908875, + -0.026682782918214798, + 0.013620201498270035, + 0.002864290028810501, + 1.5629743337631226, + 0.06437517702579498, + -0.20232005417346954, + 0.06808048486709595 + ], + [ + -1.571894645690918, + 0.49223509430885315, + 1.2358425855636597, + -2.7074813842773438, + -1.6053731441497803, + -0.9737302660942078, + 1.2407286167144775, + -0.41521748900413513, + 0.27269765734672546, + -0.08917579054832458, + 0.4081580936908722, + -1.3627166748046875, + 0.07880738377571106, + 0.7477485537528992, + -0.2180848866701126, + -0.13109977543354034, + -0.3875364661216736, + 1.8194886445999146, + -0.06430798768997192, + 0.07869948446750641, + -0.12621839344501495 + ], + [ + -1.7744590044021606, + 0.367733359336853, + 1.3591725826263428, + -3.3725903034210205, + -1.8434977531433105, + -0.8745664954185486, + 1.6967558860778809, + -0.44676536321640015, + -0.6177129149436951, + 0.03244173526763916, + 0.4171296954154968, + -1.4895254373550415, + 0.9296896457672119, + 0.15846145153045654, + -0.006281482055783272, + 0.6454329490661621, + 0.6225314140319824, + 1.6578954458236694, + 0.3061527609825134, + -0.41300463676452637, + -0.2527916431427002 + ], + [ + -1.7233139276504517, + 0.020191192626953125, + 1.0188184976577759, + -2.873849868774414, + -2.027033567428589, + -1.1650824546813965, + 2.0792155265808105, + -0.8799198865890503, + 0.6924232840538025, + -0.7650182247161865, + -0.04520058631896973, + -2.468881130218506, + -0.13257521390914917, + 0.6857031583786011, + -0.10177457332611084, + -0.7609032988548279, + -0.08249430358409882, + 1.7503252029418945, + 0.1621951013803482, + -0.22467197477817535, + 0.43942898511886597 + ], + [ + -2.3507251739501953, + 0.16114166378974915, + 1.5912489891052246, + -2.1838369369506836, + -2.061230182647705, + -1.7918524742126465, + 0.8407515287399292, + -0.3446531295776367, + -0.7727420926094055, + -0.2617931067943573, + 0.6622774004936218, + -1.05295991897583, + 0.13840271532535553, + 1.2008695602416992, + -0.509428858757019, + 0.7052024006843567, + -0.6614989042282104, + 2.0029079914093018, + -0.82997065782547, + -0.9390356540679932, + 0.7515151500701904 + ], + [ + -1.7504347562789917, + -0.13171464204788208, + 0.7620843648910522, + -3.4511537551879883, + -2.5023341178894043, + -0.8595067858695984, + 1.6361833810806274, + -0.4263937175273895, + 0.005345506593585014, + -0.8147202730178833, + 0.4142894744873047, + -0.8735606670379639, + -0.4809738099575043, + 0.9412479996681213, + -0.6222366690635681, + -0.8843234777450562, + -0.8842994570732117, + 2.4228873252868652, + 0.6301752328872681, + 0.727215051651001, + 0.5050253868103027 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.7685134410858154, + -0.04630154371261597, + -0.002802610397338867, + -1.9925076961517334, + -0.6521221399307251, + -0.013848721981048584, + 0.0, + 0.014648675918579102, + -0.0837627500295639, + -0.04761720821261406, + -0.024705946445465088, + -0.7046172618865967, + 0.0914837121963501, + 0.0, + 0.04683725908398628, + 0.048148274421691895, + 0.03868643194437027, + 0.5629743337631226, + -0.043637245893478394, + -0.08927876502275467, + 0.0 + ], + [ + -0.7685134410858154, + 0.1268729865550995, + -0.0878140926361084, + -1.9925076961517334, + -0.6521221399307251, + -0.08497345447540283, + 0.0, + -0.23278282582759857, + 0.1505891978740692, + -0.0029508620500564575, + -0.03223156929016113, + -0.7046172618865967, + -0.18528565764427185, + 0.0, + 0.009110454469919205, + 0.006383728235960007, + -0.012130538001656532, + 0.5629743337631226, + 0.23608635365962982, + -0.06009632349014282, + 0.0 + ], + [ + -0.7685134410858154, + 0.6076252460479736, + 0.2745842933654785, + -1.9925076961517334, + -0.6521221399307251, + 0.03038167953491211, + 0.0, + 0.33581337332725525, + 0.35011863708496094, + -0.06728862971067429, + -0.4030081629753113, + -0.7046172618865967, + -0.44776007533073425, + 0.0, + -0.01595483534038067, + -0.040477462112903595, + 0.28290560841560364, + 0.5629743337631226, + -0.42285358905792236, + -0.11823103576898575, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.881978750228882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.410499930381775, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.1361609697341919 + ], + [ + -1.5711963176727295, + 0.5461994409561157, + 0.8384940028190613, + -2.960416078567505, + -1.9055238962173462, + -0.7765143513679504, + 1.4308912754058838, + 0.10734349489212036, + 0.011199884116649628, + -0.028424743562936783, + 0.5893871784210205, + -1.7020739316940308, + 0.48640453815460205, + 0.9950923323631287, + 0.20675025880336761, + 0.11047534644603729, + 0.09110014140605927, + 1.3974875211715698, + -0.21079741418361664, + -0.3005509674549103, + 0.09192514419555664 + ], + [ + -2.12906813621521, + 0.3435881733894348, + 0.7164700627326965, + -2.750021457672119, + -1.3681434392929077, + -0.8937081694602966, + 1.6475064754486084, + -0.1447279453277588, + -0.44103574752807617, + -0.5038806200027466, + 0.9006609320640564, + -1.8256494998931885, + 0.433420330286026, + 0.3506351411342621, + 0.4619242548942566, + -0.3905109167098999, + -0.01803053542971611, + 1.9505069255828857, + -0.339522123336792, + -0.20715753734111786, + -0.410531610250473 + ], + [ + -1.194382667541504, + 0.9652969837188721, + 0.41722428798675537, + -2.6203644275665283, + -1.139541506767273, + -1.5412108898162842, + 1.667648434638977, + -0.5256404876708984, + 0.17716272175312042, + 0.30056339502334595, + 0.09983009099960327, + -1.8219844102859497, + -0.23339274525642395, + 0.5763673782348633, + -0.4285883605480194, + 0.1476086974143982, + -0.3515554666519165, + 1.2458031177520752, + 0.3135184645652771, + -0.33711761236190796, + -0.09264840185642242 + ], + [ + -2.0614774227142334, + -0.4080510139465332, + 0.5476135015487671, + -3.0662269592285156, + -1.497420310974121, + -0.11606889963150024, + 1.03195321559906, + -0.3406473994255066, + 0.12844708561897278, + -0.4317864775657654, + 1.035879135131836, + -0.7729326486587524, + 1.0862895250320435, + 0.16315871477127075, + -0.14380013942718506, + -0.8498450517654419, + 0.9780882000923157, + 1.0358695983886719, + -0.5399289727210999, + -0.10386796295642853, + 0.8144621849060059 + ] + ], + "predicted_scores": [ + 8.302830696105957, + 8.272442817687988, + 8.218302726745605, + 8.129446029663086, + 7.833746433258057, + 7.824558734893799, + 7.328679084777832, + 7.000413417816162 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "794181626/train0", + "selected_index": 0, + "state_hash": "4043b3c43ca4f115e1ea4fbcd98c7d08f797df2efbfe6b67bcffa5592a8aa36d", + "stored_base_utility": 0.17512565851211548, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.054018765687942505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054018765687942505 + }, + { + "contact_quality": null, + "progress": 5.517899990081787e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 5.517899990081787e-05 + }, + { + "contact_quality": null, + "progress": 0.058512330055236816, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058512330055236816 + }, + { + "contact_quality": null, + "progress": 0.00018827617168426514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00018827617168426514 + }, + { + "contact_quality": null, + "progress": 0.00025878846645355225, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00025878846645355225 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 7.644295692443848e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 7.644295692443848e-05 + } + ], + "candidate_progress": [ + 1.0, + 0.054018765687942505, + 5.517899990081787e-05, + 0.058512330055236816, + 0.00018827617168426514, + 0.00025878846645355225, + 1.0, + 7.644295692443848e-05 + ], + "candidate_source_chart_ids": [ + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-58950fed04d203b97ab9", + "generated_tangents": [ + [ + -0.1778576821088791, + 0.0036302395164966583, + 1.1676632165908813, + -0.7546417713165283, + -1.697277307510376, + -0.2011466920375824, + 0.298480749130249, + -0.07482165098190308, + -0.12784148752689362, + 0.13857874274253845, + -0.21941524744033813, + -1.5108556747436523, + 0.477994829416275, + 0.13235871493816376, + 0.2661297619342804, + 0.06012414023280144, + -0.3290677070617676, + -0.36622101068496704, + -0.9605446457862854, + -0.30346766114234924, + -0.2351449429988861 + ], + [ + -0.28704923391342163, + 0.09078311920166016, + 1.114987850189209, + -0.7498209476470947, + -1.8012349605560303, + -0.28389838337898254, + 0.18818894028663635, + -0.037162914872169495, + -0.0017223749309778214, + 0.14139193296432495, + -0.08193399757146835, + -1.5852880477905273, + 0.19839653372764587, + 0.2623569071292877, + 0.2278982251882553, + 0.01609227992594242, + -0.08104647696018219, + -0.13260115683078766, + -0.7025322318077087, + -0.2369544357061386, + -0.14419075846672058 + ], + [ + -0.2206813246011734, + 0.026308827102184296, + 1.2314008474349976, + -0.7984839677810669, + -1.924837350845337, + -0.368415504693985, + 0.1726488173007965, + -0.1132161021232605, + -0.023435836657881737, + 0.05976184457540512, + -0.1202930435538292, + -1.4768600463867188, + 0.1491192877292633, + 0.24533262848854065, + 0.22463300824165344, + -0.023225639015436172, + -0.18822930753231049, + -0.13598276674747467, + -0.7937994599342346, + -0.10444749891757965, + -0.28017333149909973 + ], + [ + -0.4899563789367676, + 0.043576642870903015, + 1.2521061897277832, + -0.92316734790802, + -1.7282285690307617, + -0.0303565114736557, + 0.5548646450042725, + -0.08239397406578064, + -0.07094340771436691, + 0.15895822644233704, + -0.1991833746433258, + -1.516670823097229, + -0.07032427191734314, + 0.3291274309158325, + 0.23166821897029877, + 0.09363971650600433, + -0.19648267328739166, + -0.3459843397140503, + -0.6647997498512268, + -0.04420539736747742, + 0.18072429299354553 + ], + [ + -0.1396852284669876, + 0.001727461814880371, + 1.2981458902359009, + -0.7096518278121948, + -1.8154969215393066, + -0.21825021505355835, + 0.14054328203201294, + -0.07004091143608093, + -0.13817119598388672, + 0.07811032235622406, + -0.22212743759155273, + -1.3904515504837036, + 0.5396687984466553, + -0.12960778176784515, + 0.19688141345977783, + 0.14898556470870972, + -0.3176721930503845, + -0.3955593705177307, + -0.9047334790229797, + -0.3793342411518097, + -0.38841238617897034 + ], + [ + -0.18346960842609406, + 0.04333057999610901, + 1.108190655708313, + -0.7183141708374023, + -1.9171998500823975, + -0.12311094999313354, + 0.10123905539512634, + -0.02744145691394806, + -0.0041778963059186935, + 0.109732985496521, + -0.13936829566955566, + -1.401587963104248, + 0.2145768702030182, + 0.16982248425483704, + 0.22343409061431885, + 0.0016652271151542664, + -0.1294211596250534, + -0.15836508572101593, + -0.742404043674469, + -0.14056962728500366, + -0.22674208879470825 + ], + [ + -0.15506576001644135, + 0.031700484454631805, + 1.0520659685134888, + -0.7341622114181519, + -1.780832052230835, + -0.44813862442970276, + 0.11466643214225769, + -0.002584487199783325, + -0.00038519036024808884, + 0.15540660917758942, + -0.10375405102968216, + -1.5776569843292236, + 0.21429434418678284, + 0.1920042335987091, + 0.33942121267318726, + 0.002302626147866249, + -0.13589848577976227, + -0.14653809368610382, + -0.8399917483329773, + -0.07796430587768555, + -0.07958358526229858 + ], + [ + -0.20411308109760284, + 0.02717725932598114, + 1.1939626932144165, + -0.866493821144104, + -1.8202178478240967, + -0.44825616478919983, + -0.0890854001045227, + -0.22147828340530396, + -0.02458440326154232, + 0.012295819818973541, + -8.651614189147949e-05, + -1.3786108493804932, + 0.38345596194267273, + -0.01583552360534668, + -0.014947537332773209, + 0.07425405085086823, + -0.43137431144714355, + -0.19424985349178314, + -0.73204106092453, + 0.02016991376876831, + -0.2934549152851105 + ] + ], + "generated_utilities": [ + 2.0, + 0.054018765687942505, + 5.517899990081787e-05, + 0.058512330055236816, + 0.00018827617168426514, + 0.00025878846645355225, + 2.0, + 7.644295692443848e-05 + ], + "group_id": "ms-58950fed04d203b97ab9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20183078944683075, + 0.18959736824035645, + 2.0, + 0.0385412722826004, + 0.15877258777618408, + 0.03365510702133179, + 0.17151890695095062, + 0.13845136761665344, + 0.11539244651794434, + 0.1388089954853058, + 0.15365298092365265, + 0.0967668741941452, + 0.0908290445804596, + 0.06094485521316528, + 0.0377749502658844, + 0.028643101453781128 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.028361663222312927, + -0.489355206489563, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.16713106632232666, + 0.0, + 0.5545767545700073, + -0.18274790048599243, + 0.2772042155265808, + -0.45601847767829895, + -1.3811633586883545, + -0.1571006327867508, + 0.0, + -0.004826217889785767, + 0.2916078567504883, + 0.34807145595550537, + -0.37882840633392334, + 0.08759653568267822, + -0.07658697664737701, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.797813892364502, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.654926061630249, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4572639465332031 + ], + [ + 0.17793144285678864, + 0.008656054735183716, + 2.343496322631836, + -1.9455183744430542, + -2.4697093963623047, + 1.0036901235580444, + 2.5124316215515137, + -0.29308056831359863, + -0.02035735547542572, + -0.059014320373535156, + -0.5863867402076721, + -2.254472255706787, + 0.3354390263557434, + 1.381989598274231, + -0.27729421854019165, + 0.5023907423019409, + -0.8654648661613464, + 0.39588332176208496, + -0.5682056546211243, + 0.20400357246398926, + 0.48529666662216187 + ], + [ + -0.5227808952331543, + 0.12448178231716156, + 2.4855635166168213, + -1.6602094173431396, + -2.3857760429382324, + 0.323077529668808, + 1.9053025245666504, + -0.698577344417572, + 0.619890034198761, + 0.3500756621360779, + -0.1476593315601349, + -2.8024404048919678, + -0.21084021031856537, + 1.0928113460540771, + -0.3938775062561035, + 0.4228009581565857, + -0.39060449600219727, + -0.31665533781051636, + -0.9205541610717773, + 0.03774743899703026, + 0.3333621621131897 + ], + [ + 0.10275222361087799, + 0.08925776183605194, + 2.0934102535247803, + -1.379586935043335, + -3.3245811462402344, + 0.6494860649108887, + 1.990661382675171, + -0.021198302507400513, + 0.4400750398635864, + 0.39176517724990845, + -0.8639137148857117, + -1.981496810913086, + 0.01414397917687893, + 1.2019504308700562, + -0.7839368581771851, + 0.108054518699646, + -0.17177742719650269, + -0.36239588260650635, + -0.5078206062316895, + 0.4236416518688202, + 0.25084057450294495 + ], + [ + -0.8110343217849731, + -0.11216431856155396, + 2.8897132873535156, + -1.319732427597046, + -3.1141648292541504, + 0.416613906621933, + 2.3244988918304443, + -0.6579729914665222, + -0.39938226342201233, + -0.08646327257156372, + -0.3429824113845825, + -2.4165658950805664, + -0.6488300561904907, + 1.4793217182159424, + -0.8628695011138916, + 0.6845848560333252, + -0.7794754505157471, + -0.014068186283111572, + -0.27444708347320557, + 0.8030790686607361, + 0.08975183963775635 + ], + [ + -0.23147638142108917, + -0.26693814992904663, + 2.8227345943450928, + -2.3573813438415527, + -3.3905396461486816, + 0.38961973786354065, + 1.9831907749176025, + -1.2505497932434082, + 0.00624750554561615, + 0.9689428210258484, + 0.11981219053268433, + -1.5865073204040527, + 0.4901374876499176, + 0.6158997416496277, + 0.3767610788345337, + -0.025944411754608154, + -1.4681568145751953, + 0.48217374086380005, + -1.5973502397537231, + -0.7199738621711731, + 0.5119494795799255 + ], + [ + -0.19429966807365417, + 0.7059361934661865, + 3.2694599628448486, + -2.656050682067871, + -2.5502617359161377, + -0.23801833391189575, + 1.628230333328247, + 0.36624830961227417, + -0.08827590942382812, + -0.2098810374736786, + -1.1745270490646362, + -2.2196781635284424, + 0.44475334882736206, + 0.48411858081817627, + -0.25864526629447937, + 0.2009485512971878, + -0.9008907079696655, + -0.3571908473968506, + -0.16894888877868652, + -0.6256314516067505, + -0.4078131914138794 + ], + [ + 0.22205571830272675, + 0.6026778221130371, + 1.8579424619674683, + -2.404672384262085, + -3.728808641433716, + -0.19806396961212158, + 2.8292319774627686, + -0.2435564398765564, + 0.7903643846511841, + -0.376501202583313, + -1.0294175148010254, + -1.7672805786132812, + 0.5992995500564575, + 2.044847249984741, + 0.0011147260665893555, + 0.18739356100559235, + -0.952850341796875, + 0.8702673316001892, + 0.038269996643066406, + -0.4372258484363556, + -0.37835830450057983 + ], + [ + -0.3783646821975708, + 0.08899453282356262, + 2.721517324447632, + -2.3160932064056396, + -2.0679585933685303, + 0.549463152885437, + 3.141833543777466, + 0.09782099723815918, + 0.5830706357955933, + 0.7134941816329956, + 0.2986952066421509, + -2.9980335235595703, + -0.6391466856002808, + 1.075132131576538, + -0.7696593403816223, + 0.4343666732311249, + -0.911625862121582, + -0.31454378366470337, + -1.5935450792312622, + -0.8993679881095886, + -0.3811922073364258 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.1209426149725914, + 0.10675475001335144, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.0875239372253418, + 0.0, + -0.0486987829208374, + 0.06580346822738647, + 0.24258655309677124, + -0.20229026675224304, + -1.3811633586883545, + 0.4185009002685547, + 0.0, + -0.1431063860654831, + 0.15154658257961273, + 0.17190229892730713, + 0.10405468940734863, + 0.036865830421447754, + -0.1988307237625122, + 0.0 + ] + ], + "predicted_scores": [ + 8.322562217712402, + 8.284337043762207, + 8.195613861083984, + 8.095207214355469, + 8.043414115905762, + 7.969996929168701, + 7.789015293121338, + 7.55466365814209 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113371261/train0", + "selected_index": 0, + "state_hash": "1b9018834a77531260bf8dfa74a9832916fe215bda78b5367b061bb4e0ce2333", + "stored_base_utility": 0.20183078944683075, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.01641605794429779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01641605794429779 + }, + "base_progress": 0.01641605794429779, + "base_success": false, + "base_utility": 0.01641605794429779, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11001592874526978, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11001592874526978 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 7.614493370056152e-06, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 7.614493370056152e-06 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.11001592874526978, + 1.0, + 1.0, + 1.0, + 7.614493370056152e-06, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-9f61e09c05b4c8f74366", + "ms-9f61e09c05b4c8f74366", + "ms-74704688fb63757619e1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + false, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-651bc0395d78d2819158", + "generated_tangents": [ + [ + -0.28055286407470703, + -0.12427914142608643, + 0.7458057403564453, + -0.8203845024108887, + -1.9704015254974365, + -0.5069563388824463, + 0.5400291681289673, + 0.056122586131095886, + -0.19652147591114044, + -0.3618527948856354, + 0.003855101764202118, + -1.6388840675354004, + 0.09804847836494446, + -0.4239356517791748, + -0.07491517066955566, + -0.09474967420101166, + 0.18653123080730438, + 0.7591179013252258, + -0.27929621934890747, + -0.4226912260055542, + 0.41058146953582764 + ], + [ + -0.25913751125335693, + -0.15451189875602722, + 0.857841968536377, + -0.7576884031295776, + -1.992818832397461, + -0.5041130781173706, + 0.3823484778404236, + 0.11783941835165024, + -0.09992609173059464, + -0.3692428767681122, + -0.042962513864040375, + -1.6736167669296265, + 0.15392783284187317, + -0.5406212210655212, + -0.04191606119275093, + -0.1567104160785675, + 0.16465872526168823, + 0.7287470102310181, + -0.266432523727417, + -0.49043866991996765, + 0.2968205213546753 + ], + [ + -0.14534591138362885, + -0.1459084153175354, + 1.0194928646087646, + -0.7109029293060303, + -1.8421368598937988, + 0.12687039375305176, + 0.4298076629638672, + -0.02768939733505249, + -0.0852721706032753, + -0.32993319630622864, + 0.12060986459255219, + -1.4413669109344482, + -0.022756069898605347, + -0.3273512125015259, + -0.018921706825494766, + -0.2069738656282425, + 0.13648608326911926, + 0.6646034717559814, + -0.16806504130363464, + -0.3461107611656189, + 0.047406673431396484 + ], + [ + -0.19669298827648163, + -0.16399496793746948, + 0.7689091563224792, + -0.4982009828090668, + -1.6042232513427734, + -0.23108428716659546, + 0.8815572261810303, + 0.11060038208961487, + -0.0574159249663353, + -0.46911656856536865, + 0.22009345889091492, + -1.443450689315796, + 0.21278473734855652, + -0.31276553869247437, + 0.10376255214214325, + -0.2275824397802353, + 0.2820473313331604, + 0.47658276557922363, + -0.14072933793067932, + -0.17545510828495026, + 0.009726181626319885 + ], + [ + -0.14645330607891083, + -0.15107746422290802, + 0.8798984289169312, + -0.7744108438491821, + -1.9667823314666748, + -0.6426529884338379, + 0.3072587549686432, + 0.010424420237541199, + -0.11162305623292923, + -0.3278885781764984, + -0.09279123693704605, + -1.683987021446228, + 0.20445826649665833, + -0.5105334520339966, + -0.033252593129873276, + -0.09740671515464783, + 0.2381344586610794, + 0.7486972212791443, + -0.14820578694343567, + -0.28766438364982605, + 0.19942304491996765 + ], + [ + -0.2849956750869751, + -0.12620314955711365, + 0.9757925271987915, + -0.6153578758239746, + -1.86106538772583, + 0.03962574899196625, + 0.4314451217651367, + 0.059229254722595215, + -0.020434828475117683, + -0.47481411695480347, + 0.016673069447278976, + -1.5006448030471802, + -0.29565802216529846, + -0.21682687103748322, + 0.12796173989772797, + -0.09710533916950226, + 0.11769160628318787, + 0.6002511382102966, + -0.2779631018638611, + -0.4966781735420227, + 0.055744290351867676 + ], + [ + -0.32628291845321655, + -0.16823628544807434, + 0.9088062644004822, + -0.6385703682899475, + -1.7459666728973389, + -3.316998481750488e-05, + 0.257204532623291, + 0.09736340492963791, + 0.07916215062141418, + -0.42485809326171875, + 0.018571551889181137, + -1.442797064781189, + -0.1283630132675171, + -0.3043360114097595, + 0.022733233869075775, + 0.0018946034833788872, + 0.27441564202308655, + 0.6587740778923035, + -0.2499735951423645, + -0.7256455421447754, + 0.12833906710147858 + ], + [ + -0.20989848673343658, + -0.09311650693416595, + 0.9274475574493408, + -0.9091185331344604, + -2.068138837814331, + -0.6280347108840942, + 0.29668983817100525, + 0.04231889545917511, + -0.07097811251878738, + -0.27199476957321167, + 0.0860631912946701, + -1.5460503101348877, + 0.16375216841697693, + -0.5114734172821045, + 0.004433806985616684, + -0.11124520003795624, + 0.3245386481285095, + 0.8809368014335632, + -0.11035174131393433, + -0.3040013313293457, + 0.12515383958816528 + ] + ], + "generated_utilities": [ + 0.11001592874526978, + 2.0, + 2.0, + 2.0, + 7.614493370056152e-06, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-651bc0395d78d2819158", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 7.152557373046875e-07, + 2.0, + 0.0, + 0.0, + 0.18490922451019287, + 0.0, + 0.18801423907279968, + 2.0, + 0.18292954564094543, + 0.06743006408214569, + 2.0, + 0.08718416094779968, + 0.0016145259141921997, + 2.0, + 0.0, + 0.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.13822472095489502, + 0.052123308181762695, + 1.2542681694030762, + -0.5262794494628906, + -1.515254020690918, + -0.022995710372924805, + 0.0, + 0.07829460501670837, + 0.056280869990587234, + -0.006563127040863037, + 0.00594717264175415, + -1.0464823246002197, + 0.028334707021713257, + 0.0, + 0.04214516654610634, + -0.0032968670129776, + 0.025241270661354065, + 1.0898823738098145, + -0.09987176954746246, + 0.009208589792251587, + 0.0 + ], + [ + -1.138224720954895, + -0.12241673469543457, + 2.254268169403076, + -1.5262794494628906, + -2.515254020690918, + -0.4036164879798889, + 2.5164785385131836, + -0.48240265250205994, + 0.09632497280836105, + -1.0002868175506592, + 0.6668505668640137, + -2.0464823246002197, + 0.5158075094223022, + 0.48990970849990845, + 0.02257649227976799, + -0.23748205602169037, + -0.2213379293680191, + 2.0898823738098145, + 0.1738651841878891, + -0.37470543384552, + -0.06145218014717102 + ], + [ + -1.0695267915725708, + -0.1775755137205124, + 2.1683619022369385, + -1.4577077627182007, + -2.7917678356170654, + -0.632304310798645, + 2.3911960124969482, + -0.4637458324432373, + 0.35316434502601624, + -0.8419650793075562, + 0.8253071308135986, + -1.9648017883300781, + 0.2602742314338684, + 0.4741981625556946, + 0.17542533576488495, + -0.15374207496643066, + -0.4857789874076843, + 2.119079828262329, + 0.44966256618499756, + -0.3760189712047577, + 0.18533895909786224 + ], + [ + -0.7955029010772705, + 0.23637625575065613, + 1.9978463649749756, + -1.8726613521575928, + -2.2562904357910156, + -0.3048732876777649, + 2.261690378189087, + -0.6620026230812073, + 0.06043636053800583, + -1.091504693031311, + 1.038455605506897, + -1.9805728197097778, + 0.7370907068252563, + 0.5352527499198914, + 0.26298987865448, + -0.2739749252796173, + 0.14720074832439423, + 2.0173659324645996, + -0.16336800158023834, + -0.5858949422836304, + -0.1647021770477295 + ], + [ + -1.6335437297821045, + -0.08274295926094055, + 1.833235502243042, + -1.758880853652954, + -2.0250279903411865, + -0.8149738311767578, + 2.4238271713256836, + -0.33182430267333984, + -0.16191768646240234, + -1.4929053783416748, + 0.8467401266098022, + -2.4791038036346436, + 0.2757382392883301, + 0.16471439599990845, + 0.18122689425945282, + -0.2995648682117462, + -0.34926944971084595, + 2.063732147216797, + 0.3726923167705536, + -0.259366512298584, + 0.36020517349243164 + ], + [ + -0.9912663698196411, + -0.19695568084716797, + 2.342681646347046, + -2.1077044010162354, + -3.031508684158325, + -0.6591198444366455, + 2.5516788959503174, + -0.8631293773651123, + 0.2481168806552887, + -1.107275366783142, + 0.5796921253204346, + -2.479954719543457, + 1.0449233055114746, + 0.40612131357192993, + -0.12985309958457947, + -0.18829946219921112, + -0.6196849346160889, + 1.6295442581176758, + 0.4197757840156555, + 0.02911210060119629, + 0.5378711223602295 + ], + [ + -0.6689683198928833, + -0.5626275539398193, + 1.738625407218933, + -1.919468879699707, + -2.6463799476623535, + -0.32590651512145996, + 2.287154197692871, + -0.9975759983062744, + 0.7224722504615784, + -1.427150845527649, + 0.572014331817627, + -2.2473490238189697, + 0.8538937568664551, + 0.0030456185340881348, + -0.1059482991695404, + -0.003734409809112549, + -0.2650519907474518, + 2.554786205291748, + -0.24461163580417633, + -0.6881535649299622, + 0.5301691293716431 + ], + [ + -1.5373890399932861, + 0.42636638879776, + 2.9180266857147217, + -2.126905679702759, + -1.825490117073059, + -0.6280819773674011, + 2.593716621398926, + -0.6465503573417664, + 0.5798410177230835, + -0.5376753807067871, + 0.5435326099395752, + -2.610151767730713, + 1.0700397491455078, + 0.7953996658325195, + 0.5072687864303589, + -0.02665230631828308, + -0.6953800320625305, + 2.0521392822265625, + 0.751386821269989, + -0.7565212249755859, + -0.5263289213180542 + ], + [ + -0.7768699526786804, + -0.18900753557682037, + 1.7192409038543701, + -2.2835047245025635, + -1.713951826095581, + 0.004747450351715088, + 3.2709672451019287, + -1.1866822242736816, + 0.38759592175483704, + -0.7857802510261536, + 1.539168119430542, + -1.3250088691711426, + 0.8453556299209595, + 0.5691420435905457, + -0.701140820980072, + -0.6115058660507202, + -0.8829987645149231, + 1.5742011070251465, + -0.3939879536628723, + 0.45414644479751587, + -0.23744438588619232 + ] + ], + "predicted_scores": [ + 9.421090126037598, + 9.321776390075684, + 8.89632797241211, + 8.840374946594238, + 8.587821960449219, + 8.185254096984863, + 8.08885669708252, + 7.968245983123779 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1483645336/train0", + "selected_index": 0, + "state_hash": "857c28f61d13d7ee5a59956a87579f6c17f6c601c1680e574691e095e373d439", + "stored_base_utility": 7.152557373046875e-07, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15461373329162598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15461373329162598 + }, + "base_progress": 0.15461373329162598, + "base_success": false, + "base_utility": 0.15461373329162598, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06999577581882477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06999577581882477 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03132614493370056, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03132614493370056 + }, + { + "contact_quality": null, + "progress": 0.023272648453712463, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023272648453712463 + }, + { + "contact_quality": null, + "progress": 0.030094027519226074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030094027519226074 + }, + { + "contact_quality": null, + "progress": 0.04187875986099243, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04187875986099243 + }, + { + "contact_quality": null, + "progress": 0.0486045777797699, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0486045777797699 + } + ], + "candidate_progress": [ + 0.06999577581882477, + 1.0, + 1.0, + 0.03132614493370056, + 0.023272648453712463, + 0.030094027519226074, + 0.04187875986099243, + 0.0486045777797699 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9fad82fd288e4296e354", + "generated_tangents": [ + [ + -0.1998446136713028, + -0.13613846898078918, + 1.1954774856567383, + -0.19622287154197693, + -1.2181291580200195, + -0.09228421747684479, + -0.5131031274795532, + -0.09343284368515015, + 0.034097351133823395, + 0.5444074869155884, + -0.06611961126327515, + -1.2418341636657715, + -0.07458993792533875, + 0.09996648132801056, + -0.08733539283275604, + 0.02461063489317894, + 0.11653249710798264, + -0.5559351444244385, + -0.22178617119789124, + -0.018946915864944458, + 0.09585287421941757 + ], + [ + 0.10768678784370422, + -0.14886173605918884, + 1.122004747390747, + -0.30997174978256226, + -1.2846577167510986, + -0.4357328712940216, + -0.6459855437278748, + 0.028352245688438416, + 0.13504968583583832, + 0.46079155802726746, + 0.015527933835983276, + -1.360116720199585, + 0.17759987711906433, + -0.19781215488910675, + 0.034844718873500824, + 0.016100937500596046, + 0.08097121119499207, + -0.40727776288986206, + -0.3540729284286499, + -0.2883093059062958, + -0.13371184468269348 + ], + [ + 0.018745824694633484, + -0.2062220275402069, + 1.1598033905029297, + -0.1009679064154625, + -1.332383632659912, + -0.4028060734272003, + -0.9239940047264099, + 0.0017495453357696533, + 0.06034529209136963, + 0.5863930583000183, + -0.13289305567741394, + -1.2339847087860107, + 0.06810715794563293, + -0.2841215133666992, + -0.10978451371192932, + 0.14976097643375397, + 0.16245397925376892, + -0.4142678380012512, + -0.37755507230758667, + 0.04395031929016113, + 0.09613826125860214 + ], + [ + 0.1592612862586975, + -0.1608738750219345, + 1.1876754760742188, + -0.33564597368240356, + -1.2400803565979004, + -0.4422418773174286, + -0.6213474869728088, + 0.00048807263374328613, + 0.0908799022436142, + 0.4733821749687195, + 0.018075991421937943, + -1.3230524063110352, + 0.12521758675575256, + -0.26577723026275635, + 0.10867087543010712, + 0.09838718175888062, + 0.14024819433689117, + -0.38509494066238403, + -0.3966064453125, + -0.11265681684017181, + 0.008435025811195374 + ], + [ + 0.09822267293930054, + -0.14454904198646545, + 1.1871142387390137, + -0.0997898131608963, + -1.2865939140319824, + -0.485890656709671, + -0.9018321633338928, + -0.009793668985366821, + 0.08223867416381836, + 0.5608381032943726, + -0.08273427933454514, + -1.2424618005752563, + 0.03088495135307312, + -0.25332319736480713, + -0.0384976752102375, + 0.13081185519695282, + 0.16568070650100708, + -0.4293673038482666, + -0.37687379121780396, + 0.0026108622550964355, + 0.11234897375106812 + ], + [ + 0.10180012881755829, + -0.1550700068473816, + 1.188444972038269, + -0.25300154089927673, + -1.3772516250610352, + -0.4051682651042938, + -0.8206649422645569, + 0.031919628381729126, + 0.06656257808208466, + 0.5998708009719849, + 0.17621514201164246, + -1.5509395599365234, + -0.40087807178497314, + -0.2322075217962265, + -0.05952772870659828, + 0.1748443990945816, + 0.08963941037654877, + -0.37823981046676636, + -0.3356948494911194, + 0.21281248331069946, + 0.004816889762878418 + ], + [ + 0.2217009961605072, + -0.25093889236450195, + 1.0543618202209473, + -0.1766597479581833, + -1.1321866512298584, + -0.5494941473007202, + -0.55526202917099, + -0.0037053674459457397, + -0.013644784688949585, + 0.5054594874382019, + -0.29368942975997925, + -1.4012982845306396, + 0.36760565638542175, + -0.32580989599227905, + 0.1359534114599228, + 0.25731244683265686, + 0.052984341979026794, + -0.7190414667129517, + -0.3065682649612427, + -0.07599075138568878, + -0.3095443546772003 + ], + [ + 0.2995772361755371, + -0.19727033376693726, + 1.2040073871612549, + -0.15190288424491882, + -1.2154841423034668, + -0.327819287776947, + -0.9828824400901794, + 0.12254729866981506, + -0.12098874151706696, + 0.6633365750312805, + -0.18690431118011475, + -1.6477895975112915, + 0.1402067244052887, + 0.2056141048669815, + 0.04854009672999382, + 0.2965231239795685, + 0.13651099801063538, + -0.5691934823989868, + -0.36577552556991577, + 0.3864631652832031, + 0.024560481309890747 + ] + ], + "generated_utilities": [ + 0.06999577581882477, + 2.0, + 2.0, + 0.03132614493370056, + 0.023272648453712463, + 0.030094027519226074, + 0.04187875986099243, + 0.0486045777797699 + ], + "group_id": "ms-9fad82fd288e4296e354", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.19493231177330017, + 0.06623455882072449, + 2.0, + 0.15396031737327576, + 0.16184639930725098, + 0.025735855102539062, + 0.134642094373703, + 0.15606965124607086, + 0.12982505559921265, + 0.14038166403770447, + 0.09718823432922363, + 0.015769004821777344, + 0.09577034413814545, + 0.057760089635849, + 0.047366172075271606, + 0.035309627652168274 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.0622546412050724, + -0.09277644753456116, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + -0.005287081003189087, + 0.0, + -0.005681931972503662, + 0.03343720734119415, + 1.0536105632781982, + -0.00917971134185791, + -1.333775520324707, + 0.03716959059238434, + 0.0, + 0.019381746649742126, + 0.026990801095962524, + 0.21505475044250488, + -0.029777705669403076, + -0.14875531196594238, + -0.048092663288116455, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.959449768066406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.103323459625244, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.591951370239258 + ], + [ + -0.13909977674484253, + -0.26507115364074707, + 2.099381446838379, + -1.432984709739685, + -2.215683698654175, + 0.18431386351585388, + 2.42065691947937, + 0.3679453730583191, + -0.24726781249046326, + 2.055530071258545, + -0.4911385774612427, + -2.0511183738708496, + 0.04109602048993111, + 2.784976005554199, + 0.005537793040275574, + 0.05149966478347778, + 1.124292016029358, + -0.6457813382148743, + -1.226535677909851, + -0.28760284185409546, + 1.0270601511001587 + ], + [ + 0.30531278252601624, + -0.4336698055267334, + 2.6217315196990967, + -1.5083816051483154, + -2.5322670936584473, + 0.06784650683403015, + 2.3866119384765625, + -0.09851980209350586, + 0.2774229049682617, + 2.157179355621338, + -0.9796892404556274, + -2.8201849460601807, + 0.08135517686605453, + 2.818830728530884, + -0.6848310828208923, + 0.37913888692855835, + 0.7420558929443359, + -0.22611916065216064, + -1.5158166885375977, + 0.0614812970161438, + 1.3468891382217407 + ], + [ + 0.27444392442703247, + -0.22534464299678802, + 2.616544246673584, + -1.2606401443481445, + -2.0848917961120605, + -0.23496872186660767, + 2.1798665523529053, + 0.7968885898590088, + 0.26037585735321045, + 1.5473439693450928, + -0.233917236328125, + -1.867544412612915, + 0.48288947343826294, + 1.9728566408157349, + -0.4071806073188782, + 0.5528238415718079, + 1.6528295278549194, + -1.0936471223831177, + -1.062675952911377, + -0.5311015248298645, + 0.9778987765312195 + ], + [ + 0.5435068607330322, + 0.24469684064388275, + 2.6045663356781006, + -1.284223198890686, + -2.6095967292785645, + 0.9732104539871216, + 2.741159200668335, + 0.6031707525253296, + -0.5612202882766724, + 2.4774889945983887, + 0.1548280119895935, + -2.5516507625579834, + -0.0651187151670456, + 2.3864798545837402, + 0.130750373005867, + -0.1318313330411911, + 1.6559979915618896, + -1.0673086643218994, + -0.5477704405784607, + 0.18809780478477478, + 0.7076419591903687 + ], + [ + 0.31294435262680054, + 0.0263526514172554, + 3.0396268367767334, + -1.3165022134780884, + -1.8981198072433472, + 0.5049136281013489, + 2.31935715675354, + -0.4664327800273895, + -0.56757652759552, + 2.7719295024871826, + 0.21425002813339233, + -2.3925602436065674, + -0.26789456605911255, + 2.008218288421631, + -0.1879691481590271, + -0.34721577167510986, + 1.6638673543930054, + -0.30173182487487793, + -1.8719977140426636, + -0.8951039910316467, + 2.094865083694458 + ], + [ + 0.533157467842102, + 0.308005690574646, + 1.7770951986312866, + -0.6794775724411011, + -3.2953453063964844, + -0.15337547659873962, + 2.5854313373565674, + 0.07320006191730499, + -0.24033141136169434, + 2.5622174739837646, + -0.9168426990509033, + -1.5317132472991943, + -0.5942863821983337, + 2.2703769207000732, + -1.0709311962127686, + 0.6969255208969116, + 1.5304101705551147, + -0.7959219217300415, + -0.9165043830871582, + -0.8046965003013611, + 0.638719916343689 + ], + [ + 0.7393907904624939, + -0.13299384713172913, + 2.4281864166259766, + -1.861674189567566, + -3.3330533504486084, + -0.18516618013381958, + 1.6492295265197754, + -0.6932566165924072, + 0.06020718812942505, + 1.1755363941192627, + -0.016285300254821777, + -3.0671546459198, + -0.6052039861679077, + 2.6829426288604736, + 0.7021514177322388, + -0.477348268032074, + 0.466697633266449, + -0.9403301477432251, + -0.8690205812454224, + -1.1888904571533203, + 0.5360406041145325 + ], + [ + -0.0713023692369461, + -0.9776724576950073, + 1.2895376682281494, + -1.6489125490188599, + -1.4475497007369995, + -0.6822370886802673, + 3.14093017578125, + -0.5356632471084595, + -0.7523763179779053, + 2.766268253326416, + -0.8215572834014893, + -2.753042697906494, + 0.10929550230503082, + 2.570046901702881, + -0.34730663895606995, + 0.8763631582260132, + 1.3356308937072754, + -0.14330342411994934, + -1.1863114833831787, + -0.30006271600723267, + 0.7264611721038818 + ], + [ + 0.35943418741226196, + 0.22802312672138214, + 1.7158434391021729, + -1.4605827331542969, + -3.3450541496276855, + 0.23581582307815552, + 1.759531855583191, + 0.7012510299682617, + 0.0034519433975219727, + 1.4857903718948364, + -1.2866599559783936, + -2.1787426471710205, + 0.8807055354118347, + 1.9066205024719238, + -0.9918697476387024, + -0.32221919298171997, + 1.1986194849014282, + -1.187239646911621, + -0.5752912163734436, + -0.015687286853790283, + 1.775393009185791 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16282694041728973, + -0.1309112012386322, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + 0.15446226298809052, + 0.0, + -0.15669959783554077, + -0.08878482133150101, + 1.0536105632781982, + -0.09701520204544067, + -1.333775520324707, + 0.10988402366638184, + 0.0, + -0.06573517620563507, + -0.2108776718378067, + 0.21505475044250488, + -0.16805344820022583, + -0.14875531196594238, + 0.12529203295707703, + 0.0 + ] + ], + "predicted_scores": [ + 9.313835144042969, + 9.259336471557617, + 8.602334022521973, + 8.550065040588379, + 8.497767448425293, + 7.882334232330322, + 7.811857223510742, + 6.602944374084473 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1176943004/train0", + "selected_index": 0, + "state_hash": "0d923438bf2b227e3521cb937d011f5bddf9b553a882853098b702ec31090614", + "stored_base_utility": 0.19493231177330017, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06781657040119171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06781657040119171 + }, + "base_progress": 0.06781657040119171, + "base_success": false, + "base_utility": 0.06781657040119171, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09167696535587311, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09167696535587311 + }, + { + "contact_quality": null, + "progress": 0.00022964179515838623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00022964179515838623 + }, + { + "contact_quality": null, + "progress": 0.0006267279386520386, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006267279386520386 + }, + { + "contact_quality": null, + "progress": 6.0498714447021484e-06, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 6.0498714447021484e-06 + }, + { + "contact_quality": null, + "progress": 0.00010013580322265625, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00010013580322265625 + }, + { + "contact_quality": null, + "progress": 1.5795230865478516e-06, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.5795230865478516e-06 + }, + { + "contact_quality": null, + "progress": 0.002237573266029358, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.002237573266029358 + }, + { + "contact_quality": null, + "progress": 0.09276536107063293, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09276536107063293 + } + ], + "candidate_progress": [ + 0.09167696535587311, + 0.00022964179515838623, + 0.0006267279386520386, + 6.0498714447021484e-06, + 0.00010013580322265625, + 1.5795230865478516e-06, + 0.002237573266029358, + 0.09276536107063293 + ], + "candidate_source_chart_ids": [ + "ms-dec13f2e40baaf5c8d40", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "generated_tangents": [ + [ + -0.07401131093502045, + -0.04983069375157356, + 1.0861337184906006, + -0.7663307189941406, + -1.9048068523406982, + -0.47873374819755554, + 0.08673620223999023, + -0.015595436096191406, + 0.059554338455200195, + 0.11692804843187332, + -0.01721273362636566, + -1.3845481872558594, + 0.2936886250972748, + -0.06758728623390198, + -0.11758345365524292, + 0.41535380482673645, + -0.004899993538856506, + -0.6369214057922363, + -0.5080356597900391, + -0.43489518761634827, + 0.021485134959220886 + ], + [ + -0.11963720619678497, + 0.023982718586921692, + 1.0234575271606445, + -0.956607460975647, + -1.6449265480041504, + -0.47221317887306213, + -0.10534799098968506, + 0.1610494703054428, + 0.13252530992031097, + -0.015598975121974945, + 0.043594684451818466, + -1.275907039642334, + 0.17958441376686096, + 0.04892846941947937, + -0.0540248341858387, + 0.3318483233451843, + 0.08719268441200256, + -0.8543687462806702, + -0.43198680877685547, + -0.03569546341896057, + -0.11463424563407898 + ], + [ + -0.1281290501356125, + 0.07537854462862015, + 1.1528681516647339, + -0.9410957098007202, + -1.7254095077514648, + -0.3934711515903473, + -0.19175148010253906, + 0.12348711490631104, + 0.08994673937559128, + -0.037045590579509735, + -0.03958366811275482, + -1.3818869590759277, + 0.09270164370536804, + -0.035149917006492615, + -0.04040056839585304, + 0.38400712609291077, + 0.028642013669013977, + -0.8570398688316345, + -0.5078204274177551, + -0.09519390761852264, + -0.0698556900024414 + ], + [ + -0.10154861211776733, + -0.05857856199145317, + 1.125746250152588, + -0.6748766899108887, + -1.605238676071167, + -0.7067629098892212, + -0.12017327547073364, + 0.0111217200756073, + 0.07350906729698181, + -0.033876873552799225, + -0.07184863835573196, + -1.2392839193344116, + 0.20886299014091492, + 0.2580723762512207, + 0.03831881284713745, + 0.2756733000278473, + 0.2141614854335785, + -0.6981217861175537, + -0.5911544561386108, + -0.3402017652988434, + -0.33051034808158875 + ], + [ + -0.1350162774324417, + 0.04171042889356613, + 0.7024990320205688, + -0.6363195776939392, + -1.4881393909454346, + -0.32491403818130493, + 0.13575655221939087, + 0.06681013107299805, + 0.05388261377811432, + -0.00012364238500595093, + 0.05719592794775963, + -1.1104090213775635, + 0.5736833810806274, + 0.1972261369228363, + -0.16631761193275452, + 0.17855453491210938, + 0.019907936453819275, + -0.5646957159042358, + 0.04457584023475647, + -0.1348414570093155, + 0.7787601947784424 + ], + [ + -0.09109653532505035, + -0.10604146867990494, + 1.1458743810653687, + -0.6595280170440674, + -1.6466548442840576, + -0.6165168285369873, + -0.24602818489074707, + -0.02194201946258545, + 0.030948543921113014, + -0.13117703795433044, + -0.23636949062347412, + -1.1983355283737183, + 0.08925655484199524, + 0.24316507577896118, + 0.10543453693389893, + 0.33899813890457153, + 0.18528787791728973, + -0.680165708065033, + -0.6733214259147644, + -0.20355458557605743, + -0.26392191648483276 + ], + [ + -0.14924006164073944, + 0.0018317140638828278, + 1.1730269193649292, + -0.9112308025360107, + -1.6417372226715088, + -0.4951991140842438, + -0.2560999393463135, + 0.004106879234313965, + 0.1028987318277359, + -0.035963691771030426, + 0.01761562004685402, + -1.3713557720184326, + 0.4157550036907196, + 0.2217365801334381, + 0.0577625073492527, + 0.36985331773757935, + 0.23366516828536987, + -0.949657142162323, + -0.6014523506164551, + 0.19395634531974792, + 0.06467265635728836 + ], + [ + -0.06714653968811035, + 0.0016189664602279663, + 1.1881128549575806, + -0.6453325748443604, + -1.7248032093048096, + -0.7292300462722778, + -0.04763680696487427, + 0.030370160937309265, + 0.06427609175443649, + 0.07249383628368378, + -0.13684776425361633, + -1.2338018417358398, + 0.1924196183681488, + 0.21586701273918152, + 0.034250739961862564, + 0.3291316628456116, + 0.20251800119876862, + -0.7360188364982605, + -0.508668839931488, + -0.14231330156326294, + -0.2979564368724823 + ] + ], + "generated_utilities": [ + 0.09167696535587311, + 0.00022964179515838623, + 0.0006267279386520386, + 6.0498714447021484e-06, + 0.00010013580322265625, + 1.5795230865478516e-06, + 0.002237573266029358, + 0.09276536107063293 + ], + "group_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0793772041797638, + 2.0, + 2.0, + 2.0, + 0.1144598126411438, + 0.04584880173206329, + 0.08543911576271057, + 0.1307348757982254, + 0.10944780707359314, + 0.07284757494926453, + 0.05719476938247681, + 0.09351089596748352, + 0.13337698578834534, + 0.039893895387649536, + 0.09353269636631012, + 0.08224260807037354 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.016482055187225342, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.0903974175453186, + 0.0, + 0.012217208743095398, + 0.010632793419063091, + 0.03624187409877777, + 0.04524834081530571, + -1.3283450603485107, + -0.00880381464958191, + 0.0, + -0.014520794153213501, + 0.5632922649383545, + 0.003262042999267578, + -0.11586570739746094, + 0.0014255642890930176, + 0.015446990728378296, + 0.0 + ], + [ + 0.04007059335708618, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.12278485298156738, + 0.0, + -0.0962398424744606, + -0.08766138553619385, + 0.29553186893463135, + 0.10519622266292572, + -1.3283450603485107, + 0.00032788515090942383, + 0.0, + -0.13836456835269928, + 0.5632922649383545, + -0.13551044464111328, + -0.11586570739746094, + 0.07183873653411865, + -0.03684847429394722, + 0.0 + ], + [ + 0.08037775754928589, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.33122140169143677, + 0.0, + 0.20786377787590027, + -0.42098182439804077, + -0.17684945464134216, + 0.1062583178281784, + -1.3283450603485107, + 0.28051087260246277, + 0.0, + 0.7470210194587708, + 0.5632922649383545, + 0.03247350454330444, + -0.11586570739746094, + 0.4809897243976593, + -0.1744043231010437, + 0.0 + ], + [ + -0.4189682602882385, + 1.1229554414749146, + 2.8264856338500977, + -2.2389113903045654, + -2.4226768016815186, + 1.2001769542694092, + 2.1568119525909424, + -0.016906127333641052, + -0.24083517491817474, + -0.23574936389923096, + -0.13933053612709045, + -2.458132028579712, + 0.40251442790031433, + 0.870932936668396, + -0.28115370869636536, + 1.4463695287704468, + 0.9248121380805969, + -0.9938455820083618, + -0.680131196975708, + -0.051702484488487244, + -0.4663417339324951 + ], + [ + -0.6373537182807922, + 1.3288449048995972, + 3.1669085025787354, + -1.4171695709228516, + -2.242492198944092, + 1.6900161504745483, + 2.778874158859253, + 0.47899991273880005, + 0.3402186334133148, + 0.4507899582386017, + 0.8237879276275635, + -1.7865383625030518, + 1.1732149124145508, + 1.5826115608215332, + 0.4416889548301697, + 2.222722053527832, + 0.6443713307380676, + -1.8598005771636963, + -0.9690301418304443, + 0.3504742383956909, + -1.1371408700942993 + ] + ], + "predicted_scores": [ + 9.256302833557129, + 8.243694305419922, + 8.192558288574219, + 8.030529022216797, + 7.7356977462768555, + 7.631898880004883, + 7.5903639793396, + 7.43136739730835 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "998761162/train0", + "selected_index": 0, + "state_hash": "78cc63bdeba9f4d6d6da2efd23c0f7f6631321321e917a70299a7c8025e71d47", + "stored_base_utility": 0.0793772041797638, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11926940083503723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11926940083503723 + }, + { + "contact_quality": null, + "progress": 0.12008294463157654, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12008294463157654 + }, + { + "contact_quality": null, + "progress": 0.0918070524930954, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0918070524930954 + }, + { + "contact_quality": null, + "progress": 0.12258778512477875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12258778512477875 + }, + { + "contact_quality": null, + "progress": 0.04670017957687378, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04670017957687378 + }, + { + "contact_quality": null, + "progress": 0.10160571336746216, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10160571336746216 + }, + { + "contact_quality": null, + "progress": 0.085929274559021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.085929274559021 + }, + { + "contact_quality": null, + "progress": 0.13679209351539612, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13679209351539612 + } + ], + "candidate_progress": [ + 0.11926940083503723, + 0.12008294463157654, + 0.0918070524930954, + 0.12258778512477875, + 0.04670017957687378, + 0.10160571336746216, + 0.085929274559021, + 0.13679209351539612 + ], + "candidate_source_chart_ids": [ + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-fc8084d5cd7da937fc02", + "generated_tangents": [ + [ + 0.03669255971908569, + -0.056140247732400894, + 1.241540551185608, + -0.5289525985717773, + -1.5482816696166992, + -0.28787001967430115, + -0.15741264820098877, + 0.08146894723176956, + -0.14238183200359344, + 0.09228740632534027, + 0.047836124897003174, + -1.544621229171753, + 0.6215853691101074, + -0.5542036890983582, + 0.12395864725112915, + -0.016038352623581886, + -0.002518489956855774, + -0.09731416404247284, + -0.4493175745010376, + -0.407426655292511, + -0.2785944640636444 + ], + [ + 0.013331606984138489, + -0.046189721673727036, + 1.1529415845870972, + -0.5309498310089111, + -1.622788667678833, + -0.27052074670791626, + -0.1664489507675171, + 0.09679500013589859, + -0.16496385633945465, + 0.12681393325328827, + 0.046891264617443085, + -1.5165820121765137, + 0.4988780915737152, + -0.4027971029281616, + 0.15612731873989105, + 0.02184528112411499, + 0.08176298439502716, + -0.08559386432170868, + -0.5042839646339417, + -0.25376665592193604, + -0.10230544209480286 + ], + [ + -0.06215904653072357, + -0.07553570717573166, + 1.1406681537628174, + -0.6841037273406982, + -1.4110968112945557, + -0.3699903190135956, + -0.28840702772140503, + 0.026922211050987244, + -0.039488911628723145, + 0.11670631170272827, + 0.1159554272890091, + -1.5273253917694092, + 0.06421884894371033, + -0.15283460915088654, + 0.18330299854278564, + -0.0704086571931839, + -0.029702797532081604, + -0.1403348594903946, + -0.21635115146636963, + -0.15562571585178375, + -0.23336240649223328 + ], + [ + 0.018672600388526917, + -0.08738701045513153, + 1.168372392654419, + -0.6658843755722046, + -1.414001226425171, + -0.34462401270866394, + -0.29870617389678955, + 0.07341969758272171, + -0.07476446777582169, + 0.08924899995326996, + -0.03867051750421524, + -1.5284569263458252, + -0.03962814807891846, + -0.18334807455539703, + 0.23609218001365662, + -0.03537214547395706, + -0.06439022719860077, + -0.12337668240070343, + -0.32380378246307373, + -0.20941540598869324, + -0.1552450954914093 + ], + [ + 0.07217048108577728, + -0.0567023865878582, + 1.09761381149292, + -0.5650160312652588, + -1.452897071838379, + -0.3664596974849701, + -0.2504051923751831, + 0.018995091319084167, + -0.05408535897731781, + 0.08457290381193161, + 0.004011616110801697, + -1.353523850440979, + 0.0910172164440155, + -0.14501617848873138, + 0.1858120709657669, + -0.05699671059846878, + -0.03672941029071808, + -0.05039876699447632, + -0.26932191848754883, + -0.1295747309923172, + -0.24405178427696228 + ], + [ + 0.0012893974781036377, + -0.030742498114705086, + 1.1018292903900146, + -0.6289820671081543, + -1.3554205894470215, + -0.4472218453884125, + -0.3116702437400818, + 0.022877201437950134, + -0.042790912091732025, + 0.08544452488422394, + -0.017446249723434448, + -1.4039801359176636, + 0.2083112895488739, + -0.14769314229488373, + 0.15416696667671204, + -0.0833895355463028, + -0.07448841631412506, + -0.10906194150447845, + -0.2639598250389099, + -0.24095386266708374, + -0.2905418872833252 + ], + [ + -0.019429758191108704, + -0.06427917629480362, + 1.1811469793319702, + -0.6200525760650635, + -1.438051700592041, + -0.448493629693985, + -0.38918638229370117, + 0.04172700643539429, + -0.09341886639595032, + 0.15284757316112518, + 0.014197561889886856, + -1.3975614309310913, + 0.1681613028049469, + -0.11636488139629364, + 0.2413283884525299, + -0.02814667858183384, + 0.039856329560279846, + -0.15920402109622955, + -0.34145277738571167, + 0.05902218818664551, + -0.23194381594657898 + ], + [ + -0.004085913300514221, + 0.04330699145793915, + 0.8578941822052002, + -0.21346747875213623, + -1.3380348682403564, + -0.4132215082645416, + -0.14985018968582153, + -0.006819739937782288, + -0.15147656202316284, + 0.03852641582489014, + 0.12215299904346466, + -1.341388463973999, + 0.7584611177444458, + -0.15727286040782928, + 0.0740395188331604, + -0.14490386843681335, + -0.09812299907207489, + -0.028176352381706238, + -0.22412532567977905, + 0.20315155386924744, + -0.004033252596855164 + ] + ], + "generated_utilities": [ + 0.11926940083503723, + 0.12008294463157654, + 0.0918070524930954, + 0.12258778512477875, + 0.04670017957687378, + 0.10160571336746216, + 0.085929274559021, + 0.13679209351539612 + ], + "group_id": "ms-fc8084d5cd7da937fc02", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.120394766330719, + 0.1662210375070572, + 2.0, + 0.0270967036485672, + 0.16474007070064545, + 0.014024153351783752, + 0.13986149430274963, + 0.17214345932006836, + 0.19527268409729004, + 0.08535367250442505, + 0.1354738473892212, + 0.11415943503379822, + 0.05034071207046509, + 0.10602898895740509, + 0.01804502308368683, + 0.09680917859077454 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.4742112457752228, + -0.6877930760383606, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.0012008249759674072, + 0.0, + 0.04633669555187225, + -0.307714581489563, + -0.22357922792434692, + -0.35245198011398315, + -1.486854076385498, + 0.15573161840438843, + 0.0, + 0.5157145261764526, + 0.243257537484169, + 0.2085912823677063, + 0.48896148800849915, + -0.06008961796760559, + 0.4603208899497986, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.883587837219238, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6201469898223877, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.5597648620605469 + ], + [ + -0.8389496207237244, + 0.5873794555664062, + 2.327263116836548, + -2.175445795059204, + -1.844232439994812, + 0.15723112225532532, + 2.920079231262207, + -0.5335484743118286, + -0.6761462688446045, + 0.48151493072509766, + 0.2968217730522156, + -2.99812650680542, + -0.27805766463279724, + 2.0262699127197266, + 0.10151885449886322, + -0.7244771122932434, + 1.0176090002059937, + 1.278473973274231, + -0.2260074019432068, + -1.0207180976867676, + 0.7682583928108215 + ], + [ + 0.2503487765789032, + 0.20628705620765686, + 2.289511203765869, + -0.8669976592063904, + -1.9807379245758057, + 1.1820356845855713, + 2.8384368419647217, + 0.43699324131011963, + -0.688888669013977, + 1.0812339782714844, + -0.18395745754241943, + -1.6906647682189941, + -0.040969282388687134, + 1.254612684249878, + 0.24077603220939636, + -1.0562812089920044, + 1.2436273097991943, + 0.2388666272163391, + -0.5016235709190369, + 0.4570234417915344, + 1.355008840560913 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.10285450518131256, + -0.2634444832801819, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.12851905822753906, + 0.0, + 0.06062778830528259, + 0.006933145225048065, + -0.08638572692871094, + 0.32057398557662964, + -1.486854076385498, + -0.1531560719013214, + 0.0, + 0.14251592755317688, + 0.07826228439807892, + -0.24055713415145874, + -0.11116695404052734, + 0.200389564037323, + -0.07051187753677368, + 0.0 + ], + [ + -0.07637833058834076, + 0.6696901321411133, + 2.130587100982666, + -1.9006763696670532, + -2.6654670238494873, + 0.06729818880558014, + 2.65816593170166, + -0.0612010583281517, + -0.1226595938205719, + 0.9200000166893005, + -0.4352770149707794, + -2.266615390777588, + 0.48697614669799805, + 1.5162358283996582, + -0.014945849776268005, + -0.4459138512611389, + 1.1448073387145996, + 0.8031747937202454, + -0.6817312240600586, + -0.20262596011161804, + 1.1690938472747803 + ], + [ + 0.519248902797699, + 0.27624276280403137, + 1.911887764930725, + -1.8784401416778564, + -2.9212372303009033, + 0.7762739658355713, + 2.547428846359253, + 0.08362685143947601, + -0.1671719253063202, + 0.8596282601356506, + -0.17510351538658142, + -2.220954656600952, + 0.4918195605278015, + 1.357650876045227, + 0.3462192416191101, + 0.2476332038640976, + 0.6965686082839966, + 1.0915483236312866, + -0.6818809509277344, + -0.7465203404426575, + 0.724744439125061 + ] + ], + "predicted_scores": [ + 9.291539192199707, + 9.234026908874512, + 9.123018264770508, + 8.854728698730469, + 8.799765586853027, + 8.680414199829102, + 8.657164573669434, + 8.29908561706543 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1991414747/train0", + "selected_index": 0, + "state_hash": "127cf975da326f174c815484811a6acc4855bf07460a2aa5b0d728cb2a1f41c7", + "stored_base_utility": 0.120394766330719, + "task_id": "PushCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20275163650512695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20275163650512695 + }, + "base_progress": 0.20275163650512695, + "base_success": false, + "base_utility": 0.20275163650512695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.22510160505771637, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22510160505771637 + }, + { + "contact_quality": null, + "progress": 0.20951656997203827, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20951656997203827 + }, + { + "contact_quality": null, + "progress": 0.22272604703903198, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22272604703903198 + }, + { + "contact_quality": null, + "progress": 0.22567233443260193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22567233443260193 + }, + { + "contact_quality": null, + "progress": 0.2283252328634262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2283252328634262 + }, + { + "contact_quality": null, + "progress": 0.22638562321662903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22638562321662903 + }, + { + "contact_quality": null, + "progress": 0.1809503734111786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1809503734111786 + }, + { + "contact_quality": null, + "progress": 0.12205810844898224, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12205810844898224 + } + ], + "candidate_progress": [ + 0.22510160505771637, + 0.20951656997203827, + 0.22272604703903198, + 0.22567233443260193, + 0.2283252328634262, + 0.22638562321662903, + 0.1809503734111786, + 0.12205810844898224 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3afd8959363e917591d4", + "generated_tangents": [ + [ + 0.21758851408958435, + 0.45839619636535645, + -0.36980870366096497, + -1.0366790294647217, + 0.02991729974746704, + -0.614129900932312, + 0.618423581123352, + -0.06544056534767151, + -0.12036630511283875, + -0.11159460991621017, + 0.43743187189102173, + -0.1392863690853119, + -1.8090202808380127, + -0.5089260935783386, + -0.16651156544685364, + -0.015876447781920433, + 0.00580163300037384, + 0.04044273495674133, + -0.34339046478271484, + -1.698594093322754, + -0.002629324793815613 + ], + [ + 0.26656264066696167, + 0.43753552436828613, + -0.4475541412830353, + -1.1809070110321045, + 0.05887360870838165, + -0.204149067401886, + 0.40684348344802856, + -0.0066803693771362305, + -0.15968716144561768, + -0.183632493019104, + 0.1343637853860855, + -0.37479788064956665, + -1.9812617301940918, + -0.5234700441360474, + -0.22862210869789124, + -0.0699257180094719, + 0.1735115647315979, + -0.015770673751831055, + -0.42854249477386475, + -1.456526756286621, + 0.2175431102514267 + ], + [ + 0.09575708955526352, + 0.4790728688240051, + -0.3521564304828644, + -0.9997414350509644, + 0.20344986021518707, + -0.42155060172080994, + 0.46642380952835083, + -0.022058755159378052, + -0.14270851016044617, + -0.16610124707221985, + 0.1806793063879013, + -0.2857516407966614, + -1.8942155838012695, + -0.30111968517303467, + -0.21404579281806946, + -0.029661960899829865, + 0.2486855685710907, + 0.01654106378555298, + -0.2938593626022339, + -1.6678509712219238, + 0.15769878029823303 + ], + [ + 0.026058882474899292, + 0.3004976511001587, + -0.12490341067314148, + -0.9865210056304932, + 0.4065275490283966, + -0.15159425139427185, + 0.33865076303482056, + 0.023164525628089905, + -0.05884639173746109, + -0.09854967147111893, + -0.0044835880398750305, + -0.25652483105659485, + -1.9442698955535889, + 0.14995869994163513, + 0.015987791121006012, + -0.07364068925380707, + 0.22663886845111847, + 0.040101274847984314, + 0.10516948997974396, + -1.4604427814483643, + 0.30864012241363525 + ], + [ + 0.08715064823627472, + 0.3317517638206482, + -0.08255305886268616, + -1.070832371711731, + 0.43441343307495117, + -0.0382179319858551, + 0.03654316067695618, + 0.0064200758934021, + -0.17202529311180115, + -0.14261490106582642, + -0.034951142966747284, + -0.10838693380355835, + -1.9902634620666504, + 0.012336239218711853, + -0.006129477173089981, + -0.04287237673997879, + 0.15029937028884888, + -0.08352373540401459, + -0.09005069732666016, + -1.4715790748596191, + 0.23539552092552185 + ], + [ + 0.2370307594537735, + 0.4244530200958252, + -0.28319671750068665, + -1.0386545658111572, + 0.18457840383052826, + -0.17398586869239807, + 0.5261819362640381, + -0.016083568334579468, + -0.07158034294843674, + 0.018106073141098022, + 0.00092262402176857, + -0.11308425664901733, + -1.7991769313812256, + -0.10645650327205658, + 0.026809033006429672, + -0.016156738623976707, + 0.17475959658622742, + -0.07639913260936737, + -0.19715937972068787, + -1.747654914855957, + 0.29403311014175415 + ], + [ + 0.43924206495285034, + 0.44182294607162476, + -0.085472971200943, + -0.9050775766372681, + 0.6168959140777588, + -0.44606825709342957, + 0.03122866153717041, + 0.01404985785484314, + -0.03282729536294937, + -0.04009737819433212, + -0.03906179219484329, + -0.3012073338031769, + -2.002332925796509, + -0.370714008808136, + -0.17952914535999298, + 0.08427910506725311, + 0.3502427935600281, + 0.162503182888031, + -0.3028104305267334, + -1.8390069007873535, + 0.01095251739025116 + ], + [ + 0.23200273513793945, + 0.23491232097148895, + 0.3147900104522705, + -1.0627516508102417, + 0.6707007884979248, + -0.14352652430534363, + 0.6275544166564941, + 0.1423129439353943, + -0.057731397449970245, + -0.14694270491600037, + -0.03461664170026779, + -0.12505418062210083, + -0.9721747636795044, + -0.5255884528160095, + 0.08521084487438202, + -0.13859756290912628, + 0.4756892919540405, + -0.2449180632829666, + -0.1088409423828125, + -0.9841295480728149, + -0.07133238017559052 + ] + ], + "generated_utilities": [ + 0.22510160505771637, + 0.20951656997203827, + 0.22272604703903198, + 0.22567233443260193, + 0.2283252328634262, + 0.22638562321662903, + 0.1809503734111786, + 0.12205810844898224 + ], + "group_id": "ms-3afd8959363e917591d4", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8582140207290649, + 2.0, + 0.18031787872314453, + 0.07887399196624756, + 0.5568947792053223, + 0.16715207695960999, + 0.5635193586349487, + 0.18913890421390533, + 0.5682768225669861, + 0.10102970898151398, + 0.09163376688957214, + 0.12909311056137085, + 0.12299326062202454, + 0.09030237793922424, + 0.13531166315078735, + 0.06874948740005493 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.15699657797813416, + 0.6247961521148682, + 0.107974112033844, + -0.3284463882446289, + 0.22832846641540527, + -0.12728703022003174, + 0.0, + -0.2739208936691284, + -0.051639966666698456, + -0.2660524249076843, + -0.2539072632789612, + 0.09800386428833008, + -1.84090256690979, + 0.0, + -0.017676634714007378, + -0.018830953165888786, + 0.1583891659975052, + -0.07512116432189941, + -0.01770937442779541, + -1.5237011909484863, + 0.0 + ], + [ + 0.4675478935241699, + 0.6247961521148682, + 0.3345896601676941, + -0.5404787063598633, + -0.06834304332733154, + -0.12728703022003174, + 0.0, + -0.14999517798423767, + -0.2730908691883087, + 0.08837036788463593, + 0.45600926876068115, + 0.410434365272522, + -1.84090256690979, + 0.0, + 0.4969322681427002, + 0.22034725546836853, + 0.2921968400478363, + 0.5872196555137634, + 0.19443652033805847, + -1.5237011909484863, + 0.0 + ], + [ + 0.41189882159233093, + 1.6247961521148682, + 0.12157746404409409, + -1.328446388244629, + 0.9316569566726685, + -1.1272870302200317, + 1.276085376739502, + -0.029595039784908295, + -0.014941371977329254, + 0.004283940885215998, + 0.6188332438468933, + -0.49464237689971924, + -2.84090256690979, + -1.8046140670776367, + 0.0007966384291648865, + -0.006994657218456268, + 0.014263113960623741, + 0.6869955658912659, + -0.2918644845485687, + -2.5237011909484863, + -1.7481608390808105 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.552170753479004, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.6092281341552734, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.496321678161621 + ], + [ + 0.44720953702926636, + 1.4417266845703125, + 0.4182493984699249, + -1.547792911529541, + 0.990667462348938, + -0.9114025831222534, + 1.535681962966919, + 0.21791213750839233, + -0.30513259768486023, + 0.013669610023498535, + 0.7976462841033936, + -0.6395556926727295, + -2.5809314250946045, + -1.571488618850708, + 0.038753002882003784, + 0.04824339970946312, + -0.11262937635183334, + 0.9187417030334473, + -0.1308273822069168, + -2.59043288230896, + -1.9625922441482544 + ], + [ + 0.5162960290908813, + 1.2868248224258423, + 0.2058304250240326, + -1.4812198877334595, + 1.085485816001892, + -1.05157470703125, + 1.1480523347854614, + 0.2796477973461151, + -0.1532609462738037, + 0.12988577783107758, + 0.8581418395042419, + -0.6760798692703247, + -2.769819736480713, + -2.192568778991699, + 0.10523252189159393, + 0.05066283047199249, + -0.08061028271913528, + 0.7438116669654846, + -0.07841359078884125, + -2.3079848289489746, + -1.4200246334075928 + ], + [ + 0.4112767279148102, + 1.8745536804199219, + 0.5301340222358704, + -0.9917724132537842, + 0.8905197978019714, + -0.8391705751419067, + 1.3120794296264648, + 0.3723032474517822, + -0.19477176666259766, + -0.35689839720726013, + 0.2843710780143738, + -0.45066511631011963, + -2.812030792236328, + -1.588818073272705, + -0.09205310046672821, + -0.5067525506019592, + -0.41988417506217957, + 1.0423402786254883, + 0.1759319305419922, + -2.945967197418213, + -2.05134916305542 + ], + [ + 0.46365559101104736, + 1.105017900466919, + 0.466808021068573, + -1.5382812023162842, + 0.8287478089332581, + -1.4617033004760742, + 0.8807560205459595, + 0.2107943296432495, + 0.2954367995262146, + 0.45928943157196045, + 0.6575880646705627, + -0.5414218902587891, + -2.711836099624634, + -1.6827114820480347, + 0.3175691068172455, + -0.38371264934539795, + 0.34041139483451843, + 0.34132614731788635, + -0.8783347606658936, + -2.133782148361206, + -1.3974260091781616 + ], + [ + 0.13586324453353882, + 1.6158640384674072, + 0.24489957094192505, + -1.4942185878753662, + 1.1602565050125122, + -1.648895502090454, + 1.1836220026016235, + 0.17831489443778992, + -0.5400142669677734, + -0.42907604575157166, + 0.5951722264289856, + -0.7885013818740845, + -3.2272911071777344, + -2.208972692489624, + -0.4857475459575653, + 0.5623884201049805, + 0.02566027268767357, + 0.590702474117279, + -0.6581307649612427, + -2.7863473892211914, + -2.031818151473999 + ], + [ + 0.21801596879959106, + 1.0595136880874634, + 0.9040011763572693, + -1.1076470613479614, + 1.2744677066802979, + -0.7150566577911377, + 1.0042598247528076, + 0.5841938257217407, + -0.2832033336162567, + -0.45343834161758423, + -0.17037296295166016, + -0.5152367353439331, + -2.1963798999786377, + -1.5652427673339844, + 0.03223085403442383, + 0.07496664673089981, + -0.1515023559331894, + 1.233866572380066, + 0.4301607310771942, + -2.489328384399414, + -1.8620245456695557 + ], + [ + -0.15804454684257507, + 2.2168471813201904, + 0.9374236464500427, + -1.8158314228057861, + 1.114590048789978, + -0.9279311895370483, + 1.890993356704712, + -0.8653054237365723, + -0.5551189184188843, + 0.4295695126056671, + 1.1614964008331299, + -0.5840407609939575, + -2.900725841522217, + -2.579883575439453, + -0.1900041103363037, + 0.42368441820144653, + 0.38376376032829285, + 0.9654008150100708, + 0.10608989000320435, + -3.332181453704834, + -1.8879828453063965 + ], + [ + 1.0901068449020386, + 2.576328754425049, + 0.5122992396354675, + -1.9627916812896729, + 0.026246964931488037, + -0.3622438311576843, + 0.9286589622497559, + 0.644849956035614, + 0.9211093187332153, + 0.3528344929218292, + 0.3661169707775116, + 0.20941221714019775, + -2.582620143890381, + -1.6867890357971191, + 0.09869468957185745, + -0.8795469403266907, + -0.2239246666431427, + 0.5206944942474365, + -0.20716872811317444, + -2.8072235584259033, + -2.3385443687438965 + ], + [ + -0.0005125105381011963, + 1.8920408487319946, + 1.0018010139465332, + -2.2556979656219482, + 0.7338991165161133, + -0.4559611678123474, + 1.027335524559021, + -1.0165901184082031, + -0.14792552590370178, + 0.11212217807769775, + 0.8827763795852661, + 0.1683982014656067, + -3.2136244773864746, + -2.4300284385681152, + -0.9183067083358765, + 0.10752613097429276, + -0.2955099046230316, + 0.31867098808288574, + 0.06515705585479736, + -3.416110038757324, + -1.5137704610824585 + ], + [ + -0.39508435130119324, + 2.55599308013916, + 0.128604456782341, + -1.8821169137954712, + 0.6769157648086548, + -0.7477259635925293, + 1.6593644618988037, + -0.005859369412064552, + 0.22320973873138428, + -0.2906110882759094, + 0.22343486547470093, + -1.488371729850769, + -2.81829833984375, + -0.939697265625, + 0.9867495894432068, + 0.2955887019634247, + -0.11908379942178726, + 0.473803848028183, + -0.11059392988681793, + -3.471217155456543, + -2.3480191230773926 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.018576741218566895, + 0.6247961521148682, + -0.017397306859493256, + -0.3284463882446289, + 0.044771432876586914, + -0.12728703022003174, + 0.0, + -0.005512619391083717, + 0.03525955602526665, + -0.005881692748516798, + 0.04493492841720581, + 0.056808531284332275, + -1.84090256690979, + 0.0, + 0.11075251549482346, + -0.05448189005255699, + -0.041690655052661896, + 0.010008454322814941, + 0.056893616914749146, + -1.5237011909484863, + 0.0 + ] + ], + "predicted_scores": [ + 5.160774230957031, + 4.728915691375732, + 4.570479393005371, + 4.122161865234375, + 4.062873840332031, + 4.021884441375732, + 3.392411708831787, + 3.2204978466033936 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "75070641/train0", + "selected_index": 0, + "state_hash": "bd1bbc057450972b9fa6e08a54ebd8c6d82d09997dad16cecf9584632b705b7b", + "stored_base_utility": 0.8582140207290649, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1717667579650879, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1717667579650879 + }, + { + "contact_quality": null, + "progress": 0.18597783148288727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18597783148288727 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.19216947257518768, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19216947257518768 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.16492807865142822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16492807865142822 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.1717667579650879, + 0.18597783148288727, + 1.0, + 1.0, + 0.19216947257518768, + 1.0, + 0.16492807865142822, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919", + "ms-0949097d19e597fc7f63", + "ms-4cab292bc65fa59274ef" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + true, + true, + false, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-3ff1aedb72f65a182b58", + "generated_tangents": [ + [ + 0.09392966330051422, + -0.073333740234375, + -0.02382710576057434, + -0.1307896375656128, + -0.09160763025283813, + -2.3890573978424072, + -0.2402210235595703, + 0.03725676238536835, + 0.06001857668161392, + 0.05920320004224777, + 0.06732793897390366, + -0.04652625322341919, + -1.9502712488174438, + -0.13314096629619598, + 0.018359988927841187, + 0.0077648162841796875, + 0.06510114669799805, + -0.1600084751844406, + -0.006948888301849365, + -1.5971033573150635, + -0.03555750846862793 + ], + [ + 0.056459277868270874, + -0.026371078565716743, + -0.008917242288589478, + -0.21969135105609894, + -0.04966193437576294, + -2.2545814514160156, + -0.24377870559692383, + 0.07215207815170288, + 0.03315239027142525, + 0.011191889643669128, + 0.05912366136908531, + -0.011779963970184326, + -1.8740211725234985, + -0.15244542062282562, + 0.03680367395281792, + 0.031709812581539154, + 0.06003528833389282, + -0.15856648981571198, + 0.009031146764755249, + -1.582228660583496, + -0.04679661989212036 + ], + [ + 0.12764057517051697, + -0.056162089109420776, + 0.11235526204109192, + -0.1247410848736763, + 0.15366747975349426, + -2.1857078075408936, + -0.08992600440979004, + 0.07578996568918228, + 0.0003266548737883568, + 0.10295122861862183, + 0.04417681694030762, + 0.13179272413253784, + -1.7686823606491089, + 0.03846485912799835, + -0.04094785079360008, + -0.006326349917799234, + 0.008371815085411072, + -0.135220006108284, + 0.06420838832855225, + -1.5203099250793457, + -0.06928402185440063 + ], + [ + -0.08496171236038208, + -0.09299589693546295, + 0.08713719248771667, + -0.14648938179016113, + -0.06924819946289062, + -2.3240087032318115, + -0.17085039615631104, + 0.018144264817237854, + -0.017813703045248985, + -0.0343199297785759, + -0.07212626188993454, + 0.02971920371055603, + -1.9930564165115356, + -0.24653057754039764, + -0.007997963577508926, + 0.02391284704208374, + -0.07883070409297943, + -0.27574771642684937, + -0.19631525874137878, + -1.6504130363464355, + -0.043965622782707214 + ], + [ + -0.20934848487377167, + -0.1266692578792572, + 0.10706019401550293, + 0.0272170752286911, + -0.21940195560455322, + -2.659048318862915, + -0.4746842384338379, + 0.2459060251712799, + 0.05170027166604996, + -0.23570626974105835, + 0.12484186142683029, + -0.28754469752311707, + -1.6175321340560913, + -0.2846195697784424, + 0.20102715492248535, + 0.09623122215270996, + -0.05053211748600006, + -0.2426890879869461, + -0.06533008813858032, + -2.0795533657073975, + -0.16265299916267395 + ], + [ + 0.0054766833782196045, + -0.12461771816015244, + 0.09641531109809875, + -0.12070398777723312, + -0.6008101105690002, + -2.37394642829895, + -0.8811764121055603, + -0.03970348834991455, + -0.08527133613824844, + 0.08193513005971909, + -0.07997668534517288, + 0.16318264603614807, + -2.3331503868103027, + 0.425695538520813, + 0.0248819999396801, + 0.20716316998004913, + 0.12654536962509155, + -0.11810736358165741, + -0.4127483367919922, + -1.7783329486846924, + -0.47662726044654846 + ], + [ + 0.15129965543746948, + -0.05064266920089722, + 0.1171606183052063, + -0.08955720067024231, + -0.023857414722442627, + -2.461094856262207, + -0.45744675397872925, + 0.013203874230384827, + 0.017858093604445457, + -0.03413312882184982, + 0.24308684468269348, + 0.05985631048679352, + -1.8113385438919067, + -0.14572568237781525, + -0.019261468201875687, + 0.057120610028505325, + 0.11785639822483063, + -0.22399063408374786, + 0.053573429584503174, + -1.7720108032226562, + 0.08494687080383301 + ], + [ + -0.10472635924816132, + 0.09618918597698212, + 0.13700050115585327, + -0.026022180914878845, + -0.195050448179245, + -2.350626230239868, + -0.5292054414749146, + 0.12431493401527405, + 0.09990887343883514, + -0.0625324472784996, + 0.29243823885917664, + -0.2240816056728363, + -1.6519718170166016, + -0.2509976029396057, + 0.013827253133058548, + 0.18157435953617096, + 0.2608470320701599, + -0.015192806720733643, + 0.14551664888858795, + -2.0923616886138916, + 0.056387029588222504 + ] + ], + "generated_utilities": [ + 0.1717667579650879, + 0.18597783148288727, + 2.0, + 2.0, + 0.19216947257518768, + 2.0, + 0.16492807865142822, + 2.0 + ], + "group_id": "ms-3ff1aedb72f65a182b58", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.851725161075592, + 2.0, + 0.8126396536827087, + 2.0, + 0.7855525612831116, + 0.7821746468544006, + 0.6115735173225403, + 0.5732660889625549, + 0.7798006534576416, + 0.09109212458133698, + 0.8456447124481201, + 0.16636842489242554, + 0.0955686867237091, + 0.0822453498840332, + 0.0018809139728546143, + 0.08579078316688538 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.09319563210010529, + 0.18433167040348053, + 0.03857926279306412, + 0.20899039506912231, + -0.19732166826725006, + -2.7716264724731445, + -0.9089346528053284, + -0.012062903493642807, + -0.001710914308205247, + 0.02447224222123623, + 0.6588084697723389, + -0.43759873509407043, + -2.7224881649017334, + -1.560755968093872, + -0.002114389091730118, + -0.004199527204036713, + 0.015975257381796837, + 0.7779304385185242, + -0.31706690788269043, + -2.4826252460479736, + -1.5341289043426514 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.8178693056106567, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.121511936187744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0682578086853027 + ], + [ + -0.10748189687728882, + 0.29221633076667786, + 0.12325333058834076, + 0.21938443183898926, + 0.017790749669075012, + -2.485055446624756, + -1.1110891103744507, + 0.062413301318883896, + -0.2567306160926819, + -0.15453597903251648, + 0.549060583114624, + -0.18322163820266724, + -2.5428390502929688, + -1.8537542819976807, + -0.07324761152267456, + 0.28765758872032166, + 0.16924789547920227, + 0.6205734610557556, + -0.4268453121185303, + -2.6385459899902344, + -1.5760513544082642 + ], + [ + 0.44683152437210083, + 0.3713763356208801, + 0.1084144338965416, + 0.2590586543083191, + 0.1997859627008438, + -2.787759780883789, + -1.1921976804733276, + 0.33878061175346375, + 0.019987009465694427, + 0.021556159481406212, + 0.36814823746681213, + -0.2135106772184372, + -2.416769027709961, + -1.3776189088821411, + -0.3500843346118927, + -0.26303720474243164, + 0.10322260856628418, + 0.9827795624732971, + -0.3747403025627136, + -2.6419029235839844, + -1.4116339683532715 + ], + [ + -0.2883707880973816, + 0.02873578667640686, + -0.11175497621297836, + -0.106661856174469, + 0.03059367835521698, + -2.3899149894714355, + -1.0789995193481445, + 0.2610729932785034, + 0.4800889790058136, + -0.45478329062461853, + 1.0867195129394531, + -0.7417076826095581, + -3.09529185295105, + -1.730720043182373, + -0.33508142828941345, + 0.10841497033834457, + -0.38902348279953003, + 0.44348275661468506, + 0.0854053795337677, + -2.3334603309631348, + -1.7851121425628662 + ], + [ + 0.16367590427398682, + 0.05515189468860626, + -0.3197028636932373, + 0.5501840114593506, + -0.3621746301651001, + -3.0582237243652344, + -1.135798454284668, + -0.15054848790168762, + -0.5578444004058838, + 0.3691062331199646, + 0.9388304352760315, + -0.19103051722049713, + -2.7536559104919434, + -2.093773603439331, + -0.037153881043195724, + 0.19014233350753784, + -0.36205345392227173, + 0.24469107389450073, + -0.5916419625282288, + -2.7417092323303223, + -1.637229323387146 + ], + [ + -0.3646707534790039, + 0.6696967482566833, + -0.44303250312805176, + -0.4559982419013977, + -0.6542845368385315, + -2.9630653858184814, + -1.4721050262451172, + 0.5010843873023987, + 0.5162685513496399, + 0.42702627182006836, + 0.6571387648582458, + -0.08207055926322937, + -3.278073310852051, + -1.7742136716842651, + -0.6448181867599487, + -0.45361432433128357, + 0.3905653953552246, + 0.7770582437515259, + -0.19482973217964172, + -2.094736337661743, + -1.083003282546997 + ], + [ + 0.9472348093986511, + 0.6904511451721191, + -0.2936761677265167, + -0.34556156396865845, + -0.7593099474906921, + -1.9670193195343018, + -0.7599589824676514, + 0.04467743635177612, + 0.6557444334030151, + 0.2765021026134491, + 0.042136549949645996, + -0.37541040778160095, + -3.161308765411377, + -0.8803613185882568, + 0.38967010378837585, + 0.4423010051250458, + 0.019825218245387077, + 0.9605399370193481, + -0.874176561832428, + -3.0791730880737305, + -1.3198856115341187 + ], + [ + 0.8605663180351257, + 0.04459291696548462, + -0.8886744976043701, + -0.38333380222320557, + -1.0364235639572144, + -3.0104973316192627, + -0.889693021774292, + -0.08674328029155731, + 0.20159615576267242, + 0.1069621816277504, + 1.4727201461791992, + 0.09883639216423035, + -3.129849433898926, + -0.8773520588874817, + -0.2628254294395447, + -0.3415844738483429, + -0.35313117504119873, + 0.3155616521835327, + -0.556646466255188, + -2.0992307662963867, + -0.7895610928535461 + ], + [ + 0.39962780475616455, + 0.3406272530555725, + 0.05363208055496216, + -0.06018832325935364, + -0.5703235268592834, + -2.5974044799804688, + -0.6783060431480408, + 0.7815009355545044, + 0.35641810297966003, + -0.00838596560060978, + 0.6228271126747131, + 0.25608590245246887, + -2.5914058685302734, + -0.8502807021141052, + 0.726007878780365, + -0.6774719953536987, + 0.6261520981788635, + 1.149846076965332, + 0.6782730221748352, + -3.391972541809082, + -0.7993651032447815 + ], + [ + -0.29702335596084595, + 0.6166410446166992, + 0.6319320201873779, + -0.6542804837226868, + -0.8892693519592285, + -1.8599720001220703, + -1.3999924659729004, + -0.3017314672470093, + 0.9357973337173462, + -0.5596981048583984, + 1.3143360614776611, + -0.30643409490585327, + -2.028008460998535, + -1.3395459651947021, + 0.6212120056152344, + 0.6739580631256104, + -0.5840877294540405, + -0.06664055585861206, + -0.23446696996688843, + -2.0225906372070312, + -2.438906669616699 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0649428591132164, + 0.09363124519586563, + -0.014398600906133652, + -0.012044146656990051, + 0.03117428719997406, + -1.7716264724731445, + 0.0, + 0.05836518481373787, + 0.012665675021708012, + -0.013142907992005348, + -0.06252282857894897, + -0.06281104683876038, + -1.7224881649017334, + 0.0, + -0.06448972970247269, + -0.03696601092815399, + 0.030633779242634773, + -0.0031400322914123535, + 0.0327644944190979, + -1.4826252460479736, + 0.0 + ], + [ + -0.04005520045757294, + 0.2738151252269745, + -0.05346817523241043, + 0.20275527238845825, + 0.10101960599422455, + -1.7716264724731445, + 0.0, + -0.36066195368766785, + -0.21821273863315582, + 0.22022001445293427, + -0.0596311092376709, + -0.17951729893684387, + -1.7224881649017334, + 0.0, + 0.3205486238002777, + 0.25526633858680725, + 0.17392843961715698, + 0.5170043706893921, + 0.43821561336517334, + -1.4826252460479736, + 0.0 + ] + ], + "predicted_scores": [ + 3.9676198959350586, + 3.801675796508789, + 3.7893948554992676, + 3.2359251976013184, + 3.0078318119049072, + 2.753028392791748, + 2.7207984924316406, + 1.9320051670074463 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055567618/train0", + "selected_index": 0, + "state_hash": "027cc529c99c00db47a01ecfc12ab432557fbd911e8ff3c8a56adf2d3404549f", + "stored_base_utility": 0.851725161075592, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19648407399654388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19648407399654388 + }, + "base_progress": 0.19648407399654388, + "base_success": false, + "base_utility": 0.19648407399654388, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12073369324207306, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12073369324207306 + }, + { + "contact_quality": null, + "progress": 0.1906120628118515, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1906120628118515 + }, + { + "contact_quality": null, + "progress": 0.21016791462898254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21016791462898254 + }, + { + "contact_quality": null, + "progress": 0.2101641744375229, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2101641744375229 + }, + { + "contact_quality": null, + "progress": 0.1435815393924713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1435815393924713 + }, + { + "contact_quality": null, + "progress": 0.5575173497200012, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5575173497200012 + }, + { + "contact_quality": null, + "progress": 0.19436763226985931, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19436763226985931 + }, + { + "contact_quality": null, + "progress": 0.5705926418304443, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5705926418304443 + } + ], + "candidate_progress": [ + 0.12073369324207306, + 0.1906120628118515, + 0.21016791462898254, + 0.2101641744375229, + 0.1435815393924713, + 0.5575173497200012, + 0.19436763226985931, + 0.5705926418304443 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4048f8f6e047422e3c79", + "generated_tangents": [ + [ + 0.2171844244003296, + 0.18487316370010376, + 0.28260764479637146, + -0.36783677339553833, + 0.22836995124816895, + -0.14998310804367065, + 0.3650948703289032, + 0.11603033542633057, + -0.016029154881834984, + -0.09612519294023514, + 0.059937648475170135, + -0.03008727729320526, + -0.9651647806167603, + -0.7220410108566284, + 0.07607698440551758, + -0.14111168682575226, + 0.2967831492424011, + -0.19809038937091827, + -0.22854703664779663, + -1.0838415622711182, + -0.5010473728179932 + ], + [ + -0.03858329355716705, + 0.1825340986251831, + 0.020320653915405273, + -0.2719494700431824, + 0.17369604110717773, + -0.19225314259529114, + 0.12003076076507568, + 0.10427724570035934, + 0.02255987748503685, + -0.041600100696086884, + 0.13357362151145935, + -0.21711459755897522, + -1.9956270456314087, + -0.045915767550468445, + 0.04842653125524521, + -0.12164908647537231, + 0.08355529606342316, + 0.05983982980251312, + 0.05341607332229614, + -1.5519073009490967, + 0.013342559337615967 + ], + [ + 0.002469167113304138, + 0.1645757257938385, + 0.030765265226364136, + -0.31909507513046265, + 0.1670299768447876, + 0.10972249507904053, + -0.14881157875061035, + 0.034435123205184937, + -0.09460742026567459, + -0.06192999333143234, + 0.09626126289367676, + -0.17766708135604858, + -1.998361349105835, + -0.03360585868358612, + 0.023616105318069458, + -0.0715348869562149, + 0.06650449335575104, + -0.1089114099740982, + -0.19733011722564697, + -1.566237211227417, + -0.005705744028091431 + ], + [ + 0.24950110912322998, + 0.21605072915554047, + -0.08998242020606995, + -0.1326679289340973, + -0.04281589388847351, + -0.35822418332099915, + -0.1337767243385315, + -0.1644410490989685, + -0.05146455764770508, + -0.14123421907424927, + 0.5631891489028931, + -0.36045610904693604, + -1.8442262411117554, + -0.4992050528526306, + 0.09674850106239319, + -0.09498344361782074, + -0.24699492752552032, + 0.025555402040481567, + -0.3607638478279114, + -1.7779898643493652, + -0.07849517464637756 + ], + [ + 0.2732873558998108, + 0.36256277561187744, + -0.02408924698829651, + -0.1519061177968979, + 0.2774462103843689, + -0.5360255241394043, + -0.05277550220489502, + 0.00856037437915802, + 0.11335992813110352, + 0.07843641936779022, + 0.11148088425397873, + -0.47118204832077026, + -1.9787909984588623, + -0.653018057346344, + -0.12463939189910889, + 0.0651550441980362, + 0.1642739474773407, + 0.1699945628643036, + -0.45995843410491943, + -1.791959524154663, + -0.14612862467765808 + ], + [ + 0.30226635932922363, + 0.18178465962409973, + -0.22026470303535461, + -0.3560119867324829, + -0.13338452577590942, + -0.07905139029026031, + -0.1558377742767334, + -0.06049463152885437, + -0.06358654797077179, + -0.20067763328552246, + 0.2741665542125702, + -0.6450058221817017, + -1.9496437311172485, + -0.498136043548584, + -0.020649496465921402, + -0.04878104850649834, + -0.13140223920345306, + -0.036294564604759216, + -0.45630335807800293, + -1.4932804107666016, + 0.04894767701625824 + ], + [ + 0.30841732025146484, + 0.26397138833999634, + -0.2059001624584198, + -0.300531804561615, + -0.06339272856712341, + -0.21772345900535583, + 0.10650348663330078, + 0.004072010517120361, + -0.0054857488721609116, + 0.10548824071884155, + 0.08915750682353973, + -0.18550986051559448, + -1.8369694948196411, + -0.32808464765548706, + 0.05239670351147652, + -0.019515590742230415, + -0.003979310393333435, + -0.09749935567378998, + -0.2767573595046997, + -1.8588802814483643, + -0.06719134747982025 + ], + [ + 0.09183335304260254, + 0.2539528012275696, + -0.1101885735988617, + -0.16674894094467163, + -0.07158789038658142, + -0.3414709270000458, + -0.062735915184021, + -0.062052249908447266, + -0.025212986394762993, + -0.1355210840702057, + 0.2859673798084259, + -0.4291049838066101, + -1.8351331949234009, + -0.4200328588485718, + -0.026011284440755844, + -0.05623798072338104, + -0.03581221401691437, + 0.0038288086652755737, + -0.42838382720947266, + -1.7482473850250244, + -0.0438215434551239 + ] + ], + "generated_utilities": [ + 0.12073369324207306, + 0.1906120628118515, + 0.21016791462898254, + 0.2101641744375229, + 0.1435815393924713, + 0.5575173497200012, + 0.19436763226985931, + 0.5705926418304443 + ], + "group_id": "ms-4048f8f6e047422e3c79", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558006286621094, + 2.0, + 0.1986379623413086, + 0.03769078850746155, + 0.5361635684967041, + 0.18274834752082825, + 0.17300468683242798, + 0.12688571214675903, + 0.13836368918418884, + 0.07454231381416321, + 0.0899321436882019, + 0.10402637720108032, + 0.1711854636669159, + 0.17255032062530518, + 0.039613887667655945, + 0.10826331377029419 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1422826200723648, + 0.21033799648284912, + 0.1978604793548584, + 0.059162259101867676, + -0.2529524564743042, + -0.15161612629890442, + 0.0, + -0.2264404594898224, + -0.2209387719631195, + -0.025619791820645332, + 0.06467145681381226, + 0.07422274351119995, + -1.7048513889312744, + 0.0, + 0.15056702494621277, + -0.06247536092996597, + -0.1033044084906578, + 0.1334916353225708, + 0.2784581184387207, + -1.4811434745788574, + 0.0 + ], + [ + -0.44999396800994873, + 0.271686851978302, + -0.4267011880874634, + 0.2730966806411743, + -0.01798880100250244, + -0.04680868983268738, + 0.0, + -0.09472770243883133, + -0.06993720680475235, + -0.06752754002809525, + -0.13884133100509644, + 0.24737784266471863, + -1.7048513889312744, + 0.0, + -0.09056920558214188, + 0.4132460653781891, + 0.04097161442041397, + -0.25072288513183594, + 0.49742037057876587, + -1.4811434745788574, + 0.0 + ], + [ + 0.17392832040786743, + 1.2103379964828491, + -0.3708428144454956, + -0.36093437671661377, + 0.744896650314331, + 0.23929360508918762, + 1.4388959407806396, + -0.04149632900953293, + 0.015302501618862152, + 0.023577755317091942, + 0.5954978466033936, + -0.3274606168270111, + -2.7048513889312744, + -1.646976351737976, + 0.0006552413105964661, + -0.008917607367038727, + 0.022643378004431725, + 0.7492771148681641, + -0.26030659675598145, + -2.4811434745788574, + -1.684959888458252 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8777918815612793, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.293952703475952, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.369919776916504 + ], + [ + 0.1792769730091095, + 0.9774190187454224, + -0.5587559342384338, + -0.5885956883430481, + 0.9904338717460632, + 0.41647347807884216, + 1.4140102863311768, + -0.2462461292743683, + -0.1271664798259735, + -0.2736872732639313, + 0.8877567052841187, + -0.6213628053665161, + -2.937077283859253, + -1.3732452392578125, + -0.0015218837652355433, + -0.27707284688949585, + -0.16698139905929565, + 0.4978373646736145, + -0.479316771030426, + -2.7234396934509277, + -1.6759265661239624 + ], + [ + 0.1658722460269928, + 1.3070863485336304, + -0.4893108606338501, + -0.04209986329078674, + 0.7449368238449097, + 0.3420809805393219, + 1.4472918510437012, + 0.08132945746183395, + 0.13181470334529877, + 0.22494959831237793, + 0.22593030333518982, + -0.49233078956604004, + -2.7544617652893066, + -1.7597702741622925, + -0.365922212600708, + -0.17204320430755615, + -0.1821347028017044, + 0.6334611177444458, + -0.33006516098976135, + -2.2188992500305176, + -1.5452594757080078 + ], + [ + 0.18608316779136658, + 1.23563551902771, + -0.5727517604827881, + 0.10049733519554138, + 0.9073957204818726, + 0.5650919675827026, + 1.8946568965911865, + -0.17719107866287231, + 0.34025683999061584, + 0.44525983929634094, + 0.6176816821098328, + -0.2832425534725189, + -3.1802117824554443, + -1.8703548908233643, + 0.07433810085058212, + 0.1340292990207672, + 0.497944176197052, + 1.0071699619293213, + -0.44033437967300415, + -2.7992849349975586, + -1.8935935497283936 + ], + [ + 0.40270811319351196, + 1.5102664232254028, + -0.8070739507675171, + -0.16390679776668549, + 0.3215266466140747, + -0.16522419452667236, + 1.0900052785873413, + 0.07780961692333221, + 0.5627153515815735, + 0.5386761426925659, + 0.2949375510215759, + 0.059314221143722534, + -3.2417359352111816, + -1.899662971496582, + 0.4160631597042084, + 0.44336366653442383, + 0.47819244861602783, + 1.162381649017334, + -0.7831735014915466, + -2.625349521636963, + -1.2278814315795898 + ], + [ + 0.8529313802719116, + 1.5106499195098877, + 0.19478559494018555, + -0.6338159441947937, + 0.7529481053352356, + -0.24222904443740845, + 0.9723314046859741, + -0.14078867435455322, + -0.5563283562660217, + 0.5440018177032471, + 0.004017293453216553, + 0.019366413354873657, + -3.1521430015563965, + -2.0841636657714844, + -0.2760852575302124, + -0.5531485676765442, + 0.22684869170188904, + 1.100138545036316, + 0.046634942293167114, + -2.429711103439331, + -2.280839443206787 + ], + [ + -0.4760962128639221, + 1.8579951524734497, + 0.2322864532470703, + 0.19953304529190063, + 0.48002323508262634, + 0.7484984397888184, + 2.073361396789551, + -0.04567806422710419, + 0.04004429653286934, + -0.5635936260223389, + 0.5135918259620667, + 0.3908633887767792, + -1.9335949420928955, + -2.442507743835449, + -0.6620182394981384, + 0.4175874888896942, + 0.7711085677146912, + 1.2790143489837646, + -0.5464571714401245, + -3.2030673027038574, + -2.3768434524536133 + ], + [ + 1.0231709480285645, + 1.1804075241088867, + 0.11513572931289673, + -1.23623526096344, + 1.4848686456680298, + 1.1120561361312866, + 0.9504602551460266, + 0.8234015703201294, + -0.04407603666186333, + -0.37310001254081726, + 0.24512693285942078, + 0.4242170751094818, + -2.4985997676849365, + -2.1206297874450684, + 0.5509956479072571, + 0.8686066269874573, + -0.5043656229972839, + 1.3957195281982422, + -0.07662208378314972, + -2.371795654296875, + -2.2404346466064453 + ], + [ + 0.49735015630722046, + 0.3600291609764099, + 0.3103446364402771, + 0.6051294803619385, + -0.15457379817962646, + -0.5570405721664429, + 1.2383109331130981, + 0.9059417247772217, + 0.7879754900932312, + -0.31492647528648376, + 1.5871094465255737, + -0.8906484842300415, + -1.8698136806488037, + -2.401073694229126, + 0.3560212254524231, + -0.6580788493156433, + 0.9037240147590637, + 1.614617109298706, + 0.3788090944290161, + -1.6909503936767578, + -0.8765208125114441 + ], + [ + 0.10883396118879318, + 1.8993752002716064, + -0.7673929333686829, + -0.0936923623085022, + -0.15508884191513062, + -0.1517968773841858, + 0.465425968170166, + 0.43258219957351685, + 0.46641960740089417, + -0.3528187572956085, + 1.09894597530365, + -0.6569450497627258, + -2.025486946105957, + -1.2820806503295898, + -0.21141839027404785, + -0.9515489339828491, + 0.7230286002159119, + -0.21062928438186646, + -0.32822558283805847, + -3.289642333984375, + -1.7125959396362305 + ], + [ + 0.37358880043029785, + 2.052347183227539, + -1.212618112564087, + -0.6321570873260498, + 0.7031511068344116, + -0.3477393090724945, + 1.2687753438949585, + 0.8023800253868103, + -0.031013749539852142, + 0.35525181889533997, + 0.5262036323547363, + -0.9288194179534912, + -1.7686026096343994, + -2.128396987915039, + -0.7001423239707947, + -0.12872236967086792, + -0.012955078855156898, + 1.0237239599227905, + 0.03636348247528076, + -2.0397119522094727, + -0.9761244058609009 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.060203760862350464, + 0.21033799648284912, + 0.0356404185295105, + -0.03044089674949646, + 0.024639129638671875, + -0.032671064138412476, + 0.0, + 0.016544528305530548, + 0.04633430764079094, + 0.005854343995451927, + -0.014876246452331543, + 0.0028293728828430176, + -1.7048513889312744, + 0.0, + -0.05953434109687805, + -0.04181046783924103, + -0.0018185675144195557, + -0.09216523170471191, + 0.019338399171829224, + -1.4811434745788574, + 0.0 + ] + ], + "predicted_scores": [ + 6.748522758483887, + 6.704971790313721, + 5.85119104385376, + 5.7100701332092285, + 5.581671714782715, + 5.393603324890137, + 4.9805073738098145, + 4.962723255157471 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055157354/train0", + "selected_index": 0, + "state_hash": "ae21d2b71722381a530d3a9259d02ea6728eb8a4a84fafc40ccd653d62ad3968", + "stored_base_utility": 0.8558006286621094, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19844311475753784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844311475753784 + }, + "base_progress": 0.19844311475753784, + "base_success": false, + "base_utility": 0.19844311475753784, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.181088387966156, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.181088387966156 + }, + { + "contact_quality": null, + "progress": 0.19361381232738495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19361381232738495 + }, + { + "contact_quality": null, + "progress": 0.1885397732257843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1885397732257843 + }, + { + "contact_quality": null, + "progress": 0.19568920135498047, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19568920135498047 + }, + { + "contact_quality": null, + "progress": 0.1507587730884552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1507587730884552 + }, + { + "contact_quality": null, + "progress": 0.15693092346191406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15693092346191406 + }, + { + "contact_quality": null, + "progress": 0.21208032965660095, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21208032965660095 + }, + { + "contact_quality": null, + "progress": 0.19735300540924072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19735300540924072 + } + ], + "candidate_progress": [ + 0.181088387966156, + 0.19361381232738495, + 0.1885397732257843, + 0.19568920135498047, + 0.1507587730884552, + 0.15693092346191406, + 0.21208032965660095, + 0.19735300540924072 + ], + "candidate_source_chart_ids": [ + "ms-5bddb39f792f20e5e6d4", + "ms-626ac2ad9d0dd95f1a2e", + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-5b26c33feed6992386b1", + "generated_tangents": [ + [ + -0.2862510681152344, + -0.13250859081745148, + -0.16434618830680847, + 0.282477468252182, + -0.2499389946460724, + -0.7596268653869629, + -0.1956716775894165, + -0.005874142050743103, + -0.031128281727433205, + 0.052492037415504456, + 0.08428576588630676, + 0.20952987670898438, + -1.914440393447876, + -0.3057788610458374, + 0.23437592387199402, + 0.031218454241752625, + 0.03350205719470978, + 0.01368558406829834, + 0.12202651798725128, + -1.9188876152038574, + -0.2509079575538635 + ], + [ + -0.0856473296880722, + -0.14153450727462769, + -0.13054463267326355, + 0.28511396050453186, + -0.16349881887435913, + -1.1399821043014526, + -0.2611961364746094, + -0.00705265998840332, + 0.03750584274530411, + 0.03383596986532211, + 0.18552809953689575, + 0.008926361799240112, + -2.0112624168395996, + -0.29566138982772827, + 0.1691342443227768, + 0.012138821184635162, + -0.016451910138130188, + -0.007437646389007568, + 0.08463667333126068, + -1.8638122081756592, + -0.2581925094127655 + ], + [ + -0.13828109204769135, + -0.10704736411571503, + -0.11944708228111267, + 0.29184237122535706, + -0.19101271033287048, + -0.832897424697876, + -0.25124406814575195, + 0.008945763111114502, + -0.003039782866835594, + 0.04226904362440109, + 0.13229313492774963, + 0.08428864926099777, + -2.043454885482788, + -0.2798768877983093, + 0.18739157915115356, + -0.03497983515262604, + -0.051990047097206116, + -0.052511975169181824, + 0.030673429369926453, + -2.039942741394043, + -0.20155519247055054 + ], + [ + -0.06138770282268524, + -0.17644724249839783, + -0.15302732586860657, + 0.3200128972530365, + -0.1305902898311615, + -0.7829512357711792, + -0.25700312852859497, + -0.039596185088157654, + -0.033647067844867706, + 0.06793227791786194, + 0.04071666672825813, + 0.14053404331207275, + -2.0391485691070557, + -0.16965924203395844, + 0.17837156355381012, + -0.038668420165777206, + -0.02522064745426178, + -0.060609444975852966, + 0.06050428748130798, + -1.9583921432495117, + -0.07141707837581635 + ], + [ + -0.09505705535411835, + -0.15290483832359314, + -0.048787087202072144, + 0.4441513121128082, + 0.2212580144405365, + -0.4604974687099457, + -0.1289454698562622, + -0.04884445667266846, + -0.027616044506430626, + 0.17209866642951965, + 0.015292763710021973, + 0.21872572600841522, + -1.8226814270019531, + 0.11185229569673538, + 0.1978069543838501, + -0.13484697043895721, + -0.02377013862133026, + 0.019368737936019897, + 0.07270364463329315, + -2.320669412612915, + -0.161236971616745 + ], + [ + 0.020276054739952087, + -0.10329604893922806, + -0.088380366563797, + 0.3229021728038788, + 0.04564817249774933, + -0.4610654413700104, + -0.3161851763725281, + -0.1608552634716034, + -0.053467437624931335, + 0.11612476408481598, + -0.058851636946201324, + -0.006638064980506897, + -2.034417152404785, + -0.3028770089149475, + 0.3163405656814575, + -0.013032857328653336, + 0.11392523348331451, + 0.09095031023025513, + 0.041872262954711914, + -2.2858188152313232, + -0.2532116174697876 + ], + [ + -0.1948942095041275, + -0.26143571734428406, + -0.41711851954460144, + 0.43631741404533386, + 0.027200281620025635, + -0.8053562641143799, + -0.6398534178733826, + 0.09320799261331558, + -0.03754061460494995, + 0.052125826478004456, + 0.018615175038576126, + 0.5453782081604004, + -1.841666340827942, + -0.027788981795310974, + 0.14161910116672516, + -0.034317318350076675, + 0.01090770959854126, + -0.06977881491184235, + 0.18669134378433228, + -2.060138463973999, + -0.02414742112159729 + ], + [ + -0.35576683282852173, + -0.22342397272586823, + -0.12707075476646423, + 0.6335762739181519, + 0.29755714535713196, + -0.39805683493614197, + -0.1325763463973999, + -0.044789984822273254, + -0.07913310080766678, + 0.017936237156391144, + 0.02942764386534691, + 0.22497786581516266, + -1.5915617942810059, + 0.07978734374046326, + 0.10442276298999786, + -0.21446844935417175, + -0.12771053612232208, + -0.05680327117443085, + 0.15677303075790405, + -2.2263312339782715, + 0.07280028611421585 + ] + ], + "generated_utilities": [ + 0.181088387966156, + 0.19361381232738495, + 0.1885397732257843, + 0.19568920135498047, + 0.1507587730884552, + 0.15693092346191406, + 0.21208032965660095, + 0.19735300540924072 + ], + "group_id": "ms-5b26c33feed6992386b1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8326650857925415, + 2.0, + 0.15131878852844238, + 0.15007886290550232, + 0.5293327569961548, + 0.06490398943424225, + 0.5425760746002197, + 0.20163774490356445, + 0.15259261429309845, + 0.23924465477466583, + 0.12766429781913757, + 0.08033241331577301, + 0.07788355648517609, + 0.13886235654354095, + 0.09716498851776123, + 0.11682072281837463 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.06662726402282715, + -0.14569896459579468, + 0.04974943399429321, + 0.539892315864563, + -0.19018621742725372, + -0.5884796380996704, + 0.0, + 0.12748029828071594, + 0.11038697510957718, + -0.03454907238483429, + 0.08700031787157059, + -0.13105249404907227, + -1.9082190990447998, + 0.0, + -0.0033895689994096756, + -0.15730009973049164, + 0.15846982598304749, + 0.11786291003227234, + -0.02387581765651703, + -1.5966379642486572, + 0.0 + ], + [ + -0.06662726402282715, + 0.059319257736206055, + -0.19934123754501343, + 0.539892315864563, + 0.07902508974075317, + -0.5884796380996704, + 0.0, + -0.3428957760334015, + 0.08558038622140884, + 0.28825634717941284, + -0.32335156202316284, + -0.3631777763366699, + -1.9082190990447998, + 0.0, + -0.3765835762023926, + -0.5554376840591431, + 0.3468284010887146, + 0.875933825969696, + 0.11397302150726318, + -1.5966379642486572, + 0.0 + ], + [ + -1.0666272640228271, + -0.6187554597854614, + -0.8818012475967407, + 1.539892315864563, + -0.428971529006958, + -1.5884796380996704, + 2.0647666454315186, + -0.03600159287452698, + 0.032059766352176666, + 0.030456876382231712, + 0.14377687871456146, + 0.20562076568603516, + -2.9082190990448, + -1.332404375076294, + -0.014788046479225159, + -0.0006958844605833292, + 0.023885300382971764, + 0.5519534349441528, + 0.1840958297252655, + -2.5966379642486572, + -1.4879226684570312 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.129533290863037, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.664808750152588, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9758453369140625 + ], + [ + -1.2971861362457275, + -0.7044790983200073, + -1.134492039680481, + 1.5821952819824219, + -0.7159937620162964, + -1.7615629434585571, + 2.031708002090454, + -0.25998222827911377, + 0.10180196911096573, + -0.24491730332374573, + 0.2074550986289978, + 0.2793378233909607, + -2.752415657043457, + -1.3360638618469238, + 0.03524862602353096, + -0.1854405552148819, + -0.1696440726518631, + 0.7172544598579407, + 0.0028989464044570923, + -2.680950164794922, + -1.2071388959884644 + ], + [ + -1.465846300125122, + -0.34887754917144775, + -1.046374797821045, + 1.703241229057312, + -0.19744430482387543, + -1.2604204416275024, + 1.9822474718093872, + 0.18854136765003204, + -0.146264910697937, + 0.059789031744003296, + 0.39534950256347656, + 0.29636046290397644, + -2.6612584590911865, + -1.5028973817825317, + 0.059081435203552246, + -0.11711112409830093, + -0.07238245755434036, + 0.25233787298202515, + 0.12625277042388916, + -2.2362401485443115, + -1.7053974866867065 + ], + [ + -1.1892943382263184, + -0.20831987261772156, + -0.8013461828231812, + 1.7197901010513306, + -0.39134201407432556, + -1.5160430669784546, + 2.4594008922576904, + -0.1669330894947052, + -0.337530255317688, + 0.15848031640052795, + 0.05738166719675064, + 0.41757503151893616, + -3.1393940448760986, + -0.9257140755653381, + -0.5068305134773254, + 0.1864723265171051, + 0.45369866490364075, + 0.6036283373832703, + -0.018199875950813293, + -2.8028440475463867, + -1.3847744464874268 + ], + [ + -1.1111165285110474, + -1.191490888595581, + -1.361661434173584, + 1.197826623916626, + -0.29219919443130493, + -1.7355475425720215, + 2.0579750537872314, + -0.08019472658634186, + 0.5140348672866821, + -0.5474883317947388, + -0.03311571478843689, + -0.060686588287353516, + -2.3427765369415283, + -1.8552747964859009, + -0.36303937435150146, + 0.01966947503387928, + -0.5651369094848633, + 0.22617560625076294, + -0.16322728991508484, + -2.421419143676758, + -0.921785831451416 + ], + [ + -1.012363076210022, + -0.6483663320541382, + -0.7951939702033997, + 1.2563079595565796, + -0.5323562622070312, + -1.5913673639297485, + 1.406038761138916, + -0.39036238193511963, + -0.12943506240844727, + -0.4373972415924072, + -0.20462383329868317, + 0.12806488573551178, + -3.570098400115967, + -1.8897531032562256, + 0.6369156837463379, + 0.2483828067779541, + 0.6809477210044861, + 0.8138958215713501, + 0.6604747176170349, + -2.717491865158081, + -1.9953765869140625 + ], + [ + -0.378162145614624, + -0.23438718914985657, + -1.2589956521987915, + 1.844052791595459, + -0.8936926126480103, + -1.230797290802002, + 2.5580525398254395, + -0.21314577758312225, + -0.05935201793909073, + 0.46811148524284363, + 0.38594305515289307, + 1.0049593448638916, + -2.491497278213501, + -1.3096134662628174, + 0.15605805814266205, + 0.7682130932807922, + -0.4010901153087616, + 0.8638691902160645, + 0.5755372047424316, + -3.324826240539551, + -1.6860322952270508 + ], + [ + -0.6920164227485657, + -0.5060986280441284, + -0.5146759748458862, + 2.3206489086151123, + 0.26810598373413086, + -1.7696658372879028, + 1.4922571182250977, + -0.6155052185058594, + -0.5581206679344177, + 0.4068271219730377, + 0.5812678337097168, + -0.2056950330734253, + -2.227752923965454, + -0.7079744935035706, + 0.3153572678565979, + 0.019053615629673004, + -0.10691303759813309, + -0.18750637769699097, + 0.5490041971206665, + -2.7611911296844482, + -1.0861625671386719 + ], + [ + -0.9623568654060364, + -0.821997880935669, + -0.0017323493957519531, + 1.8667385578155518, + 0.39793866872787476, + -1.3673040866851807, + 2.8636348247528076, + -0.739641547203064, + -0.4186078906059265, + -0.21357670426368713, + 0.6820193529129028, + 0.09538085013628006, + -3.242215633392334, + -1.023385763168335, + -0.2367737591266632, + 0.9248462915420532, + 0.04536557197570801, + 0.2294020652770996, + 0.15174128115177155, + -2.509904146194458, + -0.5901334881782532 + ], + [ + -0.565070390701294, + 0.37745314836502075, + -1.7464253902435303, + 1.4869437217712402, + -1.187619924545288, + -2.3802757263183594, + 2.5987753868103027, + 0.5777139663696289, + -0.04772312939167023, + -0.49332448840141296, + -0.515097975730896, + 0.751485288143158, + -2.7030012607574463, + -1.5739474296569824, + 0.8266244530677795, + -0.9454089403152466, + 0.7678747177124023, + 1.0557427406311035, + 0.0584852397441864, + -2.8590199947357178, + -0.8677505850791931 + ], + [ + -0.2763708233833313, + 0.28772038221359253, + -0.7008944153785706, + 1.3402774333953857, + -0.78607177734375, + -2.1210508346557617, + 1.4908761978149414, + -0.4155166447162628, + 0.40930652618408203, + -0.1769891083240509, + -0.20092366635799408, + 0.5321645736694336, + -3.2144649028778076, + -1.2440845966339111, + -0.247700035572052, + 0.7174559235572815, + 0.11071103811264038, + -0.023232340812683105, + 0.6976174116134644, + -2.9463140964508057, + -0.5927516222000122 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.06662726402282715, + -0.05619156360626221, + -0.02257448434829712, + 0.539892315864563, + 0.053197890520095825, + -0.5884796380996704, + 0.0, + -0.04219311475753784, + 0.010949447751045227, + 0.027476036921143532, + -0.02041013538837433, + -0.01791653037071228, + -1.9082190990447998, + 0.0, + 0.01256529800593853, + 0.014024228788912296, + 0.03545428439974785, + 0.019800782203674316, + -0.022738635540008545, + -1.5966379642486572, + 0.0 + ] + ], + "predicted_scores": [ + 6.4563469886779785, + 6.449648857116699, + 6.180249214172363, + 6.115482330322266, + 5.748321533203125, + 5.519894599914551, + 5.004046440124512, + 4.754415035247803 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1522272686/train0", + "selected_index": 0, + "state_hash": "10034ca87ce4b5c491c0aa23d2a30389ba8265fa695d65198bf719c54565cf2a", + "stored_base_utility": 0.8326650857925415, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8388344049453735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8388344049453735 + }, + "base_progress": 0.8388344049453735, + "base_success": false, + "base_utility": 0.8388344049453735, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8135772347450256, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8135772347450256 + }, + { + "contact_quality": null, + "progress": 0.1985817849636078, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1985817849636078 + }, + { + "contact_quality": null, + "progress": 0.20784005522727966, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20784005522727966 + }, + { + "contact_quality": null, + "progress": 0.20173269510269165, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20173269510269165 + }, + { + "contact_quality": null, + "progress": 0.2171078473329544, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2171078473329544 + }, + { + "contact_quality": null, + "progress": 0.18976537883281708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18976537883281708 + }, + { + "contact_quality": null, + "progress": 0.20651468634605408, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20651468634605408 + }, + { + "contact_quality": null, + "progress": 0.22675341367721558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22675341367721558 + } + ], + "candidate_progress": [ + 0.8135772347450256, + 0.1985817849636078, + 0.20784005522727966, + 0.20173269510269165, + 0.2171078473329544, + 0.18976537883281708, + 0.20651468634605408, + 0.22675341367721558 + ], + "candidate_source_chart_ids": [ + "ms-eb7189a4f2b0f0a5d4bd", + "ms-5a456ddf97d294548cf1", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-0b226017db7f07c5307e", + "ms-5dfbb16e173e1a3e2bd6" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-63ac245579b1188935ad", + "generated_tangents": [ + [ + 0.024005480110645294, + -0.03487171605229378, + -0.08501949906349182, + 0.1531437635421753, + 0.059081971645355225, + -1.9487231969833374, + -0.12814170122146606, + -0.027122944593429565, + -0.06875290721654892, + -0.05074097961187363, + 0.21672600507736206, + 0.17542648315429688, + -1.233362078666687, + -0.028380826115608215, + 0.049982257187366486, + 0.07889854907989502, + 0.02847789227962494, + -0.18392501771450043, + 0.09474422037601471, + -1.1280198097229004, + -0.10618850588798523 + ], + [ + 0.0801134705543518, + -0.04132566228508949, + -0.07826349139213562, + 0.05996397137641907, + -0.05156397819519043, + -1.947234034538269, + -0.08109259605407715, + 0.011223122477531433, + -0.07242417335510254, + 0.03295724838972092, + 0.09064805507659912, + 0.08040205389261246, + -1.288273811340332, + -0.036930009722709656, + -0.016599636524915695, + 0.052263420075178146, + 0.09989877045154572, + -0.20034591853618622, + 0.056101441383361816, + -1.1894292831420898, + -0.04864327609539032 + ], + [ + 0.11546728014945984, + -0.01809103786945343, + -0.11312094330787659, + 0.02970258891582489, + -0.06853586435317993, + -2.0808277130126953, + -0.22655653953552246, + 0.04426082968711853, + -0.09063653647899628, + -0.07344181090593338, + 0.1791953146457672, + -0.04234561324119568, + -1.3234821557998657, + -0.15130607783794403, + 0.024940162897109985, + 0.07820416986942291, + 0.10006025433540344, + -0.10336704552173615, + 0.03928545117378235, + -1.260929822921753, + -0.13187727332115173 + ], + [ + 0.08999529480934143, + -0.017735548317432404, + -0.07061442732810974, + 0.00416797399520874, + -0.1770777702331543, + -1.8116111755371094, + -0.2593742609024048, + 0.06652607023715973, + -0.12665021419525146, + -0.08371938019990921, + 0.11404116451740265, + 0.046173423528671265, + -1.135612964630127, + -0.18963705003261566, + 0.07155746966600418, + 0.08745545148849487, + 0.1271779090166092, + -0.10870565474033356, + -0.06397175788879395, + -1.120802640914917, + -0.18013185262680054 + ], + [ + 0.04854058474302292, + -0.03224082291126251, + -0.04689648747444153, + -0.010309815406799316, + -0.11356344819068909, + -1.9703437089920044, + -0.23080027103424072, + 0.06358575820922852, + -0.11792464554309845, + -0.03149311989545822, + 0.08329933136701584, + 0.0213300883769989, + -1.308495283126831, + -0.12958507239818573, + 0.044057831168174744, + 0.06347276270389557, + 0.07978248596191406, + -0.1787070482969284, + -0.034421712160110474, + -1.2446329593658447, + -0.1634177267551422 + ], + [ + 0.1383705884218216, + 0.010324206203222275, + -0.07752159237861633, + -0.0030374079942703247, + -0.01030801236629486, + -1.7836098670959473, + -0.18338048458099365, + 0.1067870482802391, + -0.16640116274356842, + 0.031050056219100952, + 0.22548094391822815, + 0.028106689453125, + -1.0555660724639893, + -0.08256937563419342, + -0.003275398164987564, + 0.034911006689071655, + 0.05868661403656006, + -0.023772746324539185, + -0.0549464225769043, + -1.021216630935669, + -0.07599213719367981 + ], + [ + 0.03586271405220032, + -0.052382975816726685, + -0.016012579202651978, + -0.00796923041343689, + -0.16297471523284912, + -2.0362632274627686, + -0.3756687641143799, + 0.04673069715499878, + -0.10398165136575699, + -0.06536772102117538, + 0.16496363282203674, + 0.08488186448812485, + -1.3018834590911865, + -0.12473703920841217, + 0.0551728680729866, + 0.14191022515296936, + 0.16451120376586914, + -0.09062518179416656, + -0.04602450132369995, + -1.2958133220672607, + -0.1575165092945099 + ], + [ + 0.0409599244594574, + -0.10623904317617416, + -0.14421579241752625, + -0.2416801005601883, + -0.1338302195072174, + -2.4233953952789307, + -0.45108890533447266, + 0.2016845941543579, + -0.12244339287281036, + -0.1424170434474945, + 0.3580399751663208, + -0.14196431636810303, + -1.1082514524459839, + -0.19893445074558258, + -0.08443394303321838, + 0.1523725390434265, + 0.13660648465156555, + -0.1062401682138443, + -0.09151360392570496, + -1.2155197858810425, + -0.29695650935173035 + ] + ], + "generated_utilities": [ + 0.8135772347450256, + 0.1985817849636078, + 0.20784005522727966, + 0.20173269510269165, + 0.2171078473329544, + 0.18976537883281708, + 0.20651468634605408, + 0.22675341367721558 + ], + "group_id": "ms-63ac245579b1188935ad", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.855198860168457, + 0.8138954639434814, + 0.20312510430812836, + 2.0, + 0.7862945795059204, + 0.7843606472015381, + 0.6004383563995361, + 0.15506848692893982, + 0.15924105048179626, + 0.132452130317688, + 0.05381816625595093, + 0.09561765193939209, + 0.07140937447547913, + 0.10806754231452942, + 0.023168057203292847, + 0.07957816123962402 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.007633227854967117, + 0.02389371022582054, + 0.2181050330400467, + 0.026324495673179626, + 0.1209479570388794, + -1.3937149047851562, + 0.0, + -0.2493116855621338, + -0.17256586253643036, + -0.06812826544046402, + 0.019445210695266724, + -0.2657848000526428, + -1.0657541751861572, + 0.0, + -0.04306507110595703, + -0.07170329242944717, + 0.035314902663230896, + 0.06211268901824951, + 0.18158195912837982, + -0.7849693298339844, + 0.0 + ], + [ + -0.011555496603250504, + 0.0016381663735955954, + 0.019193550571799278, + 0.26171422004699707, + 0.08122420310974121, + -2.3937149047851562, + -1.53977632522583, + -0.00580335408449173, + -0.0020052569452673197, + 0.01787502132356167, + 0.4615197777748108, + 0.1938573122024536, + -2.0657541751861572, + -1.4660639762878418, + -0.00267898291349411, + -0.0038005562964826822, + 0.0179074015468359, + 0.5644202828407288, + 0.2690354883670807, + -1.7849693298339844, + -1.3308851718902588 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.07955265045166, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9321279525756836, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.6617703437805176 + ], + [ + -0.16036809980869293, + 0.11436319351196289, + -0.14857570827007294, + 0.09414434432983398, + 0.02274160459637642, + -2.426297903060913, + -1.7686878442764282, + -0.26862773299217224, + 0.16241224110126495, + 0.24208402633666992, + 0.4166663587093353, + 0.07841069251298904, + -2.2318098545074463, + -1.2228643894195557, + -0.08888694643974304, + 0.20051127672195435, + -0.14729085564613342, + 0.5799955129623413, + 0.33039355278015137, + -2.00492262840271, + -1.4948352575302124 + ], + [ + -0.3820815682411194, + -0.3679526746273041, + 0.18845602869987488, + 0.4168931245803833, + -0.007936321198940277, + -2.700572967529297, + -1.795093059539795, + 0.1203194335103035, + 0.19610334932804108, + 0.2749679982662201, + 0.8331882357597351, + -0.06589585542678833, + -2.333136558532715, + -1.1008005142211914, + -0.02170141413807869, + 0.24186654388904572, + -0.3734443485736847, + 0.5144574046134949, + 0.3134346902370453, + -1.5088386535644531, + -1.1978046894073486 + ], + [ + 0.13625949621200562, + 0.47836095094680786, + -0.05790846049785614, + -0.09614765644073486, + -0.14281350374221802, + -2.522264003753662, + -1.0473889112472534, + 0.4792231023311615, + -0.4869813621044159, + 0.17644789814949036, + 0.8425257205963135, + -0.28899794816970825, + -2.5508456230163574, + -0.9762126207351685, + -0.16381770372390747, + 0.1586131900548935, + -0.3460317850112915, + 0.6056831479072571, + 0.7327044010162354, + -1.494289755821228, + -1.4671605825424194 + ], + [ + -0.40018847584724426, + -0.028111951425671577, + 0.31324630975723267, + -0.22690656781196594, + 0.2902344763278961, + -2.9841673374176025, + -1.4654184579849243, + -0.1155615746974945, + -0.4125809073448181, + 0.6098153591156006, + 0.8544195890426636, + 0.22702345252037048, + -1.7853796482086182, + -1.3474400043487549, + -0.454882949590683, + -0.3122820556163788, + 0.22399993240833282, + 0.05299121141433716, + 0.07802119851112366, + -1.3896000385284424, + -1.894580602645874 + ], + [ + -0.6587541699409485, + -0.6905457973480225, + 0.10385943204164505, + -0.2785833477973938, + 0.2321375161409378, + -2.565701723098755, + -1.1605603694915771, + -0.6775721907615662, + -0.30887091159820557, + 0.1186944767832756, + 1.0924456119537354, + 0.5009081363677979, + -1.8667830228805542, + -1.2491121292114258, + -0.11290804296731949, + -0.03346216678619385, + -0.28778207302093506, + 1.2255009412765503, + 0.6856998801231384, + -1.1723419427871704, + -1.1220875978469849 + ], + [ + -0.5019480586051941, + 0.07311037182807922, + 0.5101989507675171, + 0.7159117460250854, + 0.005071915686130524, + -2.6556949615478516, + -2.321233034133911, + 0.6035679578781128, + -0.13785676658153534, + -0.4309522807598114, + 0.9664275050163269, + -0.020235419273376465, + -1.6894025802612305, + -0.7511875033378601, + 0.5713916420936584, + 0.24962951242923737, + 0.40067923069000244, + 0.12209588289260864, + 0.03222101926803589, + -2.27744460105896, + -1.9113109111785889 + ], + [ + 0.7552750110626221, + -0.8085685968399048, + 0.20896124839782715, + -0.6043995022773743, + 0.733860969543457, + -1.610467791557312, + -1.0368051528930664, + -0.02292630262672901, + 0.6320546269416809, + 0.14528857171535492, + 0.8372507095336914, + 0.41523146629333496, + -2.339751958847046, + -0.6185142993927002, + -0.5750091075897217, + 0.12324154376983643, + 0.2804809510707855, + 1.0503896474838257, + -0.15392154455184937, + -0.9555137753486633, + -1.517809510231018 + ], + [ + 0.8853163123130798, + 0.09475164860486984, + -0.13681088387966156, + 0.08429434895515442, + 0.13555681705474854, + -1.5859344005584717, + -2.1957859992980957, + 0.9337366819381714, + -1.0016963481903076, + 0.11365362256765366, + -0.31175100803375244, + 0.7412776350975037, + -1.0797133445739746, + -0.8527018427848816, + 0.6110302209854126, + 0.006565961055457592, + 0.38363590836524963, + 0.5695288181304932, + -0.2547180950641632, + -2.6268043518066406, + -1.748514175415039 + ], + [ + 0.513801097869873, + 0.7078304886817932, + 0.7309613823890686, + -0.32746708393096924, + -0.36263394355773926, + -3.3305888175964355, + -1.2108653783798218, + 0.5104080438613892, + -0.26447102427482605, + 0.6361017823219299, + 0.5986918210983276, + -0.48598653078079224, + -2.4424209594726562, + -1.4694169759750366, + -0.5649402737617493, + -0.22020848095417023, + 0.6405193209648132, + -0.0698668360710144, + 0.7155715227127075, + -2.352860450744629, + -0.7502654194831848 + ], + [ + -0.9911022782325745, + 0.9677192568778992, + 0.5195661187171936, + 0.6733791828155518, + 0.7589308023452759, + -3.3654959201812744, + -0.6360958814620972, + 0.5926312208175659, + -0.06773044914007187, + 0.3785405457019806, + 0.5934506058692932, + 0.6434512138366699, + -2.891094923019409, + -0.8924227356910706, + 0.8036434650421143, + 0.6109268665313721, + -0.6607317924499512, + 1.0173940658569336, + 0.2546094059944153, + -1.4653143882751465, + -2.0709643363952637 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16686837375164032, + 0.1354454904794693, + -0.11150697618722916, + 0.7247087359428406, + 0.2886054813861847, + -1.3937149047851562, + 0.0, + 0.6050093770027161, + 0.20291854441165924, + 0.5827622413635254, + -0.2687305808067322, + -0.25171223282814026, + -1.0657541751861572, + 0.0, + 0.29065752029418945, + -0.023413484916090965, + -0.24587245285511017, + 0.4600236713886261, + -0.4278547465801239, + -0.7849693298339844, + 0.0 + ] + ], + "predicted_scores": [ + 3.353055715560913, + 3.3517229557037354, + 3.3192176818847656, + 3.2793989181518555, + 3.27476167678833, + 2.8849124908447266, + 2.8019933700561523, + 2.0151007175445557 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "211245842/train0", + "selected_index": 0, + "state_hash": "fb52e84b6addb8bdddf02a80d5c8567e0fa27195729e8440271b710441061b7f", + "stored_base_utility": 0.855198860168457, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-875a4668671c02054a3c", + "ms-20f3cfcf183c69853b1e", + "ms-0949097d19e597fc7f63", + "ms-875a4668671c02054a3c", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "generated_tangents": [ + [ + 0.006176412105560303, + 0.015031468123197556, + -0.04347166419029236, + -0.2196698635816574, + -0.20213130116462708, + -2.1178388595581055, + -0.2735292911529541, + 0.053335875272750854, + 0.023975348100066185, + 0.04375610500574112, + 0.06376722455024719, + 0.05684523284435272, + -1.8778741359710693, + -0.11852577328681946, + 0.09389108419418335, + 0.05454123392701149, + 0.007951110601425171, + -0.1374041885137558, + 0.05853183567523956, + -1.7561705112457275, + -0.08032986521720886 + ], + [ + 0.05350024253129959, + -0.008277975022792816, + -0.011172980070114136, + -0.19443228840827942, + -0.06973081827163696, + -2.4749886989593506, + -0.21836239099502563, + -0.0009081363677978516, + 0.04159802198410034, + 0.03741426020860672, + 0.06233856454491615, + -0.03803148865699768, + -1.9127192497253418, + -0.11715845763683319, + 0.03506483882665634, + 0.05396481230854988, + 0.05986599624156952, + -0.1762569695711136, + 0.054152652621269226, + -1.5897595882415771, + -0.030290141701698303 + ], + [ + 0.10290880501270294, + -0.007280625402927399, + 0.07842746376991272, + -0.1832699030637741, + 0.05195482075214386, + -2.537594795227051, + -0.46784335374832153, + -0.02841256558895111, + 0.006892894394695759, + -0.015205807983875275, + 0.2386656403541565, + 0.1220480352640152, + -1.7291165590286255, + -0.07446907460689545, + -0.008053001016378403, + 0.07738210260868073, + 0.08646567165851593, + -0.17380063235759735, + 0.03588852286338806, + -1.7457711696624756, + 0.07436027377843857 + ], + [ + 0.02357722818851471, + 0.01157565787434578, + -0.058383435010910034, + -0.20330168306827545, + -0.13086026906967163, + -2.556213140487671, + -0.6527280211448669, + 0.12240959703922272, + 0.010187995620071888, + -0.015915729105472565, + 0.1939363181591034, + 0.03682531416416168, + -1.8583874702453613, + -0.553052544593811, + 0.03224678337574005, + 0.11126010119915009, + 0.0709986686706543, + -0.048767030239105225, + 0.21647703647613525, + -1.8499352931976318, + -0.08911240100860596 + ], + [ + -0.0158156156539917, + -0.025778014212846756, + 0.046597957611083984, + -0.2615222632884979, + 0.09938192367553711, + -2.2407608032226562, + -0.43458765745162964, + 0.21323370933532715, + 0.059951357543468475, + 0.04161942005157471, + 0.023588914424180984, + 0.14518174529075623, + -1.6217304468154907, + -0.10320907831192017, + 0.19596292078495026, + 0.11118914186954498, + 0.06349106132984161, + -0.1608113795518875, + 0.2863720953464508, + -1.8274946212768555, + -0.01357463002204895 + ], + [ + 0.011767670512199402, + -0.06779482215642929, + 0.2283744513988495, + -0.4103007912635803, + -0.03333577513694763, + -2.293379545211792, + -0.6917316317558289, + 0.25185275077819824, + -0.027487674728035927, + 0.011037394404411316, + 0.22671815752983093, + 0.16355301439762115, + -1.3806061744689941, + -0.060945481061935425, + 0.20913317799568176, + 0.03424268215894699, + 0.07699966430664062, + -0.22638840973377228, + 0.2093367576599121, + -1.8172316551208496, + -0.06600260734558105 + ], + [ + 0.043054208159446716, + 0.006995994597673416, + 0.040226101875305176, + -0.20659546554088593, + 0.016365110874176025, + -2.440964698791504, + -0.5692809224128723, + 0.08967659622430801, + -0.005274813622236252, + -0.007912807166576385, + 0.11555744707584381, + 0.13973061740398407, + -1.6668994426727295, + -0.20925067365169525, + 0.01616767793893814, + 0.10505478084087372, + 0.05943283438682556, + -0.22194214165210724, + 0.15125006437301636, + -1.7992901802062988, + 0.15188738703727722 + ], + [ + 0.12475170940160751, + -0.069279246032238, + -0.30603304505348206, + -0.3186453878879547, + 0.4094564914703369, + -1.6150081157684326, + -0.6829964518547058, + 0.06369796395301819, + 0.06468655169010162, + 0.0865817442536354, + -0.09368977695703506, + -0.0324195921421051, + -0.980171799659729, + -0.3847846984863281, + 0.31843870878219604, + 0.08126239478588104, + 0.1735626757144928, + -0.2221849113702774, + 0.3698451817035675, + -1.811368465423584, + 0.22155342996120453 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-6c8d06bace98a28a8e4e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8583254218101501, + 2.0, + 0.8125, + 0.12893107533454895, + 0.8582976460456848, + 0.15346482396125793, + 0.18961156904697418, + 0.13901273906230927, + 0.10113435983657837, + 0.11161506175994873, + 0.09923169016838074, + 0.16662254929542542, + 0.04211682081222534, + 0.024710848927497864, + 0.05722464621067047, + 0.8584086298942566 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.10696407407522202, + 0.3957799971103668, + 0.15132969617843628, + -0.32772913575172424, + 0.005261177197098732, + -2.2650365829467773, + 0.0, + 0.3958877623081207, + -0.5120954513549805, + -0.4317946135997772, + -0.3970779776573181, + 0.5317854285240173, + -1.8659565448760986, + 0.0, + 0.1198807880282402, + -0.11760073900222778, + -0.6048114895820618, + 0.15180528163909912, + -0.532695472240448, + -1.6115882396697998, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.120756149291992, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.070481538772583, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0495758056640625 + ], + [ + 0.30105674266815186, + 0.4776385426521301, + -0.22267001867294312, + -0.13300693035125732, + -0.11095686256885529, + -3.138023853302002, + -0.793931245803833, + 0.16969743371009827, + -0.16339385509490967, + 0.02508675307035446, + 0.654399573802948, + -0.22830897569656372, + -3.1443004608154297, + -1.4291013479232788, + 0.005021120421588421, + 0.005560946650803089, + -0.1901366114616394, + 1.0211858749389648, + -0.09353887289762497, + -2.3457469940185547, + -1.406853199005127 + ], + [ + 0.4160586893558502, + 0.13316196203231812, + -0.21056225895881653, + -0.25006812810897827, + 0.060951750725507736, + -2.8735690116882324, + -0.9062713980674744, + -0.28707224130630493, + -0.2396206259727478, + -0.19859735667705536, + 0.29039445519447327, + -0.24077445268630981, + -2.644789934158325, + -1.8330551385879517, + -0.23438936471939087, + 0.04620177671313286, + 0.1472887098789215, + 0.9629149436950684, + -0.020509399473667145, + -2.7985036373138428, + -1.2463163137435913 + ], + [ + 0.2486386001110077, + -0.022333532571792603, + 0.14628465473651886, + -0.8381847143173218, + 0.21282373368740082, + -3.585387945175171, + -1.2156991958618164, + -0.08670558780431747, + -0.3178575336933136, + 0.4322021007537842, + 0.6421236395835876, + -0.33922863006591797, + -3.004714012145996, + -1.19637131690979, + -0.015640778467059135, + 0.10077434033155441, + -0.449941486120224, + 1.0431067943572998, + 0.2899783253669739, + -2.500624179840088, + -1.6148788928985596 + ], + [ + 0.46770766377449036, + 0.03902164101600647, + 0.2556130588054657, + -0.45893940329551697, + 0.08939866721630096, + -3.163339853286743, + -0.7599828243255615, + 0.3327450156211853, + 0.3252364993095398, + 0.3067358434200287, + 1.1394517421722412, + 0.472858726978302, + -3.1504058837890625, + -1.9780522584915161, + -0.26712900400161743, + -0.262444406747818, + 0.20269782841205597, + 1.336982250213623, + -0.06957009434700012, + -2.0651021003723145, + -1.8484716415405273 + ], + [ + 0.48348987102508545, + 0.19676931202411652, + -0.20689666271209717, + -0.21521174907684326, + -0.5048230886459351, + -3.491732358932495, + -0.7390058636665344, + 0.18321916460990906, + -0.1509065330028534, + 0.27166980504989624, + 1.004181981086731, + -0.5531666874885559, + -2.531594753265381, + -2.0243897438049316, + -0.44593361020088196, + 0.49786314368247986, + -0.006632372736930847, + 0.24423635005950928, + -0.6331305503845215, + -2.7707719802856445, + -2.1364808082580566 + ], + [ + -0.5684993863105774, + 0.0623701810836792, + -0.10449839383363724, + -0.7858190536499023, + -0.057080868631601334, + -3.981018304824829, + -0.6721940636634827, + 0.10191962122917175, + -0.7216448783874512, + 0.3870577812194824, + 0.5508396625518799, + -0.6811783313751221, + -3.6131227016448975, + -1.7484180927276611, + 0.34220433235168457, + -0.16193144023418427, + 0.16876991093158722, + 0.6509957909584045, + 0.2548854351043701, + -2.0779964923858643, + -1.5782216787338257 + ], + [ + -0.1224658265709877, + 0.22953394055366516, + -0.41326335072517395, + 0.19802969694137573, + 0.6866055130958557, + -2.6415462493896484, + -1.049781322479248, + -0.6788480281829834, + -0.8187242746353149, + 0.6141759157180786, + 1.3649951219558716, + -0.36378854513168335, + -2.2142889499664307, + -1.3759111166000366, + 0.583359956741333, + 0.6860965490341187, + 0.8285356760025024, + 1.4575884342193604, + -0.4101440906524658, + -2.218319892883301, + -1.5390111207962036 + ], + [ + 0.3368571698665619, + 0.4528505206108093, + -0.2985517382621765, + 0.37766069173812866, + 0.2064778208732605, + -2.8070714473724365, + -1.5598305463790894, + -0.9032005071640015, + -0.48420336842536926, + 0.639835000038147, + -0.3801422119140625, + -0.1376074254512787, + -3.369077205657959, + -1.9231767654418945, + -0.16211053729057312, + 0.53387051820755, + 0.6583098769187927, + 0.7611914873123169, + -0.46404945850372314, + -2.7554476261138916, + -1.4307246208190918 + ], + [ + 0.6002402901649475, + -0.14213827252388, + 0.30183425545692444, + -0.20118463039398193, + -0.16981711983680725, + -3.4212875366210938, + -0.2252577543258667, + 0.34123149514198303, + 0.003176456317305565, + 0.7433648109436035, + 1.1403348445892334, + 0.5709706544876099, + -1.8894051313400269, + -0.7200812101364136, + -0.7235198020935059, + 0.2147149294614792, + -0.28744715452194214, + 1.0323235988616943, + 0.5815067291259766, + -2.582993984222412, + -1.2160475254058838 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.019069351255893707, + 0.012574762105941772, + 0.07413499802350998, + -0.04097828269004822, + -0.03991875424981117, + -2.2650365829467773, + 0.0, + -0.06082560494542122, + 0.05821063742041588, + 0.07896332442760468, + -0.037061095237731934, + -0.08879614621400833, + -1.8659565448760986, + 0.0, + -0.022465867921710014, + -0.026843247935175896, + 0.04789978265762329, + -0.019972383975982666, + -0.04040021449327469, + -1.6115882396697998, + 0.0 + ] + ], + "predicted_scores": [ + 3.6132442951202393, + 3.3841521739959717, + 2.2444205284118652, + 2.1902849674224854, + 2.1477720737457275, + 2.0814521312713623, + 1.9497334957122803, + 0.9199833869934082 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1665739511/train0", + "selected_index": 0, + "state_hash": "0f77663511063f17b0962024501b863c62b881513459d8bfa3ac00bbaed67de4", + "stored_base_utility": 0.8583254218101501, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1245432198047638, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1245432198047638 + }, + { + "contact_quality": null, + "progress": 0.574478030204773, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.574478030204773 + }, + { + "contact_quality": null, + "progress": 0.23411184549331665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23411184549331665 + }, + { + "contact_quality": null, + "progress": 0.23169663548469543, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23169663548469543 + }, + { + "contact_quality": null, + "progress": 0.1756683886051178, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1756683886051178 + }, + { + "contact_quality": null, + "progress": 0.1700567752122879, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1700567752122879 + }, + { + "contact_quality": null, + "progress": 0.20359350740909576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20359350740909576 + }, + { + "contact_quality": null, + "progress": 0.1943017691373825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1943017691373825 + } + ], + "candidate_progress": [ + 0.1245432198047638, + 0.574478030204773, + 0.23411184549331665, + 0.23169663548469543, + 0.1756683886051178, + 0.1700567752122879, + 0.20359350740909576, + 0.1943017691373825 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-72777024349c2aacd5e0", + "generated_tangents": [ + [ + 0.15045036375522614, + 0.1372452974319458, + 0.29213792085647583, + -0.8182337284088135, + 0.47608843445777893, + -0.38249924778938293, + 0.5203595161437988, + 0.14830099046230316, + -0.013280622661113739, + -0.10061783343553543, + -0.06293529272079468, + 0.06389458477497101, + -0.6957372426986694, + -0.5106804966926575, + 0.1213463544845581, + -0.08478164672851562, + 0.32867300510406494, + -0.3360060453414917, + -0.11113515496253967, + -1.101579189300537, + -0.2569182813167572 + ], + [ + 0.17874856293201447, + 0.2526323199272156, + -0.24080786108970642, + -0.6450339555740356, + -0.06338244676589966, + -0.6732321977615356, + 0.23784828186035156, + -0.06323722004890442, + -0.06856551021337509, + -0.13422897458076477, + 0.34170064330101013, + -0.10431668162345886, + -1.5888500213623047, + -0.36079859733581543, + -0.029090922325849533, + -0.0446627214550972, + -0.056386008858680725, + -0.07992379367351532, + -0.31195753812789917, + -1.8153178691864014, + -0.0052912235260009766 + ], + [ + 0.20602259039878845, + 0.1785765141248703, + -0.4026523530483246, + -0.8586504459381104, + -0.08055484294891357, + -0.41225311160087585, + 0.023905158042907715, + -0.006460845470428467, + -0.09959809482097626, + -0.2161354124546051, + 0.09675844013690948, + -0.2766232490539551, + -1.6789705753326416, + -0.3824644088745117, + -0.10450944304466248, + -0.02792646549642086, + -0.011458426713943481, + -0.1315893679857254, + -0.37362349033355713, + -1.5534007549285889, + 0.09506411105394363 + ], + [ + 0.03876104950904846, + 0.2657865285873413, + -0.26557520031929016, + -0.6566753387451172, + -0.04386025667190552, + -0.6304527521133423, + 0.1770125925540924, + -0.00988796353340149, + -0.043510593473911285, + -0.11370385438203812, + 0.1605052351951599, + -0.1115913987159729, + -1.4697049856185913, + -0.2920401692390442, + -0.09245873987674713, + -0.010158668272197247, + 0.12769964337348938, + -0.08878220617771149, + -0.31268852949142456, + -1.7868881225585938, + 0.08572258055210114 + ], + [ + -0.08376432955265045, + 0.18322238326072693, + -0.050839394330978394, + -0.6801820993423462, + 0.17961299419403076, + -0.4552444517612457, + 0.05471804738044739, + 0.07211660593748093, + 0.003387349657714367, + -0.038424305617809296, + -0.04805950075387955, + -0.15914759039878845, + -1.7790008783340454, + 0.07511967420578003, + -0.0040402524173259735, + -0.07099927216768265, + 0.16272948682308197, + -0.023879870772361755, + 0.09710381925106049, + -1.4573769569396973, + 0.11534436047077179 + ], + [ + 0.025695331394672394, + 0.19272837042808533, + -0.015701264142990112, + -0.692416787147522, + 0.17331527173519135, + -0.2513943612575531, + -0.05424690246582031, + -0.001001805067062378, + -0.012060513719916344, + -0.058356873691082, + -0.06549026072025299, + -0.09043657779693604, + -1.683553695678711, + 0.008678898215293884, + 0.0041976794600486755, + 0.012104487046599388, + 0.1945856362581253, + -0.135456845164299, + -0.016347736120224, + -1.4026732444763184, + 0.06931431591510773 + ], + [ + 0.07290415465831757, + 0.17387908697128296, + -0.07721421122550964, + -0.7396707534790039, + 0.13897329568862915, + -0.23482796549797058, + -0.13062971830368042, + 0.02382473647594452, + -0.02543010748922825, + -0.015188314020633698, + 0.0024499744176864624, + -0.19603782892227173, + -1.6690139770507812, + 0.0981595516204834, + -0.09673325717449188, + -0.021517161279916763, + 0.07098709046840668, + -0.11696939170360565, + -0.00607028603553772, + -1.488534927368164, + 0.052709437906742096 + ], + [ + 0.18209514021873474, + 0.2496199756860733, + -0.23308709263801575, + -0.7942858934402466, + -0.0037521719932556152, + -0.45543625950813293, + 0.12795934081077576, + -0.000754043459892273, + 0.0014778561890125275, + 0.055383406579494476, + -0.03958898037672043, + 0.01392875611782074, + -1.5706225633621216, + -0.0915948748588562, + 0.05942995101213455, + 0.00810825452208519, + 0.08427196741104126, + -0.14305253326892853, + -0.24507790803909302, + -1.9048233032226562, + 0.16264042258262634 + ] + ], + "generated_utilities": [ + 0.1245432198047638, + 0.574478030204773, + 0.23411184549331665, + 0.23169663548469543, + 0.1756683886051178, + 0.1700567752122879, + 0.20359350740909576, + 0.1943017691373825 + ], + "group_id": "ms-72777024349c2aacd5e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8314920663833618, + 0.1595238745212555, + 2.0, + 0.16971935331821442, + 0.5892879366874695, + 0.1617850512266159, + 0.1877371072769165, + 0.5782577395439148, + 0.12074722349643707, + 0.11237785220146179, + 0.08792310953140259, + 0.1209893673658371, + 0.13739405572414398, + 0.0773405134677887, + 0.09497407078742981, + 0.047447383403778076 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.002835560590028763, + 0.3854118585586548, + -0.06168194115161896, + -0.16570186614990234, + -0.05051702260971069, + -0.43052077293395996, + 0.0, + 0.024141468107700348, + -0.011737043969333172, + -0.05502588301897049, + 0.013536334037780762, + -0.000911470502614975, + -1.6284780502319336, + 0.0, + 0.030459173023700714, + -0.05656937509775162, + 0.03393230587244034, + 0.06894856691360474, + 0.00450655072927475, + -1.2762084007263184, + 0.0 + ], + [ + 0.2191334217786789, + 0.3854118585586548, + 0.5048428177833557, + -0.16570186614990234, + -0.20026534795761108, + -0.43052077293395996, + 0.0, + -0.026082046329975128, + 0.03163425251841545, + -0.04950854927301407, + 0.1969367265701294, + -0.05762401595711708, + -1.6284780502319336, + 0.0, + 0.002695085946470499, + 0.34764212369918823, + 0.6051180362701416, + 0.9295560717582703, + 0.38285934925079346, + -1.2762084007263184, + 0.0 + ], + [ + -0.04624907672405243, + 1.3854118585586548, + 0.05864633619785309, + -1.1657018661499023, + 0.7997346520423889, + -1.43052077293396, + 1.1980153322219849, + -0.023269198834896088, + -0.026228107511997223, + -0.009256521239876747, + 0.6858966946601868, + 0.03280947729945183, + -2.6284780502319336, + -1.7920862436294556, + -0.0004090145230293274, + -0.00536443293094635, + 0.007614931557327509, + 0.7401981949806213, + 0.11622340977191925, + -2.2762084007263184, + -1.6813035011291504 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.3960306644439697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.584172487258911, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.362607002258301 + ], + [ + -0.13329119980335236, + 1.1171538829803467, + -0.19378040730953217, + -0.9341158866882324, + 0.8714147806167603, + -1.659934163093567, + 1.1658833026885986, + 0.23766514658927917, + 0.21810689568519592, + -0.05730185657739639, + 0.4946695566177368, + 0.17369553446769714, + -2.559515953063965, + -1.5605767965316772, + 0.08475297689437866, + 0.22485606372356415, + -0.2591644823551178, + 0.9637633562088013, + -0.08890970051288605, + -2.1015453338623047, + -1.5709789991378784 + ], + [ + -0.3507236838340759, + 1.1737966537475586, + 0.10341192036867142, + -0.7761039137840271, + 0.47493064403533936, + -1.2297213077545166, + 1.4181653261184692, + -0.1981543004512787, + -0.18248644471168518, + 0.17768341302871704, + 0.843591034412384, + -0.07443197071552277, + -2.6062240600585938, + -1.750958800315857, + -0.15503963828086853, + 0.18218432366847992, + 0.002181168645620346, + 0.8641887903213501, + -0.026101917028427124, + -1.9398812055587769, + -1.8561092615127563 + ], + [ + -0.22363319993019104, + 1.533349871635437, + 0.4454386830329895, + -1.5271000862121582, + 0.8945828080177307, + -1.105146884918213, + 0.7171626687049866, + -0.20207461714744568, + 0.31458696722984314, + 0.20214562118053436, + 1.0593571662902832, + -0.4655032455921173, + -2.8809642791748047, + -1.580764651298523, + -0.22098299860954285, + -0.08731096982955933, + -0.15165574848651886, + 0.89030921459198, + 0.611029326915741, + -2.2792654037475586, + -1.798964262008667 + ], + [ + -0.3702698349952698, + 0.9771865606307983, + 0.27564477920532227, + -1.7643829584121704, + 0.7298657894134521, + -1.875503420829773, + 1.4598875045776367, + 0.47600361704826355, + 0.13439032435417175, + -0.23905222117900848, + 1.2192013263702393, + -0.44705769419670105, + -2.603621482849121, + -1.3898506164550781, + 0.16782721877098083, + -0.2559890151023865, + 0.2965841293334961, + 0.8750308752059937, + 0.1521819680929184, + -2.3417282104492188, + -2.2663588523864746 + ], + [ + -0.31705379486083984, + 0.866703450679779, + -0.09125347435474396, + -1.5959886312484741, + 0.8041162490844727, + -1.7037310600280762, + 1.6304850578308105, + 0.11580469459295273, + 0.2654762864112854, + 0.12200001627206802, + 0.6904033422470093, + 0.19859254360198975, + -2.7615463733673096, + -2.059410333633423, + 0.5591834783554077, + -0.4801292419433594, + -0.3343847393989563, + 0.8787070512771606, + -0.14748375117778778, + -2.0334713459014893, + -1.1611082553863525 + ], + [ + -0.11324095726013184, + 0.8777624368667603, + 0.8215147256851196, + -1.5343353748321533, + 0.881126344203949, + -1.0607860088348389, + 0.5398488640785217, + 0.6020828485488892, + -0.5065795183181763, + 0.02798180840909481, + 0.33880186080932617, + 0.320717990398407, + -3.3552045822143555, + -1.1435439586639404, + 0.19279444217681885, + -0.5245973467826843, + 0.30647164583206177, + 1.3726601600646973, + 0.3123217523097992, + -2.9675674438476562, + -1.7007204294204712 + ], + [ + -0.6432619094848633, + 0.79607754945755, + -0.49864423274993896, + -1.0710605382919312, + 1.0709084272384644, + -1.612175703048706, + 1.2555015087127686, + 0.2873077392578125, + -0.36858057975769043, + 0.05837755650281906, + 1.1025807857513428, + 0.7949575185775757, + -3.2468795776367188, + -2.556741237640381, + -0.6579093337059021, + -0.8874800801277161, + 0.7922345399856567, + 0.4745541512966156, + 0.07610362768173218, + -2.041800022125244, + -2.2069954872131348 + ], + [ + -0.9264093637466431, + 1.9828436374664307, + 0.7907096147537231, + -1.260971188545227, + 1.7548933029174805, + -2.2685132026672363, + 1.1821955442428589, + -0.49968039989471436, + 0.10982713848352432, + 0.2384604662656784, + 0.2075445055961609, + -0.7872775793075562, + -2.076058864593506, + -1.1888585090637207, + 0.7447717785835266, + 0.2355569452047348, + -0.3163020610809326, + 0.3282558023929596, + -0.4075327515602112, + -3.2451019287109375, + -2.4086263179779053 + ], + [ + -0.7190575003623962, + 1.3702023029327393, + 0.8307373523712158, + -0.7533494234085083, + 0.574251651763916, + -2.3376822471618652, + 0.5512753129005432, + -0.04391670227050781, + 0.2242431342601776, + -0.46594029664993286, + 0.5667901039123535, + -0.12832416594028473, + -2.9870448112487793, + -1.2032369375228882, + 0.17313259840011597, + 0.5932936668395996, + 0.7428228259086609, + 0.6570835113525391, + -0.1891915649175644, + -1.3703107833862305, + -1.7630409002304077 + ], + [ + 0.799586832523346, + 1.0164330005645752, + -0.49519193172454834, + -0.6983597278594971, + 1.7632484436035156, + -0.9710203409194946, + 1.5416830778121948, + 0.9653489589691162, + 0.06498706340789795, + 0.2132291942834854, + -0.1650814414024353, + 0.06786654889583588, + -1.8782389163970947, + -2.3576126098632812, + 0.9410911798477173, + -0.576595664024353, + -0.8031080365180969, + 0.7120062112808228, + 0.6479252576828003, + -2.3928208351135254, + -1.4952012300491333 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.11725310236215591, + 0.3854118585586548, + 0.06966311484575272, + -0.16570186614990234, + 0.06442362070083618, + -0.43052077293395996, + 0.0, + 0.11858775466680527, + 0.0573427751660347, + -0.14938217401504517, + -0.01657569408416748, + 0.060478124767541885, + -1.6284780502319336, + 0.0, + 0.016584808006882668, + 0.18205130100250244, + -0.13064442574977875, + 0.15621143579483032, + 0.056775350123643875, + -1.2762084007263184, + 0.0 + ] + ], + "predicted_scores": [ + 5.09656286239624, + 4.6442461013793945, + 4.363131999969482, + 4.087330341339111, + 4.079741477966309, + 3.85636568069458, + 3.712235927581787, + 3.2982747554779053 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "256770874/train0", + "selected_index": 0, + "state_hash": "3a7aa4ba27a5b34cdeb0f31ce9818588b1dd4ede96bf546f07302616c2e9b3a6", + "stored_base_utility": 0.8314920663833618, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20372186601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20372186601161957 + }, + "base_progress": 0.20372186601161957, + "base_success": false, + "base_utility": 0.20372186601161957, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2191106230020523, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2191106230020523 + }, + { + "contact_quality": null, + "progress": 0.5739269256591797, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5739269256591797 + }, + { + "contact_quality": null, + "progress": 0.5799027681350708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5799027681350708 + }, + { + "contact_quality": null, + "progress": 0.2002597153186798, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2002597153186798 + }, + { + "contact_quality": null, + "progress": 0.1847754418849945, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1847754418849945 + }, + { + "contact_quality": null, + "progress": 0.5602454543113708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5602454543113708 + }, + { + "contact_quality": null, + "progress": 0.17806151509284973, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17806151509284973 + }, + { + "contact_quality": null, + "progress": 0.5666472911834717, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5666472911834717 + } + ], + "candidate_progress": [ + 0.2191106230020523, + 0.5739269256591797, + 0.5799027681350708, + 0.2002597153186798, + 0.1847754418849945, + 0.5602454543113708, + 0.17806151509284973, + 0.5666472911834717 + ], + "candidate_source_chart_ids": [ + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-626ac2ad9d0dd95f1a2e", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "generated_tangents": [ + [ + -0.021352455019950867, + -0.06388239562511444, + -0.19936755299568176, + 0.2723495066165924, + 0.05314575135707855, + -0.6790894269943237, + -0.08313643932342529, + 0.03967951238155365, + 0.04698224365711212, + 0.0017345771193504333, + -0.04005324840545654, + -0.02530290186405182, + -1.682046890258789, + -0.021227702498435974, + 0.09034234285354614, + 0.06362509727478027, + -0.06847070157527924, + -0.17661263048648834, + 0.03995142877101898, + -1.524829387664795, + 0.0024154335260391235 + ], + [ + 0.016905948519706726, + -0.12380242347717285, + -0.20470955967903137, + 0.24490293860435486, + 0.10680311918258667, + -0.6448712348937988, + -0.013684302568435669, + -0.006054997444152832, + 0.0004276838153600693, + 0.008075475692749023, + -0.13646629452705383, + 0.03656740486621857, + -1.6498140096664429, + 0.05820601433515549, + 0.09693363308906555, + 0.051125843077898026, + -0.0516250878572464, + -0.17521561682224274, + 0.04456280171871185, + -1.4436814785003662, + 0.041915059089660645 + ], + [ + -0.1459028571844101, + -0.09840792417526245, + -0.2813153564929962, + 0.24227577447891235, + -0.024093836545944214, + -0.5552691221237183, + -0.1000216007232666, + 0.054895058274269104, + -0.002643140032887459, + 0.021926842629909515, + -0.10481435805559158, + 0.16555088758468628, + -1.4554897546768188, + -0.04020188748836517, + 0.14717628061771393, + 0.10132670402526855, + -0.011192530393600464, + -0.18386848270893097, + 0.07136976718902588, + -1.3921089172363281, + -0.03706677258014679 + ], + [ + 0.01372237503528595, + -0.09023064374923706, + -0.19493785500526428, + 0.2167317271232605, + 0.05171874165534973, + -0.9290481805801392, + -0.1885535717010498, + 0.0333847850561142, + 0.06037026643753052, + 0.02035164088010788, + -0.0007323399186134338, + -0.08101320266723633, + -1.5808125734329224, + -0.04795512557029724, + 0.05824476107954979, + 0.08652494847774506, + -0.056111469864845276, + -0.12156517803668976, + 0.06414395570755005, + -1.3584651947021484, + -0.030695393681526184 + ], + [ + -0.014965668320655823, + -0.13042062520980835, + -0.13815221190452576, + 0.30335959792137146, + 0.18823647499084473, + -0.1961337924003601, + -0.11898910999298096, + -0.12108290195465088, + -0.07559175789356232, + 0.10593551397323608, + -0.17041346430778503, + -0.02239827811717987, + -1.5221431255340576, + -0.010524049401283264, + 0.2232595533132553, + 0.06306511163711548, + 0.048533231019973755, + -0.010736271739006042, + 0.04079015552997589, + -1.7418251037597656, + -0.013632670044898987 + ], + [ + -0.0961618423461914, + -0.15962864458560944, + -0.370579332113266, + 0.5263097286224365, + 0.09191268682479858, + -0.6222783327102661, + -0.43542051315307617, + 0.11238838732242584, + -0.036678895354270935, + -0.06750737875699997, + -0.037780873477458954, + 0.33894994854927063, + -1.5700141191482544, + 0.05773831158876419, + 0.15092140436172485, + 0.14408482611179352, + 0.003930628299713135, + -0.1570245772600174, + 0.24228744208812714, + -1.5104854106903076, + 0.13777191936969757 + ], + [ + -0.09765744209289551, + -0.14905546605587006, + -0.08675655722618103, + 0.4377281963825226, + 0.31750836968421936, + -0.19685977697372437, + -0.016438215970993042, + 0.027669772505760193, + -0.043940894305706024, + 0.11480231583118439, + -0.07276531308889389, + 0.16765084862709045, + -1.4150536060333252, + 0.29519912600517273, + 0.0946052148938179, + -0.056313324719667435, + -0.06864838302135468, + -0.07631354033946991, + 0.06261126697063446, + -1.8008344173431396, + 0.0017832964658737183 + ], + [ + -0.15826822817325592, + -0.24093644320964813, + -0.45203807950019836, + 0.39025309681892395, + 0.23050133883953094, + -0.5757313966751099, + -0.4597776532173157, + 0.11452986299991608, + -0.1091751903295517, + 0.022243544459342957, + -0.1284382939338684, + 0.5329670906066895, + -1.376935601234436, + 0.16788709163665771, + 0.05883866921067238, + 0.07033094763755798, + -0.11124368011951447, + -0.1769326776266098, + 0.1647515594959259, + -1.5973732471466064, + 0.18438316881656647 + ] + ], + "generated_utilities": [ + 0.2191106230020523, + 0.5739269256591797, + 0.5799027681350708, + 0.2002597153186798, + 0.1847754418849945, + 0.5602454543113708, + 0.17806151509284973, + 0.5666472911834717 + ], + "group_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8363267779350281, + 2.0, + 2.0, + 0.16206762194633484, + 0.5390529036521912, + 0.08151453733444214, + 0.5261932611465454, + 0.5265869498252869, + 0.5459305047988892, + 0.18360589444637299, + 0.16689825057983398, + 0.15426510572433472, + 0.0028923004865646362, + 0.15516109764575958, + 0.0506085604429245, + 0.19578850269317627 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.0005862116813659668, + 0.18783891201019287, + -0.02734696865081787, + 0.09201061725616455, + 0.010171383619308472, + -0.4856022596359253, + 0.0, + 0.10349468141794205, + 0.4567214846611023, + -0.1893925815820694, + 0.12298360466957092, + 0.3369159996509552, + -1.6010875701904297, + 0.0, + 0.304210901260376, + -0.3049007058143616, + 0.3435661494731903, + 0.3158445656299591, + 0.2931552231311798, + -1.2709691524505615, + 0.0 + ], + [ + -0.726529061794281, + -0.8121610879898071, + -1.0273469686508179, + 1.0920106172561646, + 0.3926190733909607, + -1.4856022596359253, + 1.8979665040969849, + -0.029249273240566254, + 0.008005961775779724, + 0.013417234644293785, + -0.10860864818096161, + 0.43491366505622864, + -2.6010875701904297, + -1.493168592453003, + -0.01047949492931366, + -0.0018908085767179728, + 0.019280990585684776, + 0.20726272463798523, + 0.45750224590301514, + -2.2709691524505615, + -1.4645930528640747 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7959330081939697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.986337184906006, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9291861057281494 + ], + [ + -0.9019005298614502, + -0.5901641845703125, + -0.920559287071228, + 0.991828978061676, + 0.2407551407814026, + -1.2655587196350098, + 1.7652356624603271, + 0.03909382224082947, + 0.04142814502120018, + 0.12125641852617264, + -0.2822783887386322, + 0.4936577081680298, + -2.5754144191741943, + -1.5686157941818237, + -0.22672900557518005, + -0.08431098610162735, + 0.09805622696876526, + 0.2479260265827179, + 0.4450404644012451, + -2.1972453594207764, + -1.4420135021209717 + ], + [ + -0.8822115063667297, + -0.9738405346870422, + -0.7449414730072021, + 0.8784142732620239, + 0.4583987295627594, + -1.3989503383636475, + 1.8331178426742554, + -0.006558071821928024, + 0.30383312702178955, + 0.2117801308631897, + -0.2538969814777374, + 0.13478082418441772, + -2.584805727005005, + -1.325882911682129, + 0.23593956232070923, + -0.2618964612483978, + -0.30780693888664246, + 0.24083535373210907, + 0.430472195148468, + -1.946203589439392, + -1.5077860355377197 + ], + [ + -0.39491432905197144, + -0.8895903825759888, + -0.9160035848617554, + 0.7220491170883179, + 0.36837494373321533, + -1.4429160356521606, + 1.4078493118286133, + 0.03969770669937134, + 0.007204015273600817, + 0.056278251111507416, + -0.4780465364456177, + 0.9121701717376709, + -2.512721538543701, + -1.6225675344467163, + -0.42074650526046753, + 0.15076839923858643, + -0.24544812738895416, + 0.08152849972248077, + 0.1987975835800171, + -2.712414264678955, + -1.9573136568069458 + ], + [ + -0.6879937648773193, + -0.8859440088272095, + -1.1152405738830566, + 0.8750674724578857, + -0.05955067276954651, + -1.5194538831710815, + 1.876329779624939, + -0.2068542242050171, + -0.5498799085617065, + 0.46419233083724976, + 0.42569708824157715, + -0.02127784490585327, + -2.5905418395996094, + -1.411050796508789, + -0.21327723562717438, + 0.20716798305511475, + 0.20993508398532867, + 0.33476364612579346, + -0.09872657060623169, + -2.5958731174468994, + -1.6374839544296265 + ], + [ + -0.5723236799240112, + -0.28514552116394043, + -0.6157487630844116, + 0.6151646375656128, + 0.32639649510383606, + -1.7217803001403809, + 2.5557403564453125, + 0.532052218914032, + 0.5977330207824707, + -0.5707970261573792, + -0.21707558631896973, + 0.8829224109649658, + -3.1287553310394287, + -1.0988328456878662, + 0.38229066133499146, + 0.0032835237216204405, + -0.3569248616695404, + 0.0031215548515319824, + 1.1060712337493896, + -2.3536932468414307, + -1.383341670036316 + ], + [ + -0.7901747822761536, + -0.8977189660072327, + -1.4548863172531128, + 0.5700533986091614, + -0.13018959760665894, + -0.8057670593261719, + 2.4019269943237305, + 0.5368256568908691, + 0.5227565765380859, + 0.28199371695518494, + -0.1637011021375656, + -0.2898564040660858, + -2.9142980575561523, + -1.8492696285247803, + 0.7177971601486206, + 0.2360217124223709, + -0.13862104713916779, + 0.9262527227401733, + 0.39495596289634705, + -2.470649480819702, + -1.3260751962661743 + ], + [ + -0.6053685545921326, + -0.3819047510623932, + -0.258634090423584, + 0.19338607788085938, + 0.19874510169029236, + -1.3407407999038696, + 2.5238022804260254, + -0.6922282576560974, + 0.8137098550796509, + 0.39421388506889343, + 0.09789760410785675, + 0.12572914361953735, + -2.518153667449951, + -1.6892600059509277, + 0.503250002861023, + 0.88390052318573, + -0.7520112991333008, + -0.3356115520000458, + 0.15291047096252441, + -2.5788943767547607, + -0.9403089284896851 + ], + [ + -0.464814156293869, + -0.6034223437309265, + -1.6505372524261475, + 0.6883972883224487, + -0.3758969306945801, + -1.785051941871643, + 2.605452060699463, + 0.550828218460083, + 0.4742209315299988, + -0.19333000481128693, + -1.002321481704712, + 0.48782211542129517, + -1.8015108108520508, + -0.8254865407943726, + -0.7333722710609436, + -0.4948355555534363, + 0.6016485095024109, + 0.5632485151290894, + -0.24367755651474, + -2.201673984527588, + -0.5806979537010193 + ], + [ + -1.4628195762634277, + -0.4433446228504181, + -1.9926375150680542, + 0.33977705240249634, + 0.03379333019256592, + -1.6042804718017578, + 2.221484422683716, + 0.7110810279846191, + -0.41500210762023926, + 0.08095695078372955, + -0.23683495819568634, + -0.43207624554634094, + -3.080759286880493, + -0.9071659445762634, + 0.47082334756851196, + -0.9117310047149658, + -0.06435621529817581, + -0.6813669204711914, + -0.2741369605064392, + -2.3763599395751953, + -1.000203013420105 + ], + [ + -1.382751226425171, + -1.7026821374893188, + -0.11846715211868286, + 1.5398613214492798, + -0.5885447859764099, + -1.8825289011001587, + 1.5456308126449585, + -0.320659339427948, + -0.964028000831604, + -0.1812019646167755, + 0.14226128160953522, + 0.4527861177921295, + -2.714468002319336, + -1.9451607465744019, + 0.195057213306427, + 0.16905252635478973, + -0.9247990250587463, + 0.9112424850463867, + 0.79449862241745, + -2.714893341064453, + -1.6473028659820557 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.02016383409500122, + 0.02509528398513794, + -0.02734696865081787, + 0.09201061725616455, + -0.04331648349761963, + -0.4856022596359253, + 0.0, + -0.022040890529751778, + -0.003832761198282242, + -0.01408458687365055, + 0.06165061891078949, + 0.02554956078529358, + -1.6010875701904297, + 0.0, + 0.009550465270876884, + 0.03566932678222656, + 0.013732164166867733, + 0.017749279737472534, + 0.0699148178100586, + -1.2709691524505615, + 0.0 + ], + [ + 0.023195505142211914, + 0.029890060424804688, + -0.02734696865081787, + 0.09201061725616455, + 0.06108534336090088, + -0.4856022596359253, + 0.0, + -0.034551952034235, + -0.15152660012245178, + -0.21811580657958984, + -0.063543401658535, + 0.23667243123054504, + -1.6010875701904297, + 0.0, + -0.1614403873682022, + -0.17222382128238678, + -0.09706380218267441, + 0.3660350441932678, + 0.13013160228729248, + -1.2709691524505615, + 0.0 + ] + ], + "predicted_scores": [ + 6.202249526977539, + 6.037703990936279, + 5.914402008056641, + 5.794589042663574, + 5.336828708648682, + 5.235112190246582, + 5.21646785736084, + 3.7788166999816895 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "957101489/train0", + "selected_index": 0, + "state_hash": "e8473553a2b17f5adf09e08e7271b5502d53a51225c17abac3d9f78906cf8c46", + "stored_base_utility": 0.8363267779350281, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8431589603424072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8431589603424072 + }, + "base_progress": 0.8431589603424072, + "base_success": false, + "base_utility": 0.8431589603424072, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1996680349111557, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1996680349111557 + }, + { + "contact_quality": null, + "progress": 0.22532063722610474, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22532063722610474 + }, + { + "contact_quality": null, + "progress": 0.2148892730474472, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2148892730474472 + }, + { + "contact_quality": null, + "progress": 0.21345362067222595, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21345362067222595 + }, + { + "contact_quality": null, + "progress": 0.20482183992862701, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20482183992862701 + }, + { + "contact_quality": null, + "progress": 0.18579784035682678, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18579784035682678 + }, + { + "contact_quality": null, + "progress": 0.20295663177967072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20295663177967072 + }, + { + "contact_quality": null, + "progress": 0.22642898559570312, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22642898559570312 + } + ], + "candidate_progress": [ + 0.1996680349111557, + 0.22532063722610474, + 0.2148892730474472, + 0.21345362067222595, + 0.20482183992862701, + 0.18579784035682678, + 0.20295663177967072, + 0.22642898559570312 + ], + "candidate_source_chart_ids": [ + "ms-9d12ad4ebdf37419994c", + "ms-9228f8295b9891cf011c", + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-9d12ad4ebdf37419994c", + "ms-5bddb39f792f20e5e6d4", + "ms-ce9505086ea115f86649", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-9a14552b7db695dee92c", + "generated_tangents": [ + [ + 0.2624432146549225, + -0.2312541902065277, + -0.01603803038597107, + 0.716766357421875, + -0.08071556687355042, + -1.8082318305969238, + -0.09603911638259888, + 0.05549733340740204, + -0.05502775311470032, + -0.0594683513045311, + 0.20835864543914795, + -0.016604065895080566, + -1.7020565271377563, + -0.13113079965114594, + 0.038984961807727814, + 0.049232371151447296, + 0.011512726545333862, + 0.029467463493347168, + 0.15997520089149475, + -1.5683104991912842, + -0.21798065304756165 + ], + [ + 0.12213128805160522, + -0.21399933099746704, + -0.08174243569374084, + 0.6623992919921875, + 0.045203059911727905, + -1.9158293008804321, + -0.1745433807373047, + 0.04145631194114685, + -0.002330976538360119, + -0.0541447177529335, + 0.22507616877555847, + -0.08192059397697449, + -1.7410130500793457, + -0.28673940896987915, + 0.08530785143375397, + 0.0473250113427639, + -0.011274099349975586, + 0.001980200409889221, + -0.016880542039871216, + -1.586552619934082, + -0.27615809440612793 + ], + [ + -0.02724890410900116, + -0.1699056327342987, + -0.24820461869239807, + 0.4899890720844269, + -0.13699546456336975, + -1.2355618476867676, + -0.2498578429222107, + 0.06585212051868439, + -0.08921295404434204, + 0.006838686764240265, + 0.2097664773464203, + -0.07441318035125732, + -1.7632848024368286, + -0.23597757518291473, + 0.06457287818193436, + 0.047926463186740875, + -0.003697410225868225, + -0.00036013126373291016, + 0.01808789372444153, + -1.5946176052093506, + -0.18573492765426636 + ], + [ + -0.08218064904212952, + -0.1922997385263443, + -0.46739181876182556, + 0.7826329469680786, + -0.13812682032585144, + -0.9043611288070679, + -0.5052298903465271, + 0.08566302806138992, + -0.05732395499944687, + -0.00764002650976181, + 0.18206840753555298, + 0.30042794346809387, + -1.774909496307373, + -0.04345497488975525, + 0.23592454195022583, + 0.09190230071544647, + 0.1109326034784317, + -0.06977240741252899, + 0.32663580775260925, + -1.7537684440612793, + -0.058904170989990234 + ], + [ + 0.18238164484500885, + -0.23363518714904785, + -0.058075159788131714, + 0.6516423225402832, + 0.05666317045688629, + -2.0269320011138916, + -0.1774006485939026, + 0.012360140681266785, + -0.031836606562137604, + -0.08709903806447983, + 0.277927041053772, + -0.093505859375, + -1.7426353693008423, + -0.20086981356143951, + 0.10358453541994095, + 0.04527905955910683, + -0.07806704938411713, + -0.02976386249065399, + -0.0159989595413208, + -1.6452622413635254, + -0.277252197265625 + ], + [ + -0.15696273744106293, + -0.16912096738815308, + -0.3014816343784332, + 0.5433331727981567, + -0.2681531310081482, + -0.8654876947402954, + -0.15608221292495728, + 0.06168109178543091, + -0.14395484328269958, + 0.05676373839378357, + 0.12987977266311646, + 0.142566978931427, + -1.6192100048065186, + -0.20651580393314362, + 0.12550833821296692, + 0.07648462057113647, + 0.05763666331768036, + 0.0032028555870056152, + 0.030257001519203186, + -1.6643896102905273, + -0.10184425115585327 + ], + [ + -0.03403349220752716, + -0.11452681571245193, + -0.13719698786735535, + 0.7563047409057617, + -0.036255210638046265, + -1.2274441719055176, + -0.2351142168045044, + -0.03501904010772705, + -0.0992419496178627, + 0.02351989597082138, + 0.035086922347545624, + -0.05853426456451416, + -2.0476763248443604, + 0.052276626229286194, + 0.043744999915361404, + 0.03687640279531479, + -0.0037261992692947388, + -0.014840468764305115, + -0.06056523323059082, + -1.8509268760681152, + -0.05957487225532532 + ], + [ + -0.12679947912693024, + -0.23132160305976868, + -0.5529159307479858, + 0.6806193590164185, + -0.028687238693237305, + -0.8835886716842651, + -0.6139217019081116, + 0.14418193697929382, + -0.1342320442199707, + 0.0951058641076088, + 0.0733126774430275, + 0.4877215325832367, + -1.648722767829895, + 0.0299447700381279, + 0.11717789620161057, + 0.023823797702789307, + -0.003589138388633728, + -0.08864752948284149, + 0.26730063557624817, + -1.8048813343048096, + 0.03818659484386444 + ] + ], + "generated_utilities": [ + 0.1996680349111557, + 0.22532063722610474, + 0.2148892730474472, + 0.21345362067222595, + 0.20482183992862701, + 0.18579784035682678, + 0.20295663177967072, + 0.22642898559570312 + ], + "group_id": "ms-9a14552b7db695dee92c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.838127613067627, + 2.0, + 0.15671375393867493, + 0.17482513189315796, + 0.5684618353843689, + 0.19147612154483795, + 0.5643423795700073, + 0.1653006672859192, + 0.11670731008052826, + 0.14578887820243835, + 0.16058449447155, + 0.10923929512500763, + 0.13408902287483215, + 0.06845971941947937, + 0.02954423427581787, + 0.08425161242485046 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.22581616044044495, + -0.32600927352905273, + -0.054001130163669586, + 1.2553811073303223, + -0.1383839249610901, + -1.9753923416137695, + 0.0, + 0.09536236524581909, + 0.017635054886341095, + -0.06946105509996414, + 0.06630897521972656, + -0.10150641947984695, + -1.9112331867218018, + 0.0, + -0.23092225193977356, + 0.04748937487602234, + 0.3013715445995331, + -0.22072738409042358, + 0.0733548104763031, + -1.6514825820922852, + 0.0 + ], + [ + 0.22913068532943726, + -0.32600927352905273, + 0.24692901968955994, + 1.2553811073303223, + 0.3704858124256134, + -1.9753923416137695, + 0.0, + 0.036500319838523865, + 0.12820330262184143, + -0.09674224257469177, + 0.2668613791465759, + -0.14372897148132324, + -1.9112331867218018, + 0.0, + -0.27894535660743713, + -0.3322452902793884, + 0.16300754249095917, + -0.24123603105545044, + -0.2359180599451065, + -1.6514825820922852, + 0.0 + ], + [ + 0.0198141410946846, + -1.3260092735290527, + 0.011403528973460197, + 2.2553811073303223, + 0.5258303284645081, + -2.9753923416137695, + 1.8956809043884277, + -0.028990529477596283, + 0.007858656346797943, + 0.03128144145011902, + 0.5457771420478821, + -0.14793801307678223, + -2.9112331867218018, + -1.5743238925933838, + -0.011861175298690796, + -0.002163209253922105, + 0.015663353726267815, + 0.7587639689445496, + -0.07801014184951782, + -2.651482582092285, + -1.5696079730987549 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7913618087768555, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1486477851867676, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1392159461975098 + ], + [ + 0.13539990782737732, + -1.2862521409988403, + 0.28441768884658813, + 2.3658432960510254, + 0.4678441286087036, + -2.8791494369506836, + 1.8408029079437256, + -0.11893181502819061, + -0.1259741485118866, + -0.20943574607372284, + 0.4765416979789734, + -0.40862777829170227, + -2.8372273445129395, + -1.4697245359420776, + -0.08606886118650436, + -0.06554404646158218, + -0.27424973249435425, + 0.907739520072937, + -0.10130079835653305, + -2.6951611042022705, + -1.807652235031128 + ], + [ + -0.15379676222801208, + -1.7091000080108643, + 0.31153741478919983, + 2.2179949283599854, + 0.859725832939148, + -2.6038026809692383, + 1.98358952999115, + 0.22934654355049133, + -0.16202330589294434, + 0.02916553243994713, + 0.7702362537384033, + -0.0907563641667366, + -2.6495683193206787, + -1.394291877746582, + 0.2534489035606384, + -0.07077484577894211, + 0.10660970956087112, + 0.7866172194480896, + -0.41323983669281006, + -2.4847500324249268, + -1.5881152153015137 + ], + [ + 0.14051103591918945, + -0.8684865236282349, + 0.45214012265205383, + 2.499347686767578, + 0.310441792011261, + -2.875823497772217, + 1.4898443222045898, + 0.3396439850330353, + -0.033532168716192245, + 0.20189781486988068, + 0.3506202697753906, + 0.04831346869468689, + -2.4730942249298096, + -1.4369138479232788, + -0.4840375483036041, + -0.17814871668815613, + -0.1863224357366562, + 0.47034892439842224, + -0.5315045714378357, + -3.019744396209717, + -1.2571722269058228 + ], + [ + 0.5503659844398499, + -1.517655372619629, + -0.20272362232208252, + 2.5588791370391846, + 0.7160627841949463, + -2.551017999649048, + 2.423107385635376, + 0.18953359127044678, + -0.4268995523452759, + 0.014377228915691376, + 0.969605028629303, + -0.2440275251865387, + -2.385143756866455, + -1.4786417484283447, + 0.44389280676841736, + -0.11543146520853043, + -0.5315021872520447, + 1.2627880573272705, + 0.24797001481056213, + -3.17976713180542, + -1.8759949207305908 + ], + [ + -0.07551594823598862, + -1.6101253032684326, + 0.3926098048686981, + 1.726672649383545, + 1.0158472061157227, + -3.367068290710449, + 1.4499058723449707, + -0.5616514086723328, + 0.002434568479657173, + -0.5033466815948486, + -0.12249338626861572, + -0.07335222512483597, + -3.5959439277648926, + -1.0662813186645508, + -0.034436360001564026, + 0.512275218963623, + -0.04822078347206116, + 0.6117934584617615, + -0.745093822479248, + -2.2700355052948, + -2.090803623199463 + ], + [ + 0.6751457452774048, + -0.7857800126075745, + 0.11759950965642929, + 2.7748122215270996, + 1.3096001148223877, + -2.3352341651916504, + 1.4677345752716064, + 0.7219796180725098, + 0.21788907051086426, + -0.3245394825935364, + 0.08372032642364502, + -0.6393120288848877, + -2.8002729415893555, + -2.157836437225342, + 0.11339983344078064, + -0.2834126651287079, + -0.3824191987514496, + 1.130112648010254, + -0.8028184175491333, + -2.3846678733825684, + -1.8259286880493164 + ], + [ + -0.01608770713210106, + -2.1245148181915283, + -0.175740584731102, + 2.754368782043457, + 1.0032098293304443, + -3.7700157165527344, + 1.3539870977401733, + -0.7906507849693298, + -0.8311541080474854, + 0.8180835247039795, + 0.10450628399848938, + 0.01135176420211792, + -3.396463394165039, + -0.6868918538093567, + 0.22346650063991547, + 0.693323016166687, + -0.47104692459106445, + 0.8188157081604004, + -0.6246469020843506, + -2.4006571769714355, + -2.431541919708252 + ], + [ + -0.7931793928146362, + -1.7645766735076904, + 0.23125068843364716, + 2.816321849822998, + 1.1080517768859863, + -3.8820090293884277, + 1.8420056104660034, + 0.4984684884548187, + -0.9149360656738281, + -0.028868291527032852, + 0.14621934294700623, + 0.546608567237854, + -2.7883894443511963, + -1.3650472164154053, + -0.055590901523828506, + 0.41940492391586304, + -0.7235082983970642, + 1.1734092235565186, + -0.6126058101654053, + -3.217271566390991, + -2.512387275695801 + ], + [ + -0.5335642695426941, + -2.139784336090088, + 0.9540140628814697, + 1.7781559228897095, + 1.2117886543273926, + -3.71681809425354, + 2.461766242980957, + 0.4546531140804291, + 0.09786976873874664, + -0.8281612396240234, + -0.19971781969070435, + -1.1042157411575317, + -3.047807216644287, + -1.0697336196899414, + -0.22256404161453247, + -0.3800235986709595, + 0.3603542447090149, + 0.7543611526489258, + -0.23184219002723694, + -3.5156431198120117, + -2.1170644760131836 + ], + [ + 0.27338743209838867, + -0.9654052257537842, + -0.2700650095939636, + 2.6957814693450928, + 0.0643424391746521, + -3.094449281692505, + 0.8981494307518005, + 0.19603312015533447, + 0.5969407558441162, + -0.45200687646865845, + 1.466981291770935, + -0.45456382632255554, + -2.997983455657959, + -1.1903986930847168, + 0.5722758769989014, + -0.9932370781898499, + 0.28074830770492554, + 0.6244409084320068, + 0.6411240696907043, + -3.2179243564605713, + -1.905179738998413 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.025789255276322365, + -0.32600927352905273, + -0.02888326905667782, + 1.2553811073303223, + 0.013092517852783203, + -1.9753923416137695, + 0.0, + 0.04008308798074722, + -0.05201057717204094, + -0.008613351732492447, + -0.06225985288619995, + -0.0397564172744751, + -1.9112331867218018, + 0.0, + -0.02836347185075283, + -0.0055325529538095, + 0.053034864366054535, + 0.009740114212036133, + -0.006228044629096985, + -1.6514825820922852, + 0.0 + ] + ], + "predicted_scores": [ + 4.801419258117676, + 4.317750453948975, + 4.280590534210205, + 4.22405481338501, + 3.9708096981048584, + 3.954923629760742, + 3.847562074661255, + 2.8120131492614746 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264/train0", + "selected_index": 0, + "state_hash": "49d38eabb7404e0028c75dd98e1239274b5dc2f04b9db8653001e44369ca8f3a", + "stored_base_utility": 0.838127613067627, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8326728343963623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8326728343963623 + }, + "base_progress": 0.8326728343963623, + "base_success": false, + "base_utility": 0.8326728343963623, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.14438587427139282, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14438587427139282 + }, + { + "contact_quality": null, + "progress": 0.21023166179656982, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21023166179656982 + }, + { + "contact_quality": null, + "progress": 0.19636064767837524, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19636064767837524 + }, + { + "contact_quality": null, + "progress": 0.2200891673564911, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2200891673564911 + }, + { + "contact_quality": null, + "progress": 0.21172496676445007, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21172496676445007 + }, + { + "contact_quality": null, + "progress": 0.17683273553848267, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17683273553848267 + }, + { + "contact_quality": null, + "progress": 0.2003195583820343, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2003195583820343 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.14438587427139282, + 0.21023166179656982, + 0.19636064767837524, + 0.2200891673564911, + 0.21172496676445007, + 0.17683273553848267, + 0.2003195583820343, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-5dfbb16e173e1a3e2bd6", + "ms-c80e71e55d51d8595499" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-9f5d57fe2153fd267774", + "generated_tangents": [ + [ + 0.2602660059928894, + -0.03273323178291321, + 0.07987892627716064, + 0.5466212034225464, + -0.20089450478553772, + -1.8387469053268433, + -0.059253692626953125, + -0.018537238240242004, + -0.1549907624721527, + -0.029300756752490997, + 0.41915199160575867, + 0.014187842607498169, + -1.1461056470870972, + -0.13178829848766327, + 0.06668057292699814, + -0.007875459268689156, + 0.03469371795654297, + 0.0015375465154647827, + 0.1495690643787384, + -1.0548312664031982, + -0.08297193050384521 + ], + [ + 0.09643995761871338, + -0.01801915094256401, + -0.08633384108543396, + 0.3083895742893219, + -0.22981655597686768, + -1.6308103799819946, + -0.17506033182144165, + 0.1317121833562851, + -0.1870492845773697, + -0.08348088711500168, + 0.4701962172985077, + -0.0415484756231308, + -1.1528005599975586, + -0.17862139642238617, + 0.09915521740913391, + 0.003914322704076767, + 0.1445608139038086, + -0.07952277362346649, + -0.09453406929969788, + -1.1416213512420654, + -0.1206238865852356 + ], + [ + 0.07473111152648926, + 0.019286468625068665, + -0.06443783640861511, + 0.3161158263683319, + -0.17376276850700378, + -1.7811124324798584, + -0.18677401542663574, + 0.06483151018619537, + -0.21161282062530518, + -0.0615890696644783, + 0.5126396417617798, + -0.005297720432281494, + -1.2125425338745117, + -0.16410191357135773, + 0.05861067771911621, + 0.011137725785374641, + 0.1303870677947998, + 0.02194388210773468, + -0.011279016733169556, + -1.0515601634979248, + -0.0638728141784668 + ], + [ + 0.09147374331951141, + -0.01692088134586811, + -0.07399740815162659, + 0.431353896856308, + -0.18794980645179749, + -1.7875381708145142, + -0.19772344827651978, + 0.035040900111198425, + -0.20439089834690094, + -0.12228672951459885, + 0.4379032850265503, + -0.014443904161453247, + -1.1933002471923828, + -0.1691025346517563, + 0.07060874998569489, + 0.015172602608799934, + 0.16315990686416626, + -0.08178214728832245, + -0.09501567482948303, + -1.133695363998413, + -0.08170938491821289 + ], + [ + 0.06649884581565857, + -0.021549489349126816, + -0.1452367603778839, + 0.49567893147468567, + -0.09795928001403809, + -1.7211921215057373, + -0.14746838808059692, + 0.02174447476863861, + -0.15541866421699524, + -0.12002352625131607, + 0.5594443678855896, + 0.09868992865085602, + -1.1431653499603271, + -0.1553684026002884, + 0.03961511701345444, + 0.009050954133272171, + 0.09205007553100586, + -0.07196493446826935, + 0.009062260389328003, + -1.06088125705719, + -0.035056307911872864 + ], + [ + 0.14043836295604706, + 0.01007155328989029, + -0.0518205463886261, + 0.4260527789592743, + -0.07622635364532471, + -1.8494645357131958, + -0.15775960683822632, + 0.014868944883346558, + -0.1826332062482834, + -0.05402038246393204, + 0.5112128257751465, + 0.04463024437427521, + -1.2181001901626587, + -0.13149280846118927, + -0.0059579648077487946, + 0.03604526072740555, + 0.031237870454788208, + 0.05012226104736328, + -0.027166098356246948, + -1.0655229091644287, + -0.06652700901031494 + ], + [ + 0.09830405563116074, + -0.07416731864213943, + -0.19190940260887146, + 0.050100743770599365, + -0.1889578104019165, + -2.2374842166900635, + -0.3995264172554016, + 0.20491744577884674, + -0.14394374191761017, + -0.1517687439918518, + 0.7002518773078918, + -0.04169680178165436, + -1.1539170742034912, + -0.3284742832183838, + -0.026428338140249252, + 0.12713925540447235, + 0.17761936783790588, + 0.0869329571723938, + -0.015213251113891602, + -1.076972484588623, + -0.31907981634140015 + ], + [ + 0.1832084208726883, + 0.04683810472488403, + -0.03905823826789856, + 0.5987484455108643, + -0.1558729112148285, + -1.8848572969436646, + -0.8485514521598816, + -0.0488336980342865, + -0.14652352035045624, + 0.34101128578186035, + 0.5387577414512634, + -0.03178811073303223, + -1.0788077116012573, + -0.6728173494338989, + 0.27982887625694275, + 0.18009895086288452, + 0.21637894213199615, + 0.3476164937019348, + 0.16768789291381836, + -1.624605417251587, + 0.2797800898551941 + ] + ], + "generated_utilities": [ + 0.14438587427139282, + 0.21023166179656982, + 0.19636064767837524, + 0.2200891673564911, + 0.21172496676445007, + 0.17683273553848267, + 0.2003195583820343, + 2.0 + ], + "group_id": "ms-9f5d57fe2153fd267774", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8544998168945312, + 2.0, + 0.14427810907363892, + 0.1566883623600006, + 0.7847082018852234, + 0.6161996126174927, + 0.5900160074234009, + 0.12891459465026855, + 0.5811972618103027, + 0.12705880403518677, + 0.12002001702785492, + 0.07508957386016846, + 0.1335657238960266, + 2.0, + 0.5254277586936951, + 0.1602712869644165 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.07957560569047928, + 0.1778574287891388, + -0.06101015955209732, + 0.08770006895065308, + -0.13665570318698883, + -1.1242008209228516, + 0.0, + 0.20265339314937592, + 0.03117954358458519, + -0.08283126354217529, + 0.051615357398986816, + 0.006725620478391647, + -0.7304134368896484, + 0.0, + -0.03756266087293625, + 0.051428791135549545, + 0.13959813117980957, + 0.040047526359558105, + -0.04084143042564392, + -0.31500017642974854, + 0.0 + ], + [ + 0.31541678309440613, + -0.16606420278549194, + 0.09380461275577545, + 0.031209349632263184, + -0.10083025693893433, + -1.1242008209228516, + 0.0, + 0.09663133323192596, + -0.095388263463974, + 0.4012776017189026, + 0.051615357398986816, + 0.167547807097435, + -0.7304134368896484, + 0.0, + -0.6302167177200317, + 0.05524101108312607, + -0.14198890328407288, + 0.040047526359558105, + 0.5201720595359802, + -0.31500017642974854, + 0.0 + ], + [ + 0.004154659807682037, + -0.0034282731357961893, + 0.005892834160476923, + 1.0312093496322632, + -0.15828494727611542, + -2.1242008209228516, + -1.4059314727783203, + 0.002088826149702072, + 0.0016306412871927023, + 0.005124530289322138, + 1.0516153573989868, + -0.05936579033732414, + -1.7304134368896484, + -1.2081255912780762, + 0.010161396116018295, + 0.0017865521367639303, + -0.006222852971404791, + 1.040047526359558, + 0.07346899807453156, + -1.3150001764297485, + -0.9752891659736633 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.8118629455566406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.4162511825561523, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9505783319473267 + ], + [ + 0.05903958901762962, + -0.05713164061307907, + 0.27611011266708374, + 0.9773679971694946, + -0.2122824341058731, + -2.198127031326294, + -1.1234419345855713, + 0.22609303891658783, + 0.14432841539382935, + 0.13455358147621155, + 0.8475942611694336, + 0.19007176160812378, + -1.4536203145980835, + -1.208316445350647, + -0.24895474314689636, + 0.08610261976718903, + 0.16216596961021423, + 1.2356702089309692, + 0.07108174264431, + -1.4601069688796997, + -1.0005295276641846 + ], + [ + -0.2862337827682495, + -0.34814661741256714, + -0.3317800462245941, + 1.383265495300293, + -0.3485583961009979, + -1.7799506187438965, + -1.6264256238937378, + -0.24356596171855927, + -0.18337340652942657, + 0.3351142108440399, + 1.36115562915802, + -0.4404119849205017, + -1.4858057498931885, + -0.9229021668434143, + 0.3078569769859314, + -0.18743832409381866, + -0.3414495885372162, + 1.0045359134674072, + -0.1438894122838974, + -1.5133963823318481, + -0.7331042885780334 + ], + [ + 0.14600041508674622, + -0.10555070638656616, + 0.06090353801846504, + 1.1715545654296875, + 0.14829085767269135, + -1.8377182483673096, + -1.2019046545028687, + -0.33082208037376404, + 0.27619102597236633, + 0.3936208188533783, + 1.4077775478363037, + -0.2165551781654358, + -2.094696283340454, + -0.7843748331069946, + -0.13239514827728271, + 0.208020880818367, + 0.17500364780426025, + 0.5608562231063843, + 0.5641188025474548, + -1.6192113161087036, + -1.1969176530838013 + ], + [ + -0.03659747168421745, + -0.2400680035352707, + 0.49718841910362244, + 1.0422266721725464, + 0.13088186085224152, + -2.068427324295044, + -1.1215412616729736, + 0.06697356700897217, + 0.19986751675605774, + -0.5827334523200989, + 0.9792195558547974, + -0.5810520648956299, + -1.7176907062530518, + -0.6760571002960205, + -0.08971267938613892, + 0.36716267466545105, + 0.4829793870449066, + 0.7045803070068359, + -0.3341261148452759, + -1.3750282526016235, + -1.086517095565796 + ], + [ + 0.09393255412578583, + 0.6277099251747131, + 0.653377115726471, + 0.5602235198020935, + -0.8483365178108215, + -1.7264833450317383, + -1.698084831237793, + -0.3303340971469879, + 0.43070027232170105, + -0.13159427046775818, + 1.0274523496627808, + 0.4760425388813019, + -2.1360630989074707, + -0.6506720185279846, + 0.3130929470062256, + 0.6801184415817261, + 0.6575731039047241, + 0.9053056240081787, + 0.18805859982967377, + -1.3225764036178589, + -1.0653810501098633 + ], + [ + 0.536294162273407, + -0.026106586679816246, + 0.11593607813119888, + 0.2775042653083801, + 0.1801595836877823, + -1.590940237045288, + -0.6846873164176941, + 0.5080465078353882, + -0.7678324580192566, + 0.24115782976150513, + 0.2675415277481079, + -0.3879689872264862, + -2.1869571208953857, + -0.7176487445831299, + -0.15474224090576172, + -0.5326727032661438, + -0.12406099587678909, + 0.411626398563385, + 0.3519209027290344, + -1.6924495697021484, + -1.4648362398147583 + ], + [ + 0.2688097357749939, + -0.7928301095962524, + -0.39608004689216614, + 1.3961963653564453, + -0.026196151971817017, + -2.263073682785034, + -2.044499635696411, + 0.055598195642232895, + -0.485431969165802, + -0.058985911309719086, + 1.781836986541748, + 0.7104123830795288, + -2.0781984329223633, + -0.6968748569488525, + -0.1437186300754547, + 0.8155919313430786, + 0.5188283920288086, + 0.23954623937606812, + -0.6596511602401733, + -1.3867920637130737, + -0.7448391318321228 + ], + [ + 0.11259422451257706, + 0.43877506256103516, + -0.585435152053833, + 0.12099790573120117, + 0.5070493817329407, + -2.101219654083252, + -1.3759371042251587, + -0.7516231536865234, + 0.7018498778343201, + 0.7899036407470703, + 0.9772143959999084, + -0.7032701373100281, + -1.4699928760528564, + -1.4471495151519775, + 0.4750489890575409, + 0.8803708553314209, + -0.25032687187194824, + 1.3429648876190186, + 0.7985087037086487, + -0.5786607265472412, + -1.563289761543274 + ], + [ + 0.4965847432613373, + -0.8452511429786682, + 0.21300016343593597, + 0.28797537088394165, + -1.0059970617294312, + -3.040114402770996, + -1.1050269603729248, + 0.7826927304267883, + 0.8645586371421814, + -0.9447845816612244, + 1.0121192932128906, + 0.5123239755630493, + -1.5470938682556152, + -0.6170141100883484, + 0.2538997530937195, + 0.912939727306366, + -0.8613666892051697, + 0.9517492055892944, + 0.7185115218162537, + -2.1534924507141113, + -0.6818757057189941 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025899745523929596, + -0.03433045744895935, + -0.044045113027095795, + 0.031209349632263184, + -0.014532476663589478, + -1.1242008209228516, + 0.0, + 0.10768167674541473, + -0.020356399938464165, + 0.09216367453336716, + 0.051615357398986816, + -0.011370409280061722, + -0.7304134368896484, + 0.0, + 0.04558054730296135, + 0.038496289402246475, + 0.02504396252334118, + 0.040047526359558105, + 0.01217309758067131, + -0.31500017642974854, + 0.0 + ], + [ + -0.5198070406913757, + -0.8798871040344238, + 0.04596509411931038, + 1.55931556224823, + 0.31027060747146606, + -1.5283797979354858, + -1.6053413152694702, + -0.2754230797290802, + -0.012456716038286686, + 0.7998818159103394, + 1.5184144973754883, + 0.05059843137860298, + -1.85283362865448, + -0.9533926248550415, + 0.9978112578392029, + 0.8525571227073669, + -0.9611027836799622, + 0.32179969549179077, + -0.448661744594574, + -1.9452965259552002, + -0.7132223844528198 + ] + ], + "predicted_scores": [ + 3.1715168952941895, + 2.7965338230133057, + 2.589705467224121, + 2.4722023010253906, + 2.296574831008911, + 2.2875022888183594, + 1.6957511901855469, + -1.3588550090789795 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1607773860/train0", + "selected_index": 0, + "state_hash": "fdc7379ab4491c9f65860cadd3b2f3c9f2335f26aae91fb1c06d3a4edf858d6a", + "stored_base_utility": 0.8544998168945312, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8272150158882141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8272150158882141 + }, + "base_progress": 0.8272150158882141, + "base_success": false, + "base_utility": 0.8272150158882141, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.179222971200943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.179222971200943 + }, + { + "contact_quality": null, + "progress": 0.22463779151439667, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22463779151439667 + }, + { + "contact_quality": null, + "progress": 0.8147144317626953, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8147144317626953 + }, + { + "contact_quality": null, + "progress": 0.19045117497444153, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19045117497444153 + }, + { + "contact_quality": null, + "progress": 0.18127290904521942, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18127290904521942 + }, + { + "contact_quality": null, + "progress": 0.18905499577522278, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18905499577522278 + }, + { + "contact_quality": null, + "progress": 0.19813549518585205, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19813549518585205 + }, + { + "contact_quality": null, + "progress": 0.16648955643177032, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16648955643177032 + } + ], + "candidate_progress": [ + 0.179222971200943, + 0.22463779151439667, + 0.8147144317626953, + 0.19045117497444153, + 0.18127290904521942, + 0.18905499577522278, + 0.19813549518585205, + 0.16648955643177032 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-20f3cfcf183c69853b1e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b0fee26e1db967f90b81", + "generated_tangents": [ + [ + 0.22818443179130554, + -0.03238482400774956, + 0.18031135201454163, + -0.1053600162267685, + 0.07715210318565369, + -2.2039506435394287, + 0.03612273931503296, + 0.0057865530252456665, + -0.013363216072320938, + -0.04724108427762985, + 0.07863190770149231, + 0.00023005902767181396, + -1.5760533809661865, + -0.1736038774251938, + 0.04172223061323166, + 0.0436229482293129, + 0.0007335245609283447, + -0.022069111466407776, + 0.2518843412399292, + -1.3511199951171875, + -0.3136128783226013 + ], + [ + 0.06199656426906586, + -0.003600809723138809, + -0.03485104441642761, + -0.19633445143699646, + 0.04201178252696991, + -2.1112866401672363, + -0.12216496467590332, + -0.014988809823989868, + -0.08607646077871323, + -0.06515321880578995, + 0.08061842620372772, + 0.008123427629470825, + -1.598246693611145, + -0.21602292358875275, + 0.04981159791350365, + 0.057903606444597244, + 0.1086626797914505, + -0.11851386725902557, + 0.005406826734542847, + -1.420619249343872, + -0.20315900444984436 + ], + [ + 0.03715181350708008, + 0.0040266066789627075, + -0.055608779191970825, + -0.17337001860141754, + 0.104716956615448, + -1.9267939329147339, + -0.06935179233551025, + 0.01847061514854431, + -0.03166525810956955, + -0.03275337070226669, + 0.22659337520599365, + 0.13304156064987183, + -1.5220907926559448, + -0.21739737689495087, + 0.09191690385341644, + 0.08474767208099365, + 0.027780652046203613, + -0.054191216826438904, + 0.17941749095916748, + -1.3500516414642334, + -0.213168203830719 + ], + [ + 0.1229734867811203, + 0.010669179260730743, + 0.0094451904296875, + -0.22407174110412598, + 0.003824472427368164, + -2.3158624172210693, + -0.1022217869758606, + 0.012917712330818176, + -0.08186297118663788, + -0.03311104327440262, + 0.15838569402694702, + -0.09856075048446655, + -1.838870644569397, + -0.16266612708568573, + -0.018731724470853806, + 0.07653182744979858, + 0.08485358953475952, + -0.12221826612949371, + 0.0437079519033432, + -1.5114665031433105, + -0.1361807882785797 + ], + [ + 0.08229903131723404, + 0.012069184333086014, + 0.0042723119258880615, + -0.2510314881801605, + -0.11223533749580383, + -1.9687868356704712, + -0.24648147821426392, + 0.09049665182828903, + -0.07600291073322296, + 0.009642921388149261, + 0.1038266196846962, + -0.015335500240325928, + -1.5055532455444336, + -0.3190021514892578, + 0.08912461251020432, + 0.06337188184261322, + 0.0903693288564682, + -0.10074125230312347, + 0.0035130679607391357, + -1.4228956699371338, + -0.2583085894584656 + ], + [ + 0.15582038462162018, + 0.054556794464588165, + 0.06342050433158875, + -0.21441645920276642, + 0.06938904523849487, + -2.3516125679016113, + -0.31208133697509766, + -0.015061333775520325, + -0.13063524663448334, + -0.06840226799249649, + 0.30069854855537415, + 0.1009671688079834, + -1.6926590204238892, + -0.16687323153018951, + -0.026025038212537766, + 0.08407551050186157, + 0.08806981146335602, + -0.06830160319805145, + 0.11908502876758575, + -1.464280605316162, + -0.05905193090438843 + ], + [ + 0.08990472555160522, + 0.05539984256029129, + 0.03419211506843567, + -0.2913701832294464, + 0.10797059535980225, + -2.244950532913208, + -0.382861852645874, + 0.08609195798635483, + -0.1347731053829193, + -0.070514015853405, + 0.14696359634399414, + 0.05582806468009949, + -1.6561474800109863, + -0.2793393135070801, + 0.02065729722380638, + 0.13093791902065277, + 0.09945663809776306, + -0.10855282843112946, + 0.20223209261894226, + -1.6661033630371094, + 0.05775855481624603 + ], + [ + 0.16831718385219574, + -0.0490262508392334, + -0.2795744240283966, + -0.38708996772766113, + 0.5014309883117676, + -1.5272479057312012, + -0.5076436996459961, + 0.09016288071870804, + -0.03740059584379196, + -0.01181013137102127, + -0.016439422965049744, + -0.0940324068069458, + -0.9069281816482544, + -0.48866915702819824, + 0.26338136196136475, + 0.103096604347229, + 0.16522401571273804, + -0.2173214703798294, + 0.3400283455848694, + -1.6648454666137695, + 0.0634608268737793 + ] + ], + "generated_utilities": [ + 0.179222971200943, + 0.22463779151439667, + 0.8147144317626953, + 0.19045117497444153, + 0.18127290904521942, + 0.18905499577522278, + 0.19813549518585205, + 0.16648955643177032 + ], + "group_id": "ms-b0fee26e1db967f90b81", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8397466540336609, + 0.8594310283660889, + 0.14638642966747284, + 0.09418028593063354, + 0.7838175296783447, + 0.5982922315597534, + 0.5913184881210327, + 2.0, + 0.20996521413326263, + 0.17419946193695068, + 0.11757439374923706, + 0.1645697057247162, + 0.05191062390804291, + 0.08658662438392639, + 0.04989185929298401, + 0.05867189168930054 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1510201096534729, + -0.21724483370780945, + 0.14240525662899017, + 0.003108598291873932, + -0.11609727144241333, + -1.6516358852386475, + 0.0, + -0.09976008534431458, + 0.061503566801548004, + 0.3070257902145386, + -0.08051937818527222, + 0.05735237896442413, + -1.6075372695922852, + 0.0, + -0.19649454951286316, + -0.15815469622612, + 0.2621642053127289, + -0.17578476667404175, + -0.034231506288051605, + -1.3808586597442627, + 0.0 + ], + [ + 0.014505341649055481, + 0.0403006337583065, + 0.5529788732528687, + 0.0714028924703598, + -0.41683781147003174, + -1.6516358852386475, + 0.0, + 0.41446545720100403, + -0.16278114914894104, + -0.4971221685409546, + -0.3640303611755371, + 0.02483367919921875, + -1.6075372695922852, + 0.0, + 0.5711643695831299, + -0.5244585275650024, + -0.5069849491119385, + 0.31709733605384827, + -0.4254770278930664, + -1.3808586597442627, + 0.0 + ], + [ + -0.044587142765522, + 0.03136333078145981, + 0.014485275372862816, + 0.04301099479198456, + 0.47328460216522217, + -2.6516358852386475, + -0.32303211092948914, + -0.01855158433318138, + -0.003608003491535783, + 0.029763570055365562, + 0.6359696388244629, + -0.1304762363433838, + -2.607537269592285, + -1.7957698106765747, + -0.009414590895175934, + -0.000939749414101243, + 0.018994441255927086, + 0.7486392855644226, + -0.06401278078556061, + -2.3808586597442627, + -1.722450852394104 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.6460642218589783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.5915396213531494, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.444901704788208 + ], + [ + 0.18585285544395447, + 0.17557260394096375, + 0.08478302508592606, + -0.17644454538822174, + 0.7477058172225952, + -2.614673137664795, + -0.3018328547477722, + 0.2204062044620514, + -0.0497702918946743, + 0.1961878538131714, + 0.5304322242736816, + -0.15872131288051605, + -2.604066848754883, + -2.053436517715454, + 0.18444111943244934, + -0.0024187257513403893, + 0.0054171085357666016, + 1.0256288051605225, + 0.02724038064479828, + -2.426236391067505, + -1.6594898700714111 + ], + [ + -0.5335133075714111, + 0.12298225611448288, + -0.12197553366422653, + 0.0424814410507679, + 0.002416938543319702, + -2.8428826332092285, + -0.17827168107032776, + -0.3377644121646881, + 0.04258471727371216, + -0.15910103917121887, + 0.9169985055923462, + 0.003966793417930603, + -3.069180727005005, + -1.407257318496704, + -0.47542956471443176, + -0.2210284322500229, + -0.10021048784255981, + 0.7806655168533325, + 0.3228837251663208, + -2.058253049850464, + -1.556677222251892 + ], + [ + 0.20687156915664673, + -0.5511468052864075, + 0.3945963978767395, + -0.16104432940483093, + 0.4936041831970215, + -2.1271684169769287, + 0.02855244278907776, + 0.544975996017456, + -0.3336343765258789, + 0.029475202783942223, + 0.8120307326316833, + -0.3246060609817505, + -2.885145425796509, + -2.2030622959136963, + -0.28730231523513794, + 0.19937433302402496, + 0.19840951263904572, + 1.1192030906677246, + -0.3995746374130249, + -2.5594136714935303, + -1.1545796394348145 + ], + [ + -0.5497553944587708, + -0.39125651121139526, + 0.2725587487220764, + -0.27965104579925537, + 1.0584466457366943, + -2.1044363975524902, + -0.6898342370986938, + 0.36552220582962036, + -0.2289373278617859, + 0.24627351760864258, + 0.7968363165855408, + 0.14776766300201416, + -2.988917112350464, + -1.6111457347869873, + -0.5116392374038696, + 0.030796781182289124, + -0.08262427151203156, + 1.0816104412078857, + 0.336497962474823, + -1.684936285018921, + -2.290850877761841 + ], + [ + 0.18322327733039856, + 0.6938241720199585, + 0.5252001881599426, + 0.06770005077123642, + 1.120344877243042, + -1.8558356761932373, + 0.18930694460868835, + -0.7033196687698364, + 0.7861698865890503, + -0.0965423658490181, + 0.015094935894012451, + -0.5303559303283691, + -2.6036770343780518, + -2.589250087738037, + -0.468231737613678, + -0.5901691317558289, + -0.7109078764915466, + 0.986366331577301, + -0.7634903192520142, + -2.984590530395508, + -1.1651530265808105 + ], + [ + 0.7039692401885986, + -0.5346179008483887, + 0.1466517597436905, + -0.6284777522087097, + -0.07994896173477173, + -2.4640438556671143, + -1.102164626121521, + -0.26685386896133423, + -0.3165925443172455, + 0.8653677701950073, + 0.232782244682312, + -0.5054740905761719, + -3.400317668914795, + -1.4077497720718384, + -0.09987740218639374, + -0.27442091703414917, + 0.5428174138069153, + 1.5128605365753174, + 0.747156023979187, + -2.0367631912231445, + -2.464085578918457 + ], + [ + 0.4474923610687256, + -0.7824500203132629, + 0.5928975343704224, + 0.57558673620224, + -0.17630118131637573, + -2.176387310028076, + -0.3299017548561096, + -0.6172446608543396, + 0.5065999627113342, + 0.46063295006752014, + 0.5951727628707886, + -0.02366364747285843, + -3.254279136657715, + -0.8880517482757568, + 0.3714323043823242, + 0.12449074536561966, + 0.6677311658859253, + -0.20713549852371216, + 0.5062628984451294, + -3.2156763076782227, + -1.9652986526489258 + ], + [ + -0.437069296836853, + 0.615723192691803, + -0.7743021249771118, + 0.20228458940982819, + 0.4320293366909027, + -2.0403428077697754, + -0.32251936197280884, + -0.7280393838882446, + 0.10878247022628784, + 0.9395551681518555, + 0.6021312475204468, + -0.3748174011707306, + -2.0175063610076904, + -1.9166245460510254, + -0.10793138295412064, + 0.6512030959129333, + 0.3747653365135193, + 1.5698660612106323, + -0.7995363473892212, + -1.8378328084945679, + -1.0564453601837158 + ], + [ + -0.7214477062225342, + -0.5686705708503723, + 0.8819299936294556, + 0.7319017648696899, + 1.0695421695709229, + -2.0338172912597656, + 0.2973436415195465, + -0.19576850533485413, + 0.4506514370441437, + 0.9037411212921143, + 0.0996239185333252, + 0.5076434016227722, + -2.611851930618286, + -2.7078182697296143, + 0.022706281393766403, + -0.4812851846218109, + -0.40778082609176636, + 1.6509652137756348, + -0.5721896886825562, + -1.8604621887207031, + -1.2943172454833984 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.017527949064970016, + -0.13547566533088684, + -0.31396082043647766, + -0.06045208126306534, + 0.7718918323516846, + -2.996134042739868, + -0.0199548602104187, + 0.26194432377815247, + -0.303554505109787, + -0.09068091958761215, + 0.3007984459400177, + -0.39815279841423035, + -2.626121997833252, + -1.8776893615722656, + 0.25959014892578125, + 0.20619672536849976, + -0.25346171855926514, + 0.40209779143333435, + 0.1943899542093277, + -2.464694023132324, + -1.5338099002838135 + ] + ], + "predicted_scores": [ + 4.6016645431518555, + 3.48230242729187, + 3.2867255210876465, + 3.1944432258605957, + 3.0854947566986084, + 2.9843802452087402, + 2.575968027114868, + 0.7800743579864502 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1358421325/train0", + "selected_index": 0, + "state_hash": "b8e04d64f31138fff8f83f1bb017f9e8515565a2ada2ba5787ad5f06b796589c", + "stored_base_utility": 0.8397466540336609, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8307622671127319, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8307622671127319 + }, + "base_progress": 0.8307622671127319, + "base_success": false, + "base_utility": 0.8307622671127319, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.18349629640579224, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18349629640579224 + }, + { + "contact_quality": null, + "progress": 0.18907088041305542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18907088041305542 + }, + { + "contact_quality": null, + "progress": 0.1815730631351471, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1815730631351471 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.5584076046943665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5584076046943665 + }, + { + "contact_quality": null, + "progress": 0.18897385895252228, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18897385895252228 + }, + { + "contact_quality": null, + "progress": 0.19263410568237305, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19263410568237305 + }, + { + "contact_quality": null, + "progress": 0.5586480498313904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5586480498313904 + } + ], + "candidate_progress": [ + 0.18349629640579224, + 0.18907088041305542, + 0.1815730631351471, + 1.0, + 0.5584076046943665, + 0.18897385895252228, + 0.19263410568237305, + 0.5586480498313904 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-852c671b0adeccfa4bf1", + "ms-5bddb39f792f20e5e6d4", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "generated_tangents": [ + [ + -0.020635083317756653, + 0.0959254801273346, + -0.061238616704940796, + -0.08016014099121094, + 0.44623851776123047, + -0.3494376242160797, + -0.14195102453231812, + 0.13645543158054352, + 0.08305901288986206, + 0.052140943706035614, + -0.12193156033754349, + -0.10969516634941101, + -1.6263598203659058, + 0.05815311521291733, + -0.0007215999066829681, + -0.03338552638888359, + -0.050850823521614075, + 0.02344834804534912, + 0.04952099919319153, + -1.2113022804260254, + 0.039293646812438965 + ], + [ + 0.028915219008922577, + 0.09982654452323914, + -0.12753590941429138, + -0.01064649224281311, + 0.43138062953948975, + -0.23808756470680237, + -0.06415492296218872, + 0.014027386903762817, + 0.03954826295375824, + 0.07355596125125885, + -0.12208562344312668, + -0.07834076881408691, + -1.5096139907836914, + -0.004875048995018005, + -0.022102344781160355, + 0.055252332240343094, + -0.03557021915912628, + -0.104021355509758, + -0.08897733688354492, + -1.2019795179367065, + -0.006463631987571716 + ], + [ + -0.041307464241981506, + 0.12543150782585144, + -0.1795656383037567, + -0.04052796959877014, + 0.36637312173843384, + -0.09606052935123444, + -0.13462674617767334, + 0.10015849024057388, + -0.032246269285678864, + 0.08517506718635559, + -0.21736830472946167, + 0.10787816345691681, + -1.371263861656189, + 0.057300083339214325, + 0.02319614589214325, + 0.1050652414560318, + 0.0813579112291336, + -0.12381233274936676, + -0.05572065711021423, + -1.125626802444458, + 0.01024523377418518 + ], + [ + 0.04413406550884247, + 0.08551359176635742, + 0.035018593072891235, + -0.02496318519115448, + 0.7133876085281372, + -0.4000389873981476, + -0.07011795043945312, + 0.04560220241546631, + -0.05485475808382034, + 0.07572954893112183, + -0.09889846295118332, + -0.09824007749557495, + -1.4547662734985352, + -0.0967516154050827, + 0.0005850046873092651, + 0.06128402426838875, + -0.010074526071548462, + -0.1729082614183426, + -0.21908676624298096, + -1.2497552633285522, + -0.03272366523742676 + ], + [ + -0.09880837798118591, + 0.005370769649744034, + -0.3337397277355194, + 0.2283633053302765, + 0.4611598253250122, + -0.1822780966758728, + -0.2319657802581787, + 0.00840727984905243, + -0.07833203673362732, + -0.1108061745762825, + -0.11645384877920151, + 0.1898232251405716, + -1.433670163154602, + 0.13133937120437622, + 0.028921213001012802, + 0.1270018070936203, + 0.0030322521924972534, + -0.13117165863513947, + -0.04420021176338196, + -1.1167993545532227, + 0.14613139629364014 + ], + [ + 0.11469144374132156, + 0.08521609008312225, + -0.20930549502372742, + -0.0015750080347061157, + 0.3572162389755249, + -0.14677658677101135, + -0.17239797115325928, + 0.06843464076519012, + 0.003756226971745491, + 0.09487643092870712, + -0.029724866151809692, + -0.15236535668373108, + -1.5197081565856934, + 0.03441619127988815, + -0.061104562133550644, + 0.03566605970263481, + -0.14348895847797394, + -0.09357191622257233, + -0.12039792537689209, + -1.241206169128418, + 0.01101510226726532 + ], + [ + 0.1544441431760788, + 0.14982743561267853, + -0.14339640736579895, + 0.1214568018913269, + 0.17490744590759277, + -0.32321736216545105, + -0.1192970871925354, + -0.06034785509109497, + -0.029587386175990105, + 0.00627436488866806, + -0.1050943061709404, + -0.3003830909729004, + -1.4825079441070557, + -0.22405023872852325, + 0.017843198031187057, + 0.03236152231693268, + -0.18885256350040436, + 0.053827688097953796, + -0.4133561849594116, + -1.4220597743988037, + -0.03589838743209839 + ], + [ + -0.10528199374675751, + -0.07495167851448059, + -0.4740738570690155, + 0.14678460359573364, + 0.6004019975662231, + -0.06772100925445557, + -0.30438971519470215, + 0.03796494007110596, + -0.1418292373418808, + -0.009805478155612946, + -0.18407884240150452, + 0.3589443564414978, + -1.2774512767791748, + 0.1350625902414322, + -0.04852801188826561, + 0.07202960550785065, + -0.11233554780483246, + -0.1494644433259964, + -0.13059982657432556, + -1.2425318956375122, + 0.13258422911167145 + ] + ], + "generated_utilities": [ + 0.18349629640579224, + 0.18907088041305542, + 0.1815730631351471, + 2.0, + 0.5584076046943665, + 0.18897385895252228, + 0.19263410568237305, + 0.5586480498313904 + ], + "group_id": "ms-bc2d3b425d9163a62dbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8266608119010925, + 2.0, + 0.15905821323394775, + 0.13648143410682678, + 0.5621776580810547, + 0.18381908535957336, + 0.20885342359542847, + 0.18184903264045715, + 0.19253113865852356, + 0.5574395656585693, + 0.10362428426742554, + 0.0659959614276886, + 0.09997051954269409, + 0.2210000902414322, + 0.00010728836059570312, + 0.12723565101623535 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.059883326292037964, + -0.16710972785949707, + -0.04027438163757324, + -0.23492489755153656, + 0.34960758686065674, + 0.16551673412322998, + 0.0, + -0.27484941482543945, + 0.07304387539625168, + -0.38491085171699524, + 0.03056102991104126, + 0.16416949033737183, + -1.5270280838012695, + 0.0, + 0.01279987208545208, + -0.07740817964076996, + 0.07519636303186417, + 0.14440757036209106, + 0.0022157281637191772, + -1.3374013900756836, + 0.0 + ], + [ + -0.6122239828109741, + -0.0734642744064331, + -0.4064018726348877, + 0.11902926862239838, + 0.34960758686065674, + -0.07764682173728943, + 0.0, + 0.37407442927360535, + -0.08740249276161194, + -0.20959049463272095, + -0.37996697425842285, + -0.304497092962265, + -1.5270280838012695, + 0.0, + -0.9113190174102783, + 0.3964778184890747, + 0.4811243414878845, + 0.2385387420654297, + -0.11476142704486847, + -1.3374013900756836, + 0.0 + ], + [ + 0.3877759873867035, + 0.6924911141395569, + -1.0402743816375732, + 0.005685340613126755, + 1.3496075868606567, + -0.2760518193244934, + 1.2840001583099365, + -0.028063178062438965, + 0.0023769063409417868, + 0.03187502175569534, + 0.04654876887798309, + 0.3263346254825592, + -2.5270280838012695, + -1.7457892894744873, + -0.012772172689437866, + -0.008647367358207703, + 0.019657021388411522, + 0.3050904870033264, + 0.19037143886089325, + -2.3374013900756836, + -1.738369345664978 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.568000316619873, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.4915785789489746, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.476738691329956 + ], + [ + 0.1989155411720276, + 0.7169155478477478, + -0.9921301603317261, + 0.22762086987495422, + 1.516188144683838, + -0.23096826672554016, + 1.0414973497390747, + 0.0012522991746664047, + -0.0518387071788311, + -0.23540246486663818, + 0.08515512943267822, + 0.5060473084449768, + -2.5231378078460693, + -1.4593229293823242, + -0.09671962261199951, + 0.09134317189455032, + -0.07944301515817642, + 0.18403387069702148, + 0.2437569499015808, + -2.270580291748047, + -1.9640144109725952 + ], + [ + 0.256519615650177, + 0.7805030941963196, + -1.3583617210388184, + 0.3310847878456116, + 1.2735480070114136, + -0.4127388000488281, + 1.2665152549743652, + -0.19997301697731018, + -0.11129427701234818, + 0.1647017002105713, + 0.3767858147621155, + 0.4831879138946533, + -2.752143621444702, + -1.6790910959243774, + 0.2554209232330322, + 0.3251851797103882, + -0.12605378031730652, + 0.696884274482727, + 0.29535919427871704, + -2.726069688796997, + -1.704551100730896 + ], + [ + 0.49772778153419495, + 1.0026791095733643, + -0.5513590574264526, + 0.008379313163459301, + 1.7768750190734863, + -0.6102496981620789, + 1.7048192024230957, + -0.29564282298088074, + -0.20420490205287933, + -0.23934513330459595, + -0.34791648387908936, + 0.0690477192401886, + -2.602295398712158, + -1.8138929605484009, + -0.38315677642822266, + -0.16421082615852356, + 0.17502839863300323, + -0.13996875286102295, + -0.30929630994796753, + -2.544666290283203, + -1.2411268949508667 + ], + [ + 0.07701912522315979, + 0.9529868364334106, + -1.497549295425415, + -0.3016125559806824, + 1.8504154682159424, + -0.3136579394340515, + 1.6619524955749512, + -0.28860214352607727, + 0.20993559062480927, + 0.6072397232055664, + 0.04243222624063492, + -0.133962482213974, + -2.0948100090026855, + -2.1691184043884277, + 0.44940999150276184, + -0.4183289408683777, + 0.5662978291511536, + 0.5016863942146301, + 0.7054548263549805, + -2.282663345336914, + -1.8892805576324463 + ], + [ + -0.2981111705303192, + 0.038420140743255615, + -1.687281847000122, + -0.37934571504592896, + 1.1300733089447021, + -0.6208293437957764, + 1.8144867420196533, + 0.16701459884643555, + 0.5508663654327393, + 0.4797254800796509, + -0.09421360492706299, + -0.3499920070171356, + -2.5030596256256104, + -1.5871793031692505, + -0.33045580983161926, + -0.29071635007858276, + 0.5385350584983826, + 0.9102991223335266, + 0.3121117353439331, + -2.2699851989746094, + -2.081907272338867 + ], + [ + -0.28236815333366394, + 0.18060839176177979, + -0.32270634174346924, + 0.033799149096012115, + 1.128151774406433, + -0.7577929496765137, + 1.6937967538833618, + -0.813525378704071, + 0.48178502917289734, + -0.6699209809303284, + -0.03810972720384598, + 0.8273124694824219, + -2.2768704891204834, + -1.317199945449829, + 0.0046694036573171616, + 0.5900158286094666, + -0.12453534454107285, + 1.046316385269165, + 0.2770566940307617, + -2.0448076725006104, + -2.4180057048797607 + ], + [ + 0.26116815209388733, + 1.305420994758606, + -0.5357515811920166, + 0.6523149013519287, + 0.5250536203384399, + -0.6821175217628479, + 1.2918037176132202, + 0.3275931477546692, + -0.8845832943916321, + -0.6987056136131287, + 0.10023711621761322, + -0.2849085032939911, + -1.7303776741027832, + -1.0117536783218384, + -0.06847482174634933, + 0.6783800721168518, + -0.04633811116218567, + 0.0702061653137207, + -0.3159322142601013, + -3.004882335662842, + -2.0691566467285156 + ], + [ + 0.5418367385864258, + 1.1724635362625122, + -1.717900276184082, + 0.9217556118965149, + 2.316706657409668, + -0.2411489486694336, + 0.720655083656311, + -0.6330894231796265, + 0.03703225776553154, + 0.38938266038894653, + -0.8305367827415466, + -0.2152116596698761, + -2.614929437637329, + -2.045497179031372, + -0.9044661521911621, + 0.9391879439353943, + -0.2323634922504425, + 0.8392186164855957, + -0.6889055371284485, + -3.0831263065338135, + -1.0035808086395264 + ], + [ + 1.0110303163528442, + 1.0131109952926636, + -1.3200907707214355, + -0.433234840631485, + 1.9466626644134521, + 0.46447479724884033, + 1.528633952140808, + 0.40949776768684387, + 0.07322453707456589, + -0.12349443882703781, + 0.24514153599739075, + 0.005379408597946167, + -2.487516164779663, + -1.1890358924865723, + -0.026332437992095947, + -0.6102214455604553, + 0.6559669971466064, + 0.4371780753135681, + -0.6651540994644165, + -2.8161611557006836, + -1.4271454811096191 + ], + [ + 0.9640108346939087, + 1.0122193098068237, + -1.9809117317199707, + 0.40601760149002075, + 0.9296187162399292, + 0.601173460483551, + 2.229055643081665, + -0.4972975254058838, + 0.6548057794570923, + 0.6165774464607239, + -0.7309268116950989, + 0.697066068649292, + -3.0948128700256348, + -2.4828031063079834, + 0.16930773854255676, + -0.11851751059293747, + -0.8277174830436707, + 0.8252183794975281, + -0.4829612374305725, + -2.922985076904297, + -1.1981642246246338 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03792491555213928, + -0.040427446365356445, + -0.04027438163757324, + 0.02325565740466118, + 0.34960758686065674, + -0.05261008441448212, + 0.0, + 0.018791399896144867, + 0.05303370952606201, + 0.01128687709569931, + 0.01692008785903454, + 0.06139722466468811, + -1.5270280838012695, + 0.0, + -0.004229618236422539, + 0.017936373129487038, + 0.06901376694440842, + -0.04424762725830078, + -0.033029720187187195, + -1.3374013900756836, + 0.0 + ] + ], + "predicted_scores": [ + 5.32202672958374, + 5.083075046539307, + 5.033701419830322, + 4.686803340911865, + 4.6638054847717285, + 4.2977118492126465, + 3.8724560737609863, + 3.265869617462158 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007/train0", + "selected_index": 0, + "state_hash": "58b3d0c1cafc199ad58508912becbaef59038a655cef0caafd263ade5c9b53fa", + "stored_base_utility": 0.8266608119010925, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20032641291618347, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20032641291618347 + }, + "base_progress": 0.20032641291618347, + "base_success": false, + "base_utility": 0.20032641291618347, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2363920658826828, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2363920658826828 + }, + { + "contact_quality": null, + "progress": 0.5597574710845947, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5597574710845947 + }, + { + "contact_quality": null, + "progress": 0.581784188747406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.581784188747406 + }, + { + "contact_quality": null, + "progress": 0.21587182581424713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21587182581424713 + }, + { + "contact_quality": null, + "progress": 0.23852916061878204, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23852916061878204 + }, + { + "contact_quality": null, + "progress": 0.21416914463043213, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21416914463043213 + }, + { + "contact_quality": null, + "progress": 0.22261354327201843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22261354327201843 + }, + { + "contact_quality": null, + "progress": 0.23042941093444824, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23042941093444824 + } + ], + "candidate_progress": [ + 0.2363920658826828, + 0.5597574710845947, + 0.581784188747406, + 0.21587182581424713, + 0.23852916061878204, + 0.21416914463043213, + 0.22261354327201843, + 0.23042941093444824 + ], + "candidate_source_chart_ids": [ + "ms-9ae5c6c4132218ecf49e", + "ms-95028134c6e3670b89d5", + "ms-5ac1bf2d7d319955d45d", + "ms-b8492a3a812095651d75", + "ms-ee2fcbc94d41385713d0", + "ms-95028134c6e3670b89d5", + "ms-c8706d0aa14a930236de", + "ms-016c5de39ed3c111889a" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank6", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank5", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-d1960368ef76bbb10669", + "generated_tangents": [ + [ + -0.009002715349197388, + 0.08356291055679321, + -0.20533940196037292, + -0.1263590306043625, + 0.2406829297542572, + 0.06444787234067917, + 0.6394894123077393, + -0.08873367309570312, + 0.027290765196084976, + -0.16187569499015808, + -0.01088329404592514, + -0.02162127196788788, + -1.5415910482406616, + 0.676408052444458, + 0.19243642687797546, + -0.07440962642431259, + -0.11381717026233673, + -0.1056600958108902, + -0.2868006229400635, + -1.6187689304351807, + 0.2273474633693695 + ], + [ + -0.021199747920036316, + 0.12840774655342102, + -0.16464844346046448, + -0.15847478806972504, + 0.1140025183558464, + 0.10073689371347427, + 0.7886719703674316, + -0.13427844643592834, + 0.012372835539281368, + -0.11800023168325424, + -0.007419198751449585, + 0.04618501663208008, + -1.3953291177749634, + 0.6757482886314392, + 0.2294318526983261, + -0.032656021416187286, + -0.06413744390010834, + -0.05787228047847748, + -0.19111958146095276, + -1.6142559051513672, + 0.09649721533060074 + ], + [ + 0.04666370153427124, + 0.17349736392498016, + -0.10926619172096252, + -0.2744497060775757, + -0.05658543109893799, + 0.648232102394104, + 0.8430483341217041, + -0.20604443550109863, + 0.0634075179696083, + -0.14573881030082703, + -0.19670140743255615, + -0.007543712854385376, + -1.698667049407959, + 0.7503986358642578, + 0.0491400808095932, + 0.022863559424877167, + -0.14207951724529266, + -0.10934339463710785, + -0.24875205755233765, + -1.8660626411437988, + 0.21983829140663147 + ], + [ + 0.020145952701568604, + 0.20961762964725494, + -0.17780622839927673, + -0.07486733794212341, + 0.26312339305877686, + 0.17808207869529724, + 0.5103111863136292, + -0.2040085792541504, + -0.02692084200680256, + -0.09121080487966537, + 0.04213942587375641, + 0.19140036404132843, + -1.3332122564315796, + 0.5214415192604065, + 0.2798238694667816, + 0.008504919707775116, + -0.1379687637090683, + -0.09148384630680084, + -0.18972328305244446, + -1.5710396766662598, + -0.033500656485557556 + ], + [ + -0.004896685481071472, + 0.20563440024852753, + -0.2885300815105438, + -0.1926400363445282, + 0.05656532943248749, + 0.5185076594352722, + 0.8056787252426147, + -0.11853313446044922, + 0.03304257243871689, + -0.008739106357097626, + -0.08999045938253403, + -0.03273311257362366, + -1.4083976745605469, + 0.622584342956543, + 0.06208796426653862, + -0.019653087481856346, + -0.040728285908699036, + -0.08429069817066193, + -0.19704973697662354, + -1.7740581035614014, + 0.3122518062591553 + ], + [ + -0.12031464278697968, + 0.06919682770967484, + 0.00935259461402893, + -0.1544983685016632, + 0.37343141436576843, + 0.3457547724246979, + 0.8786454200744629, + -0.12150397896766663, + -0.06065675616264343, + -0.09790251404047012, + 0.06326153874397278, + 0.30927544832229614, + -1.5503307580947876, + 0.6292473077774048, + -0.06414148211479187, + -0.1344526708126068, + -0.08821617066860199, + 0.03610348701477051, + -0.2107485830783844, + -1.6647887229919434, + 0.1340775191783905 + ], + [ + -0.19174478948116302, + 0.18334776163101196, + -0.1974191963672638, + -0.20941951870918274, + 0.10067513585090637, + 0.5494449734687805, + 0.6163052916526794, + -0.17247524857521057, + 0.03027583658695221, + 0.05010638386011124, + -0.07675803452730179, + -0.14911431074142456, + -1.1016347408294678, + 0.7832385301589966, + 0.23437924683094025, + -0.093446746468544, + -0.009180739521980286, + -0.04516807198524475, + -0.08957105875015259, + -1.637145757675171, + 0.07488099485635757 + ], + [ + -0.009413033723831177, + 0.18980033695697784, + -0.1679915487766266, + -0.07231934368610382, + 0.4116837978363037, + 0.036274492740631104, + 0.6768482327461243, + -0.10792264342308044, + -0.021929940208792686, + -0.07781047374010086, + 0.11434338986873627, + 0.0351085364818573, + -1.3289271593093872, + 0.6333006620407104, + 0.10635165125131607, + -0.07997269928455353, + 0.007293254137039185, + -0.19972731173038483, + -0.2883608937263489, + -1.587573766708374, + 0.32571089267730713 + ] + ], + "generated_utilities": [ + 0.2363920658826828, + 0.5597574710845947, + 0.581784188747406, + 0.21587182581424713, + 0.23852916061878204, + 0.21416914463043213, + 0.22261354327201843, + 0.23042941093444824 + ], + "group_id": "ms-d1960368ef76bbb10669", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.842734158039093, + 2.0, + 0.1342049241065979, + 0.18085119128227234, + 0.5091985464096069, + 0.016112878918647766, + 0.5080842971801758, + 0.19944033026695251, + 0.5132148265838623, + 0.5265648365020752, + 0.11437654495239258, + 0.04028162360191345, + 0.04675377905368805, + 0.13603349030017853, + 0.12877747416496277, + 0.05102705955505371 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2770041823387146, + 0.08894228935241699, + 0.07033950090408325, + 0.12426833808422089, + 0.16520383954048157, + 0.023725181818008423, + 0.0, + -0.25569218397140503, + -0.057803913950920105, + 0.06703128665685654, + 0.1732010841369629, + 0.07705201953649521, + -1.0374658107757568, + 0.0, + 0.04586680978536606, + 0.028830496594309807, + -0.0374501496553421, + 0.10374182462692261, + -0.007100850343704224, + -1.7975506782531738, + 0.0 + ], + [ + -0.4268343448638916, + 0.13483470678329468, + -0.06075429916381836, + 0.19817759096622467, + 0.09922409057617188, + 0.6547578573226929, + 0.0, + 0.15026617050170898, + 0.002472013235092163, + -0.31349992752075195, + 0.5119487047195435, + 0.571077287197113, + -1.0374658107757568, + 0.0, + -0.5828977823257446, + -0.20166385173797607, + -0.7097158432006836, + 0.8187870979309082, + -0.0694284737110138, + -1.7975506782531738, + 0.0 + ], + [ + -0.23462048172950745, + 1.088942289352417, + -0.5585125088691711, + 0.10042184591293335, + 0.4514904320240021, + -0.12210102379322052, + 2.176525354385376, + -0.10348781198263168, + 0.19529865682125092, + 0.12132670730352402, + 0.7056414484977722, + 0.12544797360897064, + -2.037465810775757, + 0.6289246678352356, + -0.008351176977157593, + 0.0010435802396386862, + 0.017314812168478966, + 0.6334809064865112, + -0.3720659911632538, + -2.797550678253174, + -1.5395829677581787 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.353050708770752, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.2578493356704712, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0791659355163574 + ], + [ + -0.09280626475811005, + 1.1662368774414062, + -0.8228501081466675, + 0.24497312307357788, + 0.5648186802864075, + -0.03522863984107971, + 2.259786367416382, + 0.14498579502105713, + 0.2030920535326004, + 0.39727938175201416, + 0.8023015856742859, + 0.3385764956474304, + -1.8738088607788086, + 0.7300182580947876, + 0.20887598395347595, + 0.23107871413230896, + 0.3028872311115265, + 0.7558343410491943, + -0.5975744724273682, + -3.0329887866973877, + -1.8127071857452393 + ], + [ + -0.13783538341522217, + 0.9487118721008301, + -0.3492928743362427, + -0.2956767678260803, + 0.2793389856815338, + -0.16702969372272491, + 2.558225631713867, + 0.17572781443595886, + 0.570004940032959, + 0.27895650267601013, + 0.6778483986854553, + -0.27156615257263184, + -1.8747575283050537, + 1.0240070819854736, + 0.27303546667099, + 0.33387133479118347, + 0.1442956030368805, + 0.24661725759506226, + -0.5561860799789429, + -2.9813332557678223, + -1.833754062652588 + ], + [ + -0.18553051352500916, + 1.0909820795059204, + -0.27226096391677856, + 0.1659364104270935, + 0.7108361721038818, + -0.27768880128860474, + 2.239938974380493, + 0.019483201205730438, + 0.21263465285301208, + -0.09839647263288498, + 0.9676799178123474, + -0.36985379457473755, + -2.133316993713379, + 0.3758644461631775, + -0.47750571370124817, + 0.17470555007457733, + -0.17035381495952606, + 0.2707958221435547, + -0.5672953128814697, + -2.394361972808838, + -1.668647289276123 + ], + [ + -0.3557406961917877, + 0.9564932584762573, + -1.0014276504516602, + -0.38222843408584595, + 0.41349664330482483, + -0.05637756735086441, + 1.6011608839035034, + -0.31295540928840637, + -0.15234358608722687, + 0.4648304879665375, + 0.2744198739528656, + -0.44805359840393066, + -2.1544065475463867, + 1.166503667831421, + 0.014861157163977623, + -0.23438170552253723, + 0.1278134286403656, + 0.5812121033668518, + -0.7571046352386475, + -2.5030438899993896, + -2.100955009460449 + ], + [ + -0.16500982642173767, + 0.7556273937225342, + -0.988593339920044, + 0.6213532090187073, + 0.15840065479278564, + 0.5245341062545776, + 2.452624797821045, + -0.5693200826644897, + -0.32241708040237427, + 0.49458175897598267, + 1.3341540098190308, + 0.059507817029953, + -2.304706335067749, + 0.7334492206573486, + -0.20702646672725677, + 0.32367274165153503, + -0.5396079421043396, + 0.9809319972991943, + -0.6172815561294556, + -3.197409152984619, + -0.9358728528022766 + ], + [ + -0.07666900753974915, + 1.8442692756652832, + -0.6603910326957703, + -0.40882760286331177, + 0.22092406451702118, + -0.23551997542381287, + 1.6105436086654663, + 0.09144749492406845, + 0.8141599893569946, + 0.8528898358345032, + 1.0663814544677734, + 0.3283998966217041, + -1.7124712467193604, + 1.2278027534484863, + 0.7039424180984497, + 0.3645748496055603, + -0.2968631386756897, + 0.4998055696487427, + -0.9362069368362427, + -3.461055278778076, + -0.8184676766395569 + ], + [ + -0.1520652174949646, + 1.405963659286499, + -0.7959750294685364, + -0.39229461550712585, + 0.46590372920036316, + -0.8267906904220581, + 2.952258586883545, + 0.5923218727111816, + -0.6996292471885681, + 0.8352785706520081, + -0.15220212936401367, + -0.1307106763124466, + -2.849807024002075, + 0.4983477294445038, + 0.5902999639511108, + -0.5390225648880005, + 0.13897128403186798, + 0.21545273065567017, + -1.1186240911483765, + -3.1762759685516357, + -1.0118725299835205 + ], + [ + -0.564094066619873, + 0.4751617908477783, + -1.3977731466293335, + 0.8189605474472046, + 0.17298781871795654, + 0.2476775199174881, + 2.8234164714813232, + -0.4518245458602905, + -0.7904537916183472, + -0.39086827635765076, + -0.05634337663650513, + -0.09099806845188141, + -1.9690911769866943, + 1.1869969367980957, + -0.5615661144256592, + 0.32865890860557556, + -0.8041903376579285, + 1.3324832916259766, + -0.6190662384033203, + -2.4903852939605713, + -2.3098156452178955 + ], + [ + -0.2328718602657318, + 0.9724385142326355, + -1.5535115003585815, + 0.7688977718353271, + 0.4409410357475281, + -0.8374075889587402, + 1.2719230651855469, + 0.653096616268158, + -0.2887840270996094, + 0.4892227351665497, + 0.9943453669548035, + 0.5566720962524414, + -2.496413469314575, + 0.6299278140068054, + -0.08689186722040176, + 0.024782167747616768, + 0.5747005939483643, + 0.023068726062774658, + -1.111369252204895, + -2.187150001525879, + -1.2047184705734253 + ], + [ + -1.1679872274398804, + 1.9997228384017944, + -0.45194220542907715, + -0.8861345648765564, + -0.5369991064071655, + 0.46434593200683594, + 2.175499439239502, + 0.7599186301231384, + 0.38914406299591064, + 0.6877046823501587, + -0.004208564758300781, + -0.1321776658296585, + -1.3474420309066772, + 0.8496907353401184, + 0.4218248426914215, + -0.9354817271232605, + 0.945868730545044, + 1.60379958152771, + -1.195396065711975, + -3.597724676132202, + -1.9041136503219604 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04476936161518097, + 0.08894228935241699, + 0.06436485052108765, + -0.0775555968284607, + 0.04825553297996521, + -0.060782790184020996, + 0.0, + -0.07688627392053604, + -0.06308497488498688, + -0.03861912339925766, + 0.1235572099685669, + -0.0017572194337844849, + -1.0374658107757568, + 0.0, + 0.03638586029410362, + 0.0556478351354599, + 0.0387292355298996, + 0.044096171855926514, + 0.029627740383148193, + -1.7975506782531738, + 0.0 + ] + ], + "predicted_scores": [ + 8.316713333129883, + 8.251832962036133, + 8.109463691711426, + 7.959565162658691, + 7.918623924255371, + 7.686976909637451, + 7.613646030426025, + 7.346455097198486 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "416183106/train0", + "selected_index": 0, + "state_hash": "cd187560c1c39db9e6bf7d9a611a8b7108dc88766de58fd62b4317f0bbcc1b50", + "stored_base_utility": 0.842734158039093, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8350887894630432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8350887894630432 + }, + "base_progress": 0.8350887894630432, + "base_success": false, + "base_utility": 0.8350887894630432, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07415902614593506, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07415902614593506 + }, + { + "contact_quality": null, + "progress": 0.21383549273014069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21383549273014069 + }, + { + "contact_quality": null, + "progress": 0.17038175463676453, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17038175463676453 + }, + { + "contact_quality": null, + "progress": 0.12335775792598724, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12335775792598724 + }, + { + "contact_quality": null, + "progress": 0.1851280927658081, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1851280927658081 + }, + { + "contact_quality": null, + "progress": 0.15958069264888763, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15958069264888763 + }, + { + "contact_quality": null, + "progress": 0.12082469463348389, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12082469463348389 + }, + { + "contact_quality": null, + "progress": 0.21975377202033997, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21975377202033997 + } + ], + "candidate_progress": [ + 0.07415902614593506, + 0.21383549273014069, + 0.17038175463676453, + 0.12335775792598724, + 0.1851280927658081, + 0.15958069264888763, + 0.12082469463348389, + 0.21975377202033997 + ], + "candidate_source_chart_ids": [ + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-5dfbb16e173e1a3e2bd6" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-efdc07c985935d871adf", + "generated_tangents": [ + [ + 0.3686366677284241, + 0.043556421995162964, + -0.15216979384422302, + -0.013488903641700745, + 0.309867262840271, + -2.2373733520507812, + -0.4430053234100342, + 0.35782021284103394, + 0.1588248312473297, + -0.13483312726020813, + 0.5717215538024902, + 0.402296245098114, + -1.5094940662384033, + -0.0051206499338150024, + 0.2405378669500351, + 0.049752142280340195, + 0.14417493343353271, + -0.08577902615070343, + 0.8010730743408203, + -0.8206570744514465, + -0.3179226517677307 + ], + [ + 0.012061640620231628, + -0.04155142232775688, + -0.06605860590934753, + 0.1513940393924713, + 0.011064141988754272, + -1.8977504968643188, + -0.11028212308883667, + 0.06433641910552979, + 0.014126881957054138, + -0.03595695644617081, + 0.21009963750839233, + 0.18688516318798065, + -1.3424303531646729, + -0.04337623715400696, + 0.05805295705795288, + 0.09326297044754028, + 0.014324337244033813, + -0.1732342392206192, + 0.16666626930236816, + -1.0746729373931885, + -0.015538901090621948 + ], + [ + 0.04373123496770859, + -0.026497136801481247, + 0.0027818381786346436, + -0.020371943712234497, + -0.13714423775672913, + -1.9406591653823853, + -0.2620019316673279, + 0.15213267505168915, + -0.04027959704399109, + 0.01317993551492691, + 0.06469055265188217, + 0.05747249722480774, + -1.3779916763305664, + -0.11872003972530365, + 0.05823354423046112, + 0.07354122400283813, + 0.08978244662284851, + -0.1656661480665207, + 0.028914421796798706, + -1.2013511657714844, + -0.08568954467773438 + ], + [ + 0.07770948857069016, + 0.09926199913024902, + -0.08355829119682312, + -0.23897604644298553, + -0.11167275905609131, + -2.064149856567383, + -0.6689459681510925, + 0.2126466929912567, + -0.07760970294475555, + -0.264264851808548, + 0.410615473985672, + 0.2747187912464142, + -1.1980723142623901, + -0.3064813017845154, + 0.07732196152210236, + 0.11608351767063141, + -0.06674216687679291, + -0.051100775599479675, + 0.43780913949012756, + -1.3022537231445312, + -0.2767101228237152 + ], + [ + 0.08911243081092834, + -0.01669311337172985, + -0.05099359154701233, + -0.04101960361003876, + -0.19033390283584595, + -1.7867720127105713, + -0.2750092148780823, + 0.15249690413475037, + -0.04276160150766373, + -0.028678081929683685, + 0.10094863921403885, + 0.11168846487998962, + -1.1930760145187378, + -0.21459124982357025, + 0.07177359610795975, + 0.10467469692230225, + 0.11782080680131912, + -0.10453735291957855, + 0.0208817720413208, + -1.0487130880355835, + -0.07126760482788086 + ], + [ + 0.11916574835777283, + -0.019409148022532463, + 0.004142403602600098, + -0.025014802813529968, + -0.031183749437332153, + -1.8143374919891357, + -0.254702627658844, + 0.18722927570343018, + -0.0933188870549202, + 0.07084082067012787, + 0.20232877135276794, + 0.10192329436540604, + -1.0879679918289185, + -0.05684955418109894, + -0.0026487447321414948, + 0.06035435572266579, + 0.06609466671943665, + -0.03150962293148041, + 0.03563234210014343, + -1.0091068744659424, + 0.0016781985759735107 + ], + [ + 0.3439967632293701, + 0.10091355443000793, + -0.22781983017921448, + -0.0903288871049881, + -0.08182793855667114, + -2.4931893348693848, + -0.6921324133872986, + 0.34496474266052246, + 0.0554824024438858, + 0.018603794276714325, + 0.31898757815361023, + 0.1792585402727127, + -1.3971823453903198, + -0.6852246522903442, + 0.13927975296974182, + 0.12209755182266235, + 0.29126888513565063, + -0.08204825222492218, + 0.39308786392211914, + -1.060652732849121, + -0.22201550006866455 + ], + [ + -0.02788279950618744, + -0.12240372598171234, + -0.09374865889549255, + -0.18003249168395996, + -0.21576184034347534, + -2.4772815704345703, + -0.6488150954246521, + 0.2751406133174896, + -0.0911252498626709, + -0.12399519234895706, + 0.4060862958431244, + -0.11341851949691772, + -1.222923994064331, + -0.26216763257980347, + -0.05457804724574089, + 0.15642470121383667, + 0.1643677055835724, + -0.03815515339374542, + 0.02476915717124939, + -1.194394588470459, + -0.12704795598983765 + ] + ], + "generated_utilities": [ + 0.07415902614593506, + 0.21383549273014069, + 0.17038175463676453, + 0.12335775792598724, + 0.1851280927658081, + 0.15958069264888763, + 0.12082469463348389, + 0.21975377202033997 + ], + "group_id": "ms-efdc07c985935d871adf", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8197344541549683, + 2.0, + 0.8125, + 0.8125142455101013, + 0.7847581505775452, + 0.7794750332832336, + 0.5920823216438293, + 0.5697357058525085, + 2.0, + 0.5847549438476562, + 0.09384849667549133, + 0.07791829109191895, + 0.011395931243896484, + 0.09286168217658997, + 0.10894183814525604, + 0.0616525262594223 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2827143669128418, + -0.12206736207008362, + 0.1543778032064438, + 0.27268126606941223, + -0.17288367450237274, + -2.5338973999023438, + -1.7615662813186646, + -0.19161763787269592, + 0.1886080503463745, + 0.31686434149742126, + 0.5112922191619873, + 0.14046527445316315, + -2.17134952545166, + -1.8021173477172852, + 0.11115366220474243, + 0.25490811467170715, + -0.025668112561106682, + 0.4787765145301819, + 0.46609628200531006, + -1.711870551109314, + -1.5312864780426025 + ], + [ + 0.03046044334769249, + 0.042622197419404984, + -0.12103251367807388, + 0.7814357876777649, + 0.5093902349472046, + -2.648094654083252, + -1.5764049291610718, + -0.16382060945034027, + 0.15227201581001282, + -0.29746392369270325, + 0.394713819026947, + 0.044284895062446594, + -2.150869131088257, + -1.853789210319519, + 0.24474957585334778, + 0.16074210405349731, + -0.30960795283317566, + 0.986261248588562, + 0.4512072801589966, + -1.9782602787017822, + -1.814382553100586 + ], + [ + -0.47424501180648804, + 0.4193490147590637, + 0.31025299429893494, + 0.436926007270813, + 0.07080172002315521, + -2.2750933170318604, + -2.139326572418213, + 0.19072748720645905, + -0.05066118389368057, + -0.38944289088249207, + 0.11913672089576721, + -0.4728909432888031, + -2.630631446838379, + -1.9070930480957031, + 0.5720394849777222, + 0.1898895502090454, + -0.0872373953461647, + 0.3081148862838745, + 0.6700773239135742, + -1.8990029096603394, + -1.655537486076355 + ], + [ + -0.112115278840065, + -0.5390006899833679, + -0.27749133110046387, + -0.08230990171432495, + 0.18304404616355896, + -2.5003390312194824, + -1.177594542503357, + -0.37152859568595886, + 0.1827065348625183, + -0.1805598884820938, + 0.8884490728378296, + 0.3141748607158661, + -2.7738473415374756, + -1.8585447072982788, + -0.6611859798431396, + 0.5365979075431824, + 0.024639926850795746, + 1.0166820287704468, + 0.6261310577392578, + -1.3820469379425049, + -1.1859028339385986 + ], + [ + -0.013153300620615482, + 0.6059781908988953, + -0.7563084363937378, + 0.6436030864715576, + 0.4828413128852844, + -1.8755569458007812, + -1.896214246749878, + -0.445474773645401, + -0.6578570604324341, + -0.21610267460346222, + 1.3274874687194824, + -0.6251860857009888, + -2.005418300628662, + -1.463393211364746, + -0.40788042545318604, + -0.5932772159576416, + -0.6501284837722778, + 0.1491221785545349, + 0.8418278694152832, + -2.578186511993408, + -1.628711462020874 + ], + [ + -0.051258739084005356, + 0.6121965050697327, + -0.4303005337715149, + 0.378423810005188, + -0.4187525510787964, + -2.0054633617401123, + -1.5839252471923828, + 0.38306933641433716, + -0.8548648953437805, + 0.5034631490707397, + 0.3682137727737427, + 0.867935061454773, + -2.5900886058807373, + -1.8283436298370361, + 0.3810080289840698, + -0.6684208512306213, + 0.5362142324447632, + -0.17851078510284424, + 1.083348035812378, + -1.9181195497512817, + -2.306365728378296 + ], + [ + 0.5487861037254333, + 0.6947635412216187, + -0.3694673776626587, + 1.3571128845214844, + -0.20053730905056, + -2.9099156856536865, + -1.493881106376648, + 0.9321863055229187, + -0.355974406003952, + 0.6561991572380066, + -0.29231804609298706, + 0.4913230240345001, + -1.8409770727157593, + -2.0504229068756104, + 0.14528872072696686, + 0.483799010515213, + -0.32654285430908203, + 1.6123273372650146, + 0.03987070918083191, + -0.8449068665504456, + -2.3494603633880615 + ], + [ + -0.3259732127189636, + -0.5385614633560181, + 0.7304226160049438, + -0.12314724922180176, + 0.03945803642272949, + -1.5919837951660156, + -1.907224416732788, + 0.3930681645870209, + -0.8835994005203247, + -0.3352096974849701, + 0.17923185229301453, + 0.17222784459590912, + -2.512176752090454, + -1.859424352645874, + 0.536429762840271, + 0.11454571038484573, + 0.3300909101963043, + 0.5491204261779785, + -0.04762554168701172, + -1.1976318359375, + -0.5251960158348083 + ], + [ + 0.33097749948501587, + -0.04192434251308441, + -0.34520047903060913, + -0.04110339283943176, + -0.5535940527915955, + -2.2057979106903076, + -2.5934860706329346, + -0.9023091793060303, + 0.5658158659934998, + -0.6125177145004272, + 1.095580816268921, + -0.05609257519245148, + -1.606997013092041, + -2.0733447074890137, + -0.5881220102310181, + 0.8953573703765869, + 0.9258373975753784, + 0.27281180024147034, + 0.7141706943511963, + -0.9142226576805115, + -1.3141751289367676 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05978316813707352, + 0.057473838329315186, + -0.09624473005533218, + -0.011721253395080566, + 0.028141841292381287, + -1.4311916828155518, + 0.0, + -0.024008113890886307, + 0.07049623876810074, + 0.03528204560279846, + -0.010002851486206055, + 0.010910652577877045, + -1.1452367305755615, + 0.0, + 0.020785929635167122, + -0.03374411538243294, + 0.11616832762956619, + 0.05772918462753296, + 0.01671847701072693, + -0.828272819519043, + 0.0 + ], + [ + -0.13237658143043518, + -0.3393171727657318, + 0.29588890075683594, + 0.8627858757972717, + -0.1666169911623001, + -2.520320177078247, + -1.6265922784805298, + -0.031582750380039215, + 0.11019003391265869, + -0.2699315845966339, + 0.9513242244720459, + 0.15523536503314972, + -1.7896702289581299, + -1.2776744365692139, + -0.3942914605140686, + -0.30590808391571045, + -0.3098926246166229, + 0.8258905410766602, + 0.29909753799438477, + -1.5711073875427246, + -1.5744072198867798 + ] + ], + "predicted_scores": [ + 3.255740165710449, + 2.8353159427642822, + 2.6269359588623047, + 2.6108710765838623, + 2.6059200763702393, + 2.0757269859313965, + 1.4323501586914062, + 1.3334894180297852 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "706479600/train0", + "selected_index": 0, + "state_hash": "b482f902b03ca480898798a8b5f19263d7cbb18c066305330b0796b607427fa7", + "stored_base_utility": 0.8197344541549683, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03719887137413025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03719887137413025 + }, + "base_progress": 0.03719887137413025, + "base_success": false, + "base_utility": 0.03719887137413025, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07726143300533295, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07726143300533295 + }, + { + "contact_quality": null, + "progress": 0.06953388452529907, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06953388452529907 + }, + { + "contact_quality": null, + "progress": 0.046124204993247986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.046124204993247986 + }, + { + "contact_quality": null, + "progress": 0.09917314350605011, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09917314350605011 + }, + { + "contact_quality": null, + "progress": 0.06149482727050781, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06149482727050781 + }, + { + "contact_quality": null, + "progress": 0.04581362009048462, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04581362009048462 + }, + { + "contact_quality": null, + "progress": 0.0635814368724823, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0635814368724823 + }, + { + "contact_quality": null, + "progress": 0.05730763077735901, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05730763077735901 + } + ], + "candidate_progress": [ + 0.07726143300533295, + 0.06953388452529907, + 0.046124204993247986, + 0.09917314350605011, + 0.06149482727050781, + 0.04581362009048462, + 0.0635814368724823, + 0.05730763077735901 + ], + "candidate_source_chart_ids": [ + "ms-d4c5d7eea19d94314e2e", + "ms-0a4d4909900fb0bc4a9a", + "ms-5ac1bf2d7d319955d45d", + "ms-ee2fcbc94d41385713d0", + "ms-07b479748a91a9c92b20", + "ms-2fef2f43af8009bb88f1", + "ms-bb299baa34e49eafb9fc", + "ms-0b1dd0e538b8df15a6e4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank7" + ], + "candidates_evaluated": true, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "generated_tangents": [ + [ + -0.06495420634746552, + 0.3671957850456238, + 0.11139822006225586, + -1.7162952423095703, + -0.44263845682144165, + 0.37888675928115845, + 0.06493577361106873, + 0.20768825709819794, + -0.4021793603897095, + -0.16209673881530762, + 1.0889604091644287, + 0.1378367841243744, + -1.6365530490875244, + -0.23112384974956512, + -0.035061415284872055, + 0.15449780225753784, + 0.05566084384918213, + -0.3404231667518616, + -0.08311712741851807, + -1.6378886699676514, + -0.43263307213783264 + ], + [ + -0.09358946979045868, + 0.6348069310188293, + 0.17672815918922424, + -1.7474277019500732, + -0.270841121673584, + 0.7005739212036133, + -0.39553868770599365, + -0.0020561963319778442, + -0.38483887910842896, + -0.09940668195486069, + 1.285354733467102, + 0.31022900342941284, + -1.4367177486419678, + 0.034482650458812714, + -0.06608331203460693, + -0.06098440662026405, + 0.09633795917034149, + -0.2390567809343338, + -0.12952619791030884, + -1.5066163539886475, + -0.15631580352783203 + ], + [ + 0.01595573127269745, + 0.6981759667396545, + 0.25221866369247437, + -1.8958021402359009, + -0.30718374252319336, + 0.6601626873016357, + -0.33606773614883423, + -0.16047650575637817, + -0.518339991569519, + -0.08485382050275803, + 1.1284399032592773, + 0.06395082175731659, + -1.854662299156189, + -0.09885132312774658, + 0.08961603790521622, + 0.1571999043226242, + 0.1967727392911911, + -0.3777223229408264, + -0.04145368933677673, + -1.6753199100494385, + 0.10013195127248764 + ], + [ + -0.023770973086357117, + 0.6930935978889465, + 0.18124186992645264, + -1.7630515098571777, + -0.09737259149551392, + 0.8673893213272095, + -0.19814473390579224, + -0.0702391266822815, + -0.6056856513023376, + -0.07049527019262314, + 1.1719478368759155, + 0.19473861157894135, + -1.3805370330810547, + 0.011462561786174774, + 0.03564780205488205, + 0.13740402460098267, + 0.17615480720996857, + -0.35321688652038574, + -0.03179416060447693, + -1.5796303749084473, + 0.09967153519392014 + ], + [ + -0.0340493768453598, + 0.5901545286178589, + 0.1768604815006256, + -1.8761394023895264, + -0.2016609013080597, + 0.9064714908599854, + -0.05415022373199463, + -0.011286810040473938, + -0.5596218109130859, + -0.11609595268964767, + 1.1898362636566162, + 0.3228369355201721, + -1.4279693365097046, + -0.09022350609302521, + 0.08991342782974243, + 0.09004552662372589, + 0.16198056936264038, + -0.35726600885391235, + 0.043157368898391724, + -1.5128161907196045, + 0.0026268362998962402 + ], + [ + 0.021356597542762756, + 0.6034295558929443, + 0.36402377486228943, + -1.8414827585220337, + -0.3594658374786377, + 0.7170552015304565, + -0.1328151822090149, + -0.12948063015937805, + -0.48487037420272827, + -0.10393033176660538, + 1.1427063941955566, + 0.17752020061016083, + -1.6741441488265991, + -0.18471203744411469, + 0.09598825871944427, + 0.12697991728782654, + 0.17526964843273163, + -0.3313748240470886, + 0.02222144603729248, + -1.4967906475067139, + 0.025683149695396423 + ], + [ + 0.0121297687292099, + 0.5627127885818481, + 0.2642773985862732, + -2.0473899841308594, + -0.398726224899292, + 0.7902873754501343, + -0.04045999050140381, + -0.06379231810569763, + -0.5675560832023621, + -0.11296277493238449, + 1.1654249429702759, + 0.12139623612165451, + -1.8295652866363525, + -0.07097220420837402, + 0.10972034931182861, + 0.06632433831691742, + 0.103777214884758, + -0.2688276767730713, + 0.0077446699142456055, + -1.3927452564239502, + -0.1138572096824646 + ], + [ + 0.07004734128713608, + 0.6754263639450073, + 0.25278669595718384, + -1.6248010396957397, + -0.00012731552124023438, + 0.3310000002384186, + -0.10137492418289185, + -0.09762006998062134, + -0.623462975025177, + -0.09329239279031754, + 1.2168601751327515, + 0.25074687600135803, + -1.4885960817337036, + 0.014419980347156525, + 0.04936787486076355, + 0.1198219507932663, + 0.22938913106918335, + -0.3071914315223694, + -0.03162345290184021, + -1.460853099822998, + 0.0734221562743187 + ] + ], + "generated_utilities": [ + 0.07726143300533295, + 0.06953388452529907, + 0.046124204993247986, + 0.09917314350605011, + 0.06149482727050781, + 0.04581362009048462, + 0.0635814368724823, + 0.05730763077735901 + ], + "group_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558897972106934, + 2.0, + 0.04248148202896118, + 0.05394089221954346, + 0.057141318917274475, + 0.03495943546295166, + 0.08633947372436523, + 0.046462297439575195, + 0.043458908796310425, + 0.023027777671813965, + 0.061922311782836914, + 0.034134700894355774, + 0.010651648044586182, + 0.01158277690410614, + 0.02622869610786438, + 0.012083128094673157 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.4298704266548157, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + 0.09995961934328079, + 1.162198781967163, + 0.0, + -0.00648990273475647, + -0.3028862476348877, + -0.0050805360078811646, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + 0.23185759782791138, + -0.11529958993196487, + -0.058992013335227966, + -0.010845988988876343, + 0.06423944234848022, + -2.0730173587799072, + 0.0 + ], + [ + -0.48041507601737976, + 0.6002404689788818, + 0.5398462414741516, + -2.198392152786255, + 0.1505453884601593, + 1.162198781967163, + 0.0, + 0.32028457522392273, + -0.264798641204834, + -0.27968189120292664, + 1.143568515777588, + 0.623285710811615, + -1.6758570671081543, + 0.0, + 0.3762620687484741, + 0.20924779772758484, + 0.033039893954992294, + -0.22763806581497192, + 0.8496624827384949, + -2.0730173587799072, + 0.0 + ], + [ + -0.4385460615158081, + 1.6002404689788818, + 1.1335551738739014, + -3.198392152786255, + 0.11579504609107971, + 2.162198781967163, + -1.8792641162872314, + 0.44874686002731323, + -1.264798641204834, + 0.08758684247732162, + 2.143568515777588, + 1.074564814567566, + -2.6758570671081543, + 1.4318934679031372, + -0.01953941211104393, + 0.012724362313747406, + 0.02134235017001629, + 0.39346790313720703, + -0.15033751726150513, + -3.0730173587799072, + -1.573193907737732 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.758528232574463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8637869358062744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.146387815475464 + ], + [ + -0.4582313299179077, + 1.465004801750183, + 0.9327377080917358, + -2.900977373123169, + -0.12230202555656433, + 2.23574161529541, + -2.1357157230377197, + 0.42294827103614807, + -1.506612777709961, + 0.29363107681274414, + 2.034971237182617, + 1.023054599761963, + -2.559215784072876, + 1.1742838621139526, + 0.026679620146751404, + -0.0811251625418663, + 0.16039283573627472, + 0.3527735471725464, + 0.041033580899238586, + -2.8834660053253174, + -1.2895927429199219 + ], + [ + -0.8261920809745789, + 1.8162921667099, + 1.1455696821212769, + -3.5306012630462646, + -0.15104833245277405, + 2.447983741760254, + -2.128190755844116, + 0.19818204641342163, + -1.146593689918518, + 0.03477605804800987, + 2.4299705028533936, + 0.8797258734703064, + -2.331338882446289, + 1.5516352653503418, + -0.19862954318523407, + -0.2736409604549408, + -0.21770769357681274, + 0.47925806045532227, + -0.15124429762363434, + -3.0052826404571533, + -1.2108670473098755 + ], + [ + -0.2760573625564575, + 1.6254181861877441, + 0.8188510537147522, + -3.0690624713897705, + -0.22635167837142944, + 2.051481246948242, + -1.417292833328247, + 0.026889145374298096, + -1.3347617387771606, + -0.15878626704216003, + 1.9778507947921753, + 0.7640928030014038, + -3.1165366172790527, + 1.862673282623291, + -0.451316773891449, + -0.15708383917808533, + 0.26080578565597534, + 0.6509285569190979, + 0.23991608619689941, + -3.50675892829895, + -1.297291874885559 + ], + [ + -0.8321853280067444, + 1.9553383588790894, + 1.5495907068252563, + -3.790785312652588, + 0.3716217279434204, + 2.5994434356689453, + -2.1380205154418945, + -0.10642725229263306, + -0.7562419772148132, + -0.442053884267807, + 2.3244006633758545, + 1.161316990852356, + -2.699451208114624, + 1.6399294137954712, + 0.31003186106681824, + 0.5301566123962402, + -0.16996291279792786, + 0.9281869530677795, + 0.3683878183364868, + -2.650294780731201, + -1.1098875999450684 + ], + [ + -0.6997023820877075, + 1.4820023775100708, + 1.3613333702087402, + -3.8556365966796875, + -0.5036565065383911, + 1.4896399974822998, + -2.322740077972412, + 0.03597790002822876, + -1.1858596801757812, + -0.018024958670139313, + 2.0489397048950195, + 0.8914362192153931, + -2.3395535945892334, + 1.1835731267929077, + 0.5559698343276978, + 0.5892979502677917, + 0.4496755301952362, + 0.6100684404373169, + 0.22263532876968384, + -3.0098915100097656, + -1.455836296081543 + ], + [ + -0.594323456287384, + 1.846432089805603, + 0.8105549812316895, + -2.6211562156677246, + 0.2629232406616211, + 1.4025108814239502, + -2.3828163146972656, + 0.530456006526947, + -1.3874722719192505, + 0.49274298548698425, + 1.5545053482055664, + 0.4927254915237427, + -3.1490466594696045, + 1.5643255710601807, + 0.5331913232803345, + -0.53307044506073, + 0.7433652281761169, + 0.6546258926391602, + -0.74479740858078, + -3.721141815185547, + -0.9948303699493408 + ], + [ + -0.4409070312976837, + 2.4138050079345703, + 1.3223544359207153, + -3.3717033863067627, + -0.03033369779586792, + 2.350304365158081, + -2.112061023712158, + -0.22452247142791748, + -1.420605182647705, + -0.026168949902057648, + 1.4026775360107422, + 0.9532885551452637, + -2.6276464462280273, + 2.1231069564819336, + -0.49286600947380066, + -0.3450928032398224, + 0.5427718758583069, + -0.03872644901275635, + -0.6895387172698975, + -2.357243061065674, + -2.3816022872924805 + ], + [ + 0.5156354308128357, + 2.2980997562408447, + 2.0945849418640137, + -3.690303087234497, + 1.0696191787719727, + 2.266188144683838, + -1.0399069786071777, + 0.012727499008178711, + -1.2163764238357544, + 0.5646419525146484, + 1.1664416790008545, + 1.563025951385498, + -3.425304412841797, + 0.5126914978027344, + -0.18885818123817444, + -0.5277795791625977, + -0.1741226613521576, + 0.2763793468475342, + 0.8411282896995544, + -3.2598915100097656, + -2.2270073890686035 + ], + [ + -1.2896747589111328, + 2.3828718662261963, + 1.191500186920166, + -3.9747776985168457, + 0.7604506015777588, + 2.298020839691162, + -1.0116384029388428, + 1.177309513092041, + -1.6719520092010498, + -0.4176469147205353, + 2.709542989730835, + 0.26378321647644043, + -2.119570255279541, + 0.7668893933296204, + 0.6772738099098206, + -0.5338667035102844, + -0.7434203028678894, + 0.2167283445596695, + 0.03269544243812561, + -3.895913600921631, + -0.9124676585197449 + ], + [ + -1.323930263519287, + 2.338547468185425, + 1.4313098192214966, + -2.960175037384033, + 1.1079072952270508, + 1.5746114253997803, + -1.8946435451507568, + -0.4213535189628601, + -1.922096848487854, + 0.6645004153251648, + 2.1448850631713867, + 0.2334994077682495, + -3.4603066444396973, + 1.5447293519973755, + 0.25094738602638245, + 0.7163193821907043, + -0.10547596216201782, + -0.42469942569732666, + -0.49689921736717224, + -2.8712027072906494, + -1.6389087438583374 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027082443237304688, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + -0.03311736881732941, + 1.162198781967163, + 0.0, + 0.043807268142700195, + -0.264798641204834, + 0.0069312527775764465, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + -0.01150114368647337, + -0.007074322551488876, + -0.018292216584086418, + -0.035625845193862915, + 0.05120308697223663, + -2.0730173587799072, + 0.0 + ] + ], + "predicted_scores": [ + 6.755773067474365, + 5.860231876373291, + 5.797102928161621, + 5.742587089538574, + 5.7348408699035645, + 5.577351093292236, + 5.468424320220947, + 5.338929653167725 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1919198188/train0", + "selected_index": 0, + "state_hash": "6b9e4c034c70f8e93116064dbcb48aa5fea37958e04fa30734f597c38aefea8b", + "stored_base_utility": 0.8558897972106934, + "task_id": "StackCube-v1", + "train_seed": "0" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7270061373710632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7270061373710632 + }, + "base_progress": 0.7270061373710632, + "base_success": false, + "base_utility": 0.7270061373710632, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7210943102836609, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7210943102836609 + }, + { + "contact_quality": null, + "progress": 0.2108287811279297, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2108287811279297 + }, + { + "contact_quality": null, + "progress": 0.5322304368019104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5322304368019104 + }, + { + "contact_quality": null, + "progress": 0.4597426652908325, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4597426652908325 + }, + { + "contact_quality": null, + "progress": 0.5059094429016113, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5059094429016113 + }, + { + "contact_quality": null, + "progress": 0.5588076114654541, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5588076114654541 + }, + { + "contact_quality": null, + "progress": 0.5301380157470703, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5301380157470703 + } + ], + "candidate_progress": [ + 1.0, + 0.7210943102836609, + 0.2108287811279297, + 0.5322304368019104, + 0.4597426652908325, + 0.5059094429016113, + 0.5588076114654541, + 0.5301380157470703 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-df961fe24a43e56d1716", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-484ea653bbca2de3a4a2", + "generated_tangents": [ + [ + -0.1740616410970688, + -0.16297969222068787, + 0.3772149682044983, + 0.7074242830276489, + 1.0587588548660278, + -0.018649250268936157, + -0.3348178267478943, + 0.004648327827453613, + -0.04392655938863754, + 0.20681168138980865, + -0.08656638115644455, + 1.245284914970398, + -0.06480005383491516, + 0.1428927332162857, + 0.2228441834449768, + -0.015324490144848824, + 0.0933811217546463, + 0.022266343235969543, + 1.2185115814208984, + 0.05619916319847107, + -0.020767703652381897 + ], + [ + -0.12723924219608307, + -0.04269175976514816, + 0.3771720826625824, + 0.6217846870422363, + 1.2183494567871094, + 0.1744009107351303, + -0.3855992555618286, + -0.025565162301063538, + -0.04204275459051132, + 0.1668231189250946, + -0.10222139209508896, + 1.104973316192627, + 0.021443843841552734, + 0.3351121246814728, + -0.0014446936547756195, + -0.05466555431485176, + 0.05066956579685211, + 0.1515975147485733, + 1.4147889614105225, + 0.030329227447509766, + 0.015080541372299194 + ], + [ + -0.21145252883434296, + -0.16480331122875214, + 0.3572320342063904, + 0.44956710934638977, + 1.2530062198638916, + 0.2183370441198349, + -0.15831303596496582, + -0.19049617648124695, + -0.03198334574699402, + 0.027901150286197662, + -0.02730824053287506, + 1.1245977878570557, + -0.09650805592536926, + 0.4930015802383423, + -0.05702989175915718, + -0.020286599174141884, + 0.1561012864112854, + 0.10981842875480652, + 1.1236357688903809, + 0.09788975119590759, + 0.21223600208759308 + ], + [ + -0.21923606097698212, + -0.08897554874420166, + 0.020573318004608154, + 0.4900379478931427, + 1.4295775890350342, + 0.36279985308647156, + -0.6743503212928772, + -0.1906130313873291, + -0.01927298493683338, + 0.1211085319519043, + -0.17763853073120117, + 0.7397709488868713, + -0.09032189846038818, + 0.2276231050491333, + -0.05368965491652489, + 0.10843819379806519, + 0.17454448342323303, + -0.09307925403118134, + 1.185486078262329, + -0.16541698575019836, + -0.05778945982456207 + ], + [ + -0.2743591070175171, + -0.21796829998493195, + 0.18193334341049194, + 0.5038225650787354, + 1.3598613739013672, + 0.12141576409339905, + -0.6088373064994812, + -0.1826959252357483, + -0.08674545586109161, + 0.08357211947441101, + -0.07634951919317245, + 1.0663490295410156, + -0.07500910758972168, + 0.36990493535995483, + 0.07827837765216827, + -0.07535171508789062, + -0.033485040068626404, + 0.03464619815349579, + 1.285482406616211, + -0.1284266859292984, + 0.15608970820903778 + ], + [ + -0.2402840405702591, + -0.16710659861564636, + 0.3115612268447876, + 0.5065975189208984, + 1.0599058866500854, + 0.15393267571926117, + -0.33368104696273804, + -0.19817450642585754, + 0.0020447690039873123, + 0.08052072674036026, + -0.04305257648229599, + 1.2484194040298462, + 0.007736295461654663, + 0.2714783549308777, + -0.04003826901316643, + -0.09442171454429626, + 0.11606928706169128, + 0.0512305349111557, + 1.2150667905807495, + 0.06886446475982666, + 0.08871621638536453 + ], + [ + -0.22926358878612518, + -0.12117899954319, + 0.3107870817184448, + 0.5271912813186646, + 1.063497543334961, + 0.049579598009586334, + -0.4694610834121704, + -0.2212059199810028, + -0.007535628974437714, + 0.10281668603420258, + -0.14292535185813904, + 1.1758768558502197, + -0.0920785665512085, + 0.1954551786184311, + -0.09687457978725433, + -0.03090819902718067, + 0.2185717076063156, + 0.07259805500507355, + 1.1084318161010742, + 0.046909987926483154, + 0.0721399113535881 + ], + [ + -0.26756858825683594, + -0.1288309246301651, + 0.2943589389324188, + 0.6360929012298584, + 1.0180401802062988, + -0.037748053669929504, + -0.28414690494537354, + -0.09688475728034973, + 0.03188071772456169, + 0.1313062161207199, + -0.17894881963729858, + 1.2895761728286743, + 0.04807606339454651, + 0.30058038234710693, + -0.22448518872261047, + -0.05610359460115433, + 0.18521246314048767, + 0.12355383485555649, + 1.268186092376709, + -0.10017992556095123, + 0.07076788693666458 + ] + ], + "generated_utilities": [ + 2.0, + 0.7210943102836609, + 0.2108287811279297, + 0.5322304368019104, + 0.4597426652908325, + 0.5059094429016113, + 0.5588076114654541, + 0.5301380157470703 + ], + "group_id": "ms-484ea653bbca2de3a4a2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7242962718009949, + 0.7209752798080444, + 2.0, + 0.19684189558029175, + 0.21780464053153992, + 0.19326990842819214, + 0.22509369254112244, + 0.20856516063213348, + 0.21041584014892578, + 0.19898515939712524, + 0.20721960067749023, + 0.20830264687538147, + 0.19554394483566284, + 0.22549311816692352, + 0.19481703639030457, + 0.2172507494688034 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.4380870461463928, + 0.4186934530735016, + 0.6294705867767334, + -0.08003801107406616, + 0.5882754325866699, + 0.006411761045455933, + 0.0, + 0.27439412474632263, + -0.43702489137649536, + -0.26345914602279663, + -0.3102925717830658, + 1.1419475078582764, + -0.05133828520774841, + 0.0, + 0.11964668333530426, + -0.04283810034394264, + 0.021698474884033203, + -0.3213534653186798, + 1.444929838180542, + 0.058529436588287354, + 0.0 + ], + [ + 0.18277202546596527, + -0.31313952803611755, + 1.5374140739440918, + 0.9199619889259338, + 1.58827543258667, + -0.42588773369789124, + -1.307411789894104, + -0.1648276150226593, + 0.17999683320522308, + 0.7365408539772034, + -0.2825929522514343, + 2.1419475078582764, + -0.44205278158187866, + 0.3557547330856323, + 0.11787930130958557, + 0.04347073659300804, + 0.5306029915809631, + -0.6201959848403931, + 2.444929838180542, + -0.7078790068626404, + 0.7410526275634766 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.614823579788208, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7115094661712646, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4821052551269531 + ], + [ + 0.2415536344051361, + -0.2574644386768341, + 1.3600168228149414, + 0.8914169669151306, + 1.311052918434143, + -0.13483268022537231, + -1.4109176397323608, + -0.007070749998092651, + 0.30199557542800903, + 0.867316484451294, + -0.0111103355884552, + 2.228789806365967, + -0.26305240392684937, + 0.4046628475189209, + 0.25534987449645996, + 0.29946526885032654, + 0.777583122253418, + -0.535997748374939, + 2.2955121994018555, + -0.6670190095901489, + 0.8460665941238403 + ], + [ + -0.212661013007164, + -0.6653347015380859, + 1.8750245571136475, + 0.7373600006103516, + 1.2695744037628174, + -0.7584145069122314, + -1.695502758026123, + 0.05721297860145569, + -0.04310476779937744, + 0.4696756601333618, + -0.2954420745372772, + 1.9897286891937256, + -0.6554491519927979, + 0.5805841088294983, + 0.17026324570178986, + 0.3433569669723511, + 0.3734014630317688, + -0.8102566003799438, + 2.792973279953003, + -0.8463170528411865, + 1.0759955644607544 + ], + [ + -0.039009034633636475, + -0.7773740291595459, + 1.064732313156128, + 0.712108850479126, + 1.270837426185608, + -0.5491334199905396, + -1.7919384241104126, + -0.5782418251037598, + 0.4058297872543335, + 0.5231752395629883, + -0.521438479423523, + 2.2066850662231445, + -0.5258626341819763, + -0.09100466966629028, + 0.24944378435611725, + -0.2271452248096466, + 0.9009795784950256, + -0.8140718340873718, + 2.6494152545928955, + -0.5153427124023438, + 0.671539306640625 + ], + [ + -0.3804699182510376, + -0.5808351039886475, + 1.8389031887054443, + 0.7141245603561401, + 1.0997881889343262, + -0.1678517758846283, + -1.884540319442749, + -0.3411717414855957, + 0.4830858111381531, + 0.8356180787086487, + -0.8238832950592041, + 1.743321418762207, + -0.3751307427883148, + 0.5275082588195801, + -0.08622422814369202, + 0.23666565120220184, + 0.802552342414856, + -0.8389871120452881, + 1.9817423820495605, + -0.4874984323978424, + 0.4390762150287628 + ], + [ + 0.4497724771499634, + -0.7880887389183044, + 1.4436085224151611, + 1.083756446838379, + 2.2298290729522705, + -0.03008332848548889, + -1.0250177383422852, + -0.6266056299209595, + 0.3751387298107147, + 1.1566712856292725, + -0.6914000511169434, + 1.953734278678894, + 0.08129626512527466, + 0.9812837839126587, + -0.4981238543987274, + 0.6697847247123718, + -0.05150604248046875, + -0.06941354274749756, + 1.9621039628982544, + -1.345557451248169, + 0.43397778272628784 + ], + [ + 0.03950639069080353, + -1.037135124206543, + 1.1041697263717651, + 1.109439730644226, + 1.144017219543457, + -0.07355684041976929, + -0.8942177295684814, + -0.16419151425361633, + -0.12125559151172638, + 0.36935141682624817, + -0.7298048734664917, + 1.4954023361206055, + -0.018926143646240234, + 0.4787553548812866, + -0.3127076327800751, + 0.6351315379142761, + 0.5993886590003967, + -0.5143972635269165, + 2.429173469543457, + -0.6674217581748962, + 0.5048977732658386 + ], + [ + 0.9763806462287903, + -1.1549197435379028, + 2.0378684997558594, + 1.2418556213378906, + 2.427785634994507, + -1.1326193809509277, + -1.5491721630096436, + -0.8937685489654541, + 0.2583051919937134, + -0.10687285661697388, + 0.2628750801086426, + 1.6517728567123413, + -0.21639958024024963, + 0.7581590414047241, + -0.29044294357299805, + -0.08927425742149353, + 1.3009071350097656, + -0.9239004850387573, + 2.5070056915283203, + -1.3246500492095947, + 0.24751555919647217 + ], + [ + 0.9781984090805054, + -0.13759100437164307, + 1.8698725700378418, + 1.4942535161972046, + 2.5587658882141113, + -1.1335766315460205, + -0.6910277605056763, + -0.1434302181005478, + 0.4021325707435608, + -0.12275522947311401, + 0.6571680307388306, + 1.9964357614517212, + -0.1442476212978363, + 1.3201656341552734, + 0.2586786448955536, + -0.38222962617874146, + 0.9950553774833679, + -0.8371140956878662, + 1.8977141380310059, + -0.795723021030426, + 1.505234956741333 + ], + [ + -0.6250126361846924, + 0.4231366813182831, + 1.701646089553833, + 1.6233667135238647, + 1.0861127376556396, + -0.3387698233127594, + -0.9825164079666138, + -1.1511213779449463, + 0.11523068696260452, + 1.5823121070861816, + 0.5840664505958557, + 2.145885705947876, + -1.2145874500274658, + 1.1558246612548828, + -0.5131542682647705, + -0.4715161919593811, + 1.0824741125106812, + -0.7667124271392822, + 1.8457963466644287, + -1.228658676147461, + 1.2112116813659668 + ], + [ + 1.1640653610229492, + -1.1697030067443848, + 1.6035369634628296, + 0.2161661982536316, + 1.4819366931915283, + 0.2569611370563507, + -1.3529595136642456, + 0.41208329796791077, + 0.03121529519557953, + 1.731370210647583, + -0.8131548166275024, + 2.996675729751587, + -1.1173620223999023, + 0.7474988102912903, + 0.21459203958511353, + -0.2133086621761322, + -0.22307389974594116, + -0.6306187510490417, + 2.0490896701812744, + -0.49808698892593384, + 0.8432145714759827 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.014628380537033081, + -0.18207009136676788, + 0.5374140739440918, + -0.04374772310256958, + 0.5882754325866699, + 0.25251278281211853, + 0.0, + -0.23516394197940826, + 0.12562739849090576, + 0.3381730020046234, + 0.033050984144210815, + 1.1419475078582764, + -0.3290551006793976, + 0.0, + 0.04625481367111206, + -0.0026901662349700928, + 0.0754578709602356, + -0.1481514573097229, + 1.444929838180542, + 0.049897611141204834, + 0.0 + ] + ], + "predicted_scores": [ + 4.409553050994873, + 3.6382431983947754, + 3.63432240486145, + 3.5137364864349365, + 3.2669029235839844, + 3.1935641765594482, + 3.182760238647461, + 2.740438222885132 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1231780997/train1", + "selected_index": 0, + "state_hash": "771fb752a186163f05a49a9b404284bde55d89838769a764fca1d6d1a9419652", + "stored_base_utility": 0.7242962718009949, + "task_id": "LiftPegUpright-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.528373658657074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.528373658657074 + }, + "base_progress": 0.528373658657074, + "base_success": false, + "base_utility": 0.528373658657074, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.21175099909305573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21175099909305573 + }, + { + "contact_quality": null, + "progress": 0.20219288766384125, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20219288766384125 + }, + { + "contact_quality": null, + "progress": 0.19861702620983124, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19861702620983124 + }, + { + "contact_quality": null, + "progress": 0.20537595450878143, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20537595450878143 + }, + { + "contact_quality": null, + "progress": 0.19838964939117432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19838964939117432 + }, + { + "contact_quality": null, + "progress": 0.2148684412240982, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2148684412240982 + }, + { + "contact_quality": null, + "progress": 0.2115357667207718, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2115357667207718 + }, + { + "contact_quality": null, + "progress": 0.2025710493326187, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2025710493326187 + } + ], + "candidate_progress": [ + 0.21175099909305573, + 0.20219288766384125, + 0.19861702620983124, + 0.20537595450878143, + 0.19838964939117432, + 0.2148684412240982, + 0.2115357667207718, + 0.2025710493326187 + ], + "candidate_source_chart_ids": [ + "ms-df961fe24a43e56d1716", + "ms-13d8bae55d3c3710138f", + "ms-502dba1003c09ec60ba0", + "ms-0660d5f65df5fc4b6132", + "ms-8f3d4e1d07bb5dcc7c57", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb", + "ms-e1b6bcb41532a800d3cb" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank6", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-4b4cd12d70c0615e0930", + "generated_tangents": [ + [ + -0.12567324936389923, + -0.054757699370384216, + 1.0230728387832642, + 0.8730899095535278, + 0.21579238772392273, + 0.4350303113460541, + 0.12024950981140137, + 0.12823566794395447, + 0.01874651573598385, + -0.24843716621398926, + -0.25908493995666504, + 1.3926934003829956, + -0.17717412114143372, + 0.11273403465747833, + 0.09128392487764359, + -0.11082588136196136, + 0.130533367395401, + -0.07427425682544708, + 1.185415506362915, + -0.3137575387954712, + -0.017678260803222656 + ], + [ + -0.12893421947956085, + -0.14310193061828613, + 1.139750599861145, + 0.8206130266189575, + 0.20182399451732635, + 0.28020334243774414, + 0.2846207618713379, + -0.021337375044822693, + 0.12925907969474792, + -0.18140903115272522, + -0.21781983971595764, + 1.3943719863891602, + -0.07744288444519043, + 0.08842577040195465, + 0.07450462877750397, + -0.08268576860427856, + 0.08181801438331604, + -0.08562545478343964, + 1.1875466108322144, + -0.07574416697025299, + -0.08042079210281372 + ], + [ + -0.4525997042655945, + -0.13613227009773254, + 0.6503928303718567, + 0.931410551071167, + 0.35046908259391785, + 0.9532989263534546, + -0.06574034690856934, + 0.012613564729690552, + 0.042612239718437195, + -0.17629987001419067, + -0.01874808967113495, + 1.4223198890686035, + -0.21591919660568237, + 0.19703838229179382, + -0.11318287253379822, + -0.05161397159099579, + 0.24200108647346497, + -0.19728456437587738, + 1.2009154558181763, + -0.18358485400676727, + -0.12039703130722046 + ], + [ + -0.010334953665733337, + -0.08441674709320068, + 1.1696398258209229, + 0.9284533262252808, + 0.26577818393707275, + 0.2814810872077942, + 0.18289008736610413, + 0.05493806302547455, + 0.11425623297691345, + -0.25960254669189453, + -0.3063563406467438, + 1.5965182781219482, + -0.11598813533782959, + 0.14736324548721313, + 0.09953732788562775, + -0.06987813115119934, + 0.08677676320075989, + -0.07662118971347809, + 1.0562846660614014, + -0.10489632189273834, + -0.24773052334785461 + ], + [ + -0.23086850345134735, + -0.15534354746341705, + 1.0204118490219116, + 1.0651427507400513, + 0.17302680015563965, + 0.35944873094558716, + -0.08795249462127686, + -0.09896808862686157, + 0.11917176097631454, + -0.17487865686416626, + -0.12784412503242493, + 1.5219006538391113, + 0.06432041525840759, + 0.1888321042060852, + -0.0535670630633831, + -0.12329638004302979, + 0.20388533174991608, + -0.09481458365917206, + 1.2969107627868652, + -0.14185336232185364, + 0.1616964340209961 + ], + [ + -0.08113265037536621, + -0.05291309952735901, + 0.9791842103004456, + 0.922683835029602, + 0.3093123435974121, + 0.33274784684181213, + -0.05834740400314331, + 0.048475250601768494, + 0.10372401773929596, + -0.2137639820575714, + -0.28827419877052307, + 1.6551964282989502, + -0.11236998438835144, + 0.48802006244659424, + 0.04900050908327103, + -0.1199604868888855, + 0.20941756665706635, + 0.03269365429878235, + 1.160142183303833, + -0.09298726916313171, + -0.13338792324066162 + ], + [ + -0.18320231139659882, + -0.19712179899215698, + 1.0537443161010742, + 0.8514035940170288, + 0.14848870038986206, + 0.31337594985961914, + 0.08552402257919312, + -0.005987539887428284, + 0.17298458516597748, + -0.2623000144958496, + -0.22141033411026, + 1.4968657493591309, + -0.3683769106864929, + 0.4715902805328369, + 0.1347920298576355, + -0.15936803817749023, + -0.08030088245868683, + -0.10295622050762177, + 1.1988164186477661, + -0.12891122698783875, + 0.13912439346313477 + ], + [ + 0.03518760949373245, + -0.17133961617946625, + 1.2623594999313354, + 0.9834651947021484, + 0.05818794667720795, + 0.12489740550518036, + 0.09438782930374146, + 0.08768085390329361, + 0.23926232755184174, + -0.20809391140937805, + -0.37154942750930786, + 1.4849345684051514, + 0.056320399045944214, + 0.32554954290390015, + 0.0560346283018589, + -0.08457501232624054, + 0.021852806210517883, + -0.11967034637928009, + 1.2465505599975586, + 0.10155147314071655, + -0.07965892553329468 + ] + ], + "generated_utilities": [ + 0.21175099909305573, + 0.20219288766384125, + 0.19861702620983124, + 0.20537595450878143, + 0.19838964939117432, + 0.2148684412240982, + 0.2115357667207718, + 0.2025710493326187 + ], + "group_id": "ms-4b4cd12d70c0615e0930", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7225138545036316, + 2.0, + 0.6951258182525635, + 0.18915215134620667, + 0.20867900550365448, + 0.20041285455226898, + 0.20335882902145386, + 0.20815956592559814, + 0.2024221420288086, + 0.1940193623304367, + 0.20689067244529724, + 0.19995209574699402, + 0.21901433169841766, + 0.19443893432617188, + 0.20502614974975586, + 0.1959807276725769 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.09018868207931519, + 0.3819495439529419, + 0.9586174488067627, + 1.0247726440429688, + -0.179018035531044, + 0.125870943069458, + 0.0, + -0.1166897565126419, + 0.07711493968963623, + -0.24461698532104492, + -0.005436837673187256, + 1.737370491027832, + -0.4425608515739441, + 0.0, + 0.24725346267223358, + -0.03395397961139679, + 0.643236517906189, + 0.05422210693359375, + 1.2738127708435059, + 0.18164724111557007, + 0.0 + ], + [ + 0.6095315217971802, + 0.35729360580444336, + 1.9586174488067627, + 2.0247726440429688, + -0.2820541560649872, + 1.044631004333496, + -2.239265203475952, + -0.14250506460666656, + -0.4363628029823303, + -1.244616985321045, + -0.32413747906684875, + 2.737370491027832, + 0.24581307172775269, + -0.053983256220817566, + 0.2877395749092102, + 0.18538890779018402, + 0.45247530937194824, + -0.9457778930664062, + 2.273812770843506, + -0.8183527588844299, + 0.5866778492927551 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -4.478530406951904, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.10796651244163513, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1733556985855103 + ], + [ + 0.8415398597717285, + 0.17323988676071167, + 1.9669679403305054, + 2.188286304473877, + -0.386857271194458, + 1.3415166139602661, + -2.06968355178833, + -0.38555508852005005, + -0.6971946954727173, + -1.0110756158828735, + -0.19943451881408691, + 3.0229673385620117, + 0.03575621545314789, + -0.26961493492126465, + 0.5202459692955017, + 0.0921676978468895, + 0.7439858913421631, + -0.724668562412262, + 2.116668939590454, + -1.0585715770721436, + 0.7851569652557373 + ], + [ + 0.6909375190734863, + -0.018118590116500854, + 1.677350401878357, + 1.9505902528762817, + 0.11725521087646484, + 1.2839276790618896, + -2.0543243885040283, + -0.38843441009521484, + -0.6865814328193665, + -1.5103397369384766, + -0.2952713072299957, + 2.384809970855713, + -0.10558199882507324, + 0.29018354415893555, + 0.33942633867263794, + -0.19196103513240814, + 0.35257238149642944, + -0.8657878637313843, + 2.5445473194122314, + -0.706183910369873, + 0.7214455008506775 + ], + [ + 0.8977934122085571, + 0.7311913371086121, + 2.203964948654175, + 1.686698079109192, + -0.338996022939682, + 0.631161630153656, + -2.3877103328704834, + 0.07195566594600677, + -0.6892462968826294, + -0.9634823799133301, + -0.5621621012687683, + 2.479537010192871, + 0.675940752029419, + -0.047649919986724854, + 0.14082925021648407, + 0.19517391920089722, + 0.15626585483551025, + -1.3974462747573853, + 2.759931802749634, + -1.166107416152954, + 0.18018442392349243 + ], + [ + 0.7796773910522461, + 0.06440207362174988, + 1.5159449577331543, + 1.6007665395736694, + -0.12459242343902588, + 1.5781673192977905, + -2.5707991123199463, + 0.034798189997673035, + -0.44772639870643616, + -1.3837177753448486, + -0.5199699401855469, + 3.1566030979156494, + -0.22805768251419067, + -0.5881848931312561, + 0.49021050333976746, + -0.08376060426235199, + 0.5395568609237671, + -1.3038995265960693, + 2.37483811378479, + -0.615712583065033, + 0.7884383201599121 + ], + [ + 0.4830939471721649, + -0.12824741005897522, + 2.133068561553955, + 2.42085337638855, + -0.1641906201839447, + 1.6598048210144043, + -2.5325045585632324, + 0.06623834371566772, + 0.14292877912521362, + -1.85994553565979, + 0.01158285140991211, + 3.2622194290161133, + -0.017848491668701172, + -0.08653604984283447, + 0.2199593186378479, + -0.0463244765996933, + -0.04547742009162903, + -0.5160569548606873, + 2.1955723762512207, + -0.364714115858078, + 1.0323584079742432 + ], + [ + 0.7637265920639038, + -0.4020084738731384, + 2.4714059829711914, + 1.33060622215271, + 0.1464400589466095, + 0.7656213045120239, + -1.9972842931747437, + 0.42333751916885376, + -1.118815302848816, + -1.5639967918395996, + -0.32944348454475403, + 2.0334420204162598, + 0.37165334820747375, + -0.2861292064189911, + 0.6376298666000366, + 0.8521975874900818, + 0.6845490336418152, + -1.515796422958374, + 1.5925071239471436, + -0.4112323224544525, + 1.1011840105056763 + ], + [ + 1.456933617591858, + 1.0030901432037354, + 1.1997551918029785, + 1.7652757167816162, + 0.03292572498321533, + 1.302201271057129, + -2.748408317565918, + 0.7148768305778503, + -0.5089412331581116, + -0.5701506733894348, + -0.40410247445106506, + 3.586395502090454, + -0.37424683570861816, + -0.6850852966308594, + 0.9945629835128784, + 0.5821471810340881, + 0.8666700124740601, + -1.1607441902160645, + 2.1460094451904297, + -0.7912135720252991, + 0.6301633715629578 + ], + [ + -0.09475094079971313, + -0.28330767154693604, + 1.8353716135025024, + 2.3373594284057617, + 0.7110240459442139, + 1.9665697813034058, + -1.8502212762832642, + -0.40577346086502075, + -0.03506562113761902, + -0.7167142033576965, + -0.39153769612312317, + 2.3472883701324463, + 0.08497899770736694, + -0.9910531640052795, + -0.32254326343536377, + -0.11388416588306427, + -0.4873260259628296, + -0.8018304109573364, + 2.295731782913208, + -1.325164794921875, + 1.2513983249664307 + ], + [ + -0.32657402753829956, + 0.8198520541191101, + 1.211392879486084, + 1.974273443222046, + -0.11595214903354645, + 1.954041838645935, + -1.5547080039978027, + 0.06433123350143433, + -0.079216867685318, + -0.5887003540992737, + -0.6015424728393555, + 2.756274938583374, + 0.641853928565979, + -0.8642687201499939, + 0.07965350151062012, + 0.11391039192676544, + 1.10663902759552, + -1.506749153137207, + 3.222418785095215, + -1.0799858570098877, + -0.12158209085464478 + ], + [ + 0.9412136077880859, + -0.5255860090255737, + 1.9096710681915283, + 2.113077402114868, + -1.0082788467407227, + 0.1505374312400818, + -3.0413341522216797, + -0.24721917510032654, + -0.20225940644741058, + -1.6247544288635254, + -0.33320704102516174, + 1.8443769216537476, + 0.7569764256477356, + -0.5270920395851135, + 1.1813421249389648, + 1.0764858722686768, + 0.01533237099647522, + -1.4125463962554932, + 1.82404363155365, + 0.054037630558013916, + 0.45250433683395386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.056152522563934326, + 0.005679905414581299, + 0.9586174488067627, + 1.0247726440429688, + -0.054595887660980225, + 0.044631004333496094, + 0.0, + -0.09190673381090164, + 0.09592300653457642, + -0.24461698532104492, + -0.007907181978225708, + 1.737370491027832, + -0.029382914304733276, + 0.0, + 0.06305502355098724, + 0.03764393925666809, + 0.0785532295703888, + 0.005711078643798828, + 1.2738127708435059, + -0.04058480262756348, + 0.0 + ] + ], + "predicted_scores": [ + 0.612500786781311, + 0.4029788672924042, + 0.06764078140258789, + -0.08695489168167114, + -0.1073904037475586, + -0.16207697987556458, + -0.37850794196128845, + -0.9772553443908691 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1025740212/train1", + "selected_index": 0, + "state_hash": "5b394fca1712c0667d126198527a4f0ba81893d536a5fcbf7b90f5b13fa703dc", + "stored_base_utility": 0.7225138545036316, + "task_id": "LiftPegUpright-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7181311249732971, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7181311249732971 + }, + { + "contact_quality": null, + "progress": 0.655661404132843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.655661404132843 + }, + { + "contact_quality": null, + "progress": 0.6736156344413757, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6736156344413757 + }, + { + "contact_quality": null, + "progress": 0.7093296051025391, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7093296051025391 + }, + { + "contact_quality": null, + "progress": 0.7143670916557312, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7143670916557312 + }, + { + "contact_quality": null, + "progress": 0.5601552128791809, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5601552128791809 + }, + { + "contact_quality": null, + "progress": 0.7090306282043457, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7090306282043457 + }, + { + "contact_quality": null, + "progress": 0.5006552338600159, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5006552338600159 + } + ], + "candidate_progress": [ + 0.7181311249732971, + 0.655661404132843, + 0.6736156344413757, + 0.7093296051025391, + 0.7143670916557312, + 0.5601552128791809, + 0.7090306282043457, + 0.5006552338600159 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6e268d1e2915740816a1", + "generated_tangents": [ + [ + -0.0008221715688705444, + -0.15447860956192017, + 0.06838101148605347, + 0.3342396318912506, + 1.789905071258545, + 0.028334319591522217, + -0.055450379848480225, + 0.0549965500831604, + -0.048723749816417694, + 0.21258844435214996, + -0.10498260706663132, + 0.9088835716247559, + -0.0790761411190033, + 0.09338332712650299, + 0.08490000665187836, + 0.0025108912959694862, + 0.03347918391227722, + -0.059780195355415344, + 0.752911388874054, + -0.1534477174282074, + -0.13718003034591675 + ], + [ + -0.16093336045742035, + -0.11970759928226471, + 0.17358586192131042, + 0.05782929062843323, + 1.6275742053985596, + -0.09557829797267914, + 0.18253520131111145, + -0.15749138593673706, + 0.043381333351135254, + 0.12774048745632172, + 0.12142147123813629, + 0.9491201043128967, + -0.02470564842224121, + -0.012363627552986145, + -0.28332972526550293, + -0.02575576677918434, + 0.014457911252975464, + -0.098793163895607, + 0.9115554094314575, + -0.36201727390289307, + -0.13929247856140137 + ], + [ + -0.24218355119228363, + -0.0708344355225563, + 0.12442180514335632, + -0.18104180693626404, + 1.7634239196777344, + -0.052961185574531555, + -0.30474936962127686, + 0.026587843894958496, + -0.08119779080152512, + 0.07087457180023193, + 0.11938828229904175, + 0.9018123745918274, + -0.07314527034759521, + 0.40944308042526245, + -0.1938321590423584, + -0.021801553666591644, + 0.07719245553016663, + 0.01955544948577881, + 0.7524850368499756, + -0.05436146259307861, + -0.09793844819068909 + ], + [ + -0.05114002525806427, + -0.10038959980010986, + 0.006561249494552612, + 0.12133711576461792, + 1.7032814025878906, + 6.608664989471436e-05, + -0.0943724513053894, + -0.13517552614212036, + -0.028819995000958443, + 0.1371464729309082, + -0.12953326106071472, + 0.9723052382469177, + -0.0935530960559845, + 0.12432572990655899, + -0.21406587958335876, + -0.031927481293678284, + 0.16789288818836212, + 0.07786032557487488, + 0.7261624336242676, + -0.14685063064098358, + -0.10996684432029724 + ], + [ + -0.04979391396045685, + -0.11791586130857468, + -0.004456192255020142, + 0.05855506658554077, + 1.7235608100891113, + 0.11105094105005264, + -0.024040699005126953, + -0.12752288579940796, + -0.014224421232938766, + 0.07338318228721619, + 0.010501477867364883, + 1.0183629989624023, + 0.0008610785007476807, + 0.2317407727241516, + -0.18003568053245544, + -0.062449511140584946, + 0.06018295884132385, + 0.04666796326637268, + 0.8320387005805969, + -0.10812510550022125, + -0.11861711740493774 + ], + [ + -0.11588315665721893, + -0.1507032811641693, + 0.13556641340255737, + 0.21135267615318298, + 1.7329940795898438, + 0.10381265729665756, + 0.4070972204208374, + -0.1715426743030548, + -0.005405336618423462, + 0.11465444415807724, + 0.0011274665594100952, + 0.9490469098091125, + -0.02240428328514099, + 0.30045509338378906, + -0.1598256230354309, + -0.02153543382883072, + 0.1829117089509964, + 0.0506272166967392, + 0.8583177924156189, + -0.06775277853012085, + 0.08185550570487976 + ], + [ + -0.12218575179576874, + -0.05966152995824814, + -0.03340598940849304, + 0.17769727110862732, + 1.6792542934417725, + -0.022392183542251587, + 0.007063388824462891, + -0.023685961961746216, + 0.008037716150283813, + 0.13172943890094757, + -0.10718705505132675, + 1.0250279903411865, + -0.05941393971443176, + 0.2312353253364563, + -0.3399696350097656, + -0.00429912656545639, + 0.15994682908058167, + 0.10264082252979279, + 0.8822011947631836, + -0.2537164092063904, + -0.16122901439666748 + ], + [ + -0.3161572813987732, + -0.08427201211452484, + 0.16332033276557922, + -0.06050175428390503, + 1.7552568912506104, + -0.23629158735275269, + 0.22806505858898163, + -0.0823492705821991, + 0.0047934032045304775, + -0.02825797349214554, + -0.10792965441942215, + 1.0314810276031494, + 0.23326674103736877, + 0.4444473385810852, + -0.30170097947120667, + -0.020457323640584946, + 0.05768272280693054, + -0.07658644020557404, + 1.0223991870880127, + -0.09513147175312042, + 0.1868864893913269 + ] + ], + "generated_utilities": [ + 0.7181311249732971, + 0.655661404132843, + 0.6736156344413757, + 0.7093296051025391, + 0.7143670916557312, + 0.5601552128791809, + 0.7090306282043457, + 0.5006552338600159 + ], + "group_id": "ms-6e268d1e2915740816a1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7257392406463623, + 0.7165254950523376, + 2.0, + 0.72465980052948, + 0.7087006568908691, + 0.2306610643863678, + 0.7143489122390747, + 0.7045385837554932, + 0.5905889272689819, + 0.1919487714767456, + 0.1914421170949936, + 0.2230607271194458, + 0.20312398672103882, + 0.19757536053657532, + 0.19724789261817932, + 0.19653666019439697 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.44810131192207336, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0264720916748047, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.9882198572158813 + ], + [ + -0.4707597494125366, + 0.09937268495559692, + 0.6395220160484314, + -0.40935325622558594, + 2.4514670372009277, + -0.02397046983242035, + -0.21874001622200012, + 0.16799744963645935, + 0.0360843725502491, + -0.09719255566596985, + -0.8619975447654724, + 2.2499821186065674, + -0.4933375120162964, + 0.35259634256362915, + -0.34735018014907837, + -0.517992377281189, + 0.6015943288803101, + -0.7677568793296814, + 1.5267367362976074, + -0.16888000071048737, + 0.3771851658821106 + ], + [ + -0.8530820608139038, + -0.6707327961921692, + 0.7970488667488098, + 0.34902632236480713, + 2.1393990516662598, + 0.5651285648345947, + 0.113044872879982, + 0.2987540364265442, + 0.4139185845851898, + 0.41691792011260986, + -0.4906405806541443, + 1.5687955617904663, + -0.8899085521697998, + -0.01883232593536377, + 0.24473188817501068, + 0.554391622543335, + 0.23424841463565826, + -0.10042615234851837, + 2.1214542388916016, + -0.15780393779277802, + 0.5807479023933411 + ], + [ + -0.6562768220901489, + -0.059059903025627136, + 0.24766850471496582, + 0.06101328134536743, + 2.5052053928375244, + 0.5204365849494934, + 0.1595638245344162, + 0.47176462411880493, + 0.3036114573478699, + 0.13197286427021027, + -0.2551935017108917, + 2.185537099838257, + -1.0984127521514893, + 0.3213496804237366, + 0.49875718355178833, + 0.1223822832107544, + 0.2575722932815552, + -0.7017092704772949, + 1.7495591640472412, + -0.19501343369483948, + 1.1214004755020142 + ], + [ + -0.45427173376083374, + -0.041353583335876465, + 0.8143320083618164, + -0.39680930972099304, + 2.4542014598846436, + 0.382559597492218, + -0.9993512630462646, + 0.5202391147613525, + 0.5282149314880371, + 0.4165073335170746, + 0.43153735995292664, + 2.5625123977661133, + -0.721943199634552, + 0.6765530109405518, + -0.6697012186050415, + -0.0637422502040863, + 0.6919896602630615, + -1.033813714981079, + 2.317282199859619, + -0.13503669202327728, + -0.07009828090667725 + ], + [ + -0.548885703086853, + -0.609938383102417, + 0.7181543707847595, + 0.11203774809837341, + 2.478057861328125, + -0.6170704960823059, + -1.0383654832839966, + 0.3524467349052429, + 0.5748242139816284, + 0.42987847328186035, + 0.013119548559188843, + 1.3954555988311768, + -0.6403790712356567, + 0.025414258241653442, + -0.7780460119247437, + -0.23124903440475464, + 0.48347827792167664, + 0.05266502499580383, + 1.692553997039795, + 0.362108051776886, + 0.6330842971801758 + ], + [ + -1.3070502281188965, + -0.947432816028595, + 0.4198402762413025, + -1.1768442392349243, + 1.7843327522277832, + 0.7587301731109619, + 0.7447960376739502, + 0.9411643147468567, + 0.3013244867324829, + -0.3862171769142151, + -1.0263384580612183, + 2.97957181930542, + 0.1849566102027893, + 0.08799406886100769, + 0.1984272301197052, + 0.5277765393257141, + 0.2931878864765167, + -1.1385071277618408, + 1.884846568107605, + 0.23083388805389404, + 0.6359583139419556 + ], + [ + -0.5375385880470276, + 0.16159841418266296, + 1.260543942451477, + -0.6744770407676697, + 2.1646244525909424, + 0.6190755367279053, + -0.746461808681488, + -0.38674360513687134, + 0.8897944092750549, + -0.594306468963623, + -0.05647379159927368, + 1.7390328645706177, + -0.037059009075164795, + 0.470071941614151, + 0.010003047063946724, + 0.08421775698661804, + -0.4613104462623596, + -0.22854508459568024, + 1.772307276725769, + -0.29165884852409363, + 0.6179300546646118 + ], + [ + -0.6489957571029663, + 0.4526708126068115, + 0.5115790367126465, + 0.31459903717041016, + 2.9655239582061768, + 0.07586031407117844, + -1.1667817831039429, + -0.7146965861320496, + 1.0295621156692505, + 0.1932714432477951, + -0.2703063488006592, + 2.9046523571014404, + -1.5882594585418701, + 0.3030351996421814, + 0.6543201208114624, + -0.448961466550827, + 0.4115087389945984, + -0.4087216854095459, + 2.6432881355285645, + -0.6797424554824829, + 0.2669999599456787 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.22510427236557007, + -0.26737168431282043, + 0.10375632345676422, + 0.12666502594947815, + 1.379244327545166, + -0.01088210940361023, + 0.0, + 0.23721180856227875, + -0.10981421172618866, + -0.021816372871398926, + -0.071512371301651, + 1.0360438823699951, + 0.09019589424133301, + 0.0, + 0.03181244805455208, + -0.06466590613126755, + 0.2068846970796585, + 0.14229199290275574, + 0.7610219717025757, + -0.03211131691932678, + 0.0 + ] + ], + "predicted_scores": [ + 4.490880966186523, + 4.102152347564697, + 3.7473788261413574, + 3.7030088901519775, + 3.611773729324341, + 3.593114137649536, + 3.482645034790039, + 3.4754390716552734 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1475440973/train1", + "selected_index": 0, + "state_hash": "38ff5a264c3c39ff1ec8756d84e622963a11ff1ac61568ba57fbea0c114c3140", + "stored_base_utility": 0.7257392406463623, + "task_id": "LiftPegUpright-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7239725589752197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7239725589752197 + }, + "base_progress": 0.7239725589752197, + "base_success": false, + "base_utility": 0.7239725589752197, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7220139503479004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7220139503479004 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.650579035282135, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.650579035282135 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7230307459831238, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7230307459831238 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.7220139503479004, + 1.0, + 1.0, + 0.650579035282135, + 1.0, + 0.7230307459831238, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-8864796e31bbe93d7c5d", + "generated_tangents": [ + [ + 0.08788160979747772, + -0.2383939027786255, + -0.032137900590896606, + -0.24412868916988373, + 1.4860432147979736, + -0.21342450380325317, + 0.09996834397315979, + 0.1382390707731247, + -0.15099269151687622, + -0.020627014338970184, + -0.142374187707901, + 1.287447214126587, + -0.051568299531936646, + 0.22556555271148682, + -0.1834896206855774, + -0.0788751095533371, + 0.1916874647140503, + 0.022342637181282043, + 1.0566089153289795, + -0.5432512760162354, + -0.18783268332481384 + ], + [ + -0.05326157808303833, + -0.22410079836845398, + 0.09589865803718567, + -0.13422852754592896, + 1.596210241317749, + -0.4352465569972992, + 0.4212307929992676, + 0.16992929577827454, + -0.09806247055530548, + -0.0010492801666259766, + 0.17228838801383972, + 1.011678695678711, + -0.0015822947025299072, + 0.22310665249824524, + -0.19998541474342346, + -0.005391527898609638, + 0.14887210726737976, + -0.1188158243894577, + 1.1391692161560059, + -0.7205156087875366, + -0.13284984230995178 + ], + [ + -0.04978486895561218, + -0.19524957239627838, + 0.026632994413375854, + -0.16501560807228088, + 1.507580280303955, + -0.34590357542037964, + 0.40128493309020996, + 0.12344897538423538, + -0.10562635958194733, + -0.0436982586979866, + 0.033674925565719604, + 1.1360576152801514, + -0.11289465427398682, + 0.13492539525032043, + -0.2354370653629303, + -0.05626790225505829, + 0.17636601626873016, + -0.12393470108509064, + 1.126055121421814, + -0.7666268348693848, + -0.3608725368976593 + ], + [ + -0.01939263939857483, + -0.24679048359394073, + 0.03107011318206787, + -0.4890434145927429, + 1.533837080001831, + -0.4929920732975006, + 0.25275158882141113, + 0.2871931791305542, + -0.21316978335380554, + -0.0804167315363884, + 0.1186186671257019, + 1.0964572429656982, + -0.24419304728507996, + 0.5774073004722595, + -0.31122124195098877, + -0.08002664148807526, + 0.3209894001483917, + -0.00044849514961242676, + 1.0110256671905518, + -0.17076098918914795, + -0.4503469169139862 + ], + [ + 0.06518993526697159, + -0.2147245556116104, + -0.0637262761592865, + -0.17908763885498047, + 1.4877252578735352, + -0.3291391134262085, + -0.018538206815719604, + 0.11814204603433609, + -0.14750580489635468, + 0.061548177152872086, + -0.23010414838790894, + 1.2080109119415283, + -0.09372454881668091, + 0.09325084090232849, + -0.22445476055145264, + -0.09234264492988586, + 0.2706632912158966, + 0.044122859835624695, + 0.9624550342559814, + -0.6028077602386475, + -0.22579365968704224 + ], + [ + -0.00460207462310791, + -0.2785096764564514, + 0.15270927548408508, + -0.18722903728485107, + 1.50177001953125, + -0.19188493490219116, + 0.4806599020957947, + -0.026199862360954285, + -0.05291987955570221, + 0.03865250200033188, + -0.1597832441329956, + 1.3016701936721802, + 0.031150877475738525, + 0.22909942269325256, + -0.23353204131126404, + -0.06382535398006439, + 0.25345098972320557, + -0.08922843635082245, + 1.1146588325500488, + -0.5792299509048462, + 0.06494437158107758 + ], + [ + -0.21892715990543365, + -0.1725882887840271, + 0.11764785647392273, + -0.29704079031944275, + 1.6328375339508057, + -0.6228445768356323, + 0.6463073492050171, + 0.14286065101623535, + -0.18001601099967957, + -0.2367975413799286, + -0.14022648334503174, + 1.2718926668167114, + 0.10375061631202698, + 0.6114175319671631, + -0.3505202531814575, + -0.05591515079140663, + 0.232391819357872, + -0.10737200081348419, + 1.3376154899597168, + -0.4605124592781067, + -0.07286329567432404 + ], + [ + -0.022343099117279053, + -0.16998058557510376, + -0.030634313821792603, + -0.08502086997032166, + 1.4842581748962402, + -0.37542346119880676, + 0.16227364540100098, + 0.20214256644248962, + -0.11115231364965439, + -0.009610556066036224, + -0.2007465362548828, + 1.3154736757278442, + -0.12881764769554138, + 0.2673048675060272, + -0.31555965542793274, + -0.03339279443025589, + 0.26468798518180847, + 0.05638900399208069, + 1.1049151420593262, + -0.7345172762870789, + -0.23666653037071228 + ] + ], + "generated_utilities": [ + 0.7220139503479004, + 2.0, + 2.0, + 0.650579035282135, + 2.0, + 0.7230307459831238, + 2.0, + 2.0 + ], + "group_id": "ms-8864796e31bbe93d7c5d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.728268027305603, + 0.7257949113845825, + 2.0, + 0.7287554740905762, + 0.72548508644104, + 0.7011849880218506, + 0.7089968919754028, + 0.689879298210144, + 0.722068727016449, + 0.7046384811401367, + 0.1933039277791977, + 0.629345178604126, + 0.18965840339660645, + 0.18740439414978027, + 0.2032846063375473, + 0.20379847288131714 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.3859826922416687, + 0.1119421124458313, + 0.6443181037902832, + 0.11836245656013489, + 1.863237738609314, + -1.166780948638916, + 0.8025261163711548, + -0.5342166423797607, + -0.2057022899389267, + 0.8987585306167603, + -0.525823712348938, + 1.6212486028671265, + 0.25674968957901, + 0.4037417471408844, + -0.4561460018157959, + -0.18429945409297943, + -0.1778680682182312, + -0.22491087019443512, + 2.095893144607544, + -0.3484030067920685, + 0.47628670930862427 + ], + [ + -0.446367084980011, + -0.19229409098625183, + -0.03271818161010742, + 0.39123907685279846, + 1.1786221265792847, + -0.16883885860443115, + -0.3337137699127197, + 0.7789723873138428, + -0.12953178584575653, + -0.028340458869934082, + -0.871502161026001, + 0.7702060341835022, + -0.645424485206604, + 1.0858956575393677, + -0.05413459241390228, + -0.14592209458351135, + -0.03685358166694641, + -0.686826229095459, + 2.084512710571289, + 0.11898508667945862, + 0.20845119655132294 + ], + [ + 0.09377387166023254, + -0.03508002683520317, + -0.29283326864242554, + -0.9559528827667236, + 2.0994086265563965, + 0.02743762731552124, + 0.6819101572036743, + 0.607413113117218, + 0.4378519356250763, + 0.6497513055801392, + -0.5086554288864136, + 0.9556822180747986, + -0.8518581986427307, + 0.31636691093444824, + -0.4196639060974121, + 0.6866374015808105, + 0.5175505876541138, + -0.572692334651947, + 1.651871681213379, + -0.7227057814598083, + -0.33861052989959717 + ], + [ + -0.42610836029052734, + 0.10144555568695068, + 0.7105271220207214, + -1.291037678718567, + 2.2307684421539307, + 0.17725014686584473, + -0.08436906337738037, + 0.9681323766708374, + 0.5482062697410583, + 1.368379831314087, + -1.021532416343689, + 2.5197110176086426, + -0.3508763909339905, + 0.7584415078163147, + 1.0978648662567139, + 0.2626119554042816, + 0.6115584373474121, + -0.7777479290962219, + 1.7278802394866943, + -0.15240566432476044, + 0.4239119291305542 + ], + [ + 1.0207308530807495, + 0.8557561039924622, + 0.4489574134349823, + -1.0117379426956177, + 0.8262714147567749, + -0.630092442035675, + 0.09812727570533752, + 0.21663640439510345, + 0.23771141469478607, + -0.47456812858581543, + -1.3189021348953247, + 2.269665002822876, + -0.7066116333007812, + 1.1826255321502686, + 1.0058563947677612, + 0.1694127470254898, + -0.5649446845054626, + -0.32043421268463135, + 0.46698564291000366, + -0.9052557945251465, + -0.12436902523040771 + ], + [ + -0.5202209949493408, + 0.30583277344703674, + 1.2163622379302979, + 0.4305659234523773, + 1.7276779413223267, + -0.1535550355911255, + -0.5289345383644104, + -0.632349967956543, + 0.9393473267555237, + 0.38391292095184326, + -1.1998018026351929, + 1.9739618301391602, + -1.3405330181121826, + 0.048939138650894165, + 0.6369897127151489, + 0.629721999168396, + 1.388192892074585, + -1.065996766090393, + 1.9119879007339478, + -1.0432264804840088, + 0.9627590179443359 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.13860583305358887, + -0.06957411766052246, + 0.1504676342010498, + 0.1671028435230255, + 0.5675687789916992, + 0.16587841510772705, + 0.0, + -0.21137550473213196, + -0.21542736887931824, + 0.2565074861049652, + -0.00923699140548706, + 0.5251069068908691, + -0.06964048743247986, + 0.0, + -0.19547803699970245, + 0.10278619825839996, + -0.19206219911575317, + -0.1487000733613968, + 0.44575393199920654, + 0.0725468099117279, + 0.0 + ] + ], + "predicted_scores": [ + 3.7122392654418945, + 3.7117695808410645, + 3.6351206302642822, + 3.6117537021636963, + 3.4393064975738525, + 3.282926082611084, + 3.219355344772339, + 2.964003086090088 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "551504635/train1", + "selected_index": 0, + "state_hash": "ab6587e0b2a3b5f21f23c56daf8894249774912a3b20794bd49dc1b78ec736a7", + "stored_base_utility": 0.728268027305603, + "task_id": "LiftPegUpright-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.42231252789497375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.42231252789497375 + }, + "base_progress": 0.42231252789497375, + "base_success": false, + "base_utility": 0.42231252789497375, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.6624154448509216, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6624154448509216 + }, + { + "contact_quality": null, + "progress": 0.22238026559352875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22238026559352875 + }, + { + "contact_quality": null, + "progress": 0.7256770730018616, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7256770730018616 + }, + { + "contact_quality": null, + "progress": 0.20876221358776093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20876221358776093 + }, + { + "contact_quality": null, + "progress": 0.39452430605888367, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.39452430605888367 + }, + { + "contact_quality": null, + "progress": 0.2241959571838379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2241959571838379 + }, + { + "contact_quality": null, + "progress": 0.21765725314617157, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21765725314617157 + }, + { + "contact_quality": null, + "progress": 0.2077002078294754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2077002078294754 + } + ], + "candidate_progress": [ + 0.6624154448509216, + 0.22238026559352875, + 0.7256770730018616, + 0.20876221358776093, + 0.39452430605888367, + 0.2241959571838379, + 0.21765725314617157, + 0.2077002078294754 + ], + "candidate_source_chart_ids": [ + "ms-8bc7aa419d17368d81b5", + "ms-13d8bae55d3c3710138f", + "ms-df961fe24a43e56d1716", + "ms-0660d5f65df5fc4b6132", + "ms-8bc7aa419d17368d81b5", + "ms-e1b6bcb41532a800d3cb", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank0", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b04b489f4fbad93b65ca", + "generated_tangents": [ + [ + -0.473727285861969, + -0.20458243787288666, + 0.3475378751754761, + 0.8252710103988647, + 0.24466358125209808, + 0.17423851788043976, + -0.03964203596115112, + -0.0001462697982788086, + 0.006239580921828747, + -0.043422676622867584, + -0.26038965582847595, + 1.1473183631896973, + -0.1878947913646698, + 0.04016990214586258, + 0.15174512565135956, + -0.06292980164289474, + -0.0837620347738266, + 0.11048615723848343, + 1.1570909023284912, + -0.30989381670951843, + 0.030177026987075806 + ], + [ + -0.1426912099123001, + -0.19952040910720825, + 0.7745941877365112, + 0.8301906585693359, + 0.20359830558300018, + -0.17863845825195312, + 0.18299150466918945, + -0.12927794456481934, + 0.16794560849666595, + -0.04269062727689743, + -0.29995664954185486, + 1.075852394104004, + -0.1310909390449524, + 0.10846757143735886, + 0.11009477078914642, + 0.006240988150238991, + -0.03404878079891205, + -0.02286781370639801, + 1.2750709056854248, + -0.24338920414447784, + -0.14805647730827332 + ], + [ + -0.17457570135593414, + -0.11718995124101639, + 0.7044237852096558, + 0.8478370904922485, + 0.11586553603410721, + 0.13718518614768982, + 0.03240591287612915, + -0.013515517115592957, + 0.06891965121030807, + -0.1174798533320427, + -0.29726698994636536, + 1.138895034790039, + -0.26907211542129517, + 0.06772702932357788, + 0.19067975878715515, + -0.008191946893930435, + 0.05094030499458313, + -0.040464892983436584, + 1.3260605335235596, + -0.41114822030067444, + -0.24100342392921448 + ], + [ + -0.09082283079624176, + -0.15988582372665405, + 0.8348315954208374, + 0.9153009653091431, + 0.21004009246826172, + -0.11935034394264221, + 0.026137322187423706, + -0.10805058479309082, + 0.16905617713928223, + -0.1175743117928505, + -0.39345020055770874, + 1.2242575883865356, + -0.10758864879608154, + 0.12500585615634918, + 0.13719435036182404, + 0.03037506714463234, + 0.014696374535560608, + -0.006907284259796143, + 1.1555595397949219, + -0.337271511554718, + -0.4327709376811981 + ], + [ + -0.4098537564277649, + -0.21750971674919128, + 0.232845276594162, + 0.7502044439315796, + 0.006616652011871338, + -0.00454305112361908, + -0.010837703943252563, + -0.08243179321289062, + 0.06806407123804092, + -0.16616925597190857, + -0.07266721874475479, + 1.1514806747436523, + -0.29811978340148926, + 0.0646565705537796, + 0.10928624868392944, + -0.1019655168056488, + 0.12408410757780075, + 0.06523144245147705, + 1.180974006652832, + -0.2171332836151123, + 0.07417988777160645 + ], + [ + -0.24468393623828888, + -0.16898521780967712, + 0.7371393442153931, + 0.8363014459609985, + 0.13242918252944946, + 0.010619372129440308, + -0.013288170099258423, + -0.12980088591575623, + 0.21491104364395142, + -0.04816807061433792, + -0.3035173714160919, + 1.2466926574707031, + -0.3947046399116516, + 0.41484034061431885, + 0.16444221138954163, + -0.039556294679641724, + -0.06334884464740753, + 0.02021704614162445, + 1.27815580368042, + -0.2796225845813751, + -0.030325546860694885 + ], + [ + -0.09441088140010834, + -0.08173932135105133, + 0.6346042156219482, + 0.961563229560852, + 0.24901160597801208, + -0.10678330063819885, + -0.11417979001998901, + -0.07817527651786804, + 0.16771452128887177, + -0.09400541335344315, + -0.3943471610546112, + 1.3348896503448486, + -0.2699192762374878, + 0.4237706661224365, + 0.029781069606542587, + -0.04438348859548569, + 0.06384485960006714, + 0.05673319101333618, + 1.1843684911727905, + -0.4529908299446106, + -0.2420814037322998 + ], + [ + 0.008482202887535095, + -0.18334625661373138, + 0.8706309795379639, + 0.8725862503051758, + 0.1143718883395195, + -0.0940537303686142, + -0.001456618309020996, + -0.021944910287857056, + 0.2779668867588043, + -0.08584963530302048, + -0.44209742546081543, + 1.2187726497650146, + 0.06693956255912781, + 0.26411381363868713, + 0.07009257376194, + 0.015542935580015182, + -0.03625498712062836, + 0.016178786754608154, + 1.3165087699890137, + -0.25032639503479004, + -0.20187541842460632 + ] + ], + "generated_utilities": [ + 0.6624154448509216, + 0.22238026559352875, + 0.7256770730018616, + 0.20876221358776093, + 0.39452430605888367, + 0.2241959571838379, + 0.21765725314617157, + 0.2077002078294754 + ], + "group_id": "ms-b04b489f4fbad93b65ca", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7198216915130615, + 0.7247374057769775, + 2.0, + 0.22730140388011932, + 0.21130704879760742, + 0.2076549530029297, + 0.2006116509437561, + 0.21285679936408997, + 0.19614991545677185, + 0.20440611243247986, + 0.2015489637851715, + 0.19511334598064423, + 0.20091164112091064, + 0.19681888818740845, + 0.19675713777542114, + 0.20853900909423828 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.30706143379211426, + 0.2797066569328308, + 0.4565920829772949, + 0.7342244386672974, + -0.28428468108177185, + 0.012420237064361572, + 0.0, + 0.19500797986984253, + -0.1289721131324768, + -0.3524337112903595, + 0.07460063695907593, + 1.7684156894683838, + 0.11335399746894836, + 0.0, + -0.5014548301696777, + 0.4537479281425476, + 0.1963198333978653, + -0.025568604469299316, + 1.6286182403564453, + -0.0963512659072876, + 0.0 + ], + [ + 1.3070614337921143, + -0.7202933430671692, + 1.456592082977295, + 1.7342244386672974, + -0.15130555629730225, + 0.9519147872924805, + -1.4541058540344238, + -0.6432010531425476, + -0.26959508657455444, + -0.3406839668750763, + -0.4419664740562439, + 2.768415689468384, + -0.3271861970424652, + -0.13685251772403717, + 0.23842881619930267, + 0.20406432449817657, + 0.3414191007614136, + -1.0255686044692993, + 2.6286182403564453, + -0.9051820039749146, + 0.6850019693374634 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9082117080688477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.27370503544807434, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.3700039386749268 + ], + [ + 1.0634682178497314, + -0.8590207099914551, + 1.6810569763183594, + 1.8525642156600952, + 0.06716938316822052, + 1.0774534940719604, + -1.177553415298462, + -0.7412493228912354, + -0.3193308115005493, + -0.4253617525100708, + -0.4890587329864502, + 2.624048948287964, + -0.109145388007164, + -0.10380380600690842, + 0.20137116312980652, + 0.20110605657100677, + 0.06929725408554077, + -0.9908366203308105, + 2.5783159732818604, + -0.7248227000236511, + 0.919882595539093 + ], + [ + 1.2083125114440918, + -0.9112091064453125, + 1.1255995035171509, + 1.9678500890731812, + -0.3424606919288635, + 1.350451946258545, + -1.5799542665481567, + -1.014689326286316, + -0.3427739441394806, + -0.24094250798225403, + -0.4992668032646179, + 2.403038740158081, + -0.29080134630203247, + -0.1569681167602539, + 0.4877905249595642, + -0.16464389860630035, + 0.41957730054855347, + -0.6997033357620239, + 2.80957293510437, + -1.0509651899337769, + 0.8937826156616211 + ], + [ + 1.2133954763412476, + -1.0171141624450684, + 1.1572445631027222, + 1.8050156831741333, + -0.29570114612579346, + 0.7672240734100342, + -0.976697564125061, + -0.8219587802886963, + -0.746895432472229, + -0.346658855676651, + -0.7942779064178467, + 3.0403196811676025, + -0.6172577738761902, + 0.3615778088569641, + 0.546725332736969, + -0.06415437161922455, + 0.43834370374679565, + -1.332592248916626, + 3.029940605163574, + -0.5720348358154297, + 1.1660562753677368 + ], + [ + 0.7937494516372681, + -0.45201122760772705, + 2.0367026329040527, + 2.1590728759765625, + 0.38935917615890503, + 0.5424834489822388, + -1.391097903251648, + -1.0657228231430054, + -0.44411367177963257, + -0.3797546923160553, + -0.03612968325614929, + 2.867069721221924, + -0.26322364807128906, + -0.32477179169654846, + 0.007963955402374268, + 0.053216055035591125, + 0.36466139554977417, + -0.7477234601974487, + 2.170649766921997, + -0.46266868710517883, + 0.7669195532798767 + ], + [ + 1.0920023918151855, + -1.0678586959838867, + 1.4391454458236694, + 2.155820608139038, + -0.7347199320793152, + 0.6208673715591431, + -1.6380563974380493, + -0.6589961051940918, + -0.8267081379890442, + 0.011062979698181152, + -0.1169244647026062, + 2.743579626083374, + -0.41342735290527344, + -0.5980290770530701, + 0.49618369340896606, + -0.09863747656345367, + 0.2179805338382721, + -0.6006474494934082, + 2.2965025901794434, + -0.97348952293396, + 1.2447597980499268 + ], + [ + 1.8286218643188477, + -1.486896276473999, + 1.8610713481903076, + 1.792364478111267, + -0.06209900975227356, + 0.7138283252716064, + -1.0023046731948853, + -0.8588749170303345, + -0.30936315655708313, + 0.33444294333457947, + 0.2771298885345459, + 3.0452492237091064, + -0.6618144512176514, + -0.747363805770874, + 0.19540274143218994, + -0.05051566660404205, + 0.5057645440101624, + -0.3770519495010376, + 3.028423309326172, + -0.19366812705993652, + 1.1766406297683716 + ], + [ + 1.057653784751892, + -0.15160071849822998, + 1.1646175384521484, + 2.517653226852417, + 0.037323400378227234, + 1.4844036102294922, + -1.2659661769866943, + -0.9694168567657471, + -0.792635440826416, + 0.15484732389450073, + -0.6592323780059814, + 2.426063060760498, + -0.8063763976097107, + 0.37280529737472534, + -0.1984816938638687, + 0.3488052487373352, + 0.36773377656936646, + -0.9600323438644409, + 1.801730990409851, + -1.1618295907974243, + 0.526383101940155 + ], + [ + 0.8403785228729248, + -1.7131881713867188, + 1.7484346628189087, + 1.3915244340896606, + 0.038706690073013306, + 1.7859117984771729, + -0.6996253132820129, + -0.12596702575683594, + -0.07181461155414581, + -0.8556877374649048, + -0.6567700505256653, + 2.935109853744507, + 0.3808797299861908, + 0.1733608990907669, + 0.9097857475280762, + 0.7001722455024719, + 0.14087507128715515, + -0.32659411430358887, + 2.278188943862915, + -1.87721586227417, + 1.438266634941101 + ], + [ + 1.259635329246521, + -0.5244767665863037, + 2.201894998550415, + 2.2337045669555664, + -0.333431601524353, + 0.7420244216918945, + -2.0896377563476562, + 0.00931471586227417, + 0.6261725425720215, + -0.49785688519477844, + -0.9404669404029846, + 3.756554365158081, + 0.5252982378005981, + -0.3158212900161743, + 0.4053819179534912, + 0.8204222917556763, + 0.5041900277137756, + -0.43809741735458374, + 3.519717216491699, + -1.0316743850708008, + 0.3069072663784027 + ], + [ + 0.9530733823776245, + -0.011007547378540039, + 2.396453380584717, + 2.6382436752319336, + 0.29263433814048767, + 1.6715352535247803, + -0.819398045539856, + -0.5916767120361328, + 0.006437480449676514, + -0.3942071795463562, + -0.5187541842460632, + 3.677276849746704, + -0.4376804232597351, + 0.3331298828125, + -0.3693196773529053, + -0.3432707190513611, + 1.104022741317749, + -1.0780245065689087, + 3.0859882831573486, + -0.26956111192703247, + 0.8377041816711426 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.30706143379211426, + 0.024911105632781982, + 0.4565920829772949, + 0.7342244386672974, + 0.01698923110961914, + -0.04808521270751953, + 0.0, + 0.10375595092773438, + 0.23247671127319336, + -0.12021404504776001, + -0.19280612468719482, + 1.7684156894683838, + -0.23127278685569763, + 0.0, + -0.1010780781507492, + 0.00952194631099701, + -0.16307592391967773, + -0.14496707916259766, + 1.6286182403564453, + -0.28656232357025146, + 0.0 + ] + ], + "predicted_scores": [ + 1.8552892208099365, + 1.3998653888702393, + 1.3820189237594604, + 0.7555887699127197, + 0.6081521511077881, + 0.341136634349823, + -0.13221046328544617, + -0.282153457403183 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113268940/train1", + "selected_index": 0, + "state_hash": "76e9a962970055fa7565c007db1b35e003cd352908be4cf573066d8b58e2d443", + "stored_base_utility": 0.7198216915130615, + "task_id": "LiftPegUpright-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7171411514282227, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7171411514282227 + }, + { + "contact_quality": null, + "progress": 0.7212136387825012, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7212136387825012 + }, + { + "contact_quality": null, + "progress": 0.6921262741088867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6921262741088867 + }, + { + "contact_quality": null, + "progress": 0.7184799313545227, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7184799313545227 + }, + { + "contact_quality": null, + "progress": 0.7070601582527161, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7070601582527161 + }, + { + "contact_quality": null, + "progress": 0.7001742720603943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7001742720603943 + }, + { + "contact_quality": null, + "progress": 0.7255622744560242, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7255622744560242 + }, + { + "contact_quality": null, + "progress": 0.6983166337013245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6983166337013245 + } + ], + "candidate_progress": [ + 0.7171411514282227, + 0.7212136387825012, + 0.6921262741088867, + 0.7184799313545227, + 0.7070601582527161, + 0.7001742720603943, + 0.7255622744560242, + 0.6983166337013245 + ], + "candidate_source_chart_ids": [ + "ms-062addb9da32a0c2ea97", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69", + "ms-b483cac145e16ccdf42a", + "ms-efbde8652e75274fbd69", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69", + "ms-b483cac145e16ccdf42a" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-c663f7e73e7e8376a05c", + "generated_tangents": [ + [ + 0.018496453762054443, + -0.08482980728149414, + -0.21568617224693298, + 0.09971177577972412, + 1.6243064403533936, + 0.10376907140016556, + 0.020362496376037598, + -0.014710485935211182, + -0.0076336730271577835, + -0.028992556035518646, + -0.09916447848081589, + 1.0256421566009521, + -0.04061979055404663, + 0.05338793247938156, + -0.09438146650791168, + -0.0546598955988884, + 0.10169292986392975, + 0.04009826481342316, + 1.0276858806610107, + -0.12354743480682373, + -0.05365872383117676 + ], + [ + -0.05339689552783966, + -0.10680296272039413, + -0.07668808102607727, + 0.1379472017288208, + 1.5675547122955322, + 0.1497737318277359, + -0.010280996561050415, + -0.055953145027160645, + -0.01853945478796959, + 0.0033985525369644165, + -0.09700677543878555, + 0.9669918417930603, + -0.15829509496688843, + 0.37960100173950195, + -0.019056398421525955, + -0.04062753543257713, + 0.07642152905464172, + 0.07597062736749649, + 0.8810792565345764, + 0.0319751501083374, + -0.0648588091135025 + ], + [ + -0.03835053741931915, + -0.10996833443641663, + -0.09504029154777527, + 0.09926941990852356, + 1.6195778846740723, + 0.17754143476486206, + 0.08033245801925659, + -0.030462920665740967, + 0.0072348159737885, + 0.016059696674346924, + -0.10309233516454697, + 1.0032544136047363, + -0.18356046080589294, + 0.33337730169296265, + -0.037213798612356186, + -0.03972916305065155, + 0.10433965921401978, + 0.08191090077161789, + 0.888705313205719, + 0.03629118204116821, + 0.0033277571201324463 + ], + [ + 0.09127193689346313, + -0.055760692805051804, + -0.1374853551387787, + 0.07743537425994873, + 1.5472488403320312, + -0.00636233389377594, + 0.04363471269607544, + -0.00877659022808075, + 0.029708439484238625, + 0.05226445198059082, + -0.05625463277101517, + 0.8850756883621216, + 0.035779207944869995, + 0.015579409897327423, + -0.09338870644569397, + -0.0615127794444561, + 0.1847681999206543, + 0.05029042065143585, + 1.1262969970703125, + -0.029379457235336304, + 0.02394700050354004 + ], + [ + 0.04843737930059433, + -0.0003103986382484436, + -0.2789051830768585, + 0.16971105337142944, + 1.7634544372558594, + 0.504429280757904, + -0.27973997592926025, + -0.01923424005508423, + -0.021087726578116417, + 0.08221027255058289, + -0.2855764627456665, + 0.6937445402145386, + -0.07779139280319214, + 0.1338072568178177, + -0.13744857907295227, + 0.08373387157917023, + 0.19297969341278076, + -0.04320131242275238, + 0.9147657752037048, + -0.14992932975292206, + -0.1147456169128418 + ], + [ + 0.030949018895626068, + -0.05405410751700401, + -0.2067166268825531, + 0.09814414381980896, + 1.584350347518921, + 0.12914511561393738, + 0.032954901456832886, + -0.10106584429740906, + -0.015615474432706833, + 0.0002103671431541443, + -0.1916314959526062, + 0.9277292490005493, + -0.03392323851585388, + 0.12778586149215698, + -0.11368101835250854, + -0.07895809412002563, + 0.08963510394096375, + 0.051576659083366394, + 0.8910223245620728, + -0.002006441354751587, + -0.08677718043327332 + ], + [ + -0.01074892282485962, + -0.1378961205482483, + -0.2911452353000641, + 0.17210707068443298, + 1.687772274017334, + 0.11322876811027527, + -0.07297337055206299, + -0.05377556383609772, + -0.026861606165766716, + 0.023169003427028656, + -0.09588069468736649, + 0.8563389182090759, + -0.10155078768730164, + 0.2857053577899933, + 0.054930489510297775, + -0.07985828816890717, + -0.07473190128803253, + 0.060383960604667664, + 1.02923583984375, + -0.10958270728588104, + 0.020274892449378967 + ], + [ + -0.059429556131362915, + -0.07215353101491928, + -0.23873195052146912, + 0.1298384666442871, + 1.5628283023834229, + 0.12706680595874786, + 0.03786039352416992, + -0.09929674863815308, + 0.02840629406273365, + -0.03223598748445511, + -0.10132608562707901, + 0.8569584488868713, + -0.037724435329437256, + 0.07865964621305466, + -0.10110992193222046, + -0.07663080096244812, + 0.02457106113433838, + -0.005532324314117432, + 1.0890499353408813, + -0.1768168807029724, + -0.013077616691589355 + ] + ], + "generated_utilities": [ + 0.7171411514282227, + 0.7212136387825012, + 0.6921262741088867, + 0.7184799313545227, + 0.7070601582527161, + 0.7001742720603943, + 0.7255622744560242, + 0.6983166337013245 + ], + "group_id": "ms-c663f7e73e7e8376a05c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7233090400695801, + 0.7198824882507324, + 2.0, + 0.7120869159698486, + 0.27145418524742126, + 0.23090600967407227, + 0.2526220679283142, + 0.2656436562538147, + 0.264312207698822, + 0.27572086453437805, + 0.296668142080307, + 0.2128777951002121, + 0.22965002059936523, + 0.2196153998374939, + 0.22027263045310974, + 0.19939398765563965 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.038699887692928314, + -0.3685438632965088, + -1.3628361225128174, + 0.0675731897354126, + 2.4906270503997803, + 0.8971554040908813, + 0.06863358616828918, + 0.14689946174621582, + 0.0500560887157917, + 0.5852231383323669, + -0.5345680713653564, + 1.7433128356933594, + -0.5520187020301819, + 0.36285334825515747, + 0.06837204098701477, + 0.04091557487845421, + 0.26646411418914795, + -0.3870337903499603, + 1.7901486158370972, + -0.46956348419189453, + 0.5412080883979797 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.13726717233657837, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7257066965103149, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0824161767959595 + ], + [ + -0.09956050664186478, + -0.6439464092254639, + -1.5331499576568604, + -0.2275703251361847, + 2.2570624351501465, + 1.145668387413025, + -0.19933581352233887, + 0.15889883041381836, + -0.07524599134922028, + 0.6580387353897095, + -0.5754004716873169, + 1.6105921268463135, + -0.25637149810791016, + 0.37256744503974915, + 0.17931225895881653, + 0.286465048789978, + 0.07884949445724487, + -0.35266172885894775, + 1.548264980316162, + -0.2917717695236206, + 0.3307930827140808 + ], + [ + 0.13072630763053894, + -0.5927069783210754, + -1.4328954219818115, + 0.1252535879611969, + 2.5439467430114746, + 1.119347095489502, + -0.12134809792041779, + -0.09485669434070587, + -0.040146272629499435, + 0.4683093726634979, + -0.6791773438453674, + 2.1314187049865723, + -0.4368498623371124, + 0.5876930952072144, + 0.27027207612991333, + -0.1808907389640808, + 0.037005022168159485, + -0.6130290627479553, + 1.60811185836792, + -0.8427046537399292, + 0.5648807883262634 + ], + [ + -0.019807856529951096, + -0.40178078413009644, + -1.5710430145263672, + 0.3568149507045746, + 2.8693275451660156, + 1.2080440521240234, + 0.45903119444847107, + -0.07479061186313629, + 0.2587376832962036, + 0.37667450308799744, + -0.5607666373252869, + 1.9825769662857056, + -0.7874732613563538, + 0.48894837498664856, + -0.04161303490400314, + 0.2237108200788498, + 0.12043289840221405, + -0.8541218638420105, + 1.3131444454193115, + -0.8318716287612915, + 0.7653946876525879 + ], + [ + -0.039324238896369934, + 0.18316024541854858, + -1.2052042484283447, + -0.020045489072799683, + 2.441659927368164, + 1.3116326332092285, + 0.04364071041345596, + 0.617156982421875, + -0.5366188287734985, + 0.5837841629981995, + -0.6157102584838867, + 1.946446180343628, + -0.44197604060173035, + -0.12030342221260071, + 0.2635759711265564, + -0.34203875064849854, + 0.3788418769836426, + -0.01673591136932373, + 1.7612254619598389, + -0.8283271193504333, + 0.44542378187179565 + ], + [ + 0.020479684695601463, + -0.3728349208831787, + -1.2566280364990234, + -0.4903675317764282, + 2.7149407863616943, + 1.1410136222839355, + 0.18438707292079926, + 0.15751159191131592, + 0.5994289517402649, + 0.7441410422325134, + -0.5757432579994202, + 2.025078058242798, + -1.1044142246246338, + -0.24153441190719604, + 0.7623711824417114, + -0.07102327048778534, + 0.7491155862808228, + -0.853058397769928, + 1.1346031427383423, + -0.4604499638080597, + 0.3380756974220276 + ], + [ + -0.0438276082277298, + 0.21775567531585693, + -0.9180121421813965, + 0.2164689004421234, + 1.7669851779937744, + 0.7701845765113831, + -0.4171431064605713, + -0.44690096378326416, + -0.6473215818405151, + 0.5592721700668335, + 0.1537061333656311, + 1.2201651334762573, + -0.02862715721130371, + 0.759445309638977, + -0.08159622550010681, + -0.038715001195669174, + 0.8136870265007019, + -1.129607081413269, + 1.5099153518676758, + -1.2117259502410889, + 1.2496943473815918 + ], + [ + -0.6884663105010986, + -0.7670764923095703, + -1.3375780582427979, + 0.42161574959754944, + 2.4925031661987305, + 0.18111640214920044, + -0.4373469054698944, + 0.6228322386741638, + -0.3739543557167053, + 1.3811917304992676, + 0.05884122848510742, + 1.0582032203674316, + -0.38174134492874146, + 0.7282494902610779, + 0.9550073146820068, + -0.6478474736213684, + -0.467768132686615, + -0.22179017961025238, + 2.1456754207611084, + -0.39938730001449585, + 1.08339524269104 + ], + [ + 0.7042056322097778, + -0.4828507900238037, + -1.914047122001648, + -0.7084186673164368, + 1.87782621383667, + 0.42005008459091187, + 0.511604905128479, + -0.2043081521987915, + 0.9001545906066895, + 1.083567500114441, + -0.9796027541160583, + 1.3277345895767212, + -0.5564201474189758, + 0.6932777166366577, + -0.5934457778930664, + -0.296670138835907, + 0.14519014954566956, + -0.8612968325614929, + 0.8968887329101562, + -0.5584146976470947, + -0.434501588344574 + ], + [ + 0.19210004806518555, + 0.5024477243423462, + -0.4993329644203186, + -0.826042115688324, + 1.6430644989013672, + 0.9877098798751831, + 0.49057987332344055, + -0.13611945509910583, + 0.7228546738624573, + 1.4937405586242676, + 0.18012672662734985, + 2.188062906265259, + -0.6084107160568237, + 1.0874407291412354, + 0.5744171142578125, + 0.9375161528587341, + -0.43115919828414917, + 0.19787153601646423, + 2.652085781097412, + -0.529538631439209, + 0.03626328706741333 + ], + [ + -0.014053341001272202, + -1.351341724395752, + -0.7293765544891357, + 0.9410406947135925, + 3.139275074005127, + 1.3463419675827026, + -0.728021502494812, + -0.6353646516799927, + -0.19685329496860504, + 1.5315749645233154, + -1.2550475597381592, + 1.7411813735961914, + 0.14418578147888184, + 0.7038239240646362, + 0.1004495620727539, + 0.7590892910957336, + 0.9287852048873901, + -0.23726919293403625, + 0.8381044864654541, + -1.2906644344329834, + 0.9584261178970337 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03644278272986412, + -0.12137487530708313, + -0.3628361225128174, + -0.027813076972961426, + 1.4906270503997803, + -0.10284459590911865, + 0.0, + 0.15174832940101624, + 0.015595663338899612, + -0.10680150985717773, + -0.14723241329193115, + 0.7433128356933594, + 0.25961190462112427, + 0.0, + -0.23801246285438538, + -0.10784421861171722, + 0.09500086307525635, + -0.06586915254592896, + 0.7901486158370972, + -0.10978007316589355, + 0.0 + ] + ], + "predicted_scores": [ + 3.528031587600708, + 3.484841823577881, + 3.465104341506958, + 3.426609754562378, + 3.389573335647583, + 3.3381567001342773, + 3.1684746742248535, + 3.0973165035247803 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1528740417/train1", + "selected_index": 0, + "state_hash": "c02fdfdd2339e4215d60b2221be925e066a7763f9e3bba3ff33a15f20e286f5a", + "stored_base_utility": 0.7233090400695801, + "task_id": "LiftPegUpright-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15342965722084045, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15342965722084045 + }, + "base_progress": 0.15342965722084045, + "base_success": false, + "base_utility": 0.15342965722084045, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0375710129737854, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0375710129737854 + }, + { + "contact_quality": null, + "progress": 0.4434976577758789, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4434976577758789 + }, + { + "contact_quality": null, + "progress": 0.025715351104736328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025715351104736328 + }, + { + "contact_quality": null, + "progress": 0.025355923920869827, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025355923920869827 + }, + { + "contact_quality": null, + "progress": 0.035843171179294586, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035843171179294586 + }, + { + "contact_quality": null, + "progress": 0.46439141035079956, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.46439141035079956 + }, + { + "contact_quality": null, + "progress": 0.022827064618468285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022827064618468285 + }, + { + "contact_quality": null, + "progress": 0.021351909264922142, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021351909264922142 + } + ], + "candidate_progress": [ + 0.0375710129737854, + 0.4434976577758789, + 0.025715351104736328, + 0.025355923920869827, + 0.035843171179294586, + 0.46439141035079956, + 0.022827064618468285, + 0.021351909264922142 + ], + "candidate_source_chart_ids": [ + "ms-a509a572b3871e591d26", + "ms-b7e7fe66a6c86898eec2", + "ms-307f9067d5b66f55b3a6", + "ms-4015c296239564108da2", + "ms-0541a144afe57b5a7ae6", + "ms-b7e7fe66a6c86898eec2", + "ms-5df54a8baaceb8c73c8a", + "ms-23a355317920e192147f" + ], + "candidate_source_task_ids": [ + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank6", + "ctt_transport_rank3", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "generated_tangents": [ + [ + -0.04052545130252838, + -0.2974761128425598, + 0.9359121322631836, + 0.10208287835121155, + 0.8169476389884949, + 0.31597915291786194, + 1.6885921955108643, + 0.35994845628738403, + -0.1680283099412918, + 0.12475381046533585, + 0.18452417850494385, + 0.3003499507904053, + 0.5231201648712158, + 3.625150680541992, + -0.0822700560092926, + -0.13376817107200623, + 0.465038001537323, + 0.08038262277841568, + 0.4713194966316223, + 0.05367746949195862, + 3.2971887588500977 + ], + [ + -0.1903712898492813, + -0.37053966522216797, + 0.6688979268074036, + -0.09197451919317245, + 0.6773920059204102, + -0.33712926506996155, + 1.4484882354736328, + 0.5138263702392578, + -0.09785021096467972, + 0.03438278287649155, + 0.12166035175323486, + 0.31852588057518005, + 0.380852609872818, + 2.716904878616333, + -0.4005071222782135, + 0.046017542481422424, + 0.6103469729423523, + 0.3996133804321289, + 0.5668272972106934, + -0.316438764333725, + 2.8927831649780273 + ], + [ + -0.21704740822315216, + -0.330172061920166, + 1.100143313407898, + -0.07240776717662811, + 0.5212385654449463, + -0.10958275198936462, + 1.6004505157470703, + 0.07333774119615555, + -0.12773768603801727, + 0.06542205810546875, + 0.07344947755336761, + 0.17908638715744019, + 0.11419609189033508, + 3.368091344833374, + -0.08816054463386536, + -0.17370833456516266, + 0.5846867561340332, + 0.07153494656085968, + 0.4638330936431885, + 0.6930655837059021, + 3.6226603984832764 + ], + [ + -0.12587983906269073, + -0.38620221614837646, + 1.1821459531784058, + -0.057351693511009216, + 0.3969283103942871, + -0.13062044978141785, + 1.5191221237182617, + 0.05112460255622864, + -0.12875741720199585, + 0.08850830793380737, + 0.12764394283294678, + 0.24970731139183044, + 0.2700720727443695, + 3.2572789192199707, + -0.09465007483959198, + -0.2016996443271637, + 0.47044676542282104, + 0.07808393239974976, + 0.44408103823661804, + 0.5866963863372803, + 3.515298366546631 + ], + [ + -0.29369062185287476, + -0.3233182430267334, + 1.038780927658081, + 0.031081125140190125, + 0.8599079847335815, + 0.02698427438735962, + 1.6414964199066162, + 0.28055518865585327, + -0.10770316421985626, + 0.03712392598390579, + 0.08561195433139801, + 0.17864517867565155, + 0.18073830008506775, + 3.289355754852295, + -0.14902560412883759, + -0.24006064236164093, + 0.5561586618423462, + 0.10910765826702118, + 0.6538932919502258, + 0.8370549082756042, + 3.7762296199798584 + ], + [ + -0.2538868188858032, + -0.40941011905670166, + 0.7809090614318848, + -0.07721935212612152, + 0.5145487785339355, + -0.44785603880882263, + 1.4584362506866455, + 0.524945080280304, + -0.13737700879573822, + -0.18502366542816162, + 0.16651324927806854, + 0.3745180368423462, + 0.3228949010372162, + 2.64192271232605, + -0.46043604612350464, + 0.10471723973751068, + 0.6481974124908447, + 0.3921470642089844, + 0.5390013456344604, + 0.0029841959476470947, + 2.9022462368011475 + ], + [ + -0.14602552354335785, + -0.2900434732437134, + 1.1657943725585938, + -0.02164776623249054, + 0.6434773206710815, + 0.05961035192012787, + 1.5957074165344238, + 0.2755030691623688, + -0.09505859017372131, + 0.09135580062866211, + 0.18444988131523132, + 0.20256099104881287, + 0.17112037539482117, + 3.125737428665161, + -0.21520298719406128, + -0.1718806028366089, + 0.6482832431793213, + 0.10868270695209503, + 0.675834596157074, + 0.9381958842277527, + 3.60792875289917 + ], + [ + -0.13458235561847687, + -0.33202123641967773, + 1.1956603527069092, + -0.06914085149765015, + 0.22907499969005585, + -0.234523206949234, + 2.0782017707824707, + 0.20605845749378204, + -0.14640741050243378, + 0.1175045520067215, + 0.12905830144882202, + 0.2542654871940613, + 0.1759064495563507, + 3.1949563026428223, + -0.2317231297492981, + -0.22757183015346527, + 0.5861898064613342, + 0.24816322326660156, + 0.29490092396736145, + 0.5639578700065613, + 3.59490966796875 + ] + ], + "generated_utilities": [ + 0.0375710129737854, + 0.4434976577758789, + 0.025715351104736328, + 0.025355923920869827, + 0.035843171179294586, + 0.46439141035079956, + 0.022827064618468285, + 0.021351909264922142 + ], + "group_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.6040759086608887, + 2.0, + 0.14582239091396332, + 0.5723002552986145, + 0.10770338028669357, + 0.10128920525312424, + 0.08499570190906525, + 0.14736615121364594, + 0.07181884348392487, + 0.11773071438074112, + 0.06549974530935287, + 0.06568789482116699, + 0.10681070387363434, + 0.05432777479290962, + 0.0625491738319397, + 0.06991603225469589 + ], + "instruction": "Pick up the cube and move it to the goal position.", + "negative_tangents": [ + [ + -0.072060227394104, + 0.2043294906616211, + 0.8692872524261475, + 0.020746350288391113, + -0.004742175340652466, + 0.05779099464416504, + 0.0, + -0.0498790442943573, + -0.20218299329280853, + 0.41556206345558167, + -0.2306707501411438, + 0.32090064883232117, + -0.2089032530784607, + 0.0, + 0.05318737402558327, + -0.04637153446674347, + 0.04412045329809189, + 0.08670947700738907, + -0.10918038338422775, + -0.059010982513427734, + 0.0 + ], + [ + -0.5282122492790222, + 1.204329490661621, + 1.8692872524261475, + -0.8456907868385315, + 0.42858073115348816, + -0.3134727478027344, + -2.5308754444122314, + 0.1655428111553192, + -0.12024828046560287, + 0.0010203085839748383, + 0.2592625319957733, + 0.2679504156112671, + 0.25922709703445435, + 2.851924419403076, + 0.0039860643446445465, + -0.014717570506036282, + 0.01173020526766777, + 0.10578173398971558, + 0.05138798803091049, + 0.3216937780380249, + 2.9243662357330322 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.061750888824463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.703848838806152, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.8487324714660645 + ], + [ + -0.6558206081390381, + 1.3553937673568726, + 2.057647466659546, + -1.0405915975570679, + 0.2629950940608978, + -0.28233209252357483, + -2.589219093322754, + -0.06961745023727417, + 0.02406928688287735, + 0.2341437190771103, + 0.49766814708709717, + 0.4930538535118103, + 0.370309442281723, + 3.111422061920166, + 0.06151486933231354, + -0.2028706818819046, + 0.12007936835289001, + 0.2164750099182129, + 0.16723006963729858, + 0.26430171728134155, + 3.0643396377563477 + ], + [ + -0.18174728751182556, + 0.8890945315361023, + 1.5388343334197998, + -0.4532086253166199, + 0.3864310085773468, + -0.45410382747650146, + -2.623199462890625, + 0.5360933542251587, + -0.04706605523824692, + 0.11160072684288025, + 0.5460608005523682, + 0.39992716908454895, + 0.17028246819972992, + 2.8760411739349365, + 0.29450932145118713, + -0.3028962016105652, + 0.06744809448719025, + 0.1687500774860382, + 0.42097336053848267, + 0.3965923488140106, + 3.28550386428833 + ], + [ + -0.4313088357448578, + 0.9752410054206848, + 2.0068275928497314, + -0.5783756375312805, + 0.5761726498603821, + -0.5289677381515503, + -2.321643590927124, + 0.0023015737533569336, + -0.07372284680604935, + 0.05664224550127983, + 0.03747282922267914, + 0.4900178611278534, + 0.31588616967201233, + 3.018609046936035, + 0.12529893219470978, + -0.2872275412082672, + -0.2500917911529541, + 0.0006694421172142029, + -0.39133700728416443, + 0.6382719874382019, + 2.810041904449463 + ], + [ + -0.5560051202774048, + 1.4042110443115234, + 1.662226915359497, + -1.4059122800827026, + 0.8242433071136475, + -0.10998553037643433, + -2.7556281089782715, + -0.26875895261764526, + -0.4201813042163849, + -0.371376633644104, + 0.7718684673309326, + 0.6452833414077759, + 0.027421817183494568, + 2.3879575729370117, + -0.47588983178138733, + -0.045049820095300674, + 0.0838315486907959, + 0.10257662832736969, + 0.4181908369064331, + 0.2650004029273987, + 3.006909132003784 + ], + [ + -0.4896228015422821, + 1.0570447444915771, + 1.408416509628296, + -0.7269805669784546, + 1.023213267326355, + -0.22378163039684296, + -2.1348509788513184, + -0.4423682987689972, + 0.26834720373153687, + -0.5835619568824768, + 0.3675343692302704, + 0.3307996988296509, + -0.01537257432937622, + 3.4422543048858643, + 0.3622148334980011, + -0.26806333661079407, + 0.4238347113132477, + 0.2583734393119812, + 0.10431534051895142, + 0.2077663242816925, + 2.446356773376465 + ], + [ + -1.1077957153320312, + 0.4289678931236267, + 1.9882663488388062, + -0.3021383285522461, + -0.006195276975631714, + -0.22295325994491577, + -1.8697566986083984, + 0.9590625762939453, + -0.0024877265095710754, + -0.3088582456111908, + 0.6517632007598877, + 0.4271944761276245, + 0.34532302618026733, + 2.704721450805664, + 0.7557594776153564, + -0.8073300719261169, + -0.7479258179664612, + -0.19068273901939392, + 0.14240311086177826, + 0.2233867347240448, + 2.1774275302886963 + ], + [ + -0.9354162812232971, + 1.5971527099609375, + 1.0168101787567139, + -0.7164653539657593, + 0.519247829914093, + -0.7477964162826538, + -3.338376045227051, + 0.7212944030761719, + -0.6477807760238647, + 0.38193029165267944, + -0.1568622589111328, + 0.47457069158554077, + -0.30742186307907104, + 2.6848907470703125, + -0.4251534044742584, + 0.6623334884643555, + 0.03348249942064285, + -0.2996619939804077, + -0.8319172263145447, + 0.7883892059326172, + 3.5870752334594727 + ], + [ + -0.44698429107666016, + 0.3201255202293396, + 1.5405346155166626, + -0.9956957101821899, + 0.8944320678710938, + -0.4196317791938782, + -2.010188102722168, + 0.7199084758758545, + -0.49276289343833923, + 0.4512734115123749, + 0.48276829719543457, + 0.22657281160354614, + 0.8205416798591614, + 3.6653342247009277, + 0.6818231344223022, + 0.4660317897796631, + -0.14916014671325684, + 0.5328372716903687, + -0.5979922413825989, + 0.3124805986881256, + 3.110726833343506 + ], + [ + -0.5630171298980713, + 1.934722900390625, + 1.4954721927642822, + -1.684873342514038, + 0.133146733045578, + -0.4359630346298218, + -2.681238889694214, + 0.8904907703399658, + -0.48221057653427124, + -0.7635119557380676, + -0.23804441094398499, + 0.5691602230072021, + -0.23042798042297363, + 3.760842800140381, + 0.3684825003147125, + -0.20475278794765472, + 0.04838913306593895, + 0.17439094185829163, + 0.07042267918586731, + -0.1653938889503479, + 3.844882011413574 + ], + [ + -0.3705383241176605, + 1.5082495212554932, + 0.9002141356468201, + -1.3291171789169312, + 0.15676641464233398, + -1.2286558151245117, + -1.7006758451461792, + 0.34438908100128174, + 0.32962289452552795, + -0.6556114554405212, + 0.3502740263938904, + 0.7518025636672974, + -0.49080890417099, + 2.10355806350708, + 0.36426910758018494, + -0.4822288453578949, + 0.7496572732925415, + 0.43558254837989807, + 0.34650692343711853, + 0.9977220296859741, + 1.9853770732879639 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.005563676357269287, + 0.2043294906616211, + 0.8692872524261475, + 0.08163481950759888, + 0.05995750427246094, + 0.012857973575592041, + 0.0, + 0.01583634316921234, + 0.04724548012018204, + 0.018815450370311737, + 0.02742627263069153, + -0.03381502628326416, + 0.007930099964141846, + 0.0, + 0.01774722896516323, + 0.08064591884613037, + -0.06814983487129211, + -0.002621002495288849, + 0.05759798362851143, + 0.05164051055908203, + 0.0 + ] + ], + "predicted_scores": [ + 0.6592466831207275, + -0.18288078904151917, + -0.2230769693851471, + -0.33697351813316345, + -0.6092725992202759, + -0.8914775848388672, + -0.931504487991333, + -1.0792096853256226 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1273247072/train1", + "selected_index": 0, + "state_hash": "237fafab6bfdb561d66c862026f803fbd19d4007f6313bdb3d79b6dd74f69327", + "stored_base_utility": 0.6040759086608887, + "task_id": "PickCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.050487834960222244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.050487834960222244 + }, + "base_progress": 0.050487834960222244, + "base_success": false, + "base_utility": 0.050487834960222244, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.019472381100058556, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019472381100058556 + }, + { + "contact_quality": null, + "progress": 0.05719713494181633, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05719713494181633 + }, + { + "contact_quality": null, + "progress": 0.06821080297231674, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06821080297231674 + }, + { + "contact_quality": null, + "progress": 0.028275012969970703, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.028275012969970703 + }, + { + "contact_quality": null, + "progress": 0.01960599422454834, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01960599422454834 + }, + { + "contact_quality": null, + "progress": 0.11701449006795883, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11701449006795883 + }, + { + "contact_quality": null, + "progress": 0.05281931161880493, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05281931161880493 + }, + { + "contact_quality": null, + "progress": 0.06312280893325806, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06312280893325806 + } + ], + "candidate_progress": [ + 0.019472381100058556, + 0.05719713494181633, + 0.06821080297231674, + 0.028275012969970703, + 0.01960599422454834, + 0.11701449006795883, + 0.05281931161880493, + 0.06312280893325806 + ], + "candidate_source_chart_ids": [ + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0345d8ac530f88cefd61", + "generated_tangents": [ + [ + 0.444324791431427, + 0.10367916524410248, + 0.012270689010620117, + 0.8968409299850464, + 0.2659638524055481, + 1.0732413530349731, + -0.21330034732818604, + -0.09056296944618225, + 0.6343738436698914, + 0.874171257019043, + -1.1261749267578125, + -0.17651215195655823, + -0.6824302673339844, + 0.21632888913154602, + -0.8118916153907776, + 0.3421381711959839, + 0.47831493616104126, + 0.186706081032753, + -1.1168158054351807, + 0.15592551231384277, + -0.3649512827396393 + ], + [ + 0.582933783531189, + 0.07501409947872162, + 0.3667686879634857, + 0.6543165445327759, + 0.3305560350418091, + 1.1277859210968018, + -0.5874252915382385, + -0.17955923080444336, + 0.48543867468833923, + 0.6742415428161621, + -1.0333203077316284, + -0.10667729377746582, + -0.5693666934967041, + -0.16236604750156403, + -0.6417573690414429, + 0.35614097118377686, + 0.472110778093338, + 0.25263339281082153, + -1.2747080326080322, + -0.06723153591156006, + -0.5342777967453003 + ], + [ + 0.4774540066719055, + 0.18879170715808868, + 0.052444279193878174, + 0.7714061737060547, + 0.2214621752500534, + 0.9115142822265625, + -0.4279770851135254, + -0.04090888798236847, + 0.6480913758277893, + 0.8157578110694885, + -1.042747974395752, + -0.2258063554763794, + -0.696947455406189, + 0.05961869657039642, + -0.8991007804870605, + 0.30625736713409424, + 0.5233581066131592, + 0.20415931940078735, + -1.189032793045044, + 0.15570494532585144, + -0.2326734960079193 + ], + [ + 0.44649672508239746, + 0.09742087125778198, + 0.13266673684120178, + 0.7603572607040405, + 0.282711386680603, + 0.9564565420150757, + -0.46242356300354004, + -0.03456386923789978, + 0.5279194712638855, + 0.7715417146682739, + -1.0219554901123047, + -0.18551328778266907, + -0.4615861773490906, + -0.11304500699043274, + -0.8485602736473083, + 0.314506471157074, + 0.5243592858314514, + 0.30175650119781494, + -1.1856787204742432, + 0.014193445444107056, + -0.30259281396865845 + ], + [ + 0.38412991166114807, + 0.102728471159935, + 0.06790709495544434, + 0.6844552755355835, + 0.18033036589622498, + 1.2175039052963257, + -0.2505999803543091, + -0.10151886940002441, + 0.5712025761604309, + 0.8772667050361633, + -0.9988381862640381, + -0.10765290260314941, + -0.45664942264556885, + 0.0903095155954361, + -0.9287569522857666, + 0.3178274631500244, + 0.523028552532196, + 0.10677517205476761, + -0.9136781096458435, + 0.3343772292137146, + -0.033195048570632935 + ], + [ + 0.4742353558540344, + 0.1852404773235321, + 0.27466344833374023, + 0.3616723120212555, + 0.31565362215042114, + 0.30515414476394653, + -0.34123581647872925, + -0.07583582401275635, + 0.5641093254089355, + 0.5736047029495239, + -1.1779217720031738, + -0.012221962213516235, + 0.10746058821678162, + -0.6907521486282349, + -0.7577171921730042, + 0.38214731216430664, + 0.21964217722415924, + 0.2696443796157837, + -1.424448013305664, + -0.14528992772102356, + -0.24489116668701172 + ], + [ + 0.3809819221496582, + 0.2657419443130493, + 0.24924719333648682, + 0.44967177510261536, + 0.6870844960212708, + 0.7106812000274658, + -0.5713879466056824, + -0.0011958479881286621, + 0.48955443501472473, + 0.8217517137527466, + -1.05900239944458, + -0.6630968451499939, + -0.3793579936027527, + -0.5009958148002625, + -1.023776650428772, + 0.21992097795009613, + 0.5339664220809937, + 0.10243518650531769, + -1.1657459735870361, + 0.12472578883171082, + -0.18491411209106445 + ], + [ + 0.5386635065078735, + 0.11535128951072693, + 0.250006765127182, + 0.8108882904052734, + 0.6256585717201233, + 0.8495814800262451, + -0.23052239418029785, + -0.1735210120677948, + 0.5670768022537231, + 0.7462118864059448, + -1.0848647356033325, + -0.3679772615432739, + -0.16919347643852234, + -0.35942530632019043, + -0.8746117949485779, + 0.2851692736148834, + 0.38496536016464233, + 0.25257858633995056, + -1.0041775703430176, + -0.1322382092475891, + -0.18745839595794678 + ] + ], + "generated_utilities": [ + 0.019472381100058556, + 0.05719713494181633, + 0.06821080297231674, + 0.028275012969970703, + 0.01960599422454834, + 0.11701449006795883, + 0.05281931161880493, + 0.06312280893325806 + ], + "group_id": "ms-0345d8ac530f88cefd61", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.007898053154349327, + 0.0058197579346597195, + 0.052657388150691986, + 0.016109248623251915, + 0.1698339879512787, + 0.008177022449672222, + 0.12333428859710693, + 0.1150994747877121, + 0.15141737461090088, + 0.08515926450490952, + 0.1523766964673996, + 0.04073335975408554, + 0.027175527065992355, + 0.1798098385334015, + 0.08658158779144287, + 0.0618208646774292 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2244938164949417, + -0.708204448223114, + 1.4058912992477417, + 0.19480958580970764, + 1.1751772165298462, + -1.2265020608901978, + -0.8636159896850586, + -0.07021944224834442, + 1.2708500623703003, + 1.0246398448944092, + -1.7615737915039062, + -0.44495686888694763, + -1.2927910089492798, + 0.27074551582336426, + -1.6349679231643677, + 0.9748592376708984, + 1.0210484266281128, + 0.36906859278678894, + -2.528050661087036, + 0.7326674461364746, + 1.553327202796936 + ], + [ + 0.35595428943634033, + -0.8224175572395325, + 1.1192705631256104, + 0.4673205614089966, + 1.1115962266921997, + -1.4676826000213623, + -1.057816505432129, + -0.008461203426122665, + 0.9729399085044861, + 0.8019976615905762, + -1.4944581985473633, + -0.21817980706691742, + -1.3722625970840454, + 0.3557305932044983, + -1.8202308416366577, + 1.2552388906478882, + 0.7855625748634338, + 0.5709130167961121, + -2.4140331745147705, + 0.9063631296157837, + 1.282220721244812 + ], + [ + 0.19927366077899933, + -0.7607958912849426, + 1.4928451776504517, + 0.38265493512153625, + 1.2970150709152222, + -1.2964622974395752, + -0.5547049045562744, + -0.2585906684398651, + 1.3020356893539429, + 1.0737547874450684, + -1.5593467950820923, + -0.1617565155029297, + -1.4868489503860474, + 0.15243440866470337, + -1.4100472927093506, + 0.7991768717765808, + 1.0339381694793701, + 0.6228882074356079, + -2.8132376670837402, + 0.7685560584068298, + 1.8755284547805786 + ], + [ + 0.6393168568611145, + -0.2793915271759033, + 1.4690577983856201, + 0.4745514690876007, + 0.9989874362945557, + -0.7359552979469299, + -1.2171094417572021, + -0.39386802911758423, + 1.1646839380264282, + 0.6661151051521301, + -1.9162347316741943, + -0.43809595704078674, + -1.3973355293273926, + 0.7664861679077148, + -1.553026795387268, + 1.1995127201080322, + 1.285676121711731, + 0.822955846786499, + -2.574894666671753, + 0.3542059361934662, + 1.5986168384552002 + ], + [ + 0.26509445905685425, + -0.5443360805511475, + 1.2502005100250244, + 0.6112543940544128, + 0.924185037612915, + -1.0554168224334717, + -0.7927505373954773, + -0.07152043282985687, + 0.9228929281234741, + 1.545881986618042, + -1.6973015069961548, + -0.496639221906662, + -0.8727940320968628, + 0.5596445202827454, + -2.1939897537231445, + 1.239556074142456, + 1.4699029922485352, + -0.182618647813797, + -2.7263355255126953, + 0.976334810256958, + 1.0402796268463135 + ], + [ + 0.1353418529033661, + -1.1921336650848389, + 1.6913124322891235, + -0.43279919028282166, + 1.1331026554107666, + -1.8503069877624512, + -0.46699997782707214, + 0.5785945653915405, + 1.5516390800476074, + 0.38185298442840576, + -2.192314386367798, + -0.6304612159729004, + -1.6237419843673706, + -0.06383657455444336, + -1.2674099206924438, + 0.7967972755432129, + 1.2453874349594116, + 0.01532512903213501, + -2.6341590881347656, + 0.8366512656211853, + 1.9056174755096436 + ], + [ + 1.0916199684143066, + 0.23665857315063477, + 1.7737994194030762, + -0.8046588897705078, + 1.944507360458374, + -1.3606563806533813, + -0.022103488445281982, + 0.5769228935241699, + 1.0605480670928955, + 1.3100652694702148, + -1.9243974685668945, + 0.43040379881858826, + -1.9046732187271118, + -0.49132758378982544, + -1.9095335006713867, + 1.8850300312042236, + 1.366606593132019, + 0.37651723623275757, + -2.86037540435791, + 0.7753443717956543, + 2.072996139526367 + ], + [ + 0.7277938723564148, + 0.23822063207626343, + 0.4208102226257324, + -0.18717050552368164, + 1.1951282024383545, + -1.1622966527938843, + -0.6617333292961121, + -0.6339871883392334, + 0.8901646137237549, + 0.9977425932884216, + -1.8568902015686035, + 0.3291373550891876, + -0.9377886056900024, + 1.0705609321594238, + -0.90934818983078, + 1.7013394832611084, + 1.5952999591827393, + -0.05327540636062622, + -2.546459197998047, + 0.15792781114578247, + 1.2277774810791016 + ], + [ + 0.9283657073974609, + -1.421413779258728, + 1.6497737169265747, + -0.1984737515449524, + 0.9705085158348083, + -0.8917888402938843, + 0.026184558868408203, + 0.09181642532348633, + 0.6335604786872864, + 1.6531281471252441, + -1.4918432235717773, + -0.01735416054725647, + -0.5425058603286743, + 0.7866935133934021, + -0.7011347413063049, + 0.9797741770744324, + 1.6596866846084595, + -0.13051137328147888, + -2.692556858062744, + -0.05617690086364746, + 0.9771407842636108 + ] + ], + "predicted_scores": [ + 7.805641174316406, + 7.689274787902832, + 7.685960292816162, + 7.47231388092041, + 7.412343978881836, + 7.375511169433594, + 6.744673728942871, + 6.324068069458008 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train1", + "selected_index": 0, + "state_hash": "1ca7c651b2dc4638488078318fe8aa293aa6f6da78672d440752daa9f2d4e8a3", + "stored_base_utility": 0.007898053154349327, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07743936777114868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07743936777114868 + }, + "base_progress": 0.07743936777114868, + "base_success": false, + "base_utility": 0.07743936777114868, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008482594974339008, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008482594974339008 + }, + { + "contact_quality": null, + "progress": 0.01349447201937437, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01349447201937437 + }, + { + "contact_quality": null, + "progress": 0.01016932725906372, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01016932725906372 + }, + { + "contact_quality": null, + "progress": 0.009373247623443604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009373247623443604 + }, + { + "contact_quality": null, + "progress": 0.010430972091853619, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010430972091853619 + }, + { + "contact_quality": null, + "progress": 0.009613573551177979, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009613573551177979 + }, + { + "contact_quality": null, + "progress": 0.013550917617976665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013550917617976665 + }, + { + "contact_quality": null, + "progress": 0.012914061546325684, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012914061546325684 + } + ], + "candidate_progress": [ + 0.008482594974339008, + 0.01349447201937437, + 0.01016932725906372, + 0.009373247623443604, + 0.010430972091853619, + 0.009613573551177979, + 0.013550917617976665, + 0.012914061546325684 + ], + "candidate_source_chart_ids": [ + "ms-61bde0225d6f4175b7cb", + "ms-d6da14bffe59c0265db7", + "ms-61bde0225d6f4175b7cb", + "ms-02311c7783bc67e188b4", + "ms-61bde0225d6f4175b7cb", + "ms-02311c7783bc67e188b4", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-0819eb89e627595e7e1b", + "generated_tangents": [ + [ + -0.48170340061187744, + -0.21298520267009735, + 1.315117359161377, + 0.6920210123062134, + 0.8527080416679382, + 2.968205213546753, + 2.1501686573028564, + -0.38050714135169983, + -0.048633918166160583, + 1.1450988054275513, + 0.2232743203639984, + 0.8262364268302917, + 2.342627763748169, + 2.3650028705596924, + -0.36532288789749146, + -0.06950965523719788, + 1.331042766571045, + 0.2704000473022461, + 0.6595650911331177, + 0.8094047904014587, + 1.3616341352462769 + ], + [ + -0.17983202636241913, + 0.14181523025035858, + 1.1113064289093018, + 0.33009806275367737, + 1.7241795063018799, + 2.376654624938965, + 1.1097676753997803, + 0.05465759336948395, + 0.09483547508716583, + 1.7570924758911133, + -0.1716618537902832, + 0.960301399230957, + 1.9596649408340454, + 2.3049609661102295, + -0.7374430894851685, + -0.03532176837325096, + 0.5762314796447754, + 0.7728564143180847, + 0.9117649793624878, + 1.0454025268554688, + 0.3472132682800293 + ], + [ + -0.31008750200271606, + -0.12151334434747696, + 1.2201170921325684, + 0.19237235188484192, + 0.738303542137146, + 1.855515956878662, + 1.6113858222961426, + -0.37851646542549133, + 0.030104465782642365, + 1.3002598285675049, + 0.01885172352194786, + 0.9363909959793091, + 2.283543109893799, + 1.0001118183135986, + -0.49234700202941895, + -0.06432265043258667, + 1.016754388809204, + 0.5973330736160278, + 0.5052860975265503, + 1.2347865104675293, + 0.29192376136779785 + ], + [ + -0.2867894172668457, + -0.13257068395614624, + 1.4791605472564697, + 0.12920957803726196, + 0.975943386554718, + 2.172316312789917, + 1.9733011722564697, + -0.2797960937023163, + -0.054635435342788696, + 1.6207730770111084, + 0.18025602400302887, + 0.7074676156044006, + 1.452094554901123, + 2.140620708465576, + -0.23117363452911377, + 0.001552867703139782, + 0.9780837297439575, + 0.4659976363182068, + 0.7417269945144653, + 1.0483758449554443, + 0.8611186742782593 + ], + [ + -0.40189963579177856, + -0.1369675099849701, + 1.2212437391281128, + 0.1413283348083496, + 0.8650162220001221, + 1.8716392517089844, + 1.6848230361938477, + -0.4276892840862274, + 0.04519989341497421, + 1.122190237045288, + -0.018321841955184937, + 0.7497153878211975, + 2.335747480392456, + 1.097080945968628, + -0.4920797348022461, + -0.019546911120414734, + 1.1053532361984253, + 0.5397530198097229, + 0.4506194591522217, + 1.1697564125061035, + 0.3101489543914795 + ], + [ + -0.3615391254425049, + -0.14622633159160614, + 1.4674757719039917, + 0.10478755831718445, + 0.9942489266395569, + 2.164763927459717, + 1.7419607639312744, + -0.27240440249443054, + -0.04424624145030975, + 1.5607510805130005, + 0.17398256063461304, + 0.7515172958374023, + 1.4062234163284302, + 2.099848747253418, + -0.1732201874256134, + -0.03870294988155365, + 0.9826485514640808, + 0.5652058720588684, + 0.8408015370368958, + 0.9496859908103943, + 0.6621719598770142 + ], + [ + -0.046975865960121155, + 0.024709030985832214, + 1.1837114095687866, + 0.10256123542785645, + 1.2242372035980225, + 2.677042245864868, + 1.2298593521118164, + -0.22442549467086792, + -0.009645089507102966, + 1.488405466079712, + 0.21199652552604675, + 0.42241954803466797, + 1.7646032571792603, + 1.6054023504257202, + -0.39166170358657837, + -0.08306765556335449, + 0.7367458939552307, + 0.873846709728241, + 0.7609628438949585, + 0.3406764268875122, + -0.1312771737575531 + ], + [ + -0.08826951682567596, + 0.086859330534935, + 1.1238945722579956, + -0.005096703767776489, + 1.1703311204910278, + 2.7813732624053955, + 1.3670010566711426, + -0.2854817807674408, + 0.013575018383562565, + 1.4255802631378174, + 0.27501311898231506, + 0.44776588678359985, + 1.9851607084274292, + 1.3412727117538452, + -0.36598777770996094, + -0.09238594770431519, + 0.7466386556625366, + 0.863097071647644, + 0.8219146132469177, + 0.35138940811157227, + -0.11985167860984802 + ] + ], + "generated_utilities": [ + 0.008482594974339008, + 0.01349447201937437, + 0.01016932725906372, + 0.009373247623443604, + 0.010430972091853619, + 0.009613573551177979, + 0.013550917617976665, + 0.012914061546325684 + ], + "group_id": "ms-0819eb89e627595e7e1b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07638250291347504, + 0.11176031827926636, + 0.139996737241745, + 0.06282445043325424, + 0.08466696739196777, + 0.0763862133026123, + 0.05953681468963623, + 0.06854937970638275, + 0.06834080070257187, + 0.09309522807598114, + 0.14572584629058838, + 0.07182137668132782, + 0.020047685131430626, + 0.05106111615896225, + 0.03756682202219963, + 0.061605654656887054 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.04522436857223511, + -0.3834587037563324, + 2.32004976272583, + 1.6386330127716064, + 1.508955717086792, + 1.681371808052063, + 2.9060468673706055, + -0.5984516143798828, + 0.36428770422935486, + 0.7442611455917358, + 1.163742184638977, + 0.4921322464942932, + 1.9586588144302368, + 3.335318088531494, + -0.02929070219397545, + 0.5442408919334412, + 2.4529502391815186, + 0.17471881210803986, + 1.804263710975647, + 1.5498225688934326, + 1.9392809867858887 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.19627264142036438, + 0.015629824250936508, + 0.723259687423706, + 0.0313839316368103, + 0.02149224281311035, + 0.7142634391784668, + 0.0, + -0.29767167568206787, + -0.10304851830005646, + 0.6168417930603027, + 0.0271989107131958, + 0.369077205657959, + 1.5924487113952637, + 0.0, + -0.11617732048034668, + -0.1419914811849594, + 0.9400725364685059, + -0.04139697551727295, + 0.7166774272918701, + 1.092864990234375, + 0.0 + ], + [ + -0.18197131156921387, + 0.30914804339408875, + 1.4557228088378906, + 1.4375734329223633, + 0.9600651264190674, + 1.7448713779449463, + 3.031123399734497, + -0.6073551177978516, + -0.4749099016189575, + 1.3016932010650635, + 0.32392996549606323, + 0.6961254477500916, + 2.1961381435394287, + 3.730100154876709, + -0.6154190897941589, + -0.3908892273902893, + 2.0994153022766113, + 0.18937554955482483, + 1.5752670764923096, + 2.2457275390625, + 1.5629570484161377 + ] + ], + "predicted_scores": [ + 8.962811470031738, + 8.02236557006836, + 7.837157249450684, + 7.748305797576904, + 7.6356892585754395, + 7.552002429962158, + 7.480719566345215, + 7.393957614898682 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train1", + "selected_index": 0, + "state_hash": "582daf047d78a1b01018d54f5798143c4be6129fd7b6cbc025fc85818e519f3c", + "stored_base_utility": 0.07638250291347504, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19400137662887573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19400137662887573 + }, + "base_progress": 0.19400137662887573, + "base_success": false, + "base_utility": 0.19400137662887573, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03977566957473755, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03977566957473755 + }, + { + "contact_quality": null, + "progress": 0.035518765449523926, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035518765449523926 + }, + { + "contact_quality": null, + "progress": 0.037184059619903564, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037184059619903564 + }, + { + "contact_quality": null, + "progress": 0.04727194830775261, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04727194830775261 + }, + { + "contact_quality": null, + "progress": 0.1163921132683754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1163921132683754 + }, + { + "contact_quality": null, + "progress": 0.039057254791259766, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.039057254791259766 + }, + { + "contact_quality": null, + "progress": 0.03655803203582764, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03655803203582764 + }, + { + "contact_quality": null, + "progress": 0.059245508164167404, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.059245508164167404 + } + ], + "candidate_progress": [ + 0.03977566957473755, + 0.035518765449523926, + 0.037184059619903564, + 0.04727194830775261, + 0.1163921132683754, + 0.039057254791259766, + 0.03655803203582764, + 0.059245508164167404 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-25ab20bb3da8ba34be95", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "generated_tangents": [ + [ + 1.4961535930633545, + 0.05711425095796585, + -0.8380736112594604, + -0.07539501786231995, + 0.31329917907714844, + -2.6488101482391357, + -1.0738818645477295, + -0.968364953994751, + -0.2544800043106079, + 0.4324849545955658, + 0.6627989411354065, + -0.28087976574897766, + -0.10343340039253235, + -0.02613060176372528, + -0.7529232501983643, + -0.30540382862091064, + 1.263474702835083, + 1.0633282661437988, + -0.3979564309120178, + -0.3800346851348877, + 0.41512221097946167 + ], + [ + 1.3072959184646606, + 0.1105508804321289, + -1.073121428489685, + 0.022990912199020386, + 0.5359436273574829, + -2.6330926418304443, + -0.9600021243095398, + -0.8445543050765991, + -0.1756526082754135, + 0.4960522949695587, + 0.9204374551773071, + -0.1704983115196228, + 0.166866272687912, + 0.08802354335784912, + -0.9753002524375916, + -0.4139297604560852, + 1.4275983572006226, + 1.2034046649932861, + -0.23188930749893188, + 0.3031361699104309, + 0.40181833505630493 + ], + [ + 1.308129906654358, + 0.13780780136585236, + -0.9447993040084839, + 0.24220460653305054, + 0.3176955580711365, + -2.2120866775512695, + -0.9479312300682068, + -1.114196538925171, + -0.15016527473926544, + 0.4687451422214508, + 0.6838050484657288, + -0.22989246249198914, + 0.43679264187812805, + -0.037165701389312744, + -0.783089280128479, + -0.2977108359336853, + 1.2360730171203613, + 1.1537435054779053, + -0.3863477110862732, + -0.001897364854812622, + 0.1392533779144287 + ], + [ + 1.3749992847442627, + 0.2382805496454239, + -1.0212786197662354, + 0.15463030338287354, + 0.38636261224746704, + -2.4216132164001465, + -0.9702731966972351, + -0.7767324447631836, + -0.18187862634658813, + 0.48575687408447266, + 0.7979539036750793, + -0.17321276664733887, + 0.08161547780036926, + -0.05776302516460419, + -0.9654657244682312, + -0.3173145651817322, + 1.4733184576034546, + 1.1904693841934204, + -0.20466378331184387, + 0.3063088655471802, + 0.34561169147491455 + ], + [ + 1.6630033254623413, + 0.061981506645679474, + -1.2176945209503174, + 0.16619658470153809, + 0.615795373916626, + -2.1796154975891113, + -1.4509468078613281, + -0.5507602691650391, + -0.08558899164199829, + 0.42375171184539795, + 0.5340151786804199, + -0.13156241178512573, + 0.11130085587501526, + -0.16660358011722565, + -1.1031697988510132, + -0.2901856601238251, + 1.0634026527404785, + 1.2348135709762573, + 0.20170143246650696, + 0.18915006518363953, + -0.20062124729156494 + ], + [ + 1.2891536951065063, + 0.1130342036485672, + -1.0005327463150024, + 0.2583622932434082, + 0.5921269655227661, + -2.4853010177612305, + -0.9217981696128845, + -0.747860312461853, + -0.18771840631961823, + 0.4645868241786957, + 0.9687589406967163, + -0.21854183077812195, + -0.012618690729141235, + 0.22672508656978607, + -0.97975093126297, + -0.2944067418575287, + 1.4229469299316406, + 1.1545466184616089, + -0.2660664916038513, + 0.30267637968063354, + 0.28235435485839844 + ], + [ + 1.1726384162902832, + 0.014311477541923523, + -0.8084231615066528, + 0.16792312264442444, + 0.046148285269737244, + -2.0844624042510986, + -0.8476629853248596, + -1.0396668910980225, + 0.054600343108177185, + 0.7791575193405151, + 0.3479359447956085, + -0.40113091468811035, + -0.083014577627182, + 0.16177353262901306, + -1.2966257333755493, + -0.18281859159469604, + 1.1668022871017456, + 1.0944393873214722, + -0.06395062804222107, + 0.33366262912750244, + 0.6274118423461914 + ], + [ + 1.6430341005325317, + 0.23615755140781403, + -0.6891779899597168, + 0.22973570227622986, + 0.5483315587043762, + -2.0632615089416504, + -1.6444897651672363, + -0.6461735963821411, + -0.22430431842803955, + 0.5634874105453491, + 0.8024416565895081, + 0.06881408393383026, + 0.5734598636627197, + -0.7092761993408203, + -0.931363046169281, + -0.1820114254951477, + 1.4591095447540283, + 1.3638323545455933, + 0.09794610738754272, + -0.029864966869354248, + 0.16142325103282928 + ] + ], + "generated_utilities": [ + 0.03977566957473755, + 0.035518765449523926, + 0.037184059619903564, + 0.04727194830775261, + 0.1163921132683754, + 0.039057254791259766, + 0.03655803203582764, + 0.059245508164167404 + ], + "group_id": "ms-0dee962a5901fb8b5bbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23161475360393524, + 0.23611454665660858, + 0.1380448043346405, + 2.0, + 0.25444933772087097, + 0.25831863284111023, + 0.22753827273845673, + 0.2483263909816742, + 0.21622240543365479, + 0.17534737288951874, + 0.23816657066345215, + 0.09714888036251068, + 0.05844185873866081, + 0.12679120898246765, + 0.0472835898399353, + 0.03277532383799553 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7612967491149902, + 0.192471444606781, + -0.2699308395385742, + 0.017076104879379272, + -0.14400714635849, + -1.9032442569732666, + 0.0, + -0.28746497631073, + -0.2857179641723633, + -0.09024286270141602, + -0.03538453206419945, + 0.06432074308395386, + -0.2535901665687561, + 0.0, + 0.06180626153945923, + 0.16256888210773468, + -0.031329914927482605, + 1.563896656036377, + 0.04795600473880768, + 0.02485179901123047, + 0.0 + ], + [ + 1.2132649421691895, + 0.536715567111969, + -1.4273607730865479, + -1.0538015365600586, + -0.5170869827270508, + -3.4203505516052246, + -2.3889083862304688, + -0.7717514634132385, + -1.1465628147125244, + 0.8393621444702148, + -0.4419952630996704, + -0.23069629073143005, + -0.0637764036655426, + -1.5193647146224976, + -1.0273863077163696, + 0.39812424778938293, + -0.02874515950679779, + 2.7962026596069336, + 0.2370568960905075, + 1.5884850025177002, + 0.9987253546714783 + ], + [ + 2.1451404094696045, + 0.09629477560520172, + -1.8500688076019287, + 0.012462526559829712, + 0.13087880611419678, + -2.4062256813049316, + -3.303264856338501, + -1.3855981826782227, + -0.0292893648147583, + -0.17000514268875122, + 0.0005041323602199554, + -0.7487195134162903, + 0.06868582963943481, + -2.107095718383789, + -0.5890907645225525, + 0.1777246743440628, + -0.5576009750366211, + 2.767899990081787, + -0.3420543074607849, + 1.1357430219650269, + 1.8840694427490234 + ], + [ + 2.032343626022339, + 0.4649246335029602, + -0.5597869753837585, + 0.37935349345207214, + 0.14823919534683228, + -3.389478921890259, + -1.972822904586792, + -0.0014220476150512695, + -0.6147069334983826, + 0.9483420252799988, + 0.641323983669281, + -0.7033213973045349, + 0.4629128575325012, + -1.6774024963378906, + -1.1420938968658447, + 0.5891244411468506, + -0.712251603603363, + 2.1924612522125244, + 0.18694229423999786, + 0.2795591354370117, + 2.4723148345947266 + ], + [ + 2.299381732940674, + -0.6742787957191467, + -0.886760950088501, + -0.1637115478515625, + 0.32663029432296753, + -1.9267126321792603, + -1.9424545764923096, + -0.3586595058441162, + -1.1000827550888062, + 0.45532336831092834, + -0.4314403235912323, + -0.12869995832443237, + -0.032513588666915894, + -2.046614170074463, + -1.2942235469818115, + 0.3962012231349945, + 0.797533392906189, + 2.202794313430786, + 0.4938191771507263, + 0.15863806009292603, + 1.745660662651062 + ], + [ + 2.107815742492676, + 0.963295042514801, + -1.2979395389556885, + -1.0134333372116089, + -1.319197177886963, + -3.176518440246582, + -2.4782180786132812, + -1.3465216159820557, + -1.0473096370697021, + -0.22631770372390747, + 0.7881518006324768, + -1.4033093452453613, + 0.3855947256088257, + -1.3816941976547241, + -1.1257284879684448, + -0.7653472423553467, + 0.6845007538795471, + 2.9796371459960938, + 0.1967305988073349, + 1.4200057983398438, + 1.2843091487884521 + ], + [ + 1.7303649187088013, + 1.0728741884231567, + -1.1676844358444214, + -0.20355769991874695, + -0.8773125410079956, + -3.2701919078826904, + -2.1479928493499756, + -0.044036924839019775, + -1.252698302268982, + -0.01393243670463562, + 0.6447213292121887, + -0.7088868618011475, + 0.11592492461204529, + -2.1431162357330322, + -0.8284967541694641, + 0.4642922580242157, + -0.5207862854003906, + 1.8997660875320435, + -0.6910213828086853, + 0.2906065583229065, + 1.4081878662109375 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7612967491149902, + -0.5331565737724304, + -0.2699308395385742, + -0.20421931147575378, + -0.22613269090652466, + -1.9032442569732666, + 0.0, + 0.09110367298126221, + 0.009229481220245361, + 0.004811525344848633, + 0.656868577003479, + -0.28485387563705444, + -0.31901538372039795, + 0.0, + 0.08918565511703491, + 0.3355104923248291, + 0.026827186346054077, + 1.563896656036377, + 0.13480885326862335, + 0.02485179901123047, + 0.0 + ] + ], + "predicted_scores": [ + 8.347381591796875, + 8.0756196975708, + 7.89379358291626, + 7.591610431671143, + 7.5797038078308105, + 7.3529510498046875, + 7.292175769805908, + 6.160061359405518 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train1", + "selected_index": 0, + "state_hash": "73f2826502af62b561f13c32614947588918c3deb631165739e66d787008fa98", + "stored_base_utility": 0.23161475360393524, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.04583283141255379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04583283141255379 + }, + "base_progress": 0.04583283141255379, + "base_success": false, + "base_utility": 0.04583283141255379, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10429390519857407, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10429390519857407 + }, + { + "contact_quality": null, + "progress": 0.08981126546859741, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08981126546859741 + }, + { + "contact_quality": null, + "progress": 0.09254121780395508, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09254121780395508 + }, + { + "contact_quality": null, + "progress": 0.09323513507843018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09323513507843018 + }, + { + "contact_quality": null, + "progress": 0.1434544175863266, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1434544175863266 + }, + { + "contact_quality": null, + "progress": 0.17300955951213837, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17300955951213837 + }, + { + "contact_quality": null, + "progress": 0.07903516292572021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07903516292572021 + }, + { + "contact_quality": null, + "progress": 0.09951392561197281, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09951392561197281 + } + ], + "candidate_progress": [ + 0.10429390519857407, + 0.08981126546859741, + 0.09254121780395508, + 0.09323513507843018, + 0.1434544175863266, + 0.17300955951213837, + 0.07903516292572021, + 0.09951392561197281 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-7a044cb6c7bb03f3ea10", + "ms-38dff128761905bb1507" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-126eb8b36769f9ba4671", + "generated_tangents": [ + [ + 1.369253158569336, + -0.06315869092941284, + -1.080492377281189, + 0.09179672598838806, + -0.2259550392627716, + -2.9094576835632324, + -2.2019600868225098, + 0.937443733215332, + -0.023995475843548775, + 0.0738721489906311, + -0.1052713468670845, + 0.8359808921813965, + -1.4244933128356934, + -0.34113651514053345, + -1.115409016609192, + -0.15458451211452484, + 0.9955348372459412, + 0.5989454984664917, + 1.2079029083251953, + 1.6112141609191895, + 1.669655203819275 + ], + [ + 1.2912083864212036, + -0.19512198865413666, + -0.994770884513855, + 0.07655856013298035, + -0.049782752990722656, + -2.7013258934020996, + -1.6116347312927246, + 0.7569692134857178, + -0.018285728991031647, + 0.1288003921508789, + -0.2150503396987915, + 0.7750152349472046, + -1.0736428499221802, + -0.5305038690567017, + -0.9757967591285706, + -0.22260208427906036, + 1.1594197750091553, + 0.4849216938018799, + 0.7854194045066833, + 0.9466932415962219, + 1.5160409212112427 + ], + [ + 1.578649640083313, + 0.02785012125968933, + -1.048768162727356, + 0.10640284419059753, + -0.25175657868385315, + -3.3711423873901367, + -2.2970876693725586, + 0.8743171691894531, + 0.06778844445943832, + 0.1173376590013504, + -0.26985153555870056, + 0.9261730909347534, + -1.1985284090042114, + -0.24936388432979584, + -1.0118955373764038, + -0.11626236140727997, + 0.8804306983947754, + 0.7422818541526794, + 1.240435004234314, + 1.416860580444336, + 1.6711128950119019 + ], + [ + 1.4255388975143433, + -0.0031580328941345215, + -1.0690832138061523, + 0.10249030590057373, + -0.2593812644481659, + -3.1884021759033203, + -2.243687868118286, + 0.930073618888855, + 0.011781872250139713, + 0.10563268512487411, + -0.11607181280851364, + 0.9018707275390625, + -1.4074382781982422, + -0.26366329193115234, + -1.119073510169983, + -0.16597124934196472, + 1.0663888454437256, + 0.6679109334945679, + 1.2316689491271973, + 1.572657585144043, + 1.5834970474243164 + ], + [ + 1.5968835353851318, + -0.11812939494848251, + -0.951828122138977, + 0.28660598397254944, + 0.2038881480693817, + -2.8580515384674072, + -2.4814679622650146, + 0.6503607034683228, + -0.05494828522205353, + -0.08175159245729446, + -0.18631088733673096, + 0.776267409324646, + -0.5315053462982178, + 0.2381022572517395, + -0.4410700798034668, + 0.003059820272028446, + 0.4818568229675293, + 0.5917049646377563, + 1.395589828491211, + 0.8140818476676941, + 1.0847562551498413 + ], + [ + 1.5958155393600464, + -0.14150354266166687, + -1.0692274570465088, + 0.45605435967445374, + 0.0012110322713851929, + -2.7148196697235107, + -2.5512993335723877, + 0.570069432258606, + -0.04990261793136597, + -0.1281956136226654, + -0.10317870229482651, + 0.8229105472564697, + -0.5673810243606567, + 0.19114671647548676, + -0.4908102750778198, + -0.04287712275981903, + 0.4610198438167572, + 0.6574446558952332, + 1.4220271110534668, + 0.718690812587738, + 0.9527000188827515 + ], + [ + 1.1580145359039307, + -0.18998830020427704, + -0.8047693967819214, + 0.0764504075050354, + -0.06095534563064575, + -2.9506022930145264, + -1.5477747917175293, + 0.726123571395874, + -0.0107507836073637, + 0.12318387627601624, + -0.031517282128334045, + 0.9004573225975037, + -1.2188684940338135, + -0.2621389627456665, + -0.974763810634613, + -0.2896084785461426, + 1.1361896991729736, + 0.5284179449081421, + 0.9712277054786682, + 1.2001430988311768, + 1.5238471031188965 + ], + [ + 1.5085656642913818, + -0.12596197426319122, + -0.7328596115112305, + 0.3691893517971039, + 0.1983453780412674, + -3.1568164825439453, + -1.879227876663208, + 0.5138417482376099, + 0.11119545996189117, + -0.16166532039642334, + 0.4548158347606659, + 0.729375422000885, + -0.1761901080608368, + -0.025958046317100525, + -0.8443230390548706, + 0.015648137778043747, + 1.0883651971817017, + 0.6242193579673767, + 0.7178710699081421, + 1.7087934017181396, + 1.506760597229004 + ] + ], + "generated_utilities": [ + 0.10429390519857407, + 0.08981126546859741, + 0.09254121780395508, + 0.09323513507843018, + 0.1434544175863266, + 0.17300955951213837, + 0.07903516292572021, + 0.09951392561197281 + ], + "group_id": "ms-126eb8b36769f9ba4671", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05304628610610962, + 0.14190539717674255, + 0.1985912322998047, + 0.1210482120513916, + 0.18570420145988464, + 0.05259696766734123, + 0.1827520728111267, + 0.15291261672973633, + 0.07772710174322128, + 0.15551692247390747, + 0.08802877366542816, + 0.07544025033712387, + 0.08058679103851318, + 0.10843702405691147, + 0.023751219734549522, + 0.00727109145373106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7799564599990845, + -0.0863567665219307, + -0.04370701313018799, + -0.04603701829910278, + -0.003975361585617065, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.02617853879928589, + 0.009054642170667648, + -0.050068795680999756, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + -0.007249712944030762, + 0.0760425329208374, + 1.0404269695281982, + -0.04351449012756348, + -0.010379016399383545, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + 0.005863763391971588, + -0.04370701313018799, + 0.010433673858642578, + -0.05825299024581909, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.22243881225585938, + -0.007246788591146469, + 0.19961372017860413, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.11177664995193481, + -0.1665668487548828, + 1.0404269695281982, + -0.09226274490356445, + -0.18212640285491943, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + -0.7482962012290955, + -0.04370701313018799, + -0.2370874285697937, + 0.3763168752193451, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + -0.441777765750885, + -0.44057023525238037, + -0.6443575024604797, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.008161306381225586, + 0.22021520137786865, + 1.0404269695281982, + -0.09226274490356445, + -0.35497862100601196, + 1.0522968769073486, + 0.0 + ], + [ + 1.7799564599990845, + 0.04087676852941513, + -1.043707013130188, + -0.8482787609100342, + 0.3640388548374176, + -3.3430333137512207, + -2.8715221881866455, + 1.8326367139816284, + -0.4197058081626892, + -0.040397901087999344, + -0.37471577525138855, + 1.7451434135437012, + -1.6306462287902832, + -0.7785218954086304, + -0.5121169090270996, + -0.7797847986221313, + 2.0404269695281982, + 0.9077372550964355, + 0.645021378993988, + 2.0522968769073486, + 1.2749258279800415 + ], + [ + 1.6519862413406372, + 0.1747584044933319, + -0.9129671454429626, + -0.6869257092475891, + 0.12357090413570404, + -3.2319328784942627, + -2.704270839691162, + 1.7347972393035889, + -0.40612348914146423, + 0.23568139970302582, + -0.3912501931190491, + 1.8630532026290894, + -1.8548930883407593, + -0.9794526100158691, + -0.25177696347236633, + -0.6884964108467102, + 2.237109422683716, + 1.1246912479400635, + 0.7490227222442627, + 2.320941686630249, + 1.2039268016815186 + ], + [ + 1.7142105102539062, + 0.3941279351711273, + -1.021533489227295, + -1.1548089981079102, + 0.5052449107170105, + -3.714951992034912, + -3.04028058052063, + 1.529634952545166, + -0.22215746343135834, + 0.19690367579460144, + -0.12262976169586182, + 1.4847477674484253, + -1.490212082862854, + -0.7427945137023926, + -0.7506002187728882, + -0.9221256375312805, + 1.9211885929107666, + 0.5114957094192505, + 0.30887648463249207, + 2.4437777996063232, + 1.3625030517578125 + ], + [ + 1.3140592575073242, + -0.1129835918545723, + -0.9130957722663879, + -1.1089756488800049, + 0.7560886144638062, + -3.3889732360839844, + -3.1182656288146973, + 1.7436503171920776, + -0.9962512254714966, + 0.43109408020973206, + -0.8453629016876221, + 1.2057833671569824, + -2.194112777709961, + -0.22479140758514404, + -0.7258806824684143, + -0.9702881574630737, + 1.8160195350646973, + 0.649415135383606, + 0.6846981048583984, + 2.1997463703155518, + 1.7655411958694458 + ], + [ + 2.6430320739746094, + -0.7696453332901001, + -1.765415906906128, + -0.8281408548355103, + -0.1440155804157257, + -3.3335886001586914, + -3.398141860961914, + 2.4138498306274414, + 0.47537559270858765, + -0.9514593482017517, + 0.2674855887889862, + 2.6307411193847656, + -0.7876039743423462, + -0.35013434290885925, + -0.7511636018753052, + -0.04397457838058472, + 2.123702049255371, + 1.1604151725769043, + 0.7323322892189026, + 1.88338303565979, + 1.5078264474868774 + ] + ], + "predicted_scores": [ + 6.472078800201416, + 6.394108295440674, + 6.16207218170166, + 6.00885009765625, + 5.819354057312012, + 5.714887619018555, + 5.364053249359131, + 5.0096964836120605 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train1", + "selected_index": 0, + "state_hash": "2390fc23ce8b50bdcdcbb1ab5f0583d2dad2b94fb871db73b1a8401265f59987", + "stored_base_utility": 0.05304628610610962, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-cad6645f8175e38ad155", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-babaeb20129c46551281", + "ms-1f19fd22562f87bdb776", + "ms-c1620854ba52f09e24d5", + "ms-067cac2e4dbeddb12239", + "ms-067cac2e4dbeddb12239" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-16661b2fd2c85f57a66c", + "generated_tangents": [ + [ + -0.18380220234394073, + -0.24929334223270416, + 1.262943148612976, + 0.03600206971168518, + 0.6148139834403992, + 2.245206356048584, + 2.704965114593506, + 1.3962924480438232, + 0.010732992552220821, + -0.047270484268665314, + -0.05522475391626358, + 0.6706928014755249, + 2.471439838409424, + 1.5786356925964355, + 0.8490850925445557, + -0.3888009786605835, + -1.2239018678665161, + -0.33014625310897827, + -0.20082581043243408, + -1.5346240997314453, + -2.185572624206543 + ], + [ + -0.315105676651001, + -0.5645406246185303, + 1.2408192157745361, + -0.16152966022491455, + 0.7254892587661743, + 1.8949651718139648, + 2.4952545166015625, + 1.769245982170105, + 0.08672025799751282, + 0.04487885534763336, + -0.36564403772354126, + 0.8081681728363037, + 1.9177643060684204, + 2.0841763019561768, + 0.9675657153129578, + 0.02687590941786766, + -1.5230522155761719, + -0.45724719762802124, + -0.15677958726882935, + -1.3005516529083252, + -2.746328353881836 + ], + [ + -0.37123948335647583, + -0.5263715982437134, + 1.122902274131775, + -0.22985431551933289, + 0.8552618026733398, + 1.766284465789795, + 2.3940672874450684, + 1.7402807474136353, + 0.06402721256017685, + -0.005862034857273102, + -0.3764722943305969, + 0.6854610443115234, + 1.8561197519302368, + 2.278413772583008, + 1.0196105241775513, + 0.04680681228637695, + -1.5317866802215576, + -0.403542160987854, + -0.19488957524299622, + -1.4346392154693604, + -2.7704172134399414 + ], + [ + -0.626388430595398, + -0.346606969833374, + 1.2113229036331177, + -0.6287609338760376, + 1.2463550567626953, + 2.577420711517334, + 2.93330979347229, + 1.5317102670669556, + -0.28259995579719543, + 0.018710888922214508, + -0.1620306372642517, + 0.8240635395050049, + 1.637794852256775, + 1.1724345684051514, + 1.263065218925476, + -0.29181909561157227, + -1.5987719297409058, + -0.3505403995513916, + -0.4393131136894226, + -1.3932018280029297, + -2.506209135055542 + ], + [ + -0.4232896566390991, + -0.6276735663414001, + 1.3259775638580322, + -0.5200741291046143, + 1.3620741367340088, + 2.0042545795440674, + 3.035946846008301, + 1.632419228553772, + -0.5040328502655029, + -0.16054072976112366, + -0.14927446842193604, + 0.37346094846725464, + 1.4454445838928223, + 1.481978416442871, + 1.261528730392456, + -0.20199286937713623, + -1.7897307872772217, + -0.1670408695936203, + -0.10157498717308044, + -1.161111831665039, + -2.15732741355896 + ], + [ + -0.40514546632766724, + -0.10850674659013748, + 0.8810243606567383, + -0.7218068838119507, + 1.0397483110427856, + 2.2089436054229736, + 4.277473449707031, + 0.968049168586731, + -0.20366573333740234, + 0.2481432557106018, + -0.09063944965600967, + 0.19565847516059875, + 0.7704454660415649, + 2.1513984203338623, + 0.9093822836875916, + -0.2663203477859497, + -0.7597174048423767, + -0.0018222182989120483, + -0.27158141136169434, + -0.4617255926132202, + -1.760399341583252 + ], + [ + -0.4684767723083496, + -0.2348737120628357, + 0.8931154608726501, + -0.22056005895137787, + 1.024080514907837, + 2.415644645690918, + 3.497183084487915, + 1.203537940979004, + -0.14209936559200287, + 0.6001217365264893, + 0.0164608433842659, + 0.21254536509513855, + 1.8652161359786987, + 2.435795545578003, + 1.3397101163864136, + -0.16474942862987518, + -1.0032472610473633, + 0.19162122905254364, + -0.17032292485237122, + -1.4556047916412354, + -2.5731472969055176 + ], + [ + -0.3986818790435791, + -0.32030463218688965, + 0.6388684511184692, + -0.2922161817550659, + 1.1398611068725586, + 2.398139238357544, + 3.4355506896972656, + 1.4486161470413208, + -0.1690330058336258, + 0.20826128125190735, + -0.14162737131118774, + 0.22350215911865234, + 1.9856444597244263, + 2.3961915969848633, + 1.4219110012054443, + -0.10985557734966278, + -1.0580624341964722, + 0.24057930707931519, + -0.3399960994720459, + -1.33854079246521, + -2.2894551753997803 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-16661b2fd2c85f57a66c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23635749518871307, + 0.1864171028137207, + 0.18280422687530518, + 0.16576561331748962, + 0.2967458665370941, + 0.2833873927593231, + 0.2587948143482208, + 0.2695174813270569, + 0.2239263951778412, + 0.15686756372451782, + 0.2505500912666321, + 0.12250886857509613, + 0.16453635692596436, + 2.0, + 0.13450095057487488, + 0.11914549767971039 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.12099093198776245, + 0.06886856257915497, + 0.2952747344970703, + -0.1764048933982849, + -0.12411242723464966, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + -0.046127498149871826, + 0.12581980228424072, + 0.042072445154190063, + -0.06942430138587952, + 0.7517287731170654, + 0.0, + 0.690804123878479, + -0.06023237854242325, + 0.049633681774139404, + 0.01943105459213257, + 0.013327360153198242, + -0.8525059223175049, + 0.0 + ], + [ + 0.3599843978881836, + -0.34296315908432007, + 0.2952747344970703, + -0.3797188997268677, + -0.41771262884140015, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + 0.06634362041950226, + 0.138022780418396, + -0.2342309057712555, + -0.36017337441444397, + 0.7517287731170654, + 0.0, + 0.690804123878479, + 0.10550708323717117, + -0.35431206226348877, + 0.16382712125778198, + 0.06674730777740479, + -0.8525059223175049, + 0.0 + ], + [ + 0.567882239818573, + -0.4729353189468384, + 1.6256722211837769, + -1.5372763872146606, + 1.0487556457519531, + 1.5327277183532715, + 2.2847697734832764, + 2.6703484058380127, + 0.4013517200946808, + -1.3629915714263916, + 0.6385155916213989, + 0.2664203345775604, + 1.2691267728805542, + 1.2942520380020142, + 2.2084813117980957, + 0.6982719302177429, + -0.8422143459320068, + 1.070263385772705, + -0.9558742046356201, + -1.3996182680130005, + -1.4242117404937744 + ], + [ + 0.6830785274505615, + -0.5584439635276794, + 0.841848611831665, + -0.5293771624565125, + -0.14354348182678223, + 2.097932815551758, + 2.9733188152313232, + 2.603226661682129, + -0.1639159619808197, + -0.918163537979126, + 1.002030849456787, + 0.40132570266723633, + 1.1751198768615723, + 1.455662727355957, + 2.3202755451202393, + -0.18584474921226501, + -1.3483099937438965, + 0.0660635232925415, + 0.4330971837043762, + -1.1989169120788574, + -2.0896859169006348 + ], + [ + 0.44418102502822876, + 0.07469291985034943, + 1.6960139274597168, + -1.329486608505249, + 1.3067576885223389, + 2.2694292068481445, + 2.0299551486968994, + 1.6317954063415527, + 0.7472026348114014, + -0.12518703937530518, + 0.3035789430141449, + -0.5889836549758911, + 2.345867395401001, + 1.9470158815383911, + 0.8398547172546387, + -0.6090301871299744, + -1.5060018301010132, + 0.5695335865020752, + -0.6025367975234985, + -1.8703374862670898, + -2.728482246398926 + ], + [ + 0.31904691457748413, + 0.04995492100715637, + 1.3823881149291992, + -0.5106721520423889, + -0.2236303687095642, + 0.9921484589576721, + 1.5933295488357544, + 2.741320848464966, + -0.21808990836143494, + -0.7775652408599854, + 1.224307894706726, + -0.08168366551399231, + 2.692474842071533, + 2.269430160522461, + 2.3270838260650635, + -0.2901480495929718, + -1.2098140716552734, + -0.07570821046829224, + -1.1340655088424683, + -1.9999847412109375, + -1.6802332401275635 + ], + [ + -0.6257733106613159, + 0.3840647339820862, + 0.9012271165847778, + -1.248098611831665, + -0.1650761365890503, + 2.7645747661590576, + 2.7588071823120117, + 3.0172548294067383, + 0.4120163321495056, + -1.6110656261444092, + -0.10423526167869568, + 0.5334746241569519, + 1.7809263467788696, + 0.5690644383430481, + 1.5881801843643188, + 0.8860509991645813, + -1.1502506732940674, + 0.7821243405342102, + -0.549727201461792, + -1.326378345489502, + -1.5100535154342651 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2773973345756531, + -0.13796104490756989, + 1.2952747344970703, + -1.0323810577392578, + 0.5605664849281311, + 1.8606208562850952, + 2.439758777618408, + 2.353257894515991, + 0.24642756581306458, + -0.861977219581604, + 0.4799116551876068, + 0.264199823141098, + 1.7517287731170654, + 1.4282559156417847, + 1.690804123878479, + 0.1215878501534462, + -0.9322434067726135, + 0.8536378145217896, + -0.35758620500564575, + -1.8525059223175049, + -1.910990595817566 + ], + [ + -0.4916617274284363, + -0.5934253931045532, + 2.074575424194336, + -0.9512085318565369, + 0.3705800175666809, + 1.691048264503479, + 2.2907681465148926, + 1.532329797744751, + 0.8991785049438477, + -0.7733803987503052, + 1.3577321767807007, + 0.9844847917556763, + 1.7379451990127563, + 2.050845146179199, + 0.7804086804389954, + -0.8400509357452393, + -1.6339564323425293, + 0.9922879934310913, + 0.6006971597671509, + -1.0612030029296875, + -1.3134464025497437 + ] + ], + "predicted_scores": [ + 8.758014678955078, + 8.409639358520508, + 8.26655387878418, + 7.458409309387207, + 7.388648509979248, + 7.007550239562988, + 6.89915657043457, + 6.521210670471191 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train1", + "selected_index": 0, + "state_hash": "74b116fd9ceddd5b2261edd96a14934a4b4a78a40191a654d6743661b819f407", + "stored_base_utility": 0.23635749518871307, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.00048063197755254805, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00048063197755254805 + }, + { + "contact_quality": null, + "progress": 0.0005253751878626645, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0005253751878626645 + }, + { + "contact_quality": null, + "progress": 0.0004003842768725008, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0004003842768725008 + }, + { + "contact_quality": null, + "progress": 0.0003489255905151367, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0003489255905151367 + }, + { + "contact_quality": null, + "progress": 0.00048579773283563554, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00048579773283563554 + }, + { + "contact_quality": null, + "progress": 0.001480837701819837, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001480837701819837 + }, + { + "contact_quality": null, + "progress": 0.0006714065675623715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006714065675623715 + }, + { + "contact_quality": null, + "progress": 0.00032782554626464844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00032782554626464844 + } + ], + "candidate_progress": [ + 0.00048063197755254805, + 0.0005253751878626645, + 0.0004003842768725008, + 0.0003489255905151367, + 0.00048579773283563554, + 0.001480837701819837, + 0.0006714065675623715, + 0.00032782554626464844 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-8e12f7ad446482627dd3", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-20e1efbb745fd02abc95", + "generated_tangents": [ + [ + -0.12346334755420685, + -0.19985295832157135, + 2.1784703731536865, + -0.31124866008758545, + 1.529472827911377, + 2.947808265686035, + 2.6920390129089355, + -0.6265994310379028, + 0.13566288352012634, + 1.5676236152648926, + 0.112566739320755, + 1.0675935745239258, + 2.570864677429199, + 2.6500132083892822, + -1.4526606798171997, + -0.22069929540157318, + 2.1557509899139404, + 0.96132892370224, + 0.9685867428779602, + 2.9575047492980957, + 2.7185823917388916 + ], + [ + -0.43995237350463867, + -0.2050541639328003, + 1.7482885122299194, + -0.251895934343338, + 1.8548591136932373, + 3.3218228816986084, + 2.2479407787323, + -0.3862256109714508, + 0.015753012150526047, + 1.78085196018219, + 0.24729987978935242, + 1.187187910079956, + 2.186312437057495, + 3.038965940475464, + -1.5351717472076416, + -0.28550881147384644, + 1.970921516418457, + 0.9219973087310791, + 0.912335216999054, + 2.5899159908294678, + 2.3246777057647705 + ], + [ + -0.250826358795166, + -0.18446028232574463, + 1.9133923053741455, + -0.46357783675193787, + 1.6548621654510498, + 3.0725228786468506, + 2.4449257850646973, + -0.5827616453170776, + -0.009590886533260345, + 1.732719898223877, + 0.17942655086517334, + 1.117271065711975, + 2.3779704570770264, + 2.668149948120117, + -1.646291732788086, + -0.323905885219574, + 2.1908156871795654, + 1.0647403001785278, + 0.8036691546440125, + 2.8001701831817627, + 2.425461769104004 + ], + [ + -0.28640180826187134, + -0.2839859127998352, + 2.0572218894958496, + -0.42631280422210693, + 1.5923094749450684, + 2.8822576999664307, + 2.911482572555542, + -0.6053820848464966, + -0.04821009188890457, + 1.8190683126449585, + 0.24333569407463074, + 1.1664683818817139, + 2.697253704071045, + 2.843883991241455, + -1.71109139919281, + -0.26010245084762573, + 1.9914753437042236, + 1.162591814994812, + 0.9151231646537781, + 2.488668441772461, + 2.022063732147217 + ], + [ + -0.13244841992855072, + -0.17563793063163757, + 1.8345932960510254, + -0.5187920331954956, + 1.943190336227417, + 3.448361396789551, + 2.255242109298706, + -0.4262901842594147, + -0.10808656364679337, + 1.8234599828720093, + 0.2276066541671753, + 1.2756623029708862, + 1.9931296110153198, + 2.609802007675171, + -1.434948205947876, + -0.32289913296699524, + 2.1416492462158203, + 0.8319113254547119, + 0.8000109791755676, + 2.658656120300293, + 2.234731912612915 + ], + [ + -0.38166344165802, + -0.15863287448883057, + 1.6745790243148804, + -0.5549147725105286, + 1.5172958374023438, + 2.7655439376831055, + 2.8303303718566895, + -0.28214988112449646, + 0.051939524710178375, + 1.2991156578063965, + 0.2312173843383789, + 0.6783193349838257, + 1.7863367795944214, + 3.8908531665802, + -1.1424500942230225, + -0.4551764726638794, + 1.4787436723709106, + 0.5913362503051758, + 1.0201154947280884, + 2.6787302494049072, + 2.5280520915985107 + ], + [ + -0.6116804480552673, + -0.2950592041015625, + 1.8596454858779907, + -0.35463833808898926, + 1.1682342290878296, + 2.9465863704681396, + 2.540924072265625, + -0.2936297357082367, + -0.08103088289499283, + 1.50174880027771, + 0.025975782424211502, + 0.493008017539978, + 2.202176570892334, + 1.8654035329818726, + -1.3604581356048584, + -0.29839661717414856, + 1.7678908109664917, + 0.6027181148529053, + 0.622581958770752, + 2.6514248847961426, + 1.7808696031570435 + ], + [ + 0.1034335121512413, + -0.25065457820892334, + 1.947960615158081, + -0.7060551643371582, + 1.1494879722595215, + 2.454498767852783, + 2.7128725051879883, + -0.9030486345291138, + -0.08790617436170578, + 1.8212023973464966, + 0.2397160530090332, + 1.1197707653045654, + 2.8676016330718994, + 2.4723026752471924, + -1.5656120777130127, + -0.5339487791061401, + 2.1438119411468506, + 0.7354217767715454, + 0.49033674597740173, + 2.791179656982422, + 1.8338626623153687 + ] + ], + "generated_utilities": [ + 0.00048063197755254805, + 0.0005253751878626645, + 0.0004003842768725008, + 0.0003489255905151367, + 0.00048579773283563554, + 0.001480837701819837, + 0.0006714065675623715, + 0.00032782554626464844 + ], + "group_id": "ms-20e1efbb745fd02abc95", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13665315508842468, + 0.1696699559688568, + 0.049180448055267334, + 0.05942712351679802, + 0.22736775875091553, + 0.0799722671508789, + 0.17261533439159393, + 0.21202586591243744, + 0.18980810046195984, + 0.24345096945762634, + 0.08349347114562988, + 0.11624568700790405, + 0.17245106399059296, + 0.07060472667217255, + 0.041495781391859055, + 0.028585990890860558 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.08645056188106537, + -0.060760222375392914, + 0.6493666172027588, + -0.016124725341796875, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + -0.2971414625644684, + 0.023367643356323242, + 0.6914786100387573, + -0.1027369350194931, + 0.1733318567276001, + 1.2948026657104492, + 0.0, + -0.30476057529449463, + -0.10394972562789917, + 0.9822354316711426, + 0.055961012840270996, + 0.004969358444213867, + 2.122485637664795, + 0.0 + ], + [ + 0.046016186475753784, + -0.24098780751228333, + 0.6493666172027588, + 0.05737954378128052, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + 0.22233793139457703, + 0.3896067142486572, + 0.6914786100387573, + -0.061455681920051575, + 0.17170286178588867, + 1.2948026657104492, + 0.0, + -0.4717581272125244, + -0.5722612738609314, + 0.9822354316711426, + 0.3690842390060425, + 0.33333829045295715, + 2.122485637664795, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.590132236480713, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.670657634735107, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.27303409576416 + ], + [ + -0.4654892086982727, + -0.6659128069877625, + 2.053840160369873, + -0.17420679330825806, + 1.1238669157028198, + 2.773953914642334, + 2.9706993103027344, + -0.9652835726737976, + -0.6603021621704102, + 1.957632303237915, + 0.30636119842529297, + 1.0932379961013794, + 2.4988603591918945, + 2.1735024452209473, + -0.7229697108268738, + -0.6565687656402588, + 2.1097376346588135, + 0.79427170753479, + 1.1421128511428833, + 3.200641393661499, + 3.3294520378112793 + ], + [ + 0.8783296346664429, + -0.6790766716003418, + 1.3789844512939453, + -1.578566551208496, + 1.2084856033325195, + 1.906846046447754, + 2.7278928756713867, + -0.8570022583007812, + -0.3114888072013855, + 2.660332202911377, + 0.4615887999534607, + -0.16199523210525513, + 1.9830265045166016, + 2.4224376678466797, + -0.9260885715484619, + 0.5555675029754639, + 2.5474588871002197, + 0.0070996880531311035, + 1.627825379371643, + 3.9487287998199463, + 3.03330135345459 + ], + [ + -0.1274794638156891, + -0.26015421748161316, + 2.5414626598358154, + -1.0220305919647217, + 1.4708960056304932, + 2.0982167720794678, + 3.1457836627960205, + -1.0992621183395386, + -0.9075721502304077, + 0.8330968022346497, + 0.15631377696990967, + 1.3095502853393555, + 1.8582203388214111, + 2.101317882537842, + -0.9184049367904663, + 0.4676415026187897, + 2.784534454345703, + 0.4529607892036438, + 0.6403366327285767, + 3.1566896438598633, + 2.5268735885620117 + ], + [ + 0.9289348125457764, + 0.1545984447002411, + 1.6792887449264526, + -0.6400681138038635, + 0.5956984758377075, + 2.5183756351470947, + 1.6261346340179443, + -0.019961506128311157, + -0.2638699412345886, + 1.310987114906311, + 0.4996453523635864, + -0.06606829166412354, + 1.9830868244171143, + 2.392476797103882, + -0.3871336579322815, + -0.7797489166259766, + 1.927225947380066, + 0.4438522458076477, + 1.0257132053375244, + 3.211881160736084, + 3.374328851699829 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05862678587436676, + -0.042836256325244904, + 1.6493666172027588, + -0.7465556859970093, + 1.1865841150283813, + 2.873413562774658, + 2.2950661182403564, + -0.48127153515815735, + -0.11381363868713379, + 1.6914786100387573, + 0.06965510547161102, + 0.8288453221321106, + 2.294802665710449, + 2.3353288173675537, + -1.3047605752944946, + -0.39009055495262146, + 1.9822354316711426, + 0.9993376731872559, + 0.81731116771698, + 3.122485637664795, + 2.63651704788208 + ], + [ + -0.045653510838747025, + 0.00397084653377533, + 1.3591042757034302, + -0.6414353847503662, + 0.8377937078475952, + 2.8277816772460938, + 2.203674554824829, + -0.1951642632484436, + 0.12272481620311737, + 1.7411555051803589, + -0.004818864166736603, + 0.8769108057022095, + 2.243906021118164, + 2.4328396320343018, + -1.6330946683883667, + -0.4807296395301819, + 2.2988247871398926, + 0.8021057844161987, + 0.6970767378807068, + 2.8728103637695312, + 2.288278102874756 + ], + [ + -0.4273069500923157, + 0.16743427515029907, + 1.538269281387329, + -0.7002415657043457, + 0.7622614502906799, + 2.805328369140625, + 2.267277479171753, + -0.6541028022766113, + -0.2980780303478241, + 1.456940770149231, + 0.5105806589126587, + 1.040317177772522, + 2.4321720600128174, + 2.696331024169922, + -1.1555191278457642, + -0.025772064924240112, + 1.6384800672531128, + 1.4557092189788818, + 0.6594157814979553, + 3.1446406841278076, + 2.249678134918213 + ], + [ + -0.6431475877761841, + -0.24224388599395752, + 1.3122833967208862, + -0.16711556911468506, + 0.9232780337333679, + 2.850522994995117, + 2.645284414291382, + -0.379825621843338, + -0.6750738620758057, + 1.1236441135406494, + 0.4938291311264038, + 1.1075668334960938, + 2.203859806060791, + 2.799032688140869, + -1.359815239906311, + -0.5104091763496399, + 2.0970702171325684, + 1.057171106338501, + 1.1167995929718018, + 2.8190994262695312, + 2.2184524536132812 + ] + ], + "predicted_scores": [ + 7.416408061981201, + 7.266858100891113, + 7.034653663635254, + 6.855306625366211, + 6.838082313537598, + 6.716786861419678, + 6.6193695068359375, + 6.177856922149658 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1375492786/train1", + "selected_index": 0, + "state_hash": "56acb365a02ebac0ac4893531e168df3c430b486416156feeea40eeb178bda3e", + "stored_base_utility": 0.13665315508842468, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2dc5138251afd26a040d", + "generated_tangents": [ + [ + 1.7014609575271606, + -0.18039368093013763, + -1.2970715761184692, + -0.6534698605537415, + 0.3687649965286255, + 0.38120630383491516, + -0.239515483379364, + 0.751300573348999, + -0.02512221969664097, + -0.17351478338241577, + -0.7087518572807312, + -0.45235753059387207, + -3.3236494064331055, + -2.2830886840820312, + 0.26985061168670654, + 0.1224251389503479, + 0.3557276725769043, + -1.178897738456726, + 1.1012275218963623, + -2.4061923027038574, + -1.28949773311615 + ], + [ + 1.600577712059021, + -0.31889867782592773, + -1.0949269533157349, + -0.7782318592071533, + -0.03460490703582764, + 0.11920750141143799, + -0.9855067133903503, + 0.6091313362121582, + 0.1639746129512787, + -0.45358240604400635, + -0.3156372904777527, + -0.21796345710754395, + -3.02553129196167, + -2.235438823699951, + -0.003896709531545639, + 0.19991938769817352, + 0.7951566576957703, + -0.8402144312858582, + 1.218689203262329, + -2.647265672683716, + -1.5202679634094238 + ], + [ + 1.7732620239257812, + -0.08437895774841309, + -1.4931981563568115, + -0.1378997415304184, + -0.0962563157081604, + 0.34288352727890015, + -0.7458742260932922, + 0.5131850242614746, + 0.03603658825159073, + -0.2769846022129059, + -0.28220295906066895, + -0.19899293780326843, + -3.198810338973999, + -2.617182493209839, + 0.13802781701087952, + 0.05464818701148033, + 0.3968123197555542, + -1.293387532234192, + 1.251478910446167, + -2.528707265853882, + -1.2376025915145874 + ], + [ + 1.7733588218688965, + -0.07844860851764679, + -1.2190992832183838, + -0.2284640371799469, + -0.32739758491516113, + 0.6094720363616943, + -0.29514384269714355, + 0.4748133718967438, + 0.062315359711647034, + -0.05596692115068436, + -0.4144563674926758, + -0.36419641971588135, + -3.6123297214508057, + -2.5737216472625732, + 0.10197815299034119, + 0.17784062027931213, + 0.5930182933807373, + -1.0416651964187622, + 0.9636204838752747, + -2.3183629512786865, + -1.5335577726364136 + ], + [ + 1.7985514402389526, + -0.037546902894973755, + -1.2421534061431885, + -0.5172708034515381, + 0.053566426038742065, + 0.3809960186481476, + -0.05482292175292969, + 0.6663963794708252, + -0.03202047199010849, + -0.12443280965089798, + -0.5220405459403992, + -0.44478660821914673, + -3.471993923187256, + -2.3812077045440674, + 0.18697009980678558, + 0.08937080204486847, + 0.308956503868103, + -1.1457538604736328, + 1.0770564079284668, + -2.7350029945373535, + -1.3568449020385742 + ], + [ + 1.7711821794509888, + -0.11741582304239273, + -1.2946592569351196, + -0.36421170830726624, + -0.1146346926689148, + 0.456108033657074, + -0.23670202493667603, + 0.5579202175140381, + -0.0011934908106923103, + -0.12236564606428146, + -0.39571714401245117, + -0.28584516048431396, + -3.3783440589904785, + -2.509171724319458, + 0.22266556322574615, + 0.08078525960445404, + 0.35258013010025024, + -1.213260293006897, + 1.0058355331420898, + -2.471781015396118, + -1.4205639362335205 + ], + [ + 1.9132753610610962, + -0.04391380399465561, + -1.0892757177352905, + -0.30472588539123535, + 0.13552846014499664, + 0.15591594576835632, + -0.6045941710472107, + 0.36314308643341064, + 0.1025685966014862, + -0.09091076999902725, + -0.15325510501861572, + -0.19675669074058533, + -2.6142191886901855, + -1.9814730882644653, + 0.12692448496818542, + 0.010162748396396637, + 0.4250797629356384, + -0.9183775782585144, + 1.0481324195861816, + -2.7949914932250977, + -1.3203645944595337 + ], + [ + 1.7334052324295044, + -0.057985819876194, + -0.8837970495223999, + -0.24913650751113892, + -0.10631182789802551, + -0.24190834164619446, + 0.056062906980514526, + 0.6961460113525391, + 0.07683011889457703, + -0.2773747444152832, + -0.3860301375389099, + -0.47417378425598145, + -3.164853811264038, + -2.0257632732391357, + 0.2156425416469574, + 0.07730689644813538, + 0.34279489517211914, + -1.0879709720611572, + 1.0649265050888062, + -2.7084548473358154, + -1.4397176504135132 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-2dc5138251afd26a040d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1373138427734375, + 0.1398601531982422, + 0.11154793202877045, + 0.07755142450332642, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7500379085540771, + -0.32870030403137207, + -0.3281364440917969, + 0.020404458045959473, + 0.03884005546569824, + 0.26651138067245483, + 0.0, + 0.2614295184612274, + 0.02006034180521965, + 0.6256309151649475, + 0.10119366645812988, + 0.0365104079246521, + -1.9807136058807373, + 0.0, + -0.4040032625198364, + -0.3390212059020996, + 0.444367915391922, + -1.3744113445281982, + 0.3090496063232422, + -1.6372854709625244, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7500379085540771, + 0.1347455233335495, + -1.3281364440917969, + -0.3984496593475342, + -0.6640693545341492, + -0.7334886193275452, + -1.63351309299469, + 0.4696621894836426, + -0.03721961751580238, + 0.0896599143743515, + -0.8988063335418701, + -0.047963276505470276, + -2.9807136058807373, + -2.542311906814575, + 0.5875393152236938, + 0.41857796907424927, + 0.32660895586013794, + -2.3744113445281982, + 0.9575246572494507, + -2.6372854709625244, + -1.9586280584335327 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.26702618598938, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.08462381362915, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.9172561168670654 + ], + [ + 1.7928142547607422, + -0.046942055225372314, + -1.3733142614364624, + -0.4212188422679901, + -0.5566414594650269, + -0.9966021180152893, + -1.7926092147827148, + 0.37284746766090393, + 0.18181614577770233, + -0.056899696588516235, + -0.8793746829032898, + -0.06206277757883072, + -3.2539288997650146, + -2.3651227951049805, + 0.543649435043335, + 0.4969414472579956, + 0.3953157365322113, + -2.6153955459594727, + 1.239845633506775, + -2.9289329051971436, + -2.1138057708740234 + ], + [ + 1.4313925504684448, + 0.3287010192871094, + -1.4217545986175537, + -0.11546039581298828, + -0.38684383034706116, + -0.3866080045700073, + -1.4945378303527832, + 0.4180656969547272, + 0.08673545718193054, + -0.30675017833709717, + -0.8370212316513062, + -0.3124745488166809, + -3.2861690521240234, + -2.5144338607788086, + 0.7133253216743469, + 0.5272455215454102, + 0.6288780570030212, + -2.7024707794189453, + 1.0730262994766235, + -2.603350877761841, + -1.8104162216186523 + ], + [ + 2.0841901302337646, + 0.18348857760429382, + -1.4159170389175415, + -0.5868262052536011, + -0.3290694057941437, + -0.8189557790756226, + -1.811798334121704, + 0.8927561044692993, + -0.4510294198989868, + -0.1373635083436966, + -0.46978816390037537, + -0.2017078548669815, + -3.1228857040405273, + -2.619961977005005, + 0.7154320478439331, + 0.4623824954032898, + 0.8127838373184204, + -2.5683043003082275, + 0.8921399116516113, + -2.799823522567749, + -1.6220401525497437 + ], + [ + 1.5681514739990234, + -0.11813957989215851, + -1.066882848739624, + -0.4835556447505951, + -0.4833664894104004, + -0.1345365047454834, + -1.872452735900879, + 1.059093952178955, + -0.3537854254245758, + 0.5130321979522705, + -0.7769420146942139, + -0.4462882876396179, + -3.249842882156372, + -2.2144932746887207, + 0.6907999515533447, + 0.14545467495918274, + 0.8816275596618652, + -2.4357430934906006, + 1.424092411994934, + -2.656961441040039, + -2.3529839515686035 + ], + [ + 2.0935373306274414, + 0.20671290159225464, + -1.0638595819473267, + -0.37751641869544983, + -0.5913090705871582, + -0.3702518939971924, + -1.3611384630203247, + 0.8432458639144897, + 0.4395832121372223, + -0.19578908383846283, + -1.2028591632843018, + -0.07535962760448456, + -3.2909016609191895, + -2.9216225147247314, + 1.1581575870513916, + 0.9284459352493286, + -0.3347962498664856, + -1.9277539253234863, + 0.8971017003059387, + -3.245394229888916, + -2.1499102115631104 + ], + [ + 2.5108895301818848, + -0.2977973222732544, + -1.458537220954895, + -0.5705423355102539, + -1.4453105926513672, + -0.4854125380516052, + -2.383378028869629, + 0.39735814929008484, + -0.3523370325565338, + 0.4586251974105835, + -1.2388068437576294, + 0.1082226037979126, + -3.5858566761016846, + -1.8809643983840942, + -0.18688076734542847, + 0.0871247947216034, + 0.0062812864780426025, + -3.012958526611328, + 1.0266033411026, + -3.1649136543273926, + -2.6460556983947754 + ], + [ + 1.2948591709136963, + 0.21422013640403748, + -1.3904463052749634, + -0.13111025094985962, + -0.1661812961101532, + -0.4983506500720978, + -2.0006139278411865, + -0.4048727750778198, + 0.10130663216114044, + 0.1955089420080185, + -0.27839744091033936, + 0.8026529550552368, + -3.2792744636535645, + -1.6839708089828491, + 0.6468846201896667, + -0.2772523760795593, + 0.45975640416145325, + -2.612403631210327, + 1.6571701765060425, + -1.8971503973007202, + -1.2593979835510254 + ], + [ + 0.9154124855995178, + -0.2073056846857071, + -2.1218466758728027, + -0.4230979084968567, + -1.2478010654449463, + -0.5439814925193787, + -1.185287594795227, + -0.06685006618499756, + -0.7776275873184204, + -0.015365853905677795, + -1.3256038427352905, + 0.46163398027420044, + -2.8853814601898193, + -2.387786865234375, + 1.2300693988800049, + 0.9541484117507935, + 0.29864931106567383, + -2.931433916091919, + 1.3643410205841064, + -2.001633882522583, + -2.6548502445220947 + ], + [ + 2.035454750061035, + 0.03935282677412033, + -1.9080866575241089, + -0.17657285928726196, + -1.4843900203704834, + -0.4207392632961273, + -1.554489016532898, + 0.1909814178943634, + -0.5061874389648438, + -0.420978844165802, + -0.7634562849998474, + -0.7818551063537598, + -3.374903678894043, + -2.5324442386627197, + 1.2213574647903442, + 0.9676649570465088, + 0.5052346587181091, + -2.0974624156951904, + 0.9588823914527893, + -3.6367201805114746, + -1.953636646270752 + ], + [ + 1.4856661558151245, + 0.3259274363517761, + -2.1635096073150635, + -0.8793277740478516, + 0.2686290740966797, + -0.340938925743103, + -0.8516338467597961, + -0.3830271363258362, + 0.5376158356666565, + -0.2721732258796692, + -1.2276334762573242, + -0.38624608516693115, + -3.346294403076172, + -3.093522071838379, + 0.6735774278640747, + 0.5564011931419373, + -0.5450864434242249, + -1.5815953016281128, + 1.4068235158920288, + -2.7531042098999023, + -2.515698194503784 + ] + ], + "predicted_scores": [ + 6.710046291351318, + 6.684288024902344, + 6.168863773345947, + 6.0246429443359375, + 5.804898262023926, + 5.693931579589844, + 5.307399272918701, + 5.256102085113525 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565621826/train1", + "selected_index": 0, + "state_hash": "f6edd849b0b51788d7be25e402fb514cb777ccf220797faaf769000814df02cb", + "stored_base_utility": 0.1373138427734375, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08252689987421036, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08252689987421036 + }, + "base_progress": 0.08252689987421036, + "base_success": false, + "base_utility": 0.08252689987421036, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06141128018498421, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06141128018498421 + }, + { + "contact_quality": null, + "progress": 0.06181597709655762, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06181597709655762 + }, + { + "contact_quality": null, + "progress": 0.14589862525463104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14589862525463104 + }, + { + "contact_quality": null, + "progress": 0.05644259974360466, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05644259974360466 + }, + { + "contact_quality": null, + "progress": 0.061918895691633224, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061918895691633224 + }, + { + "contact_quality": null, + "progress": 0.058040279895067215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058040279895067215 + }, + { + "contact_quality": null, + "progress": 0.04945683479309082, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04945683479309082 + }, + { + "contact_quality": null, + "progress": 0.04462442919611931, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04462442919611931 + } + ], + "candidate_progress": [ + 0.06141128018498421, + 0.06181597709655762, + 0.14589862525463104, + 0.05644259974360466, + 0.061918895691633224, + 0.058040279895067215, + 0.04945683479309082, + 0.04462442919611931 + ], + "candidate_source_chart_ids": [ + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-206099bab6bfe7c84fce", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-387e6771a55be98f1a59", + "generated_tangents": [ + [ + 0.06006329506635666, + -0.08010709285736084, + 0.11080029606819153, + 0.32980427145957947, + 1.7240791320800781, + -0.362089604139328, + -0.2584540843963623, + -0.19833067059516907, + 0.30151551961898804, + 0.07162415981292725, + 0.2010030746459961, + 0.20961271226406097, + -0.03765740990638733, + -0.08270052075386047, + -0.09435108304023743, + -0.049414895474910736, + 0.108904629945755, + -0.16463418304920197, + -0.03854423761367798, + 0.15900826454162598, + -0.04328756034374237 + ], + [ + 0.009599551558494568, + -0.16914266347885132, + -0.1386336386203766, + 0.30693379044532776, + 1.6530368328094482, + -0.07611559331417084, + -0.1791226863861084, + 0.02553841471672058, + 0.19469773769378662, + 0.027957111597061157, + 0.45823749899864197, + 0.32478460669517517, + -0.1743784248828888, + 0.21550936996936798, + -0.10836927592754364, + 0.054596196860075, + 0.16475197672843933, + 0.03313766419887543, + 0.32187891006469727, + 0.3524985909461975, + 0.12549921870231628 + ], + [ + 0.6465438008308411, + -0.3374776244163513, + 0.12796393036842346, + 0.2827477753162384, + 1.7577989101409912, + -0.5565595626831055, + -0.16577357053756714, + 0.01457950472831726, + 0.24365079402923584, + 0.04987485706806183, + 0.8056530356407166, + 0.12356974929571152, + 0.26063719391822815, + 0.6298707127571106, + -0.145112082362175, + 0.06279179453849792, + 0.17149415612220764, + -0.29395294189453125, + 0.5806790590286255, + 0.426554799079895, + -0.4817831814289093 + ], + [ + -0.053341493010520935, + -0.1865396499633789, + 0.16841968894004822, + 0.11713874340057373, + 1.4590277671813965, + 0.3799755573272705, + -0.681486189365387, + 0.0640789270401001, + 0.1813700795173645, + 0.0997859388589859, + 0.4823245406150818, + 0.2732934057712555, + -0.20591291785240173, + -0.20029090344905853, + -0.24545499682426453, + 0.04443333297967911, + 0.1731298565864563, + 0.21911995112895966, + 0.1807205229997635, + 0.3559965491294861, + 0.2999149560928345 + ], + [ + -0.027210667729377747, + -0.03206159919500351, + -0.052471309900283813, + 0.01967671513557434, + 0.7426930665969849, + -0.8253703117370605, + -0.018247127532958984, + 0.12169646471738815, + 0.19982509315013885, + 0.18069101870059967, + 0.22702598571777344, + -0.37949854135513306, + -0.7075455188751221, + 0.003782108426094055, + -0.10478651523590088, + -0.0896352082490921, + 0.09950701892375946, + 0.07654224336147308, + 0.5347628593444824, + -0.2050948143005371, + 0.24008597433567047 + ], + [ + -0.10568918287754059, + -0.07966116070747375, + 0.003670215606689453, + -0.08083397150039673, + 0.8740999698638916, + -0.6896226406097412, + 0.08204978704452515, + 0.07617051899433136, + 0.1713121086359024, + 0.2213243693113327, + 0.31011971831321716, + -0.4065733551979065, + -0.656135618686676, + 0.05357886850833893, + -0.1855739802122116, + -0.11018005013465881, + -0.04411895573139191, + 0.08190812170505524, + 0.586898922920227, + -0.3115938901901245, + 0.23166456818580627 + ], + [ + -0.11853678524494171, + -0.060251377522945404, + 0.036512553691864014, + -0.0002815127372741699, + 0.7336574792861938, + -0.8709181547164917, + 0.08950820565223694, + 0.04132549464702606, + 0.16058886051177979, + 0.2268449366092682, + 0.2850486934185028, + -0.40542882680892944, + -0.6018311977386475, + 0.1696666181087494, + -0.09339651465415955, + -0.09056708216667175, + 0.09688669443130493, + 0.05742751061916351, + 0.5989552736282349, + -0.15574336051940918, + 0.22451287508010864 + ], + [ + -0.22560490667819977, + -0.031416166573762894, + 0.11794865131378174, + -0.3758751153945923, + 1.021733283996582, + -1.0908676385879517, + 0.201805979013443, + -0.062230855226516724, + 0.10675548762083054, + 0.26547670364379883, + 0.044261425733566284, + -0.1574232280254364, + -0.8401191234588623, + -1.262336254119873, + 0.0014948472380638123, + -0.12965939939022064, + -0.27413487434387207, + -0.11683200299739838, + 0.5190190076828003, + -0.29211071133613586, + -0.27497655153274536 + ] + ], + "generated_utilities": [ + 0.06141128018498421, + 0.06181597709655762, + 0.14589862525463104, + 0.05644259974360466, + 0.061918895691633224, + 0.058040279895067215, + 0.04945683479309082, + 0.04462442919611931 + ], + "group_id": "ms-387e6771a55be98f1a59", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.03822460025548935, + 0.18199902772903442, + 0.20744429528713226, + 0.05679210275411606, + 0.07646497339010239, + 0.038853906095027924, + 0.0824488028883934, + 0.05751838535070419, + 0.02475176379084587, + 0.040490686893463135, + 0.027167975902557373, + 0.06515108048915863, + 0.04604649543762207, + 0.014139334671199322, + 0.0258652176707983, + 0.034187138080596924 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0643576979637146, + 0.02293895184993744, + 0.03272289037704468, + 0.02799379825592041, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + 0.05611860752105713, + -0.010578691959381104, + 0.03396539017558098, + 0.754948616027832, + -0.12030506134033203, + 0.05740344524383545, + 0.0, + 0.01674991101026535, + 0.0017660856246948242, + 0.1110122799873352, + -0.29033517837524414, + -0.015073239803314209, + -0.062492311000823975, + 0.0 + ], + [ + 0.03256732225418091, + 0.029803231358528137, + -0.11524808406829834, + -0.03975573182106018, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + -0.11635321378707886, + -0.12916040420532227, + -0.10934318602085114, + 0.754948616027832, + -0.10127544403076172, + 0.10553282499313354, + 0.0, + 0.2124219685792923, + -0.13985675573349, + 0.20122885704040527, + -0.29033517837524414, + -0.09104716777801514, + -0.33053046464920044, + 0.0 + ] + ], + "predicted_scores": [ + 7.301035404205322, + 7.189646244049072, + 6.6587982177734375, + 6.655889987945557, + 6.1351423263549805, + 6.031546115875244, + 5.582923889160156, + 4.6993608474731445 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train1", + "selected_index": 0, + "state_hash": "6b753a027f9378ff308946331f6bb658d452d8e77220ab0cd6fc813be0ac75cc", + "stored_base_utility": 0.03822460025548935, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03917040303349495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03917040303349495 + }, + "base_progress": 0.03917040303349495, + "base_success": false, + "base_utility": 0.03917040303349495, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11718571186065674, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11718571186065674 + }, + { + "contact_quality": null, + "progress": 0.08475744724273682, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08475744724273682 + }, + { + "contact_quality": null, + "progress": 0.06714028120040894, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06714028120040894 + }, + { + "contact_quality": null, + "progress": 0.035049181431531906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035049181431531906 + }, + { + "contact_quality": null, + "progress": 0.03447306156158447, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03447306156158447 + }, + { + "contact_quality": null, + "progress": 0.02684718370437622, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02684718370437622 + }, + { + "contact_quality": null, + "progress": 0.02741086483001709, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02741086483001709 + }, + { + "contact_quality": null, + "progress": 0.0036935408134013414, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0036935408134013414 + } + ], + "candidate_progress": [ + 0.11718571186065674, + 0.08475744724273682, + 0.06714028120040894, + 0.035049181431531906, + 0.03447306156158447, + 0.02684718370437622, + 0.02741086483001709, + 0.0036935408134013414 + ], + "candidate_source_chart_ids": [ + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-be219977a9db8dbb1e58", + "ms-98eb81debb5186ebbe38", + "ms-7dedf60ad41ea2d92ee5" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-38f7298184a2c5005e94", + "generated_tangents": [ + [ + 0.6369849443435669, + 0.11492690443992615, + 0.0840025246143341, + -0.7244514226913452, + 0.8756675124168396, + -1.535658836364746, + 0.612249493598938, + 0.3143526613712311, + 0.04067815840244293, + 0.2646087110042572, + -0.8137031197547913, + 0.3005238175392151, + -1.509477138519287, + -0.9800235033035278, + 0.00546921044588089, + 0.1681281328201294, + 0.005790531635284424, + 0.14234024286270142, + -0.4064610004425049, + -1.5683271884918213, + -0.9911508560180664 + ], + [ + 0.5584995150566101, + -0.05000460147857666, + -0.1664465367794037, + -0.5341358184814453, + 1.0766503810882568, + -1.296900987625122, + 0.026495516300201416, + 0.017091497778892517, + -0.03880733251571655, + 0.20170888304710388, + -0.6933715343475342, + 0.6240185499191284, + -1.3246278762817383, + -0.7883918881416321, + -0.15517935156822205, + 0.212482288479805, + 0.05112868547439575, + 0.20218025147914886, + -0.336897611618042, + -1.534984827041626, + -0.8085364103317261 + ], + [ + 0.41416382789611816, + -0.06269974261522293, + -0.047496646642684937, + -0.6089473962783813, + 1.2396233081817627, + -1.3737013339996338, + -0.11145180463790894, + 0.05461525917053223, + -0.07937335968017578, + 0.1598551869392395, + -0.6420341730117798, + 0.6470789909362793, + -1.4348926544189453, + -0.8785654306411743, + -0.29571256041526794, + 0.17546021938323975, + -0.1828174740076065, + 0.14676928520202637, + -0.29687660932540894, + -1.3849177360534668, + -0.8743813037872314 + ], + [ + 0.4911530613899231, + -0.042739931493997574, + -0.3767957389354706, + -0.6375311613082886, + 1.306060791015625, + -1.2735764980316162, + -0.08663177490234375, + -0.23911377787590027, + 0.09168751537799835, + 0.38278016448020935, + -0.6272723078727722, + 0.5073744058609009, + -1.6354384422302246, + -0.742485761642456, + -0.10793615877628326, + 0.2449941486120224, + -0.21595244109630585, + 0.2828562259674072, + -0.22117012739181519, + -1.4895732402801514, + -0.644891619682312 + ], + [ + 0.2277255356311798, + -0.16269108653068542, + 0.18027782440185547, + -0.10532813519239426, + 0.7146583199501038, + -1.3409571647644043, + 0.010533004999160767, + 0.09202136844396591, + 0.0896114632487297, + 0.23388461768627167, + -0.4258403480052948, + 1.0549230575561523, + -0.8153125047683716, + -0.25280433893203735, + -0.3791922926902771, + 0.1472392976284027, + 0.007149025797843933, + 0.072731614112854, + -0.04462990164756775, + -1.410667896270752, + -0.78948974609375 + ], + [ + -0.20279093086719513, + -0.20314247906208038, + -0.0337354838848114, + 0.0771024227142334, + 0.8623231649398804, + -1.8566323518753052, + 0.12431898713111877, + 0.006875365972518921, + 0.18823981285095215, + 0.36663636565208435, + -0.35092443227767944, + 0.3065592050552368, + -1.2975765466690063, + -0.06951725482940674, + -0.21218442916870117, + 0.20042358338832855, + 0.06910833716392517, + 0.05749613046646118, + 0.10933664441108704, + -1.560270071029663, + -1.0320936441421509 + ], + [ + 0.13047151267528534, + -0.22059795260429382, + 0.020146965980529785, + 0.19127678871154785, + 0.9241453409194946, + -1.7760614156723022, + 0.15420421957969666, + -0.04240788519382477, + 0.18307676911354065, + 0.22342398762702942, + -0.33989664912223816, + 0.9492338299751282, + -1.38172447681427, + 0.3790102005004883, + -0.2612534761428833, + 0.15426896512508392, + 0.12225785851478577, + 0.1312655657529831, + 0.02084413170814514, + -1.6546058654785156, + -0.9936606884002686 + ], + [ + -0.4820966124534607, + 0.016768746078014374, + -0.1510896384716034, + -0.04225653409957886, + 2.0557963848114014, + -2.255887269973755, + 0.05668759346008301, + -0.1425926387310028, + 0.1668226569890976, + 0.40121230483055115, + -0.6604657173156738, + 0.6876393556594849, + -1.0268025398254395, + 0.43825095891952515, + -0.3937377631664276, + 0.44952335953712463, + 0.25725775957107544, + 0.053308889269828796, + 0.01985764503479004, + -1.5205094814300537, + -0.5356847047805786 + ] + ], + "generated_utilities": [ + 0.11718571186065674, + 0.08475744724273682, + 0.06714028120040894, + 0.035049181431531906, + 0.03447306156158447, + 0.02684718370437622, + 0.02741086483001709, + 0.0036935408134013414 + ], + "group_id": "ms-38f7298184a2c5005e94", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1194220781326294, + 0.1767096221446991, + 0.0505807027220726, + 0.026383202522993088, + 0.09035396575927734, + 0.11899666488170624, + 0.059947751462459564, + 0.08427222818136215, + 0.041255753487348557, + 0.05034176632761955, + 0.02779809758067131, + 0.011037330143153667, + 0.020422041416168213, + 0.049474477767944336, + 0.03517866134643555, + 0.02796018123626709 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.08923250436782837, + 0.05675226449966431, + -0.1250607967376709, + -0.07566308975219727, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.001595139503479004, + 0.026048749685287476, + 0.1681501865386963, + -0.11568224430084229, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.18507611751556396, + -0.15127414464950562, + 0.07637521624565125, + -0.15284772217273712, + 0.0320390909910202, + -1.8774652481079102, + 0.0 + ], + [ + -0.2749255299568176, + 0.2578675150871277, + -0.46759605407714844, + -0.5041444897651672, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.13614141941070557, + 0.11120480298995972, + 0.15478569269180298, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.01827770471572876, + -0.38639169931411743, + -0.18168723583221436, + 0.3532869219779968, + -0.5437159538269043, + -1.8774652481079102, + 0.0 + ], + [ + 0.4466920495033264, + 0.07532927393913269, + 0.8145812749862671, + -0.963495135307312, + 1.3846542835235596, + -3.0133423805236816, + -1.0476078987121582, + 0.6759306788444519, + 0.5485053658485413, + 0.634345293045044, + -0.7682492733001709, + 2.2281322479248047, + -2.7100830078125, + -0.8660445213317871, + 0.7648183703422546, + 0.607989490032196, + -0.6848516464233398, + -0.06598936021327972, + 0.2030458152294159, + -2.9672060012817383, + -1.4293488264083862 + ], + [ + 1.0058056116104126, + 0.3426892161369324, + 0.6375484466552734, + -0.5828341841697693, + 1.7963712215423584, + -3.3760194778442383, + -0.9086123704910278, + 0.20509907603263855, + 0.10472571849822998, + 1.1158980131149292, + -1.0710231065750122, + 2.203950881958008, + -2.4466300010681152, + -0.3184358477592468, + 0.43084120750427246, + 0.32133060693740845, + -0.6005205512046814, + 0.07880236953496933, + 0.52874755859375, + -2.7805917263031006, + -1.6972103118896484 + ], + [ + 0.48786461353302, + -0.007949680089950562, + 0.41629794239997864, + -0.2277727723121643, + 1.0953291654586792, + -2.7825636863708496, + -1.7178279161453247, + 0.8412191271781921, + 0.6389099359512329, + 0.7100303769111633, + -1.1701807975769043, + 2.8546230792999268, + -2.570579767227173, + -0.2383948564529419, + 0.4282584488391876, + 0.27555936574935913, + -0.4073549509048462, + 0.49247825145721436, + 0.6483681797981262, + -3.3716540336608887, + -1.891394853591919 + ], + [ + 0.17139381170272827, + 0.831203818321228, + 0.1966383457183838, + -1.5011540651321411, + 1.811257004737854, + -2.6416001319885254, + -1.8791863918304443, + 0.03218984603881836, + 1.0182924270629883, + 1.5643268823623657, + -1.2954871654510498, + 2.249680757522583, + -1.9383416175842285, + -1.4303685426712036, + 0.48937559127807617, + 0.2848057746887207, + -0.6113857626914978, + 0.6453920006752014, + -0.13643476366996765, + -2.514732599258423, + -1.8046306371688843 + ], + [ + 0.1380026936531067, + 0.5165722370147705, + 0.9639596939086914, + -0.3682268261909485, + 2.2811193466186523, + -3.5902557373046875, + -1.047336220741272, + 1.3064022064208984, + 0.7264143228530884, + 1.3356540203094482, + -0.41242116689682007, + 1.8184869289398193, + -3.016441822052002, + -0.8501555919647217, + 1.0147737264633179, + 1.0417438745498657, + -0.24422766268253326, + 0.3441629707813263, + 1.075059175491333, + -3.178764820098877, + -1.7388081550598145 + ], + [ + 1.0805304050445557, + -0.5906903743743896, + -0.32329434156417847, + -1.7024246454238892, + 1.3012806177139282, + -3.3655173778533936, + -2.0100932121276855, + 0.28595322370529175, + -0.42309290170669556, + 0.9014679193496704, + -0.11985284090042114, + 3.033172607421875, + -1.890408992767334, + -0.049424707889556885, + 0.8836197853088379, + 0.07151538133621216, + -0.6898007392883301, + 0.6614744663238525, + 0.7956362962722778, + -3.344180107116699, + -1.4918344020843506 + ], + [ + 1.4192681312561035, + -0.5825785994529724, + 0.11329725384712219, + -0.08465814590454102, + 1.0504729747772217, + -2.438842535018921, + -0.9858216047286987, + -0.35852980613708496, + 1.4301284551620483, + 0.37676453590393066, + -1.3725404739379883, + 3.114079475402832, + -2.0786290168762207, + -0.5835205912590027, + 0.1985606551170349, + 0.260464072227478, + 0.41686397790908813, + -0.104665607213974, + 0.6382055282592773, + -2.7474825382232666, + -1.663126826286316 + ], + [ + 1.0832164287567139, + 0.11555392295122147, + 1.2108526229858398, + -1.5446858406066895, + 0.9112894535064697, + -3.3027408123016357, + -1.2419135570526123, + -0.057880282402038574, + -0.07349961996078491, + 0.026286423206329346, + -0.897603452205658, + 3.2857306003570557, + -2.232046365737915, + -1.2509676218032837, + 1.1900405883789062, + 0.8446143269538879, + -1.339659333229065, + -0.07882180064916611, + 0.1166829764842987, + -2.662961721420288, + -1.0079293251037598 + ], + [ + 0.43863287568092346, + -0.2024492472410202, + 0.0567094087600708, + -0.3029903173446655, + 1.612892746925354, + -3.26739764213562, + -1.7898839712142944, + -0.1649491786956787, + 0.2927369475364685, + 1.020188808441162, + -0.10171866416931152, + 2.3230679035186768, + -2.615206003189087, + -0.08625847101211548, + 0.9057100415229797, + 1.4201092720031738, + -0.3173455595970154, + 0.09508261829614639, + 0.9565507173538208, + -2.2954390048980713, + -1.2229193449020386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.03360384702682495, + -0.028338372707366943, + -0.12341004610061646, + -0.04655510187149048, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + 0.032517433166503906, + -0.02922782301902771, + -0.033792078495025635, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + -0.06465893983840942, + -0.026740312576293945, + 0.046701133251190186, + -0.06953387707471848, + 0.02321198582649231, + -1.8774652481079102, + 0.0 + ] + ], + "predicted_scores": [ + 7.032159805297852, + 6.676544189453125, + 6.1592559814453125, + 6.096284866333008, + 3.760338306427002, + 3.450634002685547, + 3.272002696990967, + 0.3568611443042755 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train1", + "selected_index": 0, + "state_hash": "dee046e5a3c9b5e78be27f1df1f4f235344ec818a1a7439afe5f4cb30139cf34", + "stored_base_utility": 0.1194220781326294, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.009124159812927246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009124159812927246 + }, + "base_progress": 0.009124159812927246, + "base_success": false, + "base_utility": 0.009124159812927246, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008222500793635845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008222500793635845 + }, + { + "contact_quality": null, + "progress": 0.0046975016593933105, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0046975016593933105 + }, + { + "contact_quality": null, + "progress": 0.0073589482344686985, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0073589482344686985 + }, + { + "contact_quality": null, + "progress": 0.0043724775314331055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0043724775314331055 + }, + { + "contact_quality": null, + "progress": 0.021352291107177734, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021352291107177734 + }, + { + "contact_quality": null, + "progress": 0.00515224551782012, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00515224551782012 + }, + { + "contact_quality": null, + "progress": 0.00400706147775054, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00400706147775054 + }, + { + "contact_quality": null, + "progress": 0.007424016948789358, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007424016948789358 + } + ], + "candidate_progress": [ + 0.008222500793635845, + 0.0046975016593933105, + 0.0073589482344686985, + 0.0043724775314331055, + 0.021352291107177734, + 0.00515224551782012, + 0.00400706147775054, + 0.007424016948789358 + ], + "candidate_source_chart_ids": [ + "ms-6758deb06644a14aa722", + "ms-6758deb06644a14aa722", + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-397e502908d16b7804b0", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-482c5c8d208b5907a2c2", + "generated_tangents": [ + [ + -0.017186760902404785, + 0.09956637024879456, + 0.20753830671310425, + -0.20426759123802185, + 1.5893971920013428, + -0.5114763975143433, + 0.11638316512107849, + 0.10756231844425201, + 0.1158156618475914, + 0.5714477300643921, + -0.904312252998352, + 1.3716343641281128, + 0.6861156225204468, + 0.3100910186767578, + -0.3616923689842224, + 0.09996774792671204, + 0.5461421608924866, + 0.715141773223877, + 0.2816997766494751, + 0.738487184047699, + -0.15864297747612 + ], + [ + 0.23175430297851562, + -0.17619842290878296, + 0.929126501083374, + -0.018297672271728516, + 1.4659910202026367, + -0.45366737246513367, + 0.301045686006546, + -0.08241283893585205, + 0.21580976247787476, + 0.5826863646507263, + -0.8205847144126892, + 1.3259111642837524, + 0.6864414215087891, + 0.6886419057846069, + -0.6587576866149902, + 0.08108201622962952, + 0.47818803787231445, + 0.7640861868858337, + 0.0036778151988983154, + 0.6387270092964172, + -0.2671339213848114 + ], + [ + -0.04621201753616333, + -0.005012210458517075, + 0.2569892406463623, + -0.2733756899833679, + 1.7086939811706543, + -0.38714930415153503, + 0.15246009826660156, + 0.09361942857503891, + 0.12655338644981384, + 0.5312187671661377, + -0.8678405284881592, + 1.317182183265686, + 0.7126821279525757, + 0.5091283917427063, + -0.472143292427063, + 0.17529332637786865, + 0.5172862410545349, + 0.7350562214851379, + 0.10154630243778229, + 0.7145820260047913, + 0.010804355144500732 + ], + [ + -0.3709810972213745, + -0.18172338604927063, + 0.4522005617618561, + -0.09949532151222229, + 1.7164051532745361, + 0.190280020236969, + -0.047384679317474365, + 0.06561936438083649, + 0.2878030240535736, + 0.7176128029823303, + -0.9203264713287354, + 1.4038488864898682, + 1.487691044807434, + 0.42478305101394653, + -0.5498186945915222, + 0.22853001952171326, + 0.904656171798706, + 0.65569669008255, + -0.36996662616729736, + 0.20872294902801514, + -0.3177125155925751 + ], + [ + 0.31387752294540405, + 0.20017869770526886, + 0.2086392045021057, + -0.32494959235191345, + 0.8306301236152649, + 0.37293708324432373, + 0.10171589255332947, + 0.06858208775520325, + 0.294421523809433, + 0.4255017936229706, + -1.0117416381835938, + 1.2504631280899048, + 0.6738172769546509, + 0.4711087942123413, + -0.5773358941078186, + 0.0011362014338374138, + 0.13909029960632324, + 0.8214422464370728, + 0.10278940200805664, + 0.5087759494781494, + -0.17116862535476685 + ], + [ + -0.32143181562423706, + -0.05319594591856003, + 0.39582347869873047, + -0.03692084550857544, + 1.5359551906585693, + 0.23442868888378143, + 0.028827577829360962, + 0.07867465913295746, + 0.2888440787792206, + 0.7149316072463989, + -0.7780476212501526, + 1.5521657466888428, + 1.118637204170227, + 0.5799349546432495, + -0.6084085702896118, + 0.24349932372570038, + 0.7352807521820068, + 0.8135769367218018, + -0.30793333053588867, + 0.23084181547164917, + -0.18994954228401184 + ], + [ + -0.2828400135040283, + -0.06221558153629303, + 0.38384413719177246, + -0.02375875413417816, + 1.6100142002105713, + 0.2748015224933624, + 0.18585950136184692, + 0.019401252269744873, + 0.2511083781719208, + 0.6879197955131531, + -0.8648056983947754, + 1.5771504640579224, + 1.1772615909576416, + 0.5997141599655151, + -0.6419427394866943, + 0.23534314334392548, + 0.7537817358970642, + 0.9064120054244995, + -0.3188903331756592, + 0.45716267824172974, + -0.2197788953781128 + ], + [ + -0.5289236307144165, + 0.050729669630527496, + 0.5570802092552185, + -0.1773999184370041, + 1.289454698562622, + 0.27887582778930664, + -0.08914440870285034, + 0.11143083870410919, + 0.5173144936561584, + 0.4815208911895752, + -1.0125471353530884, + 0.9513201117515564, + 1.7803000211715698, + 0.7554020285606384, + -0.7401578426361084, + 0.2986412048339844, + 0.5340263247489929, + 1.0491249561309814, + 0.06565281748771667, + -1.1585078239440918, + -0.2989015281200409 + ] + ], + "generated_utilities": [ + 0.008222500793635845, + 0.0046975016593933105, + 0.0073589482344686985, + 0.0043724775314331055, + 0.021352291107177734, + 0.00515224551782012, + 0.00400706147775054, + 0.007424016948789358 + ], + "group_id": "ms-482c5c8d208b5907a2c2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0359329991042614, + 0.1834145486354828, + 0.04271410033106804, + 0.03202331066131592, + 0.12089761346578598, + 0.03575414419174194, + 0.07773216813802719, + 0.1062050461769104, + 0.10849495977163315, + 0.08369352668523788, + 0.14167599380016327, + 0.008830646984279156, + 0.014355997554957867, + 0.02518286369740963, + 0.07873731851577759, + 0.015250584110617638 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.007817894220352173, + 0.01831631362438202, + 0.14601469039916992, + 0.10826835036277771, + 1.6591682434082031, + -0.005238652229309082, + 0.0, + -0.02753499150276184, + -0.009624123573303223, + 0.5762790441513062, + -0.3171614408493042, + 1.4374253749847412, + 0.7921857833862305, + 0.0, + -0.037753358483314514, + -0.025671005249023438, + 0.027724146842956543, + 0.4680846929550171, + -0.08415359258651733, + 0.033902376890182495, + 0.0 + ], + [ + 0.4543180465698242, + 0.20962828397750854, + 1.14601469039917, + -0.33832865953445435, + 2.659168243408203, + -0.7141264081001282, + 0.3707960247993469, + 0.21857939660549164, + 0.6796972155570984, + 1.5762790441513062, + -1.3171614408493042, + 2.437425374984741, + 1.7921857833862305, + 1.7551628351211548, + -0.19566796720027924, + 0.6842412948608398, + 0.11985036730766296, + 1.468084692955017, + 0.29799214005470276, + 0.3715361952781677, + 0.2624160647392273 + ], + [ + 0.2115812450647354, + -0.09577503800392151, + 0.7509078979492188, + -0.6239187121391296, + 2.4163239002227783, + -0.5409671664237976, + 0.16154196858406067, + 0.5615253448486328, + 1.0765573978424072, + 1.4198161363601685, + -1.430187463760376, + 2.5212466716766357, + 1.9029841423034668, + 1.5550287961959839, + -0.5099657773971558, + 1.0103299617767334, + -0.21028786897659302, + 1.4441221952438354, + 0.5605546236038208, + 0.6799722909927368, + 0.5554409027099609 + ], + [ + 0.46792101860046387, + 0.08014003932476044, + 1.5437835454940796, + 0.08789932727813721, + 2.3046298027038574, + -1.1965231895446777, + 0.8527317047119141, + 0.22510167956352234, + 1.0473172664642334, + 1.3720369338989258, + -1.2376489639282227, + 2.066061019897461, + 2.033992052078247, + 2.166865825653076, + -0.46199846267700195, + 1.1675353050231934, + -0.22657811641693115, + 1.7576167583465576, + 0.766140878200531, + 0.1856600046157837, + 0.008163660764694214 + ], + [ + 0.9431964755058289, + 0.6587716341018677, + 0.8381389379501343, + -0.8407833576202393, + 2.3666741847991943, + -1.05813729763031, + 0.1429125964641571, + 0.10640819370746613, + 1.0277029275894165, + 1.193630576133728, + -0.7695048451423645, + 2.9484994411468506, + 1.1866973638534546, + 1.5341145992279053, + -0.1225738674402237, + 0.9121334552764893, + 0.7116963863372803, + 0.8198207020759583, + 0.6562460660934448, + 0.1118304431438446, + -0.2320157289505005 + ] + ], + "predicted_scores": [ + 3.8373167514801025, + 3.3415451049804688, + 3.233798027038574, + 3.2228729724884033, + 3.134660005569458, + 2.501694917678833, + 2.3772544860839844, + -0.7967339754104614 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train1", + "selected_index": 0, + "state_hash": "903f78d5baa541d73612cac5e063bb553fba566c0d4adcb9603aa055d710fb15", + "stored_base_utility": 0.0359329991042614, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14663122594356537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14663122594356537 + }, + "base_progress": 0.14663122594356537, + "base_success": false, + "base_utility": 0.14663122594356537, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09237110614776611, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09237110614776611 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.07816246896982193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07816246896982193 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.054367441684007645, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054367441684007645 + }, + { + "contact_quality": null, + "progress": 0.08874932676553726, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08874932676553726 + }, + { + "contact_quality": null, + "progress": 0.0857056975364685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0857056975364685 + }, + { + "contact_quality": null, + "progress": 0.08813277631998062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08813277631998062 + } + ], + "candidate_progress": [ + 0.09237110614776611, + 1.0, + 0.07816246896982193, + 1.0, + 0.054367441684007645, + 0.08874932676553726, + 0.0857056975364685, + 0.08813277631998062 + ], + "candidate_source_chart_ids": [ + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + true, + false, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-4da03300d92420e80ed0", + "generated_tangents": [ + [ + 1.7368111610412598, + -0.21339505910873413, + -0.46265074610710144, + 0.06460139155387878, + -0.7958738207817078, + 1.979134202003479, + 1.6510772705078125, + 1.8144487142562866, + 0.31472790241241455, + -1.1421655416488647, + -0.3903200626373291, + -1.0712122917175293, + -2.1565475463867188, + -2.780268669128418, + 1.7021327018737793, + 0.03109576925635338, + -0.903856098651886, + -0.03948545455932617, + -0.08760222792625427, + -1.6022825241088867, + -1.7862193584442139 + ], + [ + 1.7032325267791748, + -0.17047926783561707, + -0.16288384795188904, + -0.07070240378379822, + -0.43538016080856323, + 2.7562496662139893, + 1.8079917430877686, + 1.817337155342102, + 0.33617568016052246, + -0.7495534420013428, + -0.22707799077033997, + -0.6103841066360474, + -2.676668167114258, + -1.630541443824768, + 1.8960447311401367, + 0.28720083832740784, + -0.6609141230583191, + 0.08464326709508896, + 0.18430621922016144, + -1.458815574645996, + -1.265209674835205 + ], + [ + 1.70179283618927, + -0.03528985381126404, + -0.4249545633792877, + 0.05410349369049072, + -0.7084158658981323, + 2.627464771270752, + 1.7215704917907715, + 1.9313647747039795, + 0.2992633283138275, + -0.851211667060852, + -0.294365257024765, + -1.0870718955993652, + -2.2758162021636963, + -2.242332696914673, + 1.7648634910583496, + 0.07617968320846558, + -0.6996333599090576, + 0.046212613582611084, + 0.009328782558441162, + -1.4902095794677734, + -1.5844112634658813 + ], + [ + 1.5555816888809204, + -0.049911241978406906, + -0.3536982834339142, + -0.07961110770702362, + -0.8340397477149963, + 2.6909291744232178, + 1.3236160278320312, + 1.910525918006897, + 0.2639979422092438, + -0.9818429946899414, + -0.2043403685092926, + -0.979907751083374, + -2.3934974670410156, + -1.9812959432601929, + 1.8485350608825684, + 0.1398877203464508, + -0.735511064529419, + 0.19586089253425598, + 0.05111268162727356, + -1.325878620147705, + -1.5703483819961548 + ], + [ + 1.69390869140625, + -0.047605711966753006, + -0.3597279489040375, + 0.019434362649917603, + -0.6562896966934204, + 2.535365343093872, + 1.896613359451294, + 1.8285937309265137, + 0.3076530992984772, + -0.7159000039100647, + -0.42773759365081787, + -0.8712296485900879, + -2.540210485458374, + -2.202735662460327, + 1.726596474647522, + 0.10543574392795563, + -0.5718258619308472, + 0.1633746176958084, + 0.06928750872612, + -1.283116340637207, + -1.3769266605377197 + ], + [ + 1.8496158123016357, + -0.044849198311567307, + -0.609849214553833, + 0.21328625082969666, + -0.6746882796287537, + 2.7659895420074463, + 1.4111237525939941, + 1.929183840751648, + 0.256539911031723, + -0.8107463121414185, + -0.2945350706577301, + -0.7436599135398865, + -2.236917495727539, + -1.543531060218811, + 1.7369017601013184, + 0.1844581514596939, + -0.7065438628196716, + 0.06091751158237457, + -0.16380739212036133, + -1.0527808666229248, + -1.074257731437683 + ], + [ + 1.8270145654678345, + -0.08395204693078995, + -0.3144706189632416, + 0.05772817134857178, + -0.39466285705566406, + 2.3055927753448486, + 1.9214544296264648, + 1.8403788805007935, + 0.3090434968471527, + -0.8092614412307739, + -0.42020419239997864, + -0.915605902671814, + -2.6384880542755127, + -2.0004937648773193, + 1.7308602333068848, + 0.08055980503559113, + -0.5612534284591675, + 0.15524426102638245, + -0.05449339747428894, + -1.3552777767181396, + -1.4205656051635742 + ], + [ + 2.286088228225708, + -0.033173397183418274, + 0.03115794062614441, + 0.14390608668327332, + -0.08776077628135681, + 1.8534377813339233, + 2.061161518096924, + 1.9325462579727173, + 0.1618713140487671, + -1.0712156295776367, + -0.5439500212669373, + -0.7075928449630737, + -2.4705185890197754, + -1.7517262697219849, + 1.651858925819397, + -0.13163228332996368, + -0.5243056416511536, + 0.37641942501068115, + -0.2565738558769226, + -1.6566424369812012, + -1.2345231771469116 + ] + ], + "generated_utilities": [ + 0.09237110614776611, + 2.0, + 0.07816246896982193, + 2.0, + 0.054367441684007645, + 0.08874932676553726, + 0.0857056975364685, + 0.08813277631998062 + ], + "group_id": "ms-4da03300d92420e80ed0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18184101581573486, + 0.1803790181875229, + 0.09926676750183105, + 0.20791122317314148, + 0.26338279247283936, + 0.2025860995054245, + 0.2575649917125702, + 0.23894403874874115, + 0.2174234390258789, + 2.0, + 0.1918075978755951, + 0.10365168750286102, + 2.0, + 2.0, + 2.0, + 0.15933696925640106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.1168947219848633, + 0.13667210936546326, + -0.051466986536979675, + 0.03360617160797119, + -0.19173425436019897, + 1.7646396160125732, + 0.0, + 0.9886866807937622, + 0.19114632904529572, + -0.3927978277206421, + -0.21108940243721008, + -0.031822919845581055, + -1.6683034896850586, + 0.0, + 0.9908627271652222, + -0.05301591753959656, + 0.08875280618667603, + 0.04224884510040283, + -0.03718748688697815, + -0.06952071189880371, + 0.0 + ], + [ + 2.1713812351226807, + -0.41443270444869995, + -0.23275701701641083, + 0.6524178981781006, + -0.25828492641448975, + 3.331055164337158, + 2.47529673576355, + 1.4984867572784424, + -0.396874338388443, + -0.725897490978241, + 0.1870076060295105, + -0.4014764428138733, + -2.239036798477173, + -2.218061923980713, + 1.6540865898132324, + 0.013689309358596802, + -0.22782719135284424, + 1.2475574016571045, + 0.7231295704841614, + -0.281164288520813, + -1.4593302011489868 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 2.1168947219848633, + 0.045049574226140976, + -0.25911271572113037, + 0.6112266778945923, + -0.5530642867088318, + 2.7646396160125732, + 1.9622430801391602, + 1.9886866807937622, + 0.28067120909690857, + -1.392797827720642, + -0.5793421268463135, + -1.031822919845581, + -2.6683034896850586, + -2.812993049621582, + 1.9908627271652222, + -0.28380459547042847, + -0.7305920720100403, + 0.7978808879852295, + 0.1574108749628067, + -1.0695207118988037, + -0.9408642053604126 + ], + [ + 2.228377103805542, + -0.19656506180763245, + -0.23844876885414124, + 0.6676552295684814, + -0.7639804482460022, + 2.937652111053467, + 1.882341980934143, + 1.8207013607025146, + 0.27702635526657104, + -1.549763798713684, + -0.5955427289009094, + -1.185685157775879, + -2.8421716690063477, + -3.054234266281128, + 1.691805362701416, + -0.37051287293434143, + -0.43653741478919983, + 0.514695405960083, + -0.0132378488779068, + -0.899186372756958, + -0.8349848985671997 + ], + [ + 2.425011157989502, + -0.03216857090592384, + -0.010275468230247498, + 0.4265761971473694, + -0.7737585306167603, + 3.1123812198638916, + 1.5686779022216797, + 2.112421989440918, + 0.3475198745727539, + -1.3452436923980713, + -0.6067132949829102, + -1.1150503158569336, + -2.461210250854492, + -2.8643558025360107, + 2.185579299926758, + -0.6223857402801514, + -0.8183724880218506, + 0.8074819445610046, + 0.536148726940155, + -0.7826522588729858, + -0.6682058572769165 + ], + [ + 1.6831411123275757, + -0.19312052428722382, + 0.010244876146316528, + 1.1032377481460571, + -0.4994945824146271, + 2.9000508785247803, + 2.187715768814087, + 2.200220823287964, + 0.4744641184806824, + -1.6715843677520752, + -0.493664413690567, + -1.459894061088562, + -2.90458345413208, + -2.3254852294921875, + 1.4726771116256714, + -0.21338894963264465, + -1.293912649154663, + 1.3146823644638062, + -0.09375743567943573, + -0.5040683150291443, + -0.9596792459487915 + ], + [ + 1.6738543510437012, + 0.6350066661834717, + 0.19669359922409058, + -0.1396545171737671, + -0.9625160694122314, + 2.207883596420288, + 1.7678128480911255, + 1.319273829460144, + -0.267609566450119, + -0.5747214555740356, + 0.19376236200332642, + -1.4490230083465576, + -2.6397883892059326, + -3.2567262649536133, + 1.2646806240081787, + -0.14523190259933472, + -0.050948500633239746, + 0.6943628787994385, + -0.1503342241048813, + -0.7313232421875, + -1.7930638790130615 + ], + [ + 2.139432668685913, + 0.34125545620918274, + 0.6370852589607239, + 0.8591548204421997, + 0.14890235662460327, + 2.6835381984710693, + 2.324920892715454, + 2.0091233253479004, + 0.13480046391487122, + -0.47407835721969604, + -1.102479100227356, + -1.0710523128509521, + -2.3226847648620605, + -3.6534740924835205, + 1.8767409324645996, + -0.01776304841041565, + -1.167153000831604, + 0.693484902381897, + 0.563054084777832, + -0.17544519901275635, + -0.8315125703811646 + ], + [ + 1.1254388093948364, + -0.03027276322245598, + -0.8753920793533325, + 0.3578681945800781, + -0.4256642460823059, + 1.7686536312103271, + 2.253190517425537, + 2.8274970054626465, + 0.3938461244106293, + -1.4084759950637817, + -0.7252857685089111, + -0.07288885116577148, + -1.7749395370483398, + -2.1030378341674805, + 1.92497980594635, + 0.20398634672164917, + -0.03521168231964111, + 1.4702520370483398, + -0.36331063508987427, + -1.5291211605072021, + -0.8369957804679871 + ] + ], + "predicted_scores": [ + 5.142221450805664, + 5.011483669281006, + 4.925361156463623, + 4.686418056488037, + 4.659388542175293, + 4.544834136962891, + 4.415966510772705, + 3.512410879135132 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1747178030/train1", + "selected_index": 0, + "state_hash": "87c74ddf97c53ddeeaae98467b4a13b1e0e499bd94eddb617759d39164cab542", + "stored_base_utility": 0.18184101581573486, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.009015560150146484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009015560150146484 + }, + { + "contact_quality": null, + "progress": 0.01863894797861576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01863894797861576 + }, + { + "contact_quality": null, + "progress": 0.010752241127192974, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010752241127192974 + }, + { + "contact_quality": null, + "progress": 0.007264256477355957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007264256477355957 + }, + { + "contact_quality": null, + "progress": 0.007598539348691702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007598539348691702 + }, + { + "contact_quality": null, + "progress": 0.006039460655301809, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006039460655301809 + }, + { + "contact_quality": null, + "progress": 0.0047194757498800755, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0047194757498800755 + }, + { + "contact_quality": null, + "progress": 0.004894236568361521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004894236568361521 + } + ], + "candidate_progress": [ + 0.009015560150146484, + 0.01863894797861576, + 0.010752241127192974, + 0.007264256477355957, + 0.007598539348691702, + 0.006039460655301809, + 0.0047194757498800755, + 0.004894236568361521 + ], + "candidate_source_chart_ids": [ + "ms-e84163d9ef2e7014daca", + "ms-6a3add21affa4796f9be", + "ms-f85a653af630821f2572", + "ms-dc26b448cce4164a23e6", + "ms-e84163d9ef2e7014daca", + "ms-7eaea428dfdef7255a66", + "ms-0e592951fa32139b6dc9", + "ms-6a3add21affa4796f9be" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-57be9ae181c077e7821f", + "generated_tangents": [ + [ + -0.673485279083252, + -0.11057630181312561, + 1.162347435951233, + -0.4004848003387451, + 0.401690810918808, + 2.4316258430480957, + 1.357682704925537, + 0.14826972782611847, + -0.14617441594600677, + 0.9371858835220337, + -0.2728007435798645, + 0.8436618447303772, + 1.2050950527191162, + 1.2057795524597168, + 0.9029747247695923, + -0.2893470525741577, + -0.2767745852470398, + 0.2502163052558899, + 0.37158024311065674, + 1.3908026218414307, + -0.16359004378318787 + ], + [ + -0.60336834192276, + -0.17724356055259705, + 1.0772401094436646, + -0.3030449151992798, + 0.5300936698913574, + 2.089449882507324, + 0.9355648756027222, + -0.14148828387260437, + -0.02768166922032833, + 0.5793574452400208, + -0.39049065113067627, + 0.28496310114860535, + 0.9720772504806519, + 1.229799509048462, + 0.7502161860466003, + -0.16310334205627441, + -0.2300533801317215, + 0.15497493743896484, + 0.37238848209381104, + 1.4026505947113037, + -0.38596704602241516 + ], + [ + -0.5618137717247009, + -0.02821844071149826, + 1.1908972263336182, + -0.3832213878631592, + 0.8535086512565613, + 2.410517454147339, + 1.0895622968673706, + 0.1309642791748047, + -0.04606577008962631, + 0.7881467342376709, + -0.10882357507944107, + 0.5313479900360107, + 1.2998323440551758, + 1.5107380151748657, + 0.7132909297943115, + -0.2532687187194824, + -0.0633615106344223, + -0.10792712867259979, + 0.5407611131668091, + 1.3837144374847412, + 0.0009638965129852295 + ], + [ + -0.583907961845398, + -0.08239202201366425, + 1.225914716720581, + -0.514095664024353, + 0.5955853462219238, + 2.2436342239379883, + 1.195970058441162, + 0.15324531495571136, + -0.11245833337306976, + 0.8402239084243774, + -0.32664012908935547, + 0.7294513583183289, + 1.5859030485153198, + 0.7674247622489929, + 0.5557411313056946, + -0.4299098253250122, + 0.018211573362350464, + 0.234456866979599, + 0.24528399109840393, + 1.4658477306365967, + -0.04363994300365448 + ], + [ + -0.7100796103477478, + -0.10140429437160492, + 1.123480200767517, + -0.5277652740478516, + 0.6968957185745239, + 2.523554563522339, + 1.3010798692703247, + 0.10423418879508972, + -0.09796861559152603, + 0.9269856214523315, + -0.2893550992012024, + 0.8262448310852051, + 1.342462182044983, + 0.9518347978591919, + 0.7416843771934509, + -0.3507519066333771, + -0.06938178837299347, + 0.23946513235569, + 0.33395281434059143, + 1.284440040588379, + -0.1417977213859558 + ], + [ + -0.5256441831588745, + -0.08357777446508408, + 1.140179991722107, + -0.29000991582870483, + 0.25538700819015503, + 2.684236526489258, + 1.121407151222229, + -0.00708770751953125, + 0.023272830992937088, + 1.146262764930725, + -0.20857489109039307, + 0.5451996326446533, + 1.8633259534835815, + 0.9819211959838867, + 0.564016580581665, + -0.35496509075164795, + 0.1230098307132721, + 0.48420006036758423, + 0.3613831698894501, + 1.1079778671264648, + 0.261741042137146 + ], + [ + -0.7110994458198547, + -0.22062639892101288, + 1.2708433866500854, + -0.4487105906009674, + 1.060157299041748, + 2.3540022373199463, + 1.404695987701416, + 0.01504121720790863, + -0.03042696975171566, + 1.1687805652618408, + -0.1746625006198883, + 0.856299877166748, + 1.4687488079071045, + 2.0115089416503906, + 0.41887542605400085, + -0.12903989851474762, + -0.04874922335147858, + 0.10019809007644653, + 0.2882075607776642, + 1.2084572315216064, + 0.4652540683746338 + ], + [ + -0.7296510934829712, + -0.21878530085086823, + 1.2190682888031006, + -0.37322354316711426, + 0.9713817834854126, + 1.8391281366348267, + 1.2307020425796509, + 0.026589691638946533, + -0.03206340968608856, + 1.2204389572143555, + -0.10162902623414993, + 0.7409164905548096, + 1.6027806997299194, + 1.6609978675842285, + 0.4686562716960907, + -0.07023249566555023, + -0.049426332116127014, + 0.08257109671831131, + 0.5998162031173706, + 1.7177071571350098, + 0.5814725160598755 + ] + ], + "generated_utilities": [ + 0.009015560150146484, + 0.01863894797861576, + 0.010752241127192974, + 0.007264256477355957, + 0.007598539348691702, + 0.006039460655301809, + 0.0047194757498800755, + 0.004894236568361521 + ], + "group_id": "ms-57be9ae181c077e7821f", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.28031715750694275, + 0.19804009795188904, + 0.15024888515472412, + 0.23104076087474823, + 0.21580509841442108, + 0.013450047001242638, + 0.20318475365638733, + 0.22562944889068604, + 0.14178292453289032, + 0.15763869881629944, + 0.27735981345176697, + 0.1909358948469162, + 0.06666654348373413, + 2.0, + 0.060366373509168625, + 0.05085722729563713 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.06739383935928345, + 0.06319573521614075, + 0.8779754638671875, + -0.03682136535644531, + -0.07093041390180588, + 1.3382306098937988, + 0.0, + 0.07168813049793243, + 0.01971740648150444, + 0.4866671562194824, + 0.07402461767196655, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + 0.03574584424495697, + 0.021435141563415527, + 0.0021790266036987305, + -0.06565424799919128, + 1.1393508911132812, + 0.0 + ], + [ + -0.17397063970565796, + -0.08753994852304459, + 0.8779754638671875, + -0.09028738737106323, + -0.17196941375732422, + 1.3382306098937988, + 0.0, + 0.022144019603729248, + 0.1154828891158104, + 0.4866671562194824, + -0.09212043881416321, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + -0.2021142691373825, + -0.022606968879699707, + 0.0021790266036987305, + 0.03272634744644165, + 1.1393508911132812, + 0.0 + ], + [ + -1.0144902467727661, + -0.023572498932480812, + 1.8779754638671875, + -1.0368213653564453, + -0.03799626976251602, + 2.338230609893799, + 2.239948272705078, + 0.16168542206287384, + 0.07221447676420212, + 1.4866671562194824, + -0.3557073175907135, + 1.3622210025787354, + 1.896626591682434, + 2.0382778644561768, + 1.4886415004730225, + 0.1997920423746109, + -0.4386618137359619, + 1.0021790266036987, + -0.3174663782119751, + 2.1393508911132812, + 1.9118322134017944 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.479896545410156, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0765557289123535, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.823664426803589 + ], + [ + -1.0723698139190674, + -0.05779706686735153, + 2.0670831203460693, + -0.830497682094574, + -0.17302533984184265, + 2.2277159690856934, + 2.0435242652893066, + 0.08191673457622528, + 0.08711377531290054, + 1.416495680809021, + -0.5633984804153442, + 1.3216067552566528, + 1.8660295009613037, + 2.2198705673217773, + 1.6887608766555786, + 0.06658947467803955, + -0.4010601043701172, + 1.1803714036941528, + -0.15144284069538116, + 2.308220386505127, + 1.8460347652435303 + ], + [ + -0.8901733756065369, + 0.3378032147884369, + 2.2360360622406006, + -1.207388162612915, + -0.34048303961753845, + 2.460798978805542, + 2.100795030593872, + -0.1847335249185562, + -0.06362440437078476, + 1.2987924814224243, + -0.11972835659980774, + 1.170393943786621, + 2.0620977878570557, + 2.4285032749176025, + 1.3666276931762695, + -0.01825574040412903, + -0.4620283842086792, + 1.1138218641281128, + -0.5787107944488525, + 1.78761887550354, + 1.740113615989685 + ], + [ + -1.1796774864196777, + -0.12369386106729507, + 1.546891689300537, + -1.0500564575195312, + -0.026454759761691093, + 2.177122116088867, + 2.0003931522369385, + 0.13697585463523865, + 0.37715619802474976, + 1.9734747409820557, + -0.16338464617729187, + 1.3775254487991333, + 2.3054146766662598, + 1.5546050071716309, + 1.2645983695983887, + 0.3812609910964966, + 0.02191823720932007, + 1.0944219827651978, + 0.09844094514846802, + 1.8145865201950073, + 2.0624771118164062 + ], + [ + -0.5514534711837769, + -0.4441075921058655, + 1.8971614837646484, + -1.6112515926361084, + 0.1306135058403015, + 2.736365556716919, + 2.270737886428833, + -0.35948747396469116, + 0.03968408331274986, + 1.061530351638794, + -0.29366230964660645, + 1.5066968202590942, + 2.34443998336792, + 2.3445637226104736, + 1.2286460399627686, + 0.13919776678085327, + -0.21014492213726044, + 0.9723342061042786, + -0.5822539329528809, + 1.8932968378067017, + 2.159864664077759 + ], + [ + -0.9470903873443604, + -0.7433359622955322, + 1.798203945159912, + -0.5850675106048584, + -0.7611241340637207, + 2.2761964797973633, + 2.0114798545837402, + 0.7724847793579102, + -0.23453289270401, + 0.7384847402572632, + -1.022956132888794, + 1.0059813261032104, + 1.652310848236084, + 2.6376113891601562, + 2.148331642150879, + 0.9844839572906494, + 0.18531352281570435, + 1.6280735731124878, + -0.06812705099582672, + 2.2684829235076904, + 1.1641287803649902 + ], + [ + -1.729402780532837, + 0.15382106602191925, + 1.0736894607543945, + -0.5141006708145142, + -0.6476962566375732, + 2.0120980739593506, + 2.8703343868255615, + 0.5578619837760925, + -0.8154823184013367, + 1.4325131177902222, + -1.0468122959136963, + 0.9670607447624207, + 1.973939061164856, + 2.7364890575408936, + 1.8303502798080444, + 0.8089333772659302, + -0.41138362884521484, + 1.8146085739135742, + -0.11676108837127686, + 2.3541221618652344, + 2.3875224590301514 + ], + [ + -1.8409323692321777, + -0.2118261456489563, + 1.5354409217834473, + -0.3873547911643982, + 0.7043789029121399, + 2.2068612575531006, + 2.5864486694335938, + 0.8632195591926575, + -0.10360705107450485, + 1.6340630054473877, + -0.017105937004089355, + 1.90720796585083, + 2.608898639678955, + 2.3387808799743652, + 2.3744726181030273, + 0.13692857325077057, + -0.7716406583786011, + 1.3477450609207153, + -0.9811362028121948, + 2.15514874458313, + 2.298004150390625 + ], + [ + -1.2484530210494995, + 0.5977066159248352, + 2.8256869316101074, + -0.3418053388595581, + -1.031638503074646, + 1.7023751735687256, + 1.4487309455871582, + -0.3013373017311096, + -0.3682725131511688, + 2.4404053688049316, + -0.8263542652130127, + 2.1105899810791016, + 2.5769248008728027, + 1.6751525402069092, + 0.8367876410484314, + -0.1554209142923355, + 0.17985862493515015, + 1.0017495155334473, + -0.3176954984664917, + 2.6737403869628906, + 1.7722500562667847 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -1.0015538930892944, + 0.43126925826072693, + 1.2786648273468018, + -0.7182899713516235, + 0.06109866499900818, + 2.323270797729492, + 1.3882524967193604, + -0.2783380150794983, + 0.35085979104042053, + 0.5870303511619568, + -1.0134913921356201, + 0.4675567150115967, + 0.9185858964920044, + 1.9441449642181396, + 1.3693488836288452, + 0.05759717524051666, + 0.19058758020401, + 1.9360744953155518, + -1.0018632411956787, + 2.043489456176758, + 1.1226990222930908 + ] + ], + "predicted_scores": [ + 6.7034149169921875, + 6.5890398025512695, + 6.502756118774414, + 6.359658241271973, + 6.356502056121826, + 5.557772636413574, + 4.942564964294434, + 4.4306511878967285 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train1", + "selected_index": 0, + "state_hash": "9c4de49d50d74c8c571620759fff1d43be23093a5f66e946f7ae6d75cb26ade5", + "stored_base_utility": 0.28031715750694275, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1544450968503952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1544450968503952 + }, + "base_progress": 0.1544450968503952, + "base_success": false, + "base_utility": 0.1544450968503952, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07292797416448593, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07292797416448593 + }, + { + "contact_quality": null, + "progress": 0.06293817609548569, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06293817609548569 + }, + { + "contact_quality": null, + "progress": 0.07210147380828857, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07210147380828857 + }, + { + "contact_quality": null, + "progress": 0.057973701506853104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057973701506853104 + }, + { + "contact_quality": null, + "progress": 0.05558790639042854, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05558790639042854 + }, + { + "contact_quality": null, + "progress": 0.07254187017679214, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07254187017679214 + }, + { + "contact_quality": null, + "progress": 0.07161635160446167, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07161635160446167 + }, + { + "contact_quality": null, + "progress": 0.061803679913282394, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061803679913282394 + } + ], + "candidate_progress": [ + 0.07292797416448593, + 0.06293817609548569, + 0.07210147380828857, + 0.057973701506853104, + 0.05558790639042854, + 0.07254187017679214, + 0.07161635160446167, + 0.061803679913282394 + ], + "candidate_source_chart_ids": [ + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-ff129e39095c2861b522" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6388b5d2356c556c3e55", + "generated_tangents": [ + [ + -0.3968668580055237, + -0.38062024116516113, + 1.7399386167526245, + -0.2414710968732834, + 1.1255995035171509, + 3.039266347885132, + 3.546168088912964, + 2.1424098014831543, + -0.005538465455174446, + -0.07015720754861832, + 1.9714783430099487, + -0.36543887853622437, + 1.792925477027893, + 2.405517339706421, + -0.2426699697971344, + 0.47847115993499756, + 0.714500367641449, + 0.3246252238750458, + 1.9411358833312988, + 1.951286792755127, + 2.6931686401367188 + ], + [ + -0.4288380742073059, + -0.1759163737297058, + 1.9252293109893799, + -0.2241262048482895, + 0.9497015476226807, + 2.679028034210205, + 3.121192216873169, + 2.2287421226501465, + -0.10378557443618774, + -0.2547786235809326, + 1.5193686485290527, + 0.1322452425956726, + 1.9332810640335083, + 2.3389811515808105, + 0.043793801218271255, + 0.525780200958252, + 0.974789559841156, + 0.10781662911176682, + 1.7040464878082275, + 2.568972587585449, + 2.3547987937927246 + ], + [ + -0.6143622398376465, + -0.20890432596206665, + 1.8667266368865967, + -0.4120851755142212, + 1.2120611667633057, + 2.5719242095947266, + 3.503678798675537, + 2.175184488296509, + -0.025717923417687416, + -0.1382434070110321, + 1.5058507919311523, + 0.19175328314304352, + 2.1061060428619385, + 2.4802355766296387, + 0.09369316697120667, + 0.5243374109268188, + 0.6584591269493103, + 0.10176264494657516, + 1.612694263458252, + 2.428436279296875, + 2.6008424758911133 + ], + [ + -0.5036369562149048, + -0.18751846253871918, + 2.042149305343628, + -0.4116286039352417, + 1.0883207321166992, + 2.580353021621704, + 3.3304309844970703, + 2.2497329711914062, + -0.08341985195875168, + -0.08701028674840927, + 1.4502122402191162, + 0.2606508731842041, + 2.2266836166381836, + 2.2686614990234375, + 0.1270129680633545, + 0.5080239772796631, + 0.7286629676818848, + 0.08464661985635757, + 1.6795477867126465, + 2.7060940265655518, + 2.65683913230896 + ], + [ + -0.4743857979774475, + -0.2309812754392624, + 2.2059569358825684, + -0.2929331064224243, + 1.1745760440826416, + 2.7660915851593018, + 3.2718822956085205, + 2.2960586547851562, + -0.1819061040878296, + -0.16178375482559204, + 1.6534020900726318, + 0.2426091581583023, + 2.1609795093536377, + 2.7593913078308105, + 0.008061695843935013, + 0.8117608428001404, + 0.8180800676345825, + 0.15561014413833618, + 1.6041498184204102, + 2.6488308906555176, + 3.138012170791626 + ], + [ + -0.23429040610790253, + -0.30268633365631104, + 1.6476975679397583, + -0.2855158746242523, + 1.16886305809021, + 2.704505205154419, + 3.3356595039367676, + 2.3955564498901367, + -0.029977740719914436, + -0.06803838163614273, + 1.5985625982284546, + -0.014424756169319153, + 1.702675700187683, + 2.734076499938965, + 0.018823731690645218, + 0.8042119145393372, + 0.4351709485054016, + 0.27187758684158325, + 1.7853443622589111, + 1.9134089946746826, + 2.8341336250305176 + ], + [ + -0.6073477864265442, + -0.18093299865722656, + 2.125244379043579, + -0.2429467737674713, + 1.350313425064087, + 3.0189356803894043, + 3.2697079181671143, + 2.1868600845336914, + -0.0840703621506691, + -0.37311649322509766, + 1.5795865058898926, + 0.45110803842544556, + 2.3109750747680664, + 2.7224833965301514, + 0.04407507926225662, + 0.7730731964111328, + 0.8389228582382202, + 0.0641772449016571, + 1.5328953266143799, + 2.33486270904541, + 3.0427112579345703 + ], + [ + -0.5567232966423035, + -0.2194473296403885, + 2.038078546524048, + -0.39234840869903564, + 1.2892754077911377, + 2.6152706146240234, + 3.3098623752593994, + 2.311594247817993, + -0.19186042249202728, + -0.34591808915138245, + 1.69652259349823, + 0.16723546385765076, + 2.188263416290283, + 3.0482404232025146, + 0.06031256169080734, + 0.7795422673225403, + 0.8894026875495911, + 0.07335814833641052, + 1.4641940593719482, + 2.4214982986450195, + 3.0559797286987305 + ] + ], + "generated_utilities": [ + 0.07292797416448593, + 0.06293817609548569, + 0.07210147380828857, + 0.057973701506853104, + 0.05558790639042854, + 0.07254187017679214, + 0.07161635160446167, + 0.061803679913282394 + ], + "group_id": "ms-6388b5d2356c556c3e55", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08186312764883041, + 0.11667267978191376, + 0.09751813113689423, + 2.0, + 0.27431026101112366, + 0.11760994046926498, + 0.20386962592601776, + 0.19832774996757507, + 0.22735612094402313, + 0.1356993168592453, + 0.1713169515132904, + 0.1792232096195221, + 0.06848804652690887, + 0.1094626635313034, + 0.11555596441030502, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0018978863954544067, + -0.4448504149913788, + 0.7108399868011475, + -0.33157044649124146, + -0.062200844287872314, + 1.4758033752441406, + 0.0, + 1.4153923988342285, + 0.14116239547729492, + 0.00777280330657959, + 0.5541017055511475, + 0.07029426097869873, + 0.26837193965911865, + 0.0, + -0.015542566776275635, + 0.3573532998561859, + 0.6168417930603027, + 0.3612897992134094, + 0.369077205657959, + 1.5924487113952637, + 0.0 + ], + [ + -0.09549225866794586, + -0.20593886077404022, + 1.7108399868011475, + -0.1554051786661148, + 0.9377991557121277, + 2.4758033752441406, + 2.4156031608581543, + 2.4153923988342285, + -0.2459496259689331, + -0.9922271966934204, + 1.5541017055511475, + 0.4461512267589569, + 1.2501436471939087, + 1.7566912174224854, + -0.8014360666275024, + 0.04551727697253227, + 1.6168417930603027, + 0.5393297672271729, + 1.369077205657959, + 2.5924487113952637, + 3.1126015186309814 + ], + [ + -0.11610143631696701, + -0.3961976170539856, + 1.9081053733825684, + -0.14575888216495514, + 1.1381605863571167, + 2.7434439659118652, + 2.5136890411376953, + 2.6401467323303223, + -0.45453745126724243, + -0.9589858651161194, + 1.4083189964294434, + 0.5800129175186157, + 0.9940754175186157, + 1.7450767755508423, + -1.0145976543426514, + 0.3312625586986542, + 1.4729199409484863, + 0.2520747482776642, + 1.3623151779174805, + 2.5670394897460938, + 3.3589630126953125 + ], + [ + -0.2318952977657318, + 0.024853751063346863, + 1.829875111579895, + 0.09877045452594757, + 1.1515556573867798, + 2.2239503860473633, + 2.153250217437744, + 2.8057901859283447, + -0.1800437569618225, + -1.2225371599197388, + 1.421208143234253, + 0.6627710461616516, + 1.3895055055618286, + 1.856442928314209, + -0.5928603410720825, + 0.4364123046398163, + 1.7852745056152344, + 0.3345714807510376, + 1.1167365312576294, + 2.7676005363464355, + 3.041402578353882 + ], + [ + -0.2227625697851181, + -0.012646213173866272, + 1.7988159656524658, + 0.21457992494106293, + 0.8877373933792114, + 2.9257700443267822, + 2.2101094722747803, + 2.2874886989593506, + -0.09066365659236908, + -0.49909964203834534, + 2.042389392852783, + 0.6287553906440735, + 1.074569582939148, + 1.7805378437042236, + -1.1840007305145264, + 0.3728531002998352, + 1.165173053741455, + 0.3054528832435608, + 1.4924153089523315, + 2.5540966987609863, + 2.664964199066162 + ], + [ + -0.6864494681358337, + 0.119636669754982, + 1.2460956573486328, + -0.6367122530937195, + 1.4080986976623535, + 2.5262017250061035, + 2.508194923400879, + 2.1323020458221436, + -0.43302130699157715, + -1.2678899765014648, + 1.0870052576065063, + 0.46296897530555725, + 0.765267550945282, + 2.3093044757843018, + -0.46595194935798645, + -0.43870145082473755, + 2.028986930847168, + 0.26284897327423096, + 1.5542359352111816, + 2.530392646789551, + 2.7435970306396484 + ], + [ + 0.18175412714481354, + -0.478390634059906, + 1.286731481552124, + -0.8066962361335754, + 1.4787592887878418, + 2.1033878326416016, + 1.750453233718872, + 2.047861099243164, + 0.3163147568702698, + -0.30588728189468384, + 1.0766479969024658, + 0.5822455286979675, + 1.673675775527954, + 2.039386510848999, + -0.62891685962677, + 0.615638792514801, + 2.018416404724121, + 0.2530737817287445, + 1.34544837474823, + 3.2287912368774414, + 2.9459850788116455 + ], + [ + 0.5059412121772766, + 0.17952178418636322, + 1.3319568634033203, + -0.8850483894348145, + 0.6336387395858765, + 1.775376796722412, + 2.3280270099639893, + 1.7305657863616943, + -0.02617873251438141, + -0.7963138818740845, + 1.9142115116119385, + 0.6635866165161133, + 1.3493314981460571, + 1.9258644580841064, + -0.3795176148414612, + 0.7861210107803345, + 1.0032029151916504, + 0.8491684794425964, + 0.5918128490447998, + 2.399027109146118, + 2.3932981491088867 + ], + [ + -0.34448128938674927, + -0.2996852695941925, + 0.9233196377754211, + -0.22890916466712952, + 1.6932814121246338, + 3.465762138366699, + 2.2005038261413574, + 2.6472361087799072, + -0.8981531858444214, + -0.2658765912055969, + 1.8755900859832764, + 0.2962568402290344, + 1.8239326477050781, + 1.0176585912704468, + -0.7912051677703857, + 0.6727061867713928, + 2.199978828430176, + 1.015731692314148, + 0.43087130784988403, + 3.5876548290252686, + 3.893789052963257 + ] + ], + "predicted_scores": [ + 8.308979988098145, + 7.565439701080322, + 7.398580551147461, + 7.091735363006592, + 6.928080081939697, + 6.910342693328857, + 6.837721347808838, + 6.50447416305542 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train1", + "selected_index": 0, + "state_hash": "38d539e46140304e3f157cbc207aa0d9489e5470633c216a0379906af445a1ec", + "stored_base_utility": 0.08186312764883041, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06173701956868172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06173701956868172 + }, + "base_progress": 0.06173701956868172, + "base_success": false, + "base_utility": 0.06173701956868172, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.057138841599226, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057138841599226 + }, + { + "contact_quality": null, + "progress": 0.06841854006052017, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06841854006052017 + }, + { + "contact_quality": null, + "progress": 0.015207032673060894, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015207032673060894 + }, + { + "contact_quality": null, + "progress": 0.016131063923239708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016131063923239708 + }, + { + "contact_quality": null, + "progress": 0.08538073301315308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08538073301315308 + }, + { + "contact_quality": null, + "progress": 0.0494011826813221, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0494011826813221 + }, + { + "contact_quality": null, + "progress": 0.02298945188522339, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02298945188522339 + }, + { + "contact_quality": null, + "progress": 0.018225232139229774, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018225232139229774 + } + ], + "candidate_progress": [ + 0.057138841599226, + 0.06841854006052017, + 0.015207032673060894, + 0.016131063923239708, + 0.08538073301315308, + 0.0494011826813221, + 0.02298945188522339, + 0.018225232139229774 + ], + "candidate_source_chart_ids": [ + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-7e88428497a6b13936c4", + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-a2518826fa0a816cc46f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-638d2ba1917eff4d09e0", + "generated_tangents": [ + [ + 0.3360844850540161, + -0.5631576180458069, + 0.42826926708221436, + -0.05306026339530945, + 1.9138963222503662, + 0.6474291086196899, + 0.6063404083251953, + 0.3220643401145935, + 0.11976703256368637, + 0.2562875747680664, + 0.060126449912786484, + 0.5298478603363037, + -0.12947794795036316, + 0.5538510084152222, + 0.46762537956237793, + -0.05959991738200188, + -0.09273485839366913, + -0.21236087381839752, + 1.016845703125, + -1.1638799905776978, + -0.523855447769165 + ], + [ + 0.23037973046302795, + -0.527307391166687, + 0.47113341093063354, + -0.14062219858169556, + 1.9487664699554443, + 0.7238055467605591, + 0.21989122033119202, + 0.2621961236000061, + 0.03199471905827522, + 0.12914234399795532, + 0.31201422214508057, + 0.898644745349884, + 0.1890799105167389, + 0.3102106750011444, + 0.4614345133304596, + -0.21547609567642212, + -0.1286732405424118, + -0.11163227260112762, + 1.0411545038223267, + -1.0815834999084473, + -0.31273379921913147 + ], + [ + -0.05227935314178467, + -0.5841974020004272, + 0.8214445114135742, + -0.005569174885749817, + 1.8276751041412354, + 0.35953664779663086, + -0.018803954124450684, + -0.10537424683570862, + -0.006136782467365265, + 0.08871615678071976, + 0.25584447383880615, + 0.8758326768875122, + 0.09765169024467468, + 0.3560537099838257, + 0.45599764585494995, + -0.355105996131897, + -0.0563969761133194, + 0.12093324214220047, + 0.5417991876602173, + -1.1594804525375366, + -0.038095176219940186 + ], + [ + -0.05506591498851776, + -0.5924087762832642, + 0.6706127524375916, + 0.5257781744003296, + 1.266045331954956, + 0.20478864014148712, + 1.0375064611434937, + 0.10195954889059067, + 0.12375635653734207, + 0.2946399748325348, + 0.10719247907400131, + 0.6814927458763123, + 0.3802690804004669, + 0.8426738977432251, + 0.2701503336429596, + -0.2513330280780792, + -0.04093451797962189, + -0.1308555155992508, + 0.9174430966377258, + -0.7879283428192139, + -0.527397871017456 + ], + [ + 0.3089306950569153, + -0.6023566126823425, + 0.49269524216651917, + -0.13358056545257568, + 1.939436674118042, + 1.0642074346542358, + 0.5651950836181641, + 0.31345751881599426, + 0.03243149816989899, + 0.09227952361106873, + 0.007948838174343109, + 0.4876777231693268, + -0.012213468551635742, + 0.5890823006629944, + 0.5992427468299866, + -0.08782917261123657, + -0.2024044245481491, + -0.30906999111175537, + 1.0151417255401611, + -1.3159689903259277, + -0.5578432083129883 + ], + [ + 0.08377283066511154, + -0.6841627359390259, + 0.7012265920639038, + 0.1769711971282959, + 1.3453123569488525, + 0.32215750217437744, + 0.06348186731338501, + 0.1281505525112152, + 0.08698300272226334, + 0.2377529740333557, + 0.15218651294708252, + 0.6287205219268799, + 0.23558709025382996, + 0.38711631298065186, + 0.34391283988952637, + -0.2935248017311096, + -0.36701059341430664, + 0.19629979133605957, + 0.7564046382904053, + -1.1271963119506836, + -0.17813050746917725 + ], + [ + -0.03945416212081909, + -0.5818068981170654, + 0.6312779784202576, + 0.20385882258415222, + 1.534226417541504, + 0.47082000970840454, + -0.05443471670150757, + -0.030519947409629822, + 0.026358027011156082, + -0.0040660277009010315, + 0.1990625560283661, + 0.8969831466674805, + 0.30952897667884827, + 0.7024948596954346, + 0.29324811697006226, + -0.27860313653945923, + -0.516141414642334, + 0.17933474481105804, + 0.6370826363563538, + -1.00701904296875, + -0.35897037386894226 + ], + [ + 0.04158759117126465, + -0.60334712266922, + 0.442973792552948, + 0.1053362488746643, + 1.5350306034088135, + 1.008374571800232, + 0.4564563035964966, + 0.19211281836032867, + 0.029262414202094078, + 0.22067029774188995, + -0.0864112600684166, + 0.329892098903656, + -0.2380591779947281, + 0.4292786717414856, + 0.28826022148132324, + -0.3610919117927551, + -0.18386386334896088, + 0.20696431398391724, + -0.11754748225212097, + -1.646545171737671, + -0.08137795329093933 + ] + ], + "generated_utilities": [ + 0.057138841599226, + 0.06841854006052017, + 0.015207032673060894, + 0.016131063923239708, + 0.08538073301315308, + 0.0494011826813221, + 0.02298945188522339, + 0.018225232139229774 + ], + "group_id": "ms-638d2ba1917eff4d09e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.028925320133566856, + 0.11176212877035141, + 0.057868462055921555, + 0.09960909932851791, + 0.07349967956542969, + 0.029011588543653488, + 0.06443297863006592, + 0.05166193097829819, + 0.025834282860159874, + 0.056667208671569824, + 0.04477008432149887, + 0.019685089588165283, + 0.018628716468811035, + 0.08587563037872314, + 0.02232462167739868, + 0.010265171527862549 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04120570421218872, + 0.038339823484420776, + 0.5979268550872803, + -0.05306033045053482, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.3407015800476074, + -0.21580028533935547, + -0.036036908626556396, + -0.004389047622680664, + 0.5364620685577393, + 0.026709645986557007, + 0.0, + 1.0126078128814697, + -0.10034149885177612, + -0.16565871238708496, + 0.07125610113143921, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + -0.5276790261268616, + -0.19392354786396027, + 0.5979268550872803, + 0.05016906559467316, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.6852816939353943, + -0.21580028533935547, + 0.04531368613243103, + -0.09276384115219116, + 0.5364620685577393, + 0.021915271878242493, + 0.0, + 1.0126078128814697, + 0.29255303740501404, + -0.16565871238708496, + 0.14600259065628052, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + 0.5910290479660034, + -0.038985610008239746, + 1.5555552244186401, + -1.028314232826233, + 2.3103959560394287, + 0.8163888454437256, + 2.064336061477661, + 1.168483018875122, + -0.21718227863311768, + -0.19770365953445435, + 0.11310696601867676, + 0.6850457191467285, + -0.06804794818162918, + 0.33076921105384827, + 1.7529149055480957, + -0.562712550163269, + -1.6352436542510986, + 0.1224215030670166, + 3.354771852493286, + -1.1317338943481445, + -1.7348618507385254 + ] + ], + "predicted_scores": [ + 6.9197821617126465, + 6.764666557312012, + 6.156739234924316, + 5.908979415893555, + 5.825287818908691, + 5.521308422088623, + 5.370540618896484, + 4.671911716461182 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train1", + "selected_index": 0, + "state_hash": "5d6f4227fab3b4bb36756c9a15c32b6fe6c9a0ae4b07b3f20a2bd65c2594df13", + "stored_base_utility": 0.028925320133566856, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.007633765693753958, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007633765693753958 + }, + { + "contact_quality": null, + "progress": 0.004399994853883982, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004399994853883982 + }, + { + "contact_quality": null, + "progress": 0.005148271564394236, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005148271564394236 + }, + { + "contact_quality": null, + "progress": 0.0037710864562541246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0037710864562541246 + }, + { + "contact_quality": null, + "progress": 0.0034768779296427965, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0034768779296427965 + }, + { + "contact_quality": null, + "progress": 0.007886429317295551, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007886429317295551 + }, + { + "contact_quality": null, + "progress": 0.0032363932114094496, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0032363932114094496 + }, + { + "contact_quality": null, + "progress": 0.0019465485820546746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0019465485820546746 + } + ], + "candidate_progress": [ + 0.007633765693753958, + 0.004399994853883982, + 0.005148271564394236, + 0.0037710864562541246, + 0.0034768779296427965, + 0.007886429317295551, + 0.0032363932114094496, + 0.0019465485820546746 + ], + "candidate_source_chart_ids": [ + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-a6a6de64c39abad45f86", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-652ad552ba98d2736620", + "generated_tangents": [ + [ + 0.44328463077545166, + 0.08944632112979889, + 1.1233186721801758, + -0.6342301964759827, + 2.6212942600250244, + 2.7777037620544434, + 1.1180120706558228, + 0.3905259370803833, + -0.13039368391036987, + 0.8636714220046997, + 0.347335547208786, + 0.7396171689033508, + 0.21301797032356262, + 0.320237398147583, + -0.6468695402145386, + -0.2346024364233017, + 1.3507697582244873, + 0.013264164328575134, + 1.0822482109069824, + 2.213974952697754, + 2.575228452682495 + ], + [ + 0.22460892796516418, + 0.10911282896995544, + 1.2078988552093506, + -0.5381326675415039, + 2.284000873565674, + 2.28365421295166, + 1.9541079998016357, + 0.29657286405563354, + -0.12368787825107574, + 1.0407379865646362, + 0.520634114742279, + 1.1993212699890137, + 0.11919161677360535, + 0.37520527839660645, + -0.8313935995101929, + -0.02070493996143341, + 1.2995599508285522, + -0.13885383307933807, + 1.587677001953125, + 1.851771593093872, + 1.5285829305648804 + ], + [ + 0.11763723194599152, + 0.16928982734680176, + 1.2548750638961792, + -0.6948763132095337, + 2.777177333831787, + 2.4812123775482178, + 1.6804273128509521, + 0.44601404666900635, + 0.009550084359943867, + 0.7694616913795471, + 0.38177919387817383, + 1.2609425783157349, + 0.40404900908470154, + 0.8937294483184814, + -0.7064406275749207, + -0.07946361601352692, + 1.6611992120742798, + -0.11573787033557892, + 1.6066079139709473, + 2.0395593643188477, + 1.9090783596038818 + ], + [ + 0.014741957187652588, + 0.11712643504142761, + 1.5054949522018433, + -0.6112051010131836, + 2.2916512489318848, + 2.1692728996276855, + 0.9762204885482788, + 0.23812434077262878, + 0.027299486100673676, + 0.525099515914917, + 0.2947130799293518, + 1.2027831077575684, + 0.7017966508865356, + -0.37153053283691406, + -0.7292154431343079, + -0.28864961862564087, + 1.8713526725769043, + 0.05484373867511749, + 1.2416728734970093, + 2.16310453414917, + 1.9149621725082397 + ], + [ + 0.15705810487270355, + 0.21274326741695404, + 1.4380555152893066, + -0.7557823657989502, + 2.262181043624878, + 1.7250597476959229, + 1.7626023292541504, + 0.10116852819919586, + 0.0020106295123696327, + 0.8254652619361877, + 0.36281219124794006, + 1.2800666093826294, + 0.33038368821144104, + 0.35335612297058105, + -0.5088874697685242, + -0.11496816575527191, + 1.6460429430007935, + -0.10266251862049103, + 1.1497715711593628, + 2.248767137527466, + 1.8903944492340088 + ], + [ + 0.007033973932266235, + 0.054758474230766296, + 1.300036907196045, + -0.8114298582077026, + 2.267390489578247, + 1.9414392709732056, + 1.1136937141418457, + 0.29846006631851196, + 0.003335862420499325, + 0.5853330492973328, + 0.24868065118789673, + 1.1058058738708496, + -0.03307062387466431, + 0.25670677423477173, + -0.29727616906166077, + -0.06248842179775238, + 1.3406250476837158, + 0.08400772511959076, + 1.0439934730529785, + 2.1453020572662354, + 1.3445807695388794 + ], + [ + 0.17126376926898956, + 0.1989465355873108, + 1.42512047290802, + -0.7550135850906372, + 2.20028018951416, + 2.0143473148345947, + 1.4277539253234863, + 0.1929885447025299, + -0.022833550348877907, + 0.9543581008911133, + 0.2628089487552643, + 1.1975512504577637, + 0.07181712985038757, + 0.05113094300031662, + -0.5681172013282776, + -0.16490671038627625, + 1.6664643287658691, + -0.013347640633583069, + 1.3300714492797852, + 2.0404887199401855, + 1.8152005672454834 + ], + [ + -0.00817921757698059, + 0.002253517508506775, + 1.5720202922821045, + -0.6353524923324585, + 2.2705748081207275, + 1.2103488445281982, + 1.6786835193634033, + 0.13466280698776245, + -0.10966622084379196, + 1.0458556413650513, + 0.22026807069778442, + 1.342624306678772, + -0.26158997416496277, + 0.6853857040405273, + -0.7264761328697205, + -0.12796680629253387, + 1.7027888298034668, + 0.17405883967876434, + 1.5154500007629395, + 1.6549060344696045, + 1.3723491430282593 + ] + ], + "generated_utilities": [ + 0.007633765693753958, + 0.004399994853883982, + 0.005148271564394236, + 0.0037710864562541246, + 0.0034768779296427965, + 0.007886429317295551, + 0.0032363932114094496, + 0.0019465485820546746 + ], + "group_id": "ms-652ad552ba98d2736620", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08378833532333374, + 0.10458396375179291, + 0.19086095690727234, + 0.04039820283651352, + 0.15980230271816254, + 0.0428888201713562, + 0.2367449700832367, + 0.18185830116271973, + 0.17593276500701904, + 0.16103637218475342, + 0.15250858664512634, + 0.05226989835500717, + 0.1701633185148239, + 0.11590366065502167, + 0.05217729136347771, + 0.03575938940048218 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.09009888768196106, + 0.08798432350158691, + 0.718157172203064, + -0.6351244449615479, + 1.405301570892334, + 0.6596384048461914, + 0.0, + 0.060225605964660645, + -0.0894029438495636, + -0.10323715209960938, + 0.052752673625946045, + 0.5354657173156738, + -0.013395614922046661, + 0.0, + -0.3450838029384613, + -0.11172826588153839, + 0.7912200689315796, + 0.33702391386032104, + 0.5586241483688354, + 1.8093180656433105, + 0.0 + ], + [ + 0.557296633720398, + 1.087984323501587, + 1.718157172203064, + -1.6351244449615479, + 2.405301570892334, + 1.6596384048461914, + 1.7430553436279297, + 0.3360595703125, + -0.2521040737628937, + 0.8967628479003906, + 0.26587578654289246, + 1.5354657173156738, + -0.12294083833694458, + -0.022678710520267487, + -0.416314959526062, + -0.26016199588775635, + 1.7912200689315796, + -0.16825227439403534, + 1.5586241483688354, + 2.8093180656433105, + 2.2589924335479736 + ], + [ + 0.2635665237903595, + 1.0005124807357788, + 1.973160743713379, + -1.7548720836639404, + 2.176497459411621, + 1.5416926145553589, + 1.9556171894073486, + 0.2215275913476944, + -0.5432531833648682, + 0.8434366583824158, + 0.3559061288833618, + 1.6657240390777588, + -0.3025258779525757, + 0.04401832073926926, + -0.17062894999980927, + -0.1509227454662323, + 1.7901219129562378, + 0.0010261386632919312, + 1.726505994796753, + 2.9778714179992676, + 2.0530292987823486 + ], + [ + 0.23987123370170593, + 0.8904718160629272, + 1.467047929763794, + -1.990342378616333, + 2.7685346603393555, + 2.0486786365509033, + 2.013002395629883, + 0.006436288356781006, + -0.25436922907829285, + 1.1101897954940796, + 0.4448293447494507, + 1.3105798959732056, + 0.07367905974388123, + 0.23779401183128357, + -0.34580594301223755, + -0.35948312282562256, + 1.4435756206512451, + -0.35472720861434937, + 1.801603078842163, + 3.0683224201202393, + 2.2330009937286377 + ], + [ + 0.6738064289093018, + 1.3973212242126465, + 2.0960373878479004, + -1.735988736152649, + 2.5216798782348633, + 1.1975955963134766, + 2.234865665435791, + 0.4602587819099426, + -0.7169049382209778, + 0.9412583708763123, + 0.7081141471862793, + 1.1730585098266602, + -0.016052812337875366, + -0.10474976152181625, + -0.03458032011985779, + 0.044317394495010376, + 1.6339548826217651, + -0.5292693376541138, + 1.5412750244140625, + 3.018833875656128, + 2.6051149368286133 + ], + [ + 0.6305910348892212, + 1.540043592453003, + 1.626009464263916, + -1.8823078870773315, + 2.2431485652923584, + 2.1866872310638428, + 2.3003249168395996, + 0.25838330388069153, + -0.2936764061450958, + 1.2413102388381958, + -0.12014040350914001, + 1.743309497833252, + 0.3618801534175873, + -0.37178292870521545, + 0.11553955078125, + 0.04856392741203308, + 1.2029554843902588, + 0.04960852861404419, + 2.0190422534942627, + 2.5883281230926514, + 2.7636377811431885 + ], + [ + 0.5004369616508484, + 1.4795310497283936, + 2.325191020965576, + -1.7464677095413208, + 1.9271103143692017, + 1.2483245134353638, + 1.9852193593978882, + 0.24291110038757324, + 0.4463852345943451, + 0.5267151594161987, + 0.32179588079452515, + 1.2356117963790894, + 0.03201058506965637, + 0.2881847023963928, + -1.0345205068588257, + -0.4171831011772156, + 1.5080864429473877, + 0.11479203402996063, + 1.9481834173202515, + 2.840816020965576, + 2.427428960800171 + ], + [ + -0.3133848309516907, + 0.43174320459365845, + 2.305823802947998, + -1.657350778579712, + 2.010749578475952, + 1.9222235679626465, + 1.3648147583007812, + 0.1938892900943756, + -0.6084526777267456, + 0.7855725288391113, + -0.07715931534767151, + 0.9140809178352356, + -0.9242425560951233, + -0.4681842029094696, + -0.07827043533325195, + -0.23733587563037872, + 0.9328996539115906, + -0.6088451147079468, + 0.8040639162063599, + 2.620466947555542, + 2.5148611068725586 + ] + ], + "predicted_scores": [ + 3.608150005340576, + 3.326437473297119, + 2.825561761856079, + 2.500056743621826, + 2.399468421936035, + 2.315864086151123, + 2.0968191623687744, + 1.6071484088897705 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train1", + "selected_index": 0, + "state_hash": "8c1ab306d1e82f81085196c60b3c846d2e2eb20ebc538597af359a6252623e7e", + "stored_base_utility": 0.08378833532333374, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03784912824630737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03784912824630737 + }, + "base_progress": 0.03784912824630737, + "base_success": false, + "base_utility": 0.03784912824630737, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.04070079326629639, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04070079326629639 + }, + { + "contact_quality": null, + "progress": 0.035504478961229324, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035504478961229324 + }, + { + "contact_quality": null, + "progress": 0.016018351539969444, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016018351539969444 + }, + { + "contact_quality": null, + "progress": 0.057549476623535156, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057549476623535156 + }, + { + "contact_quality": null, + "progress": 0.029227495193481445, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029227495193481445 + }, + { + "contact_quality": null, + "progress": 0.029378632083535194, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029378632083535194 + }, + { + "contact_quality": null, + "progress": 0.03666381165385246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03666381165385246 + }, + { + "contact_quality": null, + "progress": 0.04491941258311272, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04491941258311272 + } + ], + "candidate_progress": [ + 0.04070079326629639, + 0.035504478961229324, + 0.016018351539969444, + 0.057549476623535156, + 0.029227495193481445, + 0.029378632083535194, + 0.03666381165385246, + 0.04491941258311272 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-48178f52a96de3d1ead0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-674fad0e7c042d23629e", + "generated_tangents": [ + [ + 0.9003629088401794, + 0.12425516545772552, + -0.30955418944358826, + -0.14490333199501038, + 0.5139075517654419, + -1.4620239734649658, + -0.17373090982437134, + 0.3943895101547241, + 0.12960994243621826, + -0.4282251000404358, + -0.20047026872634888, + -0.7172543406486511, + -1.6864415407180786, + -1.2767021656036377, + -0.46521663665771484, + -0.10093821585178375, + 0.3085433840751648, + -0.628913164138794, + -0.023214906454086304, + 0.03263598680496216, + 0.2237262725830078 + ], + [ + 0.9199603796005249, + 0.08735062181949615, + -0.3462953269481659, + -0.08642083406448364, + 0.5941511392593384, + -1.2761945724487305, + -0.2588474750518799, + 0.418645441532135, + 0.06355312466621399, + -0.35064005851745605, + -0.17876100540161133, + -0.647356390953064, + -1.6682637929916382, + -1.2554212808609009, + -0.4986473321914673, + -0.071780264377594, + 0.30545884370803833, + -0.6120912432670593, + -0.05120024085044861, + -0.14255347847938538, + -0.027466237545013428 + ], + [ + 1.0517892837524414, + 0.03220818191766739, + -0.12867340445518494, + 0.05941849946975708, + 0.31146305799484253, + -1.538813829421997, + -0.5194803476333618, + 0.7237954139709473, + 0.08202748745679855, + -0.2483474314212799, + 0.05738503858447075, + -0.7206576466560364, + -1.5642082691192627, + -0.7618886232376099, + -0.4593932628631592, + -0.1864432692527771, + 0.15552642941474915, + -0.3930931091308594, + -0.008394837379455566, + -0.05159834027290344, + 0.005968749523162842 + ], + [ + 1.182978630065918, + 0.15921711921691895, + -0.2721805274486542, + -0.20114116370677948, + 0.4701843857765198, + -1.541680932044983, + -0.27385467290878296, + 0.41247761249542236, + 0.12139921635389328, + -0.45550674200057983, + -0.18808534741401672, + -0.9331966638565063, + -1.6188327074050903, + -1.374698519706726, + -0.4295809268951416, + -0.13094736635684967, + 0.24121223390102386, + -0.6172925233840942, + 0.25906991958618164, + -0.11272430419921875, + 0.3728640079498291 + ], + [ + 1.0054574012756348, + -0.14592623710632324, + -0.30663397908210754, + 0.03191231191158295, + 0.6533571481704712, + -1.5023176670074463, + -0.08405900001525879, + 0.3093627095222473, + 0.1181986853480339, + -0.21082532405853271, + -0.27991580963134766, + -0.4078807830810547, + -1.3492491245269775, + -1.0489988327026367, + -0.4215231239795685, + -0.05872068554162979, + 0.28000956773757935, + -0.563254177570343, + -0.07488834857940674, + -0.4309867024421692, + -0.27577027678489685 + ], + [ + 1.010310173034668, + -0.12428206205368042, + -0.3011971414089203, + 0.1363234519958496, + 0.6821796298027039, + -1.4914907217025757, + -0.21904844045639038, + 0.3578451871871948, + 0.14482904970645905, + -0.28952568769454956, + -0.14728403091430664, + -0.42694759368896484, + -1.5940645933151245, + -0.9595520496368408, + -0.377424418926239, + -0.04055410623550415, + 0.26222702860832214, + -0.6909810900688171, + 0.010868072509765625, + -0.3265906870365143, + -0.3279358446598053 + ], + [ + 0.842851459980011, + -0.10780809819698334, + -0.22249743342399597, + 0.1693369448184967, + 0.5579236149787903, + -1.3317362070083618, + -0.11668068170547485, + 0.2850192189216614, + 0.10159546136856079, + -0.27994829416275024, + -0.16368147730827332, + -0.4000256061553955, + -0.9346339106559753, + -0.8347570300102234, + -0.3967498540878296, + -0.12279963493347168, + 0.300127774477005, + -0.4666268825531006, + -0.2992909550666809, + -0.48686856031417847, + -0.16061455011367798 + ], + [ + 1.1743879318237305, + -0.09086474776268005, + -0.5411816835403442, + 0.09675449132919312, + 0.8503625392913818, + -1.6534641981124878, + 0.3966398239135742, + 0.40719252824783325, + 0.12983600795269012, + -0.48168712854385376, + -0.04600173979997635, + -0.3031688332557678, + -1.4280933141708374, + -0.6938177347183228, + -0.17034503817558289, + -0.04279131442308426, + 0.4690910577774048, + -0.7339193224906921, + 0.25339069962501526, + -0.31196919083595276, + -0.19958674907684326 + ] + ], + "generated_utilities": [ + 0.04070079326629639, + 0.035504478961229324, + 0.016018351539969444, + 0.057549476623535156, + 0.029227495193481445, + 0.029378632083535194, + 0.03666381165385246, + 0.04491941258311272 + ], + "group_id": "ms-674fad0e7c042d23629e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07829316705465317, + 0.080319844186306, + 0.11335399001836777, + 0.15264075994491577, + 0.14950451254844666, + 0.07767044007778168, + 0.12988285720348358, + 0.10878288745880127, + 0.15573063492774963, + 0.10951457917690277, + 0.06582474708557129, + 0.02477661892771721, + 0.07745532691478729, + 0.03348777815699577, + 0.10050909221172333, + 0.033770203590393066 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.5533217191696167, + 0.5758762359619141, + -0.6480545997619629, + -0.10892429947853088, + 0.7943102717399597, + -2.7399702072143555, + -1.8395006656646729, + 1.447904109954834, + 1.4495785236358643, + -0.055675506591796875, + -1.3537942171096802, + 0.38944748044013977, + -1.6196211576461792, + -1.2710431814193726, + -0.9744676947593689, + 0.057668983936309814, + 1.8350472450256348, + -1.2618876695632935, + -1.2385435104370117, + -1.7976703643798828, + -1.9491409063339233 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9072847366333008, + 0.06923278421163559, + -0.14729690551757812, + 0.15558937191963196, + 0.40412187576293945, + -2.342223644256592, + 0.0, + 0.2589508295059204, + 0.35182610154151917, + 0.10176032781600952, + -0.3489869236946106, + 0.13150231540203094, + -1.3123579025268555, + 0.0, + 0.10376164317131042, + 0.0909266471862793, + 0.1919306516647339, + 0.28928786516189575, + -0.4625590741634369, + -0.30475670099258423, + 0.0 + ], + [ + 1.9072847366333008, + 0.05373973771929741, + -1.1472969055175781, + -0.47107377648353577, + 0.3914180397987366, + -3.342223644256592, + -2.4980990886688232, + 1.2589508295059204, + 0.8455543518066406, + -0.7174267172813416, + -1.0694091320037842, + 0.33346641063690186, + -2.3123579025268555, + -1.1108808517456055, + -0.33852219581604004, + -0.7232105731964111, + 1.145590901374817, + -0.7107121348381042, + -0.5049349069595337, + -1.112295150756836, + -1.244770884513855 + ], + [ + 1.808228611946106, + -0.18174676597118378, + -1.2057973146438599, + -0.6115378737449646, + 0.11874309182167053, + -3.3499772548675537, + -2.3861043453216553, + 1.2540870904922485, + 1.1210845708847046, + -0.953934371471405, + -1.3015953302383423, + 0.6089286804199219, + -2.1655566692352295, + -0.8375923037528992, + -0.26793593168258667, + -0.998324990272522, + 1.2481704950332642, + -0.8320856690406799, + -0.376935750246048, + -1.0882012844085693, + -0.9857641458511353 + ], + [ + 2.291844606399536, + 0.05574151128530502, + -0.9760321974754333, + -0.5710926055908203, + 0.8478215336799622, + -3.22356915473938, + -2.591245651245117, + 1.2184754610061646, + 0.855300784111023, + -1.0267595052719116, + -1.3318730592727661, + 0.4197496771812439, + -2.0368778705596924, + -1.3685163259506226, + -0.582176685333252, + -0.8082455992698669, + 1.3435251712799072, + -0.44022125005722046, + -0.06775334477424622, + -0.7692400217056274, + -0.8863754272460938 + ] + ], + "predicted_scores": [ + 9.506608009338379, + 9.447779655456543, + 9.234846115112305, + 9.209221839904785, + 8.154670715332031, + 8.111454963684082, + 7.987610340118408, + 6.852998733520508 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train1", + "selected_index": 0, + "state_hash": "6802b2d1fb8a5afa54b7321e2bb938176de48d74f28e9b2a9c741c23547f0ef1", + "stored_base_utility": 0.07829316705465317, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.12791956961154938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12791956961154938 + }, + "base_progress": 0.12791956961154938, + "base_success": false, + "base_utility": 0.12791956961154938, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07348597049713135, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07348597049713135 + }, + { + "contact_quality": null, + "progress": 0.06691998243331909, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06691998243331909 + }, + { + "contact_quality": null, + "progress": 0.0687488317489624, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0687488317489624 + }, + { + "contact_quality": null, + "progress": 0.07327929884195328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07327929884195328 + }, + { + "contact_quality": null, + "progress": 0.03650514408946037, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03650514408946037 + }, + { + "contact_quality": null, + "progress": 0.05396809056401253, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05396809056401253 + }, + { + "contact_quality": null, + "progress": 0.07279130071401596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07279130071401596 + }, + { + "contact_quality": null, + "progress": 0.13260090351104736, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13260090351104736 + } + ], + "candidate_progress": [ + 0.07348597049713135, + 0.06691998243331909, + 0.0687488317489624, + 0.07327929884195328, + 0.03650514408946037, + 0.05396809056401253, + 0.07279130071401596, + 0.13260090351104736 + ], + "candidate_source_chart_ids": [ + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-1ba37e30b90a3b28e62f", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-728b865677134d3788e3", + "generated_tangents": [ + [ + -0.39034903049468994, + -0.37187659740448, + 0.3935436010360718, + -0.15706869959831238, + 0.9356510043144226, + -0.19463494420051575, + 0.4388798177242279, + 0.7160687446594238, + 0.19778768718242645, + 0.15543249249458313, + -1.5249929428100586, + 0.8594302535057068, + -0.7428938150405884, + -0.5552669167518616, + -1.482895016670227, + 0.9974061250686646, + 1.7214030027389526, + 0.01351088285446167, + -0.2140149176120758, + -0.6090185642242432, + 0.4767541289329529 + ], + [ + -0.4475897550582886, + -0.3542618751525879, + 0.5799123048782349, + 0.12671992182731628, + 0.9840754866600037, + -0.1267329454421997, + 0.6425981521606445, + 0.8072195053100586, + 0.3213188648223877, + 0.23512426018714905, + -1.4188973903656006, + 0.6924213171005249, + -0.7114902138710022, + -0.19320417940616608, + -1.6145719289779663, + 0.9080706238746643, + 1.6354997158050537, + -0.19772334396839142, + -0.10790109634399414, + -0.6084801554679871, + 0.5870579481124878 + ], + [ + -0.4405338168144226, + -0.3784378170967102, + 0.5646163821220398, + 0.09684327244758606, + 0.8023883700370789, + -0.29672256112098694, + 0.5263015627861023, + 0.7598861455917358, + 0.30964475870132446, + 0.22663670778274536, + -1.4080426692962646, + 0.7018111944198608, + -0.7842860817909241, + -0.44861429929733276, + -1.646507740020752, + 0.8910717964172363, + 1.6700502634048462, + -0.20267723500728607, + -0.17231038212776184, + -0.5862141847610474, + 0.39029455184936523 + ], + [ + -0.34456372261047363, + -0.41968315839767456, + 0.7650783061981201, + 0.20784354209899902, + 0.6980795860290527, + -0.3027670979499817, + 0.8314871788024902, + 0.3897552490234375, + 0.30897587537765503, + 0.24327963590621948, + -1.3717589378356934, + 0.8708892464637756, + -0.5835776329040527, + -0.5545505881309509, + -1.8845280408859253, + 0.8902562260627747, + 1.5212655067443848, + -0.349348247051239, + -0.12915486097335815, + -0.6258441209793091, + 0.06586413085460663 + ], + [ + -0.6155652403831482, + -0.3373787999153137, + 1.1049389839172363, + 0.24397826194763184, + 1.1622910499572754, + 0.4856007695198059, + 0.5087839961051941, + 0.5048795342445374, + 0.21637371182441711, + 0.2952132821083069, + -0.8826477527618408, + 0.8144375681877136, + -1.2008814811706543, + -0.34453094005584717, + -1.3298373222351074, + 0.8847464323043823, + 1.4294888973236084, + -0.2635078430175781, + 0.4943919777870178, + -1.6388778686523438, + 0.4366621971130371 + ], + [ + -0.1289636343717575, + -0.4173429608345032, + 1.0081381797790527, + -0.022463232278823853, + 0.6412431597709656, + -0.6562385559082031, + 0.38539278507232666, + 0.7804857492446899, + 0.3084360659122467, + 0.006881922483444214, + -1.3052064180374146, + 0.37727031111717224, + -1.4309247732162476, + -0.7981573343276978, + -1.413253664970398, + 0.8792598843574524, + 1.9208170175552368, + -0.3883278965950012, + -0.5910097360610962, + -1.0852289199829102, + 0.38884368538856506 + ], + [ + -0.9370386600494385, + -0.3018230199813843, + 0.8576704263687134, + 0.3421913683414459, + 0.9998913407325745, + -0.38233909010887146, + 0.663680374622345, + 0.6609971523284912, + 0.42053672671318054, + 0.07217412441968918, + -1.364300012588501, + 1.2068727016448975, + -1.0331658124923706, + -0.07561425864696503, + -1.806665301322937, + 0.7029359340667725, + 1.1926920413970947, + -0.3325579762458801, + 0.057507649064064026, + -0.39939096570014954, + 0.16372086107730865 + ], + [ + -0.3966712951660156, + -0.34475117921829224, + 0.453570157289505, + 0.05661824345588684, + 0.8598929047584534, + -0.015782281756401062, + 0.1667470633983612, + 0.6924062967300415, + 0.2899540662765503, + -0.09931730479001999, + -1.3228330612182617, + 0.12798099219799042, + -1.021296739578247, + -0.6155245304107666, + -1.633471131324768, + 0.8296821117401123, + 1.7047876119613647, + -0.4488670229911804, + -0.6817136406898499, + -0.7421263456344604, + -0.2532697319984436 + ] + ], + "generated_utilities": [ + 0.07348597049713135, + 0.06691998243331909, + 0.0687488317489624, + 0.07327929884195328, + 0.03650514408946037, + 0.05396809056401253, + 0.07279130071401596, + 0.13260090351104736 + ], + "group_id": "ms-728b865677134d3788e3", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05257302522659302, + 0.05559448525309563, + 0.11835493892431259, + 0.04238405078649521, + 0.16897141933441162, + 0.02001136541366577, + 0.1609041690826416, + 0.14248299598693848, + 0.14091376960277557, + 0.17822234332561493, + 0.22658847272396088, + 0.1736571490764618, + 0.18659374117851257, + 0.12753915786743164, + 0.07164359092712402, + 0.0063618822023272514 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04065108299255371, + 0.05943702906370163, + 0.0039327144622802734, + 0.10760968923568726, + -0.3465921878814697, + -0.6843758821487427, + 0.0, + 0.3034271001815796, + -0.38217779994010925, + 0.019724853336811066, + -1.1125431060791016, + -0.016437113285064697, + 0.09826856851577759, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + 0.055904269218444824, + -0.2721228003501892, + 0.07830148935317993, + 0.0 + ], + [ + -0.9075424671173096, + 0.06243014335632324, + 1.0039327144622803, + -0.6845037937164307, + 0.050797268748283386, + -1.6843758821487427, + -1.3340954780578613, + 1.3034271001815796, + -0.29618290066719055, + -0.035879287868738174, + -2.1125431060791016, + 0.9835628867149353, + -0.9017314314842224, + -1.01132333278656, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233 + ], + [ + -1.117501139640808, + 0.23503510653972626, + 0.9669153094291687, + -0.8317459225654602, + 0.06331060081720352, + -1.5474717617034912, + -1.3840656280517578, + 1.240980863571167, + -0.053158730268478394, + 0.10333722829818726, + -2.2084054946899414, + 1.1802411079406738, + -0.8987206816673279, + -1.136216402053833, + -1.9180536270141602, + 1.4563629627227783, + 2.625094413757324, + -0.1296394169330597, + 0.3340006470680237, + -0.7212131023406982, + 0.9384214878082275 + ], + [ + -1.2640053033828735, + 0.2415706366300583, + 0.9424604177474976, + -0.8472410440444946, + 0.35185110569000244, + -1.8022713661193848, + -1.2391949892044067, + 0.9333417415618896, + 0.03309231996536255, + 0.22933605313301086, + -1.7521884441375732, + 1.0360857248306274, + -0.6365935802459717, + -0.7614871859550476, + -2.103987216949463, + 1.5381183624267578, + 2.11069917678833, + -0.5904175639152527, + -0.07874824106693268, + -0.3884509205818176, + 1.0602728128433228 + ], + [ + -0.7705078721046448, + 0.18350359797477722, + 0.579454779624939, + -0.5434049963951111, + 0.2274666726589203, + -1.7208446264266968, + -1.5971848964691162, + 1.6145182847976685, + -0.6915329694747925, + 0.20122811198234558, + -1.8584421873092651, + 0.9026989340782166, + -0.5741626024246216, + -0.5299967527389526, + -1.544856309890747, + 1.7236231565475464, + 2.1288580894470215, + -0.7592856287956238, + 0.3358893394470215, + -0.2929669916629791, + 1.67848801612854 + ], + [ + -0.3686429262161255, + 0.3043383061885834, + 0.7294905781745911, + -0.8736650347709656, + 0.3122289776802063, + -1.5051836967468262, + -1.8593363761901855, + 0.8389579653739929, + -0.8785254955291748, + -0.5520821213722229, + -1.8517990112304688, + 1.3417646884918213, + -0.32390326261520386, + -0.42145586013793945, + -1.6530441045761108, + 1.7202519178390503, + 2.180105209350586, + -0.4262692928314209, + -0.1161729246377945, + -0.08550512790679932, + 1.1130309104919434 + ], + [ + -0.9971213936805725, + -0.4085577130317688, + 0.9404422044754028, + -0.9977414011955261, + 0.4963066577911377, + -1.4445728063583374, + -1.8557989597320557, + 1.3813822269439697, + -0.9350934028625488, + 0.6622016429901123, + -2.0982701778411865, + 0.5712680220603943, + -1.3476169109344482, + -0.8056140542030334, + -2.2595674991607666, + 1.7601569890975952, + 2.5158329010009766, + -0.8607662916183472, + -0.010314732789993286, + -0.9802933931350708, + 0.587103545665741 + ], + [ + -0.538765549659729, + -0.2632336914539337, + 0.574018120765686, + -1.3315669298171997, + 0.23096206784248352, + -1.4029772281646729, + -1.2746275663375854, + 1.674896478652954, + -0.6276504993438721, + -0.06763898581266403, + -1.44939124584198, + 1.1256837844848633, + -0.6540690064430237, + -0.7850072979927063, + -2.472240686416626, + 1.201231837272644, + 2.056978702545166, + 0.4725829064846039, + 0.7320148944854736, + -0.47291314601898193, + 1.2244172096252441 + ], + [ + -0.5474913120269775, + 0.3537777066230774, + 0.1717698574066162, + -0.9464929103851318, + -0.24061335623264313, + -2.5145106315612793, + -1.9605224132537842, + 2.171875476837158, + 0.5462924242019653, + 0.2324821650981903, + -2.4616305828094482, + 1.3455290794372559, + -0.703314483165741, + -0.6289631128311157, + -1.848944902420044, + 1.755210518836975, + 2.649005174636841, + -0.931267499923706, + -0.3109248876571655, + 0.21283608675003052, + 0.6994417309761047 + ], + [ + -1.4273285865783691, + 0.48591291904449463, + 1.4724129438400269, + -1.1116547584533691, + -0.04224221408367157, + -1.1567587852478027, + -1.0686161518096924, + 1.7429871559143066, + -1.1814582347869873, + 0.07730802893638611, + -2.1004433631896973, + 0.45253968238830566, + -0.7293854355812073, + -0.4477439522743225, + -1.986203908920288, + 1.1158549785614014, + 2.0316240787506104, + 0.5763667821884155, + -0.48857587575912476, + -1.07709801197052, + 0.5611822605133057 + ] + ], + "predicted_scores": [ + 2.4478979110717773, + 2.437049627304077, + 2.4245622158050537, + 2.25917649269104, + 1.7486274242401123, + 1.7416589260101318, + 1.6707737445831299, + 1.5316259860992432 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296/train1", + "selected_index": 0, + "state_hash": "62e00728e448102460275bc2c35150dbf429ab0d5670a6706fa41256273bac3c", + "stored_base_utility": 0.05257302522659302, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20529556274414062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20529556274414062 + }, + "base_progress": 0.20529556274414062, + "base_success": false, + "base_utility": 0.20529556274414062, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03273266553878784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03273266553878784 + }, + { + "contact_quality": null, + "progress": 0.03373098373413086, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03373098373413086 + }, + { + "contact_quality": null, + "progress": 0.04705389216542244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04705389216542244 + }, + { + "contact_quality": null, + "progress": 0.05021713301539421, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05021713301539421 + }, + { + "contact_quality": null, + "progress": 0.04120592400431633, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04120592400431633 + }, + { + "contact_quality": null, + "progress": 0.09449902921915054, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09449902921915054 + }, + { + "contact_quality": null, + "progress": 0.051247138530015945, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.051247138530015945 + }, + { + "contact_quality": null, + "progress": 0.04602358862757683, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04602358862757683 + } + ], + "candidate_progress": [ + 0.03273266553878784, + 0.03373098373413086, + 0.04705389216542244, + 0.05021713301539421, + 0.04120592400431633, + 0.09449902921915054, + 0.051247138530015945, + 0.04602358862757683 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-6ddecc7f074b9e267409", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "generated_tangents": [ + [ + 0.9338359832763672, + -0.027668345719575882, + -0.15829375386238098, + 0.25817638635635376, + 0.500524640083313, + -2.3095037937164307, + -1.6588237285614014, + -0.8597820997238159, + -0.04362007975578308, + 0.9941971898078918, + 0.5525499582290649, + 0.25136032700538635, + 1.3738535642623901, + -0.22684724628925323, + -0.536362886428833, + -0.22348515689373016, + 1.0505791902542114, + 1.003051519393921, + 0.3198562264442444, + -0.801285445690155, + -0.5484545230865479 + ], + [ + 0.7688387632369995, + -0.11683923006057739, + -0.6016196012496948, + 0.21707341074943542, + 0.31842467188835144, + -2.13503098487854, + -1.047135353088379, + -1.1803016662597656, + -0.016026286408305168, + 0.7579044103622437, + 0.4563840329647064, + -0.14257311820983887, + 1.113267421722412, + 0.3017989993095398, + -0.44573020935058594, + -0.40615108609199524, + 0.9448165893554688, + 0.7917150259017944, + -0.0023890435695648193, + -1.0027339458465576, + -0.49909844994544983 + ], + [ + 0.9584357738494873, + -0.08497299998998642, + -0.4230339825153351, + 0.1546584665775299, + 0.354657381772995, + -2.7537901401519775, + -1.4100141525268555, + -0.8773391246795654, + -0.050078339874744415, + 0.8806989789009094, + 0.7529194951057434, + 0.0011425763368606567, + 0.4982249438762665, + 0.3286147713661194, + -0.3247344195842743, + -0.3099651634693146, + 0.9654372930526733, + 0.6347294449806213, + 0.03375589847564697, + -1.0854305028915405, + -0.07017646729946136 + ], + [ + 0.8489947319030762, + -0.11245530843734741, + -0.5541179180145264, + 0.26276764273643494, + 0.3777993321418762, + -2.4220778942108154, + -1.3048346042633057, + -0.8763859272003174, + -0.015903599560260773, + 0.8433221578598022, + 0.6016063690185547, + -0.022166937589645386, + 0.8738051652908325, + 0.5042930245399475, + -0.4785439372062683, + -0.39346086978912354, + 0.9362626671791077, + 0.687313437461853, + 0.2061292827129364, + -0.8717151284217834, + -0.2886638343334198 + ], + [ + 0.7763317823410034, + -0.18735788762569427, + -0.49614909291267395, + 0.1925407350063324, + 0.45760008692741394, + -2.593526840209961, + -1.2330846786499023, + -0.9266390800476074, + -0.04245000332593918, + 0.7759215831756592, + 0.6631491780281067, + -0.09900155663490295, + 0.869513750076294, + 0.6292188167572021, + -0.44907987117767334, + -0.465933233499527, + 0.9126246571540833, + 0.6969174742698669, + 0.19138744473457336, + -0.9333896636962891, + -0.2863849997520447 + ], + [ + 0.9015936851501465, + -0.24430496990680695, + -0.47113338112831116, + 0.26044490933418274, + 0.6222957372665405, + -1.9738842248916626, + -1.7597627639770508, + -0.5862915515899658, + 0.05656706541776657, + 0.7552425265312195, + 0.3856826424598694, + 0.1264311671257019, + 1.013861060142517, + 0.677474856376648, + -0.594429612159729, + -0.3007287085056305, + 0.5179783701896667, + 0.7459592223167419, + 0.47634759545326233, + -0.7555156946182251, + -0.5965601205825806 + ], + [ + 0.8412076830863953, + -0.14837463200092316, + -0.5811554193496704, + 0.550194263458252, + 0.07614865899085999, + -2.2806615829467773, + -1.4382452964782715, + -0.7184593677520752, + 0.2265947163105011, + 1.004583477973938, + 0.593015193939209, + -0.2310028076171875, + 0.7951648235321045, + 0.6527782678604126, + -0.9342862963676453, + -0.1985575556755066, + 0.900602400302887, + 0.8346605896949768, + 0.1322655975818634, + -0.5390253067016602, + -0.08271494507789612 + ], + [ + 0.6555678844451904, + -0.3034543991088867, + -0.32602766156196594, + 0.2733928859233856, + 0.41943785548210144, + -2.308108329772949, + -1.034306287765503, + -0.8407673835754395, + -0.06308501958847046, + 0.7547457218170166, + 0.611559271812439, + -0.22087693214416504, + 0.7729818820953369, + 0.7875890731811523, + -0.4229137897491455, + -0.3362942934036255, + 0.7964543700218201, + 0.6681042313575745, + 0.1465258151292801, + -0.9407565593719482, + -0.21448808908462524 + ] + ], + "generated_utilities": [ + 0.03273266553878784, + 0.03373098373413086, + 0.04705389216542244, + 0.05021713301539421, + 0.04120592400431633, + 0.09449902921915054, + 0.051247138530015945, + 0.04602358862757683 + ], + "group_id": "ms-7ea98b8f4d2220347ee0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18600693345069885, + 0.07475636899471283, + 0.27452313899993896, + 0.20920448005199432, + 0.18411576747894287, + 0.27989473938941956, + 0.19416475296020508, + 0.10948207229375839, + 0.123555026948452, + 0.1468026340007782, + 0.12693151831626892, + 0.11466284841299057, + 0.07585643231868744, + 0.12908481061458588, + 0.22501108050346375, + 0.06431269645690918 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.06974434852600098, + -0.02351950854063034, + 0.04567508399486542, + -0.07042121887207031, + -0.038941264152526855, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.08617860078811646, + 0.9112892150878906, + 0.7374391555786133, + 0.06539928913116455, + 0.03787863254547119, + 0.0, + -0.021528124809265137, + -0.04566385596990585, + -0.03331831097602844, + 1.6466763019561768, + 0.04444807767868042, + -0.27494049072265625, + 0.0 + ], + [ + 1.1160619258880615, + -0.10018755495548248, + 0.19327251613140106, + -1.3990073204040527, + 0.672177255153656, + -3.303851842880249, + -2.832740545272827, + -1.902563214302063, + 0.39503830671310425, + 1.542332410812378, + 1.6850805282592773, + 0.6842766404151917, + 1.1400834321975708, + 1.670598030090332, + 0.631821870803833, + -0.26601675152778625, + -0.2725638151168823, + 3.003178119659424, + 0.23009130358695984, + -1.0388810634613037, + -0.19119825959205627 + ], + [ + 1.031497836112976, + -0.2943698763847351, + -0.11846622079610825, + -1.3361037969589233, + 0.6225074529647827, + -3.545761823654175, + -2.5078811645507812, + -1.4024183750152588, + 0.6736462116241455, + 2.093475580215454, + 1.9168713092803955, + 0.6377919316291809, + 0.5105260014533997, + 1.8172619342803955, + 1.1300766468048096, + 0.1616329848766327, + 0.03408849239349365, + 2.2435190677642822, + 0.735376238822937, + -1.5760226249694824, + -0.7514586448669434 + ], + [ + 1.7542359828948975, + -0.7351009249687195, + -0.854544997215271, + -1.223644495010376, + 0.07327532768249512, + -3.150663137435913, + -2.5486767292022705, + -2.110039472579956, + 1.0206904411315918, + 2.1492340564727783, + 2.187993288040161, + 1.2547765970230103, + 0.8607165217399597, + 1.1150517463684082, + 0.9828771948814392, + 0.5030426383018494, + -0.4691219627857208, + 2.2485060691833496, + -0.14985057711601257, + -1.9480454921722412, + -0.05157876014709473 + ], + [ + 0.6040669679641724, + 0.5665496587753296, + 0.24913550913333893, + -1.6600215435028076, + -0.10813313722610474, + -3.8031911849975586, + -3.226392984390259, + -1.5098689794540405, + 1.0684192180633545, + 2.1557064056396484, + 1.545849084854126, + 1.011389970779419, + 1.113816499710083, + 0.9937302470207214, + 1.372757911682129, + -0.4055754244327545, + -0.23403912782669067, + 3.0650274753570557, + 0.1330145299434662, + -0.7907523512840271, + -0.6428619623184204 + ], + [ + 0.30788952112197876, + 0.6807127594947815, + -0.035938769578933716, + -0.8788743615150452, + -0.30191969871520996, + -4.111589431762695, + -2.432647705078125, + -1.7479791641235352, + -0.321926474571228, + 1.9823087453842163, + 1.6617395877838135, + 1.3319180011749268, + 1.5937504768371582, + 2.0502309799194336, + 0.63725745677948, + 0.7232187390327454, + -0.23885302245616913, + 1.8472312688827515, + 0.3300834596157074, + -0.43588966131210327, + -1.2869892120361328 + ], + [ + 0.3636443018913269, + -0.4284004867076874, + 0.2675324082374573, + -0.4861376881599426, + 0.2239738255739212, + -2.813422918319702, + -2.4891929626464844, + -1.165071725845337, + 0.6442201733589172, + 1.1830427646636963, + 1.177597999572754, + 0.33577069640159607, + 0.6362329721450806, + 1.5409071445465088, + 0.29718804359436035, + -0.5603961944580078, + -0.8089720010757446, + 2.6271419525146484, + -0.5456377267837524, + -0.2824941873550415, + -0.7960987091064453 + ], + [ + 1.1001086235046387, + -0.9762114882469177, + -0.00277864933013916, + -1.704659104347229, + 1.2373628616333008, + -3.385840654373169, + -1.6910454034805298, + -1.470743179321289, + 0.9354362487792969, + 1.9378679990768433, + 1.515415906906128, + -0.3002503514289856, + 0.21796274185180664, + 1.542279839515686, + 1.7353785037994385, + 0.23501873016357422, + -0.16601301729679108, + 2.1989498138427734, + 0.8671687841415405, + -1.6252647638320923, + -1.2141560316085815 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06974434852600098, + -0.15391214191913605, + -0.22159501910209656, + -0.07042121887207031, + -0.09706336259841919, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.1400253176689148, + 0.9112892150878906, + 0.7374391555786133, + 0.13415807485580444, + -0.10778343677520752, + 0.0, + -0.08470147848129272, + 0.1231045201420784, + 0.08352774381637573, + 1.6466763019561768, + -0.005736887454986572, + -0.27494049072265625, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.125826835632324, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.152101516723633, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.8942345976829529 + ] + ], + "predicted_scores": [ + 6.813526153564453, + 5.844024658203125, + 5.787663459777832, + 5.626963138580322, + 5.517650127410889, + 5.478634834289551, + 5.153843402862549, + 5.113950729370117 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train1", + "selected_index": 0, + "state_hash": "a2b9f3b3ea6458c6734081cea861cf54026796a0a594bf9be3657d28f12e9b3c", + "stored_base_utility": 0.18600693345069885, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18213583528995514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18213583528995514 + }, + "base_progress": 0.18213583528995514, + "base_success": false, + "base_utility": 0.18213583528995514, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0009908875217661262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009908875217661262 + }, + { + "contact_quality": null, + "progress": 0.0009231368894688785, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009231368894688785 + }, + { + "contact_quality": null, + "progress": 0.0015073418617248535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0015073418617248535 + }, + { + "contact_quality": null, + "progress": 0.0008463859558105469, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008463859558105469 + }, + { + "contact_quality": null, + "progress": 0.0013398727169260383, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013398727169260383 + }, + { + "contact_quality": null, + "progress": 0.003711601020768285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003711601020768285 + }, + { + "contact_quality": null, + "progress": 0.0017419258365407586, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0017419258365407586 + }, + { + "contact_quality": null, + "progress": 0.0007060368661768734, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007060368661768734 + } + ], + "candidate_progress": [ + 0.0009908875217661262, + 0.0009231368894688785, + 0.0015073418617248535, + 0.0008463859558105469, + 0.0013398727169260383, + 0.003711601020768285, + 0.0017419258365407586, + 0.0007060368661768734 + ], + "candidate_source_chart_ids": [ + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3", + "ms-02311c7783bc67e188b4", + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7f57045c114ddad153b5", + "generated_tangents": [ + [ + -0.2520824670791626, + -0.12511616945266724, + 1.573218584060669, + -0.14429140090942383, + 1.4808824062347412, + 2.862978458404541, + 1.6637825965881348, + -0.03156566619873047, + 0.019331930205225945, + 1.0417245626449585, + 0.6316964626312256, + 0.9478929042816162, + 1.1161257028579712, + 1.8866373300552368, + -0.7375503182411194, + 0.019532859325408936, + 1.4928796291351318, + -0.3573065996170044, + 1.1581472158432007, + 1.3615963459014893, + 1.4436187744140625 + ], + [ + -0.2731807231903076, + -0.24097032845020294, + 1.4526458978652954, + -0.4595770239830017, + 1.4015209674835205, + 2.5857861042022705, + 1.8386640548706055, + 0.08633071929216385, + -0.1831485480070114, + 1.0925453901290894, + 0.4738825261592865, + 0.8631149530410767, + 1.3541682958602905, + 1.2599204778671265, + -0.7336021661758423, + -0.03448072075843811, + 1.5850486755371094, + -0.1531260758638382, + 1.0067245960235596, + 1.349545955657959, + 1.4664242267608643 + ], + [ + -0.10694728791713715, + -0.3016720414161682, + 1.4750951528549194, + -0.4589652121067047, + 1.6018683910369873, + 2.2743873596191406, + 2.048210859298706, + 0.18516378104686737, + -0.16088849306106567, + 0.9327735900878906, + 0.24240219593048096, + 0.8371343612670898, + 1.4592478275299072, + 2.012094736099243, + -0.5596979260444641, + 0.09212283790111542, + 1.5698059797286987, + -0.07501949369907379, + 1.0781712532043457, + 1.850982904434204, + 1.4062294960021973 + ], + [ + -0.1476268321275711, + -0.22388184070587158, + 1.3541926145553589, + -0.2221907526254654, + 1.7256944179534912, + 2.4829025268554688, + 1.8799159526824951, + 0.05385105311870575, + -0.2196202278137207, + 1.1014631986618042, + 0.4526349902153015, + 0.8024947047233582, + 1.130302906036377, + 1.5131126642227173, + -0.8158919215202332, + 0.031249038875102997, + 1.8092244863510132, + 0.014496967196464539, + 1.0285019874572754, + 1.4472846984863281, + 1.641628623008728 + ], + [ + 0.09493135660886765, + -0.28286826610565186, + 1.3929522037506104, + -0.5253861546516418, + 1.688690185546875, + 2.262230396270752, + 2.0344462394714355, + 0.09672358632087708, + -0.13036280870437622, + 1.1764590740203857, + 0.35523638129234314, + 0.9216479063034058, + 1.0209664106369019, + 1.6644783020019531, + -0.5358803272247314, + 0.0413481704890728, + 1.5671550035476685, + -0.1557922512292862, + 1.1109559535980225, + 1.4154980182647705, + 1.7753100395202637 + ], + [ + -0.30561399459838867, + -0.2675154209136963, + 1.2992031574249268, + -0.46828600764274597, + 1.1049857139587402, + 2.2260189056396484, + 1.5147428512573242, + 0.4077915549278259, + -0.11710146069526672, + 0.6827633380889893, + 0.5014147162437439, + 0.5580408573150635, + 1.3217647075653076, + 0.8362540602684021, + -0.33639612793922424, + -0.002032805234193802, + 1.166864037513733, + -0.2901707887649536, + 0.7729716897010803, + 1.439924955368042, + 1.058825969696045 + ], + [ + -0.0660819411277771, + -0.2754341959953308, + 1.2881159782409668, + -0.43712514638900757, + 1.6505377292633057, + 2.328268051147461, + 2.0120089054107666, + 0.24228665232658386, + -0.08295539766550064, + 1.0956268310546875, + 0.14585113525390625, + 0.9994098544120789, + 1.4498875141143799, + 1.777565836906433, + -0.49545717239379883, + 0.1572200208902359, + 1.5885310173034668, + -0.29035139083862305, + 1.238010287284851, + 1.7241621017456055, + 1.612553358078003 + ], + [ + -0.1133287101984024, + -0.18578703701496124, + 1.5263078212738037, + -0.3058190941810608, + 1.4516069889068604, + 2.3268725872039795, + 1.9249842166900635, + -0.07982030510902405, + -0.13312064111232758, + 1.1845663785934448, + 0.624777615070343, + 0.9806438088417053, + 1.4735302925109863, + 1.2689952850341797, + -0.848861575126648, + -0.012123910710215569, + 1.6709089279174805, + -0.17237071692943573, + 1.0478421449661255, + 1.3791472911834717, + 1.4720901250839233 + ] + ], + "generated_utilities": [ + 0.0009908875217661262, + 0.0009231368894688785, + 0.0015073418617248535, + 0.0008463859558105469, + 0.0013398727169260383, + 0.003711601020768285, + 0.0017419258365407586, + 0.0007060368661768734 + ], + "group_id": "ms-7f57045c114ddad153b5", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.11277659982442856, + 0.1479199379682541, + 0.07541074603796005, + 0.042439721524715424, + 0.15517297387123108, + 0.03584214299917221, + 0.1660110056400299, + 0.12549421191215515, + 0.227622389793396, + 0.09924932569265366, + 0.18198737502098083, + 0.11130931228399277, + 0.10854844748973846, + 0.01617596670985222, + 0.08437560498714447, + 0.04182587191462517 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.013166621327400208, + -0.12399804592132568, + 0.48011910915374756, + 0.4925293028354645, + 0.6256281137466431, + 2.213102102279663, + 0.0, + -0.4044480323791504, + 0.1722363382577896, + 0.3389366865158081, + -0.08761294186115265, + 0.21791136264801025, + 0.9396321773529053, + 0.0, + 0.06717276573181152, + -0.5456919074058533, + 0.722014307975769, + -0.051606684923172, + 0.7784175276756287, + 0.5338386297225952, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.9143741130828857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.455221176147461, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.263336420059204 + ], + [ + -0.47406601905822754, + -0.37954479455947876, + 1.123671293258667, + -0.8903511762619019, + 1.5515481233596802, + 3.7488584518432617, + 0.9699987173080444, + -0.23581115901470184, + -0.2149372398853302, + 0.8883978128433228, + 0.5803861021995544, + 1.7047908306121826, + 2.149672746658325, + 2.2741973400115967, + 0.2513730227947235, + 0.6982267498970032, + 2.1354174613952637, + -1.151021122932434, + 1.9550246000289917, + 0.5981679558753967, + 1.338004469871521 + ], + [ + -0.1836935132741928, + -0.42300617694854736, + 1.7664438486099243, + -0.5576877593994141, + 1.0626983642578125, + 2.5276265144348145, + 2.3329849243164062, + -0.4834023118019104, + -0.25722599029541016, + 0.5209662914276123, + 0.24240870773792267, + 0.8533134460449219, + 1.5531291961669922, + 2.304234504699707, + 0.7609332799911499, + 1.0344312191009521, + 2.270921230316162, + -0.926997184753418, + 0.9014018177986145, + 2.259265422821045, + 2.172292470932007 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3910996615886688, + -0.9467402100563049, + 1.4744372367858887, + -0.2748112082481384, + 1.753147840499878, + 3.3321292400360107, + 1.7170560359954834, + 0.09920699894428253, + -0.11250317096710205, + 1.5213767290115356, + 0.0891318991780281, + 1.0193004608154297, + 1.7396382093429565, + 2.202378988265991, + 0.1176290512084961, + 0.5433868765830994, + 1.6113250255584717, + -0.2701912522315979, + 1.7789868116378784, + 1.246854305267334, + 1.6763029098510742 + ], + [ + 0.3669140636920929, + -0.8853410482406616, + 1.1054010391235352, + -0.008406088687479496, + 1.683157205581665, + 2.8090970516204834, + 2.1524839401245117, + -0.1140800267457962, + -0.16068093478679657, + 1.3338439464569092, + -0.34659242630004883, + 1.4494925737380981, + 2.0485284328460693, + 2.244007110595703, + 0.4862309694290161, + 0.6211890578269958, + 1.5708913803100586, + -0.22552455961704254, + 1.7964838743209839, + 1.386763095855713, + 1.3543281555175781 + ], + [ + 0.7718826532363892, + -0.5710152983665466, + 2.151604175567627, + -0.5648711323738098, + 2.1273765563964844, + 3.657623291015625, + 1.9845168590545654, + -0.1599704772233963, + -0.6097310781478882, + 1.135812520980835, + -0.09587362408638, + 1.555975079536438, + 2.3309824466705322, + 1.9832960367202759, + 0.46482378244400024, + 0.6467846035957336, + 1.7182108163833618, + -0.0750286877155304, + 1.9485927820205688, + 0.959426760673523, + 0.9474185109138489 + ] + ], + "predicted_scores": [ + 5.310606956481934, + 5.179131507873535, + 5.123929977416992, + 4.9957756996154785, + 4.984136581420898, + 4.951163291931152, + 4.9353485107421875, + 4.836304664611816 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524/train1", + "selected_index": 0, + "state_hash": "4885f9acf2601b3d2c201bc8e00ef617845ad2a6d08f96d585249345f457be89", + "stored_base_utility": 0.11277659982442856, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0056487321853637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056487321853637695 + }, + "base_progress": 0.0056487321853637695, + "base_success": false, + "base_utility": 0.0056487321853637695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.020783662796020508, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020783662796020508 + }, + { + "contact_quality": null, + "progress": 0.006764829158782959, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006764829158782959 + }, + { + "contact_quality": null, + "progress": 0.007911324501037598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007911324501037598 + }, + { + "contact_quality": null, + "progress": 0.008485655300319195, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008485655300319195 + }, + { + "contact_quality": null, + "progress": 0.012919445522129536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012919445522129536 + }, + { + "contact_quality": null, + "progress": 0.010243098251521587, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010243098251521587 + }, + { + "contact_quality": null, + "progress": 0.005925814155489206, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005925814155489206 + }, + { + "contact_quality": null, + "progress": 0.011702279560267925, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011702279560267925 + } + ], + "candidate_progress": [ + 0.020783662796020508, + 0.006764829158782959, + 0.007911324501037598, + 0.008485655300319195, + 0.012919445522129536, + 0.010243098251521587, + 0.005925814155489206, + 0.011702279560267925 + ], + "candidate_source_chart_ids": [ + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "generated_tangents": [ + [ + -0.9797090291976929, + -0.5788966417312622, + 0.5761638879776001, + -0.4251173734664917, + 0.8595467209815979, + 2.4331881999969482, + 2.6097702980041504, + 0.20743997395038605, + -0.1089591458439827, + 0.5487211346626282, + -0.6175524592399597, + 0.2773182690143585, + 1.6332672834396362, + 0.7114748954772949, + 0.8582735061645508, + -0.3945067524909973, + -1.4017373323440552, + 0.9111741185188293, + 0.6223698854446411, + -0.056299954652786255, + -0.10510754585266113 + ], + [ + -0.6735737323760986, + -0.5392934679985046, + 0.7878432273864746, + -0.6181210279464722, + 0.8703170418739319, + 2.469273805618286, + 2.7541191577911377, + 0.2942640483379364, + 0.007552893366664648, + 0.7389305830001831, + -0.5672551989555359, + 0.3732055425643921, + 1.6452428102493286, + 0.7301114201545715, + 0.8220322728157043, + -0.3352580666542053, + -1.3437676429748535, + 0.9306057691574097, + 0.5657291412353516, + -0.26815271377563477, + -0.3904983699321747 + ], + [ + -0.5472486615180969, + -0.5470925569534302, + 1.0276633501052856, + -0.4575492739677429, + 0.9992425441741943, + 2.3985705375671387, + 3.057952404022217, + 0.2488958090543747, + -0.049797892570495605, + 0.5111393332481384, + -0.6462520956993103, + 0.3110223114490509, + 1.320737361907959, + 1.1416326761245728, + 0.7001436948776245, + -0.3477555811405182, + -1.1263864040374756, + 0.8989945650100708, + 0.8363519906997681, + -0.2654411494731903, + -1.2923177480697632 + ], + [ + -0.6852919459342957, + -0.6334052085876465, + 0.8921171426773071, + -0.6946732997894287, + 0.9066221714019775, + 2.20363712310791, + 3.036403179168701, + 0.3311169147491455, + -0.05185820162296295, + 0.6375935077667236, + -0.44766172766685486, + 0.4774964153766632, + 1.9750186204910278, + 0.5735405087471008, + 0.7067320942878723, + -0.3437744081020355, + -1.3345177173614502, + 0.8593479990959167, + 0.49963071942329407, + 0.052784264087677, + -0.4160422384738922 + ], + [ + -0.7206347584724426, + -0.3388293981552124, + 1.3264834880828857, + -1.1894015073776245, + 1.3139410018920898, + 2.424651861190796, + 2.700961112976074, + 0.4642212390899658, + -0.3942778408527374, + 0.03071877360343933, + -0.7663148641586304, + 0.41479724645614624, + 1.3833167552947998, + 0.9138265252113342, + 0.7195596694946289, + -0.3014485538005829, + -1.6018716096878052, + 1.0623793601989746, + 0.7466589212417603, + -0.04536920785903931, + -1.7899843454360962 + ], + [ + -0.6311175227165222, + -0.35981082916259766, + 1.4574148654937744, + -1.164613127708435, + 1.1249423027038574, + 2.357187509536743, + 2.628722906112671, + 0.5517063736915588, + -0.33276084065437317, + 0.3055139183998108, + -0.6316373944282532, + 0.2753061056137085, + 1.4839552640914917, + 0.6826318502426147, + 0.8178143501281738, + -0.25516846776008606, + -1.5381749868392944, + 0.9670345187187195, + 0.7789851427078247, + -0.31182414293289185, + -1.4498058557510376 + ], + [ + -0.5135475397109985, + -0.21973976492881775, + 1.2926387786865234, + -0.8528269529342651, + 1.0104479789733887, + 2.3066048622131348, + 2.498617172241211, + 0.5369632840156555, + -0.23872369527816772, + 0.3226558268070221, + -0.34781765937805176, + 0.4087653160095215, + 1.2539739608764648, + 1.2790660858154297, + 0.9861194491386414, + -0.2710059583187103, + -1.2331801652908325, + 0.7592136263847351, + 0.9374886751174927, + -0.45170706510543823, + -0.9161456823348999 + ], + [ + -0.8477624654769897, + -0.25506559014320374, + 1.6048439741134644, + -0.9381506443023682, + 1.2483799457550049, + 2.365689754486084, + 2.4276623725891113, + 0.45283880829811096, + -0.4671296179294586, + 0.06111642345786095, + -0.6230047941207886, + 0.23739725351333618, + 1.2195711135864258, + 1.1104482412338257, + 1.0702121257781982, + -0.44958576560020447, + -1.368348479270935, + 0.8212172389030457, + 0.6473431587219238, + -0.1580667793750763, + -1.0589468479156494 + ] + ], + "generated_utilities": [ + 0.020783662796020508, + 0.006764829158782959, + 0.007911324501037598, + 0.008485655300319195, + 0.012919445522129536, + 0.010243098251521587, + 0.005925814155489206, + 0.011702279560267925 + ], + "group_id": "ms-81f4e8858b2ba90aea2e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.01361372135579586, + 0.007307033054530621, + 0.021326780319213867, + 0.04093001410365105, + 0.15059307217597961, + 0.013598322868347168, + 0.09240063279867172, + 0.1631626933813095, + 0.05027441307902336, + 0.05712141841650009, + 0.1538398563861847, + 0.06023097038269043, + 0.020811915397644043, + 0.019836824387311935, + 0.1318017691373825, + 0.051341354846954346 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06500570476055145, + 0.046472612768411636, + 1.2439939975738525, + -1.446747064590454, + 1.7077040672302246, + 2.291205883026123, + 2.1016674041748047, + -0.6354803442955017, + -0.4465000331401825, + 0.9666666984558105, + -1.1514010429382324, + 0.45639365911483765, + 1.613925814628601, + 1.0568119287490845, + 1.3792463541030884, + -0.7852597832679749, + -1.576931118965149, + 1.1258286237716675, + 0.9614027738571167, + -0.9620904922485352, + -1.7528356313705444 + ], + [ + -0.23400194942951202, + -0.21453839540481567, + 1.458961844444275, + -1.701543927192688, + 1.4158955812454224, + 2.2648162841796875, + 2.2270467281341553, + -0.5239131450653076, + -0.44378146529197693, + 1.0904332399368286, + -1.223331093788147, + 0.45784294605255127, + 1.6057316064834595, + 0.9251953363418579, + 1.308751106262207, + -0.6351475715637207, + -1.5950077772140503, + 1.0428481101989746, + 0.6781296730041504, + -0.9007717370986938, + -1.880659818649292 + ], + [ + 0.2324782907962799, + 0.05129373073577881, + 1.1239540576934814, + -1.1853233575820923, + 1.3290393352508545, + 1.933064341545105, + 2.192976951599121, + -0.23718085885047913, + -0.279358446598053, + 1.3028839826583862, + -1.4676742553710938, + 0.16475489735603333, + 1.67984938621521, + 0.6600161790847778, + 1.5654886960983276, + -1.1545368432998657, + -1.5593464374542236, + 1.1976646184921265, + 0.5715436935424805, + -0.9047394394874573, + -1.9614591598510742 + ], + [ + 0.7452290058135986, + 0.614987313747406, + 0.7383754849433899, + -1.5856380462646484, + 1.264894723892212, + 1.7337512969970703, + 2.502504825592041, + 0.040616750717163086, + -1.1382783651351929, + 0.7095704674720764, + -1.014717698097229, + 0.5126742720603943, + 1.3401073217391968, + 1.4275505542755127, + 1.4739655256271362, + -0.7308655381202698, + -0.9416123032569885, + 0.661049485206604, + 0.9453439712524414, + -1.5968079566955566, + -1.715583324432373 + ], + [ + 0.15219944715499878, + 0.5165310502052307, + 1.0654370784759521, + -0.5756344199180603, + 1.2834129333496094, + 1.6370232105255127, + 2.414426326751709, + -1.1256691217422485, + -1.1171015501022339, + 0.5337101817131042, + -0.9850769639015198, + -0.35428309440612793, + 2.4350547790527344, + 1.8563504219055176, + 2.1929399967193604, + -0.1821107268333435, + -1.4600040912628174, + 1.5430631637573242, + 1.4937412738800049, + -0.8152608275413513, + -2.020042896270752 + ] + ], + "predicted_scores": [ + 7.898220062255859, + 7.632265567779541, + 7.578484535217285, + 7.495541572570801, + 7.006176471710205, + 6.1332879066467285, + 5.817028522491455, + 5.645639896392822 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train1", + "selected_index": 0, + "state_hash": "562e8bc3804d1d310e0eaf612dbc110b8efc94514458dacf551ef33f4f0b8ee7", + "stored_base_utility": 0.01361372135579586, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.00698717450723052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00698717450723052 + }, + "base_progress": 0.00698717450723052, + "base_success": false, + "base_utility": 0.00698717450723052, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06249414011836052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06249414011836052 + }, + { + "contact_quality": null, + "progress": 0.05666528269648552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05666528269648552 + }, + { + "contact_quality": null, + "progress": 0.06223900988698006, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06223900988698006 + }, + { + "contact_quality": null, + "progress": 0.0536121129989624, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0536121129989624 + }, + { + "contact_quality": null, + "progress": 0.07487839460372925, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07487839460372925 + }, + { + "contact_quality": null, + "progress": 0.058169882744550705, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058169882744550705 + }, + { + "contact_quality": null, + "progress": 0.08651334047317505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08651334047317505 + }, + { + "contact_quality": null, + "progress": 0.052170634269714355, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.052170634269714355 + } + ], + "candidate_progress": [ + 0.06249414011836052, + 0.05666528269648552, + 0.06223900988698006, + 0.0536121129989624, + 0.07487839460372925, + 0.058169882744550705, + 0.08651334047317505, + 0.052170634269714355 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-48178f52a96de3d1ead0", + "ms-45fbd749d5ed3fe37c13", + "ms-9229ce141dd41a11fac1", + "ms-9229ce141dd41a11fac1" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8a72b73ac452bc190aeb", + "generated_tangents": [ + [ + 0.7237734794616699, + -0.3463215231895447, + -0.2813967168331146, + -0.7398946285247803, + 0.4126674234867096, + -2.4756922721862793, + -1.0576188564300537, + 0.20339497923851013, + -0.15946611762046814, + 0.041883088648319244, + 0.3290709853172302, + 0.38147589564323425, + 0.6676110029220581, + -0.22153611481189728, + -0.5712802410125732, + -0.49059584736824036, + 0.556411623954773, + 0.8972428441047668, + 0.23936396837234497, + -0.05750294029712677, + 0.5669397115707397 + ], + [ + 0.5318090915679932, + -0.2204359918832779, + -0.23124584555625916, + -0.6424493789672852, + 0.3152037262916565, + -2.814680337905884, + -0.6020680069923401, + 0.3010162115097046, + -0.2979036867618561, + -0.022432439029216766, + 0.6397415399551392, + -0.01696258783340454, + 0.44693610072135925, + 0.908657968044281, + -0.3844669461250305, + -0.6219145655632019, + 0.6164681315422058, + 0.609011709690094, + 0.45481422543525696, + -0.10028764605522156, + -0.10990327596664429 + ], + [ + 0.4268692135810852, + -0.1830245703458786, + -0.28259745240211487, + -0.6784018278121948, + 0.41127753257751465, + -2.7967660427093506, + -0.5581141114234924, + 0.29672208428382874, + -0.3161696493625641, + -0.02425564080476761, + 0.6270208954811096, + -0.03540460765361786, + 0.4001066982746124, + 0.8597201704978943, + -0.3618530333042145, + -0.6466072201728821, + 0.4462053179740906, + 0.5887701511383057, + 0.4716850519180298, + -0.07090075314044952, + -0.05993114411830902 + ], + [ + 0.42195379734039307, + -0.17336474359035492, + -0.1927911937236786, + -0.6151036024093628, + 0.32705777883529663, + -2.8485512733459473, + -0.4864002466201782, + 0.3123778998851776, + -0.3475083112716675, + -0.043378494679927826, + 0.5556626915931702, + -0.04166272282600403, + 0.49940237402915955, + 0.845285952091217, + -0.3649314343929291, + -0.6653127670288086, + 0.5573483109474182, + 0.6050751209259033, + 0.47031983733177185, + -0.06376247107982635, + -0.06030736863613129 + ], + [ + 0.8675592541694641, + -0.17233529686927795, + -0.1716870367527008, + -0.622383177280426, + 0.4599394202232361, + -2.6053483486175537, + -0.8403750061988831, + 0.08160973340272903, + -0.17525219917297363, + -0.15138384699821472, + 0.9165785312652588, + 0.2648965120315552, + 0.340472549200058, + 0.5224915146827698, + -0.08234900236129761, + -0.4862605929374695, + 0.8968313932418823, + 0.4751356244087219, + 0.7731359004974365, + 0.3415035009384155, + -0.01751391589641571 + ], + [ + 0.22726790606975555, + -0.18214154243469238, + 0.15191107988357544, + -1.019789218902588, + 0.2853069603443146, + -3.1014511585235596, + -0.4352756142616272, + 0.30375105142593384, + -0.2323322594165802, + -0.2173788845539093, + 0.534632682800293, + 0.16938892006874084, + -0.14465510845184326, + -0.04447142779827118, + -0.24358361959457397, + -0.513020396232605, + 0.3160269856452942, + 0.4808957576751709, + 0.6584339141845703, + -0.32977449893951416, + -0.24890568852424622 + ], + [ + 0.6190431118011475, + -0.3089068531990051, + -0.32904091477394104, + -0.6154882907867432, + 0.42806077003479004, + -2.5397346019744873, + -1.0147783756256104, + 0.12143614888191223, + -0.2780408561229706, + -0.153457909822464, + 0.8323661088943481, + 0.2215633988380432, + 0.37340179085731506, + 0.4195871949195862, + -0.7584972381591797, + -0.7267059087753296, + 0.5783241987228394, + 0.7702854871749878, + 0.2830463647842407, + -0.3344297707080841, + 0.25418585538864136 + ], + [ + 0.5480592250823975, + -0.31458598375320435, + -0.47533008456230164, + -0.6371848583221436, + 0.6769095063209534, + -2.691488742828369, + -1.0633811950683594, + 0.1997028887271881, + -0.29389750957489014, + -0.10647069662809372, + 0.8385688066482544, + 0.32284706830978394, + 0.4793175756931305, + 0.4523414969444275, + -0.7055636048316956, + -0.7248621582984924, + 0.5520171523094177, + 0.8804596662521362, + 0.24362146854400635, + -0.1687656044960022, + 0.17645736038684845 + ] + ], + "generated_utilities": [ + 0.06249414011836052, + 0.05666528269648552, + 0.06223900988698006, + 0.0536121129989624, + 0.07487839460372925, + 0.058169882744550705, + 0.08651334047317505, + 0.052170634269714355 + ], + "group_id": "ms-8a72b73ac452bc190aeb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.02298194169998169, + 0.01763375662267208, + 0.009361605159938335, + 0.02526181936264038, + 0.09214292466640472, + 0.022978028282523155, + 0.07704488933086395, + 0.056830525398254395, + 0.045355141162872314, + 0.06095697730779648, + 0.10329639911651611, + 0.05954315513372421, + 0.03003152273595333, + 0.008136491291224957, + 0.05599620193243027, + 0.011116823181509972 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.0913797616958618, + 0.3305599093437195, + -0.0778055489063263, + -2.223818778991699, + 0.9291321635246277, + -2.905466318130493, + -2.4575929641723633, + 1.1732350587844849, + 0.22933529317378998, + -0.8654288053512573, + 2.1122353076934814, + 1.5987088680267334, + 1.144788146018982, + 0.40604642033576965, + 0.10343833267688751, + -0.7513935565948486, + -1.054006814956665, + 1.396103024482727, + -0.16642586886882782, + -0.04196041822433472, + -1.2630513906478882 + ], + [ + 1.3690472841262817, + 0.2738865315914154, + -0.27599766850471497, + -2.431976556777954, + 0.7025187015533447, + -3.155372142791748, + -2.6898910999298096, + 1.1788215637207031, + 0.19074484705924988, + -0.7378551959991455, + 2.3089826107025146, + 1.3478807210922241, + 1.3458092212677002, + 0.3417958617210388, + 0.24155153334140778, + -1.004831314086914, + -1.3499912023544312, + 1.311987042427063, + -0.25646817684173584, + -0.3180008828639984, + -1.3290650844573975 + ], + [ + 0.9032705426216125, + -0.3011789321899414, + -0.21911907196044922, + -2.884286403656006, + 0.319735586643219, + -2.613290548324585, + -2.175532817840576, + 1.5185720920562744, + -0.1741684526205063, + -0.4879068434238434, + 2.721827507019043, + 2.1753995418548584, + 0.8214534521102905, + 0.7874165177345276, + 0.013686276972293854, + -0.7485443949699402, + -0.764183521270752, + 0.9452257752418518, + 0.07210072875022888, + 0.2788098156452179, + -1.8349425792694092 + ] + ], + "predicted_scores": [ + 4.453597068786621, + 4.2043070793151855, + 4.053958892822266, + 4.031412601470947, + 3.84814715385437, + 3.5510828495025635, + 3.0950734615325928, + 3.057717800140381 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train1", + "selected_index": 0, + "state_hash": "722240dc5e14c5a30badc5fbec9f15ecf388cf6402fb77013c8f9dbdcd100846", + "stored_base_utility": 0.02298194169998169, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08285123109817505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08285123109817505 + }, + "base_progress": 0.08285123109817505, + "base_success": false, + "base_utility": 0.08285123109817505, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.08203518390655518, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08203518390655518 + }, + { + "contact_quality": null, + "progress": 0.09714335203170776, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09714335203170776 + }, + { + "contact_quality": null, + "progress": 0.0648718848824501, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0648718848824501 + }, + { + "contact_quality": null, + "progress": 0.06826772540807724, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06826772540807724 + }, + { + "contact_quality": null, + "progress": 0.03956645727157593, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03956645727157593 + }, + { + "contact_quality": null, + "progress": 0.042625367641448975, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.042625367641448975 + }, + { + "contact_quality": null, + "progress": 0.04983709380030632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04983709380030632 + }, + { + "contact_quality": null, + "progress": 0.05635470151901245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05635470151901245 + } + ], + "candidate_progress": [ + 0.08203518390655518, + 0.09714335203170776, + 0.0648718848824501, + 0.06826772540807724, + 0.03956645727157593, + 0.042625367641448975, + 0.04983709380030632, + 0.05635470151901245 + ], + "candidate_source_chart_ids": [ + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-a96bca1ef0b74f686018", + "generated_tangents": [ + [ + 0.5346310138702393, + 0.10992571711540222, + 0.6128008961677551, + -0.2714889645576477, + 0.3584376573562622, + 0.25554028153419495, + 0.1413051187992096, + 0.11463594436645508, + 0.011281419545412064, + -0.11737047880887985, + -0.9030520915985107, + -0.049557045102119446, + -0.29371047019958496, + -0.6933405995368958, + -0.3945266306400299, + 0.11014603078365326, + 0.2626229524612427, + -0.27674490213394165, + -0.7403897643089294, + -0.3901497721672058, + -0.7334052324295044 + ], + [ + 0.14372357726097107, + 0.04398868978023529, + 0.5261341333389282, + -0.2815788984298706, + 0.6882809400558472, + 0.42881765961647034, + -0.21752256155014038, + 0.15735551714897156, + 0.04994235932826996, + -0.037333764135837555, + -0.8268375396728516, + -0.032546281814575195, + -0.6295778751373291, + -0.45782583951950073, + -0.402691513299942, + 0.07683300971984863, + 0.39632636308670044, + -0.23021282255649567, + -0.7886610627174377, + -0.5787997245788574, + -0.6151546239852905 + ], + [ + 0.16368357837200165, + 0.08141957223415375, + 0.7052232623100281, + -0.5786411762237549, + 0.8425356149673462, + 0.6810296773910522, + 0.21797403693199158, + 0.3181007206439972, + 0.1156037375330925, + 0.12721092998981476, + -0.9957047700881958, + -0.3546638488769531, + -0.9460336565971375, + -1.3903577327728271, + -0.4220317304134369, + 0.15521006286144257, + 0.5781416296958923, + -0.23526884615421295, + -1.3471155166625977, + -1.1188604831695557, + -0.5476807355880737 + ], + [ + 0.22467413544654846, + 0.024278812110424042, + 0.8184769749641418, + -0.648155927658081, + 0.848567008972168, + 0.735680341720581, + 0.06917771697044373, + 0.2127572000026703, + 0.13531038165092468, + 0.08184245228767395, + -1.00473952293396, + -0.3373028635978699, + -0.866546630859375, + -1.5128990411758423, + -0.36625927686691284, + 0.14370355010032654, + 0.5097633004188538, + -0.30661845207214355, + -1.2713730335235596, + -1.129964828491211, + -0.5226327180862427 + ], + [ + 0.1036408394575119, + -0.011204499751329422, + 0.9784748554229736, + -0.3923414349555969, + 1.0034300088882446, + 1.2108299732208252, + 0.0016484856605529785, + 0.03118811547756195, + 0.0231051966547966, + 0.26200950145721436, + -0.9888348579406738, + -0.37069010734558105, + -0.6912773847579956, + -0.6350822448730469, + -0.3739113211631775, + 0.08453267812728882, + 0.1155906394124031, + -0.18932513892650604, + -1.0536084175109863, + -0.9525995850563049, + -0.4120785892009735 + ], + [ + 0.19068212807178497, + -0.002145826816558838, + 1.1100620031356812, + -0.543897807598114, + 0.7654709815979004, + 0.928135871887207, + 0.01588571071624756, + 0.2787608206272125, + 0.03331562876701355, + 0.13259316980838776, + -0.9413942098617554, + -0.4548082947731018, + -0.7601237893104553, + -1.2860647439956665, + -0.5193300247192383, + 0.05155609920620918, + 0.5452077388763428, + -0.17883668839931488, + -1.3284802436828613, + -0.9959831833839417, + -0.5929776430130005 + ], + [ + 0.04121547192335129, + 0.31499946117401123, + 0.8823180198669434, + -0.37555986642837524, + 1.2201275825500488, + 0.557954728603363, + 0.321776807308197, + 0.3943336009979248, + 0.13487793505191803, + 0.1472679078578949, + -1.0082677602767944, + -0.1733517348766327, + -0.7004083395004272, + -1.5310494899749756, + -0.31477823853492737, + -0.015043312683701515, + 0.7494775652885437, + -0.4257533550262451, + -1.37750244140625, + -1.2458014488220215, + -0.699881911277771 + ], + [ + 0.3205135464668274, + -0.0065469890832901, + 0.8198845386505127, + -0.42302414774894714, + 0.42646801471710205, + 0.5846861600875854, + 0.21706122159957886, + 0.26053255796432495, + -0.09887508302927017, + -0.2193397581577301, + -0.8402920961380005, + -0.2699149549007416, + -0.8194273710250854, + -0.502998948097229, + -0.6821227669715881, + -0.03476265072822571, + 0.7032755017280579, + 0.36467689275741577, + -1.1770315170288086, + -1.1577614545822144, + -1.343900442123413 + ] + ], + "generated_utilities": [ + 0.08203518390655518, + 0.09714335203170776, + 0.0648718848824501, + 0.06826772540807724, + 0.03956645727157593, + 0.042625367641448975, + 0.04983709380030632, + 0.05635470151901245 + ], + "group_id": "ms-a96bca1ef0b74f686018", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07923875749111176, + 0.1507580727338791, + 0.09708476066589355, + 0.1572066992521286, + 0.09626718610525131, + 0.0781460627913475, + 0.07252912223339081, + 0.04244875907897949, + 0.026137471199035645, + 0.03892441838979721, + 0.04886416718363762, + 0.10530619323253632, + 0.0477486252784729, + 0.0898323506116867, + 0.01968318223953247, + 0.0289560966193676 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.48996758460998535, + 0.9053895473480225, + 1.8985445499420166, + -1.3594893217086792, + 1.6097791194915771, + 1.132408618927002, + 1.8319320678710938, + 0.46463528275489807, + 0.13208985328674316, + 0.8792686462402344, + -1.868321418762207, + 1.1490275859832764, + -2.0193891525268555, + -2.0637927055358887, + 0.11448347568511963, + 1.2957178354263306, + 1.293219804763794, + -1.7789109945297241, + -0.4312591254711151, + -1.7441130876541138, + -0.06536102294921875 + ], + [ + 0.03654795140028, + 1.7676925659179688, + 1.048339605331421, + -1.366391658782959, + 2.5093846321105957, + 2.160305976867676, + 2.8675124645233154, + -0.309950590133667, + -0.5841768980026245, + 1.3030188083648682, + -1.2228018045425415, + 1.1373907327651978, + -2.831526041030884, + -2.2111129760742188, + -0.14044174551963806, + 1.660491704940796, + 1.5215414762496948, + -1.0137104988098145, + -0.40579986572265625, + -1.9571893215179443, + 1.241256833076477 + ], + [ + -0.2826732099056244, + 2.2520718574523926, + 2.418309450149536, + -1.7572941780090332, + 1.5832278728485107, + 0.6361246705055237, + 2.2009639739990234, + 0.9109495878219604, + 0.22027403116226196, + 0.7075455188751221, + -1.2762477397918701, + 0.8630691766738892, + -1.2986351251602173, + -2.567821979522705, + -0.4690782427787781, + 2.1363346576690674, + 0.1853562593460083, + -1.541505217552185, + -0.033681899309158325, + -1.0161254405975342, + 0.06877446174621582 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03534998744726181, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.3504420518875122, + 0.0, + -0.013733148574829102, + 0.021970629692077637, + 0.014389097690582275, + -0.7432765960693359, + 0.006671249866485596, + -0.9755387306213379, + 0.0, + -0.027152668684720993, + 0.2708500623703003, + 0.052558183670043945, + -0.7615737915039062, + 0.020878344774246216, + -0.2927910089492798, + 0.0 + ], + [ + -0.1400495320558548, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.5714048147201538, + 0.0, + -0.3271818161010742, + -0.04413123428821564, + 0.16598933935165405, + -0.7432765960693359, + 0.2282017469406128, + -0.9755387306213379, + 0.0, + -0.4382812976837158, + 0.2708500623703003, + 0.2094714641571045, + -0.7615737915039062, + 0.23577484488487244, + -0.2927910089492798, + 0.0 + ] + ], + "predicted_scores": [ + 4.269467353820801, + 3.8671813011169434, + 3.827446937561035, + 3.6798555850982666, + 3.2768988609313965, + 2.685136318206787, + 2.673711061477661, + 1.6891785860061646 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train1", + "selected_index": 0, + "state_hash": "61714505b9ac6cbb1c6deff698bb3e96fd54c7cdb8f0d4307bfdd236634525c0", + "stored_base_utility": 0.07923875749111176, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1380254477262497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1380254477262497 + }, + { + "contact_quality": null, + "progress": 0.06922783702611923, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06922783702611923 + }, + { + "contact_quality": null, + "progress": 0.102073073387146, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.102073073387146 + }, + { + "contact_quality": null, + "progress": 0.12990880012512207, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12990880012512207 + }, + { + "contact_quality": null, + "progress": 0.11760101467370987, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11760101467370987 + }, + { + "contact_quality": null, + "progress": 0.11701671034097672, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11701671034097672 + }, + { + "contact_quality": null, + "progress": 0.14365969598293304, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14365969598293304 + }, + { + "contact_quality": null, + "progress": 0.10479718446731567, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10479718446731567 + } + ], + "candidate_progress": [ + 0.1380254477262497, + 0.06922783702611923, + 0.102073073387146, + 0.12990880012512207, + 0.11760101467370987, + 0.11701671034097672, + 0.14365969598293304, + 0.10479718446731567 + ], + "candidate_source_chart_ids": [ + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-ac4fb33f3ab4db853995", + "generated_tangents": [ + [ + 1.8757209777832031, + -0.2798987627029419, + -0.491509348154068, + 0.037609606981277466, + -0.8015756011009216, + 2.0208706855773926, + 1.6965241432189941, + 1.9253427982330322, + 0.4048178791999817, + -1.3228524923324585, + -0.4634227454662323, + -1.3139301538467407, + -2.516993284225464, + -3.13143253326416, + 1.5752646923065186, + 0.16432787477970123, + -0.923065721988678, + 0.17932958900928497, + -0.18268033862113953, + -1.8566339015960693, + -1.7394579648971558 + ], + [ + 1.877700686454773, + -0.27677154541015625, + -0.2079942524433136, + -0.09529922902584076, + -0.356905460357666, + 2.6387462615966797, + 1.9372978210449219, + 2.0541677474975586, + 0.4495166540145874, + -0.9983639717102051, + -0.34587231278419495, + -0.8706223964691162, + -2.940394639968872, + -1.9517396688461304, + 1.815815806388855, + 0.4263441562652588, + -0.6645351648330688, + 0.3294494152069092, + 0.1476365178823471, + -1.818439245223999, + -1.324389934539795 + ], + [ + 1.7460854053497314, + -0.14227360486984253, + -0.44563987851142883, + -0.10550718754529953, + -0.7599748969078064, + 2.783186912536621, + 1.4664199352264404, + 2.082426071166992, + 0.38119378685951233, + -1.1542381048202515, + -0.32885488867759705, + -1.1708210706710815, + -2.741654396057129, + -2.316239833831787, + 1.7502933740615845, + 0.27585113048553467, + -0.762572169303894, + 0.3741384446620941, + 0.0003533363342285156, + -1.8107967376708984, + -1.5808926820755005 + ], + [ + 1.9009313583374023, + -0.11730101704597473, + -0.517284631729126, + 0.05539482831954956, + -0.6659588813781738, + 2.6490137577056885, + 1.847862720489502, + 2.114100933074951, + 0.42314252257347107, + -1.0808172225952148, + -0.45225653052330017, + -1.3092527389526367, + -2.716824769973755, + -2.6231470108032227, + 1.593016266822815, + 0.22196482121944427, + -0.7252891659736633, + 0.31342220306396484, + -0.057871073484420776, + -1.9106976985931396, + -1.616092562675476 + ], + [ + 1.8682342767715454, + -0.1502419114112854, + -0.4085184633731842, + -0.021281033754348755, + -0.5494136214256287, + 2.4912312030792236, + 2.0649428367614746, + 2.020284652709961, + 0.4087589979171753, + -0.9137766361236572, + -0.5366708636283875, + -1.094678282737732, + -2.9595882892608643, + -2.596893310546875, + 1.6457347869873047, + 0.20525941252708435, + -0.6062728762626648, + 0.3353874087333679, + 0.02441948652267456, + -1.6362388134002686, + -1.350596308708191 + ], + [ + 1.9847166538238525, + -0.15845981240272522, + -0.395762175321579, + 0.08717381954193115, + -0.3476361632347107, + 2.264495849609375, + 2.0956692695617676, + 2.025761365890503, + 0.4366755485534668, + -1.0068161487579346, + -0.5347964763641357, + -1.0649518966674805, + -2.9739842414855957, + -2.2935690879821777, + 1.5841917991638184, + 0.1982099413871765, + -0.6307572722434998, + 0.3693506717681885, + -0.1537345051765442, + -1.7523667812347412, + -1.5008445978164673 + ], + [ + 1.9415417909622192, + -0.13299265503883362, + -0.6431517601013184, + 0.1701856553554535, + -0.5841852426528931, + 2.8527026176452637, + 1.545745849609375, + 2.0428261756896973, + 0.34041354060173035, + -1.0155531167984009, + -0.48445674777030945, + -0.9317442178726196, + -2.4904329776763916, + -1.8709954023361206, + 1.6128383874893188, + 0.28925085067749023, + -0.7339987754821777, + 0.13303901255130768, + -0.27472424507141113, + -1.367954969406128, + -1.1774617433547974 + ], + [ + 2.4350316524505615, + -0.1626695990562439, + -0.07594266533851624, + 0.18432152271270752, + -0.015824690461158752, + 1.9474067687988281, + 2.2089436054229736, + 2.021188974380493, + 0.19317109882831573, + -1.33009934425354, + -0.6722767353057861, + -0.8716684579849243, + -2.8951737880706787, + -1.9773768186569214, + 1.5057123899459839, + -0.029739627614617348, + -0.5092992782592773, + 0.4653567671775818, + -0.3313637971878052, + -1.94559645652771, + -1.1394827365875244 + ] + ], + "generated_utilities": [ + 0.1380254477262497, + 0.06922783702611923, + 0.102073073387146, + 0.12990880012512207, + 0.11760101467370987, + 0.11701671034097672, + 0.14365969598293304, + 0.10479718446731567 + ], + "group_id": "ms-ac4fb33f3ab4db853995", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.2651830017566681, + 0.22676818072795868, + 0.22454051673412323, + 2.0, + 0.31214088201522827, + 0.21535459160804749, + 0.18962346017360687, + 2.0, + 0.1830430030822754, + 0.15030252933502197, + 0.12050267308950424, + 0.17022761702537537, + 0.11389490216970444, + 0.07058519124984741, + 0.07076358795166016, + 0.18658748269081116 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 2.0962133407592773, + 0.057441189885139465, + -0.2733909487724304, + 0.4170292317867279, + -0.7772165536880493, + 2.858414888381958, + 2.1098427772521973, + 1.8366810083389282, + 0.017507493495941162, + -1.6127345561981201, + -0.39152991771698, + -1.017486333847046, + -2.815659999847412, + -2.787520408630371, + 1.6748408079147339, + 0.2986273169517517, + -0.6001855134963989, + 0.5475705862045288, + -0.28012242913246155, + -1.5382447242736816, + -1.6652274131774902 + ], + [ + 2.1338062286376953, + -0.07656055688858032, + -0.7531881332397461, + 0.48830646276474, + -0.36628106236457825, + 2.381558418273926, + 1.7315943241119385, + 1.838928461074829, + -0.050897032022476196, + -1.171897292137146, + -0.8350631594657898, + -0.8261303901672363, + -2.2635834217071533, + -3.2190167903900146, + 1.3416705131530762, + 0.44279617071151733, + -0.7299901247024536, + 1.2929750680923462, + -0.0032969415187835693, + -1.5000369548797607, + -1.6060972213745117 + ], + [ + 1.5521352291107178, + 0.03659695386886597, + -0.44382959604263306, + 0.21355122327804565, + -1.0471938848495483, + 3.194620370864868, + 1.9892739057540894, + 2.5084128379821777, + 0.531404972076416, + -0.9168282747268677, + -0.49052953720092773, + -1.1588059663772583, + -3.156189441680908, + -3.278008460998535, + 0.921509325504303, + 0.49181532859802246, + -1.0228296518325806, + 0.5866082310676575, + -0.4882396459579468, + -0.7946261763572693, + -2.119061231613159 + ], + [ + 2.7138938903808594, + -0.3406146764755249, + -0.32174432277679443, + 0.9962238669395447, + -0.558576226234436, + 2.049264907836914, + 1.8703808784484863, + 2.113438367843628, + 0.9615883827209473, + -1.9615708589553833, + -0.0062177181243896484, + -0.5808464288711548, + -2.1758205890655518, + -2.219890594482422, + 1.022050142288208, + 0.7201904654502869, + -1.3177021741867065, + 0.4089192748069763, + -0.0315951406955719, + -0.7215378880500793, + -2.069262981414795 + ], + [ + 2.5701913833618164, + 0.3137266933917999, + -0.5592924356460571, + 0.0026398897171020508, + -0.19283491373062134, + 2.2903807163238525, + 2.637770652770996, + 2.5494186878204346, + 0.9258015155792236, + -0.7146614193916321, + -0.686233639717102, + -0.7985042929649353, + -3.431988000869751, + -3.2178471088409424, + 1.319100260734558, + -0.15109702944755554, + -0.2694774866104126, + 1.0912322998046875, + -0.6380462050437927, + -2.0780839920043945, + -2.1209774017333984 + ], + [ + 2.6195321083068848, + -0.2646539509296417, + 0.15159690380096436, + 0.8749752044677734, + -1.1848887205123901, + 2.8789353370666504, + 1.6037814617156982, + 1.3343384265899658, + -0.2254078984260559, + -2.041776657104492, + -0.9811336994171143, + -1.3441768884658813, + -2.2502636909484863, + -1.9091612100601196, + 0.9959704279899597, + 0.42473238706588745, + -0.5330828428268433, + 1.6945891380310059, + 0.4069111943244934, + -0.5568764805793762, + -1.472819447517395 + ], + [ + 2.9485361576080322, + -0.6907311677932739, + -0.057036399841308594, + 1.2457408905029297, + 0.09488356113433838, + 2.6509125232696533, + 2.0378005504608154, + 2.278454303741455, + 1.1734973192214966, + -1.853393316268921, + 0.09614759683609009, + -0.35674989223480225, + -2.1597909927368164, + -1.9672894477844238, + 1.9760816097259521, + -0.47744297981262207, + 0.021782994270324707, + 1.6328153610229492, + -0.8569514751434326, + -1.0814869403839111, + -1.7780723571777344 + ], + [ + 2.1897926330566406, + -0.3883178234100342, + -0.6492674946784973, + 1.3223958015441895, + 0.30954307317733765, + 2.031817674636841, + 1.878535270690918, + 2.5030624866485596, + 0.6644855737686157, + -1.7382179498672485, + -1.1862120628356934, + -0.7967990636825562, + -2.8425841331481934, + -2.3635308742523193, + 2.0061349868774414, + -0.6015456318855286, + -1.429253339767456, + 1.2773160934448242, + 0.04122643172740936, + -2.0497689247131348, + -1.1791472434997559 + ], + [ + 1.586853265762329, + 0.06395213305950165, + -0.8944701552391052, + 0.5118889808654785, + -1.1310638189315796, + 2.865907669067383, + 1.3295278549194336, + 1.0862370729446411, + 0.6303088665008545, + -0.3754369616508484, + -0.04129624366760254, + -0.7044278979301453, + -2.923245429992676, + -1.7837438583374023, + 1.5932514667510986, + 1.2467985153198242, + 0.13401156663894653, + 1.6706414222717285, + -0.816297709941864, + -1.2893165349960327, + -1.0985345840454102 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.1383795738220215, + -0.1483764797449112, + -0.27178800106048584, + 0.26738643646240234, + -0.12845781445503235, + 1.6495440006256104, + 0.0, + 0.9713280200958252, + -0.050046056509017944, + -0.36543774604797363, + -0.022124826908111572, + 0.08494043350219727, + -1.671510934829712, + 0.0, + 0.5271386504173279, + 0.3898613452911377, + 0.21280205249786377, + -0.2029280662536621, + -0.01802508533000946, + -0.35994601249694824, + 0.0 + ], + [ + 1.7570452690124512, + 0.07137972116470337, + 0.02318662405014038, + 0.2907020151615143, + -0.2308352291584015, + 2.89892578125, + 2.537220001220703, + 2.238605499267578, + 0.16207315027713776, + -1.6996564865112305, + -0.8050198554992676, + -1.2014062404632568, + -2.5764517784118652, + -2.393839120864868, + 1.1276613473892212, + 0.31928181648254395, + -1.036102533340454, + 1.1266506910324097, + -0.1944657564163208, + -1.5998979806900024, + -2.151465892791748 + ] + ], + "predicted_scores": [ + 4.508659362792969, + 4.334898948669434, + 4.305202007293701, + 4.188601493835449, + 4.163142204284668, + 3.8224165439605713, + 3.804182529449463, + 3.057375431060791 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train1", + "selected_index": 0, + "state_hash": "0b62c42c3beda0eb959692d55792514223681b522edd0a5e73247682cf0bfe95", + "stored_base_utility": 0.2651830017566681, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "generated_tangents": [ + [ + 1.5776278972625732, + -0.1062684953212738, + -1.2771344184875488, + -0.49981874227523804, + 0.2987932562828064, + -0.07345153391361237, + -0.3306434750556946, + 0.6023690104484558, + -0.031822457909584045, + -0.06422949582338333, + -0.8920311331748962, + -0.25830867886543274, + -3.039278984069824, + -2.0247509479522705, + 0.04372836649417877, + 0.1492304801940918, + 0.49795687198638916, + -0.8894179463386536, + 0.9733495116233826, + -2.2327277660369873, + -1.1248815059661865 + ], + [ + 1.6623307466506958, + -0.02801577001810074, + -1.2038532495498657, + -0.416922390460968, + -0.03998517990112305, + -0.010470360517501831, + -0.17300570011138916, + 0.550474226474762, + -0.02714109607040882, + -0.04892220348119736, + -0.7027192711830139, + -0.3052824139595032, + -3.2295403480529785, + -2.228353500366211, + -0.1067059189081192, + 0.07792852818965912, + 0.47764676809310913, + -0.8404160141944885, + 0.9186916947364807, + -2.3990609645843506, + -1.2066744565963745 + ], + [ + 1.656821370124817, + -0.0799698457121849, + -1.4930826425552368, + -0.01621711254119873, + -0.22033074498176575, + 0.0406893715262413, + -1.0481154918670654, + 0.41119152307510376, + 0.033923156559467316, + -0.2097930908203125, + -0.40410226583480835, + -0.000824272632598877, + -2.937533378601074, + -2.3680219650268555, + -0.1172158420085907, + 0.029197171330451965, + 0.5653212070465088, + -1.001155138015747, + 1.157625436782837, + -2.386183023452759, + -1.1381199359893799 + ], + [ + 1.6274237632751465, + -0.06096906214952469, + -1.1917328834533691, + -0.09866362065076828, + -0.43721234798431396, + 0.2296428680419922, + -0.4563702344894409, + 0.37857958674430847, + 0.05558808147907257, + 0.02650347352027893, + -0.5547478795051575, + -0.15131524205207825, + -3.2839534282684326, + -2.348494291305542, + -0.15734140574932098, + 0.17000021040439606, + 0.7693645358085632, + -0.7704904675483704, + 0.860443651676178, + -2.102745294570923, + -1.4043538570404053 + ], + [ + 1.6805323362350464, + -0.10195273160934448, + -1.293046236038208, + -0.20656415820121765, + -0.23258602619171143, + 0.06793885678052902, + -0.46606773138046265, + 0.4363333582878113, + -0.003076324239373207, + -0.040726251900196075, + -0.5399624705314636, + -0.11603021621704102, + -3.1387341022491455, + -2.31392240524292, + -0.04789737984538078, + 0.05541032925248146, + 0.5314755439758301, + -0.8912479281425476, + 0.912960946559906, + -2.218932628631592, + -1.2900742292404175 + ], + [ + 1.548425555229187, + -0.29463839530944824, + -1.0629558563232422, + -0.6135776042938232, + -0.1539422869682312, + -0.18624693155288696, + -1.1895885467529297, + 0.4904552102088928, + 0.17714644968509674, + -0.40043675899505615, + -0.40535709261894226, + -0.14600098133087158, + -2.7475781440734863, + -2.0295424461364746, + -0.2928716540336609, + 0.1679936647415161, + 0.9342039823532104, + -0.6481351256370544, + 1.1030620336532593, + -2.437485456466675, + -1.411930799484253 + ], + [ + 1.6877155303955078, + -0.033127088099718094, + -0.8584600687026978, + -0.12578636407852173, + -0.3914439082145691, + -0.6135951280593872, + -0.2536333203315735, + 0.5843752026557922, + 0.07948585599660873, + -0.23855450749397278, + -0.49482861161231995, + -0.3175610303878784, + -2.8263332843780518, + -1.8088093996047974, + -0.07201462984085083, + 0.061782415956258774, + 0.5911457538604736, + -0.8975171446800232, + 0.8092808127403259, + -2.417825937271118, + -1.2020542621612549 + ], + [ + 1.8183302879333496, + -0.07024002820253372, + -1.0422594547271729, + -0.10619249194860458, + -0.06513074040412903, + -0.27954554557800293, + -0.9251890778541565, + 0.3007574677467346, + 0.09522706270217896, + 0.02375529706478119, + -0.23301813006401062, + -0.11444693803787231, + -2.2615537643432617, + -1.7828730344772339, + -0.1611703336238861, + 0.018185578286647797, + 0.6350998282432556, + -0.7584947943687439, + 0.9100536108016968, + -2.5712146759033203, + -1.1600972414016724 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.149173304438591, + 0.06221481412649155, + 0.1385849118232727, + 0.06656398624181747, + 2.0, + 0.10397814214229584, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7398372888565063, + -0.02836926281452179, + -0.4292973279953003, + 0.028730571269989014, + 0.009982943534851074, + -0.3464317321777344, + 0.0, + 0.009800434112548828, + -0.06816069781780243, + 0.030653484165668488, + -0.00554502010345459, + -0.06216755136847496, + -1.9802618026733398, + 0.0, + 0.01717478036880493, + -0.05601567029953003, + 0.010390102863311768, + -0.8072932958602905, + -0.05775028467178345, + -1.9342114925384521, + 0.0 + ], + [ + 0.7398372888565063, + -0.1961192637681961, + -0.4292973279953003, + -0.0017886161804199219, + 0.06461763381958008, + -0.3464317321777344, + 0.0, + 0.42972221970558167, + -0.5797654986381531, + 0.19436834752559662, + -0.23844504356384277, + 0.1733923852443695, + -1.9802618026733398, + 0.0, + -0.27269166707992554, + 0.04145494103431702, + 0.12319430708885193, + -0.8072932958602905, + 0.4949256181716919, + -1.9342114925384521, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7398372888565063, + 0.2242223471403122, + -1.4292973279953003, + -0.537040650844574, + -0.9353823661804199, + -1.3464317321777344, + -2.159137487411499, + 0.489885151386261, + -0.006211067084223032, + 0.09238272905349731, + -1.0055450201034546, + -0.04778099060058594, + -2.98026180267334, + -2.576570987701416, + 0.2992324233055115, + 0.34874510765075684, + 0.5165960192680359, + -1.8072932958602905, + 0.834364652633667, + -2.934211492538452, + -2.0790040493011475 + ], + [ + 1.9740948677062988, + 0.4187958538532257, + -1.2225801944732666, + -0.6112513542175293, + -1.0962133407592773, + -1.4875872135162354, + -2.1883857250213623, + 0.6603437662124634, + -0.20142723619937897, + 0.1697872281074524, + -0.8377580046653748, + -0.16444604098796844, + -2.87237286567688, + -2.4153482913970947, + 0.03898555040359497, + 0.5175161957740784, + 0.7254846096038818, + -1.9598647356033325, + 1.1053451299667358, + -3.172945976257324, + -1.9334450960159302 + ], + [ + 1.5938913822174072, + -0.018363401293754578, + -1.1851862668991089, + -0.20344951748847961, + -0.9226735234260559, + -1.3553601503372192, + -1.932102084159851, + 0.6446930170059204, + 0.30503973364830017, + 0.3522501289844513, + -0.9527953267097473, + -0.15980565547943115, + -2.9574379920959473, + -2.9056527614593506, + 0.6069583892822266, + 0.3722725808620453, + 0.7255810499191284, + -1.8503519296646118, + 0.5613387823104858, + -3.095244884490967, + -2.023747444152832 + ], + [ + 1.6626678705215454, + -0.02834661304950714, + -1.0125243663787842, + -0.5629516839981079, + -1.230878233909607, + -1.414373755455017, + -2.42856764793396, + 0.8520264625549316, + -0.1296539008617401, + -0.22410845756530762, + -1.0539721250534058, + -0.060277245938777924, + -2.740527391433716, + -2.1194698810577393, + 0.6826534867286682, + 0.7535779476165771, + 0.35573869943618774, + -1.9124289751052856, + 0.7101565003395081, + -3.326658010482788, + -1.8195855617523193 + ], + [ + 2.2881288528442383, + 0.6951426863670349, + -1.0978585481643677, + -0.008003771305084229, + -1.0754073858261108, + -1.7734596729278564, + -2.139570474624634, + 0.9205271601676941, + 0.29560548067092896, + 0.49158430099487305, + -1.2606573104858398, + 0.43956148624420166, + -2.831833600997925, + -2.960129976272583, + 0.6832801103591919, + -0.03894898295402527, + 1.036036491394043, + -2.127277135848999, + 0.6759494543075562, + -3.1690073013305664, + -1.9895020723342896 + ], + [ + 1.9020261764526367, + 0.6575945019721985, + -1.3994193077087402, + -0.5040701031684875, + -0.6857678890228271, + -0.7056208252906799, + -1.5344107151031494, + 0.7615087032318115, + 0.6105715036392212, + 0.19082021713256836, + -0.6303516626358032, + -0.4614318609237671, + -2.3375303745269775, + -2.188478469848633, + 0.6886799335479736, + -0.03237125277519226, + 0.483882874250412, + -1.3982734680175781, + 0.23996800184249878, + -3.3969180583953857, + -1.5116117000579834 + ], + [ + 1.666227102279663, + 0.6915197968482971, + -1.6392340660095215, + -0.9650112390518188, + -0.2591104507446289, + -1.5417896509170532, + -1.695828914642334, + 1.130251169204712, + -0.555443525314331, + -0.4872768521308899, + -1.0994479656219482, + 0.5934816002845764, + -3.5218958854675293, + -1.8239338397979736, + 0.7150859832763672, + 0.8003170490264893, + 1.0113730430603027, + -2.1905770301818848, + 0.8595032095909119, + -3.039506673812866, + -2.3675880432128906 + ], + [ + 1.59856116771698, + -0.036071136593818665, + -1.700652837753296, + 0.27602267265319824, + -1.608276128768921, + -0.875938355922699, + -1.6728798151016235, + 0.8851535320281982, + 0.7461313605308533, + -0.7497223019599915, + -1.0928030014038086, + -0.4735495150089264, + -2.223254680633545, + -2.5322465896606445, + 0.044906944036483765, + 0.6160990595817566, + 1.3697642087936401, + -1.6367510557174683, + 0.086475670337677, + -3.323115825653076, + -2.058189630508423 + ], + [ + 2.285855531692505, + 0.6314746141433716, + -1.2119637727737427, + 0.039707839488983154, + -0.9751565456390381, + -1.7796330451965332, + -1.5261995792388916, + 0.5450726747512817, + -0.8933275938034058, + 0.7903657555580139, + -1.5466060638427734, + 0.5994364023208618, + -3.370340585708618, + -3.1641077995300293, + 0.3225497603416443, + -0.39524197578430176, + 1.152822732925415, + -1.508494257926941, + 1.1221402883529663, + -3.361729860305786, + -1.9251339435577393 + ], + [ + 2.2599079608917236, + -0.005553290247917175, + -2.4053664207458496, + 0.39961689710617065, + -0.40980857610702515, + -0.9905421733856201, + -1.7244036197662354, + -0.06417369842529297, + 0.6376181840896606, + 0.5168956518173218, + -1.6740490198135376, + -0.1833474189043045, + -3.9368975162506104, + -1.7146861553192139, + 0.9366075992584229, + 0.701168417930603, + -0.23131346702575684, + -2.740941286087036, + 0.908729076385498, + -3.6231396198272705, + -1.3161351680755615 + ], + [ + 1.819408655166626, + -0.5504165291786194, + -1.302573800086975, + -0.6927691698074341, + -0.7548186779022217, + -2.1824703216552734, + -2.4921669960021973, + 0.6337187886238098, + 0.1691494882106781, + 0.7547447681427002, + -0.4371253252029419, + -0.6562469601631165, + -2.313655376434326, + -2.4974589347839355, + 0.05105690658092499, + 0.3473445773124695, + -0.4244192838668823, + -1.8233305215835571, + -0.1157217025756836, + -2.0408761501312256, + -2.2010600566864014 + ] + ], + "predicted_scores": [ + 7.347308158874512, + 7.290104866027832, + 7.171411991119385, + 6.953240871429443, + 6.723456382751465, + 6.7069830894470215, + 5.547750949859619, + 5.501445293426514 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2054899126/train1", + "selected_index": 0, + "state_hash": "b26c5f64bc7e877098323281f2b58e7850275b30405bc411ed7ffc7cf69f4e26", + "stored_base_utility": 0.149173304438591, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.037537217140197754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037537217140197754 + }, + "base_progress": 0.037537217140197754, + "base_success": false, + "base_utility": 0.037537217140197754, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01185232400894165, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01185232400894165 + }, + { + "contact_quality": null, + "progress": 0.015052755363285542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015052755363285542 + }, + { + "contact_quality": null, + "progress": 0.01309619378298521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01309619378298521 + }, + { + "contact_quality": null, + "progress": 0.01102795172482729, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01102795172482729 + }, + { + "contact_quality": null, + "progress": 0.018680056557059288, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018680056557059288 + }, + { + "contact_quality": null, + "progress": 0.014400660991668701, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014400660991668701 + }, + { + "contact_quality": null, + "progress": 0.0062745013274252415, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0062745013274252415 + }, + { + "contact_quality": null, + "progress": 0.025354644283652306, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025354644283652306 + } + ], + "candidate_progress": [ + 0.01185232400894165, + 0.015052755363285542, + 0.01309619378298521, + 0.01102795172482729, + 0.018680056557059288, + 0.014400660991668701, + 0.0062745013274252415, + 0.025354644283652306 + ], + "candidate_source_chart_ids": [ + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-6f26118aa94bacd82250" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c832439976a37f52d094", + "generated_tangents": [ + [ + 0.01794205605983734, + -0.39928489923477173, + 0.3047671914100647, + 0.640389084815979, + 0.7059146165847778, + 0.5030750036239624, + 0.16233357787132263, + 1.2212294340133667, + -0.27749332785606384, + -0.5608906745910645, + -1.718010663986206, + 0.508609414100647, + -1.7132834196090698, + -1.3535130023956299, + -0.03722262755036354, + 0.2721763551235199, + 0.49909234046936035, + -0.5537380576133728, + 1.2883248329162598, + 2.141218900680542, + 1.7727383375167847 + ], + [ + 0.02859903872013092, + -0.45314890146255493, + 0.33511263132095337, + 0.5635606050491333, + 0.6740097999572754, + 0.4368559718132019, + -0.17657595872879028, + 1.1521787643432617, + -0.20417176187038422, + -0.40049248933792114, + -1.694342017173767, + 0.7616839408874512, + -1.620996117591858, + -1.0670340061187744, + 0.032699525356292725, + 0.24202735722064972, + 0.5946812033653259, + -0.3700817823410034, + 1.4160525798797607, + 2.2401106357574463, + 1.7921828031539917 + ], + [ + 0.05263262987136841, + -0.3874813914299011, + 0.23047152161598206, + 0.7403726577758789, + 0.7350418567657471, + 0.2908567190170288, + -0.19339340925216675, + 1.2352060079574585, + -0.17305763065814972, + -0.21827661991119385, + -1.5216920375823975, + 0.8042436838150024, + -1.7348999977111816, + -0.8976197242736816, + 0.06708423048257828, + 0.27360913157463074, + 0.537377119064331, + -0.3399698734283447, + 1.4630978107452393, + 2.1092727184295654, + 1.8591285943984985 + ], + [ + -0.4147792458534241, + -0.2259741574525833, + 0.46644696593284607, + 1.522256851196289, + 0.5814188718795776, + 0.4100750982761383, + 0.19133654236793518, + 1.4422869682312012, + -0.31723716855049133, + -0.39828479290008545, + -2.055227041244507, + 1.1731226444244385, + -1.3083711862564087, + -1.6941227912902832, + 0.27829301357269287, + 0.3670850992202759, + 0.4991532564163208, + -0.01967155933380127, + 1.4013078212738037, + 2.4548299312591553, + 2.3025670051574707 + ], + [ + -0.0542437881231308, + -0.43662893772125244, + 0.34782475233078003, + 0.6523323059082031, + 0.6649909019470215, + 0.12254741787910461, + -0.16712021827697754, + 1.0428181886672974, + -0.14330950379371643, + -0.2890681028366089, + -1.5606493949890137, + 0.8559022545814514, + -1.7993046045303345, + -0.7523421049118042, + 0.08717244863510132, + 0.24994580447673798, + 0.5766638517379761, + -0.32058560848236084, + 1.4456861019134521, + 2.2457237243652344, + 1.8380358219146729 + ], + [ + -0.38087594509124756, + -0.26499029994010925, + 0.06930813193321228, + 1.2164201736450195, + 0.5663763284683228, + 0.603212833404541, + 0.4198690354824066, + 1.1472017765045166, + -0.37199628353118896, + -0.4269323945045471, + -1.9943755865097046, + 1.146256685256958, + -1.4226115942001343, + -1.6951652765274048, + 0.04609443247318268, + 0.2929285168647766, + 0.8115304708480835, + -0.18005909025669098, + 1.7425360679626465, + 2.6117398738861084, + 1.821550726890564 + ], + [ + -0.35506951808929443, + -0.3280073404312134, + 0.29627925157546997, + 1.6572011709213257, + 0.8197150230407715, + 0.3277698755264282, + 0.41143620014190674, + 1.386468529701233, + -0.2305210828781128, + -0.5448424220085144, + -1.7443983554840088, + 1.1193647384643555, + -1.1633388996124268, + -1.593414306640625, + 0.4279046952724457, + 0.31593361496925354, + 0.4422547221183777, + 0.10983540117740631, + 1.2414162158966064, + 2.3535211086273193, + 2.1370372772216797 + ], + [ + -0.40747785568237305, + -0.13480070233345032, + 0.7814823985099792, + 1.509238362312317, + 0.5356488227844238, + 0.6098292469978333, + 0.35830628871917725, + 0.8087968826293945, + -0.07927663624286652, + -0.06282895058393478, + -2.015357732772827, + 1.3790664672851562, + -1.7812633514404297, + -1.6387113332748413, + 0.15258343517780304, + 0.23151299357414246, + 0.7743162512779236, + -0.21275068819522858, + 1.7635765075683594, + 2.566110610961914, + 1.2568166255950928 + ] + ], + "generated_utilities": [ + 0.01185232400894165, + 0.015052755363285542, + 0.01309619378298521, + 0.01102795172482729, + 0.018680056557059288, + 0.014400660991668701, + 0.0062745013274252415, + 0.025354644283652306 + ], + "group_id": "ms-c832439976a37f52d094", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0808674544095993, + 0.13129448890686035, + 0.11199682950973511, + 0.2594742774963379, + 0.17684690654277802, + 0.11511725187301636, + 0.17100229859352112, + 0.12906593084335327, + 0.12404869496822357, + 0.10653237998485565, + 0.15492436289787292, + 0.05800905078649521, + 0.04767048358917236, + 0.09533219039440155, + 0.03300365060567856, + 0.07704520225524902 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025296777486801147, + 0.11629299819469452, + -0.024434387683868408, + 1.0812468528747559, + 0.04022188484668732, + -0.01547396183013916, + 0.0, + 0.05167186260223389, + 0.07629208266735077, + -0.00870610773563385, + -0.5777420997619629, + 0.07687681913375854, + -0.6336902379989624, + 0.0, + -0.043449461460113525, + -0.029798656702041626, + 0.18944299221038818, + 0.03022998571395874, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + 0.05685463547706604, + -0.3914306163787842, + 0.4578048586845398, + 1.0812468528747559, + 0.10990457236766815, + -0.18159747123718262, + 0.0, + 0.05167186260223389, + -0.5002377033233643, + 0.04299694299697876, + -0.5777420997619629, + 0.02747189998626709, + -0.6336902379989624, + 0.0, + -0.08638033270835876, + 0.17561247944831848, + 0.37304359674453735, + 0.2538231611251831, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + -0.3902576267719269, + 0.03852551057934761, + 0.16114424169063568, + 2.081246852874756, + 0.22516529262065887, + 0.8184025287628174, + 0.3388257324695587, + 1.0516718626022339, + -0.061541248112916946, + 0.22279328107833862, + -1.577742099761963, + 1.027471899986267, + -1.6336902379989624, + -1.6284003257751465, + -0.24836044013500214, + -0.3427097499370575, + 1.1894429922103882, + -0.15580247342586517, + 1.7312084436416626, + 2.8623785972595215, + 1.0248357057571411 + ], + [ + -0.5903857946395874, + -0.10628272593021393, + -0.07594148814678192, + 2.194340944290161, + 0.11632551997900009, + 0.942036509513855, + 0.06351110339164734, + 0.7966359853744507, + 0.1702205240726471, + 0.1549067199230194, + -1.439386248588562, + 1.3076335191726685, + -1.3458164930343628, + -1.90281081199646, + -0.2967699468135834, + -0.27783676981925964, + 1.0556988716125488, + 0.06510581076145172, + 1.6170521974563599, + 3.027111530303955, + 1.154527187347412 + ], + [ + -0.7559880614280701, + 0.694898247718811, + -0.3551136255264282, + 1.8615511655807495, + 0.9249891042709351, + 0.969862699508667, + 0.4452719986438751, + 1.5801937580108643, + 0.13303470611572266, + -0.13816028833389282, + -1.2399333715438843, + 1.5769362449645996, + -1.5071980953216553, + -2.0578060150146484, + -0.8449342846870422, + 0.3327537477016449, + 1.6781622171401978, + 0.4016150236129761, + 1.8172147274017334, + 3.1609320640563965, + 1.4916398525238037 + ] + ], + "predicted_scores": [ + 3.0802533626556396, + 2.8079328536987305, + 2.694960117340088, + 2.5704212188720703, + 2.3174843788146973, + 2.289219379425049, + 1.906691074371338, + 1.363757848739624 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train1", + "selected_index": 0, + "state_hash": "0f2810f22383259329f6e94fdc8384176c664f2ed476924e59db7cda4f4a4863", + "stored_base_utility": 0.0808674544095993, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09138330072164536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09138330072164536 + }, + "base_progress": 0.09138330072164536, + "base_success": false, + "base_utility": 0.09138330072164536, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.011022846214473248, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011022846214473248 + }, + { + "contact_quality": null, + "progress": 0.005815426353365183, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005815426353365183 + }, + { + "contact_quality": null, + "progress": 0.006854832172393799, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006854832172393799 + }, + { + "contact_quality": null, + "progress": 0.004729151725769043, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004729151725769043 + }, + { + "contact_quality": null, + "progress": 0.004355867858976126, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004355867858976126 + }, + { + "contact_quality": null, + "progress": 0.0065706572495400906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0065706572495400906 + }, + { + "contact_quality": null, + "progress": 0.0038128693122416735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0038128693122416735 + }, + { + "contact_quality": null, + "progress": 0.00446764612570405, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00446764612570405 + } + ], + "candidate_progress": [ + 0.011022846214473248, + 0.005815426353365183, + 0.006854832172393799, + 0.004729151725769043, + 0.004355867858976126, + 0.0065706572495400906, + 0.0038128693122416735, + 0.00446764612570405 + ], + "candidate_source_chart_ids": [ + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c96b415f083f60857334", + "generated_tangents": [ + [ + -0.8631720542907715, + 0.20969156920909882, + 0.476644366979599, + -0.04481770098209381, + -0.12367299199104309, + 2.0115714073181152, + 2.2624404430389404, + 0.13628721237182617, + 0.3867899477481842, + 1.0296491384506226, + 0.0583997406065464, + -0.09586712718009949, + 1.128807544708252, + 0.6050926446914673, + -0.4575827121734619, + -0.020532377064228058, + 0.11341233551502228, + 1.2037279605865479, + -1.155188798904419, + -0.1706058233976364, + -0.18583622574806213 + ], + [ + -0.39789462089538574, + 0.15221968293190002, + 1.0981792211532593, + -0.16742363572120667, + 0.3057832717895508, + 2.5257771015167236, + 1.3183159828186035, + -0.17978835105895996, + -0.012723429128527641, + 1.254101037979126, + 0.31563514471054077, + -0.06348013877868652, + 1.7093688249588013, + 1.2359894514083862, + -0.3803277015686035, + 0.06949549913406372, + 0.30669891834259033, + 1.182504415512085, + -0.2994729280471802, + 0.037694960832595825, + -0.916029691696167 + ], + [ + -0.3888220191001892, + 0.12620966136455536, + 1.0046055316925049, + -0.19541215896606445, + 0.3842652142047882, + 2.5759148597717285, + 1.3602116107940674, + -0.18259736895561218, + 0.047947175800800323, + 1.186171054840088, + 0.3989625871181488, + 0.0273311585187912, + 1.7371598482131958, + 0.9497014880180359, + -0.33922454714775085, + 0.028795022517442703, + 0.2447018325328827, + 1.1618549823760986, + -0.17465215921401978, + 0.10633102059364319, + -0.841964602470398 + ], + [ + -1.1787970066070557, + 0.01931753009557724, + 0.9009875655174255, + -0.049869269132614136, + -0.04317525029182434, + 2.61747145652771, + 2.009066343307495, + -0.18116849660873413, + 0.3759390413761139, + 1.1102315187454224, + 0.13905367255210876, + 0.17136947810649872, + 1.3530523777008057, + 1.0468418598175049, + -0.4049355685710907, + 0.07246808707714081, + 0.2904800772666931, + 1.3014835119247437, + -0.9384945034980774, + -0.20737358927726746, + -0.6260594129562378 + ], + [ + -1.0669664144515991, + -0.05625876039266586, + 0.8500634431838989, + 0.025441870093345642, + -0.23586323857307434, + 2.4919440746307373, + 2.0146644115448, + -0.2825126349925995, + 0.34640994668006897, + 1.0405712127685547, + 0.27461907267570496, + 0.2203475683927536, + 1.263142466545105, + 0.81635981798172, + -0.28405964374542236, + 0.007491793483495712, + 0.40737515687942505, + 1.2977722883224487, + -0.9127889275550842, + 0.06070518493652344, + -0.7309354543685913 + ], + [ + -1.635523796081543, + -0.009781613945960999, + 0.7616024017333984, + 0.02520376443862915, + 0.05396144092082977, + 2.5448250770568848, + 2.040884494781494, + -0.298483282327652, + 0.3895706236362457, + 1.0330209732055664, + 0.4657478332519531, + -0.09435263276100159, + 1.0137053728103638, + 1.0496138334274292, + -0.1867525279521942, + 0.08877302706241608, + -0.019419148564338684, + 1.3915016651153564, + -0.559948742389679, + -0.1923726201057434, + -0.7642843723297119 + ], + [ + -1.2364280223846436, + -0.05301504582166672, + 0.8328337669372559, + 0.09453615546226501, + -0.14185777306556702, + 2.3135950565338135, + 1.8596365451812744, + -0.4325282871723175, + 0.3867894411087036, + 1.0150127410888672, + 0.16291148960590363, + 0.18627028167247772, + 1.208469271659851, + 0.7966751456260681, + -0.4114797115325928, + -0.056879088282585144, + 0.3379962742328644, + 1.3211537599563599, + -0.9629276394844055, + 0.21863311529159546, + -0.7556114196777344 + ], + [ + -1.4423089027404785, + -0.10011017322540283, + 1.0570999383926392, + -0.004705637693405151, + -0.1491108536720276, + 2.23101806640625, + 1.9499225616455078, + -0.38821807503700256, + 0.3906157314777374, + 0.9672142267227173, + -0.07669848948717117, + 0.36660289764404297, + 1.3273799419403076, + 0.3710804283618927, + -0.30547404289245605, + 0.005290810018777847, + 0.1930892914533615, + 1.2460944652557373, + -1.0674211978912354, + -0.03995099663734436, + -0.8713643550872803 + ] + ], + "generated_utilities": [ + 0.011022846214473248, + 0.005815426353365183, + 0.006854832172393799, + 0.004729151725769043, + 0.004355867858976126, + 0.0065706572495400906, + 0.0038128693122416735, + 0.00446764612570405 + ], + "group_id": "ms-c96b415f083f60857334", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15946325659751892, + 0.16313008964061737, + 0.2226802557706833, + 0.18744397163391113, + 0.21545825898647308, + 0.15985511243343353, + 0.16218391060829163, + 0.26426148414611816, + 0.184181809425354, + 0.24352896213531494, + 0.14179718494415283, + 0.1026589497923851, + 0.1274908483028412, + 0.05275428295135498, + 0.08544860780239105, + 0.07143914699554443 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.16191565990448, + 0.4561867117881775, + 1.1241415739059448, + 0.08947351574897766, + 0.32140541076660156, + 3.6895532608032227, + 3.6284518241882324, + -1.8666775226593018, + -0.22328677773475647, + 1.48662531375885, + 1.4737720489501953, + 1.35750150680542, + 2.507124423980713, + 1.9315069913864136, + -0.8371121883392334, + 0.5339290499687195, + 1.0061794519424438, + 2.149812698364258, + 0.08911214768886566, + 0.17410686612129211, + 0.3549031913280487 + ], + [ + -0.14804881811141968, + -0.5333454608917236, + 0.7481307983398438, + 0.22411420941352844, + 1.7204210758209229, + 3.908766746520996, + 3.3698034286499023, + -1.6747719049453735, + 0.2769082486629486, + 2.042428970336914, + 0.44748249650001526, + 1.9492043256759644, + 2.918121576309204, + 3.2413060665130615, + -0.6994609832763672, + -0.3854917883872986, + -0.6702244281768799, + 2.408869504928589, + -0.5672509670257568, + 0.5941196084022522, + 0.46011871099472046 + ], + [ + -0.46204906702041626, + -0.4236203730106354, + 1.8582539558410645, + 0.8416532278060913, + 1.2386386394500732, + 3.7320151329040527, + 2.242549180984497, + -1.8652616739273071, + -1.0259326696395874, + 1.3383660316467285, + 1.7068204879760742, + 0.7941814661026001, + 3.3389508724212646, + 2.7797679901123047, + -0.9363516569137573, + 0.17812880873680115, + 0.6111975908279419, + 2.066016435623169, + -0.28941091895103455, + 1.501695990562439, + -0.35273995995521545 + ], + [ + -0.7471081614494324, + -0.029650092124938965, + 1.019482135772705, + -0.30564722418785095, + 1.380143642425537, + 3.247985601425171, + 3.0548973083496094, + -0.5582134127616882, + -0.920264482498169, + 2.3859832286834717, + 1.1080355644226074, + 1.5716153383255005, + 3.089156150817871, + 2.043484687805176, + 0.1913408637046814, + -0.7557333707809448, + -0.6297664642333984, + 1.7829738855361938, + -0.6967166662216187, + -0.10757219791412354, + 0.229112908244133 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027880847454071045, + -0.2692106068134308, + 0.12925803661346436, + -0.1745016872882843, + -0.1231425404548645, + 1.9967715740203857, + 0.0, + -0.1433020830154419, + 0.2667914927005768, + 0.8867976665496826, + -0.057856738567352295, + 0.5789351463317871, + 1.7831475734710693, + 0.0, + -0.35565105080604553, + 0.293251097202301, + 0.012226536870002747, + 1.121243953704834, + -0.15060651302337646, + 0.03409510850906372, + 0.0 + ], + [ + -0.7738120555877686, + 0.08056297898292542, + 1.1292580366134644, + 0.4734484851360321, + 0.8768574595451355, + 2.9967715740203857, + 3.0509116649627686, + -1.143302083015442, + -0.1310054361820221, + 1.8867976665496826, + 0.9421432614326477, + 1.578935146331787, + 2.7831475734710693, + 2.6037895679473877, + -0.5597631931304932, + 0.11406822502613068, + 0.2593899667263031, + 2.121243953704834, + -0.1791832000017166, + 0.5773167610168457, + 0.41381677985191345 + ], + [ + -0.6024693250656128, + 0.29007619619369507, + 1.3939473628997803, + 0.3474765419960022, + 1.1848478317260742, + 3.127882719039917, + 3.3329083919525146, + -1.2855982780456543, + 0.20972535014152527, + 1.8329110145568848, + 1.267270803451538, + 1.709198236465454, + 2.927372455596924, + 2.9659509658813477, + -0.7555317282676697, + 0.015305742621421814, + 0.6009905338287354, + 2.188507556915283, + 0.07961152493953705, + 0.4547080397605896, + 0.20378917455673218 + ], + [ + -1.2702982425689697, + 0.06162632256746292, + 0.9447290897369385, + 0.837199330329895, + 0.5848468542098999, + 3.4801836013793945, + 2.8138537406921387, + -1.5083554983139038, + -0.45768383145332336, + 2.08561110496521, + 1.4951050281524658, + 1.8484808206558228, + 2.5617129802703857, + 2.942486047744751, + -0.562404215335846, + 0.17441512644290924, + 0.7880480289459229, + 1.8807158470153809, + -0.043379828333854675, + 0.39090824127197266, + 0.43096962571144104 + ] + ], + "predicted_scores": [ + 8.127728462219238, + 7.716030597686768, + 7.650069236755371, + 7.320975303649902, + 7.23729133605957, + 6.941411018371582, + 6.868640422821045, + 6.600250244140625 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train1", + "selected_index": 0, + "state_hash": "bf24d506cf320effed2cd7faf94d8618b0626bf8d5c3abc1f837513597c18231", + "stored_base_utility": 0.15946325659751892, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0626688003540039, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0626688003540039 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 0.0626688003540039, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ebacae0135af2832dfbb", + "ms-003a2225c0892dcda683", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683", + "ms-3d090ac2459c4b891dd1", + "ms-ebacae0135af2832dfbb", + "ms-003a2225c0892dcda683", + "ms-6d46dae312649c977b64" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + false, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "generated_tangents": [ + [ + 0.06849304586648941, + 0.020963147282600403, + 0.3397200107574463, + -0.21450698375701904, + 1.7600600719451904, + 0.3978930115699768, + 0.08446690440177917, + 0.04985088109970093, + 0.009671914391219616, + 0.1878652274608612, + 0.06474080681800842, + 0.43082863092422485, + 0.01677122712135315, + -0.09526801109313965, + -0.9732409119606018, + -0.11426205933094025, + 1.0602449178695679, + 0.355584055185318, + -0.368932843208313, + 1.6154060363769531, + 0.22390800714492798 + ], + [ + 0.15004707872867584, + -0.08960972726345062, + 0.33218637108802795, + -0.31179484724998474, + 1.9093825817108154, + 0.37549805641174316, + 0.18935957551002502, + -0.00799529254436493, + 0.011870434507727623, + 0.1430848091840744, + 0.12518903613090515, + 0.36604756116867065, + 0.08603277802467346, + -0.06902790069580078, + -1.046745777130127, + -0.18465444445610046, + 1.1558502912521362, + 0.2851211130619049, + -0.33572494983673096, + 1.7115893363952637, + 0.13107410073280334 + ], + [ + 0.19285762310028076, + -0.04629793018102646, + 0.389565646648407, + -0.19866038858890533, + 1.7930231094360352, + 0.46391668915748596, + 0.06795725226402283, + 0.051617443561553955, + 0.06245651841163635, + 0.12304775416851044, + 0.04547211155295372, + 0.43040698766708374, + 0.028770118951797485, + 0.04088864475488663, + -0.9964492917060852, + -0.06462139636278152, + 1.080371618270874, + 0.2642951011657715, + -0.34562748670578003, + 1.6415646076202393, + 0.2722638249397278 + ], + [ + 0.11304552853107452, + -0.03762324899435043, + 0.4021502137184143, + -0.2493474781513214, + 1.8291187286376953, + 0.3095042109489441, + 0.09451401233673096, + 0.05497932434082031, + 0.006364226806908846, + 0.13321825861930847, + 0.12507623434066772, + 0.307923287153244, + 0.11472919583320618, + -0.15573225915431976, + -1.0519554615020752, + -0.13943922519683838, + 1.1995705366134644, + 0.30124446749687195, + -0.24065834283828735, + 1.7303709983825684, + 0.11594261974096298 + ], + [ + 0.038671910762786865, + 0.021258823573589325, + 0.3150809109210968, + -0.3030160069465637, + 1.8387234210968018, + 0.4528977870941162, + -0.08211082220077515, + 0.12862318754196167, + -0.009231509640812874, + 0.114659883081913, + 0.08934962749481201, + 0.4509127736091614, + 0.013727247714996338, + -0.1062411218881607, + -0.982166588306427, + -0.0884760320186615, + 1.0436427593231201, + 0.4198005199432373, + -0.24441659450531006, + 1.7569730281829834, + 0.18441389501094818 + ], + [ + 0.2280730903148651, + -0.06717413663864136, + 0.2869489789009094, + -0.13354362547397614, + 1.5840575695037842, + 0.29217830300331116, + -0.1405428647994995, + 0.1144861951470375, + 0.09890282154083252, + 0.18530166149139404, + 0.10732823610305786, + 0.5616626739501953, + -0.0902877151966095, + 0.09286195039749146, + -0.9488819241523743, + 0.04549604281783104, + 1.0309386253356934, + 0.4310721457004547, + -0.028140664100646973, + 1.790179967880249, + 0.22344520688056946 + ], + [ + 0.024200841784477234, + -0.027662798762321472, + 0.20946550369262695, + -0.2739817500114441, + 1.816767692565918, + 0.4621601700782776, + -0.015732914209365845, + -0.08524897694587708, + 0.07004743069410324, + 0.17961004376411438, + 0.04790934547781944, + 0.45547717809677124, + 0.062212854623794556, + 0.14574454724788666, + -0.9427071213722229, + -0.13454841077327728, + 1.087843894958496, + 0.16174688935279846, + -0.2898705005645752, + 1.7132596969604492, + 0.19047726690769196 + ], + [ + 0.09942897409200668, + -0.024153687059879303, + 0.44161221385002136, + -0.17972099781036377, + 1.815826416015625, + 0.5074065923690796, + 0.1741277277469635, + -0.06894567608833313, + 0.06373905390501022, + 0.11198487877845764, + 0.06291908025741577, + 0.397016704082489, + 0.1716410219669342, + 0.0040760338306427, + -1.018132209777832, + -0.0712416023015976, + 1.095371127128601, + 0.2742389440536499, + -0.36693036556243896, + 1.5890514850616455, + 0.356515109539032 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 0.0626688003540039, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20931994915008545, + 2.0, + 2.0, + 0.2008315920829773, + 0.09476469457149506, + 0.05857282876968384, + 0.10009106248617172, + 0.07648500055074692, + 0.05889680236577988, + 0.014406224712729454, + 0.09262995421886444, + 0.15997587144374847, + 0.04682888835668564, + 0.09751574695110321, + 0.06459556519985199, + 0.20335909724235535 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7537296414375305, + 0.4158197343349457, + 1.4326180219650269, + -1.4107877016067505, + 2.9579341411590576, + 0.9669660925865173, + 1.4339183568954468, + 0.5987398624420166, + -0.2018759399652481, + 0.7105501890182495, + -0.03802938759326935, + 1.3799947500228882, + -0.6568997502326965, + -0.37473973631858826, + -1.857657790184021, + 0.038770925253629684, + 1.931185007095337, + 0.6317195296287537, + 0.1380508691072464, + 2.954265832901001, + 2.9641003608703613 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8678367137908936, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.7494794726371765, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.928200721740723 + ], + [ + 0.6226351857185364, + 0.34334811568260193, + 1.492361068725586, + -1.5917733907699585, + 3.1334922313690186, + 1.0313001871109009, + 1.2075668573379517, + 0.3046378493309021, + -0.11550238728523254, + 0.4499495327472687, + 0.16646838188171387, + 1.5877203941345215, + -0.6488803625106812, + -0.5760968327522278, + -1.8349132537841797, + 0.14188142120838165, + 2.0770444869995117, + 0.5806879997253418, + 0.23913151025772095, + 2.8127381801605225, + 3.0649962425231934 + ], + [ + 0.5975998044013977, + 0.722756028175354, + 1.7548110485076904, + -1.166775107383728, + 2.6060047149658203, + 0.6675758361816406, + 1.125187635421753, + 0.3373170793056488, + -0.1842741072177887, + 0.8186168670654297, + -0.31481224298477173, + 1.2082113027572632, + -0.9804694652557373, + -0.14131735265254974, + -1.700500249862671, + 0.07075637578964233, + 1.5488730669021606, + 0.549576461315155, + 0.034750498831272125, + 3.227431297302246, + 2.9464566707611084 + ], + [ + 0.844982922077179, + 0.7132244110107422, + 1.4430484771728516, + -1.3331272602081299, + 3.1703803539276123, + 1.1851513385772705, + 1.355688452720642, + 0.6115275025367737, + 0.2616361379623413, + 1.2030918598175049, + 0.05625858157873154, + 1.6050300598144531, + -0.7155078053474426, + -0.021488159894943237, + -2.073806047439575, + 0.5387547016143799, + 1.6223118305206299, + 0.7140727043151855, + -0.06027664244174957, + 2.7503156661987305, + 2.6344075202941895 + ], + [ + 0.5821706056594849, + 0.78206866979599, + 1.8119354248046875, + -0.8252442479133606, + 3.038909435272217, + 1.3900940418243408, + 1.7462186813354492, + 0.005083680152893066, + -0.27262064814567566, + 0.867202877998352, + -0.5555376410484314, + 1.172202706336975, + -1.2016760110855103, + -0.06355220079421997, + -2.2925045490264893, + 0.6078628301620483, + 1.7895314693450928, + 0.8720078468322754, + 0.472573459148407, + 3.377012014389038, + 2.6039178371429443 + ], + [ + 0.3789674639701843, + 0.3270051181316376, + 1.056584358215332, + -2.0788497924804688, + 2.948819875717163, + 1.6022355556488037, + 0.8599810004234314, + 0.9570736885070801, + -0.6380544900894165, + 0.3103100061416626, + -0.42896610498428345, + 1.7150276899337769, + -0.8975871205329895, + -0.278381884098053, + -1.305256724357605, + 0.3637259304523468, + 1.8004443645477295, + 0.16446983814239502, + 0.19097819924354553, + 3.43778657913208, + 3.538135528564453 + ], + [ + 0.6903814673423767, + 1.1327197551727295, + 1.8523926734924316, + -2.0428731441497803, + 3.1870431900024414, + 0.5352929830551147, + 0.6004904508590698, + 1.2120530605316162, + 0.16280107200145721, + 1.119707465171814, + 0.7612010836601257, + 0.7890661954879761, + -1.3447637557983398, + -0.1703537553548813, + -1.1010353565216064, + 0.32831913232803345, + 1.5534476041793823, + -0.2119135856628418, + -0.562728762626648, + 2.4702467918395996, + 2.5528204441070557 + ], + [ + 0.9177048206329346, + -0.04751536250114441, + 1.4819984436035156, + -2.1542677879333496, + 3.522390842437744, + 1.8177759647369385, + 2.2213237285614014, + 0.8223017454147339, + -0.1973809152841568, + -0.053272128105163574, + -0.5862784385681152, + 0.6421516537666321, + -0.8713911771774292, + -0.9495017528533936, + -1.6454768180847168, + -0.38891172409057617, + 1.648432970046997, + 1.4951133728027344, + 0.5893608331680298, + 2.449934482574463, + 3.419659376144409 + ], + [ + 1.2956775426864624, + -0.3119873106479645, + 1.5930066108703613, + -1.7682520151138306, + 2.4385876655578613, + 1.4679458141326904, + 1.4536705017089844, + 0.31068816781044006, + 0.660710871219635, + 0.8104690909385681, + 0.483506441116333, + 2.1112046241760254, + -0.22927385568618774, + 0.17858651280403137, + -2.485126495361328, + -0.9187790155410767, + 1.2810213565826416, + 1.3727718591690063, + 0.44044917821884155, + 3.4018898010253906, + 3.39054799079895 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0012856721878051758, + 0.00795695185661316, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + 0.11055737733840942, + 0.0, + 0.02143383026123047, + 0.0025591105222702026, + -0.030314385890960693, + -0.0125969760119915, + 0.3799947500228882, + 0.0076233744621276855, + 0.0, + -0.857657790184021, + 0.013764806091785431, + 0.9311850070953369, + 0.07110422849655151, + 0.03423507511615753, + 1.954265832901001, + 0.0 + ], + [ + -0.24627035856246948, + -0.16095152497291565, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + -0.023127973079681396, + 0.0, + -0.24312204122543335, + 0.1389055997133255, + 0.11457735300064087, + 0.06618259102106094, + 0.3799947500228882, + 0.07616078853607178, + 0.0, + -0.857657790184021, + 0.09948430210351944, + 0.9311850070953369, + 0.006249547004699707, + -0.33439844846725464, + 1.954265832901001, + 0.0 + ] + ], + "predicted_scores": [ + 4.393341541290283, + 4.325108528137207, + 4.305380344390869, + 4.273581027984619, + 4.09778356552124, + 3.936275005340576, + 3.919905424118042, + 3.758706569671631 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "534717141/train1", + "selected_index": 0, + "state_hash": "5318995ba859471b8678d6594274c7c0565a1ccfd83ea302378cc2884dc5cc32", + "stored_base_utility": 0.20931994915008545, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14809735119342804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14809735119342804 + }, + "base_progress": 0.14809735119342804, + "base_success": false, + "base_utility": 0.14809735119342804, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0006723999977111816, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006723999977111816 + }, + { + "contact_quality": null, + "progress": 0.000547130883205682, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000547130883205682 + }, + { + "contact_quality": null, + "progress": 0.000514427840244025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000514427840244025 + }, + { + "contact_quality": null, + "progress": 0.0005189180374145508, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0005189180374145508 + }, + { + "contact_quality": null, + "progress": 0.001536289812065661, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001536289812065661 + }, + { + "contact_quality": null, + "progress": 0.0004230936465319246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0004230936465319246 + }, + { + "contact_quality": null, + "progress": 0.00036319097853265703, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00036319097853265703 + }, + { + "contact_quality": null, + "progress": 0.0003688931465148926, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0003688931465148926 + } + ], + "candidate_progress": [ + 0.0006723999977111816, + 0.000547130883205682, + 0.000514427840244025, + 0.0005189180374145508, + 0.001536289812065661, + 0.0004230936465319246, + 0.00036319097853265703, + 0.0003688931465148926 + ], + "candidate_source_chart_ids": [ + "ms-f2b2b2a1aa2863c3676f", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-da142025733b53fcf7eb", + "generated_tangents": [ + [ + -0.3432873487472534, + -0.4166366457939148, + 1.7469239234924316, + 0.30275484919548035, + 1.5726878643035889, + 2.5822014808654785, + 2.0489354133605957, + -0.5852302312850952, + 0.1062394306063652, + 1.4572888612747192, + -0.3047958016395569, + 1.0039513111114502, + 2.0345330238342285, + 2.8583405017852783, + -1.4076637029647827, + -0.2795407772064209, + 1.7358763217926025, + 1.0890778303146362, + 0.5476433038711548, + 2.262739658355713, + 1.6884771585464478 + ], + [ + -0.44322335720062256, + -0.26510465145111084, + 1.6426684856414795, + 0.22649741172790527, + 1.5892977714538574, + 3.0924112796783447, + 1.9100933074951172, + -0.5058478116989136, + 0.1344156563282013, + 1.6353282928466797, + 0.02294306457042694, + 0.9665766954421997, + 1.906577706336975, + 2.7980704307556152, + -1.6457966566085815, + -0.30786964297294617, + 1.8597486019134521, + 1.1984976530075073, + 0.6639366149902344, + 1.9491913318634033, + 1.9728260040283203 + ], + [ + -0.14347891509532928, + -0.29903554916381836, + 2.006197690963745, + 0.12976646423339844, + 1.1190180778503418, + 2.7899868488311768, + 2.288774013519287, + -0.7226301431655884, + 0.24866513907909393, + 1.41183602809906, + -0.07595058530569077, + 0.925747811794281, + 2.2851498126983643, + 2.5220324993133545, + -1.641467809677124, + -0.29011446237564087, + 2.005598306655884, + 1.2708896398544312, + 0.7066189050674438, + 2.3896970748901367, + 2.3964414596557617 + ], + [ + -0.05751855671405792, + -0.253709077835083, + 1.705299973487854, + -0.1536070853471756, + 1.6768748760223389, + 3.253661632537842, + 1.874450922012329, + -0.5398855209350586, + -0.022087017074227333, + 1.677821159362793, + -0.002833440899848938, + 1.1556262969970703, + 1.8816803693771362, + 2.575993776321411, + -1.5646936893463135, + -0.34078413248062134, + 2.111616373062134, + 1.0539075136184692, + 0.5210786461830139, + 2.0769670009613037, + 2.0071918964385986 + ], + [ + -0.387844443321228, + -0.3073009252548218, + 1.5910156965255737, + -0.09482955187559128, + 1.246790885925293, + 2.478865623474121, + 2.3712637424468994, + -0.3380632698535919, + 0.1314142942428589, + 1.1512407064437866, + 0.058152489364147186, + 0.5780254006385803, + 1.717081904411316, + 3.6755905151367188, + -1.2382253408432007, + -0.4301864802837372, + 1.4169405698776245, + 0.7797537446022034, + 0.803180456161499, + 2.1079227924346924, + 2.019592761993408 + ], + [ + -0.20618392527103424, + -0.2934543788433075, + 1.82016122341156, + -0.037838131189346313, + 1.3345353603363037, + 2.859126567840576, + 2.06943678855896, + -0.713410496711731, + 0.09354620426893234, + 1.6099059581756592, + -0.027280479669570923, + 0.8799170255661011, + 2.104429244995117, + 2.486814260482788, + -1.7883286476135254, + -0.37900424003601074, + 2.04034423828125, + 1.3327462673187256, + 0.5964900255203247, + 2.144643783569336, + 2.1147611141204834 + ], + [ + -0.21850784122943878, + -0.36162781715393066, + 1.9714752435684204, + -0.03462591767311096, + 1.3450486660003662, + 2.7020928859710693, + 2.5649144649505615, + -0.732650876045227, + 0.048771344125270844, + 1.6956018209457397, + 0.029450014233589172, + 0.9816755652427673, + 2.4808883666992188, + 2.607961654663086, + -1.8436342477798462, + -0.269349604845047, + 1.900127649307251, + 1.4064507484436035, + 0.6604788899421692, + 1.8866121768951416, + 1.769606590270996 + ], + [ + 0.10653827339410782, + -0.3439488410949707, + 1.8145986795425415, + -0.35351812839508057, + 0.9562007784843445, + 2.389096736907959, + 2.441350221633911, + -1.0330595970153809, + -0.026264620944857597, + 1.7622023820877075, + 0.0689915269613266, + 0.9659565687179565, + 2.6354854106903076, + 2.3394391536712646, + -1.6571424007415771, + -0.5705158710479736, + 2.0164456367492676, + 0.9216766357421875, + 0.38031086325645447, + 2.2025082111358643, + 1.607088327407837 + ] + ], + "generated_utilities": [ + 0.0006723999977111816, + 0.000547130883205682, + 0.000514427840244025, + 0.0005189180374145508, + 0.001536289812065661, + 0.0004230936465319246, + 0.00036319097853265703, + 0.0003688931465148926 + ], + "group_id": "ms-da142025733b53fcf7eb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.14843207597732544, + 0.1717694252729416, + 0.09356185048818588, + 0.23231086134910583, + 0.19341367483139038, + 0.04822012037038803, + 0.23024770617485046, + 0.16561973094940186, + 0.2643269896507263, + 0.12403462827205658, + 0.12689702212810516, + 0.16619956493377686, + 0.13481859862804413, + 0.07537531852722168, + 0.11867459863424301, + 0.04740651696920395 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.04848463833332062, + -0.18164527416229248, + 0.6558789014816284, + 0.038211286067962646, + 0.08492374420166016, + 2.265157699584961, + 0.0, + -0.09011825919151306, + 0.0018594712018966675, + 0.6884076595306396, + 0.027596401050686836, + 0.06117302179336548, + 1.278116226196289, + 0.0, + -0.3951960802078247, + -0.08061015605926514, + 1.07906174659729, + 0.3229694366455078, + -0.21076995134353638, + 1.8564467430114746, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0903544425964355, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.570276260375977, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.736253261566162 + ], + [ + 0.08095964789390564, + -0.7926032543182373, + 1.571103811264038, + 0.07894467562437057, + 0.2701656222343445, + 3.806621789932251, + 2.203669786453247, + -0.042464032769203186, + 0.7168105244636536, + 1.8671371936798096, + -0.30838707089424133, + 1.7242767810821533, + 1.8847527503967285, + 1.305934190750122, + -0.42665380239486694, + -0.2973874807357788, + 1.1881210803985596, + 1.6921244859695435, + 1.070060133934021, + 2.067784309387207, + 1.916501760482788 + ], + [ + 0.7620636224746704, + -2.1605899333953857, + 2.1443262100219727, + 0.17886832356452942, + 1.6259945631027222, + 3.787412166595459, + 1.3663169145584106, + 0.49799659848213196, + -0.6301277279853821, + 2.275850534439087, + -0.6735645532608032, + 1.7504758834838867, + 2.7512195110321045, + 1.7462939023971558, + -0.6835377216339111, + 0.36623528599739075, + 1.405979871749878, + 0.6472548842430115, + 1.4791045188903809, + 2.7838869094848633, + 3.0856707096099854 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.29792410135269165, + -0.30587875843048096, + 0.6558789014816284, + -0.4171115756034851, + -0.1733894944190979, + 2.265157699584961, + 0.0, + 0.14607194066047668, + 0.05651267617940903, + 0.6884076595306396, + -0.0433836504817009, + 0.07522618770599365, + 1.278116226196289, + 0.0, + -0.41462820768356323, + 0.26706579327583313, + 1.07906174659729, + 0.44956159591674805, + -0.37072819471359253, + 1.8564467430114746, + 0.0 + ], + [ + 0.18432100117206573, + -0.9473621249198914, + 1.5389783382415771, + -0.041053857654333115, + 0.6282865405082703, + 3.276425838470459, + 2.1839330196380615, + -0.3612370491027832, + 0.23081421852111816, + 1.8094429969787598, + 0.10581490397453308, + 0.8865737915039062, + 2.5146288871765137, + 2.2134878635406494, + -1.6751070022583008, + -0.08268362283706665, + 2.303063154220581, + 1.3061904907226562, + 0.4770132899284363, + 3.109205722808838, + 2.509927272796631 + ], + [ + -0.5288646817207336, + -1.5067847967147827, + 1.376499056816101, + 0.13231416046619415, + 1.048660159111023, + 3.5048489570617676, + 1.9678237438201904, + -0.14848726987838745, + 0.05475667119026184, + 1.5529061555862427, + 0.1597936749458313, + 0.7318928241729736, + 2.17749285697937, + 2.0782582759857178, + -1.5473355054855347, + -0.6715520620346069, + 1.7043644189834595, + 1.1190917491912842, + 0.4214729070663452, + 3.3400330543518066, + 2.3147284984588623 + ] + ], + "predicted_scores": [ + 6.292049884796143, + 6.263817310333252, + 6.2627787590026855, + 5.9403767585754395, + 5.853697299957275, + 5.806769371032715, + 5.642268180847168, + 4.94965934753418 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train1", + "selected_index": 0, + "state_hash": "0026cc5b3762b520baadbda7bf138a2d24fffa5b2306f245ddbc1d1556cccc22", + "stored_base_utility": 0.14843207597732544, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1362423449754715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1362423449754715 + }, + "base_progress": 0.1362423449754715, + "base_success": false, + "base_utility": 0.1362423449754715, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0132790207862854, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0132790207862854 + }, + { + "contact_quality": null, + "progress": 0.018407067283988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018407067283988 + }, + { + "contact_quality": null, + "progress": 0.012948970310389996, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012948970310389996 + }, + { + "contact_quality": null, + "progress": 0.010082741267979145, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010082741267979145 + }, + { + "contact_quality": null, + "progress": 0.010856767185032368, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010856767185032368 + }, + { + "contact_quality": null, + "progress": 0.009846587665379047, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009846587665379047 + }, + { + "contact_quality": null, + "progress": 0.006558021064847708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006558021064847708 + }, + { + "contact_quality": null, + "progress": 0.012989739887416363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012989739887416363 + } + ], + "candidate_progress": [ + 0.0132790207862854, + 0.018407067283988, + 0.012948970310389996, + 0.010082741267979145, + 0.010856767185032368, + 0.009846587665379047, + 0.006558021064847708, + 0.012989739887416363 + ], + "candidate_source_chart_ids": [ + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-76f27d9a935b25e15c38", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-76f27d9a935b25e15c38", + "ms-cf4b63d109a88a4c42bf", + "ms-a2518826fa0a816cc46f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-e2bad6e99e8a281207cc", + "generated_tangents": [ + [ + -0.7656202912330627, + -0.5836631059646606, + 1.9936965703964233, + -0.25754499435424805, + 1.807642936706543, + 2.8049769401550293, + 2.9517674446105957, + 0.8411537408828735, + -0.3716524541378021, + 0.4764257073402405, + 0.8208474516868591, + 0.7875684499740601, + 0.23823460936546326, + 0.508261501789093, + 1.0950506925582886, + 0.2447732537984848, + -0.6680842041969299, + -1.4543466567993164, + 1.4240951538085938, + -1.0104715824127197, + -1.3162401914596558 + ], + [ + -0.5956717729568481, + -0.3503824472427368, + 2.43707537651062, + -0.17598645389080048, + 1.5012032985687256, + 2.822638750076294, + 2.460059642791748, + 0.8111618757247925, + -0.4014160633087158, + 0.16674283146858215, + 0.9034915566444397, + 1.0867396593093872, + 0.3830781877040863, + 0.3912196755409241, + 0.9169946908950806, + 0.25626760721206665, + -0.6009308099746704, + -1.4893838167190552, + 1.5943150520324707, + -0.9501956105232239, + -0.7393081188201904 + ], + [ + -0.8630020618438721, + -0.4366946816444397, + 1.6524393558502197, + -1.1390119791030884, + 2.5286006927490234, + 2.949692487716675, + 2.2771573066711426, + 1.2820297479629517, + -0.40615230798721313, + 0.5594807267189026, + 0.6452037692070007, + 0.6294150352478027, + -0.3940696716308594, + 0.023169033229351044, + 0.9016968011856079, + -0.031307801604270935, + -0.6735029220581055, + -0.6876105666160583, + 1.6249415874481201, + -1.031238079071045, + -1.4563292264938354 + ], + [ + -0.832272469997406, + -0.5986323356628418, + 2.341871976852417, + -0.19841353595256805, + 1.8155076503753662, + 2.685831308364868, + 2.803917407989502, + 0.8019734621047974, + -0.3884523808956146, + 0.48511818051338196, + 0.8000732660293579, + 0.9763474464416504, + 0.22082993388175964, + 0.5804189443588257, + 1.028027892112732, + 0.17335057258605957, + -0.5768880844116211, + -1.429967999458313, + 1.5494601726531982, + -1.0149294137954712, + -1.4201065301895142 + ], + [ + -0.6742351055145264, + -0.6398972868919373, + 2.234661102294922, + -0.05855080485343933, + 1.8243978023529053, + 2.7808637619018555, + 2.83819317817688, + 0.7586811780929565, + -0.38342413306236267, + 0.353824257850647, + 0.7527822256088257, + 0.9232964515686035, + 0.30970773100852966, + 0.364149272441864, + 1.1255438327789307, + 0.18194139003753662, + -0.4114264249801636, + -1.3955891132354736, + 1.3439805507659912, + -0.8595713973045349, + -1.4836050271987915 + ], + [ + -0.8021159768104553, + -0.44574612379074097, + 1.7953410148620605, + -1.218800663948059, + 2.4733567237854004, + 3.018044948577881, + 2.455137014389038, + 1.213944435119629, + -0.4098770022392273, + 0.5149925351142883, + 0.5362481474876404, + 0.6469517946243286, + -0.38892173767089844, + -0.03415808081626892, + 0.9222234487533569, + -0.038790859282016754, + -0.5011613368988037, + -0.7243276834487915, + 1.4363141059875488, + -1.137031078338623, + -1.4899466037750244 + ], + [ + -0.8298236131668091, + -0.7571960687637329, + 2.4112040996551514, + -0.7422088384628296, + 2.5757224559783936, + 2.3544819355010986, + 2.362041711807251, + 1.3503568172454834, + -0.32672879099845886, + 0.36957573890686035, + 0.42405596375465393, + 0.7359752655029297, + -0.19414299726486206, + 1.5960707664489746, + 1.154114007949829, + 0.02908744290471077, + -0.5586186051368713, + -1.3680403232574463, + 1.2902143001556396, + -0.6922940015792847, + -1.7041513919830322 + ], + [ + -0.8196470141410828, + -0.4303642511367798, + 1.8968323469161987, + -0.3930642902851105, + 2.3315858840942383, + 2.9916892051696777, + 2.4671459197998047, + 0.9331362247467041, + -0.4813539385795593, + 0.3563936650753021, + 0.9445043206214905, + 0.8429049849510193, + -0.7790067195892334, + -0.4683154821395874, + 0.867400586605072, + -0.05478433519601822, + -0.5078667402267456, + -1.0063856840133667, + 0.9468156099319458, + -2.225613832473755, + -1.4846243858337402 + ] + ], + "generated_utilities": [ + 0.0132790207862854, + 0.018407067283988, + 0.012948970310389996, + 0.010082741267979145, + 0.010856767185032368, + 0.009846587665379047, + 0.006558021064847708, + 0.012989739887416363 + ], + "group_id": "ms-e2bad6e99e8a281207cc", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13993772864341736, + 0.16482847929000854, + 0.20825156569480896, + 0.16188287734985352, + 0.13454246520996094, + 0.14753298461437225, + 0.14504572749137878, + 0.06676968187093735, + 0.09772863239049911, + 0.0585557222366333, + 0.12863430380821228, + 0.10292482376098633, + 0.07993738353252411, + 0.03641533851623535, + 0.04985213279724121, + 0.1516878604888916 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.3249012231826782, + -0.9610954523086548, + 2.054325819015503, + 1.018629789352417, + 2.393625497817993, + 2.2159433364868164, + 2.098443031311035, + -0.49140021204948425, + 0.044472046196460724, + 0.8045110702514648, + 1.4511574506759644, + -0.3077920377254486, + -0.052587300539016724, + 0.036380887031555176, + 0.7875679731369019, + -0.3928021192550659, + -0.09456783533096313, + -1.4901847839355469, + 1.1711406707763672, + -2.1349995136260986, + -2.076193332672119 + ], + [ + -1.4271517992019653, + -1.114187479019165, + 1.618905782699585, + 1.0621737241744995, + 1.6005314588546753, + 3.017716407775879, + 2.3444156646728516, + -1.0368298292160034, + 0.6155713796615601, + 1.0049463510513306, + 1.3708372116088867, + -0.27396243810653687, + -0.19098177552223206, + -0.5735776424407959, + 0.863328218460083, + -0.4340485632419586, + 0.09502258896827698, + -1.8856427669525146, + 0.9514620304107666, + -2.3912711143493652, + -2.6571128368377686 + ], + [ + -1.0088777542114258, + -1.803802251815796, + 2.6037826538085938, + 0.763943612575531, + 1.7241847515106201, + 2.433204174041748, + 2.273603916168213, + 0.06359493732452393, + -0.17274422943592072, + -0.23299545049667358, + 1.7126038074493408, + -0.3797045052051544, + 0.7567765712738037, + -0.9981052875518799, + 1.0371832847595215, + -0.5120593309402466, + -0.059397101402282715, + -1.9757622480392456, + 0.7378945350646973, + -2.6195693016052246, + -2.178041458129883 + ], + [ + -0.09093856811523438, + -0.14735746383666992, + 1.350311517715454, + 1.477137565612793, + 2.134960174560547, + 1.7331886291503906, + 2.24615740776062, + -1.7148966789245605, + -0.3949584364891052, + 0.4433448612689972, + 0.9034961462020874, + -0.23445925116539001, + -0.6371403932571411, + -0.038229942321777344, + 2.010648250579834, + 0.14472149312496185, + -0.26303571462631226, + -2.3003270626068115, + 0.565727949142456, + -2.586710214614868, + -2.098479747772217 + ], + [ + -1.8506121635437012, + -1.4700038433074951, + 2.4313056468963623, + 1.6264371871948242, + 1.1548577547073364, + 2.9678268432617188, + 2.990251302719116, + -0.25485146045684814, + -0.36889538168907166, + -0.14847010374069214, + 0.6971698999404907, + -0.6752342581748962, + 0.44545215368270874, + 0.663374125957489, + 0.6998850107192993, + -0.6500043869018555, + -0.04876291751861572, + -2.446995258331299, + 0.7760334014892578, + -1.38918137550354, + -1.5372083187103271 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.053252458572387695, + -0.006605625152587891, + 1.0103552341461182, + 0.25736117362976074, + 1.010573148727417, + 1.4690883159637451, + 0.0, + -0.0840916633605957, + -0.24044206738471985, + -0.06801849603652954, + 0.061444878578186035, + 0.11697901785373688, + 0.03979811072349548, + 0.0, + 0.054485201835632324, + -0.11758939921855927, + 0.31161248683929443, + -0.602575421333313, + -0.021049559116363525, + -1.3257782459259033, + 0.0 + ] + ], + "predicted_scores": [ + 5.009342670440674, + 4.948630332946777, + 4.570794105529785, + 4.540657043457031, + 4.3454999923706055, + 4.340945720672607, + 4.03071403503418, + 3.644404888153076 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train1", + "selected_index": 0, + "state_hash": "f054ce13f25dcda8808ebde2bee98ca84bfbdc3fc328a0eb831e21faada3fa4c", + "stored_base_utility": 0.13993772864341736, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09695577621459961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09695577621459961 + }, + "base_progress": 0.09695577621459961, + "base_success": false, + "base_utility": 0.09695577621459961, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.023269018158316612, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023269018158316612 + }, + { + "contact_quality": null, + "progress": 0.019655028358101845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019655028358101845 + }, + { + "contact_quality": null, + "progress": 0.0087813138961792, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0087813138961792 + }, + { + "contact_quality": null, + "progress": 0.00755995512008667, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00755995512008667 + }, + { + "contact_quality": null, + "progress": 0.0067829289473593235, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0067829289473593235 + }, + { + "contact_quality": null, + "progress": 0.007289707660675049, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007289707660675049 + }, + { + "contact_quality": null, + "progress": 0.007393876556307077, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007393876556307077 + }, + { + "contact_quality": null, + "progress": 0.006907304283231497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006907304283231497 + } + ], + "candidate_progress": [ + 0.023269018158316612, + 0.019655028358101845, + 0.0087813138961792, + 0.00755995512008667, + 0.0067829289473593235, + 0.007289707660675049, + 0.007393876556307077, + 0.006907304283231497 + ], + "candidate_source_chart_ids": [ + "ms-206099bab6bfe7c84fce", + "ms-6e27d292a9cbea108fc9", + "ms-6e27d292a9cbea108fc9", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e30fd80adc089e9f697a", + "generated_tangents": [ + [ + 0.6366334557533264, + -0.11214189976453781, + -0.4352327287197113, + 0.7022861242294312, + 0.18992498517036438, + -0.21438473463058472, + -1.4528279304504395, + -0.11280220746994019, + 0.31944581866264343, + -0.01835080236196518, + 1.1632746458053589, + -0.19994431734085083, + -0.3558635711669922, + -0.00420457124710083, + -1.0711610317230225, + 0.19876058399677277, + 1.2058967351913452, + 0.17612312734127045, + -0.0037953853607177734, + 0.8510705828666687, + 0.757475733757019 + ], + [ + 0.6585342288017273, + -0.07911577820777893, + -0.5741208791732788, + 0.9375303983688354, + 0.1638108491897583, + -0.8045713901519775, + -1.2619781494140625, + -0.13599571585655212, + 0.40331730246543884, + 0.05331644415855408, + 1.224998116493225, + -0.17791005969047546, + -0.42211830615997314, + -0.6020296216011047, + -1.0580703020095825, + 0.20305536687374115, + 1.3169106245040894, + 0.062639981508255, + 0.12919948995113373, + 0.728853166103363, + 0.4308769702911377 + ], + [ + 0.12071378529071808, + 0.059990979731082916, + -0.6425563097000122, + 1.2545583248138428, + 0.4109281003475189, + -0.17255926132202148, + -1.140244483947754, + -0.349547415971756, + 0.40955424308776855, + 0.14000645279884338, + 1.4145702123641968, + -0.18921741843223572, + 0.016987502574920654, + -0.194206103682518, + -1.1323565244674683, + 0.17514415085315704, + 1.281287670135498, + 0.19112029671669006, + 0.1002194881439209, + 0.47168827056884766, + 0.5684411525726318 + ], + [ + 1.006769061088562, + -0.12850505113601685, + -0.028462618589401245, + 1.168389081954956, + -0.12616848945617676, + -0.8076877593994141, + -1.4858224391937256, + -0.3880215585231781, + 0.842804491519928, + 0.21694205701351166, + 1.554677128791809, + 0.2051052302122116, + 0.5004428625106812, + -0.5582298636436462, + -1.3873852491378784, + 0.1259801685810089, + 1.7126113176345825, + 0.6904522776603699, + 0.5261470675468445, + 0.9457487463951111, + 1.440118432044983 + ], + [ + 1.0136741399765015, + -0.18631194531917572, + -0.0763501226902008, + 1.2448190450668335, + 0.15769533812999725, + -0.3354029357433319, + -1.5129380226135254, + -0.4687686264514923, + 0.8125385642051697, + 0.3846355378627777, + 1.539689064025879, + 0.115292489528656, + 0.4957970678806305, + -0.7441828846931458, + -1.591048240661621, + 0.0773932933807373, + 1.7895101308822632, + 0.7715845108032227, + 0.3651684820652008, + 0.7716142535209656, + 1.3820078372955322 + ], + [ + 1.105075716972351, + -0.18184205889701843, + -0.16723933815956116, + 0.927869439125061, + 0.06029707193374634, + -1.5241950750350952, + -1.7320661544799805, + -0.46492311358451843, + 0.895631730556488, + 0.45828625559806824, + 1.648598313331604, + 0.16798025369644165, + 0.4141702950000763, + -0.5209448933601379, + -1.7534955739974976, + -0.09653037786483765, + 1.6231496334075928, + 0.5987874865531921, + 0.5790414810180664, + 0.9431485533714294, + 1.6359766721725464 + ], + [ + 0.9705630540847778, + -0.22713980078697205, + 0.02578490972518921, + 1.078351378440857, + -0.0194435715675354, + -0.8012266159057617, + -1.3823676109313965, + -0.4644630253314972, + 0.8608686327934265, + 0.3069694936275482, + 1.529152274131775, + 0.18881826102733612, + 0.41185715794563293, + -0.6967524886131287, + -1.5064692497253418, + 0.028784722089767456, + 1.7600440979003906, + 0.6636291742324829, + 0.4427874684333801, + 0.9378055930137634, + 1.3367422819137573 + ], + [ + 0.8924802541732788, + -0.19990727305412292, + -0.023123115301132202, + 1.1342636346817017, + -0.09567445516586304, + -0.7180756330490112, + -1.235316514968872, + -0.5094244480133057, + 0.9015052318572998, + 0.27604198455810547, + 1.4410655498504639, + 0.30212128162384033, + 0.6821057796478271, + -0.8145237565040588, + -1.5531606674194336, + 0.01960892602801323, + 1.6762187480926514, + 0.7282889485359192, + 0.4589124917984009, + 0.8844152092933655, + 1.1597541570663452 + ] + ], + "generated_utilities": [ + 0.023269018158316612, + 0.019655028358101845, + 0.0087813138961792, + 0.00755995512008667, + 0.0067829289473593235, + 0.007289707660675049, + 0.007393876556307077, + 0.006907304283231497 + ], + "group_id": "ms-e30fd80adc089e9f697a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.12557005882263184, + 0.11941087245941162, + 0.10894306749105453, + 0.060310762375593185, + 0.18920521438121796, + 0.11678676307201385, + 0.14083251357078552, + 0.12988364696502686, + 0.18350017070770264, + 0.1286114901304245, + 0.06744732707738876, + 0.04428015276789665, + 0.08399941772222519, + 0.0742691159248352, + 0.05598282814025879, + 0.09703926742076874 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.044197678565979004, + -0.12298870831727982, + -0.37176433205604553, + 0.5503368377685547, + 0.5411782264709473, + 0.6186751127243042, + 0.0, + 0.28674131631851196, + 0.08053566515445709, + 0.3295105993747711, + 0.8157306909561157, + 0.1715114712715149, + -0.43090736865997314, + 0.0, + -0.5639103651046753, + -0.48903688788414, + 0.7485904693603516, + -0.4474331736564636, + -0.18787053227424622, + 1.2062294483184814, + 0.0 + ], + [ + 1.3153167963027954, + 0.7455203533172607, + -1.0113484859466553, + 2.097999334335327, + 0.7455009818077087, + 0.3843202590942383, + -0.16713692247867584, + -0.8400754928588867, + -0.3845864534378052, + 1.065396785736084, + 1.712794542312622, + 0.803345263004303, + 0.6529266238212585, + 0.6022698879241943, + -1.1222561597824097, + 0.06260442733764648, + 1.101211428642273, + -0.133493572473526, + -0.8626402616500854, + 1.8406728506088257, + 0.8011860847473145 + ], + [ + 0.23532289266586304, + -0.516789972782135, + -0.3119599223136902, + 1.501338005065918, + 1.4930639266967773, + -0.419913113117218, + -0.6910054683685303, + 0.30246037244796753, + 0.041916340589523315, + 1.2685117721557617, + 2.5912232398986816, + 1.2864888906478882, + 0.3821069598197937, + 0.4677737355232239, + -1.84663724899292, + -0.8953573703765869, + 2.3256053924560547, + 0.18948839604854584, + -0.5580936670303345, + 1.528401494026184, + 1.4914395809173584 + ], + [ + 0.6903802752494812, + 0.3962534964084625, + -0.8207939863204956, + 0.5973623991012573, + 1.310564398765564, + 0.6411572694778442, + -0.2032768577337265, + 0.25504088401794434, + 0.6581551432609558, + 0.3938753306865692, + 1.0127826929092407, + 1.6796927452087402, + 0.4867250621318817, + 0.05769670009613037, + -2.026149034500122, + -0.17722094058990479, + 0.9414948225021362, + 0.6044259667396545, + 0.03290170431137085, + 2.9423327445983887, + -0.060306549072265625 + ], + [ + 0.47530579566955566, + -0.6085019111633301, + 0.12691906094551086, + 0.5673632025718689, + 0.08076304197311401, + 0.8255819082260132, + -0.9781221151351929, + -0.796129584312439, + -0.6745100021362305, + 0.7784937024116516, + 1.7613102197647095, + 1.4703174829483032, + -0.36175650358200073, + 0.16939619183540344, + -0.9399533271789551, + -1.2696547508239746, + 1.3314080238342285, + -0.3690394163131714, + -1.0323668718338013, + 2.305772304534912, + 0.27392566204071045 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.6770331859588623, + 0.09077263623476028, + -0.3141186833381653, + 1.5503368377685547, + 1.0133107900619507, + 0.12400172650814056, + -0.2399108111858368, + -0.4914711117744446, + -0.11917559802532196, + 0.7716305255889893, + 1.8157306909561157, + 0.7167991995811462, + 0.5690926313400269, + 0.6154549717903137, + -1.5639103651046753, + -0.5166093111038208, + 1.7485904693603516, + -0.15158627927303314, + -0.46638885140419006, + 2.2062294483184814, + 0.7902159690856934 + ], + [ + 0.9456368684768677, + -0.3964601159095764, + -0.5845282077789307, + 1.2133667469024658, + 0.5843623876571655, + -0.02685314416885376, + -0.006928771734237671, + -0.2159697711467743, + -0.3017333149909973, + 0.40500110387802124, + 1.9734654426574707, + 0.46303731203079224, + 0.33380845189094543, + 0.7758987545967102, + -2.060479164123535, + -0.955610990524292, + 1.942940592765808, + -0.23636659979820251, + -0.10507094860076904, + 1.8367226123809814, + 1.2234702110290527 + ] + ], + "predicted_scores": [ + 5.982050895690918, + 5.516965866088867, + 4.392928600311279, + 4.120024681091309, + 4.05231237411499, + 3.795178174972534, + 3.6413815021514893, + 3.2022411823272705 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train1", + "selected_index": 0, + "state_hash": "ca8873454d7cf9723b1e743355bf255813544ae3212d80abc954fd347d7adf27", + "stored_base_utility": 0.12557005882263184, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07411724328994751, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07411724328994751 + }, + { + "contact_quality": null, + "progress": 0.09047380834817886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09047380834817886 + }, + { + "contact_quality": null, + "progress": 0.10569536685943604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10569536685943604 + }, + { + "contact_quality": null, + "progress": 0.14950889348983765, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14950889348983765 + }, + { + "contact_quality": null, + "progress": 0.09969130903482437, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09969130903482437 + }, + { + "contact_quality": null, + "progress": 0.08115202188491821, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08115202188491821 + }, + { + "contact_quality": null, + "progress": 0.13061003386974335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13061003386974335 + }, + { + "contact_quality": null, + "progress": 0.08743151277303696, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08743151277303696 + } + ], + "candidate_progress": [ + 0.07411724328994751, + 0.09047380834817886, + 0.10569536685943604, + 0.14950889348983765, + 0.09969130903482437, + 0.08115202188491821, + 0.13061003386974335, + 0.08743151277303696 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "generated_tangents": [ + [ + 1.6360913515090942, + -0.18474355340003967, + -0.03315272927284241, + -0.35874706506729126, + -0.23833909630775452, + 2.607821464538574, + 1.9053452014923096, + 2.068326711654663, + 0.44662436842918396, + -0.9570363759994507, + -0.5049992799758911, + -0.6835821270942688, + -2.8723480701446533, + -1.9566744565963745, + 2.0397720336914062, + 0.3789702355861664, + -1.0628761053085327, + 0.10153849422931671, + 0.13642095029354095, + -1.9212422370910645, + -1.6213682889938354 + ], + [ + 1.6279715299606323, + -0.04317447543144226, + -0.33085086941719055, + -0.22415505349636078, + -0.4503195285797119, + 2.551999568939209, + 1.8113257884979248, + 2.193518877029419, + 0.39876192808151245, + -1.102653980255127, + -0.5429314970970154, + -1.158248782157898, + -2.6006979942321777, + -2.550006628036499, + 1.8820801973342896, + 0.12466619908809662, + -1.0567195415496826, + 0.018112868070602417, + -0.06478580832481384, + -2.0482516288757324, + -1.9957387447357178 + ], + [ + 1.6535975933074951, + -0.0813259556889534, + -0.30771657824516296, + -0.26517102122306824, + -0.40425264835357666, + 2.509902238845825, + 2.004323720932007, + 2.105619192123413, + 0.40929552912712097, + -0.9626333713531494, + -0.636191189289093, + -0.9176632165908813, + -2.89888858795166, + -2.5282859802246094, + 1.9083274602890015, + 0.1351604163646698, + -0.9505917429924011, + 0.09362629055976868, + -0.04731979966163635, + -1.804837942123413, + -1.690934181213379 + ], + [ + 1.7347869873046875, + -0.1579766422510147, + -0.4947598874568939, + -0.055146366357803345, + -0.4505763649940491, + 2.7247345447540283, + 1.3791768550872803, + 2.0401041507720947, + 0.31636542081832886, + -1.0958385467529297, + -0.6046792268753052, + -0.8150632381439209, + -2.4669017791748047, + -1.9254041910171509, + 1.8212796449661255, + 0.2587243318557739, + -0.9763078093528748, + -0.07706935703754425, + -0.25398099422454834, + -1.5144741535186768, + -1.5001959800720215 + ], + [ + 1.7521071434020996, + -0.11762430518865585, + -0.216354638338089, + -0.23457713425159454, + -0.18162685632705688, + 2.2410361766815186, + 2.021794557571411, + 2.1394143104553223, + 0.4362010955810547, + -1.0363415479660034, + -0.697080671787262, + -0.9890108108520508, + -2.9245851039886475, + -2.2857704162597656, + 1.9089828729629517, + 0.16052402555942535, + -0.9754478335380554, + 0.13712504506111145, + -0.11094871163368225, + -1.9151535034179688, + -1.7413411140441895 + ], + [ + 1.4861899614334106, + -0.15332084894180298, + -0.3576917350292206, + -0.24738585948944092, + -0.22742795944213867, + 2.1283457279205322, + 1.9224722385406494, + 2.095451831817627, + 0.44908690452575684, + -1.2427468299865723, + -0.548572301864624, + -1.1474500894546509, + -2.6523094177246094, + -2.038431406021118, + 1.9554672241210938, + 0.2681941092014313, + -0.8971112370491028, + 0.2167607545852661, + -0.17372861504554749, + -1.7628464698791504, + -1.8130526542663574 + ], + [ + 1.969941258430481, + -0.055220190435647964, + -0.21834620833396912, + -0.3075636625289917, + -0.4150499701499939, + 2.355985164642334, + 2.297492265701294, + 2.101154327392578, + 0.2317568063735962, + -1.137794017791748, + -0.5858346819877625, + -1.0449132919311523, + -3.045583963394165, + -2.1876230239868164, + 1.9643102884292603, + 0.13033004105091095, + -0.7871422171592712, + -0.08854912221431732, + 0.10471479594707489, + -1.6661694049835205, + -1.8294498920440674 + ], + [ + 2.2868597507476807, + -0.15640106797218323, + 0.23856395483016968, + -0.0673302561044693, + 0.002996191382408142, + 1.9117286205291748, + 1.9574682712554932, + 2.075385808944702, + 0.137465700507164, + -1.3756444454193115, + -0.7577681541442871, + -0.8775328397750854, + -2.7816855907440186, + -2.069620370864868, + 1.7326915264129639, + -0.07838009297847748, + -0.7329941391944885, + 0.2459639608860016, + -0.30372434854507446, + -1.9334983825683594, + -1.538361668586731 + ] + ], + "generated_utilities": [ + 0.07411724328994751, + 0.09047380834817886, + 0.10569536685943604, + 0.14950889348983765, + 0.09969130903482437, + 0.08115202188491821, + 0.13061003386974335, + 0.08743151277303696 + ], + "group_id": "ms-e9ed5d488afdfd6449d1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.22621813416481018, + 0.26591962575912476, + 0.22478708624839783, + 0.074407659471035, + 0.3134628236293793, + 0.19173070788383484, + 0.2438776195049286, + 0.26689577102661133, + 0.24477621912956238, + 0.24051737785339355, + 0.2738616168498993, + 0.04947046563029289, + 2.0, + 0.04723116010427475, + 0.08354459702968597, + 0.06540697813034058 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.6524851322174072, + -0.023243099451065063, + -0.05794792249798775, + 0.1783304661512375, + -0.3767814338207245, + 1.4893875122070312, + 0.0, + 1.024657964706421, + 0.3163401186466217, + -0.49971193075180054, + 0.05114656686782837, + -0.13273626565933228, + -1.9796202182769775, + 0.0, + 0.818771481513977, + -0.13705189526081085, + -0.21289467811584473, + 0.3914462625980377, + -0.6477798819541931, + -1.0717217922210693, + 0.0 + ], + [ + 2.363477945327759, + -1.042197585105896, + -0.8369066119194031, + 0.17341986298561096, + 0.6040215492248535, + 1.9536304473876953, + 2.2788424491882324, + 1.5015904903411865, + -0.07916292548179626, + -0.6816228628158569, + -1.3893373012542725, + -0.2375274896621704, + -2.5662975311279297, + -2.7615673542022705, + 1.151218295097351, + 0.29418760538101196, + -0.4583754539489746, + -0.4555256962776184, + -0.6620294451713562, + -2.534846067428589, + -1.916935682296753 + ], + [ + 2.0104916095733643, + -0.539351224899292, + 0.13553066551685333, + 0.9507300853729248, + 0.6636125445365906, + 1.5649433135986328, + 2.0495975017547607, + 2.832026958465576, + 1.1830073595046997, + -1.6508865356445312, + -0.26773035526275635, + -0.3988863527774811, + -2.8235108852386475, + -2.2897520065307617, + 1.4448039531707764, + -0.5226297378540039, + -2.1615242958068848, + 0.8214395046234131, + -0.755327045917511, + -2.571093797683716, + -1.7457375526428223 + ], + [ + 2.166736602783203, + -0.3969564735889435, + 0.5171664357185364, + 0.12789201736450195, + -0.9176173806190491, + 1.8307013511657715, + 1.834255576133728, + 1.3594005107879639, + 0.13751044869422913, + -0.9923363327980042, + -1.136932611465454, + -1.2923126220703125, + -2.445897102355957, + -2.4779889583587646, + 2.6843042373657227, + 0.10015779733657837, + -2.0766196250915527, + 0.1269409954547882, + 0.16698843240737915, + -1.2709240913391113, + -2.7113101482391357 + ], + [ + 1.0476312637329102, + -1.2484432458877563, + -0.8652145862579346, + 0.8706620335578918, + 0.6091254353523254, + 1.9634168148040771, + 2.105522394180298, + 1.2906999588012695, + -0.3275601267814636, + -1.0110361576080322, + -1.6514918804168701, + -1.376278281211853, + -2.345550060272217, + -2.4931728839874268, + 1.00270676612854, + 0.04506785422563553, + -0.9420130252838135, + -0.6248484253883362, + -1.469360589981079, + -3.059006929397583, + -3.495709180831909 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.6524851322174072, + -0.3197403848171234, + -0.07008056342601776, + 0.38692954182624817, + -0.013657404109835625, + 2.4893875122070312, + 2.3340368270874023, + 2.024657964706421, + 0.35216206312179565, + -1.4125897884368896, + -0.8262880444526672, + -0.49936625361442566, + -2.9796202182769775, + -2.9234721660614014, + 1.818771481513977, + 0.09626653790473938, + -1.2128946781158447, + 0.10123217105865479, + -0.7020833492279053, + -2.0717217922210693, + -2.5944693088531494 + ], + [ + 0.9812780022621155, + 0.2612687647342682, + 0.8081930875778198, + 0.3689844310283661, + -0.16453352570533752, + 2.8411240577697754, + 2.2401933670043945, + 1.7904633283615112, + -0.04837304353713989, + -0.6139949560165405, + -0.8549875020980835, + 0.3574656546115875, + -2.8409454822540283, + -2.1011672019958496, + 1.382948875427246, + 0.6656897068023682, + -1.5090553760528564, + 0.4825518727302551, + -0.3345044255256653, + -2.26375150680542, + -2.561906576156616 + ] + ], + "predicted_scores": [ + 4.722866058349609, + 4.667067050933838, + 4.577579498291016, + 4.560670375823975, + 4.475306510925293, + 4.332956314086914, + 3.9019358158111572, + 3.673485279083252 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train1", + "selected_index": 0, + "state_hash": "5608eea5b7e99b5fed6b316f742ecd34044df87286b2338705d8472043fc4375", + "stored_base_utility": 0.22621813416481018, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09514755010604858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09514755010604858 + }, + "base_progress": 0.09514755010604858, + "base_success": false, + "base_utility": 0.09514755010604858, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.042074382305145264, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.042074382305145264 + }, + { + "contact_quality": null, + "progress": 0.0623781681060791, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0623781681060791 + }, + { + "contact_quality": null, + "progress": 0.0336853452026844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0336853452026844 + }, + { + "contact_quality": null, + "progress": 0.00692981481552124, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00692981481552124 + }, + { + "contact_quality": null, + "progress": 0.03991667553782463, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03991667553782463 + }, + { + "contact_quality": null, + "progress": 0.03802645206451416, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03802645206451416 + }, + { + "contact_quality": null, + "progress": 0.07131137698888779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07131137698888779 + }, + { + "contact_quality": null, + "progress": 0.0379769392311573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0379769392311573 + } + ], + "candidate_progress": [ + 0.042074382305145264, + 0.0623781681060791, + 0.0336853452026844, + 0.00692981481552124, + 0.03991667553782463, + 0.03802645206451416, + 0.07131137698888779, + 0.0379769392311573 + ], + "candidate_source_chart_ids": [ + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ec013396e4aff6170ee1", + "generated_tangents": [ + [ + 0.4611658453941345, + -0.2873786985874176, + -0.07860538363456726, + 0.752954363822937, + 0.3158806264400482, + -1.1932144165039062, + -0.1740119457244873, + -0.6403985023498535, + 0.8440989255905151, + 1.2669352293014526, + -0.9001718759536743, + -0.8188923597335815, + -0.23858115077018738, + 1.0344558954238892, + -1.2292507886886597, + 0.6932545900344849, + 0.7712217569351196, + 0.6993335485458374, + 0.37747806310653687, + -2.2667934894561768, + -0.9057581424713135 + ], + [ + 0.39145201444625854, + -0.343828022480011, + 0.15941423177719116, + 0.7066751718521118, + 0.14841711521148682, + -0.40486249327659607, + -0.396170973777771, + -1.064365267753601, + 0.946660578250885, + 1.1499437093734741, + 0.005964670330286026, + -1.1780930757522583, + -0.300137460231781, + 0.7000108957290649, + -1.237982988357544, + 0.7131509780883789, + 0.7316395044326782, + 0.7953580021858215, + -0.03141322731971741, + -1.3973627090454102, + -0.8751441240310669 + ], + [ + 0.4933302402496338, + -0.44363027811050415, + -0.5209032297134399, + 0.6641716957092285, + 0.1200324147939682, + -0.9006333351135254, + -0.2204020619392395, + -0.599134087562561, + 0.9412590265274048, + 1.5974762439727783, + -0.8537541031837463, + -0.7195894122123718, + 0.21629825234413147, + 0.8109884262084961, + -0.9227805733680725, + 0.6385137438774109, + 0.8771730065345764, + 0.5583455562591553, + 0.38907456398010254, + -2.736330986022949, + -0.6876829862594604 + ], + [ + 0.580592930316925, + -0.4862269163131714, + -0.2889409363269806, + 0.5998220443725586, + -0.006053805351257324, + -1.1369870901107788, + 0.5363792777061462, + -0.7946500778198242, + 1.0766104459762573, + 1.6346819400787354, + -1.053938388824463, + -0.575809895992279, + -0.29470497369766235, + 0.9115384817123413, + -0.5781656503677368, + 0.5710615515708923, + 0.7287531495094299, + 0.40843814611434937, + 0.09897521138191223, + -2.882838249206543, + -0.6922982931137085 + ], + [ + 0.4615640640258789, + -0.5395740866661072, + -0.4543243944644928, + 0.7728146314620972, + -0.055400550365448, + -0.8873399496078491, + -0.1899268627166748, + -0.7802066802978516, + 0.9693182706832886, + 1.6709871292114258, + -0.8543050289154053, + -0.7539804577827454, + 0.06042942404747009, + 0.8117468357086182, + -1.0044642686843872, + 0.688340961933136, + 0.9000950455665588, + 0.5628482103347778, + 0.24863645434379578, + -2.7251012325286865, + -0.5936688184738159 + ], + [ + 0.469229519367218, + -0.49117475748062134, + -0.5210379362106323, + 0.7249901294708252, + 0.04420997202396393, + -0.7754863500595093, + -0.16021692752838135, + -0.704777717590332, + 0.9539754390716553, + 1.630562663078308, + -0.8701502084732056, + -0.7330318093299866, + 0.02480393648147583, + 0.8277038931846619, + -0.9628077149391174, + 0.6552332043647766, + 0.9154318571090698, + 0.545397937297821, + 0.26340970396995544, + -2.6805334091186523, + -0.6075135469436646 + ], + [ + 0.19687849283218384, + -0.6519871950149536, + 0.2553129196166992, + 0.574405312538147, + 0.1867075264453888, + -0.6016026735305786, + -0.4330863356590271, + -0.9723740816116333, + 1.0654808282852173, + 0.950093150138855, + -0.17083856463432312, + -1.5525258779525757, + -0.48422351479530334, + 0.5935631990432739, + -1.1726895570755005, + 0.6031124591827393, + 0.8103089332580566, + 0.9438478350639343, + -0.31401270627975464, + -1.7797350883483887, + -0.5831897258758545 + ], + [ + 0.41737931966781616, + -0.540668785572052, + -0.6665561199188232, + 0.5068308115005493, + 0.22947601974010468, + -0.8732736110687256, + -0.032761573791503906, + -0.6664251089096069, + 0.9593477845191956, + 1.7728954553604126, + -1.0882853269577026, + -0.7038149833679199, + 0.3145146667957306, + 0.8262359499931335, + -1.0099703073501587, + 0.7345770001411438, + 0.9556244015693665, + 0.45601099729537964, + 0.2565111815929413, + -3.041599750518799, + -0.5785009860992432 + ] + ], + "generated_utilities": [ + 0.042074382305145264, + 0.0623781681060791, + 0.0336853452026844, + 0.00692981481552124, + 0.03991667553782463, + 0.03802645206451416, + 0.07131137698888779, + 0.0379769392311573 + ], + "group_id": "ms-ec013396e4aff6170ee1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.010581731796264648, + 0.049839019775390625, + 0.06402494758367538, + 0.012959301471710205, + 0.11614581197500229, + 0.012509982101619244, + 0.09607907384634018, + 0.1605721116065979, + 0.1708059310913086, + 0.055588722229003906, + 0.2052522599697113, + 0.0690404623746872, + 0.11059975624084473, + 0.03047112748026848, + 0.24502670764923096, + 0.22897732257843018 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.16510963439941406, + -0.011389851570129395, + -0.16074393689632416, + 0.8547754287719727, + -0.09704409539699554, + 0.09471678733825684, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + -0.29076793789863586, + 0.0122823566198349, + 0.059903502464294434, + 0.0, + 0.13894814252853394, + 0.2961620092391968, + 0.025543689727783203, + 0.01881265640258789, + 0.052319109439849854, + -1.1815040111541748, + 0.0 + ], + [ + -0.6906574964523315, + 0.5966935753822327, + 0.1873793751001358, + 1.8547754287719727, + 0.16492335498332977, + 0.4949548840522766, + 0.624785304069519, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233, + -0.44174641370773315, + 0.866999626159668, + 1.0255436897277832, + 1.018812656402588, + 0.6033775210380554, + -2.181504011154175, + -0.4328075349330902 + ], + [ + -0.68684321641922, + 0.35380983352661133, + 0.41641929745674133, + 2.0589241981506348, + 0.20610027015209198, + 0.2780776619911194, + 0.6000017523765564, + -1.8711607456207275, + 1.71079421043396, + 2.484286069869995, + -0.12494567036628723, + 0.3767848014831543, + -0.59480881690979, + 1.3215627670288086, + -0.18650269508361816, + 0.5971667766571045, + 1.00047767162323, + 1.2905888557434082, + 0.7682754993438721, + -2.280778408050537, + -0.26339399814605713 + ], + [ + -1.0669870376586914, + 0.5963312387466431, + 0.24800772964954376, + 1.5324974060058594, + 0.049914538860321045, + 0.33535104990005493, + 0.9774697422981262, + -2.318913698196411, + 1.14731764793396, + 2.1673643589019775, + -0.3369846045970917, + -0.13161931931972504, + -0.7527784705162048, + 1.2647147178649902, + -0.5229569673538208, + 0.6999921798706055, + 0.654599666595459, + 0.9745514988899231, + 0.5995839238166809, + -2.3844704627990723, + -0.7404599189758301 + ], + [ + -0.660759449005127, + 0.560627818107605, + -0.19398994743824005, + 1.7401045560836792, + 0.006708189845085144, + 0.2270365059375763, + 0.9201371073722839, + -1.6917380094528198, + 1.8382139205932617, + 2.184737205505371, + -0.07137502729892731, + 0.13004565238952637, + -0.8376798629760742, + 1.3597201108932495, + -0.8258764147758484, + 0.521450400352478, + 1.4799309968948364, + 0.7742871046066284, + 1.007819414138794, + -2.398472785949707, + -0.7120757102966309 + ], + [ + -0.3511212170124054, + 0.6072589755058289, + 0.6547060012817383, + 2.306784152984619, + 0.03337647020816803, + 1.1676279306411743, + 0.48420602083206177, + -1.7057392597198486, + 1.7822117805480957, + 2.741852283477783, + -0.970782995223999, + 0.3188708424568176, + -0.1370975375175476, + 1.740339994430542, + 0.18519049882888794, + 0.2771400809288025, + 0.8003557920455933, + 0.8813821077346802, + 0.9829010963439941, + -1.9104408025741577, + -0.7000454664230347 + ], + [ + -1.1815872192382812, + 1.0524637699127197, + 0.5610368251800537, + 2.407832622528076, + -0.2576238512992859, + 0.1467430293560028, + 0.27881303429603577, + -1.6810011863708496, + 1.5921515226364136, + 2.929525375366211, + 0.19559642672538757, + 0.6552202105522156, + -1.3889155387878418, + 1.1560360193252563, + -0.4104887843132019, + 0.8887493014335632, + 1.283969759941101, + 0.7811828851699829, + -0.10630232095718384, + -1.4329519271850586, + -0.5646982192993164 + ], + [ + -1.4926873445510864, + 0.11535564064979553, + -0.6240940690040588, + 1.631784200668335, + 0.7930782437324524, + -0.35493338108062744, + 0.20104485750198364, + -1.5626122951507568, + 1.3110854625701904, + 3.0686309337615967, + -0.8419785499572754, + -0.0010939836502075195, + -0.49860918521881104, + 1.5949748754501343, + -0.938766598701477, + 1.762406587600708, + 0.625543475151062, + 0.20789378881454468, + 1.1513795852661133, + -2.3980860710144043, + -0.36970049142837524 + ], + [ + -1.0271471738815308, + 0.6133835911750793, + -0.3870598077774048, + 1.9299802780151367, + 0.9606215357780457, + -0.19876420497894287, + 0.29684266448020935, + -1.6164112091064453, + 1.3925114870071411, + 1.9474635124206543, + -1.2669438123703003, + -0.6821421980857849, + -0.7869906425476074, + 1.3602250814437866, + -0.14778050780296326, + 0.9938437938690186, + 0.6618988513946533, + 0.1858283281326294, + 0.8312947750091553, + -1.7043554782867432, + -0.23048581182956696 + ], + [ + -0.409736305475235, + 1.5572996139526367, + 1.0024399757385254, + 1.6385810375213623, + 0.83731609582901, + -0.1629825234413147, + -0.05952054262161255, + -1.0968945026397705, + 0.728961706161499, + 3.1370975971221924, + -0.44591856002807617, + 1.149456262588501, + -1.3439809083938599, + 1.738822340965271, + 0.4063607454299927, + 0.4372418224811554, + 1.276847004890442, + 1.4612212181091309, + 0.6917726993560791, + -1.914740800857544, + 0.5415292978286743 + ] + ], + "predicted_scores": [ + 3.913698673248291, + 3.4669954776763916, + 3.412660837173462, + 3.3919060230255127, + 3.289900302886963, + 3.232205867767334, + 3.2286953926086426, + 3.156780242919922 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296/train1", + "selected_index": 0, + "state_hash": "f5a19cf07e6a6cc71c95ce01e976d34d1eeae68c8a93ecc985fd0a78cb0c1a5e", + "stored_base_utility": 0.010581731796264648, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.010638435371220112, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010638435371220112 + }, + { + "contact_quality": null, + "progress": 0.008713643066585064, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008713643066585064 + }, + { + "contact_quality": null, + "progress": 0.00959573220461607, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00959573220461607 + }, + { + "contact_quality": null, + "progress": 0.008243958465754986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008243958465754986 + }, + { + "contact_quality": null, + "progress": 0.011892616748809814, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011892616748809814 + }, + { + "contact_quality": null, + "progress": 0.015581388957798481, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015581388957798481 + }, + { + "contact_quality": null, + "progress": 0.01546515990048647, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01546515990048647 + }, + { + "contact_quality": null, + "progress": 0.014228303916752338, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014228303916752338 + } + ], + "candidate_progress": [ + 0.010638435371220112, + 0.008713643066585064, + 0.00959573220461607, + 0.008243958465754986, + 0.011892616748809814, + 0.015581388957798481, + 0.01546515990048647, + 0.014228303916752338 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-003a2225c0892dcda683", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-0285f5815c210ac00bec", + "ms-0285f5815c210ac00bec", + "ms-be219977a9db8dbb1e58" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ee30ebaf98622759d888", + "generated_tangents": [ + [ + 0.8207482695579529, + -0.7308071255683899, + 0.4467986226081848, + -0.5334179401397705, + 2.624948263168335, + 0.5768913626670837, + 1.3773689270019531, + 0.401658833026886, + -0.25230899453163147, + 0.5512906908988953, + 0.06494399905204773, + 0.5098161101341248, + -1.6437782049179077, + -0.833240807056427, + -0.7026548385620117, + 0.3165251314640045, + 1.241769790649414, + -0.34677284955978394, + 1.7037112712860107, + -2.0828754901885986, + -1.2344740629196167 + ], + [ + 0.7710335850715637, + -0.7674400806427002, + 0.5049636960029602, + -0.6269207000732422, + 2.7183949947357178, + 0.6444479823112488, + 1.2842581272125244, + 0.3312671482563019, + -0.23908039927482605, + 0.5727101564407349, + 0.09744159877300262, + 0.4604488015174866, + -1.661820411682129, + -0.7683685421943665, + -0.7152273654937744, + 0.27507102489471436, + 1.1888011693954468, + -0.3454775810241699, + 1.620046615600586, + -2.0872716903686523, + -1.3310085535049438 + ], + [ + 0.6475136280059814, + -0.8064519762992859, + 0.15487346053123474, + -0.6583085656166077, + 3.0244264602661133, + 0.6566344499588013, + 1.0700241327285767, + 0.1811072677373886, + 0.10540540516376495, + 0.697022020816803, + -0.19983208179473877, + -0.18796342611312866, + -1.7084141969680786, + -0.6339097023010254, + -0.7315196394920349, + 0.42225131392478943, + 1.3073558807373047, + -0.5681096911430359, + 1.7822608947753906, + -1.9828650951385498, + -0.6053508520126343 + ], + [ + 0.29915285110473633, + -0.8396425843238831, + 0.20700645446777344, + -0.414063036441803, + 2.8157193660736084, + 0.6158472895622253, + 0.6684472560882568, + 0.008738711476325989, + -0.05560358613729477, + 0.3467683792114258, + -0.06346379220485687, + 0.3750460743904114, + -1.9538394212722778, + -0.40554577112197876, + -0.431382954120636, + 0.3815584182739258, + 1.0538214445114136, + -0.5597825646400452, + 1.3493223190307617, + -2.371721029281616, + -0.935927152633667 + ], + [ + 0.41692185401916504, + -0.8120754957199097, + 0.8173114657402039, + -0.4108160734176636, + 2.753605365753174, + -0.0567987859249115, + 1.0259952545166016, + 0.23923759162425995, + 0.10675296187400818, + 0.27663564682006836, + -0.009385555982589722, + -0.25673994421958923, + -1.5019313097000122, + -1.0777069330215454, + -0.17191405594348907, + 0.5699157118797302, + 1.1479648351669312, + -0.564826250076294, + 1.2419342994689941, + -2.078266143798828, + -1.4185867309570312 + ], + [ + 0.9519003033638, + -0.8409497737884521, + 0.5436072945594788, + -0.33745941519737244, + 2.6066904067993164, + 1.4436250925064087, + 1.1239246129989624, + 0.553543210029602, + -0.11368918418884277, + 0.634536623954773, + 0.004998352378606796, + 0.24750663340091705, + -1.602338194847107, + -0.6645449995994568, + -0.3664837181568146, + 0.25969475507736206, + 0.9689200520515442, + -0.46969062089920044, + 1.658595085144043, + -2.1279382705688477, + -1.1879879236221313 + ], + [ + 0.9801964163780212, + -0.8598592281341553, + 0.5351187586784363, + -0.3519566059112549, + 2.586333751678467, + 1.453399896621704, + 1.0645297765731812, + 0.5686471462249756, + -0.10753000527620316, + 0.6423960328102112, + -0.048477739095687866, + 0.23440031707286835, + -1.6481211185455322, + -0.7002952098846436, + -0.3652022182941437, + 0.24543322622776031, + 1.0314792394638062, + -0.382108211517334, + 1.6628272533416748, + -2.21238374710083, + -1.1133114099502563 + ], + [ + 0.6216107606887817, + -0.5996929407119751, + 0.2552225589752197, + -0.4646618962287903, + 1.9257380962371826, + 0.7422885894775391, + 1.1481587886810303, + 0.2559194564819336, + 0.05205418914556503, + 0.47481974959373474, + 0.2907453179359436, + -0.0073999762535095215, + -1.6014293432235718, + -0.3677322268486023, + -0.4897036552429199, + 0.12193399667739868, + 1.0392929315567017, + -0.4940047264099121, + 1.0735681056976318, + -2.2955286502838135, + -1.1116067171096802 + ] + ], + "generated_utilities": [ + 0.010638435371220112, + 0.008713643066585064, + 0.00959573220461607, + 0.008243958465754986, + 0.011892616748809814, + 0.015581388957798481, + 0.01546515990048647, + 0.014228303916752338 + ], + "group_id": "ms-ee30ebaf98622759d888", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08405601978302002, + 0.09846439212560654, + 0.04595305770635605, + 2.0, + 0.08787640184164047, + 0.032481253147125244, + 0.08755072206258774, + 0.058936577290296555, + 0.10809408128261566, + 0.06723630428314209, + 0.0625009536743164, + 0.1415095031261444, + 0.08326160907745361, + 0.10944461822509766, + 0.0673408955335617, + 0.033725105226039886 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.4581542015075684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9406367540359497, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.470216989517212 + ], + [ + 1.6340367794036865, + -0.30366095900535583, + 0.2830277681350708, + -1.1627482175827026, + 3.58767032623291, + 0.6565636396408081, + 1.4321975708007812, + 0.8704822659492493, + 0.14327874779701233, + 1.0190284252166748, + 0.3078446090221405, + 0.7927982807159424, + -1.9305998086929321, + -1.7108646631240845, + 0.10729888081550598, + 0.9427294731140137, + 0.41770225763320923, + -1.7673753499984741, + 1.3209179639816284, + -2.8053033351898193, + -1.5330681800842285 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.00628507137298584, + 0.0595628023147583, + 0.39481037855148315, + -0.006641983985900879, + 1.8433213233947754, + 0.14830267429351807, + 0.0, + -0.24425631761550903, + 0.42130333185195923, + 0.41399094462394714, + 0.478841632604599, + -0.007461488246917725, + -0.5437495708465576, + 0.0, + 0.255215585231781, + -0.07331699132919312, + -0.1736500859260559, + -0.285358190536499, + 0.567956805229187, + -1.7103407382965088, + 0.0 + ], + [ + 1.0371580123901367, + -0.33467406034469604, + 0.9327473640441895, + -0.5396467447280884, + 2.1487879753112793, + 0.6294846534729004, + 0.6252679228782654, + 0.5140941143035889, + -0.20645928382873535, + -0.04528486728668213, + 0.14780870079994202, + 0.430325984954834, + -0.9293508529663086, + -1.5147639513015747, + 1.1104007959365845, + -0.05476301908493042, + 0.1755150556564331, + -1.3470268249511719, + 1.675830364227295, + -3.2944674491882324, + -1.9277915954589844 + ] + ], + "predicted_scores": [ + 5.316679000854492, + 5.273787975311279, + 5.226407527923584, + 4.974737644195557, + 4.743207931518555, + 4.480126857757568, + 4.326788902282715, + 4.008129119873047 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "636680137/train1", + "selected_index": 0, + "state_hash": "2de607a9c69005130e51ff2b551c0eeedc85c58fef796682e6527bb5f719ff7b", + "stored_base_utility": 0.08405601978302002, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.026694953441619873, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026694953441619873 + }, + "base_progress": 0.026694953441619873, + "base_success": false, + "base_utility": 0.026694953441619873, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07018190622329712, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07018190622329712 + }, + { + "contact_quality": null, + "progress": 0.0666942223906517, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0666942223906517 + }, + { + "contact_quality": null, + "progress": 0.055874746292829514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.055874746292829514 + }, + { + "contact_quality": null, + "progress": 0.04966646432876587, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04966646432876587 + }, + { + "contact_quality": null, + "progress": 0.05415312573313713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05415312573313713 + }, + { + "contact_quality": null, + "progress": 0.07510259002447128, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07510259002447128 + }, + { + "contact_quality": null, + "progress": 0.060902971774339676, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060902971774339676 + }, + { + "contact_quality": null, + "progress": 0.060523372143507004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060523372143507004 + } + ], + "candidate_progress": [ + 0.07018190622329712, + 0.0666942223906517, + 0.055874746292829514, + 0.04966646432876587, + 0.05415312573313713, + 0.07510259002447128, + 0.060902971774339676, + 0.060523372143507004 + ], + "candidate_source_chart_ids": [ + "ms-1a4979e969aa331fb126", + "ms-53a86042af3c0887e556", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-38dff128761905bb1507" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-f3f1d07cfd59b823da47", + "generated_tangents": [ + [ + 1.470210075378418, + -0.01580379530787468, + -0.7283998727798462, + 0.055736392736434937, + 0.24472132325172424, + -1.1284493207931519, + -0.8600251078605652, + 0.3851696252822876, + 0.3760157823562622, + 0.12302403151988983, + -0.6882843971252441, + 0.061757877469062805, + -0.3473249077796936, + -0.3466219902038574, + 0.0132053904235363, + -0.13766108453273773, + 0.5573180913925171, + -0.3471088409423828, + 0.34647753834724426, + -0.36234331130981445, + 0.2744932174682617 + ], + [ + 1.5544204711914062, + -0.205126091837883, + -0.8731956481933594, + 0.2661944329738617, + -0.2335115373134613, + -0.869574785232544, + -0.7200550436973572, + 0.6391658782958984, + 0.3534799814224243, + -0.03703191131353378, + -0.453675240278244, + 0.24169741570949554, + -0.5494473576545715, + -0.46163302659988403, + 0.10946279764175415, + -0.02392527088522911, + 0.4711993932723999, + -0.33371812105178833, + 0.49010616540908813, + -0.19618189334869385, + 0.27094411849975586 + ], + [ + 1.329495906829834, + 0.027531512081623077, + -0.8849279880523682, + 0.3290958106517792, + 0.33424481749534607, + -1.3913294076919556, + -0.16062289476394653, + 0.22481419146060944, + 0.28562772274017334, + -0.21471309661865234, + -0.7556926012039185, + 0.1228054016828537, + -0.2151045799255371, + -0.048544853925704956, + 0.04217083379626274, + 0.12947796285152435, + 0.14930139482021332, + -0.47361016273498535, + 0.8292598724365234, + 0.23861980438232422, + 0.3284529149532318 + ], + [ + 1.4044618606567383, + 0.005583565682172775, + -0.7097935676574707, + 0.2668057680130005, + 0.25619059801101685, + -1.5797830820083618, + -0.1833033561706543, + 0.32690665125846863, + 0.2806861102581024, + -0.08240661770105362, + -0.6182770133018494, + -0.02741682529449463, + -0.10848620533943176, + -0.20988096296787262, + 0.03131095692515373, + 0.09103158116340637, + 0.20413587987422943, + -0.45211684703826904, + 0.4914407730102539, + 0.17220258712768555, + 0.2225353717803955 + ], + [ + 1.4604007005691528, + 0.05853419005870819, + -0.6606404781341553, + 0.36987581849098206, + 0.22956697642803192, + -1.6750952005386353, + -0.29594236612319946, + 0.22983431816101074, + 0.25976860523223877, + -0.0760643407702446, + -0.6543765068054199, + 0.08448074758052826, + 0.0864664614200592, + -0.3004239797592163, + 0.0643731877207756, + 0.09490689635276794, + 0.21696867048740387, + -0.41205132007598877, + 0.4456923007965088, + 0.07428351044654846, + 0.196548730134964 + ], + [ + 1.6114256381988525, + -0.22972920536994934, + -0.8154833316802979, + -0.016338661313056946, + 0.3510611951351166, + -0.9570256471633911, + -1.0122497081756592, + 0.44647374749183655, + 0.22919197380542755, + -0.22349590063095093, + -1.0027257204055786, + -0.20639553666114807, + -0.21644142270088196, + -0.27758657932281494, + 0.40001967549324036, + 0.10671809315681458, + -0.2702145576477051, + -0.3088212013244629, + 1.0284866094589233, + -1.0518794059753418, + -0.12535235285758972 + ], + [ + 1.6862655878067017, + -0.19620074331760406, + -0.7826117277145386, + -0.13826322555541992, + 0.46432220935821533, + -1.1855347156524658, + -1.1719279289245605, + 0.4701594412326813, + 0.24714942276477814, + -0.21692991256713867, + -1.0339958667755127, + -0.25352704524993896, + -0.2177039086818695, + -0.25296932458877563, + 0.45234382152557373, + 0.11219623684883118, + -0.23155592381954193, + -0.34111160039901733, + 1.069090485572815, + -1.0809414386749268, + -0.07589679956436157 + ], + [ + 1.6199008226394653, + 0.004985533654689789, + -0.6341373920440674, + 0.2836281359195709, + 0.20008470118045807, + -1.8935352563858032, + -0.021761029958724976, + 0.08875052630901337, + 0.3210280239582062, + 0.01811107248067856, + -0.6016245484352112, + -0.14748689532279968, + 0.020007193088531494, + -0.16946063935756683, + 0.07198408246040344, + -0.05364494025707245, + 0.07550117373466492, + -0.3124585747718811, + 0.7698216438293457, + -0.2978877127170563, + 0.28643712401390076 + ] + ], + "generated_utilities": [ + 0.07018190622329712, + 0.0666942223906517, + 0.055874746292829514, + 0.04966646432876587, + 0.05415312573313713, + 0.07510259002447128, + 0.060902971774339676, + 0.060523372143507004 + ], + "group_id": "ms-f3f1d07cfd59b823da47", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07767709344625473, + 0.07153411954641342, + 0.08480638265609741, + 0.06087514013051987, + 0.15252648293972015, + 0.0776997059583664, + 0.1476559340953827, + 0.17124073207378387, + 0.12327007949352264, + 0.12037092447280884, + 0.08562581241130829, + 0.09917601197957993, + 0.07137986272573471, + 0.039922356605529785, + 0.04496842622756958, + 0.05561637878417969 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.9664052724838257, + 0.20301294326782227, + -1.1196715831756592, + -0.17260856926441193, + -0.5341777801513672, + -2.678128242492676, + -2.431448459625244, + 1.5603530406951904, + 0.03585217520594597, + -0.3520800471305847, + -2.473386287689209, + 1.0162299871444702, + -0.7868502736091614, + -0.9102090001106262, + 0.9442894458770752, + -0.41473180055618286, + 0.7349075675010681, + -0.8863289952278137, + 0.8796874284744263, + 0.028114937245845795, + 0.24920660257339478 + ], + [ + 1.900488257408142, + -0.00838443636894226, + -1.2311820983886719, + -0.361179918050766, + -0.3987593352794647, + -2.551008462905884, + -2.701385021209717, + 1.7117550373077393, + -0.20423506200313568, + -0.3429180979728699, + -2.6584055423736572, + 0.8018259406089783, + -0.856230616569519, + -0.901038408279419, + 1.2131764888763428, + -0.6892215609550476, + 0.6726197600364685, + -0.7798839211463928, + 0.6868253946304321, + 0.07731680572032928, + 0.30107203125953674 + ], + [ + 1.6257587671279907, + 0.275219202041626, + -0.7838636636734009, + -0.5118123292922974, + -0.3429456353187561, + -2.342111349105835, + -2.7158243656158447, + 1.7038744688034058, + -0.1860528439283371, + -0.11617843806743622, + -2.1894750595092773, + 1.2306283712387085, + -0.8203369379043579, + -0.5157363414764404, + 0.7951346635818481, + -0.464976966381073, + 0.8823708295822144, + -0.7671404480934143, + 0.9700374007225037, + 0.21184596419334412, + -0.11244213581085205 + ] + ], + "predicted_scores": [ + 5.896638870239258, + 5.579070568084717, + 5.217473030090332, + 4.840173721313477, + 4.642172336578369, + 3.875715970993042, + 3.8069164752960205, + 3.643287420272827 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train1", + "selected_index": 0, + "state_hash": "3f477a29208b02558b99bd0f3305d0f1845c277d32e0b5afdafa71012673383b", + "stored_base_utility": 0.07767709344625473, + "task_id": "PullCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07080724835395813, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07080724835395813 + }, + "base_progress": 0.07080724835395813, + "base_success": false, + "base_utility": 0.07080724835395813, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06590981781482697, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06590981781482697 + }, + { + "contact_quality": null, + "progress": 0.05606792867183685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05606792867183685 + }, + { + "contact_quality": null, + "progress": 0.06440688669681549, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06440688669681549 + }, + { + "contact_quality": null, + "progress": 0.056011125445365906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.056011125445365906 + }, + { + "contact_quality": null, + "progress": 0.07260851562023163, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07260851562023163 + }, + { + "contact_quality": null, + "progress": 0.07159297168254852, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07159297168254852 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.07599961757659912, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07599961757659912 + } + ], + "candidate_progress": [ + 0.06590981781482697, + 0.05606792867183685, + 0.06440688669681549, + 0.056011125445365906, + 0.07260851562023163, + 0.07159297168254852, + 1.0, + 0.07599961757659912 + ], + "candidate_source_chart_ids": [ + "ms-dab80e1bb7987577e33d", + "ms-bfe867e8d518b9f8333a", + "ms-bfe867e8d518b9f8333a", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-081952a63087fc8f83cd", + "generated_tangents": [ + [ + 0.03375854343175888, + -0.04900426045060158, + 1.412408471107483, + -0.725378155708313, + -1.2397005558013916, + 0.14692215621471405, + -0.09779292345046997, + -0.13894176483154297, + 0.02070929855108261, + 0.1501135230064392, + -0.03075685352087021, + -1.318574070930481, + 0.20384398102760315, + 0.12198208272457123, + 0.06595951318740845, + -0.13841527700424194, + 0.14516130089759827, + 0.28113842010498047, + -0.8382633328437805, + -0.33905622363090515, + -0.09004026651382446 + ], + [ + -0.02888302505016327, + 0.00361696258187294, + 1.3578001260757446, + -0.7423804998397827, + -1.3380978107452393, + 0.24790945649147034, + -0.1035570502281189, + -0.03865240514278412, + 0.05394285172224045, + 0.16857779026031494, + -0.013176776468753815, + -1.4046103954315186, + 0.1299164593219757, + 0.24455326795578003, + -0.00902726873755455, + -0.11642973124980927, + 0.15607956051826477, + 0.36178815364837646, + -0.761135995388031, + -0.15585646033287048, + -0.08467072248458862 + ], + [ + -0.019810110330581665, + 0.007485195994377136, + 1.3393781185150146, + -0.7851817607879639, + -1.3124110698699951, + 0.2607835531234741, + -0.13392162322998047, + -0.0397568941116333, + 0.04677657037973404, + 0.16579826176166534, + 0.04543188214302063, + -1.401707410812378, + 0.14570406079292297, + 0.223608136177063, + -0.044904474169015884, + -0.11624743044376373, + 0.20332372188568115, + 0.3622475266456604, + -0.7356930375099182, + -0.14989958703517914, + -0.05062338709831238 + ], + [ + 0.04672608524560928, + -0.06441795825958252, + 1.452209234237671, + -0.6598774194717407, + -0.9569069743156433, + 0.3073361814022064, + -0.018876492977142334, + 0.0018425732851028442, + 0.054364390671253204, + 0.12689965963363647, + -0.10602091997861862, + -1.0276288986206055, + 0.2296665608882904, + 0.0032745152711868286, + -0.02115141972899437, + -0.20479948818683624, + 0.2083372324705124, + 0.532421886920929, + -0.6153956651687622, + -0.10267405211925507, + -0.1843487024307251 + ], + [ + 0.18768692016601562, + -0.08090052008628845, + 1.4347021579742432, + -0.6663864254951477, + -1.2294483184814453, + 0.08292755484580994, + 0.0018686950206756592, + -0.01563642919063568, + 0.09956304728984833, + 0.31591761112213135, + -0.014304853975772858, + -1.1225000619888306, + 0.4266897737979889, + -0.1096440851688385, + -0.00591939315199852, + -0.19483500719070435, + 0.13410940766334534, + 0.34820324182510376, + -0.6537925601005554, + -0.20497937500476837, + -0.020299524068832397 + ], + [ + 0.1646152287721634, + -0.18114203214645386, + 1.7602574825286865, + -0.620148777961731, + -1.4567086696624756, + 0.4231996536254883, + -0.2829062342643738, + 0.1680794060230255, + 0.1296631395816803, + 0.1684209704399109, + 0.05274005979299545, + -0.9986904859542847, + 0.13165435194969177, + 0.2405969798564911, + -0.10494042932987213, + -0.26405394077301025, + 0.0836157500743866, + 0.45522111654281616, + -0.9746655821800232, + -0.14122386276721954, + -0.11801105737686157 + ], + [ + -0.2704783082008362, + -0.1814546436071396, + 1.643018364906311, + -0.703923225402832, + -1.0324947834014893, + 0.9079009294509888, + -0.27390748262405396, + 0.007359892129898071, + 0.0805899053812027, + 0.18501780927181244, + -0.022130578756332397, + -1.0192875862121582, + 0.43525174260139465, + 0.07570907473564148, + -0.09988804161548615, + -0.2945282757282257, + -0.024339601397514343, + 0.4812130331993103, + -0.7962149977684021, + -0.3062295615673065, + 0.00640125572681427 + ], + [ + 0.08928649872541428, + -0.15247970819473267, + 1.6031819581985474, + -0.7279216051101685, + -1.0372061729431152, + 0.47576549649238586, + -0.16235554218292236, + 0.18651781976222992, + 0.05945601314306259, + 0.1023881584405899, + 0.076483353972435, + -1.1296696662902832, + 0.20422998070716858, + -0.07566863298416138, + -0.051602523773908615, + -0.2550467550754547, + -0.15156693756580353, + 0.5413152575492859, + -0.7147372961044312, + -0.09824743866920471, + 0.015150845050811768 + ] + ], + "generated_utilities": [ + 0.06590981781482697, + 0.05606792867183685, + 0.06440688669681549, + 0.056011125445365906, + 0.07260851562023163, + 0.07159297168254852, + 2.0, + 0.07599961757659912 + ], + "group_id": "ms-081952a63087fc8f83cd", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0006054937839508057, + 2.0, + 2.0, + 2.0, + 0.09041815996170044, + 0.03755016624927521, + 0.10023769736289978, + 0.08501018583774567, + 0.10316208004951477, + 0.13556814193725586, + 0.0687420666217804, + 0.16420090198516846, + 0.07440806925296783, + 0.03177754580974579, + 0.05512912571430206, + 0.05822136998176575 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05530714988708496, + -0.0015276968479156494, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.028481781482696533, + 0.0, + 0.07634823024272919, + 0.10118435323238373, + 0.01472395658493042, + -0.021184463053941727, + -1.3624117374420166, + 0.06702810525894165, + 0.0, + 0.064424529671669, + -0.13110561668872833, + -0.04677841067314148, + -0.0666094422340393, + -0.8611308336257935, + -0.02982822060585022, + 0.0 + ], + [ + -0.13999104499816895, + -0.16139763593673706, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.17549771070480347, + 0.0, + -0.15019088983535767, + 0.2207401990890503, + -0.06357991695404053, + 0.10177183151245117, + -1.3624117374420166, + -0.05549207329750061, + 0.0, + -0.1271420419216156, + -0.057942673563957214, + -0.279840350151062, + -0.11312717199325562, + -0.8611308336257935, + 0.1586582064628601, + 0.0 + ], + [ + 0.2317524552345276, + -0.2474955916404724, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.2997032403945923, + 0.0, + -0.3180278241634369, + 0.1991673707962036, + 0.26833486557006836, + 0.005899060517549515, + -1.3624117374420166, + 0.032782405614852905, + 0.0, + 0.17653466761112213, + 0.5550464987754822, + 0.13015127182006836, + -0.11312717199325562, + -0.8611308336257935, + 0.020897403359413147, + 0.0 + ], + [ + -0.7016198635101318, + 0.312343955039978, + 2.582733154296875, + -1.5283455848693848, + -2.530975103378296, + 0.7002967596054077, + 2.0224196910858154, + -0.031223494559526443, + -0.07945594191551208, + 0.7471928000450134, + -0.05505431815981865, + -2.3624117374420166, + 0.4337144196033478, + 1.385794997215271, + -0.18244768679141998, + -0.22281835973262787, + -0.5003347992897034, + 0.8868728280067444, + -1.8611308336257935, + 0.19736243784427643, + 0.2646070420742035 + ], + [ + -0.41503679752349854, + 0.3199126422405243, + 2.3341803550720215, + -1.5432449579238892, + -2.6036009788513184, + 0.8186125755310059, + 1.889835238456726, + -0.20609386265277863, + 0.0827421247959137, + 0.8381537795066833, + -0.0944519191980362, + -2.4805798530578613, + 0.3916063606739044, + 1.2392640113830566, + -0.2266693413257599, + -0.33516526222229004, + -0.3536234498023987, + 0.9749420881271362, + -2.1439476013183594, + 0.3261662721633911, + 0.13869160413742065 + ], + [ + -0.6778035759925842, + 0.3674989640712738, + 2.3216195106506348, + -1.6129508018493652, + -2.3233790397644043, + 0.37060582637786865, + 2.3460428714752197, + 0.2269098311662674, + -0.269074022769928, + 0.9810001254081726, + 0.023842941969633102, + -2.478595733642578, + 0.46739575266838074, + 1.1671457290649414, + -0.41542381048202515, + 0.030661001801490784, + -0.48762595653533936, + 0.7803086638450623, + -1.7778995037078857, + 0.10219369828701019, + 0.08611755073070526 + ], + [ + -1.0292376279830933, + 0.05653715133666992, + 3.017642021179199, + -1.2965388298034668, + -2.1434991359710693, + 0.8112732172012329, + 2.3553943634033203, + 0.14745590090751648, + -0.42473679780960083, + 0.8983602523803711, + -0.5389888286590576, + -1.9595260620117188, + 0.8665074110031128, + 1.7057592868804932, + -0.36306142807006836, + -0.6694610714912415, + -0.3699086308479309, + 0.7089512944221497, + -2.2496888637542725, + 0.1759258657693863, + 0.13106243312358856 + ], + [ + -0.2738665044307709, + 0.13772070407867432, + 2.3146424293518066, + -1.9326081275939941, + -2.760838508605957, + 0.10491085052490234, + 1.9823331832885742, + 0.3494338393211365, + 0.3550114631652832, + 0.7471076846122742, + -0.06264704465866089, + -2.2210469245910645, + 0.5386381149291992, + 1.431454062461853, + 0.1620958298444748, + -0.00858105719089508, + -0.8883554935455322, + 1.1764272451400757, + -1.8280467987060547, + 0.30690017342567444, + -0.32479730248451233 + ], + [ + -0.8548018932342529, + -0.2078946828842163, + 2.740286111831665, + -1.6552648544311523, + -2.470123291015625, + 1.0607044696807861, + 2.4267592430114746, + -0.16535070538520813, + -0.2640931308269501, + 0.2447068691253662, + 0.5616287589073181, + -2.808340311050415, + 1.0820176601409912, + 1.2134929895401, + 0.2653927206993103, + -0.454708456993103, + 0.03163468837738037, + 0.8172736763954163, + -1.8861355781555176, + 0.45078086853027344, + 0.29641321301460266 + ], + [ + -0.7025277018547058, + 0.5556451678276062, + 2.3374006748199463, + -1.9256110191345215, + -1.7495018243789673, + 0.9721168279647827, + 1.8326293230056763, + 0.5355191826820374, + -0.5788277983665466, + 0.19061005115509033, + -0.6322742700576782, + -3.031383514404297, + -0.1651831567287445, + 0.8584551215171814, + 0.2280312329530716, + 0.27221786975860596, + -0.3728860020637512, + 1.102063775062561, + -2.5995376110076904, + 0.7578867673873901, + -0.3784477412700653 + ], + [ + -7.212162017822266e-06, + 0.552659273147583, + 2.0397679805755615, + -1.8501746654510498, + -2.5457160472869873, + 1.0514975786209106, + 2.5270917415618896, + -0.7367458343505859, + 0.7676748037338257, + 0.7428221702575684, + -0.538831353187561, + -2.7423243522644043, + 0.07969537377357483, + 1.0069079399108887, + -0.8100140690803528, + -0.9174093008041382, + -0.9411231875419617, + 1.0430617332458496, + -1.4683198928833008, + 1.0334573984146118, + 0.4721405506134033 + ], + [ + -0.41159626841545105, + 0.41878798604011536, + 1.6363914012908936, + -2.4960880279541016, + -3.049954891204834, + -0.09567022323608398, + 2.3269245624542236, + 0.11159682273864746, + -0.7885257005691528, + 0.44582536816596985, + -0.47802892327308655, + -1.5574463605880737, + 1.0153859853744507, + 2.258706569671631, + -0.36591836810112, + 0.49115514755249023, + -0.9523675441741943, + 0.5892280340194702, + -2.1512646675109863, + -0.7744256854057312, + 0.592117428779602 + ], + [ + 0.1343594193458557, + 0.45838433504104614, + 2.6735801696777344, + -2.005676746368408, + -3.340463161468506, + 1.0152086019515991, + 1.429718255996704, + 0.7929191589355469, + -0.2598758339881897, + 1.6483545303344727, + -0.7411167025566101, + -2.8978958129882812, + 1.3759692907333374, + 1.4385837316513062, + 0.2961248755455017, + -0.042098239064216614, + -0.6380948424339294, + 1.0523303747177124, + -2.1783251762390137, + -0.045405834913253784, + 0.7960561513900757 + ] + ], + "predicted_scores": [ + 8.38859748840332, + 7.865166664123535, + 7.739889621734619, + 7.458866596221924, + 7.421019554138184, + 7.341989994049072, + 7.153594970703125, + 6.231618881225586 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2141802816/train1", + "selected_index": 0, + "state_hash": "d1cf06bba4675ad6a6942dc4f6fbade890fbf21b5c3068f8bbe30cfc60c54d9b", + "stored_base_utility": 0.0006054937839508057, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-2ba02dcefa15a9371706", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-121bc2e6676c1ac10911", + "generated_tangents": [ + [ + -1.1428537368774414, + -0.33681827783584595, + 0.0769180953502655, + -0.5968054533004761, + -0.571502685546875, + 0.25492945313453674, + 1.8293170928955078, + -0.05916261672973633, + 0.2201186865568161, + -0.46115565299987793, + 0.1269274204969406, + -0.9132931232452393, + -0.09839442372322083, + 0.7744705677032471, + -0.06471119821071625, + -0.14481939375400543, + -0.20126263797283173, + 0.7576097846031189, + 0.2796335816383362, + 0.01933467388153076, + 0.22309339046478271 + ], + [ + -0.43985962867736816, + -0.023176170885562897, + 0.3756997585296631, + -0.24748322367668152, + -0.6525080800056458, + -0.2818254232406616, + 2.2367281913757324, + 0.08405214548110962, + 0.15601615607738495, + -0.06820777803659439, + -0.1090114489197731, + -0.1929427981376648, + -0.14624667167663574, + 0.92344731092453, + -0.08865220844745636, + 0.05167548730969429, + -0.40641969442367554, + 0.49160391092300415, + 0.0954277366399765, + 0.14885711669921875, + 0.6119989156723022 + ], + [ + -0.6306641101837158, + -0.1735406219959259, + 0.028355330228805542, + -0.8203088045120239, + -0.5015040040016174, + -0.03623749315738678, + 1.7209930419921875, + -0.052967727184295654, + 0.2429560273885727, + -0.29548078775405884, + -0.2207215130329132, + -0.5763352513313293, + -0.6688967943191528, + 0.4102092385292053, + 0.20549078285694122, + -0.10159313678741455, + -0.16518981754779816, + 0.543495774269104, + -0.06937819719314575, + 0.16048121452331543, + 0.6708518266677856 + ], + [ + -0.33497750759124756, + -0.034328460693359375, + -0.29067251086235046, + -0.6124038696289062, + -1.2642409801483154, + 0.3559309244155884, + 1.6858468055725098, + -0.061229854822158813, + 0.16996949911117554, + -0.09348472207784653, + 0.07988788187503815, + -0.7375262379646301, + -0.446968674659729, + 0.48836642503738403, + -0.1606544852256775, + -0.1050918698310852, + -0.5172917246818542, + 0.659605860710144, + -0.5452120900154114, + 0.11138984560966492, + 0.6398909091949463 + ], + [ + -0.8535206317901611, + -0.03217511251568794, + -0.45911410450935364, + -0.9529755115509033, + -0.764872670173645, + 0.03175976872444153, + 1.339918613433838, + -0.03322230279445648, + 0.3067573606967926, + -0.3839905261993408, + -0.28448960185050964, + -0.6497014164924622, + -0.5434535145759583, + 0.2834795415401459, + -0.3409568667411804, + -0.09876580536365509, + -0.08029772341251373, + 0.8834956288337708, + -0.282196581363678, + -0.38060763478279114, + 0.9672904014587402 + ], + [ + -0.9470295906066895, + -0.159866064786911, + -0.022715657949447632, + -0.5708368420600891, + -0.43117791414260864, + 0.011789724230766296, + 1.926875352859497, + -0.05345553159713745, + 0.2844993472099304, + -0.3308572471141815, + -0.25868457555770874, + -0.4164997339248657, + -0.8099724054336548, + 0.42035311460494995, + -0.0224350206553936, + -0.06410782039165497, + -0.04351837933063507, + 0.6609387397766113, + -0.12267306447029114, + 0.2131757140159607, + 1.0377119779586792 + ], + [ + -0.7540832161903381, + -0.18264012038707733, + -0.12035420536994934, + -0.6487157940864563, + -0.6398594975471497, + 0.12603461742401123, + 1.8564980030059814, + -0.11238265037536621, + 0.20606793463230133, + -0.31462809443473816, + -0.10243957489728928, + -0.5333985090255737, + -0.7816828489303589, + 0.5273482799530029, + -0.06624205410480499, + -0.14748543500900269, + -0.18167315423488617, + 0.6917931437492371, + -0.18754485249519348, + 0.11445602774620056, + 1.0951846837997437 + ], + [ + -0.6739012598991394, + -0.43815380334854126, + 0.03737080097198486, + -0.7623136043548584, + -0.26687294244766235, + -0.7074288129806519, + 1.153752088546753, + -0.20895999670028687, + 0.41342222690582275, + -0.5485003590583801, + -0.32419297099113464, + -0.5121111869812012, + 0.003171265125274658, + -0.05138956010341644, + -0.19933244585990906, + -0.012433379888534546, + 0.0928570032119751, + 0.8527677655220032, + -0.1134442389011383, + -0.04123127460479736, + 0.26661014556884766 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-121bc2e6676c1ac10911", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15086030960083008, + 0.08354458212852478, + 2.0, + 0.1453719437122345, + 0.19646552205085754, + 0.10343040525913239, + 0.1443832516670227, + 0.13976123929023743, + 0.17431864142417908, + 2.0, + 0.12949708104133606, + 2.0, + 2.0, + 0.11397694051265717, + 0.1271887570619583, + 0.025320395827293396 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.39665424823760986, + 0.08244594186544418, + -0.026901081204414368, + -0.959688663482666, + -1.2648048400878906, + 0.05568990111351013, + 0.0, + -0.048488348722457886, + 0.03576555848121643, + -0.012020409107208252, + 0.08576416969299316, + -0.25812816619873047, + 0.03589746356010437, + 0.0, + 0.0016258209943771362, + -0.04025750607252121, + 0.010670006275177002, + 0.5142596960067749, + 0.06220144033432007, + 0.07906517386436462, + 0.0 + ], + [ + -2.15720796585083, + 0.13025060296058655, + 0.41441529989242554, + -1.4498568773269653, + -2.6386165618896484, + 0.5572659969329834, + 0.7093658447265625, + -1.0835764408111572, + -0.896172285079956, + 0.07744038105010986, + 1.1224509477615356, + -1.6321502923965454, + 0.877888560295105, + 0.4978562295436859, + 0.5774499177932739, + 0.8093571662902832, + -0.8521628379821777, + 2.054164171218872, + -0.6021197438240051, + 1.0350128412246704, + 0.823830246925354 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.39665424823760986, + -0.2284793257713318, + -0.027931194752454758, + -0.959688663482666, + -1.2648048400878906, + -0.17197783291339874, + 0.0, + -0.06679429113864899, + 0.1646350920200348, + -0.11601293087005615, + 0.08576416969299316, + -0.25812816619873047, + 0.1479925513267517, + 0.0, + -0.04833713173866272, + 0.22201494872570038, + -0.16177666187286377, + 0.5142596960067749, + -0.10178101062774658, + 0.21631890535354614, + 0.0 + ], + [ + -0.8824495077133179, + 0.0034109503030776978, + -0.5904110074043274, + -1.400491714477539, + -2.4447474479675293, + -0.00789790228009224, + 1.6208159923553467, + 0.26819390058517456, + -0.23575741052627563, + -0.7184416651725769, + 0.9072303771972656, + -1.2077033519744873, + -0.21496516466140747, + 0.8258882761001587, + -0.4575563669204712, + -0.3655202388763428, + -0.5908474922180176, + 2.0210399627685547, + -1.2132787704467773, + 0.9719676971435547, + 0.24625355005264282 + ], + [ + -0.6953714489936829, + 0.4026735723018646, + 0.6648998856544495, + -1.356613278388977, + -2.953742742538452, + 0.037713564932346344, + 1.53464937210083, + 0.027994811534881592, + 0.03435474634170532, + -1.1939163208007812, + 0.9877840280532837, + -1.180513620376587, + 0.4064505994319916, + 0.5743880867958069, + 0.1558205783367157, + 0.3796072006225586, + -0.6676555275917053, + 1.4665511846542358, + 0.04170459508895874, + 0.17645099759101868, + 0.2388516366481781 + ], + [ + -1.4095232486724854, + -0.06358731538057327, + -0.7774888873100281, + -2.806776285171509, + -2.44041109085083, + -0.8258867859840393, + 2.383592128753662, + -0.093393474817276, + -0.24701429903507233, + -0.5286431312561035, + 0.7741218209266663, + -2.111828565597534, + -0.31807881593704224, + 1.3003538846969604, + -0.9306488037109375, + -0.8708829283714294, + -0.3694953918457031, + 2.169797420501709, + -0.7153271436691284, + 0.029519766569137573, + -0.5629553198814392 + ] + ], + "predicted_scores": [ + 3.663315534591675, + 2.5850982666015625, + 1.273209810256958, + 0.41306161880493164, + 0.03598073124885559, + -0.15662100911140442, + -0.1887272298336029, + -0.43141189217567444 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264/train1", + "selected_index": 0, + "state_hash": "d1280a9e9362f8da3290de84fdd0f0b78475d470cd0230e590ccaf8f5428548a", + "stored_base_utility": 0.15086030960083008, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-22f153f0d4fa656018e6", + "generated_tangents": [ + [ + -0.24887557327747345, + -0.027390284463763237, + 0.5010677576065063, + -0.9348056316375732, + -1.5222833156585693, + -0.04675397276878357, + 0.5733861327171326, + -0.15539279580116272, + 0.08523792028427124, + 0.10814549028873444, + -0.18766281008720398, + -0.9836422204971313, + 0.05895397067070007, + 0.2705014944076538, + -0.14306843280792236, + -0.103380486369133, + -0.055262550711631775, + -0.014810875058174133, + -0.4295213222503662, + 0.024319976568222046, + -0.1443595290184021 + ], + [ + -0.6374868750572205, + -0.17066138982772827, + 0.5716707110404968, + -0.8789207935333252, + -1.7522239685058594, + -0.16476508975028992, + 0.44131967425346375, + -0.15306147933006287, + 0.058665692806243896, + -0.11219052225351334, + -0.07430637627840042, + -1.1802771091461182, + 0.40951308608055115, + 0.21393123269081116, + -0.2618885040283203, + -0.22483773529529572, + 0.10827963054180145, + 0.059042468667030334, + -0.25649404525756836, + -0.10458812117576599, + 0.4529889225959778 + ], + [ + -0.6623293161392212, + -0.07495532929897308, + 0.21865785121917725, + -1.1720353364944458, + -1.9080228805541992, + 0.6288790106773376, + -0.21692156791687012, + -0.04327043890953064, + 0.01988872140645981, + -0.02546333521604538, + 0.10489708185195923, + -1.2112760543823242, + 0.010973095893859863, + 0.2623225748538971, + -0.2310495674610138, + -0.34748461842536926, + 0.128828763961792, + -0.1510370522737503, + -0.2398608922958374, + 0.15933069586753845, + 0.51133131980896 + ], + [ + -0.47105252742767334, + -0.20062479376792908, + 0.5346060395240784, + -1.0590448379516602, + -1.8472039699554443, + 0.0377831757068634, + -0.05174052715301514, + 0.034425780177116394, + 0.027260316535830498, + -0.053590960800647736, + 0.14692896604537964, + -1.1483964920043945, + 0.03161340951919556, + 0.3398374915122986, + -0.2776331603527069, + -0.16293086111545563, + -0.075893834233284, + -0.055674538016319275, + -0.4315546154975891, + 0.21798521280288696, + 0.35815271735191345 + ], + [ + -0.6245862245559692, + -0.20884281396865845, + 0.6665024161338806, + -0.9951554536819458, + -1.9768967628479004, + 0.18941600620746613, + -0.0761062502861023, + -0.03308330476284027, + 0.02400665357708931, + -0.013298399746418, + 0.13496777415275574, + -1.254115343093872, + 0.07984712719917297, + 0.32589688897132874, + -0.2589893341064453, + -0.18994814157485962, + -0.045686766505241394, + -0.06592737138271332, + -0.43024200201034546, + 0.15589860081672668, + 0.16848532855510712 + ], + [ + -0.6612503528594971, + -0.20185846090316772, + 0.6815662980079651, + -1.04066801071167, + -1.7090041637420654, + 0.4628135859966278, + -0.09681999683380127, + -0.15471616387367249, + 0.04341607540845871, + 0.06519214808940887, + 0.14938311278820038, + -1.0014803409576416, + 0.19993892312049866, + 0.5284703969955444, + -0.23039305210113525, + -0.24038387835025787, + -0.13825036585330963, + -0.046910956501960754, + -0.34253668785095215, + -0.30616962909698486, + 0.05990007519721985 + ], + [ + -0.7707524299621582, + -0.05616701394319534, + 0.7455941438674927, + -0.9382203817367554, + -1.903231143951416, + -0.1696189045906067, + 0.017838507890701294, + -0.20468854904174805, + 0.11596565693616867, + 0.15395723283290863, + 0.08011648803949356, + -1.3235644102096558, + -0.08012253046035767, + 0.30108708143234253, + -0.3165774345397949, + -0.191693976521492, + -0.04941864311695099, + -0.0692630261182785, + -0.5177285075187683, + 0.3827189803123474, + 0.12820693850517273 + ], + [ + -0.6125665307044983, + -0.24611927568912506, + 0.7538248300552368, + -1.0436112880706787, + -2.0779056549072266, + -0.08610925078392029, + -0.04694008827209473, + -0.026910915970802307, + 0.049526311457157135, + 0.07596208900213242, + 0.06273334473371506, + -1.0408450365066528, + -0.15906131267547607, + 0.47979414463043213, + -0.09796953201293945, + -0.26035764813423157, + -0.37497830390930176, + -0.10670261085033417, + -0.4887915253639221, + -0.07884900271892548, + 0.03693243861198425 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-22f153f0d4fa656018e6", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 2.60770320892334e-06, + 3.129243850708008e-07, + 6.109476089477539e-07, + 2.0, + 0.20781123638153076, + 2.0, + 0.16136802732944489, + 0.15920132398605347, + 0.16996727883815765, + 0.14405608177185059, + 0.15874098241329193, + 0.08526891469955444, + 0.0828208178281784, + 2.0, + 0.034265756607055664, + 2.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.3937166929244995, + 0.17073306441307068, + -0.5225132703781128, + -0.8479964733123779, + -1.713785171508789, + -0.19215916097164154, + 0.0, + 0.8004336953163147, + 0.4925512969493866, + 0.3161412179470062, + 0.10414126515388489, + -0.9857386350631714, + -0.4383167624473572, + 0.0, + 0.05412057042121887, + 0.021832358092069626, + 0.5262433290481567, + -0.24631118774414062, + -0.20008409023284912, + -0.04257836565375328, + 0.0 + ], + [ + -1.3937166929244995, + 0.2384568303823471, + 0.3933030962944031, + -1.847996473312378, + -2.713785171508789, + 0.20405007898807526, + 1.2454378604888916, + 0.0006654821336269379, + -0.07783810049295425, + -0.002305123955011368, + 0.18527865409851074, + -1.9857386350631714, + -0.009703370742499828, + 0.7078995108604431, + -0.08121174573898315, + -0.031056255102157593, + -0.40927961468696594, + 0.5954965949058533, + -0.8039777874946594, + 0.05187072232365608, + 0.2283477634191513 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.490875720977783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4157990217208862, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4566955268383026 + ], + [ + -1.5219146013259888, + 0.23501667380332947, + 0.5663670301437378, + -1.892876386642456, + -2.744781494140625, + -0.04844887554645538, + 1.3053655624389648, + -0.11823110282421112, + -0.02477371320128441, + -0.17061075568199158, + 0.22949494421482086, + -1.854048252105713, + -0.025548618286848068, + 0.7958164215087891, + 0.12320645153522491, + -0.23218455910682678, + -0.6995364427566528, + 0.8081986308097839, + -0.558222234249115, + -0.18823161721229553, + 0.04578563570976257 + ], + [ + -1.608424186706543, + 0.022089049220085144, + 0.15639334917068481, + -1.982436180114746, + -2.6509876251220703, + -0.01820056140422821, + 0.95000159740448, + -0.08617040514945984, + -0.46914470195770264, + 0.3453706204891205, + -0.020491018891334534, + -1.738605260848999, + -0.0859340950846672, + 0.443394273519516, + 0.09907735884189606, + -0.38721394538879395, + -0.3415156602859497, + 0.524202287197113, + -0.7343614101409912, + -0.22556106746196747, + 0.4535057842731476 + ], + [ + -1.7957028150558472, + 0.593864381313324, + 0.06846150755882263, + -2.0563552379608154, + -2.2595136165618896, + -0.12188448011875153, + 0.9899688959121704, + -0.14783401787281036, + -0.22327497601509094, + -0.32116562128067017, + -0.07331573963165283, + -1.910165786743164, + 0.264303058385849, + 1.0567166805267334, + 0.016188345849514008, + 0.159999281167984, + -0.22885318100452423, + 0.705758273601532, + -0.8778964877128601, + 0.4381847381591797, + 0.19897118210792542 + ], + [ + -1.499897837638855, + -0.2043466418981552, + 0.42094048857688904, + -1.430327296257019, + -3.1506710052490234, + -0.08664844930171967, + 1.4003938436508179, + 0.24064289033412933, + -0.35471564531326294, + -0.37839531898498535, + 0.47059693932533264, + -1.7703200578689575, + 0.101543128490448, + 0.8470064401626587, + 0.16171279549598694, + -0.0019259154796600342, + -0.6455138325691223, + 0.5694639682769775, + -0.9997703433036804, + 0.5366826057434082, + -0.11106891930103302 + ], + [ + -1.8847153186798096, + 0.21543154120445251, + 0.9309400320053101, + -2.1095237731933594, + -2.136512517929077, + 0.6948637366294861, + 1.233637809753418, + 0.577111542224884, + -0.5727867484092712, + -0.6055393815040588, + 0.8738593459129333, + -2.6654083728790283, + -0.171565979719162, + 0.8934992551803589, + 0.17394188046455383, + 0.6209473609924316, + -0.1149752140045166, + 0.6160256266593933, + -0.5519447326660156, + 0.7195507287979126, + 0.5175096392631531 + ], + [ + -2.138639450073242, + 0.36738795042037964, + -0.33481496572494507, + -1.9255597591400146, + -2.2420146465301514, + 0.31966179609298706, + 1.3172208070755005, + -0.48551350831985474, + 0.27687638998031616, + -0.41806119680404663, + 0.05362778902053833, + -2.3665032386779785, + -0.47635504603385925, + 0.081889808177948, + 0.542697548866272, + -0.7215265035629272, + -0.906980037689209, + 0.6977691054344177, + -1.4041197299957275, + -0.11211369931697845, + 0.5149455070495605 + ], + [ + -1.4869698286056519, + -0.30130720138549805, + 1.28191077709198, + -0.9839465618133545, + -2.2501890659332275, + -0.20459605753421783, + 1.3607591390609741, + 0.6349532008171082, + -0.8627415895462036, + -0.623334527015686, + 0.21189503371715546, + -2.2093186378479004, + -0.4655338525772095, + 0.7016672492027283, + 0.6916180849075317, + -0.0807163268327713, + 0.1465948522090912, + 0.05083376169204712, + -0.8381134271621704, + 0.015291888266801834, + 0.05565965175628662 + ], + [ + -2.365525245666504, + 0.45377761125564575, + 0.2984481453895569, + -1.3690096139907837, + -2.843494176864624, + 0.32462257146835327, + 0.5082458257675171, + 0.7033615112304688, + 0.31817564368247986, + -0.4890395700931549, + 0.09036337584257126, + -2.904804229736328, + 0.560937762260437, + 1.649811029434204, + -0.9795587062835693, + -0.46419358253479004, + -0.4739357829093933, + 0.34800490736961365, + -1.0712809562683105, + 0.9167320132255554, + -0.48252081871032715 + ], + [ + -1.2987136840820312, + 0.7370830774307251, + -0.49668365716934204, + -1.201794147491455, + -3.552318572998047, + -0.5406968593597412, + 2.1693286895751953, + 0.5925421714782715, + 0.7903083562850952, + 0.991620659828186, + 0.7936639785766602, + -2.3832802772521973, + -0.621701180934906, + 0.43542394042015076, + -0.14415577054023743, + 0.15921397507190704, + -0.395956814289093, + -0.2778439521789551, + -1.069305419921875, + -0.5347616076469421, + -0.26581722497940063 + ] + ], + "predicted_scores": [ + 3.5058493614196777, + 3.4791247844696045, + 3.096482038497925, + 2.2233731746673584, + 2.211974620819092, + 1.8748934268951416, + 1.7593693733215332, + 1.1505695581436157 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007/train1", + "selected_index": 0, + "state_hash": "3139b3ac90f6fb958f14d875d082c95c8569f039086e0789e6c9e079d602b945", + "stored_base_utility": 2.60770320892334e-06, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0320269912481308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0320269912481308 + }, + "base_progress": 0.0320269912481308, + "base_success": false, + "base_utility": 0.0320269912481308, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03511819243431091, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03511819243431091 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03361482918262482, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03361482918262482 + }, + { + "contact_quality": null, + "progress": 0.024507999420166016, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024507999420166016 + }, + { + "contact_quality": null, + "progress": 0.02013157308101654, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02013157308101654 + }, + { + "contact_quality": null, + "progress": 0.03976954519748688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03976954519748688 + }, + { + "contact_quality": null, + "progress": 0.03125926852226257, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03125926852226257 + }, + { + "contact_quality": null, + "progress": 0.017352595925331116, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017352595925331116 + } + ], + "candidate_progress": [ + 0.03511819243431091, + 1.0, + 0.03361482918262482, + 0.024507999420166016, + 0.02013157308101654, + 0.03976954519748688, + 0.03125926852226257, + 0.017352595925331116 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1", + "ms-2423891634e7b469eed8", + "ms-ad5107846b45eae5a0ab", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "generated_tangents": [ + [ + -0.004802003502845764, + 0.2759769558906555, + 0.6961096525192261, + -1.6054247617721558, + 0.24079948663711548, + 0.44251441955566406, + 0.6913051605224609, + -0.21571427583694458, + 0.21488012373447418, + 0.13592529296875, + -0.28689852356910706, + -0.7922946214675903, + 0.25793585181236267, + 0.3460880517959595, + 0.2650951147079468, + -0.18585707247257233, + -0.016776978969573975, + 0.05982615053653717, + -0.6826059818267822, + 0.22354018688201904, + 0.2553693652153015 + ], + [ + -0.04533401131629944, + 0.2646772861480713, + 0.5887936949729919, + -1.6549843549728394, + 0.3267868161201477, + 0.3581558167934418, + 0.31044822931289673, + -0.11962872743606567, + 0.20861420035362244, + 0.23288512229919434, + -0.17209821939468384, + -1.001847743988037, + 0.16749736666679382, + 0.5874131321907043, + 0.004180684685707092, + -0.21890732645988464, + -0.09830166399478912, + 0.20511867105960846, + -0.6198068857192993, + 0.19094139337539673, + 0.10831240564584732 + ], + [ + 0.04220472276210785, + 0.22109778225421906, + 0.8459436893463135, + -1.8022898435592651, + 0.2696196436882019, + 0.42059656977653503, + 0.5853767991065979, + -0.1801830530166626, + 0.19578561186790466, + 0.03888130933046341, + -0.07634315639734268, + -0.7004210948944092, + -0.04044097661972046, + 0.5887544751167297, + -0.06970055401325226, + -0.24952712655067444, + 0.0805043876171112, + 0.26777398586273193, + -0.7303264737129211, + 0.4835692048072815, + 0.367800235748291 + ], + [ + 0.047049954533576965, + 0.13327544927597046, + 0.8369796872138977, + -1.7121621370315552, + 0.4671766757965088, + 0.3856254518032074, + 0.8522987961769104, + -0.08665236830711365, + 0.2445538192987442, + 0.20203416049480438, + -0.2101040482521057, + -0.45600658655166626, + 0.41030821204185486, + 0.2601064443588257, + 0.0742850974202156, + -0.1906077265739441, + 0.11767930537462234, + 0.060356974601745605, + -0.5563361048698425, + 0.3837745785713196, + 0.4662744104862213 + ], + [ + 0.0006069839000701904, + 0.14383821189403534, + 0.877548336982727, + -1.7358375787734985, + 0.12807147204875946, + 0.3142473101615906, + 0.5067259669303894, + -0.10054326057434082, + 0.22263167798519135, + 0.06229785829782486, + -0.14894098043441772, + -0.7591889500617981, + 0.23409780859947205, + 0.30805134773254395, + 0.06809236854314804, + -0.3235054016113281, + -0.12104277312755585, + 0.29004836082458496, + -0.8910035490989685, + 0.4976123571395874, + 0.2881072759628296 + ], + [ + -0.02980591356754303, + 0.07565620541572571, + 0.8593282699584961, + -1.7921754121780396, + 0.2695343494415283, + 0.5140743255615234, + 0.5875495076179504, + -0.1574336290359497, + 0.2650357484817505, + 0.12073545157909393, + -0.23395144939422607, + -0.7751498818397522, + 0.42846986651420593, + 0.2149398922920227, + 0.01605881378054619, + -0.31510379910469055, + -0.07233808934688568, + 0.21103742718696594, + -0.6775739192962646, + 0.5546218156814575, + 0.290467232465744 + ], + [ + 0.02690904587507248, + 0.058692567050457, + 0.8474045991897583, + -1.8158884048461914, + 0.3521440923213959, + 0.5875942707061768, + 0.5325207710266113, + -0.020325347781181335, + 0.25149399042129517, + 0.0481838583946228, + -0.17914846539497375, + -0.7939544916152954, + 0.19633254408836365, + 0.35983195900917053, + -0.10683563351631165, + -0.3130522668361664, + -0.09634657204151154, + 0.24662891030311584, + -0.6263746619224548, + 0.6420769095420837, + 0.35792800784111023 + ], + [ + 0.10404586791992188, + 0.08910278975963593, + 1.0987014770507812, + -1.8449687957763672, + 0.5307130813598633, + 0.5096020698547363, + 0.7059164643287659, + -0.104676753282547, + 0.21691840887069702, + 0.05176727473735809, + -0.2859335243701935, + -0.5509306192398071, + -0.05278941988945007, + 0.3020511865615845, + -0.043806713074445724, + -0.29579946398735046, + -0.16649185121059418, + 0.051625266671180725, + -0.6595144867897034, + 0.19266369938850403, + 0.3640836477279663 + ] + ], + "generated_utilities": [ + 0.03511819243431091, + 2.0, + 0.03361482918262482, + 0.024507999420166016, + 0.02013157308101654, + 0.03976954519748688, + 0.03125926852226257, + 0.017352595925331116 + ], + "group_id": "ms-27c9f70e97b8850cc3ab", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.09440004825592041, + 0.14095592498779297, + 2.0, + 0.09219714999198914, + 0.05765572190284729, + 0.02530825138092041, + 0.07676126062870026, + 0.06643831729888916, + 0.047928065061569214, + 0.07107676565647125, + 0.05561619997024536, + 0.02124820649623871, + 0.08948668837547302, + 0.10203869640827179, + 0.06119762361049652, + 0.04547174274921417 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.984761953353882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.7230467796325684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.554086923599243 + ], + [ + 1.0343247652053833, + 1.287368655204773, + 1.867376446723938, + -2.4465248584747314, + 1.036996841430664, + -1.2664310932159424, + 1.4399157762527466, + -0.8265262246131897, + -0.28319495916366577, + 0.16331201791763306, + -1.5917816162109375, + -1.5816115140914917, + -0.1664504110813141, + 0.7195659279823303, + 0.29144588112831116, + -0.18517816066741943, + -0.08826595544815063, + -0.8266173601150513, + -0.7933371067047119, + -0.938305139541626, + 0.5359267592430115 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3968086242675781, + 0.5359851121902466, + 0.7176156044006348, + -1.52370023727417, + -0.1481943428516388, + 0.10887271165847778, + 0.0, + -0.12331700325012207, + 0.14385993778705597, + -0.041522324085235596, + 0.16089874505996704, + -0.6189663410186768, + -0.01699581742286682, + 0.0, + -0.07216246426105499, + -0.15073662996292114, + -0.134138286113739, + -0.06049776077270508, + -0.26776671409606934, + -0.019470810890197754, + 0.0 + ] + ], + "predicted_scores": [ + 5.22308874130249, + 4.920225143432617, + 4.658702850341797, + 4.609931945800781, + 4.31458854675293, + 4.129289150238037, + 3.913191795349121, + 3.680939197540283 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1881223807/train1", + "selected_index": 0, + "state_hash": "0f58a11c36c331c9abab00795189dccbc7e0aebf985ca9df67bf9da756f154e3", + "stored_base_utility": 0.09440004825592041, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.10841864347457886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10841864347457886 + }, + "base_progress": 0.10841864347457886, + "base_success": false, + "base_utility": 0.10841864347457886, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0735837072134018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0735837072134018 + }, + { + "contact_quality": null, + "progress": 0.03691600263118744, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03691600263118744 + }, + { + "contact_quality": null, + "progress": 0.04955050349235535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04955050349235535 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0934627503156662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0934627503156662 + }, + { + "contact_quality": null, + "progress": 0.03192968666553497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03192968666553497 + }, + { + "contact_quality": null, + "progress": 0.03504982590675354, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03504982590675354 + } + ], + "candidate_progress": [ + 0.0735837072134018, + 0.03691600263118744, + 0.04955050349235535, + 1.0, + 1.0, + 0.0934627503156662, + 0.03192968666553497, + 0.03504982590675354 + ], + "candidate_source_chart_ids": [ + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + true, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-2867deaca7e98582620b", + "generated_tangents": [ + [ + -0.20429296791553497, + -0.14876165986061096, + 1.3290013074874878, + -1.352597713470459, + -1.7126870155334473, + 0.14912866055965424, + 0.07702407240867615, + -0.3980390727519989, + 0.06364188343286514, + -0.1360224485397339, + 0.11478905379772186, + -1.6165939569473267, + 0.2900989353656769, + 0.009237505495548248, + -0.11541570723056793, + 0.05542559549212456, + 0.420320600271225, + 0.6107426285743713, + -0.5372239351272583, + -0.5682095289230347, + -0.25948551297187805 + ], + [ + -0.10409478843212128, + -0.12906688451766968, + 1.1847866773605347, + -1.091176986694336, + -1.3266043663024902, + 0.37149542570114136, + 0.48397350311279297, + -0.33552107214927673, + 0.19832918047904968, + 0.18402515351772308, + 0.03066684491932392, + -1.435766577720642, + -0.03562536835670471, + 0.27757394313812256, + -0.29364967346191406, + -0.005042250733822584, + 0.4285711348056793, + 0.28734320402145386, + -0.42920011281967163, + -0.5686603784561157, + -0.49396613240242004 + ], + [ + -0.13803307712078094, + -0.1262870877981186, + 1.149946928024292, + -1.1331455707550049, + -1.315546989440918, + 0.3669312596321106, + 0.599021315574646, + -0.28965067863464355, + 0.1906404346227646, + 0.17868971824645996, + 0.013034090399742126, + -1.4281377792358398, + -0.04523172974586487, + 0.11331256479024887, + -0.2622690796852112, + 0.03134184330701828, + 0.3662893772125244, + 0.2786247134208679, + -0.48805147409439087, + -0.6206440925598145, + -0.62861168384552 + ], + [ + -0.06871075928211212, + -0.11510556936264038, + 1.2270950078964233, + -1.3125711679458618, + -1.0540797710418701, + 0.5840364694595337, + 0.32354772090911865, + -0.39099714159965515, + 0.053422100841999054, + 0.1083730161190033, + 0.026813482865691185, + -1.2090234756469727, + 0.19731321930885315, + 0.13857580721378326, + -0.3366085886955261, + 0.05450739338994026, + 0.7534447908401489, + 0.5692466497421265, + -0.5715690851211548, + -0.510240912437439, + -0.04246090352535248 + ], + [ + -0.11054052412509918, + -0.052226945757865906, + 1.1467020511627197, + -1.2065396308898926, + -1.3488514423370361, + 0.520066499710083, + 0.20574578642845154, + -0.23715922236442566, + 0.10530725866556168, + 0.25740188360214233, + 0.005739815533161163, + -1.4566844701766968, + -0.03515699505805969, + 0.46808958053588867, + -0.2493971586227417, + -0.010118776001036167, + 0.5703317523002625, + 0.5680418610572815, + -0.627251386642456, + -0.46288031339645386, + -0.06490163505077362 + ], + [ + -0.04272349178791046, + -0.10831273347139359, + 1.4016040563583374, + -1.3271615505218506, + -1.4304304122924805, + 0.4315873682498932, + 0.05909159779548645, + -0.1402525007724762, + 0.08273803442716599, + -0.12461645156145096, + 0.17579922080039978, + -1.3578174114227295, + 0.18397226929664612, + -0.09607651829719543, + -0.05287511274218559, + 0.07294613122940063, + 0.438185453414917, + 0.6284604072570801, + -0.4433620572090149, + -0.5687687397003174, + -0.2140549123287201 + ], + [ + -0.18141140043735504, + -0.1062038391828537, + 1.3859483003616333, + -1.3377981185913086, + -1.3497600555419922, + 0.41398143768310547, + -0.22816812992095947, + -0.15845686197280884, + 0.10322701930999756, + 0.03054284304380417, + 0.2757967412471771, + -1.4107712507247925, + 0.39524540305137634, + 0.19922864437103271, + -0.0873475968837738, + 0.06450699269771576, + 0.5114575624465942, + 0.6249605417251587, + -0.5176184773445129, + -0.6053492426872253, + -0.32059231400489807 + ], + [ + -0.1258784979581833, + -0.1408306062221527, + 1.1380115747451782, + -1.2596936225891113, + -1.4162650108337402, + 0.22117295861244202, + -0.3289192318916321, + -0.31275537610054016, + 0.07880052924156189, + 0.31489723920822144, + -0.15400105714797974, + -1.4290878772735596, + 0.11055895686149597, + 0.4349272847175598, + -0.3734752833843231, + 0.10176998376846313, + 0.545526385307312, + 0.583720862865448, + -0.6139087080955505, + -0.31675440073013306, + -0.017446845769882202 + ] + ], + "generated_utilities": [ + 0.0735837072134018, + 0.03691600263118744, + 0.04955050349235535, + 2.0, + 2.0, + 0.0934627503156662, + 0.03192968666553497, + 0.03504982590675354 + ], + "group_id": "ms-2867deaca7e98582620b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0924256443977356, + 0.05646410584449768, + 2.0, + 2.0, + 0.183049738407135, + 2.0, + 0.16824382543563843, + 0.1424575299024582, + 0.08230158686637878, + 0.15871968865394592, + 0.16382521390914917, + 0.05336849391460419, + 0.07674244046211243, + 0.10727545619010925, + 0.06594061851501465, + 0.045542001724243164 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.44511327147483826, + 0.05827039107680321, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.2619132101535797, + 0.0, + -0.37528133392333984, + -0.07968637347221375, + -0.07514983415603638, + 0.19317352771759033, + 0.03868463635444641, + -0.06793206930160522, + 0.0, + 0.13187909126281738, + -0.20905672013759613, + 0.030073702335357666, + 0.21597933769226074, + -0.0761370062828064, + -0.11662125587463379, + 0.0 + ], + [ + -0.15486939251422882, + 0.24427275359630585, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.44790399074554443, + 0.0, + -0.37528133392333984, + -0.29508399963378906, + 0.7311322093009949, + 0.19317352771759033, + 0.09925723075866699, + 0.1767415851354599, + 0.0, + -0.5768744349479675, + 0.1300974041223526, + 0.21151533722877502, + 0.21597933769226074, + 0.04431194067001343, + -0.040631115436553955, + 0.0 + ], + [ + -0.29166290163993835, + 0.09681949019432068, + 2.2608249187469482, + -1.3577959537506104, + -2.195330858230591, + -0.13884210586547852, + 2.5595173835754395, + -1.3752813339233398, + 0.390796959400177, + 0.673005223274231, + 1.1931735277175903, + -0.41860872507095337, + -0.24474714696407318, + 0.3015621602535248, + -0.2400326132774353, + 0.14207373559474945, + -0.26512715220451355, + 1.2159793376922607, + -0.3237651586532593, + -0.7466862201690674, + 0.30001604557037354 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.119034767150879, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6031243205070496, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6000320911407471 + ], + [ + -0.3580354154109955, + 0.03320832550525665, + 2.5546932220458984, + -1.138798713684082, + -2.260084629058838, + -0.04964215308427811, + 2.8354909420013428, + -1.6732244491577148, + 0.21277914941310883, + 0.5097208023071289, + 1.387328028678894, + -0.691602349281311, + -0.28483790159225464, + 0.35576167702674866, + -0.0605899840593338, + 0.3764685094356537, + -0.28031954169273376, + 1.063202977180481, + -0.03841540217399597, + -0.5410391092300415, + 0.5580100417137146 + ], + [ + -0.5578174591064453, + 0.3727092146873474, + 2.1929497718811035, + -1.577579140663147, + -1.8962993621826172, + -0.2802738845348358, + 2.1670117378234863, + -1.0752147436141968, + 0.16372516751289368, + 0.6087679862976074, + 0.8854888081550598, + -0.18080148100852966, + -0.18821784853935242, + 0.010591715574264526, + -0.12560874223709106, + 0.15850795805454254, + -0.1898641288280487, + 1.1823316812515259, + 0.06655246019363403, + -0.4014361798763275, + 0.4376043379306793 + ], + [ + -0.43790310621261597, + -0.12778332829475403, + 2.6857643127441406, + -1.6293541193008423, + -1.9850949048995972, + -0.7167636752128601, + 2.465301513671875, + -1.4102312326431274, + 0.2071499079465866, + 0.5742072463035583, + 0.9525313377380371, + -0.34116798639297485, + -0.04760183393955231, + -0.05569759011268616, + -0.4777999520301819, + 0.17186680436134338, + 0.0841393768787384, + 0.7708102464675903, + -0.2784478962421417, + -1.2458651065826416, + 0.46140795946121216 + ], + [ + 0.06170770525932312, + 0.45555436611175537, + 1.6506444215774536, + -0.9394457340240479, + -1.9117670059204102, + -0.6910014152526855, + 2.40145206451416, + -1.7531975507736206, + 0.10217079520225525, + 1.2514894008636475, + 1.3236911296844482, + -0.6504120826721191, + -0.6943472027778625, + 0.7029284238815308, + -0.7387814521789551, + -0.33777421712875366, + -0.8187699317932129, + 1.8448829650878906, + 0.00849190354347229, + -1.394152045249939, + 0.918946385383606 + ] + ], + "predicted_scores": [ + 5.722990989685059, + 5.510813236236572, + 5.459251403808594, + 5.288208961486816, + 5.2233147621154785, + 5.001323223114014, + 4.794158935546875, + 3.848588466644287 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1189392917/train1", + "selected_index": 0, + "state_hash": "e8cf9b95f09f32d8e7942b114792725bd89c392a97ff7e6468ebf09df1bb0d21", + "stored_base_utility": 0.0924256443977356, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.13927479088306427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13927479088306427 + }, + "base_progress": 0.13927479088306427, + "base_success": false, + "base_utility": 0.13927479088306427, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06193861365318298, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06193861365318298 + }, + { + "contact_quality": null, + "progress": 0.060320109128952026, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060320109128952026 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06495748460292816, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06495748460292816 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.06193861365318298, + 0.060320109128952026, + 1.0, + 0.06495748460292816, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + false, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-365d8c6de6de16f66744", + "generated_tangents": [ + [ + -0.0793064683675766, + -0.04591337963938713, + 1.4583491086959839, + -0.593203067779541, + -1.700812816619873, + -0.0134030282497406, + 0.04298475384712219, + -0.10123550891876221, + 0.006442335434257984, + 0.4131436049938202, + -0.08257017284631729, + -1.2423640489578247, + 0.16897401213645935, + 0.18572698533535004, + -0.08300364017486572, + -0.030812395736575127, + 0.3930715322494507, + 0.3161470293998718, + -0.4375561475753784, + -0.6313263177871704, + -0.21668821573257446 + ], + [ + 0.0020146816968917847, + -0.023522598668932915, + 1.520439863204956, + -0.4879358112812042, + -1.5140752792358398, + -0.13699603080749512, + -0.0178375244140625, + -0.12501052021980286, + 0.01563800685107708, + 0.3766712546348572, + 0.0915779173374176, + -1.244942307472229, + 0.11321845650672913, + 0.25958627462387085, + -0.06212689355015755, + -0.009170486591756344, + 0.30351322889328003, + 0.3310752511024475, + -0.4037807583808899, + -0.2220911830663681, + -0.1283075511455536 + ], + [ + -0.20166529715061188, + -0.11073032766580582, + 1.2266387939453125, + -0.7656883001327515, + -1.1065425872802734, + 0.10294624418020248, + -0.036587536334991455, + -0.3598079979419708, + 0.07628034055233002, + 0.19569899141788483, + 0.10280755907297134, + -1.2576781511306763, + 0.2309914529323578, + 0.2545529007911682, + -0.08116030693054199, + -0.1079796850681305, + 0.23373401165008545, + 0.3920135200023651, + -0.257668137550354, + -0.26219671964645386, + 0.03141540288925171 + ], + [ + 0.1136198490858078, + -0.04602309688925743, + 1.4528672695159912, + -0.4521344006061554, + -1.4777216911315918, + -0.03439553081989288, + 0.07159629464149475, + -0.03103436529636383, + 0.07056211680173874, + 0.2610095739364624, + -0.04178794473409653, + -1.2863779067993164, + -0.16831350326538086, + 0.10637831687927246, + -0.06459861993789673, + -0.016607005149126053, + 0.32327282428741455, + 0.37286895513534546, + -0.4181337356567383, + -0.3303793668746948, + -0.00886024534702301 + ], + [ + 0.07044044137001038, + 0.013210754841566086, + 1.4144206047058105, + -0.5527143478393555, + -1.404444932937622, + 0.08353669941425323, + -0.02880951762199402, + -0.011849001049995422, + 0.02369382232427597, + 0.24913308024406433, + -0.10446097701787949, + -1.3512086868286133, + 0.0598885715007782, + 0.15145491063594818, + -0.10982738435268402, + -0.08647167682647705, + 0.40291428565979004, + 0.3543710708618164, + -0.4759449362754822, + -0.4454573392868042, + 0.06395728886127472 + ], + [ + -0.06900635361671448, + 0.12228089570999146, + 1.208706021308899, + -0.48433423042297363, + -0.9781535267829895, + 0.22853729128837585, + -0.11004656553268433, + -0.05800589919090271, + 0.0071481578052043915, + 0.12424708902835846, + -0.11923506110906601, + -1.197339653968811, + -0.20530453324317932, + 0.15419232845306396, + -0.29404327273368835, + -0.114983931183815, + 0.430389940738678, + 0.3056219220161438, + -0.43837523460388184, + -0.37814196944236755, + -0.016082659363746643 + ], + [ + -0.029048293828964233, + -0.01479404792189598, + 1.4270713329315186, + -0.5820334553718567, + -1.4748575687408447, + 0.20198768377304077, + 0.02460220456123352, + -0.16334596276283264, + 0.041051752865314484, + 0.2036067247390747, + -0.1663990616798401, + -1.2124314308166504, + -0.010271072387695312, + 0.19432416558265686, + -0.0633213222026825, + -0.04832817614078522, + 0.40308159589767456, + 0.462817907333374, + -0.4925278425216675, + -0.2808868885040283, + 0.11546452343463898 + ], + [ + 0.04306834936141968, + -0.014999547973275185, + 1.445180058479309, + -0.603166937828064, + -1.408811092376709, + 0.10178462415933609, + 0.0807824432849884, + -0.12137889862060547, + 0.06345739215612411, + 0.24071167409420013, + -0.17246851325035095, + -1.1719390153884888, + -0.12095233798027039, + 0.2868489623069763, + -0.1355857104063034, + -0.0820237398147583, + 0.31586211919784546, + 0.4032604992389679, + -0.5581992268562317, + -0.2176150232553482, + 0.003830462694168091 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.06193861365318298, + 0.060320109128952026, + 2.0, + 0.06495748460292816, + 2.0, + 2.0 + ], + "group_id": "ms-365d8c6de6de16f66744", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20231953263282776, + 0.12319359183311462, + 2.0, + 2.0, + 0.1590207815170288, + 0.019343823194503784, + 0.1322992593050003, + 0.09479346871376038, + 0.1768283247947693, + 0.1054161787033081, + 0.16278749704360962, + 0.08510464429855347, + 0.09564858675003052, + 0.07681193947792053, + 0.1334480196237564, + 0.08148902654647827 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.09058855473995209, + -0.022760659456253052, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.07837581634521484, + 0.0, + -0.05228453874588013, + -0.0679071694612503, + 0.026890814304351807, + 0.03908693790435791, + -1.4526653289794922, + 0.04786187410354614, + 0.0, + 0.019858457148075104, + -0.05534793436527252, + 0.17386651039123535, + -0.024165034294128418, + -0.05880159139633179, + 0.044484615325927734, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.063174724578857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6561365127563477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0078073740005493 + ], + [ + 0.05168074369430542, + 0.27186232805252075, + 2.0911388397216797, + -1.6812132596969604, + -2.3280625343322754, + 0.38023167848587036, + 2.5090978145599365, + -0.2604735791683197, + 0.04725922644138336, + 0.8121852874755859, + -0.6659450531005859, + -2.2462568283081055, + 0.49988994002342224, + 1.9786770343780518, + 0.02324119210243225, + -0.45050501823425293, + 1.1065142154693604, + 0.5724118947982788, + -0.6320602893829346, + -0.7826599478721619, + 0.47463753819465637 + ], + [ + 0.46382713317871094, + -0.06331050395965576, + 2.657292366027832, + -1.1560972929000854, + -2.765941619873047, + 0.7805119752883911, + 2.3117406368255615, + -0.3644295036792755, + -0.25628525018692017, + 0.7623568773269653, + -0.4176616668701172, + -2.646364688873291, + 0.40904325246810913, + 1.7654002904891968, + -0.23315654695034027, + 0.026929572224617004, + 0.8236149549484253, + 0.8322737812995911, + -0.7612741589546204, + -0.4725078046321869, + 0.5895571708679199 + ], + [ + -0.26226741075515747, + 0.3922410011291504, + 2.450082302093506, + -0.9020822048187256, + -3.028742551803589, + 1.0746917724609375, + 3.104015350341797, + 0.3752234876155853, + -0.009036362171173096, + 0.3652568459510803, + -0.7608495950698853, + -2.337242603302002, + -0.03944078087806702, + 1.972575306892395, + -0.32381975650787354, + -0.45855021476745605, + 1.6927499771118164, + 0.8358478546142578, + -0.8610806465148926, + -0.6741821765899658, + 0.4013170003890991 + ], + [ + 0.5790615677833557, + 0.8308188915252686, + 3.141986846923828, + -1.218785285949707, + -2.0300495624542236, + 0.4300342798233032, + 3.0737316608428955, + -0.09167198091745377, + 0.4237035810947418, + 0.7421274185180664, + -0.637946367263794, + -2.1245827674865723, + 0.4184945523738861, + 1.2377147674560547, + -0.4068492650985718, + -0.03615438938140869, + 1.5385099649429321, + 0.51850825548172, + -0.5737811923027039, + -1.472043752670288, + 0.45800793170928955 + ], + [ + 0.38714414834976196, + 0.25513744354248047, + 2.9400248527526855, + -1.8116681575775146, + -1.7500431537628174, + 1.403240442276001, + 1.9709672927856445, + 0.6755956411361694, + 0.49445489048957825, + 0.19490671157836914, + -1.1990764141082764, + -2.803361177444458, + 0.499578058719635, + 1.164682149887085, + 0.9588776230812073, + 0.5154257416725159, + 1.0328537225723267, + 1.1309876441955566, + -0.9365650415420532, + -0.9663279056549072, + -0.032733798027038574 + ], + [ + 0.7417005896568298, + -0.12406697869300842, + 3.0325844287872314, + -0.5656485557556152, + -2.5084800720214844, + 0.27374324202537537, + 2.2886011600494385, + 0.5997821092605591, + 0.9623891115188599, + 0.38451504707336426, + -1.4501185417175293, + -1.6908833980560303, + 0.6743149757385254, + 1.8593690395355225, + -0.018024832010269165, + -0.20444278419017792, + 1.364742636680603, + 0.34230074286460876, + -1.4427728652954102, + -1.5577378273010254, + 0.5589802265167236 + ], + [ + 0.3448556661605835, + -0.5947309732437134, + 2.3888885974884033, + -1.0376405715942383, + -2.218167781829834, + 1.568857192993164, + 3.0995821952819824, + -0.4682580530643463, + 0.3072889745235443, + 0.14532917737960815, + -0.567415177822113, + -3.085655689239502, + -0.6656016111373901, + 1.6447856426239014, + 0.8135664463043213, + 0.3758860230445862, + 1.399398922920227, + 0.11528515815734863, + -0.531513512134552, + -0.6104787588119507, + 1.4474445581436157 + ], + [ + -0.18966330587863922, + -0.7208725214004517, + 2.586766481399536, + -0.6973344087600708, + -3.2187445163726807, + 1.4117884635925293, + 3.0797414779663086, + -0.29149553179740906, + 0.09620378166437149, + 0.874613881111145, + -0.8004729151725769, + -2.0231733322143555, + -0.5948910713195801, + 1.5080900192260742, + -0.750367283821106, + 0.8155628442764282, + 1.6562600135803223, + 0.8091130256652832, + -1.1569771766662598, + -1.4121384620666504, + 0.3738836348056793 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06428352743387222, + -0.015524566173553467, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + -0.1607142686843872, + 0.0, + 0.27928173542022705, + 0.05664636939764023, + -0.03914147615432739, + 0.1700872778892517, + -1.4526653289794922, + 0.05555404722690582, + 0.0, + 0.18240894377231598, + -0.05239863693714142, + 0.17386651039123535, + -0.21921080350875854, + -0.3728579580783844, + 0.11250120401382446, + 0.0 + ], + [ + 0.008915487676858902, + 0.540733277797699, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.22902190685272217, + 0.0, + -0.3544532060623169, + -0.2348114550113678, + 0.04456782341003418, + 0.2907915711402893, + -1.4526653289794922, + -0.1788860559463501, + 0.0, + 0.07762694358825684, + -0.07040916383266449, + 0.17386651039123535, + 0.44059625267982483, + -0.19159233570098877, + 0.09149748086929321, + 0.0 + ] + ], + "predicted_scores": [ + 8.251832962036133, + 8.237282752990723, + 7.584033012390137, + 7.5751519203186035, + 7.472279071807861, + 7.430319786071777, + 7.395534515380859, + 7.284758567810059 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1997090716/train1", + "selected_index": 0, + "state_hash": "98870eab9f394b355af9b8a6d689b3bcd574af94db5d1a9139d0aed0b633b959", + "stored_base_utility": 0.20231953263282776, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03280790150165558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03280790150165558 + }, + "base_progress": 0.03280790150165558, + "base_success": false, + "base_utility": 0.03280790150165558, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.02276340126991272, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02276340126991272 + }, + { + "contact_quality": null, + "progress": 0.022152185440063477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022152185440063477 + }, + { + "contact_quality": null, + "progress": 0.0349019318819046, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0349019318819046 + }, + { + "contact_quality": null, + "progress": 0.022732555866241455, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022732555866241455 + }, + { + "contact_quality": null, + "progress": 0.021209537982940674, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021209537982940674 + }, + { + "contact_quality": null, + "progress": 0.023332014679908752, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023332014679908752 + }, + { + "contact_quality": null, + "progress": 0.020442917943000793, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020442917943000793 + }, + { + "contact_quality": null, + "progress": 0.022646263241767883, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022646263241767883 + } + ], + "candidate_progress": [ + 0.02276340126991272, + 0.022152185440063477, + 0.0349019318819046, + 0.022732555866241455, + 0.021209537982940674, + 0.023332014679908752, + 0.020442917943000793, + 0.022646263241767883 + ], + "candidate_source_chart_ids": [ + "ms-3f6cd47af5e9e6781b73", + "ms-fe38b164397dd669f391", + "ms-2423891634e7b469eed8", + "ms-2cbe83da02847008bf24", + "ms-2cbe83da02847008bf24", + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-3c326d05fec66e1d6bea", + "generated_tangents": [ + [ + 0.40442317724227905, + -0.7725453972816467, + 1.1501598358154297, + 0.17600640654563904, + -0.30155789852142334, + -0.39580318331718445, + -0.01875162124633789, + 0.26155561208724976, + 0.4048941731452942, + -0.02681455761194229, + -0.3096141219139099, + -0.6872912049293518, + -0.36405548453330994, + 0.06340265274047852, + 0.1773771047592163, + -0.10477298498153687, + 0.015425518155097961, + 0.20689745247364044, + 0.04955346882343292, + -0.3902435600757599, + 0.24647946655750275 + ], + [ + 0.07905979454517365, + -0.909687876701355, + 0.9376540184020996, + 0.43800172209739685, + -0.023324698209762573, + -0.23537349700927734, + -0.45633143186569214, + 0.2921690344810486, + 0.6057440638542175, + -0.07589032500982285, + -0.563623309135437, + -0.505800187587738, + -0.07873103022575378, + 0.17948520183563232, + -0.20254218578338623, + -0.10596145689487457, + 0.1744166612625122, + 0.10523907840251923, + 0.24562320113182068, + -0.25142624974250793, + 0.3499114215373993 + ], + [ + 0.12745730578899384, + -0.8414797782897949, + 1.0682847499847412, + 0.2258983850479126, + -0.2101956009864807, + -0.13761919736862183, + -0.23796921968460083, + 0.1290353685617447, + 0.5689228773117065, + -0.03213111311197281, + -0.42360377311706543, + -0.6594093441963196, + 0.03247526288032532, + 0.12020916491746902, + -0.26870110630989075, + -0.13109737634658813, + 0.28618288040161133, + 0.21626964211463928, + 0.12039704620838165, + -0.22269728779792786, + 0.14697611331939697 + ], + [ + 0.29885730147361755, + -0.911459743976593, + 1.0089999437332153, + 0.2503260672092438, + -0.20167016983032227, + -0.3472200036048889, + -0.6033332943916321, + 0.09250596910715103, + 0.5052357912063599, + 0.009694404900074005, + -0.46168193221092224, + -0.6105486154556274, + -0.10397589206695557, + 0.11901242285966873, + -0.3067343533039093, + -0.0861128717660904, + 0.0857761800289154, + 0.08606605976819992, + 0.10337814688682556, + -0.34056398272514343, + 0.31238046288490295 + ], + [ + 0.3344447910785675, + -0.9550344944000244, + 1.1186020374298096, + 0.22837793827056885, + -0.14751920104026794, + -0.3237929940223694, + -0.5763834118843079, + 0.20042864978313446, + 0.5201164484024048, + 0.004939809441566467, + -0.4772302806377411, + -0.6474586725234985, + -0.18470191955566406, + 0.19634193181991577, + -0.2683328688144684, + -0.03984227776527405, + 0.1518973708152771, + 0.03422252833843231, + 0.15046000480651855, + -0.20712175965309143, + 0.3553316593170166 + ], + [ + 0.20436400175094604, + -0.9022265076637268, + 1.1143196821212769, + 0.43258896470069885, + 0.17536607384681702, + -0.18271684646606445, + -0.692637026309967, + 0.212854266166687, + 0.5240234732627869, + 0.03801193833351135, + -0.5399991869926453, + -0.6230814456939697, + -0.15395104885101318, + 0.21651341021060944, + -0.31853556632995605, + -0.025904826819896698, + 0.07033790647983551, + 0.1579502373933792, + 0.06720687448978424, + -0.1857805848121643, + 0.3302036225795746 + ], + [ + 0.16572201251983643, + -0.9039570093154907, + 0.9914646148681641, + 0.3511832654476166, + 0.0878506749868393, + -0.038403719663619995, + -0.7526358962059021, + 0.19371949136257172, + 0.523228645324707, + -0.07204826921224594, + -0.5434116125106812, + -0.7394533157348633, + -0.10109508037567139, + -0.06650365889072418, + -0.10940064489841461, + 0.08765175938606262, + 0.30040937662124634, + 0.18968750536441803, + -0.09119495749473572, + -0.35894227027893066, + 0.4195023477077484 + ], + [ + 0.25426024198532104, + -0.9746049046516418, + 0.9145423173904419, + 0.2769142985343933, + -0.01973554491996765, + -0.06788142025470734, + -0.673834502696991, + 0.0617547333240509, + 0.5495027303695679, + -0.057230062782764435, + -0.5406963229179382, + -0.6831774115562439, + -0.18949377536773682, + 0.19629457592964172, + -0.2762845456600189, + -0.09926141798496246, + 0.1114891842007637, + 0.07077242434024811, + -0.08504712581634521, + -0.26889699697494507, + 0.45010533928871155 + ] + ], + "generated_utilities": [ + 0.02276340126991272, + 0.022152185440063477, + 0.0349019318819046, + 0.022732555866241455, + 0.021209537982940674, + 0.023332014679908752, + 0.020442917943000793, + 0.022646263241767883 + ], + "group_id": "ms-3c326d05fec66e1d6bea", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18246883153915405, + 2.0, + 2.0, + 0.11388173699378967, + 0.035728439688682556, + 0.05159728229045868, + 0.043176501989364624, + 0.04580867290496826, + 0.02321702241897583, + 0.040450215339660645, + 0.04055213928222656, + 0.024657413363456726, + 0.017660364508628845, + 0.044850513339042664, + 0.031234994530677795, + 0.06195336580276489 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.018136948347091675, + 0.10074883699417114, + 0.11574286222457886, + 0.0, + 0.1410461664199829, + 0.38714247941970825, + 0.2381301373243332, + 0.06927800178527832, + -0.6138286590576172, + -0.33484986424446106, + 0.0, + 0.012610852718353271, + 0.5407634973526001, + 0.07784794270992279, + 0.3537199795246124, + -0.1602279543876648, + 0.002093076705932617, + 0.0 + ], + [ + 1.9327397346496582, + -1.7429183721542358, + 1.8877019882202148, + -0.33001917600631714, + 0.49746543169021606, + 0.47390231490135193, + 1.7802670001983643, + -0.8589538335800171, + 0.37976425886154175, + -0.18104670941829681, + -0.8859321475028992, + -1.6138286590576172, + 0.34242650866508484, + 1.2023069858551025, + 0.555707573890686, + -0.35764527320861816, + -0.10027600824832916, + 0.4702257812023163, + -0.6294202208518982, + 0.1926015019416809, + 0.5916687846183777 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.5605340003967285, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.404613971710205, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1833375692367554 + ], + [ + 1.7389304637908936, + -1.4791877269744873, + 2.158487319946289, + -0.3266654312610626, + 0.5907376408576965, + 0.43260636925697327, + 1.9136874675750732, + -0.5760036110877991, + 0.3071907162666321, + -0.4073694348335266, + -0.8863769769668579, + -1.8163083791732788, + 0.5969879627227783, + 1.196010947227478, + 0.7674136161804199, + -0.2368052750825882, + -0.12123071402311325, + 0.36782756447792053, + -0.8187789916992188, + 0.15793970227241516, + 0.43327897787094116 + ], + [ + 1.9560860395431519, + -2.008063793182373, + 1.6910814046859741, + -0.2576735019683838, + 0.647503137588501, + 0.5322431921958923, + 1.9268598556518555, + -1.2501428127288818, + 0.775363564491272, + -0.0965004414319992, + -0.5331029891967773, + -1.639766812324524, + 0.44450435042381287, + 1.5980249643325806, + 0.9402694702148438, + -0.1454993337392807, + -0.2541998028755188, + 0.6709543466567993, + -0.44338259100914, + -0.043697476387023926, + 0.7131086587905884 + ], + [ + 1.4609386920928955, + -1.6896755695343018, + 2.174467086791992, + -0.34207087755203247, + 0.962078332901001, + 0.28904104232788086, + 2.0044631958007812, + -1.184704303741455, + 0.8424319624900818, + -0.6237084865570068, + -0.9171007871627808, + -1.5616872310638428, + 0.3556978702545166, + 1.5233795642852783, + 0.7210954427719116, + 0.028062552213668823, + 0.21759141981601715, + 0.2823620140552521, + -0.588395357131958, + -0.12581703066825867, + 0.7275858521461487 + ], + [ + 2.087728500366211, + -2.0203187465667725, + 2.4627456665039062, + 0.09322953224182129, + 0.4771752953529358, + 0.2439241260290146, + 2.3431313037872314, + -0.7176190614700317, + 0.8298519849777222, + -0.47738587856292725, + -1.4635096788406372, + -1.2598087787628174, + 0.008061617612838745, + 0.9835944175720215, + 0.10656380653381348, + -0.1548040360212326, + 0.20115382969379425, + 0.4581896960735321, + -1.197885274887085, + -0.28882738947868347, + 0.17229679226875305 + ], + [ + 2.535038471221924, + -2.2647459506988525, + 1.280318260192871, + -0.8243153095245361, + 0.36149507761001587, + 0.01678559184074402, + 1.259200096130371, + -0.5383518934249878, + 1.0661393404006958, + -0.1134992390871048, + -1.0009663105010986, + -1.8629896640777588, + 0.5329263806343079, + 1.1952301263809204, + 0.5380017757415771, + -0.21264730393886566, + 0.33232176303863525, + 0.9111320972442627, + -1.0945085287094116, + 0.7546109557151794, + 0.8692140579223633 + ], + [ + 2.3966352939605713, + -2.3338406085968018, + 2.0661728382110596, + -0.1174439787864685, + -0.28071242570877075, + 0.8010929822921753, + 2.485050678253174, + -0.5360689163208008, + 0.9712552428245544, + 0.5487139821052551, + -0.46108368039131165, + -2.0631186962127686, + 0.6046813726425171, + 0.9991111159324646, + 1.299267292022705, + 0.3742367625236511, + -0.7156072854995728, + 0.7805436849594116, + -1.3091955184936523, + -0.5538330674171448, + -0.20684683322906494 + ], + [ + 2.614837884902954, + -2.24919056892395, + 2.2911505699157715, + -0.2939787209033966, + 1.1876089572906494, + 0.15644106268882751, + 1.7348854541778564, + -0.7635170221328735, + 0.5141214728355408, + -0.17918013036251068, + -0.8421157598495483, + -1.5084272623062134, + -0.18822845816612244, + 1.8389434814453125, + 0.24652135372161865, + 0.38509100675582886, + 0.5761538147926331, + -0.15904095768928528, + 0.0013433098793029785, + 0.6120359897613525, + 1.3977106809616089 + ], + [ + 1.5386989116668701, + -2.0177698135375977, + 1.8581953048706055, + -1.2861346006393433, + 1.0945217609405518, + 0.8363389372825623, + 2.708517074584961, + -0.9201098084449768, + 0.8546208739280701, + -0.058566831052303314, + -1.5932974815368652, + -2.015472412109375, + 0.2565937638282776, + 1.988462209701538, + -0.04372602701187134, + -0.8754178881645203, + 0.27551382780075073, + -0.29693683981895447, + -0.2824576497077942, + 0.40926772356033325, + 0.7658611536026001 + ], + [ + 2.8735361099243164, + -2.675261974334717, + 1.459324836730957, + 0.380817174911499, + 0.6231281757354736, + 0.03267902135848999, + 2.296022415161133, + -1.587423324584961, + 0.7833051681518555, + 0.6618785262107849, + -1.7110093832015991, + -1.436572551727295, + -0.14784568548202515, + 1.525026559829712, + 1.4909214973449707, + -1.0601696968078613, + -0.5454426407814026, + -0.09859517216682434, + -1.4344520568847656, + -0.058256953954696655, + -0.3444262146949768 + ], + [ + 2.0854334831237793, + -1.3952012062072754, + 2.651411771774292, + -0.7663257122039795, + -0.05179548263549805, + 0.8797686100006104, + 2.0904030799865723, + -0.8943268656730652, + 1.0378066301345825, + 0.08841629326343536, + -0.36119532585144043, + -1.236596703529358, + 1.0752983093261719, + 0.349973201751709, + -0.19527214765548706, + -0.8867040872573853, + 0.18460334837436676, + 1.454195261001587, + -0.6220338940620422, + 0.9919598698616028, + 1.229588508605957 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.009577363729476929, + -0.06759434938430786, + 0.05406871438026428, + 0.0, + 0.003663182258605957, + 0.0492824912071228, + -0.038241639733314514, + -0.02580159902572632, + -0.6138286590576172, + -0.056222498416900635, + 0.0, + -0.026217222213745117, + -0.051223963499069214, + -0.031640589237213135, + -0.010870158672332764, + 0.08728909492492676, + -0.020528942346572876, + 0.0 + ], + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.06735062599182129, + -0.023807168006896973, + 0.1439460813999176, + 0.0, + -0.19209599494934082, + -0.14847975969314575, + -0.22449488937854767, + 0.09710657596588135, + -0.6138286590576172, + -0.1854294240474701, + 0.0, + -0.17072844505310059, + 0.09961000084877014, + -0.017846792936325073, + 0.22415585815906525, + -0.18760734796524048, + -0.060083091259002686, + 0.0 + ] + ], + "predicted_scores": [ + 5.217313289642334, + 4.2235822677612305, + 4.106942653656006, + 3.900104522705078, + 3.644498348236084, + 3.5433480739593506, + 3.4885756969451904, + 3.16633939743042 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524/train1", + "selected_index": 0, + "state_hash": "6333fcf80f76890398820f27d1a99efe459cdb6ff795201d73e3bcedc966664f", + "stored_base_utility": 0.18246883153915405, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18095605075359344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18095605075359344 + }, + "base_progress": 0.18095605075359344, + "base_success": false, + "base_utility": 0.18095605075359344, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03734047710895538, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03734047710895538 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.077694833278656, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.077694833278656 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.03734047710895538, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.077694833278656, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "generated_tangents": [ + [ + -0.7903228998184204, + -0.18409757316112518, + -0.14013144373893738, + -2.5101399421691895, + -0.9746879935264587, + -0.32710710167884827, + 0.23014850914478302, + -0.35718467831611633, + -0.18316799402236938, + -0.11985679715871811, + 0.04373310133814812, + -1.4271233081817627, + 0.26793184876441956, + 0.49707841873168945, + -0.33715230226516724, + -0.6178942322731018, + 0.7486673593521118, + 0.5280073881149292, + -0.09113270044326782, + -0.6513424515724182, + 0.3264918029308319 + ], + [ + -0.6438853144645691, + -0.21396349370479584, + 0.1571500301361084, + -2.6774978637695312, + -0.9535077214241028, + -1.0579779148101807, + -0.03666716814041138, + -0.33191463351249695, + -0.08599188923835754, + -0.1596742570400238, + 0.0914076641201973, + -0.9398535490036011, + 0.6325937509536743, + 0.6105518341064453, + -0.3370731770992279, + -0.5366656184196472, + 0.6772626042366028, + 0.4953550100326538, + -0.21307837963104248, + -0.4033760130405426, + -0.02836097776889801 + ], + [ + -0.8709462881088257, + -0.32414430379867554, + 0.20400968194007874, + -2.6513242721557617, + -0.7906050086021423, + -1.0159655809402466, + 0.037367284297943115, + -0.05022421479225159, + -0.07747355103492737, + -0.1263035535812378, + 0.0053431689739227295, + -1.0356826782226562, + 0.7024953365325928, + 0.5128731727600098, + -0.34005260467529297, + -0.4671139121055603, + 0.5004689693450928, + 0.5561464428901672, + 0.11805525422096252, + -0.11512330174446106, + -0.008380204439163208 + ], + [ + -0.8215875029563904, + -0.37770628929138184, + -0.01451060175895691, + -2.6621549129486084, + -1.2418372631072998, + -0.8316209316253662, + 0.04187417030334473, + -0.17782306671142578, + -0.043381065130233765, + -0.1426493227481842, + 0.02887875773012638, + -0.9600236415863037, + 0.3839133679866791, + 0.539447009563446, + -0.28831005096435547, + -0.5900147557258606, + 0.35402023792266846, + 0.5745357871055603, + -0.0773160457611084, + -0.39950305223464966, + 0.1341574490070343 + ], + [ + -0.5483207702636719, + -0.2609912157058716, + 0.2517930567264557, + -2.571366310119629, + -1.1163084506988525, + -0.960135817527771, + 0.07333788275718689, + -0.3056282103061676, + -0.052837468683719635, + 0.17141781747341156, + -0.024515755474567413, + -0.7675437927246094, + 0.7875030040740967, + 0.323081374168396, + -0.3825334310531616, + -0.43633607029914856, + 0.32293570041656494, + 0.5625973343849182, + -0.003449380397796631, + -0.2669273614883423, + 0.16390341520309448 + ], + [ + -0.5376284122467041, + -0.23660890758037567, + 0.18803945183753967, + -2.50242018699646, + -1.0672125816345215, + -0.782788872718811, + 0.03912481665611267, + -0.23303145170211792, + -0.03477063030004501, + 0.1316252201795578, + -0.022717773914337158, + -0.8641567230224609, + 0.5775023698806763, + 0.19517473876476288, + -0.3026233911514282, + -0.570724368095398, + 0.3069618344306946, + 0.5117232203483582, + 0.04628857970237732, + -0.13032598793506622, + 0.0027305632829666138 + ], + [ + -0.7411304116249084, + -0.45104676485061646, + -0.06603893637657166, + -2.5610299110412598, + -1.1784672737121582, + -0.6976543664932251, + -0.02025526762008667, + -0.11350211501121521, + -0.05391823500394821, + -0.04498853534460068, + 0.17176511883735657, + -0.9517258405685425, + 0.384677916765213, + 0.39348429441452026, + -0.21921876072883606, + -0.548501193523407, + 0.2978198528289795, + 0.41421887278556824, + -0.1791597306728363, + -0.34533971548080444, + 0.08193863928318024 + ], + [ + -0.960375189781189, + -0.22278209030628204, + 0.1317155659198761, + -2.5167276859283447, + -1.2622530460357666, + -0.9951598644256592, + 0.19187024235725403, + -0.15784722566604614, + 0.027605026960372925, + -0.22090288996696472, + -0.1198325976729393, + -0.900932788848877, + 0.5397241115570068, + 0.024562105536460876, + -0.33324357867240906, + -0.5202938914299011, + 0.4540517032146454, + 0.48335450887680054, + 0.2966420650482178, + -0.1541118025779724, + 0.2457282841205597 + ] + ], + "generated_utilities": [ + 0.03734047710895538, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.077694833278656, + 2.0 + ], + "group_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.17512565851211548, + 2.0, + 2.0, + 2.0, + 0.10078734159469604, + 2.0, + 2.0, + 0.06518048048019409, + 2.0, + 2.0, + 0.05268773436546326, + 0.056524619460105896, + 0.07100173830986023, + 0.16304254531860352, + 2.0, + 0.07095202803611755 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -1.7685134410858154, + 0.4561518430709839, + 0.8554723262786865, + -2.9925076961517334, + -1.652122139930725, + -0.9696183204650879, + 1.440989375114441, + -0.1663060486316681, + 0.005464707035571337, + -0.08078178763389587, + 0.5969918370246887, + -1.7046172618865967, + 0.3472420275211334, + 0.7052499651908875, + -0.026682782918214798, + 0.013620201498270035, + 0.002864290028810501, + 1.5629743337631226, + 0.06437517702579498, + -0.20232005417346954, + 0.06808048486709595 + ], + [ + -1.571894645690918, + 0.49223509430885315, + 1.2358425855636597, + -2.7074813842773438, + -1.6053731441497803, + -0.9737302660942078, + 1.2407286167144775, + -0.41521748900413513, + 0.27269765734672546, + -0.08917579054832458, + 0.4081580936908722, + -1.3627166748046875, + 0.07880738377571106, + 0.7477485537528992, + -0.2180848866701126, + -0.13109977543354034, + -0.3875364661216736, + 1.8194886445999146, + -0.06430798768997192, + 0.07869948446750641, + -0.12621839344501495 + ], + [ + -1.7744590044021606, + 0.367733359336853, + 1.3591725826263428, + -3.3725903034210205, + -1.8434977531433105, + -0.8745664954185486, + 1.6967558860778809, + -0.44676536321640015, + -0.6177129149436951, + 0.03244173526763916, + 0.4171296954154968, + -1.4895254373550415, + 0.9296896457672119, + 0.15846145153045654, + -0.006281482055783272, + 0.6454329490661621, + 0.6225314140319824, + 1.6578954458236694, + 0.3061527609825134, + -0.41300463676452637, + -0.2527916431427002 + ], + [ + -1.7233139276504517, + 0.020191192626953125, + 1.0188184976577759, + -2.873849868774414, + -2.027033567428589, + -1.1650824546813965, + 2.0792155265808105, + -0.8799198865890503, + 0.6924232840538025, + -0.7650182247161865, + -0.04520058631896973, + -2.468881130218506, + -0.13257521390914917, + 0.6857031583786011, + -0.10177457332611084, + -0.7609032988548279, + -0.08249430358409882, + 1.7503252029418945, + 0.1621951013803482, + -0.22467197477817535, + 0.43942898511886597 + ], + [ + -2.3507251739501953, + 0.16114166378974915, + 1.5912489891052246, + -2.1838369369506836, + -2.061230182647705, + -1.7918524742126465, + 0.8407515287399292, + -0.3446531295776367, + -0.7727420926094055, + -0.2617931067943573, + 0.6622774004936218, + -1.05295991897583, + 0.13840271532535553, + 1.2008695602416992, + -0.509428858757019, + 0.7052024006843567, + -0.6614989042282104, + 2.0029079914093018, + -0.82997065782547, + -0.9390356540679932, + 0.7515151500701904 + ], + [ + -1.7504347562789917, + -0.13171464204788208, + 0.7620843648910522, + -3.4511537551879883, + -2.5023341178894043, + -0.8595067858695984, + 1.6361833810806274, + -0.4263937175273895, + 0.005345506593585014, + -0.8147202730178833, + 0.4142894744873047, + -0.8735606670379639, + -0.4809738099575043, + 0.9412479996681213, + -0.6222366690635681, + -0.8843234777450562, + -0.8842994570732117, + 2.4228873252868652, + 0.6301752328872681, + 0.727215051651001, + 0.5050253868103027 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.7685134410858154, + -0.04630154371261597, + -0.002802610397338867, + -1.9925076961517334, + -0.6521221399307251, + -0.013848721981048584, + 0.0, + 0.014648675918579102, + -0.0837627500295639, + -0.04761720821261406, + -0.024705946445465088, + -0.7046172618865967, + 0.0914837121963501, + 0.0, + 0.04683725908398628, + 0.048148274421691895, + 0.03868643194437027, + 0.5629743337631226, + -0.043637245893478394, + -0.08927876502275467, + 0.0 + ], + [ + -0.7685134410858154, + 0.1268729865550995, + -0.0878140926361084, + -1.9925076961517334, + -0.6521221399307251, + -0.08497345447540283, + 0.0, + -0.23278282582759857, + 0.1505891978740692, + -0.0029508620500564575, + -0.03223156929016113, + -0.7046172618865967, + -0.18528565764427185, + 0.0, + 0.009110454469919205, + 0.006383728235960007, + -0.012130538001656532, + 0.5629743337631226, + 0.23608635365962982, + -0.06009632349014282, + 0.0 + ], + [ + -0.7685134410858154, + 0.6076252460479736, + 0.2745842933654785, + -1.9925076961517334, + -0.6521221399307251, + 0.03038167953491211, + 0.0, + 0.33581337332725525, + 0.35011863708496094, + -0.06728862971067429, + -0.4030081629753113, + -0.7046172618865967, + -0.44776007533073425, + 0.0, + -0.01595483534038067, + -0.040477462112903595, + 0.28290560841560364, + 0.5629743337631226, + -0.42285358905792236, + -0.11823103576898575, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.881978750228882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.410499930381775, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.1361609697341919 + ], + [ + -1.5711963176727295, + 0.5461994409561157, + 0.8384940028190613, + -2.960416078567505, + -1.9055238962173462, + -0.7765143513679504, + 1.4308912754058838, + 0.10734349489212036, + 0.011199884116649628, + -0.028424743562936783, + 0.5893871784210205, + -1.7020739316940308, + 0.48640453815460205, + 0.9950923323631287, + 0.20675025880336761, + 0.11047534644603729, + 0.09110014140605927, + 1.3974875211715698, + -0.21079741418361664, + -0.3005509674549103, + 0.09192514419555664 + ], + [ + -2.12906813621521, + 0.3435881733894348, + 0.7164700627326965, + -2.750021457672119, + -1.3681434392929077, + -0.8937081694602966, + 1.6475064754486084, + -0.1447279453277588, + -0.44103574752807617, + -0.5038806200027466, + 0.9006609320640564, + -1.8256494998931885, + 0.433420330286026, + 0.3506351411342621, + 0.4619242548942566, + -0.3905109167098999, + -0.01803053542971611, + 1.9505069255828857, + -0.339522123336792, + -0.20715753734111786, + -0.410531610250473 + ], + [ + -1.194382667541504, + 0.9652969837188721, + 0.41722428798675537, + -2.6203644275665283, + -1.139541506767273, + -1.5412108898162842, + 1.667648434638977, + -0.5256404876708984, + 0.17716272175312042, + 0.30056339502334595, + 0.09983009099960327, + -1.8219844102859497, + -0.23339274525642395, + 0.5763673782348633, + -0.4285883605480194, + 0.1476086974143982, + -0.3515554666519165, + 1.2458031177520752, + 0.3135184645652771, + -0.33711761236190796, + -0.09264840185642242 + ], + [ + -2.0614774227142334, + -0.4080510139465332, + 0.5476135015487671, + -3.0662269592285156, + -1.497420310974121, + -0.11606889963150024, + 1.03195321559906, + -0.3406473994255066, + 0.12844708561897278, + -0.4317864775657654, + 1.035879135131836, + -0.7729326486587524, + 1.0862895250320435, + 0.16315871477127075, + -0.14380013942718506, + -0.8498450517654419, + 0.9780882000923157, + 1.0358695983886719, + -0.5399289727210999, + -0.10386796295642853, + 0.8144621849060059 + ] + ], + "predicted_scores": [ + 7.710109710693359, + 6.939748764038086, + 6.586607933044434, + 6.2206501960754395, + 6.1800737380981445, + 6.078464984893799, + 5.686131000518799, + 5.517265319824219 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "794181626/train1", + "selected_index": 0, + "state_hash": "4043b3c43ca4f115e1ea4fbcd98c7d08f797df2efbfe6b67bcffa5592a8aa36d", + "stored_base_utility": 0.17512565851211548, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08194400370121002, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08194400370121002 + }, + { + "contact_quality": null, + "progress": 0.09846144914627075, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09846144914627075 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.009868413209915161, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009868413209915161 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.2353062629699707e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.2353062629699707e-05 + } + ], + "candidate_progress": [ + 1.0, + 0.08194400370121002, + 0.09846144914627075, + 1.0, + 0.009868413209915161, + 1.0, + 1.0, + 1.2353062629699707e-05 + ], + "candidate_source_chart_ids": [ + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + false, + true, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-58950fed04d203b97ab9", + "generated_tangents": [ + [ + -0.1266256421804428, + -0.002259131520986557, + 1.6213842630386353, + -0.7543456554412842, + -1.6134655475616455, + 0.1789816915988922, + 0.22414876520633698, + -0.13272294402122498, + 0.005732003599405289, + 0.31355682015419006, + 0.10824506729841232, + -1.4673373699188232, + 0.34103503823280334, + 0.4597412943840027, + 0.1806682050228119, + -0.0032630981877446175, + -0.08528648316860199, + -0.011055469512939453, + -0.704582929611206, + 0.005777209997177124, + 0.024259299039840698 + ], + [ + -0.26199430227279663, + 0.005657803267240524, + 1.5621333122253418, + -0.6827386617660522, + -1.7485694885253906, + 0.5028383731842041, + 0.3064003586769104, + -0.09494590759277344, + -0.04572809487581253, + 0.3929176330566406, + -0.07712054997682571, + -1.3926876783370972, + 0.45926740765571594, + 0.45114612579345703, + 0.1326114982366562, + 0.008343080058693886, + 0.029313594102859497, + -0.019701898097991943, + -0.5967046618461609, + -0.2046085149049759, + -0.039422422647476196 + ], + [ + -0.030679523944854736, + 0.10015365481376648, + 1.5865848064422607, + -0.7421698570251465, + -1.6257531642913818, + 0.35852259397506714, + 0.30258017778396606, + 0.0038167834281921387, + 0.05335818976163864, + 0.26822400093078613, + -0.003624599426984787, + -1.505874752998352, + 0.12448570132255554, + 0.1746404618024826, + 0.17631147801876068, + 0.019554976373910904, + -0.028907790780067444, + 0.029448434710502625, + -0.6635255813598633, + 0.011510729789733887, + 0.060159601271152496 + ], + [ + -0.37533628940582275, + -0.009217534214258194, + 1.356131911277771, + -0.9588888883590698, + -1.2638320922851562, + 0.2709219753742218, + 0.1071615219116211, + -0.28787726163864136, + -0.024955475702881813, + 0.18631234765052795, + -0.004405900835990906, + -1.5419903993606567, + 0.4139575660228729, + 0.3009853959083557, + 0.09969498217105865, + -0.07751719653606415, + -0.08473540842533112, + 0.14005310833454132, + -0.4224899411201477, + -0.0025836527347564697, + -0.0714116245508194 + ], + [ + -0.24527902901172638, + 0.20554082095623016, + 1.314713478088379, + -0.7239284515380859, + -1.1570618152618408, + 0.689300537109375, + 0.32333698868751526, + 0.02252228558063507, + -0.08762882649898529, + 0.07853666692972183, + -0.09377797693014145, + -1.4810137748718262, + 0.02623167634010315, + 0.2645219564437866, + -0.027446914464235306, + -0.059286776930093765, + -0.014142453670501709, + -0.022253021597862244, + -0.6485402584075928, + 0.06886330246925354, + 0.055783092975616455 + ], + [ + -0.18931667506694794, + 0.11410576105117798, + 1.4706931114196777, + -0.8522800207138062, + -1.6707148551940918, + 0.5215132832527161, + 0.09922018647193909, + -0.11198189854621887, + 0.00743284123018384, + 0.1963292956352234, + -0.09166524559259415, + -1.4035396575927734, + 0.2550733983516693, + 0.3028809428215027, + 0.10759639739990234, + -0.017149310559034348, + 0.11416398733854294, + 0.16294322907924652, + -0.5847540497779846, + -0.00019565224647521973, + 0.02579958736896515 + ], + [ + -0.1178145557641983, + 0.11055108904838562, + 1.5651174783706665, + -0.8255586624145508, + -1.5410065650939941, + 0.4679739773273468, + 0.1807832419872284, + 0.025051862001419067, + -0.01573147065937519, + 0.2161959707736969, + -0.04220806807279587, + -1.5332847833633423, + 0.3336579501628876, + 0.25811365246772766, + 0.053960882127285004, + -0.017516309395432472, + 0.06422293186187744, + 0.04104362428188324, + -0.6408693194389343, + -0.12048624455928802, + 0.13836044073104858 + ], + [ + -0.11564154922962189, + 0.105094775557518, + 1.4515327215194702, + -0.8464374542236328, + -1.6001012325286865, + 0.3799157440662384, + 0.13040688633918762, + -0.06551563739776611, + 0.01518826000392437, + 0.24243757128715515, + -0.11430519074201584, + -1.3762128353118896, + 0.15279600024223328, + 0.3622298836708069, + 0.05797265097498894, + -0.039949700236320496, + 0.04767826199531555, + 0.1508132517337799, + -0.6215991377830505, + 0.07235613465309143, + -0.13865765929222107 + ] + ], + "generated_utilities": [ + 2.0, + 0.08194400370121002, + 0.09846144914627075, + 2.0, + 0.009868413209915161, + 2.0, + 2.0, + 1.2353062629699707e-05 + ], + "group_id": "ms-58950fed04d203b97ab9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20183078944683075, + 0.18959736824035645, + 2.0, + 0.0385412722826004, + 0.15877258777618408, + 0.03365510702133179, + 0.17151890695095062, + 0.13845136761665344, + 0.11539244651794434, + 0.1388089954853058, + 0.15365298092365265, + 0.0967668741941452, + 0.0908290445804596, + 0.06094485521316528, + 0.0377749502658844, + 0.028643101453781128 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.028361663222312927, + -0.489355206489563, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.16713106632232666, + 0.0, + 0.5545767545700073, + -0.18274790048599243, + 0.2772042155265808, + -0.45601847767829895, + -1.3811633586883545, + -0.1571006327867508, + 0.0, + -0.004826217889785767, + 0.2916078567504883, + 0.34807145595550537, + -0.37882840633392334, + 0.08759653568267822, + -0.07658697664737701, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.797813892364502, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.654926061630249, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4572639465332031 + ], + [ + 0.17793144285678864, + 0.008656054735183716, + 2.343496322631836, + -1.9455183744430542, + -2.4697093963623047, + 1.0036901235580444, + 2.5124316215515137, + -0.29308056831359863, + -0.02035735547542572, + -0.059014320373535156, + -0.5863867402076721, + -2.254472255706787, + 0.3354390263557434, + 1.381989598274231, + -0.27729421854019165, + 0.5023907423019409, + -0.8654648661613464, + 0.39588332176208496, + -0.5682056546211243, + 0.20400357246398926, + 0.48529666662216187 + ], + [ + -0.5227808952331543, + 0.12448178231716156, + 2.4855635166168213, + -1.6602094173431396, + -2.3857760429382324, + 0.323077529668808, + 1.9053025245666504, + -0.698577344417572, + 0.619890034198761, + 0.3500756621360779, + -0.1476593315601349, + -2.8024404048919678, + -0.21084021031856537, + 1.0928113460540771, + -0.3938775062561035, + 0.4228009581565857, + -0.39060449600219727, + -0.31665533781051636, + -0.9205541610717773, + 0.03774743899703026, + 0.3333621621131897 + ], + [ + 0.10275222361087799, + 0.08925776183605194, + 2.0934102535247803, + -1.379586935043335, + -3.3245811462402344, + 0.6494860649108887, + 1.990661382675171, + -0.021198302507400513, + 0.4400750398635864, + 0.39176517724990845, + -0.8639137148857117, + -1.981496810913086, + 0.01414397917687893, + 1.2019504308700562, + -0.7839368581771851, + 0.108054518699646, + -0.17177742719650269, + -0.36239588260650635, + -0.5078206062316895, + 0.4236416518688202, + 0.25084057450294495 + ], + [ + -0.8110343217849731, + -0.11216431856155396, + 2.8897132873535156, + -1.319732427597046, + -3.1141648292541504, + 0.416613906621933, + 2.3244988918304443, + -0.6579729914665222, + -0.39938226342201233, + -0.08646327257156372, + -0.3429824113845825, + -2.4165658950805664, + -0.6488300561904907, + 1.4793217182159424, + -0.8628695011138916, + 0.6845848560333252, + -0.7794754505157471, + -0.014068186283111572, + -0.27444708347320557, + 0.8030790686607361, + 0.08975183963775635 + ], + [ + -0.23147638142108917, + -0.26693814992904663, + 2.8227345943450928, + -2.3573813438415527, + -3.3905396461486816, + 0.38961973786354065, + 1.9831907749176025, + -1.2505497932434082, + 0.00624750554561615, + 0.9689428210258484, + 0.11981219053268433, + -1.5865073204040527, + 0.4901374876499176, + 0.6158997416496277, + 0.3767610788345337, + -0.025944411754608154, + -1.4681568145751953, + 0.48217374086380005, + -1.5973502397537231, + -0.7199738621711731, + 0.5119494795799255 + ], + [ + -0.19429966807365417, + 0.7059361934661865, + 3.2694599628448486, + -2.656050682067871, + -2.5502617359161377, + -0.23801833391189575, + 1.628230333328247, + 0.36624830961227417, + -0.08827590942382812, + -0.2098810374736786, + -1.1745270490646362, + -2.2196781635284424, + 0.44475334882736206, + 0.48411858081817627, + -0.25864526629447937, + 0.2009485512971878, + -0.9008907079696655, + -0.3571908473968506, + -0.16894888877868652, + -0.6256314516067505, + -0.4078131914138794 + ], + [ + 0.22205571830272675, + 0.6026778221130371, + 1.8579424619674683, + -2.404672384262085, + -3.728808641433716, + -0.19806396961212158, + 2.8292319774627686, + -0.2435564398765564, + 0.7903643846511841, + -0.376501202583313, + -1.0294175148010254, + -1.7672805786132812, + 0.5992995500564575, + 2.044847249984741, + 0.0011147260665893555, + 0.18739356100559235, + -0.952850341796875, + 0.8702673316001892, + 0.038269996643066406, + -0.4372258484363556, + -0.37835830450057983 + ], + [ + -0.3783646821975708, + 0.08899453282356262, + 2.721517324447632, + -2.3160932064056396, + -2.0679585933685303, + 0.549463152885437, + 3.141833543777466, + 0.09782099723815918, + 0.5830706357955933, + 0.7134941816329956, + 0.2986952066421509, + -2.9980335235595703, + -0.6391466856002808, + 1.075132131576538, + -0.7696593403816223, + 0.4343666732311249, + -0.911625862121582, + -0.31454378366470337, + -1.5935450792312622, + -0.8993679881095886, + -0.3811922073364258 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.1209426149725914, + 0.10675475001335144, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.0875239372253418, + 0.0, + -0.0486987829208374, + 0.06580346822738647, + 0.24258655309677124, + -0.20229026675224304, + -1.3811633586883545, + 0.4185009002685547, + 0.0, + -0.1431063860654831, + 0.15154658257961273, + 0.17190229892730713, + 0.10405468940734863, + 0.036865830421447754, + -0.1988307237625122, + 0.0 + ] + ], + "predicted_scores": [ + 7.98698091506958, + 7.975624084472656, + 7.712305545806885, + 7.680033206939697, + 7.596920013427734, + 7.543159008026123, + 7.519227504730225, + 7.204469203948975 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113371261/train1", + "selected_index": 0, + "state_hash": "1b9018834a77531260bf8dfa74a9832916fe215bda78b5367b061bb4e0ce2333", + "stored_base_utility": 0.20183078944683075, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.01641605794429779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01641605794429779 + }, + "base_progress": 0.01641605794429779, + "base_success": false, + "base_utility": 0.01641605794429779, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1069791316986084, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1069791316986084 + }, + { + "contact_quality": null, + "progress": 0.05026264488697052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05026264488697052 + }, + { + "contact_quality": null, + "progress": 0.09185349941253662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09185349941253662 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08999918401241302, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08999918401241302 + }, + { + "contact_quality": null, + "progress": 0.1404910683631897, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1404910683631897 + } + ], + "candidate_progress": [ + 1.0, + 0.1069791316986084, + 0.05026264488697052, + 0.09185349941253662, + 1.0, + 1.0, + 0.08999918401241302, + 0.1404910683631897 + ], + "candidate_source_chart_ids": [ + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-9f61e09c05b4c8f74366", + "ms-9f61e09c05b4c8f74366" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-651bc0395d78d2819158", + "generated_tangents": [ + [ + -0.504991352558136, + -0.21939893066883087, + 1.3444803953170776, + -0.743743896484375, + -1.3850276470184326, + -0.031286150217056274, + 1.1078267097473145, + -0.10409319400787354, + 0.06178978830575943, + -0.17713001370429993, + -0.0072335004806518555, + -1.1815955638885498, + 0.18793776631355286, + 0.005217045545578003, + -0.26193767786026, + -0.16495974361896515, + 0.25650161504745483, + 1.0371205806732178, + 0.04905816912651062, + -0.4537602663040161, + 0.5095398426055908 + ], + [ + -0.35190099477767944, + -0.2557993531227112, + 1.6199679374694824, + -0.9254664182662964, + -1.6567935943603516, + -0.3687204420566559, + 1.2310676574707031, + -0.07828259468078613, + -0.05586294084787369, + -0.5629380345344543, + 0.2418472170829773, + -1.8919719457626343, + 0.3157327473163605, + 0.3827357888221741, + -0.23709455132484436, + -0.34630805253982544, + 0.32932648062705994, + 1.38849675655365, + -0.07390868663787842, + -0.10740266740322113, + 0.666833758354187 + ], + [ + -0.34218829870224, + -0.27783042192459106, + 1.4953527450561523, + -1.1027888059616089, + -1.684415340423584, + -0.512134313583374, + 1.1624705791473389, + -0.330881804227829, + -0.15998603403568268, + -0.36967188119888306, + 0.10534857213497162, + -1.759790301322937, + 0.3465334475040436, + 0.548625111579895, + -0.18405956029891968, + -0.38847607374191284, + 0.3852667808532715, + 1.4972847700119019, + -0.4169442057609558, + -0.24077847599983215, + 0.5337702035903931 + ], + [ + -0.34306859970092773, + -0.2906871736049652, + 1.4421676397323608, + -1.031981348991394, + -1.768282413482666, + -0.7896502017974854, + 1.0640686750411987, + -0.1864977478981018, + -0.10034267604351044, + -0.377536416053772, + 0.04089305177330971, + -1.8937641382217407, + 0.4347366392612457, + 0.3884339928627014, + -0.15927137434482574, + -0.3172740340232849, + 0.4115046262741089, + 1.4296550750732422, + -0.29879748821258545, + -0.2692466974258423, + 0.6491756439208984 + ], + [ + -0.4007565379142761, + -0.19153250753879547, + 1.4736835956573486, + -0.9515355825424194, + -1.1882541179656982, + -0.3834197223186493, + 0.5090863108634949, + -0.011358320713043213, + -0.0645751953125, + -0.3433118462562561, + 0.08590733259916306, + -1.5041146278381348, + 0.4623602330684662, + 0.184320330619812, + -0.04183097556233406, + -0.29643651843070984, + 0.5616769194602966, + 1.236170768737793, + -0.08876863121986389, + -0.4893401861190796, + 0.5256282091140747 + ], + [ + -0.5128392577171326, + -0.2719690799713135, + 1.5210754871368408, + -1.0445642471313477, + -1.494227409362793, + -0.4725043475627899, + 0.9738064408302307, + -0.22282347083091736, + -0.1320306956768036, + -0.44933080673217773, + 0.061698026955127716, + -1.736612319946289, + 0.35390862822532654, + 0.40314024686813354, + -0.10215525329113007, + -0.39843082427978516, + 0.468929260969162, + 1.4518229961395264, + -0.31548869609832764, + -0.2277843952178955, + 0.580947995185852 + ], + [ + -0.3465387225151062, + -0.32379454374313354, + 1.3886327743530273, + -0.7451854944229126, + -1.4972631931304932, + 0.1540457159280777, + 0.280448317527771, + 0.03724488615989685, + 0.0381455272436142, + -0.16173213720321655, + 0.10003554075956345, + -1.3670969009399414, + 0.22218278050422668, + 0.24737313389778137, + -0.3057235777378082, + -0.20689114928245544, + 0.4633018970489502, + 1.149208903312683, + -0.19519853591918945, + -0.471104234457016, + 0.6061217784881592 + ], + [ + -0.3432949185371399, + -0.2844322621822357, + 1.3986402750015259, + -0.8788204193115234, + -1.512843370437622, + -0.16554951667785645, + 0.3919046223163605, + 0.0830933079123497, + 0.02433175966143608, + -0.36533844470977783, + -0.01762431114912033, + -1.3434616327285767, + 0.05337068438529968, + 0.29428839683532715, + -0.1534549444913864, + -0.22810466587543488, + 0.5390974879264832, + 1.1905592679977417, + -0.23922199010849, + -0.4439091384410858, + 0.5905386209487915 + ] + ], + "generated_utilities": [ + 2.0, + 0.1069791316986084, + 0.05026264488697052, + 0.09185349941253662, + 2.0, + 2.0, + 0.08999918401241302, + 0.1404910683631897 + ], + "group_id": "ms-651bc0395d78d2819158", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 7.152557373046875e-07, + 2.0, + 0.0, + 0.0, + 0.18490922451019287, + 0.0, + 0.18801423907279968, + 2.0, + 0.18292954564094543, + 0.06743006408214569, + 2.0, + 0.08718416094779968, + 0.0016145259141921997, + 2.0, + 0.0, + 0.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.13822472095489502, + 0.052123308181762695, + 1.2542681694030762, + -0.5262794494628906, + -1.515254020690918, + -0.022995710372924805, + 0.0, + 0.07829460501670837, + 0.056280869990587234, + -0.006563127040863037, + 0.00594717264175415, + -1.0464823246002197, + 0.028334707021713257, + 0.0, + 0.04214516654610634, + -0.0032968670129776, + 0.025241270661354065, + 1.0898823738098145, + -0.09987176954746246, + 0.009208589792251587, + 0.0 + ], + [ + -1.138224720954895, + -0.12241673469543457, + 2.254268169403076, + -1.5262794494628906, + -2.515254020690918, + -0.4036164879798889, + 2.5164785385131836, + -0.48240265250205994, + 0.09632497280836105, + -1.0002868175506592, + 0.6668505668640137, + -2.0464823246002197, + 0.5158075094223022, + 0.48990970849990845, + 0.02257649227976799, + -0.23748205602169037, + -0.2213379293680191, + 2.0898823738098145, + 0.1738651841878891, + -0.37470543384552, + -0.06145218014717102 + ], + [ + -1.0695267915725708, + -0.1775755137205124, + 2.1683619022369385, + -1.4577077627182007, + -2.7917678356170654, + -0.632304310798645, + 2.3911960124969482, + -0.4637458324432373, + 0.35316434502601624, + -0.8419650793075562, + 0.8253071308135986, + -1.9648017883300781, + 0.2602742314338684, + 0.4741981625556946, + 0.17542533576488495, + -0.15374207496643066, + -0.4857789874076843, + 2.119079828262329, + 0.44966256618499756, + -0.3760189712047577, + 0.18533895909786224 + ], + [ + -0.7955029010772705, + 0.23637625575065613, + 1.9978463649749756, + -1.8726613521575928, + -2.2562904357910156, + -0.3048732876777649, + 2.261690378189087, + -0.6620026230812073, + 0.06043636053800583, + -1.091504693031311, + 1.038455605506897, + -1.9805728197097778, + 0.7370907068252563, + 0.5352527499198914, + 0.26298987865448, + -0.2739749252796173, + 0.14720074832439423, + 2.0173659324645996, + -0.16336800158023834, + -0.5858949422836304, + -0.1647021770477295 + ], + [ + -1.6335437297821045, + -0.08274295926094055, + 1.833235502243042, + -1.758880853652954, + -2.0250279903411865, + -0.8149738311767578, + 2.4238271713256836, + -0.33182430267333984, + -0.16191768646240234, + -1.4929053783416748, + 0.8467401266098022, + -2.4791038036346436, + 0.2757382392883301, + 0.16471439599990845, + 0.18122689425945282, + -0.2995648682117462, + -0.34926944971084595, + 2.063732147216797, + 0.3726923167705536, + -0.259366512298584, + 0.36020517349243164 + ], + [ + -0.9912663698196411, + -0.19695568084716797, + 2.342681646347046, + -2.1077044010162354, + -3.031508684158325, + -0.6591198444366455, + 2.5516788959503174, + -0.8631293773651123, + 0.2481168806552887, + -1.107275366783142, + 0.5796921253204346, + -2.479954719543457, + 1.0449233055114746, + 0.40612131357192993, + -0.12985309958457947, + -0.18829946219921112, + -0.6196849346160889, + 1.6295442581176758, + 0.4197757840156555, + 0.02911210060119629, + 0.5378711223602295 + ], + [ + -0.6689683198928833, + -0.5626275539398193, + 1.738625407218933, + -1.919468879699707, + -2.6463799476623535, + -0.32590651512145996, + 2.287154197692871, + -0.9975759983062744, + 0.7224722504615784, + -1.427150845527649, + 0.572014331817627, + -2.2473490238189697, + 0.8538937568664551, + 0.0030456185340881348, + -0.1059482991695404, + -0.003734409809112549, + -0.2650519907474518, + 2.554786205291748, + -0.24461163580417633, + -0.6881535649299622, + 0.5301691293716431 + ], + [ + -1.5373890399932861, + 0.42636638879776, + 2.9180266857147217, + -2.126905679702759, + -1.825490117073059, + -0.6280819773674011, + 2.593716621398926, + -0.6465503573417664, + 0.5798410177230835, + -0.5376753807067871, + 0.5435326099395752, + -2.610151767730713, + 1.0700397491455078, + 0.7953996658325195, + 0.5072687864303589, + -0.02665230631828308, + -0.6953800320625305, + 2.0521392822265625, + 0.751386821269989, + -0.7565212249755859, + -0.5263289213180542 + ], + [ + -0.7768699526786804, + -0.18900753557682037, + 1.7192409038543701, + -2.2835047245025635, + -1.713951826095581, + 0.004747450351715088, + 3.2709672451019287, + -1.1866822242736816, + 0.38759592175483704, + -0.7857802510261536, + 1.539168119430542, + -1.3250088691711426, + 0.8453556299209595, + 0.5691420435905457, + -0.701140820980072, + -0.6115058660507202, + -0.8829987645149231, + 1.5742011070251465, + -0.3939879536628723, + 0.45414644479751587, + -0.23744438588619232 + ] + ], + "predicted_scores": [ + 7.084019184112549, + 7.08164119720459, + 7.0393757820129395, + 6.938310623168945, + 6.919251918792725, + 6.6739912033081055, + 6.482789039611816, + 6.373105525970459 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1483645336/train1", + "selected_index": 0, + "state_hash": "857c28f61d13d7ee5a59956a87579f6c17f6c601c1680e574691e095e373d439", + "stored_base_utility": 7.152557373046875e-07, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15461373329162598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15461373329162598 + }, + "base_progress": 0.15461373329162598, + "base_success": false, + "base_utility": 0.15461373329162598, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.04685409367084503, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04685409367084503 + }, + { + "contact_quality": null, + "progress": 0.025345027446746826, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025345027446746826 + }, + { + "contact_quality": null, + "progress": 0.04131391644477844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04131391644477844 + }, + { + "contact_quality": null, + "progress": 0.04660575091838837, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04660575091838837 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.04232649505138397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04232649505138397 + }, + { + "contact_quality": null, + "progress": 0.01882496476173401, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01882496476173401 + } + ], + "candidate_progress": [ + 0.04685409367084503, + 0.025345027446746826, + 0.04131391644477844, + 0.04660575091838837, + 1.0, + 1.0, + 0.04232649505138397, + 0.01882496476173401 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-1c4df81cce643ca4fb3e" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-9fad82fd288e4296e354", + "generated_tangents": [ + [ + 0.20766592025756836, + -0.09938950836658478, + 1.423433780670166, + -0.17177465558052063, + -1.3757376670837402, + -0.15777432918548584, + -0.24454718828201294, + -0.14580971002578735, + 0.26739832758903503, + 0.43735209107398987, + 0.041180569678545, + -1.3525785207748413, + -0.08127036690711975, + -0.23888324201107025, + -0.17473816871643066, + 0.14641612768173218, + 0.3547530770301819, + 0.006695836782455444, + -0.4463213086128235, + -0.29608821868896484, + -0.10081437230110168 + ], + [ + 0.34539926052093506, + -0.1247783973813057, + 1.5992226600646973, + -0.4015488028526306, + -1.4918618202209473, + -0.004178911447525024, + -0.8372355103492737, + 0.056121423840522766, + 0.23947948217391968, + 0.6515242457389832, + -0.2394605576992035, + -1.434688925743103, + -0.3629516065120697, + 0.14668413996696472, + -0.3211246132850647, + 0.26727861166000366, + 0.40521925687789917, + -0.09060023725032806, + -0.1244821846485138, + -0.5918140411376953, + 0.25310218334198 + ], + [ + 0.19805660843849182, + -0.1373877376317978, + 1.395082950592041, + -0.11858675628900528, + -1.5370681285858154, + -0.11043974757194519, + -0.12877482175827026, + -0.15525361895561218, + 0.3476954400539398, + 0.6211431622505188, + -0.1982797384262085, + -1.2383601665496826, + -0.15672820806503296, + 0.04531017690896988, + -0.16811484098434448, + 0.14060468971729279, + 0.2006876915693283, + -0.06664876639842987, + -0.22397667169570923, + -0.5180864930152893, + 0.097849041223526 + ], + [ + 0.17813198268413544, + -0.16313879191875458, + 1.4762564897537231, + -0.0008421540260314941, + -1.3843040466308594, + -0.09591269493103027, + -0.381478488445282, + -0.16069743037223816, + 0.2906929552555084, + 0.4442158341407776, + 0.0563124343752861, + -1.268372654914856, + -0.1029319167137146, + -0.24049584567546844, + -0.3363857865333557, + 0.12519130110740662, + 0.4553292989730835, + -0.04446130990982056, + -0.4481552839279175, + -0.33072778582572937, + -0.06801219284534454 + ], + [ + 0.09426534920930862, + 0.038272030651569366, + 1.1666783094406128, + -0.16440847516059875, + -1.0368421077728271, + 0.07006518542766571, + -0.32122868299484253, + -0.22001880407333374, + 0.3071807622909546, + 0.3413297235965729, + -0.10435640066862106, + -1.1479319334030151, + -0.3591143488883972, + -0.2282538264989853, + -0.413745641708374, + 0.06576137244701385, + 0.4322214126586914, + -0.18121947348117828, + -0.3420878052711487, + -0.4371933341026306, + -0.12038460373878479 + ], + [ + 0.2011847198009491, + -0.018461860716342926, + 1.3517429828643799, + -0.25713682174682617, + -1.3855352401733398, + 0.030164361000061035, + -0.38910025358200073, + -0.11146190762519836, + 0.2730139493942261, + 0.4387199878692627, + -0.025306664407253265, + -1.3061028718948364, + -0.04865357279777527, + -0.15673215687274933, + -0.2664736807346344, + 0.08021986484527588, + 0.40260010957717896, + -0.03546345233917236, + -0.4927268624305725, + -0.45318055152893066, + -0.07600349187850952 + ], + [ + 0.20030882954597473, + -0.16111081838607788, + 1.4788992404937744, + 0.09693899750709534, + -1.4145588874816895, + 0.15468072891235352, + -0.5813713669776917, + -0.047689273953437805, + 0.2658916711807251, + 0.45151978731155396, + 0.08848687261343002, + -1.2820138931274414, + -0.13745823502540588, + -0.2831106185913086, + -0.390105277299881, + 0.1235877275466919, + 0.5074005126953125, + 0.023937925696372986, + -0.3375319242477417, + -0.36059510707855225, + -0.15831956267356873 + ], + [ + 0.3738653361797333, + -0.1765938252210617, + 1.487797498703003, + -0.28906771540641785, + -1.2941184043884277, + 0.12946821749210358, + -0.5777290463447571, + 0.020496457815170288, + 0.2621426284313202, + 0.4477204978466034, + 0.17428931593894958, + -1.3336917161941528, + -0.3482901453971863, + -0.01711834967136383, + -0.3248338997364044, + 0.15900002419948578, + 0.4918249845504761, + 0.05082593858242035, + -0.3648627996444702, + -0.19320496916770935, + 0.061057500541210175 + ] + ], + "generated_utilities": [ + 0.04685409367084503, + 0.025345027446746826, + 0.04131391644477844, + 0.04660575091838837, + 2.0, + 2.0, + 0.04232649505138397, + 0.01882496476173401 + ], + "group_id": "ms-9fad82fd288e4296e354", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.19493231177330017, + 0.06623455882072449, + 2.0, + 0.15396031737327576, + 0.16184639930725098, + 0.025735855102539062, + 0.134642094373703, + 0.15606965124607086, + 0.12982505559921265, + 0.14038166403770447, + 0.09718823432922363, + 0.015769004821777344, + 0.09577034413814545, + 0.057760089635849, + 0.047366172075271606, + 0.035309627652168274 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.0622546412050724, + -0.09277644753456116, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + -0.005287081003189087, + 0.0, + -0.005681931972503662, + 0.03343720734119415, + 1.0536105632781982, + -0.00917971134185791, + -1.333775520324707, + 0.03716959059238434, + 0.0, + 0.019381746649742126, + 0.026990801095962524, + 0.21505475044250488, + -0.029777705669403076, + -0.14875531196594238, + -0.048092663288116455, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.959449768066406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.103323459625244, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.591951370239258 + ], + [ + -0.13909977674484253, + -0.26507115364074707, + 2.099381446838379, + -1.432984709739685, + -2.215683698654175, + 0.18431386351585388, + 2.42065691947937, + 0.3679453730583191, + -0.24726781249046326, + 2.055530071258545, + -0.4911385774612427, + -2.0511183738708496, + 0.04109602048993111, + 2.784976005554199, + 0.005537793040275574, + 0.05149966478347778, + 1.124292016029358, + -0.6457813382148743, + -1.226535677909851, + -0.28760284185409546, + 1.0270601511001587 + ], + [ + 0.30531278252601624, + -0.4336698055267334, + 2.6217315196990967, + -1.5083816051483154, + -2.5322670936584473, + 0.06784650683403015, + 2.3866119384765625, + -0.09851980209350586, + 0.2774229049682617, + 2.157179355621338, + -0.9796892404556274, + -2.8201849460601807, + 0.08135517686605453, + 2.818830728530884, + -0.6848310828208923, + 0.37913888692855835, + 0.7420558929443359, + -0.22611916065216064, + -1.5158166885375977, + 0.0614812970161438, + 1.3468891382217407 + ], + [ + 0.27444392442703247, + -0.22534464299678802, + 2.616544246673584, + -1.2606401443481445, + -2.0848917961120605, + -0.23496872186660767, + 2.1798665523529053, + 0.7968885898590088, + 0.26037585735321045, + 1.5473439693450928, + -0.233917236328125, + -1.867544412612915, + 0.48288947343826294, + 1.9728566408157349, + -0.4071806073188782, + 0.5528238415718079, + 1.6528295278549194, + -1.0936471223831177, + -1.062675952911377, + -0.5311015248298645, + 0.9778987765312195 + ], + [ + 0.5435068607330322, + 0.24469684064388275, + 2.6045663356781006, + -1.284223198890686, + -2.6095967292785645, + 0.9732104539871216, + 2.741159200668335, + 0.6031707525253296, + -0.5612202882766724, + 2.4774889945983887, + 0.1548280119895935, + -2.5516507625579834, + -0.0651187151670456, + 2.3864798545837402, + 0.130750373005867, + -0.1318313330411911, + 1.6559979915618896, + -1.0673086643218994, + -0.5477704405784607, + 0.18809780478477478, + 0.7076419591903687 + ], + [ + 0.31294435262680054, + 0.0263526514172554, + 3.0396268367767334, + -1.3165022134780884, + -1.8981198072433472, + 0.5049136281013489, + 2.31935715675354, + -0.4664327800273895, + -0.56757652759552, + 2.7719295024871826, + 0.21425002813339233, + -2.3925602436065674, + -0.26789456605911255, + 2.008218288421631, + -0.1879691481590271, + -0.34721577167510986, + 1.6638673543930054, + -0.30173182487487793, + -1.8719977140426636, + -0.8951039910316467, + 2.094865083694458 + ], + [ + 0.533157467842102, + 0.308005690574646, + 1.7770951986312866, + -0.6794775724411011, + -3.2953453063964844, + -0.15337547659873962, + 2.5854313373565674, + 0.07320006191730499, + -0.24033141136169434, + 2.5622174739837646, + -0.9168426990509033, + -1.5317132472991943, + -0.5942863821983337, + 2.2703769207000732, + -1.0709311962127686, + 0.6969255208969116, + 1.5304101705551147, + -0.7959219217300415, + -0.9165043830871582, + -0.8046965003013611, + 0.638719916343689 + ], + [ + 0.7393907904624939, + -0.13299384713172913, + 2.4281864166259766, + -1.861674189567566, + -3.3330533504486084, + -0.18516618013381958, + 1.6492295265197754, + -0.6932566165924072, + 0.06020718812942505, + 1.1755363941192627, + -0.016285300254821777, + -3.0671546459198, + -0.6052039861679077, + 2.6829426288604736, + 0.7021514177322388, + -0.477348268032074, + 0.466697633266449, + -0.9403301477432251, + -0.8690205812454224, + -1.1888904571533203, + 0.5360406041145325 + ], + [ + -0.0713023692369461, + -0.9776724576950073, + 1.2895376682281494, + -1.6489125490188599, + -1.4475497007369995, + -0.6822370886802673, + 3.14093017578125, + -0.5356632471084595, + -0.7523763179779053, + 2.766268253326416, + -0.8215572834014893, + -2.753042697906494, + 0.10929550230503082, + 2.570046901702881, + -0.34730663895606995, + 0.8763631582260132, + 1.3356308937072754, + -0.14330342411994934, + -1.1863114833831787, + -0.30006271600723267, + 0.7264611721038818 + ], + [ + 0.35943418741226196, + 0.22802312672138214, + 1.7158434391021729, + -1.4605827331542969, + -3.3450541496276855, + 0.23581582307815552, + 1.759531855583191, + 0.7012510299682617, + 0.0034519433975219727, + 1.4857903718948364, + -1.2866599559783936, + -2.1787426471710205, + 0.8807055354118347, + 1.9066205024719238, + -0.9918697476387024, + -0.32221919298171997, + 1.1986194849014282, + -1.187239646911621, + -0.5752912163734436, + -0.015687286853790283, + 1.775393009185791 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16282694041728973, + -0.1309112012386322, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + 0.15446226298809052, + 0.0, + -0.15669959783554077, + -0.08878482133150101, + 1.0536105632781982, + -0.09701520204544067, + -1.333775520324707, + 0.10988402366638184, + 0.0, + -0.06573517620563507, + -0.2108776718378067, + 0.21505475044250488, + -0.16805344820022583, + -0.14875531196594238, + 0.12529203295707703, + 0.0 + ] + ], + "predicted_scores": [ + 7.332526206970215, + 7.2481184005737305, + 7.235976696014404, + 7.188525676727295, + 7.012310028076172, + 6.965161323547363, + 6.905152320861816, + 6.820371150970459 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1176943004/train1", + "selected_index": 0, + "state_hash": "0d923438bf2b227e3521cb937d011f5bddf9b553a882853098b702ec31090614", + "stored_base_utility": 0.19493231177330017, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06781657040119171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06781657040119171 + }, + "base_progress": 0.06781657040119171, + "base_success": false, + "base_utility": 0.06781657040119171, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.00024640560150146484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00024640560150146484 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05664622783660889, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05664622783660889 + }, + { + "contact_quality": null, + "progress": 0.00476934015750885, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00476934015750885 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.00024640560150146484, + 1.0, + 1.0, + 0.05664622783660889, + 0.00476934015750885, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + true, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "generated_tangents": [ + [ + -0.2442924529314041, + 0.1922399252653122, + 1.2617125511169434, + -1.0334233045578003, + -1.6436376571655273, + 0.2199554741382599, + -0.06904351711273193, + -0.16711318492889404, + 0.05172731727361679, + 0.07282762229442596, + 0.28897908329963684, + -1.388846516609192, + 0.1773185431957245, + 0.13332222402095795, + -0.046502482146024704, + 0.21015708148479462, + 0.28959494829177856, + -0.45105117559432983, + -0.6802242994308472, + -0.24409805238246918, + -0.05618833005428314 + ], + [ + -0.30237674713134766, + 0.15285466611385345, + 0.983101487159729, + -0.8516850471496582, + -1.0975096225738525, + 0.6081944704055786, + 0.6999843716621399, + -0.12977352738380432, + 0.20687785744667053, + 0.16245761513710022, + 0.10769888758659363, + -1.2781031131744385, + 0.3068641722202301, + -0.5050348043441772, + -0.06470249593257904, + 0.29483672976493835, + 0.2722031772136688, + -0.5721697807312012, + -0.0724286139011383, + -0.6215910911560059, + 0.49704527854919434 + ], + [ + -0.18169476091861725, + 0.19710227847099304, + 1.2476732730865479, + -1.0144259929656982, + -1.620957374572754, + 0.2072249799966812, + -0.06506145000457764, + -0.08163696527481079, + 0.08300820738077164, + 0.04539757966995239, + 0.2998795807361603, + -1.4227242469787598, + -0.027084678411483765, + 0.1919259876012802, + -0.07722999155521393, + 0.16494181752204895, + 0.22130902111530304, + -0.33830904960632324, + -0.7020812630653381, + -0.33060234785079956, + -0.13652271032333374 + ], + [ + -0.3333394527435303, + 0.1445809006690979, + 1.3304860591888428, + -0.9550267457962036, + -1.5173897743225098, + -0.11261460185050964, + 0.14770564436912537, + -0.23509421944618225, + 0.1560247838497162, + -0.012800522148609161, + 0.2957661747932434, + -1.4266231060028076, + 0.15834972262382507, + 0.08385638892650604, + 0.012017462402582169, + 0.2219429463148117, + 0.3547346293926239, + -0.4124802350997925, + -0.6573758125305176, + -0.023219317197799683, + -0.1346246898174286 + ], + [ + -0.1128702312707901, + 0.1393568515777588, + 1.321090817451477, + -0.9430903196334839, + -1.5495328903198242, + -0.13003066182136536, + -0.1664838194847107, + -0.17644405364990234, + 0.12526345252990723, + 0.11281096935272217, + 0.27010899782180786, + -1.229363203048706, + -0.0460125207901001, + 0.18365833163261414, + 0.009704198688268661, + 0.2022450566291809, + 0.3173149824142456, + -0.4220997095108032, + -0.8078009486198425, + -0.23880226910114288, + -0.21057868003845215 + ], + [ + -0.1460224837064743, + 0.12933729588985443, + 1.2848962545394897, + -0.9112169742584229, + -1.6112854480743408, + 0.058030061423778534, + -0.17862147092819214, + -0.1863740086555481, + 0.15170565247535706, + 0.07708920538425446, + 0.16633179783821106, + -1.0974202156066895, + -0.05451437830924988, + 0.17298021912574768, + 0.008827831596136093, + 0.20871904492378235, + 0.2916363477706909, + -0.4815630316734314, + -0.7385351061820984, + -0.08161972463130951, + -0.08039110898971558 + ], + [ + -0.08636783063411713, + 0.1789580136537552, + 1.2403063774108887, + -0.9093590974807739, + -1.5502665042877197, + -0.08800885081291199, + -0.1913967728614807, + -0.10208499431610107, + 0.11641716212034225, + 0.15578410029411316, + 0.25646504759788513, + -1.179133415222168, + 0.002452850341796875, + 0.2192654311656952, + 0.02191675826907158, + 0.242428258061409, + 0.3431023955345154, + -0.4159812927246094, + -0.8238541483879089, + -0.04872231185436249, + -0.16364553570747375 + ], + [ + -0.3311907649040222, + 0.1384386271238327, + 1.4478973150253296, + -0.9109632968902588, + -1.7710649967193604, + 0.5832236409187317, + -0.21054184436798096, + -0.179454505443573, + 0.21354767680168152, + 0.2505671977996826, + 0.5328578352928162, + -1.3277925252914429, + -0.054305195808410645, + -0.03966668248176575, + -0.14212284982204437, + 0.14360544085502625, + 0.5255115628242493, + -0.058621689677238464, + -0.5408797860145569, + -0.33775684237480164, + 0.0875023603439331 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.00024640560150146484, + 2.0, + 2.0, + 0.05664622783660889, + 0.00476934015750885, + 2.0 + ], + "group_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0793772041797638, + 2.0, + 2.0, + 2.0, + 0.1144598126411438, + 0.04584880173206329, + 0.08543911576271057, + 0.1307348757982254, + 0.10944780707359314, + 0.07284757494926453, + 0.05719476938247681, + 0.09351089596748352, + 0.13337698578834534, + 0.039893895387649536, + 0.09353269636631012, + 0.08224260807037354 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.016482055187225342, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.0903974175453186, + 0.0, + 0.012217208743095398, + 0.010632793419063091, + 0.03624187409877777, + 0.04524834081530571, + -1.3283450603485107, + -0.00880381464958191, + 0.0, + -0.014520794153213501, + 0.5632922649383545, + 0.003262042999267578, + -0.11586570739746094, + 0.0014255642890930176, + 0.015446990728378296, + 0.0 + ], + [ + 0.04007059335708618, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.12278485298156738, + 0.0, + -0.0962398424744606, + -0.08766138553619385, + 0.29553186893463135, + 0.10519622266292572, + -1.3283450603485107, + 0.00032788515090942383, + 0.0, + -0.13836456835269928, + 0.5632922649383545, + -0.13551044464111328, + -0.11586570739746094, + 0.07183873653411865, + -0.03684847429394722, + 0.0 + ], + [ + 0.08037775754928589, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.33122140169143677, + 0.0, + 0.20786377787590027, + -0.42098182439804077, + -0.17684945464134216, + 0.1062583178281784, + -1.3283450603485107, + 0.28051087260246277, + 0.0, + 0.7470210194587708, + 0.5632922649383545, + 0.03247350454330444, + -0.11586570739746094, + 0.4809897243976593, + -0.1744043231010437, + 0.0 + ], + [ + -0.4189682602882385, + 1.1229554414749146, + 2.8264856338500977, + -2.2389113903045654, + -2.4226768016815186, + 1.2001769542694092, + 2.1568119525909424, + -0.016906127333641052, + -0.24083517491817474, + -0.23574936389923096, + -0.13933053612709045, + -2.458132028579712, + 0.40251442790031433, + 0.870932936668396, + -0.28115370869636536, + 1.4463695287704468, + 0.9248121380805969, + -0.9938455820083618, + -0.680131196975708, + -0.051702484488487244, + -0.4663417339324951 + ], + [ + -0.6373537182807922, + 1.3288449048995972, + 3.1669085025787354, + -1.4171695709228516, + -2.242492198944092, + 1.6900161504745483, + 2.778874158859253, + 0.47899991273880005, + 0.3402186334133148, + 0.4507899582386017, + 0.8237879276275635, + -1.7865383625030518, + 1.1732149124145508, + 1.5826115608215332, + 0.4416889548301697, + 2.222722053527832, + 0.6443713307380676, + -1.8598005771636963, + -0.9690301418304443, + 0.3504742383956909, + -1.1371408700942993 + ] + ], + "predicted_scores": [ + 9.808351516723633, + 9.605087280273438, + 9.494207382202148, + 9.47028923034668, + 9.351375579833984, + 9.00699234008789, + 8.98760986328125, + 7.942086219787598 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "998761162/train1", + "selected_index": 0, + "state_hash": "78cc63bdeba9f4d6d6da2efd23c0f7f6631321321e917a70299a7c8025e71d47", + "stored_base_utility": 0.0793772041797638, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05380085110664368, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05380085110664368 + }, + { + "contact_quality": null, + "progress": 0.045698270201683044, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.045698270201683044 + }, + { + "contact_quality": null, + "progress": 0.07195276021957397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07195276021957397 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.07060344517230988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07060344517230988 + }, + { + "contact_quality": null, + "progress": 0.07577565312385559, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07577565312385559 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.05380085110664368, + 0.045698270201683044, + 0.07195276021957397, + 1.0, + 0.07060344517230988, + 0.07577565312385559, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-fc8084d5cd7da937fc02", + "generated_tangents": [ + [ + -0.25766533613204956, + -0.12600606679916382, + 1.309084415435791, + -0.5940244197845459, + -1.1092166900634766, + 0.5150706171989441, + -0.052182137966156006, + -0.018023774027824402, + 0.027493583038449287, + 0.243815615773201, + -0.08884450048208237, + -1.4417210817337036, + 0.0016799569129943848, + 0.07788446545600891, + -0.0633881539106369, + -0.05702465400099754, + 0.3619932532310486, + 0.22092172503471375, + -0.33220547437667847, + -0.20806413888931274, + 0.058834560215473175 + ], + [ + -0.24930612742900848, + -0.10163244605064392, + 1.2350733280181885, + -0.5576799511909485, + -0.9712830185890198, + 0.23886027932167053, + -0.014269083738327026, + -0.05562072992324829, + 0.0043869102373719215, + 0.2794634997844696, + -0.03582601994276047, + -1.3612873554229736, + 0.17686626315116882, + -0.24185006320476532, + 0.008196085691452026, + -0.03828829526901245, + 0.23057204484939575, + 0.1730470061302185, + -0.35135817527770996, + -0.18045026063919067, + -0.09966912865638733 + ], + [ + -0.1390257030725479, + -0.07180877774953842, + 1.373719573020935, + -0.7151130437850952, + -1.1378800868988037, + -0.11026304960250854, + 0.05199959874153137, + -0.137319415807724, + 0.06817563623189926, + 0.06592569500207901, + -0.063407763838768, + -1.4731884002685547, + -0.0001252293586730957, + 0.04567163437604904, + 0.0496208593249321, + -0.04812358692288399, + 0.3729042708873749, + 0.20860879123210907, + -0.3157980442047119, + 0.04793208837509155, + -0.033076465129852295 + ], + [ + -0.08036945760250092, + 0.00599384680390358, + 1.1883916854858398, + -0.5798259973526001, + -0.9847074151039124, + 0.38269490003585815, + 0.5433768630027771, + -0.11317121982574463, + 0.09527748823165894, + 0.021453171968460083, + -0.06421821564435959, + -1.3068289756774902, + 0.279157429933548, + -0.27088087797164917, + 0.0208757221698761, + 0.04182553291320801, + 0.11511608958244324, + -0.009401455521583557, + -0.058999449014663696, + -0.36277535557746887, + -0.017688199877738953 + ], + [ + -0.06852550804615021, + 0.004076119512319565, + 1.2988475561141968, + -0.7716856002807617, + -1.3796987533569336, + 0.12243293225765228, + -0.1760903000831604, + -0.0433368980884552, + 0.04176665097475052, + 0.15399578213691711, + -0.12742996215820312, + -1.4812989234924316, + -0.01767835021018982, + 0.14924274384975433, + 0.007744971662759781, + -0.032446056604385376, + 0.2803436517715454, + 0.27891480922698975, + -0.3602532744407654, + -0.09992602467536926, + 0.1479855626821518 + ], + [ + -0.12720204889774323, + -0.027446024119853973, + 1.2578574419021606, + -0.685033917427063, + -1.2497880458831787, + 0.11687909066677094, + -0.1411479115486145, + -0.08844837546348572, + 0.012235293164849281, + 0.13517729938030243, + -0.0583876296877861, + -1.3469491004943848, + 0.08236351609230042, + 0.25899428129196167, + 0.0673624724149704, + 0.0028378991410136223, + 0.22267533838748932, + 0.28155064582824707, + -0.24954187870025635, + -0.1778385043144226, + 0.08610621094703674 + ], + [ + -0.0639880895614624, + -0.002814624458551407, + 1.2676571607589722, + -0.7168252468109131, + -1.4181642532348633, + 0.12310458719730377, + -0.21598517894744873, + -0.040600910782814026, + 0.012664916925132275, + 0.16312536597251892, + -0.09044265002012253, + -1.3518496751785278, + 0.005789220333099365, + 0.18833549320697784, + 0.044131435453891754, + -0.012436660937964916, + 0.1918671429157257, + 0.3091844916343689, + -0.3346096873283386, + -0.016804546117782593, + 0.03484569489955902 + ], + [ + -0.010094091296195984, + -0.0019205734133720398, + 1.295836091041565, + -0.780786395072937, + -1.3846473693847656, + 0.20473794639110565, + -0.160369873046875, + -0.0183841735124588, + 0.03672788664698601, + 0.11521317064762115, + -0.08856336027383804, + -1.5128268003463745, + -0.15641868114471436, + 0.14340858161449432, + -0.0047133006155490875, + -0.0030567143112421036, + 0.22386585175991058, + 0.3224785625934601, + -0.4089009165763855, + -0.1121964156627655, + 0.03727264702320099 + ] + ], + "generated_utilities": [ + 0.05380085110664368, + 0.045698270201683044, + 0.07195276021957397, + 2.0, + 0.07060344517230988, + 0.07577565312385559, + 2.0, + 2.0 + ], + "group_id": "ms-fc8084d5cd7da937fc02", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.120394766330719, + 0.1662210375070572, + 2.0, + 0.0270967036485672, + 0.16474007070064545, + 0.014024153351783752, + 0.13986149430274963, + 0.17214345932006836, + 0.19527268409729004, + 0.08535367250442505, + 0.1354738473892212, + 0.11415943503379822, + 0.05034071207046509, + 0.10602898895740509, + 0.01804502308368683, + 0.09680917859077454 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.4742112457752228, + -0.6877930760383606, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.0012008249759674072, + 0.0, + 0.04633669555187225, + -0.307714581489563, + -0.22357922792434692, + -0.35245198011398315, + -1.486854076385498, + 0.15573161840438843, + 0.0, + 0.5157145261764526, + 0.243257537484169, + 0.2085912823677063, + 0.48896148800849915, + -0.06008961796760559, + 0.4603208899497986, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.883587837219238, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6201469898223877, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.5597648620605469 + ], + [ + -0.8389496207237244, + 0.5873794555664062, + 2.327263116836548, + -2.175445795059204, + -1.844232439994812, + 0.15723112225532532, + 2.920079231262207, + -0.5335484743118286, + -0.6761462688446045, + 0.48151493072509766, + 0.2968217730522156, + -2.99812650680542, + -0.27805766463279724, + 2.0262699127197266, + 0.10151885449886322, + -0.7244771122932434, + 1.0176090002059937, + 1.278473973274231, + -0.2260074019432068, + -1.0207180976867676, + 0.7682583928108215 + ], + [ + 0.2503487765789032, + 0.20628705620765686, + 2.289511203765869, + -0.8669976592063904, + -1.9807379245758057, + 1.1820356845855713, + 2.8384368419647217, + 0.43699324131011963, + -0.688888669013977, + 1.0812339782714844, + -0.18395745754241943, + -1.6906647682189941, + -0.040969282388687134, + 1.254612684249878, + 0.24077603220939636, + -1.0562812089920044, + 1.2436273097991943, + 0.2388666272163391, + -0.5016235709190369, + 0.4570234417915344, + 1.355008840560913 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.10285450518131256, + -0.2634444832801819, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.12851905822753906, + 0.0, + 0.06062778830528259, + 0.006933145225048065, + -0.08638572692871094, + 0.32057398557662964, + -1.486854076385498, + -0.1531560719013214, + 0.0, + 0.14251592755317688, + 0.07826228439807892, + -0.24055713415145874, + -0.11116695404052734, + 0.200389564037323, + -0.07051187753677368, + 0.0 + ], + [ + -0.07637833058834076, + 0.6696901321411133, + 2.130587100982666, + -1.9006763696670532, + -2.6654670238494873, + 0.06729818880558014, + 2.65816593170166, + -0.0612010583281517, + -0.1226595938205719, + 0.9200000166893005, + -0.4352770149707794, + -2.266615390777588, + 0.48697614669799805, + 1.5162358283996582, + -0.014945849776268005, + -0.4459138512611389, + 1.1448073387145996, + 0.8031747937202454, + -0.6817312240600586, + -0.20262596011161804, + 1.1690938472747803 + ], + [ + 0.519248902797699, + 0.27624276280403137, + 1.911887764930725, + -1.8784401416778564, + -2.9212372303009033, + 0.7762739658355713, + 2.547428846359253, + 0.08362685143947601, + -0.1671719253063202, + 0.8596282601356506, + -0.17510351538658142, + -2.220954656600952, + 0.4918195605278015, + 1.357650876045227, + 0.3462192416191101, + 0.2476332038640976, + 0.6965686082839966, + 1.0915483236312866, + -0.6818809509277344, + -0.7465203404426575, + 0.724744439125061 + ] + ], + "predicted_scores": [ + 7.541661739349365, + 7.425927639007568, + 7.374334335327148, + 7.179238319396973, + 6.926530838012695, + 6.9085612297058105, + 6.650173664093018, + 6.590436935424805 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1991414747/train1", + "selected_index": 0, + "state_hash": "127cf975da326f174c815484811a6acc4855bf07460a2aa5b0d728cb2a1f41c7", + "stored_base_utility": 0.120394766330719, + "task_id": "PushCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20275163650512695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20275163650512695 + }, + "base_progress": 0.20275163650512695, + "base_success": false, + "base_utility": 0.20275163650512695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2330203801393509, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2330203801393509 + }, + { + "contact_quality": null, + "progress": 0.15594589710235596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15594589710235596 + }, + { + "contact_quality": null, + "progress": 0.23662996292114258, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23662996292114258 + }, + { + "contact_quality": null, + "progress": 0.24087366461753845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24087366461753845 + }, + { + "contact_quality": null, + "progress": 0.22966185212135315, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22966185212135315 + }, + { + "contact_quality": null, + "progress": 0.22573722898960114, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22573722898960114 + }, + { + "contact_quality": null, + "progress": 0.2381056249141693, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2381056249141693 + }, + { + "contact_quality": null, + "progress": 0.16225522756576538, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16225522756576538 + } + ], + "candidate_progress": [ + 0.2330203801393509, + 0.15594589710235596, + 0.23662996292114258, + 0.24087366461753845, + 0.22966185212135315, + 0.22573722898960114, + 0.2381056249141693, + 0.16225522756576538 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3afd8959363e917591d4", + "generated_tangents": [ + [ + 0.032345257699489594, + 0.041534118354320526, + -0.24840888381004333, + -1.115391731262207, + 0.43810999393463135, + -0.3190232217311859, + 1.095834493637085, + 0.0745425820350647, + 0.1691260039806366, + -0.14132744073867798, + 0.06961572170257568, + -0.23097893595695496, + -2.2369840145111084, + -0.4671887159347534, + -0.019553054124116898, + -0.044790834188461304, + -0.010421991348266602, + 0.2782835066318512, + -0.2650759816169739, + -1.255564570426941, + -0.2518557012081146 + ], + [ + 0.052113763988018036, + 0.0841289609670639, + 0.16394153237342834, + -1.261644959449768, + 0.39749136567115784, + -0.4661403000354767, + 0.7681432962417603, + 0.11206910014152527, + 0.16414090991020203, + -0.08916645497083664, + 0.011096607893705368, + -0.444663941860199, + -2.5357465744018555, + -0.42860186100006104, + -0.05279140546917915, + 0.059909138828516006, + 0.09880596399307251, + 0.38401365280151367, + -0.3973008990287781, + -1.5253477096557617, + -0.2788736820220947 + ], + [ + -0.14080624282360077, + 0.12565378844738007, + -0.20969948172569275, + -1.1504992246627808, + 0.746421754360199, + -0.6103098392486572, + 1.072847843170166, + 0.1000339463353157, + 0.14912262558937073, + -0.18910285830497742, + 0.030119994655251503, + -0.42367202043533325, + -2.2265522480010986, + -0.28670209646224976, + 0.07863615453243256, + -0.1160237044095993, + 0.049855709075927734, + -0.07195411622524261, + -0.2410619854927063, + -1.5276737213134766, + -0.1106393039226532 + ], + [ + -0.13328681886196136, + 0.1056923121213913, + -0.14332684874534607, + -1.0966047048568726, + 0.6499765515327454, + -0.5430305004119873, + 0.929847240447998, + 0.06825277209281921, + 0.1386578381061554, + -0.1759842038154602, + 0.0729876309633255, + -0.282988965511322, + -2.3079235553741455, + -0.14611150324344635, + -0.03398582711815834, + -0.10941538214683533, + 0.055496156215667725, + 0.08245377987623215, + -0.1715477705001831, + -1.4508602619171143, + -0.12715402245521545 + ], + [ + -0.06993065774440765, + 0.1536913365125656, + -0.16998520493507385, + -0.8637540340423584, + 0.7124572992324829, + -0.5082904100418091, + 1.2236427068710327, + -0.03231893479824066, + 0.157712921500206, + -0.18595099449157715, + 0.22639310359954834, + 0.059861332178115845, + -2.0127789974212646, + -0.3475462794303894, + -0.1140376627445221, + -0.09047801792621613, + -0.10816304385662079, + 0.08630485087633133, + 0.33980488777160645, + -1.3979809284210205, + -0.17245376110076904 + ], + [ + -0.06523267924785614, + 0.1315969079732895, + -0.28416815400123596, + -1.009912133216858, + 0.7519988417625427, + -0.5380936861038208, + 0.3884163498878479, + -0.014463379979133606, + 0.12786871194839478, + -0.10217460244894028, + -0.056670717895030975, + -0.13899099826812744, + -2.1328392028808594, + 0.08389313519001007, + 0.08188736438751221, + -0.07830917835235596, + -0.04422105848789215, + 0.14191380143165588, + -0.10145944356918335, + -1.582294225692749, + 0.05496809631586075 + ], + [ + -0.15150649845600128, + 0.17804746329784393, + -0.07315006852149963, + -1.1448009014129639, + 0.7629889249801636, + -0.5661367177963257, + 0.521133303642273, + 0.08745068311691284, + 0.1308426409959793, + -0.19754517078399658, + 0.1423840969800949, + -0.17677173018455505, + -2.2040855884552, + 0.31156861782073975, + 0.03675226494669914, + -0.13841646909713745, + 0.06883640587329865, + 0.3033646047115326, + -0.09109467267990112, + -1.582838535308838, + 0.06170547008514404 + ], + [ + 0.18227604031562805, + 0.12305095791816711, + 0.29672008752822876, + -1.265738606452942, + 0.3126491904258728, + -1.3182971477508545, + 1.3083362579345703, + 0.07891415059566498, + 0.13188202679157257, + -0.1381712555885315, + 0.18966078758239746, + -0.22495943307876587, + -1.6865249872207642, + -0.4436098337173462, + -0.06243528053164482, + -0.2181418091058731, + 0.18943697214126587, + 0.13939020037651062, + -0.350314199924469, + -1.6874771118164062, + 0.05776550620794296 + ] + ], + "generated_utilities": [ + 0.2330203801393509, + 0.15594589710235596, + 0.23662996292114258, + 0.24087366461753845, + 0.22966185212135315, + 0.22573722898960114, + 0.2381056249141693, + 0.16225522756576538 + ], + "group_id": "ms-3afd8959363e917591d4", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8582140207290649, + 2.0, + 0.18031787872314453, + 0.07887399196624756, + 0.5568947792053223, + 0.16715207695960999, + 0.5635193586349487, + 0.18913890421390533, + 0.5682768225669861, + 0.10102970898151398, + 0.09163376688957214, + 0.12909311056137085, + 0.12299326062202454, + 0.09030237793922424, + 0.13531166315078735, + 0.06874948740005493 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.15699657797813416, + 0.6247961521148682, + 0.107974112033844, + -0.3284463882446289, + 0.22832846641540527, + -0.12728703022003174, + 0.0, + -0.2739208936691284, + -0.051639966666698456, + -0.2660524249076843, + -0.2539072632789612, + 0.09800386428833008, + -1.84090256690979, + 0.0, + -0.017676634714007378, + -0.018830953165888786, + 0.1583891659975052, + -0.07512116432189941, + -0.01770937442779541, + -1.5237011909484863, + 0.0 + ], + [ + 0.4675478935241699, + 0.6247961521148682, + 0.3345896601676941, + -0.5404787063598633, + -0.06834304332733154, + -0.12728703022003174, + 0.0, + -0.14999517798423767, + -0.2730908691883087, + 0.08837036788463593, + 0.45600926876068115, + 0.410434365272522, + -1.84090256690979, + 0.0, + 0.4969322681427002, + 0.22034725546836853, + 0.2921968400478363, + 0.5872196555137634, + 0.19443652033805847, + -1.5237011909484863, + 0.0 + ], + [ + 0.41189882159233093, + 1.6247961521148682, + 0.12157746404409409, + -1.328446388244629, + 0.9316569566726685, + -1.1272870302200317, + 1.276085376739502, + -0.029595039784908295, + -0.014941371977329254, + 0.004283940885215998, + 0.6188332438468933, + -0.49464237689971924, + -2.84090256690979, + -1.8046140670776367, + 0.0007966384291648865, + -0.006994657218456268, + 0.014263113960623741, + 0.6869955658912659, + -0.2918644845485687, + -2.5237011909484863, + -1.7481608390808105 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.552170753479004, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.6092281341552734, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.496321678161621 + ], + [ + 0.44720953702926636, + 1.4417266845703125, + 0.4182493984699249, + -1.547792911529541, + 0.990667462348938, + -0.9114025831222534, + 1.535681962966919, + 0.21791213750839233, + -0.30513259768486023, + 0.013669610023498535, + 0.7976462841033936, + -0.6395556926727295, + -2.5809314250946045, + -1.571488618850708, + 0.038753002882003784, + 0.04824339970946312, + -0.11262937635183334, + 0.9187417030334473, + -0.1308273822069168, + -2.59043288230896, + -1.9625922441482544 + ], + [ + 0.5162960290908813, + 1.2868248224258423, + 0.2058304250240326, + -1.4812198877334595, + 1.085485816001892, + -1.05157470703125, + 1.1480523347854614, + 0.2796477973461151, + -0.1532609462738037, + 0.12988577783107758, + 0.8581418395042419, + -0.6760798692703247, + -2.769819736480713, + -2.192568778991699, + 0.10523252189159393, + 0.05066283047199249, + -0.08061028271913528, + 0.7438116669654846, + -0.07841359078884125, + -2.3079848289489746, + -1.4200246334075928 + ], + [ + 0.4112767279148102, + 1.8745536804199219, + 0.5301340222358704, + -0.9917724132537842, + 0.8905197978019714, + -0.8391705751419067, + 1.3120794296264648, + 0.3723032474517822, + -0.19477176666259766, + -0.35689839720726013, + 0.2843710780143738, + -0.45066511631011963, + -2.812030792236328, + -1.588818073272705, + -0.09205310046672821, + -0.5067525506019592, + -0.41988417506217957, + 1.0423402786254883, + 0.1759319305419922, + -2.945967197418213, + -2.05134916305542 + ], + [ + 0.46365559101104736, + 1.105017900466919, + 0.466808021068573, + -1.5382812023162842, + 0.8287478089332581, + -1.4617033004760742, + 0.8807560205459595, + 0.2107943296432495, + 0.2954367995262146, + 0.45928943157196045, + 0.6575880646705627, + -0.5414218902587891, + -2.711836099624634, + -1.6827114820480347, + 0.3175691068172455, + -0.38371264934539795, + 0.34041139483451843, + 0.34132614731788635, + -0.8783347606658936, + -2.133782148361206, + -1.3974260091781616 + ], + [ + 0.13586324453353882, + 1.6158640384674072, + 0.24489957094192505, + -1.4942185878753662, + 1.1602565050125122, + -1.648895502090454, + 1.1836220026016235, + 0.17831489443778992, + -0.5400142669677734, + -0.42907604575157166, + 0.5951722264289856, + -0.7885013818740845, + -3.2272911071777344, + -2.208972692489624, + -0.4857475459575653, + 0.5623884201049805, + 0.02566027268767357, + 0.590702474117279, + -0.6581307649612427, + -2.7863473892211914, + -2.031818151473999 + ], + [ + 0.21801596879959106, + 1.0595136880874634, + 0.9040011763572693, + -1.1076470613479614, + 1.2744677066802979, + -0.7150566577911377, + 1.0042598247528076, + 0.5841938257217407, + -0.2832033336162567, + -0.45343834161758423, + -0.17037296295166016, + -0.5152367353439331, + -2.1963798999786377, + -1.5652427673339844, + 0.03223085403442383, + 0.07496664673089981, + -0.1515023559331894, + 1.233866572380066, + 0.4301607310771942, + -2.489328384399414, + -1.8620245456695557 + ], + [ + -0.15804454684257507, + 2.2168471813201904, + 0.9374236464500427, + -1.8158314228057861, + 1.114590048789978, + -0.9279311895370483, + 1.890993356704712, + -0.8653054237365723, + -0.5551189184188843, + 0.4295695126056671, + 1.1614964008331299, + -0.5840407609939575, + -2.900725841522217, + -2.579883575439453, + -0.1900041103363037, + 0.42368441820144653, + 0.38376376032829285, + 0.9654008150100708, + 0.10608989000320435, + -3.332181453704834, + -1.8879828453063965 + ], + [ + 1.0901068449020386, + 2.576328754425049, + 0.5122992396354675, + -1.9627916812896729, + 0.026246964931488037, + -0.3622438311576843, + 0.9286589622497559, + 0.644849956035614, + 0.9211093187332153, + 0.3528344929218292, + 0.3661169707775116, + 0.20941221714019775, + -2.582620143890381, + -1.6867890357971191, + 0.09869468957185745, + -0.8795469403266907, + -0.2239246666431427, + 0.5206944942474365, + -0.20716872811317444, + -2.8072235584259033, + -2.3385443687438965 + ], + [ + -0.0005125105381011963, + 1.8920408487319946, + 1.0018010139465332, + -2.2556979656219482, + 0.7338991165161133, + -0.4559611678123474, + 1.027335524559021, + -1.0165901184082031, + -0.14792552590370178, + 0.11212217807769775, + 0.8827763795852661, + 0.1683982014656067, + -3.2136244773864746, + -2.4300284385681152, + -0.9183067083358765, + 0.10752613097429276, + -0.2955099046230316, + 0.31867098808288574, + 0.06515705585479736, + -3.416110038757324, + -1.5137704610824585 + ], + [ + -0.39508435130119324, + 2.55599308013916, + 0.128604456782341, + -1.8821169137954712, + 0.6769157648086548, + -0.7477259635925293, + 1.6593644618988037, + -0.005859369412064552, + 0.22320973873138428, + -0.2906110882759094, + 0.22343486547470093, + -1.488371729850769, + -2.81829833984375, + -0.939697265625, + 0.9867495894432068, + 0.2955887019634247, + -0.11908379942178726, + 0.473803848028183, + -0.11059392988681793, + -3.471217155456543, + -2.3480191230773926 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.018576741218566895, + 0.6247961521148682, + -0.017397306859493256, + -0.3284463882446289, + 0.044771432876586914, + -0.12728703022003174, + 0.0, + -0.005512619391083717, + 0.03525955602526665, + -0.005881692748516798, + 0.04493492841720581, + 0.056808531284332275, + -1.84090256690979, + 0.0, + 0.11075251549482346, + -0.05448189005255699, + -0.041690655052661896, + 0.010008454322814941, + 0.056893616914749146, + -1.5237011909484863, + 0.0 + ] + ], + "predicted_scores": [ + 2.6485273838043213, + 2.5635921955108643, + 1.288849949836731, + 1.2780762910842896, + 1.010661244392395, + 0.6195508241653442, + 0.10887739062309265, + -0.662818193435669 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "75070641/train1", + "selected_index": 0, + "state_hash": "bd1bbc057450972b9fa6e08a54ebd8c6d82d09997dad16cecf9584632b705b7b", + "stored_base_utility": 0.8582140207290649, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8343751430511475, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8343751430511475 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8134292960166931, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8134292960166931 + }, + { + "contact_quality": null, + "progress": 0.8216007947921753, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8216007947921753 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1820054054260254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1820054054260254 + }, + { + "contact_quality": null, + "progress": 0.8494137525558472, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8494137525558472 + } + ], + "candidate_progress": [ + 1.0, + 0.8343751430511475, + 1.0, + 0.8134292960166931, + 0.8216007947921753, + 1.0, + 0.1820054054260254, + 0.8494137525558472 + ], + "candidate_source_chart_ids": [ + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-0949097d19e597fc7f63", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + true, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3ff1aedb72f65a182b58", + "generated_tangents": [ + [ + -0.062439724802970886, + -0.10267174243927002, + -0.08697965741157532, + -0.19292812049388885, + 0.04958716034889221, + -1.9831255674362183, + 0.0265466570854187, + 0.09313465654850006, + 0.17003506422042847, + -0.06069370359182358, + -0.008215725421905518, + -0.1732316017150879, + -1.980164885520935, + -0.14224843680858612, + -0.05720065161585808, + -0.04062411189079285, + 0.02359837293624878, + 0.06030084192752838, + -0.06397882103919983, + -1.6451139450073242, + -0.21163910627365112 + ], + [ + 0.03654243052005768, + -0.08168097585439682, + -0.04141226410865784, + -0.14411795139312744, + 0.12869243323802948, + -2.002037525177002, + 0.007157176733016968, + 0.06532534956932068, + 0.17071294784545898, + -0.03254286199808121, + -0.017103321850299835, + -0.14140141010284424, + -1.9333590269088745, + -0.06879577040672302, + -0.021911438554525375, + -0.07094591110944748, + 0.03496462106704712, + 0.07464239746332169, + -0.0960855782032013, + -1.587310552597046, + -0.1497659981250763 + ], + [ + -0.08239680528640747, + -0.16734471917152405, + 0.009770750999450684, + -0.19258294999599457, + 0.15593227744102478, + -1.92607581615448, + 0.13474443554878235, + 0.07099077105522156, + 0.22003060579299927, + 0.013289451599121094, + 0.038875825703144073, + -0.1812708079814911, + -1.9331516027450562, + -0.14263375103473663, + -0.11658428609371185, + -0.09669330716133118, + -0.08281369507312775, + 0.034088134765625, + -0.03991749882698059, + -1.654839038848877, + -0.25345224142074585 + ], + [ + -0.11683295667171478, + -0.1470959484577179, + -0.015281647443771362, + -0.09597066044807434, + 0.03893783688545227, + -1.9797533750534058, + -0.15978527069091797, + 0.010778605937957764, + 0.14734525978565216, + -0.09725389629602432, + -0.2448519766330719, + -0.14496874809265137, + -1.9619742631912231, + -0.06363658607006073, + -0.09696879982948303, + -0.06293077766895294, + 0.02727481722831726, + -0.05131717026233673, + -0.3007820248603821, + -1.738222360610962, + -0.42688825726509094 + ], + [ + -0.12231095135211945, + -0.04192669689655304, + 0.03312569856643677, + -0.2702970802783966, + 0.24234843254089355, + -1.9157201051712036, + 0.19673866033554077, + 0.009997710585594177, + 0.15686525404453278, + -0.06183413416147232, + -0.09602666646242142, + -0.1328202784061432, + -1.9324674606323242, + -0.2542661428451538, + -0.05770087614655495, + -0.06654544174671173, + -0.0003515481948852539, + 0.004999503493309021, + -0.03302639722824097, + -1.501014232635498, + -0.4634285271167755 + ], + [ + -0.2972021698951721, + -0.10989681631326675, + 0.01714882254600525, + -0.004400297999382019, + -0.022506117820739746, + -1.9709454774856567, + 0.08642077445983887, + 0.223464697599411, + 0.15293675661087036, + -0.1213032677769661, + 0.06315916776657104, + -0.3256334662437439, + -2.0076959133148193, + -0.11435651779174805, + -0.04275403544306755, + 0.03250744193792343, + 0.10640215128660202, + -0.08467166125774384, + -0.0460490882396698, + -1.8005802631378174, + -0.41277238726615906 + ], + [ + -0.32271140813827515, + -0.19143876433372498, + -0.14872053265571594, + -0.08058717846870422, + -0.2637145221233368, + -1.8081475496292114, + -0.21086162328720093, + 0.23122139275074005, + 0.10887749493122101, + -0.06519048660993576, + -0.1338232159614563, + -0.4990749955177307, + -2.027585029602051, + -0.10016399621963501, + 0.12119326740503311, + -0.04883003234863281, + -0.0022823214530944824, + -0.14061720669269562, + -0.3593866229057312, + -1.7351727485656738, + -0.5648698806762695 + ], + [ + -0.1143321543931961, + -0.21972617506980896, + -0.23139998316764832, + -0.20535056293010712, + 0.11902617663145065, + -1.966908574104309, + -0.08657747507095337, + -0.27134639024734497, + 0.17673714458942413, + -0.24420922994613647, + -0.014107845723628998, + -0.5075947642326355, + -2.299177646636963, + -0.0649813711643219, + 0.07382363826036453, + -0.05025056004524231, + 0.11171112954616547, + -0.0808776468038559, + -0.43903303146362305, + -1.7131781578063965, + -0.6973515748977661 + ] + ], + "generated_utilities": [ + 2.0, + 0.8343751430511475, + 2.0, + 0.8134292960166931, + 0.8216007947921753, + 2.0, + 0.1820054054260254, + 0.8494137525558472 + ], + "group_id": "ms-3ff1aedb72f65a182b58", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.851725161075592, + 2.0, + 0.8126396536827087, + 2.0, + 0.7855525612831116, + 0.7821746468544006, + 0.6115735173225403, + 0.5732660889625549, + 0.7798006534576416, + 0.09109212458133698, + 0.8456447124481201, + 0.16636842489242554, + 0.0955686867237091, + 0.0822453498840332, + 0.0018809139728546143, + 0.08579078316688538 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.09319563210010529, + 0.18433167040348053, + 0.03857926279306412, + 0.20899039506912231, + -0.19732166826725006, + -2.7716264724731445, + -0.9089346528053284, + -0.012062903493642807, + -0.001710914308205247, + 0.02447224222123623, + 0.6588084697723389, + -0.43759873509407043, + -2.7224881649017334, + -1.560755968093872, + -0.002114389091730118, + -0.004199527204036713, + 0.015975257381796837, + 0.7779304385185242, + -0.31706690788269043, + -2.4826252460479736, + -1.5341289043426514 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.8178693056106567, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.121511936187744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0682578086853027 + ], + [ + -0.10748189687728882, + 0.29221633076667786, + 0.12325333058834076, + 0.21938443183898926, + 0.017790749669075012, + -2.485055446624756, + -1.1110891103744507, + 0.062413301318883896, + -0.2567306160926819, + -0.15453597903251648, + 0.549060583114624, + -0.18322163820266724, + -2.5428390502929688, + -1.8537542819976807, + -0.07324761152267456, + 0.28765758872032166, + 0.16924789547920227, + 0.6205734610557556, + -0.4268453121185303, + -2.6385459899902344, + -1.5760513544082642 + ], + [ + 0.44683152437210083, + 0.3713763356208801, + 0.1084144338965416, + 0.2590586543083191, + 0.1997859627008438, + -2.787759780883789, + -1.1921976804733276, + 0.33878061175346375, + 0.019987009465694427, + 0.021556159481406212, + 0.36814823746681213, + -0.2135106772184372, + -2.416769027709961, + -1.3776189088821411, + -0.3500843346118927, + -0.26303720474243164, + 0.10322260856628418, + 0.9827795624732971, + -0.3747403025627136, + -2.6419029235839844, + -1.4116339683532715 + ], + [ + -0.2883707880973816, + 0.02873578667640686, + -0.11175497621297836, + -0.106661856174469, + 0.03059367835521698, + -2.3899149894714355, + -1.0789995193481445, + 0.2610729932785034, + 0.4800889790058136, + -0.45478329062461853, + 1.0867195129394531, + -0.7417076826095581, + -3.09529185295105, + -1.730720043182373, + -0.33508142828941345, + 0.10841497033834457, + -0.38902348279953003, + 0.44348275661468506, + 0.0854053795337677, + -2.3334603309631348, + -1.7851121425628662 + ], + [ + 0.16367590427398682, + 0.05515189468860626, + -0.3197028636932373, + 0.5501840114593506, + -0.3621746301651001, + -3.0582237243652344, + -1.135798454284668, + -0.15054848790168762, + -0.5578444004058838, + 0.3691062331199646, + 0.9388304352760315, + -0.19103051722049713, + -2.7536559104919434, + -2.093773603439331, + -0.037153881043195724, + 0.19014233350753784, + -0.36205345392227173, + 0.24469107389450073, + -0.5916419625282288, + -2.7417092323303223, + -1.637229323387146 + ], + [ + -0.3646707534790039, + 0.6696967482566833, + -0.44303250312805176, + -0.4559982419013977, + -0.6542845368385315, + -2.9630653858184814, + -1.4721050262451172, + 0.5010843873023987, + 0.5162685513496399, + 0.42702627182006836, + 0.6571387648582458, + -0.08207055926322937, + -3.278073310852051, + -1.7742136716842651, + -0.6448181867599487, + -0.45361432433128357, + 0.3905653953552246, + 0.7770582437515259, + -0.19482973217964172, + -2.094736337661743, + -1.083003282546997 + ], + [ + 0.9472348093986511, + 0.6904511451721191, + -0.2936761677265167, + -0.34556156396865845, + -0.7593099474906921, + -1.9670193195343018, + -0.7599589824676514, + 0.04467743635177612, + 0.6557444334030151, + 0.2765021026134491, + 0.042136549949645996, + -0.37541040778160095, + -3.161308765411377, + -0.8803613185882568, + 0.38967010378837585, + 0.4423010051250458, + 0.019825218245387077, + 0.9605399370193481, + -0.874176561832428, + -3.0791730880737305, + -1.3198856115341187 + ], + [ + 0.8605663180351257, + 0.04459291696548462, + -0.8886744976043701, + -0.38333380222320557, + -1.0364235639572144, + -3.0104973316192627, + -0.889693021774292, + -0.08674328029155731, + 0.20159615576267242, + 0.1069621816277504, + 1.4727201461791992, + 0.09883639216423035, + -3.129849433898926, + -0.8773520588874817, + -0.2628254294395447, + -0.3415844738483429, + -0.35313117504119873, + 0.3155616521835327, + -0.556646466255188, + -2.0992307662963867, + -0.7895610928535461 + ], + [ + 0.39962780475616455, + 0.3406272530555725, + 0.05363208055496216, + -0.06018832325935364, + -0.5703235268592834, + -2.5974044799804688, + -0.6783060431480408, + 0.7815009355545044, + 0.35641810297966003, + -0.00838596560060978, + 0.6228271126747131, + 0.25608590245246887, + -2.5914058685302734, + -0.8502807021141052, + 0.726007878780365, + -0.6774719953536987, + 0.6261520981788635, + 1.149846076965332, + 0.6782730221748352, + -3.391972541809082, + -0.7993651032447815 + ], + [ + -0.29702335596084595, + 0.6166410446166992, + 0.6319320201873779, + -0.6542804837226868, + -0.8892693519592285, + -1.8599720001220703, + -1.3999924659729004, + -0.3017314672470093, + 0.9357973337173462, + -0.5596981048583984, + 1.3143360614776611, + -0.30643409490585327, + -2.028008460998535, + -1.3395459651947021, + 0.6212120056152344, + 0.6739580631256104, + -0.5840877294540405, + -0.06664055585861206, + -0.23446696996688843, + -2.0225906372070312, + -2.438906669616699 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0649428591132164, + 0.09363124519586563, + -0.014398600906133652, + -0.012044146656990051, + 0.03117428719997406, + -1.7716264724731445, + 0.0, + 0.05836518481373787, + 0.012665675021708012, + -0.013142907992005348, + -0.06252282857894897, + -0.06281104683876038, + -1.7224881649017334, + 0.0, + -0.06448972970247269, + -0.03696601092815399, + 0.030633779242634773, + -0.0031400322914123535, + 0.0327644944190979, + -1.4826252460479736, + 0.0 + ], + [ + -0.04005520045757294, + 0.2738151252269745, + -0.05346817523241043, + 0.20275527238845825, + 0.10101960599422455, + -1.7716264724731445, + 0.0, + -0.36066195368766785, + -0.21821273863315582, + 0.22022001445293427, + -0.0596311092376709, + -0.17951729893684387, + -1.7224881649017334, + 0.0, + 0.3205486238002777, + 0.25526633858680725, + 0.17392843961715698, + 0.5170043706893921, + 0.43821561336517334, + -1.4826252460479736, + 0.0 + ] + ], + "predicted_scores": [ + 3.6719894409179688, + 3.591271162033081, + 3.2816109657287598, + 2.9038634300231934, + 2.802067756652832, + 2.5342326164245605, + 1.3963682651519775, + 0.9829980134963989 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055567618/train1", + "selected_index": 0, + "state_hash": "027cc529c99c00db47a01ecfc12ab432557fbd911e8ff3c8a56adf2d3404549f", + "stored_base_utility": 0.851725161075592, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19648407399654388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19648407399654388 + }, + "base_progress": 0.19648407399654388, + "base_success": false, + "base_utility": 0.19648407399654388, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.16174951195716858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16174951195716858 + }, + { + "contact_quality": null, + "progress": 0.5578349828720093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5578349828720093 + }, + { + "contact_quality": null, + "progress": 0.5522037148475647, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5522037148475647 + }, + { + "contact_quality": null, + "progress": 0.5545214414596558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5545214414596558 + }, + { + "contact_quality": null, + "progress": 0.5589519739151001, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5589519739151001 + }, + { + "contact_quality": null, + "progress": 0.23065254092216492, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23065254092216492 + }, + { + "contact_quality": null, + "progress": 0.812659502029419, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.812659502029419 + }, + { + "contact_quality": null, + "progress": 0.12922577559947968, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12922577559947968 + } + ], + "candidate_progress": [ + 0.16174951195716858, + 0.5578349828720093, + 0.5522037148475647, + 0.5545214414596558, + 0.5589519739151001, + 0.23065254092216492, + 0.812659502029419, + 0.12922577559947968 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4048f8f6e047422e3c79", + "generated_tangents": [ + [ + 0.11197315156459808, + 0.006205298006534576, + 0.14926928281784058, + -0.7924947738647461, + 0.2881510853767395, + -0.07056090235710144, + 0.8740236759185791, + 0.11901829391717911, + 0.17200277745723724, + -0.11694551259279251, + -0.08643137663602829, + -0.4421881437301636, + -2.138794422149658, + -0.139741912484169, + 0.003947321325540543, + 0.0577222965657711, + 0.09705755114555359, + 0.5465931296348572, + -0.4082576036453247, + -1.3069524765014648, + -0.2806330621242523 + ], + [ + 0.07726870477199554, + -0.07570934295654297, + -0.20511028170585632, + -0.6203000545501709, + 0.45177608728408813, + 0.07481899857521057, + 0.9583245515823364, + 0.1163330152630806, + 0.17529936134815216, + -0.16057321429252625, + -0.03535432368516922, + -0.08222764730453491, + -1.8852554559707642, + -0.3469465970993042, + 0.053696177899837494, + -0.024394530802965164, + -0.046366795897483826, + 0.426835834980011, + -0.2268209457397461, + -1.2934815883636475, + -0.207806795835495 + ], + [ + -0.08031533658504486, + 0.01641693338751793, + -0.2207373082637787, + -0.5965136885643005, + 0.6481350660324097, + -0.04586876928806305, + 0.9781064987182617, + 0.04301615059375763, + 0.15092270076274872, + -0.1742050051689148, + -0.006424359977245331, + -0.3280750513076782, + -1.7509621381759644, + -0.025145605206489563, + 0.09848727285861969, + -0.07737380266189575, + 0.00167161226272583, + 0.11688224971294403, + -0.29587846994400024, + -1.4826886653900146, + -0.18090125918388367 + ], + [ + -0.05299849808216095, + 0.05311792343854904, + -0.26579973101615906, + -0.5007749795913696, + 0.6133293509483337, + -0.04339100420475006, + 0.5321841239929199, + -0.03950536251068115, + 0.08820459991693497, + -0.03519446402788162, + -0.07128775864839554, + -0.1457923948764801, + -1.879436731338501, + 0.19897763431072235, + 0.05538950860500336, + -0.04888976365327835, + -0.007486552000045776, + 0.2481449842453003, + -0.04760858416557312, + -1.4677531719207764, + -0.024621397256851196 + ], + [ + -0.09349143505096436, + 0.004638437181711197, + -0.14963886141777039, + -0.5424951314926147, + 0.5620851516723633, + -0.10637667775154114, + 0.8044445514678955, + 0.0581660270690918, + 0.15918996930122375, + -0.16691237688064575, + 0.020559489727020264, + -0.15404486656188965, + -1.784014344215393, + 0.04818182438611984, + 0.03167671337723732, + -0.10784728825092316, + -0.032836392521858215, + 0.2474530190229416, + -0.20835059881210327, + -1.3472936153411865, + -0.16309085488319397 + ], + [ + -0.06473344564437866, + 0.028126433491706848, + -0.1466488540172577, + -0.3547529876232147, + 0.5873199701309204, + -0.0310448557138443, + 1.0894575119018555, + -0.025085359811782837, + 0.17570744454860687, + -0.19183149933815002, + 0.13417312502861023, + 0.1717030555009842, + -1.63825261592865, + -0.21345572173595428, + -0.019779633730649948, + -0.10790978372097015, + -0.1684931069612503, + 0.27209532260894775, + 0.2623617649078369, + -1.3589808940887451, + -0.17549178004264832 + ], + [ + -0.07936891913414001, + 0.07177864760160446, + 0.0379214882850647, + -0.49842533469200134, + 0.6703141927719116, + -0.004151239991188049, + 0.5484176874160767, + 0.05874086916446686, + 0.0968005433678627, + -0.14093080163002014, + 0.030639145523309708, + -0.1879361867904663, + -1.7977654933929443, + 0.4428902268409729, + 0.013650596141815186, + -0.1640920341014862, + 0.07403738796710968, + 0.35591626167297363, + -0.08696019649505615, + -1.4611952304840088, + -0.044204503297805786 + ], + [ + 0.25739431381225586, + 0.022951796650886536, + 0.2976742386817932, + -0.8237043619155884, + 0.2583261728286743, + -0.7832725048065186, + 1.4842185974121094, + 0.10086319595575333, + 0.11956450343132019, + -0.06298255175352097, + 0.12854579091072083, + -0.29761233925819397, + -1.3621857166290283, + -0.11717723309993744, + 0.00656508281826973, + -0.2573537826538086, + 0.13421666622161865, + 0.31478530168533325, + -0.2815800905227661, + -1.6235404014587402, + 0.0009451210498809814 + ] + ], + "generated_utilities": [ + 0.16174951195716858, + 0.5578349828720093, + 0.5522037148475647, + 0.5545214414596558, + 0.5589519739151001, + 0.23065254092216492, + 0.812659502029419, + 0.12922577559947968 + ], + "group_id": "ms-4048f8f6e047422e3c79", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558006286621094, + 2.0, + 0.1986379623413086, + 0.03769078850746155, + 0.5361635684967041, + 0.18274834752082825, + 0.17300468683242798, + 0.12688571214675903, + 0.13836368918418884, + 0.07454231381416321, + 0.0899321436882019, + 0.10402637720108032, + 0.1711854636669159, + 0.17255032062530518, + 0.039613887667655945, + 0.10826331377029419 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1422826200723648, + 0.21033799648284912, + 0.1978604793548584, + 0.059162259101867676, + -0.2529524564743042, + -0.15161612629890442, + 0.0, + -0.2264404594898224, + -0.2209387719631195, + -0.025619791820645332, + 0.06467145681381226, + 0.07422274351119995, + -1.7048513889312744, + 0.0, + 0.15056702494621277, + -0.06247536092996597, + -0.1033044084906578, + 0.1334916353225708, + 0.2784581184387207, + -1.4811434745788574, + 0.0 + ], + [ + -0.44999396800994873, + 0.271686851978302, + -0.4267011880874634, + 0.2730966806411743, + -0.01798880100250244, + -0.04680868983268738, + 0.0, + -0.09472770243883133, + -0.06993720680475235, + -0.06752754002809525, + -0.13884133100509644, + 0.24737784266471863, + -1.7048513889312744, + 0.0, + -0.09056920558214188, + 0.4132460653781891, + 0.04097161442041397, + -0.25072288513183594, + 0.49742037057876587, + -1.4811434745788574, + 0.0 + ], + [ + 0.17392832040786743, + 1.2103379964828491, + -0.3708428144454956, + -0.36093437671661377, + 0.744896650314331, + 0.23929360508918762, + 1.4388959407806396, + -0.04149632900953293, + 0.015302501618862152, + 0.023577755317091942, + 0.5954978466033936, + -0.3274606168270111, + -2.7048513889312744, + -1.646976351737976, + 0.0006552413105964661, + -0.008917607367038727, + 0.022643378004431725, + 0.7492771148681641, + -0.26030659675598145, + -2.4811434745788574, + -1.684959888458252 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8777918815612793, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.293952703475952, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.369919776916504 + ], + [ + 0.1792769730091095, + 0.9774190187454224, + -0.5587559342384338, + -0.5885956883430481, + 0.9904338717460632, + 0.41647347807884216, + 1.4140102863311768, + -0.2462461292743683, + -0.1271664798259735, + -0.2736872732639313, + 0.8877567052841187, + -0.6213628053665161, + -2.937077283859253, + -1.3732452392578125, + -0.0015218837652355433, + -0.27707284688949585, + -0.16698139905929565, + 0.4978373646736145, + -0.479316771030426, + -2.7234396934509277, + -1.6759265661239624 + ], + [ + 0.1658722460269928, + 1.3070863485336304, + -0.4893108606338501, + -0.04209986329078674, + 0.7449368238449097, + 0.3420809805393219, + 1.4472918510437012, + 0.08132945746183395, + 0.13181470334529877, + 0.22494959831237793, + 0.22593030333518982, + -0.49233078956604004, + -2.7544617652893066, + -1.7597702741622925, + -0.365922212600708, + -0.17204320430755615, + -0.1821347028017044, + 0.6334611177444458, + -0.33006516098976135, + -2.2188992500305176, + -1.5452594757080078 + ], + [ + 0.18608316779136658, + 1.23563551902771, + -0.5727517604827881, + 0.10049733519554138, + 0.9073957204818726, + 0.5650919675827026, + 1.8946568965911865, + -0.17719107866287231, + 0.34025683999061584, + 0.44525983929634094, + 0.6176816821098328, + -0.2832425534725189, + -3.1802117824554443, + -1.8703548908233643, + 0.07433810085058212, + 0.1340292990207672, + 0.497944176197052, + 1.0071699619293213, + -0.44033437967300415, + -2.7992849349975586, + -1.8935935497283936 + ], + [ + 0.40270811319351196, + 1.5102664232254028, + -0.8070739507675171, + -0.16390679776668549, + 0.3215266466140747, + -0.16522419452667236, + 1.0900052785873413, + 0.07780961692333221, + 0.5627153515815735, + 0.5386761426925659, + 0.2949375510215759, + 0.059314221143722534, + -3.2417359352111816, + -1.899662971496582, + 0.4160631597042084, + 0.44336366653442383, + 0.47819244861602783, + 1.162381649017334, + -0.7831735014915466, + -2.625349521636963, + -1.2278814315795898 + ], + [ + 0.8529313802719116, + 1.5106499195098877, + 0.19478559494018555, + -0.6338159441947937, + 0.7529481053352356, + -0.24222904443740845, + 0.9723314046859741, + -0.14078867435455322, + -0.5563283562660217, + 0.5440018177032471, + 0.004017293453216553, + 0.019366413354873657, + -3.1521430015563965, + -2.0841636657714844, + -0.2760852575302124, + -0.5531485676765442, + 0.22684869170188904, + 1.100138545036316, + 0.046634942293167114, + -2.429711103439331, + -2.280839443206787 + ], + [ + -0.4760962128639221, + 1.8579951524734497, + 0.2322864532470703, + 0.19953304529190063, + 0.48002323508262634, + 0.7484984397888184, + 2.073361396789551, + -0.04567806422710419, + 0.04004429653286934, + -0.5635936260223389, + 0.5135918259620667, + 0.3908633887767792, + -1.9335949420928955, + -2.442507743835449, + -0.6620182394981384, + 0.4175874888896942, + 0.7711085677146912, + 1.2790143489837646, + -0.5464571714401245, + -3.2030673027038574, + -2.3768434524536133 + ], + [ + 1.0231709480285645, + 1.1804075241088867, + 0.11513572931289673, + -1.23623526096344, + 1.4848686456680298, + 1.1120561361312866, + 0.9504602551460266, + 0.8234015703201294, + -0.04407603666186333, + -0.37310001254081726, + 0.24512693285942078, + 0.4242170751094818, + -2.4985997676849365, + -2.1206297874450684, + 0.5509956479072571, + 0.8686066269874573, + -0.5043656229972839, + 1.3957195281982422, + -0.07662208378314972, + -2.371795654296875, + -2.2404346466064453 + ], + [ + 0.49735015630722046, + 0.3600291609764099, + 0.3103446364402771, + 0.6051294803619385, + -0.15457379817962646, + -0.5570405721664429, + 1.2383109331130981, + 0.9059417247772217, + 0.7879754900932312, + -0.31492647528648376, + 1.5871094465255737, + -0.8906484842300415, + -1.8698136806488037, + -2.401073694229126, + 0.3560212254524231, + -0.6580788493156433, + 0.9037240147590637, + 1.614617109298706, + 0.3788090944290161, + -1.6909503936767578, + -0.8765208125114441 + ], + [ + 0.10883396118879318, + 1.8993752002716064, + -0.7673929333686829, + -0.0936923623085022, + -0.15508884191513062, + -0.1517968773841858, + 0.465425968170166, + 0.43258219957351685, + 0.46641960740089417, + -0.3528187572956085, + 1.09894597530365, + -0.6569450497627258, + -2.025486946105957, + -1.2820806503295898, + -0.21141839027404785, + -0.9515489339828491, + 0.7230286002159119, + -0.21062928438186646, + -0.32822558283805847, + -3.289642333984375, + -1.7125959396362305 + ], + [ + 0.37358880043029785, + 2.052347183227539, + -1.212618112564087, + -0.6321570873260498, + 0.7031511068344116, + -0.3477393090724945, + 1.2687753438949585, + 0.8023800253868103, + -0.031013749539852142, + 0.35525181889533997, + 0.5262036323547363, + -0.9288194179534912, + -1.7686026096343994, + -2.128396987915039, + -0.7001423239707947, + -0.12872236967086792, + -0.012955078855156898, + 1.0237239599227905, + 0.03636348247528076, + -2.0397119522094727, + -0.9761244058609009 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.060203760862350464, + 0.21033799648284912, + 0.0356404185295105, + -0.03044089674949646, + 0.024639129638671875, + -0.032671064138412476, + 0.0, + 0.016544528305530548, + 0.04633430764079094, + 0.005854343995451927, + -0.014876246452331543, + 0.0028293728828430176, + -1.7048513889312744, + 0.0, + -0.05953434109687805, + -0.04181046783924103, + -0.0018185675144195557, + -0.09216523170471191, + 0.019338399171829224, + -1.4811434745788574, + 0.0 + ] + ], + "predicted_scores": [ + 3.702712059020996, + 2.98641037940979, + 2.442595958709717, + 1.7719107866287231, + 1.769231915473938, + 1.3229467868804932, + 0.3650352358818054, + -0.19932720065116882 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055157354/train1", + "selected_index": 0, + "state_hash": "ae21d2b71722381a530d3a9259d02ea6728eb8a4a84fafc40ccd653d62ad3968", + "stored_base_utility": 0.8558006286621094, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19844311475753784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844311475753784 + }, + "base_progress": 0.19844311475753784, + "base_success": false, + "base_utility": 0.19844311475753784, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.20458371937274933, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20458371937274933 + }, + { + "contact_quality": null, + "progress": 0.24029652774333954, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24029652774333954 + }, + { + "contact_quality": null, + "progress": 0.22018474340438843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22018474340438843 + }, + { + "contact_quality": null, + "progress": 0.19370315968990326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19370315968990326 + }, + { + "contact_quality": null, + "progress": 0.2239307016134262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2239307016134262 + }, + { + "contact_quality": null, + "progress": 0.20504295825958252, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20504295825958252 + }, + { + "contact_quality": null, + "progress": 0.22744446992874146, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22744446992874146 + }, + { + "contact_quality": null, + "progress": 0.2148163914680481, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2148163914680481 + } + ], + "candidate_progress": [ + 0.20458371937274933, + 0.24029652774333954, + 0.22018474340438843, + 0.19370315968990326, + 0.2239307016134262, + 0.20504295825958252, + 0.22744446992874146, + 0.2148163914680481 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-5b26c33feed6992386b1", + "generated_tangents": [ + [ + -0.07354520261287689, + -0.1256038099527359, + -0.2062130868434906, + 0.5030757188796997, + 0.02786637842655182, + -0.8338708877563477, + 0.06525328755378723, + -0.013071492314338684, + 0.11664233356714249, + -0.05924617499113083, + 0.03110441565513611, + 0.14111444354057312, + -1.826629400253296, + -0.10948537290096283, + -0.10933530330657959, + -0.020234961062669754, + 0.06310255825519562, + 0.09728805720806122, + 0.033154457807540894, + -1.6914780139923096, + -0.2264210283756256 + ], + [ + -0.2372485250234604, + -0.20508919656276703, + -0.29013410210609436, + 0.6007887125015259, + 0.13013434410095215, + -0.689579963684082, + 0.348114550113678, + -0.03366383910179138, + 0.04903995245695114, + -0.07776796072721481, + -0.16477221250534058, + 0.09789817035198212, + -2.2579095363616943, + -0.1864905208349228, + -0.10923396050930023, + -0.12563833594322205, + 0.1276419758796692, + -0.016522720456123352, + 0.18242934346199036, + -2.0922954082489014, + -0.45239725708961487 + ], + [ + -0.20454083383083344, + -0.15289542078971863, + -0.12778088450431824, + 0.40291568636894226, + 0.024638861417770386, + -0.5686707496643066, + -0.06602460145950317, + -0.0829559862613678, + 0.04839726537466049, + -0.028551049530506134, + -0.0814475491642952, + 0.05080345273017883, + -2.0805954933166504, + -0.04724736511707306, + 0.0011115968227386475, + -0.041859667748212814, + 0.05227665603160858, + 0.14725975692272186, + -0.08195555210113525, + -1.9680421352386475, + -0.20420211553573608 + ], + [ + -0.09986400604248047, + -0.11947405338287354, + -0.08526220917701721, + 0.4571826159954071, + 0.07406111061573029, + -0.5261439085006714, + 0.0399547815322876, + -0.02421511709690094, + 0.09648419171571732, + -0.037118442356586456, + 0.055190496146678925, + -0.016269028186798096, + -2.046588659286499, + -0.06308583915233612, + -0.0685170441865921, + -0.03924035653471947, + 0.1361769139766693, + 0.08701697736978531, + -0.024386346340179443, + -1.9275929927825928, + -0.1816137433052063 + ], + [ + -0.25593799352645874, + -0.17109103500843048, + -0.4388103187084198, + 0.3898616135120392, + 0.155392125248909, + -0.4137926399707794, + 0.2990950644016266, + -0.1853753626346588, + 0.10374453663825989, + -0.06233047693967819, + -0.25568926334381104, + -0.10958299040794373, + -2.4645986557006836, + -0.13059793412685394, + 0.07097207754850388, + -0.06706021726131439, + 0.056096628308296204, + 0.03948955237865448, + -0.05719456076622009, + -2.224764585494995, + -0.3590715825557709 + ], + [ + -0.254397988319397, + -0.09702186286449432, + -0.1946195662021637, + 0.5826245546340942, + -0.06090956926345825, + -0.5958397388458252, + -0.012904822826385498, + -0.008259877562522888, + 0.0874367207288742, + 0.018531017005443573, + -0.05752967298030853, + 0.22654378414154053, + -1.7283936738967896, + -0.08846975862979889, + -0.03372752293944359, + -0.016586128622293472, + 0.0731029212474823, + 0.06927242875099182, + 0.03214584290981293, + -1.7384560108184814, + -0.271386057138443 + ], + [ + -0.5649675130844116, + -0.19363243877887726, + -0.306924432516098, + 0.6226949691772461, + 0.36562594771385193, + -0.5397567749023438, + 0.45113444328308105, + -0.16688323020935059, + 0.09275840222835541, + -0.08188348263502121, + -0.2544727623462677, + 0.08943998068571091, + -2.107145071029663, + -0.08943602442741394, + -0.046965304762125015, + -0.17391133308410645, + 0.005414009094238281, + -0.10378427803516388, + 0.016249656677246094, + -2.1992218494415283, + -0.3649754822254181 + ], + [ + -0.3835487365722656, + -0.17246809601783752, + -0.22917434573173523, + 0.48758628964424133, + 0.16649049520492554, + -0.5598640441894531, + 0.3399524688720703, + -0.1241578757762909, + 0.11243285983800888, + -0.0111432746052742, + -0.12439332157373428, + -0.014379605650901794, + -2.411860942840576, + 0.00608748197555542, + -0.0877176821231842, + -0.18000683188438416, + 0.10511918365955353, + -0.008005529642105103, + -0.0656428337097168, + -2.1491360664367676, + -0.4590318500995636 + ] + ], + "generated_utilities": [ + 0.20458371937274933, + 0.24029652774333954, + 0.22018474340438843, + 0.19370315968990326, + 0.2239307016134262, + 0.20504295825958252, + 0.22744446992874146, + 0.2148163914680481 + ], + "group_id": "ms-5b26c33feed6992386b1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8326650857925415, + 2.0, + 0.15131878852844238, + 0.15007886290550232, + 0.5293327569961548, + 0.06490398943424225, + 0.5425760746002197, + 0.20163774490356445, + 0.15259261429309845, + 0.23924465477466583, + 0.12766429781913757, + 0.08033241331577301, + 0.07788355648517609, + 0.13886235654354095, + 0.09716498851776123, + 0.11682072281837463 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.06662726402282715, + -0.14569896459579468, + 0.04974943399429321, + 0.539892315864563, + -0.19018621742725372, + -0.5884796380996704, + 0.0, + 0.12748029828071594, + 0.11038697510957718, + -0.03454907238483429, + 0.08700031787157059, + -0.13105249404907227, + -1.9082190990447998, + 0.0, + -0.0033895689994096756, + -0.15730009973049164, + 0.15846982598304749, + 0.11786291003227234, + -0.02387581765651703, + -1.5966379642486572, + 0.0 + ], + [ + -0.06662726402282715, + 0.059319257736206055, + -0.19934123754501343, + 0.539892315864563, + 0.07902508974075317, + -0.5884796380996704, + 0.0, + -0.3428957760334015, + 0.08558038622140884, + 0.28825634717941284, + -0.32335156202316284, + -0.3631777763366699, + -1.9082190990447998, + 0.0, + -0.3765835762023926, + -0.5554376840591431, + 0.3468284010887146, + 0.875933825969696, + 0.11397302150726318, + -1.5966379642486572, + 0.0 + ], + [ + -1.0666272640228271, + -0.6187554597854614, + -0.8818012475967407, + 1.539892315864563, + -0.428971529006958, + -1.5884796380996704, + 2.0647666454315186, + -0.03600159287452698, + 0.032059766352176666, + 0.030456876382231712, + 0.14377687871456146, + 0.20562076568603516, + -2.9082190990448, + -1.332404375076294, + -0.014788046479225159, + -0.0006958844605833292, + 0.023885300382971764, + 0.5519534349441528, + 0.1840958297252655, + -2.5966379642486572, + -1.4879226684570312 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.129533290863037, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.664808750152588, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9758453369140625 + ], + [ + -1.2971861362457275, + -0.7044790983200073, + -1.134492039680481, + 1.5821952819824219, + -0.7159937620162964, + -1.7615629434585571, + 2.031708002090454, + -0.25998222827911377, + 0.10180196911096573, + -0.24491730332374573, + 0.2074550986289978, + 0.2793378233909607, + -2.752415657043457, + -1.3360638618469238, + 0.03524862602353096, + -0.1854405552148819, + -0.1696440726518631, + 0.7172544598579407, + 0.0028989464044570923, + -2.680950164794922, + -1.2071388959884644 + ], + [ + -1.465846300125122, + -0.34887754917144775, + -1.046374797821045, + 1.703241229057312, + -0.19744430482387543, + -1.2604204416275024, + 1.9822474718093872, + 0.18854136765003204, + -0.146264910697937, + 0.059789031744003296, + 0.39534950256347656, + 0.29636046290397644, + -2.6612584590911865, + -1.5028973817825317, + 0.059081435203552246, + -0.11711112409830093, + -0.07238245755434036, + 0.25233787298202515, + 0.12625277042388916, + -2.2362401485443115, + -1.7053974866867065 + ], + [ + -1.1892943382263184, + -0.20831987261772156, + -0.8013461828231812, + 1.7197901010513306, + -0.39134201407432556, + -1.5160430669784546, + 2.4594008922576904, + -0.1669330894947052, + -0.337530255317688, + 0.15848031640052795, + 0.05738166719675064, + 0.41757503151893616, + -3.1393940448760986, + -0.9257140755653381, + -0.5068305134773254, + 0.1864723265171051, + 0.45369866490364075, + 0.6036283373832703, + -0.018199875950813293, + -2.8028440475463867, + -1.3847744464874268 + ], + [ + -1.1111165285110474, + -1.191490888595581, + -1.361661434173584, + 1.197826623916626, + -0.29219919443130493, + -1.7355475425720215, + 2.0579750537872314, + -0.08019472658634186, + 0.5140348672866821, + -0.5474883317947388, + -0.03311571478843689, + -0.060686588287353516, + -2.3427765369415283, + -1.8552747964859009, + -0.36303937435150146, + 0.01966947503387928, + -0.5651369094848633, + 0.22617560625076294, + -0.16322728991508484, + -2.421419143676758, + -0.921785831451416 + ], + [ + -1.012363076210022, + -0.6483663320541382, + -0.7951939702033997, + 1.2563079595565796, + -0.5323562622070312, + -1.5913673639297485, + 1.406038761138916, + -0.39036238193511963, + -0.12943506240844727, + -0.4373972415924072, + -0.20462383329868317, + 0.12806488573551178, + -3.570098400115967, + -1.8897531032562256, + 0.6369156837463379, + 0.2483828067779541, + 0.6809477210044861, + 0.8138958215713501, + 0.6604747176170349, + -2.717491865158081, + -1.9953765869140625 + ], + [ + -0.378162145614624, + -0.23438718914985657, + -1.2589956521987915, + 1.844052791595459, + -0.8936926126480103, + -1.230797290802002, + 2.5580525398254395, + -0.21314577758312225, + -0.05935201793909073, + 0.46811148524284363, + 0.38594305515289307, + 1.0049593448638916, + -2.491497278213501, + -1.3096134662628174, + 0.15605805814266205, + 0.7682130932807922, + -0.4010901153087616, + 0.8638691902160645, + 0.5755372047424316, + -3.324826240539551, + -1.6860322952270508 + ], + [ + -0.6920164227485657, + -0.5060986280441284, + -0.5146759748458862, + 2.3206489086151123, + 0.26810598373413086, + -1.7696658372879028, + 1.4922571182250977, + -0.6155052185058594, + -0.5581206679344177, + 0.4068271219730377, + 0.5812678337097168, + -0.2056950330734253, + -2.227752923965454, + -0.7079744935035706, + 0.3153572678565979, + 0.019053615629673004, + -0.10691303759813309, + -0.18750637769699097, + 0.5490041971206665, + -2.7611911296844482, + -1.0861625671386719 + ], + [ + -0.9623568654060364, + -0.821997880935669, + -0.0017323493957519531, + 1.8667385578155518, + 0.39793866872787476, + -1.3673040866851807, + 2.8636348247528076, + -0.739641547203064, + -0.4186078906059265, + -0.21357670426368713, + 0.6820193529129028, + 0.09538085013628006, + -3.242215633392334, + -1.023385763168335, + -0.2367737591266632, + 0.9248462915420532, + 0.04536557197570801, + 0.2294020652770996, + 0.15174128115177155, + -2.509904146194458, + -0.5901334881782532 + ], + [ + -0.565070390701294, + 0.37745314836502075, + -1.7464253902435303, + 1.4869437217712402, + -1.187619924545288, + -2.3802757263183594, + 2.5987753868103027, + 0.5777139663696289, + -0.04772312939167023, + -0.49332448840141296, + -0.515097975730896, + 0.751485288143158, + -2.7030012607574463, + -1.5739474296569824, + 0.8266244530677795, + -0.9454089403152466, + 0.7678747177124023, + 1.0557427406311035, + 0.0584852397441864, + -2.8590199947357178, + -0.8677505850791931 + ], + [ + -0.2763708233833313, + 0.28772038221359253, + -0.7008944153785706, + 1.3402774333953857, + -0.78607177734375, + -2.1210508346557617, + 1.4908761978149414, + -0.4155166447162628, + 0.40930652618408203, + -0.1769891083240509, + -0.20092366635799408, + 0.5321645736694336, + -3.2144649028778076, + -1.2440845966339111, + -0.247700035572052, + 0.7174559235572815, + 0.11071103811264038, + -0.023232340812683105, + 0.6976174116134644, + -2.9463140964508057, + -0.5927516222000122 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.06662726402282715, + -0.05619156360626221, + -0.02257448434829712, + 0.539892315864563, + 0.053197890520095825, + -0.5884796380996704, + 0.0, + -0.04219311475753784, + 0.010949447751045227, + 0.027476036921143532, + -0.02041013538837433, + -0.01791653037071228, + -1.9082190990447998, + 0.0, + 0.01256529800593853, + 0.014024228788912296, + 0.03545428439974785, + 0.019800782203674316, + -0.022738635540008545, + -1.5966379642486572, + 0.0 + ] + ], + "predicted_scores": [ + 5.118069648742676, + 5.116032600402832, + 4.928212642669678, + 4.704899311065674, + 4.558428764343262, + 4.518015384674072, + 3.5779058933258057, + 3.563319206237793 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1522272686/train1", + "selected_index": 0, + "state_hash": "10034ca87ce4b5c491c0aa23d2a30389ba8265fa695d65198bf719c54565cf2a", + "stored_base_utility": 0.8326650857925415, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8388344049453735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8388344049453735 + }, + "base_progress": 0.8388344049453735, + "base_success": false, + "base_utility": 0.8388344049453735, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8183318376541138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8183318376541138 + }, + { + "contact_quality": null, + "progress": 0.22806134819984436, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22806134819984436 + }, + { + "contact_quality": null, + "progress": 0.832514762878418, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.832514762878418 + }, + { + "contact_quality": null, + "progress": 0.19844911992549896, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844911992549896 + }, + { + "contact_quality": null, + "progress": 0.22729773819446564, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22729773819446564 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.20421144366264343, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20421144366264343 + }, + { + "contact_quality": null, + "progress": 0.21390068531036377, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21390068531036377 + } + ], + "candidate_progress": [ + 0.8183318376541138, + 0.22806134819984436, + 0.832514762878418, + 0.19844911992549896, + 0.22729773819446564, + 1.0, + 0.20421144366264343, + 0.21390068531036377 + ], + "candidate_source_chart_ids": [ + "ms-5dfbb16e173e1a3e2bd6", + "ms-0b226017db7f07c5307e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-5a456ddf97d294548cf1", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-63ac245579b1188935ad", + "generated_tangents": [ + [ + 0.07592315971851349, + -0.13614404201507568, + 0.24823014438152313, + -0.01298859715461731, + -0.09450054168701172, + -1.8092811107635498, + 0.15673872828483582, + 0.037664398550987244, + -0.0279239509254694, + -0.15241870284080505, + -0.010709114372730255, + 0.02850356698036194, + -1.1655819416046143, + 0.08682677149772644, + -0.022702503949403763, + -0.06397686898708344, + 0.0005460083484649658, + 0.23710030317306519, + 0.17263144254684448, + -1.1171483993530273, + -0.3485688865184784 + ], + [ + 0.07267247140407562, + -0.0802840143442154, + -0.08107021450996399, + 0.07116827368736267, + 0.08196194469928741, + -1.653926968574524, + 0.07945936918258667, + 0.01596371829509735, + -0.00013408716768026352, + -0.047363825142383575, + 0.06136802211403847, + -0.02504594624042511, + -1.290826678276062, + -0.026758089661598206, + -0.11897893249988556, + 0.00425499863922596, + 0.12613841891288757, + 0.06169748306274414, + -0.061475396156311035, + -1.1233642101287842, + -0.1260521113872528 + ], + [ + 0.060978569090366364, + -0.02506651170551777, + -0.024420887231826782, + 0.19880282878875732, + 0.08790360391139984, + -1.5615947246551514, + 0.10512334108352661, + -0.00031866133213043213, + 0.03373006731271744, + -0.10571563988924026, + 0.06250378489494324, + 0.11451734602451324, + -1.1461238861083984, + -0.03493809700012207, + -0.13956454396247864, + -0.050033316016197205, + 0.09889145195484161, + 0.06260648369789124, + -0.029381930828094482, + -1.1260510683059692, + -0.1741926670074463 + ], + [ + 0.09063458442687988, + -0.07695324718952179, + -0.05717006325721741, + 0.036723554134368896, + 0.13756972551345825, + -1.5772385597229004, + 0.06240609288215637, + 0.03608326613903046, + -0.026225006207823753, + -0.027511008083820343, + 0.05826742202043533, + -0.012191027402877808, + -1.243361234664917, + -0.067467600107193, + -0.0990309864282608, + -0.020708836615085602, + 0.17371058464050293, + 0.04112726449966431, + -0.059579998254776, + -1.1515450477600098, + -0.16000548005104065 + ], + [ + 0.06826478242874146, + -0.09602241218090057, + -0.08379694819450378, + 0.06933766603469849, + 0.19838720560073853, + -1.4886646270751953, + 0.05686655640602112, + 0.013776406645774841, + 0.016440080478787422, + -0.03296954184770584, + -0.009033150970935822, + 0.07672037929296494, + -1.180536150932312, + 0.029635459184646606, + -0.10914972424507141, + -0.047345105558633804, + 0.13566727936267853, + 0.020698681473731995, + -0.06398943066596985, + -1.1226403713226318, + -0.05157192051410675 + ], + [ + 0.0007637739181518555, + -0.12595009803771973, + 0.04650917649269104, + -0.0979643166065216, + -0.017885595560073853, + -1.7403672933578491, + -0.022388488054275513, + -0.009271219372749329, + 0.018568366765975952, + -0.052760593593120575, + 0.06903066486120224, + -0.07851454615592957, + -1.2405943870544434, + 0.019612088799476624, + -0.04794960841536522, + 0.026756152510643005, + 0.20309510827064514, + 0.09076309204101562, + -0.23707938194274902, + -1.091101884841919, + -0.15543323755264282 + ], + [ + 0.09207674860954285, + -0.08228801935911179, + -0.04116174578666687, + -0.0035973042249679565, + 0.15192490816116333, + -1.4146941900253296, + -0.02971857786178589, + 0.1076693907380104, + -0.055346980690956116, + -0.0685649886727333, + 0.10692813992500305, + 0.1420077085494995, + -1.13576078414917, + -0.02274516224861145, + -0.09511969983577728, + 0.022226441651582718, + 0.02580389380455017, + 0.16851267218589783, + -0.054017484188079834, + -1.016629695892334, + -0.20070743560791016 + ], + [ + 0.04372203350067139, + -0.06514435261487961, + -0.08617410063743591, + 0.029892414808273315, + 0.06151294708251953, + -1.5051560401916504, + -0.042172372341156006, + 0.059974148869514465, + -0.004093525931239128, + -0.0804525837302208, + 0.046910591423511505, + 0.027034729719161987, + -1.095505952835083, + -0.10103793442249298, + -0.09881904721260071, + 0.0494321808218956, + 0.11947246640920639, + 0.06747278571128845, + -0.074625164270401, + -1.0337462425231934, + -0.2033267319202423 + ] + ], + "generated_utilities": [ + 0.8183318376541138, + 0.22806134819984436, + 0.832514762878418, + 0.19844911992549896, + 0.22729773819446564, + 2.0, + 0.20421144366264343, + 0.21390068531036377 + ], + "group_id": "ms-63ac245579b1188935ad", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.855198860168457, + 0.8138954639434814, + 0.20312510430812836, + 2.0, + 0.7862945795059204, + 0.7843606472015381, + 0.6004383563995361, + 0.15506848692893982, + 0.15924105048179626, + 0.132452130317688, + 0.05381816625595093, + 0.09561765193939209, + 0.07140937447547913, + 0.10806754231452942, + 0.023168057203292847, + 0.07957816123962402 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.007633227854967117, + 0.02389371022582054, + 0.2181050330400467, + 0.026324495673179626, + 0.1209479570388794, + -1.3937149047851562, + 0.0, + -0.2493116855621338, + -0.17256586253643036, + -0.06812826544046402, + 0.019445210695266724, + -0.2657848000526428, + -1.0657541751861572, + 0.0, + -0.04306507110595703, + -0.07170329242944717, + 0.035314902663230896, + 0.06211268901824951, + 0.18158195912837982, + -0.7849693298339844, + 0.0 + ], + [ + -0.011555496603250504, + 0.0016381663735955954, + 0.019193550571799278, + 0.26171422004699707, + 0.08122420310974121, + -2.3937149047851562, + -1.53977632522583, + -0.00580335408449173, + -0.0020052569452673197, + 0.01787502132356167, + 0.4615197777748108, + 0.1938573122024536, + -2.0657541751861572, + -1.4660639762878418, + -0.00267898291349411, + -0.0038005562964826822, + 0.0179074015468359, + 0.5644202828407288, + 0.2690354883670807, + -1.7849693298339844, + -1.3308851718902588 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.07955265045166, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9321279525756836, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.6617703437805176 + ], + [ + -0.16036809980869293, + 0.11436319351196289, + -0.14857570827007294, + 0.09414434432983398, + 0.02274160459637642, + -2.426297903060913, + -1.7686878442764282, + -0.26862773299217224, + 0.16241224110126495, + 0.24208402633666992, + 0.4166663587093353, + 0.07841069251298904, + -2.2318098545074463, + -1.2228643894195557, + -0.08888694643974304, + 0.20051127672195435, + -0.14729085564613342, + 0.5799955129623413, + 0.33039355278015137, + -2.00492262840271, + -1.4948352575302124 + ], + [ + -0.3820815682411194, + -0.3679526746273041, + 0.18845602869987488, + 0.4168931245803833, + -0.007936321198940277, + -2.700572967529297, + -1.795093059539795, + 0.1203194335103035, + 0.19610334932804108, + 0.2749679982662201, + 0.8331882357597351, + -0.06589585542678833, + -2.333136558532715, + -1.1008005142211914, + -0.02170141413807869, + 0.24186654388904572, + -0.3734443485736847, + 0.5144574046134949, + 0.3134346902370453, + -1.5088386535644531, + -1.1978046894073486 + ], + [ + 0.13625949621200562, + 0.47836095094680786, + -0.05790846049785614, + -0.09614765644073486, + -0.14281350374221802, + -2.522264003753662, + -1.0473889112472534, + 0.4792231023311615, + -0.4869813621044159, + 0.17644789814949036, + 0.8425257205963135, + -0.28899794816970825, + -2.5508456230163574, + -0.9762126207351685, + -0.16381770372390747, + 0.1586131900548935, + -0.3460317850112915, + 0.6056831479072571, + 0.7327044010162354, + -1.494289755821228, + -1.4671605825424194 + ], + [ + -0.40018847584724426, + -0.028111951425671577, + 0.31324630975723267, + -0.22690656781196594, + 0.2902344763278961, + -2.9841673374176025, + -1.4654184579849243, + -0.1155615746974945, + -0.4125809073448181, + 0.6098153591156006, + 0.8544195890426636, + 0.22702345252037048, + -1.7853796482086182, + -1.3474400043487549, + -0.454882949590683, + -0.3122820556163788, + 0.22399993240833282, + 0.05299121141433716, + 0.07802119851112366, + -1.3896000385284424, + -1.894580602645874 + ], + [ + -0.6587541699409485, + -0.6905457973480225, + 0.10385943204164505, + -0.2785833477973938, + 0.2321375161409378, + -2.565701723098755, + -1.1605603694915771, + -0.6775721907615662, + -0.30887091159820557, + 0.1186944767832756, + 1.0924456119537354, + 0.5009081363677979, + -1.8667830228805542, + -1.2491121292114258, + -0.11290804296731949, + -0.03346216678619385, + -0.28778207302093506, + 1.2255009412765503, + 0.6856998801231384, + -1.1723419427871704, + -1.1220875978469849 + ], + [ + -0.5019480586051941, + 0.07311037182807922, + 0.5101989507675171, + 0.7159117460250854, + 0.005071915686130524, + -2.6556949615478516, + -2.321233034133911, + 0.6035679578781128, + -0.13785676658153534, + -0.4309522807598114, + 0.9664275050163269, + -0.020235419273376465, + -1.6894025802612305, + -0.7511875033378601, + 0.5713916420936584, + 0.24962951242923737, + 0.40067923069000244, + 0.12209588289260864, + 0.03222101926803589, + -2.27744460105896, + -1.9113109111785889 + ], + [ + 0.7552750110626221, + -0.8085685968399048, + 0.20896124839782715, + -0.6043995022773743, + 0.733860969543457, + -1.610467791557312, + -1.0368051528930664, + -0.02292630262672901, + 0.6320546269416809, + 0.14528857171535492, + 0.8372507095336914, + 0.41523146629333496, + -2.339751958847046, + -0.6185142993927002, + -0.5750091075897217, + 0.12324154376983643, + 0.2804809510707855, + 1.0503896474838257, + -0.15392154455184937, + -0.9555137753486633, + -1.517809510231018 + ], + [ + 0.8853163123130798, + 0.09475164860486984, + -0.13681088387966156, + 0.08429434895515442, + 0.13555681705474854, + -1.5859344005584717, + -2.1957859992980957, + 0.9337366819381714, + -1.0016963481903076, + 0.11365362256765366, + -0.31175100803375244, + 0.7412776350975037, + -1.0797133445739746, + -0.8527018427848816, + 0.6110302209854126, + 0.006565961055457592, + 0.38363590836524963, + 0.5695288181304932, + -0.2547180950641632, + -2.6268043518066406, + -1.748514175415039 + ], + [ + 0.513801097869873, + 0.7078304886817932, + 0.7309613823890686, + -0.32746708393096924, + -0.36263394355773926, + -3.3305888175964355, + -1.2108653783798218, + 0.5104080438613892, + -0.26447102427482605, + 0.6361017823219299, + 0.5986918210983276, + -0.48598653078079224, + -2.4424209594726562, + -1.4694169759750366, + -0.5649402737617493, + -0.22020848095417023, + 0.6405193209648132, + -0.0698668360710144, + 0.7155715227127075, + -2.352860450744629, + -0.7502654194831848 + ], + [ + -0.9911022782325745, + 0.9677192568778992, + 0.5195661187171936, + 0.6733791828155518, + 0.7589308023452759, + -3.3654959201812744, + -0.6360958814620972, + 0.5926312208175659, + -0.06773044914007187, + 0.3785405457019806, + 0.5934506058692932, + 0.6434512138366699, + -2.891094923019409, + -0.8924227356910706, + 0.8036434650421143, + 0.6109268665313721, + -0.6607317924499512, + 1.0173940658569336, + 0.2546094059944153, + -1.4653143882751465, + -2.0709643363952637 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16686837375164032, + 0.1354454904794693, + -0.11150697618722916, + 0.7247087359428406, + 0.2886054813861847, + -1.3937149047851562, + 0.0, + 0.6050093770027161, + 0.20291854441165924, + 0.5827622413635254, + -0.2687305808067322, + -0.25171223282814026, + -1.0657541751861572, + 0.0, + 0.29065752029418945, + -0.023413484916090965, + -0.24587245285511017, + 0.4600236713886261, + -0.4278547465801239, + -0.7849693298339844, + 0.0 + ] + ], + "predicted_scores": [ + 3.951364278793335, + 3.529320478439331, + 3.5285229682922363, + 3.487807273864746, + 3.3132495880126953, + 3.2474350929260254, + 3.2326605319976807, + 3.150263547897339 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "211245842/train1", + "selected_index": 0, + "state_hash": "fb52e84b6addb8bdddf02a80d5c8567e0fa27195729e8440271b710441061b7f", + "stored_base_utility": 0.855198860168457, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8184402585029602, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8184402585029602 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8130596876144409, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8130596876144409 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8125015497207642, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8125015497207642 + }, + { + "contact_quality": null, + "progress": 0.8125571608543396, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8125571608543396 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8167412877082825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8167412877082825 + } + ], + "candidate_progress": [ + 0.8184402585029602, + 1.0, + 0.8130596876144409, + 1.0, + 0.8125015497207642, + 0.8125571608543396, + 1.0, + 0.8167412877082825 + ], + "candidate_source_chart_ids": [ + "ms-875a4668671c02054a3c", + "ms-20f3cfcf183c69853b1e", + "ms-875a4668671c02054a3c", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + true, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "generated_tangents": [ + [ + -0.01070578396320343, + -0.033470772206783295, + -0.15495410561561584, + -0.14183250069618225, + 0.07164767384529114, + -1.9417678117752075, + -0.14187884330749512, + 0.08544448018074036, + 0.14689677953720093, + -0.052584730088710785, + 0.03513490408658981, + -0.05415037274360657, + -1.8725054264068604, + -0.1501309722661972, + -0.03508613631129265, + -0.05812845751643181, + 0.0007632821798324585, + 0.0408368855714798, + 0.020252972841262817, + -1.7189569473266602, + -0.2775786817073822 + ], + [ + -0.012276023626327515, + -0.028644850477576256, + -0.09336325526237488, + -0.14155516028404236, + 0.09984803199768066, + -2.1003611087799072, + -0.07957345247268677, + 0.02338235080242157, + 0.18073223531246185, + -0.0357944592833519, + -0.012997716665267944, + -0.08227553963661194, + -1.8983495235443115, + -0.14400701224803925, + -0.05123366788029671, + -0.02659599855542183, + 0.0619058758020401, + 0.08809824287891388, + -0.07122787833213806, + -1.5974328517913818, + -0.20015138387680054 + ], + [ + -0.11515431106090546, + -0.17193876206874847, + 0.10262271761894226, + -0.28938037157058716, + 0.2151399850845337, + -2.1313931941986084, + -0.0048177242279052734, + 0.16256849467754364, + 0.14841464161872864, + -0.09813383966684341, + -0.12788528203964233, + -8.824467658996582e-05, + -1.9873358011245728, + -0.5370582938194275, + -0.07476823031902313, + 0.007620302960276604, + -0.1157131940126419, + 0.0862252488732338, + 0.14170050621032715, + -2.0510191917419434, + -0.8264507055282593 + ], + [ + -0.08699685335159302, + -0.06080082803964615, + 0.035507649183273315, + -0.30851852893829346, + 0.21045854687690735, + -1.9924968481063843, + -0.0053428709506988525, + 0.0917435809969902, + 0.15357144176959991, + -0.022615395486354828, + -0.041588179767131805, + -0.14054468274116516, + -1.841922640800476, + -0.34916096925735474, + -0.0639040470123291, + 0.004495611414313316, + 0.10388081520795822, + 0.044780537486076355, + 0.0870068371295929, + -1.650801420211792, + -0.46704408526420593 + ], + [ + 0.06315898150205612, + -0.0010309480130672455, + -0.05888590216636658, + -0.12963604927062988, + 0.1410176157951355, + -2.07487154006958, + -0.0625162124633789, + 0.038195788860321045, + 0.1573835015296936, + -0.21881094574928284, + 0.008190367370843887, + -0.23996657133102417, + -1.8577485084533691, + -0.29064762592315674, + -0.3194462060928345, + -0.15451104938983917, + 0.1985035091638565, + -0.03591302037239075, + -0.2407069206237793, + -1.6201457977294922, + 0.11997450143098831 + ], + [ + -0.051500990986824036, + -0.1544715166091919, + -0.09336391091346741, + -0.3871646225452423, + 0.26705265045166016, + -1.8083488941192627, + 0.11108162999153137, + 0.19076180458068848, + 0.12387299537658691, + -0.03509994596242905, + 0.12095065414905548, + -0.3259950280189514, + -1.9535678625106812, + -0.30912458896636963, + 0.030186571180820465, + -0.03327091783285141, + 0.07141990959644318, + -0.007241472601890564, + 0.20521318912506104, + -1.9050300121307373, + -0.5703409910202026 + ], + [ + -0.14716701209545135, + -0.05128244683146477, + -0.017126411199569702, + -0.20498310029506683, + 0.2128160148859024, + -1.8568142652511597, + -0.07626485824584961, + 0.13453879952430725, + 0.16995331645011902, + 0.013863474130630493, + -0.11347896605730057, + -0.14959144592285156, + -1.8593852519989014, + -0.23283414542675018, + -0.0664626955986023, + -0.004108800552785397, + 0.10919519513845444, + -0.053353384137153625, + -0.07882529497146606, + -1.6868853569030762, + -0.5271542072296143 + ], + [ + -0.06839938461780548, + -0.005638685077428818, + -0.0015881359577178955, + -0.326743483543396, + 0.29128238558769226, + -1.9975172281265259, + 0.08928051590919495, + 0.019716590642929077, + 0.15808995068073273, + -0.07500874251127243, + -0.057167477905750275, + -0.04409746825695038, + -1.923764705657959, + -0.2519254684448242, + -0.06714454293251038, + -0.05821181833744049, + 0.044128432869911194, + 0.06208115816116333, + -0.020185887813568115, + -1.5056889057159424, + -0.45270946621894836 + ] + ], + "generated_utilities": [ + 0.8184402585029602, + 2.0, + 0.8130596876144409, + 2.0, + 0.8125015497207642, + 0.8125571608543396, + 2.0, + 0.8167412877082825 + ], + "group_id": "ms-6c8d06bace98a28a8e4e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8583254218101501, + 2.0, + 0.8125, + 0.12893107533454895, + 0.8582976460456848, + 0.15346482396125793, + 0.18961156904697418, + 0.13901273906230927, + 0.10113435983657837, + 0.11161506175994873, + 0.09923169016838074, + 0.16662254929542542, + 0.04211682081222534, + 0.024710848927497864, + 0.05722464621067047, + 0.8584086298942566 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.10696407407522202, + 0.3957799971103668, + 0.15132969617843628, + -0.32772913575172424, + 0.005261177197098732, + -2.2650365829467773, + 0.0, + 0.3958877623081207, + -0.5120954513549805, + -0.4317946135997772, + -0.3970779776573181, + 0.5317854285240173, + -1.8659565448760986, + 0.0, + 0.1198807880282402, + -0.11760073900222778, + -0.6048114895820618, + 0.15180528163909912, + -0.532695472240448, + -1.6115882396697998, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.120756149291992, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.070481538772583, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0495758056640625 + ], + [ + 0.30105674266815186, + 0.4776385426521301, + -0.22267001867294312, + -0.13300693035125732, + -0.11095686256885529, + -3.138023853302002, + -0.793931245803833, + 0.16969743371009827, + -0.16339385509490967, + 0.02508675307035446, + 0.654399573802948, + -0.22830897569656372, + -3.1443004608154297, + -1.4291013479232788, + 0.005021120421588421, + 0.005560946650803089, + -0.1901366114616394, + 1.0211858749389648, + -0.09353887289762497, + -2.3457469940185547, + -1.406853199005127 + ], + [ + 0.4160586893558502, + 0.13316196203231812, + -0.21056225895881653, + -0.25006812810897827, + 0.060951750725507736, + -2.8735690116882324, + -0.9062713980674744, + -0.28707224130630493, + -0.2396206259727478, + -0.19859735667705536, + 0.29039445519447327, + -0.24077445268630981, + -2.644789934158325, + -1.8330551385879517, + -0.23438936471939087, + 0.04620177671313286, + 0.1472887098789215, + 0.9629149436950684, + -0.020509399473667145, + -2.7985036373138428, + -1.2463163137435913 + ], + [ + 0.2486386001110077, + -0.022333532571792603, + 0.14628465473651886, + -0.8381847143173218, + 0.21282373368740082, + -3.585387945175171, + -1.2156991958618164, + -0.08670558780431747, + -0.3178575336933136, + 0.4322021007537842, + 0.6421236395835876, + -0.33922863006591797, + -3.004714012145996, + -1.19637131690979, + -0.015640778467059135, + 0.10077434033155441, + -0.449941486120224, + 1.0431067943572998, + 0.2899783253669739, + -2.500624179840088, + -1.6148788928985596 + ], + [ + 0.46770766377449036, + 0.03902164101600647, + 0.2556130588054657, + -0.45893940329551697, + 0.08939866721630096, + -3.163339853286743, + -0.7599828243255615, + 0.3327450156211853, + 0.3252364993095398, + 0.3067358434200287, + 1.1394517421722412, + 0.472858726978302, + -3.1504058837890625, + -1.9780522584915161, + -0.26712900400161743, + -0.262444406747818, + 0.20269782841205597, + 1.336982250213623, + -0.06957009434700012, + -2.0651021003723145, + -1.8484716415405273 + ], + [ + 0.48348987102508545, + 0.19676931202411652, + -0.20689666271209717, + -0.21521174907684326, + -0.5048230886459351, + -3.491732358932495, + -0.7390058636665344, + 0.18321916460990906, + -0.1509065330028534, + 0.27166980504989624, + 1.004181981086731, + -0.5531666874885559, + -2.531594753265381, + -2.0243897438049316, + -0.44593361020088196, + 0.49786314368247986, + -0.006632372736930847, + 0.24423635005950928, + -0.6331305503845215, + -2.7707719802856445, + -2.1364808082580566 + ], + [ + -0.5684993863105774, + 0.0623701810836792, + -0.10449839383363724, + -0.7858190536499023, + -0.057080868631601334, + -3.981018304824829, + -0.6721940636634827, + 0.10191962122917175, + -0.7216448783874512, + 0.3870577812194824, + 0.5508396625518799, + -0.6811783313751221, + -3.6131227016448975, + -1.7484180927276611, + 0.34220433235168457, + -0.16193144023418427, + 0.16876991093158722, + 0.6509957909584045, + 0.2548854351043701, + -2.0779964923858643, + -1.5782216787338257 + ], + [ + -0.1224658265709877, + 0.22953394055366516, + -0.41326335072517395, + 0.19802969694137573, + 0.6866055130958557, + -2.6415462493896484, + -1.049781322479248, + -0.6788480281829834, + -0.8187242746353149, + 0.6141759157180786, + 1.3649951219558716, + -0.36378854513168335, + -2.2142889499664307, + -1.3759111166000366, + 0.583359956741333, + 0.6860965490341187, + 0.8285356760025024, + 1.4575884342193604, + -0.4101440906524658, + -2.218319892883301, + -1.5390111207962036 + ], + [ + 0.3368571698665619, + 0.4528505206108093, + -0.2985517382621765, + 0.37766069173812866, + 0.2064778208732605, + -2.8070714473724365, + -1.5598305463790894, + -0.9032005071640015, + -0.48420336842536926, + 0.639835000038147, + -0.3801422119140625, + -0.1376074254512787, + -3.369077205657959, + -1.9231767654418945, + -0.16211053729057312, + 0.53387051820755, + 0.6583098769187927, + 0.7611914873123169, + -0.46404945850372314, + -2.7554476261138916, + -1.4307246208190918 + ], + [ + 0.6002402901649475, + -0.14213827252388, + 0.30183425545692444, + -0.20118463039398193, + -0.16981711983680725, + -3.4212875366210938, + -0.2252577543258667, + 0.34123149514198303, + 0.003176456317305565, + 0.7433648109436035, + 1.1403348445892334, + 0.5709706544876099, + -1.8894051313400269, + -0.7200812101364136, + -0.7235198020935059, + 0.2147149294614792, + -0.28744715452194214, + 1.0323235988616943, + 0.5815067291259766, + -2.582993984222412, + -1.2160475254058838 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.019069351255893707, + 0.012574762105941772, + 0.07413499802350998, + -0.04097828269004822, + -0.03991875424981117, + -2.2650365829467773, + 0.0, + -0.06082560494542122, + 0.05821063742041588, + 0.07896332442760468, + -0.037061095237731934, + -0.08879614621400833, + -1.8659565448760986, + 0.0, + -0.022465867921710014, + -0.026843247935175896, + 0.04789978265762329, + -0.019972383975982666, + -0.04040021449327469, + -1.6115882396697998, + 0.0 + ] + ], + "predicted_scores": [ + 2.5245156288146973, + 2.4051806926727295, + 2.306995391845703, + 1.821117877960205, + 1.7071008682250977, + 1.702606201171875, + 1.620788812637329, + 1.4197536706924438 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1665739511/train1", + "selected_index": 0, + "state_hash": "0f77663511063f17b0962024501b863c62b881513459d8bfa3ac00bbaed67de4", + "stored_base_utility": 0.8583254218101501, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.19993644952774048, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19993644952774048 + }, + { + "contact_quality": null, + "progress": 0.23993994295597076, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23993994295597076 + }, + { + "contact_quality": null, + "progress": 0.22335931658744812, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22335931658744812 + }, + { + "contact_quality": null, + "progress": 0.23456653952598572, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23456653952598572 + }, + { + "contact_quality": null, + "progress": 0.23101213574409485, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23101213574409485 + }, + { + "contact_quality": null, + "progress": 0.2455206662416458, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2455206662416458 + }, + { + "contact_quality": null, + "progress": 0.8128684163093567, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8128684163093567 + }, + { + "contact_quality": null, + "progress": 0.14169609546661377, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14169609546661377 + } + ], + "candidate_progress": [ + 0.19993644952774048, + 0.23993994295597076, + 0.22335931658744812, + 0.23456653952598572, + 0.23101213574409485, + 0.2455206662416458, + 0.8128684163093567, + 0.14169609546661377 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-72777024349c2aacd5e0", + "generated_tangents": [ + [ + 0.055865637958049774, + -0.055816806852817535, + -0.20907989144325256, + -0.769936203956604, + 0.4880841374397278, + -0.25204890966415405, + 0.6985225677490234, + 0.12398729473352432, + 0.16473756730556488, + -0.21713915467262268, + 0.05861528217792511, + -0.06095767021179199, + -1.8979934453964233, + -0.37624984979629517, + -0.006911199539899826, + 0.010136563330888748, + 0.02695821225643158, + 0.22647030651569366, + -0.2013503909111023, + -1.2692904472351074, + -0.1853887140750885 + ], + [ + -0.020478785037994385, + 0.044863805174827576, + -0.18460753560066223, + -0.6313502788543701, + 0.7127556204795837, + -0.28846850991249084, + 0.3333684802055359, + 0.016965195536613464, + 0.09148606657981873, + -0.08868777006864548, + 0.012398604303598404, + -0.08372506499290466, + -1.7294925451278687, + 0.13085003197193146, + -0.06228635832667351, + -0.050725340843200684, + 0.06951582431793213, + 0.03120531141757965, + -0.1471652388572693, + -1.3888170719146729, + 0.022654831409454346 + ], + [ + 0.05123429745435715, + 0.010771278291940689, + -0.3759067952632904, + -0.5952674150466919, + 0.7524579763412476, + -0.19433516263961792, + 0.3558697998523712, + 0.07158229500055313, + 0.08915022015571594, + -0.0989876464009285, + 0.0979427918791771, + -0.15328934788703918, + -1.7377667427062988, + 0.19969624280929565, + -0.13484250009059906, + -0.04985309764742851, + -0.009539350867271423, + 0.005046576261520386, + -0.09007668495178223, + -1.4380772113800049, + 0.043121084570884705 + ], + [ + -0.12803484499454498, + 0.0055353716015815735, + -0.20564386248588562, + -0.741146445274353, + 0.7627297639846802, + -0.4963206946849823, + 0.6548652052879333, + 0.05438297986984253, + 0.1399819552898407, + -0.24557217955589294, + 0.0346328541636467, + -0.2714666724205017, + -1.7525880336761475, + -0.13117019832134247, + 0.07491924613714218, + -0.04336576163768768, + 0.06672023236751556, + -0.11204515397548676, + -0.2286921739578247, + -1.4248881340026855, + -0.15738847851753235 + ], + [ + -0.07982590794563293, + 0.044089801609516144, + -0.13705822825431824, + -0.5091069340705872, + 0.707309365272522, + -0.37458983063697815, + 0.8120944499969482, + -0.00697346031665802, + 0.16757282614707947, + -0.2451404333114624, + 0.22887039184570312, + 0.20996689796447754, + -1.660861611366272, + -0.24643395841121674, + -0.09035596251487732, + -0.04664681851863861, + -0.10761897265911102, + 0.05824191868305206, + 0.2820228338241577, + -1.3844363689422607, + -0.12130829691886902 + ], + [ + -0.13600154221057892, + 0.015058614313602448, + -0.13416323065757751, + -0.7091516256332397, + 0.6443045139312744, + -0.4889087975025177, + 0.5162445902824402, + 0.04842694103717804, + 0.13782557845115662, + -0.2266004979610443, + 0.08909663558006287, + -0.11199542880058289, + -1.8013991117477417, + -0.019768044352531433, + -0.024277430027723312, + -0.04220620170235634, + 0.033375561237335205, + 0.03389929234981537, + -0.1650262475013733, + -1.3381359577178955, + -0.12047871947288513 + ], + [ + -0.15009067952632904, + 0.05454045534133911, + 0.029337078332901, + -0.6240274906158447, + 0.6465781927108765, + -0.4041072428226471, + 0.3130103349685669, + 0.07021210342645645, + 0.09378987550735474, + -0.21047285199165344, + 0.11681286990642548, + -0.14802634716033936, + -1.8183677196502686, + 0.3068639636039734, + -0.010608140379190445, + -0.11435514688491821, + 0.07564397156238556, + 0.13713252544403076, + -0.13190573453903198, + -1.4660191535949707, + -0.04035669565200806 + ], + [ + 0.19345687329769135, + 0.036093391478061676, + 0.3382440209388733, + -0.9522699117660522, + 0.27695274353027344, + -1.1500868797302246, + 1.2254087924957275, + 0.09756684303283691, + 0.10196477919816971, + -0.11644334346055984, + 0.16047650575637817, + -0.19358840584754944, + -1.3692970275878906, + -0.3326490521430969, + -0.07179197669029236, + -0.18614377081394196, + 0.1879936307668686, + 0.07348190248012543, + -0.2861347198486328, + -1.5783159732818604, + -0.08773162961006165 + ] + ], + "generated_utilities": [ + 0.19993644952774048, + 0.23993994295597076, + 0.22335931658744812, + 0.23456653952598572, + 0.23101213574409485, + 0.2455206662416458, + 0.8128684163093567, + 0.14169609546661377 + ], + "group_id": "ms-72777024349c2aacd5e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8314920663833618, + 0.1595238745212555, + 2.0, + 0.16971935331821442, + 0.5892879366874695, + 0.1617850512266159, + 0.1877371072769165, + 0.5782577395439148, + 0.12074722349643707, + 0.11237785220146179, + 0.08792310953140259, + 0.1209893673658371, + 0.13739405572414398, + 0.0773405134677887, + 0.09497407078742981, + 0.047447383403778076 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.002835560590028763, + 0.3854118585586548, + -0.06168194115161896, + -0.16570186614990234, + -0.05051702260971069, + -0.43052077293395996, + 0.0, + 0.024141468107700348, + -0.011737043969333172, + -0.05502588301897049, + 0.013536334037780762, + -0.000911470502614975, + -1.6284780502319336, + 0.0, + 0.030459173023700714, + -0.05656937509775162, + 0.03393230587244034, + 0.06894856691360474, + 0.00450655072927475, + -1.2762084007263184, + 0.0 + ], + [ + 0.2191334217786789, + 0.3854118585586548, + 0.5048428177833557, + -0.16570186614990234, + -0.20026534795761108, + -0.43052077293395996, + 0.0, + -0.026082046329975128, + 0.03163425251841545, + -0.04950854927301407, + 0.1969367265701294, + -0.05762401595711708, + -1.6284780502319336, + 0.0, + 0.002695085946470499, + 0.34764212369918823, + 0.6051180362701416, + 0.9295560717582703, + 0.38285934925079346, + -1.2762084007263184, + 0.0 + ], + [ + -0.04624907672405243, + 1.3854118585586548, + 0.05864633619785309, + -1.1657018661499023, + 0.7997346520423889, + -1.43052077293396, + 1.1980153322219849, + -0.023269198834896088, + -0.026228107511997223, + -0.009256521239876747, + 0.6858966946601868, + 0.03280947729945183, + -2.6284780502319336, + -1.7920862436294556, + -0.0004090145230293274, + -0.00536443293094635, + 0.007614931557327509, + 0.7401981949806213, + 0.11622340977191925, + -2.2762084007263184, + -1.6813035011291504 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.3960306644439697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.584172487258911, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.362607002258301 + ], + [ + -0.13329119980335236, + 1.1171538829803467, + -0.19378040730953217, + -0.9341158866882324, + 0.8714147806167603, + -1.659934163093567, + 1.1658833026885986, + 0.23766514658927917, + 0.21810689568519592, + -0.05730185657739639, + 0.4946695566177368, + 0.17369553446769714, + -2.559515953063965, + -1.5605767965316772, + 0.08475297689437866, + 0.22485606372356415, + -0.2591644823551178, + 0.9637633562088013, + -0.08890970051288605, + -2.1015453338623047, + -1.5709789991378784 + ], + [ + -0.3507236838340759, + 1.1737966537475586, + 0.10341192036867142, + -0.7761039137840271, + 0.47493064403533936, + -1.2297213077545166, + 1.4181653261184692, + -0.1981543004512787, + -0.18248644471168518, + 0.17768341302871704, + 0.843591034412384, + -0.07443197071552277, + -2.6062240600585938, + -1.750958800315857, + -0.15503963828086853, + 0.18218432366847992, + 0.002181168645620346, + 0.8641887903213501, + -0.026101917028427124, + -1.9398812055587769, + -1.8561092615127563 + ], + [ + -0.22363319993019104, + 1.533349871635437, + 0.4454386830329895, + -1.5271000862121582, + 0.8945828080177307, + -1.105146884918213, + 0.7171626687049866, + -0.20207461714744568, + 0.31458696722984314, + 0.20214562118053436, + 1.0593571662902832, + -0.4655032455921173, + -2.8809642791748047, + -1.580764651298523, + -0.22098299860954285, + -0.08731096982955933, + -0.15165574848651886, + 0.89030921459198, + 0.611029326915741, + -2.2792654037475586, + -1.798964262008667 + ], + [ + -0.3702698349952698, + 0.9771865606307983, + 0.27564477920532227, + -1.7643829584121704, + 0.7298657894134521, + -1.875503420829773, + 1.4598875045776367, + 0.47600361704826355, + 0.13439032435417175, + -0.23905222117900848, + 1.2192013263702393, + -0.44705769419670105, + -2.603621482849121, + -1.3898506164550781, + 0.16782721877098083, + -0.2559890151023865, + 0.2965841293334961, + 0.8750308752059937, + 0.1521819680929184, + -2.3417282104492188, + -2.2663588523864746 + ], + [ + -0.31705379486083984, + 0.866703450679779, + -0.09125347435474396, + -1.5959886312484741, + 0.8041162490844727, + -1.7037310600280762, + 1.6304850578308105, + 0.11580469459295273, + 0.2654762864112854, + 0.12200001627206802, + 0.6904033422470093, + 0.19859254360198975, + -2.7615463733673096, + -2.059410333633423, + 0.5591834783554077, + -0.4801292419433594, + -0.3343847393989563, + 0.8787070512771606, + -0.14748375117778778, + -2.0334713459014893, + -1.1611082553863525 + ], + [ + -0.11324095726013184, + 0.8777624368667603, + 0.8215147256851196, + -1.5343353748321533, + 0.881126344203949, + -1.0607860088348389, + 0.5398488640785217, + 0.6020828485488892, + -0.5065795183181763, + 0.02798180840909481, + 0.33880186080932617, + 0.320717990398407, + -3.3552045822143555, + -1.1435439586639404, + 0.19279444217681885, + -0.5245973467826843, + 0.30647164583206177, + 1.3726601600646973, + 0.3123217523097992, + -2.9675674438476562, + -1.7007204294204712 + ], + [ + -0.6432619094848633, + 0.79607754945755, + -0.49864423274993896, + -1.0710605382919312, + 1.0709084272384644, + -1.612175703048706, + 1.2555015087127686, + 0.2873077392578125, + -0.36858057975769043, + 0.05837755650281906, + 1.1025807857513428, + 0.7949575185775757, + -3.2468795776367188, + -2.556741237640381, + -0.6579093337059021, + -0.8874800801277161, + 0.7922345399856567, + 0.4745541512966156, + 0.07610362768173218, + -2.041800022125244, + -2.2069954872131348 + ], + [ + -0.9264093637466431, + 1.9828436374664307, + 0.7907096147537231, + -1.260971188545227, + 1.7548933029174805, + -2.2685132026672363, + 1.1821955442428589, + -0.49968039989471436, + 0.10982713848352432, + 0.2384604662656784, + 0.2075445055961609, + -0.7872775793075562, + -2.076058864593506, + -1.1888585090637207, + 0.7447717785835266, + 0.2355569452047348, + -0.3163020610809326, + 0.3282558023929596, + -0.4075327515602112, + -3.2451019287109375, + -2.4086263179779053 + ], + [ + -0.7190575003623962, + 1.3702023029327393, + 0.8307373523712158, + -0.7533494234085083, + 0.574251651763916, + -2.3376822471618652, + 0.5512753129005432, + -0.04391670227050781, + 0.2242431342601776, + -0.46594029664993286, + 0.5667901039123535, + -0.12832416594028473, + -2.9870448112487793, + -1.2032369375228882, + 0.17313259840011597, + 0.5932936668395996, + 0.7428228259086609, + 0.6570835113525391, + -0.1891915649175644, + -1.3703107833862305, + -1.7630409002304077 + ], + [ + 0.799586832523346, + 1.0164330005645752, + -0.49519193172454834, + -0.6983597278594971, + 1.7632484436035156, + -0.9710203409194946, + 1.5416830778121948, + 0.9653489589691162, + 0.06498706340789795, + 0.2132291942834854, + -0.1650814414024353, + 0.06786654889583588, + -1.8782389163970947, + -2.3576126098632812, + 0.9410911798477173, + -0.576595664024353, + -0.8031080365180969, + 0.7120062112808228, + 0.6479252576828003, + -2.3928208351135254, + -1.4952012300491333 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.11725310236215591, + 0.3854118585586548, + 0.06966311484575272, + -0.16570186614990234, + 0.06442362070083618, + -0.43052077293395996, + 0.0, + 0.11858775466680527, + 0.0573427751660347, + -0.14938217401504517, + -0.01657569408416748, + 0.060478124767541885, + -1.6284780502319336, + 0.0, + 0.016584808006882668, + 0.18205130100250244, + -0.13064442574977875, + 0.15621143579483032, + 0.056775350123643875, + -1.2762084007263184, + 0.0 + ] + ], + "predicted_scores": [ + 2.7105655670166016, + 1.4601608514785767, + 1.3950504064559937, + 1.3776217699050903, + 1.21635103225708, + 0.9927445650100708, + 0.2479965090751648, + -0.32676640152931213 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "256770874/train1", + "selected_index": 0, + "state_hash": "3a7aa4ba27a5b34cdeb0f31ce9818588b1dd4ede96bf546f07302616c2e9b3a6", + "stored_base_utility": 0.8314920663833618, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20372186601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20372186601161957 + }, + "base_progress": 0.20372186601161957, + "base_success": false, + "base_utility": 0.20372186601161957, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.23941433429718018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23941433429718018 + }, + { + "contact_quality": null, + "progress": 0.2004055678844452, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2004055678844452 + }, + { + "contact_quality": null, + "progress": 0.21979865431785583, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21979865431785583 + }, + { + "contact_quality": null, + "progress": 0.5798611640930176, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5798611640930176 + }, + { + "contact_quality": null, + "progress": 0.20053516328334808, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20053516328334808 + }, + { + "contact_quality": null, + "progress": 0.5538582801818848, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5538582801818848 + }, + { + "contact_quality": null, + "progress": 0.5764124393463135, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5764124393463135 + }, + { + "contact_quality": null, + "progress": 0.2330804020166397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2330804020166397 + } + ], + "candidate_progress": [ + 0.23941433429718018, + 0.2004055678844452, + 0.21979865431785583, + 0.5798611640930176, + 0.20053516328334808, + 0.5538582801818848, + 0.5764124393463135, + 0.2330804020166397 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-ecb37ebbab21a506d8ca", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "generated_tangents": [ + [ + -0.038756996393203735, + -0.13550937175750732, + -0.2540381848812103, + 0.33375242352485657, + 0.4462161660194397, + -0.5883610248565674, + 0.053501129150390625, + 0.07820597290992737, + 0.13572408258914948, + -0.07118258625268936, + -0.10641970485448837, + 0.1457289159297943, + -1.7172857522964478, + -0.010731622576713562, + -0.06439921259880066, + 0.042354196310043335, + 0.03466056287288666, + -0.004692539572715759, + -0.011387646198272705, + -1.3083875179290771, + -0.1264006495475769 + ], + [ + -0.1580553501844406, + -0.13328713178634644, + -0.19537189602851868, + 0.41709479689598083, + 0.3933730721473694, + -0.3096438944339752, + 0.07337459921836853, + 0.09225863218307495, + 0.08545009046792984, + -0.016713984310626984, + -0.21659067273139954, + 0.23247233033180237, + -1.602565884590149, + 0.052193187177181244, + 0.00322757288813591, + 0.06677122414112091, + 0.056176647543907166, + -0.03963209688663483, + -0.003780156373977661, + -1.3334197998046875, + -0.15710443258285522 + ], + [ + -0.14634673297405243, + -0.1670856475830078, + -0.1505264937877655, + 0.22932136058807373, + 0.46648484468460083, + -0.29467034339904785, + -0.030024707317352295, + 0.024578526616096497, + 0.10371974855661392, + -0.07803692668676376, + -0.17516982555389404, + 0.09712421149015427, + -1.818817377090454, + -0.0137137770652771, + 0.06714317202568054, + 0.05008891597390175, + 0.014884591102600098, + -0.0014598816633224487, + -0.03815457224845886, + -1.4960157871246338, + -0.1800711750984192 + ], + [ + -0.13525371253490448, + -0.21858741343021393, + -0.27348700165748596, + 0.4115125834941864, + 0.47482720017433167, + -0.3542366921901703, + 0.4323655068874359, + 0.07173498719930649, + 0.10754535347223282, + -0.1506441831588745, + -0.38532698154449463, + -0.014619454741477966, + -1.9378687143325806, + 0.0027394890785217285, + -0.14780347049236298, + 0.008556980639696121, + -0.02571706473827362, + -0.15478719770908356, + 0.15509457886219025, + -1.6268761157989502, + -0.2344937026500702 + ], + [ + -0.07550613582134247, + -0.14450731873512268, + -0.1280311644077301, + 0.2581559419631958, + 0.4942222833633423, + -0.2489149570465088, + -0.0064125657081604, + 0.0913766473531723, + 0.10777343064546585, + -0.060408152639865875, + -0.05955008417367935, + -0.009624660015106201, + -1.820928931236267, + -0.0243748277425766, + 0.004844978451728821, + 0.022344592958688736, + 0.12104836106300354, + -0.03878173232078552, + -0.03758198022842407, + -1.5328521728515625, + -0.18792152404785156 + ], + [ + 0.021220877766609192, + -0.24586030840873718, + -0.46019771695137024, + 0.3124466836452484, + 0.41348084807395935, + -0.2183077037334442, + 0.5402253270149231, + 0.1507055014371872, + 0.054333195090293884, + -0.17292925715446472, + -0.3890281021595001, + -0.02466900646686554, + -2.089529037475586, + -0.1611459106206894, + -0.1178891658782959, + 0.03982258588075638, + -0.05764146149158478, + -0.16406626999378204, + 0.03445519506931305, + -1.498215913772583, + -0.35786673426628113 + ], + [ + -0.17654405534267426, + -0.2302348017692566, + -0.4488113820552826, + 0.20745614171028137, + 0.5087523460388184, + -0.03024144470691681, + 0.3668936491012573, + -0.029347702860832214, + 0.14453846216201782, + -0.1354864239692688, + -0.3882730007171631, + -0.07349419593811035, + -2.1526310443878174, + -0.08613543212413788, + 0.04476073384284973, + 0.06055107340216637, + -0.021915599703788757, + -0.04606534540653229, + -0.059882551431655884, + -1.699329137802124, + -0.21382218599319458 + ], + [ + -0.308411180973053, + -0.23533658683300018, + -0.2005605399608612, + 0.31021276116371155, + 0.46984899044036865, + -0.2295137345790863, + 0.40720218420028687, + -0.0017099827527999878, + 0.1529761403799057, + -0.09738033264875412, + -0.293779194355011, + -0.05512559413909912, + -2.0737287998199463, + 0.09915192425251007, + -0.1006896048784256, + -0.04626504331827164, + 0.010933130979537964, + -0.12180183827877045, + -0.08540168404579163, + -1.6375348567962646, + -0.426442950963974 + ] + ], + "generated_utilities": [ + 0.23941433429718018, + 0.2004055678844452, + 0.21979865431785583, + 0.5798611640930176, + 0.20053516328334808, + 0.5538582801818848, + 0.5764124393463135, + 0.2330804020166397 + ], + "group_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8363267779350281, + 2.0, + 2.0, + 0.16206762194633484, + 0.5390529036521912, + 0.08151453733444214, + 0.5261932611465454, + 0.5265869498252869, + 0.5459305047988892, + 0.18360589444637299, + 0.16689825057983398, + 0.15426510572433472, + 0.0028923004865646362, + 0.15516109764575958, + 0.0506085604429245, + 0.19578850269317627 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.0005862116813659668, + 0.18783891201019287, + -0.02734696865081787, + 0.09201061725616455, + 0.010171383619308472, + -0.4856022596359253, + 0.0, + 0.10349468141794205, + 0.4567214846611023, + -0.1893925815820694, + 0.12298360466957092, + 0.3369159996509552, + -1.6010875701904297, + 0.0, + 0.304210901260376, + -0.3049007058143616, + 0.3435661494731903, + 0.3158445656299591, + 0.2931552231311798, + -1.2709691524505615, + 0.0 + ], + [ + -0.726529061794281, + -0.8121610879898071, + -1.0273469686508179, + 1.0920106172561646, + 0.3926190733909607, + -1.4856022596359253, + 1.8979665040969849, + -0.029249273240566254, + 0.008005961775779724, + 0.013417234644293785, + -0.10860864818096161, + 0.43491366505622864, + -2.6010875701904297, + -1.493168592453003, + -0.01047949492931366, + -0.0018908085767179728, + 0.019280990585684776, + 0.20726272463798523, + 0.45750224590301514, + -2.2709691524505615, + -1.4645930528640747 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7959330081939697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.986337184906006, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9291861057281494 + ], + [ + -0.9019005298614502, + -0.5901641845703125, + -0.920559287071228, + 0.991828978061676, + 0.2407551407814026, + -1.2655587196350098, + 1.7652356624603271, + 0.03909382224082947, + 0.04142814502120018, + 0.12125641852617264, + -0.2822783887386322, + 0.4936577081680298, + -2.5754144191741943, + -1.5686157941818237, + -0.22672900557518005, + -0.08431098610162735, + 0.09805622696876526, + 0.2479260265827179, + 0.4450404644012451, + -2.1972453594207764, + -1.4420135021209717 + ], + [ + -0.8822115063667297, + -0.9738405346870422, + -0.7449414730072021, + 0.8784142732620239, + 0.4583987295627594, + -1.3989503383636475, + 1.8331178426742554, + -0.006558071821928024, + 0.30383312702178955, + 0.2117801308631897, + -0.2538969814777374, + 0.13478082418441772, + -2.584805727005005, + -1.325882911682129, + 0.23593956232070923, + -0.2618964612483978, + -0.30780693888664246, + 0.24083535373210907, + 0.430472195148468, + -1.946203589439392, + -1.5077860355377197 + ], + [ + -0.39491432905197144, + -0.8895903825759888, + -0.9160035848617554, + 0.7220491170883179, + 0.36837494373321533, + -1.4429160356521606, + 1.4078493118286133, + 0.03969770669937134, + 0.007204015273600817, + 0.056278251111507416, + -0.4780465364456177, + 0.9121701717376709, + -2.512721538543701, + -1.6225675344467163, + -0.42074650526046753, + 0.15076839923858643, + -0.24544812738895416, + 0.08152849972248077, + 0.1987975835800171, + -2.712414264678955, + -1.9573136568069458 + ], + [ + -0.6879937648773193, + -0.8859440088272095, + -1.1152405738830566, + 0.8750674724578857, + -0.05955067276954651, + -1.5194538831710815, + 1.876329779624939, + -0.2068542242050171, + -0.5498799085617065, + 0.46419233083724976, + 0.42569708824157715, + -0.02127784490585327, + -2.5905418395996094, + -1.411050796508789, + -0.21327723562717438, + 0.20716798305511475, + 0.20993508398532867, + 0.33476364612579346, + -0.09872657060623169, + -2.5958731174468994, + -1.6374839544296265 + ], + [ + -0.5723236799240112, + -0.28514552116394043, + -0.6157487630844116, + 0.6151646375656128, + 0.32639649510383606, + -1.7217803001403809, + 2.5557403564453125, + 0.532052218914032, + 0.5977330207824707, + -0.5707970261573792, + -0.21707558631896973, + 0.8829224109649658, + -3.1287553310394287, + -1.0988328456878662, + 0.38229066133499146, + 0.0032835237216204405, + -0.3569248616695404, + 0.0031215548515319824, + 1.1060712337493896, + -2.3536932468414307, + -1.383341670036316 + ], + [ + -0.7901747822761536, + -0.8977189660072327, + -1.4548863172531128, + 0.5700533986091614, + -0.13018959760665894, + -0.8057670593261719, + 2.4019269943237305, + 0.5368256568908691, + 0.5227565765380859, + 0.28199371695518494, + -0.1637011021375656, + -0.2898564040660858, + -2.9142980575561523, + -1.8492696285247803, + 0.7177971601486206, + 0.2360217124223709, + -0.13862104713916779, + 0.9262527227401733, + 0.39495596289634705, + -2.470649480819702, + -1.3260751962661743 + ], + [ + -0.6053685545921326, + -0.3819047510623932, + -0.258634090423584, + 0.19338607788085938, + 0.19874510169029236, + -1.3407407999038696, + 2.5238022804260254, + -0.6922282576560974, + 0.8137098550796509, + 0.39421388506889343, + 0.09789760410785675, + 0.12572914361953735, + -2.518153667449951, + -1.6892600059509277, + 0.503250002861023, + 0.88390052318573, + -0.7520112991333008, + -0.3356115520000458, + 0.15291047096252441, + -2.5788943767547607, + -0.9403089284896851 + ], + [ + -0.464814156293869, + -0.6034223437309265, + -1.6505372524261475, + 0.6883972883224487, + -0.3758969306945801, + -1.785051941871643, + 2.605452060699463, + 0.550828218460083, + 0.4742209315299988, + -0.19333000481128693, + -1.002321481704712, + 0.48782211542129517, + -1.8015108108520508, + -0.8254865407943726, + -0.7333722710609436, + -0.4948355555534363, + 0.6016485095024109, + 0.5632485151290894, + -0.24367755651474, + -2.201673984527588, + -0.5806979537010193 + ], + [ + -1.4628195762634277, + -0.4433446228504181, + -1.9926375150680542, + 0.33977705240249634, + 0.03379333019256592, + -1.6042804718017578, + 2.221484422683716, + 0.7110810279846191, + -0.41500210762023926, + 0.08095695078372955, + -0.23683495819568634, + -0.43207624554634094, + -3.080759286880493, + -0.9071659445762634, + 0.47082334756851196, + -0.9117310047149658, + -0.06435621529817581, + -0.6813669204711914, + -0.2741369605064392, + -2.3763599395751953, + -1.000203013420105 + ], + [ + -1.382751226425171, + -1.7026821374893188, + -0.11846715211868286, + 1.5398613214492798, + -0.5885447859764099, + -1.8825289011001587, + 1.5456308126449585, + -0.320659339427948, + -0.964028000831604, + -0.1812019646167755, + 0.14226128160953522, + 0.4527861177921295, + -2.714468002319336, + -1.9451607465744019, + 0.195057213306427, + 0.16905252635478973, + -0.9247990250587463, + 0.9112424850463867, + 0.79449862241745, + -2.714893341064453, + -1.6473028659820557 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.02016383409500122, + 0.02509528398513794, + -0.02734696865081787, + 0.09201061725616455, + -0.04331648349761963, + -0.4856022596359253, + 0.0, + -0.022040890529751778, + -0.003832761198282242, + -0.01408458687365055, + 0.06165061891078949, + 0.02554956078529358, + -1.6010875701904297, + 0.0, + 0.009550465270876884, + 0.03566932678222656, + 0.013732164166867733, + 0.017749279737472534, + 0.0699148178100586, + -1.2709691524505615, + 0.0 + ], + [ + 0.023195505142211914, + 0.029890060424804688, + -0.02734696865081787, + 0.09201061725616455, + 0.06108534336090088, + -0.4856022596359253, + 0.0, + -0.034551952034235, + -0.15152660012245178, + -0.21811580657958984, + -0.063543401658535, + 0.23667243123054504, + -1.6010875701904297, + 0.0, + -0.1614403873682022, + -0.17222382128238678, + -0.09706380218267441, + 0.3660350441932678, + 0.13013160228729248, + -1.2709691524505615, + 0.0 + ] + ], + "predicted_scores": [ + 5.028995513916016, + 4.867879390716553, + 4.800228118896484, + 4.730734348297119, + 4.398209571838379, + 4.280922889709473, + 3.719674587249756, + 3.216965436935425 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "957101489/train1", + "selected_index": 0, + "state_hash": "e8473553a2b17f5adf09e08e7271b5502d53a51225c17abac3d9f78906cf8c46", + "stored_base_utility": 0.8363267779350281, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8431589603424072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8431589603424072 + }, + "base_progress": 0.8431589603424072, + "base_success": false, + "base_utility": 0.8431589603424072, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2240862250328064, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2240862250328064 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.23456276953220367, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23456276953220367 + }, + { + "contact_quality": null, + "progress": 0.22421211004257202, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22421211004257202 + }, + { + "contact_quality": null, + "progress": 0.23834753036499023, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23834753036499023 + }, + { + "contact_quality": null, + "progress": 0.23887677490711212, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23887677490711212 + }, + { + "contact_quality": null, + "progress": 0.2396199107170105, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2396199107170105 + }, + { + "contact_quality": null, + "progress": 0.21505582332611084, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21505582332611084 + } + ], + "candidate_progress": [ + 0.2240862250328064, + 1.0, + 0.23456276953220367, + 0.22421211004257202, + 0.23834753036499023, + 0.23887677490711212, + 0.2396199107170105, + 0.21505582332611084 + ], + "candidate_source_chart_ids": [ + "ms-5bddb39f792f20e5e6d4", + "ms-9228f8295b9891cf011c", + "ms-5bddb39f792f20e5e6d4", + "ms-9d12ad4ebdf37419994c", + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-ce9505086ea115f86649", + "ms-9d12ad4ebdf37419994c" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9a14552b7db695dee92c", + "generated_tangents": [ + [ + -0.24398012459278107, + -0.31837475299835205, + -0.39673200249671936, + 0.8768613338470459, + 0.41880494356155396, + -0.886460542678833, + 0.2791355848312378, + 0.07774002850055695, + -0.05111357569694519, + -0.166623055934906, + -0.20629540085792542, + -0.10491454601287842, + -2.0003256797790527, + -0.25472474098205566, + -0.12438620626926422, + -0.10710087418556213, + -0.08980457484722137, + 0.03481768071651459, + -0.139867901802063, + -1.9910016059875488, + -0.5777000188827515 + ], + [ + -0.011119157075881958, + -0.24807676672935486, + -0.0015007555484771729, + 0.6639519929885864, + 0.3065972924232483, + -1.6503907442092896, + 0.07185950875282288, + -0.008339866995811462, + 0.026413122192025185, + -0.061842434108257294, + -0.04869193583726883, + 0.01619863510131836, + -1.5339487791061401, + -0.15631748735904694, + -0.08074803650379181, + -0.008532043546438217, + 0.04662767052650452, + 0.07369226217269897, + -0.2059488296508789, + -1.5359117984771729, + -0.08076417446136475 + ], + [ + -0.061953917145729065, + -0.32794779539108276, + -0.5045595169067383, + 0.786717414855957, + 0.33011600375175476, + -0.819882869720459, + 0.48401767015457153, + 0.1185966208577156, + -0.08473946899175644, + -0.11563808470964432, + -0.2017354667186737, + -0.15378880500793457, + -2.1040890216827393, + -0.30424827337265015, + -0.11217138171195984, + -0.05279594287276268, + -0.10907794535160065, + 0.04559807479381561, + -0.23217111825942993, + -1.800889492034912, + -0.6683998107910156 + ], + [ + -0.036740511655807495, + -0.2286078780889511, + -0.1605977714061737, + 0.5439430475234985, + 0.2564154267311096, + -1.5536032915115356, + 0.047141581773757935, + 0.03982469439506531, + 0.010757035575807095, + -0.0548592135310173, + -0.14562514424324036, + 0.12549127638339996, + -1.5212243795394897, + -0.11536450684070587, + -0.12896764278411865, + -0.032481104135513306, + 0.01559726893901825, + 0.10565237700939178, + -0.29879236221313477, + -1.5924403667449951, + 0.10036706179380417 + ], + [ + -0.037515848875045776, + -0.178071066737175, + -0.2516103684902191, + 0.6320818662643433, + 0.348877489566803, + -1.0222082138061523, + -0.011195868253707886, + 0.03477014601230621, + 0.04416806995868683, + -0.07414370030164719, + -0.0005400404334068298, + -0.012319117784500122, + -1.7142136096954346, + -0.11643184721469879, + -0.1803487241268158, + -0.03259944170713425, + -0.005131468176841736, + 0.12992960214614868, + -0.1465526521205902, + -1.624044418334961, + -0.26597124338150024 + ], + [ + -0.1794966608285904, + -0.1724695861339569, + -0.2557346522808075, + 0.7137835025787354, + 0.21937555074691772, + -0.8046422004699707, + -0.026013940572738647, + 0.05393823981285095, + 0.02173120155930519, + -0.011962853372097015, + -0.07620421797037125, + 0.0318894237279892, + -1.587060570716858, + -0.11730077862739563, + -0.10359573364257812, + -0.03643292933702469, + -0.018543973565101624, + 0.11555609107017517, + -0.16775581240653992, + -1.6180734634399414, + -0.24911972880363464 + ], + [ + -0.1586340218782425, + -0.16423314809799194, + -0.2150205671787262, + 0.77131187915802, + 0.4593883752822876, + -1.0750209093093872, + 0.08453720808029175, + 0.035815492272377014, + 0.035750649869441986, + -0.10813844949007034, + -0.026688888669013977, + -0.0509973019361496, + -2.2673110961914062, + -0.061440303921699524, + -0.0829656720161438, + -0.07193587720394135, + -0.03457425534725189, + 0.07937327027320862, + -0.21057280898094177, + -1.794555902481079, + -0.21171388030052185 + ], + [ + 0.01266525685787201, + -0.2515493631362915, + -0.032141655683517456, + 0.6023002862930298, + 0.32514604926109314, + -1.6528481245040894, + 0.06609141826629639, + -0.0161876380443573, + 0.010724145919084549, + -0.06089041382074356, + 0.013896647840738297, + 0.0018012821674346924, + -1.564214825630188, + -0.09429937601089478, + -0.04476979002356529, + -0.05024196580052376, + 0.016884565353393555, + 0.0802685022354126, + -0.22384709119796753, + -1.529163122177124, + -0.02373480796813965 + ] + ], + "generated_utilities": [ + 0.2240862250328064, + 2.0, + 0.23456276953220367, + 0.22421211004257202, + 0.23834753036499023, + 0.23887677490711212, + 0.2396199107170105, + 0.21505582332611084 + ], + "group_id": "ms-9a14552b7db695dee92c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.838127613067627, + 2.0, + 0.15671375393867493, + 0.17482513189315796, + 0.5684618353843689, + 0.19147612154483795, + 0.5643423795700073, + 0.1653006672859192, + 0.11670731008052826, + 0.14578887820243835, + 0.16058449447155, + 0.10923929512500763, + 0.13408902287483215, + 0.06845971941947937, + 0.02954423427581787, + 0.08425161242485046 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.22581616044044495, + -0.32600927352905273, + -0.054001130163669586, + 1.2553811073303223, + -0.1383839249610901, + -1.9753923416137695, + 0.0, + 0.09536236524581909, + 0.017635054886341095, + -0.06946105509996414, + 0.06630897521972656, + -0.10150641947984695, + -1.9112331867218018, + 0.0, + -0.23092225193977356, + 0.04748937487602234, + 0.3013715445995331, + -0.22072738409042358, + 0.0733548104763031, + -1.6514825820922852, + 0.0 + ], + [ + 0.22913068532943726, + -0.32600927352905273, + 0.24692901968955994, + 1.2553811073303223, + 0.3704858124256134, + -1.9753923416137695, + 0.0, + 0.036500319838523865, + 0.12820330262184143, + -0.09674224257469177, + 0.2668613791465759, + -0.14372897148132324, + -1.9112331867218018, + 0.0, + -0.27894535660743713, + -0.3322452902793884, + 0.16300754249095917, + -0.24123603105545044, + -0.2359180599451065, + -1.6514825820922852, + 0.0 + ], + [ + 0.0198141410946846, + -1.3260092735290527, + 0.011403528973460197, + 2.2553811073303223, + 0.5258303284645081, + -2.9753923416137695, + 1.8956809043884277, + -0.028990529477596283, + 0.007858656346797943, + 0.03128144145011902, + 0.5457771420478821, + -0.14793801307678223, + -2.9112331867218018, + -1.5743238925933838, + -0.011861175298690796, + -0.002163209253922105, + 0.015663353726267815, + 0.7587639689445496, + -0.07801014184951782, + -2.651482582092285, + -1.5696079730987549 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7913618087768555, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1486477851867676, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1392159461975098 + ], + [ + 0.13539990782737732, + -1.2862521409988403, + 0.28441768884658813, + 2.3658432960510254, + 0.4678441286087036, + -2.8791494369506836, + 1.8408029079437256, + -0.11893181502819061, + -0.1259741485118866, + -0.20943574607372284, + 0.4765416979789734, + -0.40862777829170227, + -2.8372273445129395, + -1.4697245359420776, + -0.08606886118650436, + -0.06554404646158218, + -0.27424973249435425, + 0.907739520072937, + -0.10130079835653305, + -2.6951611042022705, + -1.807652235031128 + ], + [ + -0.15379676222801208, + -1.7091000080108643, + 0.31153741478919983, + 2.2179949283599854, + 0.859725832939148, + -2.6038026809692383, + 1.98358952999115, + 0.22934654355049133, + -0.16202330589294434, + 0.02916553243994713, + 0.7702362537384033, + -0.0907563641667366, + -2.6495683193206787, + -1.394291877746582, + 0.2534489035606384, + -0.07077484577894211, + 0.10660970956087112, + 0.7866172194480896, + -0.41323983669281006, + -2.4847500324249268, + -1.5881152153015137 + ], + [ + 0.14051103591918945, + -0.8684865236282349, + 0.45214012265205383, + 2.499347686767578, + 0.310441792011261, + -2.875823497772217, + 1.4898443222045898, + 0.3396439850330353, + -0.033532168716192245, + 0.20189781486988068, + 0.3506202697753906, + 0.04831346869468689, + -2.4730942249298096, + -1.4369138479232788, + -0.4840375483036041, + -0.17814871668815613, + -0.1863224357366562, + 0.47034892439842224, + -0.5315045714378357, + -3.019744396209717, + -1.2571722269058228 + ], + [ + 0.5503659844398499, + -1.517655372619629, + -0.20272362232208252, + 2.5588791370391846, + 0.7160627841949463, + -2.551017999649048, + 2.423107385635376, + 0.18953359127044678, + -0.4268995523452759, + 0.014377228915691376, + 0.969605028629303, + -0.2440275251865387, + -2.385143756866455, + -1.4786417484283447, + 0.44389280676841736, + -0.11543146520853043, + -0.5315021872520447, + 1.2627880573272705, + 0.24797001481056213, + -3.17976713180542, + -1.8759949207305908 + ], + [ + -0.07551594823598862, + -1.6101253032684326, + 0.3926098048686981, + 1.726672649383545, + 1.0158472061157227, + -3.367068290710449, + 1.4499058723449707, + -0.5616514086723328, + 0.002434568479657173, + -0.5033466815948486, + -0.12249338626861572, + -0.07335222512483597, + -3.5959439277648926, + -1.0662813186645508, + -0.034436360001564026, + 0.512275218963623, + -0.04822078347206116, + 0.6117934584617615, + -0.745093822479248, + -2.2700355052948, + -2.090803623199463 + ], + [ + 0.6751457452774048, + -0.7857800126075745, + 0.11759950965642929, + 2.7748122215270996, + 1.3096001148223877, + -2.3352341651916504, + 1.4677345752716064, + 0.7219796180725098, + 0.21788907051086426, + -0.3245394825935364, + 0.08372032642364502, + -0.6393120288848877, + -2.8002729415893555, + -2.157836437225342, + 0.11339983344078064, + -0.2834126651287079, + -0.3824191987514496, + 1.130112648010254, + -0.8028184175491333, + -2.3846678733825684, + -1.8259286880493164 + ], + [ + -0.01608770713210106, + -2.1245148181915283, + -0.175740584731102, + 2.754368782043457, + 1.0032098293304443, + -3.7700157165527344, + 1.3539870977401733, + -0.7906507849693298, + -0.8311541080474854, + 0.8180835247039795, + 0.10450628399848938, + 0.01135176420211792, + -3.396463394165039, + -0.6868918538093567, + 0.22346650063991547, + 0.693323016166687, + -0.47104692459106445, + 0.8188157081604004, + -0.6246469020843506, + -2.4006571769714355, + -2.431541919708252 + ], + [ + -0.7931793928146362, + -1.7645766735076904, + 0.23125068843364716, + 2.816321849822998, + 1.1080517768859863, + -3.8820090293884277, + 1.8420056104660034, + 0.4984684884548187, + -0.9149360656738281, + -0.028868291527032852, + 0.14621934294700623, + 0.546608567237854, + -2.7883894443511963, + -1.3650472164154053, + -0.055590901523828506, + 0.41940492391586304, + -0.7235082983970642, + 1.1734092235565186, + -0.6126058101654053, + -3.217271566390991, + -2.512387275695801 + ], + [ + -0.5335642695426941, + -2.139784336090088, + 0.9540140628814697, + 1.7781559228897095, + 1.2117886543273926, + -3.71681809425354, + 2.461766242980957, + 0.4546531140804291, + 0.09786976873874664, + -0.8281612396240234, + -0.19971781969070435, + -1.1042157411575317, + -3.047807216644287, + -1.0697336196899414, + -0.22256404161453247, + -0.3800235986709595, + 0.3603542447090149, + 0.7543611526489258, + -0.23184219002723694, + -3.5156431198120117, + -2.1170644760131836 + ], + [ + 0.27338743209838867, + -0.9654052257537842, + -0.2700650095939636, + 2.6957814693450928, + 0.0643424391746521, + -3.094449281692505, + 0.8981494307518005, + 0.19603312015533447, + 0.5969407558441162, + -0.45200687646865845, + 1.466981291770935, + -0.45456382632255554, + -2.997983455657959, + -1.1903986930847168, + 0.5722758769989014, + -0.9932370781898499, + 0.28074830770492554, + 0.6244409084320068, + 0.6411240696907043, + -3.2179243564605713, + -1.905179738998413 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.025789255276322365, + -0.32600927352905273, + -0.02888326905667782, + 1.2553811073303223, + 0.013092517852783203, + -1.9753923416137695, + 0.0, + 0.04008308798074722, + -0.05201057717204094, + -0.008613351732492447, + -0.06225985288619995, + -0.0397564172744751, + -1.9112331867218018, + 0.0, + -0.02836347185075283, + -0.0055325529538095, + 0.053034864366054535, + 0.009740114212036133, + -0.006228044629096985, + -1.6514825820922852, + 0.0 + ] + ], + "predicted_scores": [ + 5.9339399337768555, + 5.8429741859436035, + 5.83306884765625, + 5.777254581451416, + 5.714264392852783, + 5.678617000579834, + 5.648623943328857, + 5.636460304260254 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264/train1", + "selected_index": 0, + "state_hash": "49d38eabb7404e0028c75dd98e1239274b5dc2f04b9db8653001e44369ca8f3a", + "stored_base_utility": 0.838127613067627, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8326728343963623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8326728343963623 + }, + "base_progress": 0.8326728343963623, + "base_success": false, + "base_utility": 0.8326728343963623, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8636302947998047, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8636302947998047 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.8636302947998047, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-c80e71e55d51d8595499", + "ms-5dfbb16e173e1a3e2bd6", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-19a3669e096e98232ec8", + "ms-c80e71e55d51d8595499", + "ms-19a3669e096e98232ec8" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-9f5d57fe2153fd267774", + "generated_tangents": [ + [ + 0.08413292467594147, + -0.08230706304311752, + -0.1254267394542694, + 0.2186259925365448, + -0.004199966788291931, + -1.5646209716796875, + 0.17910218238830566, + -0.059523820877075195, + -0.042556025087833405, + -0.08612646907567978, + 0.30891841650009155, + 0.00041578710079193115, + -1.151727557182312, + -0.004442453384399414, + -0.20556923747062683, + -0.05447538197040558, + 0.06253249943256378, + 0.21837827563285828, + -0.07886528968811035, + -0.8268441557884216, + -0.10379719734191895 + ], + [ + 0.005435451865196228, + -0.15224504470825195, + 0.2607381045818329, + 0.0070578306913375854, + -0.23332613706588745, + -1.6677610874176025, + 0.15069997310638428, + 0.02010825276374817, + -0.03953012824058533, + -0.14700442552566528, + 0.18694975972175598, + 0.08739671856164932, + -1.0308794975280762, + 0.17629662156105042, + 0.01297353208065033, + -0.11081932485103607, + -0.05898500978946686, + 0.36896753311157227, + 0.12984052300453186, + -0.7175573110580444, + -0.30426302552223206 + ], + [ + 0.06530517339706421, + -0.10118457674980164, + -0.09261497855186462, + 0.12752801179885864, + 0.029137805104255676, + -1.4337760210037231, + 0.16093605756759644, + 0.010899156332015991, + -0.05387471616268158, + -0.03502560406923294, + 0.2865021824836731, + -0.025219827890396118, + -1.016312599182129, + 0.0006787031888961792, + -0.11459082365036011, + -0.06721201539039612, + 0.22419260442256927, + 0.14956511557102203, + -0.07550835609436035, + -0.8552228212356567, + -0.09761044383049011 + ], + [ + -0.010899603366851807, + -0.09557810425758362, + -0.07494232058525085, + 0.11577621102333069, + -0.04265552759170532, + -1.4771164655685425, + 0.13611218333244324, + -0.030279889702796936, + -0.07786482572555542, + -0.1113763228058815, + 0.2983352243900299, + -0.02195592224597931, + -1.1468371152877808, + -0.09548032283782959, + -0.14314137399196625, + -0.05801401287317276, + 0.13833624124526978, + 0.19742122292518616, + -0.14581972360610962, + -0.8729411363601685, + -0.09893390536308289 + ], + [ + 0.04760611802339554, + -0.04970837011933327, + -0.025667399168014526, + 0.2508976459503174, + -0.04721599817276001, + -1.3973768949508667, + 0.19110479950904846, + -0.019008412957191467, + -0.013932688161730766, + -0.08488116413354874, + 0.2985996603965759, + 0.07291415333747864, + -0.9242931604385376, + 0.02658630907535553, + -0.12224647402763367, + -0.06941595673561096, + 0.1371287703514099, + 0.1649552285671234, + -0.05273208022117615, + -0.8101409077644348, + -0.06959079205989838 + ], + [ + 0.011269211769104004, + -0.09086939692497253, + -0.10519930720329285, + 0.167340487241745, + -0.06838038563728333, + -1.5224459171295166, + 0.16651907563209534, + -0.0974019467830658, + -0.057201795279979706, + -0.12136388570070267, + 0.19467008113861084, + -0.015901654958724976, + -1.0124092102050781, + -0.044310301542282104, + -0.10065591335296631, + -0.06727073341608047, + 0.19224049150943756, + 0.1459444910287857, + -0.2303667664527893, + -0.8499047160148621, + -0.06792102754116058 + ], + [ + 0.16562232375144958, + -0.10833266377449036, + 0.09600090980529785, + -0.1374075561761856, + 0.30824172496795654, + -1.4698164463043213, + -0.30176299810409546, + -0.06370925903320312, + 0.010365153662860394, + -0.0443614199757576, + 0.32736533880233765, + -0.3612250089645386, + -1.0171478986740112, + -0.5918875932693481, + -0.16154873371124268, + -0.059748485684394836, + 0.29379719495773315, + 0.39021843671798706, + -0.05616903305053711, + -1.054394006729126, + 0.27992331981658936 + ], + [ + 0.04429534822702408, + -0.108091801404953, + 0.08059525489807129, + 0.2930544316768646, + -0.06860846281051636, + -1.504648208618164, + 0.1273336410522461, + -0.021768197417259216, + 0.005642636679112911, + -0.08264502137899399, + 0.25489139556884766, + 0.10974524915218353, + -0.8973052501678467, + -0.04159153997898102, + -0.042293477803468704, + -0.12202993035316467, + 0.050917476415634155, + 0.17859452962875366, + -0.14225858449935913, + -0.8196880221366882, + -0.11021232604980469 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.8636302947998047, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-9f5d57fe2153fd267774", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8544998168945312, + 2.0, + 0.14427810907363892, + 0.1566883623600006, + 0.7847082018852234, + 0.6161996126174927, + 0.5900160074234009, + 0.12891459465026855, + 0.5811972618103027, + 0.12705880403518677, + 0.12002001702785492, + 0.07508957386016846, + 0.1335657238960266, + 2.0, + 0.5254277586936951, + 0.1602712869644165 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.07957560569047928, + 0.1778574287891388, + -0.06101015955209732, + 0.08770006895065308, + -0.13665570318698883, + -1.1242008209228516, + 0.0, + 0.20265339314937592, + 0.03117954358458519, + -0.08283126354217529, + 0.051615357398986816, + 0.006725620478391647, + -0.7304134368896484, + 0.0, + -0.03756266087293625, + 0.051428791135549545, + 0.13959813117980957, + 0.040047526359558105, + -0.04084143042564392, + -0.31500017642974854, + 0.0 + ], + [ + 0.31541678309440613, + -0.16606420278549194, + 0.09380461275577545, + 0.031209349632263184, + -0.10083025693893433, + -1.1242008209228516, + 0.0, + 0.09663133323192596, + -0.095388263463974, + 0.4012776017189026, + 0.051615357398986816, + 0.167547807097435, + -0.7304134368896484, + 0.0, + -0.6302167177200317, + 0.05524101108312607, + -0.14198890328407288, + 0.040047526359558105, + 0.5201720595359802, + -0.31500017642974854, + 0.0 + ], + [ + 0.004154659807682037, + -0.0034282731357961893, + 0.005892834160476923, + 1.0312093496322632, + -0.15828494727611542, + -2.1242008209228516, + -1.4059314727783203, + 0.002088826149702072, + 0.0016306412871927023, + 0.005124530289322138, + 1.0516153573989868, + -0.05936579033732414, + -1.7304134368896484, + -1.2081255912780762, + 0.010161396116018295, + 0.0017865521367639303, + -0.006222852971404791, + 1.040047526359558, + 0.07346899807453156, + -1.3150001764297485, + -0.9752891659736633 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.8118629455566406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.4162511825561523, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9505783319473267 + ], + [ + 0.05903958901762962, + -0.05713164061307907, + 0.27611011266708374, + 0.9773679971694946, + -0.2122824341058731, + -2.198127031326294, + -1.1234419345855713, + 0.22609303891658783, + 0.14432841539382935, + 0.13455358147621155, + 0.8475942611694336, + 0.19007176160812378, + -1.4536203145980835, + -1.208316445350647, + -0.24895474314689636, + 0.08610261976718903, + 0.16216596961021423, + 1.2356702089309692, + 0.07108174264431, + -1.4601069688796997, + -1.0005295276641846 + ], + [ + -0.2862337827682495, + -0.34814661741256714, + -0.3317800462245941, + 1.383265495300293, + -0.3485583961009979, + -1.7799506187438965, + -1.6264256238937378, + -0.24356596171855927, + -0.18337340652942657, + 0.3351142108440399, + 1.36115562915802, + -0.4404119849205017, + -1.4858057498931885, + -0.9229021668434143, + 0.3078569769859314, + -0.18743832409381866, + -0.3414495885372162, + 1.0045359134674072, + -0.1438894122838974, + -1.5133963823318481, + -0.7331042885780334 + ], + [ + 0.14600041508674622, + -0.10555070638656616, + 0.06090353801846504, + 1.1715545654296875, + 0.14829085767269135, + -1.8377182483673096, + -1.2019046545028687, + -0.33082208037376404, + 0.27619102597236633, + 0.3936208188533783, + 1.4077775478363037, + -0.2165551781654358, + -2.094696283340454, + -0.7843748331069946, + -0.13239514827728271, + 0.208020880818367, + 0.17500364780426025, + 0.5608562231063843, + 0.5641188025474548, + -1.6192113161087036, + -1.1969176530838013 + ], + [ + -0.03659747168421745, + -0.2400680035352707, + 0.49718841910362244, + 1.0422266721725464, + 0.13088186085224152, + -2.068427324295044, + -1.1215412616729736, + 0.06697356700897217, + 0.19986751675605774, + -0.5827334523200989, + 0.9792195558547974, + -0.5810520648956299, + -1.7176907062530518, + -0.6760571002960205, + -0.08971267938613892, + 0.36716267466545105, + 0.4829793870449066, + 0.7045803070068359, + -0.3341261148452759, + -1.3750282526016235, + -1.086517095565796 + ], + [ + 0.09393255412578583, + 0.6277099251747131, + 0.653377115726471, + 0.5602235198020935, + -0.8483365178108215, + -1.7264833450317383, + -1.698084831237793, + -0.3303340971469879, + 0.43070027232170105, + -0.13159427046775818, + 1.0274523496627808, + 0.4760425388813019, + -2.1360630989074707, + -0.6506720185279846, + 0.3130929470062256, + 0.6801184415817261, + 0.6575731039047241, + 0.9053056240081787, + 0.18805859982967377, + -1.3225764036178589, + -1.0653810501098633 + ], + [ + 0.536294162273407, + -0.026106586679816246, + 0.11593607813119888, + 0.2775042653083801, + 0.1801595836877823, + -1.590940237045288, + -0.6846873164176941, + 0.5080465078353882, + -0.7678324580192566, + 0.24115782976150513, + 0.2675415277481079, + -0.3879689872264862, + -2.1869571208953857, + -0.7176487445831299, + -0.15474224090576172, + -0.5326727032661438, + -0.12406099587678909, + 0.411626398563385, + 0.3519209027290344, + -1.6924495697021484, + -1.4648362398147583 + ], + [ + 0.2688097357749939, + -0.7928301095962524, + -0.39608004689216614, + 1.3961963653564453, + -0.026196151971817017, + -2.263073682785034, + -2.044499635696411, + 0.055598195642232895, + -0.485431969165802, + -0.058985911309719086, + 1.781836986541748, + 0.7104123830795288, + -2.0781984329223633, + -0.6968748569488525, + -0.1437186300754547, + 0.8155919313430786, + 0.5188283920288086, + 0.23954623937606812, + -0.6596511602401733, + -1.3867920637130737, + -0.7448391318321228 + ], + [ + 0.11259422451257706, + 0.43877506256103516, + -0.585435152053833, + 0.12099790573120117, + 0.5070493817329407, + -2.101219654083252, + -1.3759371042251587, + -0.7516231536865234, + 0.7018498778343201, + 0.7899036407470703, + 0.9772143959999084, + -0.7032701373100281, + -1.4699928760528564, + -1.4471495151519775, + 0.4750489890575409, + 0.8803708553314209, + -0.25032687187194824, + 1.3429648876190186, + 0.7985087037086487, + -0.5786607265472412, + -1.563289761543274 + ], + [ + 0.4965847432613373, + -0.8452511429786682, + 0.21300016343593597, + 0.28797537088394165, + -1.0059970617294312, + -3.040114402770996, + -1.1050269603729248, + 0.7826927304267883, + 0.8645586371421814, + -0.9447845816612244, + 1.0121192932128906, + 0.5123239755630493, + -1.5470938682556152, + -0.6170141100883484, + 0.2538997530937195, + 0.912939727306366, + -0.8613666892051697, + 0.9517492055892944, + 0.7185115218162537, + -2.1534924507141113, + -0.6818757057189941 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025899745523929596, + -0.03433045744895935, + -0.044045113027095795, + 0.031209349632263184, + -0.014532476663589478, + -1.1242008209228516, + 0.0, + 0.10768167674541473, + -0.020356399938464165, + 0.09216367453336716, + 0.051615357398986816, + -0.011370409280061722, + -0.7304134368896484, + 0.0, + 0.04558054730296135, + 0.038496289402246475, + 0.02504396252334118, + 0.040047526359558105, + 0.01217309758067131, + -0.31500017642974854, + 0.0 + ], + [ + -0.5198070406913757, + -0.8798871040344238, + 0.04596509411931038, + 1.55931556224823, + 0.31027060747146606, + -1.5283797979354858, + -1.6053413152694702, + -0.2754230797290802, + -0.012456716038286686, + 0.7998818159103394, + 1.5184144973754883, + 0.05059843137860298, + -1.85283362865448, + -0.9533926248550415, + 0.9978112578392029, + 0.8525571227073669, + -0.9611027836799622, + 0.32179969549179077, + -0.448661744594574, + -1.9452965259552002, + -0.7132223844528198 + ] + ], + "predicted_scores": [ + 3.505180835723877, + 3.2975637912750244, + 3.262356996536255, + 3.181797504425049, + 3.159857749938965, + 3.157637357711792, + 3.1126620769500732, + 2.846342086791992 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1607773860/train1", + "selected_index": 0, + "state_hash": "fdc7379ab4491c9f65860cadd3b2f3c9f2335f26aae91fb1c06d3a4edf858d6a", + "stored_base_utility": 0.8544998168945312, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8272150158882141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8272150158882141 + }, + "base_progress": 0.8272150158882141, + "base_success": false, + "base_utility": 0.8272150158882141, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8165659308433533, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8165659308433533 + }, + { + "contact_quality": null, + "progress": 0.21969212591648102, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21969212591648102 + }, + { + "contact_quality": null, + "progress": 0.20318710803985596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20318710803985596 + }, + { + "contact_quality": null, + "progress": 0.21245209872722626, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21245209872722626 + }, + { + "contact_quality": null, + "progress": 0.20310749113559723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20310749113559723 + }, + { + "contact_quality": null, + "progress": 0.24723665416240692, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24723665416240692 + }, + { + "contact_quality": null, + "progress": 0.8245300054550171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8245300054550171 + }, + { + "contact_quality": null, + "progress": 0.20159128308296204, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20159128308296204 + } + ], + "candidate_progress": [ + 0.8165659308433533, + 0.21969212591648102, + 0.20318710803985596, + 0.21245209872722626, + 0.20310749113559723, + 0.24723665416240692, + 0.8245300054550171, + 0.20159128308296204 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-20f3cfcf183c69853b1e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b0fee26e1db967f90b81", + "generated_tangents": [ + [ + -0.016336917877197266, + -0.013360157608985901, + -0.07621291279792786, + -0.16371239721775055, + 0.0668400228023529, + -1.894248366355896, + 0.008704543113708496, + -0.06707262992858887, + 0.0719880759716034, + -0.15066036581993103, + -0.08019492775201797, + -0.0692489743232727, + -1.5693426132202148, + -0.240192249417305, + -0.0804758071899414, + -0.01940889097750187, + 0.033196404576301575, + 0.04620952904224396, + -0.1469961702823639, + -1.5181262493133545, + -0.29363399744033813 + ], + [ + 0.027893297374248505, + -0.008285533636808395, + -0.08378562331199646, + -0.17221945524215698, + 0.21182218194007874, + -1.9948073625564575, + 0.03760448098182678, + 0.035435259342193604, + 0.0717722699046135, + -0.08506845682859421, + 0.04238452389836311, + -0.03463172912597656, + -1.7876969575881958, + -0.13388626277446747, + -0.0628129243850708, + -0.01397827547043562, + 0.034210845828056335, + 0.04611770808696747, + -0.11658340692520142, + -1.548569917678833, + -0.21235740184783936 + ], + [ + 0.041436389088630676, + -0.07316534221172333, + -0.045554667711257935, + -0.22406762838363647, + 0.07410885393619537, + -1.6889369487762451, + -0.057614028453826904, + 0.10750889033079147, + 0.03681034594774246, + -0.06785174459218979, + 0.005821295082569122, + -0.10103631019592285, + -1.4945741891860962, + -0.16003330051898956, + -0.04352296516299248, + -0.0019179899245500565, + 0.0614660382270813, + 0.03905145823955536, + -0.05980569124221802, + -1.4910571575164795, + -0.31295046210289 + ], + [ + 0.1022481769323349, + -0.06372478604316711, + 0.09434723854064941, + -0.08511318266391754, + 0.16531969606876373, + -1.787405014038086, + 0.029011934995651245, + -0.0098990797996521, + 0.11715134978294373, + -0.12257704883813858, + 0.04646285995841026, + 0.013985782861709595, + -1.4870665073394775, + -0.20644120872020721, + -0.029565926641225815, + -0.041694074869155884, + -0.07414861023426056, + 0.018175959587097168, + -0.06981590390205383, + -1.47420334815979, + -0.30480608344078064 + ], + [ + 0.016742020845413208, + 0.015241581946611404, + -0.0390644371509552, + -0.12326564639806747, + 0.11400949209928513, + -1.7120423316955566, + -0.029703915119171143, + 0.02878272533416748, + 0.1282055675983429, + -0.13336879014968872, + 0.06595300137996674, + 0.07015843689441681, + -1.4451978206634521, + -0.16225583851337433, + -0.0900433212518692, + -0.04399387910962105, + -0.04886709153652191, + 0.07016231119632721, + -0.03312358260154724, + -1.4111404418945312, + -0.27658745646476746 + ], + [ + 0.0614650622010231, + 0.017464391887187958, + 0.005555987358093262, + -0.21283464133739471, + 0.2246522158384323, + -2.001433849334717, + 0.09043154120445251, + 0.10920168459415436, + 0.03578871488571167, + -0.2796453833580017, + 0.08667211979627609, + -0.17847344279289246, + -1.6167254447937012, + -0.32877546548843384, + -0.3370673954486847, + -0.12383782863616943, + 0.16609779000282288, + -0.03975054621696472, + -0.3205159902572632, + -1.37526273727417, + -0.07098342478275299 + ], + [ + -0.020552828907966614, + -0.0018603764474391937, + 0.048248082399368286, + -0.4000368118286133, + 0.3300015926361084, + -2.0215983390808105, + 0.23077704012393951, + 0.048726677894592285, + 0.02718084678053856, + -0.18375235795974731, + 0.020289212465286255, + -0.02788648009300232, + -1.767735481262207, + -0.16084523499011993, + -0.07335028052330017, + -0.03634614497423172, + 0.015561208128929138, + 0.05741630494594574, + -0.14920943975448608, + -1.3750662803649902, + -0.5028982162475586 + ], + [ + -0.05880250036716461, + -0.042254041880369186, + 0.07410252094268799, + -0.39490199089050293, + 0.25762513279914856, + -1.9657152891159058, + 0.1290152668952942, + 0.10992196947336197, + 0.03848196193575859, + -0.12064739316701889, + 0.03580477833747864, + -0.09004402160644531, + -1.7064241170883179, + -0.24991311132907867, + -0.09742112457752228, + 0.009243233129382133, + 0.09109720587730408, + 0.027836203575134277, + -0.0063446760177612305, + -1.503725290298462, + -0.5057412385940552 + ] + ], + "generated_utilities": [ + 0.8165659308433533, + 0.21969212591648102, + 0.20318710803985596, + 0.21245209872722626, + 0.20310749113559723, + 0.24723665416240692, + 0.8245300054550171, + 0.20159128308296204 + ], + "group_id": "ms-b0fee26e1db967f90b81", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8397466540336609, + 0.8594310283660889, + 0.14638642966747284, + 0.09418028593063354, + 0.7838175296783447, + 0.5982922315597534, + 0.5913184881210327, + 2.0, + 0.20996521413326263, + 0.17419946193695068, + 0.11757439374923706, + 0.1645697057247162, + 0.05191062390804291, + 0.08658662438392639, + 0.04989185929298401, + 0.05867189168930054 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1510201096534729, + -0.21724483370780945, + 0.14240525662899017, + 0.003108598291873932, + -0.11609727144241333, + -1.6516358852386475, + 0.0, + -0.09976008534431458, + 0.061503566801548004, + 0.3070257902145386, + -0.08051937818527222, + 0.05735237896442413, + -1.6075372695922852, + 0.0, + -0.19649454951286316, + -0.15815469622612, + 0.2621642053127289, + -0.17578476667404175, + -0.034231506288051605, + -1.3808586597442627, + 0.0 + ], + [ + 0.014505341649055481, + 0.0403006337583065, + 0.5529788732528687, + 0.0714028924703598, + -0.41683781147003174, + -1.6516358852386475, + 0.0, + 0.41446545720100403, + -0.16278114914894104, + -0.4971221685409546, + -0.3640303611755371, + 0.02483367919921875, + -1.6075372695922852, + 0.0, + 0.5711643695831299, + -0.5244585275650024, + -0.5069849491119385, + 0.31709733605384827, + -0.4254770278930664, + -1.3808586597442627, + 0.0 + ], + [ + -0.044587142765522, + 0.03136333078145981, + 0.014485275372862816, + 0.04301099479198456, + 0.47328460216522217, + -2.6516358852386475, + -0.32303211092948914, + -0.01855158433318138, + -0.003608003491535783, + 0.029763570055365562, + 0.6359696388244629, + -0.1304762363433838, + -2.607537269592285, + -1.7957698106765747, + -0.009414590895175934, + -0.000939749414101243, + 0.018994441255927086, + 0.7486392855644226, + -0.06401278078556061, + -2.3808586597442627, + -1.722450852394104 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.6460642218589783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.5915396213531494, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.444901704788208 + ], + [ + 0.18585285544395447, + 0.17557260394096375, + 0.08478302508592606, + -0.17644454538822174, + 0.7477058172225952, + -2.614673137664795, + -0.3018328547477722, + 0.2204062044620514, + -0.0497702918946743, + 0.1961878538131714, + 0.5304322242736816, + -0.15872131288051605, + -2.604066848754883, + -2.053436517715454, + 0.18444111943244934, + -0.0024187257513403893, + 0.0054171085357666016, + 1.0256288051605225, + 0.02724038064479828, + -2.426236391067505, + -1.6594898700714111 + ], + [ + -0.5335133075714111, + 0.12298225611448288, + -0.12197553366422653, + 0.0424814410507679, + 0.002416938543319702, + -2.8428826332092285, + -0.17827168107032776, + -0.3377644121646881, + 0.04258471727371216, + -0.15910103917121887, + 0.9169985055923462, + 0.003966793417930603, + -3.069180727005005, + -1.407257318496704, + -0.47542956471443176, + -0.2210284322500229, + -0.10021048784255981, + 0.7806655168533325, + 0.3228837251663208, + -2.058253049850464, + -1.556677222251892 + ], + [ + 0.20687156915664673, + -0.5511468052864075, + 0.3945963978767395, + -0.16104432940483093, + 0.4936041831970215, + -2.1271684169769287, + 0.02855244278907776, + 0.544975996017456, + -0.3336343765258789, + 0.029475202783942223, + 0.8120307326316833, + -0.3246060609817505, + -2.885145425796509, + -2.2030622959136963, + -0.28730231523513794, + 0.19937433302402496, + 0.19840951263904572, + 1.1192030906677246, + -0.3995746374130249, + -2.5594136714935303, + -1.1545796394348145 + ], + [ + -0.5497553944587708, + -0.39125651121139526, + 0.2725587487220764, + -0.27965104579925537, + 1.0584466457366943, + -2.1044363975524902, + -0.6898342370986938, + 0.36552220582962036, + -0.2289373278617859, + 0.24627351760864258, + 0.7968363165855408, + 0.14776766300201416, + -2.988917112350464, + -1.6111457347869873, + -0.5116392374038696, + 0.030796781182289124, + -0.08262427151203156, + 1.0816104412078857, + 0.336497962474823, + -1.684936285018921, + -2.290850877761841 + ], + [ + 0.18322327733039856, + 0.6938241720199585, + 0.5252001881599426, + 0.06770005077123642, + 1.120344877243042, + -1.8558356761932373, + 0.18930694460868835, + -0.7033196687698364, + 0.7861698865890503, + -0.0965423658490181, + 0.015094935894012451, + -0.5303559303283691, + -2.6036770343780518, + -2.589250087738037, + -0.468231737613678, + -0.5901691317558289, + -0.7109078764915466, + 0.986366331577301, + -0.7634903192520142, + -2.984590530395508, + -1.1651530265808105 + ], + [ + 0.7039692401885986, + -0.5346179008483887, + 0.1466517597436905, + -0.6284777522087097, + -0.07994896173477173, + -2.4640438556671143, + -1.102164626121521, + -0.26685386896133423, + -0.3165925443172455, + 0.8653677701950073, + 0.232782244682312, + -0.5054740905761719, + -3.400317668914795, + -1.4077497720718384, + -0.09987740218639374, + -0.27442091703414917, + 0.5428174138069153, + 1.5128605365753174, + 0.747156023979187, + -2.0367631912231445, + -2.464085578918457 + ], + [ + 0.4474923610687256, + -0.7824500203132629, + 0.5928975343704224, + 0.57558673620224, + -0.17630118131637573, + -2.176387310028076, + -0.3299017548561096, + -0.6172446608543396, + 0.5065999627113342, + 0.46063295006752014, + 0.5951727628707886, + -0.02366364747285843, + -3.254279136657715, + -0.8880517482757568, + 0.3714323043823242, + 0.12449074536561966, + 0.6677311658859253, + -0.20713549852371216, + 0.5062628984451294, + -3.2156763076782227, + -1.9652986526489258 + ], + [ + -0.437069296836853, + 0.615723192691803, + -0.7743021249771118, + 0.20228458940982819, + 0.4320293366909027, + -2.0403428077697754, + -0.32251936197280884, + -0.7280393838882446, + 0.10878247022628784, + 0.9395551681518555, + 0.6021312475204468, + -0.3748174011707306, + -2.0175063610076904, + -1.9166245460510254, + -0.10793138295412064, + 0.6512030959129333, + 0.3747653365135193, + 1.5698660612106323, + -0.7995363473892212, + -1.8378328084945679, + -1.0564453601837158 + ], + [ + -0.7214477062225342, + -0.5686705708503723, + 0.8819299936294556, + 0.7319017648696899, + 1.0695421695709229, + -2.0338172912597656, + 0.2973436415195465, + -0.19576850533485413, + 0.4506514370441437, + 0.9037411212921143, + 0.0996239185333252, + 0.5076434016227722, + -2.611851930618286, + -2.7078182697296143, + 0.022706281393766403, + -0.4812851846218109, + -0.40778082609176636, + 1.6509652137756348, + -0.5721896886825562, + -1.8604621887207031, + -1.2943172454833984 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.017527949064970016, + -0.13547566533088684, + -0.31396082043647766, + -0.06045208126306534, + 0.7718918323516846, + -2.996134042739868, + -0.0199548602104187, + 0.26194432377815247, + -0.303554505109787, + -0.09068091958761215, + 0.3007984459400177, + -0.39815279841423035, + -2.626121997833252, + -1.8776893615722656, + 0.25959014892578125, + 0.20619672536849976, + -0.25346171855926514, + 0.40209779143333435, + 0.1943899542093277, + -2.464694023132324, + -1.5338099002838135 + ] + ], + "predicted_scores": [ + 3.2439491748809814, + 3.1261205673217773, + 3.0300378799438477, + 2.903808116912842, + 2.8831100463867188, + 2.7466049194335938, + 2.478484869003296, + 2.3214616775512695 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1358421325/train1", + "selected_index": 0, + "state_hash": "b8e04d64f31138fff8f83f1bb017f9e8515565a2ada2ba5787ad5f06b796589c", + "stored_base_utility": 0.8397466540336609, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8307622671127319, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8307622671127319 + }, + "base_progress": 0.8307622671127319, + "base_success": false, + "base_utility": 0.8307622671127319, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.20941472053527832, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20941472053527832 + }, + { + "contact_quality": null, + "progress": 0.24078069627285004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24078069627285004 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.16757088899612427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16757088899612427 + }, + { + "contact_quality": null, + "progress": 0.20940162241458893, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20940162241458893 + }, + { + "contact_quality": null, + "progress": 0.22608521580696106, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22608521580696106 + }, + { + "contact_quality": null, + "progress": 0.8135863542556763, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8135863542556763 + }, + { + "contact_quality": null, + "progress": 0.8580727577209473, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8580727577209473 + } + ], + "candidate_progress": [ + 0.20941472053527832, + 0.24078069627285004, + 1.0, + 0.16757088899612427, + 0.20940162241458893, + 0.22608521580696106, + 0.8135863542556763, + 0.8580727577209473 + ], + "candidate_source_chart_ids": [ + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-852c671b0adeccfa4bf1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + true, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "generated_tangents": [ + [ + -0.007829621434211731, + 0.01891593635082245, + -0.19329121708869934, + -0.005221456289291382, + 0.8205686807632446, + 0.10241826623678207, + 0.26574987173080444, + 0.008135095238685608, + 0.07516167312860489, + -0.02350630611181259, + -0.11826420575380325, + -0.04830430448055267, + -1.471059799194336, + 0.09093007445335388, + -0.04718199744820595, + -0.014181646518409252, + 0.0020204484462738037, + -0.017738118767738342, + -0.05534696578979492, + -1.0768465995788574, + -0.037861645221710205 + ], + [ + 0.02366016060113907, + -0.06201127916574478, + -0.32058408856391907, + -0.002542644739151001, + 0.7281009554862976, + 0.3995829224586487, + 0.8869548439979553, + 0.11554411798715591, + 0.02445787936449051, + -0.1949370801448822, + -0.267350435256958, + -0.24438369274139404, + -1.6549205780029297, + 0.12907633185386658, + -0.23692730069160461, + -0.05998361483216286, + -0.07307992875576019, + -0.08992688357830048, + -0.20582860708236694, + -1.2334067821502686, + -0.3705994188785553 + ], + [ + -0.04514935612678528, + -0.013184353709220886, + -0.04945489764213562, + -0.04480566084384918, + 0.7700527906417847, + 0.00011171400547027588, + 0.2241833209991455, + 0.02297687530517578, + 0.04185006767511368, + 0.049113623797893524, + -0.18785682320594788, + -0.1471807062625885, + -1.612541675567627, + -0.0796525776386261, + -0.03811833634972572, + 0.0030864905565977097, + -0.042008593678474426, + 0.014863431453704834, + -0.12185788154602051, + -1.2912936210632324, + -0.05844557285308838 + ], + [ + -0.020709052681922913, + 0.0016984492540359497, + -0.10843166708946228, + -0.05650597810745239, + 0.7973015308380127, + -0.06671230494976044, + 0.39062896370887756, + 0.10322524607181549, + 0.11411242932081223, + -0.2520296573638916, + -0.2025318145751953, + 0.10886607319116592, + -1.5927718877792358, + -0.04515893757343292, + 0.008747205138206482, + -0.016902105882763863, + -0.0842159241437912, + 0.06362536549568176, + -0.18833810091018677, + -1.1561708450317383, + -0.04434993863105774 + ], + [ + -0.06029124557971954, + 0.02984365075826645, + 0.02232035994529724, + 0.08511334657669067, + 0.758011519908905, + 0.08001004159450531, + 0.0702548623085022, + 0.08262918889522552, + 0.08510931581258774, + -0.137472003698349, + -0.09991353005170822, + -0.08376145362854004, + -1.514747142791748, + 0.18701723217964172, + 0.014249257743358612, + -0.06069842353463173, + -0.030516967177391052, + 0.06509275734424591, + -0.05639326572418213, + -1.169434905052185, + 0.05315692722797394 + ], + [ + 0.06198125332593918, + -0.022218966856598854, + -0.30206623673439026, + 0.06313300132751465, + 0.874203085899353, + 0.14009854197502136, + 0.2918381690979004, + 0.06727392971515656, + 0.04293106496334076, + -0.01505974680185318, + -0.06775247305631638, + -0.15592628717422485, + -1.4857100248336792, + 0.13839712738990784, + -0.14423401653766632, + -0.041462600231170654, + -0.1070481389760971, + -0.05557180941104889, + -0.01071956753730774, + -1.1340205669403076, + -0.06076802313327789 + ], + [ + -0.13456524908542633, + -0.02907385863363743, + -0.1418074071407318, + 0.17192599177360535, + 0.697289228439331, + 0.38130325078964233, + 0.6507576704025269, + 0.005012795329093933, + 0.09296613186597824, + -0.21925926208496094, + -0.2505892217159271, + -0.34688031673431396, + -1.4939807653427124, + 0.1576516330242157, + -0.26408883929252625, + -0.07235299050807953, + -0.06471963226795197, + -0.11286158859729767, + -0.059493839740753174, + -1.4128203392028809, + -0.2227497696876526 + ], + [ + -0.12860457599163055, + 0.04124286025762558, + -0.11609110236167908, + 0.10706081986427307, + 0.7368475794792175, + 0.262528657913208, + 0.23853906989097595, + 0.07235468178987503, + 0.018470266833901405, + 0.014972291886806488, + -0.1195920780301094, + 0.08717256784439087, + -1.4545354843139648, + 0.1598609983921051, + -0.04632589593529701, + -0.014307182282209396, + 0.054265186190605164, + 0.019666001200675964, + -0.1396884322166443, + -1.1443967819213867, + -0.05688692629337311 + ] + ], + "generated_utilities": [ + 0.20941472053527832, + 0.24078069627285004, + 2.0, + 0.16757088899612427, + 0.20940162241458893, + 0.22608521580696106, + 0.8135863542556763, + 0.8580727577209473 + ], + "group_id": "ms-bc2d3b425d9163a62dbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8266608119010925, + 2.0, + 0.15905821323394775, + 0.13648143410682678, + 0.5621776580810547, + 0.18381908535957336, + 0.20885342359542847, + 0.18184903264045715, + 0.19253113865852356, + 0.5574395656585693, + 0.10362428426742554, + 0.0659959614276886, + 0.09997051954269409, + 0.2210000902414322, + 0.00010728836059570312, + 0.12723565101623535 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.059883326292037964, + -0.16710972785949707, + -0.04027438163757324, + -0.23492489755153656, + 0.34960758686065674, + 0.16551673412322998, + 0.0, + -0.27484941482543945, + 0.07304387539625168, + -0.38491085171699524, + 0.03056102991104126, + 0.16416949033737183, + -1.5270280838012695, + 0.0, + 0.01279987208545208, + -0.07740817964076996, + 0.07519636303186417, + 0.14440757036209106, + 0.0022157281637191772, + -1.3374013900756836, + 0.0 + ], + [ + -0.6122239828109741, + -0.0734642744064331, + -0.4064018726348877, + 0.11902926862239838, + 0.34960758686065674, + -0.07764682173728943, + 0.0, + 0.37407442927360535, + -0.08740249276161194, + -0.20959049463272095, + -0.37996697425842285, + -0.304497092962265, + -1.5270280838012695, + 0.0, + -0.9113190174102783, + 0.3964778184890747, + 0.4811243414878845, + 0.2385387420654297, + -0.11476142704486847, + -1.3374013900756836, + 0.0 + ], + [ + 0.3877759873867035, + 0.6924911141395569, + -1.0402743816375732, + 0.005685340613126755, + 1.3496075868606567, + -0.2760518193244934, + 1.2840001583099365, + -0.028063178062438965, + 0.0023769063409417868, + 0.03187502175569534, + 0.04654876887798309, + 0.3263346254825592, + -2.5270280838012695, + -1.7457892894744873, + -0.012772172689437866, + -0.008647367358207703, + 0.019657021388411522, + 0.3050904870033264, + 0.19037143886089325, + -2.3374013900756836, + -1.738369345664978 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.568000316619873, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.4915785789489746, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.476738691329956 + ], + [ + 0.1989155411720276, + 0.7169155478477478, + -0.9921301603317261, + 0.22762086987495422, + 1.516188144683838, + -0.23096826672554016, + 1.0414973497390747, + 0.0012522991746664047, + -0.0518387071788311, + -0.23540246486663818, + 0.08515512943267822, + 0.5060473084449768, + -2.5231378078460693, + -1.4593229293823242, + -0.09671962261199951, + 0.09134317189455032, + -0.07944301515817642, + 0.18403387069702148, + 0.2437569499015808, + -2.270580291748047, + -1.9640144109725952 + ], + [ + 0.256519615650177, + 0.7805030941963196, + -1.3583617210388184, + 0.3310847878456116, + 1.2735480070114136, + -0.4127388000488281, + 1.2665152549743652, + -0.19997301697731018, + -0.11129427701234818, + 0.1647017002105713, + 0.3767858147621155, + 0.4831879138946533, + -2.752143621444702, + -1.6790910959243774, + 0.2554209232330322, + 0.3251851797103882, + -0.12605378031730652, + 0.696884274482727, + 0.29535919427871704, + -2.726069688796997, + -1.704551100730896 + ], + [ + 0.49772778153419495, + 1.0026791095733643, + -0.5513590574264526, + 0.008379313163459301, + 1.7768750190734863, + -0.6102496981620789, + 1.7048192024230957, + -0.29564282298088074, + -0.20420490205287933, + -0.23934513330459595, + -0.34791648387908936, + 0.0690477192401886, + -2.602295398712158, + -1.8138929605484009, + -0.38315677642822266, + -0.16421082615852356, + 0.17502839863300323, + -0.13996875286102295, + -0.30929630994796753, + -2.544666290283203, + -1.2411268949508667 + ], + [ + 0.07701912522315979, + 0.9529868364334106, + -1.497549295425415, + -0.3016125559806824, + 1.8504154682159424, + -0.3136579394340515, + 1.6619524955749512, + -0.28860214352607727, + 0.20993559062480927, + 0.6072397232055664, + 0.04243222624063492, + -0.133962482213974, + -2.0948100090026855, + -2.1691184043884277, + 0.44940999150276184, + -0.4183289408683777, + 0.5662978291511536, + 0.5016863942146301, + 0.7054548263549805, + -2.282663345336914, + -1.8892805576324463 + ], + [ + -0.2981111705303192, + 0.038420140743255615, + -1.687281847000122, + -0.37934571504592896, + 1.1300733089447021, + -0.6208293437957764, + 1.8144867420196533, + 0.16701459884643555, + 0.5508663654327393, + 0.4797254800796509, + -0.09421360492706299, + -0.3499920070171356, + -2.5030596256256104, + -1.5871793031692505, + -0.33045580983161926, + -0.29071635007858276, + 0.5385350584983826, + 0.9102991223335266, + 0.3121117353439331, + -2.2699851989746094, + -2.081907272338867 + ], + [ + -0.28236815333366394, + 0.18060839176177979, + -0.32270634174346924, + 0.033799149096012115, + 1.128151774406433, + -0.7577929496765137, + 1.6937967538833618, + -0.813525378704071, + 0.48178502917289734, + -0.6699209809303284, + -0.03810972720384598, + 0.8273124694824219, + -2.2768704891204834, + -1.317199945449829, + 0.0046694036573171616, + 0.5900158286094666, + -0.12453534454107285, + 1.046316385269165, + 0.2770566940307617, + -2.0448076725006104, + -2.4180057048797607 + ], + [ + 0.26116815209388733, + 1.305420994758606, + -0.5357515811920166, + 0.6523149013519287, + 0.5250536203384399, + -0.6821175217628479, + 1.2918037176132202, + 0.3275931477546692, + -0.8845832943916321, + -0.6987056136131287, + 0.10023711621761322, + -0.2849085032939911, + -1.7303776741027832, + -1.0117536783218384, + -0.06847482174634933, + 0.6783800721168518, + -0.04633811116218567, + 0.0702061653137207, + -0.3159322142601013, + -3.004882335662842, + -2.0691566467285156 + ], + [ + 0.5418367385864258, + 1.1724635362625122, + -1.717900276184082, + 0.9217556118965149, + 2.316706657409668, + -0.2411489486694336, + 0.720655083656311, + -0.6330894231796265, + 0.03703225776553154, + 0.38938266038894653, + -0.8305367827415466, + -0.2152116596698761, + -2.614929437637329, + -2.045497179031372, + -0.9044661521911621, + 0.9391879439353943, + -0.2323634922504425, + 0.8392186164855957, + -0.6889055371284485, + -3.0831263065338135, + -1.0035808086395264 + ], + [ + 1.0110303163528442, + 1.0131109952926636, + -1.3200907707214355, + -0.433234840631485, + 1.9466626644134521, + 0.46447479724884033, + 1.528633952140808, + 0.40949776768684387, + 0.07322453707456589, + -0.12349443882703781, + 0.24514153599739075, + 0.005379408597946167, + -2.487516164779663, + -1.1890358924865723, + -0.026332437992095947, + -0.6102214455604553, + 0.6559669971466064, + 0.4371780753135681, + -0.6651540994644165, + -2.8161611557006836, + -1.4271454811096191 + ], + [ + 0.9640108346939087, + 1.0122193098068237, + -1.9809117317199707, + 0.40601760149002075, + 0.9296187162399292, + 0.601173460483551, + 2.229055643081665, + -0.4972975254058838, + 0.6548057794570923, + 0.6165774464607239, + -0.7309268116950989, + 0.697066068649292, + -3.0948128700256348, + -2.4828031063079834, + 0.16930773854255676, + -0.11851751059293747, + -0.8277174830436707, + 0.8252183794975281, + -0.4829612374305725, + -2.922985076904297, + -1.1981642246246338 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03792491555213928, + -0.040427446365356445, + -0.04027438163757324, + 0.02325565740466118, + 0.34960758686065674, + -0.05261008441448212, + 0.0, + 0.018791399896144867, + 0.05303370952606201, + 0.01128687709569931, + 0.01692008785903454, + 0.06139722466468811, + -1.5270280838012695, + 0.0, + -0.004229618236422539, + 0.017936373129487038, + 0.06901376694440842, + -0.04424762725830078, + -0.033029720187187195, + -1.3374013900756836, + 0.0 + ] + ], + "predicted_scores": [ + 3.066279649734497, + 2.8866214752197266, + 2.7994003295898438, + 2.7754523754119873, + 2.6756951808929443, + 2.520692825317383, + 2.500575542449951, + 2.316096305847168 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007/train1", + "selected_index": 0, + "state_hash": "58b3d0c1cafc199ad58508912becbaef59038a655cef0caafd263ade5c9b53fa", + "stored_base_utility": 0.8266608119010925, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20032641291618347, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20032641291618347 + }, + "base_progress": 0.20032641291618347, + "base_success": false, + "base_utility": 0.20032641291618347, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.20836615562438965, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20836615562438965 + }, + { + "contact_quality": null, + "progress": 0.21678751707077026, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21678751707077026 + }, + { + "contact_quality": null, + "progress": 0.22490769624710083, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22490769624710083 + }, + { + "contact_quality": null, + "progress": 0.21572625637054443, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21572625637054443 + }, + { + "contact_quality": null, + "progress": 0.2215946912765503, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2215946912765503 + }, + { + "contact_quality": null, + "progress": 0.20464813709259033, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20464813709259033 + }, + { + "contact_quality": null, + "progress": 0.21880853176116943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21880853176116943 + }, + { + "contact_quality": null, + "progress": 0.22272410988807678, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22272410988807678 + } + ], + "candidate_progress": [ + 0.20836615562438965, + 0.21678751707077026, + 0.22490769624710083, + 0.21572625637054443, + 0.2215946912765503, + 0.20464813709259033, + 0.21880853176116943, + 0.22272410988807678 + ], + "candidate_source_chart_ids": [ + "ms-95028134c6e3670b89d5", + "ms-b8492a3a812095651d75", + "ms-95028134c6e3670b89d5", + "ms-ee2fcbc94d41385713d0", + "ms-016c5de39ed3c111889a", + "ms-c8706d0aa14a930236de", + "ms-9ae5c6c4132218ecf49e", + "ms-5ac1bf2d7d319955d45d" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank6", + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-d1960368ef76bbb10669", + "generated_tangents": [ + [ + -0.2732497453689575, + 0.10623323917388916, + -0.5341019630432129, + -0.52424156665802, + 0.38650238513946533, + 0.11353766918182373, + 1.2099891901016235, + -0.2784939110279083, + 0.29466575384140015, + -0.1788938045501709, + 0.012427084147930145, + -0.3214123249053955, + -1.5456247329711914, + 0.4183884859085083, + 0.17805370688438416, + -0.06384320557117462, + -0.3878546357154846, + 0.13299499452114105, + -0.2544509172439575, + -1.7263820171356201, + -0.864246129989624 + ], + [ + -0.21580372750759125, + 0.1762966513633728, + -0.6667286157608032, + -0.2037983387708664, + 0.8459116220474243, + 0.08843044936656952, + 0.694096565246582, + -0.11482015252113342, + 0.08122444152832031, + -0.2178671956062317, + 0.07332239300012589, + -0.40485692024230957, + -1.284013271331787, + 0.5618535876274109, + 0.2824944853782654, + 0.0454351082444191, + -0.4142775535583496, + 0.21625806391239166, + -0.06423258781433105, + -1.5438718795776367, + -0.5496693849563599 + ], + [ + -0.2495129257440567, + 0.0910559594631195, + -0.49450036883354187, + -0.3702200949192047, + 0.5745404362678528, + 0.018759846687316895, + 0.8391996622085571, + -0.12035444378852844, + 0.14337728917598724, + -0.07287713140249252, + -0.05631840229034424, + -0.06944793462753296, + -1.3841220140457153, + 0.22425532341003418, + 0.18419459462165833, + 0.013678228482604027, + -0.2958596348762512, + 0.21296192705631256, + -0.14534270763397217, + -1.3982222080230713, + -0.3495023548603058 + ], + [ + -0.2663615942001343, + 0.14154016971588135, + -0.38258078694343567, + -0.4201826751232147, + 0.55460125207901, + 0.7031106948852539, + 0.6285395622253418, + -0.11529606580734253, + 0.13880234956741333, + -0.016407139599323273, + 0.008387640118598938, + -0.3320184350013733, + -1.4349403381347656, + 0.11576896905899048, + 0.10303384065628052, + 0.06527340412139893, + -0.2843952178955078, + 0.33774620294570923, + -0.18962779641151428, + -1.386549949645996, + -0.11612573266029358 + ], + [ + -0.0685390830039978, + 0.10524150729179382, + -0.47070881724357605, + -0.5226870775222778, + 0.56330406665802, + -0.07589875161647797, + 0.6408642530441284, + -0.23171091079711914, + 0.12112893909215927, + 0.007567659020423889, + -0.08336234837770462, + -0.33656448125839233, + -1.3480051755905151, + 0.020533017814159393, + 0.1174524575471878, + -0.013492795638740063, + -0.3253559470176697, + 0.21753256022930145, + -0.12693777680397034, + -1.4755010604858398, + -0.41111454367637634 + ], + [ + -0.46328067779541016, + 0.14907579123973846, + -0.46756860613822937, + -0.3170608878135681, + 0.6231703758239746, + 0.03959271311759949, + 0.9835152626037598, + 0.017133131623268127, + 0.10476913303136826, + -0.14825591444969177, + -0.054740145802497864, + -0.347636878490448, + -1.119163155555725, + 0.571229100227356, + 0.2677895128726959, + 0.0102913286536932, + -0.21753139793872833, + 0.15240247547626495, + -0.03422674536705017, + -1.409219741821289, + -0.11958730220794678 + ], + [ + -0.21189619600772858, + 0.07958297431468964, + -0.490036278963089, + -0.42205339670181274, + 0.5753673315048218, + 0.016161277890205383, + 0.7361229658126831, + -0.1892039179801941, + 0.13392025232315063, + -0.025577999651432037, + -0.018562987446784973, + -0.21219003200531006, + -1.445809006690979, + 0.18592366576194763, + 0.17200149595737457, + -0.014207066036760807, + -0.28478527069091797, + 0.2607681155204773, + -0.1231447160243988, + -1.417062520980835, + -0.33946332335472107 + ], + [ + -0.23061718046665192, + 0.1084190160036087, + -0.44667258858680725, + -0.40127190947532654, + 0.5795594453811646, + 0.4705345034599304, + 0.6456956267356873, + -0.2400834858417511, + 0.17875860631465912, + -0.05076836794614792, + 0.0344834104180336, + -0.1540597677230835, + -1.8119797706604004, + 0.1843666434288025, + 0.07184919714927673, + 0.0648496001958847, + -0.23051045835018158, + 0.43014973402023315, + -0.2404172420501709, + -1.547011137008667, + -0.19852378964424133 + ] + ], + "generated_utilities": [ + 0.20836615562438965, + 0.21678751707077026, + 0.22490769624710083, + 0.21572625637054443, + 0.2215946912765503, + 0.20464813709259033, + 0.21880853176116943, + 0.22272410988807678 + ], + "group_id": "ms-d1960368ef76bbb10669", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.842734158039093, + 2.0, + 0.1342049241065979, + 0.18085119128227234, + 0.5091985464096069, + 0.016112878918647766, + 0.5080842971801758, + 0.19944033026695251, + 0.5132148265838623, + 0.5265648365020752, + 0.11437654495239258, + 0.04028162360191345, + 0.04675377905368805, + 0.13603349030017853, + 0.12877747416496277, + 0.05102705955505371 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2770041823387146, + 0.08894228935241699, + 0.07033950090408325, + 0.12426833808422089, + 0.16520383954048157, + 0.023725181818008423, + 0.0, + -0.25569218397140503, + -0.057803913950920105, + 0.06703128665685654, + 0.1732010841369629, + 0.07705201953649521, + -1.0374658107757568, + 0.0, + 0.04586680978536606, + 0.028830496594309807, + -0.0374501496553421, + 0.10374182462692261, + -0.007100850343704224, + -1.7975506782531738, + 0.0 + ], + [ + -0.4268343448638916, + 0.13483470678329468, + -0.06075429916381836, + 0.19817759096622467, + 0.09922409057617188, + 0.6547578573226929, + 0.0, + 0.15026617050170898, + 0.002472013235092163, + -0.31349992752075195, + 0.5119487047195435, + 0.571077287197113, + -1.0374658107757568, + 0.0, + -0.5828977823257446, + -0.20166385173797607, + -0.7097158432006836, + 0.8187870979309082, + -0.0694284737110138, + -1.7975506782531738, + 0.0 + ], + [ + -0.23462048172950745, + 1.088942289352417, + -0.5585125088691711, + 0.10042184591293335, + 0.4514904320240021, + -0.12210102379322052, + 2.176525354385376, + -0.10348781198263168, + 0.19529865682125092, + 0.12132670730352402, + 0.7056414484977722, + 0.12544797360897064, + -2.037465810775757, + 0.6289246678352356, + -0.008351176977157593, + 0.0010435802396386862, + 0.017314812168478966, + 0.6334809064865112, + -0.3720659911632538, + -2.797550678253174, + -1.5395829677581787 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.353050708770752, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.2578493356704712, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0791659355163574 + ], + [ + -0.09280626475811005, + 1.1662368774414062, + -0.8228501081466675, + 0.24497312307357788, + 0.5648186802864075, + -0.03522863984107971, + 2.259786367416382, + 0.14498579502105713, + 0.2030920535326004, + 0.39727938175201416, + 0.8023015856742859, + 0.3385764956474304, + -1.8738088607788086, + 0.7300182580947876, + 0.20887598395347595, + 0.23107871413230896, + 0.3028872311115265, + 0.7558343410491943, + -0.5975744724273682, + -3.0329887866973877, + -1.8127071857452393 + ], + [ + -0.13783538341522217, + 0.9487118721008301, + -0.3492928743362427, + -0.2956767678260803, + 0.2793389856815338, + -0.16702969372272491, + 2.558225631713867, + 0.17572781443595886, + 0.570004940032959, + 0.27895650267601013, + 0.6778483986854553, + -0.27156615257263184, + -1.8747575283050537, + 1.0240070819854736, + 0.27303546667099, + 0.33387133479118347, + 0.1442956030368805, + 0.24661725759506226, + -0.5561860799789429, + -2.9813332557678223, + -1.833754062652588 + ], + [ + -0.18553051352500916, + 1.0909820795059204, + -0.27226096391677856, + 0.1659364104270935, + 0.7108361721038818, + -0.27768880128860474, + 2.239938974380493, + 0.019483201205730438, + 0.21263465285301208, + -0.09839647263288498, + 0.9676799178123474, + -0.36985379457473755, + -2.133316993713379, + 0.3758644461631775, + -0.47750571370124817, + 0.17470555007457733, + -0.17035381495952606, + 0.2707958221435547, + -0.5672953128814697, + -2.394361972808838, + -1.668647289276123 + ], + [ + -0.3557406961917877, + 0.9564932584762573, + -1.0014276504516602, + -0.38222843408584595, + 0.41349664330482483, + -0.05637756735086441, + 1.6011608839035034, + -0.31295540928840637, + -0.15234358608722687, + 0.4648304879665375, + 0.2744198739528656, + -0.44805359840393066, + -2.1544065475463867, + 1.166503667831421, + 0.014861157163977623, + -0.23438170552253723, + 0.1278134286403656, + 0.5812121033668518, + -0.7571046352386475, + -2.5030438899993896, + -2.100955009460449 + ], + [ + -0.16500982642173767, + 0.7556273937225342, + -0.988593339920044, + 0.6213532090187073, + 0.15840065479278564, + 0.5245341062545776, + 2.452624797821045, + -0.5693200826644897, + -0.32241708040237427, + 0.49458175897598267, + 1.3341540098190308, + 0.059507817029953, + -2.304706335067749, + 0.7334492206573486, + -0.20702646672725677, + 0.32367274165153503, + -0.5396079421043396, + 0.9809319972991943, + -0.6172815561294556, + -3.197409152984619, + -0.9358728528022766 + ], + [ + -0.07666900753974915, + 1.8442692756652832, + -0.6603910326957703, + -0.40882760286331177, + 0.22092406451702118, + -0.23551997542381287, + 1.6105436086654663, + 0.09144749492406845, + 0.8141599893569946, + 0.8528898358345032, + 1.0663814544677734, + 0.3283998966217041, + -1.7124712467193604, + 1.2278027534484863, + 0.7039424180984497, + 0.3645748496055603, + -0.2968631386756897, + 0.4998055696487427, + -0.9362069368362427, + -3.461055278778076, + -0.8184676766395569 + ], + [ + -0.1520652174949646, + 1.405963659286499, + -0.7959750294685364, + -0.39229461550712585, + 0.46590372920036316, + -0.8267906904220581, + 2.952258586883545, + 0.5923218727111816, + -0.6996292471885681, + 0.8352785706520081, + -0.15220212936401367, + -0.1307106763124466, + -2.849807024002075, + 0.4983477294445038, + 0.5902999639511108, + -0.5390225648880005, + 0.13897128403186798, + 0.21545273065567017, + -1.1186240911483765, + -3.1762759685516357, + -1.0118725299835205 + ], + [ + -0.564094066619873, + 0.4751617908477783, + -1.3977731466293335, + 0.8189605474472046, + 0.17298781871795654, + 0.2476775199174881, + 2.8234164714813232, + -0.4518245458602905, + -0.7904537916183472, + -0.39086827635765076, + -0.05634337663650513, + -0.09099806845188141, + -1.9690911769866943, + 1.1869969367980957, + -0.5615661144256592, + 0.32865890860557556, + -0.8041903376579285, + 1.3324832916259766, + -0.6190662384033203, + -2.4903852939605713, + -2.3098156452178955 + ], + [ + -0.2328718602657318, + 0.9724385142326355, + -1.5535115003585815, + 0.7688977718353271, + 0.4409410357475281, + -0.8374075889587402, + 1.2719230651855469, + 0.653096616268158, + -0.2887840270996094, + 0.4892227351665497, + 0.9943453669548035, + 0.5566720962524414, + -2.496413469314575, + 0.6299278140068054, + -0.08689186722040176, + 0.024782167747616768, + 0.5747005939483643, + 0.023068726062774658, + -1.111369252204895, + -2.187150001525879, + -1.2047184705734253 + ], + [ + -1.1679872274398804, + 1.9997228384017944, + -0.45194220542907715, + -0.8861345648765564, + -0.5369991064071655, + 0.46434593200683594, + 2.175499439239502, + 0.7599186301231384, + 0.38914406299591064, + 0.6877046823501587, + -0.004208564758300781, + -0.1321776658296585, + -1.3474420309066772, + 0.8496907353401184, + 0.4218248426914215, + -0.9354817271232605, + 0.945868730545044, + 1.60379958152771, + -1.195396065711975, + -3.597724676132202, + -1.9041136503219604 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04476936161518097, + 0.08894228935241699, + 0.06436485052108765, + -0.0775555968284607, + 0.04825553297996521, + -0.060782790184020996, + 0.0, + -0.07688627392053604, + -0.06308497488498688, + -0.03861912339925766, + 0.1235572099685669, + -0.0017572194337844849, + -1.0374658107757568, + 0.0, + 0.03638586029410362, + 0.0556478351354599, + 0.0387292355298996, + 0.044096171855926514, + 0.029627740383148193, + -1.7975506782531738, + 0.0 + ] + ], + "predicted_scores": [ + 4.358102321624756, + 4.097344398498535, + 4.023711204528809, + 3.9058315753936768, + 3.8040666580200195, + 3.7624096870422363, + 3.7143890857696533, + 3.3574204444885254 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "416183106/train1", + "selected_index": 0, + "state_hash": "cd187560c1c39db9e6bf7d9a611a8b7108dc88766de58fd62b4317f0bbcc1b50", + "stored_base_utility": 0.842734158039093, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8350887894630432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8350887894630432 + }, + "base_progress": 0.8350887894630432, + "base_success": false, + "base_utility": 0.8350887894630432, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.814424455165863, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.814424455165863 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.22760358452796936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22760358452796936 + }, + { + "contact_quality": null, + "progress": 0.2006266564130783, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2006266564130783 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.21153436601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21153436601161957 + }, + { + "contact_quality": null, + "progress": 0.19750897586345673, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19750897586345673 + }, + { + "contact_quality": null, + "progress": 0.15966159105300903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15966159105300903 + } + ], + "candidate_progress": [ + 0.814424455165863, + 1.0, + 0.22760358452796936, + 0.2006266564130783, + 1.0, + 0.21153436601161957, + 0.19750897586345673, + 0.15966159105300903 + ], + "candidate_source_chart_ids": [ + "ms-5dfbb16e173e1a3e2bd6", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + true, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-efdc07c985935d871adf", + "generated_tangents": [ + [ + -0.013332024216651917, + -0.09877345710992813, + 0.26450932025909424, + -0.02948884665966034, + -0.12492042779922485, + -1.7737995386123657, + 0.08859455585479736, + 0.06688976287841797, + 0.06304284185171127, + -0.1261521875858307, + 0.024119026958942413, + -0.05646693706512451, + -1.324194073677063, + 0.04585892707109451, + -0.01973208412528038, + -0.07470755279064178, + -0.0056900084018707275, + 0.1738128662109375, + 0.2990410029888153, + -1.220249891281128, + -0.2839846611022949 + ], + [ + 0.03280780464410782, + -0.05737597495317459, + -0.022029787302017212, + -0.001036912202835083, + 0.004179731011390686, + -1.5749638080596924, + -0.024465858936309814, + 0.08278200030326843, + 0.05197286605834961, + -0.028704054653644562, + 0.06959202885627747, + -0.03036804497241974, + -1.372532606124878, + -0.12132520973682404, + -0.057227808982133865, + -0.02337384596467018, + 0.1442265510559082, + 0.03886638581752777, + -0.016652077436447144, + -1.168245553970337, + -0.1577155888080597 + ], + [ + -0.0032540112733840942, + -0.04752686992287636, + -0.04314276576042175, + 0.0059797316789627075, + -0.07244718074798584, + -1.5057730674743652, + -0.11828851699829102, + 0.07665592432022095, + 0.05501272529363632, + -0.07879000157117844, + 0.04016869515180588, + -0.004581347107887268, + -1.195197343826294, + -0.1345781832933426, + -0.0619640089571476, + 0.03739229217171669, + 0.11475878953933716, + 0.04048973321914673, + -0.04752197861671448, + -1.0660927295684814, + -0.18573719263076782 + ], + [ + -0.008654564619064331, + 0.0036018006503582, + 0.02833312749862671, + 0.14265021681785583, + -0.0014200806617736816, + -1.5085525512695312, + 0.048266440629959106, + 0.008562147617340088, + 0.1230761930346489, + -0.09674461930990219, + 0.07801411300897598, + 0.12203509360551834, + -1.2971980571746826, + -0.08735552430152893, + -0.08992095291614532, + -0.06161929666996002, + 0.06254541873931885, + 0.043771058320999146, + 0.004117995500564575, + -1.1442360877990723, + -0.1357654631137848 + ], + [ + 0.12913350760936737, + 0.08720725774765015, + 0.0193556547164917, + -0.2315005660057068, + 0.18963700532913208, + -1.8492833375930786, + -0.20759832859039307, + 0.2419825941324234, + 0.03273481875658035, + -0.0850699320435524, + 0.08927270025014877, + 0.14676442742347717, + -1.4844624996185303, + -0.6773268580436707, + -0.14903348684310913, + 0.12158150970935822, + 0.35294899344444275, + 0.05146056413650513, + 0.13797593116760254, + -1.037107229232788, + -0.24696126580238342 + ], + [ + 0.027051463723182678, + -0.06046345829963684, + 0.03760448098182678, + -0.012343600392341614, + 0.021400943398475647, + -1.4144668579101562, + -0.09972888231277466, + 0.12906959652900696, + 0.015865160152316093, + -0.04949576407670975, + 0.13024920225143433, + 0.09889984875917435, + -1.251424789428711, + -0.09391428530216217, + -0.07430841028690338, + 0.009700924158096313, + 0.03600558638572693, + 0.15948447585105896, + 0.010383576154708862, + -1.0674681663513184, + -0.19397443532943726 + ], + [ + -0.19073574244976044, + 0.06492596864700317, + 0.09132397174835205, + -0.2691749930381775, + 0.500572919845581, + -1.6856186389923096, + 0.4800916314125061, + 0.283923864364624, + -0.1709602326154709, + -0.35143622756004333, + 0.08284296840429306, + -0.07589542865753174, + -1.6209856271743774, + -0.32679450511932373, + 0.06518667191267014, + 0.04909523203969002, + 0.07296597957611084, + -0.12730474770069122, + 0.23832252621650696, + -0.8187363743782043, + -0.5636305809020996 + ], + [ + -0.2372923344373703, + 0.07363962382078171, + 0.15948879718780518, + -0.17178823053836823, + 0.1485951989889145, + -1.4346777200698853, + -0.2271597981452942, + 0.04705031216144562, + -0.034805335104465485, + -0.37829411029815674, + 0.014230944216251373, + 0.33765649795532227, + -1.5487416982650757, + -0.5470280051231384, + -0.2538008391857147, + -0.015747789293527603, + -0.04476819932460785, + 0.1644936501979828, + 0.24060587584972382, + -1.043912410736084, + -0.5292702913284302 + ] + ], + "generated_utilities": [ + 0.814424455165863, + 2.0, + 0.22760358452796936, + 0.2006266564130783, + 2.0, + 0.21153436601161957, + 0.19750897586345673, + 0.15966159105300903 + ], + "group_id": "ms-efdc07c985935d871adf", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8197344541549683, + 2.0, + 0.8125, + 0.8125142455101013, + 0.7847581505775452, + 0.7794750332832336, + 0.5920823216438293, + 0.5697357058525085, + 2.0, + 0.5847549438476562, + 0.09384849667549133, + 0.07791829109191895, + 0.011395931243896484, + 0.09286168217658997, + 0.10894183814525604, + 0.0616525262594223 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2827143669128418, + -0.12206736207008362, + 0.1543778032064438, + 0.27268126606941223, + -0.17288367450237274, + -2.5338973999023438, + -1.7615662813186646, + -0.19161763787269592, + 0.1886080503463745, + 0.31686434149742126, + 0.5112922191619873, + 0.14046527445316315, + -2.17134952545166, + -1.8021173477172852, + 0.11115366220474243, + 0.25490811467170715, + -0.025668112561106682, + 0.4787765145301819, + 0.46609628200531006, + -1.711870551109314, + -1.5312864780426025 + ], + [ + 0.03046044334769249, + 0.042622197419404984, + -0.12103251367807388, + 0.7814357876777649, + 0.5093902349472046, + -2.648094654083252, + -1.5764049291610718, + -0.16382060945034027, + 0.15227201581001282, + -0.29746392369270325, + 0.394713819026947, + 0.044284895062446594, + -2.150869131088257, + -1.853789210319519, + 0.24474957585334778, + 0.16074210405349731, + -0.30960795283317566, + 0.986261248588562, + 0.4512072801589966, + -1.9782602787017822, + -1.814382553100586 + ], + [ + -0.47424501180648804, + 0.4193490147590637, + 0.31025299429893494, + 0.436926007270813, + 0.07080172002315521, + -2.2750933170318604, + -2.139326572418213, + 0.19072748720645905, + -0.05066118389368057, + -0.38944289088249207, + 0.11913672089576721, + -0.4728909432888031, + -2.630631446838379, + -1.9070930480957031, + 0.5720394849777222, + 0.1898895502090454, + -0.0872373953461647, + 0.3081148862838745, + 0.6700773239135742, + -1.8990029096603394, + -1.655537486076355 + ], + [ + -0.112115278840065, + -0.5390006899833679, + -0.27749133110046387, + -0.08230990171432495, + 0.18304404616355896, + -2.5003390312194824, + -1.177594542503357, + -0.37152859568595886, + 0.1827065348625183, + -0.1805598884820938, + 0.8884490728378296, + 0.3141748607158661, + -2.7738473415374756, + -1.8585447072982788, + -0.6611859798431396, + 0.5365979075431824, + 0.024639926850795746, + 1.0166820287704468, + 0.6261310577392578, + -1.3820469379425049, + -1.1859028339385986 + ], + [ + -0.013153300620615482, + 0.6059781908988953, + -0.7563084363937378, + 0.6436030864715576, + 0.4828413128852844, + -1.8755569458007812, + -1.896214246749878, + -0.445474773645401, + -0.6578570604324341, + -0.21610267460346222, + 1.3274874687194824, + -0.6251860857009888, + -2.005418300628662, + -1.463393211364746, + -0.40788042545318604, + -0.5932772159576416, + -0.6501284837722778, + 0.1491221785545349, + 0.8418278694152832, + -2.578186511993408, + -1.628711462020874 + ], + [ + -0.051258739084005356, + 0.6121965050697327, + -0.4303005337715149, + 0.378423810005188, + -0.4187525510787964, + -2.0054633617401123, + -1.5839252471923828, + 0.38306933641433716, + -0.8548648953437805, + 0.5034631490707397, + 0.3682137727737427, + 0.867935061454773, + -2.5900886058807373, + -1.8283436298370361, + 0.3810080289840698, + -0.6684208512306213, + 0.5362142324447632, + -0.17851078510284424, + 1.083348035812378, + -1.9181195497512817, + -2.306365728378296 + ], + [ + 0.5487861037254333, + 0.6947635412216187, + -0.3694673776626587, + 1.3571128845214844, + -0.20053730905056, + -2.9099156856536865, + -1.493881106376648, + 0.9321863055229187, + -0.355974406003952, + 0.6561991572380066, + -0.29231804609298706, + 0.4913230240345001, + -1.8409770727157593, + -2.0504229068756104, + 0.14528872072696686, + 0.483799010515213, + -0.32654285430908203, + 1.6123273372650146, + 0.03987070918083191, + -0.8449068665504456, + -2.3494603633880615 + ], + [ + -0.3259732127189636, + -0.5385614633560181, + 0.7304226160049438, + -0.12314724922180176, + 0.03945803642272949, + -1.5919837951660156, + -1.907224416732788, + 0.3930681645870209, + -0.8835994005203247, + -0.3352096974849701, + 0.17923185229301453, + 0.17222784459590912, + -2.512176752090454, + -1.859424352645874, + 0.536429762840271, + 0.11454571038484573, + 0.3300909101963043, + 0.5491204261779785, + -0.04762554168701172, + -1.1976318359375, + -0.5251960158348083 + ], + [ + 0.33097749948501587, + -0.04192434251308441, + -0.34520047903060913, + -0.04110339283943176, + -0.5535940527915955, + -2.2057979106903076, + -2.5934860706329346, + -0.9023091793060303, + 0.5658158659934998, + -0.6125177145004272, + 1.095580816268921, + -0.05609257519245148, + -1.606997013092041, + -2.0733447074890137, + -0.5881220102310181, + 0.8953573703765869, + 0.9258373975753784, + 0.27281180024147034, + 0.7141706943511963, + -0.9142226576805115, + -1.3141751289367676 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05978316813707352, + 0.057473838329315186, + -0.09624473005533218, + -0.011721253395080566, + 0.028141841292381287, + -1.4311916828155518, + 0.0, + -0.024008113890886307, + 0.07049623876810074, + 0.03528204560279846, + -0.010002851486206055, + 0.010910652577877045, + -1.1452367305755615, + 0.0, + 0.020785929635167122, + -0.03374411538243294, + 0.11616832762956619, + 0.05772918462753296, + 0.01671847701072693, + -0.828272819519043, + 0.0 + ], + [ + -0.13237658143043518, + -0.3393171727657318, + 0.29588890075683594, + 0.8627858757972717, + -0.1666169911623001, + -2.520320177078247, + -1.6265922784805298, + -0.031582750380039215, + 0.11019003391265869, + -0.2699315845966339, + 0.9513242244720459, + 0.15523536503314972, + -1.7896702289581299, + -1.2776744365692139, + -0.3942914605140686, + -0.30590808391571045, + -0.3098926246166229, + 0.8258905410766602, + 0.29909753799438477, + -1.5711073875427246, + -1.5744072198867798 + ] + ], + "predicted_scores": [ + 3.636545181274414, + 3.3324379920959473, + 3.0866990089416504, + 2.9894821643829346, + 2.8972675800323486, + 2.6028425693511963, + 1.7788119316101074, + 1.3280467987060547 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "706479600/train1", + "selected_index": 0, + "state_hash": "b482f902b03ca480898798a8b5f19263d7cbb18c066305330b0796b607427fa7", + "stored_base_utility": 0.8197344541549683, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03719887137413025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03719887137413025 + }, + "base_progress": 0.03719887137413025, + "base_success": false, + "base_utility": 0.03719887137413025, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03183290362358093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03183290362358093 + }, + { + "contact_quality": null, + "progress": 0.019060388207435608, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019060388207435608 + }, + { + "contact_quality": null, + "progress": 0.1717282235622406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1717282235622406 + }, + { + "contact_quality": null, + "progress": 0.044950783252716064, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044950783252716064 + }, + { + "contact_quality": null, + "progress": 0.012144356966018677, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012144356966018677 + }, + { + "contact_quality": null, + "progress": 0.032083868980407715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.032083868980407715 + }, + { + "contact_quality": null, + "progress": 0.03983277082443237, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03983277082443237 + }, + { + "contact_quality": null, + "progress": 0.2203868180513382, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2203868180513382 + } + ], + "candidate_progress": [ + 0.03183290362358093, + 0.019060388207435608, + 0.1717282235622406, + 0.044950783252716064, + 0.012144356966018677, + 0.032083868980407715, + 0.03983277082443237, + 0.2203868180513382 + ], + "candidate_source_chart_ids": [ + "ms-0a4d4909900fb0bc4a9a", + "ms-0b1dd0e538b8df15a6e4", + "ms-ee2fcbc94d41385713d0", + "ms-bb299baa34e49eafb9fc", + "ms-07b479748a91a9c92b20", + "ms-2fef2f43af8009bb88f1", + "ms-d4c5d7eea19d94314e2e", + "ms-5ac1bf2d7d319955d45d" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank7", + "ctt_transport_rank6", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "generated_tangents": [ + [ + 0.16383254528045654, + 0.42179369926452637, + 0.16461342573165894, + -2.1338515281677246, + 0.08883903920650482, + 0.5972244739532471, + 0.10003271698951721, + -0.2376829981803894, + -0.26638832688331604, + -0.07205671817064285, + 0.9734528064727783, + 0.11627578735351562, + -1.5066206455230713, + 0.2632649540901184, + -0.2858303189277649, + -0.019674409180879593, + 0.1757156401872635, + 0.048682138323783875, + -0.12139785289764404, + -1.718052625656128, + -0.2075815498828888 + ], + [ + -0.10916204750537872, + 0.6658870577812195, + 0.12393003702163696, + -2.1848275661468506, + 0.17044630646705627, + 0.4360089898109436, + 0.26449263095855713, + -0.34011539816856384, + -0.2549479007720947, + -0.1827726662158966, + 1.1058545112609863, + -0.21791580319404602, + -1.6534463167190552, + 0.33526065945625305, + -0.292739599943161, + 0.03876979649066925, + 0.13243404030799866, + 0.24716481566429138, + -0.20260515809059143, + -2.3194258213043213, + -0.5518152713775635 + ], + [ + -0.17379315197467804, + 0.6714427471160889, + 0.12935209274291992, + -2.3667335510253906, + 0.142299085855484, + 0.7797820568084717, + 0.14695042371749878, + -0.3271960914134979, + -0.2474137544631958, + -0.12848272919654846, + 1.0801074504852295, + -0.19133707880973816, + -1.5834693908691406, + 0.47572988271713257, + -0.2721030116081238, + 0.0743202418088913, + 0.14934219419956207, + 0.2220892310142517, + -0.18336063623428345, + -2.328981399536133, + -0.32035452127456665 + ], + [ + -0.17327190935611725, + 0.648026704788208, + 0.23139452934265137, + -2.241300106048584, + 0.21182551980018616, + 0.4038086235523224, + 0.2869844436645508, + -0.2783063054084778, + -0.3269444406032562, + -0.2556919455528259, + 1.1562069654464722, + -0.09216046333312988, + -1.6788058280944824, + 0.34088200330734253, + -0.28028857707977295, + -0.09964674711227417, + 0.2195674479007721, + 0.3082142174243927, + -0.1626938283443451, + -2.2618508338928223, + -0.5362296104431152 + ], + [ + -0.14865826070308685, + 0.6718940138816833, + 0.1439131200313568, + -2.2066173553466797, + 0.1279124617576599, + 0.664729118347168, + 0.14038732647895813, + -0.2893095314502716, + -0.260088175535202, + -0.1551385521888733, + 1.1699298620224, + -0.03954972326755524, + -1.5320545434951782, + 0.28058522939682007, + -0.30633148550987244, + 0.01826954074203968, + 0.2276003062725067, + 0.2617988586425781, + -0.11420491337776184, + -2.2845611572265625, + -0.3470449149608612 + ], + [ + -0.13937751948833466, + 0.6060163378715515, + 0.1937028467655182, + -2.2179930210113525, + 0.23789609968662262, + 0.5366073846817017, + 0.35657814145088196, + -0.3390844166278839, + -0.2873803675174713, + -0.1804806888103485, + 1.081243634223938, + -0.0493759959936142, + -1.562054991722107, + 0.36232563853263855, + -0.28738126158714294, + 0.010354973375797272, + 0.2523888945579529, + 0.2718321681022644, + -0.20946475863456726, + -2.313732862472534, + -0.53669273853302 + ], + [ + -0.07790698111057281, + 0.39966386556625366, + 0.16148176789283752, + -2.022827625274658, + 0.0058796703815460205, + 0.6123775839805603, + -0.19460445642471313, + -0.18917885422706604, + -0.21152812242507935, + -0.24591153860092163, + 0.8451730608940125, + 0.22218969464302063, + -1.4132885932922363, + -0.14920301735401154, + -0.13171988725662231, + 0.02767074480652809, + 0.02690739929676056, + -0.03900305926799774, + -0.3170877695083618, + -2.0181307792663574, + -0.5251866579055786 + ], + [ + -0.19375617802143097, + 0.6010558009147644, + 0.05264982581138611, + -2.406672954559326, + 0.336072713136673, + 0.7744350433349609, + 0.5523248314857483, + -0.468593567609787, + -0.2784287929534912, + -0.22781330347061157, + 0.9399668574333191, + -0.0835561454296112, + -1.832580327987671, + 0.4588161110877991, + -0.3360155522823334, + 0.054078880697488785, + 0.1387353241443634, + 0.21493403613567352, + -0.30240166187286377, + -2.459296941757202, + -0.6023001670837402 + ] + ], + "generated_utilities": [ + 0.03183290362358093, + 0.019060388207435608, + 0.1717282235622406, + 0.044950783252716064, + 0.012144356966018677, + 0.032083868980407715, + 0.03983277082443237, + 0.2203868180513382 + ], + "group_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558897972106934, + 2.0, + 0.04248148202896118, + 0.05394089221954346, + 0.057141318917274475, + 0.03495943546295166, + 0.08633947372436523, + 0.046462297439575195, + 0.043458908796310425, + 0.023027777671813965, + 0.061922311782836914, + 0.034134700894355774, + 0.010651648044586182, + 0.01158277690410614, + 0.02622869610786438, + 0.012083128094673157 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.4298704266548157, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + 0.09995961934328079, + 1.162198781967163, + 0.0, + -0.00648990273475647, + -0.3028862476348877, + -0.0050805360078811646, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + 0.23185759782791138, + -0.11529958993196487, + -0.058992013335227966, + -0.010845988988876343, + 0.06423944234848022, + -2.0730173587799072, + 0.0 + ], + [ + -0.48041507601737976, + 0.6002404689788818, + 0.5398462414741516, + -2.198392152786255, + 0.1505453884601593, + 1.162198781967163, + 0.0, + 0.32028457522392273, + -0.264798641204834, + -0.27968189120292664, + 1.143568515777588, + 0.623285710811615, + -1.6758570671081543, + 0.0, + 0.3762620687484741, + 0.20924779772758484, + 0.033039893954992294, + -0.22763806581497192, + 0.8496624827384949, + -2.0730173587799072, + 0.0 + ], + [ + -0.4385460615158081, + 1.6002404689788818, + 1.1335551738739014, + -3.198392152786255, + 0.11579504609107971, + 2.162198781967163, + -1.8792641162872314, + 0.44874686002731323, + -1.264798641204834, + 0.08758684247732162, + 2.143568515777588, + 1.074564814567566, + -2.6758570671081543, + 1.4318934679031372, + -0.01953941211104393, + 0.012724362313747406, + 0.02134235017001629, + 0.39346790313720703, + -0.15033751726150513, + -3.0730173587799072, + -1.573193907737732 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.758528232574463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8637869358062744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.146387815475464 + ], + [ + -0.4582313299179077, + 1.465004801750183, + 0.9327377080917358, + -2.900977373123169, + -0.12230202555656433, + 2.23574161529541, + -2.1357157230377197, + 0.42294827103614807, + -1.506612777709961, + 0.29363107681274414, + 2.034971237182617, + 1.023054599761963, + -2.559215784072876, + 1.1742838621139526, + 0.026679620146751404, + -0.0811251625418663, + 0.16039283573627472, + 0.3527735471725464, + 0.041033580899238586, + -2.8834660053253174, + -1.2895927429199219 + ], + [ + -0.8261920809745789, + 1.8162921667099, + 1.1455696821212769, + -3.5306012630462646, + -0.15104833245277405, + 2.447983741760254, + -2.128190755844116, + 0.19818204641342163, + -1.146593689918518, + 0.03477605804800987, + 2.4299705028533936, + 0.8797258734703064, + -2.331338882446289, + 1.5516352653503418, + -0.19862954318523407, + -0.2736409604549408, + -0.21770769357681274, + 0.47925806045532227, + -0.15124429762363434, + -3.0052826404571533, + -1.2108670473098755 + ], + [ + -0.2760573625564575, + 1.6254181861877441, + 0.8188510537147522, + -3.0690624713897705, + -0.22635167837142944, + 2.051481246948242, + -1.417292833328247, + 0.026889145374298096, + -1.3347617387771606, + -0.15878626704216003, + 1.9778507947921753, + 0.7640928030014038, + -3.1165366172790527, + 1.862673282623291, + -0.451316773891449, + -0.15708383917808533, + 0.26080578565597534, + 0.6509285569190979, + 0.23991608619689941, + -3.50675892829895, + -1.297291874885559 + ], + [ + -0.8321853280067444, + 1.9553383588790894, + 1.5495907068252563, + -3.790785312652588, + 0.3716217279434204, + 2.5994434356689453, + -2.1380205154418945, + -0.10642725229263306, + -0.7562419772148132, + -0.442053884267807, + 2.3244006633758545, + 1.161316990852356, + -2.699451208114624, + 1.6399294137954712, + 0.31003186106681824, + 0.5301566123962402, + -0.16996291279792786, + 0.9281869530677795, + 0.3683878183364868, + -2.650294780731201, + -1.1098875999450684 + ], + [ + -0.6997023820877075, + 1.4820023775100708, + 1.3613333702087402, + -3.8556365966796875, + -0.5036565065383911, + 1.4896399974822998, + -2.322740077972412, + 0.03597790002822876, + -1.1858596801757812, + -0.018024958670139313, + 2.0489397048950195, + 0.8914362192153931, + -2.3395535945892334, + 1.1835731267929077, + 0.5559698343276978, + 0.5892979502677917, + 0.4496755301952362, + 0.6100684404373169, + 0.22263532876968384, + -3.0098915100097656, + -1.455836296081543 + ], + [ + -0.594323456287384, + 1.846432089805603, + 0.8105549812316895, + -2.6211562156677246, + 0.2629232406616211, + 1.4025108814239502, + -2.3828163146972656, + 0.530456006526947, + -1.3874722719192505, + 0.49274298548698425, + 1.5545053482055664, + 0.4927254915237427, + -3.1490466594696045, + 1.5643255710601807, + 0.5331913232803345, + -0.53307044506073, + 0.7433652281761169, + 0.6546258926391602, + -0.74479740858078, + -3.721141815185547, + -0.9948303699493408 + ], + [ + -0.4409070312976837, + 2.4138050079345703, + 1.3223544359207153, + -3.3717033863067627, + -0.03033369779586792, + 2.350304365158081, + -2.112061023712158, + -0.22452247142791748, + -1.420605182647705, + -0.026168949902057648, + 1.4026775360107422, + 0.9532885551452637, + -2.6276464462280273, + 2.1231069564819336, + -0.49286600947380066, + -0.3450928032398224, + 0.5427718758583069, + -0.03872644901275635, + -0.6895387172698975, + -2.357243061065674, + -2.3816022872924805 + ], + [ + 0.5156354308128357, + 2.2980997562408447, + 2.0945849418640137, + -3.690303087234497, + 1.0696191787719727, + 2.266188144683838, + -1.0399069786071777, + 0.012727499008178711, + -1.2163764238357544, + 0.5646419525146484, + 1.1664416790008545, + 1.563025951385498, + -3.425304412841797, + 0.5126914978027344, + -0.18885818123817444, + -0.5277795791625977, + -0.1741226613521576, + 0.2763793468475342, + 0.8411282896995544, + -3.2598915100097656, + -2.2270073890686035 + ], + [ + -1.2896747589111328, + 2.3828718662261963, + 1.191500186920166, + -3.9747776985168457, + 0.7604506015777588, + 2.298020839691162, + -1.0116384029388428, + 1.177309513092041, + -1.6719520092010498, + -0.4176469147205353, + 2.709542989730835, + 0.26378321647644043, + -2.119570255279541, + 0.7668893933296204, + 0.6772738099098206, + -0.5338667035102844, + -0.7434203028678894, + 0.2167283445596695, + 0.03269544243812561, + -3.895913600921631, + -0.9124676585197449 + ], + [ + -1.323930263519287, + 2.338547468185425, + 1.4313098192214966, + -2.960175037384033, + 1.1079072952270508, + 1.5746114253997803, + -1.8946435451507568, + -0.4213535189628601, + -1.922096848487854, + 0.6645004153251648, + 2.1448850631713867, + 0.2334994077682495, + -3.4603066444396973, + 1.5447293519973755, + 0.25094738602638245, + 0.7163193821907043, + -0.10547596216201782, + -0.42469942569732666, + -0.49689921736717224, + -2.8712027072906494, + -1.6389087438583374 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027082443237304688, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + -0.03311736881732941, + 1.162198781967163, + 0.0, + 0.043807268142700195, + -0.264798641204834, + 0.0069312527775764465, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + -0.01150114368647337, + -0.007074322551488876, + -0.018292216584086418, + -0.035625845193862915, + 0.05120308697223663, + -2.0730173587799072, + 0.0 + ] + ], + "predicted_scores": [ + 4.430130481719971, + 3.745222568511963, + 3.388216495513916, + 3.2002310752868652, + 3.1686899662017822, + 3.1340255737304688, + 3.0641114711761475, + 3.0597243309020996 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1919198188/train1", + "selected_index": 0, + "state_hash": "6b9e4c034c70f8e93116064dbcb48aa5fea37958e04fa30734f597c38aefea8b", + "stored_base_utility": 0.8558897972106934, + "task_id": "StackCube-v1", + "train_seed": "1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7270061373710632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7270061373710632 + }, + "base_progress": 0.7270061373710632, + "base_success": false, + "base_utility": 0.7270061373710632, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.725460946559906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.725460946559906 + }, + { + "contact_quality": null, + "progress": 0.7120364308357239, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7120364308357239 + }, + { + "contact_quality": null, + "progress": 0.723896324634552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.723896324634552 + }, + { + "contact_quality": null, + "progress": 0.6907892227172852, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6907892227172852 + }, + { + "contact_quality": null, + "progress": 0.7161080837249756, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7161080837249756 + }, + { + "contact_quality": null, + "progress": 0.707915723323822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.707915723323822 + }, + { + "contact_quality": null, + "progress": 0.7119319438934326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7119319438934326 + }, + { + "contact_quality": null, + "progress": 0.6006918549537659, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6006918549537659 + } + ], + "candidate_progress": [ + 0.725460946559906, + 0.7120364308357239, + 0.723896324634552, + 0.6907892227172852, + 0.7161080837249756, + 0.707915723323822, + 0.7119319438934326, + 0.6006918549537659 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-efbde8652e75274fbd69", + "ms-df961fe24a43e56d1716", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-484ea653bbca2de3a4a2", + "generated_tangents": [ + [ + -0.04396830499172211, + -0.08343648910522461, + 0.6609010100364685, + 0.3912830054759979, + 1.2738001346588135, + -0.22729763388633728, + -0.41784465312957764, + 0.12113326787948608, + -0.1098494827747345, + 0.16556432843208313, + 0.14648115634918213, + 1.5712038278579712, + -0.33531808853149414, + 0.0032642632722854614, + 0.25243619084358215, + 0.09833438694477081, + 0.2740958034992218, + -0.08344472944736481, + 1.2964844703674316, + 0.07571494579315186, + -0.11336424946784973 + ], + [ + -0.17958395183086395, + -0.09769926965236664, + 0.3501026928424835, + 0.030126169323921204, + 1.3105278015136719, + -0.3103365898132324, + -0.5545125603675842, + -0.061156630516052246, + 0.028589868918061256, + 0.07429073750972748, + 0.156553715467453, + 1.4471940994262695, + -0.296586275100708, + 0.0018916875123977661, + 0.08276046067476273, + 0.09654374420642853, + -0.014714613556861877, + -0.09888015687465668, + 1.5025501251220703, + -0.20295250415802002, + -0.008486762642860413 + ], + [ + -0.173442080616951, + -0.10092069953680038, + 0.5204766392707825, + 0.47542688250541687, + 1.328404188156128, + -0.3512614071369171, + -0.35438400506973267, + -0.12318980693817139, + -0.0074684638530015945, + 0.15452751517295837, + -0.05886925756931305, + 1.3658742904663086, + -0.009514480829238892, + 0.05464161932468414, + 0.18910977244377136, + 0.17883265018463135, + 0.16515031456947327, + -0.11293287575244904, + 1.421081781387329, + -0.26805827021598816, + -0.08112901449203491 + ], + [ + -0.2532610297203064, + -0.10047822445631027, + 0.46251213550567627, + 0.2273329794406891, + 1.3086507320404053, + -0.01985238492488861, + -0.5974724888801575, + -0.06729739904403687, + -0.006806142628192902, + 0.16290780901908875, + 0.02308567240834236, + 1.3942630290985107, + -0.2884600758552551, + -0.05897437036037445, + 0.14326632022857666, + 0.056614089757204056, + 0.13427653908729553, + -0.014556601643562317, + 1.436126470565796, + -0.11865103244781494, + -0.22375035285949707 + ], + [ + -0.2011590152978897, + -0.07051295042037964, + 0.4562249779701233, + 0.23362600803375244, + 1.2472186088562012, + -0.25279319286346436, + -0.5657718777656555, + 0.0708199143409729, + 0.03144624084234238, + 0.25414907932281494, + -0.13450783491134644, + 1.3995730876922607, + -0.28993117809295654, + -0.12428523600101471, + -0.006003204733133316, + 0.044804804027080536, + 0.05686257779598236, + 0.06919267773628235, + 1.4648306369781494, + -0.08783349394798279, + -0.191087007522583 + ], + [ + -0.19128648936748505, + -0.09508658200502396, + 0.4334104359149933, + 0.28650733828544617, + 1.3387606143951416, + -0.07222752273082733, + -0.6596245169639587, + -0.11855903267860413, + -0.004214378073811531, + 0.23318347334861755, + -0.0008311532437801361, + 1.3847993612289429, + -0.26629433035850525, + -0.08290413022041321, + 0.22168152034282684, + 0.07008364796638489, + 0.1424579918384552, + 0.05860574543476105, + 1.4799671173095703, + -0.20615199208259583, + -0.1873224675655365 + ], + [ + -0.14408321678638458, + 0.014559552073478699, + 0.25759410858154297, + 0.10650721192359924, + 1.38962984085083, + 0.3713410496711731, + -0.6855922341346741, + 0.006837889552116394, + -0.05902540683746338, + 0.07595857232809067, + -0.11012532562017441, + 1.2790130376815796, + -0.41708293557167053, + 0.016162410378456116, + 0.1911972463130951, + 0.2482377141714096, + 0.2648971676826477, + -0.3591464161872864, + 1.7145140171051025, + -0.20835676789283752, + 0.2266080379486084 + ], + [ + -0.27097994089126587, + -0.1293676495552063, + 0.2697438597679138, + 0.07886222004890442, + 1.445467472076416, + -0.12780547142028809, + -0.556123673915863, + -0.11891081929206848, + -0.056843534111976624, + 0.18678805232048035, + -0.06028033047914505, + 1.3125343322753906, + -0.36576956510543823, + -0.07183168828487396, + 0.1435416340827942, + 0.07444605231285095, + -0.03573937714099884, + 0.13283783197402954, + 1.7500474452972412, + -0.3045712411403656, + -0.04804620146751404 + ] + ], + "generated_utilities": [ + 0.725460946559906, + 0.7120364308357239, + 0.723896324634552, + 0.6907892227172852, + 0.7161080837249756, + 0.707915723323822, + 0.7119319438934326, + 0.6006918549537659 + ], + "group_id": "ms-484ea653bbca2de3a4a2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7242962718009949, + 0.7209752798080444, + 2.0, + 0.19684189558029175, + 0.21780464053153992, + 0.19326990842819214, + 0.22509369254112244, + 0.20856516063213348, + 0.21041584014892578, + 0.19898515939712524, + 0.20721960067749023, + 0.20830264687538147, + 0.19554394483566284, + 0.22549311816692352, + 0.19481703639030457, + 0.2172507494688034 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.4380870461463928, + 0.4186934530735016, + 0.6294705867767334, + -0.08003801107406616, + 0.5882754325866699, + 0.006411761045455933, + 0.0, + 0.27439412474632263, + -0.43702489137649536, + -0.26345914602279663, + -0.3102925717830658, + 1.1419475078582764, + -0.05133828520774841, + 0.0, + 0.11964668333530426, + -0.04283810034394264, + 0.021698474884033203, + -0.3213534653186798, + 1.444929838180542, + 0.058529436588287354, + 0.0 + ], + [ + 0.18277202546596527, + -0.31313952803611755, + 1.5374140739440918, + 0.9199619889259338, + 1.58827543258667, + -0.42588773369789124, + -1.307411789894104, + -0.1648276150226593, + 0.17999683320522308, + 0.7365408539772034, + -0.2825929522514343, + 2.1419475078582764, + -0.44205278158187866, + 0.3557547330856323, + 0.11787930130958557, + 0.04347073659300804, + 0.5306029915809631, + -0.6201959848403931, + 2.444929838180542, + -0.7078790068626404, + 0.7410526275634766 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.614823579788208, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7115094661712646, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4821052551269531 + ], + [ + 0.2415536344051361, + -0.2574644386768341, + 1.3600168228149414, + 0.8914169669151306, + 1.311052918434143, + -0.13483268022537231, + -1.4109176397323608, + -0.007070749998092651, + 0.30199557542800903, + 0.867316484451294, + -0.0111103355884552, + 2.228789806365967, + -0.26305240392684937, + 0.4046628475189209, + 0.25534987449645996, + 0.29946526885032654, + 0.777583122253418, + -0.535997748374939, + 2.2955121994018555, + -0.6670190095901489, + 0.8460665941238403 + ], + [ + -0.212661013007164, + -0.6653347015380859, + 1.8750245571136475, + 0.7373600006103516, + 1.2695744037628174, + -0.7584145069122314, + -1.695502758026123, + 0.05721297860145569, + -0.04310476779937744, + 0.4696756601333618, + -0.2954420745372772, + 1.9897286891937256, + -0.6554491519927979, + 0.5805841088294983, + 0.17026324570178986, + 0.3433569669723511, + 0.3734014630317688, + -0.8102566003799438, + 2.792973279953003, + -0.8463170528411865, + 1.0759955644607544 + ], + [ + -0.039009034633636475, + -0.7773740291595459, + 1.064732313156128, + 0.712108850479126, + 1.270837426185608, + -0.5491334199905396, + -1.7919384241104126, + -0.5782418251037598, + 0.4058297872543335, + 0.5231752395629883, + -0.521438479423523, + 2.2066850662231445, + -0.5258626341819763, + -0.09100466966629028, + 0.24944378435611725, + -0.2271452248096466, + 0.9009795784950256, + -0.8140718340873718, + 2.6494152545928955, + -0.5153427124023438, + 0.671539306640625 + ], + [ + -0.3804699182510376, + -0.5808351039886475, + 1.8389031887054443, + 0.7141245603561401, + 1.0997881889343262, + -0.1678517758846283, + -1.884540319442749, + -0.3411717414855957, + 0.4830858111381531, + 0.8356180787086487, + -0.8238832950592041, + 1.743321418762207, + -0.3751307427883148, + 0.5275082588195801, + -0.08622422814369202, + 0.23666565120220184, + 0.802552342414856, + -0.8389871120452881, + 1.9817423820495605, + -0.4874984323978424, + 0.4390762150287628 + ], + [ + 0.4497724771499634, + -0.7880887389183044, + 1.4436085224151611, + 1.083756446838379, + 2.2298290729522705, + -0.03008332848548889, + -1.0250177383422852, + -0.6266056299209595, + 0.3751387298107147, + 1.1566712856292725, + -0.6914000511169434, + 1.953734278678894, + 0.08129626512527466, + 0.9812837839126587, + -0.4981238543987274, + 0.6697847247123718, + -0.05150604248046875, + -0.06941354274749756, + 1.9621039628982544, + -1.345557451248169, + 0.43397778272628784 + ], + [ + 0.03950639069080353, + -1.037135124206543, + 1.1041697263717651, + 1.109439730644226, + 1.144017219543457, + -0.07355684041976929, + -0.8942177295684814, + -0.16419151425361633, + -0.12125559151172638, + 0.36935141682624817, + -0.7298048734664917, + 1.4954023361206055, + -0.018926143646240234, + 0.4787553548812866, + -0.3127076327800751, + 0.6351315379142761, + 0.5993886590003967, + -0.5143972635269165, + 2.429173469543457, + -0.6674217581748962, + 0.5048977732658386 + ], + [ + 0.9763806462287903, + -1.1549197435379028, + 2.0378684997558594, + 1.2418556213378906, + 2.427785634994507, + -1.1326193809509277, + -1.5491721630096436, + -0.8937685489654541, + 0.2583051919937134, + -0.10687285661697388, + 0.2628750801086426, + 1.6517728567123413, + -0.21639958024024963, + 0.7581590414047241, + -0.29044294357299805, + -0.08927425742149353, + 1.3009071350097656, + -0.9239004850387573, + 2.5070056915283203, + -1.3246500492095947, + 0.24751555919647217 + ], + [ + 0.9781984090805054, + -0.13759100437164307, + 1.8698725700378418, + 1.4942535161972046, + 2.5587658882141113, + -1.1335766315460205, + -0.6910277605056763, + -0.1434302181005478, + 0.4021325707435608, + -0.12275522947311401, + 0.6571680307388306, + 1.9964357614517212, + -0.1442476212978363, + 1.3201656341552734, + 0.2586786448955536, + -0.38222962617874146, + 0.9950553774833679, + -0.8371140956878662, + 1.8977141380310059, + -0.795723021030426, + 1.505234956741333 + ], + [ + -0.6250126361846924, + 0.4231366813182831, + 1.701646089553833, + 1.6233667135238647, + 1.0861127376556396, + -0.3387698233127594, + -0.9825164079666138, + -1.1511213779449463, + 0.11523068696260452, + 1.5823121070861816, + 0.5840664505958557, + 2.145885705947876, + -1.2145874500274658, + 1.1558246612548828, + -0.5131542682647705, + -0.4715161919593811, + 1.0824741125106812, + -0.7667124271392822, + 1.8457963466644287, + -1.228658676147461, + 1.2112116813659668 + ], + [ + 1.1640653610229492, + -1.1697030067443848, + 1.6035369634628296, + 0.2161661982536316, + 1.4819366931915283, + 0.2569611370563507, + -1.3529595136642456, + 0.41208329796791077, + 0.03121529519557953, + 1.731370210647583, + -0.8131548166275024, + 2.996675729751587, + -1.1173620223999023, + 0.7474988102912903, + 0.21459203958511353, + -0.2133086621761322, + -0.22307389974594116, + -0.6306187510490417, + 2.0490896701812744, + -0.49808698892593384, + 0.8432145714759827 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.014628380537033081, + -0.18207009136676788, + 0.5374140739440918, + -0.04374772310256958, + 0.5882754325866699, + 0.25251278281211853, + 0.0, + -0.23516394197940826, + 0.12562739849090576, + 0.3381730020046234, + 0.033050984144210815, + 1.1419475078582764, + -0.3290551006793976, + 0.0, + 0.04625481367111206, + -0.0026901662349700928, + 0.0754578709602356, + -0.1481514573097229, + 1.444929838180542, + 0.049897611141204834, + 0.0 + ] + ], + "predicted_scores": [ + 6.357418537139893, + 5.67083740234375, + 5.565855503082275, + 5.436758041381836, + 5.376450538635254, + 5.213481426239014, + 5.070798397064209, + 4.92073392868042 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1231780997/train2", + "selected_index": 0, + "state_hash": "771fb752a186163f05a49a9b404284bde55d89838769a764fca1d6d1a9419652", + "stored_base_utility": 0.7242962718009949, + "task_id": "LiftPegUpright-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.528373658657074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.528373658657074 + }, + "base_progress": 0.528373658657074, + "base_success": false, + "base_utility": 0.528373658657074, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.6627864837646484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6627864837646484 + }, + { + "contact_quality": null, + "progress": 0.7146708369255066, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7146708369255066 + }, + { + "contact_quality": null, + "progress": 0.2057957649230957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2057957649230957 + }, + { + "contact_quality": null, + "progress": 0.20426128804683685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20426128804683685 + }, + { + "contact_quality": null, + "progress": 0.7040143609046936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7040143609046936 + }, + { + "contact_quality": null, + "progress": 0.1966126710176468, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1966126710176468 + }, + { + "contact_quality": null, + "progress": 0.6997706890106201, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6997706890106201 + }, + { + "contact_quality": null, + "progress": 0.6655330061912537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6655330061912537 + } + ], + "candidate_progress": [ + 0.6627864837646484, + 0.7146708369255066, + 0.2057957649230957, + 0.20426128804683685, + 0.7040143609046936, + 0.1966126710176468, + 0.6997706890106201, + 0.6655330061912537 + ], + "candidate_source_chart_ids": [ + "ms-8f3d4e1d07bb5dcc7c57", + "ms-df961fe24a43e56d1716", + "ms-e1b6bcb41532a800d3cb", + "ms-0660d5f65df5fc4b6132", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb", + "ms-13d8bae55d3c3710138f", + "ms-502dba1003c09ec60ba0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank6" + ], + "candidates_evaluated": true, + "chart_id": "ms-4b4cd12d70c0615e0930", + "generated_tangents": [ + [ + -0.300254762172699, + 0.053849607706069946, + 0.7199182510375977, + 0.5696301460266113, + 0.37969574332237244, + 0.132772296667099, + 0.06084561347961426, + 0.275797963142395, + 0.015088420361280441, + -0.04985090345144272, + -0.09546753019094467, + 1.8002060651779175, + -0.07629215717315674, + 0.005778871476650238, + -0.0030481629073619843, + 0.04588718339800835, + 0.15246544778347015, + -0.14120404422283173, + 1.568716287612915, + -0.19701126217842102, + -0.03143902122974396 + ], + [ + -0.1923649162054062, + 0.05242970585823059, + 0.8130223751068115, + 0.5171482563018799, + 0.28202661871910095, + -0.2814216613769531, + 0.06353339552879333, + 0.1968238651752472, + -0.02700730599462986, + -0.2716432511806488, + -0.1141243502497673, + 1.760290503501892, + -0.165985107421875, + 0.12487167119979858, + 0.08824179321527481, + 0.19057968258857727, + 0.31001168489456177, + -0.1510704606771469, + 1.5847454071044922, + -0.10264299809932709, + -0.22121229767799377 + ], + [ + 0.02517171949148178, + 0.022989556193351746, + 1.021344780921936, + 0.5466411113739014, + 0.21092377603054047, + -0.07494795322418213, + -0.29042571783065796, + 0.4393842816352844, + -0.0019921306520700455, + -0.1441338062286377, + 0.04217243567109108, + 1.7934508323669434, + -0.4576164186000824, + 0.17146851122379303, + 0.11672595888376236, + 0.024392124265432358, + 0.1892388015985489, + -0.156822070479393, + 1.6405470371246338, + -0.2620435059070587, + 0.035926759243011475 + ], + [ + -0.20293591916561127, + 0.054996930062770844, + 1.0815178155899048, + 0.7565701007843018, + 0.39579933881759644, + -0.12166270613670349, + 0.010693728923797607, + 0.2836955189704895, + 0.033144909888505936, + -0.19373682141304016, + -0.02088361233472824, + 1.722155213356018, + -0.22261345386505127, + -0.040947794914245605, + 0.22898752987384796, + 0.12912122905254364, + 0.2327134609222412, + -0.19339342415332794, + 1.43467116355896, + -0.09849818050861359, + -0.06645791232585907 + ], + [ + -0.33093082904815674, + 0.08323894441127777, + 0.7557040452957153, + 0.6216579675674438, + 0.2939087450504303, + -0.051970675587654114, + 0.17446112632751465, + 0.19948093593120575, + -0.029680700972676277, + -0.03263171762228012, + -0.1835663914680481, + 1.7952388525009155, + -0.47616440057754517, + 0.1938922107219696, + 0.18788287043571472, + 0.19288389384746552, + 0.16428688168525696, + -0.15009815990924835, + 1.4842219352722168, + -0.1300111562013626, + -0.18514320254325867 + ], + [ + 0.09189243614673615, + -0.051281724125146866, + 1.1037657260894775, + 0.6742314100265503, + 0.07488898932933807, + 0.19092682003974915, + -0.10823947191238403, + 0.3776037096977234, + 0.13911443948745728, + -0.05855753272771835, + 0.10154823958873749, + 1.7754346132278442, + -0.15086999535560608, + 0.20505103468894958, + 0.2659764289855957, + 0.06166105344891548, + 0.1542041003704071, + -0.23311443626880646, + 1.5005040168762207, + 0.006253182888031006, + -0.16563260555267334 + ], + [ + -0.12978534400463104, + 0.07043270766735077, + 1.0013962984085083, + 0.5269284248352051, + 0.25870266556739807, + -0.24719852209091187, + 0.04909998178482056, + 0.234236478805542, + 0.008548240177333355, + -0.20292186737060547, + -0.09823883324861526, + 1.694366455078125, + -0.059740275144577026, + -0.0781109631061554, + 0.02889179438352585, + 0.0746571272611618, + 0.07848958671092987, + -0.18098969757556915, + 1.4529037475585938, + -0.02616429328918457, + -0.2298535704612732 + ], + [ + -0.16542251408100128, + 0.1198706179857254, + 0.6873865127563477, + 0.6309127807617188, + 0.1625741422176361, + -0.0017057806253433228, + -0.25552499294281006, + 0.14617007970809937, + 0.057443417608737946, + -0.007251046597957611, + -0.1327020525932312, + 1.8942509889602661, + -0.1863330900669098, + -0.14612554013729095, + 0.06294852495193481, + 0.042082853615283966, + 0.18045394122600555, + -0.08017493784427643, + 1.432365894317627, + -0.16892123222351074, + -0.009519800543785095 + ] + ], + "generated_utilities": [ + 0.6627864837646484, + 0.7146708369255066, + 0.2057957649230957, + 0.20426128804683685, + 0.7040143609046936, + 0.1966126710176468, + 0.6997706890106201, + 0.6655330061912537 + ], + "group_id": "ms-4b4cd12d70c0615e0930", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7225138545036316, + 2.0, + 0.6951258182525635, + 0.18915215134620667, + 0.20867900550365448, + 0.20041285455226898, + 0.20335882902145386, + 0.20815956592559814, + 0.2024221420288086, + 0.1940193623304367, + 0.20689067244529724, + 0.19995209574699402, + 0.21901433169841766, + 0.19443893432617188, + 0.20502614974975586, + 0.1959807276725769 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.09018868207931519, + 0.3819495439529419, + 0.9586174488067627, + 1.0247726440429688, + -0.179018035531044, + 0.125870943069458, + 0.0, + -0.1166897565126419, + 0.07711493968963623, + -0.24461698532104492, + -0.005436837673187256, + 1.737370491027832, + -0.4425608515739441, + 0.0, + 0.24725346267223358, + -0.03395397961139679, + 0.643236517906189, + 0.05422210693359375, + 1.2738127708435059, + 0.18164724111557007, + 0.0 + ], + [ + 0.6095315217971802, + 0.35729360580444336, + 1.9586174488067627, + 2.0247726440429688, + -0.2820541560649872, + 1.044631004333496, + -2.239265203475952, + -0.14250506460666656, + -0.4363628029823303, + -1.244616985321045, + -0.32413747906684875, + 2.737370491027832, + 0.24581307172775269, + -0.053983256220817566, + 0.2877395749092102, + 0.18538890779018402, + 0.45247530937194824, + -0.9457778930664062, + 2.273812770843506, + -0.8183527588844299, + 0.5866778492927551 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -4.478530406951904, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.10796651244163513, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1733556985855103 + ], + [ + 0.8415398597717285, + 0.17323988676071167, + 1.9669679403305054, + 2.188286304473877, + -0.386857271194458, + 1.3415166139602661, + -2.06968355178833, + -0.38555508852005005, + -0.6971946954727173, + -1.0110756158828735, + -0.19943451881408691, + 3.0229673385620117, + 0.03575621545314789, + -0.26961493492126465, + 0.5202459692955017, + 0.0921676978468895, + 0.7439858913421631, + -0.724668562412262, + 2.116668939590454, + -1.0585715770721436, + 0.7851569652557373 + ], + [ + 0.6909375190734863, + -0.018118590116500854, + 1.677350401878357, + 1.9505902528762817, + 0.11725521087646484, + 1.2839276790618896, + -2.0543243885040283, + -0.38843441009521484, + -0.6865814328193665, + -1.5103397369384766, + -0.2952713072299957, + 2.384809970855713, + -0.10558199882507324, + 0.29018354415893555, + 0.33942633867263794, + -0.19196103513240814, + 0.35257238149642944, + -0.8657878637313843, + 2.5445473194122314, + -0.706183910369873, + 0.7214455008506775 + ], + [ + 0.8977934122085571, + 0.7311913371086121, + 2.203964948654175, + 1.686698079109192, + -0.338996022939682, + 0.631161630153656, + -2.3877103328704834, + 0.07195566594600677, + -0.6892462968826294, + -0.9634823799133301, + -0.5621621012687683, + 2.479537010192871, + 0.675940752029419, + -0.047649919986724854, + 0.14082925021648407, + 0.19517391920089722, + 0.15626585483551025, + -1.3974462747573853, + 2.759931802749634, + -1.166107416152954, + 0.18018442392349243 + ], + [ + 0.7796773910522461, + 0.06440207362174988, + 1.5159449577331543, + 1.6007665395736694, + -0.12459242343902588, + 1.5781673192977905, + -2.5707991123199463, + 0.034798189997673035, + -0.44772639870643616, + -1.3837177753448486, + -0.5199699401855469, + 3.1566030979156494, + -0.22805768251419067, + -0.5881848931312561, + 0.49021050333976746, + -0.08376060426235199, + 0.5395568609237671, + -1.3038995265960693, + 2.37483811378479, + -0.615712583065033, + 0.7884383201599121 + ], + [ + 0.4830939471721649, + -0.12824741005897522, + 2.133068561553955, + 2.42085337638855, + -0.1641906201839447, + 1.6598048210144043, + -2.5325045585632324, + 0.06623834371566772, + 0.14292877912521362, + -1.85994553565979, + 0.01158285140991211, + 3.2622194290161133, + -0.017848491668701172, + -0.08653604984283447, + 0.2199593186378479, + -0.0463244765996933, + -0.04547742009162903, + -0.5160569548606873, + 2.1955723762512207, + -0.364714115858078, + 1.0323584079742432 + ], + [ + 0.7637265920639038, + -0.4020084738731384, + 2.4714059829711914, + 1.33060622215271, + 0.1464400589466095, + 0.7656213045120239, + -1.9972842931747437, + 0.42333751916885376, + -1.118815302848816, + -1.5639967918395996, + -0.32944348454475403, + 2.0334420204162598, + 0.37165334820747375, + -0.2861292064189911, + 0.6376298666000366, + 0.8521975874900818, + 0.6845490336418152, + -1.515796422958374, + 1.5925071239471436, + -0.4112323224544525, + 1.1011840105056763 + ], + [ + 1.456933617591858, + 1.0030901432037354, + 1.1997551918029785, + 1.7652757167816162, + 0.03292572498321533, + 1.302201271057129, + -2.748408317565918, + 0.7148768305778503, + -0.5089412331581116, + -0.5701506733894348, + -0.40410247445106506, + 3.586395502090454, + -0.37424683570861816, + -0.6850852966308594, + 0.9945629835128784, + 0.5821471810340881, + 0.8666700124740601, + -1.1607441902160645, + 2.1460094451904297, + -0.7912135720252991, + 0.6301633715629578 + ], + [ + -0.09475094079971313, + -0.28330767154693604, + 1.8353716135025024, + 2.3373594284057617, + 0.7110240459442139, + 1.9665697813034058, + -1.8502212762832642, + -0.40577346086502075, + -0.03506562113761902, + -0.7167142033576965, + -0.39153769612312317, + 2.3472883701324463, + 0.08497899770736694, + -0.9910531640052795, + -0.32254326343536377, + -0.11388416588306427, + -0.4873260259628296, + -0.8018304109573364, + 2.295731782913208, + -1.325164794921875, + 1.2513983249664307 + ], + [ + -0.32657402753829956, + 0.8198520541191101, + 1.211392879486084, + 1.974273443222046, + -0.11595214903354645, + 1.954041838645935, + -1.5547080039978027, + 0.06433123350143433, + -0.079216867685318, + -0.5887003540992737, + -0.6015424728393555, + 2.756274938583374, + 0.641853928565979, + -0.8642687201499939, + 0.07965350151062012, + 0.11391039192676544, + 1.10663902759552, + -1.506749153137207, + 3.222418785095215, + -1.0799858570098877, + -0.12158209085464478 + ], + [ + 0.9412136077880859, + -0.5255860090255737, + 1.9096710681915283, + 2.113077402114868, + -1.0082788467407227, + 0.1505374312400818, + -3.0413341522216797, + -0.24721917510032654, + -0.20225940644741058, + -1.6247544288635254, + -0.33320704102516174, + 1.8443769216537476, + 0.7569764256477356, + -0.5270920395851135, + 1.1813421249389648, + 1.0764858722686768, + 0.01533237099647522, + -1.4125463962554932, + 1.82404363155365, + 0.054037630558013916, + 0.45250433683395386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.056152522563934326, + 0.005679905414581299, + 0.9586174488067627, + 1.0247726440429688, + -0.054595887660980225, + 0.044631004333496094, + 0.0, + -0.09190673381090164, + 0.09592300653457642, + -0.24461698532104492, + -0.007907181978225708, + 1.737370491027832, + -0.029382914304733276, + 0.0, + 0.06305502355098724, + 0.03764393925666809, + 0.0785532295703888, + 0.005711078643798828, + 1.2738127708435059, + -0.04058480262756348, + 0.0 + ] + ], + "predicted_scores": [ + 4.831881999969482, + 4.637731552124023, + 4.629426956176758, + 4.454403400421143, + 4.3109450340271, + 4.067284107208252, + 3.851375102996826, + 3.8106584548950195 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1025740212/train2", + "selected_index": 0, + "state_hash": "5b394fca1712c0667d126198527a4f0ba81893d536a5fcbf7b90f5b13fa703dc", + "stored_base_utility": 0.7225138545036316, + "task_id": "LiftPegUpright-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7040655612945557, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7040655612945557 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7247136235237122, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7247136235237122 + }, + { + "contact_quality": null, + "progress": 0.717224657535553, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.717224657535553 + }, + { + "contact_quality": null, + "progress": 0.7159295678138733, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7159295678138733 + }, + { + "contact_quality": null, + "progress": 0.40124979615211487, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.40124979615211487 + } + ], + "candidate_progress": [ + 0.7040655612945557, + 1.0, + 1.0, + 1.0, + 0.7247136235237122, + 0.717224657535553, + 0.7159295678138733, + 0.40124979615211487 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6e268d1e2915740816a1", + "generated_tangents": [ + [ + 0.026772454380989075, + -0.1296411007642746, + 0.3895333707332611, + 0.1391119360923767, + 1.9083693027496338, + -0.08746890723705292, + 0.11865496635437012, + 0.18726663291454315, + -0.15920095145702362, + 0.10431478917598724, + 0.06600998342037201, + 1.333777904510498, + -0.43381696939468384, + 0.159135103225708, + 0.17815954983234406, + 0.10608826577663422, + 0.3377605974674225, + -0.08688642084598541, + 0.8666390180587769, + -0.21693208813667297, + -0.4241900146007538 + ], + [ + -0.1682554930448532, + -0.12048175930976868, + 0.19391068816184998, + -0.04517236351966858, + 1.7990427017211914, + 0.08498160541057587, + 0.16223889589309692, + 0.02773110568523407, + -0.09519593417644501, + 0.006628185510635376, + 0.005198635160923004, + 1.3731099367141724, + -0.032558292150497437, + 0.06638997048139572, + 0.04230092465877533, + 0.09292496740818024, + 0.1456332504749298, + -0.1668664664030075, + 1.0234153270721436, + 0.023253291845321655, + -0.09180307388305664 + ], + [ + -0.15021051466464996, + -0.16460613906383514, + 0.22778600454330444, + -0.052551835775375366, + 1.9887595176696777, + -0.11002078652381897, + 0.09651431441307068, + 0.08139576762914658, + -0.059139251708984375, + 0.1783117949962616, + -0.09237655252218246, + 1.2773078680038452, + -0.20209002494812012, + 0.15842197835445404, + -0.1090216189622879, + 0.10600344836711884, + 0.19155451655387878, + -0.016486629843711853, + 1.0678369998931885, + -0.3012024760246277, + -0.1768263578414917 + ], + [ + -0.27181482315063477, + -0.1355568915605545, + 0.38721534609794617, + -0.14984719455242157, + 1.761793851852417, + -0.27559980750083923, + 0.09324458241462708, + 0.09733743965625763, + -0.02516706846654415, + 0.24463962018489838, + -0.013168446719646454, + 1.2979891300201416, + -0.6586089730262756, + 0.2973225712776184, + -0.01377873495221138, + 0.136238694190979, + 0.17095285654067993, + -0.0614003986120224, + 1.0419929027557373, + -0.18981415033340454, + -0.013761118054389954 + ], + [ + -0.18508736789226532, + -0.12962208688259125, + 0.25904661417007446, + -0.10958153009414673, + 1.9822971820831299, + 0.016127049922943115, + 0.0933694839477539, + -0.022003725171089172, + -0.0713600441813469, + 0.07369425147771835, + -0.03565383702516556, + 1.3851419687271118, + -0.3443012833595276, + 0.1290782243013382, + 0.023380320519208908, + 0.11314325034618378, + 0.2110779583454132, + -0.08936159312725067, + 1.0043904781341553, + -0.2254841923713684, + -0.17989367246627808 + ], + [ + -0.11984036862850189, + -0.15093642473220825, + 0.0017314553260803223, + -0.21155759692192078, + 2.0077171325683594, + -0.21363839507102966, + 0.008436620235443115, + -0.030026763677597046, + -0.03960853070020676, + 0.19732311367988586, + -0.1784546971321106, + 1.2282062768936157, + -0.1912350058555603, + 0.11905762553215027, + -0.0025265179574489594, + 0.05166539177298546, + 0.2459152638912201, + -0.14281965792179108, + 1.1461100578308105, + -0.2580501139163971, + -0.030434131622314453 + ], + [ + -0.14233838021755219, + -0.11305731534957886, + 0.23599496483802795, + -0.03712493181228638, + 2.0280590057373047, + 0.026946067810058594, + 0.008028149604797363, + -0.08196964859962463, + -0.054160602390766144, + 0.12734714150428772, + -0.08615278452634811, + 1.3986033201217651, + -0.39400535821914673, + 0.09593387693166733, + 0.05878238007426262, + 0.11810895800590515, + 0.2561825215816498, + -0.027173668146133423, + 1.033850908279419, + -0.2784793972969055, + -0.1574062705039978 + ], + [ + -0.3736599087715149, + -0.12925270199775696, + 0.10042649507522583, + -0.15230914950370789, + 1.8361225128173828, + 0.1269616186618805, + -0.20964395999908447, + -0.18080934882164001, + 0.034028373658657074, + 0.06390392780303955, + -0.17515945434570312, + 1.378110647201538, + -0.4343094527721405, + 0.20561745762825012, + -0.1421905905008316, + 0.05440641567111015, + 0.0006457120180130005, + -0.36248528957366943, + 1.0916004180908203, + -0.424134224653244, + -0.06055821478366852 + ] + ], + "generated_utilities": [ + 0.7040655612945557, + 2.0, + 2.0, + 2.0, + 0.7247136235237122, + 0.717224657535553, + 0.7159295678138733, + 0.40124979615211487 + ], + "group_id": "ms-6e268d1e2915740816a1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7257392406463623, + 0.7165254950523376, + 2.0, + 0.72465980052948, + 0.7087006568908691, + 0.2306610643863678, + 0.7143489122390747, + 0.7045385837554932, + 0.5905889272689819, + 0.1919487714767456, + 0.1914421170949936, + 0.2230607271194458, + 0.20312398672103882, + 0.19757536053657532, + 0.19724789261817932, + 0.19653666019439697 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.44810131192207336, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0264720916748047, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.9882198572158813 + ], + [ + -0.4707597494125366, + 0.09937268495559692, + 0.6395220160484314, + -0.40935325622558594, + 2.4514670372009277, + -0.02397046983242035, + -0.21874001622200012, + 0.16799744963645935, + 0.0360843725502491, + -0.09719255566596985, + -0.8619975447654724, + 2.2499821186065674, + -0.4933375120162964, + 0.35259634256362915, + -0.34735018014907837, + -0.517992377281189, + 0.6015943288803101, + -0.7677568793296814, + 1.5267367362976074, + -0.16888000071048737, + 0.3771851658821106 + ], + [ + -0.8530820608139038, + -0.6707327961921692, + 0.7970488667488098, + 0.34902632236480713, + 2.1393990516662598, + 0.5651285648345947, + 0.113044872879982, + 0.2987540364265442, + 0.4139185845851898, + 0.41691792011260986, + -0.4906405806541443, + 1.5687955617904663, + -0.8899085521697998, + -0.01883232593536377, + 0.24473188817501068, + 0.554391622543335, + 0.23424841463565826, + -0.10042615234851837, + 2.1214542388916016, + -0.15780393779277802, + 0.5807479023933411 + ], + [ + -0.6562768220901489, + -0.059059903025627136, + 0.24766850471496582, + 0.06101328134536743, + 2.5052053928375244, + 0.5204365849494934, + 0.1595638245344162, + 0.47176462411880493, + 0.3036114573478699, + 0.13197286427021027, + -0.2551935017108917, + 2.185537099838257, + -1.0984127521514893, + 0.3213496804237366, + 0.49875718355178833, + 0.1223822832107544, + 0.2575722932815552, + -0.7017092704772949, + 1.7495591640472412, + -0.19501343369483948, + 1.1214004755020142 + ], + [ + -0.45427173376083374, + -0.041353583335876465, + 0.8143320083618164, + -0.39680930972099304, + 2.4542014598846436, + 0.382559597492218, + -0.9993512630462646, + 0.5202391147613525, + 0.5282149314880371, + 0.4165073335170746, + 0.43153735995292664, + 2.5625123977661133, + -0.721943199634552, + 0.6765530109405518, + -0.6697012186050415, + -0.0637422502040863, + 0.6919896602630615, + -1.033813714981079, + 2.317282199859619, + -0.13503669202327728, + -0.07009828090667725 + ], + [ + -0.548885703086853, + -0.609938383102417, + 0.7181543707847595, + 0.11203774809837341, + 2.478057861328125, + -0.6170704960823059, + -1.0383654832839966, + 0.3524467349052429, + 0.5748242139816284, + 0.42987847328186035, + 0.013119548559188843, + 1.3954555988311768, + -0.6403790712356567, + 0.025414258241653442, + -0.7780460119247437, + -0.23124903440475464, + 0.48347827792167664, + 0.05266502499580383, + 1.692553997039795, + 0.362108051776886, + 0.6330842971801758 + ], + [ + -1.3070502281188965, + -0.947432816028595, + 0.4198402762413025, + -1.1768442392349243, + 1.7843327522277832, + 0.7587301731109619, + 0.7447960376739502, + 0.9411643147468567, + 0.3013244867324829, + -0.3862171769142151, + -1.0263384580612183, + 2.97957181930542, + 0.1849566102027893, + 0.08799406886100769, + 0.1984272301197052, + 0.5277765393257141, + 0.2931878864765167, + -1.1385071277618408, + 1.884846568107605, + 0.23083388805389404, + 0.6359583139419556 + ], + [ + -0.5375385880470276, + 0.16159841418266296, + 1.260543942451477, + -0.6744770407676697, + 2.1646244525909424, + 0.6190755367279053, + -0.746461808681488, + -0.38674360513687134, + 0.8897944092750549, + -0.594306468963623, + -0.05647379159927368, + 1.7390328645706177, + -0.037059009075164795, + 0.470071941614151, + 0.010003047063946724, + 0.08421775698661804, + -0.4613104462623596, + -0.22854508459568024, + 1.772307276725769, + -0.29165884852409363, + 0.6179300546646118 + ], + [ + -0.6489957571029663, + 0.4526708126068115, + 0.5115790367126465, + 0.31459903717041016, + 2.9655239582061768, + 0.07586031407117844, + -1.1667817831039429, + -0.7146965861320496, + 1.0295621156692505, + 0.1932714432477951, + -0.2703063488006592, + 2.9046523571014404, + -1.5882594585418701, + 0.3030351996421814, + 0.6543201208114624, + -0.448961466550827, + 0.4115087389945984, + -0.4087216854095459, + 2.6432881355285645, + -0.6797424554824829, + 0.2669999599456787 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.22510427236557007, + -0.26737168431282043, + 0.10375632345676422, + 0.12666502594947815, + 1.379244327545166, + -0.01088210940361023, + 0.0, + 0.23721180856227875, + -0.10981421172618866, + -0.021816372871398926, + -0.071512371301651, + 1.0360438823699951, + 0.09019589424133301, + 0.0, + 0.03181244805455208, + -0.06466590613126755, + 0.2068846970796585, + 0.14229199290275574, + 0.7610219717025757, + -0.03211131691932678, + 0.0 + ] + ], + "predicted_scores": [ + 6.741085052490234, + 6.448919773101807, + 6.381951332092285, + 6.351556777954102, + 6.2756218910217285, + 6.065993785858154, + 5.817976474761963, + 5.585904598236084 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1475440973/train2", + "selected_index": 0, + "state_hash": "38ff5a264c3c39ff1ec8756d84e622963a11ff1ac61568ba57fbea0c114c3140", + "stored_base_utility": 0.7257392406463623, + "task_id": "LiftPegUpright-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7239725589752197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7239725589752197 + }, + "base_progress": 0.7239725589752197, + "base_success": false, + "base_utility": 0.7239725589752197, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.712515652179718, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.712515652179718 + }, + { + "contact_quality": null, + "progress": 0.7240610718727112, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7240610718727112 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7171879410743713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7171879410743713 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.712515652179718, + 0.7240610718727112, + 1.0, + 1.0, + 1.0, + 1.0, + 0.7171879410743713, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-dd32ce9859c365c535b4", + "ms-dfc38c43e9a87f99ef7f", + "ms-dfc38c43e9a87f99ef7f", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8864796e31bbe93d7c5d", + "generated_tangents": [ + [ + -0.10783638060092926, + -0.09325660765171051, + 0.2980384826660156, + -0.33395111560821533, + 2.0316874980926514, + -0.6613283157348633, + 0.13582468032836914, + 0.30387037992477417, + -0.18794894218444824, + 0.021270275115966797, + 0.05030153691768646, + 1.67851984500885, + -0.704618513584137, + 0.8027870059013367, + 0.21700423955917358, + 0.010395519435405731, + 0.34987202286720276, + -0.39984625577926636, + 1.33915376663208, + -0.44861865043640137, + 0.2573627531528473 + ], + [ + -0.012575432658195496, + -0.15628373622894287, + 0.14575940370559692, + -0.3477315604686737, + 2.1024715900421143, + -0.21605506539344788, + 0.3728860020637512, + 0.04174472391605377, + -0.1807873398065567, + 0.2590184509754181, + 0.023569080978631973, + 1.4081878662109375, + -0.41216424107551575, + 0.3406746983528137, + -0.03466382250189781, + 0.05362289026379585, + 0.3251451551914215, + -0.2605082392692566, + 1.3668866157531738, + -0.5721875429153442, + 0.18283721804618835 + ], + [ + 0.12038832157850266, + -0.07027330249547958, + 0.09921270608901978, + -0.39877015352249146, + 2.0046935081481934, + -0.3416796624660492, + 0.400138795375824, + -0.002096042037010193, + -0.13842645287513733, + 0.07764597237110138, + -0.0947338119149208, + 1.4713890552520752, + -0.31413087248802185, + 0.4171268343925476, + 0.0804351419210434, + -0.03479379415512085, + 0.3605607748031616, + -0.21419651806354523, + 1.3085267543792725, + -0.5157297849655151, + 0.22035014629364014 + ], + [ + 0.012208536267280579, + -0.017652850598096848, + 0.2961108684539795, + -0.2688570022583008, + 1.965496301651001, + -0.2678772807121277, + 0.1856738030910492, + -0.002307325601577759, + -0.19012604653835297, + 0.00589403510093689, + -0.03486165404319763, + 1.8909276723861694, + -0.4045448899269104, + 0.5490383505821228, + 0.14483818411827087, + 0.06391429901123047, + 0.4477684497833252, + -0.2726069688796997, + 1.3530051708221436, + -0.508605420589447, + -0.12967249751091003 + ], + [ + -0.005118951201438904, + -0.008849266916513443, + 0.2604236602783203, + -0.19897401332855225, + 1.9821579456329346, + -0.406417578458786, + 0.2835600972175598, + 0.15547087788581848, + -0.18190845847129822, + 0.10572440922260284, + -0.11708914488554001, + 1.7370109558105469, + -0.3137834668159485, + 0.4488229751586914, + -0.06477105617523193, + -0.006240908522158861, + 0.39373844861984253, + -0.20722119510173798, + 1.36594557762146, + -0.561301052570343, + -0.1171288788318634 + ], + [ + -0.11272348463535309, + -0.02308821864426136, + -0.028663665056228638, + -0.248068705201149, + 1.9404525756835938, + -0.24219614267349243, + 0.588707685470581, + -0.0058912187814712524, + -0.1755552589893341, + -0.00034246593713760376, + -0.04061531275510788, + 1.8202155828475952, + -0.2156980037689209, + 0.37749236822128296, + 0.08020981401205063, + 0.024505749344825745, + 0.2673306465148926, + -0.2767093777656555, + 1.2514865398406982, + -0.3545854687690735, + 0.15575112402439117 + ], + [ + -0.290314257144928, + -0.09416916966438293, + 0.040746986865997314, + -0.402606338262558, + 1.936187744140625, + -0.27139273285865784, + 0.023540079593658447, + -0.05686098337173462, + -0.09082868695259094, + -0.010172031819820404, + -0.06936090439558029, + 1.7161526679992676, + -0.6132203936576843, + 0.6554793119430542, + -0.07010607421398163, + -0.0399882048368454, + 0.30135107040405273, + -0.5384387969970703, + 1.3447628021240234, + -0.642231822013855, + 0.1492292732000351 + ], + [ + 0.03728649765253067, + 0.0138520747423172, + 0.28487300872802734, + -0.2022971510887146, + 2.0122439861297607, + -0.30780452489852905, + 0.18874788284301758, + -0.010944128036499023, + -0.19370464980602264, + 0.053227126598358154, + -0.09578505903482437, + 1.8564132452011108, + -0.3929198086261749, + 0.5121991038322449, + 0.1994069665670395, + 0.054055098444223404, + 0.4637346863746643, + -0.22313715517520905, + 1.37990140914917, + -0.5531796216964722, + -0.16263440251350403 + ] + ], + "generated_utilities": [ + 0.712515652179718, + 0.7240610718727112, + 2.0, + 2.0, + 2.0, + 2.0, + 0.7171879410743713, + 2.0 + ], + "group_id": "ms-8864796e31bbe93d7c5d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.728268027305603, + 0.7257949113845825, + 2.0, + 0.7287554740905762, + 0.72548508644104, + 0.7011849880218506, + 0.7089968919754028, + 0.689879298210144, + 0.722068727016449, + 0.7046384811401367, + 0.1933039277791977, + 0.629345178604126, + 0.18965840339660645, + 0.18740439414978027, + 0.2032846063375473, + 0.20379847288131714 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.3859826922416687, + 0.1119421124458313, + 0.6443181037902832, + 0.11836245656013489, + 1.863237738609314, + -1.166780948638916, + 0.8025261163711548, + -0.5342166423797607, + -0.2057022899389267, + 0.8987585306167603, + -0.525823712348938, + 1.6212486028671265, + 0.25674968957901, + 0.4037417471408844, + -0.4561460018157959, + -0.18429945409297943, + -0.1778680682182312, + -0.22491087019443512, + 2.095893144607544, + -0.3484030067920685, + 0.47628670930862427 + ], + [ + -0.446367084980011, + -0.19229409098625183, + -0.03271818161010742, + 0.39123907685279846, + 1.1786221265792847, + -0.16883885860443115, + -0.3337137699127197, + 0.7789723873138428, + -0.12953178584575653, + -0.028340458869934082, + -0.871502161026001, + 0.7702060341835022, + -0.645424485206604, + 1.0858956575393677, + -0.05413459241390228, + -0.14592209458351135, + -0.03685358166694641, + -0.686826229095459, + 2.084512710571289, + 0.11898508667945862, + 0.20845119655132294 + ], + [ + 0.09377387166023254, + -0.03508002683520317, + -0.29283326864242554, + -0.9559528827667236, + 2.0994086265563965, + 0.02743762731552124, + 0.6819101572036743, + 0.607413113117218, + 0.4378519356250763, + 0.6497513055801392, + -0.5086554288864136, + 0.9556822180747986, + -0.8518581986427307, + 0.31636691093444824, + -0.4196639060974121, + 0.6866374015808105, + 0.5175505876541138, + -0.572692334651947, + 1.651871681213379, + -0.7227057814598083, + -0.33861052989959717 + ], + [ + -0.42610836029052734, + 0.10144555568695068, + 0.7105271220207214, + -1.291037678718567, + 2.2307684421539307, + 0.17725014686584473, + -0.08436906337738037, + 0.9681323766708374, + 0.5482062697410583, + 1.368379831314087, + -1.021532416343689, + 2.5197110176086426, + -0.3508763909339905, + 0.7584415078163147, + 1.0978648662567139, + 0.2626119554042816, + 0.6115584373474121, + -0.7777479290962219, + 1.7278802394866943, + -0.15240566432476044, + 0.4239119291305542 + ], + [ + 1.0207308530807495, + 0.8557561039924622, + 0.4489574134349823, + -1.0117379426956177, + 0.8262714147567749, + -0.630092442035675, + 0.09812727570533752, + 0.21663640439510345, + 0.23771141469478607, + -0.47456812858581543, + -1.3189021348953247, + 2.269665002822876, + -0.7066116333007812, + 1.1826255321502686, + 1.0058563947677612, + 0.1694127470254898, + -0.5649446845054626, + -0.32043421268463135, + 0.46698564291000366, + -0.9052557945251465, + -0.12436902523040771 + ], + [ + -0.5202209949493408, + 0.30583277344703674, + 1.2163622379302979, + 0.4305659234523773, + 1.7276779413223267, + -0.1535550355911255, + -0.5289345383644104, + -0.632349967956543, + 0.9393473267555237, + 0.38391292095184326, + -1.1998018026351929, + 1.9739618301391602, + -1.3405330181121826, + 0.048939138650894165, + 0.6369897127151489, + 0.629721999168396, + 1.388192892074585, + -1.065996766090393, + 1.9119879007339478, + -1.0432264804840088, + 0.9627590179443359 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.13860583305358887, + -0.06957411766052246, + 0.1504676342010498, + 0.1671028435230255, + 0.5675687789916992, + 0.16587841510772705, + 0.0, + -0.21137550473213196, + -0.21542736887931824, + 0.2565074861049652, + -0.00923699140548706, + 0.5251069068908691, + -0.06964048743247986, + 0.0, + -0.19547803699970245, + 0.10278619825839996, + -0.19206219911575317, + -0.1487000733613968, + 0.44575393199920654, + 0.0725468099117279, + 0.0 + ] + ], + "predicted_scores": [ + 6.7557806968688965, + 6.573371887207031, + 5.993346691131592, + 5.651432037353516, + 5.6416802406311035, + 5.634435176849365, + 5.356122016906738, + 5.133599758148193 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "551504635/train2", + "selected_index": 0, + "state_hash": "ab6587e0b2a3b5f21f23c56daf8894249774912a3b20794bd49dc1b78ec736a7", + "stored_base_utility": 0.728268027305603, + "task_id": "LiftPegUpright-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.42231252789497375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.42231252789497375 + }, + "base_progress": 0.42231252789497375, + "base_success": false, + "base_utility": 0.42231252789497375, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.21454350650310516, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21454350650310516 + }, + { + "contact_quality": null, + "progress": 0.2053624391555786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2053624391555786 + }, + { + "contact_quality": null, + "progress": 0.4355689287185669, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4355689287185669 + }, + { + "contact_quality": null, + "progress": 0.20295369625091553, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20295369625091553 + }, + { + "contact_quality": null, + "progress": 0.6990174651145935, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6990174651145935 + }, + { + "contact_quality": null, + "progress": 0.19902725517749786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19902725517749786 + }, + { + "contact_quality": null, + "progress": 0.4670216143131256, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4670216143131256 + }, + { + "contact_quality": null, + "progress": 0.513749361038208, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.513749361038208 + } + ], + "candidate_progress": [ + 0.21454350650310516, + 0.2053624391555786, + 0.4355689287185669, + 0.20295369625091553, + 0.6990174651145935, + 0.19902725517749786, + 0.4670216143131256, + 0.513749361038208 + ], + "candidate_source_chart_ids": [ + "ms-13d8bae55d3c3710138f", + "ms-e1b6bcb41532a800d3cb", + "ms-8bc7aa419d17368d81b5", + "ms-0660d5f65df5fc4b6132", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb", + "ms-8bc7aa419d17368d81b5", + "ms-df961fe24a43e56d1716" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-b04b489f4fbad93b65ca", + "generated_tangents": [ + [ + -0.10732845962047577, + -0.07998903095722198, + 0.9089713096618652, + 0.43404635787010193, + 0.5116224884986877, + -0.4192449152469635, + -0.0853036642074585, + 0.21688437461853027, + 0.060581207275390625, + -0.10935025662183762, + -0.08304091542959213, + 1.5341674089431763, + -0.2520782947540283, + -0.30165374279022217, + 0.21617785096168518, + 0.03804491087794304, + 0.029590457677841187, + -0.28844237327575684, + 1.558558702468872, + -0.1732308715581894, + -0.41648069024086 + ], + [ + 0.10789267718791962, + -0.025463012978434563, + 0.9054484367370605, + 0.5132200717926025, + 0.43973252177238464, + -0.28184157609939575, + -0.4851565361022949, + 0.35337552428245544, + 0.020470203831791878, + -0.13234728574752808, + 0.09148307144641876, + 1.60129714012146, + -0.8142393827438354, + -0.14194037020206451, + 0.31978920102119446, + 0.0017017032951116562, + 0.15934930741786957, + -0.2463037520647049, + 1.7043859958648682, + -0.3596044182777405, + -0.21034398674964905 + ], + [ + -0.2914814352989197, + 0.0723748505115509, + 0.6198295950889587, + 0.5085612535476685, + 0.7717130184173584, + -0.44099000096321106, + -0.5913452506065369, + 0.42901328206062317, + 0.023957092314958572, + -0.10156137496232986, + -0.03212863951921463, + 1.4908665418624878, + -0.13424307107925415, + -0.479592502117157, + 0.20987318456172943, + 0.0969122052192688, + 0.03166326880455017, + 0.1798076629638672, + 1.7433905601501465, + -0.05927984416484833, + -0.09309232234954834 + ], + [ + -0.1918175369501114, + -0.03785164654254913, + 1.0073039531707764, + 0.5900051593780518, + 0.622347891330719, + -0.3132612407207489, + -0.18197572231292725, + 0.24078305065631866, + 0.11385473608970642, + -0.11136939376592636, + -0.00994226336479187, + 1.4678353071212769, + -0.3760998547077179, + -0.24490900337696075, + 0.4038201570510864, + 0.09390346705913544, + 0.18737035989761353, + -0.3093724846839905, + 1.491455078125, + -0.20497477054595947, + -0.24525514245033264 + ], + [ + -0.34775400161743164, + 0.06690829992294312, + 0.6236593127250671, + 0.6907037496566772, + 0.5310370326042175, + -0.2572833299636841, + -0.08047932386398315, + 0.17149080336093903, + 0.013812781311571598, + 0.05431506037712097, + -0.21567025780677795, + 1.558121681213379, + -0.4854121804237366, + -0.08453099429607391, + 0.30467668175697327, + 0.17470823228359222, + 0.04356248676776886, + -0.2433718889951706, + 1.6160027980804443, + -0.17445962131023407, + -0.3500150740146637 + ], + [ + 0.17549237608909607, + -0.11710525304079056, + 0.9921681880950928, + 0.5986083745956421, + 0.24832230806350708, + 0.100563645362854, + -0.4224139451980591, + 0.34710606932640076, + 0.13641080260276794, + -0.015819929540157318, + 0.10837534070014954, + 1.5618746280670166, + -0.31603893637657166, + 0.0254497230052948, + 0.4129064381122589, + 0.02422311156988144, + 0.10954248160123825, + -0.30140072107315063, + 1.6172378063201904, + -0.12455418705940247, + -0.3618107736110687 + ], + [ + -0.19824378192424774, + 0.02349727600812912, + 0.5157075524330139, + 0.477207213640213, + 0.5088697671890259, + -0.05654667317867279, + -0.30953019857406616, + 0.22353291511535645, + 0.0600614920258522, + -0.09159446507692337, + -0.03458709269762039, + 1.5768359899520874, + -0.34343600273132324, + -0.5828196406364441, + 0.11493048071861267, + 0.004861665889620781, + 0.13417184352874756, + -0.06164027750492096, + 1.639733076095581, + 0.25797146558761597, + -0.29030829668045044 + ], + [ + -0.2250225991010666, + 0.035239078104496, + 0.6645710468292236, + 0.5496208667755127, + 0.50779128074646, + -0.4300542175769806, + -0.11587768793106079, + 0.10986512154340744, + 0.04473678022623062, + -0.07210289686918259, + -0.10999809950590134, + 1.5273667573928833, + -0.27252569794654846, + -0.44839179515838623, + 0.16335031390190125, + 0.12081894278526306, + 0.19093842804431915, + -0.22147516906261444, + 1.8027544021606445, + -0.18689005076885223, + -0.3028786778450012 + ] + ], + "generated_utilities": [ + 0.21454350650310516, + 0.2053624391555786, + 0.4355689287185669, + 0.20295369625091553, + 0.6990174651145935, + 0.19902725517749786, + 0.4670216143131256, + 0.513749361038208 + ], + "group_id": "ms-b04b489f4fbad93b65ca", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7198216915130615, + 0.7247374057769775, + 2.0, + 0.22730140388011932, + 0.21130704879760742, + 0.2076549530029297, + 0.2006116509437561, + 0.21285679936408997, + 0.19614991545677185, + 0.20440611243247986, + 0.2015489637851715, + 0.19511334598064423, + 0.20091164112091064, + 0.19681888818740845, + 0.19675713777542114, + 0.20853900909423828 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.30706143379211426, + 0.2797066569328308, + 0.4565920829772949, + 0.7342244386672974, + -0.28428468108177185, + 0.012420237064361572, + 0.0, + 0.19500797986984253, + -0.1289721131324768, + -0.3524337112903595, + 0.07460063695907593, + 1.7684156894683838, + 0.11335399746894836, + 0.0, + -0.5014548301696777, + 0.4537479281425476, + 0.1963198333978653, + -0.025568604469299316, + 1.6286182403564453, + -0.0963512659072876, + 0.0 + ], + [ + 1.3070614337921143, + -0.7202933430671692, + 1.456592082977295, + 1.7342244386672974, + -0.15130555629730225, + 0.9519147872924805, + -1.4541058540344238, + -0.6432010531425476, + -0.26959508657455444, + -0.3406839668750763, + -0.4419664740562439, + 2.768415689468384, + -0.3271861970424652, + -0.13685251772403717, + 0.23842881619930267, + 0.20406432449817657, + 0.3414191007614136, + -1.0255686044692993, + 2.6286182403564453, + -0.9051820039749146, + 0.6850019693374634 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9082117080688477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.27370503544807434, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.3700039386749268 + ], + [ + 1.0634682178497314, + -0.8590207099914551, + 1.6810569763183594, + 1.8525642156600952, + 0.06716938316822052, + 1.0774534940719604, + -1.177553415298462, + -0.7412493228912354, + -0.3193308115005493, + -0.4253617525100708, + -0.4890587329864502, + 2.624048948287964, + -0.109145388007164, + -0.10380380600690842, + 0.20137116312980652, + 0.20110605657100677, + 0.06929725408554077, + -0.9908366203308105, + 2.5783159732818604, + -0.7248227000236511, + 0.919882595539093 + ], + [ + 1.2083125114440918, + -0.9112091064453125, + 1.1255995035171509, + 1.9678500890731812, + -0.3424606919288635, + 1.350451946258545, + -1.5799542665481567, + -1.014689326286316, + -0.3427739441394806, + -0.24094250798225403, + -0.4992668032646179, + 2.403038740158081, + -0.29080134630203247, + -0.1569681167602539, + 0.4877905249595642, + -0.16464389860630035, + 0.41957730054855347, + -0.6997033357620239, + 2.80957293510437, + -1.0509651899337769, + 0.8937826156616211 + ], + [ + 1.2133954763412476, + -1.0171141624450684, + 1.1572445631027222, + 1.8050156831741333, + -0.29570114612579346, + 0.7672240734100342, + -0.976697564125061, + -0.8219587802886963, + -0.746895432472229, + -0.346658855676651, + -0.7942779064178467, + 3.0403196811676025, + -0.6172577738761902, + 0.3615778088569641, + 0.546725332736969, + -0.06415437161922455, + 0.43834370374679565, + -1.332592248916626, + 3.029940605163574, + -0.5720348358154297, + 1.1660562753677368 + ], + [ + 0.7937494516372681, + -0.45201122760772705, + 2.0367026329040527, + 2.1590728759765625, + 0.38935917615890503, + 0.5424834489822388, + -1.391097903251648, + -1.0657228231430054, + -0.44411367177963257, + -0.3797546923160553, + -0.03612968325614929, + 2.867069721221924, + -0.26322364807128906, + -0.32477179169654846, + 0.007963955402374268, + 0.053216055035591125, + 0.36466139554977417, + -0.7477234601974487, + 2.170649766921997, + -0.46266868710517883, + 0.7669195532798767 + ], + [ + 1.0920023918151855, + -1.0678586959838867, + 1.4391454458236694, + 2.155820608139038, + -0.7347199320793152, + 0.6208673715591431, + -1.6380563974380493, + -0.6589961051940918, + -0.8267081379890442, + 0.011062979698181152, + -0.1169244647026062, + 2.743579626083374, + -0.41342735290527344, + -0.5980290770530701, + 0.49618369340896606, + -0.09863747656345367, + 0.2179805338382721, + -0.6006474494934082, + 2.2965025901794434, + -0.97348952293396, + 1.2447597980499268 + ], + [ + 1.8286218643188477, + -1.486896276473999, + 1.8610713481903076, + 1.792364478111267, + -0.06209900975227356, + 0.7138283252716064, + -1.0023046731948853, + -0.8588749170303345, + -0.30936315655708313, + 0.33444294333457947, + 0.2771298885345459, + 3.0452492237091064, + -0.6618144512176514, + -0.747363805770874, + 0.19540274143218994, + -0.05051566660404205, + 0.5057645440101624, + -0.3770519495010376, + 3.028423309326172, + -0.19366812705993652, + 1.1766406297683716 + ], + [ + 1.057653784751892, + -0.15160071849822998, + 1.1646175384521484, + 2.517653226852417, + 0.037323400378227234, + 1.4844036102294922, + -1.2659661769866943, + -0.9694168567657471, + -0.792635440826416, + 0.15484732389450073, + -0.6592323780059814, + 2.426063060760498, + -0.8063763976097107, + 0.37280529737472534, + -0.1984816938638687, + 0.3488052487373352, + 0.36773377656936646, + -0.9600323438644409, + 1.801730990409851, + -1.1618295907974243, + 0.526383101940155 + ], + [ + 0.8403785228729248, + -1.7131881713867188, + 1.7484346628189087, + 1.3915244340896606, + 0.038706690073013306, + 1.7859117984771729, + -0.6996253132820129, + -0.12596702575683594, + -0.07181461155414581, + -0.8556877374649048, + -0.6567700505256653, + 2.935109853744507, + 0.3808797299861908, + 0.1733608990907669, + 0.9097857475280762, + 0.7001722455024719, + 0.14087507128715515, + -0.32659411430358887, + 2.278188943862915, + -1.87721586227417, + 1.438266634941101 + ], + [ + 1.259635329246521, + -0.5244767665863037, + 2.201894998550415, + 2.2337045669555664, + -0.333431601524353, + 0.7420244216918945, + -2.0896377563476562, + 0.00931471586227417, + 0.6261725425720215, + -0.49785688519477844, + -0.9404669404029846, + 3.756554365158081, + 0.5252982378005981, + -0.3158212900161743, + 0.4053819179534912, + 0.8204222917556763, + 0.5041900277137756, + -0.43809741735458374, + 3.519717216491699, + -1.0316743850708008, + 0.3069072663784027 + ], + [ + 0.9530733823776245, + -0.011007547378540039, + 2.396453380584717, + 2.6382436752319336, + 0.29263433814048767, + 1.6715352535247803, + -0.819398045539856, + -0.5916767120361328, + 0.006437480449676514, + -0.3942071795463562, + -0.5187541842460632, + 3.677276849746704, + -0.4376804232597351, + 0.3331298828125, + -0.3693196773529053, + -0.3432707190513611, + 1.104022741317749, + -1.0780245065689087, + 3.0859882831573486, + -0.26956111192703247, + 0.8377041816711426 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.30706143379211426, + 0.024911105632781982, + 0.4565920829772949, + 0.7342244386672974, + 0.01698923110961914, + -0.04808521270751953, + 0.0, + 0.10375595092773438, + 0.23247671127319336, + -0.12021404504776001, + -0.19280612468719482, + 1.7684156894683838, + -0.23127278685569763, + 0.0, + -0.1010780781507492, + 0.00952194631099701, + -0.16307592391967773, + -0.14496707916259766, + 1.6286182403564453, + -0.28656232357025146, + 0.0 + ] + ], + "predicted_scores": [ + 6.294991493225098, + 6.1743950843811035, + 5.736081123352051, + 5.545506000518799, + 5.456851482391357, + 5.102646350860596, + 4.967200756072998, + 4.937523365020752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113268940/train2", + "selected_index": 0, + "state_hash": "76e9a962970055fa7565c007db1b35e003cd352908be4cf573066d8b58e2d443", + "stored_base_utility": 0.7198216915130615, + "task_id": "LiftPegUpright-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7267584800720215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7267584800720215 + }, + { + "contact_quality": null, + "progress": 0.7201066613197327, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7201066613197327 + }, + { + "contact_quality": null, + "progress": 0.7173653244972229, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7173653244972229 + }, + { + "contact_quality": null, + "progress": 0.7081647515296936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7081647515296936 + }, + { + "contact_quality": null, + "progress": 0.7245497107505798, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7245497107505798 + }, + { + "contact_quality": null, + "progress": 0.722693145275116, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.722693145275116 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.7267584800720215, + 0.7201066613197327, + 0.7173653244972229, + 0.7081647515296936, + 0.7245497107505798, + 0.722693145275116 + ], + "candidate_source_chart_ids": [ + "ms-b483cac145e16ccdf42a", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69", + "ms-062addb9da32a0c2ea97", + "ms-baa36ee75ec2de550b9b", + "ms-b483cac145e16ccdf42a", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + true, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-c663f7e73e7e8376a05c", + "generated_tangents": [ + [ + 0.10946595668792725, + -0.11374206840991974, + -0.30370935797691345, + -0.09413708001375198, + 2.146193504333496, + -0.004189819097518921, + 0.13687065243721008, + 0.10518065094947815, + -0.06227639317512512, + 0.10917890071868896, + 0.008878372609615326, + 1.1038117408752441, + -0.07490143179893494, + -0.17087791860103607, + 0.06763358414173126, + 0.003116860054433346, + 0.17291629314422607, + -0.11192534863948822, + 1.142543911933899, + 0.18380320072174072, + -0.16685497760772705 + ], + [ + 0.026685722172260284, + -0.04553455114364624, + -0.26302388310432434, + -0.10152976214885712, + 2.136355400085449, + -0.057701557874679565, + -0.14895033836364746, + 0.05181865394115448, + -0.0992811992764473, + -0.043570078909397125, + 0.07364282011985779, + 1.3353840112686157, + -0.15444502234458923, + -0.2545003294944763, + 0.05045922100543976, + 0.047208771109580994, + 0.06242549419403076, + -0.21155910193920135, + 1.0452136993408203, + 0.030841559171676636, + -0.11804291605949402 + ], + [ + -0.009969472885131836, + -0.03685017302632332, + -0.30377474427223206, + -0.10030913352966309, + 2.1355435848236084, + -0.07478314638137817, + -0.06573319435119629, + 0.05628681182861328, + -0.10190877318382263, + -0.05741732567548752, + 0.06106235459446907, + 1.3288439512252808, + -0.16746309399604797, + -0.25997209548950195, + 0.036219522356987, + 0.04209299385547638, + 0.09307558834552765, + -0.21978159248828888, + 1.000388503074646, + 0.03648659586906433, + -0.03894805908203125 + ], + [ + 0.020938843488693237, + -0.09490247070789337, + -0.29992005228996277, + -0.04530210793018341, + 2.1611111164093018, + -0.014455854892730713, + 0.15765157341957092, + 0.04537287354469299, + -0.0998525321483612, + 0.04522865265607834, + -0.07850425690412521, + 1.185364007949829, + -0.22974291443824768, + -0.13400767743587494, + 0.02067471295595169, + 0.03210567310452461, + 0.13150949776172638, + -0.12193818390369415, + 1.0589673519134521, + 0.04473540186882019, + -0.00128251314163208 + ], + [ + 0.03843197971582413, + -0.08653861284255981, + -0.33079013228416443, + -0.13347841799259186, + 2.185880184173584, + 0.05989789962768555, + 0.04211077094078064, + -0.02038167417049408, + -0.11486528813838959, + 0.08138403296470642, + -0.056508637964725494, + 1.2871387004852295, + -0.15371173620224, + -0.24662600457668304, + 0.018292300403118134, + 0.04085967689752579, + 0.1069001778960228, + -0.16666091978549957, + 1.0928897857666016, + 0.0013755261898040771, + -0.18731218576431274 + ], + [ + 0.05528000742197037, + -0.07219599187374115, + -0.3663255274295807, + -0.06928510963916779, + 2.1292848587036133, + -0.02273339033126831, + 0.10645395517349243, + -0.008280232548713684, + -0.11774088442325592, + 0.01741180568933487, + -0.07084622234106064, + 1.049666404724121, + -0.18437302112579346, + -0.20236153900623322, + -0.06443154811859131, + 0.012843076139688492, + 0.04830491542816162, + -0.16940997540950775, + 1.1128853559494019, + -0.009312480688095093, + -0.10608768463134766 + ], + [ + 0.05965904891490936, + -0.07622204720973969, + -0.4074612557888031, + -0.04590828716754913, + 2.1456429958343506, + -0.13582226634025574, + -0.030505597591400146, + 0.02432785928249359, + -0.19222700595855713, + 0.09118318557739258, + -0.08913388103246689, + 1.0548746585845947, + -0.37534233927726746, + -0.14381365478038788, + 0.12527123093605042, + 0.07386000454425812, + -0.015574052929878235, + -0.09169892966747284, + 1.1436439752578735, + -0.09258164465427399, + -0.10136809945106506 + ], + [ + 0.15269577503204346, + 0.010219737887382507, + -0.3038608133792877, + -0.04885582625865936, + 1.9410953521728516, + 0.5958267450332642, + -0.08891606330871582, + -0.0011883378028869629, + -0.1737256646156311, + 0.043074771761894226, + -0.1666845977306366, + 1.0102267265319824, + -0.41516172885894775, + -0.16439305245876312, + 0.0007879920303821564, + 0.17655529081821442, + 0.23835942149162292, + -0.4307820796966553, + 1.1399189233779907, + 0.07694810628890991, + 0.14124073088169098 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.7267584800720215, + 0.7201066613197327, + 0.7173653244972229, + 0.7081647515296936, + 0.7245497107505798, + 0.722693145275116 + ], + "group_id": "ms-c663f7e73e7e8376a05c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7233090400695801, + 0.7198824882507324, + 2.0, + 0.7120869159698486, + 0.27145418524742126, + 0.23090600967407227, + 0.2526220679283142, + 0.2656436562538147, + 0.264312207698822, + 0.27572086453437805, + 0.296668142080307, + 0.2128777951002121, + 0.22965002059936523, + 0.2196153998374939, + 0.22027263045310974, + 0.19939398765563965 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.038699887692928314, + -0.3685438632965088, + -1.3628361225128174, + 0.0675731897354126, + 2.4906270503997803, + 0.8971554040908813, + 0.06863358616828918, + 0.14689946174621582, + 0.0500560887157917, + 0.5852231383323669, + -0.5345680713653564, + 1.7433128356933594, + -0.5520187020301819, + 0.36285334825515747, + 0.06837204098701477, + 0.04091557487845421, + 0.26646411418914795, + -0.3870337903499603, + 1.7901486158370972, + -0.46956348419189453, + 0.5412080883979797 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.13726717233657837, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7257066965103149, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0824161767959595 + ], + [ + -0.09956050664186478, + -0.6439464092254639, + -1.5331499576568604, + -0.2275703251361847, + 2.2570624351501465, + 1.145668387413025, + -0.19933581352233887, + 0.15889883041381836, + -0.07524599134922028, + 0.6580387353897095, + -0.5754004716873169, + 1.6105921268463135, + -0.25637149810791016, + 0.37256744503974915, + 0.17931225895881653, + 0.286465048789978, + 0.07884949445724487, + -0.35266172885894775, + 1.548264980316162, + -0.2917717695236206, + 0.3307930827140808 + ], + [ + 0.13072630763053894, + -0.5927069783210754, + -1.4328954219818115, + 0.1252535879611969, + 2.5439467430114746, + 1.119347095489502, + -0.12134809792041779, + -0.09485669434070587, + -0.040146272629499435, + 0.4683093726634979, + -0.6791773438453674, + 2.1314187049865723, + -0.4368498623371124, + 0.5876930952072144, + 0.27027207612991333, + -0.1808907389640808, + 0.037005022168159485, + -0.6130290627479553, + 1.60811185836792, + -0.8427046537399292, + 0.5648807883262634 + ], + [ + -0.019807856529951096, + -0.40178078413009644, + -1.5710430145263672, + 0.3568149507045746, + 2.8693275451660156, + 1.2080440521240234, + 0.45903119444847107, + -0.07479061186313629, + 0.2587376832962036, + 0.37667450308799744, + -0.5607666373252869, + 1.9825769662857056, + -0.7874732613563538, + 0.48894837498664856, + -0.04161303490400314, + 0.2237108200788498, + 0.12043289840221405, + -0.8541218638420105, + 1.3131444454193115, + -0.8318716287612915, + 0.7653946876525879 + ], + [ + -0.039324238896369934, + 0.18316024541854858, + -1.2052042484283447, + -0.020045489072799683, + 2.441659927368164, + 1.3116326332092285, + 0.04364071041345596, + 0.617156982421875, + -0.5366188287734985, + 0.5837841629981995, + -0.6157102584838867, + 1.946446180343628, + -0.44197604060173035, + -0.12030342221260071, + 0.2635759711265564, + -0.34203875064849854, + 0.3788418769836426, + -0.01673591136932373, + 1.7612254619598389, + -0.8283271193504333, + 0.44542378187179565 + ], + [ + 0.020479684695601463, + -0.3728349208831787, + -1.2566280364990234, + -0.4903675317764282, + 2.7149407863616943, + 1.1410136222839355, + 0.18438707292079926, + 0.15751159191131592, + 0.5994289517402649, + 0.7441410422325134, + -0.5757432579994202, + 2.025078058242798, + -1.1044142246246338, + -0.24153441190719604, + 0.7623711824417114, + -0.07102327048778534, + 0.7491155862808228, + -0.853058397769928, + 1.1346031427383423, + -0.4604499638080597, + 0.3380756974220276 + ], + [ + -0.0438276082277298, + 0.21775567531585693, + -0.9180121421813965, + 0.2164689004421234, + 1.7669851779937744, + 0.7701845765113831, + -0.4171431064605713, + -0.44690096378326416, + -0.6473215818405151, + 0.5592721700668335, + 0.1537061333656311, + 1.2201651334762573, + -0.02862715721130371, + 0.759445309638977, + -0.08159622550010681, + -0.038715001195669174, + 0.8136870265007019, + -1.129607081413269, + 1.5099153518676758, + -1.2117259502410889, + 1.2496943473815918 + ], + [ + -0.6884663105010986, + -0.7670764923095703, + -1.3375780582427979, + 0.42161574959754944, + 2.4925031661987305, + 0.18111640214920044, + -0.4373469054698944, + 0.6228322386741638, + -0.3739543557167053, + 1.3811917304992676, + 0.05884122848510742, + 1.0582032203674316, + -0.38174134492874146, + 0.7282494902610779, + 0.9550073146820068, + -0.6478474736213684, + -0.467768132686615, + -0.22179017961025238, + 2.1456754207611084, + -0.39938730001449585, + 1.08339524269104 + ], + [ + 0.7042056322097778, + -0.4828507900238037, + -1.914047122001648, + -0.7084186673164368, + 1.87782621383667, + 0.42005008459091187, + 0.511604905128479, + -0.2043081521987915, + 0.9001545906066895, + 1.083567500114441, + -0.9796027541160583, + 1.3277345895767212, + -0.5564201474189758, + 0.6932777166366577, + -0.5934457778930664, + -0.296670138835907, + 0.14519014954566956, + -0.8612968325614929, + 0.8968887329101562, + -0.5584146976470947, + -0.434501588344574 + ], + [ + 0.19210004806518555, + 0.5024477243423462, + -0.4993329644203186, + -0.826042115688324, + 1.6430644989013672, + 0.9877098798751831, + 0.49057987332344055, + -0.13611945509910583, + 0.7228546738624573, + 1.4937405586242676, + 0.18012672662734985, + 2.188062906265259, + -0.6084107160568237, + 1.0874407291412354, + 0.5744171142578125, + 0.9375161528587341, + -0.43115919828414917, + 0.19787153601646423, + 2.652085781097412, + -0.529538631439209, + 0.03626328706741333 + ], + [ + -0.014053341001272202, + -1.351341724395752, + -0.7293765544891357, + 0.9410406947135925, + 3.139275074005127, + 1.3463419675827026, + -0.728021502494812, + -0.6353646516799927, + -0.19685329496860504, + 1.5315749645233154, + -1.2550475597381592, + 1.7411813735961914, + 0.14418578147888184, + 0.7038239240646362, + 0.1004495620727539, + 0.7590892910957336, + 0.9287852048873901, + -0.23726919293403625, + 0.8381044864654541, + -1.2906644344329834, + 0.9584261178970337 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03644278272986412, + -0.12137487530708313, + -0.3628361225128174, + -0.027813076972961426, + 1.4906270503997803, + -0.10284459590911865, + 0.0, + 0.15174832940101624, + 0.015595663338899612, + -0.10680150985717773, + -0.14723241329193115, + 0.7433128356933594, + 0.25961190462112427, + 0.0, + -0.23801246285438538, + -0.10784421861171722, + 0.09500086307525635, + -0.06586915254592896, + 0.7901486158370972, + -0.10978007316589355, + 0.0 + ] + ], + "predicted_scores": [ + 7.598120212554932, + 7.074145793914795, + 7.03989839553833, + 6.809096813201904, + 6.70497465133667, + 6.377159595489502, + 5.469299793243408, + 5.32899284362793 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1528740417/train2", + "selected_index": 0, + "state_hash": "c02fdfdd2339e4215d60b2221be925e066a7763f9e3bba3ff33a15f20e286f5a", + "stored_base_utility": 0.7233090400695801, + "task_id": "LiftPegUpright-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15342965722084045, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15342965722084045 + }, + "base_progress": 0.15342965722084045, + "base_success": false, + "base_utility": 0.15342965722084045, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07777997106313705, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07777997106313705 + }, + { + "contact_quality": null, + "progress": 0.04109710454940796, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04109710454940796 + }, + { + "contact_quality": null, + "progress": 0.1038902997970581, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1038902997970581 + }, + { + "contact_quality": null, + "progress": 0.4708559513092041, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4708559513092041 + }, + { + "contact_quality": null, + "progress": 0.07253576815128326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07253576815128326 + }, + { + "contact_quality": null, + "progress": 0.09191600978374481, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09191600978374481 + }, + { + "contact_quality": null, + "progress": 0.05067390203475952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05067390203475952 + }, + { + "contact_quality": null, + "progress": 0.04289213567972183, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04289213567972183 + } + ], + "candidate_progress": [ + 0.07777997106313705, + 0.04109710454940796, + 0.1038902997970581, + 0.4708559513092041, + 0.07253576815128326, + 0.09191600978374481, + 0.05067390203475952, + 0.04289213567972183 + ], + "candidate_source_chart_ids": [ + "ms-23a355317920e192147f", + "ms-a509a572b3871e591d26", + "ms-5df54a8baaceb8c73c8a", + "ms-0541a144afe57b5a7ae6", + "ms-4015c296239564108da2", + "ms-307f9067d5b66f55b3a6", + "ms-b7e7fe66a6c86898eec2", + "ms-b7e7fe66a6c86898eec2" + ], + "candidate_source_task_ids": [ + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank6", + "ctt_transport_rank6" + ], + "candidates_evaluated": true, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "generated_tangents": [ + [ + -0.2633914351463318, + -0.24415768682956696, + 0.9743947982788086, + -0.1303619146347046, + -0.024884283542633057, + -0.2420523762702942, + 1.0717332363128662, + 0.314116895198822, + -0.2180466204881668, + 0.030273988842964172, + -0.08761095255613327, + 0.18791615962982178, + -0.20732125639915466, + 2.459271192550659, + 0.02998794987797737, + 0.13088786602020264, + -0.004302307963371277, + -0.07890604436397552, + 0.2167949378490448, + -0.3225422203540802, + 2.8667795658111572 + ], + [ + -0.35856717824935913, + -0.21494071185588837, + 1.2262903451919556, + -0.038692787289619446, + 0.4024645686149597, + -0.17845800518989563, + 1.2016874551773071, + 0.05131341516971588, + -0.14427171647548676, + 0.06735620647668839, + -0.0521039217710495, + -0.01999160647392273, + -0.05861958861351013, + 2.786029100418091, + 0.14884191751480103, + 0.09314198791980743, + -0.001286521553993225, + -0.20260174572467804, + 0.23968689143657684, + 0.03133571147918701, + 2.820800542831421 + ], + [ + 0.0325254425406456, + -0.32269322872161865, + 0.6115252375602722, + -0.09641175717115402, + 0.4406045377254486, + -0.28701019287109375, + 0.6941764950752258, + 0.3170878291130066, + -0.17988522350788116, + -0.05719827860593796, + 0.02314738929271698, + 0.11381135880947113, + 0.12774470448493958, + 2.5892746448516846, + 0.22388124465942383, + 0.12328775227069855, + -0.062124744057655334, + -0.09468741714954376, + 0.36773669719696045, + -0.011422485113143921, + 2.8251781463623047 + ], + [ + -0.13648687303066254, + -0.30327659845352173, + 0.4851149916648865, + -0.1762879192829132, + 0.756318211555481, + -0.2686004936695099, + 0.7468767166137695, + 0.29756754636764526, + -0.1784353405237198, + -0.07621937245130539, + -0.04210872948169708, + 0.18523074686527252, + 0.2125585377216339, + 2.610225200653076, + 0.23728817701339722, + 0.07998001575469971, + -0.08203758299350739, + -0.011189937591552734, + 0.38997745513916016, + 0.05735740065574646, + 2.957305669784546 + ], + [ + -0.2623583674430847, + -0.19964829087257385, + 1.0483981370925903, + -0.1308472901582718, + 0.09816092252731323, + -0.21537232398986816, + 0.6463452577590942, + 0.27233392000198364, + -0.24594703316688538, + -0.041606657207012177, + 0.05039536580443382, + 0.15449047088623047, + -0.17688804864883423, + 2.6832849979400635, + 0.2820209264755249, + 0.0677497386932373, + 0.0134086012840271, + -0.07833032310009003, + 0.3621862530708313, + -0.11108329892158508, + 2.829379081726074 + ], + [ + -0.33549773693084717, + -0.17868132889270782, + 0.929031491279602, + -0.14533010125160217, + 0.22347202897071838, + -0.2668790817260742, + 0.6549648642539978, + 0.38585883378982544, + -0.18775269389152527, + -0.058350615203380585, + 0.06588498502969742, + 0.1194341629743576, + -0.018949836492538452, + 2.5024490356445312, + 0.2588111460208893, + 0.1296253502368927, + 0.008959144353866577, + -0.09163670241832733, + 0.31655794382095337, + -0.19999781250953674, + 2.834423780441284 + ], + [ + -0.4173704981803894, + -0.3724336624145508, + 0.5605266690254211, + -0.24761976301670074, + -0.15740084648132324, + -0.6621396541595459, + 0.8614896535873413, + -0.1276450753211975, + -0.2848299741744995, + 0.07669876515865326, + -0.24686670303344727, + -0.06723156571388245, + -0.17518436908721924, + 2.8572006225585938, + 0.10023102164268494, + 0.15556290745735168, + 0.09022867679595947, + -0.2503125071525574, + -0.06601586937904358, + 0.09898245334625244, + 2.4181342124938965 + ], + [ + -0.5222467184066772, + -0.3916386365890503, + 0.7598212957382202, + -0.19166620075702667, + -0.2371518909931183, + -0.6868388652801514, + 1.0427273511886597, + -0.13800588250160217, + -0.3128302991390228, + -0.07054481655359268, + -0.2948596775531769, + 0.01922476291656494, + -0.39983636140823364, + 2.7168216705322266, + 0.04087206721305847, + 0.24478109180927277, + 0.12932133674621582, + -0.46600866317749023, + -0.08169755339622498, + 0.08273890614509583, + 2.473327398300171 + ] + ], + "generated_utilities": [ + 0.07777997106313705, + 0.04109710454940796, + 0.1038902997970581, + 0.4708559513092041, + 0.07253576815128326, + 0.09191600978374481, + 0.05067390203475952, + 0.04289213567972183 + ], + "group_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.6040759086608887, + 2.0, + 0.14582239091396332, + 0.5723002552986145, + 0.10770338028669357, + 0.10128920525312424, + 0.08499570190906525, + 0.14736615121364594, + 0.07181884348392487, + 0.11773071438074112, + 0.06549974530935287, + 0.06568789482116699, + 0.10681070387363434, + 0.05432777479290962, + 0.0625491738319397, + 0.06991603225469589 + ], + "instruction": "Pick up the cube and move it to the goal position.", + "negative_tangents": [ + [ + -0.072060227394104, + 0.2043294906616211, + 0.8692872524261475, + 0.020746350288391113, + -0.004742175340652466, + 0.05779099464416504, + 0.0, + -0.0498790442943573, + -0.20218299329280853, + 0.41556206345558167, + -0.2306707501411438, + 0.32090064883232117, + -0.2089032530784607, + 0.0, + 0.05318737402558327, + -0.04637153446674347, + 0.04412045329809189, + 0.08670947700738907, + -0.10918038338422775, + -0.059010982513427734, + 0.0 + ], + [ + -0.5282122492790222, + 1.204329490661621, + 1.8692872524261475, + -0.8456907868385315, + 0.42858073115348816, + -0.3134727478027344, + -2.5308754444122314, + 0.1655428111553192, + -0.12024828046560287, + 0.0010203085839748383, + 0.2592625319957733, + 0.2679504156112671, + 0.25922709703445435, + 2.851924419403076, + 0.0039860643446445465, + -0.014717570506036282, + 0.01173020526766777, + 0.10578173398971558, + 0.05138798803091049, + 0.3216937780380249, + 2.9243662357330322 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.061750888824463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.703848838806152, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.8487324714660645 + ], + [ + -0.6558206081390381, + 1.3553937673568726, + 2.057647466659546, + -1.0405915975570679, + 0.2629950940608978, + -0.28233209252357483, + -2.589219093322754, + -0.06961745023727417, + 0.02406928688287735, + 0.2341437190771103, + 0.49766814708709717, + 0.4930538535118103, + 0.370309442281723, + 3.111422061920166, + 0.06151486933231354, + -0.2028706818819046, + 0.12007936835289001, + 0.2164750099182129, + 0.16723006963729858, + 0.26430171728134155, + 3.0643396377563477 + ], + [ + -0.18174728751182556, + 0.8890945315361023, + 1.5388343334197998, + -0.4532086253166199, + 0.3864310085773468, + -0.45410382747650146, + -2.623199462890625, + 0.5360933542251587, + -0.04706605523824692, + 0.11160072684288025, + 0.5460608005523682, + 0.39992716908454895, + 0.17028246819972992, + 2.8760411739349365, + 0.29450932145118713, + -0.3028962016105652, + 0.06744809448719025, + 0.1687500774860382, + 0.42097336053848267, + 0.3965923488140106, + 3.28550386428833 + ], + [ + -0.4313088357448578, + 0.9752410054206848, + 2.0068275928497314, + -0.5783756375312805, + 0.5761726498603821, + -0.5289677381515503, + -2.321643590927124, + 0.0023015737533569336, + -0.07372284680604935, + 0.05664224550127983, + 0.03747282922267914, + 0.4900178611278534, + 0.31588616967201233, + 3.018609046936035, + 0.12529893219470978, + -0.2872275412082672, + -0.2500917911529541, + 0.0006694421172142029, + -0.39133700728416443, + 0.6382719874382019, + 2.810041904449463 + ], + [ + -0.5560051202774048, + 1.4042110443115234, + 1.662226915359497, + -1.4059122800827026, + 0.8242433071136475, + -0.10998553037643433, + -2.7556281089782715, + -0.26875895261764526, + -0.4201813042163849, + -0.371376633644104, + 0.7718684673309326, + 0.6452833414077759, + 0.027421817183494568, + 2.3879575729370117, + -0.47588983178138733, + -0.045049820095300674, + 0.0838315486907959, + 0.10257662832736969, + 0.4181908369064331, + 0.2650004029273987, + 3.006909132003784 + ], + [ + -0.4896228015422821, + 1.0570447444915771, + 1.408416509628296, + -0.7269805669784546, + 1.023213267326355, + -0.22378163039684296, + -2.1348509788513184, + -0.4423682987689972, + 0.26834720373153687, + -0.5835619568824768, + 0.3675343692302704, + 0.3307996988296509, + -0.01537257432937622, + 3.4422543048858643, + 0.3622148334980011, + -0.26806333661079407, + 0.4238347113132477, + 0.2583734393119812, + 0.10431534051895142, + 0.2077663242816925, + 2.446356773376465 + ], + [ + -1.1077957153320312, + 0.4289678931236267, + 1.9882663488388062, + -0.3021383285522461, + -0.006195276975631714, + -0.22295325994491577, + -1.8697566986083984, + 0.9590625762939453, + -0.0024877265095710754, + -0.3088582456111908, + 0.6517632007598877, + 0.4271944761276245, + 0.34532302618026733, + 2.704721450805664, + 0.7557594776153564, + -0.8073300719261169, + -0.7479258179664612, + -0.19068273901939392, + 0.14240311086177826, + 0.2233867347240448, + 2.1774275302886963 + ], + [ + -0.9354162812232971, + 1.5971527099609375, + 1.0168101787567139, + -0.7164653539657593, + 0.519247829914093, + -0.7477964162826538, + -3.338376045227051, + 0.7212944030761719, + -0.6477807760238647, + 0.38193029165267944, + -0.1568622589111328, + 0.47457069158554077, + -0.30742186307907104, + 2.6848907470703125, + -0.4251534044742584, + 0.6623334884643555, + 0.03348249942064285, + -0.2996619939804077, + -0.8319172263145447, + 0.7883892059326172, + 3.5870752334594727 + ], + [ + -0.44698429107666016, + 0.3201255202293396, + 1.5405346155166626, + -0.9956957101821899, + 0.8944320678710938, + -0.4196317791938782, + -2.010188102722168, + 0.7199084758758545, + -0.49276289343833923, + 0.4512734115123749, + 0.48276829719543457, + 0.22657281160354614, + 0.8205416798591614, + 3.6653342247009277, + 0.6818231344223022, + 0.4660317897796631, + -0.14916014671325684, + 0.5328372716903687, + -0.5979922413825989, + 0.3124805986881256, + 3.110726833343506 + ], + [ + -0.5630171298980713, + 1.934722900390625, + 1.4954721927642822, + -1.684873342514038, + 0.133146733045578, + -0.4359630346298218, + -2.681238889694214, + 0.8904907703399658, + -0.48221057653427124, + -0.7635119557380676, + -0.23804441094398499, + 0.5691602230072021, + -0.23042798042297363, + 3.760842800140381, + 0.3684825003147125, + -0.20475278794765472, + 0.04838913306593895, + 0.17439094185829163, + 0.07042267918586731, + -0.1653938889503479, + 3.844882011413574 + ], + [ + -0.3705383241176605, + 1.5082495212554932, + 0.9002141356468201, + -1.3291171789169312, + 0.15676641464233398, + -1.2286558151245117, + -1.7006758451461792, + 0.34438908100128174, + 0.32962289452552795, + -0.6556114554405212, + 0.3502740263938904, + 0.7518025636672974, + -0.49080890417099, + 2.10355806350708, + 0.36426910758018494, + -0.4822288453578949, + 0.7496572732925415, + 0.43558254837989807, + 0.34650692343711853, + 0.9977220296859741, + 1.9853770732879639 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.005563676357269287, + 0.2043294906616211, + 0.8692872524261475, + 0.08163481950759888, + 0.05995750427246094, + 0.012857973575592041, + 0.0, + 0.01583634316921234, + 0.04724548012018204, + 0.018815450370311737, + 0.02742627263069153, + -0.03381502628326416, + 0.007930099964141846, + 0.0, + 0.01774722896516323, + 0.08064591884613037, + -0.06814983487129211, + -0.002621002495288849, + 0.05759798362851143, + 0.05164051055908203, + 0.0 + ] + ], + "predicted_scores": [ + 3.0987119674682617, + 3.048224449157715, + 3.0443451404571533, + 2.829008102416992, + 2.3250162601470947, + 2.161208152770996, + 1.4783743619918823, + 0.9284471273422241 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1273247072/train2", + "selected_index": 0, + "state_hash": "237fafab6bfdb561d66c862026f803fbd19d4007f6313bdb3d79b6dd74f69327", + "stored_base_utility": 0.6040759086608887, + "task_id": "PickCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.050487834960222244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.050487834960222244 + }, + "base_progress": 0.050487834960222244, + "base_success": false, + "base_utility": 0.050487834960222244, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12168914079666138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12168914079666138 + }, + { + "contact_quality": null, + "progress": 0.08366147428750992, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08366147428750992 + }, + { + "contact_quality": null, + "progress": 0.15701176226139069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15701176226139069 + }, + { + "contact_quality": null, + "progress": 0.12812411785125732, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12812411785125732 + }, + { + "contact_quality": null, + "progress": 0.18499283492565155, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18499283492565155 + }, + { + "contact_quality": null, + "progress": 0.08666101843118668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08666101843118668 + }, + { + "contact_quality": null, + "progress": 0.09535380452871323, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09535380452871323 + }, + { + "contact_quality": null, + "progress": 0.09223812818527222, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09223812818527222 + } + ], + "candidate_progress": [ + 0.12168914079666138, + 0.08366147428750992, + 0.15701176226139069, + 0.12812411785125732, + 0.18499283492565155, + 0.08666101843118668, + 0.09535380452871323, + 0.09223812818527222 + ], + "candidate_source_chart_ids": [ + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0345d8ac530f88cefd61", + "generated_tangents": [ + [ + 0.45177316665649414, + 0.053055040538311005, + 0.5662141442298889, + -0.08378662168979645, + 1.0378338098526, + -0.4903506934642792, + 0.11517664790153503, + -0.18985208868980408, + 0.536932110786438, + 0.612867534160614, + -0.4678453505039215, + -0.2846616804599762, + -0.7690434455871582, + -0.24230216443538666, + -0.723652720451355, + 0.37185996770858765, + -0.23070524632930756, + 0.42627060413360596, + -1.3476972579956055, + -0.23582014441490173, + -0.3316645324230194 + ], + [ + 0.15758034586906433, + 0.08929029107093811, + 0.27810442447662354, + 0.12750515341758728, + 0.8122290968894958, + -0.23212140798568726, + 0.21684962511062622, + -0.2672390043735504, + 0.5321968793869019, + 0.687800943851471, + -0.4824269711971283, + -0.4698670506477356, + -1.2611265182495117, + 0.31366878747940063, + -0.3113158047199249, + 0.4353826642036438, + -0.19363166391849518, + 0.43276938796043396, + -1.3187494277954102, + 0.3054482340812683, + 0.15071073174476624 + ], + [ + 0.09606397151947021, + 0.01897987723350525, + 0.27905046939849854, + 0.1527319848537445, + 0.9234761595726013, + -0.33946576714515686, + 0.2668255567550659, + -0.21642044186592102, + 0.5691895484924316, + 0.6546608209609985, + -0.41415712237358093, + -0.4193723201751709, + -1.2848857641220093, + 0.2762640118598938, + -0.41601526737213135, + 0.46642565727233887, + -0.1210404485464096, + 0.32488104701042175, + -1.2147209644317627, + 0.19869518280029297, + 0.1881680190563202 + ], + [ + 0.10475245118141174, + 0.043515369296073914, + 0.16247892379760742, + 0.2928580343723297, + 1.027008056640625, + -0.20103126764297485, + 0.316720187664032, + -0.2060665786266327, + 0.5696927905082703, + 0.7628860473632812, + -0.3807542026042938, + -0.24436303973197937, + -1.3558554649353027, + 0.13071060180664062, + -0.4661105275154114, + 0.49856120347976685, + 0.032164350152015686, + 0.17239071428775787, + -1.2636163234710693, + 0.018511801958084106, + -0.017703816294670105 + ], + [ + 0.041100695729255676, + 0.08698694407939911, + 0.10086056590080261, + 0.06992438435554504, + 1.004431962966919, + -0.6041911840438843, + 1.1277751922607422, + 0.41241931915283203, + 0.47663602232933044, + 0.6032533049583435, + -0.413040429353714, + -0.8885858058929443, + -1.3979250192642212, + 0.21834880113601685, + -0.4339330494403839, + 0.4801313281059265, + 0.1524370163679123, + 0.4601544141769409, + -1.3526644706726074, + 0.5546669363975525, + -0.05564048886299133 + ], + [ + 0.28570207953453064, + 0.26462239027023315, + 0.41819876432418823, + 0.200236976146698, + 1.0851836204528809, + -0.38349828124046326, + -0.07097262144088745, + -0.35254666209220886, + 0.48069003224372864, + 0.5254405736923218, + -0.6000253558158875, + -0.6434926986694336, + -0.9392081499099731, + 0.11152574419975281, + -0.4522472620010376, + 0.4310823082923889, + 0.01918680965900421, + 0.39273834228515625, + -1.1843786239624023, + -0.08674001693725586, + 0.24136638641357422 + ], + [ + 0.22639581561088562, + 0.21364231407642365, + 0.29420721530914307, + 0.29290685057640076, + 0.8009777665138245, + -0.5039989948272705, + -0.1242033839225769, + -0.18591883778572083, + 0.4670332074165344, + 0.5588225722312927, + -0.6637930870056152, + -0.48225724697113037, + -1.1685916185379028, + 0.20823800563812256, + -0.4058948755264282, + 0.3993051052093506, + 0.032337650656700134, + 0.4755861163139343, + -1.4540271759033203, + 0.04864993691444397, + -0.015121623873710632 + ], + [ + 0.15841281414031982, + 0.04041200876235962, + 0.412304162979126, + 0.35304316878318787, + 1.0737528800964355, + -0.2817305624485016, + -0.140988290309906, + -0.246922105550766, + 0.5877202749252319, + 0.7453543543815613, + -0.5220662951469421, + -0.5889569520950317, + -1.1438002586364746, + 0.05441661924123764, + -0.4762595295906067, + 0.5195172429084778, + 0.03120456635951996, + 0.3142508268356323, + -1.433992624282837, + -0.1491376906633377, + 0.13459615409374237 + ] + ], + "generated_utilities": [ + 0.12168914079666138, + 0.08366147428750992, + 0.15701176226139069, + 0.12812411785125732, + 0.18499283492565155, + 0.08666101843118668, + 0.09535380452871323, + 0.09223812818527222 + ], + "group_id": "ms-0345d8ac530f88cefd61", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.007898053154349327, + 0.0058197579346597195, + 0.052657388150691986, + 0.016109248623251915, + 0.1698339879512787, + 0.008177022449672222, + 0.12333428859710693, + 0.1150994747877121, + 0.15141737461090088, + 0.08515926450490952, + 0.1523766964673996, + 0.04073335975408554, + 0.027175527065992355, + 0.1798098385334015, + 0.08658158779144287, + 0.0618208646774292 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2244938164949417, + -0.708204448223114, + 1.4058912992477417, + 0.19480958580970764, + 1.1751772165298462, + -1.2265020608901978, + -0.8636159896850586, + -0.07021944224834442, + 1.2708500623703003, + 1.0246398448944092, + -1.7615737915039062, + -0.44495686888694763, + -1.2927910089492798, + 0.27074551582336426, + -1.6349679231643677, + 0.9748592376708984, + 1.0210484266281128, + 0.36906859278678894, + -2.528050661087036, + 0.7326674461364746, + 1.553327202796936 + ], + [ + 0.35595428943634033, + -0.8224175572395325, + 1.1192705631256104, + 0.4673205614089966, + 1.1115962266921997, + -1.4676826000213623, + -1.057816505432129, + -0.008461203426122665, + 0.9729399085044861, + 0.8019976615905762, + -1.4944581985473633, + -0.21817980706691742, + -1.3722625970840454, + 0.3557305932044983, + -1.8202308416366577, + 1.2552388906478882, + 0.7855625748634338, + 0.5709130167961121, + -2.4140331745147705, + 0.9063631296157837, + 1.282220721244812 + ], + [ + 0.19927366077899933, + -0.7607958912849426, + 1.4928451776504517, + 0.38265493512153625, + 1.2970150709152222, + -1.2964622974395752, + -0.5547049045562744, + -0.2585906684398651, + 1.3020356893539429, + 1.0737547874450684, + -1.5593467950820923, + -0.1617565155029297, + -1.4868489503860474, + 0.15243440866470337, + -1.4100472927093506, + 0.7991768717765808, + 1.0339381694793701, + 0.6228882074356079, + -2.8132376670837402, + 0.7685560584068298, + 1.8755284547805786 + ], + [ + 0.6393168568611145, + -0.2793915271759033, + 1.4690577983856201, + 0.4745514690876007, + 0.9989874362945557, + -0.7359552979469299, + -1.2171094417572021, + -0.39386802911758423, + 1.1646839380264282, + 0.6661151051521301, + -1.9162347316741943, + -0.43809595704078674, + -1.3973355293273926, + 0.7664861679077148, + -1.553026795387268, + 1.1995127201080322, + 1.285676121711731, + 0.822955846786499, + -2.574894666671753, + 0.3542059361934662, + 1.5986168384552002 + ], + [ + 0.26509445905685425, + -0.5443360805511475, + 1.2502005100250244, + 0.6112543940544128, + 0.924185037612915, + -1.0554168224334717, + -0.7927505373954773, + -0.07152043282985687, + 0.9228929281234741, + 1.545881986618042, + -1.6973015069961548, + -0.496639221906662, + -0.8727940320968628, + 0.5596445202827454, + -2.1939897537231445, + 1.239556074142456, + 1.4699029922485352, + -0.182618647813797, + -2.7263355255126953, + 0.976334810256958, + 1.0402796268463135 + ], + [ + 0.1353418529033661, + -1.1921336650848389, + 1.6913124322891235, + -0.43279919028282166, + 1.1331026554107666, + -1.8503069877624512, + -0.46699997782707214, + 0.5785945653915405, + 1.5516390800476074, + 0.38185298442840576, + -2.192314386367798, + -0.6304612159729004, + -1.6237419843673706, + -0.06383657455444336, + -1.2674099206924438, + 0.7967972755432129, + 1.2453874349594116, + 0.01532512903213501, + -2.6341590881347656, + 0.8366512656211853, + 1.9056174755096436 + ], + [ + 1.0916199684143066, + 0.23665857315063477, + 1.7737994194030762, + -0.8046588897705078, + 1.944507360458374, + -1.3606563806533813, + -0.022103488445281982, + 0.5769228935241699, + 1.0605480670928955, + 1.3100652694702148, + -1.9243974685668945, + 0.43040379881858826, + -1.9046732187271118, + -0.49132758378982544, + -1.9095335006713867, + 1.8850300312042236, + 1.366606593132019, + 0.37651723623275757, + -2.86037540435791, + 0.7753443717956543, + 2.072996139526367 + ], + [ + 0.7277938723564148, + 0.23822063207626343, + 0.4208102226257324, + -0.18717050552368164, + 1.1951282024383545, + -1.1622966527938843, + -0.6617333292961121, + -0.6339871883392334, + 0.8901646137237549, + 0.9977425932884216, + -1.8568902015686035, + 0.3291373550891876, + -0.9377886056900024, + 1.0705609321594238, + -0.90934818983078, + 1.7013394832611084, + 1.5952999591827393, + -0.05327540636062622, + -2.546459197998047, + 0.15792781114578247, + 1.2277774810791016 + ], + [ + 0.9283657073974609, + -1.421413779258728, + 1.6497737169265747, + -0.1984737515449524, + 0.9705085158348083, + -0.8917888402938843, + 0.026184558868408203, + 0.09181642532348633, + 0.6335604786872864, + 1.6531281471252441, + -1.4918432235717773, + -0.01735416054725647, + -0.5425058603286743, + 0.7866935133934021, + -0.7011347413063049, + 0.9797741770744324, + 1.6596866846084595, + -0.13051137328147888, + -2.692556858062744, + -0.05617690086364746, + 0.9771407842636108 + ] + ], + "predicted_scores": [ + 3.8818323612213135, + 3.315857172012329, + 3.1084160804748535, + 3.026110887527466, + 3.0173468589782715, + 2.8973727226257324, + 2.8117752075195312, + 2.2416369915008545 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train2", + "selected_index": 0, + "state_hash": "1ca7c651b2dc4638488078318fe8aa293aa6f6da78672d440752daa9f2d4e8a3", + "stored_base_utility": 0.007898053154349327, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07743936777114868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07743936777114868 + }, + "base_progress": 0.07743936777114868, + "base_success": false, + "base_utility": 0.07743936777114868, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.010246972553431988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010246972553431988 + }, + { + "contact_quality": null, + "progress": 0.009836931712925434, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009836931712925434 + }, + { + "contact_quality": null, + "progress": 0.010462522506713867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010462522506713867 + }, + { + "contact_quality": null, + "progress": 0.016623934730887413, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016623934730887413 + }, + { + "contact_quality": null, + "progress": 0.016291439533233643, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016291439533233643 + }, + { + "contact_quality": null, + "progress": 0.008464436046779156, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008464436046779156 + }, + { + "contact_quality": null, + "progress": 0.007733523845672607, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007733523845672607 + }, + { + "contact_quality": null, + "progress": 0.007662097457796335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007662097457796335 + } + ], + "candidate_progress": [ + 0.010246972553431988, + 0.009836931712925434, + 0.010462522506713867, + 0.016623934730887413, + 0.016291439533233643, + 0.008464436046779156, + 0.007733523845672607, + 0.007662097457796335 + ], + "candidate_source_chart_ids": [ + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-02311c7783bc67e188b4", + "ms-02311c7783bc67e188b4", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-0819eb89e627595e7e1b", + "generated_tangents": [ + [ + -0.27776890993118286, + 0.23108194768428802, + 0.7111798524856567, + 0.7763466835021973, + 0.9968110918998718, + 2.906764268875122, + 1.5565147399902344, + -0.4989220201969147, + 0.23751190304756165, + 1.8172210454940796, + -0.07352765649557114, + 1.018977403640747, + 2.1709094047546387, + 2.8995978832244873, + -0.5208884477615356, + -0.09437926113605499, + 0.8423108458518982, + 0.5564112663269043, + 0.42147862911224365, + 2.1392533779144287, + 1.0801944732666016 + ], + [ + -0.1273861676454544, + -0.1396070420742035, + 0.6843858957290649, + 0.24298560619354248, + 0.9184566140174866, + 2.1182150840759277, + 1.0224272012710571, + -0.6168533563613892, + 0.18282200396060944, + 1.6481122970581055, + 0.15509067475795746, + 1.137547254562378, + 2.268160820007324, + 1.603983759880066, + -0.5772895812988281, + -0.2041473090648651, + 0.9118913412094116, + 0.704179584980011, + 0.5581766366958618, + 1.6132352352142334, + 0.335203617811203 + ], + [ + -0.07269053161144257, + -0.07543421536684036, + 0.7156150341033936, + 0.1343604326248169, + 0.7264622449874878, + 2.192913770675659, + 1.2316503524780273, + -0.6132524013519287, + 0.22511693835258484, + 1.69596266746521, + 0.17755138874053955, + 1.3399709463119507, + 2.240403652191162, + 1.60073983669281, + -0.4946681261062622, + -0.20121850073337555, + 0.8695274591445923, + 0.6954562664031982, + 0.5624851584434509, + 1.67295241355896, + 0.40039920806884766 + ], + [ + -0.19697685539722443, + -0.05964571610093117, + 0.7484403848648071, + 0.49418261647224426, + 1.358931303024292, + 2.1272594928741455, + 1.354171633720398, + -0.20623332262039185, + 0.1664104014635086, + 1.4757176637649536, + 0.10749824345111847, + 1.0062272548675537, + 1.3730510473251343, + 1.345031499862671, + -0.4678349494934082, + -0.004985216073691845, + 0.6688669323921204, + 0.34790584444999695, + 0.3010469973087311, + 1.613962173461914, + 1.203932523727417 + ], + [ + -0.29352378845214844, + -0.030778557062149048, + 0.6847835779190063, + 0.4548744857311249, + 1.434896469116211, + 1.9511553049087524, + 1.2712209224700928, + -0.2169855535030365, + 0.16551634669303894, + 1.4247666597366333, + 0.042481109499931335, + 1.1290748119354248, + 1.3033252954483032, + 1.440265417098999, + -0.4396582841873169, + -0.036126572638750076, + 0.7003827095031738, + 0.4048020839691162, + 0.24394062161445618, + 1.579622745513916, + 1.2323073148727417 + ], + [ + -0.3463971018791199, + 0.010657303035259247, + 0.5432380437850952, + 0.30100831389427185, + 1.5649447441101074, + 1.5867369174957275, + 0.6540951132774353, + -0.5391755104064941, + -0.02725127898156643, + 1.5252676010131836, + 0.06962070614099503, + 1.1831287145614624, + 1.8866065740585327, + 1.6422253847122192, + -0.4391603469848633, + -0.12437775731086731, + 1.1717960834503174, + 0.9279109239578247, + 0.4423338770866394, + 1.3115346431732178, + 1.173444390296936 + ], + [ + -0.6493239998817444, + -0.0025205612182617188, + 0.35217803716659546, + 0.039193183183670044, + 2.0393121242523193, + 2.1324617862701416, + 0.8784822225570679, + -0.5305458307266235, + -0.0757417157292366, + 1.5512365102767944, + 0.19312569499015808, + 0.9525071382522583, + 1.7539006471633911, + 1.5624492168426514, + -0.5603730082511902, + -0.012362434528768063, + 1.032532811164856, + 0.4476777911186218, + 0.521888792514801, + 1.6565124988555908, + 1.1062722206115723 + ], + [ + -0.434856116771698, + 0.03118390589952469, + 0.4904160797595978, + 0.24183911085128784, + 1.500748872756958, + 1.5910472869873047, + 0.7564234733581543, + -0.6446245908737183, + 0.06703783571720123, + 1.3886953592300415, + 0.05164824053645134, + 1.3714869022369385, + 1.9783786535263062, + 1.5633913278579712, + -0.3470684587955475, + -0.16040073335170746, + 1.1508047580718994, + 0.7564806342124939, + 0.5159573554992676, + 1.4810285568237305, + 1.4211665391921997 + ] + ], + "generated_utilities": [ + 0.010246972553431988, + 0.009836931712925434, + 0.010462522506713867, + 0.016623934730887413, + 0.016291439533233643, + 0.008464436046779156, + 0.007733523845672607, + 0.007662097457796335 + ], + "group_id": "ms-0819eb89e627595e7e1b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07638250291347504, + 0.11176031827926636, + 0.139996737241745, + 0.06282445043325424, + 0.08466696739196777, + 0.0763862133026123, + 0.05953681468963623, + 0.06854937970638275, + 0.06834080070257187, + 0.09309522807598114, + 0.14572584629058838, + 0.07182137668132782, + 0.020047685131430626, + 0.05106111615896225, + 0.03756682202219963, + 0.061605654656887054 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.04522436857223511, + -0.3834587037563324, + 2.32004976272583, + 1.6386330127716064, + 1.508955717086792, + 1.681371808052063, + 2.9060468673706055, + -0.5984516143798828, + 0.36428770422935486, + 0.7442611455917358, + 1.163742184638977, + 0.4921322464942932, + 1.9586588144302368, + 3.335318088531494, + -0.02929070219397545, + 0.5442408919334412, + 2.4529502391815186, + 0.17471881210803986, + 1.804263710975647, + 1.5498225688934326, + 1.9392809867858887 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.19627264142036438, + 0.015629824250936508, + 0.723259687423706, + 0.0313839316368103, + 0.02149224281311035, + 0.7142634391784668, + 0.0, + -0.29767167568206787, + -0.10304851830005646, + 0.6168417930603027, + 0.0271989107131958, + 0.369077205657959, + 1.5924487113952637, + 0.0, + -0.11617732048034668, + -0.1419914811849594, + 0.9400725364685059, + -0.04139697551727295, + 0.7166774272918701, + 1.092864990234375, + 0.0 + ], + [ + -0.18197131156921387, + 0.30914804339408875, + 1.4557228088378906, + 1.4375734329223633, + 0.9600651264190674, + 1.7448713779449463, + 3.031123399734497, + -0.6073551177978516, + -0.4749099016189575, + 1.3016932010650635, + 0.32392996549606323, + 0.6961254477500916, + 2.1961381435394287, + 3.730100154876709, + -0.6154190897941589, + -0.3908892273902893, + 2.0994153022766113, + 0.18937554955482483, + 1.5752670764923096, + 2.2457275390625, + 1.5629570484161377 + ] + ], + "predicted_scores": [ + 3.1131200790405273, + 1.5541642904281616, + 1.505118727684021, + 1.397347331047058, + 0.9729986786842346, + 0.37713712453842163, + 0.31507235765457153, + 0.16730530560016632 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train2", + "selected_index": 0, + "state_hash": "582daf047d78a1b01018d54f5798143c4be6129fd7b6cbc025fc85818e519f3c", + "stored_base_utility": 0.07638250291347504, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19400137662887573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19400137662887573 + }, + "base_progress": 0.19400137662887573, + "base_success": false, + "base_utility": 0.19400137662887573, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.047912776470184326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.047912776470184326 + }, + { + "contact_quality": null, + "progress": 0.16342134773731232, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16342134773731232 + }, + { + "contact_quality": null, + "progress": 0.09078226238489151, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09078226238489151 + }, + { + "contact_quality": null, + "progress": 0.11114299297332764, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11114299297332764 + }, + { + "contact_quality": null, + "progress": 0.14748775959014893, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14748775959014893 + }, + { + "contact_quality": null, + "progress": 0.13230924308300018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13230924308300018 + }, + { + "contact_quality": null, + "progress": 0.11984594911336899, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11984594911336899 + }, + { + "contact_quality": null, + "progress": 0.0701046958565712, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0701046958565712 + } + ], + "candidate_progress": [ + 0.047912776470184326, + 0.16342134773731232, + 0.09078226238489151, + 0.11114299297332764, + 0.14748775959014893, + 0.13230924308300018, + 0.11984594911336899, + 0.0701046958565712 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "generated_tangents": [ + [ + 0.9212105870246887, + 0.2495684176683426, + -0.7139983177185059, + -0.26197585463523865, + -0.16777950525283813, + -3.125133514404297, + -1.1343600749969482, + -0.845574140548706, + 0.1403491497039795, + 1.0603828430175781, + 1.0307753086090088, + 0.25075656175613403, + -1.175748586654663, + -0.15695838630199432, + -1.2099212408065796, + 0.036964427679777145, + 0.6747310757637024, + 1.717119812965393, + -0.639168381690979, + 0.5787088871002197, + 0.5156751871109009 + ], + [ + 1.031951904296875, + 0.10456676781177521, + -0.0990132987499237, + -0.8342064619064331, + 0.048051148653030396, + -3.0087099075317383, + -1.678978443145752, + -0.4555343687534332, + -0.05653058737516403, + 0.6307278275489807, + 0.35930153727531433, + -0.1259935200214386, + -1.54985511302948, + 0.01895079016685486, + -1.026986002922058, + -0.02540828101336956, + 0.022382229566574097, + 1.5241296291351318, + -1.186220407485962, + -0.3008204996585846, + -0.03713391721248627 + ], + [ + 0.9142172336578369, + 0.28473961353302, + -0.7407100200653076, + -0.4550410211086273, + 0.16757476329803467, + -2.450864791870117, + -1.1916437149047852, + -0.7311481237411499, + 0.011652294546365738, + 1.0108829736709595, + 0.9055538177490234, + 0.27949410676956177, + -1.4278541803359985, + -0.7882205247879028, + -1.1272212266921997, + 0.030480921268463135, + 0.27419817447662354, + 1.558565378189087, + -0.9363412261009216, + 0.057448357343673706, + 0.1634758859872818 + ], + [ + 0.8624707460403442, + 0.14740881323814392, + -0.5647199153900146, + -0.3894729018211365, + 0.13196495175361633, + -3.027218818664551, + -0.9061381220817566, + -0.737486720085144, + 0.03615472838282585, + 0.8685611486434937, + 0.8174701929092407, + -0.1609852910041809, + -1.6961089372634888, + -0.34921181201934814, + -1.0282783508300781, + 0.009532948955893517, + 0.2734566330909729, + 1.357501745223999, + -0.7216119766235352, + 0.01734524965286255, + 0.07203369587659836 + ], + [ + 0.9260347485542297, + 0.17288383841514587, + -0.5587950944900513, + -0.7025783061981201, + -0.01387980580329895, + -2.9452710151672363, + -0.9583986401557922, + -0.6184329986572266, + 0.008266683667898178, + 0.7364510297775269, + 0.6600156426429749, + -0.0791592001914978, + -1.6208281517028809, + -0.3753531575202942, + -1.1459007263183594, + -0.022931182757019997, + 0.1371820569038391, + 1.3399425745010376, + -0.8164740204811096, + -0.11073678731918335, + 0.2146928310394287 + ], + [ + 0.8769559860229492, + 0.06960363686084747, + -0.3903687298297882, + -0.33665645122528076, + -0.08544418215751648, + -2.8250210285186768, + -1.2471399307250977, + -0.6057186126708984, + 0.0977524146437645, + 0.8803013563156128, + 0.4490353763103485, + 0.02901233732700348, + -1.8310905694961548, + -0.4052550792694092, + -1.017613410949707, + 0.02303798496723175, + -0.03266887366771698, + 1.3069043159484863, + -1.017390489578247, + -0.5154598951339722, + -0.33940544724464417 + ], + [ + 1.0058937072753906, + 0.21089066565036774, + -0.47070208191871643, + -0.6376264095306396, + -0.10214352607727051, + -2.9263365268707275, + -1.0027015209197998, + -0.6247806549072266, + 0.056414879858493805, + 0.8938659429550171, + 0.6290707588195801, + 0.04730623960494995, + -1.6837979555130005, + -0.3781541585922241, + -1.1397086381912231, + -0.018741093575954437, + 0.14122837781906128, + 1.416974425315857, + -0.8748225569725037, + -0.10672138631343842, + 0.0648472011089325 + ], + [ + 0.9055119752883911, + 0.2079230099916458, + -0.10746070742607117, + -0.46652254462242126, + -0.21936234831809998, + -3.202359437942505, + -1.061741590499878, + -0.6140661239624023, + 0.07765284925699234, + 1.0384798049926758, + 0.6780832409858704, + 0.13602565228939056, + -1.3173234462738037, + -0.7119632363319397, + -1.2493627071380615, + 0.1700039654970169, + 0.18064826726913452, + 1.6471635103225708, + -0.8899497389793396, + -0.11035288870334625, + -0.1744442582130432 + ] + ], + "generated_utilities": [ + 0.047912776470184326, + 0.16342134773731232, + 0.09078226238489151, + 0.11114299297332764, + 0.14748775959014893, + 0.13230924308300018, + 0.11984594911336899, + 0.0701046958565712 + ], + "group_id": "ms-0dee962a5901fb8b5bbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23161475360393524, + 0.23611454665660858, + 0.1380448043346405, + 2.0, + 0.25444933772087097, + 0.25831863284111023, + 0.22753827273845673, + 0.2483263909816742, + 0.21622240543365479, + 0.17534737288951874, + 0.23816657066345215, + 0.09714888036251068, + 0.05844185873866081, + 0.12679120898246765, + 0.0472835898399353, + 0.03277532383799553 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7612967491149902, + 0.192471444606781, + -0.2699308395385742, + 0.017076104879379272, + -0.14400714635849, + -1.9032442569732666, + 0.0, + -0.28746497631073, + -0.2857179641723633, + -0.09024286270141602, + -0.03538453206419945, + 0.06432074308395386, + -0.2535901665687561, + 0.0, + 0.06180626153945923, + 0.16256888210773468, + -0.031329914927482605, + 1.563896656036377, + 0.04795600473880768, + 0.02485179901123047, + 0.0 + ], + [ + 1.2132649421691895, + 0.536715567111969, + -1.4273607730865479, + -1.0538015365600586, + -0.5170869827270508, + -3.4203505516052246, + -2.3889083862304688, + -0.7717514634132385, + -1.1465628147125244, + 0.8393621444702148, + -0.4419952630996704, + -0.23069629073143005, + -0.0637764036655426, + -1.5193647146224976, + -1.0273863077163696, + 0.39812424778938293, + -0.02874515950679779, + 2.7962026596069336, + 0.2370568960905075, + 1.5884850025177002, + 0.9987253546714783 + ], + [ + 2.1451404094696045, + 0.09629477560520172, + -1.8500688076019287, + 0.012462526559829712, + 0.13087880611419678, + -2.4062256813049316, + -3.303264856338501, + -1.3855981826782227, + -0.0292893648147583, + -0.17000514268875122, + 0.0005041323602199554, + -0.7487195134162903, + 0.06868582963943481, + -2.107095718383789, + -0.5890907645225525, + 0.1777246743440628, + -0.5576009750366211, + 2.767899990081787, + -0.3420543074607849, + 1.1357430219650269, + 1.8840694427490234 + ], + [ + 2.032343626022339, + 0.4649246335029602, + -0.5597869753837585, + 0.37935349345207214, + 0.14823919534683228, + -3.389478921890259, + -1.972822904586792, + -0.0014220476150512695, + -0.6147069334983826, + 0.9483420252799988, + 0.641323983669281, + -0.7033213973045349, + 0.4629128575325012, + -1.6774024963378906, + -1.1420938968658447, + 0.5891244411468506, + -0.712251603603363, + 2.1924612522125244, + 0.18694229423999786, + 0.2795591354370117, + 2.4723148345947266 + ], + [ + 2.299381732940674, + -0.6742787957191467, + -0.886760950088501, + -0.1637115478515625, + 0.32663029432296753, + -1.9267126321792603, + -1.9424545764923096, + -0.3586595058441162, + -1.1000827550888062, + 0.45532336831092834, + -0.4314403235912323, + -0.12869995832443237, + -0.032513588666915894, + -2.046614170074463, + -1.2942235469818115, + 0.3962012231349945, + 0.797533392906189, + 2.202794313430786, + 0.4938191771507263, + 0.15863806009292603, + 1.745660662651062 + ], + [ + 2.107815742492676, + 0.963295042514801, + -1.2979395389556885, + -1.0134333372116089, + -1.319197177886963, + -3.176518440246582, + -2.4782180786132812, + -1.3465216159820557, + -1.0473096370697021, + -0.22631770372390747, + 0.7881518006324768, + -1.4033093452453613, + 0.3855947256088257, + -1.3816941976547241, + -1.1257284879684448, + -0.7653472423553467, + 0.6845007538795471, + 2.9796371459960938, + 0.1967305988073349, + 1.4200057983398438, + 1.2843091487884521 + ], + [ + 1.7303649187088013, + 1.0728741884231567, + -1.1676844358444214, + -0.20355769991874695, + -0.8773125410079956, + -3.2701919078826904, + -2.1479928493499756, + -0.044036924839019775, + -1.252698302268982, + -0.01393243670463562, + 0.6447213292121887, + -0.7088868618011475, + 0.11592492461204529, + -2.1431162357330322, + -0.8284967541694641, + 0.4642922580242157, + -0.5207862854003906, + 1.8997660875320435, + -0.6910213828086853, + 0.2906065583229065, + 1.4081878662109375 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7612967491149902, + -0.5331565737724304, + -0.2699308395385742, + -0.20421931147575378, + -0.22613269090652466, + -1.9032442569732666, + 0.0, + 0.09110367298126221, + 0.009229481220245361, + 0.004811525344848633, + 0.656868577003479, + -0.28485387563705444, + -0.31901538372039795, + 0.0, + 0.08918565511703491, + 0.3355104923248291, + 0.026827186346054077, + 1.563896656036377, + 0.13480885326862335, + 0.02485179901123047, + 0.0 + ] + ], + "predicted_scores": [ + 7.601155757904053, + 6.7186150550842285, + 6.455691814422607, + 6.337220668792725, + 6.190089702606201, + 5.931434631347656, + 5.5402512550354, + 5.225919246673584 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train2", + "selected_index": 0, + "state_hash": "73f2826502af62b561f13c32614947588918c3deb631165739e66d787008fa98", + "stored_base_utility": 0.23161475360393524, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.04583283141255379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04583283141255379 + }, + "base_progress": 0.04583283141255379, + "base_success": false, + "base_utility": 0.04583283141255379, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06292174011468887, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06292174011468887 + }, + { + "contact_quality": null, + "progress": 0.06575056165456772, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06575056165456772 + }, + { + "contact_quality": null, + "progress": 0.04468737170100212, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04468737170100212 + }, + { + "contact_quality": null, + "progress": 0.05525680258870125, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05525680258870125 + }, + { + "contact_quality": null, + "progress": 0.05402831360697746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05402831360697746 + }, + { + "contact_quality": null, + "progress": 0.05949234962463379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05949234962463379 + }, + { + "contact_quality": null, + "progress": 0.06155836582183838, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06155836582183838 + }, + { + "contact_quality": null, + "progress": 0.07069015502929688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07069015502929688 + } + ], + "candidate_progress": [ + 0.06292174011468887, + 0.06575056165456772, + 0.04468737170100212, + 0.05525680258870125, + 0.05402831360697746, + 0.05949234962463379, + 0.06155836582183838, + 0.07069015502929688 + ], + "candidate_source_chart_ids": [ + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-38dff128761905bb1507", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-126eb8b36769f9ba4671", + "generated_tangents": [ + [ + 0.655849277973175, + 0.30569660663604736, + -0.2553529441356659, + -0.263614296913147, + -0.017318665981292725, + -2.6994242668151855, + -1.2797112464904785, + 1.3210679292678833, + -0.20580926537513733, + -0.34731701016426086, + 0.0599541999399662, + 1.1947097778320312, + -0.9926650524139404, + -0.22014300525188446, + -0.5412349700927734, + -0.006252258084714413, + 0.7201115489006042, + 0.35588404536247253, + 0.7991909384727478, + 1.6356327533721924, + 1.4817169904708862 + ], + [ + 0.5567323565483093, + 0.2999644875526428, + -0.22638460993766785, + -0.2857460081577301, + -0.12410396337509155, + -2.6662635803222656, + -1.1602342128753662, + 1.3528891801834106, + -0.19935408234596252, + -0.3467772305011749, + 0.14726582169532776, + 1.185856819152832, + -0.9297693967819214, + -0.17500300705432892, + -0.5039250254631042, + 0.00262271985411644, + 0.7704614400863647, + 0.38718703389167786, + 0.7173773050308228, + 1.7797887325286865, + 1.5975384712219238 + ], + [ + 0.6345928907394409, + 0.22035647928714752, + -0.3399869501590729, + -0.5475633144378662, + -0.6624675393104553, + -2.795095443725586, + -1.7817482948303223, + 1.4510209560394287, + -0.07520416378974915, + -0.5234764814376831, + 0.20408061146736145, + 1.3866987228393555, + -0.9252206087112427, + -0.6048867106437683, + -0.4693869352340698, + -0.05610787868499756, + 0.9850830435752869, + 0.2763737142086029, + 1.0204551219940186, + 1.0851173400878906, + 1.6064753532409668 + ], + [ + 0.6709209680557251, + 0.22418354451656342, + -0.5388002395629883, + -0.38371536135673523, + -0.5098652243614197, + -3.042370080947876, + -1.8117797374725342, + 1.4727284908294678, + -0.1018194705247879, + -0.40387922525405884, + 0.344486266374588, + 1.4169539213180542, + -1.0792994499206543, + -0.6196965575218201, + -0.34013456106185913, + -0.010105160064995289, + 0.863153874874115, + 0.366731733083725, + 0.9567697644233704, + 1.240022897720337, + 1.4136368036270142 + ], + [ + 0.7780305743217468, + 0.15041577816009521, + -0.03117486834526062, + -0.3764033913612366, + -0.3695783019065857, + -3.130204916000366, + -1.807060718536377, + 1.5753087997436523, + -0.17455875873565674, + -0.4455745816230774, + 0.33693787455558777, + 1.3635772466659546, + -0.7788095474243164, + -0.10962678492069244, + -0.20459851622581482, + -0.09904170036315918, + 0.6942819952964783, + 0.9156771898269653, + 0.57056725025177, + 1.858638048171997, + 0.898651123046875 + ], + [ + 0.5313800573348999, + 0.1547086536884308, + -0.4544283449649811, + -0.299678236246109, + 0.184084951877594, + -3.0529234409332275, + -1.6618046760559082, + 1.3184638023376465, + -0.1364424079656601, + -0.1288485825061798, + 0.4071817696094513, + 1.2905830144882202, + -0.7680494785308838, + -0.5091203451156616, + -0.3352624177932739, + -0.1254398226737976, + 0.7647642493247986, + 0.38689929246902466, + 0.3632575571537018, + 1.2556016445159912, + 1.437532901763916 + ], + [ + 0.5951058268547058, + 0.18803085386753082, + -0.6277573108673096, + -0.3599866032600403, + -0.4260461926460266, + -3.027994394302368, + -1.897934913635254, + 1.447753667831421, + -0.15206478536128998, + -0.3560253381729126, + 0.16618674993515015, + 1.4100892543792725, + -1.074182152748108, + -0.5514003038406372, + -0.37114202976226807, + -0.009665049612522125, + 0.8715071678161621, + 0.30882224440574646, + 0.9878675937652588, + 1.1838805675506592, + 1.028620958328247 + ], + [ + 0.4412482976913452, + 0.1250564008951187, + -0.45928481221199036, + -0.23636017739772797, + 0.05779939889907837, + -3.066941738128662, + -1.807917833328247, + 1.2976337671279907, + -0.10770668089389801, + -0.17227691411972046, + 0.4780229330062866, + 1.4018090963363647, + -0.8124645352363586, + -0.39200007915496826, + -0.3194437026977539, + -0.08693481981754303, + 0.6708048582077026, + 0.3947564959526062, + 0.3704075813293457, + 1.3500699996948242, + 1.2606252431869507 + ] + ], + "generated_utilities": [ + 0.06292174011468887, + 0.06575056165456772, + 0.04468737170100212, + 0.05525680258870125, + 0.05402831360697746, + 0.05949234962463379, + 0.06155836582183838, + 0.07069015502929688 + ], + "group_id": "ms-126eb8b36769f9ba4671", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05304628610610962, + 0.14190539717674255, + 0.1985912322998047, + 0.1210482120513916, + 0.18570420145988464, + 0.05259696766734123, + 0.1827520728111267, + 0.15291261672973633, + 0.07772710174322128, + 0.15551692247390747, + 0.08802877366542816, + 0.07544025033712387, + 0.08058679103851318, + 0.10843702405691147, + 0.023751219734549522, + 0.00727109145373106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7799564599990845, + -0.0863567665219307, + -0.04370701313018799, + -0.04603701829910278, + -0.003975361585617065, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.02617853879928589, + 0.009054642170667648, + -0.050068795680999756, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + -0.007249712944030762, + 0.0760425329208374, + 1.0404269695281982, + -0.04351449012756348, + -0.010379016399383545, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + 0.005863763391971588, + -0.04370701313018799, + 0.010433673858642578, + -0.05825299024581909, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.22243881225585938, + -0.007246788591146469, + 0.19961372017860413, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.11177664995193481, + -0.1665668487548828, + 1.0404269695281982, + -0.09226274490356445, + -0.18212640285491943, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + -0.7482962012290955, + -0.04370701313018799, + -0.2370874285697937, + 0.3763168752193451, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + -0.441777765750885, + -0.44057023525238037, + -0.6443575024604797, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.008161306381225586, + 0.22021520137786865, + 1.0404269695281982, + -0.09226274490356445, + -0.35497862100601196, + 1.0522968769073486, + 0.0 + ], + [ + 1.7799564599990845, + 0.04087676852941513, + -1.043707013130188, + -0.8482787609100342, + 0.3640388548374176, + -3.3430333137512207, + -2.8715221881866455, + 1.8326367139816284, + -0.4197058081626892, + -0.040397901087999344, + -0.37471577525138855, + 1.7451434135437012, + -1.6306462287902832, + -0.7785218954086304, + -0.5121169090270996, + -0.7797847986221313, + 2.0404269695281982, + 0.9077372550964355, + 0.645021378993988, + 2.0522968769073486, + 1.2749258279800415 + ], + [ + 1.6519862413406372, + 0.1747584044933319, + -0.9129671454429626, + -0.6869257092475891, + 0.12357090413570404, + -3.2319328784942627, + -2.704270839691162, + 1.7347972393035889, + -0.40612348914146423, + 0.23568139970302582, + -0.3912501931190491, + 1.8630532026290894, + -1.8548930883407593, + -0.9794526100158691, + -0.25177696347236633, + -0.6884964108467102, + 2.237109422683716, + 1.1246912479400635, + 0.7490227222442627, + 2.320941686630249, + 1.2039268016815186 + ], + [ + 1.7142105102539062, + 0.3941279351711273, + -1.021533489227295, + -1.1548089981079102, + 0.5052449107170105, + -3.714951992034912, + -3.04028058052063, + 1.529634952545166, + -0.22215746343135834, + 0.19690367579460144, + -0.12262976169586182, + 1.4847477674484253, + -1.490212082862854, + -0.7427945137023926, + -0.7506002187728882, + -0.9221256375312805, + 1.9211885929107666, + 0.5114957094192505, + 0.30887648463249207, + 2.4437777996063232, + 1.3625030517578125 + ], + [ + 1.3140592575073242, + -0.1129835918545723, + -0.9130957722663879, + -1.1089756488800049, + 0.7560886144638062, + -3.3889732360839844, + -3.1182656288146973, + 1.7436503171920776, + -0.9962512254714966, + 0.43109408020973206, + -0.8453629016876221, + 1.2057833671569824, + -2.194112777709961, + -0.22479140758514404, + -0.7258806824684143, + -0.9702881574630737, + 1.8160195350646973, + 0.649415135383606, + 0.6846981048583984, + 2.1997463703155518, + 1.7655411958694458 + ], + [ + 2.6430320739746094, + -0.7696453332901001, + -1.765415906906128, + -0.8281408548355103, + -0.1440155804157257, + -3.3335886001586914, + -3.398141860961914, + 2.4138498306274414, + 0.47537559270858765, + -0.9514593482017517, + 0.2674855887889862, + 2.6307411193847656, + -0.7876039743423462, + -0.35013434290885925, + -0.7511636018753052, + -0.04397457838058472, + 2.123702049255371, + 1.1604151725769043, + 0.7323322892189026, + 1.88338303565979, + 1.5078264474868774 + ] + ], + "predicted_scores": [ + 6.690057754516602, + 6.604052543640137, + 6.222714900970459, + 6.147878646850586, + 5.996226787567139, + 5.941082000732422, + 5.827487468719482, + 5.642694473266602 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train2", + "selected_index": 0, + "state_hash": "2390fc23ce8b50bdcdcbb1ab5f0583d2dad2b94fb871db73b1a8401265f59987", + "stored_base_utility": 0.05304628610610962, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.025880932807922363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025880932807922363 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.025880932807922363, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-1f19fd22562f87bdb776", + "ms-babaeb20129c46551281", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-067cac2e4dbeddb12239", + "ms-c1620854ba52f09e24d5", + "ms-067cac2e4dbeddb12239", + "ms-cad6645f8175e38ad155" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-16661b2fd2c85f57a66c", + "generated_tangents": [ + [ + -0.6824389100074768, + -0.42353397607803345, + 1.9156477451324463, + 0.27319207787513733, + 1.1869573593139648, + 1.6918898820877075, + 3.035088300704956, + 1.200953722000122, + -0.2849206328392029, + -0.3664309084415436, + -0.02527526021003723, + 0.13283467292785645, + 1.7509948015213013, + 2.0090372562408447, + 0.9902731776237488, + -0.09048666059970856, + -1.5413249731063843, + -0.017665475606918335, + -0.3971679210662842, + -0.03746393322944641, + -1.81447172164917 + ], + [ + -0.5094736218452454, + -0.4551224112510681, + 1.1852335929870605, + -0.39674949645996094, + 1.1378552913665771, + 1.9335203170776367, + 3.5333211421966553, + 1.1739822626113892, + -0.08422532677650452, + -0.17456355690956116, + -0.05542765557765961, + 0.5721486210823059, + 1.4768439531326294, + 2.1573987007141113, + 0.47684183716773987, + -0.41097113490104675, + -1.140610694885254, + -0.003322198987007141, + -0.5198103189468384, + -1.5106821060180664, + -2.2797093391418457 + ], + [ + -0.17949147522449493, + -0.27099424600601196, + 1.509683609008789, + 0.11790081858634949, + 1.112421989440918, + 2.195878028869629, + 3.7887556552886963, + 1.2573126554489136, + -0.15526361763477325, + 0.2515209913253784, + -0.28459781408309937, + 0.695488691329956, + 1.9764295816421509, + 2.660031318664551, + 0.7410562634468079, + -0.15210482478141785, + -0.8240321278572083, + 0.12183477729558945, + -0.057398468255996704, + -0.7678194046020508, + -2.11991548538208 + ], + [ + -0.2508142590522766, + -0.21315686404705048, + 1.3031139373779297, + 0.30783143639564514, + 1.0800400972366333, + 2.267934799194336, + 3.4985358715057373, + 1.4066815376281738, + -0.21535561978816986, + 0.1340244561433792, + -0.3501300811767578, + 0.6035999655723572, + 1.8388158082962036, + 2.672978401184082, + 0.7043142318725586, + -0.11049807071685791, + -0.7912341952323914, + 0.2669382095336914, + 0.012259066104888916, + -0.8488905429840088, + -1.793541431427002 + ], + [ + -0.09856274724006653, + -0.5983659625053406, + 0.6540148258209229, + -0.5010594129562378, + 0.6528372764587402, + 3.1899161338806152, + 3.608482837677002, + 1.4343825578689575, + -0.012374132871627808, + 0.0454079732298851, + -0.42775359749794006, + 0.2738773226737976, + 1.513808012008667, + 2.3734359741210938, + 0.9223951697349548, + -0.2623382806777954, + -1.3865463733673096, + 0.2606728672981262, + -0.2553842067718506, + -1.0965392589569092, + -1.938814640045166 + ], + [ + -0.3856390118598938, + -0.46329641342163086, + 0.7654943466186523, + -0.19061198830604553, + 0.8493849635124207, + 2.530032157897949, + 3.7486729621887207, + 1.2035999298095703, + -0.2593596875667572, + -0.452076256275177, + -0.05605132877826691, + 0.05742599070072174, + 0.48679158091545105, + 1.7778207063674927, + 0.5767878293991089, + -0.3285136818885803, + -0.5391584634780884, + -0.20521382987499237, + -0.0013042688369750977, + -1.6672983169555664, + -2.5346899032592773 + ], + [ + -0.08828580379486084, + -0.7060015201568604, + 0.8980893492698669, + -0.5621912479400635, + 0.8083421587944031, + 2.847586154937744, + 4.094493389129639, + 1.091495394706726, + -0.12258994579315186, + -0.29970160126686096, + -0.511314868927002, + 0.5406684875488281, + 1.621792197227478, + 1.9818919897079468, + 0.7899641990661621, + -0.22454890608787537, + -1.068435549736023, + 0.565302312374115, + -0.02269512414932251, + -1.276261806488037, + -1.887265920639038 + ], + [ + -0.15478308498859406, + -0.167904794216156, + 1.6011892557144165, + 0.09527513384819031, + 1.1605334281921387, + 2.3821661472320557, + 4.203315734863281, + 1.4099047183990479, + -0.20276464521884918, + 0.10758756846189499, + -0.07489912956953049, + 0.6418052911758423, + 1.7774180173873901, + 3.06317138671875, + 0.3030961751937866, + -0.20969876646995544, + -0.37359678745269775, + -0.010742828249931335, + 0.18264299631118774, + -0.5141030550003052, + -1.8507068157196045 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.025880932807922363, + 2.0, + 2.0 + ], + "group_id": "ms-16661b2fd2c85f57a66c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23635749518871307, + 0.1864171028137207, + 0.18280422687530518, + 0.16576561331748962, + 0.2967458665370941, + 0.2833873927593231, + 0.2587948143482208, + 0.2695174813270569, + 0.2239263951778412, + 0.15686756372451782, + 0.2505500912666321, + 0.12250886857509613, + 0.16453635692596436, + 2.0, + 0.13450095057487488, + 0.11914549767971039 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.12099093198776245, + 0.06886856257915497, + 0.2952747344970703, + -0.1764048933982849, + -0.12411242723464966, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + -0.046127498149871826, + 0.12581980228424072, + 0.042072445154190063, + -0.06942430138587952, + 0.7517287731170654, + 0.0, + 0.690804123878479, + -0.06023237854242325, + 0.049633681774139404, + 0.01943105459213257, + 0.013327360153198242, + -0.8525059223175049, + 0.0 + ], + [ + 0.3599843978881836, + -0.34296315908432007, + 0.2952747344970703, + -0.3797188997268677, + -0.41771262884140015, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + 0.06634362041950226, + 0.138022780418396, + -0.2342309057712555, + -0.36017337441444397, + 0.7517287731170654, + 0.0, + 0.690804123878479, + 0.10550708323717117, + -0.35431206226348877, + 0.16382712125778198, + 0.06674730777740479, + -0.8525059223175049, + 0.0 + ], + [ + 0.567882239818573, + -0.4729353189468384, + 1.6256722211837769, + -1.5372763872146606, + 1.0487556457519531, + 1.5327277183532715, + 2.2847697734832764, + 2.6703484058380127, + 0.4013517200946808, + -1.3629915714263916, + 0.6385155916213989, + 0.2664203345775604, + 1.2691267728805542, + 1.2942520380020142, + 2.2084813117980957, + 0.6982719302177429, + -0.8422143459320068, + 1.070263385772705, + -0.9558742046356201, + -1.3996182680130005, + -1.4242117404937744 + ], + [ + 0.6830785274505615, + -0.5584439635276794, + 0.841848611831665, + -0.5293771624565125, + -0.14354348182678223, + 2.097932815551758, + 2.9733188152313232, + 2.603226661682129, + -0.1639159619808197, + -0.918163537979126, + 1.002030849456787, + 0.40132570266723633, + 1.1751198768615723, + 1.455662727355957, + 2.3202755451202393, + -0.18584474921226501, + -1.3483099937438965, + 0.0660635232925415, + 0.4330971837043762, + -1.1989169120788574, + -2.0896859169006348 + ], + [ + 0.44418102502822876, + 0.07469291985034943, + 1.6960139274597168, + -1.329486608505249, + 1.3067576885223389, + 2.2694292068481445, + 2.0299551486968994, + 1.6317954063415527, + 0.7472026348114014, + -0.12518703937530518, + 0.3035789430141449, + -0.5889836549758911, + 2.345867395401001, + 1.9470158815383911, + 0.8398547172546387, + -0.6090301871299744, + -1.5060018301010132, + 0.5695335865020752, + -0.6025367975234985, + -1.8703374862670898, + -2.728482246398926 + ], + [ + 0.31904691457748413, + 0.04995492100715637, + 1.3823881149291992, + -0.5106721520423889, + -0.2236303687095642, + 0.9921484589576721, + 1.5933295488357544, + 2.741320848464966, + -0.21808990836143494, + -0.7775652408599854, + 1.224307894706726, + -0.08168366551399231, + 2.692474842071533, + 2.269430160522461, + 2.3270838260650635, + -0.2901480495929718, + -1.2098140716552734, + -0.07570821046829224, + -1.1340655088424683, + -1.9999847412109375, + -1.6802332401275635 + ], + [ + -0.6257733106613159, + 0.3840647339820862, + 0.9012271165847778, + -1.248098611831665, + -0.1650761365890503, + 2.7645747661590576, + 2.7588071823120117, + 3.0172548294067383, + 0.4120163321495056, + -1.6110656261444092, + -0.10423526167869568, + 0.5334746241569519, + 1.7809263467788696, + 0.5690644383430481, + 1.5881801843643188, + 0.8860509991645813, + -1.1502506732940674, + 0.7821243405342102, + -0.549727201461792, + -1.326378345489502, + -1.5100535154342651 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2773973345756531, + -0.13796104490756989, + 1.2952747344970703, + -1.0323810577392578, + 0.5605664849281311, + 1.8606208562850952, + 2.439758777618408, + 2.353257894515991, + 0.24642756581306458, + -0.861977219581604, + 0.4799116551876068, + 0.264199823141098, + 1.7517287731170654, + 1.4282559156417847, + 1.690804123878479, + 0.1215878501534462, + -0.9322434067726135, + 0.8536378145217896, + -0.35758620500564575, + -1.8525059223175049, + -1.910990595817566 + ], + [ + -0.4916617274284363, + -0.5934253931045532, + 2.074575424194336, + -0.9512085318565369, + 0.3705800175666809, + 1.691048264503479, + 2.2907681465148926, + 1.532329797744751, + 0.8991785049438477, + -0.7733803987503052, + 1.3577321767807007, + 0.9844847917556763, + 1.7379451990127563, + 2.050845146179199, + 0.7804086804389954, + -0.8400509357452393, + -1.6339564323425293, + 0.9922879934310913, + 0.6006971597671509, + -1.0612030029296875, + -1.3134464025497437 + ] + ], + "predicted_scores": [ + 7.575928211212158, + 5.902347087860107, + 4.982975959777832, + 4.682953834533691, + 4.231492519378662, + 3.7651405334472656, + 3.3235254287719727, + 2.9474380016326904 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train2", + "selected_index": 0, + "state_hash": "74b116fd9ceddd5b2261edd96a14934a4b4a78a40191a654d6743661b819f407", + "stored_base_utility": 0.23635749518871307, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0008456110954284668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008456110954284668 + }, + { + "contact_quality": null, + "progress": 0.0010655125370249152, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0010655125370249152 + }, + { + "contact_quality": null, + "progress": 0.0008172591333277524, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008172591333277524 + }, + { + "contact_quality": null, + "progress": 0.0007147590513341129, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007147590513341129 + }, + { + "contact_quality": null, + "progress": 0.0007733504171483219, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007733504171483219 + }, + { + "contact_quality": null, + "progress": 0.0009190241689793766, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009190241689793766 + }, + { + "contact_quality": null, + "progress": 0.000718792260158807, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000718792260158807 + }, + { + "contact_quality": null, + "progress": 0.0012095769634470344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0012095769634470344 + } + ], + "candidate_progress": [ + 0.0008456110954284668, + 0.0010655125370249152, + 0.0008172591333277524, + 0.0007147590513341129, + 0.0007733504171483219, + 0.0009190241689793766, + 0.000718792260158807, + 0.0012095769634470344 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-8e12f7ad446482627dd3", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-20e1efbb745fd02abc95", + "generated_tangents": [ + [ + -0.2970411777496338, + -0.13891279697418213, + 1.6107505559921265, + -0.6531276106834412, + 1.6390912532806396, + 2.8007149696350098, + 1.9549026489257812, + -0.2721187472343445, + 0.03811945766210556, + 1.6218369007110596, + 0.4281376898288727, + 1.174232840538025, + 2.360860586166382, + 2.097445249557495, + -1.3558846712112427, + -0.16812588274478912, + 1.9791476726531982, + 1.03844153881073, + 0.5798172950744629, + 2.5662693977355957, + 1.8120908737182617 + ], + [ + -0.1011519581079483, + -0.2186785191297531, + 1.675442099571228, + -0.8823689222335815, + 1.8577234745025635, + 3.0255486965179443, + 2.3365561962127686, + -0.33499643206596375, + 0.13097485899925232, + 1.560232400894165, + 0.5682781934738159, + 1.2047090530395508, + 2.4729018211364746, + 1.7730436325073242, + -1.165023922920227, + -0.33508002758026123, + 1.8437622785568237, + 0.5554348230361938, + 0.1301388293504715, + 3.2331323623657227, + 1.0670866966247559 + ], + [ + -0.3184179663658142, + -0.06403876841068268, + 1.6588443517684937, + -0.5006476044654846, + 1.9847056865692139, + 3.2442283630371094, + 1.751053810119629, + -0.53663170337677, + 0.056220643222332, + 1.5616052150726318, + 0.4458616077899933, + 1.088004469871521, + 2.0531327724456787, + 2.18585205078125, + -1.1377735137939453, + -0.2569707930088043, + 1.9332804679870605, + 0.8343563079833984, + 0.5231003761291504, + 2.9470410346984863, + 2.1021714210510254 + ], + [ + -0.4304850697517395, + -0.07702029496431351, + 1.6036697626113892, + -0.7226977348327637, + 1.712554693222046, + 3.1905558109283447, + 1.81992769241333, + -0.4713207185268402, + 0.07778530567884445, + 1.6159635782241821, + 0.3994346261024475, + 1.1715067625045776, + 2.044454574584961, + 2.1256802082061768, + -1.3056920766830444, + -0.23062705993652344, + 1.9890633821487427, + 0.8633118271827698, + 0.4869081974029541, + 2.961604118347168, + 1.9759396314620972 + ], + [ + -0.3146146535873413, + -0.06958557665348053, + 1.778191089630127, + -0.53082275390625, + 1.1891772747039795, + 3.3179545402526855, + 1.4995636940002441, + -0.42483940720558167, + 0.14092330634593964, + 1.46672523021698, + 0.5188098549842834, + 1.4428155422210693, + 2.558320999145508, + 2.306028366088867, + -1.2591758966445923, + -0.2865239381790161, + 2.061889171600342, + 0.731976330280304, + 0.6196022033691406, + 3.5407614707946777, + 1.8295236825942993 + ], + [ + -0.1450331062078476, + -0.11107157170772552, + 1.6518384218215942, + -0.7693350315093994, + 1.1015067100524902, + 3.1857244968414307, + 1.6807351112365723, + -0.011720404028892517, + 0.1923666000366211, + 1.7378132343292236, + 0.44404587149620056, + 1.0362420082092285, + 2.450160503387451, + 1.815553069114685, + -1.4958609342575073, + -0.4214487671852112, + 2.145357608795166, + 0.8884361386299133, + 0.7274243831634521, + 2.626995801925659, + 1.2838307619094849 + ], + [ + -0.5335362553596497, + -0.12909740209579468, + 1.5401071310043335, + -0.6026504635810852, + 1.5971753597259521, + 3.320101022720337, + 1.6288726329803467, + -0.4474392235279083, + 0.07987426966428757, + 1.5799193382263184, + 0.5128104090690613, + 1.0947915315628052, + 1.9190994501113892, + 2.4076898097991943, + -1.3900490999221802, + -0.28468674421310425, + 1.9677813053131104, + 0.8986243605613708, + 0.6235958933830261, + 2.8599469661712646, + 2.1499834060668945 + ], + [ + -0.38367998600006104, + -0.203634575009346, + 1.4452296495437622, + -0.5168579816818237, + 1.2648968696594238, + 3.1302900314331055, + 1.5878973007202148, + -0.13402852416038513, + 0.06269306689500809, + 1.4342036247253418, + 0.7396007776260376, + 0.93980473279953, + 1.8660436868667603, + 2.7143394947052, + -1.0735108852386475, + -0.21826890110969543, + 2.111849546432495, + 1.0680543184280396, + 0.5848070383071899, + 3.2526438236236572, + 1.787776231765747 + ] + ], + "generated_utilities": [ + 0.0008456110954284668, + 0.0010655125370249152, + 0.0008172591333277524, + 0.0007147590513341129, + 0.0007733504171483219, + 0.0009190241689793766, + 0.000718792260158807, + 0.0012095769634470344 + ], + "group_id": "ms-20e1efbb745fd02abc95", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13665315508842468, + 0.1696699559688568, + 0.049180448055267334, + 0.05942712351679802, + 0.22736775875091553, + 0.0799722671508789, + 0.17261533439159393, + 0.21202586591243744, + 0.18980810046195984, + 0.24345096945762634, + 0.08349347114562988, + 0.11624568700790405, + 0.17245106399059296, + 0.07060472667217255, + 0.041495781391859055, + 0.028585990890860558 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.08645056188106537, + -0.060760222375392914, + 0.6493666172027588, + -0.016124725341796875, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + -0.2971414625644684, + 0.023367643356323242, + 0.6914786100387573, + -0.1027369350194931, + 0.1733318567276001, + 1.2948026657104492, + 0.0, + -0.30476057529449463, + -0.10394972562789917, + 0.9822354316711426, + 0.055961012840270996, + 0.004969358444213867, + 2.122485637664795, + 0.0 + ], + [ + 0.046016186475753784, + -0.24098780751228333, + 0.6493666172027588, + 0.05737954378128052, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + 0.22233793139457703, + 0.3896067142486572, + 0.6914786100387573, + -0.061455681920051575, + 0.17170286178588867, + 1.2948026657104492, + 0.0, + -0.4717581272125244, + -0.5722612738609314, + 0.9822354316711426, + 0.3690842390060425, + 0.33333829045295715, + 2.122485637664795, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.590132236480713, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.670657634735107, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.27303409576416 + ], + [ + -0.4654892086982727, + -0.6659128069877625, + 2.053840160369873, + -0.17420679330825806, + 1.1238669157028198, + 2.773953914642334, + 2.9706993103027344, + -0.9652835726737976, + -0.6603021621704102, + 1.957632303237915, + 0.30636119842529297, + 1.0932379961013794, + 2.4988603591918945, + 2.1735024452209473, + -0.7229697108268738, + -0.6565687656402588, + 2.1097376346588135, + 0.79427170753479, + 1.1421128511428833, + 3.200641393661499, + 3.3294520378112793 + ], + [ + 0.8783296346664429, + -0.6790766716003418, + 1.3789844512939453, + -1.578566551208496, + 1.2084856033325195, + 1.906846046447754, + 2.7278928756713867, + -0.8570022583007812, + -0.3114888072013855, + 2.660332202911377, + 0.4615887999534607, + -0.16199523210525513, + 1.9830265045166016, + 2.4224376678466797, + -0.9260885715484619, + 0.5555675029754639, + 2.5474588871002197, + 0.0070996880531311035, + 1.627825379371643, + 3.9487287998199463, + 3.03330135345459 + ], + [ + -0.1274794638156891, + -0.26015421748161316, + 2.5414626598358154, + -1.0220305919647217, + 1.4708960056304932, + 2.0982167720794678, + 3.1457836627960205, + -1.0992621183395386, + -0.9075721502304077, + 0.8330968022346497, + 0.15631377696990967, + 1.3095502853393555, + 1.8582203388214111, + 2.101317882537842, + -0.9184049367904663, + 0.4676415026187897, + 2.784534454345703, + 0.4529607892036438, + 0.6403366327285767, + 3.1566896438598633, + 2.5268735885620117 + ], + [ + 0.9289348125457764, + 0.1545984447002411, + 1.6792887449264526, + -0.6400681138038635, + 0.5956984758377075, + 2.5183756351470947, + 1.6261346340179443, + -0.019961506128311157, + -0.2638699412345886, + 1.310987114906311, + 0.4996453523635864, + -0.06606829166412354, + 1.9830868244171143, + 2.392476797103882, + -0.3871336579322815, + -0.7797489166259766, + 1.927225947380066, + 0.4438522458076477, + 1.0257132053375244, + 3.211881160736084, + 3.374328851699829 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05862678587436676, + -0.042836256325244904, + 1.6493666172027588, + -0.7465556859970093, + 1.1865841150283813, + 2.873413562774658, + 2.2950661182403564, + -0.48127153515815735, + -0.11381363868713379, + 1.6914786100387573, + 0.06965510547161102, + 0.8288453221321106, + 2.294802665710449, + 2.3353288173675537, + -1.3047605752944946, + -0.39009055495262146, + 1.9822354316711426, + 0.9993376731872559, + 0.81731116771698, + 3.122485637664795, + 2.63651704788208 + ], + [ + -0.045653510838747025, + 0.00397084653377533, + 1.3591042757034302, + -0.6414353847503662, + 0.8377937078475952, + 2.8277816772460938, + 2.203674554824829, + -0.1951642632484436, + 0.12272481620311737, + 1.7411555051803589, + -0.004818864166736603, + 0.8769108057022095, + 2.243906021118164, + 2.4328396320343018, + -1.6330946683883667, + -0.4807296395301819, + 2.2988247871398926, + 0.8021057844161987, + 0.6970767378807068, + 2.8728103637695312, + 2.288278102874756 + ], + [ + -0.4273069500923157, + 0.16743427515029907, + 1.538269281387329, + -0.7002415657043457, + 0.7622614502906799, + 2.805328369140625, + 2.267277479171753, + -0.6541028022766113, + -0.2980780303478241, + 1.456940770149231, + 0.5105806589126587, + 1.040317177772522, + 2.4321720600128174, + 2.696331024169922, + -1.1555191278457642, + -0.025772064924240112, + 1.6384800672531128, + 1.4557092189788818, + 0.6594157814979553, + 3.1446406841278076, + 2.249678134918213 + ], + [ + -0.6431475877761841, + -0.24224388599395752, + 1.3122833967208862, + -0.16711556911468506, + 0.9232780337333679, + 2.850522994995117, + 2.645284414291382, + -0.379825621843338, + -0.6750738620758057, + 1.1236441135406494, + 0.4938291311264038, + 1.1075668334960938, + 2.203859806060791, + 2.799032688140869, + -1.359815239906311, + -0.5104091763496399, + 2.0970702171325684, + 1.057171106338501, + 1.1167995929718018, + 2.8190994262695312, + 2.2184524536132812 + ] + ], + "predicted_scores": [ + 1.0665324926376343, + 0.9473126530647278, + 0.6300321221351624, + 0.44677799940109253, + 0.2111024409532547, + 0.19814617931842804, + 0.05749352276325226, + -0.8892081379890442 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1375492786/train2", + "selected_index": 0, + "state_hash": "56acb365a02ebac0ac4893531e168df3c430b486416156feeea40eeb178bda3e", + "stored_base_utility": 0.13665315508842468, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2dc5138251afd26a040d", + "generated_tangents": [ + [ + 1.6630635261535645, + 0.3364940285682678, + -1.1061028242111206, + -0.5212184190750122, + 0.6414185166358948, + 0.31022578477859497, + -0.9027884602546692, + 0.8580968379974365, + 0.018770955502986908, + -0.2807011306285858, + -0.08123781532049179, + 0.2720161974430084, + -2.3702707290649414, + -2.2326834201812744, + 0.6190675497055054, + 0.5045605301856995, + 0.8924949169158936, + -1.502112627029419, + 1.245079517364502, + -1.888390302658081, + -0.6553448438644409 + ], + [ + 1.4316965341567993, + 0.1998540759086609, + -1.1474429368972778, + -0.5519155263900757, + 0.22836831212043762, + -0.3073066473007202, + -0.3848276734352112, + 0.5623452663421631, + 0.14031925797462463, + -0.14810025691986084, + -0.30554065108299255, + -0.1590932011604309, + -3.0235400199890137, + -2.117691993713379, + 0.45407184958457947, + 0.37112584710121155, + 0.6169575452804565, + -1.341200828552246, + 0.9203458428382874, + -2.283836603164673, + -1.25898015499115 + ], + [ + 1.5809606313705444, + 0.3620300889015198, + -1.0180070400238037, + -0.8809640407562256, + 0.6343622207641602, + 0.7183874845504761, + -1.0426461696624756, + 1.1562525033950806, + -0.0012709833681583405, + -0.3069444000720978, + -0.26884499192237854, + 0.3377997875213623, + -2.921963930130005, + -2.3162052631378174, + 0.40847247838974, + 0.453095942735672, + 0.9378368258476257, + -1.4076868295669556, + 1.0142264366149902, + -1.9853324890136719, + -0.5875377655029297 + ], + [ + 1.580847978591919, + 0.3325806260108948, + -1.0962201356887817, + -0.6875804662704468, + 0.42038458585739136, + 0.5779291391372681, + -0.9634775519371033, + 0.9584397077560425, + -0.020952241495251656, + -0.20275816321372986, + -0.16259431838989258, + 0.35765987634658813, + -2.7152206897735596, + -2.415917158126831, + 0.557228147983551, + 0.4680039584636688, + 0.9834054708480835, + -1.5287296772003174, + 0.9148144721984863, + -1.8712751865386963, + -0.8346226215362549 + ], + [ + 1.592742681503296, + 0.34411555528640747, + -0.8949155807495117, + -0.6642281413078308, + 0.12804077565670013, + 0.6749277114868164, + -1.140320062637329, + 0.9702752828598022, + 0.03766300529241562, + -0.13944312930107117, + -0.32787424325942993, + 0.3410719633102417, + -2.911372184753418, + -2.625809907913208, + 0.4719639718532562, + 0.4915294051170349, + 1.0420522689819336, + -1.4377121925354004, + 0.8342892527580261, + -1.9092910289764404, + -0.8492064476013184 + ], + [ + 1.3482401371002197, + 0.23607315123081207, + -1.1230732202529907, + -0.791126012802124, + -0.015790238976478577, + -0.6648694276809692, + -1.387589454650879, + 0.7008053064346313, + 0.05333154648542404, + 0.018860504031181335, + 0.07875882089138031, + 0.06073237955570221, + -2.0448343753814697, + -1.5583157539367676, + 0.6191694140434265, + 0.2761877775192261, + 0.8809552788734436, + -1.0871162414550781, + 0.9719794392585754, + -1.7854135036468506, + -0.671933650970459 + ], + [ + 1.5167537927627563, + 0.3650404214859009, + -1.3342314958572388, + -0.5989227294921875, + 0.4047403037548065, + 0.6936545372009277, + -0.270136296749115, + 1.1136995553970337, + 0.04881930351257324, + -0.187240868806839, + -0.02180282026529312, + 0.333523690700531, + -2.9222593307495117, + -2.319345712661743, + 0.3447681665420532, + 0.42478007078170776, + 1.274926781654358, + -1.4496917724609375, + 0.9142850637435913, + -1.9181504249572754, + -0.8424738645553589 + ], + [ + 1.5288500785827637, + 0.28309112787246704, + -0.9288069009780884, + -0.3774929940700531, + -0.06359642744064331, + -0.5223523378372192, + -0.5713916420936584, + 0.33994153141975403, + 0.1605323851108551, + -0.13476330041885376, + -0.49160775542259216, + -0.204166442155838, + -3.1072325706481934, + -2.0889811515808105, + 0.5006641149520874, + 0.34123772382736206, + 0.6568778157234192, + -1.4446017742156982, + 1.1540147066116333, + -2.175333261489868, + -1.2963474988937378 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-2dc5138251afd26a040d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1373138427734375, + 0.1398601531982422, + 0.11154793202877045, + 0.07755142450332642, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7500379085540771, + -0.32870030403137207, + -0.3281364440917969, + 0.020404458045959473, + 0.03884005546569824, + 0.26651138067245483, + 0.0, + 0.2614295184612274, + 0.02006034180521965, + 0.6256309151649475, + 0.10119366645812988, + 0.0365104079246521, + -1.9807136058807373, + 0.0, + -0.4040032625198364, + -0.3390212059020996, + 0.444367915391922, + -1.3744113445281982, + 0.3090496063232422, + -1.6372854709625244, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7500379085540771, + 0.1347455233335495, + -1.3281364440917969, + -0.3984496593475342, + -0.6640693545341492, + -0.7334886193275452, + -1.63351309299469, + 0.4696621894836426, + -0.03721961751580238, + 0.0896599143743515, + -0.8988063335418701, + -0.047963276505470276, + -2.9807136058807373, + -2.542311906814575, + 0.5875393152236938, + 0.41857796907424927, + 0.32660895586013794, + -2.3744113445281982, + 0.9575246572494507, + -2.6372854709625244, + -1.9586280584335327 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.26702618598938, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.08462381362915, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.9172561168670654 + ], + [ + 1.7928142547607422, + -0.046942055225372314, + -1.3733142614364624, + -0.4212188422679901, + -0.5566414594650269, + -0.9966021180152893, + -1.7926092147827148, + 0.37284746766090393, + 0.18181614577770233, + -0.056899696588516235, + -0.8793746829032898, + -0.06206277757883072, + -3.2539288997650146, + -2.3651227951049805, + 0.543649435043335, + 0.4969414472579956, + 0.3953157365322113, + -2.6153955459594727, + 1.239845633506775, + -2.9289329051971436, + -2.1138057708740234 + ], + [ + 1.4313925504684448, + 0.3287010192871094, + -1.4217545986175537, + -0.11546039581298828, + -0.38684383034706116, + -0.3866080045700073, + -1.4945378303527832, + 0.4180656969547272, + 0.08673545718193054, + -0.30675017833709717, + -0.8370212316513062, + -0.3124745488166809, + -3.2861690521240234, + -2.5144338607788086, + 0.7133253216743469, + 0.5272455215454102, + 0.6288780570030212, + -2.7024707794189453, + 1.0730262994766235, + -2.603350877761841, + -1.8104162216186523 + ], + [ + 2.0841901302337646, + 0.18348857760429382, + -1.4159170389175415, + -0.5868262052536011, + -0.3290694057941437, + -0.8189557790756226, + -1.811798334121704, + 0.8927561044692993, + -0.4510294198989868, + -0.1373635083436966, + -0.46978816390037537, + -0.2017078548669815, + -3.1228857040405273, + -2.619961977005005, + 0.7154320478439331, + 0.4623824954032898, + 0.8127838373184204, + -2.5683043003082275, + 0.8921399116516113, + -2.799823522567749, + -1.6220401525497437 + ], + [ + 1.5681514739990234, + -0.11813957989215851, + -1.066882848739624, + -0.4835556447505951, + -0.4833664894104004, + -0.1345365047454834, + -1.872452735900879, + 1.059093952178955, + -0.3537854254245758, + 0.5130321979522705, + -0.7769420146942139, + -0.4462882876396179, + -3.249842882156372, + -2.2144932746887207, + 0.6907999515533447, + 0.14545467495918274, + 0.8816275596618652, + -2.4357430934906006, + 1.424092411994934, + -2.656961441040039, + -2.3529839515686035 + ], + [ + 2.0935373306274414, + 0.20671290159225464, + -1.0638595819473267, + -0.37751641869544983, + -0.5913090705871582, + -0.3702518939971924, + -1.3611384630203247, + 0.8432458639144897, + 0.4395832121372223, + -0.19578908383846283, + -1.2028591632843018, + -0.07535962760448456, + -3.2909016609191895, + -2.9216225147247314, + 1.1581575870513916, + 0.9284459352493286, + -0.3347962498664856, + -1.9277539253234863, + 0.8971017003059387, + -3.245394229888916, + -2.1499102115631104 + ], + [ + 2.5108895301818848, + -0.2977973222732544, + -1.458537220954895, + -0.5705423355102539, + -1.4453105926513672, + -0.4854125380516052, + -2.383378028869629, + 0.39735814929008484, + -0.3523370325565338, + 0.4586251974105835, + -1.2388068437576294, + 0.1082226037979126, + -3.5858566761016846, + -1.8809643983840942, + -0.18688076734542847, + 0.0871247947216034, + 0.0062812864780426025, + -3.012958526611328, + 1.0266033411026, + -3.1649136543273926, + -2.6460556983947754 + ], + [ + 1.2948591709136963, + 0.21422013640403748, + -1.3904463052749634, + -0.13111025094985962, + -0.1661812961101532, + -0.4983506500720978, + -2.0006139278411865, + -0.4048727750778198, + 0.10130663216114044, + 0.1955089420080185, + -0.27839744091033936, + 0.8026529550552368, + -3.2792744636535645, + -1.6839708089828491, + 0.6468846201896667, + -0.2772523760795593, + 0.45975640416145325, + -2.612403631210327, + 1.6571701765060425, + -1.8971503973007202, + -1.2593979835510254 + ], + [ + 0.9154124855995178, + -0.2073056846857071, + -2.1218466758728027, + -0.4230979084968567, + -1.2478010654449463, + -0.5439814925193787, + -1.185287594795227, + -0.06685006618499756, + -0.7776275873184204, + -0.015365853905677795, + -1.3256038427352905, + 0.46163398027420044, + -2.8853814601898193, + -2.387786865234375, + 1.2300693988800049, + 0.9541484117507935, + 0.29864931106567383, + -2.931433916091919, + 1.3643410205841064, + -2.001633882522583, + -2.6548502445220947 + ], + [ + 2.035454750061035, + 0.03935282677412033, + -1.9080866575241089, + -0.17657285928726196, + -1.4843900203704834, + -0.4207392632961273, + -1.554489016532898, + 0.1909814178943634, + -0.5061874389648438, + -0.420978844165802, + -0.7634562849998474, + -0.7818551063537598, + -3.374903678894043, + -2.5324442386627197, + 1.2213574647903442, + 0.9676649570465088, + 0.5052346587181091, + -2.0974624156951904, + 0.9588823914527893, + -3.6367201805114746, + -1.953636646270752 + ], + [ + 1.4856661558151245, + 0.3259274363517761, + -2.1635096073150635, + -0.8793277740478516, + 0.2686290740966797, + -0.340938925743103, + -0.8516338467597961, + -0.3830271363258362, + 0.5376158356666565, + -0.2721732258796692, + -1.2276334762573242, + -0.38624608516693115, + -3.346294403076172, + -3.093522071838379, + 0.6735774278640747, + 0.5564011931419373, + -0.5450864434242249, + -1.5815953016281128, + 1.4068235158920288, + -2.7531042098999023, + -2.515698194503784 + ] + ], + "predicted_scores": [ + 3.835603713989258, + 3.6783268451690674, + 3.3198025226593018, + 2.9785282611846924, + 2.8413028717041016, + 2.7241344451904297, + 2.5705950260162354, + 2.5176258087158203 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565621826/train2", + "selected_index": 0, + "state_hash": "f6edd849b0b51788d7be25e402fb514cb777ccf220797faaf769000814df02cb", + "stored_base_utility": 0.1373138427734375, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08252689987421036, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08252689987421036 + }, + "base_progress": 0.08252689987421036, + "base_success": false, + "base_utility": 0.08252689987421036, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09265488386154175, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09265488386154175 + }, + { + "contact_quality": null, + "progress": 0.08510921150445938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08510921150445938 + }, + { + "contact_quality": null, + "progress": 0.09707490354776382, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09707490354776382 + }, + { + "contact_quality": null, + "progress": 0.08394956588745117, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08394956588745117 + }, + { + "contact_quality": null, + "progress": 0.0661265030503273, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0661265030503273 + }, + { + "contact_quality": null, + "progress": 0.0821652039885521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0821652039885521 + }, + { + "contact_quality": null, + "progress": 0.04405144974589348, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04405144974589348 + }, + { + "contact_quality": null, + "progress": 0.09674423933029175, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09674423933029175 + } + ], + "candidate_progress": [ + 0.09265488386154175, + 0.08510921150445938, + 0.09707490354776382, + 0.08394956588745117, + 0.0661265030503273, + 0.0821652039885521, + 0.04405144974589348, + 0.09674423933029175 + ], + "candidate_source_chart_ids": [ + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-206099bab6bfe7c84fce", + "ms-4aed2a49597da545e551", + "ms-45fbd749d5ed3fe37c13" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-387e6771a55be98f1a59", + "generated_tangents": [ + [ + 0.28549665212631226, + 0.05311489850282669, + -0.38248977065086365, + -0.018529057502746582, + 0.6787716746330261, + -1.2279969453811646, + -0.6420841813087463, + 0.26857051253318787, + 0.17115972936153412, + 0.2551596462726593, + 0.7053962349891663, + 0.04692859947681427, + -0.5527177453041077, + -0.18881969153881073, + -0.23941057920455933, + -0.0982455462217331, + 0.029637441039085388, + 0.07620451599359512, + 0.23607471585273743, + -0.3256145119667053, + -0.5287892818450928 + ], + [ + 0.36704954504966736, + 0.05758073925971985, + -0.3474738299846649, + -0.011212378740310669, + 0.6217342615127563, + -1.2112942934036255, + -0.6362608075141907, + 0.2833932638168335, + 0.15666010975837708, + 0.2438482642173767, + 0.6734858155250549, + 0.020701885223388672, + -0.590171754360199, + -0.22812683880329132, + -0.17236584424972534, + -0.11846224963665009, + 0.06505389511585236, + -0.08531375229358673, + 0.1952979862689972, + -0.2484874427318573, + -0.582443118095398 + ], + [ + 0.27893614768981934, + 0.09527479112148285, + -0.3688504993915558, + 0.03827756643295288, + 0.5562700033187866, + -1.3607341051101685, + -0.5660457015037537, + 0.283571720123291, + 0.17398253083229065, + 0.26349055767059326, + 0.6594426035881042, + 0.0042448490858078, + -0.6312653422355652, + -0.34405291080474854, + -0.21913892030715942, + -0.10617777705192566, + -0.0103473961353302, + 0.024394690990447998, + 0.08682183921337128, + -0.16082853078842163, + -0.3717314898967743 + ], + [ + 0.22916582226753235, + 0.07972680032253265, + 0.1921221911907196, + -0.23081974685192108, + 1.1584932804107666, + -0.9987024068832397, + -0.8276074528694153, + -0.07720938324928284, + 0.1346161812543869, + 0.09562423825263977, + 0.9233322143554688, + 0.06993602216243744, + -0.8096259236335754, + -0.38820624351501465, + -0.15911449491977692, + -0.0007902458310127258, + 0.5113657116889954, + -0.10122306644916534, + -0.15443506836891174, + 0.20267784595489502, + -0.26166316866874695 + ], + [ + 0.6438100337982178, + 0.1824950873851776, + 0.2209223508834839, + -0.2092418521642685, + 1.2713840007781982, + -1.1816667318344116, + -1.134199619293213, + 0.0350564569234848, + 0.1381925493478775, + 0.15791894495487213, + 0.9445805549621582, + 0.15191148221492767, + -1.0084794759750366, + -0.5793180465698242, + -0.4575043320655823, + 0.04685113579034805, + 0.6560973525047302, + -0.08546553552150726, + -0.3179672360420227, + 0.302046537399292, + -0.7222995758056641 + ], + [ + 0.2856890857219696, + 0.19474300742149353, + 0.2752723693847656, + -0.36156928539276123, + 1.2245597839355469, + -0.6177957057952881, + -1.2531189918518066, + 0.02007262408733368, + 0.1324319690465927, + 0.15256844460964203, + 1.0360097885131836, + 0.4375746250152588, + -0.4120018184185028, + -0.3471463918685913, + -0.1916099190711975, + 0.049049533903598785, + 0.3769719898700714, + -0.09059213101863861, + 0.020839035511016846, + 0.5283606648445129, + -0.2865541875362396 + ], + [ + 0.17219120264053345, + 0.09193360805511475, + 0.38803911209106445, + -0.22901012003421783, + 1.144775152206421, + -0.8856706619262695, + -0.7789334654808044, + -0.2274218499660492, + 0.1517324447631836, + 0.09137210249900818, + 1.0625745058059692, + 0.21263983845710754, + -0.8271968364715576, + -0.4312806725502014, + -0.0942372977733612, + -0.02930457517504692, + 0.5402745008468628, + -0.14919717609882355, + -0.3878049850463867, + 0.1922987699508667, + -0.4372692406177521 + ], + [ + 0.17109063267707825, + -0.07279525697231293, + 0.4865674674510956, + -0.41254833340644836, + 0.8830982446670532, + -1.1640024185180664, + -1.1009790897369385, + 0.42098212242126465, + 0.25346437096595764, + 0.01581348478794098, + 0.504300057888031, + 0.5966982841491699, + -1.5934369564056396, + -1.4469102621078491, + 0.01507195457816124, + 0.049132492393255234, + -0.02747724950313568, + -0.361056387424469, + 0.44712918996810913, + -0.7449579238891602, + -0.9258217811584473 + ] + ], + "generated_utilities": [ + 0.09265488386154175, + 0.08510921150445938, + 0.09707490354776382, + 0.08394956588745117, + 0.0661265030503273, + 0.0821652039885521, + 0.04405144974589348, + 0.09674423933029175 + ], + "group_id": "ms-387e6771a55be98f1a59", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.03822460025548935, + 0.18199902772903442, + 0.20744429528713226, + 0.05679210275411606, + 0.07646497339010239, + 0.038853906095027924, + 0.0824488028883934, + 0.05751838535070419, + 0.02475176379084587, + 0.040490686893463135, + 0.027167975902557373, + 0.06515108048915863, + 0.04604649543762207, + 0.014139334671199322, + 0.0258652176707983, + 0.034187138080596924 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0643576979637146, + 0.02293895184993744, + 0.03272289037704468, + 0.02799379825592041, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + 0.05611860752105713, + -0.010578691959381104, + 0.03396539017558098, + 0.754948616027832, + -0.12030506134033203, + 0.05740344524383545, + 0.0, + 0.01674991101026535, + 0.0017660856246948242, + 0.1110122799873352, + -0.29033517837524414, + -0.015073239803314209, + -0.062492311000823975, + 0.0 + ], + [ + 0.03256732225418091, + 0.029803231358528137, + -0.11524808406829834, + -0.03975573182106018, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + -0.11635321378707886, + -0.12916040420532227, + -0.10934318602085114, + 0.754948616027832, + -0.10127544403076172, + 0.10553282499313354, + 0.0, + 0.2124219685792923, + -0.13985675573349, + 0.20122885704040527, + -0.29033517837524414, + -0.09104716777801514, + -0.33053046464920044, + 0.0 + ] + ], + "predicted_scores": [ + 6.3780999183654785, + 6.203186511993408, + 5.9015960693359375, + 4.977954387664795, + 4.771164417266846, + 4.232055187225342, + 3.805530309677124, + 1.5832422971725464 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train2", + "selected_index": 0, + "state_hash": "6b753a027f9378ff308946331f6bb658d452d8e77220ab0cd6fc813be0ac75cc", + "stored_base_utility": 0.03822460025548935, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03917040303349495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03917040303349495 + }, + "base_progress": 0.03917040303349495, + "base_success": false, + "base_utility": 0.03917040303349495, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06579937785863876, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06579937785863876 + }, + { + "contact_quality": null, + "progress": 0.08081644773483276, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08081644773483276 + }, + { + "contact_quality": null, + "progress": 0.0637521967291832, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0637521967291832 + }, + { + "contact_quality": null, + "progress": 0.04907931759953499, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04907931759953499 + }, + { + "contact_quality": null, + "progress": 0.044412415474653244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044412415474653244 + }, + { + "contact_quality": null, + "progress": 0.060444433242082596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060444433242082596 + }, + { + "contact_quality": null, + "progress": 0.11467790603637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11467790603637695 + }, + { + "contact_quality": null, + "progress": 0.05245332047343254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05245332047343254 + } + ], + "candidate_progress": [ + 0.06579937785863876, + 0.08081644773483276, + 0.0637521967291832, + 0.04907931759953499, + 0.044412415474653244, + 0.060444433242082596, + 0.11467790603637695, + 0.05245332047343254 + ], + "candidate_source_chart_ids": [ + "ms-397e502908d16b7804b0", + "ms-98eb81debb5186ebbe38", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-be219977a9db8dbb1e58", + "ms-7dedf60ad41ea2d92ee5", + "ms-397e502908d16b7804b0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-38f7298184a2c5005e94", + "generated_tangents": [ + [ + 0.06392338871955872, + 0.09199774265289307, + 0.39832690358161926, + -0.08481396734714508, + 1.1780062913894653, + -2.1410186290740967, + -0.026175349950790405, + 0.3576704263687134, + 0.007041775155812502, + 0.39361920952796936, + -0.20811009407043457, + 0.917994499206543, + -1.023305892944336, + 0.15454500913619995, + 0.17341533303260803, + 0.46468600630760193, + -0.13702644407749176, + -0.036796897649765015, + 0.2083837389945984, + -1.7337701320648193, + -1.0185716152191162 + ], + [ + 0.3518551290035248, + 0.07209115475416183, + 0.1243547797203064, + 0.07106101512908936, + 0.917472243309021, + -1.8660422563552856, + -0.34445130825042725, + 0.344399631023407, + 0.08186738193035126, + 0.19340568780899048, + -0.36229076981544495, + 0.7980875968933105, + -1.5588971376419067, + 0.033204786479473114, + 0.22450880706310272, + 0.3089137077331543, + 0.04057250916957855, + -0.17583157122135162, + 0.3140011727809906, + -1.8366100788116455, + -0.6990377902984619 + ], + [ + -0.011062920093536377, + 0.1393205225467682, + -0.058338314294815063, + -0.4927695095539093, + 1.4945178031921387, + -1.5698870420455933, + -0.9513618350028992, + 0.5369454622268677, + -0.04390205442905426, + 0.23241496086120605, + -0.3265971541404724, + 1.3889678716659546, + -1.312913179397583, + -0.11061471700668335, + 0.41059115529060364, + 0.45313361287117004, + -0.0007749646902084351, + -0.01357463002204895, + 0.10118094086647034, + -2.1971399784088135, + -1.5030590295791626 + ], + [ + 0.26104041934013367, + 0.16586053371429443, + 0.3561619818210602, + -0.24365101754665375, + 1.4226458072662354, + -1.5604000091552734, + -0.5354271531105042, + 0.3232274055480957, + -0.018214862793684006, + 0.24432046711444855, + -0.4937252700328827, + 1.5809727907180786, + -1.190437912940979, + -0.013494908809661865, + 0.19259680807590485, + 0.3032260537147522, + 0.012787118554115295, + -0.08537150919437408, + 0.2886306345462799, + -2.011969804763794, + -1.5524606704711914 + ], + [ + 0.24262487888336182, + 0.09299619495868683, + 0.44020742177963257, + -0.3339966833591461, + 1.3947012424468994, + -1.4245266914367676, + -0.5730263590812683, + 0.34654945135116577, + -0.015343474224209785, + 0.21485567092895508, + -0.5876930356025696, + 1.55846107006073, + -1.2206151485443115, + -0.006257236003875732, + 0.11838798224925995, + 0.28561991453170776, + -0.11307172477245331, + -0.14989586174488068, + 0.4280414581298828, + -1.9923348426818848, + -1.4128297567367554 + ], + [ + 0.07177262753248215, + 0.11817136406898499, + 0.19908684492111206, + -0.2735249698162079, + 0.8937707543373108, + -1.2653834819793701, + -0.20989418029785156, + 0.3359907269477844, + 0.03906693682074547, + 0.3878709375858307, + 0.18974530696868896, + 0.7361236810684204, + -1.6000571250915527, + -0.15636850893497467, + -0.060112934559583664, + 0.2737217843532562, + 0.5054886937141418, + -0.06787629425525665, + 0.2340533435344696, + -1.4242932796478271, + -0.5128096342086792 + ], + [ + -0.20007582008838654, + 0.2851930260658264, + -0.14037975668907166, + 0.19749146699905396, + 1.5596563816070557, + -1.0835773944854736, + -0.9457593560218811, + 0.7596969604492188, + 0.16349643468856812, + 0.32024094462394714, + -0.2900964915752411, + 0.9833406209945679, + -1.6681772470474243, + 0.20751577615737915, + 0.13381241261959076, + 0.47939008474349976, + 0.10001221299171448, + -0.04324066638946533, + 0.14821138978004456, + -2.220677375793457, + -1.3216150999069214 + ], + [ + 0.0864458978176117, + 0.14882588386535645, + 0.27863213419914246, + -0.4068523943424225, + 0.6527861952781677, + -1.1175181865692139, + -0.11312580108642578, + 0.2875620126724243, + -0.031250230967998505, + 0.2120424211025238, + -0.65817791223526, + 1.217991828918457, + -1.2307050228118896, + 0.47097188234329224, + 0.07097706198692322, + 0.3608599007129669, + -0.020377397537231445, + -0.4141119718551636, + 0.3017750382423401, + -1.4808273315429688, + -1.7473108768463135 + ] + ], + "generated_utilities": [ + 0.06579937785863876, + 0.08081644773483276, + 0.0637521967291832, + 0.04907931759953499, + 0.044412415474653244, + 0.060444433242082596, + 0.11467790603637695, + 0.05245332047343254 + ], + "group_id": "ms-38f7298184a2c5005e94", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1194220781326294, + 0.1767096221446991, + 0.0505807027220726, + 0.026383202522993088, + 0.09035396575927734, + 0.11899666488170624, + 0.059947751462459564, + 0.08427222818136215, + 0.041255753487348557, + 0.05034176632761955, + 0.02779809758067131, + 0.011037330143153667, + 0.020422041416168213, + 0.049474477767944336, + 0.03517866134643555, + 0.02796018123626709 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.08923250436782837, + 0.05675226449966431, + -0.1250607967376709, + -0.07566308975219727, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.001595139503479004, + 0.026048749685287476, + 0.1681501865386963, + -0.11568224430084229, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.18507611751556396, + -0.15127414464950562, + 0.07637521624565125, + -0.15284772217273712, + 0.0320390909910202, + -1.8774652481079102, + 0.0 + ], + [ + -0.2749255299568176, + 0.2578675150871277, + -0.46759605407714844, + -0.5041444897651672, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.13614141941070557, + 0.11120480298995972, + 0.15478569269180298, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.01827770471572876, + -0.38639169931411743, + -0.18168723583221436, + 0.3532869219779968, + -0.5437159538269043, + -1.8774652481079102, + 0.0 + ], + [ + 0.4466920495033264, + 0.07532927393913269, + 0.8145812749862671, + -0.963495135307312, + 1.3846542835235596, + -3.0133423805236816, + -1.0476078987121582, + 0.6759306788444519, + 0.5485053658485413, + 0.634345293045044, + -0.7682492733001709, + 2.2281322479248047, + -2.7100830078125, + -0.8660445213317871, + 0.7648183703422546, + 0.607989490032196, + -0.6848516464233398, + -0.06598936021327972, + 0.2030458152294159, + -2.9672060012817383, + -1.4293488264083862 + ], + [ + 1.0058056116104126, + 0.3426892161369324, + 0.6375484466552734, + -0.5828341841697693, + 1.7963712215423584, + -3.3760194778442383, + -0.9086123704910278, + 0.20509907603263855, + 0.10472571849822998, + 1.1158980131149292, + -1.0710231065750122, + 2.203950881958008, + -2.4466300010681152, + -0.3184358477592468, + 0.43084120750427246, + 0.32133060693740845, + -0.6005205512046814, + 0.07880236953496933, + 0.52874755859375, + -2.7805917263031006, + -1.6972103118896484 + ], + [ + 0.48786461353302, + -0.007949680089950562, + 0.41629794239997864, + -0.2277727723121643, + 1.0953291654586792, + -2.7825636863708496, + -1.7178279161453247, + 0.8412191271781921, + 0.6389099359512329, + 0.7100303769111633, + -1.1701807975769043, + 2.8546230792999268, + -2.570579767227173, + -0.2383948564529419, + 0.4282584488391876, + 0.27555936574935913, + -0.4073549509048462, + 0.49247825145721436, + 0.6483681797981262, + -3.3716540336608887, + -1.891394853591919 + ], + [ + 0.17139381170272827, + 0.831203818321228, + 0.1966383457183838, + -1.5011540651321411, + 1.811257004737854, + -2.6416001319885254, + -1.8791863918304443, + 0.03218984603881836, + 1.0182924270629883, + 1.5643268823623657, + -1.2954871654510498, + 2.249680757522583, + -1.9383416175842285, + -1.4303685426712036, + 0.48937559127807617, + 0.2848057746887207, + -0.6113857626914978, + 0.6453920006752014, + -0.13643476366996765, + -2.514732599258423, + -1.8046306371688843 + ], + [ + 0.1380026936531067, + 0.5165722370147705, + 0.9639596939086914, + -0.3682268261909485, + 2.2811193466186523, + -3.5902557373046875, + -1.047336220741272, + 1.3064022064208984, + 0.7264143228530884, + 1.3356540203094482, + -0.41242116689682007, + 1.8184869289398193, + -3.016441822052002, + -0.8501555919647217, + 1.0147737264633179, + 1.0417438745498657, + -0.24422766268253326, + 0.3441629707813263, + 1.075059175491333, + -3.178764820098877, + -1.7388081550598145 + ], + [ + 1.0805304050445557, + -0.5906903743743896, + -0.32329434156417847, + -1.7024246454238892, + 1.3012806177139282, + -3.3655173778533936, + -2.0100932121276855, + 0.28595322370529175, + -0.42309290170669556, + 0.9014679193496704, + -0.11985284090042114, + 3.033172607421875, + -1.890408992767334, + -0.049424707889556885, + 0.8836197853088379, + 0.07151538133621216, + -0.6898007392883301, + 0.6614744663238525, + 0.7956362962722778, + -3.344180107116699, + -1.4918344020843506 + ], + [ + 1.4192681312561035, + -0.5825785994529724, + 0.11329725384712219, + -0.08465814590454102, + 1.0504729747772217, + -2.438842535018921, + -0.9858216047286987, + -0.35852980613708496, + 1.4301284551620483, + 0.37676453590393066, + -1.3725404739379883, + 3.114079475402832, + -2.0786290168762207, + -0.5835205912590027, + 0.1985606551170349, + 0.260464072227478, + 0.41686397790908813, + -0.104665607213974, + 0.6382055282592773, + -2.7474825382232666, + -1.663126826286316 + ], + [ + 1.0832164287567139, + 0.11555392295122147, + 1.2108526229858398, + -1.5446858406066895, + 0.9112894535064697, + -3.3027408123016357, + -1.2419135570526123, + -0.057880282402038574, + -0.07349961996078491, + 0.026286423206329346, + -0.897603452205658, + 3.2857306003570557, + -2.232046365737915, + -1.2509676218032837, + 1.1900405883789062, + 0.8446143269538879, + -1.339659333229065, + -0.07882180064916611, + 0.1166829764842987, + -2.662961721420288, + -1.0079293251037598 + ], + [ + 0.43863287568092346, + -0.2024492472410202, + 0.0567094087600708, + -0.3029903173446655, + 1.612892746925354, + -3.26739764213562, + -1.7898839712142944, + -0.1649491786956787, + 0.2927369475364685, + 1.020188808441162, + -0.10171866416931152, + 2.3230679035186768, + -2.615206003189087, + -0.08625847101211548, + 0.9057100415229797, + 1.4201092720031738, + -0.3173455595970154, + 0.09508261829614639, + 0.9565507173538208, + -2.2954390048980713, + -1.2229193449020386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.03360384702682495, + -0.028338372707366943, + -0.12341004610061646, + -0.04655510187149048, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + 0.032517433166503906, + -0.02922782301902771, + -0.033792078495025635, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + -0.06465893983840942, + -0.026740312576293945, + 0.046701133251190186, + -0.06953387707471848, + 0.02321198582649231, + -1.8774652481079102, + 0.0 + ] + ], + "predicted_scores": [ + 3.8450777530670166, + 3.747302293777466, + 3.028522491455078, + 2.623569965362549, + 2.476485252380371, + 2.088772773742676, + 1.6912859678268433, + 1.608293056488037 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train2", + "selected_index": 0, + "state_hash": "dee046e5a3c9b5e78be27f1df1f4f235344ec818a1a7439afe5f4cb30139cf34", + "stored_base_utility": 0.1194220781326294, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.009124159812927246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009124159812927246 + }, + "base_progress": 0.009124159812927246, + "base_success": false, + "base_utility": 0.009124159812927246, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.021712401881814003, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021712401881814003 + }, + { + "contact_quality": null, + "progress": 0.01253493595868349, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01253493595868349 + }, + { + "contact_quality": null, + "progress": 0.020592987537384033, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020592987537384033 + }, + { + "contact_quality": null, + "progress": 0.024399837478995323, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024399837478995323 + }, + { + "contact_quality": null, + "progress": 0.015089293010532856, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015089293010532856 + }, + { + "contact_quality": null, + "progress": 0.01447206735610962, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01447206735610962 + }, + { + "contact_quality": null, + "progress": 0.030993500724434853, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030993500724434853 + }, + { + "contact_quality": null, + "progress": 0.023590564727783203, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023590564727783203 + } + ], + "candidate_progress": [ + 0.021712401881814003, + 0.01253493595868349, + 0.020592987537384033, + 0.024399837478995323, + 0.015089293010532856, + 0.01447206735610962, + 0.030993500724434853, + 0.023590564727783203 + ], + "candidate_source_chart_ids": [ + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-6758deb06644a14aa722", + "ms-397e502908d16b7804b0", + "ms-35a50d2822c2c3cc4516" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-482c5c8d208b5907a2c2", + "generated_tangents": [ + [ + -0.10324154794216156, + 0.11459678411483765, + 0.6651431918144226, + 0.08507943153381348, + 1.1882154941558838, + -0.012950196862220764, + 0.2219807356595993, + 0.3474201560020447, + 0.18798601627349854, + 0.6566392779350281, + -0.31874650716781616, + 1.4696160554885864, + 0.3617602288722992, + -0.10195180773735046, + -0.05230840668082237, + -0.021728409454226494, + 0.1829734444618225, + 0.44701093435287476, + 0.18698903918266296, + 0.3523678779602051, + -0.7169893980026245 + ], + [ + 0.09427937865257263, + 0.24168075621128082, + 1.1515212059020996, + -0.15542587637901306, + 1.1397836208343506, + 1.445318579673767, + -0.3245232105255127, + 0.2744519114494324, + 0.28566601872444153, + 1.0274202823638916, + -0.4358368515968323, + 1.6342661380767822, + 1.3807165622711182, + 1.3334944248199463, + -0.37339451909065247, + 0.19507186114788055, + 0.357954204082489, + 0.7042507529258728, + 0.24645519256591797, + -0.6367019414901733, + -0.6903630495071411 + ], + [ + 0.20592395961284637, + 0.020406574010849, + 0.8932121992111206, + 0.12083536386489868, + 1.3073530197143555, + 0.9037048816680908, + -0.3148799538612366, + 0.3076699376106262, + 0.2580004334449768, + 1.004162073135376, + -0.3212444484233856, + 1.5189355611801147, + 1.0994170904159546, + 0.5334300994873047, + 0.08030927181243896, + 0.16071689128875732, + 0.10693933069705963, + 0.5786799192428589, + -0.053774625062942505, + 0.05243697762489319, + -0.7388463020324707 + ], + [ + -0.1731804460287094, + 0.17591097950935364, + 0.5804473757743835, + 0.0004608035087585449, + 1.1744091510772705, + 0.0935523509979248, + 0.020604103803634644, + 0.34948986768722534, + 0.1699536144733429, + 0.6689888834953308, + -0.44994327425956726, + 1.1837468147277832, + 0.39836356043815613, + -0.1453128606081009, + -0.06979362666606903, + 0.01856297440826893, + 0.17380034923553467, + 0.3868778944015503, + 0.2002110779285431, + 0.3215237855911255, + -0.5445191860198975 + ], + [ + -0.11170406639575958, + 0.024100452661514282, + 0.914171040058136, + 0.016485899686813354, + 1.3674840927124023, + 1.1694116592407227, + -0.2093716859817505, + 0.2524982988834381, + 0.25639545917510986, + 0.9274079203605652, + -0.5004943609237671, + 1.3227384090423584, + 1.0618607997894287, + 0.43705642223358154, + 0.1531822681427002, + 0.11158479750156403, + 0.2071995735168457, + 0.5246315002441406, + -0.16538608074188232, + 0.07106602191925049, + -0.6931731700897217 + ], + [ + 0.05488795042037964, + 0.005637887865304947, + 0.9165982007980347, + 0.2725606858730316, + 1.513552188873291, + -0.292318195104599, + -0.3103640079498291, + 0.22820740938186646, + 0.1727409064769745, + 0.66246497631073, + -0.25479790568351746, + 1.590884804725647, + 0.3123844563961029, + 0.11982061713933945, + 0.12594442069530487, + 0.15583494305610657, + 0.14659681916236877, + 0.321718692779541, + 0.1898215413093567, + 0.32241296768188477, + -0.4913976490497589 + ], + [ + -0.22871951758861542, + -0.021601146087050438, + 0.4148894250392914, + 0.21790257096290588, + 0.9094608426094055, + -0.362530380487442, + -0.25824183225631714, + 0.525179922580719, + 0.21581538021564484, + 0.6320717930793762, + -0.6137592196464539, + 1.2848787307739258, + 0.17660120129585266, + 0.06205551326274872, + 0.05705440044403076, + 0.2466651350259781, + 0.06993600726127625, + 0.5956662893295288, + 0.3316118121147156, + 0.24304735660552979, + -0.030683904886245728 + ], + [ + 0.20705199241638184, + 0.009793613106012344, + 0.8095183968544006, + 0.12078338861465454, + 1.2094385623931885, + 0.6669921875, + -0.5426391959190369, + 0.49290749430656433, + 0.19917532801628113, + 1.019526481628418, + -0.38820043206214905, + 1.4178555011749268, + 1.0817474126815796, + 0.33618342876434326, + 0.022878531366586685, + 0.2310313880443573, + 0.10935914516448975, + 0.7442690134048462, + -0.14142760634422302, + 0.16419675946235657, + -0.6085910797119141 + ] + ], + "generated_utilities": [ + 0.021712401881814003, + 0.01253493595868349, + 0.020592987537384033, + 0.024399837478995323, + 0.015089293010532856, + 0.01447206735610962, + 0.030993500724434853, + 0.023590564727783203 + ], + "group_id": "ms-482c5c8d208b5907a2c2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0359329991042614, + 0.1834145486354828, + 0.04271410033106804, + 0.03202331066131592, + 0.12089761346578598, + 0.03575414419174194, + 0.07773216813802719, + 0.1062050461769104, + 0.10849495977163315, + 0.08369352668523788, + 0.14167599380016327, + 0.008830646984279156, + 0.014355997554957867, + 0.02518286369740963, + 0.07873731851577759, + 0.015250584110617638 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.007817894220352173, + 0.01831631362438202, + 0.14601469039916992, + 0.10826835036277771, + 1.6591682434082031, + -0.005238652229309082, + 0.0, + -0.02753499150276184, + -0.009624123573303223, + 0.5762790441513062, + -0.3171614408493042, + 1.4374253749847412, + 0.7921857833862305, + 0.0, + -0.037753358483314514, + -0.025671005249023438, + 0.027724146842956543, + 0.4680846929550171, + -0.08415359258651733, + 0.033902376890182495, + 0.0 + ], + [ + 0.4543180465698242, + 0.20962828397750854, + 1.14601469039917, + -0.33832865953445435, + 2.659168243408203, + -0.7141264081001282, + 0.3707960247993469, + 0.21857939660549164, + 0.6796972155570984, + 1.5762790441513062, + -1.3171614408493042, + 2.437425374984741, + 1.7921857833862305, + 1.7551628351211548, + -0.19566796720027924, + 0.6842412948608398, + 0.11985036730766296, + 1.468084692955017, + 0.29799214005470276, + 0.3715361952781677, + 0.2624160647392273 + ], + [ + 0.2115812450647354, + -0.09577503800392151, + 0.7509078979492188, + -0.6239187121391296, + 2.4163239002227783, + -0.5409671664237976, + 0.16154196858406067, + 0.5615253448486328, + 1.0765573978424072, + 1.4198161363601685, + -1.430187463760376, + 2.5212466716766357, + 1.9029841423034668, + 1.5550287961959839, + -0.5099657773971558, + 1.0103299617767334, + -0.21028786897659302, + 1.4441221952438354, + 0.5605546236038208, + 0.6799722909927368, + 0.5554409027099609 + ], + [ + 0.46792101860046387, + 0.08014003932476044, + 1.5437835454940796, + 0.08789932727813721, + 2.3046298027038574, + -1.1965231895446777, + 0.8527317047119141, + 0.22510167956352234, + 1.0473172664642334, + 1.3720369338989258, + -1.2376489639282227, + 2.066061019897461, + 2.033992052078247, + 2.166865825653076, + -0.46199846267700195, + 1.1675353050231934, + -0.22657811641693115, + 1.7576167583465576, + 0.766140878200531, + 0.1856600046157837, + 0.008163660764694214 + ], + [ + 0.9431964755058289, + 0.6587716341018677, + 0.8381389379501343, + -0.8407833576202393, + 2.3666741847991943, + -1.05813729763031, + 0.1429125964641571, + 0.10640819370746613, + 1.0277029275894165, + 1.193630576133728, + -0.7695048451423645, + 2.9484994411468506, + 1.1866973638534546, + 1.5341145992279053, + -0.1225738674402237, + 0.9121334552764893, + 0.7116963863372803, + 0.8198207020759583, + 0.6562460660934448, + 0.1118304431438446, + -0.2320157289505005 + ] + ], + "predicted_scores": [ + -0.16841058433055878, + -0.28825217485427856, + -0.3040606379508972, + -0.41618430614471436, + -0.43267136812210083, + -0.6537924408912659, + -0.8287419676780701, + -0.8810359835624695 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train2", + "selected_index": 0, + "state_hash": "903f78d5baa541d73612cac5e063bb553fba566c0d4adcb9603aa055d710fb15", + "stored_base_utility": 0.0359329991042614, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14663122594356537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14663122594356537 + }, + "base_progress": 0.14663122594356537, + "base_success": false, + "base_utility": 0.14663122594356537, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06835349649190903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06835349649190903 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.06835349649190903, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4da03300d92420e80ed0", + "generated_tangents": [ + [ + 1.5912787914276123, + -0.036956414580345154, + -0.21241113543510437, + 0.6516801118850708, + -0.19399788975715637, + 1.5948033332824707, + 1.271910548210144, + 1.4476734399795532, + 0.1724436730146408, + -0.9798520803451538, + -0.42552649974823, + -0.6116935014724731, + -2.8600406646728516, + -1.6242611408233643, + 1.367811918258667, + 0.11179615557193756, + -0.8350984454154968, + 0.33912092447280884, + -0.6248295307159424, + -1.3573098182678223, + -0.7161210775375366 + ], + [ + 1.7093102931976318, + -0.196620911359787, + 0.02189430594444275, + 0.641255259513855, + -0.4094357490539551, + 2.530367612838745, + 1.4680109024047852, + 2.0426712036132812, + 0.1967797428369522, + -1.2756084203720093, + -0.5933140516281128, + -0.6144430637359619, + -3.5528340339660645, + -2.1312100887298584, + 1.4805469512939453, + 0.2696048319339752, + -0.6461998820304871, + 0.41171813011169434, + -0.28442347049713135, + -0.7976076006889343, + -0.9309910535812378 + ], + [ + 1.4498106241226196, + -0.29270243644714355, + 0.04120686650276184, + -0.13904261589050293, + -0.21596965193748474, + 0.7187117338180542, + 2.4622488021850586, + 1.5556423664093018, + 0.1405021697282791, + -1.037904143333435, + -0.42656487226486206, + -0.9309213161468506, + -2.8924334049224854, + -1.6524156332015991, + 1.009226679801941, + 0.010251995176076889, + -0.44311296939849854, + 0.07730083912611008, + -0.2179584503173828, + -1.8193836212158203, + -0.6662688255310059 + ], + [ + 1.5806620121002197, + -0.14825692772865295, + -0.2096545398235321, + 0.583511233329773, + -0.5389400720596313, + 2.4316601753234863, + 1.4776484966278076, + 1.8064268827438354, + 0.19802790880203247, + -1.2530491352081299, + -0.5538384318351746, + -0.795089602470398, + -3.2339212894439697, + -2.5375781059265137, + 1.4592887163162231, + 0.23937417566776276, + -0.48405611515045166, + 0.3261455297470093, + -0.24709457159042358, + -0.6881916522979736, + -1.1293952465057373 + ], + [ + 1.5368729829788208, + -0.09165816009044647, + -0.11731317639350891, + 0.554122805595398, + -0.27068716287612915, + 2.2091636657714844, + 1.6179258823394775, + 1.8708558082580566, + 0.1728523224592209, + -1.3351114988327026, + -0.556918740272522, + -0.8974283933639526, + -3.302716016769409, + -2.4506871700286865, + 1.5876867771148682, + 0.22924140095710754, + -0.5252518653869629, + 0.2545866072177887, + -0.24682819843292236, + -0.8035078644752502, + -1.14055597782135 + ], + [ + 1.5098063945770264, + -0.22725751996040344, + -0.26270297169685364, + 0.5413416624069214, + -0.4311066269874573, + 2.0036556720733643, + 1.6241755485534668, + 1.839166522026062, + 0.2298468053340912, + -1.2273516654968262, + -0.7428346276283264, + -0.8687328100204468, + -3.3517966270446777, + -2.3812711238861084, + 1.5311939716339111, + 0.17737452685832977, + -0.405849814414978, + 0.26439476013183594, + -0.2840878367424011, + -0.978763997554779, + -1.39325749874115 + ], + [ + 1.6119011640548706, + -0.15155303478240967, + -0.4006286561489105, + 0.7292817831039429, + -0.4522017240524292, + 2.1315112113952637, + 1.7339651584625244, + 1.7694288492202759, + 0.27131545543670654, + -1.235857367515564, + -0.46351078152656555, + -0.911089301109314, + -3.1760571002960205, + -2.4977662563323975, + 1.5386873483657837, + 0.20270687341690063, + -0.3883051872253418, + 0.14286354184150696, + -0.23281234502792358, + -0.8912167549133301, + -1.1589937210083008 + ], + [ + 1.4282817840576172, + -0.2259243130683899, + -0.2913416922092438, + 0.6201746463775635, + -0.37133270502090454, + 1.4765207767486572, + 1.910367727279663, + 1.6940661668777466, + 0.316834956407547, + -1.439162254333496, + -0.5653513073921204, + -1.072232723236084, + -2.8804492950439453, + -2.411958932876587, + 1.7964717149734497, + 0.17439192533493042, + -0.6093809604644775, + 0.0989665761590004, + -0.2696952223777771, + -1.2158225774765015, + -1.27056884765625 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.06835349649190903, + 2.0 + ], + "group_id": "ms-4da03300d92420e80ed0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18184101581573486, + 0.1803790181875229, + 0.09926676750183105, + 0.20791122317314148, + 0.26338279247283936, + 0.2025860995054245, + 0.2575649917125702, + 0.23894403874874115, + 0.2174234390258789, + 2.0, + 0.1918075978755951, + 0.10365168750286102, + 2.0, + 2.0, + 2.0, + 0.15933696925640106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.1168947219848633, + 0.13667210936546326, + -0.051466986536979675, + 0.03360617160797119, + -0.19173425436019897, + 1.7646396160125732, + 0.0, + 0.9886866807937622, + 0.19114632904529572, + -0.3927978277206421, + -0.21108940243721008, + -0.031822919845581055, + -1.6683034896850586, + 0.0, + 0.9908627271652222, + -0.05301591753959656, + 0.08875280618667603, + 0.04224884510040283, + -0.03718748688697815, + -0.06952071189880371, + 0.0 + ], + [ + 2.1713812351226807, + -0.41443270444869995, + -0.23275701701641083, + 0.6524178981781006, + -0.25828492641448975, + 3.331055164337158, + 2.47529673576355, + 1.4984867572784424, + -0.396874338388443, + -0.725897490978241, + 0.1870076060295105, + -0.4014764428138733, + -2.239036798477173, + -2.218061923980713, + 1.6540865898132324, + 0.013689309358596802, + -0.22782719135284424, + 1.2475574016571045, + 0.7231295704841614, + -0.281164288520813, + -1.4593302011489868 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 2.1168947219848633, + 0.045049574226140976, + -0.25911271572113037, + 0.6112266778945923, + -0.5530642867088318, + 2.7646396160125732, + 1.9622430801391602, + 1.9886866807937622, + 0.28067120909690857, + -1.392797827720642, + -0.5793421268463135, + -1.031822919845581, + -2.6683034896850586, + -2.812993049621582, + 1.9908627271652222, + -0.28380459547042847, + -0.7305920720100403, + 0.7978808879852295, + 0.1574108749628067, + -1.0695207118988037, + -0.9408642053604126 + ], + [ + 2.228377103805542, + -0.19656506180763245, + -0.23844876885414124, + 0.6676552295684814, + -0.7639804482460022, + 2.937652111053467, + 1.882341980934143, + 1.8207013607025146, + 0.27702635526657104, + -1.549763798713684, + -0.5955427289009094, + -1.185685157775879, + -2.8421716690063477, + -3.054234266281128, + 1.691805362701416, + -0.37051287293434143, + -0.43653741478919983, + 0.514695405960083, + -0.0132378488779068, + -0.899186372756958, + -0.8349848985671997 + ], + [ + 2.425011157989502, + -0.03216857090592384, + -0.010275468230247498, + 0.4265761971473694, + -0.7737585306167603, + 3.1123812198638916, + 1.5686779022216797, + 2.112421989440918, + 0.3475198745727539, + -1.3452436923980713, + -0.6067132949829102, + -1.1150503158569336, + -2.461210250854492, + -2.8643558025360107, + 2.185579299926758, + -0.6223857402801514, + -0.8183724880218506, + 0.8074819445610046, + 0.536148726940155, + -0.7826522588729858, + -0.6682058572769165 + ], + [ + 1.6831411123275757, + -0.19312052428722382, + 0.010244876146316528, + 1.1032377481460571, + -0.4994945824146271, + 2.9000508785247803, + 2.187715768814087, + 2.200220823287964, + 0.4744641184806824, + -1.6715843677520752, + -0.493664413690567, + -1.459894061088562, + -2.90458345413208, + -2.3254852294921875, + 1.4726771116256714, + -0.21338894963264465, + -1.293912649154663, + 1.3146823644638062, + -0.09375743567943573, + -0.5040683150291443, + -0.9596792459487915 + ], + [ + 1.6738543510437012, + 0.6350066661834717, + 0.19669359922409058, + -0.1396545171737671, + -0.9625160694122314, + 2.207883596420288, + 1.7678128480911255, + 1.319273829460144, + -0.267609566450119, + -0.5747214555740356, + 0.19376236200332642, + -1.4490230083465576, + -2.6397883892059326, + -3.2567262649536133, + 1.2646806240081787, + -0.14523190259933472, + -0.050948500633239746, + 0.6943628787994385, + -0.1503342241048813, + -0.7313232421875, + -1.7930638790130615 + ], + [ + 2.139432668685913, + 0.34125545620918274, + 0.6370852589607239, + 0.8591548204421997, + 0.14890235662460327, + 2.6835381984710693, + 2.324920892715454, + 2.0091233253479004, + 0.13480046391487122, + -0.47407835721969604, + -1.102479100227356, + -1.0710523128509521, + -2.3226847648620605, + -3.6534740924835205, + 1.8767409324645996, + -0.01776304841041565, + -1.167153000831604, + 0.693484902381897, + 0.563054084777832, + -0.17544519901275635, + -0.8315125703811646 + ], + [ + 1.1254388093948364, + -0.03027276322245598, + -0.8753920793533325, + 0.3578681945800781, + -0.4256642460823059, + 1.7686536312103271, + 2.253190517425537, + 2.8274970054626465, + 0.3938461244106293, + -1.4084759950637817, + -0.7252857685089111, + -0.07288885116577148, + -1.7749395370483398, + -2.1030378341674805, + 1.92497980594635, + 0.20398634672164917, + -0.03521168231964111, + 1.4702520370483398, + -0.36331063508987427, + -1.5291211605072021, + -0.8369957804679871 + ] + ], + "predicted_scores": [ + 5.25183629989624, + 4.327092170715332, + 4.227852821350098, + 3.780285358428955, + 3.7225940227508545, + 3.6847755908966064, + 3.204503297805786, + 2.865238904953003 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1747178030/train2", + "selected_index": 0, + "state_hash": "87c74ddf97c53ddeeaae98467b4a13b1e0e499bd94eddb617759d39164cab542", + "stored_base_utility": 0.18184101581573486, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.009399374015629292, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009399374015629292 + }, + { + "contact_quality": null, + "progress": 0.006204982753843069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006204982753843069 + }, + { + "contact_quality": null, + "progress": 0.007237812038511038, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007237812038511038 + }, + { + "contact_quality": null, + "progress": 0.008653719909489155, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008653719909489155 + }, + { + "contact_quality": null, + "progress": 0.010276774875819683, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010276774875819683 + }, + { + "contact_quality": null, + "progress": 0.006190141197293997, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006190141197293997 + }, + { + "contact_quality": null, + "progress": 0.0069203972816467285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0069203972816467285 + }, + { + "contact_quality": null, + "progress": 0.0071913800202310085, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0071913800202310085 + } + ], + "candidate_progress": [ + 0.009399374015629292, + 0.006204982753843069, + 0.007237812038511038, + 0.008653719909489155, + 0.010276774875819683, + 0.006190141197293997, + 0.0069203972816467285, + 0.0071913800202310085 + ], + "candidate_source_chart_ids": [ + "ms-7eaea428dfdef7255a66", + "ms-dc26b448cce4164a23e6", + "ms-e84163d9ef2e7014daca", + "ms-6a3add21affa4796f9be", + "ms-6a3add21affa4796f9be", + "ms-e84163d9ef2e7014daca", + "ms-f85a653af630821f2572", + "ms-0e592951fa32139b6dc9" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-57be9ae181c077e7821f", + "generated_tangents": [ + [ + -0.30926334857940674, + -0.3646051287651062, + 1.0946054458618164, + -0.36321312189102173, + 0.9840645790100098, + 2.676570177078247, + 1.7332475185394287, + 0.04148532450199127, + 0.08790934830904007, + 0.9474417567253113, + -0.09922204166650772, + 0.810888409614563, + 1.813698649406433, + 2.016425848007202, + 0.8269513845443726, + -0.5040432810783386, + -0.19824446737766266, + 0.20947657525539398, + 0.08509057760238647, + 1.264101505279541, + 0.2216612994670868 + ], + [ + -0.6527650952339172, + -0.3242778778076172, + 1.444452166557312, + -0.5342669486999512, + 0.7839994430541992, + 2.1463117599487305, + 1.489823579788208, + 0.2299799919128418, + 0.1252458691596985, + 1.0040754079818726, + -0.11908742040395737, + 1.0013298988342285, + 1.8913668394088745, + 1.497053861618042, + 0.5943560600280762, + -0.5696547627449036, + -0.1311655193567276, + 0.21365685760974884, + 0.25112485885620117, + 1.3706295490264893, + 0.27272772789001465 + ], + [ + -0.7989112734794617, + -0.21156653761863708, + 1.441413402557373, + -0.3115503191947937, + 0.6862883567810059, + 2.3581361770629883, + 1.1852996349334717, + 0.18688702583312988, + 0.030488146468997, + 0.7526087760925293, + -0.0653768926858902, + 1.264310359954834, + 1.640783429145813, + 1.4627430438995361, + 0.5899918675422668, + -0.5287672281265259, + -0.2672613263130188, + 0.24378249049186707, + 0.20653939247131348, + 1.1917920112609863, + 0.09985701739788055 + ], + [ + -0.7405605316162109, + -0.1526736468076706, + 1.1547361612319946, + -0.19872139394283295, + 0.8518969416618347, + 2.5984432697296143, + 1.2750420570373535, + 0.3957311809062958, + 0.031122282147407532, + 1.0674495697021484, + -0.36115434765815735, + 0.8102352619171143, + 1.6482597589492798, + 1.3360583782196045, + 0.5427156686782837, + -0.40142983198165894, + -0.15791146457195282, + 0.26739078760147095, + 0.3148157298564911, + 1.5067071914672852, + 0.7017356157302856 + ], + [ + -0.7004825472831726, + -0.2538937032222748, + 1.2259334325790405, + -0.03816375136375427, + 0.7258093357086182, + 2.302774429321289, + 1.3373589515686035, + 0.5013123750686646, + -0.050833381712436676, + 0.5012713074684143, + -0.3792320787906647, + 1.0458316802978516, + 1.6910203695297241, + 1.5447372198104858, + 0.5399075150489807, + -0.6167660355567932, + -0.14431537687778473, + 0.20329181849956512, + 0.2862829864025116, + 1.4421494007110596, + -0.006353422999382019 + ], + [ + -0.8840819597244263, + -0.2599353790283203, + 1.60691499710083, + -0.1055542379617691, + 0.7680267691612244, + 2.445167303085327, + 1.0817859172821045, + 0.23314587771892548, + -0.04322775453329086, + 0.6557232737541199, + -0.260415643453598, + 1.2362195253372192, + 1.4674233198165894, + 1.7542859315872192, + 0.6350005269050598, + -0.4188421368598938, + 0.003112286329269409, + 0.23604613542556763, + 0.3358215391635895, + 1.2808003425598145, + 0.0012314319610595703 + ], + [ + -1.0495582818984985, + -0.22637467086315155, + 1.338814616203308, + -0.38610005378723145, + 1.3288910388946533, + 2.4673244953155518, + 1.1249277591705322, + 0.10093340277671814, + -0.008707791566848755, + 1.0432755947113037, + 0.36337634921073914, + 0.5582226514816284, + 1.5481916666030884, + 1.3156152963638306, + 1.004977822303772, + -0.19249799847602844, + -0.3012973666191101, + 0.14856944978237152, + 0.4736841022968292, + 1.6065986156463623, + 0.3558976650238037 + ], + [ + -0.5317566394805908, + -0.08665710687637329, + 1.5755513906478882, + -0.02031444013118744, + 1.0695438385009766, + 2.639113664627075, + 1.1690127849578857, + 0.27002424001693726, + -0.02944600023329258, + 1.0525678396224976, + -0.10891043394804001, + 0.8228455781936646, + 1.7721060514450073, + 1.7008202075958252, + 0.6315513253211975, + -0.1996202915906906, + -0.004995107650756836, + 0.22394779324531555, + 0.10063542425632477, + 1.4173269271850586, + 1.517505407333374 + ] + ], + "generated_utilities": [ + 0.009399374015629292, + 0.006204982753843069, + 0.007237812038511038, + 0.008653719909489155, + 0.010276774875819683, + 0.006190141197293997, + 0.0069203972816467285, + 0.0071913800202310085 + ], + "group_id": "ms-57be9ae181c077e7821f", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.28031715750694275, + 0.19804009795188904, + 0.15024888515472412, + 0.23104076087474823, + 0.21580509841442108, + 0.013450047001242638, + 0.20318475365638733, + 0.22562944889068604, + 0.14178292453289032, + 0.15763869881629944, + 0.27735981345176697, + 0.1909358948469162, + 0.06666654348373413, + 2.0, + 0.060366373509168625, + 0.05085722729563713 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.06739383935928345, + 0.06319573521614075, + 0.8779754638671875, + -0.03682136535644531, + -0.07093041390180588, + 1.3382306098937988, + 0.0, + 0.07168813049793243, + 0.01971740648150444, + 0.4866671562194824, + 0.07402461767196655, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + 0.03574584424495697, + 0.021435141563415527, + 0.0021790266036987305, + -0.06565424799919128, + 1.1393508911132812, + 0.0 + ], + [ + -0.17397063970565796, + -0.08753994852304459, + 0.8779754638671875, + -0.09028738737106323, + -0.17196941375732422, + 1.3382306098937988, + 0.0, + 0.022144019603729248, + 0.1154828891158104, + 0.4866671562194824, + -0.09212043881416321, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + -0.2021142691373825, + -0.022606968879699707, + 0.0021790266036987305, + 0.03272634744644165, + 1.1393508911132812, + 0.0 + ], + [ + -1.0144902467727661, + -0.023572498932480812, + 1.8779754638671875, + -1.0368213653564453, + -0.03799626976251602, + 2.338230609893799, + 2.239948272705078, + 0.16168542206287384, + 0.07221447676420212, + 1.4866671562194824, + -0.3557073175907135, + 1.3622210025787354, + 1.896626591682434, + 2.0382778644561768, + 1.4886415004730225, + 0.1997920423746109, + -0.4386618137359619, + 1.0021790266036987, + -0.3174663782119751, + 2.1393508911132812, + 1.9118322134017944 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.479896545410156, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0765557289123535, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.823664426803589 + ], + [ + -1.0723698139190674, + -0.05779706686735153, + 2.0670831203460693, + -0.830497682094574, + -0.17302533984184265, + 2.2277159690856934, + 2.0435242652893066, + 0.08191673457622528, + 0.08711377531290054, + 1.416495680809021, + -0.5633984804153442, + 1.3216067552566528, + 1.8660295009613037, + 2.2198705673217773, + 1.6887608766555786, + 0.06658947467803955, + -0.4010601043701172, + 1.1803714036941528, + -0.15144284069538116, + 2.308220386505127, + 1.8460347652435303 + ], + [ + -0.8901733756065369, + 0.3378032147884369, + 2.2360360622406006, + -1.207388162612915, + -0.34048303961753845, + 2.460798978805542, + 2.100795030593872, + -0.1847335249185562, + -0.06362440437078476, + 1.2987924814224243, + -0.11972835659980774, + 1.170393943786621, + 2.0620977878570557, + 2.4285032749176025, + 1.3666276931762695, + -0.01825574040412903, + -0.4620283842086792, + 1.1138218641281128, + -0.5787107944488525, + 1.78761887550354, + 1.740113615989685 + ], + [ + -1.1796774864196777, + -0.12369386106729507, + 1.546891689300537, + -1.0500564575195312, + -0.026454759761691093, + 2.177122116088867, + 2.0003931522369385, + 0.13697585463523865, + 0.37715619802474976, + 1.9734747409820557, + -0.16338464617729187, + 1.3775254487991333, + 2.3054146766662598, + 1.5546050071716309, + 1.2645983695983887, + 0.3812609910964966, + 0.02191823720932007, + 1.0944219827651978, + 0.09844094514846802, + 1.8145865201950073, + 2.0624771118164062 + ], + [ + -0.5514534711837769, + -0.4441075921058655, + 1.8971614837646484, + -1.6112515926361084, + 0.1306135058403015, + 2.736365556716919, + 2.270737886428833, + -0.35948747396469116, + 0.03968408331274986, + 1.061530351638794, + -0.29366230964660645, + 1.5066968202590942, + 2.34443998336792, + 2.3445637226104736, + 1.2286460399627686, + 0.13919776678085327, + -0.21014492213726044, + 0.9723342061042786, + -0.5822539329528809, + 1.8932968378067017, + 2.159864664077759 + ], + [ + -0.9470903873443604, + -0.7433359622955322, + 1.798203945159912, + -0.5850675106048584, + -0.7611241340637207, + 2.2761964797973633, + 2.0114798545837402, + 0.7724847793579102, + -0.23453289270401, + 0.7384847402572632, + -1.022956132888794, + 1.0059813261032104, + 1.652310848236084, + 2.6376113891601562, + 2.148331642150879, + 0.9844839572906494, + 0.18531352281570435, + 1.6280735731124878, + -0.06812705099582672, + 2.2684829235076904, + 1.1641287803649902 + ], + [ + -1.729402780532837, + 0.15382106602191925, + 1.0736894607543945, + -0.5141006708145142, + -0.6476962566375732, + 2.0120980739593506, + 2.8703343868255615, + 0.5578619837760925, + -0.8154823184013367, + 1.4325131177902222, + -1.0468122959136963, + 0.9670607447624207, + 1.973939061164856, + 2.7364890575408936, + 1.8303502798080444, + 0.8089333772659302, + -0.41138362884521484, + 1.8146085739135742, + -0.11676108837127686, + 2.3541221618652344, + 2.3875224590301514 + ], + [ + -1.8409323692321777, + -0.2118261456489563, + 1.5354409217834473, + -0.3873547911643982, + 0.7043789029121399, + 2.2068612575531006, + 2.5864486694335938, + 0.8632195591926575, + -0.10360705107450485, + 1.6340630054473877, + -0.017105937004089355, + 1.90720796585083, + 2.608898639678955, + 2.3387808799743652, + 2.3744726181030273, + 0.13692857325077057, + -0.7716406583786011, + 1.3477450609207153, + -0.9811362028121948, + 2.15514874458313, + 2.298004150390625 + ], + [ + -1.2484530210494995, + 0.5977066159248352, + 2.8256869316101074, + -0.3418053388595581, + -1.031638503074646, + 1.7023751735687256, + 1.4487309455871582, + -0.3013373017311096, + -0.3682725131511688, + 2.4404053688049316, + -0.8263542652130127, + 2.1105899810791016, + 2.5769248008728027, + 1.6751525402069092, + 0.8367876410484314, + -0.1554209142923355, + 0.17985862493515015, + 1.0017495155334473, + -0.3176954984664917, + 2.6737403869628906, + 1.7722500562667847 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -1.0015538930892944, + 0.43126925826072693, + 1.2786648273468018, + -0.7182899713516235, + 0.06109866499900818, + 2.323270797729492, + 1.3882524967193604, + -0.2783380150794983, + 0.35085979104042053, + 0.5870303511619568, + -1.0134913921356201, + 0.4675567150115967, + 0.9185858964920044, + 1.9441449642181396, + 1.3693488836288452, + 0.05759717524051666, + 0.19058758020401, + 1.9360744953155518, + -1.0018632411956787, + 2.043489456176758, + 1.1226990222930908 + ] + ], + "predicted_scores": [ + 5.009902000427246, + 3.7094740867614746, + 3.506354570388794, + 3.256587028503418, + 3.1261518001556396, + 2.2112104892730713, + 1.8244775533676147, + 1.5559664964675903 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train2", + "selected_index": 0, + "state_hash": "9c4de49d50d74c8c571620759fff1d43be23093a5f66e946f7ae6d75cb26ade5", + "stored_base_utility": 0.28031715750694275, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1544450968503952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1544450968503952 + }, + "base_progress": 0.1544450968503952, + "base_success": false, + "base_utility": 0.1544450968503952, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.009421487338840961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009421487338840961 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08336275815963745, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08336275815963745 + }, + { + "contact_quality": null, + "progress": 0.07322987169027328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07322987169027328 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08531469106674194, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08531469106674194 + }, + { + "contact_quality": null, + "progress": 0.08539849519729614, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08539849519729614 + } + ], + "candidate_progress": [ + 0.009421487338840961, + 1.0, + 1.0, + 0.08336275815963745, + 0.07322987169027328, + 1.0, + 0.08531469106674194, + 0.08539849519729614 + ], + "candidate_source_chart_ids": [ + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-ff129e39095c2861b522", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-6388b5d2356c556c3e55", + "generated_tangents": [ + [ + -0.1346358209848404, + -0.290164977312088, + 1.3188178539276123, + 0.1854286789894104, + 1.2796549797058105, + 3.007685899734497, + 2.3563530445098877, + 2.484987497329712, + 0.026990480720996857, + -0.27481046319007874, + 0.751373291015625, + 0.7147144079208374, + 1.8042527437210083, + 2.37459659576416, + -0.26678499579429626, + 0.02226560190320015, + 0.9388956427574158, + 0.8101158738136292, + 1.034219741821289, + 2.466493606567383, + 2.2822017669677734 + ], + [ + -0.2568861246109009, + -0.19353245198726654, + 1.4529776573181152, + -0.1390744000673294, + 1.0377912521362305, + 2.9880504608154297, + 2.1059212684631348, + 2.381761312484741, + 0.15299169719219208, + -0.21691611409187317, + 1.122802495956421, + 1.1129459142684937, + 2.4177308082580566, + 2.2433297634124756, + -0.39117854833602905, + -0.04049832001328468, + 0.876803994178772, + 0.7658475637435913, + 1.1124186515808105, + 2.7858569622039795, + 2.717148780822754 + ], + [ + -0.28434789180755615, + -0.17466460168361664, + 1.3416450023651123, + -0.20311586558818817, + 1.3057668209075928, + 2.56211256980896, + 2.5722076892852783, + 1.9682852029800415, + -0.0061507876962423325, + -0.18780440092086792, + 1.0633792877197266, + 0.8869252800941467, + 2.2866291999816895, + 2.264634132385254, + -0.6698980927467346, + 0.10091784596443176, + 0.6638089418411255, + 0.7630418539047241, + 1.1821287870407104, + 2.280217409133911, + 2.386282205581665 + ], + [ + -0.26237136125564575, + -0.23905599117279053, + 1.6208723783493042, + -0.19124546647071838, + 0.9796238541603088, + 2.8906607627868652, + 2.228334665298462, + 2.2495615482330322, + 0.1718578040599823, + -0.1611553430557251, + 1.1272178888320923, + 1.2065633535385132, + 2.335747003555298, + 2.2670536041259766, + -0.4416409134864807, + -0.01251941453665495, + 0.8514349460601807, + 0.6165656447410583, + 0.9599085450172424, + 2.9152796268463135, + 2.509885549545288 + ], + [ + -0.3158658742904663, + -0.17014434933662415, + 1.6859221458435059, + -0.25803977251052856, + 1.0942418575286865, + 2.8559882640838623, + 2.141686201095581, + 2.1352880001068115, + 0.0933176651597023, + -0.07159272581338882, + 1.4386317729949951, + 0.7763835191726685, + 2.158827304840088, + 2.4083712100982666, + -0.6753755211830139, + -0.09084516763687134, + 0.6954506635665894, + 0.6088263988494873, + 0.6035493612289429, + 2.703592538833618, + 2.6240713596343994 + ], + [ + -0.22186796367168427, + -0.04749760404229164, + 1.3944339752197266, + -0.17063748836517334, + 1.1214823722839355, + 2.5736377239227295, + 2.551818609237671, + 2.207057476043701, + 0.15786957740783691, + 0.08558044582605362, + 1.23016357421875, + 0.5588364601135254, + 2.169656991958618, + 2.2912497520446777, + -0.6412237286567688, + -0.07000379264354706, + 0.8803773522377014, + 0.5812371373176575, + 0.7415778636932373, + 2.634361505508423, + 2.1934618949890137 + ], + [ + -0.17021532356739044, + -0.08369076251983643, + 1.5253565311431885, + -0.14208582043647766, + 1.0116651058197021, + 2.639970541000366, + 2.589531660079956, + 2.143213987350464, + 0.16520118713378906, + 0.16542699933052063, + 1.3231823444366455, + 0.6546212434768677, + 2.240913152694702, + 2.1633529663085938, + -0.6388358473777771, + -0.04554488882422447, + 0.6752522587776184, + 0.5876330137252808, + 0.8056936264038086, + 2.764847755432129, + 2.196819305419922 + ], + [ + -0.015666082501411438, + -0.14996975660324097, + 1.6243469715118408, + -0.026797041296958923, + 1.1090970039367676, + 3.2096641063690186, + 1.8211236000061035, + 2.043275833129883, + 0.2226421982049942, + -0.3239434063434601, + 1.5672627687454224, + 1.3945112228393555, + 2.06219744682312, + 2.4981837272644043, + -0.3383021056652069, + -0.005654589273035526, + 1.1494463682174683, + 0.8538839221000671, + 0.9946694374084473, + 2.952598810195923, + 2.3358447551727295 + ] + ], + "generated_utilities": [ + 0.009421487338840961, + 2.0, + 2.0, + 0.08336275815963745, + 0.07322987169027328, + 2.0, + 0.08531469106674194, + 0.08539849519729614 + ], + "group_id": "ms-6388b5d2356c556c3e55", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08186312764883041, + 0.11667267978191376, + 0.09751813113689423, + 2.0, + 0.27431026101112366, + 0.11760994046926498, + 0.20386962592601776, + 0.19832774996757507, + 0.22735612094402313, + 0.1356993168592453, + 0.1713169515132904, + 0.1792232096195221, + 0.06848804652690887, + 0.1094626635313034, + 0.11555596441030502, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0018978863954544067, + -0.4448504149913788, + 0.7108399868011475, + -0.33157044649124146, + -0.062200844287872314, + 1.4758033752441406, + 0.0, + 1.4153923988342285, + 0.14116239547729492, + 0.00777280330657959, + 0.5541017055511475, + 0.07029426097869873, + 0.26837193965911865, + 0.0, + -0.015542566776275635, + 0.3573532998561859, + 0.6168417930603027, + 0.3612897992134094, + 0.369077205657959, + 1.5924487113952637, + 0.0 + ], + [ + -0.09549225866794586, + -0.20593886077404022, + 1.7108399868011475, + -0.1554051786661148, + 0.9377991557121277, + 2.4758033752441406, + 2.4156031608581543, + 2.4153923988342285, + -0.2459496259689331, + -0.9922271966934204, + 1.5541017055511475, + 0.4461512267589569, + 1.2501436471939087, + 1.7566912174224854, + -0.8014360666275024, + 0.04551727697253227, + 1.6168417930603027, + 0.5393297672271729, + 1.369077205657959, + 2.5924487113952637, + 3.1126015186309814 + ], + [ + -0.11610143631696701, + -0.3961976170539856, + 1.9081053733825684, + -0.14575888216495514, + 1.1381605863571167, + 2.7434439659118652, + 2.5136890411376953, + 2.6401467323303223, + -0.45453745126724243, + -0.9589858651161194, + 1.4083189964294434, + 0.5800129175186157, + 0.9940754175186157, + 1.7450767755508423, + -1.0145976543426514, + 0.3312625586986542, + 1.4729199409484863, + 0.2520747482776642, + 1.3623151779174805, + 2.5670394897460938, + 3.3589630126953125 + ], + [ + -0.2318952977657318, + 0.024853751063346863, + 1.829875111579895, + 0.09877045452594757, + 1.1515556573867798, + 2.2239503860473633, + 2.153250217437744, + 2.8057901859283447, + -0.1800437569618225, + -1.2225371599197388, + 1.421208143234253, + 0.6627710461616516, + 1.3895055055618286, + 1.856442928314209, + -0.5928603410720825, + 0.4364123046398163, + 1.7852745056152344, + 0.3345714807510376, + 1.1167365312576294, + 2.7676005363464355, + 3.041402578353882 + ], + [ + -0.2227625697851181, + -0.012646213173866272, + 1.7988159656524658, + 0.21457992494106293, + 0.8877373933792114, + 2.9257700443267822, + 2.2101094722747803, + 2.2874886989593506, + -0.09066365659236908, + -0.49909964203834534, + 2.042389392852783, + 0.6287553906440735, + 1.074569582939148, + 1.7805378437042236, + -1.1840007305145264, + 0.3728531002998352, + 1.165173053741455, + 0.3054528832435608, + 1.4924153089523315, + 2.5540966987609863, + 2.664964199066162 + ], + [ + -0.6864494681358337, + 0.119636669754982, + 1.2460956573486328, + -0.6367122530937195, + 1.4080986976623535, + 2.5262017250061035, + 2.508194923400879, + 2.1323020458221436, + -0.43302130699157715, + -1.2678899765014648, + 1.0870052576065063, + 0.46296897530555725, + 0.765267550945282, + 2.3093044757843018, + -0.46595194935798645, + -0.43870145082473755, + 2.028986930847168, + 0.26284897327423096, + 1.5542359352111816, + 2.530392646789551, + 2.7435970306396484 + ], + [ + 0.18175412714481354, + -0.478390634059906, + 1.286731481552124, + -0.8066962361335754, + 1.4787592887878418, + 2.1033878326416016, + 1.750453233718872, + 2.047861099243164, + 0.3163147568702698, + -0.30588728189468384, + 1.0766479969024658, + 0.5822455286979675, + 1.673675775527954, + 2.039386510848999, + -0.62891685962677, + 0.615638792514801, + 2.018416404724121, + 0.2530737817287445, + 1.34544837474823, + 3.2287912368774414, + 2.9459850788116455 + ], + [ + 0.5059412121772766, + 0.17952178418636322, + 1.3319568634033203, + -0.8850483894348145, + 0.6336387395858765, + 1.775376796722412, + 2.3280270099639893, + 1.7305657863616943, + -0.02617873251438141, + -0.7963138818740845, + 1.9142115116119385, + 0.6635866165161133, + 1.3493314981460571, + 1.9258644580841064, + -0.3795176148414612, + 0.7861210107803345, + 1.0032029151916504, + 0.8491684794425964, + 0.5918128490447998, + 2.399027109146118, + 2.3932981491088867 + ], + [ + -0.34448128938674927, + -0.2996852695941925, + 0.9233196377754211, + -0.22890916466712952, + 1.6932814121246338, + 3.465762138366699, + 2.2005038261413574, + 2.6472361087799072, + -0.8981531858444214, + -0.2658765912055969, + 1.8755900859832764, + 0.2962568402290344, + 1.8239326477050781, + 1.0176585912704468, + -0.7912051677703857, + 0.6727061867713928, + 2.199978828430176, + 1.015731692314148, + 0.43087130784988403, + 3.5876548290252686, + 3.893789052963257 + ] + ], + "predicted_scores": [ + 6.715732097625732, + 5.764956951141357, + 5.601903438568115, + 5.531642913818359, + 5.302748203277588, + 5.134200096130371, + 4.9023871421813965, + 4.442611217498779 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train2", + "selected_index": 0, + "state_hash": "38d539e46140304e3f157cbc207aa0d9489e5470633c216a0379906af445a1ec", + "stored_base_utility": 0.08186312764883041, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06173701956868172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06173701956868172 + }, + "base_progress": 0.06173701956868172, + "base_success": false, + "base_utility": 0.06173701956868172, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.109224833548069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.109224833548069 + }, + { + "contact_quality": null, + "progress": 0.057921767234802246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057921767234802246 + }, + { + "contact_quality": null, + "progress": 0.06627502292394638, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06627502292394638 + }, + { + "contact_quality": null, + "progress": 0.021286645904183388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021286645904183388 + }, + { + "contact_quality": null, + "progress": 0.06679177284240723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06679177284240723 + }, + { + "contact_quality": null, + "progress": 0.04627428576350212, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04627428576350212 + }, + { + "contact_quality": null, + "progress": 0.054965514689683914, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054965514689683914 + }, + { + "contact_quality": null, + "progress": 0.14829117059707642, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14829117059707642 + } + ], + "candidate_progress": [ + 0.109224833548069, + 0.057921767234802246, + 0.06627502292394638, + 0.021286645904183388, + 0.06679177284240723, + 0.04627428576350212, + 0.054965514689683914, + 0.14829117059707642 + ], + "candidate_source_chart_ids": [ + "ms-3a4e2d080c9c4b279895", + "ms-7e88428497a6b13936c4", + "ms-3a4e2d080c9c4b279895", + "ms-a2518826fa0a816cc46f", + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-d535b38d768633f6d922" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-638d2ba1917eff4d09e0", + "generated_tangents": [ + [ + 0.19399428367614746, + -0.19429710507392883, + 0.7175668478012085, + 0.10569876432418823, + 1.6889584064483643, + 1.2522661685943604, + 0.3853796720504761, + 0.30492085218429565, + -0.23217259347438812, + -0.04009782522916794, + 0.34059959650039673, + 0.7768262028694153, + 0.19477424025535583, + 0.8821943402290344, + 0.8789517283439636, + 0.07258066534996033, + 0.0825379490852356, + 0.24814879894256592, + 1.0100672245025635, + -1.0790424346923828, + -0.5989418029785156 + ], + [ + 0.31291431188583374, + -0.44760996103286743, + 0.5468348860740662, + 0.04534590244293213, + 1.4834082126617432, + 0.4475386142730713, + 0.15500012040138245, + 0.3648453950881958, + -0.1622944325208664, + 0.283906489610672, + 0.05420593544840813, + 0.6349453926086426, + -0.5642687082290649, + 0.5123119950294495, + 0.3089844882488251, + 0.13765224814414978, + 0.023821890354156494, + -0.1964070051908493, + 0.7212094664573669, + -1.5697035789489746, + -1.1024484634399414 + ], + [ + 0.21944445371627808, + -0.26180827617645264, + 0.7513684034347534, + 0.07701483368873596, + 1.6302812099456787, + 1.209617257118225, + 0.567581832408905, + 0.43779635429382324, + -0.13354693353176117, + 0.07422672212123871, + 0.13548581302165985, + 0.7683057188987732, + 0.16806039214134216, + 0.6201119422912598, + 0.6791425347328186, + 0.0793917179107666, + 0.247845858335495, + 0.19680571556091309, + 1.0756046772003174, + -0.9696177840232849, + -0.5063502788543701 + ], + [ + -0.004198238253593445, + -0.374566912651062, + 0.6863826513290405, + -0.3365015983581543, + 1.32783842086792, + 1.1231145858764648, + 0.16173768043518066, + 0.12974990904331207, + -0.10361332446336746, + 0.23197752237319946, + 0.2207789421081543, + 0.8557438254356384, + -0.06281271576881409, + 0.5347269773483276, + 0.2829627990722656, + -0.15411359071731567, + -0.03339408338069916, + 0.1670663207769394, + 0.570730447769165, + -1.079788327217102, + -0.38740941882133484 + ], + [ + 0.2365972101688385, + -0.41906940937042236, + 0.5717867016792297, + -0.32165977358818054, + 1.379390001296997, + 0.3509386479854584, + 0.7487069368362427, + 0.21619491279125214, + 6.029754877090454e-05, + 0.09556954354047775, + 0.30910229682922363, + 0.43290406465530396, + -0.3340233564376831, + 0.2862071990966797, + 0.5412895679473877, + 0.047336529940366745, + 0.22724208235740662, + -0.0628570169210434, + 1.2245359420776367, + -1.1665306091308594, + -0.6084585189819336 + ], + [ + 0.06348957866430283, + -0.22858506441116333, + 0.7147878408432007, + -0.23703420162200928, + 1.6729516983032227, + 0.7785142660140991, + 0.6724982261657715, + 0.34640511870384216, + -0.11504130065441132, + -0.07103797048330307, + 0.1612311154603958, + 0.614039421081543, + -0.08057567477226257, + 0.7951291799545288, + 0.4771057367324829, + 0.01192605309188366, + -0.029168590903282166, + 0.09965971857309341, + 0.7977463006973267, + -0.8623479008674622, + -0.6617336273193359 + ], + [ + 0.26776623725891113, + -0.2583177089691162, + 0.7229114770889282, + -0.25329554080963135, + 1.555058479309082, + 0.5885607004165649, + 0.5584661364555359, + 0.18812701106071472, + -0.018099995329976082, + 0.23198270797729492, + 0.2221827208995819, + 0.5615720152854919, + 0.22216364741325378, + 0.3671315908432007, + 0.30829712748527527, + 0.032752685248851776, + 0.15349799394607544, + -0.044431447982788086, + 0.9087569713592529, + -0.9487636089324951, + -0.48602381348609924 + ], + [ + 0.20732782781124115, + -0.4252946972846985, + 0.5844835042953491, + -0.27273496985435486, + 1.3516948223114014, + 0.6736023426055908, + 0.6423986554145813, + 0.35052746534347534, + -0.009159471839666367, + -0.15334001183509827, + 0.24608778953552246, + 0.47937795519828796, + -0.3758259415626526, + 0.36896395683288574, + 0.5268030166625977, + 0.0503540076315403, + 0.11380910128355026, + -0.07216577231884003, + 1.3559410572052002, + -1.0476348400115967, + -0.6348272562026978 + ] + ], + "generated_utilities": [ + 0.109224833548069, + 0.057921767234802246, + 0.06627502292394638, + 0.021286645904183388, + 0.06679177284240723, + 0.04627428576350212, + 0.054965514689683914, + 0.14829117059707642 + ], + "group_id": "ms-638d2ba1917eff4d09e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.028925320133566856, + 0.11176212877035141, + 0.057868462055921555, + 0.09960909932851791, + 0.07349967956542969, + 0.029011588543653488, + 0.06443297863006592, + 0.05166193097829819, + 0.025834282860159874, + 0.056667208671569824, + 0.04477008432149887, + 0.019685089588165283, + 0.018628716468811035, + 0.08587563037872314, + 0.02232462167739868, + 0.010265171527862549 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04120570421218872, + 0.038339823484420776, + 0.5979268550872803, + -0.05306033045053482, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.3407015800476074, + -0.21580028533935547, + -0.036036908626556396, + -0.004389047622680664, + 0.5364620685577393, + 0.026709645986557007, + 0.0, + 1.0126078128814697, + -0.10034149885177612, + -0.16565871238708496, + 0.07125610113143921, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + -0.5276790261268616, + -0.19392354786396027, + 0.5979268550872803, + 0.05016906559467316, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.6852816939353943, + -0.21580028533935547, + 0.04531368613243103, + -0.09276384115219116, + 0.5364620685577393, + 0.021915271878242493, + 0.0, + 1.0126078128814697, + 0.29255303740501404, + -0.16565871238708496, + 0.14600259065628052, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + 0.5910290479660034, + -0.038985610008239746, + 1.5555552244186401, + -1.028314232826233, + 2.3103959560394287, + 0.8163888454437256, + 2.064336061477661, + 1.168483018875122, + -0.21718227863311768, + -0.19770365953445435, + 0.11310696601867676, + 0.6850457191467285, + -0.06804794818162918, + 0.33076921105384827, + 1.7529149055480957, + -0.562712550163269, + -1.6352436542510986, + 0.1224215030670166, + 3.354771852493286, + -1.1317338943481445, + -1.7348618507385254 + ] + ], + "predicted_scores": [ + 4.312126636505127, + 4.290807723999023, + 4.064854145050049, + 3.963754892349243, + 3.7331457138061523, + 3.612520694732666, + 3.518197536468506, + 3.428798198699951 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train2", + "selected_index": 0, + "state_hash": "5d6f4227fab3b4bb36756c9a15c32b6fe6c9a0ae4b07b3f20a2bd65c2594df13", + "stored_base_utility": 0.028925320133566856, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.018323838710784912, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018323838710784912 + }, + { + "contact_quality": null, + "progress": 0.017151853069663048, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017151853069663048 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.023069700226187706, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023069700226187706 + }, + { + "contact_quality": null, + "progress": 0.005640605930238962, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005640605930238962 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.019837161526083946, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019837161526083946 + }, + { + "contact_quality": null, + "progress": 0.016323328018188477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016323328018188477 + } + ], + "candidate_progress": [ + 0.018323838710784912, + 0.017151853069663048, + 1.0, + 0.023069700226187706, + 0.005640605930238962, + 1.0, + 0.019837161526083946, + 0.016323328018188477 + ], + "candidate_source_chart_ids": [ + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-a6a6de64c39abad45f86" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + true, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-652ad552ba98d2736620", + "generated_tangents": [ + [ + 0.3993667960166931, + 0.01467999815940857, + 0.8236827850341797, + -0.701335072517395, + 2.074022054672241, + 1.8071799278259277, + 0.779463529586792, + 0.3570999205112457, + 0.06147110462188721, + 0.7161964774131775, + 0.3467694818973541, + 1.2415695190429688, + -0.12027537822723389, + -0.23912478983402252, + -0.42852506041526794, + 0.04579297453165054, + 1.4412566423416138, + -0.21752147376537323, + 1.143805980682373, + 2.98844838142395, + 1.2782543897628784 + ], + [ + -0.024377495050430298, + -0.027264267206192017, + 0.7988369464874268, + -0.7997981309890747, + 2.4030120372772217, + 2.5693132877349854, + 0.9931743741035461, + 0.28386300802230835, + -0.00976945273578167, + 1.0466086864471436, + 0.6274899840354919, + 1.215024471282959, + -1.2369223833084106, + 0.47804349660873413, + -0.31641435623168945, + 0.056881461292505264, + 1.430560827255249, + -0.3863064646720886, + 0.6832652688026428, + 2.42386531829834, + 1.7024511098861694 + ], + [ + 0.09451492875814438, + -0.017048465088009834, + 0.9759140014648438, + -0.8719127178192139, + 2.0689077377319336, + 1.8039824962615967, + 0.901330292224884, + 0.24651363492012024, + 0.0975688025355339, + 0.6806550621986389, + 0.5312308669090271, + 1.3876440525054932, + -0.11206024885177612, + 0.10132507979869843, + -0.3278767764568329, + -0.131260946393013, + 1.5182933807373047, + -0.17465271055698395, + 0.9838205575942993, + 2.670842409133911, + 1.2546682357788086 + ], + [ + 0.026769131422042847, + 0.0005880407989025116, + 0.7986657619476318, + -0.8370130062103271, + 1.9655966758728027, + 1.8510187864303589, + 0.7054927349090576, + 0.31965190172195435, + 0.08110027760267258, + 0.7890376448631287, + 0.48895692825317383, + 1.372125267982483, + -0.3302253484725952, + 0.14341162145137787, + -0.32688868045806885, + -0.07599435746669769, + 1.526108741760254, + -0.12525631487369537, + 0.8992928266525269, + 2.496145248413086, + 1.4744231700897217 + ], + [ + 0.027667418122291565, + -0.06151444464921951, + 1.2060891389846802, + -0.7992005348205566, + 1.8079657554626465, + 2.1099348068237305, + 0.6640889048576355, + 0.24634325504302979, + 0.04415634274482727, + 0.7627803683280945, + 0.47449788451194763, + 1.4929237365722656, + -0.38216859102249146, + 0.27360230684280396, + -0.3836110830307007, + -0.047117680311203, + 1.5105971097946167, + -0.23858605325222015, + 0.7320762276649475, + 2.231227159500122, + 1.9689866304397583 + ], + [ + 0.06250117719173431, + 0.04385526478290558, + 0.9026212692260742, + -0.6674528121948242, + 1.7355897426605225, + 1.8075567483901978, + 0.394543319940567, + 0.4861884117126465, + 0.09448187798261642, + 0.6392885446548462, + 0.38521555066108704, + 1.336188554763794, + 0.06261160969734192, + 0.19100789725780487, + -0.4509143829345703, + -0.05264287814497948, + 1.441964864730835, + -0.00014866888523101807, + 0.7572662830352783, + 2.3709421157836914, + 1.1847805976867676 + ], + [ + 0.09632676839828491, + -0.009599275887012482, + 0.7571297287940979, + -0.7809529304504395, + 2.08193302154541, + 1.2948377132415771, + 0.44271188974380493, + 0.16933895647525787, + -0.02538822777569294, + 0.8536251187324524, + 0.3307485580444336, + 1.5156704187393188, + -0.5239653587341309, + 0.05099817365407944, + -0.2650001049041748, + 0.08685488998889923, + 1.6421197652816772, + -0.18257303535938263, + 0.7593610882759094, + 1.9531049728393555, + 1.2956956624984741 + ], + [ + -0.12293921411037445, + 0.1826166808605194, + 0.9262709617614746, + -0.9620940685272217, + 2.1618926525115967, + 1.9479553699493408, + 0.44185671210289, + 0.18182949721813202, + 0.023798175156116486, + 0.9044660329818726, + 0.4397014081478119, + 1.205376386642456, + 0.04302605986595154, + 0.4440864324569702, + -0.33897510170936584, + -0.12255626916885376, + 1.4983183145523071, + -0.13862715661525726, + 1.0010534524917603, + 2.302675485610962, + 1.759432315826416 + ] + ], + "generated_utilities": [ + 0.018323838710784912, + 0.017151853069663048, + 2.0, + 0.023069700226187706, + 0.005640605930238962, + 2.0, + 0.019837161526083946, + 0.016323328018188477 + ], + "group_id": "ms-652ad552ba98d2736620", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08378833532333374, + 0.10458396375179291, + 0.19086095690727234, + 0.04039820283651352, + 0.15980230271816254, + 0.0428888201713562, + 0.2367449700832367, + 0.18185830116271973, + 0.17593276500701904, + 0.16103637218475342, + 0.15250858664512634, + 0.05226989835500717, + 0.1701633185148239, + 0.11590366065502167, + 0.05217729136347771, + 0.03575938940048218 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.09009888768196106, + 0.08798432350158691, + 0.718157172203064, + -0.6351244449615479, + 1.405301570892334, + 0.6596384048461914, + 0.0, + 0.060225605964660645, + -0.0894029438495636, + -0.10323715209960938, + 0.052752673625946045, + 0.5354657173156738, + -0.013395614922046661, + 0.0, + -0.3450838029384613, + -0.11172826588153839, + 0.7912200689315796, + 0.33702391386032104, + 0.5586241483688354, + 1.8093180656433105, + 0.0 + ], + [ + 0.557296633720398, + 1.087984323501587, + 1.718157172203064, + -1.6351244449615479, + 2.405301570892334, + 1.6596384048461914, + 1.7430553436279297, + 0.3360595703125, + -0.2521040737628937, + 0.8967628479003906, + 0.26587578654289246, + 1.5354657173156738, + -0.12294083833694458, + -0.022678710520267487, + -0.416314959526062, + -0.26016199588775635, + 1.7912200689315796, + -0.16825227439403534, + 1.5586241483688354, + 2.8093180656433105, + 2.2589924335479736 + ], + [ + 0.2635665237903595, + 1.0005124807357788, + 1.973160743713379, + -1.7548720836639404, + 2.176497459411621, + 1.5416926145553589, + 1.9556171894073486, + 0.2215275913476944, + -0.5432531833648682, + 0.8434366583824158, + 0.3559061288833618, + 1.6657240390777588, + -0.3025258779525757, + 0.04401832073926926, + -0.17062894999980927, + -0.1509227454662323, + 1.7901219129562378, + 0.0010261386632919312, + 1.726505994796753, + 2.9778714179992676, + 2.0530292987823486 + ], + [ + 0.23987123370170593, + 0.8904718160629272, + 1.467047929763794, + -1.990342378616333, + 2.7685346603393555, + 2.0486786365509033, + 2.013002395629883, + 0.006436288356781006, + -0.25436922907829285, + 1.1101897954940796, + 0.4448293447494507, + 1.3105798959732056, + 0.07367905974388123, + 0.23779401183128357, + -0.34580594301223755, + -0.35948312282562256, + 1.4435756206512451, + -0.35472720861434937, + 1.801603078842163, + 3.0683224201202393, + 2.2330009937286377 + ], + [ + 0.6738064289093018, + 1.3973212242126465, + 2.0960373878479004, + -1.735988736152649, + 2.5216798782348633, + 1.1975955963134766, + 2.234865665435791, + 0.4602587819099426, + -0.7169049382209778, + 0.9412583708763123, + 0.7081141471862793, + 1.1730585098266602, + -0.016052812337875366, + -0.10474976152181625, + -0.03458032011985779, + 0.044317394495010376, + 1.6339548826217651, + -0.5292693376541138, + 1.5412750244140625, + 3.018833875656128, + 2.6051149368286133 + ], + [ + 0.6305910348892212, + 1.540043592453003, + 1.626009464263916, + -1.8823078870773315, + 2.2431485652923584, + 2.1866872310638428, + 2.3003249168395996, + 0.25838330388069153, + -0.2936764061450958, + 1.2413102388381958, + -0.12014040350914001, + 1.743309497833252, + 0.3618801534175873, + -0.37178292870521545, + 0.11553955078125, + 0.04856392741203308, + 1.2029554843902588, + 0.04960852861404419, + 2.0190422534942627, + 2.5883281230926514, + 2.7636377811431885 + ], + [ + 0.5004369616508484, + 1.4795310497283936, + 2.325191020965576, + -1.7464677095413208, + 1.9271103143692017, + 1.2483245134353638, + 1.9852193593978882, + 0.24291110038757324, + 0.4463852345943451, + 0.5267151594161987, + 0.32179588079452515, + 1.2356117963790894, + 0.03201058506965637, + 0.2881847023963928, + -1.0345205068588257, + -0.4171831011772156, + 1.5080864429473877, + 0.11479203402996063, + 1.9481834173202515, + 2.840816020965576, + 2.427428960800171 + ], + [ + -0.3133848309516907, + 0.43174320459365845, + 2.305823802947998, + -1.657350778579712, + 2.010749578475952, + 1.9222235679626465, + 1.3648147583007812, + 0.1938892900943756, + -0.6084526777267456, + 0.7855725288391113, + -0.07715931534767151, + 0.9140809178352356, + -0.9242425560951233, + -0.4681842029094696, + -0.07827043533325195, + -0.23733587563037872, + 0.9328996539115906, + -0.6088451147079468, + 0.8040639162063599, + 2.620466947555542, + 2.5148611068725586 + ] + ], + "predicted_scores": [ + 2.189335346221924, + 2.0589840412139893, + 1.8798794746398926, + 1.8045814037322998, + 1.5384629964828491, + 1.338186502456665, + 1.3312212228775024, + 0.7763898968696594 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424/train2", + "selected_index": 0, + "state_hash": "8c1ab306d1e82f81085196c60b3c846d2e2eb20ebc538597af359a6252623e7e", + "stored_base_utility": 0.08378833532333374, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03784912824630737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03784912824630737 + }, + "base_progress": 0.03784912824630737, + "base_success": false, + "base_utility": 0.03784912824630737, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03371499106287956, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03371499106287956 + }, + { + "contact_quality": null, + "progress": 0.04782070592045784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04782070592045784 + }, + { + "contact_quality": null, + "progress": 0.03829113766551018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03829113766551018 + }, + { + "contact_quality": null, + "progress": 0.12104523181915283, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12104523181915283 + }, + { + "contact_quality": null, + "progress": 0.07092549651861191, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07092549651861191 + }, + { + "contact_quality": null, + "progress": 0.0719817504286766, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0719817504286766 + }, + { + "contact_quality": null, + "progress": 0.05774464085698128, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05774464085698128 + }, + { + "contact_quality": null, + "progress": 0.028163930401206017, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.028163930401206017 + } + ], + "candidate_progress": [ + 0.03371499106287956, + 0.04782070592045784, + 0.03829113766551018, + 0.12104523181915283, + 0.07092549651861191, + 0.0719817504286766, + 0.05774464085698128, + 0.028163930401206017 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-48178f52a96de3d1ead0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-674fad0e7c042d23629e", + "generated_tangents": [ + [ + 0.9901873469352722, + 0.12360922992229462, + -0.2232363522052765, + -0.38938653469085693, + 0.3337686061859131, + -2.008176326751709, + -0.3517991304397583, + 0.46582189202308655, + 0.04784628748893738, + -0.33971720933914185, + 0.017600134015083313, + -0.5129214525222778, + -1.0531117916107178, + -0.4508397579193115, + -0.203139066696167, + 0.0007231589406728745, + 0.6066197156906128, + -0.7301870584487915, + -0.27302658557891846, + -1.1774073839187622, + -0.44787415862083435 + ], + [ + 0.8991358280181885, + 0.19254790246486664, + -0.17443087697029114, + -0.44649723172187805, + 0.23242536187171936, + -1.9363666772842407, + -0.1544741988182068, + 0.4052850604057312, + 0.044559232890605927, + -0.370149165391922, + -0.07202247530221939, + -0.4836416244506836, + -1.3371537923812866, + -0.23648254573345184, + -0.21802955865859985, + 0.05119294300675392, + 0.6496686339378357, + -0.6800534725189209, + -0.19558092951774597, + -1.0557403564453125, + -0.46343186497688293 + ], + [ + 0.9861155152320862, + 0.14955280721187592, + -0.3322644531726837, + -0.46376001834869385, + 0.4665031135082245, + -1.6896737813949585, + -0.16477763652801514, + 0.3933957517147064, + 0.07449617236852646, + -0.4259939193725586, + -0.03261926770210266, + -0.4738824963569641, + -1.3679347038269043, + -0.43129962682724, + -0.2540896534919739, + -0.12142802774906158, + 0.4131118953227997, + -0.7374284267425537, + -0.3372821807861328, + -1.2552692890167236, + -0.47657260298728943 + ], + [ + 0.7735786437988281, + 0.22672899067401886, + -0.22413203120231628, + -0.7112962007522583, + -0.16793420910835266, + -1.765476942062378, + -0.6376448273658752, + 0.27754539251327515, + -0.07475936412811279, + -0.7416353821754456, + -0.32008084654808044, + -0.44644373655319214, + -0.9784222841262817, + -0.6154751777648926, + -0.38817328214645386, + 0.1537492871284485, + 0.7908267378807068, + -0.8680008053779602, + 0.06949670612812042, + -1.153836727142334, + -0.06420813500881195 + ], + [ + 0.7969854474067688, + 0.22821016609668732, + -0.2962935268878937, + -0.6233872175216675, + 0.05773313343524933, + -2.0004169940948486, + -0.5777955651283264, + 0.28147614002227783, + -0.13770993053913116, + -0.5338562726974487, + -0.24591869115829468, + -0.5146559476852417, + -1.2625700235366821, + -0.49143052101135254, + -0.3756565749645233, + 0.14758989214897156, + 0.6155756115913391, + -0.7990389466285706, + -0.022716522216796875, + -1.0248379707336426, + -0.2633824050426483 + ], + [ + 0.7150876522064209, + 0.19802415370941162, + -0.3352034389972687, + -0.4889489710330963, + 0.09766247868537903, + -1.9784432649612427, + -0.5443668961524963, + 0.3809836506843567, + -0.12667275965213776, + -0.5288137793540955, + -0.30938172340393066, + -0.47195154428482056, + -1.2304800748825073, + -0.241909459233284, + -0.34431222081184387, + 0.12775352597236633, + 0.730070173740387, + -0.9150301814079285, + 0.06360238790512085, + -0.9554764032363892, + -0.44829538464546204 + ], + [ + 0.9162852764129639, + 0.1627006232738495, + -0.5613288879394531, + -0.6743629574775696, + 0.2753516137599945, + -1.476357102394104, + -0.49550431966781616, + 0.11746828258037567, + -0.007133401930332184, + -0.24751639366149902, + -0.10520651191473007, + -0.10798293352127075, + -1.2416572570800781, + -0.41993898153305054, + -0.6537002325057983, + -0.12407141923904419, + 0.3091234564781189, + -0.7746067643165588, + 0.16305281221866608, + -1.2652614116668701, + -0.30002978444099426 + ], + [ + 0.7665120363235474, + 0.003784533590078354, + -0.741275429725647, + -0.7520970106124878, + 0.4740793704986572, + -1.833933711051941, + 0.7798678278923035, + 0.7178219556808472, + -0.09947560727596283, + -0.48181402683258057, + -0.5541530251502991, + -0.373829185962677, + -1.8973037004470825, + -0.7944494485855103, + -0.4560420513153076, + 0.35093244910240173, + 0.08538462221622467, + -0.8632031083106995, + -0.3773888945579529, + -1.253408670425415, + -0.9948717355728149 + ] + ], + "generated_utilities": [ + 0.03371499106287956, + 0.04782070592045784, + 0.03829113766551018, + 0.12104523181915283, + 0.07092549651861191, + 0.0719817504286766, + 0.05774464085698128, + 0.028163930401206017 + ], + "group_id": "ms-674fad0e7c042d23629e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07829316705465317, + 0.080319844186306, + 0.11335399001836777, + 0.15264075994491577, + 0.14950451254844666, + 0.07767044007778168, + 0.12988285720348358, + 0.10878288745880127, + 0.15573063492774963, + 0.10951457917690277, + 0.06582474708557129, + 0.02477661892771721, + 0.07745532691478729, + 0.03348777815699577, + 0.10050909221172333, + 0.033770203590393066 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.5533217191696167, + 0.5758762359619141, + -0.6480545997619629, + -0.10892429947853088, + 0.7943102717399597, + -2.7399702072143555, + -1.8395006656646729, + 1.447904109954834, + 1.4495785236358643, + -0.055675506591796875, + -1.3537942171096802, + 0.38944748044013977, + -1.6196211576461792, + -1.2710431814193726, + -0.9744676947593689, + 0.057668983936309814, + 1.8350472450256348, + -1.2618876695632935, + -1.2385435104370117, + -1.7976703643798828, + -1.9491409063339233 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9072847366333008, + 0.06923278421163559, + -0.14729690551757812, + 0.15558937191963196, + 0.40412187576293945, + -2.342223644256592, + 0.0, + 0.2589508295059204, + 0.35182610154151917, + 0.10176032781600952, + -0.3489869236946106, + 0.13150231540203094, + -1.3123579025268555, + 0.0, + 0.10376164317131042, + 0.0909266471862793, + 0.1919306516647339, + 0.28928786516189575, + -0.4625590741634369, + -0.30475670099258423, + 0.0 + ], + [ + 1.9072847366333008, + 0.05373973771929741, + -1.1472969055175781, + -0.47107377648353577, + 0.3914180397987366, + -3.342223644256592, + -2.4980990886688232, + 1.2589508295059204, + 0.8455543518066406, + -0.7174267172813416, + -1.0694091320037842, + 0.33346641063690186, + -2.3123579025268555, + -1.1108808517456055, + -0.33852219581604004, + -0.7232105731964111, + 1.145590901374817, + -0.7107121348381042, + -0.5049349069595337, + -1.112295150756836, + -1.244770884513855 + ], + [ + 1.808228611946106, + -0.18174676597118378, + -1.2057973146438599, + -0.6115378737449646, + 0.11874309182167053, + -3.3499772548675537, + -2.3861043453216553, + 1.2540870904922485, + 1.1210845708847046, + -0.953934371471405, + -1.3015953302383423, + 0.6089286804199219, + -2.1655566692352295, + -0.8375923037528992, + -0.26793593168258667, + -0.998324990272522, + 1.2481704950332642, + -0.8320856690406799, + -0.376935750246048, + -1.0882012844085693, + -0.9857641458511353 + ], + [ + 2.291844606399536, + 0.05574151128530502, + -0.9760321974754333, + -0.5710926055908203, + 0.8478215336799622, + -3.22356915473938, + -2.591245651245117, + 1.2184754610061646, + 0.855300784111023, + -1.0267595052719116, + -1.3318730592727661, + 0.4197496771812439, + -2.0368778705596924, + -1.3685163259506226, + -0.582176685333252, + -0.8082455992698669, + 1.3435251712799072, + -0.44022125005722046, + -0.06775334477424622, + -0.7692400217056274, + -0.8863754272460938 + ] + ], + "predicted_scores": [ + 2.433502674102783, + 2.286835193634033, + 1.6041948795318604, + 0.9942966103553772, + 0.9331410527229309, + 0.6499817967414856, + -0.26208561658859253, + -1.5038988590240479 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train2", + "selected_index": 0, + "state_hash": "6802b2d1fb8a5afa54b7321e2bb938176de48d74f28e9b2a9c741c23547f0ef1", + "stored_base_utility": 0.07829316705465317, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.12791956961154938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12791956961154938 + }, + "base_progress": 0.12791956961154938, + "base_success": false, + "base_utility": 0.12791956961154938, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.034705858677625656, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.034705858677625656 + }, + { + "contact_quality": null, + "progress": 0.10312310606241226, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10312310606241226 + }, + { + "contact_quality": null, + "progress": 0.06007760763168335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06007760763168335 + }, + { + "contact_quality": null, + "progress": 0.07922307401895523, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07922307401895523 + }, + { + "contact_quality": null, + "progress": 0.0422377772629261, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0422377772629261 + }, + { + "contact_quality": null, + "progress": 0.06801291555166245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06801291555166245 + }, + { + "contact_quality": null, + "progress": 0.0445721335709095, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0445721335709095 + }, + { + "contact_quality": null, + "progress": 0.028874794021248817, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.028874794021248817 + } + ], + "candidate_progress": [ + 0.034705858677625656, + 0.10312310606241226, + 0.06007760763168335, + 0.07922307401895523, + 0.0422377772629261, + 0.06801291555166245, + 0.0445721335709095, + 0.028874794021248817 + ], + "candidate_source_chart_ids": [ + "ms-1ba37e30b90a3b28e62f", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-728b865677134d3788e3", + "generated_tangents": [ + [ + -1.0635509490966797, + -0.144166499376297, + 0.802090585231781, + -0.2630089819431305, + 0.42815130949020386, + -0.6543326377868652, + 0.8752983212471008, + 0.20988425612449646, + -0.0268863532692194, + 0.2100597620010376, + -0.7060477137565613, + 0.38852477073669434, + -0.16292265057563782, + 0.12568438053131104, + -1.2479095458984375, + 0.9765241742134094, + 1.52181875705719, + -0.2224280685186386, + -0.4899129867553711, + -0.5188641548156738, + 0.4392372965812683 + ], + [ + -0.8478046655654907, + -0.0586041659116745, + 0.553169846534729, + -0.13006587326526642, + 0.22915710508823395, + -0.8670809268951416, + -0.4086799621582031, + 0.5121471285820007, + -0.12022474408149719, + -0.1769028604030609, + -1.2220101356506348, + 0.8367915749549866, + -0.6423728466033936, + 0.6282032132148743, + -1.4395055770874023, + 0.9174424409866333, + 1.0084816217422485, + -0.7866349220275879, + -0.2790687680244446, + -0.2328687459230423, + 0.18835793435573578 + ], + [ + -1.079932451248169, + 0.02097637951374054, + 0.42843595147132874, + -0.06362952291965485, + 0.5173932909965515, + -1.1870826482772827, + -0.24006062746047974, + 0.6398480534553528, + -0.09183204174041748, + -0.22705382108688354, + -0.8776602745056152, + 1.1680071353912354, + -0.47220680117607117, + 0.8978961706161499, + -1.3510977029800415, + 0.6759516000747681, + 1.1731407642364502, + -0.6885026693344116, + 0.30227115750312805, + -0.28838708996772766, + 0.6158943176269531 + ], + [ + -0.8748584985733032, + -0.047565214335918427, + 0.7239023447036743, + -0.6460307836532593, + 0.46451473236083984, + -0.8489879369735718, + -0.4247792363166809, + 0.8628618717193604, + -0.0037170033901929855, + -0.08119147270917892, + -0.8511540293693542, + 0.5110795497894287, + -0.6595770120620728, + 0.7682916522026062, + -0.9892342686653137, + 0.7929928302764893, + 1.4224151372909546, + -0.6785783171653748, + -0.4034343361854553, + -0.4234212040901184, + 0.35307013988494873 + ], + [ + -1.0285038948059082, + -0.09682001918554306, + 0.5330713987350464, + -0.41781651973724365, + 0.594927191734314, + -0.7877529859542847, + -0.3395764231681824, + 0.7509548664093018, + -0.1473914235830307, + -0.10875154286623001, + -1.1968668699264526, + 0.8369057774543762, + -0.7651267647743225, + 0.8366407155990601, + -1.185888409614563, + 0.7464142441749573, + 1.060364007949829, + -0.8087634444236755, + -0.18946054577827454, + -0.5167063474655151, + 0.14220881462097168 + ], + [ + -0.9473214149475098, + -0.13021314144134521, + 0.5214983224868774, + -0.3412332534790039, + 0.4770852327346802, + -0.9695706367492676, + -0.3040820360183716, + 0.7984873056411743, + -0.14576475322246552, + -0.07656867057085037, + -1.1646051406860352, + 0.8167257308959961, + -0.8126498460769653, + 0.8298033475875854, + -1.285340428352356, + 0.7093576788902283, + 1.071486473083496, + -0.9461115002632141, + -0.1984189748764038, + -0.32209935784339905, + 0.33333319425582886 + ], + [ + -0.9799249172210693, + 0.09466944634914398, + 0.7084234952926636, + -0.5850423574447632, + 0.565342128276825, + -0.40946367383003235, + -0.6522653698921204, + 0.62880539894104, + 0.06187302619218826, + -0.0017405301332473755, + -0.925563395023346, + 0.8420876860618591, + -0.6774611473083496, + 0.8070276975631714, + -0.9017876982688904, + 0.7470946311950684, + 0.9803131222724915, + -0.7930443286895752, + -0.2944353222846985, + -0.5137248635292053, + 0.0745958611369133 + ], + [ + -1.2287569046020508, + -0.055298492312431335, + 0.5617462992668152, + -0.7970031499862671, + 0.9275137186050415, + -0.7675570249557495, + -0.7011186480522156, + 0.6776913404464722, + -0.2211901992559433, + -0.19782641530036926, + -1.2233498096466064, + 0.7510000467300415, + -0.41770225763320923, + 0.9504981637001038, + -1.2038949728012085, + 0.7601245045661926, + 1.2908552885055542, + -0.7631882429122925, + -0.4973359704017639, + -0.44955307245254517, + 0.174300417304039 + ] + ], + "generated_utilities": [ + 0.034705858677625656, + 0.10312310606241226, + 0.06007760763168335, + 0.07922307401895523, + 0.0422377772629261, + 0.06801291555166245, + 0.0445721335709095, + 0.028874794021248817 + ], + "group_id": "ms-728b865677134d3788e3", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05257302522659302, + 0.05559448525309563, + 0.11835493892431259, + 0.04238405078649521, + 0.16897141933441162, + 0.02001136541366577, + 0.1609041690826416, + 0.14248299598693848, + 0.14091376960277557, + 0.17822234332561493, + 0.22658847272396088, + 0.1736571490764618, + 0.18659374117851257, + 0.12753915786743164, + 0.07164359092712402, + 0.0063618822023272514 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04065108299255371, + 0.05943702906370163, + 0.0039327144622802734, + 0.10760968923568726, + -0.3465921878814697, + -0.6843758821487427, + 0.0, + 0.3034271001815796, + -0.38217779994010925, + 0.019724853336811066, + -1.1125431060791016, + -0.016437113285064697, + 0.09826856851577759, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + 0.055904269218444824, + -0.2721228003501892, + 0.07830148935317993, + 0.0 + ], + [ + -0.9075424671173096, + 0.06243014335632324, + 1.0039327144622803, + -0.6845037937164307, + 0.050797268748283386, + -1.6843758821487427, + -1.3340954780578613, + 1.3034271001815796, + -0.29618290066719055, + -0.035879287868738174, + -2.1125431060791016, + 0.9835628867149353, + -0.9017314314842224, + -1.01132333278656, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233 + ], + [ + -1.117501139640808, + 0.23503510653972626, + 0.9669153094291687, + -0.8317459225654602, + 0.06331060081720352, + -1.5474717617034912, + -1.3840656280517578, + 1.240980863571167, + -0.053158730268478394, + 0.10333722829818726, + -2.2084054946899414, + 1.1802411079406738, + -0.8987206816673279, + -1.136216402053833, + -1.9180536270141602, + 1.4563629627227783, + 2.625094413757324, + -0.1296394169330597, + 0.3340006470680237, + -0.7212131023406982, + 0.9384214878082275 + ], + [ + -1.2640053033828735, + 0.2415706366300583, + 0.9424604177474976, + -0.8472410440444946, + 0.35185110569000244, + -1.8022713661193848, + -1.2391949892044067, + 0.9333417415618896, + 0.03309231996536255, + 0.22933605313301086, + -1.7521884441375732, + 1.0360857248306274, + -0.6365935802459717, + -0.7614871859550476, + -2.103987216949463, + 1.5381183624267578, + 2.11069917678833, + -0.5904175639152527, + -0.07874824106693268, + -0.3884509205818176, + 1.0602728128433228 + ], + [ + -0.7705078721046448, + 0.18350359797477722, + 0.579454779624939, + -0.5434049963951111, + 0.2274666726589203, + -1.7208446264266968, + -1.5971848964691162, + 1.6145182847976685, + -0.6915329694747925, + 0.20122811198234558, + -1.8584421873092651, + 0.9026989340782166, + -0.5741626024246216, + -0.5299967527389526, + -1.544856309890747, + 1.7236231565475464, + 2.1288580894470215, + -0.7592856287956238, + 0.3358893394470215, + -0.2929669916629791, + 1.67848801612854 + ], + [ + -0.3686429262161255, + 0.3043383061885834, + 0.7294905781745911, + -0.8736650347709656, + 0.3122289776802063, + -1.5051836967468262, + -1.8593363761901855, + 0.8389579653739929, + -0.8785254955291748, + -0.5520821213722229, + -1.8517990112304688, + 1.3417646884918213, + -0.32390326261520386, + -0.42145586013793945, + -1.6530441045761108, + 1.7202519178390503, + 2.180105209350586, + -0.4262692928314209, + -0.1161729246377945, + -0.08550512790679932, + 1.1130309104919434 + ], + [ + -0.9971213936805725, + -0.4085577130317688, + 0.9404422044754028, + -0.9977414011955261, + 0.4963066577911377, + -1.4445728063583374, + -1.8557989597320557, + 1.3813822269439697, + -0.9350934028625488, + 0.6622016429901123, + -2.0982701778411865, + 0.5712680220603943, + -1.3476169109344482, + -0.8056140542030334, + -2.2595674991607666, + 1.7601569890975952, + 2.5158329010009766, + -0.8607662916183472, + -0.010314732789993286, + -0.9802933931350708, + 0.587103545665741 + ], + [ + -0.538765549659729, + -0.2632336914539337, + 0.574018120765686, + -1.3315669298171997, + 0.23096206784248352, + -1.4029772281646729, + -1.2746275663375854, + 1.674896478652954, + -0.6276504993438721, + -0.06763898581266403, + -1.44939124584198, + 1.1256837844848633, + -0.6540690064430237, + -0.7850072979927063, + -2.472240686416626, + 1.201231837272644, + 2.056978702545166, + 0.4725829064846039, + 0.7320148944854736, + -0.47291314601898193, + 1.2244172096252441 + ], + [ + -0.5474913120269775, + 0.3537777066230774, + 0.1717698574066162, + -0.9464929103851318, + -0.24061335623264313, + -2.5145106315612793, + -1.9605224132537842, + 2.171875476837158, + 0.5462924242019653, + 0.2324821650981903, + -2.4616305828094482, + 1.3455290794372559, + -0.703314483165741, + -0.6289631128311157, + -1.848944902420044, + 1.755210518836975, + 2.649005174636841, + -0.931267499923706, + -0.3109248876571655, + 0.21283608675003052, + 0.6994417309761047 + ], + [ + -1.4273285865783691, + 0.48591291904449463, + 1.4724129438400269, + -1.1116547584533691, + -0.04224221408367157, + -1.1567587852478027, + -1.0686161518096924, + 1.7429871559143066, + -1.1814582347869873, + 0.07730802893638611, + -2.1004433631896973, + 0.45253968238830566, + -0.7293854355812073, + -0.4477439522743225, + -1.986203908920288, + 1.1158549785614014, + 2.0316240787506104, + 0.5763667821884155, + -0.48857587575912476, + -1.07709801197052, + 0.5611822605133057 + ] + ], + "predicted_scores": [ + 1.380915641784668, + 0.013786926865577698, + -0.30374717712402344, + -0.4565553069114685, + -0.6123114228248596, + -0.6671327352523804, + -1.0171722173690796, + -1.4761658906936646 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296/train2", + "selected_index": 0, + "state_hash": "62e00728e448102460275bc2c35150dbf429ab0d5670a6706fa41256273bac3c", + "stored_base_utility": 0.05257302522659302, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20529556274414062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20529556274414062 + }, + "base_progress": 0.20529556274414062, + "base_success": false, + "base_utility": 0.20529556274414062, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.14004097878932953, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14004097878932953 + }, + { + "contact_quality": null, + "progress": 0.1419321894645691, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1419321894645691 + }, + { + "contact_quality": null, + "progress": 0.12569941580295563, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12569941580295563 + }, + { + "contact_quality": null, + "progress": 0.19780242443084717, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19780242443084717 + }, + { + "contact_quality": null, + "progress": 0.19119806587696075, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19119806587696075 + }, + { + "contact_quality": null, + "progress": 0.09375361353158951, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09375361353158951 + }, + { + "contact_quality": null, + "progress": 0.07512229681015015, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07512229681015015 + }, + { + "contact_quality": null, + "progress": 0.05006565526127815, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05006565526127815 + } + ], + "candidate_progress": [ + 0.14004097878932953, + 0.1419321894645691, + 0.12569941580295563, + 0.19780242443084717, + 0.19119806587696075, + 0.09375361353158951, + 0.07512229681015015, + 0.05006565526127815 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-6ddecc7f074b9e267409" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "generated_tangents": [ + [ + 0.10673574358224869, + -0.14959247410297394, + -0.12463840842247009, + -0.37258410453796387, + -0.05057141184806824, + -2.4426519870758057, + -0.9322618842124939, + -0.7199194431304932, + -0.041174888610839844, + 0.6258273720741272, + 0.6073430776596069, + -0.09425163269042969, + -0.4072318971157074, + 0.28364187479019165, + 0.015309281647205353, + -0.19833293557167053, + -0.35566574335098267, + 0.3385075330734253, + -0.6021514534950256, + -0.8059977889060974, + 0.08888347446918488 + ], + [ + 0.20093709230422974, + -0.08732785284519196, + -0.07632574439048767, + -0.3199525475502014, + -0.15303108096122742, + -2.4377336502075195, + -1.001490831375122, + -0.7903403043746948, + -0.04812772572040558, + 0.6852710247039795, + 0.5423687696456909, + -0.0874888002872467, + -0.49707430601119995, + 0.2025846242904663, + -0.013522166758775711, + -0.1688818633556366, + -0.32763415575027466, + 0.36862581968307495, + -0.6373065710067749, + -0.7562669515609741, + -0.18745648860931396 + ], + [ + 0.22022023797035217, + -0.06436137109994888, + -0.053831905126571655, + -0.14377784729003906, + 0.011305093765258789, + -2.408085346221924, + -0.7647324204444885, + -0.8291052579879761, + -0.10818691551685333, + 0.7213743329048157, + 0.6900634765625, + -0.2628686726093292, + -0.41571927070617676, + 0.12526550889015198, + -0.062162432819604874, + -0.13022896647453308, + -0.3306620121002197, + 0.28726160526275635, + -0.4696381092071533, + -0.6505988240242004, + -0.2831553518772125 + ], + [ + 0.1806195080280304, + -0.12329690158367157, + -0.2678345739841461, + -0.0991961658000946, + -0.11933040618896484, + -2.370558023452759, + -1.0490424633026123, + -0.7236278057098389, + -0.03582095354795456, + 0.7128385901451111, + 0.2698477506637573, + 0.031962111592292786, + -0.6503012776374817, + 0.1851949691772461, + 0.036946482956409454, + -0.09476342797279358, + -0.48858946561813354, + 0.23446667194366455, + -0.6592259407043457, + -1.1064987182617188, + -0.2510225772857666 + ], + [ + 0.4808974862098694, + -0.14640262722969055, + 0.30818068981170654, + -0.7484868764877319, + -0.08179956674575806, + -2.536876916885376, + -1.3846843242645264, + -0.3886636793613434, + -0.09180399030447006, + 0.3166411817073822, + 0.2011575996875763, + -0.1790960431098938, + -0.826603889465332, + 0.3920462727546692, + 0.03207596391439438, + -0.10321949422359467, + -0.39817750453948975, + 0.41908925771713257, + -0.7173893451690674, + -0.977794349193573, + -0.27252867817878723 + ], + [ + 0.2641885280609131, + -0.09089608490467072, + 0.36994171142578125, + -0.2175145447254181, + -0.31745749711990356, + -2.5401532649993896, + -0.7345007061958313, + -0.6528854370117188, + -0.023025771602988243, + 0.8325459957122803, + 0.5940406322479248, + 0.21302993595600128, + -0.2803461253643036, + -0.42173969745635986, + -0.2120581865310669, + -0.005919860675930977, + -0.2557860016822815, + 0.5652105808258057, + -0.603173553943634, + -0.7599316835403442, + -0.4067913591861725 + ], + [ + 0.2427782416343689, + -0.04562993347644806, + -0.22766199707984924, + 0.03796195983886719, + 0.3980022370815277, + -2.0137979984283447, + -0.9746316075325012, + -0.7912865877151489, + -0.218749538064003, + 0.814417839050293, + 0.8569574356079102, + 0.25053051114082336, + -0.2706298232078552, + 0.26941999793052673, + -0.021201658993959427, + -0.24368491768836975, + -0.09445913136005402, + 0.6419159770011902, + -0.6443300843238831, + -0.5719162225723267, + -0.09823915362358093 + ], + [ + 0.347103476524353, + -0.11219358444213867, + -0.23887714743614197, + -0.2464659959077835, + 0.11277837306261063, + -2.269803524017334, + -0.8880805373191833, + -1.2211774587631226, + 0.053966037929058075, + 1.1723911762237549, + 0.7669907808303833, + 0.04349309206008911, + -0.39687860012054443, + -0.07752034068107605, + -0.18468192219734192, + -0.1400359719991684, + -0.10476179420948029, + 0.2363874614238739, + -0.7008696794509888, + -0.6770836114883423, + 0.04924654960632324 + ] + ], + "generated_utilities": [ + 0.14004097878932953, + 0.1419321894645691, + 0.12569941580295563, + 0.19780242443084717, + 0.19119806587696075, + 0.09375361353158951, + 0.07512229681015015, + 0.05006565526127815 + ], + "group_id": "ms-7ea98b8f4d2220347ee0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18600693345069885, + 0.07475636899471283, + 0.27452313899993896, + 0.20920448005199432, + 0.18411576747894287, + 0.27989473938941956, + 0.19416475296020508, + 0.10948207229375839, + 0.123555026948452, + 0.1468026340007782, + 0.12693151831626892, + 0.11466284841299057, + 0.07585643231868744, + 0.12908481061458588, + 0.22501108050346375, + 0.06431269645690918 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.06974434852600098, + -0.02351950854063034, + 0.04567508399486542, + -0.07042121887207031, + -0.038941264152526855, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.08617860078811646, + 0.9112892150878906, + 0.7374391555786133, + 0.06539928913116455, + 0.03787863254547119, + 0.0, + -0.021528124809265137, + -0.04566385596990585, + -0.03331831097602844, + 1.6466763019561768, + 0.04444807767868042, + -0.27494049072265625, + 0.0 + ], + [ + 1.1160619258880615, + -0.10018755495548248, + 0.19327251613140106, + -1.3990073204040527, + 0.672177255153656, + -3.303851842880249, + -2.832740545272827, + -1.902563214302063, + 0.39503830671310425, + 1.542332410812378, + 1.6850805282592773, + 0.6842766404151917, + 1.1400834321975708, + 1.670598030090332, + 0.631821870803833, + -0.26601675152778625, + -0.2725638151168823, + 3.003178119659424, + 0.23009130358695984, + -1.0388810634613037, + -0.19119825959205627 + ], + [ + 1.031497836112976, + -0.2943698763847351, + -0.11846622079610825, + -1.3361037969589233, + 0.6225074529647827, + -3.545761823654175, + -2.5078811645507812, + -1.4024183750152588, + 0.6736462116241455, + 2.093475580215454, + 1.9168713092803955, + 0.6377919316291809, + 0.5105260014533997, + 1.8172619342803955, + 1.1300766468048096, + 0.1616329848766327, + 0.03408849239349365, + 2.2435190677642822, + 0.735376238822937, + -1.5760226249694824, + -0.7514586448669434 + ], + [ + 1.7542359828948975, + -0.7351009249687195, + -0.854544997215271, + -1.223644495010376, + 0.07327532768249512, + -3.150663137435913, + -2.5486767292022705, + -2.110039472579956, + 1.0206904411315918, + 2.1492340564727783, + 2.187993288040161, + 1.2547765970230103, + 0.8607165217399597, + 1.1150517463684082, + 0.9828771948814392, + 0.5030426383018494, + -0.4691219627857208, + 2.2485060691833496, + -0.14985057711601257, + -1.9480454921722412, + -0.05157876014709473 + ], + [ + 0.6040669679641724, + 0.5665496587753296, + 0.24913550913333893, + -1.6600215435028076, + -0.10813313722610474, + -3.8031911849975586, + -3.226392984390259, + -1.5098689794540405, + 1.0684192180633545, + 2.1557064056396484, + 1.545849084854126, + 1.011389970779419, + 1.113816499710083, + 0.9937302470207214, + 1.372757911682129, + -0.4055754244327545, + -0.23403912782669067, + 3.0650274753570557, + 0.1330145299434662, + -0.7907523512840271, + -0.6428619623184204 + ], + [ + 0.30788952112197876, + 0.6807127594947815, + -0.035938769578933716, + -0.8788743615150452, + -0.30191969871520996, + -4.111589431762695, + -2.432647705078125, + -1.7479791641235352, + -0.321926474571228, + 1.9823087453842163, + 1.6617395877838135, + 1.3319180011749268, + 1.5937504768371582, + 2.0502309799194336, + 0.63725745677948, + 0.7232187390327454, + -0.23885302245616913, + 1.8472312688827515, + 0.3300834596157074, + -0.43588966131210327, + -1.2869892120361328 + ], + [ + 0.3636443018913269, + -0.4284004867076874, + 0.2675324082374573, + -0.4861376881599426, + 0.2239738255739212, + -2.813422918319702, + -2.4891929626464844, + -1.165071725845337, + 0.6442201733589172, + 1.1830427646636963, + 1.177597999572754, + 0.33577069640159607, + 0.6362329721450806, + 1.5409071445465088, + 0.29718804359436035, + -0.5603961944580078, + -0.8089720010757446, + 2.6271419525146484, + -0.5456377267837524, + -0.2824941873550415, + -0.7960987091064453 + ], + [ + 1.1001086235046387, + -0.9762114882469177, + -0.00277864933013916, + -1.704659104347229, + 1.2373628616333008, + -3.385840654373169, + -1.6910454034805298, + -1.470743179321289, + 0.9354362487792969, + 1.9378679990768433, + 1.515415906906128, + -0.3002503514289856, + 0.21796274185180664, + 1.542279839515686, + 1.7353785037994385, + 0.23501873016357422, + -0.16601301729679108, + 2.1989498138427734, + 0.8671687841415405, + -1.6252647638320923, + -1.2141560316085815 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06974434852600098, + -0.15391214191913605, + -0.22159501910209656, + -0.07042121887207031, + -0.09706336259841919, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.1400253176689148, + 0.9112892150878906, + 0.7374391555786133, + 0.13415807485580444, + -0.10778343677520752, + 0.0, + -0.08470147848129272, + 0.1231045201420784, + 0.08352774381637573, + 1.6466763019561768, + -0.005736887454986572, + -0.27494049072265625, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.125826835632324, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.152101516723633, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.8942345976829529 + ] + ], + "predicted_scores": [ + 5.43532133102417, + 5.334425449371338, + 5.021456718444824, + 5.008562088012695, + 4.660686016082764, + 4.276692867279053, + 3.5957794189453125, + 3.458228588104248 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train2", + "selected_index": 0, + "state_hash": "a2b9f3b3ea6458c6734081cea861cf54026796a0a594bf9be3657d28f12e9b3c", + "stored_base_utility": 0.18600693345069885, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18213583528995514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18213583528995514 + }, + "base_progress": 0.18213583528995514, + "base_success": false, + "base_utility": 0.18213583528995514, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.002675553085282445, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.002675553085282445 + }, + { + "contact_quality": null, + "progress": 0.0032987594604492188, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0032987594604492188 + }, + { + "contact_quality": null, + "progress": 0.0033065874595195055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0033065874595195055 + }, + { + "contact_quality": null, + "progress": 0.0031580727081745863, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0031580727081745863 + }, + { + "contact_quality": null, + "progress": 0.00375221180729568, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00375221180729568 + }, + { + "contact_quality": null, + "progress": 0.004366755485534668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004366755485534668 + }, + { + "contact_quality": null, + "progress": 0.004318694118410349, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004318694118410349 + }, + { + "contact_quality": null, + "progress": 0.0034571688156574965, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0034571688156574965 + } + ], + "candidate_progress": [ + 0.002675553085282445, + 0.0032987594604492188, + 0.0033065874595195055, + 0.0031580727081745863, + 0.00375221180729568, + 0.004366755485534668, + 0.004318694118410349, + 0.0034571688156574965 + ], + "candidate_source_chart_ids": [ + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-f2b2b2a1aa2863c3676f", + "ms-02311c7783bc67e188b4", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7f57045c114ddad153b5", + "generated_tangents": [ + [ + -0.39032959938049316, + -0.508104145526886, + 1.5406129360198975, + -0.4155612587928772, + 1.9475791454315186, + 3.158402681350708, + 1.9311771392822266, + 0.34415990114212036, + 0.07478859275579453, + 0.9082386493682861, + 0.5083922743797302, + 1.3319385051727295, + 1.284725546836853, + 2.5255703926086426, + -0.13781119883060455, + -0.1745506078004837, + 1.0435887575149536, + -0.3424244523048401, + 1.1724116802215576, + 1.4357433319091797, + 1.6907014846801758 + ], + [ + 0.1280473917722702, + -0.5491660833358765, + 1.4465504884719849, + -0.41509976983070374, + 1.7438325881958008, + 3.0281577110290527, + 1.7826979160308838, + 0.4565412700176239, + 0.16918210685253143, + 0.9650174379348755, + 0.3661339282989502, + 1.0957837104797363, + 1.8078123331069946, + 2.0525758266448975, + -0.269488662481308, + -0.16108833253383636, + 1.2963379621505737, + -0.1321779042482376, + 1.051758885383606, + 1.5588188171386719, + 1.7583426237106323 + ], + [ + 0.08456126600503922, + -0.5221831798553467, + 1.2618119716644287, + -0.40739238262176514, + 1.727027416229248, + 3.1535515785217285, + 1.8114442825317383, + 0.527355432510376, + 0.1626051664352417, + 1.0738208293914795, + 0.546379566192627, + 1.0603299140930176, + 1.7653990983963013, + 1.6606471538543701, + -0.12099452316761017, + -0.21441476047039032, + 1.4798492193222046, + -0.15774168074131012, + 1.2178139686584473, + 1.6470692157745361, + 1.750726342201233 + ], + [ + -0.06606529653072357, + -0.6003207564353943, + 1.4119372367858887, + -0.36464112997055054, + 1.7714884281158447, + 2.7030832767486572, + 1.7662653923034668, + 0.4509817659854889, + 0.10357790440320969, + 0.8971320986747742, + 0.442639023065567, + 1.3400689363479614, + 1.4598724842071533, + 2.1251139640808105, + -0.09128360450267792, + 0.008713070303201675, + 1.4488658905029297, + -0.06104664504528046, + 1.2843379974365234, + 1.388801097869873, + 2.0064051151275635 + ], + [ + -0.12615616619586945, + -0.6678372621536255, + 1.198927879333496, + -0.37412479519844055, + 1.5052831172943115, + 2.952237129211426, + 1.7415893077850342, + 0.46397364139556885, + 0.11576055735349655, + 0.9424229264259338, + 0.23245862126350403, + 1.0981528759002686, + 1.8414663076400757, + 1.9273345470428467, + -0.24184110760688782, + -0.032975923269987106, + 1.1227922439575195, + -0.34808146953582764, + 1.25648832321167, + 1.3888342380523682, + 1.6293386220932007 + ], + [ + 0.1342376172542572, + -0.48477381467819214, + 1.4279762506484985, + -0.3248389661312103, + 1.0237741470336914, + 2.874401330947876, + 1.9266736507415771, + 0.7250481843948364, + 0.20822665095329285, + 0.9497660398483276, + 0.49012768268585205, + 0.8773930668830872, + 2.1183817386627197, + 1.8961174488067627, + -0.31126612424850464, + -0.08741752803325653, + 1.3766226768493652, + -0.1429089456796646, + 1.5382955074310303, + 1.7073025703430176, + 0.9260591268539429 + ], + [ + -0.14159391820430756, + -0.6119739413261414, + 1.0168184041976929, + -0.47098153829574585, + 1.4086096286773682, + 3.124621629714966, + 1.6730246543884277, + 0.46711400151252747, + 0.06346733868122101, + 0.9438840746879578, + 0.0648728460073471, + 1.0331757068634033, + 1.8586443662643433, + 1.8924436569213867, + -0.2553775906562805, + -0.02012600749731064, + 1.0731415748596191, + -0.23178528249263763, + 1.2510342597961426, + 1.3793447017669678, + 1.6255412101745605 + ], + [ + -0.17344127595424652, + -0.5800595879554749, + 1.0790690183639526, + -0.2571268081665039, + 1.5217797756195068, + 2.6276838779449463, + 1.5147156715393066, + 0.3386792242527008, + 0.056403473019599915, + 0.9924086928367615, + 0.21902933716773987, + 0.8074168562889099, + 1.6257332563400269, + 1.9361677169799805, + -0.17142485082149506, + -0.02326297201216221, + 1.1998100280761719, + 0.017527207732200623, + 1.040473461151123, + 1.4633429050445557, + 1.76557457447052 + ] + ], + "generated_utilities": [ + 0.002675553085282445, + 0.0032987594604492188, + 0.0033065874595195055, + 0.0031580727081745863, + 0.00375221180729568, + 0.004366755485534668, + 0.004318694118410349, + 0.0034571688156574965 + ], + "group_id": "ms-7f57045c114ddad153b5", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.11277659982442856, + 0.1479199379682541, + 0.07541074603796005, + 0.042439721524715424, + 0.15517297387123108, + 0.03584214299917221, + 0.1660110056400299, + 0.12549421191215515, + 0.227622389793396, + 0.09924932569265366, + 0.18198737502098083, + 0.11130931228399277, + 0.10854844748973846, + 0.01617596670985222, + 0.08437560498714447, + 0.04182587191462517 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.013166621327400208, + -0.12399804592132568, + 0.48011910915374756, + 0.4925293028354645, + 0.6256281137466431, + 2.213102102279663, + 0.0, + -0.4044480323791504, + 0.1722363382577896, + 0.3389366865158081, + -0.08761294186115265, + 0.21791136264801025, + 0.9396321773529053, + 0.0, + 0.06717276573181152, + -0.5456919074058533, + 0.722014307975769, + -0.051606684923172, + 0.7784175276756287, + 0.5338386297225952, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.9143741130828857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.455221176147461, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.263336420059204 + ], + [ + -0.47406601905822754, + -0.37954479455947876, + 1.123671293258667, + -0.8903511762619019, + 1.5515481233596802, + 3.7488584518432617, + 0.9699987173080444, + -0.23581115901470184, + -0.2149372398853302, + 0.8883978128433228, + 0.5803861021995544, + 1.7047908306121826, + 2.149672746658325, + 2.2741973400115967, + 0.2513730227947235, + 0.6982267498970032, + 2.1354174613952637, + -1.151021122932434, + 1.9550246000289917, + 0.5981679558753967, + 1.338004469871521 + ], + [ + -0.1836935132741928, + -0.42300617694854736, + 1.7664438486099243, + -0.5576877593994141, + 1.0626983642578125, + 2.5276265144348145, + 2.3329849243164062, + -0.4834023118019104, + -0.25722599029541016, + 0.5209662914276123, + 0.24240870773792267, + 0.8533134460449219, + 1.5531291961669922, + 2.304234504699707, + 0.7609332799911499, + 1.0344312191009521, + 2.270921230316162, + -0.926997184753418, + 0.9014018177986145, + 2.259265422821045, + 2.172292470932007 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3910996615886688, + -0.9467402100563049, + 1.4744372367858887, + -0.2748112082481384, + 1.753147840499878, + 3.3321292400360107, + 1.7170560359954834, + 0.09920699894428253, + -0.11250317096710205, + 1.5213767290115356, + 0.0891318991780281, + 1.0193004608154297, + 1.7396382093429565, + 2.202378988265991, + 0.1176290512084961, + 0.5433868765830994, + 1.6113250255584717, + -0.2701912522315979, + 1.7789868116378784, + 1.246854305267334, + 1.6763029098510742 + ], + [ + 0.3669140636920929, + -0.8853410482406616, + 1.1054010391235352, + -0.008406088687479496, + 1.683157205581665, + 2.8090970516204834, + 2.1524839401245117, + -0.1140800267457962, + -0.16068093478679657, + 1.3338439464569092, + -0.34659242630004883, + 1.4494925737380981, + 2.0485284328460693, + 2.244007110595703, + 0.4862309694290161, + 0.6211890578269958, + 1.5708913803100586, + -0.22552455961704254, + 1.7964838743209839, + 1.386763095855713, + 1.3543281555175781 + ], + [ + 0.7718826532363892, + -0.5710152983665466, + 2.151604175567627, + -0.5648711323738098, + 2.1273765563964844, + 3.657623291015625, + 1.9845168590545654, + -0.1599704772233963, + -0.6097310781478882, + 1.135812520980835, + -0.09587362408638, + 1.555975079536438, + 2.3309824466705322, + 1.9832960367202759, + 0.46482378244400024, + 0.6467846035957336, + 1.7182108163833618, + -0.0750286877155304, + 1.9485927820205688, + 0.959426760673523, + 0.9474185109138489 + ] + ], + "predicted_scores": [ + 4.125182151794434, + 3.808912754058838, + 3.6503946781158447, + 3.3113675117492676, + 3.2645554542541504, + 3.13165020942688, + 2.9304094314575195, + 2.3975682258605957 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524/train2", + "selected_index": 0, + "state_hash": "4885f9acf2601b3d2c201bc8e00ef617845ad2a6d08f96d585249345f457be89", + "stored_base_utility": 0.11277659982442856, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0056487321853637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056487321853637695 + }, + "base_progress": 0.0056487321853637695, + "base_success": false, + "base_utility": 0.0056487321853637695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.011842290870845318, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011842290870845318 + }, + { + "contact_quality": null, + "progress": 0.01331732701510191, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01331732701510191 + }, + { + "contact_quality": null, + "progress": 0.007131179329007864, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007131179329007864 + }, + { + "contact_quality": null, + "progress": 0.007257858756929636, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007257858756929636 + }, + { + "contact_quality": null, + "progress": 0.00787546206265688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00787546206265688 + }, + { + "contact_quality": null, + "progress": 0.008778492920100689, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008778492920100689 + }, + { + "contact_quality": null, + "progress": 0.004853228572756052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004853228572756052 + }, + { + "contact_quality": null, + "progress": 0.003935039043426514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003935039043426514 + } + ], + "candidate_progress": [ + 0.011842290870845318, + 0.01331732701510191, + 0.007131179329007864, + 0.007257858756929636, + 0.00787546206265688, + 0.008778492920100689, + 0.004853228572756052, + 0.003935039043426514 + ], + "candidate_source_chart_ids": [ + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "generated_tangents": [ + [ + -0.12132076919078827, + -0.2878642678260803, + 1.0866169929504395, + -0.4333801865577698, + 1.3068206310272217, + 2.5597479343414307, + 1.9493379592895508, + -0.0816757082939148, + -0.505718469619751, + 0.20882830023765564, + -0.3755462169647217, + -0.05416959524154663, + 1.8078550100326538, + 0.6881452798843384, + 1.1576682329177856, + -0.7253053188323975, + -1.2206331491470337, + 1.11386239528656, + 0.8785176873207092, + -0.0019288063049316406, + -1.526033639907837 + ], + [ + -0.6971748471260071, + -0.16849307715892792, + 0.8937058448791504, + -0.6516115665435791, + 1.1871507167816162, + 3.069403648376465, + 2.0086045265197754, + 0.004958391189575195, + -0.38325098156929016, + -0.06779725104570389, + -0.28374800086021423, + 0.0900740697979927, + 1.780896544456482, + 0.854914665222168, + 1.1015669107437134, + -0.7410902380943298, + -1.6288267374038696, + 0.8072232007980347, + 1.0519955158233643, + -0.6125564575195312, + -0.911697506904602 + ], + [ + -0.3046749234199524, + -0.40183591842651367, + 0.9257275462150574, + -0.8784635066986084, + 1.2770981788635254, + 3.3158912658691406, + 2.2451343536376953, + 0.3959258794784546, + -0.26736778020858765, + 0.06612849235534668, + -0.3355943560600281, + 0.23737362027168274, + 1.8543399572372437, + 0.6302992105484009, + 1.222571849822998, + -0.7062807083129883, + -1.4489601850509644, + 0.8391095399856567, + 0.8286699652671814, + -0.21063582599163055, + -0.9838241338729858 + ], + [ + -0.48727017641067505, + -0.27816566824913025, + 0.897692859172821, + -0.6824548840522766, + 1.1272952556610107, + 3.0164794921875, + 2.324998378753662, + 0.1511741429567337, + -0.2782328724861145, + 0.03229042887687683, + -0.5024949312210083, + 0.5358070135116577, + 1.8460942506790161, + 0.8555551767349243, + 1.201119303703308, + -0.7147851586341858, + -1.1396129131317139, + 0.7666560411453247, + 0.9762827157974243, + -0.0225905179977417, + -1.1771522760391235 + ], + [ + -0.6487288475036621, + -0.2438424974679947, + 1.4727966785430908, + -0.48905354738235474, + 0.5148495435714722, + 2.9174039363861084, + 1.5262482166290283, + 0.10794110596179962, + -0.351114958524704, + 0.47189971804618835, + -0.6558436155319214, + 0.661037027835846, + 1.795812964439392, + 0.45406192541122437, + 0.8320794701576233, + -0.7480214834213257, + -1.1061779260635376, + 0.7629175186157227, + 0.5486004948616028, + -0.443786084651947, + -0.8721569776535034 + ], + [ + -0.6348989605903625, + -0.21784470975399017, + 1.2891793251037598, + -0.4872874319553375, + 0.6194880604743958, + 2.8650362491607666, + 1.8282687664031982, + 0.12518739700317383, + -0.4802975058555603, + 0.18316322565078735, + -0.2056957185268402, + 0.45549896359443665, + 1.0011792182922363, + 0.20390743017196655, + 0.8425129055976868, + -0.8140647411346436, + -0.958462655544281, + 0.7885424494743347, + 0.5988937616348267, + -0.2641720473766327, + -0.8488433361053467 + ], + [ + -0.5075993537902832, + -0.20654913783073425, + 1.2304483652114868, + -0.4762396812438965, + 1.3018770217895508, + 3.0701406002044678, + 2.494612455368042, + 0.3038874864578247, + -0.5493683815002441, + 0.1495422124862671, + -0.33732831478118896, + 0.7177326679229736, + 0.8174186944961548, + 0.5500130653381348, + 1.3581563234329224, + -0.7364436984062195, + -0.8602816462516785, + 0.9882403016090393, + 0.9251771569252014, + -0.3310457170009613, + -1.1166590452194214 + ], + [ + -0.464261531829834, + -0.23492270708084106, + 1.3735148906707764, + -0.43437132239341736, + 1.0023789405822754, + 2.9266200065612793, + 2.208437442779541, + 0.33880719542503357, + -0.5210384130477905, + 0.4060251712799072, + -0.4580993950366974, + 0.5641075372695923, + 1.1459509134292603, + 0.366401731967926, + 1.2579251527786255, + -0.6666497588157654, + -0.7129340171813965, + 0.9066519141197205, + 0.9991755485534668, + -0.46294230222702026, + -0.8339345455169678 + ] + ], + "generated_utilities": [ + 0.011842290870845318, + 0.01331732701510191, + 0.007131179329007864, + 0.007257858756929636, + 0.00787546206265688, + 0.008778492920100689, + 0.004853228572756052, + 0.003935039043426514 + ], + "group_id": "ms-81f4e8858b2ba90aea2e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.01361372135579586, + 0.007307033054530621, + 0.021326780319213867, + 0.04093001410365105, + 0.15059307217597961, + 0.013598322868347168, + 0.09240063279867172, + 0.1631626933813095, + 0.05027441307902336, + 0.05712141841650009, + 0.1538398563861847, + 0.06023097038269043, + 0.020811915397644043, + 0.019836824387311935, + 0.1318017691373825, + 0.051341354846954346 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06500570476055145, + 0.046472612768411636, + 1.2439939975738525, + -1.446747064590454, + 1.7077040672302246, + 2.291205883026123, + 2.1016674041748047, + -0.6354803442955017, + -0.4465000331401825, + 0.9666666984558105, + -1.1514010429382324, + 0.45639365911483765, + 1.613925814628601, + 1.0568119287490845, + 1.3792463541030884, + -0.7852597832679749, + -1.576931118965149, + 1.1258286237716675, + 0.9614027738571167, + -0.9620904922485352, + -1.7528356313705444 + ], + [ + -0.23400194942951202, + -0.21453839540481567, + 1.458961844444275, + -1.701543927192688, + 1.4158955812454224, + 2.2648162841796875, + 2.2270467281341553, + -0.5239131450653076, + -0.44378146529197693, + 1.0904332399368286, + -1.223331093788147, + 0.45784294605255127, + 1.6057316064834595, + 0.9251953363418579, + 1.308751106262207, + -0.6351475715637207, + -1.5950077772140503, + 1.0428481101989746, + 0.6781296730041504, + -0.9007717370986938, + -1.880659818649292 + ], + [ + 0.2324782907962799, + 0.05129373073577881, + 1.1239540576934814, + -1.1853233575820923, + 1.3290393352508545, + 1.933064341545105, + 2.192976951599121, + -0.23718085885047913, + -0.279358446598053, + 1.3028839826583862, + -1.4676742553710938, + 0.16475489735603333, + 1.67984938621521, + 0.6600161790847778, + 1.5654886960983276, + -1.1545368432998657, + -1.5593464374542236, + 1.1976646184921265, + 0.5715436935424805, + -0.9047394394874573, + -1.9614591598510742 + ], + [ + 0.7452290058135986, + 0.614987313747406, + 0.7383754849433899, + -1.5856380462646484, + 1.264894723892212, + 1.7337512969970703, + 2.502504825592041, + 0.040616750717163086, + -1.1382783651351929, + 0.7095704674720764, + -1.014717698097229, + 0.5126742720603943, + 1.3401073217391968, + 1.4275505542755127, + 1.4739655256271362, + -0.7308655381202698, + -0.9416123032569885, + 0.661049485206604, + 0.9453439712524414, + -1.5968079566955566, + -1.715583324432373 + ], + [ + 0.15219944715499878, + 0.5165310502052307, + 1.0654370784759521, + -0.5756344199180603, + 1.2834129333496094, + 1.6370232105255127, + 2.414426326751709, + -1.1256691217422485, + -1.1171015501022339, + 0.5337101817131042, + -0.9850769639015198, + -0.35428309440612793, + 2.4350547790527344, + 1.8563504219055176, + 2.1929399967193604, + -0.1821107268333435, + -1.4600040912628174, + 1.5430631637573242, + 1.4937412738800049, + -0.8152608275413513, + -2.020042896270752 + ] + ], + "predicted_scores": [ + 10.417702674865723, + 10.037516593933105, + 8.708206176757812, + 8.181203842163086, + 7.666223526000977, + 7.613871097564697, + 6.922053813934326, + 6.808025360107422 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train2", + "selected_index": 0, + "state_hash": "562e8bc3804d1d310e0eaf612dbc110b8efc94514458dacf551ef33f4f0b8ee7", + "stored_base_utility": 0.01361372135579586, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.00698717450723052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00698717450723052 + }, + "base_progress": 0.00698717450723052, + "base_success": false, + "base_utility": 0.00698717450723052, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.029071787372231483, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029071787372231483 + }, + { + "contact_quality": null, + "progress": 0.06671091169118881, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06671091169118881 + }, + { + "contact_quality": null, + "progress": 0.005293230060487986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005293230060487986 + }, + { + "contact_quality": null, + "progress": 0.02160499431192875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02160499431192875 + }, + { + "contact_quality": null, + "progress": 0.044261831790208817, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044261831790208817 + }, + { + "contact_quality": null, + "progress": 0.07134183496236801, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07134183496236801 + }, + { + "contact_quality": null, + "progress": 0.06265010684728622, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06265010684728622 + }, + { + "contact_quality": null, + "progress": 0.016331294551491737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016331294551491737 + } + ], + "candidate_progress": [ + 0.029071787372231483, + 0.06671091169118881, + 0.005293230060487986, + 0.02160499431192875, + 0.044261831790208817, + 0.07134183496236801, + 0.06265010684728622, + 0.016331294551491737 + ], + "candidate_source_chart_ids": [ + "ms-9229ce141dd41a11fac1", + "ms-45fbd749d5ed3fe37c13", + "ms-9229ce141dd41a11fac1", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-48178f52a96de3d1ead0", + "ms-45fbd749d5ed3fe37c13", + "ms-25ab20bb3da8ba34be95" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-8a72b73ac452bc190aeb", + "generated_tangents": [ + [ + 0.38898879289627075, + 0.048232778906822205, + -0.27594080567359924, + -0.4399867355823517, + 0.6052330732345581, + -2.289738893508911, + -0.8728967308998108, + 0.0841187983751297, + -0.12292179465293884, + 0.04051147401332855, + 0.9417217969894409, + 0.871377170085907, + -0.17005059123039246, + 0.38461577892303467, + 0.03485450893640518, + -0.18288835883140564, + 0.25484609603881836, + 0.3678521513938904, + 0.39994749426841736, + -0.0563485324382782, + 0.022962406277656555 + ], + [ + 0.42928653955459595, + 0.022410452365875244, + -0.38184383511543274, + -0.5375196933746338, + 0.31605851650238037, + -2.4681060314178467, + -0.7639077305793762, + 0.5468398332595825, + 0.012673158198595047, + -0.02565433830022812, + 0.9478268623352051, + 0.5630228519439697, + -0.7646524310112, + 0.34908342361450195, + -0.04627295210957527, + -0.35641101002693176, + 0.18813484907150269, + 0.4935140013694763, + 0.5122667551040649, + -0.23788228631019592, + 0.010100230574607849 + ], + [ + 0.23712769150733948, + 0.13500076532363892, + -0.36908820271492004, + -0.37255918979644775, + 0.5231672525405884, + -2.231074333190918, + -1.03863525390625, + 0.08374889194965363, + -0.05988902598619461, + 0.125088170170784, + 0.958392858505249, + 0.7723782062530518, + -0.29773563146591187, + 0.45239418745040894, + 0.08420394361019135, + -0.192107155919075, + 0.22629736363887787, + 0.4328534007072449, + 0.43861401081085205, + -0.14866802096366882, + 0.017510607838630676 + ], + [ + 0.4899980425834656, + 0.04783404618501663, + -0.45972272753715515, + -0.5446975231170654, + 0.25615087151527405, + -2.509995937347412, + -0.6396778225898743, + 0.4406871199607849, + 0.014913775026798248, + -0.009976811707019806, + 0.9618518948554993, + 0.5958398580551147, + -0.7680066823959351, + 0.21110044419765472, + -0.03699727728962898, + -0.366599440574646, + 0.2802734076976776, + 0.4948219656944275, + 0.4585062861442566, + -0.06579568982124329, + -0.020760580897331238 + ], + [ + 0.22369495034217834, + -0.06301255524158478, + 0.12384846806526184, + -0.8326464891433716, + 0.6406474113464355, + -2.646409273147583, + -1.1933045387268066, + 0.7598435878753662, + -0.022133609279990196, + -0.4027285575866699, + 0.6744118332862854, + 0.8667235374450684, + -1.349009394645691, + -0.4615873694419861, + -0.05838386341929436, + -0.1220080703496933, + 0.02892071008682251, + 0.3502132296562195, + 0.8222885131835938, + -0.5224537253379822, + -0.46730276942253113 + ], + [ + 0.47789353132247925, + 0.014254044741392136, + -0.3995330035686493, + -0.8906805515289307, + 0.41174107789993286, + -1.9399954080581665, + -0.6106557250022888, + 0.6918694972991943, + -0.14082100987434387, + -0.171038419008255, + 0.396634042263031, + 0.2174096703529358, + -1.219422459602356, + -0.17239926755428314, + -0.19276103377342224, + -0.0120627973228693, + 0.08096519112586975, + 0.29732704162597656, + 0.2129671275615692, + -0.7286126017570496, + -0.6878169775009155 + ], + [ + 0.48613131046295166, + 0.01594846323132515, + -0.3296055495738983, + -0.5491228103637695, + 0.2825898230075836, + -2.540189743041992, + -0.676031768321991, + 0.5068598389625549, + 0.03761497884988785, + -0.016039960086345673, + 0.9585782885551453, + 0.5539495944976807, + -0.7344855070114136, + 0.32664743065834045, + -0.007775496691465378, + -0.35730859637260437, + 0.2810666859149933, + 0.38950514793395996, + 0.45345014333724976, + -0.18090961873531342, + -0.1197081208229065 + ], + [ + 0.328957736492157, + 0.08033671975135803, + -0.1528473198413849, + -0.5800721645355225, + 0.4263259768486023, + -2.3949427604675293, + -1.0099446773529053, + 0.06037957966327667, + -0.16935035586357117, + 0.20834535360336304, + 0.8870474100112915, + 0.9208224415779114, + -0.3436633050441742, + -0.107276052236557, + -0.22534698247909546, + -0.29083573818206787, + 0.33226278424263, + 0.5033165812492371, + 0.3189806640148163, + -0.23628006875514984, + 0.1325584203004837 + ] + ], + "generated_utilities": [ + 0.029071787372231483, + 0.06671091169118881, + 0.005293230060487986, + 0.02160499431192875, + 0.044261831790208817, + 0.07134183496236801, + 0.06265010684728622, + 0.016331294551491737 + ], + "group_id": "ms-8a72b73ac452bc190aeb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.02298194169998169, + 0.01763375662267208, + 0.009361605159938335, + 0.02526181936264038, + 0.09214292466640472, + 0.022978028282523155, + 0.07704488933086395, + 0.056830525398254395, + 0.045355141162872314, + 0.06095697730779648, + 0.10329639911651611, + 0.05954315513372421, + 0.03003152273595333, + 0.008136491291224957, + 0.05599620193243027, + 0.011116823181509972 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.0913797616958618, + 0.3305599093437195, + -0.0778055489063263, + -2.223818778991699, + 0.9291321635246277, + -2.905466318130493, + -2.4575929641723633, + 1.1732350587844849, + 0.22933529317378998, + -0.8654288053512573, + 2.1122353076934814, + 1.5987088680267334, + 1.144788146018982, + 0.40604642033576965, + 0.10343833267688751, + -0.7513935565948486, + -1.054006814956665, + 1.396103024482727, + -0.16642586886882782, + -0.04196041822433472, + -1.2630513906478882 + ], + [ + 1.3690472841262817, + 0.2738865315914154, + -0.27599766850471497, + -2.431976556777954, + 0.7025187015533447, + -3.155372142791748, + -2.6898910999298096, + 1.1788215637207031, + 0.19074484705924988, + -0.7378551959991455, + 2.3089826107025146, + 1.3478807210922241, + 1.3458092212677002, + 0.3417958617210388, + 0.24155153334140778, + -1.004831314086914, + -1.3499912023544312, + 1.311987042427063, + -0.25646817684173584, + -0.3180008828639984, + -1.3290650844573975 + ], + [ + 0.9032705426216125, + -0.3011789321899414, + -0.21911907196044922, + -2.884286403656006, + 0.319735586643219, + -2.613290548324585, + -2.175532817840576, + 1.5185720920562744, + -0.1741684526205063, + -0.4879068434238434, + 2.721827507019043, + 2.1753995418548584, + 0.8214534521102905, + 0.7874165177345276, + 0.013686276972293854, + -0.7485443949699402, + -0.764183521270752, + 0.9452257752418518, + 0.07210072875022888, + 0.2788098156452179, + -1.8349425792694092 + ] + ], + "predicted_scores": [ + 3.9230997562408447, + 3.7890663146972656, + 3.765925645828247, + 3.3247056007385254, + 2.9679667949676514, + 2.929856061935425, + 2.90167498588562, + 2.408285140991211 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train2", + "selected_index": 0, + "state_hash": "722240dc5e14c5a30badc5fbec9f15ecf388cf6402fb77013c8f9dbdcd100846", + "stored_base_utility": 0.02298194169998169, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08285123109817505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08285123109817505 + }, + "base_progress": 0.08285123109817505, + "base_success": false, + "base_utility": 0.08285123109817505, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.035203736275434494, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035203736275434494 + }, + { + "contact_quality": null, + "progress": 0.032373785972595215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.032373785972595215 + }, + { + "contact_quality": null, + "progress": 0.05178316310048103, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05178316310048103 + }, + { + "contact_quality": null, + "progress": 0.0497889518737793, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0497889518737793 + }, + { + "contact_quality": null, + "progress": 0.05319662764668465, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05319662764668465 + }, + { + "contact_quality": null, + "progress": 0.07769393920898438, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07769393920898438 + }, + { + "contact_quality": null, + "progress": 0.03644818067550659, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03644818067550659 + }, + { + "contact_quality": null, + "progress": 0.06582687050104141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06582687050104141 + } + ], + "candidate_progress": [ + 0.035203736275434494, + 0.032373785972595215, + 0.05178316310048103, + 0.0497889518737793, + 0.05319662764668465, + 0.07769393920898438, + 0.03644818067550659, + 0.06582687050104141 + ], + "candidate_source_chart_ids": [ + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-a96bca1ef0b74f686018", + "generated_tangents": [ + [ + 0.24377943575382233, + 0.32708078622817993, + 0.911333441734314, + -0.5903794765472412, + 1.3735504150390625, + 0.2285427451133728, + 0.3272901475429535, + 0.21760070323944092, + 0.19221735000610352, + 0.417921781539917, + -1.0585862398147583, + 0.4535292685031891, + -1.0655913352966309, + -0.31938672065734863, + 0.24255125224590302, + 0.28544744849205017, + 0.2651522159576416, + -0.6137786507606506, + -1.0221281051635742, + -1.125736951828003, + -0.4783599078655243 + ], + [ + 0.11985970288515091, + 0.3361794352531433, + 0.6141902804374695, + -0.5244864821434021, + 0.7764644622802734, + 0.04178490489721298, + 0.6964371204376221, + 0.22153443098068237, + 0.07246668636798859, + 0.5326949954032898, + -0.9732793569564819, + 0.36889153718948364, + -1.846948504447937, + -0.19115303456783295, + 0.4001021683216095, + 0.39801299571990967, + 0.30469444394111633, + -0.6587927937507629, + -1.259711503982544, + -1.1519087553024292, + -0.7216782569885254 + ], + [ + 0.09906940907239914, + 0.3210197687149048, + 0.8100796937942505, + -0.6630990505218506, + 1.3996472358703613, + 0.08193384110927582, + 0.5416862964630127, + 0.13100376725196838, + 0.24795670807361603, + 0.25795668363571167, + -1.0583336353302002, + 0.36826959252357483, + -1.3281937837600708, + -0.1057974249124527, + 0.15827549993991852, + 0.289165735244751, + 0.2950815260410309, + -0.49331915378570557, + -0.9946284890174866, + -1.0162885189056396, + -0.2692895531654358 + ], + [ + 0.046681515872478485, + 0.26552754640579224, + 0.8814612627029419, + -0.6127615571022034, + 1.3233678340911865, + 0.13734042644500732, + 0.3281581401824951, + 0.07143212109804153, + 0.2597976624965668, + 0.28958308696746826, + -0.9637115001678467, + 0.5178719162940979, + -1.0891590118408203, + -0.3096383213996887, + 0.16210781037807465, + 0.30034083127975464, + 0.2671850919723511, + -0.505286693572998, + -1.0878467559814453, + -1.2061097621917725, + -0.2321859896183014 + ], + [ + 0.036467716097831726, + 0.2845940589904785, + 0.42028266191482544, + -0.7000330090522766, + 1.2009313106536865, + 0.4338286519050598, + 0.6932967901229858, + 0.32800519466400146, + 0.2756168246269226, + 0.1253436654806137, + -1.1826969385147095, + 0.3525141477584839, + -1.5950883626937866, + -0.22825323045253754, + 0.2078842669725418, + 0.38955697417259216, + 0.3499182462692261, + -0.3336631655693054, + -0.959294855594635, + -1.1037397384643555, + -0.7263059616088867 + ], + [ + 0.3529552221298218, + 0.3237884044647217, + 0.9772035479545593, + -0.40606868267059326, + 0.7661637663841248, + -0.01265116035938263, + 0.10729837417602539, + 0.18035152554512024, + 0.2199239432811737, + 0.26751601696014404, + -0.8885954022407532, + 0.49449723958969116, + -0.48741045594215393, + 0.284982293844223, + -0.113868847489357, + 0.32704752683639526, + -0.24657611548900604, + -0.40503430366516113, + -0.8470893502235413, + -0.9366427063941956, + -0.9513434171676636 + ], + [ + 0.13152211904525757, + 0.25801020860671997, + 0.9706121683120728, + -0.727902889251709, + 1.9322187900543213, + 0.5594120025634766, + 0.07011374831199646, + 0.27577298879623413, + 0.18436232209205627, + 0.14370256662368774, + -1.4306739568710327, + 0.6487612128257751, + -1.2129435539245605, + 0.3180162310600281, + 0.25029003620147705, + 0.32358434796333313, + 0.02250918745994568, + -0.6521103382110596, + -0.8014636635780334, + -0.4835957884788513, + -0.5101519823074341 + ], + [ + 0.21362939476966858, + 0.2772570848464966, + 0.8402403593063354, + -0.3651801347732544, + 0.9455380439758301, + -0.15807780623435974, + 0.01500430703163147, + 0.27950620651245117, + 0.11633198708295822, + 0.20554964244365692, + -0.9478622674942017, + 0.2525849938392639, + -0.6278101205825806, + 0.14455139636993408, + -0.05825187638401985, + 0.2951047718524933, + -0.1235896497964859, + -0.4522935152053833, + -0.7448015809059143, + -1.2800843715667725, + -0.6439568996429443 + ] + ], + "generated_utilities": [ + 0.035203736275434494, + 0.032373785972595215, + 0.05178316310048103, + 0.0497889518737793, + 0.05319662764668465, + 0.07769393920898438, + 0.03644818067550659, + 0.06582687050104141 + ], + "group_id": "ms-a96bca1ef0b74f686018", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07923875749111176, + 0.1507580727338791, + 0.09708476066589355, + 0.1572066992521286, + 0.09626718610525131, + 0.0781460627913475, + 0.07252912223339081, + 0.04244875907897949, + 0.026137471199035645, + 0.03892441838979721, + 0.04886416718363762, + 0.10530619323253632, + 0.0477486252784729, + 0.0898323506116867, + 0.01968318223953247, + 0.0289560966193676 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.48996758460998535, + 0.9053895473480225, + 1.8985445499420166, + -1.3594893217086792, + 1.6097791194915771, + 1.132408618927002, + 1.8319320678710938, + 0.46463528275489807, + 0.13208985328674316, + 0.8792686462402344, + -1.868321418762207, + 1.1490275859832764, + -2.0193891525268555, + -2.0637927055358887, + 0.11448347568511963, + 1.2957178354263306, + 1.293219804763794, + -1.7789109945297241, + -0.4312591254711151, + -1.7441130876541138, + -0.06536102294921875 + ], + [ + 0.03654795140028, + 1.7676925659179688, + 1.048339605331421, + -1.366391658782959, + 2.5093846321105957, + 2.160305976867676, + 2.8675124645233154, + -0.309950590133667, + -0.5841768980026245, + 1.3030188083648682, + -1.2228018045425415, + 1.1373907327651978, + -2.831526041030884, + -2.2111129760742188, + -0.14044174551963806, + 1.660491704940796, + 1.5215414762496948, + -1.0137104988098145, + -0.40579986572265625, + -1.9571893215179443, + 1.241256833076477 + ], + [ + -0.2826732099056244, + 2.2520718574523926, + 2.418309450149536, + -1.7572941780090332, + 1.5832278728485107, + 0.6361246705055237, + 2.2009639739990234, + 0.9109495878219604, + 0.22027403116226196, + 0.7075455188751221, + -1.2762477397918701, + 0.8630691766738892, + -1.2986351251602173, + -2.567821979522705, + -0.4690782427787781, + 2.1363346576690674, + 0.1853562593460083, + -1.541505217552185, + -0.033681899309158325, + -1.0161254405975342, + 0.06877446174621582 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03534998744726181, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.3504420518875122, + 0.0, + -0.013733148574829102, + 0.021970629692077637, + 0.014389097690582275, + -0.7432765960693359, + 0.006671249866485596, + -0.9755387306213379, + 0.0, + -0.027152668684720993, + 0.2708500623703003, + 0.052558183670043945, + -0.7615737915039062, + 0.020878344774246216, + -0.2927910089492798, + 0.0 + ], + [ + -0.1400495320558548, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.5714048147201538, + 0.0, + -0.3271818161010742, + -0.04413123428821564, + 0.16598933935165405, + -0.7432765960693359, + 0.2282017469406128, + -0.9755387306213379, + 0.0, + -0.4382812976837158, + 0.2708500623703003, + 0.2094714641571045, + -0.7615737915039062, + 0.23577484488487244, + -0.2927910089492798, + 0.0 + ] + ], + "predicted_scores": [ + 2.423630475997925, + 2.00319766998291, + 1.951001763343811, + 1.8348560333251953, + 1.5207659006118774, + 1.333327054977417, + 0.9061217904090881, + 0.8724957704544067 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train2", + "selected_index": 0, + "state_hash": "61714505b9ac6cbb1c6deff698bb3e96fd54c7cdb8f0d4307bfdd236634525c0", + "stored_base_utility": 0.07923875749111176, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.10657280683517456, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10657280683517456 + }, + { + "contact_quality": null, + "progress": 0.08607447147369385, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08607447147369385 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.10657280683517456, + 0.08607447147369385 + ], + "candidate_source_chart_ids": [ + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ac4fb33f3ab4db853995", + "generated_tangents": [ + [ + 1.6531691551208496, + -0.05526414141058922, + -0.1839403212070465, + 0.42755988240242004, + -0.09132128953933716, + 1.700735092163086, + 1.5142931938171387, + 1.6024315357208252, + 0.19293171167373657, + -1.11464524269104, + -0.5208871364593506, + -0.7540864944458008, + -3.1391124725341797, + -1.9721730947494507, + 1.2823563814163208, + 0.25687524676322937, + -0.9032511115074158, + 0.3206101953983307, + -0.6732171177864075, + -1.6390252113342285, + -1.0234016180038452 + ], + [ + 1.888071060180664, + -0.2116311937570572, + 0.10793489217758179, + 0.4411371648311615, + -0.3422536849975586, + 2.3450942039489746, + 1.5086116790771484, + 2.2642388343811035, + 0.20116852223873138, + -1.4900288581848145, + -0.7762719392776489, + -0.7539726495742798, + -3.886331558227539, + -2.482748031616211, + 1.4267284870147705, + 0.4085291624069214, + -0.6320739388465881, + 0.29684925079345703, + -0.3692408800125122, + -1.113175392150879, + -1.1301547288894653 + ], + [ + 1.6716148853302002, + -0.13889551162719727, + 0.007288455963134766, + 0.3261609375476837, + -0.17362120747566223, + 2.013519287109375, + 1.6892683506011963, + 2.1416587829589844, + 0.2149132788181305, + -1.5493266582489014, + -0.687951922416687, + -0.9009572267532349, + -3.6530354022979736, + -2.873899221420288, + 1.4981926679611206, + 0.37792372703552246, + -0.5116065144538879, + 0.08120601624250412, + -0.3444892168045044, + -1.060876727104187, + -1.3318877220153809 + ], + [ + 1.7461152076721191, + -0.21108204126358032, + -0.13795432448387146, + 0.32761266827583313, + -0.3691607117652893, + 2.3137362003326416, + 1.6223986148834229, + 2.0560142993927, + 0.24970006942749023, + -1.4734489917755127, + -0.7384984493255615, + -0.8993710279464722, + -3.650503635406494, + -2.9555695056915283, + 1.415787696838379, + 0.41653957962989807, + -0.4787940979003906, + 0.17419150471687317, + -0.32359981536865234, + -0.9217686653137207, + -1.3698807954788208 + ], + [ + 1.5240848064422607, + -0.3394169807434082, + 0.14871841669082642, + -0.32393109798431396, + 0.027644485235214233, + 0.5967182517051697, + 2.466012477874756, + 1.6800605058670044, + 0.18204478919506073, + -1.2379786968231201, + -0.6065686345100403, + -1.0351016521453857, + -3.2212986946105957, + -1.9640520811080933, + 0.9234760999679565, + 0.19026611745357513, + -0.473141074180603, + -0.0059905946254730225, + -0.41115814447402954, + -2.032818078994751, + -1.0137569904327393 + ], + [ + 1.647157907485962, + -0.21023760735988617, + -0.2447310984134674, + 0.34482625126838684, + -0.40069395303726196, + 1.971716284751892, + 1.8459899425506592, + 2.1531007289886475, + 0.22969676554203033, + -1.4864333868026733, + -0.9127892851829529, + -1.0147104263305664, + -3.628441572189331, + -2.7818243503570557, + 1.4673426151275635, + 0.3162531852722168, + -0.42331385612487793, + 0.1564280092716217, + -0.4080636501312256, + -1.2643910646438599, + -1.6785491704940796 + ], + [ + 1.6835044622421265, + -0.1711624711751938, + -0.322613388299942, + 0.4638591706752777, + -0.331248939037323, + 2.0816099643707275, + 1.8940846920013428, + 2.0946686267852783, + 0.26951080560684204, + -1.4435874223709106, + -0.6085704565048218, + -1.0365313291549683, + -3.5451056957244873, + -2.8089160919189453, + 1.4839354753494263, + 0.35859429836273193, + -0.43926113843917847, + -0.0053787678480148315, + -0.31296366453170776, + -1.1489890813827515, + -1.4108330011367798 + ], + [ + 1.649933934211731, + -0.24798466265201569, + -0.3144838511943817, + 0.3171953856945038, + -0.2297109067440033, + 1.523589849472046, + 1.9365217685699463, + 1.9921112060546875, + 0.2927779257297516, + -1.6297800540924072, + -0.6952162981033325, + -1.1927822828292847, + -3.184230327606201, + -2.780707359313965, + 1.660999059677124, + 0.3532601594924927, + -0.6409322023391724, + 0.05518341064453125, + -0.42419081926345825, + -1.48614501953125, + -1.4847630262374878 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.10657280683517456, + 0.08607447147369385 + ], + "group_id": "ms-ac4fb33f3ab4db853995", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.2651830017566681, + 0.22676818072795868, + 0.22454051673412323, + 2.0, + 0.31214088201522827, + 0.21535459160804749, + 0.18962346017360687, + 2.0, + 0.1830430030822754, + 0.15030252933502197, + 0.12050267308950424, + 0.17022761702537537, + 0.11389490216970444, + 0.07058519124984741, + 0.07076358795166016, + 0.18658748269081116 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 2.0962133407592773, + 0.057441189885139465, + -0.2733909487724304, + 0.4170292317867279, + -0.7772165536880493, + 2.858414888381958, + 2.1098427772521973, + 1.8366810083389282, + 0.017507493495941162, + -1.6127345561981201, + -0.39152991771698, + -1.017486333847046, + -2.815659999847412, + -2.787520408630371, + 1.6748408079147339, + 0.2986273169517517, + -0.6001855134963989, + 0.5475705862045288, + -0.28012242913246155, + -1.5382447242736816, + -1.6652274131774902 + ], + [ + 2.1338062286376953, + -0.07656055688858032, + -0.7531881332397461, + 0.48830646276474, + -0.36628106236457825, + 2.381558418273926, + 1.7315943241119385, + 1.838928461074829, + -0.050897032022476196, + -1.171897292137146, + -0.8350631594657898, + -0.8261303901672363, + -2.2635834217071533, + -3.2190167903900146, + 1.3416705131530762, + 0.44279617071151733, + -0.7299901247024536, + 1.2929750680923462, + -0.0032969415187835693, + -1.5000369548797607, + -1.6060972213745117 + ], + [ + 1.5521352291107178, + 0.03659695386886597, + -0.44382959604263306, + 0.21355122327804565, + -1.0471938848495483, + 3.194620370864868, + 1.9892739057540894, + 2.5084128379821777, + 0.531404972076416, + -0.9168282747268677, + -0.49052953720092773, + -1.1588059663772583, + -3.156189441680908, + -3.278008460998535, + 0.921509325504303, + 0.49181532859802246, + -1.0228296518325806, + 0.5866082310676575, + -0.4882396459579468, + -0.7946261763572693, + -2.119061231613159 + ], + [ + 2.7138938903808594, + -0.3406146764755249, + -0.32174432277679443, + 0.9962238669395447, + -0.558576226234436, + 2.049264907836914, + 1.8703808784484863, + 2.113438367843628, + 0.9615883827209473, + -1.9615708589553833, + -0.0062177181243896484, + -0.5808464288711548, + -2.1758205890655518, + -2.219890594482422, + 1.022050142288208, + 0.7201904654502869, + -1.3177021741867065, + 0.4089192748069763, + -0.0315951406955719, + -0.7215378880500793, + -2.069262981414795 + ], + [ + 2.5701913833618164, + 0.3137266933917999, + -0.5592924356460571, + 0.0026398897171020508, + -0.19283491373062134, + 2.2903807163238525, + 2.637770652770996, + 2.5494186878204346, + 0.9258015155792236, + -0.7146614193916321, + -0.686233639717102, + -0.7985042929649353, + -3.431988000869751, + -3.2178471088409424, + 1.319100260734558, + -0.15109702944755554, + -0.2694774866104126, + 1.0912322998046875, + -0.6380462050437927, + -2.0780839920043945, + -2.1209774017333984 + ], + [ + 2.6195321083068848, + -0.2646539509296417, + 0.15159690380096436, + 0.8749752044677734, + -1.1848887205123901, + 2.8789353370666504, + 1.6037814617156982, + 1.3343384265899658, + -0.2254078984260559, + -2.041776657104492, + -0.9811336994171143, + -1.3441768884658813, + -2.2502636909484863, + -1.9091612100601196, + 0.9959704279899597, + 0.42473238706588745, + -0.5330828428268433, + 1.6945891380310059, + 0.4069111943244934, + -0.5568764805793762, + -1.472819447517395 + ], + [ + 2.9485361576080322, + -0.6907311677932739, + -0.057036399841308594, + 1.2457408905029297, + 0.09488356113433838, + 2.6509125232696533, + 2.0378005504608154, + 2.278454303741455, + 1.1734973192214966, + -1.853393316268921, + 0.09614759683609009, + -0.35674989223480225, + -2.1597909927368164, + -1.9672894477844238, + 1.9760816097259521, + -0.47744297981262207, + 0.021782994270324707, + 1.6328153610229492, + -0.8569514751434326, + -1.0814869403839111, + -1.7780723571777344 + ], + [ + 2.1897926330566406, + -0.3883178234100342, + -0.6492674946784973, + 1.3223958015441895, + 0.30954307317733765, + 2.031817674636841, + 1.878535270690918, + 2.5030624866485596, + 0.6644855737686157, + -1.7382179498672485, + -1.1862120628356934, + -0.7967990636825562, + -2.8425841331481934, + -2.3635308742523193, + 2.0061349868774414, + -0.6015456318855286, + -1.429253339767456, + 1.2773160934448242, + 0.04122643172740936, + -2.0497689247131348, + -1.1791472434997559 + ], + [ + 1.586853265762329, + 0.06395213305950165, + -0.8944701552391052, + 0.5118889808654785, + -1.1310638189315796, + 2.865907669067383, + 1.3295278549194336, + 1.0862370729446411, + 0.6303088665008545, + -0.3754369616508484, + -0.04129624366760254, + -0.7044278979301453, + -2.923245429992676, + -1.7837438583374023, + 1.5932514667510986, + 1.2467985153198242, + 0.13401156663894653, + 1.6706414222717285, + -0.816297709941864, + -1.2893165349960327, + -1.0985345840454102 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.1383795738220215, + -0.1483764797449112, + -0.27178800106048584, + 0.26738643646240234, + -0.12845781445503235, + 1.6495440006256104, + 0.0, + 0.9713280200958252, + -0.050046056509017944, + -0.36543774604797363, + -0.022124826908111572, + 0.08494043350219727, + -1.671510934829712, + 0.0, + 0.5271386504173279, + 0.3898613452911377, + 0.21280205249786377, + -0.2029280662536621, + -0.01802508533000946, + -0.35994601249694824, + 0.0 + ], + [ + 1.7570452690124512, + 0.07137972116470337, + 0.02318662405014038, + 0.2907020151615143, + -0.2308352291584015, + 2.89892578125, + 2.537220001220703, + 2.238605499267578, + 0.16207315027713776, + -1.6996564865112305, + -0.8050198554992676, + -1.2014062404632568, + -2.5764517784118652, + -2.393839120864868, + 1.1276613473892212, + 0.31928181648254395, + -1.036102533340454, + 1.1266506910324097, + -0.1944657564163208, + -1.5998979806900024, + -2.151465892791748 + ] + ], + "predicted_scores": [ + 5.929937362670898, + 4.861267566680908, + 4.486452579498291, + 4.37121057510376, + 4.077110290527344, + 3.99444842338562, + 3.964893102645874, + 3.537386655807495 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392/train2", + "selected_index": 0, + "state_hash": "0b62c42c3beda0eb959692d55792514223681b522edd0a5e73247682cf0bfe95", + "stored_base_utility": 0.2651830017566681, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "generated_tangents": [ + [ + 1.5821378231048584, + 0.3146262764930725, + -1.0519134998321533, + -0.4090140759944916, + 0.605135440826416, + 0.02337801456451416, + -1.0089218616485596, + 0.7230665683746338, + -0.012307297438383102, + -0.23266643285751343, + -0.09701662510633469, + 0.26082491874694824, + -2.365182638168335, + -2.036797046661377, + 0.4285762310028076, + 0.4792984426021576, + 0.8540741801261902, + -1.276024341583252, + 1.1826664209365845, + -1.9315857887268066, + -0.6276378631591797 + ], + [ + 1.3737164735794067, + 0.2679445743560791, + -0.841779351234436, + -0.30625125765800476, + -0.1751491129398346, + -0.6902062892913818, + -0.6044722199440002, + 0.224562406539917, + 0.1592528223991394, + -0.09754335135221481, + -0.464151531457901, + -0.27126649022102356, + -2.840402841567993, + -1.8718410730361938, + 0.28366222977638245, + 0.2886638343334198, + 0.589530885219574, + -1.0971307754516602, + 1.037682056427002, + -2.1236624717712402, + -1.1697248220443726 + ], + [ + 1.4693235158920288, + 0.33992689847946167, + -0.9559155702590942, + -0.7759301662445068, + 0.5938560366630554, + 0.4410706162452698, + -1.1516180038452148, + 0.9748616218566895, + -0.041813597083091736, + -0.2766180634498596, + -0.27089664340019226, + 0.37276214361190796, + -2.81288480758667, + -2.1215333938598633, + 0.2626761496067047, + 0.4437684714794159, + 0.8953127861022949, + -1.1623727083206177, + 0.9205278158187866, + -1.9004206657409668, + -0.5554968118667603 + ], + [ + 1.3242058753967285, + 0.21055619418621063, + -1.0986342430114746, + -0.46126535534858704, + 0.25072017312049866, + -0.5641969442367554, + -0.49803632497787476, + 0.47351670265197754, + 0.11856383085250854, + -0.09280801564455032, + -0.29051852226257324, + -0.15622341632843018, + -2.8529722690582275, + -1.9449175596237183, + 0.3076666593551636, + 0.3464489281177521, + 0.5473067760467529, + -1.043578028678894, + 0.785569965839386, + -2.2451817989349365, + -1.1477209329605103 + ], + [ + 1.2049038410186768, + 0.21534813940525055, + -1.0089812278747559, + -0.7239872217178345, + 0.017136886715888977, + -0.9327054023742676, + -1.4726097583770752, + 0.6058399677276611, + 0.057108648121356964, + 0.04002790153026581, + 0.07871583104133606, + 0.07699061930179596, + -1.9197403192520142, + -1.3473838567733765, + 0.4497959613800049, + 0.23396480083465576, + 0.8955276608467102, + -0.8793306946754456, + 0.9313246011734009, + -1.6798534393310547, + -0.5567028522491455 + ], + [ + 1.4960994720458984, + 0.31615322828292847, + -1.0203620195388794, + -0.572618842124939, + 0.3891475796699524, + 0.29192644357681274, + -1.0681753158569336, + 0.832166314125061, + -0.06805451214313507, + -0.1598108410835266, + -0.18521764874458313, + 0.36702466011047363, + -2.688307285308838, + -2.25980544090271, + 0.3495863080024719, + 0.4443596303462982, + 0.9426625370979309, + -1.2859559059143066, + 0.8251757621765137, + -1.8694875240325928, + -0.8118435144424438 + ], + [ + 1.498408317565918, + 0.34343862533569336, + -0.7902431488037109, + -0.576141893863678, + 0.07044373452663422, + 0.3306047022342682, + -1.1915507316589355, + 0.8400086164474487, + -0.03115631826221943, + -0.07947546988725662, + -0.3417731523513794, + 0.37364697456359863, + -2.825819730758667, + -2.354111433029175, + 0.27121174335479736, + 0.4421323239803314, + 0.9562599062919617, + -1.1478915214538574, + 0.7435473799705505, + -1.878373146057129, + -0.7719675302505493 + ], + [ + 1.3977625370025635, + 0.3278428316116333, + -1.282359004020691, + -0.5069842338562012, + 0.3946380615234375, + 0.42068952322006226, + -0.3734458088874817, + 0.9397799968719482, + 0.0006347186863422394, + -0.12525802850723267, + -0.017962239682674408, + 0.35881081223487854, + -2.728942632675171, + -2.072857141494751, + 0.18919141590595245, + 0.36938372254371643, + 1.256686806678772, + -1.2285515069961548, + 0.7937507033348083, + -1.8180181980133057, + -0.7821594476699829 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.149173304438591, + 0.06221481412649155, + 0.1385849118232727, + 0.06656398624181747, + 2.0, + 0.10397814214229584, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7398372888565063, + -0.02836926281452179, + -0.4292973279953003, + 0.028730571269989014, + 0.009982943534851074, + -0.3464317321777344, + 0.0, + 0.009800434112548828, + -0.06816069781780243, + 0.030653484165668488, + -0.00554502010345459, + -0.06216755136847496, + -1.9802618026733398, + 0.0, + 0.01717478036880493, + -0.05601567029953003, + 0.010390102863311768, + -0.8072932958602905, + -0.05775028467178345, + -1.9342114925384521, + 0.0 + ], + [ + 0.7398372888565063, + -0.1961192637681961, + -0.4292973279953003, + -0.0017886161804199219, + 0.06461763381958008, + -0.3464317321777344, + 0.0, + 0.42972221970558167, + -0.5797654986381531, + 0.19436834752559662, + -0.23844504356384277, + 0.1733923852443695, + -1.9802618026733398, + 0.0, + -0.27269166707992554, + 0.04145494103431702, + 0.12319430708885193, + -0.8072932958602905, + 0.4949256181716919, + -1.9342114925384521, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7398372888565063, + 0.2242223471403122, + -1.4292973279953003, + -0.537040650844574, + -0.9353823661804199, + -1.3464317321777344, + -2.159137487411499, + 0.489885151386261, + -0.006211067084223032, + 0.09238272905349731, + -1.0055450201034546, + -0.04778099060058594, + -2.98026180267334, + -2.576570987701416, + 0.2992324233055115, + 0.34874510765075684, + 0.5165960192680359, + -1.8072932958602905, + 0.834364652633667, + -2.934211492538452, + -2.0790040493011475 + ], + [ + 1.9740948677062988, + 0.4187958538532257, + -1.2225801944732666, + -0.6112513542175293, + -1.0962133407592773, + -1.4875872135162354, + -2.1883857250213623, + 0.6603437662124634, + -0.20142723619937897, + 0.1697872281074524, + -0.8377580046653748, + -0.16444604098796844, + -2.87237286567688, + -2.4153482913970947, + 0.03898555040359497, + 0.5175161957740784, + 0.7254846096038818, + -1.9598647356033325, + 1.1053451299667358, + -3.172945976257324, + -1.9334450960159302 + ], + [ + 1.5938913822174072, + -0.018363401293754578, + -1.1851862668991089, + -0.20344951748847961, + -0.9226735234260559, + -1.3553601503372192, + -1.932102084159851, + 0.6446930170059204, + 0.30503973364830017, + 0.3522501289844513, + -0.9527953267097473, + -0.15980565547943115, + -2.9574379920959473, + -2.9056527614593506, + 0.6069583892822266, + 0.3722725808620453, + 0.7255810499191284, + -1.8503519296646118, + 0.5613387823104858, + -3.095244884490967, + -2.023747444152832 + ], + [ + 1.6626678705215454, + -0.02834661304950714, + -1.0125243663787842, + -0.5629516839981079, + -1.230878233909607, + -1.414373755455017, + -2.42856764793396, + 0.8520264625549316, + -0.1296539008617401, + -0.22410845756530762, + -1.0539721250534058, + -0.060277245938777924, + -2.740527391433716, + -2.1194698810577393, + 0.6826534867286682, + 0.7535779476165771, + 0.35573869943618774, + -1.9124289751052856, + 0.7101565003395081, + -3.326658010482788, + -1.8195855617523193 + ], + [ + 2.2881288528442383, + 0.6951426863670349, + -1.0978585481643677, + -0.008003771305084229, + -1.0754073858261108, + -1.7734596729278564, + -2.139570474624634, + 0.9205271601676941, + 0.29560548067092896, + 0.49158430099487305, + -1.2606573104858398, + 0.43956148624420166, + -2.831833600997925, + -2.960129976272583, + 0.6832801103591919, + -0.03894898295402527, + 1.036036491394043, + -2.127277135848999, + 0.6759494543075562, + -3.1690073013305664, + -1.9895020723342896 + ], + [ + 1.9020261764526367, + 0.6575945019721985, + -1.3994193077087402, + -0.5040701031684875, + -0.6857678890228271, + -0.7056208252906799, + -1.5344107151031494, + 0.7615087032318115, + 0.6105715036392212, + 0.19082021713256836, + -0.6303516626358032, + -0.4614318609237671, + -2.3375303745269775, + -2.188478469848633, + 0.6886799335479736, + -0.03237125277519226, + 0.483882874250412, + -1.3982734680175781, + 0.23996800184249878, + -3.3969180583953857, + -1.5116117000579834 + ], + [ + 1.666227102279663, + 0.6915197968482971, + -1.6392340660095215, + -0.9650112390518188, + -0.2591104507446289, + -1.5417896509170532, + -1.695828914642334, + 1.130251169204712, + -0.555443525314331, + -0.4872768521308899, + -1.0994479656219482, + 0.5934816002845764, + -3.5218958854675293, + -1.8239338397979736, + 0.7150859832763672, + 0.8003170490264893, + 1.0113730430603027, + -2.1905770301818848, + 0.8595032095909119, + -3.039506673812866, + -2.3675880432128906 + ], + [ + 1.59856116771698, + -0.036071136593818665, + -1.700652837753296, + 0.27602267265319824, + -1.608276128768921, + -0.875938355922699, + -1.6728798151016235, + 0.8851535320281982, + 0.7461313605308533, + -0.7497223019599915, + -1.0928030014038086, + -0.4735495150089264, + -2.223254680633545, + -2.5322465896606445, + 0.044906944036483765, + 0.6160990595817566, + 1.3697642087936401, + -1.6367510557174683, + 0.086475670337677, + -3.323115825653076, + -2.058189630508423 + ], + [ + 2.285855531692505, + 0.6314746141433716, + -1.2119637727737427, + 0.039707839488983154, + -0.9751565456390381, + -1.7796330451965332, + -1.5261995792388916, + 0.5450726747512817, + -0.8933275938034058, + 0.7903657555580139, + -1.5466060638427734, + 0.5994364023208618, + -3.370340585708618, + -3.1641077995300293, + 0.3225497603416443, + -0.39524197578430176, + 1.152822732925415, + -1.508494257926941, + 1.1221402883529663, + -3.361729860305786, + -1.9251339435577393 + ], + [ + 2.2599079608917236, + -0.005553290247917175, + -2.4053664207458496, + 0.39961689710617065, + -0.40980857610702515, + -0.9905421733856201, + -1.7244036197662354, + -0.06417369842529297, + 0.6376181840896606, + 0.5168956518173218, + -1.6740490198135376, + -0.1833474189043045, + -3.9368975162506104, + -1.7146861553192139, + 0.9366075992584229, + 0.701168417930603, + -0.23131346702575684, + -2.740941286087036, + 0.908729076385498, + -3.6231396198272705, + -1.3161351680755615 + ], + [ + 1.819408655166626, + -0.5504165291786194, + -1.302573800086975, + -0.6927691698074341, + -0.7548186779022217, + -2.1824703216552734, + -2.4921669960021973, + 0.6337187886238098, + 0.1691494882106781, + 0.7547447681427002, + -0.4371253252029419, + -0.6562469601631165, + -2.313655376434326, + -2.4974589347839355, + 0.05105690658092499, + 0.3473445773124695, + -0.4244192838668823, + -1.8233305215835571, + -0.1157217025756836, + -2.0408761501312256, + -2.2010600566864014 + ] + ], + "predicted_scores": [ + 3.6475484371185303, + 3.555715322494507, + 3.4646213054656982, + 3.458019495010376, + 3.0644209384918213, + 2.9289677143096924, + 2.7355568408966064, + 2.3974509239196777 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2054899126/train2", + "selected_index": 0, + "state_hash": "b26c5f64bc7e877098323281f2b58e7850275b30405bc411ed7ffc7cf69f4e26", + "stored_base_utility": 0.149173304438591, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.037537217140197754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037537217140197754 + }, + "base_progress": 0.037537217140197754, + "base_success": false, + "base_utility": 0.037537217140197754, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.02533026598393917, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02533026598393917 + }, + { + "contact_quality": null, + "progress": 0.008884906768798828, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008884906768798828 + }, + { + "contact_quality": null, + "progress": 0.008443613536655903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008443613536655903 + }, + { + "contact_quality": null, + "progress": 0.02165730856359005, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02165730856359005 + }, + { + "contact_quality": null, + "progress": 0.010604898445308208, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010604898445308208 + }, + { + "contact_quality": null, + "progress": 0.02483522891998291, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02483522891998291 + }, + { + "contact_quality": null, + "progress": 0.029142936691641808, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029142936691641808 + }, + { + "contact_quality": null, + "progress": 0.009114067070186138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009114067070186138 + } + ], + "candidate_progress": [ + 0.02533026598393917, + 0.008884906768798828, + 0.008443613536655903, + 0.02165730856359005, + 0.010604898445308208, + 0.02483522891998291, + 0.029142936691641808, + 0.009114067070186138 + ], + "candidate_source_chart_ids": [ + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-6f26118aa94bacd82250", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-c832439976a37f52d094", + "generated_tangents": [ + [ + -0.8198245167732239, + -0.03920508921146393, + 0.5819593667984009, + 0.7463089227676392, + 0.10489299893379211, + -0.10580489039421082, + -0.20906049013137817, + 1.1903159618377686, + -0.34412050247192383, + -0.6929799318313599, + -1.9149105548858643, + 0.643068253993988, + -1.6121429204940796, + -1.0431427955627441, + 0.1031402200460434, + 0.24797122180461884, + 0.7003986835479736, + 0.023552045226097107, + 0.13818491995334625, + 2.3249471187591553, + 1.3970123529434204 + ], + [ + -1.0568962097167969, + -0.18154874444007874, + 0.4242200553417206, + 1.3580749034881592, + -0.2119625210762024, + -0.10483729839324951, + -0.6275549530982971, + 1.8155874013900757, + -0.3267662525177002, + -0.7155795693397522, + -2.3674774169921875, + 1.2270745038986206, + -1.9952422380447388, + -1.1826207637786865, + 0.12356767058372498, + 0.3143351376056671, + 0.4271289110183716, + 0.07336220145225525, + 0.4373856484889984, + 3.0269980430603027, + 1.3758344650268555 + ], + [ + -1.0864646434783936, + -0.16934318840503693, + 0.3293246030807495, + 1.4138600826263428, + -0.07199227809906006, + 0.03945857286453247, + -0.4595247507095337, + 1.8685837984085083, + -0.3098562955856323, + -0.6432785987854004, + -2.4197516441345215, + 1.0945141315460205, + -2.026355504989624, + -1.45381498336792, + 0.06001746654510498, + 0.32894641160964966, + 0.45864784717559814, + 0.0468917042016983, + 0.595192551612854, + 2.9091782569885254, + 1.4164118766784668 + ], + [ + -0.6667748093605042, + -0.06050927937030792, + 0.5117385387420654, + 0.8088277578353882, + 0.022689253091812134, + -0.14834624528884888, + 0.016691237688064575, + 1.2453981637954712, + -0.3827824592590332, + -0.5151144862174988, + -1.9905287027359009, + 0.7936994433403015, + -1.7512415647506714, + -1.3275130987167358, + 0.11352592706680298, + 0.2874607741832733, + 0.6238629817962646, + -0.0001649409532546997, + 0.25904381275177, + 2.1807234287261963, + 1.6636848449707031 + ], + [ + -1.0627790689468384, + -0.06425783783197403, + 0.45271599292755127, + 1.2808951139450073, + -0.07449811697006226, + -0.08740252256393433, + -0.3336034417152405, + 1.748331904411316, + -0.25657546520233154, + -0.6822872757911682, + -2.256725311279297, + 1.1384570598602295, + -2.145827054977417, + -1.134251356124878, + 0.12506824731826782, + 0.30707207322120667, + 0.5226618051528931, + 0.044191062450408936, + 0.6069962978363037, + 2.6754794120788574, + 1.2606077194213867 + ], + [ + -0.543549120426178, + 0.05135799199342728, + 0.04146483540534973, + 0.9642932415008545, + 0.2744615077972412, + -0.7055658102035522, + 0.03691324591636658, + 0.9951686859130859, + -0.32953909039497375, + -0.23412221670150757, + -1.3947173357009888, + 0.9031991958618164, + -1.835490107536316, + -0.7074829339981079, + -0.1214071661233902, + 0.1754102259874344, + 0.7325943112373352, + -0.28040844202041626, + 0.5575394630432129, + 1.8994114398956299, + 0.8361173868179321 + ], + [ + -0.911431074142456, + 0.015191037207841873, + 0.5205184817314148, + 0.7936104536056519, + 0.05123633146286011, + -0.14767053723335266, + -0.06205552816390991, + 1.1552459001541138, + -0.4112311601638794, + -0.5587905049324036, + -1.9563528299331665, + 0.9562231302261353, + -1.4852240085601807, + -0.8329577445983887, + 0.03854535147547722, + 0.253476083278656, + 0.5263214111328125, + 0.09186801314353943, + 0.6027330756187439, + 2.5320591926574707, + 1.1744015216827393 + ], + [ + -0.8819104433059692, + -0.004576355218887329, + 0.35864776372909546, + 1.2648749351501465, + 0.19551649689674377, + -0.12527060508728027, + -0.251573383808136, + 1.5137056112289429, + -0.12731441855430603, + -0.6814642548561096, + -2.018608570098877, + 0.9902058243751526, + -2.0675392150878906, + -0.812502384185791, + 0.24994364380836487, + 0.27510279417037964, + 0.5562025904655457, + -0.038785889744758606, + 0.4682663381099701, + 2.339139461517334, + 1.2594951391220093 + ] + ], + "generated_utilities": [ + 0.02533026598393917, + 0.008884906768798828, + 0.008443613536655903, + 0.02165730856359005, + 0.010604898445308208, + 0.02483522891998291, + 0.029142936691641808, + 0.009114067070186138 + ], + "group_id": "ms-c832439976a37f52d094", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0808674544095993, + 0.13129448890686035, + 0.11199682950973511, + 0.2594742774963379, + 0.17684690654277802, + 0.11511725187301636, + 0.17100229859352112, + 0.12906593084335327, + 0.12404869496822357, + 0.10653237998485565, + 0.15492436289787292, + 0.05800905078649521, + 0.04767048358917236, + 0.09533219039440155, + 0.03300365060567856, + 0.07704520225524902 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025296777486801147, + 0.11629299819469452, + -0.024434387683868408, + 1.0812468528747559, + 0.04022188484668732, + -0.01547396183013916, + 0.0, + 0.05167186260223389, + 0.07629208266735077, + -0.00870610773563385, + -0.5777420997619629, + 0.07687681913375854, + -0.6336902379989624, + 0.0, + -0.043449461460113525, + -0.029798656702041626, + 0.18944299221038818, + 0.03022998571395874, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + 0.05685463547706604, + -0.3914306163787842, + 0.4578048586845398, + 1.0812468528747559, + 0.10990457236766815, + -0.18159747123718262, + 0.0, + 0.05167186260223389, + -0.5002377033233643, + 0.04299694299697876, + -0.5777420997619629, + 0.02747189998626709, + -0.6336902379989624, + 0.0, + -0.08638033270835876, + 0.17561247944831848, + 0.37304359674453735, + 0.2538231611251831, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + -0.3902576267719269, + 0.03852551057934761, + 0.16114424169063568, + 2.081246852874756, + 0.22516529262065887, + 0.8184025287628174, + 0.3388257324695587, + 1.0516718626022339, + -0.061541248112916946, + 0.22279328107833862, + -1.577742099761963, + 1.027471899986267, + -1.6336902379989624, + -1.6284003257751465, + -0.24836044013500214, + -0.3427097499370575, + 1.1894429922103882, + -0.15580247342586517, + 1.7312084436416626, + 2.8623785972595215, + 1.0248357057571411 + ], + [ + -0.5903857946395874, + -0.10628272593021393, + -0.07594148814678192, + 2.194340944290161, + 0.11632551997900009, + 0.942036509513855, + 0.06351110339164734, + 0.7966359853744507, + 0.1702205240726471, + 0.1549067199230194, + -1.439386248588562, + 1.3076335191726685, + -1.3458164930343628, + -1.90281081199646, + -0.2967699468135834, + -0.27783676981925964, + 1.0556988716125488, + 0.06510581076145172, + 1.6170521974563599, + 3.027111530303955, + 1.154527187347412 + ], + [ + -0.7559880614280701, + 0.694898247718811, + -0.3551136255264282, + 1.8615511655807495, + 0.9249891042709351, + 0.969862699508667, + 0.4452719986438751, + 1.5801937580108643, + 0.13303470611572266, + -0.13816028833389282, + -1.2399333715438843, + 1.5769362449645996, + -1.5071980953216553, + -2.0578060150146484, + -0.8449342846870422, + 0.3327537477016449, + 1.6781622171401978, + 0.4016150236129761, + 1.8172147274017334, + 3.1609320640563965, + 1.4916398525238037 + ] + ], + "predicted_scores": [ + 2.265834331512451, + 2.2305963039398193, + 2.0992085933685303, + 2.014530658721924, + 1.3508869409561157, + 1.0760185718536377, + 0.9567503929138184, + 0.7489107847213745 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train2", + "selected_index": 0, + "state_hash": "0f2810f22383259329f6e94fdc8384176c664f2ed476924e59db7cda4f4a4863", + "stored_base_utility": 0.0808674544095993, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09138330072164536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09138330072164536 + }, + "base_progress": 0.09138330072164536, + "base_success": false, + "base_utility": 0.09138330072164536, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.015890737995505333, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015890737995505333 + }, + { + "contact_quality": null, + "progress": 0.006532192230224609, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006532192230224609 + }, + { + "contact_quality": null, + "progress": 0.0023473103065043688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0023473103065043688 + }, + { + "contact_quality": null, + "progress": 0.007760286331176758, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007760286331176758 + }, + { + "contact_quality": null, + "progress": 0.01693703792989254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01693703792989254 + }, + { + "contact_quality": null, + "progress": 0.0026943087577819824, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0026943087577819824 + }, + { + "contact_quality": null, + "progress": 0.009350995533168316, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009350995533168316 + }, + { + "contact_quality": null, + "progress": 0.013496856205165386, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013496856205165386 + } + ], + "candidate_progress": [ + 0.015890737995505333, + 0.006532192230224609, + 0.0023473103065043688, + 0.007760286331176758, + 0.01693703792989254, + 0.0026943087577819824, + 0.009350995533168316, + 0.013496856205165386 + ], + "candidate_source_chart_ids": [ + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c96b415f083f60857334", + "generated_tangents": [ + [ + 0.06174185127019882, + 0.16033563017845154, + 0.6671453714370728, + 0.40813764929771423, + 0.3197011351585388, + 2.89839243888855, + 1.3230814933776855, + -0.14131903648376465, + 0.006032740697264671, + 1.1513893604278564, + 0.23397895693778992, + 0.8938525319099426, + 1.2062853574752808, + 1.2772371768951416, + -0.12925788760185242, + -0.6825408339500427, + -0.03556711971759796, + 1.296842098236084, + -0.601061224937439, + 0.1869063377380371, + -0.16873237490653992 + ], + [ + -0.3362332582473755, + 0.09249238669872284, + 1.1323519945144653, + 0.7928755283355713, + 0.5456668138504028, + 2.4640860557556152, + 1.2390525341033936, + -0.4458146393299103, + -0.09833692759275436, + 0.85829097032547, + -0.14441698789596558, + 1.6865679025650024, + 1.6072865724563599, + 1.194542407989502, + -0.12387286126613617, + -0.673762321472168, + 0.00941266119480133, + 1.4972132444381714, + -0.4563497304916382, + 0.774008572101593, + -0.4810538589954376 + ], + [ + -0.5711472630500793, + 0.23895512521266937, + 0.8510761260986328, + 0.5206438302993774, + 1.3842408657073975, + 2.4938137531280518, + 0.8395487070083618, + -0.5246961116790771, + -0.07523845881223679, + 1.12064790725708, + 0.49252521991729736, + 1.1939893960952759, + 1.4841113090515137, + 0.8059262633323669, + -0.19115960597991943, + -0.4434705078601837, + 0.671130359172821, + 1.4036900997161865, + -0.39283162355422974, + 0.6669578552246094, + 0.3197353482246399 + ], + [ + -0.2605810761451721, + 0.06699453294277191, + 1.0132678747177124, + 0.5877721309661865, + 0.7528336048126221, + 2.762845039367676, + 1.0420910120010376, + -0.16084054112434387, + -0.08754749596118927, + 0.9618850350379944, + 0.16109144687652588, + 1.2569996118545532, + 1.3267520666122437, + 1.039302945137024, + -0.11500556766986847, + -0.6801729202270508, + 0.20641353726387024, + 1.3454126119613647, + -0.2619008421897888, + 0.619723379611969, + -0.4462575614452362 + ], + [ + -0.4345166087150574, + 0.005640070885419846, + 0.45308390259742737, + 0.7029827833175659, + 0.520183801651001, + 2.582667350769043, + 1.0958545207977295, + -0.09384369850158691, + -0.13915623724460602, + 0.6293283104896545, + 0.21264910697937012, + 1.0392800569534302, + 1.2485343217849731, + 1.30171799659729, + -0.03129736706614494, + -0.6041085720062256, + -0.1849183291196823, + 1.6694345474243164, + -0.1887371838092804, + 0.2625390887260437, + -0.36729738116264343 + ], + [ + -0.5511574745178223, + 0.26515454053878784, + 0.9038888216018677, + 0.540073037147522, + 1.4176926612854004, + 2.3262815475463867, + 0.6459638476371765, + -0.32197311520576477, + -0.1562633365392685, + 1.2329294681549072, + 0.43738022446632385, + 1.0444974899291992, + 1.4231468439102173, + 0.9694470763206482, + -0.19568341970443726, + -0.45007869601249695, + 0.7409229874610901, + 1.4327186346054077, + -0.35214728116989136, + 0.596210241317749, + 0.23459947109222412 + ], + [ + -0.15069086849689484, + 0.06990204751491547, + 0.9320263862609863, + 0.6539721488952637, + 0.7788692116737366, + 2.5048460960388184, + 1.065310001373291, + -0.2339310646057129, + -0.07993582636117935, + 0.8268597722053528, + 0.18042296171188354, + 1.3550728559494019, + 1.321607232093811, + 0.793192446231842, + 0.03388269990682602, + -0.6482014656066895, + 0.1964799165725708, + 1.4189894199371338, + -0.29010432958602905, + 0.7080780863761902, + -0.45979389548301697 + ], + [ + -0.15596266090869904, + 0.06414393335580826, + 0.7779709100723267, + 0.7256145477294922, + 0.7950412034988403, + 2.3781542778015137, + 1.0835585594177246, + -0.21336519718170166, + -0.12509968876838684, + 0.726881206035614, + 0.025465819984674454, + 1.324411153793335, + 1.5107316970825195, + 0.8208259344100952, + -0.08363127708435059, + -0.6538136005401611, + -0.021880105137825012, + 1.5258489847183228, + -0.2999498248100281, + 0.798958957195282, + -0.4104281961917877 + ] + ], + "generated_utilities": [ + 0.015890737995505333, + 0.006532192230224609, + 0.0023473103065043688, + 0.007760286331176758, + 0.01693703792989254, + 0.0026943087577819824, + 0.009350995533168316, + 0.013496856205165386 + ], + "group_id": "ms-c96b415f083f60857334", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15946325659751892, + 0.16313008964061737, + 0.2226802557706833, + 0.18744397163391113, + 0.21545825898647308, + 0.15985511243343353, + 0.16218391060829163, + 0.26426148414611816, + 0.184181809425354, + 0.24352896213531494, + 0.14179718494415283, + 0.1026589497923851, + 0.1274908483028412, + 0.05275428295135498, + 0.08544860780239105, + 0.07143914699554443 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.16191565990448, + 0.4561867117881775, + 1.1241415739059448, + 0.08947351574897766, + 0.32140541076660156, + 3.6895532608032227, + 3.6284518241882324, + -1.8666775226593018, + -0.22328677773475647, + 1.48662531375885, + 1.4737720489501953, + 1.35750150680542, + 2.507124423980713, + 1.9315069913864136, + -0.8371121883392334, + 0.5339290499687195, + 1.0061794519424438, + 2.149812698364258, + 0.08911214768886566, + 0.17410686612129211, + 0.3549031913280487 + ], + [ + -0.14804881811141968, + -0.5333454608917236, + 0.7481307983398438, + 0.22411420941352844, + 1.7204210758209229, + 3.908766746520996, + 3.3698034286499023, + -1.6747719049453735, + 0.2769082486629486, + 2.042428970336914, + 0.44748249650001526, + 1.9492043256759644, + 2.918121576309204, + 3.2413060665130615, + -0.6994609832763672, + -0.3854917883872986, + -0.6702244281768799, + 2.408869504928589, + -0.5672509670257568, + 0.5941196084022522, + 0.46011871099472046 + ], + [ + -0.46204906702041626, + -0.4236203730106354, + 1.8582539558410645, + 0.8416532278060913, + 1.2386386394500732, + 3.7320151329040527, + 2.242549180984497, + -1.8652616739273071, + -1.0259326696395874, + 1.3383660316467285, + 1.7068204879760742, + 0.7941814661026001, + 3.3389508724212646, + 2.7797679901123047, + -0.9363516569137573, + 0.17812880873680115, + 0.6111975908279419, + 2.066016435623169, + -0.28941091895103455, + 1.501695990562439, + -0.35273995995521545 + ], + [ + -0.7471081614494324, + -0.029650092124938965, + 1.019482135772705, + -0.30564722418785095, + 1.380143642425537, + 3.247985601425171, + 3.0548973083496094, + -0.5582134127616882, + -0.920264482498169, + 2.3859832286834717, + 1.1080355644226074, + 1.5716153383255005, + 3.089156150817871, + 2.043484687805176, + 0.1913408637046814, + -0.7557333707809448, + -0.6297664642333984, + 1.7829738855361938, + -0.6967166662216187, + -0.10757219791412354, + 0.229112908244133 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027880847454071045, + -0.2692106068134308, + 0.12925803661346436, + -0.1745016872882843, + -0.1231425404548645, + 1.9967715740203857, + 0.0, + -0.1433020830154419, + 0.2667914927005768, + 0.8867976665496826, + -0.057856738567352295, + 0.5789351463317871, + 1.7831475734710693, + 0.0, + -0.35565105080604553, + 0.293251097202301, + 0.012226536870002747, + 1.121243953704834, + -0.15060651302337646, + 0.03409510850906372, + 0.0 + ], + [ + -0.7738120555877686, + 0.08056297898292542, + 1.1292580366134644, + 0.4734484851360321, + 0.8768574595451355, + 2.9967715740203857, + 3.0509116649627686, + -1.143302083015442, + -0.1310054361820221, + 1.8867976665496826, + 0.9421432614326477, + 1.578935146331787, + 2.7831475734710693, + 2.6037895679473877, + -0.5597631931304932, + 0.11406822502613068, + 0.2593899667263031, + 2.121243953704834, + -0.1791832000017166, + 0.5773167610168457, + 0.41381677985191345 + ], + [ + -0.6024693250656128, + 0.29007619619369507, + 1.3939473628997803, + 0.3474765419960022, + 1.1848478317260742, + 3.127882719039917, + 3.3329083919525146, + -1.2855982780456543, + 0.20972535014152527, + 1.8329110145568848, + 1.267270803451538, + 1.709198236465454, + 2.927372455596924, + 2.9659509658813477, + -0.7555317282676697, + 0.015305742621421814, + 0.6009905338287354, + 2.188507556915283, + 0.07961152493953705, + 0.4547080397605896, + 0.20378917455673218 + ], + [ + -1.2702982425689697, + 0.06162632256746292, + 0.9447290897369385, + 0.837199330329895, + 0.5848468542098999, + 3.4801836013793945, + 2.8138537406921387, + -1.5083554983139038, + -0.45768383145332336, + 2.08561110496521, + 1.4951050281524658, + 1.8484808206558228, + 2.5617129802703857, + 2.942486047744751, + -0.562404215335846, + 0.17441512644290924, + 0.7880480289459229, + 1.8807158470153809, + -0.043379828333854675, + 0.39090824127197266, + 0.43096962571144104 + ] + ], + "predicted_scores": [ + 2.9776570796966553, + 2.588106393814087, + 2.4356861114501953, + 2.3696303367614746, + 2.3323519229888916, + 2.198922634124756, + 2.1822986602783203, + 2.0857701301574707 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train2", + "selected_index": 0, + "state_hash": "bf24d506cf320effed2cd7faf94d8618b0626bf8d5c3abc1f837513597c18231", + "stored_base_utility": 0.15946325659751892, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0703698992729187, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0703698992729187 + }, + { + "contact_quality": null, + "progress": 0.06225750967860222, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06225750967860222 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06159693002700806, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06159693002700806 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 0.0703698992729187, + 0.06225750967860222, + 1.0, + 0.06159693002700806, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-6d46dae312649c977b64", + "ms-ebacae0135af2832dfbb", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683", + "ms-3d090ac2459c4b891dd1", + "ms-ebacae0135af2832dfbb", + "ms-003a2225c0892dcda683" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + false, + false, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "generated_tangents": [ + [ + -0.0733477771282196, + 0.2440221756696701, + 0.15181493759155273, + -0.41763055324554443, + 1.9432990550994873, + 0.5350282192230225, + -0.27695566415786743, + 0.25723034143447876, + -0.06783943623304367, + 0.3047679364681244, + 0.3456987142562866, + 0.3823747932910919, + -0.4067792296409607, + 0.14749783277511597, + -0.8008304238319397, + 0.11215963959693909, + 1.2942795753479004, + 0.21942253410816193, + -0.0323183536529541, + 1.8373687267303467, + -0.3271423280239105 + ], + [ + -0.06810550391674042, + 0.06187015026807785, + 0.49137261509895325, + -0.3761376738548279, + 1.8276028633117676, + 0.30036407709121704, + -0.24131083488464355, + 0.08209671080112457, + 0.06700944900512695, + 0.12063291668891907, + 0.3343655467033386, + 0.6434705257415771, + -0.5821419358253479, + -0.26108187437057495, + -0.8045896887779236, + 0.19552746415138245, + 1.0714962482452393, + 0.08714357018470764, + -0.18119210004806519, + 1.9759488105773926, + -0.0006811916828155518 + ], + [ + 0.0842127725481987, + 0.17406752705574036, + 0.5474393367767334, + -0.3364284634590149, + 1.5966274738311768, + 0.23887129127979279, + -0.5160813927650452, + 0.2524411678314209, + -0.032035455107688904, + 0.11147579550743103, + 0.493985116481781, + 0.5396754145622253, + -0.6150932908058167, + -0.06750963628292084, + -0.9069730043411255, + 0.20672529935836792, + 1.2836986780166626, + 0.11550749093294144, + -0.04422783851623535, + 2.0371110439300537, + -0.03447459638118744 + ], + [ + -0.13922162353992462, + 0.059459343552589417, + 0.483210027217865, + -0.43105340003967285, + 1.721055507659912, + 0.39234158396720886, + -0.17947715520858765, + -0.006537705659866333, + -0.02469947747886181, + 0.14277306199073792, + 0.2891971170902252, + 0.6004711389541626, + -0.4498223066329956, + -0.2213577777147293, + -0.7869839668273926, + 0.197745680809021, + 1.1171355247497559, + 0.02183094620704651, + -0.21498942375183105, + 2.0113275051116943, + 0.01405133306980133 + ], + [ + -0.1677147001028061, + 0.025322608649730682, + 0.4403640031814575, + -0.4360009431838989, + 1.7552154064178467, + 0.3358663320541382, + -0.004117816686630249, + 0.104227215051651, + -0.0601222962141037, + 0.18684738874435425, + 0.27332305908203125, + 0.43872106075286865, + -0.3200552463531494, + -0.2029048055410385, + -0.8302770853042603, + 0.13667984306812286, + 1.139520525932312, + 0.011928841471672058, + -0.27651447057724, + 2.025801420211792, + 0.023210197687149048 + ], + [ + -0.07179400324821472, + 0.09151734411716461, + 0.48320209980010986, + -0.40990766882896423, + 1.9464759826660156, + 0.1472827047109604, + -0.2739640474319458, + 0.032065972685813904, + -0.07190819829702377, + 0.16847270727157593, + 0.33975619077682495, + 0.5009675025939941, + -0.35624414682388306, + -0.31571322679519653, + -0.789415180683136, + 0.13318884372711182, + 1.192352056503296, + -0.0014500021934509277, + -0.15416264533996582, + 1.934866189956665, + 0.005992040038108826 + ], + [ + -0.17290262877941132, + 0.07997788488864899, + 0.4743005335330963, + -0.4328017830848694, + 1.936750888824463, + 0.28863662481307983, + -0.15091127157211304, + 0.08707568049430847, + -0.025879545137286186, + 0.14880897104740143, + 0.29611441493034363, + 0.45149537920951843, + -0.4635292589664459, + -0.24268673360347748, + -0.7780963182449341, + 0.1647162288427353, + 1.183290719985962, + -0.004053756594657898, + -0.2848284840583801, + 1.9144017696380615, + -0.0437970906496048 + ], + [ + -0.2155911773443222, + 0.04945673048496246, + 0.4276820719242096, + -0.4204087257385254, + 1.80472993850708, + 0.3480533957481384, + 0.025602787733078003, + 0.07459354400634766, + -0.050504542887210846, + 0.14902879297733307, + 0.27028581500053406, + 0.3612027168273926, + -0.3037831783294678, + -0.20018760859966278, + -0.8478841781616211, + 0.1292649358510971, + 1.168738842010498, + -0.0031080394983291626, + -0.3887249231338501, + 2.0357699394226074, + -0.007598087191581726 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 0.0703698992729187, + 0.06225750967860222, + 2.0, + 0.06159693002700806, + 2.0 + ], + "group_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20931994915008545, + 2.0, + 2.0, + 0.2008315920829773, + 0.09476469457149506, + 0.05857282876968384, + 0.10009106248617172, + 0.07648500055074692, + 0.05889680236577988, + 0.014406224712729454, + 0.09262995421886444, + 0.15997587144374847, + 0.04682888835668564, + 0.09751574695110321, + 0.06459556519985199, + 0.20335909724235535 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7537296414375305, + 0.4158197343349457, + 1.4326180219650269, + -1.4107877016067505, + 2.9579341411590576, + 0.9669660925865173, + 1.4339183568954468, + 0.5987398624420166, + -0.2018759399652481, + 0.7105501890182495, + -0.03802938759326935, + 1.3799947500228882, + -0.6568997502326965, + -0.37473973631858826, + -1.857657790184021, + 0.038770925253629684, + 1.931185007095337, + 0.6317195296287537, + 0.1380508691072464, + 2.954265832901001, + 2.9641003608703613 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8678367137908936, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.7494794726371765, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.928200721740723 + ], + [ + 0.6226351857185364, + 0.34334811568260193, + 1.492361068725586, + -1.5917733907699585, + 3.1334922313690186, + 1.0313001871109009, + 1.2075668573379517, + 0.3046378493309021, + -0.11550238728523254, + 0.4499495327472687, + 0.16646838188171387, + 1.5877203941345215, + -0.6488803625106812, + -0.5760968327522278, + -1.8349132537841797, + 0.14188142120838165, + 2.0770444869995117, + 0.5806879997253418, + 0.23913151025772095, + 2.8127381801605225, + 3.0649962425231934 + ], + [ + 0.5975998044013977, + 0.722756028175354, + 1.7548110485076904, + -1.166775107383728, + 2.6060047149658203, + 0.6675758361816406, + 1.125187635421753, + 0.3373170793056488, + -0.1842741072177887, + 0.8186168670654297, + -0.31481224298477173, + 1.2082113027572632, + -0.9804694652557373, + -0.14131735265254974, + -1.700500249862671, + 0.07075637578964233, + 1.5488730669021606, + 0.549576461315155, + 0.034750498831272125, + 3.227431297302246, + 2.9464566707611084 + ], + [ + 0.844982922077179, + 0.7132244110107422, + 1.4430484771728516, + -1.3331272602081299, + 3.1703803539276123, + 1.1851513385772705, + 1.355688452720642, + 0.6115275025367737, + 0.2616361379623413, + 1.2030918598175049, + 0.05625858157873154, + 1.6050300598144531, + -0.7155078053474426, + -0.021488159894943237, + -2.073806047439575, + 0.5387547016143799, + 1.6223118305206299, + 0.7140727043151855, + -0.06027664244174957, + 2.7503156661987305, + 2.6344075202941895 + ], + [ + 0.5821706056594849, + 0.78206866979599, + 1.8119354248046875, + -0.8252442479133606, + 3.038909435272217, + 1.3900940418243408, + 1.7462186813354492, + 0.005083680152893066, + -0.27262064814567566, + 0.867202877998352, + -0.5555376410484314, + 1.172202706336975, + -1.2016760110855103, + -0.06355220079421997, + -2.2925045490264893, + 0.6078628301620483, + 1.7895314693450928, + 0.8720078468322754, + 0.472573459148407, + 3.377012014389038, + 2.6039178371429443 + ], + [ + 0.3789674639701843, + 0.3270051181316376, + 1.056584358215332, + -2.0788497924804688, + 2.948819875717163, + 1.6022355556488037, + 0.8599810004234314, + 0.9570736885070801, + -0.6380544900894165, + 0.3103100061416626, + -0.42896610498428345, + 1.7150276899337769, + -0.8975871205329895, + -0.278381884098053, + -1.305256724357605, + 0.3637259304523468, + 1.8004443645477295, + 0.16446983814239502, + 0.19097819924354553, + 3.43778657913208, + 3.538135528564453 + ], + [ + 0.6903814673423767, + 1.1327197551727295, + 1.8523926734924316, + -2.0428731441497803, + 3.1870431900024414, + 0.5352929830551147, + 0.6004904508590698, + 1.2120530605316162, + 0.16280107200145721, + 1.119707465171814, + 0.7612010836601257, + 0.7890661954879761, + -1.3447637557983398, + -0.1703537553548813, + -1.1010353565216064, + 0.32831913232803345, + 1.5534476041793823, + -0.2119135856628418, + -0.562728762626648, + 2.4702467918395996, + 2.5528204441070557 + ], + [ + 0.9177048206329346, + -0.04751536250114441, + 1.4819984436035156, + -2.1542677879333496, + 3.522390842437744, + 1.8177759647369385, + 2.2213237285614014, + 0.8223017454147339, + -0.1973809152841568, + -0.053272128105163574, + -0.5862784385681152, + 0.6421516537666321, + -0.8713911771774292, + -0.9495017528533936, + -1.6454768180847168, + -0.38891172409057617, + 1.648432970046997, + 1.4951133728027344, + 0.5893608331680298, + 2.449934482574463, + 3.419659376144409 + ], + [ + 1.2956775426864624, + -0.3119873106479645, + 1.5930066108703613, + -1.7682520151138306, + 2.4385876655578613, + 1.4679458141326904, + 1.4536705017089844, + 0.31068816781044006, + 0.660710871219635, + 0.8104690909385681, + 0.483506441116333, + 2.1112046241760254, + -0.22927385568618774, + 0.17858651280403137, + -2.485126495361328, + -0.9187790155410767, + 1.2810213565826416, + 1.3727718591690063, + 0.44044917821884155, + 3.4018898010253906, + 3.39054799079895 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0012856721878051758, + 0.00795695185661316, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + 0.11055737733840942, + 0.0, + 0.02143383026123047, + 0.0025591105222702026, + -0.030314385890960693, + -0.0125969760119915, + 0.3799947500228882, + 0.0076233744621276855, + 0.0, + -0.857657790184021, + 0.013764806091785431, + 0.9311850070953369, + 0.07110422849655151, + 0.03423507511615753, + 1.954265832901001, + 0.0 + ], + [ + -0.24627035856246948, + -0.16095152497291565, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + -0.023127973079681396, + 0.0, + -0.24312204122543335, + 0.1389055997133255, + 0.11457735300064087, + 0.06618259102106094, + 0.3799947500228882, + 0.07616078853607178, + 0.0, + -0.857657790184021, + 0.09948430210351944, + 0.9311850070953369, + 0.006249547004699707, + -0.33439844846725464, + 1.954265832901001, + 0.0 + ] + ], + "predicted_scores": [ + 2.724388599395752, + 2.2137644290924072, + 1.8454216718673706, + 1.8319257497787476, + 1.8158799409866333, + 1.7394673824310303, + 1.7313783168792725, + 1.690064549446106 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "534717141/train2", + "selected_index": 0, + "state_hash": "5318995ba859471b8678d6594274c7c0565a1ccfd83ea302378cc2884dc5cc32", + "stored_base_utility": 0.20931994915008545, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14809735119342804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14809735119342804 + }, + "base_progress": 0.14809735119342804, + "base_success": false, + "base_utility": 0.14809735119342804, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.002109348773956299, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.002109348773956299 + }, + { + "contact_quality": null, + "progress": 0.0016142526874318719, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0016142526874318719 + }, + { + "contact_quality": null, + "progress": 0.001568953157402575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001568953157402575 + }, + { + "contact_quality": null, + "progress": 0.0013637542724609375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013637542724609375 + }, + { + "contact_quality": null, + "progress": 0.0013946691760793328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013946691760793328 + }, + { + "contact_quality": null, + "progress": 0.0013481974601745605, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013481974601745605 + }, + { + "contact_quality": null, + "progress": 0.0006553332204930484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006553332204930484 + }, + { + "contact_quality": null, + "progress": 0.0024978716392070055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0024978716392070055 + } + ], + "candidate_progress": [ + 0.002109348773956299, + 0.0016142526874318719, + 0.001568953157402575, + 0.0013637542724609375, + 0.0013946691760793328, + 0.0013481974601745605, + 0.0006553332204930484, + 0.0024978716392070055 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-f2b2b2a1aa2863c3676f", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-da142025733b53fcf7eb", + "generated_tangents": [ + [ + -0.16491813957691193, + -0.44812941551208496, + 1.522404432296753, + -0.29195141792297363, + 1.6155595779418945, + 3.036224126815796, + 2.4263293743133545, + -0.11942023038864136, + 0.156746506690979, + 1.2404955625534058, + 0.30534762144088745, + 0.9937582612037659, + 1.7859283685684204, + 1.3082846403121948, + -1.1007142066955566, + -0.2226385921239853, + 1.4320591688156128, + 0.6443809866905212, + -0.10069802403450012, + 2.5378925800323486, + 0.788082480430603 + ], + [ + -0.25906503200531006, + -0.4311758279800415, + 1.550649642944336, + -0.10462211817502975, + 1.5163681507110596, + 3.0287973880767822, + 1.9991042613983154, + -0.08224475383758545, + 0.13758185505867004, + 1.187612533569336, + 0.1380847841501236, + 1.0298229455947876, + 1.7279993295669556, + 1.705025315284729, + -1.2922234535217285, + -0.11821980774402618, + 1.7057580947875977, + 0.9846886992454529, + 0.40484148263931274, + 1.9227759838104248, + 1.367584466934204 + ], + [ + -0.3099903464317322, + -0.22394734621047974, + 1.6123816967010498, + -0.0927695482969284, + 2.086507797241211, + 3.3660175800323486, + 1.9148619174957275, + -0.24106869101524353, + 0.11848428845405579, + 1.1820400953292847, + 0.14725540578365326, + 0.9857961535453796, + 1.5172349214553833, + 1.7305700778961182, + -1.1125696897506714, + -0.21418099105358124, + 1.6485706567764282, + 0.9478051662445068, + 0.3282330632209778, + 2.321153163909912, + 1.6825212240219116 + ], + [ + -0.4207296371459961, + -0.34084856510162354, + 1.6467245817184448, + 0.06274494528770447, + 1.2211084365844727, + 3.5212364196777344, + 1.7665579319000244, + -0.32972094416618347, + 0.17988735437393188, + 0.9871339797973633, + 0.25226306915283203, + 1.225220799446106, + 1.9555734395980835, + 1.8543727397918701, + -1.2462942600250244, + -0.24397948384284973, + 1.7886381149291992, + 0.6570941805839539, + 0.3053135573863983, + 2.7333998680114746, + 1.375671148300171 + ], + [ + -0.4197947382926941, + -0.41934341192245483, + 1.5252076387405396, + -0.10965941101312637, + 1.530900001525879, + 3.3226230144500732, + 1.6510274410247803, + -0.26396510004997253, + 0.15492278337478638, + 1.2087863683700562, + 0.09346982091665268, + 0.9122577905654907, + 1.489465355873108, + 1.5815447568893433, + -1.225317120552063, + -0.16869446635246277, + 1.6537623405456543, + 0.7922667860984802, + 0.24121429026126862, + 2.372321844100952, + 1.6607167720794678 + ], + [ + -0.4949355125427246, + -0.41885125637054443, + 1.547715425491333, + 0.06952869892120361, + 1.609220266342163, + 3.6366617679595947, + 1.5721161365509033, + -0.20115438103675842, + 0.13943473994731903, + 1.2018177509307861, + 0.24125489592552185, + 0.9632784128189087, + 1.3610948324203491, + 1.7838205099105835, + -1.24807870388031, + -0.2722555100917816, + 1.6444717645645142, + 0.823582649230957, + 0.42304664850234985, + 2.1840851306915283, + 1.7703300714492798 + ], + [ + -0.4479713439941406, + -0.3628988265991211, + 1.6345138549804688, + 0.14003247022628784, + 1.279918909072876, + 3.654670238494873, + 1.973222017288208, + -0.5072991847991943, + 0.24864208698272705, + 1.4242440462112427, + 0.22237750887870789, + 1.0343232154846191, + 1.5170308351516724, + 1.8595339059829712, + -1.446240782737732, + -0.4530695378780365, + 1.9973170757293701, + 1.062402367591858, + 0.13227444887161255, + 2.6164281368255615, + 1.4536553621292114 + ], + [ + -0.2238340526819229, + -0.27119576930999756, + 1.3281177282333374, + -0.07311345636844635, + 1.2926609516143799, + 2.861276626586914, + 1.480619192123413, + -0.04855930805206299, + 0.1650649607181549, + 1.0754681825637817, + 0.6095993518829346, + 0.8284340500831604, + 1.1569561958312988, + 2.1539785861968994, + -1.074147343635559, + -0.17524687945842743, + 1.7277039289474487, + 0.9363681674003601, + 0.37941500544548035, + 2.6101033687591553, + 1.175939917564392 + ] + ], + "generated_utilities": [ + 0.002109348773956299, + 0.0016142526874318719, + 0.001568953157402575, + 0.0013637542724609375, + 0.0013946691760793328, + 0.0013481974601745605, + 0.0006553332204930484, + 0.0024978716392070055 + ], + "group_id": "ms-da142025733b53fcf7eb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.14843207597732544, + 0.1717694252729416, + 0.09356185048818588, + 0.23231086134910583, + 0.19341367483139038, + 0.04822012037038803, + 0.23024770617485046, + 0.16561973094940186, + 0.2643269896507263, + 0.12403462827205658, + 0.12689702212810516, + 0.16619956493377686, + 0.13481859862804413, + 0.07537531852722168, + 0.11867459863424301, + 0.04740651696920395 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.04848463833332062, + -0.18164527416229248, + 0.6558789014816284, + 0.038211286067962646, + 0.08492374420166016, + 2.265157699584961, + 0.0, + -0.09011825919151306, + 0.0018594712018966675, + 0.6884076595306396, + 0.027596401050686836, + 0.06117302179336548, + 1.278116226196289, + 0.0, + -0.3951960802078247, + -0.08061015605926514, + 1.07906174659729, + 0.3229694366455078, + -0.21076995134353638, + 1.8564467430114746, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0903544425964355, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.570276260375977, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.736253261566162 + ], + [ + 0.08095964789390564, + -0.7926032543182373, + 1.571103811264038, + 0.07894467562437057, + 0.2701656222343445, + 3.806621789932251, + 2.203669786453247, + -0.042464032769203186, + 0.7168105244636536, + 1.8671371936798096, + -0.30838707089424133, + 1.7242767810821533, + 1.8847527503967285, + 1.305934190750122, + -0.42665380239486694, + -0.2973874807357788, + 1.1881210803985596, + 1.6921244859695435, + 1.070060133934021, + 2.067784309387207, + 1.916501760482788 + ], + [ + 0.7620636224746704, + -2.1605899333953857, + 2.1443262100219727, + 0.17886832356452942, + 1.6259945631027222, + 3.787412166595459, + 1.3663169145584106, + 0.49799659848213196, + -0.6301277279853821, + 2.275850534439087, + -0.6735645532608032, + 1.7504758834838867, + 2.7512195110321045, + 1.7462939023971558, + -0.6835377216339111, + 0.36623528599739075, + 1.405979871749878, + 0.6472548842430115, + 1.4791045188903809, + 2.7838869094848633, + 3.0856707096099854 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.29792410135269165, + -0.30587875843048096, + 0.6558789014816284, + -0.4171115756034851, + -0.1733894944190979, + 2.265157699584961, + 0.0, + 0.14607194066047668, + 0.05651267617940903, + 0.6884076595306396, + -0.0433836504817009, + 0.07522618770599365, + 1.278116226196289, + 0.0, + -0.41462820768356323, + 0.26706579327583313, + 1.07906174659729, + 0.44956159591674805, + -0.37072819471359253, + 1.8564467430114746, + 0.0 + ], + [ + 0.18432100117206573, + -0.9473621249198914, + 1.5389783382415771, + -0.041053857654333115, + 0.6282865405082703, + 3.276425838470459, + 2.1839330196380615, + -0.3612370491027832, + 0.23081421852111816, + 1.8094429969787598, + 0.10581490397453308, + 0.8865737915039062, + 2.5146288871765137, + 2.2134878635406494, + -1.6751070022583008, + -0.08268362283706665, + 2.303063154220581, + 1.3061904907226562, + 0.4770132899284363, + 3.109205722808838, + 2.509927272796631 + ], + [ + -0.5288646817207336, + -1.5067847967147827, + 1.376499056816101, + 0.13231416046619415, + 1.048660159111023, + 3.5048489570617676, + 1.9678237438201904, + -0.14848726987838745, + 0.05475667119026184, + 1.5529061555862427, + 0.1597936749458313, + 0.7318928241729736, + 2.17749285697937, + 2.0782582759857178, + -1.5473355054855347, + -0.6715520620346069, + 1.7043644189834595, + 1.1190917491912842, + 0.4214729070663452, + 3.3400330543518066, + 2.3147284984588623 + ] + ], + "predicted_scores": [ + 1.2798094749450684, + 1.2566577196121216, + 1.0953518152236938, + 0.5736859440803528, + 0.4687597155570984, + 0.43638235330581665, + -0.01489083468914032, + -0.2645830512046814 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615/train2", + "selected_index": 0, + "state_hash": "0026cc5b3762b520baadbda7bf138a2d24fffa5b2306f245ddbc1d1556cccc22", + "stored_base_utility": 0.14843207597732544, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1362423449754715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1362423449754715 + }, + "base_progress": 0.1362423449754715, + "base_success": false, + "base_utility": 0.1362423449754715, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01934593915939331, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01934593915939331 + }, + { + "contact_quality": null, + "progress": 0.018479665741324425, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018479665741324425 + }, + { + "contact_quality": null, + "progress": 0.014796634204685688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014796634204685688 + }, + { + "contact_quality": null, + "progress": 0.016739627346396446, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016739627346396446 + }, + { + "contact_quality": null, + "progress": 0.011937320232391357, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011937320232391357 + }, + { + "contact_quality": null, + "progress": 0.009798427112400532, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009798427112400532 + }, + { + "contact_quality": null, + "progress": 0.04993142560124397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04993142560124397 + }, + { + "contact_quality": null, + "progress": 0.04576842114329338, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04576842114329338 + } + ], + "candidate_progress": [ + 0.01934593915939331, + 0.018479665741324425, + 0.014796634204685688, + 0.016739627346396446, + 0.011937320232391357, + 0.009798427112400532, + 0.04993142560124397, + 0.04576842114329338 + ], + "candidate_source_chart_ids": [ + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-a2518826fa0a816cc46f", + "ms-cf4b63d109a88a4c42bf", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-e2bad6e99e8a281207cc", + "generated_tangents": [ + [ + -0.808625340461731, + -0.3958311676979065, + 1.4513057470321655, + 0.10015183687210083, + 1.8970415592193604, + 3.1691267490386963, + 1.6807830333709717, + 0.496963232755661, + -0.593037486076355, + 0.3745551109313965, + 0.04373528063297272, + 0.5483255982398987, + -0.059080034494400024, + -0.17493756115436554, + 0.0818648487329483, + 0.27815714478492737, + 0.11154191941022873, + -1.379705786705017, + 0.6501579284667969, + -2.061927080154419, + -1.9550199508666992 + ], + [ + -0.7248064279556274, + -0.4604642987251282, + 1.5598711967468262, + 0.005166083574295044, + 1.815105676651001, + 3.014575481414795, + 1.7203545570373535, + 0.42555975914001465, + -0.6206048727035522, + 0.3805982768535614, + 0.001539815217256546, + 0.6111949682235718, + -0.14247506856918335, + 0.04190334677696228, + 0.14704649150371552, + 0.2566608488559723, + 0.07439719140529633, + -1.3993266820907593, + 0.6737163066864014, + -1.9276411533355713, + -1.82131826877594 + ], + [ + -0.40650177001953125, + -0.49699872732162476, + 1.4829405546188354, + 0.13907760381698608, + 1.8399548530578613, + 3.485158920288086, + 1.5967557430267334, + 0.41830646991729736, + -0.5379894375801086, + 0.3360911011695862, + 0.09948419779539108, + 0.812816858291626, + 0.22150996327400208, + 0.03305993974208832, + 0.06606549769639969, + 0.23715467751026154, + 0.42798539996147156, + -1.5503917932510376, + 1.3109371662139893, + -1.9463622570037842, + -1.5908902883529663 + ], + [ + -0.6315448880195618, + -0.4405671954154968, + 1.6896828413009644, + -0.003717586398124695, + 1.7352089881896973, + 3.074761152267456, + 1.769089698791504, + 0.4281931519508362, + -0.517143726348877, + 0.36459216475486755, + -0.03713090717792511, + 0.5744054913520813, + -0.23416925966739655, + 0.06707154214382172, + 0.1249435693025589, + 0.22939813137054443, + 0.18297842144966125, + -1.3903321027755737, + 0.5762546062469482, + -1.9213354587554932, + -2.0142428874969482 + ], + [ + -0.6389257311820984, + -0.443348228931427, + 1.5659390687942505, + -0.24717013537883759, + 2.3528077602386475, + 3.363032817840576, + 1.5654137134552002, + 0.32284098863601685, + -0.2637955844402313, + 0.5328659415245056, + 0.2923922538757324, + 0.7081804275512695, + -0.3174516558647156, + 0.1322580873966217, + 0.31917423009872437, + -0.058363039046525955, + 0.27812865376472473, + -1.2366472482681274, + 0.4684528112411499, + -1.9222798347473145, + -1.19365656375885 + ], + [ + -0.538007915019989, + -0.3468935489654541, + 1.5260671377182007, + -0.30296847224235535, + 1.7538366317749023, + 3.3511900901794434, + 1.2871084213256836, + -0.11565133929252625, + -0.16260549426078796, + 0.7333114147186279, + 0.0004078671336174011, + 0.3794543743133545, + -1.0234380960464478, + 0.5007558465003967, + 0.5275300145149231, + 0.27768683433532715, + 0.3729734718799591, + -1.0702588558197021, + -0.10718375444412231, + -1.9243483543395996, + -1.7530497312545776 + ], + [ + -1.092842936515808, + -0.3846888542175293, + 1.2500170469284058, + -0.7387838363647461, + 2.329988479614258, + 3.2617266178131104, + 2.0534462928771973, + 0.7335795164108276, + -0.32580500841140747, + 0.10699981451034546, + 0.121836356818676, + 0.4568679630756378, + -0.5413341522216797, + 0.11316933482885361, + 0.13814303278923035, + 0.15308858454227448, + 0.33005163073539734, + -1.6212522983551025, + 0.009338468313217163, + -1.193364143371582, + -1.317259669303894 + ], + [ + -1.0184271335601807, + -0.3951433300971985, + 1.2153401374816895, + -0.7337980270385742, + 2.3822622299194336, + 3.210197925567627, + 2.015505075454712, + 0.7741807699203491, + -0.30517974495887756, + 0.10034145414829254, + 0.2205776870250702, + 0.3355855345726013, + -0.5925508141517639, + 0.2985408902168274, + 0.11383157968521118, + 0.17892028391361237, + 0.2655179798603058, + -1.67020845413208, + 0.06308495998382568, + -1.019197702407837, + -1.18149733543396 + ] + ], + "generated_utilities": [ + 0.01934593915939331, + 0.018479665741324425, + 0.014796634204685688, + 0.016739627346396446, + 0.011937320232391357, + 0.009798427112400532, + 0.04993142560124397, + 0.04576842114329338 + ], + "group_id": "ms-e2bad6e99e8a281207cc", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13993772864341736, + 0.16482847929000854, + 0.20825156569480896, + 0.16188287734985352, + 0.13454246520996094, + 0.14753298461437225, + 0.14504572749137878, + 0.06676968187093735, + 0.09772863239049911, + 0.0585557222366333, + 0.12863430380821228, + 0.10292482376098633, + 0.07993738353252411, + 0.03641533851623535, + 0.04985213279724121, + 0.1516878604888916 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.3249012231826782, + -0.9610954523086548, + 2.054325819015503, + 1.018629789352417, + 2.393625497817993, + 2.2159433364868164, + 2.098443031311035, + -0.49140021204948425, + 0.044472046196460724, + 0.8045110702514648, + 1.4511574506759644, + -0.3077920377254486, + -0.052587300539016724, + 0.036380887031555176, + 0.7875679731369019, + -0.3928021192550659, + -0.09456783533096313, + -1.4901847839355469, + 1.1711406707763672, + -2.1349995136260986, + -2.076193332672119 + ], + [ + -1.4271517992019653, + -1.114187479019165, + 1.618905782699585, + 1.0621737241744995, + 1.6005314588546753, + 3.017716407775879, + 2.3444156646728516, + -1.0368298292160034, + 0.6155713796615601, + 1.0049463510513306, + 1.3708372116088867, + -0.27396243810653687, + -0.19098177552223206, + -0.5735776424407959, + 0.863328218460083, + -0.4340485632419586, + 0.09502258896827698, + -1.8856427669525146, + 0.9514620304107666, + -2.3912711143493652, + -2.6571128368377686 + ], + [ + -1.0088777542114258, + -1.803802251815796, + 2.6037826538085938, + 0.763943612575531, + 1.7241847515106201, + 2.433204174041748, + 2.273603916168213, + 0.06359493732452393, + -0.17274422943592072, + -0.23299545049667358, + 1.7126038074493408, + -0.3797045052051544, + 0.7567765712738037, + -0.9981052875518799, + 1.0371832847595215, + -0.5120593309402466, + -0.059397101402282715, + -1.9757622480392456, + 0.7378945350646973, + -2.6195693016052246, + -2.178041458129883 + ], + [ + -0.09093856811523438, + -0.14735746383666992, + 1.350311517715454, + 1.477137565612793, + 2.134960174560547, + 1.7331886291503906, + 2.24615740776062, + -1.7148966789245605, + -0.3949584364891052, + 0.4433448612689972, + 0.9034961462020874, + -0.23445925116539001, + -0.6371403932571411, + -0.038229942321777344, + 2.010648250579834, + 0.14472149312496185, + -0.26303571462631226, + -2.3003270626068115, + 0.565727949142456, + -2.586710214614868, + -2.098479747772217 + ], + [ + -1.8506121635437012, + -1.4700038433074951, + 2.4313056468963623, + 1.6264371871948242, + 1.1548577547073364, + 2.9678268432617188, + 2.990251302719116, + -0.25485146045684814, + -0.36889538168907166, + -0.14847010374069214, + 0.6971698999404907, + -0.6752342581748962, + 0.44545215368270874, + 0.663374125957489, + 0.6998850107192993, + -0.6500043869018555, + -0.04876291751861572, + -2.446995258331299, + 0.7760334014892578, + -1.38918137550354, + -1.5372083187103271 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.053252458572387695, + -0.006605625152587891, + 1.0103552341461182, + 0.25736117362976074, + 1.010573148727417, + 1.4690883159637451, + 0.0, + -0.0840916633605957, + -0.24044206738471985, + -0.06801849603652954, + 0.061444878578186035, + 0.11697901785373688, + 0.03979811072349548, + 0.0, + 0.054485201835632324, + -0.11758939921855927, + 0.31161248683929443, + -0.602575421333313, + -0.021049559116363525, + -1.3257782459259033, + 0.0 + ] + ], + "predicted_scores": [ + 3.989610195159912, + 3.801568031311035, + 3.771143674850464, + 3.5158884525299072, + 2.6187055110931396, + 2.271235466003418, + 1.9595327377319336, + 1.9518026113510132 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805/train2", + "selected_index": 0, + "state_hash": "f054ce13f25dcda8808ebde2bee98ca84bfbdc3fc328a0eb831e21faada3fa4c", + "stored_base_utility": 0.13993772864341736, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09695577621459961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09695577621459961 + }, + "base_progress": 0.09695577621459961, + "base_success": false, + "base_utility": 0.09695577621459961, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.027097145095467567, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027097145095467567 + }, + { + "contact_quality": null, + "progress": 0.02346259355545044, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02346259355545044 + }, + { + "contact_quality": null, + "progress": 0.04263921454548836, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04263921454548836 + }, + { + "contact_quality": null, + "progress": 0.030199527740478516, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030199527740478516 + }, + { + "contact_quality": null, + "progress": 0.02752045728266239, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02752045728266239 + }, + { + "contact_quality": null, + "progress": 0.027290603145956993, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027290603145956993 + }, + { + "contact_quality": null, + "progress": 0.021321892738342285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021321892738342285 + }, + { + "contact_quality": null, + "progress": 0.03452460095286369, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03452460095286369 + } + ], + "candidate_progress": [ + 0.027097145095467567, + 0.02346259355545044, + 0.04263921454548836, + 0.030199527740478516, + 0.02752045728266239, + 0.027290603145956993, + 0.021321892738342285, + 0.03452460095286369 + ], + "candidate_source_chart_ids": [ + "ms-206099bab6bfe7c84fce", + "ms-6e27d292a9cbea108fc9", + "ms-6e27d292a9cbea108fc9", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e30fd80adc089e9f697a", + "generated_tangents": [ + [ + 0.3590228855609894, + 0.19683429598808289, + -0.27843567728996277, + 0.6162688732147217, + 0.5412493944168091, + -0.801476240158081, + -1.189948320388794, + 0.00704619288444519, + 0.04275801032781601, + 0.010225243866443634, + 0.9955396056175232, + 0.3697530925273895, + -0.13494029641151428, + 0.037109315395355225, + -0.3661476969718933, + 0.08389641344547272, + 0.3658115267753601, + -0.05796189606189728, + -0.08679118752479553, + 1.0497918128967285, + 0.28130897879600525 + ], + [ + 0.35214918851852417, + 0.19684278964996338, + -0.23120197653770447, + 0.6600933074951172, + 0.6702309250831604, + -1.220144510269165, + -0.4723138213157654, + -0.021933957934379578, + 0.15916015207767487, + 0.03964683413505554, + 0.8672034740447998, + 0.04386952519416809, + 0.056779175996780396, + -0.014775574207305908, + -0.3643971085548401, + 0.028458911925554276, + 0.15976694226264954, + 0.07386019825935364, + -0.35446876287460327, + 1.2123723030090332, + 0.36295968294143677 + ], + [ + 0.387315034866333, + 0.17117254436016083, + -0.18575212359428406, + 0.56825852394104, + 0.7309894561767578, + -0.774138331413269, + -0.5184234976768494, + 0.0943412333726883, + 0.03323892503976822, + 0.10794530808925629, + 0.9891995787620544, + 0.020437508821487427, + 0.3686356246471405, + 0.3178538680076599, + -0.6662771105766296, + 0.03811803087592125, + 0.28423410654067993, + 0.26532724499702454, + -0.41444945335388184, + 1.1595392227172852, + 0.3615068197250366 + ], + [ + 0.4227678179740906, + 0.3649612069129944, + -0.14785698056221008, + 0.3151308000087738, + 0.7339253425598145, + -0.6750684976577759, + -0.26398754119873047, + -0.16428720951080322, + -0.05128262937068939, + 0.5043823719024658, + 0.8338589072227478, + 0.3655779957771301, + 0.9708422422409058, + 0.8090587258338928, + -0.21247264742851257, + -0.0019111288711428642, + 0.06804615259170532, + 0.17872658371925354, + -0.47385716438293457, + 1.0519821643829346, + 0.6560294032096863 + ], + [ + 0.4932646155357361, + 0.2583072781562805, + -0.13222184777259827, + 0.2968616783618927, + 0.6814801096916199, + -0.9484403133392334, + -0.9983848929405212, + 0.011778086423873901, + 0.016037452965974808, + 0.31290170550346375, + 0.7360706329345703, + 0.3139982223510742, + 1.1989713907241821, + 0.7084675431251526, + -0.1079864501953125, + 0.12537041306495667, + 0.162700355052948, + 0.22526514530181885, + -0.40702927112579346, + 0.9055491089820862, + 0.9361089468002319 + ], + [ + 0.5312609672546387, + 0.18521516025066376, + -0.08911815285682678, + 0.43024638295173645, + 0.895797073841095, + -0.813001275062561, + -1.119739294052124, + 0.03544671833515167, + 0.04468908905982971, + 0.33778032660484314, + 0.656747043132782, + 0.20835548639297485, + 1.2784655094146729, + 0.8251712322235107, + -0.028265830129384995, + 0.13416336476802826, + 0.2592344582080841, + 0.29410091042518616, + -0.394891619682312, + 0.7114847302436829, + 1.0665563344955444 + ], + [ + 0.5456377267837524, + 0.3053399920463562, + -0.2471875250339508, + 0.2321479320526123, + 0.5717474222183228, + -0.9433281421661377, + -0.907658040523529, + 0.08428370207548141, + 0.011065405793488026, + 0.27069997787475586, + 0.6413487792015076, + 0.5000815391540527, + 1.2367584705352783, + 0.6818474531173706, + -0.11301298439502716, + 0.07140019536018372, + 0.11339274048805237, + 0.23209679126739502, + -0.4480735659599304, + 0.9108291268348694, + 0.8850172758102417 + ], + [ + 0.431556761264801, + 0.22277767956256866, + -0.146419495344162, + 0.18227934837341309, + 0.5816071033477783, + -0.9192342758178711, + -1.0169851779937744, + 0.11149691045284271, + -0.018793081864714622, + 0.2590688169002533, + 0.9602459669113159, + 0.4354865550994873, + 1.364359974861145, + 0.503301739692688, + -0.2617972195148468, + 0.09783723950386047, + 0.21418017148971558, + 0.13097012042999268, + -0.2826020121574402, + 1.146355152130127, + 0.9910222291946411 + ] + ], + "generated_utilities": [ + 0.027097145095467567, + 0.02346259355545044, + 0.04263921454548836, + 0.030199527740478516, + 0.02752045728266239, + 0.027290603145956993, + 0.021321892738342285, + 0.03452460095286369 + ], + "group_id": "ms-e30fd80adc089e9f697a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.12557005882263184, + 0.11941087245941162, + 0.10894306749105453, + 0.060310762375593185, + 0.18920521438121796, + 0.11678676307201385, + 0.14083251357078552, + 0.12988364696502686, + 0.18350017070770264, + 0.1286114901304245, + 0.06744732707738876, + 0.04428015276789665, + 0.08399941772222519, + 0.0742691159248352, + 0.05598282814025879, + 0.09703926742076874 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.044197678565979004, + -0.12298870831727982, + -0.37176433205604553, + 0.5503368377685547, + 0.5411782264709473, + 0.6186751127243042, + 0.0, + 0.28674131631851196, + 0.08053566515445709, + 0.3295105993747711, + 0.8157306909561157, + 0.1715114712715149, + -0.43090736865997314, + 0.0, + -0.5639103651046753, + -0.48903688788414, + 0.7485904693603516, + -0.4474331736564636, + -0.18787053227424622, + 1.2062294483184814, + 0.0 + ], + [ + 1.3153167963027954, + 0.7455203533172607, + -1.0113484859466553, + 2.097999334335327, + 0.7455009818077087, + 0.3843202590942383, + -0.16713692247867584, + -0.8400754928588867, + -0.3845864534378052, + 1.065396785736084, + 1.712794542312622, + 0.803345263004303, + 0.6529266238212585, + 0.6022698879241943, + -1.1222561597824097, + 0.06260442733764648, + 1.101211428642273, + -0.133493572473526, + -0.8626402616500854, + 1.8406728506088257, + 0.8011860847473145 + ], + [ + 0.23532289266586304, + -0.516789972782135, + -0.3119599223136902, + 1.501338005065918, + 1.4930639266967773, + -0.419913113117218, + -0.6910054683685303, + 0.30246037244796753, + 0.041916340589523315, + 1.2685117721557617, + 2.5912232398986816, + 1.2864888906478882, + 0.3821069598197937, + 0.4677737355232239, + -1.84663724899292, + -0.8953573703765869, + 2.3256053924560547, + 0.18948839604854584, + -0.5580936670303345, + 1.528401494026184, + 1.4914395809173584 + ], + [ + 0.6903802752494812, + 0.3962534964084625, + -0.8207939863204956, + 0.5973623991012573, + 1.310564398765564, + 0.6411572694778442, + -0.2032768577337265, + 0.25504088401794434, + 0.6581551432609558, + 0.3938753306865692, + 1.0127826929092407, + 1.6796927452087402, + 0.4867250621318817, + 0.05769670009613037, + -2.026149034500122, + -0.17722094058990479, + 0.9414948225021362, + 0.6044259667396545, + 0.03290170431137085, + 2.9423327445983887, + -0.060306549072265625 + ], + [ + 0.47530579566955566, + -0.6085019111633301, + 0.12691906094551086, + 0.5673632025718689, + 0.08076304197311401, + 0.8255819082260132, + -0.9781221151351929, + -0.796129584312439, + -0.6745100021362305, + 0.7784937024116516, + 1.7613102197647095, + 1.4703174829483032, + -0.36175650358200073, + 0.16939619183540344, + -0.9399533271789551, + -1.2696547508239746, + 1.3314080238342285, + -0.3690394163131714, + -1.0323668718338013, + 2.305772304534912, + 0.27392566204071045 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.6770331859588623, + 0.09077263623476028, + -0.3141186833381653, + 1.5503368377685547, + 1.0133107900619507, + 0.12400172650814056, + -0.2399108111858368, + -0.4914711117744446, + -0.11917559802532196, + 0.7716305255889893, + 1.8157306909561157, + 0.7167991995811462, + 0.5690926313400269, + 0.6154549717903137, + -1.5639103651046753, + -0.5166093111038208, + 1.7485904693603516, + -0.15158627927303314, + -0.46638885140419006, + 2.2062294483184814, + 0.7902159690856934 + ], + [ + 0.9456368684768677, + -0.3964601159095764, + -0.5845282077789307, + 1.2133667469024658, + 0.5843623876571655, + -0.02685314416885376, + -0.006928771734237671, + -0.2159697711467743, + -0.3017333149909973, + 0.40500110387802124, + 1.9734654426574707, + 0.46303731203079224, + 0.33380845189094543, + 0.7758987545967102, + -2.060479164123535, + -0.955610990524292, + 1.942940592765808, + -0.23636659979820251, + -0.10507094860076904, + 1.8367226123809814, + 1.2234702110290527 + ] + ], + "predicted_scores": [ + 6.529160022735596, + 5.073872089385986, + 4.5338897705078125, + 4.301075458526611, + 3.954558849334717, + 3.819385290145874, + 3.693312644958496, + 3.690680980682373 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130/train2", + "selected_index": 0, + "state_hash": "ca8873454d7cf9723b1e743355bf255813544ae3212d80abc954fd347d7adf27", + "stored_base_utility": 0.12557005882263184, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1320568472146988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1320568472146988 + }, + { + "contact_quality": null, + "progress": 0.12680597603321075, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12680597603321075 + }, + { + "contact_quality": null, + "progress": 0.09484513849020004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09484513849020004 + }, + { + "contact_quality": null, + "progress": 0.10936561971902847, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10936561971902847 + }, + { + "contact_quality": null, + "progress": 0.11417748779058456, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11417748779058456 + }, + { + "contact_quality": null, + "progress": 0.06823223829269409, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06823223829269409 + }, + { + "contact_quality": null, + "progress": 0.1029265895485878, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1029265895485878 + }, + { + "contact_quality": null, + "progress": 0.0987953171133995, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0987953171133995 + } + ], + "candidate_progress": [ + 0.1320568472146988, + 0.12680597603321075, + 0.09484513849020004, + 0.10936561971902847, + 0.11417748779058456, + 0.06823223829269409, + 0.1029265895485878, + 0.0987953171133995 + ], + "candidate_source_chart_ids": [ + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "generated_tangents": [ + [ + 1.5705944299697876, + -0.06851592659950256, + -0.0494934618473053, + 0.141649067401886, + 0.3069194555282593, + 1.6204804182052612, + 1.7940168380737305, + 1.8458722829818726, + 0.12396884709596634, + -1.1453535556793213, + -0.4320434629917145, + -0.7400387525558472, + -3.26072096824646, + -2.046830654144287, + 1.4676625728607178, + 0.2676600515842438, + -0.8536955714225769, + -0.2154492288827896, + -0.33907508850097656, + -1.831056833267212, + -1.1493115425109863 + ], + [ + 1.8652468919754028, + -0.2339145839214325, + 0.17751935124397278, + 0.23989439010620117, + -0.0668235719203949, + 2.4834957122802734, + 1.7183148860931396, + 2.470745325088501, + 0.22536756098270416, + -1.5991404056549072, + -0.7004081010818481, + -0.5648285746574402, + -3.7447264194488525, + -2.515866994857788, + 1.5853447914123535, + 0.4201934337615967, + -0.581537127494812, + -0.1651209145784378, + -0.08740833401679993, + -1.26237154006958, + -1.3109184503555298 + ], + [ + 1.6519989967346191, + -0.22999686002731323, + 0.07272452116012573, + 0.17863675951957703, + 0.08630292117595673, + 2.0467116832733154, + 1.782813549041748, + 2.344526529312134, + 0.23404105007648468, + -1.7041015625, + -0.6769210696220398, + -0.682097315788269, + -3.5634939670562744, + -2.8932785987854004, + 1.603251338005066, + 0.3866955041885376, + -0.5121111273765564, + -0.3393757939338684, + -0.1352817416191101, + -1.216221570968628, + -1.4936425685882568 + ], + [ + 1.7114033699035645, + -0.24365893006324768, + 0.013003885746002197, + 0.09338992834091187, + -0.10904493927955627, + 2.304471254348755, + 1.6770527362823486, + 2.308021068572998, + 0.2794436812400818, + -1.54660964012146, + -0.7000381946563721, + -0.7109536528587341, + -3.6303679943084717, + -3.1097657680511475, + 1.5242034196853638, + 0.41823381185531616, + -0.5191966891288757, + -0.3860350251197815, + -0.07287967205047607, + -1.2147479057312012, + -1.6497353315353394 + ], + [ + 1.5470114946365356, + -0.19456952810287476, + 0.2033877670764923, + 0.06865578889846802, + -0.28358471393585205, + 1.8922683000564575, + 1.821535587310791, + 2.2622311115264893, + 0.1230999156832695, + -1.7442768812179565, + -0.6677217483520508, + -0.879165530204773, + -3.289560079574585, + -2.8239214420318604, + 1.560648798942566, + 0.2975308895111084, + -0.5076003074645996, + -0.11996276676654816, + -0.48666930198669434, + -1.3704237937927246, + -1.8300011157989502 + ], + [ + 1.7211253643035889, + -0.31464123725891113, + -0.25204816460609436, + 0.2328798770904541, + 0.4461529850959778, + 2.118340253829956, + 1.7182228565216064, + 2.352573871612549, + 0.29229557514190674, + -1.8145620822906494, + -0.6128995418548584, + -1.046897292137146, + -3.5329716205596924, + -2.6767611503601074, + 1.5114555358886719, + 0.43397367000579834, + -0.4783868193626404, + -0.34383583068847656, + 0.03508239984512329, + -1.3395237922668457, + -1.5731325149536133 + ], + [ + 1.5320279598236084, + -0.33009469509124756, + 0.29068851470947266, + -0.3612316846847534, + 0.2736901342868805, + 0.7711696624755859, + 2.45078706741333, + 1.858219861984253, + 0.18155181407928467, + -1.2958438396453857, + -0.677787184715271, + -0.911828875541687, + -3.1887199878692627, + -2.020345449447632, + 1.0366549491882324, + 0.27170655131340027, + -0.48277270793914795, + -0.322127103805542, + -0.4044927954673767, + -2.291778802871704, + -1.5043164491653442 + ], + [ + 1.6546272039413452, + -0.19025036692619324, + -0.1967904269695282, + 0.2030722200870514, + -0.020738422870635986, + 2.034555673599243, + 2.197772264480591, + 2.3154332637786865, + 0.26576685905456543, + -1.5636883974075317, + -0.7797420620918274, + -0.8226667642593384, + -3.454789876937866, + -2.9299166202545166, + 1.645264744758606, + 0.3283607065677643, + -0.43253177404403687, + -0.4816684126853943, + 0.002722412347793579, + -1.3148722648620605, + -1.615276575088501 + ] + ], + "generated_utilities": [ + 0.1320568472146988, + 0.12680597603321075, + 0.09484513849020004, + 0.10936561971902847, + 0.11417748779058456, + 0.06823223829269409, + 0.1029265895485878, + 0.0987953171133995 + ], + "group_id": "ms-e9ed5d488afdfd6449d1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.22621813416481018, + 0.26591962575912476, + 0.22478708624839783, + 0.074407659471035, + 0.3134628236293793, + 0.19173070788383484, + 0.2438776195049286, + 0.26689577102661133, + 0.24477621912956238, + 0.24051737785339355, + 0.2738616168498993, + 0.04947046563029289, + 2.0, + 0.04723116010427475, + 0.08354459702968597, + 0.06540697813034058 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.6524851322174072, + -0.023243099451065063, + -0.05794792249798775, + 0.1783304661512375, + -0.3767814338207245, + 1.4893875122070312, + 0.0, + 1.024657964706421, + 0.3163401186466217, + -0.49971193075180054, + 0.05114656686782837, + -0.13273626565933228, + -1.9796202182769775, + 0.0, + 0.818771481513977, + -0.13705189526081085, + -0.21289467811584473, + 0.3914462625980377, + -0.6477798819541931, + -1.0717217922210693, + 0.0 + ], + [ + 2.363477945327759, + -1.042197585105896, + -0.8369066119194031, + 0.17341986298561096, + 0.6040215492248535, + 1.9536304473876953, + 2.2788424491882324, + 1.5015904903411865, + -0.07916292548179626, + -0.6816228628158569, + -1.3893373012542725, + -0.2375274896621704, + -2.5662975311279297, + -2.7615673542022705, + 1.151218295097351, + 0.29418760538101196, + -0.4583754539489746, + -0.4555256962776184, + -0.6620294451713562, + -2.534846067428589, + -1.916935682296753 + ], + [ + 2.0104916095733643, + -0.539351224899292, + 0.13553066551685333, + 0.9507300853729248, + 0.6636125445365906, + 1.5649433135986328, + 2.0495975017547607, + 2.832026958465576, + 1.1830073595046997, + -1.6508865356445312, + -0.26773035526275635, + -0.3988863527774811, + -2.8235108852386475, + -2.2897520065307617, + 1.4448039531707764, + -0.5226297378540039, + -2.1615242958068848, + 0.8214395046234131, + -0.755327045917511, + -2.571093797683716, + -1.7457375526428223 + ], + [ + 2.166736602783203, + -0.3969564735889435, + 0.5171664357185364, + 0.12789201736450195, + -0.9176173806190491, + 1.8307013511657715, + 1.834255576133728, + 1.3594005107879639, + 0.13751044869422913, + -0.9923363327980042, + -1.136932611465454, + -1.2923126220703125, + -2.445897102355957, + -2.4779889583587646, + 2.6843042373657227, + 0.10015779733657837, + -2.0766196250915527, + 0.1269409954547882, + 0.16698843240737915, + -1.2709240913391113, + -2.7113101482391357 + ], + [ + 1.0476312637329102, + -1.2484432458877563, + -0.8652145862579346, + 0.8706620335578918, + 0.6091254353523254, + 1.9634168148040771, + 2.105522394180298, + 1.2906999588012695, + -0.3275601267814636, + -1.0110361576080322, + -1.6514918804168701, + -1.376278281211853, + -2.345550060272217, + -2.4931728839874268, + 1.00270676612854, + 0.04506785422563553, + -0.9420130252838135, + -0.6248484253883362, + -1.469360589981079, + -3.059006929397583, + -3.495709180831909 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.6524851322174072, + -0.3197403848171234, + -0.07008056342601776, + 0.38692954182624817, + -0.013657404109835625, + 2.4893875122070312, + 2.3340368270874023, + 2.024657964706421, + 0.35216206312179565, + -1.4125897884368896, + -0.8262880444526672, + -0.49936625361442566, + -2.9796202182769775, + -2.9234721660614014, + 1.818771481513977, + 0.09626653790473938, + -1.2128946781158447, + 0.10123217105865479, + -0.7020833492279053, + -2.0717217922210693, + -2.5944693088531494 + ], + [ + 0.9812780022621155, + 0.2612687647342682, + 0.8081930875778198, + 0.3689844310283661, + -0.16453352570533752, + 2.8411240577697754, + 2.2401933670043945, + 1.7904633283615112, + -0.04837304353713989, + -0.6139949560165405, + -0.8549875020980835, + 0.3574656546115875, + -2.8409454822540283, + -2.1011672019958496, + 1.382948875427246, + 0.6656897068023682, + -1.5090553760528564, + 0.4825518727302551, + -0.3345044255256653, + -2.26375150680542, + -2.561906576156616 + ] + ], + "predicted_scores": [ + 5.859316349029541, + 5.420124530792236, + 5.264007568359375, + 5.090690612792969, + 4.776496410369873, + 4.625821113586426, + 4.44064998626709, + 4.344982147216797 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813/train2", + "selected_index": 0, + "state_hash": "5608eea5b7e99b5fed6b316f742ecd34044df87286b2338705d8472043fc4375", + "stored_base_utility": 0.22621813416481018, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09514755010604858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09514755010604858 + }, + "base_progress": 0.09514755010604858, + "base_success": false, + "base_utility": 0.09514755010604858, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05872911214828491, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05872911214828491 + }, + { + "contact_quality": null, + "progress": 0.05446813628077507, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05446813628077507 + }, + { + "contact_quality": null, + "progress": 0.06178152561187744, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06178152561187744 + }, + { + "contact_quality": null, + "progress": 0.006611684802919626, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006611684802919626 + }, + { + "contact_quality": null, + "progress": 0.006492753978818655, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006492753978818655 + }, + { + "contact_quality": null, + "progress": 0.006466865539550781, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006466865539550781 + }, + { + "contact_quality": null, + "progress": 0.0066977739334106445, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0066977739334106445 + }, + { + "contact_quality": null, + "progress": 0.0056935749016702175, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056935749016702175 + } + ], + "candidate_progress": [ + 0.05872911214828491, + 0.05446813628077507, + 0.06178152561187744, + 0.006611684802919626, + 0.006492753978818655, + 0.006466865539550781, + 0.0066977739334106445, + 0.0056935749016702175 + ], + "candidate_source_chart_ids": [ + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ec013396e4aff6170ee1", + "generated_tangents": [ + [ + 0.5101381540298462, + -0.05360936373472214, + -0.029661327600479126, + 0.20449179410934448, + 1.0406460762023926, + -0.030199676752090454, + -0.3574920892715454, + -1.022248387336731, + 0.23367558419704437, + 1.1467262506484985, + 0.24691137671470642, + -0.724727988243103, + -0.3094742000102997, + 0.3639238774776459, + -0.165083646774292, + 0.8841742873191833, + 0.1577814817428589, + 0.3289101719856262, + -0.22462034225463867, + -1.9921259880065918, + 0.06798835098743439 + ], + [ + -0.15466274321079254, + -0.03181654214859009, + 0.3227255940437317, + 1.1556735038757324, + 0.6858857870101929, + 0.19386076927185059, + 0.4094448387622833, + -1.6155661344528198, + 0.4407074451446533, + 1.43799889087677, + -0.06252732127904892, + -0.14964619278907776, + -0.9649289846420288, + -0.15493474900722504, + -0.6133165955543518, + 0.6074562668800354, + 0.06724636256694794, + 0.28682491183280945, + 0.1895415186882019, + -1.3882741928100586, + -0.38994941115379333 + ], + [ + 0.285857230424881, + -0.13716495037078857, + 0.5049164295196533, + 0.38094261288642883, + 0.7799856066703796, + -0.0754309594631195, + -0.5730307698249817, + -1.216273546218872, + 0.24912449717521667, + 1.2083938121795654, + 0.4422561526298523, + -0.48583900928497314, + 0.34284552931785583, + 0.6974050998687744, + -0.19796833395957947, + 0.9154776334762573, + 0.48387467861175537, + 0.39983516931533813, + -0.2657671570777893, + -1.5331840515136719, + 0.41048866510391235 + ], + [ + -0.21052299439907074, + -0.13993632793426514, + -0.10644039511680603, + 1.7271068096160889, + 1.2685949802398682, + 0.4705065190792084, + -0.2037225365638733, + -1.3093360662460327, + 0.12810085713863373, + 1.2043800354003906, + 0.1598292887210846, + 0.12638956308364868, + -0.8021771907806396, + 0.5791755318641663, + -0.189354807138443, + 0.5201027989387512, + 0.17081068456172943, + 0.34929633140563965, + 0.019526392221450806, + -2.0044541358947754, + -0.7854268550872803 + ], + [ + -0.155709907412529, + -0.09890305995941162, + -0.1439400613307953, + 1.3399819135665894, + 0.7990903854370117, + -0.03894232213497162, + 0.4093298017978668, + -1.4367101192474365, + 0.27560991048812866, + 1.5024744272232056, + -0.010531358420848846, + -0.12255418300628662, + -0.7647441625595093, + -0.08253826200962067, + -0.5598908066749573, + 0.640886664390564, + 0.31670981645584106, + 0.35831886529922485, + -0.11838579177856445, + -1.8318662643432617, + -0.2831522822380066 + ], + [ + -0.20471538603305817, + -0.07208540290594101, + -0.06225898861885071, + 1.1957406997680664, + 0.8960942029953003, + 0.2189735770225525, + 0.21606463193893433, + -1.4886016845703125, + 0.31738755106925964, + 1.547255516052246, + 0.05674219876527786, + -0.2351195216178894, + -0.8285989761352539, + -0.07602265477180481, + -0.5344630479812622, + 0.6355008482933044, + 0.1701146364212036, + 0.11661215871572495, + 0.1879451423883438, + -1.9480006694793701, + -0.3432569205760956 + ], + [ + -0.17921487987041473, + -0.08891166746616364, + -0.1476530134677887, + 1.390163540840149, + 0.9013686776161194, + -0.037880584597587585, + 0.22892099618911743, + -1.4280190467834473, + 0.26878079771995544, + 1.5264595746994019, + 0.051676854491233826, + -0.23780396580696106, + -0.9902065992355347, + 0.230289489030838, + -0.48221898078918457, + 0.6269107460975647, + 0.1803981363773346, + 0.30742016434669495, + -0.015006333589553833, + -1.7922697067260742, + -0.2591914236545563 + ], + [ + -0.3252200484275818, + -0.10510633885860443, + -0.05997195839881897, + 1.5174412727355957, + 0.9277306199073792, + 0.06104784458875656, + 0.3326023519039154, + -1.1899224519729614, + 0.2765958607196808, + 1.51645028591156, + -0.09933304041624069, + -0.21646061539649963, + -0.9433207511901855, + 0.010022200644016266, + -0.5805730223655701, + 0.6100038290023804, + 0.2636059522628784, + 0.1665084958076477, + -0.010278105735778809, + -2.134931802749634, + -0.2663840651512146 + ] + ], + "generated_utilities": [ + 0.05872911214828491, + 0.05446813628077507, + 0.06178152561187744, + 0.006611684802919626, + 0.006492753978818655, + 0.006466865539550781, + 0.0066977739334106445, + 0.0056935749016702175 + ], + "group_id": "ms-ec013396e4aff6170ee1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.010581731796264648, + 0.049839019775390625, + 0.06402494758367538, + 0.012959301471710205, + 0.11614581197500229, + 0.012509982101619244, + 0.09607907384634018, + 0.1605721116065979, + 0.1708059310913086, + 0.055588722229003906, + 0.2052522599697113, + 0.0690404623746872, + 0.11059975624084473, + 0.03047112748026848, + 0.24502670764923096, + 0.22897732257843018 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.16510963439941406, + -0.011389851570129395, + -0.16074393689632416, + 0.8547754287719727, + -0.09704409539699554, + 0.09471678733825684, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + -0.29076793789863586, + 0.0122823566198349, + 0.059903502464294434, + 0.0, + 0.13894814252853394, + 0.2961620092391968, + 0.025543689727783203, + 0.01881265640258789, + 0.052319109439849854, + -1.1815040111541748, + 0.0 + ], + [ + -0.6906574964523315, + 0.5966935753822327, + 0.1873793751001358, + 1.8547754287719727, + 0.16492335498332977, + 0.4949548840522766, + 0.624785304069519, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233, + -0.44174641370773315, + 0.866999626159668, + 1.0255436897277832, + 1.018812656402588, + 0.6033775210380554, + -2.181504011154175, + -0.4328075349330902 + ], + [ + -0.68684321641922, + 0.35380983352661133, + 0.41641929745674133, + 2.0589241981506348, + 0.20610027015209198, + 0.2780776619911194, + 0.6000017523765564, + -1.8711607456207275, + 1.71079421043396, + 2.484286069869995, + -0.12494567036628723, + 0.3767848014831543, + -0.59480881690979, + 1.3215627670288086, + -0.18650269508361816, + 0.5971667766571045, + 1.00047767162323, + 1.2905888557434082, + 0.7682754993438721, + -2.280778408050537, + -0.26339399814605713 + ], + [ + -1.0669870376586914, + 0.5963312387466431, + 0.24800772964954376, + 1.5324974060058594, + 0.049914538860321045, + 0.33535104990005493, + 0.9774697422981262, + -2.318913698196411, + 1.14731764793396, + 2.1673643589019775, + -0.3369846045970917, + -0.13161931931972504, + -0.7527784705162048, + 1.2647147178649902, + -0.5229569673538208, + 0.6999921798706055, + 0.654599666595459, + 0.9745514988899231, + 0.5995839238166809, + -2.3844704627990723, + -0.7404599189758301 + ], + [ + -0.660759449005127, + 0.560627818107605, + -0.19398994743824005, + 1.7401045560836792, + 0.006708189845085144, + 0.2270365059375763, + 0.9201371073722839, + -1.6917380094528198, + 1.8382139205932617, + 2.184737205505371, + -0.07137502729892731, + 0.13004565238952637, + -0.8376798629760742, + 1.3597201108932495, + -0.8258764147758484, + 0.521450400352478, + 1.4799309968948364, + 0.7742871046066284, + 1.007819414138794, + -2.398472785949707, + -0.7120757102966309 + ], + [ + -0.3511212170124054, + 0.6072589755058289, + 0.6547060012817383, + 2.306784152984619, + 0.03337647020816803, + 1.1676279306411743, + 0.48420602083206177, + -1.7057392597198486, + 1.7822117805480957, + 2.741852283477783, + -0.970782995223999, + 0.3188708424568176, + -0.1370975375175476, + 1.740339994430542, + 0.18519049882888794, + 0.2771400809288025, + 0.8003557920455933, + 0.8813821077346802, + 0.9829010963439941, + -1.9104408025741577, + -0.7000454664230347 + ], + [ + -1.1815872192382812, + 1.0524637699127197, + 0.5610368251800537, + 2.407832622528076, + -0.2576238512992859, + 0.1467430293560028, + 0.27881303429603577, + -1.6810011863708496, + 1.5921515226364136, + 2.929525375366211, + 0.19559642672538757, + 0.6552202105522156, + -1.3889155387878418, + 1.1560360193252563, + -0.4104887843132019, + 0.8887493014335632, + 1.283969759941101, + 0.7811828851699829, + -0.10630232095718384, + -1.4329519271850586, + -0.5646982192993164 + ], + [ + -1.4926873445510864, + 0.11535564064979553, + -0.6240940690040588, + 1.631784200668335, + 0.7930782437324524, + -0.35493338108062744, + 0.20104485750198364, + -1.5626122951507568, + 1.3110854625701904, + 3.0686309337615967, + -0.8419785499572754, + -0.0010939836502075195, + -0.49860918521881104, + 1.5949748754501343, + -0.938766598701477, + 1.762406587600708, + 0.625543475151062, + 0.20789378881454468, + 1.1513795852661133, + -2.3980860710144043, + -0.36970049142837524 + ], + [ + -1.0271471738815308, + 0.6133835911750793, + -0.3870598077774048, + 1.9299802780151367, + 0.9606215357780457, + -0.19876420497894287, + 0.29684266448020935, + -1.6164112091064453, + 1.3925114870071411, + 1.9474635124206543, + -1.2669438123703003, + -0.6821421980857849, + -0.7869906425476074, + 1.3602250814437866, + -0.14778050780296326, + 0.9938437938690186, + 0.6618988513946533, + 0.1858283281326294, + 0.8312947750091553, + -1.7043554782867432, + -0.23048581182956696 + ], + [ + -0.409736305475235, + 1.5572996139526367, + 1.0024399757385254, + 1.6385810375213623, + 0.83731609582901, + -0.1629825234413147, + -0.05952054262161255, + -1.0968945026397705, + 0.728961706161499, + 3.1370975971221924, + -0.44591856002807617, + 1.149456262588501, + -1.3439809083938599, + 1.738822340965271, + 0.4063607454299927, + 0.4372418224811554, + 1.276847004890442, + 1.4612212181091309, + 0.6917726993560791, + -1.914740800857544, + 0.5415292978286743 + ] + ], + "predicted_scores": [ + 4.882284164428711, + 4.643063068389893, + 3.9711496829986572, + 3.844956398010254, + 3.3679914474487305, + 3.356393814086914, + 3.28397274017334, + 2.771803140640259 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296/train2", + "selected_index": 0, + "state_hash": "f5a19cf07e6a6cc71c95ce01e976d34d1eeae68c8a93ecc985fd0a78cb0c1a5e", + "stored_base_utility": 0.010581731796264648, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008193274028599262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008193274028599262 + }, + { + "contact_quality": null, + "progress": 0.007785916328430176, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007785916328430176 + }, + { + "contact_quality": null, + "progress": 0.010366380214691162, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010366380214691162 + }, + { + "contact_quality": null, + "progress": 0.009668509475886822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009668509475886822 + }, + { + "contact_quality": null, + "progress": 0.015532891266047955, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015532891266047955 + }, + { + "contact_quality": null, + "progress": 0.02954266406595707, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02954266406595707 + }, + { + "contact_quality": null, + "progress": 0.016943970695137978, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016943970695137978 + }, + { + "contact_quality": null, + "progress": 0.036433976143598557, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.036433976143598557 + } + ], + "candidate_progress": [ + 0.008193274028599262, + 0.007785916328430176, + 0.010366380214691162, + 0.009668509475886822, + 0.015532891266047955, + 0.02954266406595707, + 0.016943970695137978, + 0.036433976143598557 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-003a2225c0892dcda683", + "ms-0285f5815c210ac00bec", + "ms-0285f5815c210ac00bec", + "ms-be219977a9db8dbb1e58", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-ee30ebaf98622759d888", + "generated_tangents": [ + [ + 0.27170443534851074, + -0.4520937204360962, + 0.6051304340362549, + -0.11098112165927887, + 2.2225608825683594, + 0.8262457847595215, + 0.51774001121521, + 0.22463487088680267, + -0.13016048073768616, + 0.7891191840171814, + 0.10605950653553009, + 0.4298771023750305, + -1.3047176599502563, + 0.21161803603172302, + -0.13798145949840546, + 0.6419728398323059, + 0.8496493697166443, + -0.619520902633667, + 1.1715009212493896, + -2.2415504455566406, + -1.6119660139083862 + ], + [ + 0.2660503685474396, + -0.4555244445800781, + 0.5446423888206482, + -0.0777994841337204, + 2.3126749992370605, + 0.9720523357391357, + 0.578887403011322, + 0.2177036553621292, + -0.14381539821624756, + 0.8266566395759583, + 0.17044782638549805, + 0.541135311126709, + -1.3385353088378906, + 0.13335537910461426, + -0.1603492945432663, + 0.6253682971000671, + 0.925575315952301, + -0.4906349182128906, + 1.1988892555236816, + -2.2153210639953613, + -1.8029555082321167 + ], + [ + 0.3739367723464966, + -0.42600834369659424, + 0.9287277460098267, + 0.03848761320114136, + 1.9865660667419434, + 1.2307316064834595, + 0.579405665397644, + 0.3883819580078125, + -0.14122119545936584, + 0.6310248374938965, + 0.22717157006263733, + 0.33508965373039246, + -1.7190955877304077, + -0.06860573589801788, + 0.08088497817516327, + 0.5474994778633118, + 0.6855216026306152, + -0.6447333693504333, + 1.0441217422485352, + -2.1500213146209717, + -1.3579318523406982 + ], + [ + 0.35994911193847656, + -0.43802374601364136, + 0.9374743103981018, + 0.043926119804382324, + 1.9804353713989258, + 1.2484631538391113, + 0.6148548722267151, + 0.3553198575973511, + -0.13248299062252045, + 0.6398996114730835, + 0.21147191524505615, + 0.36410677433013916, + -1.8008677959442139, + -0.013217300176620483, + 0.06951896846294403, + 0.5593108534812927, + 0.6968275308609009, + -0.6997720003128052, + 1.0882527828216553, + -2.1821072101593018, + -1.3579750061035156 + ], + [ + 0.41315406560897827, + -0.25528600811958313, + 0.5363587141036987, + -0.20223447680473328, + 1.8758914470672607, + 0.8883450031280518, + 0.5835685729980469, + 0.35880497097969055, + -0.09831319004297256, + 0.2971900403499603, + 0.08581723272800446, + 0.45772281289100647, + -1.966092586517334, + -0.04350966215133667, + -0.2585785686969757, + 0.3062516748905182, + 1.0503592491149902, + -0.7279158234596252, + 0.8477835059165955, + -2.0772640705108643, + -0.6477218866348267 + ], + [ + 0.23574531078338623, + -0.16467499732971191, + 0.23795995116233826, + 0.21440979838371277, + 2.8141818046569824, + 0.8763961791992188, + -0.12586015462875366, + 0.8820905685424805, + -0.0445159450173378, + 0.2997494637966156, + 0.0844811201095581, + 0.913824737071991, + -2.358003616333008, + 0.1929166167974472, + -0.08803212642669678, + 0.5527779459953308, + 1.0282915830612183, + -0.6101901531219482, + 1.1536555290222168, + -2.258113384246826, + -1.429984211921692 + ], + [ + 0.4009239077568054, + -0.10225620120763779, + 0.6916215419769287, + -0.14502769708633423, + 2.9053778648376465, + 1.6723735332489014, + 0.6771396398544312, + 0.7002356052398682, + 0.11819807440042496, + 0.24285200238227844, + 0.4466201663017273, + 0.5122247934341431, + -1.8524523973464966, + 0.2524349093437195, + 0.027714475989341736, + 0.8019974231719971, + 1.25539231300354, + -0.5372257828712463, + 1.086941123008728, + -1.4629790782928467, + -1.634588360786438 + ], + [ + 0.7704882621765137, + -0.32318365573883057, + 0.08198440074920654, + 0.25582268834114075, + 1.919844627380371, + 1.0956228971481323, + 0.9605214595794678, + 0.5610069632530212, + -0.04221829026937485, + 0.38827699422836304, + 0.08968988060951233, + 0.30052971839904785, + -1.791446566581726, + 0.552302896976471, + 0.2858082056045532, + 0.49666696786880493, + 1.0112940073013306, + -0.35014182329177856, + 1.0454986095428467, + -1.9807567596435547, + -2.004758596420288 + ] + ], + "generated_utilities": [ + 0.008193274028599262, + 0.007785916328430176, + 0.010366380214691162, + 0.009668509475886822, + 0.015532891266047955, + 0.02954266406595707, + 0.016943970695137978, + 0.036433976143598557 + ], + "group_id": "ms-ee30ebaf98622759d888", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08405601978302002, + 0.09846439212560654, + 0.04595305770635605, + 2.0, + 0.08787640184164047, + 0.032481253147125244, + 0.08755072206258774, + 0.058936577290296555, + 0.10809408128261566, + 0.06723630428314209, + 0.0625009536743164, + 0.1415095031261444, + 0.08326160907745361, + 0.10944461822509766, + 0.0673408955335617, + 0.033725105226039886 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.4581542015075684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9406367540359497, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.470216989517212 + ], + [ + 1.6340367794036865, + -0.30366095900535583, + 0.2830277681350708, + -1.1627482175827026, + 3.58767032623291, + 0.6565636396408081, + 1.4321975708007812, + 0.8704822659492493, + 0.14327874779701233, + 1.0190284252166748, + 0.3078446090221405, + 0.7927982807159424, + -1.9305998086929321, + -1.7108646631240845, + 0.10729888081550598, + 0.9427294731140137, + 0.41770225763320923, + -1.7673753499984741, + 1.3209179639816284, + -2.8053033351898193, + -1.5330681800842285 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.00628507137298584, + 0.0595628023147583, + 0.39481037855148315, + -0.006641983985900879, + 1.8433213233947754, + 0.14830267429351807, + 0.0, + -0.24425631761550903, + 0.42130333185195923, + 0.41399094462394714, + 0.478841632604599, + -0.007461488246917725, + -0.5437495708465576, + 0.0, + 0.255215585231781, + -0.07331699132919312, + -0.1736500859260559, + -0.285358190536499, + 0.567956805229187, + -1.7103407382965088, + 0.0 + ], + [ + 1.0371580123901367, + -0.33467406034469604, + 0.9327473640441895, + -0.5396467447280884, + 2.1487879753112793, + 0.6294846534729004, + 0.6252679228782654, + 0.5140941143035889, + -0.20645928382873535, + -0.04528486728668213, + 0.14780870079994202, + 0.430325984954834, + -0.9293508529663086, + -1.5147639513015747, + 1.1104007959365845, + -0.05476301908493042, + 0.1755150556564331, + -1.3470268249511719, + 1.675830364227295, + -3.2944674491882324, + -1.9277915954589844 + ] + ], + "predicted_scores": [ + 9.16378116607666, + 8.605719566345215, + 8.50434398651123, + 8.490377426147461, + 7.924519062042236, + 7.78317928314209, + 5.880914688110352, + 2.4533824920654297 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "636680137/train2", + "selected_index": 0, + "state_hash": "2de607a9c69005130e51ff2b551c0eeedc85c58fef796682e6527bb5f719ff7b", + "stored_base_utility": 0.08405601978302002, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.026694953441619873, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026694953441619873 + }, + "base_progress": 0.026694953441619873, + "base_success": false, + "base_utility": 0.026694953441619873, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.020747045055031776, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020747045055031776 + }, + { + "contact_quality": null, + "progress": 0.018429061397910118, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018429061397910118 + }, + { + "contact_quality": null, + "progress": 0.013478715904057026, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013478715904057026 + }, + { + "contact_quality": null, + "progress": 0.015456080436706543, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015456080436706543 + }, + { + "contact_quality": null, + "progress": 0.04147064685821533, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04147064685821533 + }, + { + "contact_quality": null, + "progress": 0.044700879603624344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044700879603624344 + }, + { + "contact_quality": null, + "progress": 0.025082269683480263, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025082269683480263 + }, + { + "contact_quality": null, + "progress": 0.039384882897138596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.039384882897138596 + } + ], + "candidate_progress": [ + 0.020747045055031776, + 0.018429061397910118, + 0.013478715904057026, + 0.015456080436706543, + 0.04147064685821533, + 0.044700879603624344, + 0.025082269683480263, + 0.039384882897138596 + ], + "candidate_source_chart_ids": [ + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-53a86042af3c0887e556", + "ms-1a4979e969aa331fb126" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-f3f1d07cfd59b823da47", + "generated_tangents": [ + [ + 1.0349353551864624, + -0.06126919761300087, + -0.24474528431892395, + -0.4928235411643982, + -0.09823867678642273, + -1.5665614604949951, + -0.34603285789489746, + 0.6530897617340088, + 0.012888672761619091, + -0.3264999985694885, + -1.0608021020889282, + 0.3909127712249756, + -0.2603576183319092, + 0.6442807912826538, + 0.6143785715103149, + -0.01590779423713684, + 0.08504053950309753, + -0.449088454246521, + 0.19008608162403107, + 0.24809932708740234, + 0.5028657913208008 + ], + [ + 1.2439202070236206, + 0.016533263027668, + -0.4084916412830353, + -0.3623259961605072, + 0.3134339451789856, + -1.8045082092285156, + -0.16475188732147217, + 0.5584524869918823, + -0.12950576841831207, + -0.2849171757698059, + -0.8536432385444641, + 0.49415135383605957, + -0.3136178255081177, + 0.5371270775794983, + 0.34843894839286804, + -0.10000298917293549, + 0.18952998518943787, + -0.6882005929946899, + 0.31218549609184265, + -0.5124547481536865, + 0.5893510580062866 + ], + [ + 1.0175174474716187, + -0.0002964623272418976, + -0.15028533339500427, + -0.3534201979637146, + 0.12945501506328583, + -1.4312644004821777, + -0.27580761909484863, + 0.7022987604141235, + -0.09329473227262497, + -0.18243283033370972, + -1.097499132156372, + 0.6538478136062622, + -0.32707029581069946, + 0.74146568775177, + 0.35227170586586, + -0.11970359086990356, + 0.17694193124771118, + -0.5119301676750183, + 0.3433413803577423, + 0.23969638347625732, + 0.6604613065719604 + ], + [ + 0.9946953654289246, + -0.015589572489261627, + -0.17950591444969177, + -0.4158869981765747, + -0.03835874795913696, + -1.5828815698623657, + -0.3362371325492859, + 0.6675266623497009, + -0.024929681792855263, + -0.241695374250412, + -1.09584641456604, + 0.349558025598526, + -0.24058499932289124, + 0.5352067351341248, + 0.49236515164375305, + -0.019468313083052635, + 0.12109559029340744, + -0.5607954859733582, + 0.2185971587896347, + 0.376800000667572, + 0.48997700214385986 + ], + [ + 0.8608055114746094, + 0.26703953742980957, + -0.4005744159221649, + -0.2484826147556305, + 0.2338886708021164, + -0.9424741268157959, + 0.44213464856147766, + 0.6766794919967651, + -0.045276157557964325, + -0.07421816140413284, + -1.0170491933822632, + 0.5813264846801758, + -0.8028758764266968, + -0.21456409990787506, + 0.15495555102825165, + 0.07764653861522675, + 0.2836809456348419, + -1.0219577550888062, + 0.10980084538459778, + 0.4904273748397827, + 0.5579061508178711 + ], + [ + 0.9573588967323303, + 0.27570217847824097, + -0.3582111895084381, + -0.3116881251335144, + 0.3710542321205139, + -0.8759490251541138, + 0.30302301049232483, + 0.6119298934936523, + -0.09536349773406982, + -0.07157202810049057, + -1.150516390800476, + 0.6570000052452087, + -0.9844952821731567, + -0.21766610443592072, + 0.08414357900619507, + 0.0901758074760437, + 0.2130962461233139, + -1.0716454982757568, + 0.1417546570301056, + 0.25137341022491455, + 0.3846231698989868 + ], + [ + 1.204817771911621, + 0.09754139184951782, + -0.6386665105819702, + -0.062338411808013916, + 0.09314592182636261, + -1.5826767683029175, + -0.41071248054504395, + 0.521346926689148, + 0.25248655676841736, + -0.1181819960474968, + -0.40200352668762207, + 0.6560121774673462, + -0.41702309250831604, + 0.25525963306427, + 0.3502126634120941, + -0.22212518751621246, + 0.36393314599990845, + -0.7670950293540955, + 0.23357316851615906, + 0.18474698066711426, + 0.6281170845031738 + ], + [ + 1.2584664821624756, + 0.22888241708278656, + -0.6232999563217163, + -0.2170407772064209, + -0.11911964416503906, + -1.6025922298431396, + -0.021400898694992065, + 0.42525210976600647, + 0.42933526635169983, + 0.1212443932890892, + -0.6297640204429626, + 0.4663831293582916, + -0.8602383136749268, + 0.35413768887519836, + 0.31219640374183655, + -0.2646739184856415, + -0.16007615625858307, + -0.6059231162071228, + 0.18030160665512085, + -0.01707574725151062, + 0.014252588152885437 + ] + ], + "generated_utilities": [ + 0.020747045055031776, + 0.018429061397910118, + 0.013478715904057026, + 0.015456080436706543, + 0.04147064685821533, + 0.044700879603624344, + 0.025082269683480263, + 0.039384882897138596 + ], + "group_id": "ms-f3f1d07cfd59b823da47", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07767709344625473, + 0.07153411954641342, + 0.08480638265609741, + 0.06087514013051987, + 0.15252648293972015, + 0.0776997059583664, + 0.1476559340953827, + 0.17124073207378387, + 0.12327007949352264, + 0.12037092447280884, + 0.08562581241130829, + 0.09917601197957993, + 0.07137986272573471, + 0.039922356605529785, + 0.04496842622756958, + 0.05561637878417969 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.9664052724838257, + 0.20301294326782227, + -1.1196715831756592, + -0.17260856926441193, + -0.5341777801513672, + -2.678128242492676, + -2.431448459625244, + 1.5603530406951904, + 0.03585217520594597, + -0.3520800471305847, + -2.473386287689209, + 1.0162299871444702, + -0.7868502736091614, + -0.9102090001106262, + 0.9442894458770752, + -0.41473180055618286, + 0.7349075675010681, + -0.8863289952278137, + 0.8796874284744263, + 0.028114937245845795, + 0.24920660257339478 + ], + [ + 1.900488257408142, + -0.00838443636894226, + -1.2311820983886719, + -0.361179918050766, + -0.3987593352794647, + -2.551008462905884, + -2.701385021209717, + 1.7117550373077393, + -0.20423506200313568, + -0.3429180979728699, + -2.6584055423736572, + 0.8018259406089783, + -0.856230616569519, + -0.901038408279419, + 1.2131764888763428, + -0.6892215609550476, + 0.6726197600364685, + -0.7798839211463928, + 0.6868253946304321, + 0.07731680572032928, + 0.30107203125953674 + ], + [ + 1.6257587671279907, + 0.275219202041626, + -0.7838636636734009, + -0.5118123292922974, + -0.3429456353187561, + -2.342111349105835, + -2.7158243656158447, + 1.7038744688034058, + -0.1860528439283371, + -0.11617843806743622, + -2.1894750595092773, + 1.2306283712387085, + -0.8203369379043579, + -0.5157363414764404, + 0.7951346635818481, + -0.464976966381073, + 0.8823708295822144, + -0.7671404480934143, + 0.9700374007225037, + 0.21184596419334412, + -0.11244213581085205 + ] + ], + "predicted_scores": [ + 4.140278339385986, + 3.720200538635254, + 3.6682474613189697, + 3.4919662475585938, + 3.44966983795166, + 2.959789991378784, + 2.7394957542419434, + 2.1507058143615723 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957/train2", + "selected_index": 0, + "state_hash": "3f477a29208b02558b99bd0f3305d0f1845c277d32e0b5afdafa71012673383b", + "stored_base_utility": 0.07767709344625473, + "task_id": "PullCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07080724835395813, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07080724835395813 + }, + "base_progress": 0.07080724835395813, + "base_success": false, + "base_utility": 0.07080724835395813, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.04525895416736603, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04525895416736603 + }, + { + "contact_quality": null, + "progress": 0.06861761212348938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06861761212348938 + }, + { + "contact_quality": null, + "progress": 0.07222916185855865, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07222916185855865 + }, + { + "contact_quality": null, + "progress": 0.09644092619419098, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09644092619419098 + }, + { + "contact_quality": null, + "progress": 0.053700655698776245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.053700655698776245 + }, + { + "contact_quality": null, + "progress": 4.214048385620117e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 4.214048385620117e-05 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.04525895416736603, + 0.06861761212348938, + 0.07222916185855865, + 0.09644092619419098, + 0.053700655698776245, + 4.214048385620117e-05, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-dab80e1bb7987577e33d", + "ms-bfe867e8d518b9f8333a", + "ms-bfe867e8d518b9f8333a", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-081952a63087fc8f83cd", + "generated_tangents": [ + [ + -0.032365500926971436, + 0.10390760004520416, + 1.415757656097412, + -0.5925084352493286, + -0.8634079098701477, + -0.19858211278915405, + -0.2506406903266907, + 0.048071250319480896, + -0.11267983913421631, + 0.12202358990907669, + 0.15672841668128967, + -1.0689752101898193, + -0.06612500548362732, + 0.06195733696222305, + 0.04202529042959213, + -0.013784321956336498, + 0.016651272773742676, + 0.1811247020959854, + -0.5694010257720947, + -0.2003055214881897, + -0.03322325646877289 + ], + [ + -0.11432172358036041, + 0.08919510245323181, + 1.517054557800293, + -0.4753502309322357, + -0.8041483759880066, + -0.19328033924102783, + -0.35954850912094116, + 0.09516977518796921, + -0.0904659777879715, + 0.15756557881832123, + 0.20103389024734497, + -0.8708748817443848, + -0.07928246259689331, + -0.00747188925743103, + 0.08754106611013412, + 0.004732646048069, + 0.11909907311201096, + 0.17976027727127075, + -0.3799339532852173, + -0.008889615535736084, + -0.05170711874961853 + ], + [ + -0.11421684920787811, + 0.07243978977203369, + 1.5277025699615479, + -0.5288271307945251, + -0.8376646637916565, + -0.12296772003173828, + -0.3800280690193176, + 0.11622491478919983, + -0.10285645723342896, + 0.1337386518716812, + 0.3056662976741791, + -0.8803032636642456, + -0.049797624349594116, + -0.023733511567115784, + 0.04615781828761101, + -0.016816962510347366, + 0.13501235842704773, + 0.14333173632621765, + -0.36909955739974976, + 0.030240654945373535, + -0.10721567273139954 + ], + [ + -0.11995033919811249, + 0.060259461402893066, + 1.217244029045105, + -0.7226763963699341, + -1.1381659507751465, + 0.2173631489276886, + -0.1764562726020813, + -0.09369504451751709, + -0.0863342434167862, + 0.2310180962085724, + 0.16642993688583374, + -0.9087406396865845, + -0.24564912915229797, + 0.15980838239192963, + -0.023319657891988754, + 0.029917996376752853, + -0.04658149182796478, + 0.003416284918785095, + -0.4824352264404297, + -0.23991714417934418, + 0.09200343489646912 + ], + [ + 0.026846006512641907, + 0.03736086189746857, + 1.4077420234680176, + -0.7325681447982788, + -1.1262304782867432, + 0.045286305248737335, + -0.19949376583099365, + -0.04826843738555908, + -0.051994092762470245, + 0.1934489905834198, + 0.20467272400856018, + -0.8510723114013672, + 0.1608184278011322, + 0.023805618286132812, + -0.029557082802057266, + 0.018773630261421204, + -0.011426910758018494, + 0.12089154124259949, + -0.41795814037323, + -0.14041092991828918, + 0.010147333145141602 + ], + [ + -0.206819549202919, + 0.09014281630516052, + 1.5819236040115356, + -0.9215487241744995, + -0.7654125094413757, + 0.12386204302310944, + -0.337863028049469, + -0.05849051475524902, + -0.013000188395380974, + 0.02482248842716217, + 0.29215696454048157, + -0.6927528977394104, + -0.31674784421920776, + 0.11673110723495483, + -0.09033209085464478, + -0.06116269528865814, + -0.04351656138896942, + 0.21102769672870636, + -0.6393582224845886, + 0.10619005560874939, + -0.2805402874946594 + ], + [ + -0.34209907054901123, + -0.0786755234003067, + 1.4100205898284912, + -0.9066512584686279, + -0.8203497529029846, + 0.08083697408437729, + -0.3675681948661804, + 0.26753443479537964, + -0.11146403849124908, + 0.16693587601184845, + 0.4500954747200012, + -0.4960119128227234, + 0.08443322777748108, + 0.3699827194213867, + -0.09823426604270935, + -0.12029463052749634, + 0.10245548188686371, + 0.19952167570590973, + -0.6053216457366943, + 0.06697890162467957, + -0.2166258990764618 + ], + [ + -0.45381081104278564, + 0.18843740224838257, + 1.566582441329956, + -0.7463564872741699, + -0.7096806168556213, + 0.5056304931640625, + -0.2998039126396179, + 0.06751719117164612, + -0.1269555389881134, + 0.06923900544643402, + 0.1686798334121704, + -0.9188179969787598, + -0.040544867515563965, + 0.27317044138908386, + 0.021121639758348465, + -0.13543808460235596, + -0.08620311319828033, + 0.2963440418243408, + -0.6836603283882141, + -0.14094305038452148, + -0.08842611312866211 + ] + ], + "generated_utilities": [ + 0.04525895416736603, + 0.06861761212348938, + 0.07222916185855865, + 0.09644092619419098, + 0.053700655698776245, + 4.214048385620117e-05, + 2.0, + 2.0 + ], + "group_id": "ms-081952a63087fc8f83cd", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0006054937839508057, + 2.0, + 2.0, + 2.0, + 0.09041815996170044, + 0.03755016624927521, + 0.10023769736289978, + 0.08501018583774567, + 0.10316208004951477, + 0.13556814193725586, + 0.0687420666217804, + 0.16420090198516846, + 0.07440806925296783, + 0.03177754580974579, + 0.05512912571430206, + 0.05822136998176575 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05530714988708496, + -0.0015276968479156494, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.028481781482696533, + 0.0, + 0.07634823024272919, + 0.10118435323238373, + 0.01472395658493042, + -0.021184463053941727, + -1.3624117374420166, + 0.06702810525894165, + 0.0, + 0.064424529671669, + -0.13110561668872833, + -0.04677841067314148, + -0.0666094422340393, + -0.8611308336257935, + -0.02982822060585022, + 0.0 + ], + [ + -0.13999104499816895, + -0.16139763593673706, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.17549771070480347, + 0.0, + -0.15019088983535767, + 0.2207401990890503, + -0.06357991695404053, + 0.10177183151245117, + -1.3624117374420166, + -0.05549207329750061, + 0.0, + -0.1271420419216156, + -0.057942673563957214, + -0.279840350151062, + -0.11312717199325562, + -0.8611308336257935, + 0.1586582064628601, + 0.0 + ], + [ + 0.2317524552345276, + -0.2474955916404724, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.2997032403945923, + 0.0, + -0.3180278241634369, + 0.1991673707962036, + 0.26833486557006836, + 0.005899060517549515, + -1.3624117374420166, + 0.032782405614852905, + 0.0, + 0.17653466761112213, + 0.5550464987754822, + 0.13015127182006836, + -0.11312717199325562, + -0.8611308336257935, + 0.020897403359413147, + 0.0 + ], + [ + -0.7016198635101318, + 0.312343955039978, + 2.582733154296875, + -1.5283455848693848, + -2.530975103378296, + 0.7002967596054077, + 2.0224196910858154, + -0.031223494559526443, + -0.07945594191551208, + 0.7471928000450134, + -0.05505431815981865, + -2.3624117374420166, + 0.4337144196033478, + 1.385794997215271, + -0.18244768679141998, + -0.22281835973262787, + -0.5003347992897034, + 0.8868728280067444, + -1.8611308336257935, + 0.19736243784427643, + 0.2646070420742035 + ], + [ + -0.41503679752349854, + 0.3199126422405243, + 2.3341803550720215, + -1.5432449579238892, + -2.6036009788513184, + 0.8186125755310059, + 1.889835238456726, + -0.20609386265277863, + 0.0827421247959137, + 0.8381537795066833, + -0.0944519191980362, + -2.4805798530578613, + 0.3916063606739044, + 1.2392640113830566, + -0.2266693413257599, + -0.33516526222229004, + -0.3536234498023987, + 0.9749420881271362, + -2.1439476013183594, + 0.3261662721633911, + 0.13869160413742065 + ], + [ + -0.6778035759925842, + 0.3674989640712738, + 2.3216195106506348, + -1.6129508018493652, + -2.3233790397644043, + 0.37060582637786865, + 2.3460428714752197, + 0.2269098311662674, + -0.269074022769928, + 0.9810001254081726, + 0.023842941969633102, + -2.478595733642578, + 0.46739575266838074, + 1.1671457290649414, + -0.41542381048202515, + 0.030661001801490784, + -0.48762595653533936, + 0.7803086638450623, + -1.7778995037078857, + 0.10219369828701019, + 0.08611755073070526 + ], + [ + -1.0292376279830933, + 0.05653715133666992, + 3.017642021179199, + -1.2965388298034668, + -2.1434991359710693, + 0.8112732172012329, + 2.3553943634033203, + 0.14745590090751648, + -0.42473679780960083, + 0.8983602523803711, + -0.5389888286590576, + -1.9595260620117188, + 0.8665074110031128, + 1.7057592868804932, + -0.36306142807006836, + -0.6694610714912415, + -0.3699086308479309, + 0.7089512944221497, + -2.2496888637542725, + 0.1759258657693863, + 0.13106243312358856 + ], + [ + -0.2738665044307709, + 0.13772070407867432, + 2.3146424293518066, + -1.9326081275939941, + -2.760838508605957, + 0.10491085052490234, + 1.9823331832885742, + 0.3494338393211365, + 0.3550114631652832, + 0.7471076846122742, + -0.06264704465866089, + -2.2210469245910645, + 0.5386381149291992, + 1.431454062461853, + 0.1620958298444748, + -0.00858105719089508, + -0.8883554935455322, + 1.1764272451400757, + -1.8280467987060547, + 0.30690017342567444, + -0.32479730248451233 + ], + [ + -0.8548018932342529, + -0.2078946828842163, + 2.740286111831665, + -1.6552648544311523, + -2.470123291015625, + 1.0607044696807861, + 2.4267592430114746, + -0.16535070538520813, + -0.2640931308269501, + 0.2447068691253662, + 0.5616287589073181, + -2.808340311050415, + 1.0820176601409912, + 1.2134929895401, + 0.2653927206993103, + -0.454708456993103, + 0.03163468837738037, + 0.8172736763954163, + -1.8861355781555176, + 0.45078086853027344, + 0.29641321301460266 + ], + [ + -0.7025277018547058, + 0.5556451678276062, + 2.3374006748199463, + -1.9256110191345215, + -1.7495018243789673, + 0.9721168279647827, + 1.8326293230056763, + 0.5355191826820374, + -0.5788277983665466, + 0.19061005115509033, + -0.6322742700576782, + -3.031383514404297, + -0.1651831567287445, + 0.8584551215171814, + 0.2280312329530716, + 0.27221786975860596, + -0.3728860020637512, + 1.102063775062561, + -2.5995376110076904, + 0.7578867673873901, + -0.3784477412700653 + ], + [ + -7.212162017822266e-06, + 0.552659273147583, + 2.0397679805755615, + -1.8501746654510498, + -2.5457160472869873, + 1.0514975786209106, + 2.5270917415618896, + -0.7367458343505859, + 0.7676748037338257, + 0.7428221702575684, + -0.538831353187561, + -2.7423243522644043, + 0.07969537377357483, + 1.0069079399108887, + -0.8100140690803528, + -0.9174093008041382, + -0.9411231875419617, + 1.0430617332458496, + -1.4683198928833008, + 1.0334573984146118, + 0.4721405506134033 + ], + [ + -0.41159626841545105, + 0.41878798604011536, + 1.6363914012908936, + -2.4960880279541016, + -3.049954891204834, + -0.09567022323608398, + 2.3269245624542236, + 0.11159682273864746, + -0.7885257005691528, + 0.44582536816596985, + -0.47802892327308655, + -1.5574463605880737, + 1.0153859853744507, + 2.258706569671631, + -0.36591836810112, + 0.49115514755249023, + -0.9523675441741943, + 0.5892280340194702, + -2.1512646675109863, + -0.7744256854057312, + 0.592117428779602 + ], + [ + 0.1343594193458557, + 0.45838433504104614, + 2.6735801696777344, + -2.005676746368408, + -3.340463161468506, + 1.0152086019515991, + 1.429718255996704, + 0.7929191589355469, + -0.2598758339881897, + 1.6483545303344727, + -0.7411167025566101, + -2.8978958129882812, + 1.3759692907333374, + 1.4385837316513062, + 0.2961248755455017, + -0.042098239064216614, + -0.6380948424339294, + 1.0523303747177124, + -2.1783251762390137, + -0.045405834913253784, + 0.7960561513900757 + ] + ], + "predicted_scores": [ + 4.203157901763916, + 3.707049608230591, + 3.626953125, + 3.5704145431518555, + 3.536304473876953, + 3.1239418983459473, + 3.1084041595458984, + 2.9858627319335938 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2141802816/train2", + "selected_index": 0, + "state_hash": "d1cf06bba4675ad6a6942dc4f6fbade890fbf21b5c3068f8bbe30cfc60c54d9b", + "stored_base_utility": 0.0006054937839508057, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12855732440948486, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12855732440948486 + }, + { + "contact_quality": null, + "progress": 0.12833033502101898, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12833033502101898 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.09257189929485321, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09257189929485321 + }, + { + "contact_quality": null, + "progress": 0.0656263679265976, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0656263679265976 + }, + { + "contact_quality": null, + "progress": 0.060852110385894775, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060852110385894775 + }, + { + "contact_quality": null, + "progress": 0.1262555718421936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1262555718421936 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.12855732440948486, + 0.12833033502101898, + 1.0, + 0.09257189929485321, + 0.0656263679265976, + 0.060852110385894775, + 0.1262555718421936, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-398ddbf730ed8830b08b", + "ms-2ba02dcefa15a9371706", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + true, + false, + false, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-121bc2e6676c1ac10911", + "generated_tangents": [ + [ + -0.7257179617881775, + -0.03575143963098526, + 0.23788043856620789, + -0.9687948226928711, + -0.9038702845573425, + 0.10042782127857208, + 1.625234842300415, + 0.27071014046669006, + 0.1942325383424759, + -0.14601635932922363, + 0.39276736974716187, + -1.3444900512695312, + -0.041935503482818604, + 1.0646857023239136, + -0.3098151385784149, + -0.05423483997583389, + -0.6402682662010193, + 0.3935595154762268, + -0.45741546154022217, + -0.27093175053596497, + 0.4617878198623657 + ], + [ + -0.49222296476364136, + -0.02527921460568905, + 0.19154101610183716, + -1.0117971897125244, + -1.118886947631836, + 0.06761611253023148, + 1.7967050075531006, + -0.027564451098442078, + 0.2803365886211395, + 0.09631343185901642, + 0.2927677631378174, + -1.0953946113586426, + 0.08128461241722107, + 1.87663996219635, + -0.21061468124389648, + -0.16087670624256134, + -0.4095489978790283, + 0.2142687737941742, + -0.25936591625213623, + -0.0271415114402771, + 0.04426760971546173 + ], + [ + -0.6047983765602112, + -0.04153505340218544, + 0.033416807651519775, + -0.7612345218658447, + -1.0191209316253662, + -0.46741023659706116, + 1.0215144157409668, + -0.18060284852981567, + 0.2781020998954773, + -0.12955832481384277, + 0.24962148070335388, + -0.9737802743911743, + -0.004270076751708984, + 0.5479865074157715, + -0.11303319036960602, + -0.04215051978826523, + -0.5573393106460571, + 0.29055196046829224, + -0.832116425037384, + -0.6861945390701294, + 0.33294442296028137 + ], + [ + -0.7405837178230286, + 0.16284878551959991, + 0.13304874300956726, + -1.1297900676727295, + -1.0768933296203613, + -0.027847886085510254, + 1.5588839054107666, + 0.1773626208305359, + 0.229502871632576, + 0.03195039927959442, + 0.2792961597442627, + -0.8868368864059448, + -0.4169521927833557, + 1.0307973623275757, + -0.28355836868286133, + 0.039785824716091156, + -0.6305896043777466, + 0.32707977294921875, + -0.42981040477752686, + -0.300695538520813, + 0.6085741519927979 + ], + [ + -0.8752638101577759, + 0.15627846121788025, + -0.2642001807689667, + -1.4300812482833862, + -0.9086292386054993, + -0.40588781237602234, + 1.422707438468933, + 0.35890230536460876, + 0.32191863656044006, + 0.1711602658033371, + 0.07514350861310959, + -1.0588886737823486, + -0.037381649017333984, + 0.8639682531356812, + -0.3543846607208252, + 0.05986660346388817, + -0.5966168642044067, + 0.5338371992111206, + -0.4685141444206238, + -0.7181501388549805, + 0.8448859453201294 + ], + [ + -0.6440788507461548, + 0.26109427213668823, + 0.1002824604511261, + -1.1860899925231934, + -1.2110652923583984, + 0.2786216735839844, + 1.4428701400756836, + 0.049609556794166565, + 0.30572041869163513, + 0.013519495725631714, + 0.3673737943172455, + -0.9484045505523682, + -0.38821327686309814, + 1.107595682144165, + -0.11746107041835785, + -0.10040338337421417, + -0.514596700668335, + 0.2826034128665924, + -0.5153048038482666, + -0.17715111374855042, + 0.11471407860517502 + ], + [ + -0.7710386514663696, + 0.20785890519618988, + 0.002715498208999634, + -1.027133822441101, + -0.7781583666801453, + 0.05032403767108917, + 1.601724624633789, + 0.1300971806049347, + 0.20471708476543427, + -0.07448870688676834, + 0.30200162529945374, + -0.6621583104133606, + -0.41534000635147095, + 0.9123588800430298, + -0.32488465309143066, + -0.03074706345796585, + -0.612622082233429, + 0.38867878913879395, + -0.6351329684257507, + -0.41679513454437256, + 0.6185945868492126 + ], + [ + -0.18245281279087067, + 0.06815386563539505, + 0.19148990511894226, + -0.6779538989067078, + -0.4827861785888672, + -0.36563435196876526, + 1.8912129402160645, + 0.09653539210557938, + 0.28170448541641235, + -0.11424743384122849, + 0.30296146869659424, + -0.5772009491920471, + -0.19625002145767212, + 1.2267690896987915, + 0.05415017157793045, + 0.10929052531719208, + -0.21586258709430695, + 0.4771376848220825, + -0.36779356002807617, + -0.0263901948928833, + 0.815432071685791 + ] + ], + "generated_utilities": [ + 0.12855732440948486, + 0.12833033502101898, + 2.0, + 0.09257189929485321, + 0.0656263679265976, + 0.060852110385894775, + 0.1262555718421936, + 2.0 + ], + "group_id": "ms-121bc2e6676c1ac10911", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15086030960083008, + 0.08354458212852478, + 2.0, + 0.1453719437122345, + 0.19646552205085754, + 0.10343040525913239, + 0.1443832516670227, + 0.13976123929023743, + 0.17431864142417908, + 2.0, + 0.12949708104133606, + 2.0, + 2.0, + 0.11397694051265717, + 0.1271887570619583, + 0.025320395827293396 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.39665424823760986, + 0.08244594186544418, + -0.026901081204414368, + -0.959688663482666, + -1.2648048400878906, + 0.05568990111351013, + 0.0, + -0.048488348722457886, + 0.03576555848121643, + -0.012020409107208252, + 0.08576416969299316, + -0.25812816619873047, + 0.03589746356010437, + 0.0, + 0.0016258209943771362, + -0.04025750607252121, + 0.010670006275177002, + 0.5142596960067749, + 0.06220144033432007, + 0.07906517386436462, + 0.0 + ], + [ + -2.15720796585083, + 0.13025060296058655, + 0.41441529989242554, + -1.4498568773269653, + -2.6386165618896484, + 0.5572659969329834, + 0.7093658447265625, + -1.0835764408111572, + -0.896172285079956, + 0.07744038105010986, + 1.1224509477615356, + -1.6321502923965454, + 0.877888560295105, + 0.4978562295436859, + 0.5774499177932739, + 0.8093571662902832, + -0.8521628379821777, + 2.054164171218872, + -0.6021197438240051, + 1.0350128412246704, + 0.823830246925354 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.39665424823760986, + -0.2284793257713318, + -0.027931194752454758, + -0.959688663482666, + -1.2648048400878906, + -0.17197783291339874, + 0.0, + -0.06679429113864899, + 0.1646350920200348, + -0.11601293087005615, + 0.08576416969299316, + -0.25812816619873047, + 0.1479925513267517, + 0.0, + -0.04833713173866272, + 0.22201494872570038, + -0.16177666187286377, + 0.5142596960067749, + -0.10178101062774658, + 0.21631890535354614, + 0.0 + ], + [ + -0.8824495077133179, + 0.0034109503030776978, + -0.5904110074043274, + -1.400491714477539, + -2.4447474479675293, + -0.00789790228009224, + 1.6208159923553467, + 0.26819390058517456, + -0.23575741052627563, + -0.7184416651725769, + 0.9072303771972656, + -1.2077033519744873, + -0.21496516466140747, + 0.8258882761001587, + -0.4575563669204712, + -0.3655202388763428, + -0.5908474922180176, + 2.0210399627685547, + -1.2132787704467773, + 0.9719676971435547, + 0.24625355005264282 + ], + [ + -0.6953714489936829, + 0.4026735723018646, + 0.6648998856544495, + -1.356613278388977, + -2.953742742538452, + 0.037713564932346344, + 1.53464937210083, + 0.027994811534881592, + 0.03435474634170532, + -1.1939163208007812, + 0.9877840280532837, + -1.180513620376587, + 0.4064505994319916, + 0.5743880867958069, + 0.1558205783367157, + 0.3796072006225586, + -0.6676555275917053, + 1.4665511846542358, + 0.04170459508895874, + 0.17645099759101868, + 0.2388516366481781 + ], + [ + -1.4095232486724854, + -0.06358731538057327, + -0.7774888873100281, + -2.806776285171509, + -2.44041109085083, + -0.8258867859840393, + 2.383592128753662, + -0.093393474817276, + -0.24701429903507233, + -0.5286431312561035, + 0.7741218209266663, + -2.111828565597534, + -0.31807881593704224, + 1.3003538846969604, + -0.9306488037109375, + -0.8708829283714294, + -0.3694953918457031, + 2.169797420501709, + -0.7153271436691284, + 0.029519766569137573, + -0.5629553198814392 + ] + ], + "predicted_scores": [ + 5.867988109588623, + 5.841307640075684, + 5.431830406188965, + 5.335111618041992, + 5.190425872802734, + 5.006448268890381, + 4.667802333831787, + 4.228028297424316 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264/train2", + "selected_index": 0, + "state_hash": "d1280a9e9362f8da3290de84fdd0f0b78475d470cd0230e590ccaf8f5428548a", + "stored_base_utility": 0.15086030960083008, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.11496219038963318, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11496219038963318 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1255601942539215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1255601942539215 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 0.11496219038963318, + 1.0, + 1.0, + 0.1255601942539215, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + false, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-22f153f0d4fa656018e6", + "generated_tangents": [ + [ + -0.7708300352096558, + 0.15459537506103516, + 0.6022915840148926, + -1.4455561637878418, + -2.0376839637756348, + 0.2948944866657257, + 0.34606489539146423, + 0.020299717783927917, + 0.050667427480220795, + 0.0030802711844444275, + 0.10662112385034561, + -1.3358653783798218, + -0.06966105103492737, + 0.8762516379356384, + 0.17116057872772217, + 0.029481906443834305, + -0.3574213981628418, + 0.29465121030807495, + -0.7650296092033386, + -0.015099853277206421, + 0.5462791919708252 + ], + [ + -0.559061586856842, + 0.14502456784248352, + 0.45190829038619995, + -1.0738792419433594, + -1.5723164081573486, + -0.5392160415649414, + 0.7303147315979004, + -6.195902824401855e-05, + -0.09901300817728043, + 0.10049861669540405, + 0.051869314163923264, + -1.1138311624526978, + 0.033548980951309204, + 0.33838987350463867, + -0.11065022647380829, + 0.03068166971206665, + -0.1765679270029068, + 0.17674095928668976, + -0.6770181059837341, + -0.363102525472641, + 0.18358556926250458 + ], + [ + -0.810593843460083, + 0.18474450707435608, + 0.5427998900413513, + -1.2668769359588623, + -2.039551258087158, + 0.170603409409523, + 0.5517785549163818, + 0.030628055334091187, + -0.008206840604543686, + 0.023059621453285217, + 0.11736996471881866, + -1.446105718612671, + 0.020194947719573975, + 0.6126370429992676, + 0.044161491096019745, + 0.0003640996292233467, + -0.31732720136642456, + 0.24351078271865845, + -0.7596177458763123, + 0.2315993309020996, + 0.34795206785202026 + ], + [ + -0.8680835962295532, + 0.12034939229488373, + 0.8356432914733887, + -1.1642320156097412, + -2.1295688152313232, + -0.17883101105690002, + 0.3872016668319702, + -0.05696237087249756, + 0.0003417814150452614, + 0.014565914869308472, + -0.0069030895829200745, + -1.2308298349380493, + -0.11796659231185913, + 0.8523040413856506, + 0.11972692608833313, + 0.021913524717092514, + -0.2314680963754654, + 0.33641567826271057, + -0.6810120940208435, + 0.09716376662254333, + 0.28725558519363403 + ], + [ + -0.7122457027435303, + 0.18187060952186584, + 0.5787478089332581, + -1.2160292863845825, + -1.988480806350708, + 0.17790736258029938, + 0.5816813111305237, + 0.05637797713279724, + -0.05884978175163269, + 0.09957218915224075, + 0.14835883677005768, + -1.6232458353042603, + -0.06337305903434753, + 0.6757876873016357, + 0.21428130567073822, + -0.025791537016630173, + -0.3279285430908203, + 0.24416449666023254, + -0.9319389462471008, + 0.5412918925285339, + 0.054402247071266174 + ], + [ + -0.8143599033355713, + 0.1726515293121338, + 0.6953363418579102, + -1.250404953956604, + -2.039469003677368, + -0.08825115859508514, + 0.5160762071609497, + 0.063111811876297, + 0.07397442311048508, + -0.04727152734994888, + 0.19868794083595276, + -1.1970100402832031, + -0.25815117359161377, + 0.4863189458847046, + -0.17198443412780762, + 0.06569875776767731, + -0.2151908427476883, + 0.3011682629585266, + -0.7800189852714539, + 0.37843602895736694, + 0.6221891641616821 + ], + [ + -1.1280444860458374, + 0.2126903384923935, + 0.3045387268066406, + -1.3631364107131958, + -2.1111292839050293, + 0.27852940559387207, + 0.5967617630958557, + 0.19003501534461975, + -0.06598386913537979, + 0.22563880681991577, + 0.1391657292842865, + -1.6498486995697021, + -0.7170576453208923, + 0.4736671447753906, + 0.08535799384117126, + -0.06540680676698685, + -0.17089809477329254, + 0.21565906703472137, + -0.568437933921814, + 0.5021947622299194, + 0.39886555075645447 + ], + [ + -0.37826770544052124, + 0.1162397563457489, + 0.7141335606575012, + -0.8773645162582397, + -1.4110991954803467, + 0.015673592686653137, + 0.07687386870384216, + 0.025286629796028137, + -0.06477708369493484, + 0.0015392526984214783, + 0.06454146653413773, + -0.9371962547302246, + 0.03644940257072449, + 0.24814444780349731, + -0.11165717244148254, + 0.0296492800116539, + 0.07360316812992096, + 0.026344329118728638, + -0.5505878925323486, + 0.08845961093902588, + 0.28262630105018616 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 0.11496219038963318, + 2.0, + 2.0, + 0.1255601942539215, + 2.0 + ], + "group_id": "ms-22f153f0d4fa656018e6", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 2.60770320892334e-06, + 3.129243850708008e-07, + 6.109476089477539e-07, + 2.0, + 0.20781123638153076, + 2.0, + 0.16136802732944489, + 0.15920132398605347, + 0.16996727883815765, + 0.14405608177185059, + 0.15874098241329193, + 0.08526891469955444, + 0.0828208178281784, + 2.0, + 0.034265756607055664, + 2.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.3937166929244995, + 0.17073306441307068, + -0.5225132703781128, + -0.8479964733123779, + -1.713785171508789, + -0.19215916097164154, + 0.0, + 0.8004336953163147, + 0.4925512969493866, + 0.3161412179470062, + 0.10414126515388489, + -0.9857386350631714, + -0.4383167624473572, + 0.0, + 0.05412057042121887, + 0.021832358092069626, + 0.5262433290481567, + -0.24631118774414062, + -0.20008409023284912, + -0.04257836565375328, + 0.0 + ], + [ + -1.3937166929244995, + 0.2384568303823471, + 0.3933030962944031, + -1.847996473312378, + -2.713785171508789, + 0.20405007898807526, + 1.2454378604888916, + 0.0006654821336269379, + -0.07783810049295425, + -0.002305123955011368, + 0.18527865409851074, + -1.9857386350631714, + -0.009703370742499828, + 0.7078995108604431, + -0.08121174573898315, + -0.031056255102157593, + -0.40927961468696594, + 0.5954965949058533, + -0.8039777874946594, + 0.05187072232365608, + 0.2283477634191513 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.490875720977783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4157990217208862, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4566955268383026 + ], + [ + -1.5219146013259888, + 0.23501667380332947, + 0.5663670301437378, + -1.892876386642456, + -2.744781494140625, + -0.04844887554645538, + 1.3053655624389648, + -0.11823110282421112, + -0.02477371320128441, + -0.17061075568199158, + 0.22949494421482086, + -1.854048252105713, + -0.025548618286848068, + 0.7958164215087891, + 0.12320645153522491, + -0.23218455910682678, + -0.6995364427566528, + 0.8081986308097839, + -0.558222234249115, + -0.18823161721229553, + 0.04578563570976257 + ], + [ + -1.608424186706543, + 0.022089049220085144, + 0.15639334917068481, + -1.982436180114746, + -2.6509876251220703, + -0.01820056140422821, + 0.95000159740448, + -0.08617040514945984, + -0.46914470195770264, + 0.3453706204891205, + -0.020491018891334534, + -1.738605260848999, + -0.0859340950846672, + 0.443394273519516, + 0.09907735884189606, + -0.38721394538879395, + -0.3415156602859497, + 0.524202287197113, + -0.7343614101409912, + -0.22556106746196747, + 0.4535057842731476 + ], + [ + -1.7957028150558472, + 0.593864381313324, + 0.06846150755882263, + -2.0563552379608154, + -2.2595136165618896, + -0.12188448011875153, + 0.9899688959121704, + -0.14783401787281036, + -0.22327497601509094, + -0.32116562128067017, + -0.07331573963165283, + -1.910165786743164, + 0.264303058385849, + 1.0567166805267334, + 0.016188345849514008, + 0.159999281167984, + -0.22885318100452423, + 0.705758273601532, + -0.8778964877128601, + 0.4381847381591797, + 0.19897118210792542 + ], + [ + -1.499897837638855, + -0.2043466418981552, + 0.42094048857688904, + -1.430327296257019, + -3.1506710052490234, + -0.08664844930171967, + 1.4003938436508179, + 0.24064289033412933, + -0.35471564531326294, + -0.37839531898498535, + 0.47059693932533264, + -1.7703200578689575, + 0.101543128490448, + 0.8470064401626587, + 0.16171279549598694, + -0.0019259154796600342, + -0.6455138325691223, + 0.5694639682769775, + -0.9997703433036804, + 0.5366826057434082, + -0.11106891930103302 + ], + [ + -1.8847153186798096, + 0.21543154120445251, + 0.9309400320053101, + -2.1095237731933594, + -2.136512517929077, + 0.6948637366294861, + 1.233637809753418, + 0.577111542224884, + -0.5727867484092712, + -0.6055393815040588, + 0.8738593459129333, + -2.6654083728790283, + -0.171565979719162, + 0.8934992551803589, + 0.17394188046455383, + 0.6209473609924316, + -0.1149752140045166, + 0.6160256266593933, + -0.5519447326660156, + 0.7195507287979126, + 0.5175096392631531 + ], + [ + -2.138639450073242, + 0.36738795042037964, + -0.33481496572494507, + -1.9255597591400146, + -2.2420146465301514, + 0.31966179609298706, + 1.3172208070755005, + -0.48551350831985474, + 0.27687638998031616, + -0.41806119680404663, + 0.05362778902053833, + -2.3665032386779785, + -0.47635504603385925, + 0.081889808177948, + 0.542697548866272, + -0.7215265035629272, + -0.906980037689209, + 0.6977691054344177, + -1.4041197299957275, + -0.11211369931697845, + 0.5149455070495605 + ], + [ + -1.4869698286056519, + -0.30130720138549805, + 1.28191077709198, + -0.9839465618133545, + -2.2501890659332275, + -0.20459605753421783, + 1.3607591390609741, + 0.6349532008171082, + -0.8627415895462036, + -0.623334527015686, + 0.21189503371715546, + -2.2093186378479004, + -0.4655338525772095, + 0.7016672492027283, + 0.6916180849075317, + -0.0807163268327713, + 0.1465948522090912, + 0.05083376169204712, + -0.8381134271621704, + 0.015291888266801834, + 0.05565965175628662 + ], + [ + -2.365525245666504, + 0.45377761125564575, + 0.2984481453895569, + -1.3690096139907837, + -2.843494176864624, + 0.32462257146835327, + 0.5082458257675171, + 0.7033615112304688, + 0.31817564368247986, + -0.4890395700931549, + 0.09036337584257126, + -2.904804229736328, + 0.560937762260437, + 1.649811029434204, + -0.9795587062835693, + -0.46419358253479004, + -0.4739357829093933, + 0.34800490736961365, + -1.0712809562683105, + 0.9167320132255554, + -0.48252081871032715 + ], + [ + -1.2987136840820312, + 0.7370830774307251, + -0.49668365716934204, + -1.201794147491455, + -3.552318572998047, + -0.5406968593597412, + 2.1693286895751953, + 0.5925421714782715, + 0.7903083562850952, + 0.991620659828186, + 0.7936639785766602, + -2.3832802772521973, + -0.621701180934906, + 0.43542394042015076, + -0.14415577054023743, + 0.15921397507190704, + -0.395956814289093, + -0.2778439521789551, + -1.069305419921875, + -0.5347616076469421, + -0.26581722497940063 + ] + ], + "predicted_scores": [ + 4.533088684082031, + 4.359924793243408, + 4.246114253997803, + 3.96541428565979, + 3.6263058185577393, + 3.6202778816223145, + 3.4237430095672607, + 2.6584508419036865 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007/train2", + "selected_index": 0, + "state_hash": "3139b3ac90f6fb958f14d875d082c95c8569f039086e0789e6c9e079d602b945", + "stored_base_utility": 2.60770320892334e-06, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0320269912481308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0320269912481308 + }, + "base_progress": 0.0320269912481308, + "base_success": false, + "base_utility": 0.0320269912481308, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.040287479758262634, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.040287479758262634 + }, + { + "contact_quality": null, + "progress": 0.0174645334482193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0174645334482193 + }, + { + "contact_quality": null, + "progress": 0.021355673670768738, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021355673670768738 + }, + { + "contact_quality": null, + "progress": 0.018107041716575623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018107041716575623 + }, + { + "contact_quality": null, + "progress": 0.018239781260490417, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018239781260490417 + }, + { + "contact_quality": null, + "progress": 0.022067666053771973, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022067666053771973 + }, + { + "contact_quality": null, + "progress": 0.038215383887290955, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.038215383887290955 + }, + { + "contact_quality": null, + "progress": 0.019529670476913452, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019529670476913452 + } + ], + "candidate_progress": [ + 0.040287479758262634, + 0.0174645334482193, + 0.021355673670768738, + 0.018107041716575623, + 0.018239781260490417, + 0.022067666053771973, + 0.038215383887290955, + 0.019529670476913452 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-ad5107846b45eae5a0ab", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-543339410c02ce2269d1", + "ms-2423891634e7b469eed8" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "generated_tangents": [ + [ + 0.14018268883228302, + 0.289065420627594, + 0.8483375310897827, + -1.7791224718093872, + 0.4814610779285431, + -0.18178966641426086, + 0.12895378470420837, + 0.069358691573143, + -0.059383705258369446, + 0.035258613526821136, + 0.13515865802764893, + -0.5167298913002014, + 0.050205498933792114, + 0.11525478214025497, + 0.0007116049528121948, + -0.15094910562038422, + 0.04490123689174652, + -0.037689417600631714, + -0.5086848139762878, + -0.211318701505661, + -0.05127803981304169 + ], + [ + 0.17016763985157013, + 0.34602171182632446, + 0.9436938166618347, + -1.9131704568862915, + 0.4846016764640808, + -0.19932904839515686, + 0.4778391718864441, + -0.1454114317893982, + -0.04693783074617386, + 0.11983165889978409, + 0.05753686651587486, + -0.5236656665802002, + -0.26676976680755615, + 0.40207332372665405, + 0.04501873999834061, + -0.08188727498054504, + 0.19682645797729492, + -0.2202015370130539, + -0.7720839381217957, + -0.12631845474243164, + -0.1130034327507019 + ], + [ + 0.1542426496744156, + 0.24253727495670319, + 0.9044508934020996, + -1.724196195602417, + 0.46095937490463257, + -0.05063405632972717, + 0.19632071256637573, + -0.022570163011550903, + 0.04484015703201294, + 0.12057716399431229, + 0.12402200698852539, + -0.47842562198638916, + 0.07320907711982727, + 0.05156547576189041, + 0.004217091947793961, + -0.1772926151752472, + -0.08882392942905426, + -0.1795724779367447, + -0.670177698135376, + -0.034486472606658936, + -0.07198132574558258 + ], + [ + 0.2664527893066406, + 0.23964805901050568, + 0.7832643389701843, + -1.741670846939087, + 0.2732876241207123, + -0.0017767548561096191, + 0.42830294370651245, + 0.06510612368583679, + 0.08872084319591522, + 0.15838471055030823, + 0.2198942005634308, + -0.402571439743042, + -0.006391555070877075, + 0.34281936287879944, + -0.12248881161212921, + -0.23056839406490326, + -0.10353462398052216, + -0.16413728892803192, + -0.4925073981285095, + 0.07817566394805908, + -0.022388413548469543 + ], + [ + 0.2701839804649353, + 0.24286682903766632, + 0.7585902214050293, + -1.760545015335083, + 0.40861284732818604, + -0.050659388303756714, + 0.45358529686927795, + -0.009043246507644653, + 0.10340894758701324, + 0.21402087807655334, + 0.1538773477077484, + -0.46348661184310913, + 0.06407955288887024, + 0.2675989270210266, + -0.03547931835055351, + -0.20605412125587463, + -0.11561249196529388, + -0.21442778408527374, + -0.6482141613960266, + 0.11941751837730408, + -0.1081092357635498 + ], + [ + 0.2603517770767212, + 0.29136765003204346, + 0.8377959728240967, + -1.7505241632461548, + 0.45684218406677246, + -0.02211475372314453, + 0.1808648705482483, + -0.02677808701992035, + 0.09574203193187714, + 0.0705883800983429, + 0.11149191856384277, + -0.38531720638275146, + -0.01217949390411377, + 0.1385795623064041, + -0.01819026842713356, + -0.1604638397693634, + -0.08709345757961273, + -0.17443491518497467, + -0.6340339779853821, + -0.24893814325332642, + 0.13137471675872803 + ], + [ + 0.10570760071277618, + 0.2851371765136719, + 0.8287855982780457, + -1.7305612564086914, + 0.5774431228637695, + -0.0024221092462539673, + 0.18119779229164124, + 0.023360982537269592, + 0.08462734520435333, + 0.09469969570636749, + 0.18955576419830322, + -0.4194323420524597, + -0.09402373433113098, + 0.10175512731075287, + -0.06348559260368347, + -0.12264516949653625, + -0.009652271866798401, + -0.08393912017345428, + -0.5841110944747925, + -0.0612044632434845, + 0.002477407455444336 + ], + [ + 0.22996777296066284, + 0.24743469059467316, + 0.7988656759262085, + -1.8694814443588257, + 0.2585720121860504, + 0.1517871767282486, + 0.6424685716629028, + 0.05439126491546631, + 0.15564747154712677, + 0.11952495574951172, + 0.09387843310832977, + -0.5468964576721191, + 0.1126844584941864, + 0.07026393711566925, + -0.02239188179373741, + -0.06617730110883713, + -0.18370158970355988, + -0.178851917386055, + -0.7413548827171326, + 0.10490772128105164, + -0.14298400282859802 + ] + ], + "generated_utilities": [ + 0.040287479758262634, + 0.0174645334482193, + 0.021355673670768738, + 0.018107041716575623, + 0.018239781260490417, + 0.022067666053771973, + 0.038215383887290955, + 0.019529670476913452 + ], + "group_id": "ms-27c9f70e97b8850cc3ab", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.09440004825592041, + 0.14095592498779297, + 2.0, + 0.09219714999198914, + 0.05765572190284729, + 0.02530825138092041, + 0.07676126062870026, + 0.06643831729888916, + 0.047928065061569214, + 0.07107676565647125, + 0.05561619997024536, + 0.02124820649623871, + 0.08948668837547302, + 0.10203869640827179, + 0.06119762361049652, + 0.04547174274921417 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.984761953353882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.7230467796325684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.554086923599243 + ], + [ + 1.0343247652053833, + 1.287368655204773, + 1.867376446723938, + -2.4465248584747314, + 1.036996841430664, + -1.2664310932159424, + 1.4399157762527466, + -0.8265262246131897, + -0.28319495916366577, + 0.16331201791763306, + -1.5917816162109375, + -1.5816115140914917, + -0.1664504110813141, + 0.7195659279823303, + 0.29144588112831116, + -0.18517816066741943, + -0.08826595544815063, + -0.8266173601150513, + -0.7933371067047119, + -0.938305139541626, + 0.5359267592430115 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3968086242675781, + 0.5359851121902466, + 0.7176156044006348, + -1.52370023727417, + -0.1481943428516388, + 0.10887271165847778, + 0.0, + -0.12331700325012207, + 0.14385993778705597, + -0.041522324085235596, + 0.16089874505996704, + -0.6189663410186768, + -0.01699581742286682, + 0.0, + -0.07216246426105499, + -0.15073662996292114, + -0.134138286113739, + -0.06049776077270508, + -0.26776671409606934, + -0.019470810890197754, + 0.0 + ] + ], + "predicted_scores": [ + 2.6910204887390137, + 2.5916354656219482, + 2.4882662296295166, + 2.3931052684783936, + 2.295912742614746, + 2.16906476020813, + 2.1331605911254883, + 1.6332793235778809 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1881223807/train2", + "selected_index": 0, + "state_hash": "0f58a11c36c331c9abab00795189dccbc7e0aebf985ca9df67bf9da756f154e3", + "stored_base_utility": 0.09440004825592041, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.10841864347457886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10841864347457886 + }, + "base_progress": 0.10841864347457886, + "base_success": false, + "base_utility": 0.10841864347457886, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03331609070301056, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03331609070301056 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.03331609070301056, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2867deaca7e98582620b", + "generated_tangents": [ + [ + 0.16600991785526276, + 0.18908433616161346, + 0.33548200130462646, + -0.9058983325958252, + -1.540191411972046, + -0.1795315444469452, + -0.871669352054596, + -0.3932388126850128, + -0.15279380977153778, + 0.7445515394210815, + 0.05246046185493469, + -1.212005376815796, + 0.053169697523117065, + -0.07310402393341064, + -0.26925426721572876, + 0.06416788697242737, + 0.40964382886886597, + 0.3355107307434082, + -0.28585803508758545, + -0.3497779667377472, + -0.6202583312988281 + ], + [ + 0.07943939417600632, + 0.05913250148296356, + 0.6852797269821167, + -0.8930180072784424, + -1.4039890766143799, + -0.39491453766822815, + -1.018752098083496, + -0.18416669964790344, + 0.002063695341348648, + 0.6112512946128845, + 0.4535205662250519, + -1.0310211181640625, + -0.006541192531585693, + 0.08287917077541351, + -0.17422263324260712, + 0.04211186617612839, + 0.2024422585964203, + 0.3467860817909241, + -0.17872291803359985, + -0.18163183331489563, + -0.13720151782035828 + ], + [ + -0.04607692360877991, + 0.09997068345546722, + 0.6241083741188049, + -0.5203796625137329, + -1.1481754779815674, + -0.17407873272895813, + -1.150116205215454, + -0.22077441215515137, + 0.05634499341249466, + 0.5917983651161194, + 0.5587509870529175, + -1.1554293632507324, + 0.06377473473548889, + -0.16887415945529938, + -0.010603468865156174, + 0.09912288188934326, + 0.5224568247795105, + 0.060127824544906616, + -0.22322219610214233, + 0.021599620580673218, + -0.8569719791412354 + ], + [ + 0.09243831783533096, + 0.06287702918052673, + 0.7539478540420532, + -1.0097072124481201, + -1.5342717170715332, + -0.6428382396697998, + -1.2201895713806152, + -0.27795061469078064, + -0.02100360207259655, + 0.5988133549690247, + 0.28385302424430847, + -1.287218451499939, + 0.13356050848960876, + 0.13597771525382996, + -0.18391810357570648, + 0.07739007472991943, + 0.3036441206932068, + 0.30869847536087036, + -0.17749691009521484, + 0.09222644567489624, + -0.20486858487129211 + ], + [ + -0.08986543118953705, + 0.0525314062833786, + 0.6842445135116577, + -0.4866493046283722, + -1.1337988376617432, + -0.20357748866081238, + -1.1288089752197266, + -0.20260179042816162, + 0.05673578381538391, + 0.5497421622276306, + 0.568764328956604, + -1.12516450881958, + 0.026677489280700684, + 0.027614064514636993, + -0.026013080030679703, + 0.08397582173347473, + 0.5806421041488647, + 0.04677726328372955, + -0.19632792472839355, + 0.1386643946170807, + -0.8952764272689819 + ], + [ + -0.21484331786632538, + 0.35670602321624756, + 0.2919608950614929, + -1.321199893951416, + -1.2454345226287842, + -0.1845996379852295, + -1.170832872390747, + -0.4199824035167694, + -0.04239681363105774, + 0.45418572425842285, + 0.44296663999557495, + -1.0982940196990967, + -0.12045922875404358, + -0.027421846985816956, + -0.1989881694316864, + 0.27994588017463684, + 0.45071840286254883, + 0.3256455063819885, + -0.2464848756790161, + -0.6050560474395752, + -0.5589829683303833 + ], + [ + 0.13482598960399628, + 0.3827795386314392, + 0.5731130242347717, + -1.3230628967285156, + -1.481137752532959, + -0.6478264331817627, + -1.032254695892334, + -0.1686381697654724, + 0.022745568305253983, + 0.34474611282348633, + 0.3553159832954407, + -1.062491536140442, + -0.06841418147087097, + 0.030543625354766846, + -0.219338059425354, + 0.2924506664276123, + 0.3867171108722687, + 0.581658661365509, + -0.06119430065155029, + -0.4303423762321472, + -0.4755922257900238 + ], + [ + 0.10067836940288544, + 0.30324530601501465, + 0.6015622615814209, + -1.295541763305664, + -1.3804712295532227, + -0.4262353479862213, + -1.076366901397705, + -0.20897558331489563, + 0.02361627109348774, + 0.40963461995124817, + 0.35281074047088623, + -0.9959433078765869, + -0.1752685010433197, + 0.004814960062503815, + -0.17192019522190094, + 0.3385526239871979, + 0.4245295226573944, + 0.6542333364486694, + -0.09239929914474487, + -0.3336179554462433, + -0.453787237405777 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.03331609070301056, + 2.0, + 2.0 + ], + "group_id": "ms-2867deaca7e98582620b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0924256443977356, + 0.05646410584449768, + 2.0, + 2.0, + 0.183049738407135, + 2.0, + 0.16824382543563843, + 0.1424575299024582, + 0.08230158686637878, + 0.15871968865394592, + 0.16382521390914917, + 0.05336849391460419, + 0.07674244046211243, + 0.10727545619010925, + 0.06594061851501465, + 0.045542001724243164 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.44511327147483826, + 0.05827039107680321, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.2619132101535797, + 0.0, + -0.37528133392333984, + -0.07968637347221375, + -0.07514983415603638, + 0.19317352771759033, + 0.03868463635444641, + -0.06793206930160522, + 0.0, + 0.13187909126281738, + -0.20905672013759613, + 0.030073702335357666, + 0.21597933769226074, + -0.0761370062828064, + -0.11662125587463379, + 0.0 + ], + [ + -0.15486939251422882, + 0.24427275359630585, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.44790399074554443, + 0.0, + -0.37528133392333984, + -0.29508399963378906, + 0.7311322093009949, + 0.19317352771759033, + 0.09925723075866699, + 0.1767415851354599, + 0.0, + -0.5768744349479675, + 0.1300974041223526, + 0.21151533722877502, + 0.21597933769226074, + 0.04431194067001343, + -0.040631115436553955, + 0.0 + ], + [ + -0.29166290163993835, + 0.09681949019432068, + 2.2608249187469482, + -1.3577959537506104, + -2.195330858230591, + -0.13884210586547852, + 2.5595173835754395, + -1.3752813339233398, + 0.390796959400177, + 0.673005223274231, + 1.1931735277175903, + -0.41860872507095337, + -0.24474714696407318, + 0.3015621602535248, + -0.2400326132774353, + 0.14207373559474945, + -0.26512715220451355, + 1.2159793376922607, + -0.3237651586532593, + -0.7466862201690674, + 0.30001604557037354 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.119034767150879, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6031243205070496, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6000320911407471 + ], + [ + -0.3580354154109955, + 0.03320832550525665, + 2.5546932220458984, + -1.138798713684082, + -2.260084629058838, + -0.04964215308427811, + 2.8354909420013428, + -1.6732244491577148, + 0.21277914941310883, + 0.5097208023071289, + 1.387328028678894, + -0.691602349281311, + -0.28483790159225464, + 0.35576167702674866, + -0.0605899840593338, + 0.3764685094356537, + -0.28031954169273376, + 1.063202977180481, + -0.03841540217399597, + -0.5410391092300415, + 0.5580100417137146 + ], + [ + -0.5578174591064453, + 0.3727092146873474, + 2.1929497718811035, + -1.577579140663147, + -1.8962993621826172, + -0.2802738845348358, + 2.1670117378234863, + -1.0752147436141968, + 0.16372516751289368, + 0.6087679862976074, + 0.8854888081550598, + -0.18080148100852966, + -0.18821784853935242, + 0.010591715574264526, + -0.12560874223709106, + 0.15850795805454254, + -0.1898641288280487, + 1.1823316812515259, + 0.06655246019363403, + -0.4014361798763275, + 0.4376043379306793 + ], + [ + -0.43790310621261597, + -0.12778332829475403, + 2.6857643127441406, + -1.6293541193008423, + -1.9850949048995972, + -0.7167636752128601, + 2.465301513671875, + -1.4102312326431274, + 0.2071499079465866, + 0.5742072463035583, + 0.9525313377380371, + -0.34116798639297485, + -0.04760183393955231, + -0.05569759011268616, + -0.4777999520301819, + 0.17186680436134338, + 0.0841393768787384, + 0.7708102464675903, + -0.2784478962421417, + -1.2458651065826416, + 0.46140795946121216 + ], + [ + 0.06170770525932312, + 0.45555436611175537, + 1.6506444215774536, + -0.9394457340240479, + -1.9117670059204102, + -0.6910014152526855, + 2.40145206451416, + -1.7531975507736206, + 0.10217079520225525, + 1.2514894008636475, + 1.3236911296844482, + -0.6504120826721191, + -0.6943472027778625, + 0.7029284238815308, + -0.7387814521789551, + -0.33777421712875366, + -0.8187699317932129, + 1.8448829650878906, + 0.00849190354347229, + -1.394152045249939, + 0.918946385383606 + ] + ], + "predicted_scores": [ + 4.198071479797363, + 3.129955768585205, + 3.086650848388672, + 3.0081777572631836, + 2.726970672607422, + 2.0165863037109375, + 1.782904863357544, + 1.744734764099121 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1189392917/train2", + "selected_index": 0, + "state_hash": "e8cf9b95f09f32d8e7942b114792725bd89c392a97ff7e6468ebf09df1bb0d21", + "stored_base_utility": 0.0924256443977356, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.13927479088306427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13927479088306427 + }, + "base_progress": 0.13927479088306427, + "base_success": false, + "base_utility": 0.13927479088306427, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 3.081560134887695e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 3.081560134887695e-05 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 3.081560134887695e-05, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-365d8c6de6de16f66744", + "generated_tangents": [ + [ + -0.21114064753055573, + -0.048341378569602966, + 1.5556553602218628, + -0.46712633967399597, + -1.2199385166168213, + -0.11526814103126526, + -0.16461020708084106, + 0.05351035296916962, + -0.026479950174689293, + 0.16385047137737274, + 0.03798053413629532, + -0.9300800561904907, + -0.2876429557800293, + -0.07136794924736023, + 0.08146791905164719, + -0.026538491249084473, + 0.2775648832321167, + 0.22740747034549713, + -0.16802898049354553, + -0.20614074170589447, + 0.10743649303913116 + ], + [ + -0.1791655272245407, + -0.02371925674378872, + 1.6634361743927002, + -0.5424115657806396, + -1.3346848487854004, + 0.08280134201049805, + -0.23706042766571045, + -0.022989988327026367, + 0.016705770045518875, + 0.11423534154891968, + -0.03222975879907608, + -0.9869319200515747, + -0.16671717166900635, + 0.026621557772159576, + 0.031675536185503006, + -0.012221254408359528, + 0.2166295051574707, + 0.12822693586349487, + -0.07551956176757812, + -0.2610858082771301, + 0.051014289259910583 + ], + [ + -0.22534425556659698, + -0.08131547272205353, + 1.5393378734588623, + -0.5683905482292175, + -1.2945749759674072, + 0.07191448658704758, + -0.19457030296325684, + -0.020548522472381592, + -0.002910829149186611, + 0.16843777894973755, + -0.030726857483386993, + -1.1023364067077637, + -0.06796851754188538, + -0.17109696567058563, + -0.029572751373052597, + -0.02294149249792099, + 0.20655836164951324, + 0.15574046969413757, + -0.038037896156311035, + -0.20111629366874695, + 0.032548144459724426 + ], + [ + -0.06677797436714172, + 0.0060897208750247955, + 1.807574987411499, + -0.5565886497497559, + -1.4104347229003906, + -0.1659129559993744, + -0.35193121433258057, + 0.03796319663524628, + 0.0001925518736243248, + 0.07528190314769745, + -0.019746683537960052, + -1.056929349899292, + -0.1898556351661682, + 0.018602512776851654, + 0.027445875108242035, + -0.054877590388059616, + 0.1442004144191742, + 0.10452555865049362, + -0.09021466970443726, + -0.16082943975925446, + 0.11316534876823425 + ], + [ + -0.09619469940662384, + -0.05190189555287361, + 1.875879168510437, + -0.7030974626541138, + -1.4674580097198486, + -0.010268077254295349, + -0.5517680048942566, + 0.09250915050506592, + -0.09007631987333298, + 0.2208840250968933, + -0.04731278866529465, + -0.9954752922058105, + 0.28448495268821716, + -0.19590722024440765, + -0.08933983743190765, + 0.04485812783241272, + 0.2518169581890106, + 0.20833410322666168, + 0.01020514965057373, + -0.22327697277069092, + -0.3381204903125763 + ], + [ + -0.11923225224018097, + -0.10508862137794495, + 1.6521183252334595, + -0.7320802211761475, + -1.3820405006408691, + -0.09266479313373566, + -0.9676738381385803, + -0.013583093881607056, + 0.007066716905683279, + 0.1558513641357422, + -0.04582802206277847, + -0.9558316469192505, + 0.15382614731788635, + -0.16842050850391388, + -0.04179012402892113, + -0.013198169879615307, + 0.14783084392547607, + 0.3141729533672333, + -0.07083505392074585, + 0.0677485466003418, + -0.05980193614959717 + ], + [ + -0.2758737802505493, + 0.04080096632242203, + 1.3296016454696655, + -0.49317029118537903, + -1.1362073421478271, + 0.2666260600090027, + -0.5547595620155334, + -0.1566736102104187, + -0.08262604475021362, + 0.11275874823331833, + -0.14412081241607666, + -1.1537481546401978, + -0.27469709515571594, + 0.10920702666044235, + -0.0987781435251236, + -0.08442221581935883, + 0.212855264544487, + 0.11175546050071716, + 0.001248687505722046, + -0.29110726714134216, + -0.05398714542388916 + ], + [ + -0.07298408448696136, + -0.07261212170124054, + 1.5537549257278442, + -0.6544216871261597, + -1.4192097187042236, + -0.346295565366745, + -0.33383774757385254, + -0.0756678581237793, + -0.027217773720622063, + 0.10626167058944702, + 0.041009142994880676, + -1.1146725416183472, + -0.11862128973007202, + 0.13409720361232758, + -0.10111531615257263, + -0.11558949947357178, + 0.06267830729484558, + 0.07639651745557785, + 0.033487915992736816, + -0.17422418296337128, + 0.029652714729309082 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 3.081560134887695e-05, + 2.0 + ], + "group_id": "ms-365d8c6de6de16f66744", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20231953263282776, + 0.12319359183311462, + 2.0, + 2.0, + 0.1590207815170288, + 0.019343823194503784, + 0.1322992593050003, + 0.09479346871376038, + 0.1768283247947693, + 0.1054161787033081, + 0.16278749704360962, + 0.08510464429855347, + 0.09564858675003052, + 0.07681193947792053, + 0.1334480196237564, + 0.08148902654647827 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.09058855473995209, + -0.022760659456253052, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.07837581634521484, + 0.0, + -0.05228453874588013, + -0.0679071694612503, + 0.026890814304351807, + 0.03908693790435791, + -1.4526653289794922, + 0.04786187410354614, + 0.0, + 0.019858457148075104, + -0.05534793436527252, + 0.17386651039123535, + -0.024165034294128418, + -0.05880159139633179, + 0.044484615325927734, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.063174724578857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6561365127563477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0078073740005493 + ], + [ + 0.05168074369430542, + 0.27186232805252075, + 2.0911388397216797, + -1.6812132596969604, + -2.3280625343322754, + 0.38023167848587036, + 2.5090978145599365, + -0.2604735791683197, + 0.04725922644138336, + 0.8121852874755859, + -0.6659450531005859, + -2.2462568283081055, + 0.49988994002342224, + 1.9786770343780518, + 0.02324119210243225, + -0.45050501823425293, + 1.1065142154693604, + 0.5724118947982788, + -0.6320602893829346, + -0.7826599478721619, + 0.47463753819465637 + ], + [ + 0.46382713317871094, + -0.06331050395965576, + 2.657292366027832, + -1.1560972929000854, + -2.765941619873047, + 0.7805119752883911, + 2.3117406368255615, + -0.3644295036792755, + -0.25628525018692017, + 0.7623568773269653, + -0.4176616668701172, + -2.646364688873291, + 0.40904325246810913, + 1.7654002904891968, + -0.23315654695034027, + 0.026929572224617004, + 0.8236149549484253, + 0.8322737812995911, + -0.7612741589546204, + -0.4725078046321869, + 0.5895571708679199 + ], + [ + -0.26226741075515747, + 0.3922410011291504, + 2.450082302093506, + -0.9020822048187256, + -3.028742551803589, + 1.0746917724609375, + 3.104015350341797, + 0.3752234876155853, + -0.009036362171173096, + 0.3652568459510803, + -0.7608495950698853, + -2.337242603302002, + -0.03944078087806702, + 1.972575306892395, + -0.32381975650787354, + -0.45855021476745605, + 1.6927499771118164, + 0.8358478546142578, + -0.8610806465148926, + -0.6741821765899658, + 0.4013170003890991 + ], + [ + 0.5790615677833557, + 0.8308188915252686, + 3.141986846923828, + -1.218785285949707, + -2.0300495624542236, + 0.4300342798233032, + 3.0737316608428955, + -0.09167198091745377, + 0.4237035810947418, + 0.7421274185180664, + -0.637946367263794, + -2.1245827674865723, + 0.4184945523738861, + 1.2377147674560547, + -0.4068492650985718, + -0.03615438938140869, + 1.5385099649429321, + 0.51850825548172, + -0.5737811923027039, + -1.472043752670288, + 0.45800793170928955 + ], + [ + 0.38714414834976196, + 0.25513744354248047, + 2.9400248527526855, + -1.8116681575775146, + -1.7500431537628174, + 1.403240442276001, + 1.9709672927856445, + 0.6755956411361694, + 0.49445489048957825, + 0.19490671157836914, + -1.1990764141082764, + -2.803361177444458, + 0.499578058719635, + 1.164682149887085, + 0.9588776230812073, + 0.5154257416725159, + 1.0328537225723267, + 1.1309876441955566, + -0.9365650415420532, + -0.9663279056549072, + -0.032733798027038574 + ], + [ + 0.7417005896568298, + -0.12406697869300842, + 3.0325844287872314, + -0.5656485557556152, + -2.5084800720214844, + 0.27374324202537537, + 2.2886011600494385, + 0.5997821092605591, + 0.9623891115188599, + 0.38451504707336426, + -1.4501185417175293, + -1.6908833980560303, + 0.6743149757385254, + 1.8593690395355225, + -0.018024832010269165, + -0.20444278419017792, + 1.364742636680603, + 0.34230074286460876, + -1.4427728652954102, + -1.5577378273010254, + 0.5589802265167236 + ], + [ + 0.3448556661605835, + -0.5947309732437134, + 2.3888885974884033, + -1.0376405715942383, + -2.218167781829834, + 1.568857192993164, + 3.0995821952819824, + -0.4682580530643463, + 0.3072889745235443, + 0.14532917737960815, + -0.567415177822113, + -3.085655689239502, + -0.6656016111373901, + 1.6447856426239014, + 0.8135664463043213, + 0.3758860230445862, + 1.399398922920227, + 0.11528515815734863, + -0.531513512134552, + -0.6104787588119507, + 1.4474445581436157 + ], + [ + -0.18966330587863922, + -0.7208725214004517, + 2.586766481399536, + -0.6973344087600708, + -3.2187445163726807, + 1.4117884635925293, + 3.0797414779663086, + -0.29149553179740906, + 0.09620378166437149, + 0.874613881111145, + -0.8004729151725769, + -2.0231733322143555, + -0.5948910713195801, + 1.5080900192260742, + -0.750367283821106, + 0.8155628442764282, + 1.6562600135803223, + 0.8091130256652832, + -1.1569771766662598, + -1.4121384620666504, + 0.3738836348056793 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06428352743387222, + -0.015524566173553467, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + -0.1607142686843872, + 0.0, + 0.27928173542022705, + 0.05664636939764023, + -0.03914147615432739, + 0.1700872778892517, + -1.4526653289794922, + 0.05555404722690582, + 0.0, + 0.18240894377231598, + -0.05239863693714142, + 0.17386651039123535, + -0.21921080350875854, + -0.3728579580783844, + 0.11250120401382446, + 0.0 + ], + [ + 0.008915487676858902, + 0.540733277797699, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.22902190685272217, + 0.0, + -0.3544532060623169, + -0.2348114550113678, + 0.04456782341003418, + 0.2907915711402893, + -1.4526653289794922, + -0.1788860559463501, + 0.0, + 0.07762694358825684, + -0.07040916383266449, + 0.17386651039123535, + 0.44059625267982483, + -0.19159233570098877, + 0.09149748086929321, + 0.0 + ] + ], + "predicted_scores": [ + 4.233409404754639, + 4.0162577629089355, + 3.88030743598938, + 3.4916210174560547, + 3.4586877822875977, + 3.002124309539795, + 2.914055109024048, + 2.882695436477661 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1997090716/train2", + "selected_index": 0, + "state_hash": "98870eab9f394b355af9b8a6d689b3bcd574af94db5d1a9139d0aed0b633b959", + "stored_base_utility": 0.20231953263282776, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03280790150165558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03280790150165558 + }, + "base_progress": 0.03280790150165558, + "base_success": false, + "base_utility": 0.03280790150165558, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01805901527404785, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01805901527404785 + }, + { + "contact_quality": null, + "progress": 0.022866547107696533, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022866547107696533 + }, + { + "contact_quality": null, + "progress": 0.019091889262199402, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019091889262199402 + }, + { + "contact_quality": null, + "progress": 0.01995408535003662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01995408535003662 + }, + { + "contact_quality": null, + "progress": 0.019698604941368103, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019698604941368103 + }, + { + "contact_quality": null, + "progress": 0.019849225878715515, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019849225878715515 + }, + { + "contact_quality": null, + "progress": 0.023350223898887634, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023350223898887634 + }, + { + "contact_quality": null, + "progress": 0.02442134916782379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02442134916782379 + } + ], + "candidate_progress": [ + 0.01805901527404785, + 0.022866547107696533, + 0.019091889262199402, + 0.01995408535003662, + 0.019698604941368103, + 0.019849225878715515, + 0.023350223898887634, + 0.02442134916782379 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-2423891634e7b469eed8", + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1", + "ms-2cbe83da02847008bf24", + "ms-fe38b164397dd669f391", + "ms-3f6cd47af5e9e6781b73", + "ms-2cbe83da02847008bf24" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-3c326d05fec66e1d6bea", + "generated_tangents": [ + [ + 0.19276104867458344, + -0.6077608466148376, + 0.9373341798782349, + 0.4416253864765167, + 0.2649969458580017, + 0.46473005414009094, + -0.03351306915283203, + 0.27680742740631104, + 0.34446781873703003, + 0.24360504746437073, + -0.15033817291259766, + -0.5768869519233704, + -0.4378374516963959, + 0.08635913580656052, + -0.16154083609580994, + -0.013578517362475395, + -0.08223603665828705, + -0.14786718785762787, + -0.074046790599823, + -0.4710044860839844, + -0.2585410475730896 + ], + [ + 0.30996179580688477, + -0.625074565410614, + 0.9759076833724976, + 0.3689461052417755, + 0.09832823276519775, + 0.756193995475769, + -0.7694090008735657, + 0.3742751479148865, + 0.5233266353607178, + 0.10121019184589386, + -0.186079740524292, + -0.33603256940841675, + -0.4210893213748932, + -0.0727536678314209, + -0.08566601574420929, + -0.054874129593372345, + -0.26122283935546875, + -0.2174142748117447, + -0.13606640696525574, + -0.1925308108329773, + -0.030809640884399414 + ], + [ + 0.3530561327934265, + -0.5663501024246216, + 1.0013673305511475, + 0.5904250144958496, + 0.2888363003730774, + 0.5083624124526978, + -0.3288395404815674, + 0.19990651309490204, + 0.3434184193611145, + 0.16889619827270508, + -0.09934317320585251, + -0.6318717002868652, + -0.1830020248889923, + 0.05097272992134094, + -0.24534952640533447, + -0.0378064326941967, + -0.14884088933467865, + -0.13635729253292084, + 0.16512149572372437, + -0.3850916028022766, + -0.03494034707546234 + ], + [ + 0.24989527463912964, + -0.6553949117660522, + 0.9435427784919739, + 0.471303254365921, + 0.3789020776748657, + 0.47036027908325195, + -0.6558746695518494, + 0.3336401581764221, + 0.4827704429626465, + 0.21932020783424377, + -0.16736462712287903, + -0.4875970482826233, + -0.32751625776290894, + -0.07440099120140076, + -0.24660229682922363, + -0.009617837145924568, + -0.18763534724712372, + -0.09053583443164825, + -0.1090857982635498, + -0.2659403085708618, + -0.1171875 + ], + [ + 0.41503775119781494, + -0.677285373210907, + 1.028478980064392, + 0.5291897058486938, + 0.22892804443836212, + 0.417287677526474, + -0.7241002917289734, + 0.2863876521587372, + 0.39558354020118713, + 0.04016329348087311, + -0.1694144904613495, + -0.4067331552505493, + -0.2850966453552246, + 0.17843478918075562, + -0.1777254045009613, + -0.01690167933702469, + -0.08967171609401703, + -0.1685670167207718, + 0.06325215101242065, + -0.23178468644618988, + 0.06850159913301468 + ], + [ + 0.2314331978559494, + -0.5535585880279541, + 1.000349998474121, + 0.38280996680259705, + 0.3449220657348633, + 0.41305145621299744, + -0.3596547245979309, + 0.43198442459106445, + 0.4153168499469757, + 0.052870988845825195, + 0.04645320028066635, + -0.5818946957588196, + -0.3390159606933594, + 0.17957070469856262, + -0.16262038052082062, + -0.019297663122415543, + -0.2034837156534195, + -0.03579661250114441, + 0.17443202435970306, + -0.13197749853134155, + -0.013713151216506958 + ], + [ + 0.10679545253515244, + -0.5216257572174072, + 1.1731065511703491, + 0.2876773178577423, + 0.2409408539533615, + -0.07592439651489258, + 0.08116781711578369, + 0.2347056269645691, + 0.3631366193294525, + 0.02719423919916153, + 0.10486897826194763, + -0.7611414790153503, + -0.33368831872940063, + -0.19680024683475494, + -0.2561682164669037, + -0.08563822507858276, + -0.008296146988868713, + -0.14190365374088287, + 0.03549067676067352, + -0.12387071549892426, + 0.13013368844985962 + ], + [ + 0.4368787407875061, + -0.6942615509033203, + 1.0927714109420776, + 0.4788568913936615, + 0.18266475200653076, + 0.31222712993621826, + -0.46198225021362305, + 0.31760334968566895, + 0.4469907283782959, + 0.07939180731773376, + -0.10216853767633438, + -0.4994339346885681, + -0.2430732399225235, + 0.25602757930755615, + -0.2264256775379181, + -0.017290612682700157, + -0.06057564914226532, + -0.19999904930591583, + 0.06390033662319183, + -0.13188783824443817, + 0.009188741445541382 + ] + ], + "generated_utilities": [ + 0.01805901527404785, + 0.022866547107696533, + 0.019091889262199402, + 0.01995408535003662, + 0.019698604941368103, + 0.019849225878715515, + 0.023350223898887634, + 0.02442134916782379 + ], + "group_id": "ms-3c326d05fec66e1d6bea", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18246883153915405, + 2.0, + 2.0, + 0.11388173699378967, + 0.035728439688682556, + 0.05159728229045868, + 0.043176501989364624, + 0.04580867290496826, + 0.02321702241897583, + 0.040450215339660645, + 0.04055213928222656, + 0.024657413363456726, + 0.017660364508628845, + 0.044850513339042664, + 0.031234994530677795, + 0.06195336580276489 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.018136948347091675, + 0.10074883699417114, + 0.11574286222457886, + 0.0, + 0.1410461664199829, + 0.38714247941970825, + 0.2381301373243332, + 0.06927800178527832, + -0.6138286590576172, + -0.33484986424446106, + 0.0, + 0.012610852718353271, + 0.5407634973526001, + 0.07784794270992279, + 0.3537199795246124, + -0.1602279543876648, + 0.002093076705932617, + 0.0 + ], + [ + 1.9327397346496582, + -1.7429183721542358, + 1.8877019882202148, + -0.33001917600631714, + 0.49746543169021606, + 0.47390231490135193, + 1.7802670001983643, + -0.8589538335800171, + 0.37976425886154175, + -0.18104670941829681, + -0.8859321475028992, + -1.6138286590576172, + 0.34242650866508484, + 1.2023069858551025, + 0.555707573890686, + -0.35764527320861816, + -0.10027600824832916, + 0.4702257812023163, + -0.6294202208518982, + 0.1926015019416809, + 0.5916687846183777 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.5605340003967285, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.404613971710205, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1833375692367554 + ], + [ + 1.7389304637908936, + -1.4791877269744873, + 2.158487319946289, + -0.3266654312610626, + 0.5907376408576965, + 0.43260636925697327, + 1.9136874675750732, + -0.5760036110877991, + 0.3071907162666321, + -0.4073694348335266, + -0.8863769769668579, + -1.8163083791732788, + 0.5969879627227783, + 1.196010947227478, + 0.7674136161804199, + -0.2368052750825882, + -0.12123071402311325, + 0.36782756447792053, + -0.8187789916992188, + 0.15793970227241516, + 0.43327897787094116 + ], + [ + 1.9560860395431519, + -2.008063793182373, + 1.6910814046859741, + -0.2576735019683838, + 0.647503137588501, + 0.5322431921958923, + 1.9268598556518555, + -1.2501428127288818, + 0.775363564491272, + -0.0965004414319992, + -0.5331029891967773, + -1.639766812324524, + 0.44450435042381287, + 1.5980249643325806, + 0.9402694702148438, + -0.1454993337392807, + -0.2541998028755188, + 0.6709543466567993, + -0.44338259100914, + -0.043697476387023926, + 0.7131086587905884 + ], + [ + 1.4609386920928955, + -1.6896755695343018, + 2.174467086791992, + -0.34207087755203247, + 0.962078332901001, + 0.28904104232788086, + 2.0044631958007812, + -1.184704303741455, + 0.8424319624900818, + -0.6237084865570068, + -0.9171007871627808, + -1.5616872310638428, + 0.3556978702545166, + 1.5233795642852783, + 0.7210954427719116, + 0.028062552213668823, + 0.21759141981601715, + 0.2823620140552521, + -0.588395357131958, + -0.12581703066825867, + 0.7275858521461487 + ], + [ + 2.087728500366211, + -2.0203187465667725, + 2.4627456665039062, + 0.09322953224182129, + 0.4771752953529358, + 0.2439241260290146, + 2.3431313037872314, + -0.7176190614700317, + 0.8298519849777222, + -0.47738587856292725, + -1.4635096788406372, + -1.2598087787628174, + 0.008061617612838745, + 0.9835944175720215, + 0.10656380653381348, + -0.1548040360212326, + 0.20115382969379425, + 0.4581896960735321, + -1.197885274887085, + -0.28882738947868347, + 0.17229679226875305 + ], + [ + 2.535038471221924, + -2.2647459506988525, + 1.280318260192871, + -0.8243153095245361, + 0.36149507761001587, + 0.01678559184074402, + 1.259200096130371, + -0.5383518934249878, + 1.0661393404006958, + -0.1134992390871048, + -1.0009663105010986, + -1.8629896640777588, + 0.5329263806343079, + 1.1952301263809204, + 0.5380017757415771, + -0.21264730393886566, + 0.33232176303863525, + 0.9111320972442627, + -1.0945085287094116, + 0.7546109557151794, + 0.8692140579223633 + ], + [ + 2.3966352939605713, + -2.3338406085968018, + 2.0661728382110596, + -0.1174439787864685, + -0.28071242570877075, + 0.8010929822921753, + 2.485050678253174, + -0.5360689163208008, + 0.9712552428245544, + 0.5487139821052551, + -0.46108368039131165, + -2.0631186962127686, + 0.6046813726425171, + 0.9991111159324646, + 1.299267292022705, + 0.3742367625236511, + -0.7156072854995728, + 0.7805436849594116, + -1.3091955184936523, + -0.5538330674171448, + -0.20684683322906494 + ], + [ + 2.614837884902954, + -2.24919056892395, + 2.2911505699157715, + -0.2939787209033966, + 1.1876089572906494, + 0.15644106268882751, + 1.7348854541778564, + -0.7635170221328735, + 0.5141214728355408, + -0.17918013036251068, + -0.8421157598495483, + -1.5084272623062134, + -0.18822845816612244, + 1.8389434814453125, + 0.24652135372161865, + 0.38509100675582886, + 0.5761538147926331, + -0.15904095768928528, + 0.0013433098793029785, + 0.6120359897613525, + 1.3977106809616089 + ], + [ + 1.5386989116668701, + -2.0177698135375977, + 1.8581953048706055, + -1.2861346006393433, + 1.0945217609405518, + 0.8363389372825623, + 2.708517074584961, + -0.9201098084449768, + 0.8546208739280701, + -0.058566831052303314, + -1.5932974815368652, + -2.015472412109375, + 0.2565937638282776, + 1.988462209701538, + -0.04372602701187134, + -0.8754178881645203, + 0.27551382780075073, + -0.29693683981895447, + -0.2824576497077942, + 0.40926772356033325, + 0.7658611536026001 + ], + [ + 2.8735361099243164, + -2.675261974334717, + 1.459324836730957, + 0.380817174911499, + 0.6231281757354736, + 0.03267902135848999, + 2.296022415161133, + -1.587423324584961, + 0.7833051681518555, + 0.6618785262107849, + -1.7110093832015991, + -1.436572551727295, + -0.14784568548202515, + 1.525026559829712, + 1.4909214973449707, + -1.0601696968078613, + -0.5454426407814026, + -0.09859517216682434, + -1.4344520568847656, + -0.058256953954696655, + -0.3444262146949768 + ], + [ + 2.0854334831237793, + -1.3952012062072754, + 2.651411771774292, + -0.7663257122039795, + -0.05179548263549805, + 0.8797686100006104, + 2.0904030799865723, + -0.8943268656730652, + 1.0378066301345825, + 0.08841629326343536, + -0.36119532585144043, + -1.236596703529358, + 1.0752983093261719, + 0.349973201751709, + -0.19527214765548706, + -0.8867040872573853, + 0.18460334837436676, + 1.454195261001587, + -0.6220338940620422, + 0.9919598698616028, + 1.229588508605957 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.009577363729476929, + -0.06759434938430786, + 0.05406871438026428, + 0.0, + 0.003663182258605957, + 0.0492824912071228, + -0.038241639733314514, + -0.02580159902572632, + -0.6138286590576172, + -0.056222498416900635, + 0.0, + -0.026217222213745117, + -0.051223963499069214, + -0.031640589237213135, + -0.010870158672332764, + 0.08728909492492676, + -0.020528942346572876, + 0.0 + ], + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.06735062599182129, + -0.023807168006896973, + 0.1439460813999176, + 0.0, + -0.19209599494934082, + -0.14847975969314575, + -0.22449488937854767, + 0.09710657596588135, + -0.6138286590576172, + -0.1854294240474701, + 0.0, + -0.17072844505310059, + 0.09961000084877014, + -0.017846792936325073, + 0.22415585815906525, + -0.18760734796524048, + -0.060083091259002686, + 0.0 + ] + ], + "predicted_scores": [ + 2.515261650085449, + 2.362213611602783, + 1.9975978136062622, + 1.9524565935134888, + 1.9387346506118774, + 1.9376667737960815, + 1.6050832271575928, + 1.4873660802841187 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524/train2", + "selected_index": 0, + "state_hash": "6333fcf80f76890398820f27d1a99efe459cdb6ff795201d73e3bcedc966664f", + "stored_base_utility": 0.18246883153915405, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18095605075359344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18095605075359344 + }, + "base_progress": 0.18095605075359344, + "base_success": false, + "base_utility": 0.18095605075359344, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05759069323539734, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05759069323539734 + }, + { + "contact_quality": null, + "progress": 0.03104570508003235, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03104570508003235 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.036939144134521484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.036939144134521484 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 0.05759069323539734, + 0.03104570508003235, + 1.0, + 0.036939144134521484 + ], + "candidate_source_chart_ids": [ + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "generated_tangents": [ + [ + -0.9199861288070679, + 0.30872613191604614, + -0.11077669262886047, + -1.9874893426895142, + -0.8131400942802429, + 0.15791809558868408, + -0.5111090540885925, + -0.03193216025829315, + -0.3522268533706665, + 0.11017335951328278, + 0.3190152049064636, + -0.6767958402633667, + 0.15991482138633728, + 0.2653621435165405, + -0.5116323232650757, + -0.5092017650604248, + 0.15582670271396637, + 0.7595919966697693, + -0.08631685376167297, + -0.1302405446767807, + -0.29183804988861084 + ], + [ + -0.7484748363494873, + 0.24598805606365204, + -0.46471795439720154, + -2.0231847763061523, + -0.7788739800453186, + -0.009635865688323975, + -0.5818192362785339, + -0.005646020174026489, + -0.22873127460479736, + 0.4051384925842285, + 0.4068411886692047, + -0.30067235231399536, + -0.10813647508621216, + 0.42515963315963745, + -0.20543107390403748, + -0.3371945321559906, + 0.04766494035720825, + 1.0037654638290405, + -0.16063323616981506, + 0.03557148575782776, + -0.3789990246295929 + ], + [ + -0.731407642364502, + 0.30742979049682617, + -0.4756476581096649, + -2.045758008956909, + -0.7928981184959412, + -0.10356190800666809, + -0.4837225675582886, + -0.15372467041015625, + -0.24504807591438293, + 0.37941187620162964, + 0.416603147983551, + -0.35127514600753784, + -0.12188982963562012, + 0.5938593745231628, + -0.37876227498054504, + -0.36554548144340515, + 0.030406087636947632, + 0.9633036851882935, + -0.1396521031856537, + 0.10455045104026794, + -0.163976788520813 + ], + [ + -0.7403339743614197, + 0.29316914081573486, + -0.1711283028125763, + -2.079578161239624, + -0.9704251885414124, + 0.010900795459747314, + -0.6362676024436951, + -0.17625626921653748, + -0.16771242022514343, + 0.38578927516937256, + 0.5840837359428406, + -0.24930799007415771, + 0.18123987317085266, + 0.6146359443664551, + -0.6376851201057434, + -0.38402917981147766, + 0.2362058162689209, + 0.9600014686584473, + -0.09930005669593811, + 0.3097380995750427, + -0.1147318184375763 + ], + [ + -0.3282294273376465, + 0.16843962669372559, + -0.09394654631614685, + -1.9015899896621704, + -0.6381380558013916, + -0.18369367718696594, + -0.6958561539649963, + -0.11395466327667236, + -0.14993177354335785, + 0.28022268414497375, + 0.41690579056739807, + -0.2501760721206665, + 0.5187122821807861, + 0.45513468980789185, + -0.38766011595726013, + -0.45108339190483093, + 0.2503165304660797, + 0.7832396030426025, + -0.13590681552886963, + 0.15501433610916138, + -0.39501962065696716 + ], + [ + -0.38397884368896484, + 0.2456217259168625, + -0.005237668752670288, + -1.7133713960647583, + -0.6999824643135071, + -0.3198879361152649, + -0.528630793094635, + -0.24333840608596802, + -0.05804062634706497, + 0.4728117287158966, + 0.4355371594429016, + -0.2204369604587555, + 0.36551615595817566, + 0.25783246755599976, + -0.2590427100658417, + -0.5544210076332092, + 0.37955984473228455, + 0.5370518565177917, + -0.2108737826347351, + 0.16847515106201172, + -0.36978408694267273 + ], + [ + -0.985397458076477, + 0.37230902910232544, + -0.4251243770122528, + -2.2483575344085693, + -1.0317788124084473, + 0.0034407377243041992, + 0.1626998484134674, + -0.11172974109649658, + -0.2908857762813568, + 0.20559722185134888, + -0.13058370351791382, + -0.1935850977897644, + 0.33945032954216003, + 0.9497448205947876, + -0.35347533226013184, + -0.30820897221565247, + 0.14114263653755188, + 1.154399037361145, + -0.3007515072822571, + 0.3745461702346802, + -0.5210039615631104 + ], + [ + -0.3994852304458618, + 0.22625382244586945, + -0.08539852499961853, + -1.7654892206192017, + -0.5889594554901123, + -0.18787041306495667, + -0.8400595784187317, + -0.18711340427398682, + -0.042555905878543854, + 0.4458334445953369, + 0.4649679362773895, + -0.03487749397754669, + 0.28858545422554016, + 0.21199294924736023, + -0.21182990074157715, + -0.5362876057624817, + 0.4092741906642914, + 0.5948329567909241, + -0.22820013761520386, + 0.2394702434539795, + -0.30517613887786865 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 0.05759069323539734, + 0.03104570508003235, + 2.0, + 0.036939144134521484 + ], + "group_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.17512565851211548, + 2.0, + 2.0, + 2.0, + 0.10078734159469604, + 2.0, + 2.0, + 0.06518048048019409, + 2.0, + 2.0, + 0.05268773436546326, + 0.056524619460105896, + 0.07100173830986023, + 0.16304254531860352, + 2.0, + 0.07095202803611755 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -1.7685134410858154, + 0.4561518430709839, + 0.8554723262786865, + -2.9925076961517334, + -1.652122139930725, + -0.9696183204650879, + 1.440989375114441, + -0.1663060486316681, + 0.005464707035571337, + -0.08078178763389587, + 0.5969918370246887, + -1.7046172618865967, + 0.3472420275211334, + 0.7052499651908875, + -0.026682782918214798, + 0.013620201498270035, + 0.002864290028810501, + 1.5629743337631226, + 0.06437517702579498, + -0.20232005417346954, + 0.06808048486709595 + ], + [ + -1.571894645690918, + 0.49223509430885315, + 1.2358425855636597, + -2.7074813842773438, + -1.6053731441497803, + -0.9737302660942078, + 1.2407286167144775, + -0.41521748900413513, + 0.27269765734672546, + -0.08917579054832458, + 0.4081580936908722, + -1.3627166748046875, + 0.07880738377571106, + 0.7477485537528992, + -0.2180848866701126, + -0.13109977543354034, + -0.3875364661216736, + 1.8194886445999146, + -0.06430798768997192, + 0.07869948446750641, + -0.12621839344501495 + ], + [ + -1.7744590044021606, + 0.367733359336853, + 1.3591725826263428, + -3.3725903034210205, + -1.8434977531433105, + -0.8745664954185486, + 1.6967558860778809, + -0.44676536321640015, + -0.6177129149436951, + 0.03244173526763916, + 0.4171296954154968, + -1.4895254373550415, + 0.9296896457672119, + 0.15846145153045654, + -0.006281482055783272, + 0.6454329490661621, + 0.6225314140319824, + 1.6578954458236694, + 0.3061527609825134, + -0.41300463676452637, + -0.2527916431427002 + ], + [ + -1.7233139276504517, + 0.020191192626953125, + 1.0188184976577759, + -2.873849868774414, + -2.027033567428589, + -1.1650824546813965, + 2.0792155265808105, + -0.8799198865890503, + 0.6924232840538025, + -0.7650182247161865, + -0.04520058631896973, + -2.468881130218506, + -0.13257521390914917, + 0.6857031583786011, + -0.10177457332611084, + -0.7609032988548279, + -0.08249430358409882, + 1.7503252029418945, + 0.1621951013803482, + -0.22467197477817535, + 0.43942898511886597 + ], + [ + -2.3507251739501953, + 0.16114166378974915, + 1.5912489891052246, + -2.1838369369506836, + -2.061230182647705, + -1.7918524742126465, + 0.8407515287399292, + -0.3446531295776367, + -0.7727420926094055, + -0.2617931067943573, + 0.6622774004936218, + -1.05295991897583, + 0.13840271532535553, + 1.2008695602416992, + -0.509428858757019, + 0.7052024006843567, + -0.6614989042282104, + 2.0029079914093018, + -0.82997065782547, + -0.9390356540679932, + 0.7515151500701904 + ], + [ + -1.7504347562789917, + -0.13171464204788208, + 0.7620843648910522, + -3.4511537551879883, + -2.5023341178894043, + -0.8595067858695984, + 1.6361833810806274, + -0.4263937175273895, + 0.005345506593585014, + -0.8147202730178833, + 0.4142894744873047, + -0.8735606670379639, + -0.4809738099575043, + 0.9412479996681213, + -0.6222366690635681, + -0.8843234777450562, + -0.8842994570732117, + 2.4228873252868652, + 0.6301752328872681, + 0.727215051651001, + 0.5050253868103027 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.7685134410858154, + -0.04630154371261597, + -0.002802610397338867, + -1.9925076961517334, + -0.6521221399307251, + -0.013848721981048584, + 0.0, + 0.014648675918579102, + -0.0837627500295639, + -0.04761720821261406, + -0.024705946445465088, + -0.7046172618865967, + 0.0914837121963501, + 0.0, + 0.04683725908398628, + 0.048148274421691895, + 0.03868643194437027, + 0.5629743337631226, + -0.043637245893478394, + -0.08927876502275467, + 0.0 + ], + [ + -0.7685134410858154, + 0.1268729865550995, + -0.0878140926361084, + -1.9925076961517334, + -0.6521221399307251, + -0.08497345447540283, + 0.0, + -0.23278282582759857, + 0.1505891978740692, + -0.0029508620500564575, + -0.03223156929016113, + -0.7046172618865967, + -0.18528565764427185, + 0.0, + 0.009110454469919205, + 0.006383728235960007, + -0.012130538001656532, + 0.5629743337631226, + 0.23608635365962982, + -0.06009632349014282, + 0.0 + ], + [ + -0.7685134410858154, + 0.6076252460479736, + 0.2745842933654785, + -1.9925076961517334, + -0.6521221399307251, + 0.03038167953491211, + 0.0, + 0.33581337332725525, + 0.35011863708496094, + -0.06728862971067429, + -0.4030081629753113, + -0.7046172618865967, + -0.44776007533073425, + 0.0, + -0.01595483534038067, + -0.040477462112903595, + 0.28290560841560364, + 0.5629743337631226, + -0.42285358905792236, + -0.11823103576898575, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.881978750228882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.410499930381775, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.1361609697341919 + ], + [ + -1.5711963176727295, + 0.5461994409561157, + 0.8384940028190613, + -2.960416078567505, + -1.9055238962173462, + -0.7765143513679504, + 1.4308912754058838, + 0.10734349489212036, + 0.011199884116649628, + -0.028424743562936783, + 0.5893871784210205, + -1.7020739316940308, + 0.48640453815460205, + 0.9950923323631287, + 0.20675025880336761, + 0.11047534644603729, + 0.09110014140605927, + 1.3974875211715698, + -0.21079741418361664, + -0.3005509674549103, + 0.09192514419555664 + ], + [ + -2.12906813621521, + 0.3435881733894348, + 0.7164700627326965, + -2.750021457672119, + -1.3681434392929077, + -0.8937081694602966, + 1.6475064754486084, + -0.1447279453277588, + -0.44103574752807617, + -0.5038806200027466, + 0.9006609320640564, + -1.8256494998931885, + 0.433420330286026, + 0.3506351411342621, + 0.4619242548942566, + -0.3905109167098999, + -0.01803053542971611, + 1.9505069255828857, + -0.339522123336792, + -0.20715753734111786, + -0.410531610250473 + ], + [ + -1.194382667541504, + 0.9652969837188721, + 0.41722428798675537, + -2.6203644275665283, + -1.139541506767273, + -1.5412108898162842, + 1.667648434638977, + -0.5256404876708984, + 0.17716272175312042, + 0.30056339502334595, + 0.09983009099960327, + -1.8219844102859497, + -0.23339274525642395, + 0.5763673782348633, + -0.4285883605480194, + 0.1476086974143982, + -0.3515554666519165, + 1.2458031177520752, + 0.3135184645652771, + -0.33711761236190796, + -0.09264840185642242 + ], + [ + -2.0614774227142334, + -0.4080510139465332, + 0.5476135015487671, + -3.0662269592285156, + -1.497420310974121, + -0.11606889963150024, + 1.03195321559906, + -0.3406473994255066, + 0.12844708561897278, + -0.4317864775657654, + 1.035879135131836, + -0.7729326486587524, + 1.0862895250320435, + 0.16315871477127075, + -0.14380013942718506, + -0.8498450517654419, + 0.9780882000923157, + 1.0358695983886719, + -0.5399289727210999, + -0.10386796295642853, + 0.8144621849060059 + ] + ], + "predicted_scores": [ + 6.941451549530029, + 6.583841800689697, + 6.470824241638184, + 5.723037242889404, + 5.598393440246582, + 5.495008945465088, + 5.4782795906066895, + 5.432579517364502 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "794181626/train2", + "selected_index": 0, + "state_hash": "4043b3c43ca4f115e1ea4fbcd98c7d08f797df2efbfe6b67bcffa5592a8aa36d", + "stored_base_utility": 0.17512565851211548, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.2934207916259766e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.2934207916259766e-05 + }, + { + "contact_quality": null, + "progress": 0.05296194553375244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05296194553375244 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.000760689377784729, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000760689377784729 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.1980533599853516e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.1980533599853516e-05 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.2934207916259766e-05, + 0.05296194553375244, + 1.0, + 0.000760689377784729, + 1.0, + 1.1980533599853516e-05 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + false, + true, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-58950fed04d203b97ab9", + "generated_tangents": [ + [ + -0.31529414653778076, + 0.06719810515642166, + 1.3790019750595093, + -0.8087677955627441, + -1.5558207035064697, + 0.1901077926158905, + 0.1225493848323822, + -0.009450942277908325, + 0.020079508423805237, + 0.13775329291820526, + -0.03765364736318588, + -1.1909911632537842, + 0.10977646708488464, + 0.057752951979637146, + -0.037106942385435104, + 0.03026609867811203, + -0.009784594178199768, + -0.03841622173786163, + -0.46646183729171753, + 0.05364760756492615, + 0.2150888592004776 + ], + [ + -0.3955671191215515, + 0.08322340250015259, + 1.355778694152832, + -0.7844339609146118, + -1.5168399810791016, + -0.008477360010147095, + 0.18259158730506897, + 0.026051878929138184, + -0.010561231523752213, + 0.13007283210754395, + 0.0012517385184764862, + -1.1184790134429932, + -0.040822625160217285, + 0.0874737948179245, + 0.10217350721359253, + 0.027105841785669327, + 0.002047210931777954, + 0.07655707001686096, + -0.5974532961845398, + 0.05660775303840637, + 0.13720059394836426 + ], + [ + -0.27848654985427856, + 0.0682956725358963, + 1.5169649124145508, + -0.7754560708999634, + -1.5582163333892822, + 0.1750037521123886, + 0.14128366112709045, + 0.005196854472160339, + 0.04952076077461243, + 0.09669095277786255, + -0.02788574993610382, + -1.0915590524673462, + 0.0059854090213775635, + 0.13754324615001678, + 0.005132380872964859, + 0.04684419929981232, + -0.0276670902967453, + 0.00729253888130188, + -0.5461607575416565, + -0.038370281457901, + 0.07043653726577759 + ], + [ + -0.30860960483551025, + 0.1983822137117386, + 1.2585598230361938, + -0.8459683656692505, + -1.4227685928344727, + 0.4656536281108856, + -0.03908658027648926, + -0.18026328086853027, + -0.006392387673258781, + 0.050521716475486755, + -0.06917355209589005, + -1.3046241998672485, + -0.22434425354003906, + 0.23492836952209473, + -0.07044491171836853, + 0.015672385692596436, + -0.07792384922504425, + -0.16247378289699554, + -0.366665780544281, + -0.08924534916877747, + 0.047389835119247437 + ], + [ + -0.14354734122753143, + 0.12389950454235077, + 1.5886749029159546, + -0.7557408809661865, + -1.6064472198486328, + -0.07062575221061707, + -0.007238209247589111, + 0.051516443490982056, + 0.04593954235315323, + 0.0594705194234848, + 0.007216770201921463, + -1.180119514465332, + 0.002326071262359619, + 0.1234818547964096, + -0.020440172404050827, + -0.0020101983100175858, + -0.08072556555271149, + -0.026401802897453308, + -0.5160574913024902, + 0.08116337656974792, + 0.16265127062797546 + ], + [ + -0.1600053459405899, + -0.0019751600921154022, + 1.4490902423858643, + -0.806849479675293, + -1.5834367275238037, + -0.3417109549045563, + -0.04128307104110718, + 0.0046322643756866455, + 0.0322856679558754, + 0.057643599808216095, + 0.006958127021789551, + -1.2496333122253418, + 0.08490565419197083, + 0.046819135546684265, + -0.07280509173870087, + -0.018529821187257767, + -0.21667523682117462, + -0.07447965443134308, + -0.37272197008132935, + 0.08866557478904724, + 0.0742463618516922 + ], + [ + -0.2705276608467102, + 0.015920650213956833, + 1.833092451095581, + -0.874640703201294, + -1.805408239364624, + 0.04700533300638199, + -0.3806138038635254, + 0.11919692158699036, + -0.05380581319332123, + 0.1927267611026764, + -0.006913498044013977, + -1.1574627161026, + 0.4797649085521698, + 0.13377641141414642, + -0.09122300148010254, + 0.056952085345983505, + -0.06653980910778046, + 0.019200608134269714, + -0.36915457248687744, + -0.08732870221138, + -0.23738515377044678 + ], + [ + -0.26357823610305786, + 0.05911149084568024, + 1.5157943964004517, + -0.9628791809082031, + -1.8328700065612793, + 0.17975999414920807, + -0.5800464749336243, + -0.027612760663032532, + 0.04569070041179657, + 0.1539253294467926, + 0.03349337726831436, + -1.1308995485305786, + 0.23134228587150574, + 0.18589317798614502, + -0.031458284705877304, + 0.06552442908287048, + -0.27106863260269165, + 0.2178078144788742, + -0.4619292616844177, + 0.2917059659957886, + 0.10871528834104538 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 1.2934207916259766e-05, + 0.05296194553375244, + 2.0, + 0.000760689377784729, + 2.0, + 1.1980533599853516e-05 + ], + "group_id": "ms-58950fed04d203b97ab9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20183078944683075, + 0.18959736824035645, + 2.0, + 0.0385412722826004, + 0.15877258777618408, + 0.03365510702133179, + 0.17151890695095062, + 0.13845136761665344, + 0.11539244651794434, + 0.1388089954853058, + 0.15365298092365265, + 0.0967668741941452, + 0.0908290445804596, + 0.06094485521316528, + 0.0377749502658844, + 0.028643101453781128 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.028361663222312927, + -0.489355206489563, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.16713106632232666, + 0.0, + 0.5545767545700073, + -0.18274790048599243, + 0.2772042155265808, + -0.45601847767829895, + -1.3811633586883545, + -0.1571006327867508, + 0.0, + -0.004826217889785767, + 0.2916078567504883, + 0.34807145595550537, + -0.37882840633392334, + 0.08759653568267822, + -0.07658697664737701, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.797813892364502, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.654926061630249, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4572639465332031 + ], + [ + 0.17793144285678864, + 0.008656054735183716, + 2.343496322631836, + -1.9455183744430542, + -2.4697093963623047, + 1.0036901235580444, + 2.5124316215515137, + -0.29308056831359863, + -0.02035735547542572, + -0.059014320373535156, + -0.5863867402076721, + -2.254472255706787, + 0.3354390263557434, + 1.381989598274231, + -0.27729421854019165, + 0.5023907423019409, + -0.8654648661613464, + 0.39588332176208496, + -0.5682056546211243, + 0.20400357246398926, + 0.48529666662216187 + ], + [ + -0.5227808952331543, + 0.12448178231716156, + 2.4855635166168213, + -1.6602094173431396, + -2.3857760429382324, + 0.323077529668808, + 1.9053025245666504, + -0.698577344417572, + 0.619890034198761, + 0.3500756621360779, + -0.1476593315601349, + -2.8024404048919678, + -0.21084021031856537, + 1.0928113460540771, + -0.3938775062561035, + 0.4228009581565857, + -0.39060449600219727, + -0.31665533781051636, + -0.9205541610717773, + 0.03774743899703026, + 0.3333621621131897 + ], + [ + 0.10275222361087799, + 0.08925776183605194, + 2.0934102535247803, + -1.379586935043335, + -3.3245811462402344, + 0.6494860649108887, + 1.990661382675171, + -0.021198302507400513, + 0.4400750398635864, + 0.39176517724990845, + -0.8639137148857117, + -1.981496810913086, + 0.01414397917687893, + 1.2019504308700562, + -0.7839368581771851, + 0.108054518699646, + -0.17177742719650269, + -0.36239588260650635, + -0.5078206062316895, + 0.4236416518688202, + 0.25084057450294495 + ], + [ + -0.8110343217849731, + -0.11216431856155396, + 2.8897132873535156, + -1.319732427597046, + -3.1141648292541504, + 0.416613906621933, + 2.3244988918304443, + -0.6579729914665222, + -0.39938226342201233, + -0.08646327257156372, + -0.3429824113845825, + -2.4165658950805664, + -0.6488300561904907, + 1.4793217182159424, + -0.8628695011138916, + 0.6845848560333252, + -0.7794754505157471, + -0.014068186283111572, + -0.27444708347320557, + 0.8030790686607361, + 0.08975183963775635 + ], + [ + -0.23147638142108917, + -0.26693814992904663, + 2.8227345943450928, + -2.3573813438415527, + -3.3905396461486816, + 0.38961973786354065, + 1.9831907749176025, + -1.2505497932434082, + 0.00624750554561615, + 0.9689428210258484, + 0.11981219053268433, + -1.5865073204040527, + 0.4901374876499176, + 0.6158997416496277, + 0.3767610788345337, + -0.025944411754608154, + -1.4681568145751953, + 0.48217374086380005, + -1.5973502397537231, + -0.7199738621711731, + 0.5119494795799255 + ], + [ + -0.19429966807365417, + 0.7059361934661865, + 3.2694599628448486, + -2.656050682067871, + -2.5502617359161377, + -0.23801833391189575, + 1.628230333328247, + 0.36624830961227417, + -0.08827590942382812, + -0.2098810374736786, + -1.1745270490646362, + -2.2196781635284424, + 0.44475334882736206, + 0.48411858081817627, + -0.25864526629447937, + 0.2009485512971878, + -0.9008907079696655, + -0.3571908473968506, + -0.16894888877868652, + -0.6256314516067505, + -0.4078131914138794 + ], + [ + 0.22205571830272675, + 0.6026778221130371, + 1.8579424619674683, + -2.404672384262085, + -3.728808641433716, + -0.19806396961212158, + 2.8292319774627686, + -0.2435564398765564, + 0.7903643846511841, + -0.376501202583313, + -1.0294175148010254, + -1.7672805786132812, + 0.5992995500564575, + 2.044847249984741, + 0.0011147260665893555, + 0.18739356100559235, + -0.952850341796875, + 0.8702673316001892, + 0.038269996643066406, + -0.4372258484363556, + -0.37835830450057983 + ], + [ + -0.3783646821975708, + 0.08899453282356262, + 2.721517324447632, + -2.3160932064056396, + -2.0679585933685303, + 0.549463152885437, + 3.141833543777466, + 0.09782099723815918, + 0.5830706357955933, + 0.7134941816329956, + 0.2986952066421509, + -2.9980335235595703, + -0.6391466856002808, + 1.075132131576538, + -0.7696593403816223, + 0.4343666732311249, + -0.911625862121582, + -0.31454378366470337, + -1.5935450792312622, + -0.8993679881095886, + -0.3811922073364258 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.1209426149725914, + 0.10675475001335144, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.0875239372253418, + 0.0, + -0.0486987829208374, + 0.06580346822738647, + 0.24258655309677124, + -0.20229026675224304, + -1.3811633586883545, + 0.4185009002685547, + 0.0, + -0.1431063860654831, + 0.15154658257961273, + 0.17190229892730713, + 0.10405468940734863, + 0.036865830421447754, + -0.1988307237625122, + 0.0 + ] + ], + "predicted_scores": [ + 4.611660480499268, + 4.5243706703186035, + 4.38554048538208, + 4.170772552490234, + 3.9188783168792725, + 3.6644399166107178, + 3.558708429336548, + 3.06156849861145 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113371261/train2", + "selected_index": 0, + "state_hash": "1b9018834a77531260bf8dfa74a9832916fe215bda78b5367b061bb4e0ce2333", + "stored_base_utility": 0.20183078944683075, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.01641605794429779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01641605794429779 + }, + "base_progress": 0.01641605794429779, + "base_success": false, + "base_utility": 0.01641605794429779, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.039980679750442505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.039980679750442505 + }, + { + "contact_quality": null, + "progress": 0.060836225748062134, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060836225748062134 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06095251441001892, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06095251441001892 + }, + { + "contact_quality": null, + "progress": 0.031782910227775574, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.031782910227775574 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0629233866930008, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0629233866930008 + } + ], + "candidate_progress": [ + 1.0, + 0.039980679750442505, + 0.060836225748062134, + 1.0, + 0.06095251441001892, + 0.031782910227775574, + 1.0, + 0.0629233866930008 + ], + "candidate_source_chart_ids": [ + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-9f61e09c05b4c8f74366", + "ms-9f61e09c05b4c8f74366", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-651bc0395d78d2819158", + "generated_tangents": [ + [ + -0.4790000915527344, + -0.2623399794101715, + 1.2292762994766235, + -0.9752191305160522, + -1.6180856227874756, + -0.2288983166217804, + -0.028735220432281494, + -0.3114524185657501, + 0.01069274265319109, + -0.275970995426178, + 0.022787142544984818, + -1.5272339582443237, + -0.40187370777130127, + 0.5987383723258972, + -0.22365739941596985, + -0.07965487241744995, + 0.23476973176002502, + 0.8301016688346863, + -0.360110878944397, + -0.5283201932907104, + 0.005559906363487244 + ], + [ + -0.3233262896537781, + -0.3249558210372925, + 1.278425693511963, + -1.119921088218689, + -1.674001932144165, + -0.6046465635299683, + 0.23673884570598602, + -0.30465856194496155, + -0.02057245932519436, + -0.2387981116771698, + 0.08232123404741287, + -1.4324522018432617, + -0.35365891456604004, + 0.6753489971160889, + -0.2134622037410736, + -0.07452715933322906, + 0.26820996403694153, + 0.7786791324615479, + -0.12941446900367737, + -0.523983359336853, + 0.2599738836288452 + ], + [ + -0.3779054880142212, + -0.238853320479393, + 1.2486516237258911, + -0.7021474838256836, + -1.4407782554626465, + -0.36793550848960876, + 0.3716639280319214, + -0.054011762142181396, + 0.09405867010354996, + -0.1169712021946907, + -0.16203543543815613, + -1.2887053489685059, + -0.4147561490535736, + 0.5722816586494446, + -0.2696605622768402, + 0.006031792610883713, + 0.17731794714927673, + 0.8135117888450623, + -0.21739381551742554, + -0.4056673049926758, + 0.23164701461791992 + ], + [ + -0.3505050539970398, + -0.2582804560661316, + 1.0538042783737183, + -0.6612803936004639, + -1.555436372756958, + -0.2776733338832855, + 0.14931350946426392, + -0.07758373022079468, + -0.01656237430870533, + -0.013535372912883759, + -0.21431154012680054, + -1.051558017730713, + -0.41920626163482666, + 0.4313759207725525, + -0.34845492243766785, + -0.01781424880027771, + 0.30896419286727905, + 0.7666733860969543, + -0.20654213428497314, + -0.48529598116874695, + 0.42542150616645813 + ], + [ + -0.25688695907592773, + -0.2868007719516754, + 1.15911865234375, + -0.7883217334747314, + -1.3453056812286377, + -0.4138046205043793, + 0.5186532735824585, + -0.19082722067832947, + 0.07793162763118744, + -0.15708866715431213, + -0.1384713053703308, + -1.2223492860794067, + -0.1700986921787262, + 0.46245038509368896, + -0.35558512806892395, + -0.06017529219388962, + 0.3921342194080353, + 0.7723162770271301, + -0.3470229506492615, + -0.48607730865478516, + -0.03092317283153534 + ], + [ + -0.383963406085968, + -0.2568821907043457, + 1.3227291107177734, + -1.033108115196228, + -1.7499196529388428, + -0.3432324230670929, + -0.03116506338119507, + -0.31188544631004333, + -0.0018440410494804382, + -0.2096678614616394, + 0.09987686574459076, + -1.330317735671997, + -0.504436731338501, + 0.7273666858673096, + -0.2382347583770752, + -0.09948363900184631, + 0.25238290429115295, + 0.7306241393089294, + -0.2710176706314087, + -0.43206655979156494, + 0.07764556258916855 + ], + [ + -0.2638291120529175, + -0.15891681611537933, + 1.5607041120529175, + -1.1172218322753906, + -1.5984916687011719, + -0.021221444010734558, + 0.09867265820503235, + -0.2503146529197693, + 0.03268471360206604, + -0.45578712224960327, + -0.021439678966999054, + -1.5354739427566528, + -0.1466650664806366, + 0.7854465842247009, + -0.25617313385009766, + 0.002685592509806156, + 0.40647679567337036, + 0.9339250922203064, + -0.1025136411190033, + -0.5770187377929688, + 0.46320152282714844 + ], + [ + -0.31497132778167725, + -0.4016556143760681, + 1.2052533626556396, + -0.5810573101043701, + -1.340848684310913, + -0.36131349205970764, + 0.23329539597034454, + -0.08051857352256775, + 0.01538036111742258, + -0.16655072569847107, + -0.028443746268749237, + -1.2804170846939087, + -0.26589950919151306, + 0.033651262521743774, + -0.4386689066886902, + -0.035484254360198975, + 0.4670451879501343, + 0.5834474563598633, + -0.16527092456817627, + -0.1602071225643158, + 0.022881165146827698 + ] + ], + "generated_utilities": [ + 2.0, + 0.039980679750442505, + 0.060836225748062134, + 2.0, + 0.06095251441001892, + 0.031782910227775574, + 2.0, + 0.0629233866930008 + ], + "group_id": "ms-651bc0395d78d2819158", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 7.152557373046875e-07, + 2.0, + 0.0, + 0.0, + 0.18490922451019287, + 0.0, + 0.18801423907279968, + 2.0, + 0.18292954564094543, + 0.06743006408214569, + 2.0, + 0.08718416094779968, + 0.0016145259141921997, + 2.0, + 0.0, + 0.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.13822472095489502, + 0.052123308181762695, + 1.2542681694030762, + -0.5262794494628906, + -1.515254020690918, + -0.022995710372924805, + 0.0, + 0.07829460501670837, + 0.056280869990587234, + -0.006563127040863037, + 0.00594717264175415, + -1.0464823246002197, + 0.028334707021713257, + 0.0, + 0.04214516654610634, + -0.0032968670129776, + 0.025241270661354065, + 1.0898823738098145, + -0.09987176954746246, + 0.009208589792251587, + 0.0 + ], + [ + -1.138224720954895, + -0.12241673469543457, + 2.254268169403076, + -1.5262794494628906, + -2.515254020690918, + -0.4036164879798889, + 2.5164785385131836, + -0.48240265250205994, + 0.09632497280836105, + -1.0002868175506592, + 0.6668505668640137, + -2.0464823246002197, + 0.5158075094223022, + 0.48990970849990845, + 0.02257649227976799, + -0.23748205602169037, + -0.2213379293680191, + 2.0898823738098145, + 0.1738651841878891, + -0.37470543384552, + -0.06145218014717102 + ], + [ + -1.0695267915725708, + -0.1775755137205124, + 2.1683619022369385, + -1.4577077627182007, + -2.7917678356170654, + -0.632304310798645, + 2.3911960124969482, + -0.4637458324432373, + 0.35316434502601624, + -0.8419650793075562, + 0.8253071308135986, + -1.9648017883300781, + 0.2602742314338684, + 0.4741981625556946, + 0.17542533576488495, + -0.15374207496643066, + -0.4857789874076843, + 2.119079828262329, + 0.44966256618499756, + -0.3760189712047577, + 0.18533895909786224 + ], + [ + -0.7955029010772705, + 0.23637625575065613, + 1.9978463649749756, + -1.8726613521575928, + -2.2562904357910156, + -0.3048732876777649, + 2.261690378189087, + -0.6620026230812073, + 0.06043636053800583, + -1.091504693031311, + 1.038455605506897, + -1.9805728197097778, + 0.7370907068252563, + 0.5352527499198914, + 0.26298987865448, + -0.2739749252796173, + 0.14720074832439423, + 2.0173659324645996, + -0.16336800158023834, + -0.5858949422836304, + -0.1647021770477295 + ], + [ + -1.6335437297821045, + -0.08274295926094055, + 1.833235502243042, + -1.758880853652954, + -2.0250279903411865, + -0.8149738311767578, + 2.4238271713256836, + -0.33182430267333984, + -0.16191768646240234, + -1.4929053783416748, + 0.8467401266098022, + -2.4791038036346436, + 0.2757382392883301, + 0.16471439599990845, + 0.18122689425945282, + -0.2995648682117462, + -0.34926944971084595, + 2.063732147216797, + 0.3726923167705536, + -0.259366512298584, + 0.36020517349243164 + ], + [ + -0.9912663698196411, + -0.19695568084716797, + 2.342681646347046, + -2.1077044010162354, + -3.031508684158325, + -0.6591198444366455, + 2.5516788959503174, + -0.8631293773651123, + 0.2481168806552887, + -1.107275366783142, + 0.5796921253204346, + -2.479954719543457, + 1.0449233055114746, + 0.40612131357192993, + -0.12985309958457947, + -0.18829946219921112, + -0.6196849346160889, + 1.6295442581176758, + 0.4197757840156555, + 0.02911210060119629, + 0.5378711223602295 + ], + [ + -0.6689683198928833, + -0.5626275539398193, + 1.738625407218933, + -1.919468879699707, + -2.6463799476623535, + -0.32590651512145996, + 2.287154197692871, + -0.9975759983062744, + 0.7224722504615784, + -1.427150845527649, + 0.572014331817627, + -2.2473490238189697, + 0.8538937568664551, + 0.0030456185340881348, + -0.1059482991695404, + -0.003734409809112549, + -0.2650519907474518, + 2.554786205291748, + -0.24461163580417633, + -0.6881535649299622, + 0.5301691293716431 + ], + [ + -1.5373890399932861, + 0.42636638879776, + 2.9180266857147217, + -2.126905679702759, + -1.825490117073059, + -0.6280819773674011, + 2.593716621398926, + -0.6465503573417664, + 0.5798410177230835, + -0.5376753807067871, + 0.5435326099395752, + -2.610151767730713, + 1.0700397491455078, + 0.7953996658325195, + 0.5072687864303589, + -0.02665230631828308, + -0.6953800320625305, + 2.0521392822265625, + 0.751386821269989, + -0.7565212249755859, + -0.5263289213180542 + ], + [ + -0.7768699526786804, + -0.18900753557682037, + 1.7192409038543701, + -2.2835047245025635, + -1.713951826095581, + 0.004747450351715088, + 3.2709672451019287, + -1.1866822242736816, + 0.38759592175483704, + -0.7857802510261536, + 1.539168119430542, + -1.3250088691711426, + 0.8453556299209595, + 0.5691420435905457, + -0.701140820980072, + -0.6115058660507202, + -0.8829987645149231, + 1.5742011070251465, + -0.3939879536628723, + 0.45414644479751587, + -0.23744438588619232 + ] + ], + "predicted_scores": [ + 4.050506591796875, + 4.038098335266113, + 4.001011371612549, + 3.959012031555176, + 3.8739075660705566, + 3.815523624420166, + 3.801790237426758, + 3.5321407318115234 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1483645336/train2", + "selected_index": 0, + "state_hash": "857c28f61d13d7ee5a59956a87579f6c17f6c601c1680e574691e095e373d439", + "stored_base_utility": 7.152557373046875e-07, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15461373329162598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15461373329162598 + }, + "base_progress": 0.15461373329162598, + "base_success": false, + "base_utility": 0.15461373329162598, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.041229456663131714, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.041229456663131714 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03825731575489044, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03825731575489044 + }, + { + "contact_quality": null, + "progress": 0.03897048532962799, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03897048532962799 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03536248207092285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03536248207092285 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.041229456663131714, + 1.0, + 0.03825731575489044, + 0.03897048532962799, + 1.0, + 0.03536248207092285, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9fad82fd288e4296e354", + "generated_tangents": [ + [ + 0.12536628544330597, + -0.11647109687328339, + 1.2921411991119385, + -0.24000583589076996, + -1.2581112384796143, + 0.11144833266735077, + -0.32800525426864624, + 0.09118542820215225, + 0.097465381026268, + 0.37290751934051514, + -0.04069405049085617, + -1.1826680898666382, + -0.060546427965164185, + -0.01645784080028534, + -0.03844963386654854, + 0.16085399687290192, + 0.09477581083774567, + -0.31814146041870117, + -0.12133142352104187, + -0.14529256522655487, + 0.15311136841773987 + ], + [ + 0.018894150853157043, + -0.14952826499938965, + 1.329519271850586, + -0.2941533327102661, + -1.251598834991455, + 0.30027931928634644, + -0.3334652781486511, + 0.05567649006843567, + 0.08584731817245483, + 0.40141862630844116, + 0.0010401755571365356, + -1.2042847871780396, + -0.04038769006729126, + -0.016273587942123413, + 0.06102803722023964, + 0.17994867265224457, + 0.209195077419281, + -0.24125592410564423, + -0.2691193222999573, + -0.018109411001205444, + 0.12268800288438797 + ], + [ + 0.1202477291226387, + -0.15849904716014862, + 1.2376221418380737, + -0.19699864089488983, + -1.2386269569396973, + 0.18403024971485138, + -0.4259967803955078, + -0.021072864532470703, + 0.14224401116371155, + 0.3849940001964569, + 0.0711132138967514, + -1.222604513168335, + 0.028215348720550537, + 0.010118171572685242, + 0.17152467370033264, + 0.1558779925107956, + 0.13798077404499054, + -0.2491908222436905, + -0.3019210696220398, + 0.004700005054473877, + 0.0299319326877594 + ], + [ + 0.15748688578605652, + -0.07043987512588501, + 1.3349759578704834, + -0.20302537083625793, + -1.1472175121307373, + -0.14832288026809692, + -0.21017318964004517, + 0.16716204583644867, + 0.06764215975999832, + 0.316910058259964, + -0.018689855933189392, + -1.054552435874939, + -0.29433655738830566, + 0.025387123227119446, + 0.06591791659593582, + 0.10386522114276886, + 0.22614863514900208, + -0.19228409230709076, + -0.1799493432044983, + -0.1313530057668686, + 0.19895869493484497 + ], + [ + 0.06650558114051819, + -0.11055754870176315, + 1.0697321891784668, + -0.30425894260406494, + -1.0227339267730713, + 0.4965338110923767, + -0.7454113364219666, + -0.04227656126022339, + 0.1323845535516739, + 0.49749746918678284, + -0.08936873823404312, + -0.9663962125778198, + -0.22731533646583557, + 0.016360580921173096, + 0.0704846978187561, + 0.20341277122497559, + 0.16095131635665894, + -0.3076995015144348, + -0.2250744104385376, + -0.3158529996871948, + -0.22438421845436096 + ], + [ + 0.32339608669281006, + -0.2009962499141693, + 1.398557424545288, + -0.15897992253303528, + -1.1609511375427246, + 0.15981946885585785, + -0.26014435291290283, + 0.22751465439796448, + 0.08869170397520065, + 0.40087276697158813, + 0.3614153265953064, + -1.004456877708435, + -0.40240389108657837, + 0.13136352598667145, + 0.01480763778090477, + 0.2172158658504486, + 0.10615666210651398, + -0.38317227363586426, + -0.41943812370300293, + 0.10234701633453369, + -0.04620635509490967 + ], + [ + 0.04029425233602524, + 0.05161657929420471, + 0.9304510354995728, + -0.31075945496559143, + -1.1189072132110596, + 0.522032618522644, + -0.3789590001106262, + -0.04234050214290619, + -0.011937307193875313, + 0.3761950731277466, + -0.12726551294326782, + -1.2070890665054321, + -0.24522726237773895, + 0.29885485768318176, + -0.08305133879184723, + 0.14699284732341766, + 0.041526660323143005, + -0.41915464401245117, + -0.03320619463920593, + -0.2105114459991455, + -0.3015945255756378 + ], + [ + 0.19390295445919037, + -0.24859407544136047, + 1.6159428358078003, + -0.31207573413848877, + -0.9656159281730652, + 0.3233919143676758, + -0.939553439617157, + 0.18780739605426788, + 0.046040572226047516, + 0.5581344366073608, + -0.01026865839958191, + -1.2877377271652222, + -0.13225805759429932, + 0.1152028888463974, + -0.2035476267337799, + 0.33016154170036316, + 0.06611089408397675, + -0.2509678602218628, + -0.08846113085746765, + -0.478963702917099, + -0.016499951481819153 + ] + ], + "generated_utilities": [ + 0.041229456663131714, + 2.0, + 0.03825731575489044, + 0.03897048532962799, + 2.0, + 0.03536248207092285, + 2.0, + 2.0 + ], + "group_id": "ms-9fad82fd288e4296e354", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.19493231177330017, + 0.06623455882072449, + 2.0, + 0.15396031737327576, + 0.16184639930725098, + 0.025735855102539062, + 0.134642094373703, + 0.15606965124607086, + 0.12982505559921265, + 0.14038166403770447, + 0.09718823432922363, + 0.015769004821777344, + 0.09577034413814545, + 0.057760089635849, + 0.047366172075271606, + 0.035309627652168274 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.0622546412050724, + -0.09277644753456116, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + -0.005287081003189087, + 0.0, + -0.005681931972503662, + 0.03343720734119415, + 1.0536105632781982, + -0.00917971134185791, + -1.333775520324707, + 0.03716959059238434, + 0.0, + 0.019381746649742126, + 0.026990801095962524, + 0.21505475044250488, + -0.029777705669403076, + -0.14875531196594238, + -0.048092663288116455, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.959449768066406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.103323459625244, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.591951370239258 + ], + [ + -0.13909977674484253, + -0.26507115364074707, + 2.099381446838379, + -1.432984709739685, + -2.215683698654175, + 0.18431386351585388, + 2.42065691947937, + 0.3679453730583191, + -0.24726781249046326, + 2.055530071258545, + -0.4911385774612427, + -2.0511183738708496, + 0.04109602048993111, + 2.784976005554199, + 0.005537793040275574, + 0.05149966478347778, + 1.124292016029358, + -0.6457813382148743, + -1.226535677909851, + -0.28760284185409546, + 1.0270601511001587 + ], + [ + 0.30531278252601624, + -0.4336698055267334, + 2.6217315196990967, + -1.5083816051483154, + -2.5322670936584473, + 0.06784650683403015, + 2.3866119384765625, + -0.09851980209350586, + 0.2774229049682617, + 2.157179355621338, + -0.9796892404556274, + -2.8201849460601807, + 0.08135517686605453, + 2.818830728530884, + -0.6848310828208923, + 0.37913888692855835, + 0.7420558929443359, + -0.22611916065216064, + -1.5158166885375977, + 0.0614812970161438, + 1.3468891382217407 + ], + [ + 0.27444392442703247, + -0.22534464299678802, + 2.616544246673584, + -1.2606401443481445, + -2.0848917961120605, + -0.23496872186660767, + 2.1798665523529053, + 0.7968885898590088, + 0.26037585735321045, + 1.5473439693450928, + -0.233917236328125, + -1.867544412612915, + 0.48288947343826294, + 1.9728566408157349, + -0.4071806073188782, + 0.5528238415718079, + 1.6528295278549194, + -1.0936471223831177, + -1.062675952911377, + -0.5311015248298645, + 0.9778987765312195 + ], + [ + 0.5435068607330322, + 0.24469684064388275, + 2.6045663356781006, + -1.284223198890686, + -2.6095967292785645, + 0.9732104539871216, + 2.741159200668335, + 0.6031707525253296, + -0.5612202882766724, + 2.4774889945983887, + 0.1548280119895935, + -2.5516507625579834, + -0.0651187151670456, + 2.3864798545837402, + 0.130750373005867, + -0.1318313330411911, + 1.6559979915618896, + -1.0673086643218994, + -0.5477704405784607, + 0.18809780478477478, + 0.7076419591903687 + ], + [ + 0.31294435262680054, + 0.0263526514172554, + 3.0396268367767334, + -1.3165022134780884, + -1.8981198072433472, + 0.5049136281013489, + 2.31935715675354, + -0.4664327800273895, + -0.56757652759552, + 2.7719295024871826, + 0.21425002813339233, + -2.3925602436065674, + -0.26789456605911255, + 2.008218288421631, + -0.1879691481590271, + -0.34721577167510986, + 1.6638673543930054, + -0.30173182487487793, + -1.8719977140426636, + -0.8951039910316467, + 2.094865083694458 + ], + [ + 0.533157467842102, + 0.308005690574646, + 1.7770951986312866, + -0.6794775724411011, + -3.2953453063964844, + -0.15337547659873962, + 2.5854313373565674, + 0.07320006191730499, + -0.24033141136169434, + 2.5622174739837646, + -0.9168426990509033, + -1.5317132472991943, + -0.5942863821983337, + 2.2703769207000732, + -1.0709311962127686, + 0.6969255208969116, + 1.5304101705551147, + -0.7959219217300415, + -0.9165043830871582, + -0.8046965003013611, + 0.638719916343689 + ], + [ + 0.7393907904624939, + -0.13299384713172913, + 2.4281864166259766, + -1.861674189567566, + -3.3330533504486084, + -0.18516618013381958, + 1.6492295265197754, + -0.6932566165924072, + 0.06020718812942505, + 1.1755363941192627, + -0.016285300254821777, + -3.0671546459198, + -0.6052039861679077, + 2.6829426288604736, + 0.7021514177322388, + -0.477348268032074, + 0.466697633266449, + -0.9403301477432251, + -0.8690205812454224, + -1.1888904571533203, + 0.5360406041145325 + ], + [ + -0.0713023692369461, + -0.9776724576950073, + 1.2895376682281494, + -1.6489125490188599, + -1.4475497007369995, + -0.6822370886802673, + 3.14093017578125, + -0.5356632471084595, + -0.7523763179779053, + 2.766268253326416, + -0.8215572834014893, + -2.753042697906494, + 0.10929550230503082, + 2.570046901702881, + -0.34730663895606995, + 0.8763631582260132, + 1.3356308937072754, + -0.14330342411994934, + -1.1863114833831787, + -0.30006271600723267, + 0.7264611721038818 + ], + [ + 0.35943418741226196, + 0.22802312672138214, + 1.7158434391021729, + -1.4605827331542969, + -3.3450541496276855, + 0.23581582307815552, + 1.759531855583191, + 0.7012510299682617, + 0.0034519433975219727, + 1.4857903718948364, + -1.2866599559783936, + -2.1787426471710205, + 0.8807055354118347, + 1.9066205024719238, + -0.9918697476387024, + -0.32221919298171997, + 1.1986194849014282, + -1.187239646911621, + -0.5752912163734436, + -0.015687286853790283, + 1.775393009185791 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16282694041728973, + -0.1309112012386322, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + 0.15446226298809052, + 0.0, + -0.15669959783554077, + -0.08878482133150101, + 1.0536105632781982, + -0.09701520204544067, + -1.333775520324707, + 0.10988402366638184, + 0.0, + -0.06573517620563507, + -0.2108776718378067, + 0.21505475044250488, + -0.16805344820022583, + -0.14875531196594238, + 0.12529203295707703, + 0.0 + ] + ], + "predicted_scores": [ + 3.67533278465271, + 3.6726293563842773, + 3.551239013671875, + 3.468717575073242, + 3.2934136390686035, + 2.26446533203125, + 2.1814773082733154, + 1.823534607887268 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1176943004/train2", + "selected_index": 0, + "state_hash": "0d923438bf2b227e3521cb937d011f5bddf9b553a882853098b702ec31090614", + "stored_base_utility": 0.19493231177330017, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06781657040119171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06781657040119171 + }, + "base_progress": 0.06781657040119171, + "base_success": false, + "base_utility": 0.06781657040119171, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05735775828361511, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05735775828361511 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06561818718910217, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06561818718910217 + }, + { + "contact_quality": null, + "progress": 0.06600728631019592, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06600728631019592 + }, + { + "contact_quality": null, + "progress": 0.06556455790996552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06556455790996552 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.05735775828361511, + 1.0, + 0.06561818718910217, + 0.06600728631019592, + 0.06556455790996552, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + false, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "generated_tangents": [ + [ + -0.35940051078796387, + 0.26138144731521606, + 1.1218804121017456, + -1.1519173383712769, + -1.8066575527191162, + -0.2927666902542114, + -0.29577481746673584, + 0.049288854002952576, + -0.04845111817121506, + 0.1943264901638031, + 0.16906771063804626, + -1.1960171461105347, + 0.19308754801750183, + 0.3298017978668213, + 0.19319206476211548, + 0.14853538572788239, + 0.37110635638237, + -0.5730489492416382, + -0.4156043529510498, + 0.05488809943199158, + -0.06755031645298004 + ], + [ + -0.2960370182991028, + 0.2934601902961731, + 1.109288215637207, + -1.2864618301391602, + -1.6074607372283936, + -0.09004327654838562, + -0.2562062740325928, + -0.030219927430152893, + 0.046365439891815186, + 0.19546306133270264, + 0.2563927471637726, + -1.0252492427825928, + 0.32666411995887756, + 0.3295997083187103, + 0.10318690538406372, + 0.217575341463089, + 0.38629478216171265, + -0.7123311161994934, + -0.23487460613250732, + -0.14324526488780975, + 0.11484400182962418 + ], + [ + -0.2867763042449951, + 0.26990586519241333, + 1.1454148292541504, + -1.254696249961853, + -1.574082851409912, + -0.16223374009132385, + -0.46713531017303467, + 0.06653812527656555, + 0.004242160357534885, + 0.17129941284656525, + 0.13043415546417236, + -0.985645055770874, + 0.23969587683677673, + 0.28597861528396606, + 0.05446753650903702, + 0.24525628983974457, + 0.4033745527267456, + -0.7307695746421814, + -0.2557576894760132, + -0.16481541097164154, + 0.21549968421459198 + ], + [ + -0.37504494190216064, + 0.18524841964244843, + 0.9497373700141907, + -1.0685830116271973, + -1.176116704940796, + -0.13801464438438416, + -0.1652403473854065, + -0.017291411757469177, + 0.057547666132450104, + 0.2606729567050934, + 0.11610789597034454, + -1.0282714366912842, + 0.2768854796886444, + 0.07643024623394012, + 0.11365670710802078, + 0.22004571557044983, + 0.3535291850566864, + -0.5654693245887756, + -0.17161616683006287, + -0.11329098045825958, + -0.05826231837272644 + ], + [ + -0.2096341997385025, + 0.3217547535896301, + 1.2635153532028198, + -1.2642682790756226, + -1.6292884349822998, + 0.010172441601753235, + -0.44962286949157715, + -0.15860790014266968, + -0.05424071103334427, + 0.1652822047472, + 0.4396679699420929, + -0.8116848468780518, + 0.19552484154701233, + 0.14938408136367798, + 0.18905597925186157, + 0.207344651222229, + 0.3720264136791229, + -0.5570352077484131, + -0.5578036904335022, + -0.439746618270874, + -0.07734593749046326 + ], + [ + -0.13951198756694794, + 0.2989269495010376, + 1.293318748474121, + -1.1962066888809204, + -1.6288855075836182, + -0.2245807945728302, + -0.5613883137702942, + -0.01875273883342743, + 0.01569196954369545, + 0.32219794392585754, + 0.38344618678092957, + -0.7402890920639038, + 0.3756786286830902, + 0.19320344924926758, + 0.10246355086565018, + 0.2723497152328491, + 0.3831385374069214, + -0.5127334594726562, + -0.5140840411186218, + -0.2359667271375656, + 0.0076765865087509155 + ], + [ + -0.1366761177778244, + 0.3460448980331421, + 1.2618845701217651, + -1.1698354482650757, + -1.6935830116271973, + 0.03249320387840271, + -0.46773266792297363, + -0.08201417326927185, + -0.07811436802148819, + 0.2002832591533661, + 0.3100554347038269, + -0.7904842495918274, + 0.18730613589286804, + 0.1492442935705185, + 0.2157992571592331, + 0.24433599412441254, + 0.45189207792282104, + -0.48394668102264404, + -0.5058633685112, + -0.3069969117641449, + -0.0896095335483551 + ], + [ + -0.31444495916366577, + 0.11777603626251221, + 1.0513482093811035, + -1.2007355690002441, + -1.4836351871490479, + -0.02403835952281952, + -0.6299536824226379, + -0.013205036520957947, + 0.01766916736960411, + 0.2846088707447052, + 0.32949161529541016, + -1.0364447832107544, + 0.11130443215370178, + -0.06628595292568207, + 0.11114044487476349, + 0.347646564245224, + 0.29851871728897095, + -0.36744821071624756, + -0.7297919392585754, + -0.12092496454715729, + 0.3485521674156189 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.05735775828361511, + 2.0, + 0.06561818718910217, + 0.06600728631019592, + 0.06556455790996552, + 2.0 + ], + "group_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0793772041797638, + 2.0, + 2.0, + 2.0, + 0.1144598126411438, + 0.04584880173206329, + 0.08543911576271057, + 0.1307348757982254, + 0.10944780707359314, + 0.07284757494926453, + 0.05719476938247681, + 0.09351089596748352, + 0.13337698578834534, + 0.039893895387649536, + 0.09353269636631012, + 0.08224260807037354 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.016482055187225342, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.0903974175453186, + 0.0, + 0.012217208743095398, + 0.010632793419063091, + 0.03624187409877777, + 0.04524834081530571, + -1.3283450603485107, + -0.00880381464958191, + 0.0, + -0.014520794153213501, + 0.5632922649383545, + 0.003262042999267578, + -0.11586570739746094, + 0.0014255642890930176, + 0.015446990728378296, + 0.0 + ], + [ + 0.04007059335708618, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.12278485298156738, + 0.0, + -0.0962398424744606, + -0.08766138553619385, + 0.29553186893463135, + 0.10519622266292572, + -1.3283450603485107, + 0.00032788515090942383, + 0.0, + -0.13836456835269928, + 0.5632922649383545, + -0.13551044464111328, + -0.11586570739746094, + 0.07183873653411865, + -0.03684847429394722, + 0.0 + ], + [ + 0.08037775754928589, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.33122140169143677, + 0.0, + 0.20786377787590027, + -0.42098182439804077, + -0.17684945464134216, + 0.1062583178281784, + -1.3283450603485107, + 0.28051087260246277, + 0.0, + 0.7470210194587708, + 0.5632922649383545, + 0.03247350454330444, + -0.11586570739746094, + 0.4809897243976593, + -0.1744043231010437, + 0.0 + ], + [ + -0.4189682602882385, + 1.1229554414749146, + 2.8264856338500977, + -2.2389113903045654, + -2.4226768016815186, + 1.2001769542694092, + 2.1568119525909424, + -0.016906127333641052, + -0.24083517491817474, + -0.23574936389923096, + -0.13933053612709045, + -2.458132028579712, + 0.40251442790031433, + 0.870932936668396, + -0.28115370869636536, + 1.4463695287704468, + 0.9248121380805969, + -0.9938455820083618, + -0.680131196975708, + -0.051702484488487244, + -0.4663417339324951 + ], + [ + -0.6373537182807922, + 1.3288449048995972, + 3.1669085025787354, + -1.4171695709228516, + -2.242492198944092, + 1.6900161504745483, + 2.778874158859253, + 0.47899991273880005, + 0.3402186334133148, + 0.4507899582386017, + 0.8237879276275635, + -1.7865383625030518, + 1.1732149124145508, + 1.5826115608215332, + 0.4416889548301697, + 2.222722053527832, + 0.6443713307380676, + -1.8598005771636963, + -0.9690301418304443, + 0.3504742383956909, + -1.1371408700942993 + ] + ], + "predicted_scores": [ + 5.1568989753723145, + 4.8573408126831055, + 4.804030418395996, + 4.69545316696167, + 4.384824752807617, + 4.1982316970825195, + 4.097631931304932, + 3.475550413131714 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "998761162/train2", + "selected_index": 0, + "state_hash": "78cc63bdeba9f4d6d6da2efd23c0f7f6631321321e917a70299a7c8025e71d47", + "stored_base_utility": 0.0793772041797638, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.023216769099235535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023216769099235535 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.023216769099235535, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-fc8084d5cd7da937fc02", + "generated_tangents": [ + [ + 0.06674793362617493, + -0.017455171793699265, + 1.4833317995071411, + -0.5928583741188049, + -1.0986278057098389, + -0.07370790839195251, + -0.3972606658935547, + 0.10492485761642456, + -0.029081830754876137, + 0.08049751818180084, + 0.10041757673025131, + -1.2483304738998413, + -0.08490768074989319, + 0.040428973734378815, + 0.06932853907346725, + 0.09570556879043579, + 0.11309096217155457, + 0.1258828341960907, + -0.0645999014377594, + -0.05343492329120636, + 0.128771111369133 + ], + [ + 0.017478227615356445, + 0.04112286865711212, + 1.4941059350967407, + -0.5866529941558838, + -1.1760003566741943, + -0.08213865756988525, + -0.2005012035369873, + 0.01987043023109436, + -0.022918878123164177, + 0.1040123850107193, + 0.20422354340553284, + -1.2287273406982422, + 0.00930255651473999, + 0.05246828496456146, + 0.037923481315374374, + 0.11094716191291809, + 0.06123155355453491, + 0.10196759551763535, + -0.16752895712852478, + -0.012012243270874023, + 0.050198912620544434 + ], + [ + -0.03730447590351105, + 0.014132339507341385, + 1.4144362211227417, + -0.6733148097991943, + -1.1825482845306396, + -0.008287966251373291, + -0.35631752014160156, + -0.03764437139034271, + -0.012792257592082024, + 0.07346613705158234, + 0.19872477650642395, + -1.2307169437408447, + 0.023642033338546753, + 0.005198977887630463, + 0.0063120052218437195, + 0.06589984893798828, + 0.07383552193641663, + 0.0706728845834732, + -0.09465029835700989, + -0.0023286044597625732, + 0.0881691575050354 + ], + [ + -0.016812756657600403, + 0.04553462564945221, + 1.3833285570144653, + -0.6451879143714905, + -1.2300786972045898, + -0.14793860912322998, + -0.3728104829788208, + 0.04813587665557861, + -0.050586894154548645, + 0.08829276263713837, + 0.21551665663719177, + -1.2655410766601562, + 0.07566395401954651, + 0.14191819727420807, + 0.023812171071767807, + 0.08441734313964844, + 0.0654795914888382, + 0.11626894772052765, + -0.16875094175338745, + -0.17117637395858765, + 0.06423484534025192 + ], + [ + -0.2635042667388916, + -0.024782611057162285, + 1.2271443605422974, + -0.4885129928588867, + -0.8494554162025452, + -0.08545178174972534, + -0.2659471035003662, + 0.013631850481033325, + 0.004045146517455578, + 0.21345219016075134, + 0.2647266387939453, + -1.1775646209716797, + -0.03151434659957886, + -0.1833101063966751, + 0.1026906818151474, + 0.07188206911087036, + 0.20349493622779846, + -0.06375865638256073, + -0.2572023868560791, + 0.29140204191207886, + -0.397828608751297 + ], + [ + -0.15293361246585846, + 0.05725117772817612, + 1.328978419303894, + -0.7022752165794373, + -1.2217082977294922, + 0.010761305689811707, + -0.1359962821006775, + -0.019572749733924866, + -0.05365105718374252, + 0.02904101461172104, + 0.14346858859062195, + -1.4018242359161377, + -0.2438257783651352, + 0.16816318035125732, + 0.07734844833612442, + -0.01614791341125965, + 0.15640252828598022, + 0.13216665387153625, + -0.3579481244087219, + 0.14938241243362427, + -0.19346219301223755 + ], + [ + -0.3136376142501831, + 0.05857615917921066, + 1.4760576486587524, + -0.7016957998275757, + -0.9582225680351257, + -0.24509742856025696, + -0.5958195328712463, + -0.0006892681121826172, + -0.052986957132816315, + 0.14698907732963562, + 0.15362226963043213, + -1.0713621377944946, + -0.08330953121185303, + 0.13679146766662598, + -0.04393580183386803, + 0.07613711059093475, + 0.15425747632980347, + 0.12584447860717773, + -0.2639538049697876, + 0.09439608454704285, + 0.2678697109222412 + ], + [ + -0.3261988162994385, + 0.06708893179893494, + 1.3728224039077759, + -0.7918506860733032, + -1.060340166091919, + -0.18442299962043762, + -0.8321369290351868, + -0.04335425794124603, + -0.04527103155851364, + 0.16791337728500366, + 0.16491550207138062, + -1.0117523670196533, + 0.03638753294944763, + -0.009315937757492065, + -0.1257777065038681, + 0.1290484368801117, + 0.17142516374588013, + 0.18401315808296204, + -0.24615490436553955, + 0.025304079055786133, + 0.20221281051635742 + ] + ], + "generated_utilities": [ + 0.023216769099235535, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-fc8084d5cd7da937fc02", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.120394766330719, + 0.1662210375070572, + 2.0, + 0.0270967036485672, + 0.16474007070064545, + 0.014024153351783752, + 0.13986149430274963, + 0.17214345932006836, + 0.19527268409729004, + 0.08535367250442505, + 0.1354738473892212, + 0.11415943503379822, + 0.05034071207046509, + 0.10602898895740509, + 0.01804502308368683, + 0.09680917859077454 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.4742112457752228, + -0.6877930760383606, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.0012008249759674072, + 0.0, + 0.04633669555187225, + -0.307714581489563, + -0.22357922792434692, + -0.35245198011398315, + -1.486854076385498, + 0.15573161840438843, + 0.0, + 0.5157145261764526, + 0.243257537484169, + 0.2085912823677063, + 0.48896148800849915, + -0.06008961796760559, + 0.4603208899497986, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.883587837219238, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6201469898223877, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.5597648620605469 + ], + [ + -0.8389496207237244, + 0.5873794555664062, + 2.327263116836548, + -2.175445795059204, + -1.844232439994812, + 0.15723112225532532, + 2.920079231262207, + -0.5335484743118286, + -0.6761462688446045, + 0.48151493072509766, + 0.2968217730522156, + -2.99812650680542, + -0.27805766463279724, + 2.0262699127197266, + 0.10151885449886322, + -0.7244771122932434, + 1.0176090002059937, + 1.278473973274231, + -0.2260074019432068, + -1.0207180976867676, + 0.7682583928108215 + ], + [ + 0.2503487765789032, + 0.20628705620765686, + 2.289511203765869, + -0.8669976592063904, + -1.9807379245758057, + 1.1820356845855713, + 2.8384368419647217, + 0.43699324131011963, + -0.688888669013977, + 1.0812339782714844, + -0.18395745754241943, + -1.6906647682189941, + -0.040969282388687134, + 1.254612684249878, + 0.24077603220939636, + -1.0562812089920044, + 1.2436273097991943, + 0.2388666272163391, + -0.5016235709190369, + 0.4570234417915344, + 1.355008840560913 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.10285450518131256, + -0.2634444832801819, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.12851905822753906, + 0.0, + 0.06062778830528259, + 0.006933145225048065, + -0.08638572692871094, + 0.32057398557662964, + -1.486854076385498, + -0.1531560719013214, + 0.0, + 0.14251592755317688, + 0.07826228439807892, + -0.24055713415145874, + -0.11116695404052734, + 0.200389564037323, + -0.07051187753677368, + 0.0 + ], + [ + -0.07637833058834076, + 0.6696901321411133, + 2.130587100982666, + -1.9006763696670532, + -2.6654670238494873, + 0.06729818880558014, + 2.65816593170166, + -0.0612010583281517, + -0.1226595938205719, + 0.9200000166893005, + -0.4352770149707794, + -2.266615390777588, + 0.48697614669799805, + 1.5162358283996582, + -0.014945849776268005, + -0.4459138512611389, + 1.1448073387145996, + 0.8031747937202454, + -0.6817312240600586, + -0.20262596011161804, + 1.1690938472747803 + ], + [ + 0.519248902797699, + 0.27624276280403137, + 1.911887764930725, + -1.8784401416778564, + -2.9212372303009033, + 0.7762739658355713, + 2.547428846359253, + 0.08362685143947601, + -0.1671719253063202, + 0.8596282601356506, + -0.17510351538658142, + -2.220954656600952, + 0.4918195605278015, + 1.357650876045227, + 0.3462192416191101, + 0.2476332038640976, + 0.6965686082839966, + 1.0915483236312866, + -0.6818809509277344, + -0.7465203404426575, + 0.724744439125061 + ] + ], + "predicted_scores": [ + 3.261988639831543, + 3.237738847732544, + 3.007798194885254, + 2.962484121322632, + 2.906874179840088, + 2.7258682250976562, + 2.4441936016082764, + 2.238560438156128 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1991414747/train2", + "selected_index": 0, + "state_hash": "127cf975da326f174c815484811a6acc4855bf07460a2aa5b0d728cb2a1f41c7", + "stored_base_utility": 0.120394766330719, + "task_id": "PushCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20275163650512695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20275163650512695 + }, + "base_progress": 0.20275163650512695, + "base_success": false, + "base_utility": 0.20275163650512695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1694907248020172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1694907248020172 + }, + { + "contact_quality": null, + "progress": 0.21781498193740845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21781498193740845 + }, + { + "contact_quality": null, + "progress": 0.17819397151470184, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17819397151470184 + }, + { + "contact_quality": null, + "progress": 0.200613334774971, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.200613334774971 + }, + { + "contact_quality": null, + "progress": 0.0787760317325592, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0787760317325592 + }, + { + "contact_quality": null, + "progress": 0.1785118579864502, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1785118579864502 + }, + { + "contact_quality": null, + "progress": 0.19316479563713074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19316479563713074 + }, + { + "contact_quality": null, + "progress": 0.187234565615654, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.187234565615654 + } + ], + "candidate_progress": [ + 0.1694907248020172, + 0.21781498193740845, + 0.17819397151470184, + 0.200613334774971, + 0.0787760317325592, + 0.1785118579864502, + 0.19316479563713074, + 0.187234565615654 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-3afd8959363e917591d4", + "generated_tangents": [ + [ + 0.7377750873565674, + 0.26347267627716064, + -0.08314630389213562, + -0.9994946718215942, + 0.9082075357437134, + -0.19150912761688232, + 1.3097355365753174, + -0.16698706150054932, + 0.08360017091035843, + -0.1160506084561348, + 0.101993128657341, + -0.1766650676727295, + -1.296095609664917, + -0.5928234457969666, + -0.11710420250892639, + 0.13019615411758423, + 0.3176431357860565, + 0.09339398145675659, + -0.3347843885421753, + -0.9365300536155701, + -0.6035232543945312 + ], + [ + 0.24448202550411224, + 0.39407414197921753, + -0.3670208752155304, + -1.0298949480056763, + 0.5744311809539795, + -0.264547735452652, + 0.38746458292007446, + -0.03442789614200592, + -0.060970596969127655, + -0.03835376352071762, + 0.3443411886692047, + -0.061457276344299316, + -2.3562428951263428, + -0.35382968187332153, + -0.29803335666656494, + 0.06864343583583832, + 0.25068435072898865, + -0.1870059221982956, + -0.2646765112876892, + -1.807386875152588, + -0.28571200370788574 + ], + [ + 0.21231411397457123, + 0.32847559452056885, + -0.23204758763313293, + -1.045691728591919, + 0.6634618043899536, + 0.10659906268119812, + 0.27520033717155457, + -0.11260038614273071, + -0.06890711188316345, + 0.04264010488986969, + 0.22799637913703918, + -0.29159629344940186, + -2.414637804031372, + -0.26020193099975586, + -0.14008629322052002, + 0.08521687984466553, + 0.372461199760437, + -0.19722320139408112, + -0.409726619720459, + -1.4473233222961426, + -0.3482573330402374 + ], + [ + 0.17680992186069489, + 0.3088708519935608, + -0.4014383852481842, + -1.1345856189727783, + 0.6421109437942505, + -0.29996058344841003, + 0.4782603979110718, + -0.0847964882850647, + -0.058243148028850555, + 0.04311869293451309, + 0.22336465120315552, + -0.22831225395202637, + -2.349536180496216, + -0.2620731592178345, + -0.2297009825706482, + 0.0788859874010086, + 0.28485727310180664, + -0.14942477643489838, + -0.502982497215271, + -1.7022714614868164, + -0.32193323969841003 + ], + [ + 0.3183712959289551, + 0.1401023268699646, + -0.03044000267982483, + -1.1987438201904297, + 0.9590957760810852, + 0.3255043029785156, + 0.49064400792121887, + 0.11818192154169083, + 0.05391983687877655, + 0.2607717216014862, + 0.15072235465049744, + -0.08412399888038635, + -2.3221940994262695, + -0.5120459198951721, + -0.20447063446044922, + 0.17877575755119324, + 0.4177470803260803, + 0.06512308120727539, + -0.5801995992660522, + -1.6647918224334717, + -0.24401062726974487 + ], + [ + 0.12330439686775208, + 0.19448666274547577, + 0.08899796009063721, + -0.9709631204605103, + 0.6608500480651855, + -0.17747187614440918, + 0.20461422204971313, + 0.08426865935325623, + 0.05225510895252228, + 0.010891012847423553, + 0.3167150914669037, + 0.03324928879737854, + -2.2975637912750244, + -0.047919467091560364, + -0.1316448599100113, + -0.045330993831157684, + -0.006210550665855408, + 0.09347040951251984, + -0.2524777054786682, + -1.4940173625946045, + 0.09557963162660599 + ], + [ + 0.1730630099773407, + 0.2970139980316162, + -0.4695887267589569, + -1.067762017250061, + 0.6304346323013306, + -0.29939547181129456, + 0.44776809215545654, + 0.16391919553279877, + -0.0837835967540741, + 0.06573087722063065, + 0.22065696120262146, + -0.47530072927474976, + -2.380866289138794, + -0.3768603205680847, + -0.21038717031478882, + 0.1146782636642456, + 0.3076738119125366, + -0.07745186984539032, + -0.5000383257865906, + -1.6727688312530518, + -0.2857869267463684 + ], + [ + 0.2767746150493622, + 0.2590828537940979, + -0.013947397470474243, + -1.1364459991455078, + 0.6993545293807983, + -0.41169485449790955, + 0.053765684366226196, + -0.053581610321998596, + -0.06956262141466141, + -0.0026132240891456604, + 0.24177467823028564, + -0.0682813823223114, + -2.4299659729003906, + -0.08124825358390808, + -0.17382197082042694, + 0.06207389757037163, + 0.1271786391735077, + 0.07436180859804153, + -0.3390803933143616, + -1.6315560340881348, + 0.030501589179039 + ] + ], + "generated_utilities": [ + 0.1694907248020172, + 0.21781498193740845, + 0.17819397151470184, + 0.200613334774971, + 0.0787760317325592, + 0.1785118579864502, + 0.19316479563713074, + 0.187234565615654 + ], + "group_id": "ms-3afd8959363e917591d4", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8582140207290649, + 2.0, + 0.18031787872314453, + 0.07887399196624756, + 0.5568947792053223, + 0.16715207695960999, + 0.5635193586349487, + 0.18913890421390533, + 0.5682768225669861, + 0.10102970898151398, + 0.09163376688957214, + 0.12909311056137085, + 0.12299326062202454, + 0.09030237793922424, + 0.13531166315078735, + 0.06874948740005493 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.15699657797813416, + 0.6247961521148682, + 0.107974112033844, + -0.3284463882446289, + 0.22832846641540527, + -0.12728703022003174, + 0.0, + -0.2739208936691284, + -0.051639966666698456, + -0.2660524249076843, + -0.2539072632789612, + 0.09800386428833008, + -1.84090256690979, + 0.0, + -0.017676634714007378, + -0.018830953165888786, + 0.1583891659975052, + -0.07512116432189941, + -0.01770937442779541, + -1.5237011909484863, + 0.0 + ], + [ + 0.4675478935241699, + 0.6247961521148682, + 0.3345896601676941, + -0.5404787063598633, + -0.06834304332733154, + -0.12728703022003174, + 0.0, + -0.14999517798423767, + -0.2730908691883087, + 0.08837036788463593, + 0.45600926876068115, + 0.410434365272522, + -1.84090256690979, + 0.0, + 0.4969322681427002, + 0.22034725546836853, + 0.2921968400478363, + 0.5872196555137634, + 0.19443652033805847, + -1.5237011909484863, + 0.0 + ], + [ + 0.41189882159233093, + 1.6247961521148682, + 0.12157746404409409, + -1.328446388244629, + 0.9316569566726685, + -1.1272870302200317, + 1.276085376739502, + -0.029595039784908295, + -0.014941371977329254, + 0.004283940885215998, + 0.6188332438468933, + -0.49464237689971924, + -2.84090256690979, + -1.8046140670776367, + 0.0007966384291648865, + -0.006994657218456268, + 0.014263113960623741, + 0.6869955658912659, + -0.2918644845485687, + -2.5237011909484863, + -1.7481608390808105 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.552170753479004, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.6092281341552734, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.496321678161621 + ], + [ + 0.44720953702926636, + 1.4417266845703125, + 0.4182493984699249, + -1.547792911529541, + 0.990667462348938, + -0.9114025831222534, + 1.535681962966919, + 0.21791213750839233, + -0.30513259768486023, + 0.013669610023498535, + 0.7976462841033936, + -0.6395556926727295, + -2.5809314250946045, + -1.571488618850708, + 0.038753002882003784, + 0.04824339970946312, + -0.11262937635183334, + 0.9187417030334473, + -0.1308273822069168, + -2.59043288230896, + -1.9625922441482544 + ], + [ + 0.5162960290908813, + 1.2868248224258423, + 0.2058304250240326, + -1.4812198877334595, + 1.085485816001892, + -1.05157470703125, + 1.1480523347854614, + 0.2796477973461151, + -0.1532609462738037, + 0.12988577783107758, + 0.8581418395042419, + -0.6760798692703247, + -2.769819736480713, + -2.192568778991699, + 0.10523252189159393, + 0.05066283047199249, + -0.08061028271913528, + 0.7438116669654846, + -0.07841359078884125, + -2.3079848289489746, + -1.4200246334075928 + ], + [ + 0.4112767279148102, + 1.8745536804199219, + 0.5301340222358704, + -0.9917724132537842, + 0.8905197978019714, + -0.8391705751419067, + 1.3120794296264648, + 0.3723032474517822, + -0.19477176666259766, + -0.35689839720726013, + 0.2843710780143738, + -0.45066511631011963, + -2.812030792236328, + -1.588818073272705, + -0.09205310046672821, + -0.5067525506019592, + -0.41988417506217957, + 1.0423402786254883, + 0.1759319305419922, + -2.945967197418213, + -2.05134916305542 + ], + [ + 0.46365559101104736, + 1.105017900466919, + 0.466808021068573, + -1.5382812023162842, + 0.8287478089332581, + -1.4617033004760742, + 0.8807560205459595, + 0.2107943296432495, + 0.2954367995262146, + 0.45928943157196045, + 0.6575880646705627, + -0.5414218902587891, + -2.711836099624634, + -1.6827114820480347, + 0.3175691068172455, + -0.38371264934539795, + 0.34041139483451843, + 0.34132614731788635, + -0.8783347606658936, + -2.133782148361206, + -1.3974260091781616 + ], + [ + 0.13586324453353882, + 1.6158640384674072, + 0.24489957094192505, + -1.4942185878753662, + 1.1602565050125122, + -1.648895502090454, + 1.1836220026016235, + 0.17831489443778992, + -0.5400142669677734, + -0.42907604575157166, + 0.5951722264289856, + -0.7885013818740845, + -3.2272911071777344, + -2.208972692489624, + -0.4857475459575653, + 0.5623884201049805, + 0.02566027268767357, + 0.590702474117279, + -0.6581307649612427, + -2.7863473892211914, + -2.031818151473999 + ], + [ + 0.21801596879959106, + 1.0595136880874634, + 0.9040011763572693, + -1.1076470613479614, + 1.2744677066802979, + -0.7150566577911377, + 1.0042598247528076, + 0.5841938257217407, + -0.2832033336162567, + -0.45343834161758423, + -0.17037296295166016, + -0.5152367353439331, + -2.1963798999786377, + -1.5652427673339844, + 0.03223085403442383, + 0.07496664673089981, + -0.1515023559331894, + 1.233866572380066, + 0.4301607310771942, + -2.489328384399414, + -1.8620245456695557 + ], + [ + -0.15804454684257507, + 2.2168471813201904, + 0.9374236464500427, + -1.8158314228057861, + 1.114590048789978, + -0.9279311895370483, + 1.890993356704712, + -0.8653054237365723, + -0.5551189184188843, + 0.4295695126056671, + 1.1614964008331299, + -0.5840407609939575, + -2.900725841522217, + -2.579883575439453, + -0.1900041103363037, + 0.42368441820144653, + 0.38376376032829285, + 0.9654008150100708, + 0.10608989000320435, + -3.332181453704834, + -1.8879828453063965 + ], + [ + 1.0901068449020386, + 2.576328754425049, + 0.5122992396354675, + -1.9627916812896729, + 0.026246964931488037, + -0.3622438311576843, + 0.9286589622497559, + 0.644849956035614, + 0.9211093187332153, + 0.3528344929218292, + 0.3661169707775116, + 0.20941221714019775, + -2.582620143890381, + -1.6867890357971191, + 0.09869468957185745, + -0.8795469403266907, + -0.2239246666431427, + 0.5206944942474365, + -0.20716872811317444, + -2.8072235584259033, + -2.3385443687438965 + ], + [ + -0.0005125105381011963, + 1.8920408487319946, + 1.0018010139465332, + -2.2556979656219482, + 0.7338991165161133, + -0.4559611678123474, + 1.027335524559021, + -1.0165901184082031, + -0.14792552590370178, + 0.11212217807769775, + 0.8827763795852661, + 0.1683982014656067, + -3.2136244773864746, + -2.4300284385681152, + -0.9183067083358765, + 0.10752613097429276, + -0.2955099046230316, + 0.31867098808288574, + 0.06515705585479736, + -3.416110038757324, + -1.5137704610824585 + ], + [ + -0.39508435130119324, + 2.55599308013916, + 0.128604456782341, + -1.8821169137954712, + 0.6769157648086548, + -0.7477259635925293, + 1.6593644618988037, + -0.005859369412064552, + 0.22320973873138428, + -0.2906110882759094, + 0.22343486547470093, + -1.488371729850769, + -2.81829833984375, + -0.939697265625, + 0.9867495894432068, + 0.2955887019634247, + -0.11908379942178726, + 0.473803848028183, + -0.11059392988681793, + -3.471217155456543, + -2.3480191230773926 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.018576741218566895, + 0.6247961521148682, + -0.017397306859493256, + -0.3284463882446289, + 0.044771432876586914, + -0.12728703022003174, + 0.0, + -0.005512619391083717, + 0.03525955602526665, + -0.005881692748516798, + 0.04493492841720581, + 0.056808531284332275, + -1.84090256690979, + 0.0, + 0.11075251549482346, + -0.05448189005255699, + -0.041690655052661896, + 0.010008454322814941, + 0.056893616914749146, + -1.5237011909484863, + 0.0 + ] + ], + "predicted_scores": [ + 7.24985408782959, + 5.471922874450684, + 5.066809177398682, + 5.065141677856445, + 4.941882610321045, + 4.642519950866699, + 4.432784080505371, + 4.277013778686523 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "75070641/train2", + "selected_index": 0, + "state_hash": "bd1bbc057450972b9fa6e08a54ebd8c6d82d09997dad16cecf9584632b705b7b", + "stored_base_utility": 0.8582140207290649, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.15755623579025269, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15755623579025269 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1525605022907257, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1525605022907257 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.13584548234939575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13584548234939575 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.15755623579025269, + 1.0, + 1.0, + 0.1525605022907257, + 1.0, + 1.0, + 0.13584548234939575, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-0949097d19e597fc7f63", + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-3ff1aedb72f65a182b58", + "generated_tangents": [ + [ + 0.08186054974794388, + 0.006009191274642944, + -0.06694689393043518, + -0.0055119991302490234, + -0.010643720626831055, + -2.132002353668213, + -0.25076591968536377, + 0.0505671352148056, + 0.01362932100892067, + 0.14865826070308685, + 0.19644054770469666, + 0.04079253971576691, + -1.9053442478179932, + -0.19569317996501923, + 0.11837344616651535, + -0.027462827041745186, + 0.019821152091026306, + 0.037378594279289246, + -0.1200321614742279, + -1.6845331192016602, + -0.12577423453330994 + ], + [ + 0.13234175741672516, + 0.04501605033874512, + 0.13640928268432617, + -0.14431041479110718, + 0.037565574049949646, + -2.147111654281616, + -0.19054502248764038, + 0.0752481073141098, + -0.031138623133301735, + 0.13148614764213562, + 0.2956600487232208, + 0.05605921149253845, + -1.8617624044418335, + -0.10655191540718079, + 0.02225549891591072, + -0.050534848123788834, + 0.028844401240348816, + 0.060540422797203064, + -0.08961048722267151, + -1.5277156829833984, + -0.10681098699569702 + ], + [ + 0.05637793242931366, + 0.062057361006736755, + 0.09954336285591125, + -0.11781985312700272, + -0.06153351068496704, + -2.009679079055786, + -0.3176780343055725, + 0.06290937960147858, + -0.09404604882001877, + 0.12751705944538116, + 0.13278409838676453, + 0.1133994311094284, + -1.8079689741134644, + -0.23078136146068573, + 0.049533046782016754, + -0.01197347417473793, + 0.07555678486824036, + -0.18356020748615265, + -0.24197518825531006, + -1.5916109085083008, + -0.15395718812942505 + ], + [ + 0.07001128047704697, + -0.006341617554426193, + -0.013905733823776245, + -0.10054901242256165, + 0.004002317786216736, + -2.089357376098633, + -0.29093819856643677, + 0.07939032465219498, + 0.0024054748937487602, + 0.12677901983261108, + 0.24757447838783264, + 0.05934283137321472, + -1.8960217237472534, + -0.24146471917629242, + 0.1240246593952179, + 0.0006272131577134132, + 0.07136332988739014, + -0.013259604573249817, + -0.03865545988082886, + -1.662118911743164, + -0.1441478133201599 + ], + [ + -0.10122169554233551, + -0.014751335605978966, + 0.04853445291519165, + 0.018408209085464478, + -0.0015732795000076294, + -2.041250467300415, + -0.17305952310562134, + 0.21826361119747162, + -0.05995109677314758, + 0.15702185034751892, + 0.09051486104726791, + 0.06831461191177368, + -1.9522912502288818, + -0.027929797768592834, + 0.01918385550379753, + 0.04103182256221771, + 0.19815915822982788, + 0.033082395792007446, + -0.08686968684196472, + -1.602992296218872, + -0.1569117307662964 + ], + [ + -0.30795198678970337, + -0.09053993225097656, + 0.28962182998657227, + 0.0344066321849823, + -0.08195352554321289, + -2.2240450382232666, + -0.20861703157424927, + 0.2981802225112915, + -0.07621319591999054, + -0.11032906919717789, + 0.157115638256073, + 0.015329763293266296, + -2.1129181385040283, + 0.04135496914386749, + 0.09659168124198914, + 0.0655207633972168, + 0.15245279669761658, + 0.04563286900520325, + -0.24892079830169678, + -1.642627477645874, + 0.08329165726900101 + ], + [ + -0.013811811804771423, + -0.03245372697710991, + -0.31161126494407654, + -0.16774164140224457, + -0.24508851766586304, + -1.9709912538528442, + -0.17202073335647583, + 0.26413002610206604, + -0.07094306498765945, + 0.14067888259887695, + 0.3410402834415436, + -0.3525315523147583, + -2.244873285293579, + 0.23459428548812866, + -0.2386588752269745, + 0.031863175332546234, + 0.38297897577285767, + -0.11049143970012665, + -0.40620607137680054, + -1.3109962940216064, + -0.4964732229709625 + ], + [ + -0.039339467883110046, + -0.10939320921897888, + 0.28396040201187134, + 0.07240608334541321, + -0.12615758180618286, + -2.052049160003662, + -0.2889990210533142, + 0.2383739948272705, + -0.035172559320926666, + 0.02148519456386566, + 0.29923754930496216, + 0.002026692032814026, + -2.1962761878967285, + -0.027928397059440613, + -0.053601328283548355, + 0.14903222024440765, + 0.4132443070411682, + 0.1425221711397171, + -0.08159804344177246, + -1.587170124053955, + -0.0466499924659729 + ] + ], + "generated_utilities": [ + 0.15755623579025269, + 2.0, + 2.0, + 0.1525605022907257, + 2.0, + 2.0, + 0.13584548234939575, + 2.0 + ], + "group_id": "ms-3ff1aedb72f65a182b58", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.851725161075592, + 2.0, + 0.8126396536827087, + 2.0, + 0.7855525612831116, + 0.7821746468544006, + 0.6115735173225403, + 0.5732660889625549, + 0.7798006534576416, + 0.09109212458133698, + 0.8456447124481201, + 0.16636842489242554, + 0.0955686867237091, + 0.0822453498840332, + 0.0018809139728546143, + 0.08579078316688538 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.09319563210010529, + 0.18433167040348053, + 0.03857926279306412, + 0.20899039506912231, + -0.19732166826725006, + -2.7716264724731445, + -0.9089346528053284, + -0.012062903493642807, + -0.001710914308205247, + 0.02447224222123623, + 0.6588084697723389, + -0.43759873509407043, + -2.7224881649017334, + -1.560755968093872, + -0.002114389091730118, + -0.004199527204036713, + 0.015975257381796837, + 0.7779304385185242, + -0.31706690788269043, + -2.4826252460479736, + -1.5341289043426514 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.8178693056106567, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.121511936187744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0682578086853027 + ], + [ + -0.10748189687728882, + 0.29221633076667786, + 0.12325333058834076, + 0.21938443183898926, + 0.017790749669075012, + -2.485055446624756, + -1.1110891103744507, + 0.062413301318883896, + -0.2567306160926819, + -0.15453597903251648, + 0.549060583114624, + -0.18322163820266724, + -2.5428390502929688, + -1.8537542819976807, + -0.07324761152267456, + 0.28765758872032166, + 0.16924789547920227, + 0.6205734610557556, + -0.4268453121185303, + -2.6385459899902344, + -1.5760513544082642 + ], + [ + 0.44683152437210083, + 0.3713763356208801, + 0.1084144338965416, + 0.2590586543083191, + 0.1997859627008438, + -2.787759780883789, + -1.1921976804733276, + 0.33878061175346375, + 0.019987009465694427, + 0.021556159481406212, + 0.36814823746681213, + -0.2135106772184372, + -2.416769027709961, + -1.3776189088821411, + -0.3500843346118927, + -0.26303720474243164, + 0.10322260856628418, + 0.9827795624732971, + -0.3747403025627136, + -2.6419029235839844, + -1.4116339683532715 + ], + [ + -0.2883707880973816, + 0.02873578667640686, + -0.11175497621297836, + -0.106661856174469, + 0.03059367835521698, + -2.3899149894714355, + -1.0789995193481445, + 0.2610729932785034, + 0.4800889790058136, + -0.45478329062461853, + 1.0867195129394531, + -0.7417076826095581, + -3.09529185295105, + -1.730720043182373, + -0.33508142828941345, + 0.10841497033834457, + -0.38902348279953003, + 0.44348275661468506, + 0.0854053795337677, + -2.3334603309631348, + -1.7851121425628662 + ], + [ + 0.16367590427398682, + 0.05515189468860626, + -0.3197028636932373, + 0.5501840114593506, + -0.3621746301651001, + -3.0582237243652344, + -1.135798454284668, + -0.15054848790168762, + -0.5578444004058838, + 0.3691062331199646, + 0.9388304352760315, + -0.19103051722049713, + -2.7536559104919434, + -2.093773603439331, + -0.037153881043195724, + 0.19014233350753784, + -0.36205345392227173, + 0.24469107389450073, + -0.5916419625282288, + -2.7417092323303223, + -1.637229323387146 + ], + [ + -0.3646707534790039, + 0.6696967482566833, + -0.44303250312805176, + -0.4559982419013977, + -0.6542845368385315, + -2.9630653858184814, + -1.4721050262451172, + 0.5010843873023987, + 0.5162685513496399, + 0.42702627182006836, + 0.6571387648582458, + -0.08207055926322937, + -3.278073310852051, + -1.7742136716842651, + -0.6448181867599487, + -0.45361432433128357, + 0.3905653953552246, + 0.7770582437515259, + -0.19482973217964172, + -2.094736337661743, + -1.083003282546997 + ], + [ + 0.9472348093986511, + 0.6904511451721191, + -0.2936761677265167, + -0.34556156396865845, + -0.7593099474906921, + -1.9670193195343018, + -0.7599589824676514, + 0.04467743635177612, + 0.6557444334030151, + 0.2765021026134491, + 0.042136549949645996, + -0.37541040778160095, + -3.161308765411377, + -0.8803613185882568, + 0.38967010378837585, + 0.4423010051250458, + 0.019825218245387077, + 0.9605399370193481, + -0.874176561832428, + -3.0791730880737305, + -1.3198856115341187 + ], + [ + 0.8605663180351257, + 0.04459291696548462, + -0.8886744976043701, + -0.38333380222320557, + -1.0364235639572144, + -3.0104973316192627, + -0.889693021774292, + -0.08674328029155731, + 0.20159615576267242, + 0.1069621816277504, + 1.4727201461791992, + 0.09883639216423035, + -3.129849433898926, + -0.8773520588874817, + -0.2628254294395447, + -0.3415844738483429, + -0.35313117504119873, + 0.3155616521835327, + -0.556646466255188, + -2.0992307662963867, + -0.7895610928535461 + ], + [ + 0.39962780475616455, + 0.3406272530555725, + 0.05363208055496216, + -0.06018832325935364, + -0.5703235268592834, + -2.5974044799804688, + -0.6783060431480408, + 0.7815009355545044, + 0.35641810297966003, + -0.00838596560060978, + 0.6228271126747131, + 0.25608590245246887, + -2.5914058685302734, + -0.8502807021141052, + 0.726007878780365, + -0.6774719953536987, + 0.6261520981788635, + 1.149846076965332, + 0.6782730221748352, + -3.391972541809082, + -0.7993651032447815 + ], + [ + -0.29702335596084595, + 0.6166410446166992, + 0.6319320201873779, + -0.6542804837226868, + -0.8892693519592285, + -1.8599720001220703, + -1.3999924659729004, + -0.3017314672470093, + 0.9357973337173462, + -0.5596981048583984, + 1.3143360614776611, + -0.30643409490585327, + -2.028008460998535, + -1.3395459651947021, + 0.6212120056152344, + 0.6739580631256104, + -0.5840877294540405, + -0.06664055585861206, + -0.23446696996688843, + -2.0225906372070312, + -2.438906669616699 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0649428591132164, + 0.09363124519586563, + -0.014398600906133652, + -0.012044146656990051, + 0.03117428719997406, + -1.7716264724731445, + 0.0, + 0.05836518481373787, + 0.012665675021708012, + -0.013142907992005348, + -0.06252282857894897, + -0.06281104683876038, + -1.7224881649017334, + 0.0, + -0.06448972970247269, + -0.03696601092815399, + 0.030633779242634773, + -0.0031400322914123535, + 0.0327644944190979, + -1.4826252460479736, + 0.0 + ], + [ + -0.04005520045757294, + 0.2738151252269745, + -0.05346817523241043, + 0.20275527238845825, + 0.10101960599422455, + -1.7716264724731445, + 0.0, + -0.36066195368766785, + -0.21821273863315582, + 0.22022001445293427, + -0.0596311092376709, + -0.17951729893684387, + -1.7224881649017334, + 0.0, + 0.3205486238002777, + 0.25526633858680725, + 0.17392843961715698, + 0.5170043706893921, + 0.43821561336517334, + -1.4826252460479736, + 0.0 + ] + ], + "predicted_scores": [ + 4.094995975494385, + 4.043515205383301, + 3.7610859870910645, + 3.6654560565948486, + 3.6402814388275146, + 2.4322822093963623, + 1.8528903722763062, + 1.8338117599487305 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055567618/train2", + "selected_index": 0, + "state_hash": "027cc529c99c00db47a01ecfc12ab432557fbd911e8ff3c8a56adf2d3404549f", + "stored_base_utility": 0.851725161075592, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19648407399654388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19648407399654388 + }, + "base_progress": 0.19648407399654388, + "base_success": false, + "base_utility": 0.19648407399654388, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8651304244995117, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8651304244995117 + }, + { + "contact_quality": null, + "progress": 0.17065170407295227, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17065170407295227 + }, + { + "contact_quality": null, + "progress": 0.21422582864761353, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21422582864761353 + }, + { + "contact_quality": null, + "progress": 0.213461235165596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.213461235165596 + }, + { + "contact_quality": null, + "progress": 0.09443560242652893, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09443560242652893 + }, + { + "contact_quality": null, + "progress": 0.18556243181228638, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18556243181228638 + }, + { + "contact_quality": null, + "progress": 0.18798279762268066, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18798279762268066 + }, + { + "contact_quality": null, + "progress": 0.19312432408332825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19312432408332825 + } + ], + "candidate_progress": [ + 0.8651304244995117, + 0.17065170407295227, + 0.21422582864761353, + 0.213461235165596, + 0.09443560242652893, + 0.18556243181228638, + 0.18798279762268066, + 0.19312432408332825 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4048f8f6e047422e3c79", + "generated_tangents": [ + [ + 0.030008889734745026, + 0.17263352870941162, + 0.06290218234062195, + -0.35769036412239075, + 0.20301605761051178, + -0.013823091983795166, + 0.05073705315589905, + -0.011901825666427612, + 0.0407995879650116, + -0.06939861923456192, + 0.30223992466926575, + -0.13431647419929504, + -2.0956919193267822, + -0.06421901285648346, + 0.017468735575675964, + -0.025348056107759476, + -0.003789648413658142, + 0.11182623356580734, + -0.14155647158622742, + -1.6103920936584473, + -0.09692034125328064 + ], + [ + 0.47444403171539307, + 0.17549239099025726, + 0.01300850510597229, + -0.4280071258544922, + 0.4995863437652588, + -0.09945118427276611, + 1.1715638637542725, + -0.0569804310798645, + 0.010068010538816452, + -0.11193756014108658, + 0.2464638650417328, + -0.3236076235771179, + -1.5485386848449707, + -0.3787398934364319, + 0.03685591369867325, + 0.21671059727668762, + 0.2907164692878723, + 0.11012721806764603, + -0.14242395758628845, + -1.1806422472000122, + -0.5797140598297119 + ], + [ + 0.03376307338476181, + 0.40081632137298584, + -0.3041814863681793, + -0.25909775495529175, + 0.23389090597629547, + -0.28211724758148193, + 0.3170156478881836, + -0.08350473642349243, + -0.13009127974510193, + -0.04976027458906174, + 0.36134809255599976, + -0.3108815550804138, + -2.242213487625122, + -0.3403996229171753, + -0.11681632697582245, + -0.0013635903596878052, + 0.20280736684799194, + -0.11909632384777069, + -0.2411510944366455, + -1.9621775150299072, + -0.14653849601745605 + ], + [ + 0.01651117205619812, + 0.2903990149497986, + -0.2700941264629364, + -0.4617370367050171, + 0.33837172389030457, + -0.20104432106018066, + 0.19775182008743286, + -0.041856229305267334, + -0.08482728898525238, + -0.0354887917637825, + 0.31226062774658203, + -0.34998375177383423, + -2.218757152557373, + -0.29290342330932617, + -0.07298333942890167, + 0.044787146151065826, + 0.24641327559947968, + -0.11413340270519257, + -0.4040852189064026, + -1.803842544555664, + -0.2802719473838806 + ], + [ + 0.14583849906921387, + 0.10028946399688721, + -0.008981138467788696, + -0.4491742253303528, + 0.49456340074539185, + 0.320042222738266, + 0.535348117351532, + 0.08054546266794205, + -0.02265104465186596, + 0.18600349128246307, + 0.15243756771087646, + -0.22849059104919434, + -2.1810052394866943, + -0.567722737789154, + -0.006112057715654373, + 0.20684833824634552, + 0.33710867166519165, + 0.06878407299518585, + -0.5710155963897705, + -1.7927601337432861, + -0.26154419779777527 + ], + [ + 0.011184021830558777, + 0.29185396432876587, + -0.15513935685157776, + -0.44751620292663574, + 0.369235098361969, + 0.11543236672878265, + 0.1479061245918274, + -0.06622764468193054, + -0.097317636013031, + -0.04419920593500137, + 0.3350147306919098, + -0.38822662830352783, + -2.342780351638794, + -0.2594625949859619, + 0.002361048012971878, + 0.05833374336361885, + 0.3517112731933594, + -0.1215195506811142, + -0.3438400626182556, + -1.5425746440887451, + -0.2135961651802063 + ], + [ + 0.15981324017047882, + 0.2047366350889206, + -0.013066917657852173, + -0.49522680044174194, + 0.3404083251953125, + -0.04098178446292877, + -0.11504268646240234, + -0.02589324116706848, + -0.09764020144939423, + -0.028640396893024445, + 0.23497521877288818, + -0.16871348023414612, + -2.2523000240325928, + -0.08155752718448639, + -0.06333911418914795, + 0.08381980657577515, + 0.10687589645385742, + 0.0644548237323761, + -0.31258511543273926, + -1.6810500621795654, + -0.011160194873809814 + ], + [ + 0.09823539853096008, + 0.2795335054397583, + -0.46369877457618713, + -0.6231623888015747, + 0.29014328122138977, + -0.18661874532699585, + 0.34255513548851013, + 0.09270450472831726, + -0.11617457866668701, + 0.031556062400341034, + 0.24025917053222656, + -0.5258431434631348, + -2.278263807296753, + -0.4086526036262512, + -0.08707267045974731, + 0.10760253667831421, + 0.3566898703575134, + -0.00210687518119812, + -0.5319662094116211, + -1.8286168575286865, + -0.1939733922481537 + ] + ], + "generated_utilities": [ + 0.8651304244995117, + 0.17065170407295227, + 0.21422582864761353, + 0.213461235165596, + 0.09443560242652893, + 0.18556243181228638, + 0.18798279762268066, + 0.19312432408332825 + ], + "group_id": "ms-4048f8f6e047422e3c79", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558006286621094, + 2.0, + 0.1986379623413086, + 0.03769078850746155, + 0.5361635684967041, + 0.18274834752082825, + 0.17300468683242798, + 0.12688571214675903, + 0.13836368918418884, + 0.07454231381416321, + 0.0899321436882019, + 0.10402637720108032, + 0.1711854636669159, + 0.17255032062530518, + 0.039613887667655945, + 0.10826331377029419 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1422826200723648, + 0.21033799648284912, + 0.1978604793548584, + 0.059162259101867676, + -0.2529524564743042, + -0.15161612629890442, + 0.0, + -0.2264404594898224, + -0.2209387719631195, + -0.025619791820645332, + 0.06467145681381226, + 0.07422274351119995, + -1.7048513889312744, + 0.0, + 0.15056702494621277, + -0.06247536092996597, + -0.1033044084906578, + 0.1334916353225708, + 0.2784581184387207, + -1.4811434745788574, + 0.0 + ], + [ + -0.44999396800994873, + 0.271686851978302, + -0.4267011880874634, + 0.2730966806411743, + -0.01798880100250244, + -0.04680868983268738, + 0.0, + -0.09472770243883133, + -0.06993720680475235, + -0.06752754002809525, + -0.13884133100509644, + 0.24737784266471863, + -1.7048513889312744, + 0.0, + -0.09056920558214188, + 0.4132460653781891, + 0.04097161442041397, + -0.25072288513183594, + 0.49742037057876587, + -1.4811434745788574, + 0.0 + ], + [ + 0.17392832040786743, + 1.2103379964828491, + -0.3708428144454956, + -0.36093437671661377, + 0.744896650314331, + 0.23929360508918762, + 1.4388959407806396, + -0.04149632900953293, + 0.015302501618862152, + 0.023577755317091942, + 0.5954978466033936, + -0.3274606168270111, + -2.7048513889312744, + -1.646976351737976, + 0.0006552413105964661, + -0.008917607367038727, + 0.022643378004431725, + 0.7492771148681641, + -0.26030659675598145, + -2.4811434745788574, + -1.684959888458252 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8777918815612793, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.293952703475952, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.369919776916504 + ], + [ + 0.1792769730091095, + 0.9774190187454224, + -0.5587559342384338, + -0.5885956883430481, + 0.9904338717460632, + 0.41647347807884216, + 1.4140102863311768, + -0.2462461292743683, + -0.1271664798259735, + -0.2736872732639313, + 0.8877567052841187, + -0.6213628053665161, + -2.937077283859253, + -1.3732452392578125, + -0.0015218837652355433, + -0.27707284688949585, + -0.16698139905929565, + 0.4978373646736145, + -0.479316771030426, + -2.7234396934509277, + -1.6759265661239624 + ], + [ + 0.1658722460269928, + 1.3070863485336304, + -0.4893108606338501, + -0.04209986329078674, + 0.7449368238449097, + 0.3420809805393219, + 1.4472918510437012, + 0.08132945746183395, + 0.13181470334529877, + 0.22494959831237793, + 0.22593030333518982, + -0.49233078956604004, + -2.7544617652893066, + -1.7597702741622925, + -0.365922212600708, + -0.17204320430755615, + -0.1821347028017044, + 0.6334611177444458, + -0.33006516098976135, + -2.2188992500305176, + -1.5452594757080078 + ], + [ + 0.18608316779136658, + 1.23563551902771, + -0.5727517604827881, + 0.10049733519554138, + 0.9073957204818726, + 0.5650919675827026, + 1.8946568965911865, + -0.17719107866287231, + 0.34025683999061584, + 0.44525983929634094, + 0.6176816821098328, + -0.2832425534725189, + -3.1802117824554443, + -1.8703548908233643, + 0.07433810085058212, + 0.1340292990207672, + 0.497944176197052, + 1.0071699619293213, + -0.44033437967300415, + -2.7992849349975586, + -1.8935935497283936 + ], + [ + 0.40270811319351196, + 1.5102664232254028, + -0.8070739507675171, + -0.16390679776668549, + 0.3215266466140747, + -0.16522419452667236, + 1.0900052785873413, + 0.07780961692333221, + 0.5627153515815735, + 0.5386761426925659, + 0.2949375510215759, + 0.059314221143722534, + -3.2417359352111816, + -1.899662971496582, + 0.4160631597042084, + 0.44336366653442383, + 0.47819244861602783, + 1.162381649017334, + -0.7831735014915466, + -2.625349521636963, + -1.2278814315795898 + ], + [ + 0.8529313802719116, + 1.5106499195098877, + 0.19478559494018555, + -0.6338159441947937, + 0.7529481053352356, + -0.24222904443740845, + 0.9723314046859741, + -0.14078867435455322, + -0.5563283562660217, + 0.5440018177032471, + 0.004017293453216553, + 0.019366413354873657, + -3.1521430015563965, + -2.0841636657714844, + -0.2760852575302124, + -0.5531485676765442, + 0.22684869170188904, + 1.100138545036316, + 0.046634942293167114, + -2.429711103439331, + -2.280839443206787 + ], + [ + -0.4760962128639221, + 1.8579951524734497, + 0.2322864532470703, + 0.19953304529190063, + 0.48002323508262634, + 0.7484984397888184, + 2.073361396789551, + -0.04567806422710419, + 0.04004429653286934, + -0.5635936260223389, + 0.5135918259620667, + 0.3908633887767792, + -1.9335949420928955, + -2.442507743835449, + -0.6620182394981384, + 0.4175874888896942, + 0.7711085677146912, + 1.2790143489837646, + -0.5464571714401245, + -3.2030673027038574, + -2.3768434524536133 + ], + [ + 1.0231709480285645, + 1.1804075241088867, + 0.11513572931289673, + -1.23623526096344, + 1.4848686456680298, + 1.1120561361312866, + 0.9504602551460266, + 0.8234015703201294, + -0.04407603666186333, + -0.37310001254081726, + 0.24512693285942078, + 0.4242170751094818, + -2.4985997676849365, + -2.1206297874450684, + 0.5509956479072571, + 0.8686066269874573, + -0.5043656229972839, + 1.3957195281982422, + -0.07662208378314972, + -2.371795654296875, + -2.2404346466064453 + ], + [ + 0.49735015630722046, + 0.3600291609764099, + 0.3103446364402771, + 0.6051294803619385, + -0.15457379817962646, + -0.5570405721664429, + 1.2383109331130981, + 0.9059417247772217, + 0.7879754900932312, + -0.31492647528648376, + 1.5871094465255737, + -0.8906484842300415, + -1.8698136806488037, + -2.401073694229126, + 0.3560212254524231, + -0.6580788493156433, + 0.9037240147590637, + 1.614617109298706, + 0.3788090944290161, + -1.6909503936767578, + -0.8765208125114441 + ], + [ + 0.10883396118879318, + 1.8993752002716064, + -0.7673929333686829, + -0.0936923623085022, + -0.15508884191513062, + -0.1517968773841858, + 0.465425968170166, + 0.43258219957351685, + 0.46641960740089417, + -0.3528187572956085, + 1.09894597530365, + -0.6569450497627258, + -2.025486946105957, + -1.2820806503295898, + -0.21141839027404785, + -0.9515489339828491, + 0.7230286002159119, + -0.21062928438186646, + -0.32822558283805847, + -3.289642333984375, + -1.7125959396362305 + ], + [ + 0.37358880043029785, + 2.052347183227539, + -1.212618112564087, + -0.6321570873260498, + 0.7031511068344116, + -0.3477393090724945, + 1.2687753438949585, + 0.8023800253868103, + -0.031013749539852142, + 0.35525181889533997, + 0.5262036323547363, + -0.9288194179534912, + -1.7686026096343994, + -2.128396987915039, + -0.7001423239707947, + -0.12872236967086792, + -0.012955078855156898, + 1.0237239599227905, + 0.03636348247528076, + -2.0397119522094727, + -0.9761244058609009 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.060203760862350464, + 0.21033799648284912, + 0.0356404185295105, + -0.03044089674949646, + 0.024639129638671875, + -0.032671064138412476, + 0.0, + 0.016544528305530548, + 0.04633430764079094, + 0.005854343995451927, + -0.014876246452331543, + 0.0028293728828430176, + -1.7048513889312744, + 0.0, + -0.05953434109687805, + -0.04181046783924103, + -0.0018185675144195557, + -0.09216523170471191, + 0.019338399171829224, + -1.4811434745788574, + 0.0 + ] + ], + "predicted_scores": [ + 7.490676403045654, + 7.170323371887207, + 6.697402000427246, + 6.384732723236084, + 6.295480251312256, + 6.164147853851318, + 5.90757942199707, + 5.689966678619385 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055157354/train2", + "selected_index": 0, + "state_hash": "ae21d2b71722381a530d3a9259d02ea6728eb8a4a84fafc40ccd653d62ad3968", + "stored_base_utility": 0.8558006286621094, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19844311475753784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844311475753784 + }, + "base_progress": 0.19844311475753784, + "base_success": false, + "base_utility": 0.19844311475753784, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.17567642033100128, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17567642033100128 + }, + { + "contact_quality": null, + "progress": 0.1609627604484558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1609627604484558 + }, + { + "contact_quality": null, + "progress": 0.23198044300079346, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23198044300079346 + }, + { + "contact_quality": null, + "progress": 0.18044036626815796, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18044036626815796 + }, + { + "contact_quality": null, + "progress": 0.21981436014175415, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21981436014175415 + }, + { + "contact_quality": null, + "progress": 0.15481720864772797, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15481720864772797 + }, + { + "contact_quality": null, + "progress": 0.2075604647397995, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2075604647397995 + }, + { + "contact_quality": null, + "progress": 0.22741466760635376, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22741466760635376 + } + ], + "candidate_progress": [ + 0.17567642033100128, + 0.1609627604484558, + 0.23198044300079346, + 0.18044036626815796, + 0.21981436014175415, + 0.15481720864772797, + 0.2075604647397995, + 0.22741466760635376 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-ecb37ebbab21a506d8ca", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-5b26c33feed6992386b1", + "generated_tangents": [ + [ + -0.003956928849220276, + -0.06990145146846771, + -0.16480055451393127, + 0.30755630135536194, + -0.012048155069351196, + -1.086521029472351, + -0.14643651247024536, + 0.07404845952987671, + 0.050366371870040894, + 0.03993061184883118, + 0.13770818710327148, + 0.24159541726112366, + -1.9209805727005005, + -0.0966176986694336, + 0.08004353195428848, + 0.06546422839164734, + 0.08362357318401337, + -0.14493761956691742, + 0.2019823044538498, + -1.9210615158081055, + -0.3066258430480957 + ], + [ + -0.0462736040353775, + -0.05885855853557587, + -0.09201386570930481, + 0.23123934864997864, + -0.021240919828414917, + -0.785561203956604, + -0.19830679893493652, + 0.1271599531173706, + 0.04168267548084259, + 0.030569881200790405, + 0.15824760496616364, + 0.2760062515735626, + -2.0482239723205566, + -0.05982495844364166, + 0.1003795936703682, + 0.06336382031440735, + 0.11962166428565979, + -0.16470323503017426, + 0.1422899067401886, + -1.9973554611206055, + -0.2114710509777069 + ], + [ + -0.19363166391849518, + -0.14713603258132935, + -0.39566370844841003, + 0.4984795153141022, + 0.11760872602462769, + -1.262765645980835, + 0.09189155697822571, + 0.22310979664325714, + -0.0947994515299797, + -0.11231277137994766, + -0.017672404646873474, + 0.22374528646469116, + -2.1181323528289795, + 0.038686588406562805, + 0.005642283707857132, + 0.044049616903066635, + 0.29121851921081543, + -0.08374150097370148, + -0.028894901275634766, + -1.9262609481811523, + -0.36327096819877625 + ], + [ + 0.010901883244514465, + 0.0019717514514923096, + -0.20607814192771912, + 0.25729432702064514, + 0.029973581433296204, + -0.811781644821167, + -0.17005163431167603, + 0.1184350997209549, + -0.028967997059226036, + 0.03754618763923645, + 0.07278949022293091, + 0.33260172605514526, + -2.0846428871154785, + -0.07179608941078186, + 0.113896444439888, + 0.03850396350026131, + 0.11058761924505234, + -0.13422997295856476, + 0.1795283406972885, + -2.0686228275299072, + -0.23739710450172424 + ], + [ + -0.2527129650115967, + -0.0534171387553215, + -0.11055079102516174, + 0.287568598985672, + 0.007939234375953674, + -0.7499806880950928, + -0.17638403177261353, + 0.23254576325416565, + -0.10218941420316696, + -0.1576124131679535, + -0.07019532471895218, + 0.21945291757583618, + -2.081728219985962, + -0.04562319815158844, + 0.21058905124664307, + -0.01752379722893238, + 0.1355130672454834, + -0.0845472663640976, + 0.021035194396972656, + -2.333874464035034, + -0.1841607689857483 + ], + [ + -0.14197538793087006, + -0.10166645050048828, + -0.13611909747123718, + 0.3405918776988983, + -0.017250031232833862, + -0.8338378667831421, + -0.10513591766357422, + 0.09952886402606964, + 0.046714864671230316, + 0.09271503984928131, + 0.11503598093986511, + 0.416715145111084, + -1.8861693143844604, + -0.15263445675373077, + 0.1018257886171341, + 0.10959236323833466, + 0.09383639693260193, + -0.18258638679981232, + 0.10749277472496033, + -1.9266455173492432, + -0.3445923626422882 + ], + [ + -0.27726882696151733, + -0.06937313079833984, + -0.08735474944114685, + 0.3757733404636383, + 0.11187510937452316, + -0.7334935665130615, + -0.10866284370422363, + 0.2559641897678375, + -0.10003504902124405, + -0.07836895436048508, + 0.10444165766239166, + 0.14157049357891083, + -2.126008987426758, + 0.013141870498657227, + 0.04199111461639404, + -0.004544961266219616, + 0.1588364690542221, + -0.10955263674259186, + 0.033526286482810974, + -2.068049430847168, + -0.2482026219367981 + ], + [ + -0.2805596590042114, + -0.08894135057926178, + -0.2837352454662323, + 0.3582393229007721, + 0.25944972038269043, + -0.7279287576675415, + -0.1078721284866333, + 0.16095273196697235, + -0.11950705945491791, + -0.044760726392269135, + -0.0387117937207222, + 0.05619433522224426, + -2.170316696166992, + -0.10741914808750153, + -0.06863324344158173, + 0.025339357554912567, + 0.005009979009628296, + -0.028363361954689026, + 0.016270101070404053, + -2.294799566268921, + -0.22589710354804993 + ] + ], + "generated_utilities": [ + 0.17567642033100128, + 0.1609627604484558, + 0.23198044300079346, + 0.18044036626815796, + 0.21981436014175415, + 0.15481720864772797, + 0.2075604647397995, + 0.22741466760635376 + ], + "group_id": "ms-5b26c33feed6992386b1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8326650857925415, + 2.0, + 0.15131878852844238, + 0.15007886290550232, + 0.5293327569961548, + 0.06490398943424225, + 0.5425760746002197, + 0.20163774490356445, + 0.15259261429309845, + 0.23924465477466583, + 0.12766429781913757, + 0.08033241331577301, + 0.07788355648517609, + 0.13886235654354095, + 0.09716498851776123, + 0.11682072281837463 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.06662726402282715, + -0.14569896459579468, + 0.04974943399429321, + 0.539892315864563, + -0.19018621742725372, + -0.5884796380996704, + 0.0, + 0.12748029828071594, + 0.11038697510957718, + -0.03454907238483429, + 0.08700031787157059, + -0.13105249404907227, + -1.9082190990447998, + 0.0, + -0.0033895689994096756, + -0.15730009973049164, + 0.15846982598304749, + 0.11786291003227234, + -0.02387581765651703, + -1.5966379642486572, + 0.0 + ], + [ + -0.06662726402282715, + 0.059319257736206055, + -0.19934123754501343, + 0.539892315864563, + 0.07902508974075317, + -0.5884796380996704, + 0.0, + -0.3428957760334015, + 0.08558038622140884, + 0.28825634717941284, + -0.32335156202316284, + -0.3631777763366699, + -1.9082190990447998, + 0.0, + -0.3765835762023926, + -0.5554376840591431, + 0.3468284010887146, + 0.875933825969696, + 0.11397302150726318, + -1.5966379642486572, + 0.0 + ], + [ + -1.0666272640228271, + -0.6187554597854614, + -0.8818012475967407, + 1.539892315864563, + -0.428971529006958, + -1.5884796380996704, + 2.0647666454315186, + -0.03600159287452698, + 0.032059766352176666, + 0.030456876382231712, + 0.14377687871456146, + 0.20562076568603516, + -2.9082190990448, + -1.332404375076294, + -0.014788046479225159, + -0.0006958844605833292, + 0.023885300382971764, + 0.5519534349441528, + 0.1840958297252655, + -2.5966379642486572, + -1.4879226684570312 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.129533290863037, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.664808750152588, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9758453369140625 + ], + [ + -1.2971861362457275, + -0.7044790983200073, + -1.134492039680481, + 1.5821952819824219, + -0.7159937620162964, + -1.7615629434585571, + 2.031708002090454, + -0.25998222827911377, + 0.10180196911096573, + -0.24491730332374573, + 0.2074550986289978, + 0.2793378233909607, + -2.752415657043457, + -1.3360638618469238, + 0.03524862602353096, + -0.1854405552148819, + -0.1696440726518631, + 0.7172544598579407, + 0.0028989464044570923, + -2.680950164794922, + -1.2071388959884644 + ], + [ + -1.465846300125122, + -0.34887754917144775, + -1.046374797821045, + 1.703241229057312, + -0.19744430482387543, + -1.2604204416275024, + 1.9822474718093872, + 0.18854136765003204, + -0.146264910697937, + 0.059789031744003296, + 0.39534950256347656, + 0.29636046290397644, + -2.6612584590911865, + -1.5028973817825317, + 0.059081435203552246, + -0.11711112409830093, + -0.07238245755434036, + 0.25233787298202515, + 0.12625277042388916, + -2.2362401485443115, + -1.7053974866867065 + ], + [ + -1.1892943382263184, + -0.20831987261772156, + -0.8013461828231812, + 1.7197901010513306, + -0.39134201407432556, + -1.5160430669784546, + 2.4594008922576904, + -0.1669330894947052, + -0.337530255317688, + 0.15848031640052795, + 0.05738166719675064, + 0.41757503151893616, + -3.1393940448760986, + -0.9257140755653381, + -0.5068305134773254, + 0.1864723265171051, + 0.45369866490364075, + 0.6036283373832703, + -0.018199875950813293, + -2.8028440475463867, + -1.3847744464874268 + ], + [ + -1.1111165285110474, + -1.191490888595581, + -1.361661434173584, + 1.197826623916626, + -0.29219919443130493, + -1.7355475425720215, + 2.0579750537872314, + -0.08019472658634186, + 0.5140348672866821, + -0.5474883317947388, + -0.03311571478843689, + -0.060686588287353516, + -2.3427765369415283, + -1.8552747964859009, + -0.36303937435150146, + 0.01966947503387928, + -0.5651369094848633, + 0.22617560625076294, + -0.16322728991508484, + -2.421419143676758, + -0.921785831451416 + ], + [ + -1.012363076210022, + -0.6483663320541382, + -0.7951939702033997, + 1.2563079595565796, + -0.5323562622070312, + -1.5913673639297485, + 1.406038761138916, + -0.39036238193511963, + -0.12943506240844727, + -0.4373972415924072, + -0.20462383329868317, + 0.12806488573551178, + -3.570098400115967, + -1.8897531032562256, + 0.6369156837463379, + 0.2483828067779541, + 0.6809477210044861, + 0.8138958215713501, + 0.6604747176170349, + -2.717491865158081, + -1.9953765869140625 + ], + [ + -0.378162145614624, + -0.23438718914985657, + -1.2589956521987915, + 1.844052791595459, + -0.8936926126480103, + -1.230797290802002, + 2.5580525398254395, + -0.21314577758312225, + -0.05935201793909073, + 0.46811148524284363, + 0.38594305515289307, + 1.0049593448638916, + -2.491497278213501, + -1.3096134662628174, + 0.15605805814266205, + 0.7682130932807922, + -0.4010901153087616, + 0.8638691902160645, + 0.5755372047424316, + -3.324826240539551, + -1.6860322952270508 + ], + [ + -0.6920164227485657, + -0.5060986280441284, + -0.5146759748458862, + 2.3206489086151123, + 0.26810598373413086, + -1.7696658372879028, + 1.4922571182250977, + -0.6155052185058594, + -0.5581206679344177, + 0.4068271219730377, + 0.5812678337097168, + -0.2056950330734253, + -2.227752923965454, + -0.7079744935035706, + 0.3153572678565979, + 0.019053615629673004, + -0.10691303759813309, + -0.18750637769699097, + 0.5490041971206665, + -2.7611911296844482, + -1.0861625671386719 + ], + [ + -0.9623568654060364, + -0.821997880935669, + -0.0017323493957519531, + 1.8667385578155518, + 0.39793866872787476, + -1.3673040866851807, + 2.8636348247528076, + -0.739641547203064, + -0.4186078906059265, + -0.21357670426368713, + 0.6820193529129028, + 0.09538085013628006, + -3.242215633392334, + -1.023385763168335, + -0.2367737591266632, + 0.9248462915420532, + 0.04536557197570801, + 0.2294020652770996, + 0.15174128115177155, + -2.509904146194458, + -0.5901334881782532 + ], + [ + -0.565070390701294, + 0.37745314836502075, + -1.7464253902435303, + 1.4869437217712402, + -1.187619924545288, + -2.3802757263183594, + 2.5987753868103027, + 0.5777139663696289, + -0.04772312939167023, + -0.49332448840141296, + -0.515097975730896, + 0.751485288143158, + -2.7030012607574463, + -1.5739474296569824, + 0.8266244530677795, + -0.9454089403152466, + 0.7678747177124023, + 1.0557427406311035, + 0.0584852397441864, + -2.8590199947357178, + -0.8677505850791931 + ], + [ + -0.2763708233833313, + 0.28772038221359253, + -0.7008944153785706, + 1.3402774333953857, + -0.78607177734375, + -2.1210508346557617, + 1.4908761978149414, + -0.4155166447162628, + 0.40930652618408203, + -0.1769891083240509, + -0.20092366635799408, + 0.5321645736694336, + -3.2144649028778076, + -1.2440845966339111, + -0.247700035572052, + 0.7174559235572815, + 0.11071103811264038, + -0.023232340812683105, + 0.6976174116134644, + -2.9463140964508057, + -0.5927516222000122 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.06662726402282715, + -0.05619156360626221, + -0.02257448434829712, + 0.539892315864563, + 0.053197890520095825, + -0.5884796380996704, + 0.0, + -0.04219311475753784, + 0.010949447751045227, + 0.027476036921143532, + -0.02041013538837433, + -0.01791653037071228, + -1.9082190990447998, + 0.0, + 0.01256529800593853, + 0.014024228788912296, + 0.03545428439974785, + 0.019800782203674316, + -0.022738635540008545, + -1.5966379642486572, + 0.0 + ] + ], + "predicted_scores": [ + 6.3688836097717285, + 6.3467512130737305, + 6.275082111358643, + 6.2109055519104, + 6.087492942810059, + 6.081385612487793, + 5.703493118286133, + 5.19040060043335 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1522272686/train2", + "selected_index": 0, + "state_hash": "10034ca87ce4b5c491c0aa23d2a30389ba8265fa695d65198bf719c54565cf2a", + "stored_base_utility": 0.8326650857925415, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8388344049453735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8388344049453735 + }, + "base_progress": 0.8388344049453735, + "base_success": false, + "base_utility": 0.8388344049453735, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8181180953979492, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8181180953979492 + }, + { + "contact_quality": null, + "progress": 0.1479516476392746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1479516476392746 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1400405466556549, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1400405466556549 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.8181180953979492, + 0.1479516476392746, + 1.0, + 1.0, + 1.0, + 0.1400405466556549, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-5a456ddf97d294548cf1", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-5dfbb16e173e1a3e2bd6" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-63ac245579b1188935ad", + "generated_tangents": [ + [ + 0.0875767171382904, + 0.06667950004339218, + -0.11912426352500916, + 0.02909502387046814, + 0.04533493518829346, + -1.8368381261825562, + -0.1082448959350586, + -0.0030595362186431885, + -0.006928380578756332, + 0.10766997188329697, + 0.13685259222984314, + 0.1791902482509613, + -1.384962558746338, + -0.08337503671646118, + -0.057299669831991196, + 0.07471176981925964, + 0.12037551403045654, + 0.0058998167514801025, + -0.016657859086990356, + -1.1310760974884033, + -0.027680858969688416 + ], + [ + -0.03461998701095581, + 0.030188776552677155, + 0.02041885256767273, + 0.0791589617729187, + 0.04111059010028839, + -1.769657850265503, + -0.13236629962921143, + -0.0026993602514266968, + -0.012054651975631714, + 0.063559390604496, + 0.23516583442687988, + 0.23861247301101685, + -1.475093960762024, + -0.0302799791097641, + -0.031195703893899918, + 0.1027858704328537, + 0.029262766242027283, + -0.056897953152656555, + -0.06885898113250732, + -0.9965713620185852, + -0.056251779198646545 + ], + [ + 0.13037967681884766, + 0.044371433556079865, + -0.023115307092666626, + -0.055332034826278687, + 0.10363438725471497, + -1.8522902727127075, + -0.11580264568328857, + 0.09957293421030045, + -0.029794400557875633, + 0.06683573871850967, + 0.16266819834709167, + 0.13251268863677979, + -1.3987717628479004, + -0.11576487123966217, + -0.0020797140896320343, + 0.10281313955783844, + 0.16798318922519684, + -0.029749050736427307, + -0.02250218391418457, + -1.0928770303726196, + -0.019477322697639465 + ], + [ + 0.1336415708065033, + 0.021478980779647827, + -0.0920698344707489, + 0.06146487593650818, + 0.058310478925704956, + -1.9416826963424683, + -0.16801291704177856, + 0.011562362313270569, + -0.0352933332324028, + 0.09769804030656815, + 0.19700577855110168, + 0.0953979641199112, + -1.4475812911987305, + -0.13870875537395477, + 0.011281915009021759, + 0.10560320317745209, + 0.08078952133655548, + 0.02909603714942932, + 0.04007375240325928, + -1.1974456310272217, + -0.09815460443496704 + ], + [ + 0.14251402020454407, + 0.07695221155881882, + -0.014767199754714966, + -0.052157893776893616, + 0.11734548956155777, + -1.8062281608581543, + -0.11383187770843506, + 0.15381109714508057, + -0.07094687223434448, + 0.09402982890605927, + 0.22719672322273254, + 0.20944231748580933, + -1.2605376243591309, + -0.07241888344287872, + -0.04573053494095802, + 0.08019408583641052, + 0.1890329122543335, + 0.13442978262901306, + 0.09435175359249115, + -1.0764597654342651, + -0.06758096814155579 + ], + [ + 0.03332400321960449, + -0.02650948241353035, + -0.027606159448623657, + 0.028628617525100708, + 0.1964508593082428, + -1.9185587167739868, + -0.10465216636657715, + 0.027949780225753784, + -0.03177535533905029, + 0.07444071769714355, + 0.19828975200653076, + 0.2741793990135193, + -1.389540433883667, + -0.1669108122587204, + 0.105293408036232, + 0.10851116478443146, + 0.14594432711601257, + -0.08073602616786957, + -0.14245057106018066, + -1.159231185913086, + -0.1667010486125946 + ], + [ + 0.14103801548480988, + 0.010021902620792389, + -0.05186501145362854, + 0.01830558478832245, + 0.1603812575340271, + -1.7513757944107056, + -0.13741683959960938, + 0.09187138825654984, + -0.001362161710858345, + 0.08333832025527954, + 0.2154909074306488, + 0.19489426910877228, + -1.264728307723999, + -0.15727905929088593, + 0.007760878652334213, + 0.13821762800216675, + 0.19939735531806946, + -0.008028343319892883, + 0.012169510126113892, + -0.9980083703994751, + -0.16007345914840698 + ], + [ + -0.2055303007364273, + -0.07026727497577667, + 0.19995340704917908, + -0.1151643916964531, + 0.23799385130405426, + -1.6392040252685547, + -0.04348176717758179, + 0.1457993984222412, + -0.12437643110752106, + 0.05682114511728287, + 0.28148573637008667, + 0.23318122327327728, + -1.691398024559021, + 0.042029380798339844, + -0.0035445429384708405, + 0.08843211829662323, + 0.2996772527694702, + -0.03317791223526001, + -0.17194655537605286, + -1.0370934009552002, + 0.09168896079063416 + ] + ], + "generated_utilities": [ + 2.0, + 0.8181180953979492, + 0.1479516476392746, + 2.0, + 2.0, + 2.0, + 0.1400405466556549, + 2.0 + ], + "group_id": "ms-63ac245579b1188935ad", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.855198860168457, + 0.8138954639434814, + 0.20312510430812836, + 2.0, + 0.7862945795059204, + 0.7843606472015381, + 0.6004383563995361, + 0.15506848692893982, + 0.15924105048179626, + 0.132452130317688, + 0.05381816625595093, + 0.09561765193939209, + 0.07140937447547913, + 0.10806754231452942, + 0.023168057203292847, + 0.07957816123962402 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.007633227854967117, + 0.02389371022582054, + 0.2181050330400467, + 0.026324495673179626, + 0.1209479570388794, + -1.3937149047851562, + 0.0, + -0.2493116855621338, + -0.17256586253643036, + -0.06812826544046402, + 0.019445210695266724, + -0.2657848000526428, + -1.0657541751861572, + 0.0, + -0.04306507110595703, + -0.07170329242944717, + 0.035314902663230896, + 0.06211268901824951, + 0.18158195912837982, + -0.7849693298339844, + 0.0 + ], + [ + -0.011555496603250504, + 0.0016381663735955954, + 0.019193550571799278, + 0.26171422004699707, + 0.08122420310974121, + -2.3937149047851562, + -1.53977632522583, + -0.00580335408449173, + -0.0020052569452673197, + 0.01787502132356167, + 0.4615197777748108, + 0.1938573122024536, + -2.0657541751861572, + -1.4660639762878418, + -0.00267898291349411, + -0.0038005562964826822, + 0.0179074015468359, + 0.5644202828407288, + 0.2690354883670807, + -1.7849693298339844, + -1.3308851718902588 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.07955265045166, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9321279525756836, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.6617703437805176 + ], + [ + -0.16036809980869293, + 0.11436319351196289, + -0.14857570827007294, + 0.09414434432983398, + 0.02274160459637642, + -2.426297903060913, + -1.7686878442764282, + -0.26862773299217224, + 0.16241224110126495, + 0.24208402633666992, + 0.4166663587093353, + 0.07841069251298904, + -2.2318098545074463, + -1.2228643894195557, + -0.08888694643974304, + 0.20051127672195435, + -0.14729085564613342, + 0.5799955129623413, + 0.33039355278015137, + -2.00492262840271, + -1.4948352575302124 + ], + [ + -0.3820815682411194, + -0.3679526746273041, + 0.18845602869987488, + 0.4168931245803833, + -0.007936321198940277, + -2.700572967529297, + -1.795093059539795, + 0.1203194335103035, + 0.19610334932804108, + 0.2749679982662201, + 0.8331882357597351, + -0.06589585542678833, + -2.333136558532715, + -1.1008005142211914, + -0.02170141413807869, + 0.24186654388904572, + -0.3734443485736847, + 0.5144574046134949, + 0.3134346902370453, + -1.5088386535644531, + -1.1978046894073486 + ], + [ + 0.13625949621200562, + 0.47836095094680786, + -0.05790846049785614, + -0.09614765644073486, + -0.14281350374221802, + -2.522264003753662, + -1.0473889112472534, + 0.4792231023311615, + -0.4869813621044159, + 0.17644789814949036, + 0.8425257205963135, + -0.28899794816970825, + -2.5508456230163574, + -0.9762126207351685, + -0.16381770372390747, + 0.1586131900548935, + -0.3460317850112915, + 0.6056831479072571, + 0.7327044010162354, + -1.494289755821228, + -1.4671605825424194 + ], + [ + -0.40018847584724426, + -0.028111951425671577, + 0.31324630975723267, + -0.22690656781196594, + 0.2902344763278961, + -2.9841673374176025, + -1.4654184579849243, + -0.1155615746974945, + -0.4125809073448181, + 0.6098153591156006, + 0.8544195890426636, + 0.22702345252037048, + -1.7853796482086182, + -1.3474400043487549, + -0.454882949590683, + -0.3122820556163788, + 0.22399993240833282, + 0.05299121141433716, + 0.07802119851112366, + -1.3896000385284424, + -1.894580602645874 + ], + [ + -0.6587541699409485, + -0.6905457973480225, + 0.10385943204164505, + -0.2785833477973938, + 0.2321375161409378, + -2.565701723098755, + -1.1605603694915771, + -0.6775721907615662, + -0.30887091159820557, + 0.1186944767832756, + 1.0924456119537354, + 0.5009081363677979, + -1.8667830228805542, + -1.2491121292114258, + -0.11290804296731949, + -0.03346216678619385, + -0.28778207302093506, + 1.2255009412765503, + 0.6856998801231384, + -1.1723419427871704, + -1.1220875978469849 + ], + [ + -0.5019480586051941, + 0.07311037182807922, + 0.5101989507675171, + 0.7159117460250854, + 0.005071915686130524, + -2.6556949615478516, + -2.321233034133911, + 0.6035679578781128, + -0.13785676658153534, + -0.4309522807598114, + 0.9664275050163269, + -0.020235419273376465, + -1.6894025802612305, + -0.7511875033378601, + 0.5713916420936584, + 0.24962951242923737, + 0.40067923069000244, + 0.12209588289260864, + 0.03222101926803589, + -2.27744460105896, + -1.9113109111785889 + ], + [ + 0.7552750110626221, + -0.8085685968399048, + 0.20896124839782715, + -0.6043995022773743, + 0.733860969543457, + -1.610467791557312, + -1.0368051528930664, + -0.02292630262672901, + 0.6320546269416809, + 0.14528857171535492, + 0.8372507095336914, + 0.41523146629333496, + -2.339751958847046, + -0.6185142993927002, + -0.5750091075897217, + 0.12324154376983643, + 0.2804809510707855, + 1.0503896474838257, + -0.15392154455184937, + -0.9555137753486633, + -1.517809510231018 + ], + [ + 0.8853163123130798, + 0.09475164860486984, + -0.13681088387966156, + 0.08429434895515442, + 0.13555681705474854, + -1.5859344005584717, + -2.1957859992980957, + 0.9337366819381714, + -1.0016963481903076, + 0.11365362256765366, + -0.31175100803375244, + 0.7412776350975037, + -1.0797133445739746, + -0.8527018427848816, + 0.6110302209854126, + 0.006565961055457592, + 0.38363590836524963, + 0.5695288181304932, + -0.2547180950641632, + -2.6268043518066406, + -1.748514175415039 + ], + [ + 0.513801097869873, + 0.7078304886817932, + 0.7309613823890686, + -0.32746708393096924, + -0.36263394355773926, + -3.3305888175964355, + -1.2108653783798218, + 0.5104080438613892, + -0.26447102427482605, + 0.6361017823219299, + 0.5986918210983276, + -0.48598653078079224, + -2.4424209594726562, + -1.4694169759750366, + -0.5649402737617493, + -0.22020848095417023, + 0.6405193209648132, + -0.0698668360710144, + 0.7155715227127075, + -2.352860450744629, + -0.7502654194831848 + ], + [ + -0.9911022782325745, + 0.9677192568778992, + 0.5195661187171936, + 0.6733791828155518, + 0.7589308023452759, + -3.3654959201812744, + -0.6360958814620972, + 0.5926312208175659, + -0.06773044914007187, + 0.3785405457019806, + 0.5934506058692932, + 0.6434512138366699, + -2.891094923019409, + -0.8924227356910706, + 0.8036434650421143, + 0.6109268665313721, + -0.6607317924499512, + 1.0173940658569336, + 0.2546094059944153, + -1.4653143882751465, + -2.0709643363952637 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16686837375164032, + 0.1354454904794693, + -0.11150697618722916, + 0.7247087359428406, + 0.2886054813861847, + -1.3937149047851562, + 0.0, + 0.6050093770027161, + 0.20291854441165924, + 0.5827622413635254, + -0.2687305808067322, + -0.25171223282814026, + -1.0657541751861572, + 0.0, + 0.29065752029418945, + -0.023413484916090965, + -0.24587245285511017, + 0.4600236713886261, + -0.4278547465801239, + -0.7849693298339844, + 0.0 + ] + ], + "predicted_scores": [ + 5.630728244781494, + 5.463387966156006, + 5.200445652008057, + 5.182462215423584, + 5.181726455688477, + 5.047179698944092, + 4.899623870849609, + 4.412124156951904 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "211245842/train2", + "selected_index": 0, + "state_hash": "fb52e84b6addb8bdddf02a80d5c8567e0fa27195729e8440271b710441061b7f", + "stored_base_utility": 0.855198860168457, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.13187798857688904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13187798857688904 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.13499042391777039, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13499042391777039 + }, + { + "contact_quality": null, + "progress": 0.16052719950675964, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16052719950675964 + } + ], + "candidate_progress": [ + 0.13187798857688904, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.13499042391777039, + 0.16052719950675964 + ], + "candidate_source_chart_ids": [ + "ms-875a4668671c02054a3c", + "ms-20f3cfcf183c69853b1e", + "ms-875a4668671c02054a3c", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "generated_tangents": [ + [ + 0.05264825373888016, + 0.03665849566459656, + -0.04196348786354065, + -0.12311732769012451, + 0.32306361198425293, + -2.348386526107788, + -0.04350852966308594, + 0.05847747623920441, + -0.1053842082619667, + 0.12492304295301437, + 0.06711933761835098, + 0.211573526263237, + -2.1858420372009277, + -0.1017637699842453, + -0.007234741002321243, + -0.01731230691075325, + 0.31457260251045227, + 0.10490778833627701, + -0.11348345875740051, + -1.5930464267730713, + -0.1564388871192932 + ], + [ + 0.07825403660535812, + 0.058583058416843414, + -0.1342998445034027, + -0.03148752450942993, + 0.08402667939662933, + -2.2371621131896973, + -0.27078020572662354, + -0.024682804942131042, + 0.00805166456848383, + 0.17025572061538696, + 0.1683911830186844, + 0.07624384760856628, + -1.8365434408187866, + -0.17505960166454315, + 0.0820261612534523, + 0.018710482865571976, + 0.018737733364105225, + 0.022886931896209717, + -0.05467510223388672, + -1.6140875816345215, + -0.19445520639419556 + ], + [ + 0.05307849496603012, + 0.029461383819580078, + -0.10224887728691101, + -0.11858844757080078, + 0.0649922639131546, + -2.0333945751190186, + -0.2658430337905884, + 0.06256981194019318, + 0.036147210747003555, + 0.12818849086761475, + 0.22078293561935425, + 0.17197996377944946, + -1.9191186428070068, + -0.13697020709514618, + 0.04940533638000488, + 0.05206349864602089, + 0.027456894516944885, + -0.00018197298049926758, + 0.027761712670326233, + -1.6916732788085938, + -0.16476592421531677 + ], + [ + -0.057636141777038574, + -0.01742929220199585, + -0.036404818296432495, + -0.01711226999759674, + 0.09378434717655182, + -2.0185699462890625, + -0.22933721542358398, + 0.13657210767269135, + -0.010317999869585037, + 0.1696648895740509, + 0.01408969983458519, + 0.18841473758220673, + -1.9807454347610474, + -0.1428220421075821, + 0.006691664457321167, + 0.09006409347057343, + 0.2312215268611908, + -0.051600679755210876, + -0.07336241006851196, + -1.722533941268921, + -0.19763889908790588 + ], + [ + -0.021733596920967102, + -0.022700006142258644, + -0.09785762429237366, + -0.0392765998840332, + 0.02021634578704834, + -1.9918321371078491, + -0.29967331886291504, + 0.15010276436805725, + -0.006304899230599403, + 0.15463489294052124, + -0.00865880399942398, + 0.0901183933019638, + -2.0479912757873535, + -0.11742065846920013, + -0.005247395485639572, + 0.11192092299461365, + 0.2463504821062088, + 0.038363367319107056, + -0.07147014141082764, + -1.624755859375, + -0.32979658246040344 + ], + [ + -0.08303648233413696, + 0.005722474306821823, + 0.02876332402229309, + -0.063916876912117, + 0.09928908944129944, + -2.1052942276000977, + -0.22953367233276367, + 0.1969548612833023, + -0.052117086946964264, + 0.1749492585659027, + 0.05964469164609909, + 0.15526777505874634, + -1.8762308359146118, + 0.0742640495300293, + 0.003120254725217819, + 0.08843435347080231, + 0.21593104302883148, + 0.009374931454658508, + -0.040671706199645996, + -1.5572540760040283, + -0.12304079532623291 + ], + [ + 0.06068043410778046, + -0.07272287458181381, + 0.016918838024139404, + -0.23076801002025604, + 0.06407099962234497, + -2.143833875656128, + -0.17938441038131714, + 0.23395749926567078, + -0.08170146495103836, + -0.0024092718958854675, + 0.03844539448618889, + 0.12899602949619293, + -2.111145257949829, + -0.1777956336736679, + -0.059261541813611984, + 0.09445318579673767, + 0.30387645959854126, + -0.05118764936923981, + 0.024333566427230835, + -1.648310661315918, + -0.1620248556137085 + ], + [ + 0.18324795365333557, + -0.05526398494839668, + 0.017617106437683105, + -0.2363443225622177, + 0.20967961847782135, + -1.349843978881836, + -0.7425796389579773, + -0.05657553672790527, + -0.0720028430223465, + 0.012850567698478699, + 0.15367190539836884, + 0.1685241460800171, + -1.4467493295669556, + -0.41098105907440186, + 0.024837851524353027, + 0.1823609322309494, + 0.1010756641626358, + -0.011001542210578918, + -0.14043891429901123, + -1.534637451171875, + -0.16784745454788208 + ] + ], + "generated_utilities": [ + 0.13187798857688904, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.13499042391777039, + 0.16052719950675964 + ], + "group_id": "ms-6c8d06bace98a28a8e4e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8583254218101501, + 2.0, + 0.8125, + 0.12893107533454895, + 0.8582976460456848, + 0.15346482396125793, + 0.18961156904697418, + 0.13901273906230927, + 0.10113435983657837, + 0.11161506175994873, + 0.09923169016838074, + 0.16662254929542542, + 0.04211682081222534, + 0.024710848927497864, + 0.05722464621067047, + 0.8584086298942566 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.10696407407522202, + 0.3957799971103668, + 0.15132969617843628, + -0.32772913575172424, + 0.005261177197098732, + -2.2650365829467773, + 0.0, + 0.3958877623081207, + -0.5120954513549805, + -0.4317946135997772, + -0.3970779776573181, + 0.5317854285240173, + -1.8659565448760986, + 0.0, + 0.1198807880282402, + -0.11760073900222778, + -0.6048114895820618, + 0.15180528163909912, + -0.532695472240448, + -1.6115882396697998, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.120756149291992, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.070481538772583, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0495758056640625 + ], + [ + 0.30105674266815186, + 0.4776385426521301, + -0.22267001867294312, + -0.13300693035125732, + -0.11095686256885529, + -3.138023853302002, + -0.793931245803833, + 0.16969743371009827, + -0.16339385509490967, + 0.02508675307035446, + 0.654399573802948, + -0.22830897569656372, + -3.1443004608154297, + -1.4291013479232788, + 0.005021120421588421, + 0.005560946650803089, + -0.1901366114616394, + 1.0211858749389648, + -0.09353887289762497, + -2.3457469940185547, + -1.406853199005127 + ], + [ + 0.4160586893558502, + 0.13316196203231812, + -0.21056225895881653, + -0.25006812810897827, + 0.060951750725507736, + -2.8735690116882324, + -0.9062713980674744, + -0.28707224130630493, + -0.2396206259727478, + -0.19859735667705536, + 0.29039445519447327, + -0.24077445268630981, + -2.644789934158325, + -1.8330551385879517, + -0.23438936471939087, + 0.04620177671313286, + 0.1472887098789215, + 0.9629149436950684, + -0.020509399473667145, + -2.7985036373138428, + -1.2463163137435913 + ], + [ + 0.2486386001110077, + -0.022333532571792603, + 0.14628465473651886, + -0.8381847143173218, + 0.21282373368740082, + -3.585387945175171, + -1.2156991958618164, + -0.08670558780431747, + -0.3178575336933136, + 0.4322021007537842, + 0.6421236395835876, + -0.33922863006591797, + -3.004714012145996, + -1.19637131690979, + -0.015640778467059135, + 0.10077434033155441, + -0.449941486120224, + 1.0431067943572998, + 0.2899783253669739, + -2.500624179840088, + -1.6148788928985596 + ], + [ + 0.46770766377449036, + 0.03902164101600647, + 0.2556130588054657, + -0.45893940329551697, + 0.08939866721630096, + -3.163339853286743, + -0.7599828243255615, + 0.3327450156211853, + 0.3252364993095398, + 0.3067358434200287, + 1.1394517421722412, + 0.472858726978302, + -3.1504058837890625, + -1.9780522584915161, + -0.26712900400161743, + -0.262444406747818, + 0.20269782841205597, + 1.336982250213623, + -0.06957009434700012, + -2.0651021003723145, + -1.8484716415405273 + ], + [ + 0.48348987102508545, + 0.19676931202411652, + -0.20689666271209717, + -0.21521174907684326, + -0.5048230886459351, + -3.491732358932495, + -0.7390058636665344, + 0.18321916460990906, + -0.1509065330028534, + 0.27166980504989624, + 1.004181981086731, + -0.5531666874885559, + -2.531594753265381, + -2.0243897438049316, + -0.44593361020088196, + 0.49786314368247986, + -0.006632372736930847, + 0.24423635005950928, + -0.6331305503845215, + -2.7707719802856445, + -2.1364808082580566 + ], + [ + -0.5684993863105774, + 0.0623701810836792, + -0.10449839383363724, + -0.7858190536499023, + -0.057080868631601334, + -3.981018304824829, + -0.6721940636634827, + 0.10191962122917175, + -0.7216448783874512, + 0.3870577812194824, + 0.5508396625518799, + -0.6811783313751221, + -3.6131227016448975, + -1.7484180927276611, + 0.34220433235168457, + -0.16193144023418427, + 0.16876991093158722, + 0.6509957909584045, + 0.2548854351043701, + -2.0779964923858643, + -1.5782216787338257 + ], + [ + -0.1224658265709877, + 0.22953394055366516, + -0.41326335072517395, + 0.19802969694137573, + 0.6866055130958557, + -2.6415462493896484, + -1.049781322479248, + -0.6788480281829834, + -0.8187242746353149, + 0.6141759157180786, + 1.3649951219558716, + -0.36378854513168335, + -2.2142889499664307, + -1.3759111166000366, + 0.583359956741333, + 0.6860965490341187, + 0.8285356760025024, + 1.4575884342193604, + -0.4101440906524658, + -2.218319892883301, + -1.5390111207962036 + ], + [ + 0.3368571698665619, + 0.4528505206108093, + -0.2985517382621765, + 0.37766069173812866, + 0.2064778208732605, + -2.8070714473724365, + -1.5598305463790894, + -0.9032005071640015, + -0.48420336842536926, + 0.639835000038147, + -0.3801422119140625, + -0.1376074254512787, + -3.369077205657959, + -1.9231767654418945, + -0.16211053729057312, + 0.53387051820755, + 0.6583098769187927, + 0.7611914873123169, + -0.46404945850372314, + -2.7554476261138916, + -1.4307246208190918 + ], + [ + 0.6002402901649475, + -0.14213827252388, + 0.30183425545692444, + -0.20118463039398193, + -0.16981711983680725, + -3.4212875366210938, + -0.2252577543258667, + 0.34123149514198303, + 0.003176456317305565, + 0.7433648109436035, + 1.1403348445892334, + 0.5709706544876099, + -1.8894051313400269, + -0.7200812101364136, + -0.7235198020935059, + 0.2147149294614792, + -0.28744715452194214, + 1.0323235988616943, + 0.5815067291259766, + -2.582993984222412, + -1.2160475254058838 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.019069351255893707, + 0.012574762105941772, + 0.07413499802350998, + -0.04097828269004822, + -0.03991875424981117, + -2.2650365829467773, + 0.0, + -0.06082560494542122, + 0.05821063742041588, + 0.07896332442760468, + -0.037061095237731934, + -0.08879614621400833, + -1.8659565448760986, + 0.0, + -0.022465867921710014, + -0.026843247935175896, + 0.04789978265762329, + -0.019972383975982666, + -0.04040021449327469, + -1.6115882396697998, + 0.0 + ] + ], + "predicted_scores": [ + 4.905849456787109, + 4.465726375579834, + 4.1869683265686035, + 3.873480796813965, + 3.8334813117980957, + 3.511204957962036, + 3.3798365592956543, + 3.3796608448028564 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1665739511/train2", + "selected_index": 0, + "state_hash": "0f77663511063f17b0962024501b863c62b881513459d8bfa3ac00bbaed67de4", + "stored_base_utility": 0.8583254218101501, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11944694817066193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11944694817066193 + }, + { + "contact_quality": null, + "progress": 0.16878563165664673, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16878563165664673 + }, + { + "contact_quality": null, + "progress": 0.17180444300174713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17180444300174713 + }, + { + "contact_quality": null, + "progress": 0.17166458070278168, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17166458070278168 + }, + { + "contact_quality": null, + "progress": 0.19043110311031342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19043110311031342 + }, + { + "contact_quality": null, + "progress": 0.1885138899087906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1885138899087906 + }, + { + "contact_quality": null, + "progress": 0.19852638244628906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19852638244628906 + }, + { + "contact_quality": null, + "progress": 0.1698688566684723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1698688566684723 + } + ], + "candidate_progress": [ + 0.11944694817066193, + 0.16878563165664673, + 0.17180444300174713, + 0.17166458070278168, + 0.19043110311031342, + 0.1885138899087906, + 0.19852638244628906, + 0.1698688566684723 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-72777024349c2aacd5e0", + "generated_tangents": [ + [ + 0.46157407760620117, + 0.174520805478096, + -0.03888651728630066, + -0.5337043404579163, + 0.5942518711090088, + -0.2038794755935669, + 0.9634844064712524, + -0.15029233694076538, + 0.050802119076251984, + -0.046997345983982086, + 0.08813080191612244, + -0.10318443179130554, + -1.1648684740066528, + -0.15636833012104034, + -0.06540808081626892, + 0.2142813801765442, + 0.36294233798980713, + -0.014899656176567078, + -0.2761247158050537, + -0.8063332438468933, + -0.4218128025531769 + ], + [ + 0.1354047954082489, + 0.1665385216474533, + -0.09189078211784363, + -0.5567952394485474, + 0.4742484986782074, + -0.36955758929252625, + -0.047621965408325195, + -0.15154656767845154, + 0.0809565931558609, + 0.06719428300857544, + 0.19103571772575378, + 0.06663753092288971, + -1.800724744796753, + -0.04190605878829956, + -0.1072264164686203, + 0.0981844812631607, + 0.07242068648338318, + -0.10282446444034576, + -0.16536816954612732, + -1.1918113231658936, + 0.11890645325183868 + ], + [ + -0.029327720403671265, + 0.1477586179971695, + -0.07216331362724304, + -0.41563108563423157, + 0.41079404950141907, + -0.08326615393161774, + -0.04381251335144043, + 0.024007931351661682, + 0.09756569564342499, + 0.0832497775554657, + 0.24985820055007935, + 0.06584241986274719, + -1.7538155317306519, + 0.1450558453798294, + -0.03608652576804161, + 0.03242557495832443, + 0.061501696705818176, + -0.04694490134716034, + -0.12731868028640747, + -1.2788195610046387, + 0.11673453450202942 + ], + [ + 0.12438970059156418, + 0.14528720080852509, + -0.12435182929039001, + -0.5101529359817505, + 0.47771918773651123, + -0.32751479744911194, + -0.1798877716064453, + -0.0757099986076355, + 0.061697252094745636, + 0.07973569631576538, + 0.2250921130180359, + -0.01808471977710724, + -1.8440006971359253, + -0.08621540665626526, + -0.2479267716407776, + 0.07480540871620178, + 0.03405578434467316, + -0.07880456745624542, + -0.061933428049087524, + -1.2551229000091553, + 0.19224265217781067 + ], + [ + 0.01854190230369568, + 0.2946150302886963, + -0.3474480211734772, + -0.3782707452774048, + 0.42811042070388794, + -0.283975750207901, + 0.007112085819244385, + -0.08178913593292236, + -0.05934348702430725, + -0.011700280010700226, + 0.24963107705116272, + -0.017295151948928833, + -1.9101580381393433, + -0.0944548100233078, + -0.21573540568351746, + 0.0623256154358387, + 0.23162756860256195, + -0.27955859899520874, + -0.11699864268302917, + -1.5937581062316895, + 0.09011290222406387 + ], + [ + -0.028540179133415222, + 0.24281413853168488, + -0.19422301650047302, + -0.5394165515899658, + 0.5835269689559937, + 0.06668397784233093, + -0.047837793827056885, + -0.0931469202041626, + -0.06707969307899475, + -0.021994300186634064, + 0.17776334285736084, + -0.17167231440544128, + -2.0242831707000732, + 0.004569120705127716, + -0.06618009507656097, + 0.0867065042257309, + 0.32150667905807495, + -0.2761147618293762, + -0.2576178312301636, + -1.239419937133789, + -0.07899683713912964 + ], + [ + -0.03803916275501251, + 0.2134903520345688, + -0.3332861363887787, + -0.5010786056518555, + 0.48019707202911377, + -0.2275649607181549, + -0.011795222759246826, + -0.053437620401382446, + -0.04079996794462204, + 0.050588078796863556, + 0.16244184970855713, + -0.09413272142410278, + -1.840475082397461, + -0.056180715560913086, + -0.17065294086933136, + 0.08698076009750366, + 0.2513794004917145, + -0.3058536648750305, + -0.34668129682540894, + -1.406508445739746, + -0.022539496421813965 + ], + [ + -0.01849667727947235, + 0.21552197635173798, + -0.4834817945957184, + -0.5902851223945618, + 0.3741875886917114, + -0.27450865507125854, + 0.10583284497261047, + 0.1401326060295105, + -0.05798898637294769, + 0.10467419773340225, + 0.11828957498073578, + -0.26767292618751526, + -1.872161865234375, + -0.09646837413311005, + -0.18259958922863007, + 0.17331455647945404, + 0.345462441444397, + -0.1606987565755844, + -0.43198955059051514, + -1.382481336593628, + 0.18469150364398956 + ] + ], + "generated_utilities": [ + 0.11944694817066193, + 0.16878563165664673, + 0.17180444300174713, + 0.17166458070278168, + 0.19043110311031342, + 0.1885138899087906, + 0.19852638244628906, + 0.1698688566684723 + ], + "group_id": "ms-72777024349c2aacd5e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8314920663833618, + 0.1595238745212555, + 2.0, + 0.16971935331821442, + 0.5892879366874695, + 0.1617850512266159, + 0.1877371072769165, + 0.5782577395439148, + 0.12074722349643707, + 0.11237785220146179, + 0.08792310953140259, + 0.1209893673658371, + 0.13739405572414398, + 0.0773405134677887, + 0.09497407078742981, + 0.047447383403778076 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.002835560590028763, + 0.3854118585586548, + -0.06168194115161896, + -0.16570186614990234, + -0.05051702260971069, + -0.43052077293395996, + 0.0, + 0.024141468107700348, + -0.011737043969333172, + -0.05502588301897049, + 0.013536334037780762, + -0.000911470502614975, + -1.6284780502319336, + 0.0, + 0.030459173023700714, + -0.05656937509775162, + 0.03393230587244034, + 0.06894856691360474, + 0.00450655072927475, + -1.2762084007263184, + 0.0 + ], + [ + 0.2191334217786789, + 0.3854118585586548, + 0.5048428177833557, + -0.16570186614990234, + -0.20026534795761108, + -0.43052077293395996, + 0.0, + -0.026082046329975128, + 0.03163425251841545, + -0.04950854927301407, + 0.1969367265701294, + -0.05762401595711708, + -1.6284780502319336, + 0.0, + 0.002695085946470499, + 0.34764212369918823, + 0.6051180362701416, + 0.9295560717582703, + 0.38285934925079346, + -1.2762084007263184, + 0.0 + ], + [ + -0.04624907672405243, + 1.3854118585586548, + 0.05864633619785309, + -1.1657018661499023, + 0.7997346520423889, + -1.43052077293396, + 1.1980153322219849, + -0.023269198834896088, + -0.026228107511997223, + -0.009256521239876747, + 0.6858966946601868, + 0.03280947729945183, + -2.6284780502319336, + -1.7920862436294556, + -0.0004090145230293274, + -0.00536443293094635, + 0.007614931557327509, + 0.7401981949806213, + 0.11622340977191925, + -2.2762084007263184, + -1.6813035011291504 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.3960306644439697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.584172487258911, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.362607002258301 + ], + [ + -0.13329119980335236, + 1.1171538829803467, + -0.19378040730953217, + -0.9341158866882324, + 0.8714147806167603, + -1.659934163093567, + 1.1658833026885986, + 0.23766514658927917, + 0.21810689568519592, + -0.05730185657739639, + 0.4946695566177368, + 0.17369553446769714, + -2.559515953063965, + -1.5605767965316772, + 0.08475297689437866, + 0.22485606372356415, + -0.2591644823551178, + 0.9637633562088013, + -0.08890970051288605, + -2.1015453338623047, + -1.5709789991378784 + ], + [ + -0.3507236838340759, + 1.1737966537475586, + 0.10341192036867142, + -0.7761039137840271, + 0.47493064403533936, + -1.2297213077545166, + 1.4181653261184692, + -0.1981543004512787, + -0.18248644471168518, + 0.17768341302871704, + 0.843591034412384, + -0.07443197071552277, + -2.6062240600585938, + -1.750958800315857, + -0.15503963828086853, + 0.18218432366847992, + 0.002181168645620346, + 0.8641887903213501, + -0.026101917028427124, + -1.9398812055587769, + -1.8561092615127563 + ], + [ + -0.22363319993019104, + 1.533349871635437, + 0.4454386830329895, + -1.5271000862121582, + 0.8945828080177307, + -1.105146884918213, + 0.7171626687049866, + -0.20207461714744568, + 0.31458696722984314, + 0.20214562118053436, + 1.0593571662902832, + -0.4655032455921173, + -2.8809642791748047, + -1.580764651298523, + -0.22098299860954285, + -0.08731096982955933, + -0.15165574848651886, + 0.89030921459198, + 0.611029326915741, + -2.2792654037475586, + -1.798964262008667 + ], + [ + -0.3702698349952698, + 0.9771865606307983, + 0.27564477920532227, + -1.7643829584121704, + 0.7298657894134521, + -1.875503420829773, + 1.4598875045776367, + 0.47600361704826355, + 0.13439032435417175, + -0.23905222117900848, + 1.2192013263702393, + -0.44705769419670105, + -2.603621482849121, + -1.3898506164550781, + 0.16782721877098083, + -0.2559890151023865, + 0.2965841293334961, + 0.8750308752059937, + 0.1521819680929184, + -2.3417282104492188, + -2.2663588523864746 + ], + [ + -0.31705379486083984, + 0.866703450679779, + -0.09125347435474396, + -1.5959886312484741, + 0.8041162490844727, + -1.7037310600280762, + 1.6304850578308105, + 0.11580469459295273, + 0.2654762864112854, + 0.12200001627206802, + 0.6904033422470093, + 0.19859254360198975, + -2.7615463733673096, + -2.059410333633423, + 0.5591834783554077, + -0.4801292419433594, + -0.3343847393989563, + 0.8787070512771606, + -0.14748375117778778, + -2.0334713459014893, + -1.1611082553863525 + ], + [ + -0.11324095726013184, + 0.8777624368667603, + 0.8215147256851196, + -1.5343353748321533, + 0.881126344203949, + -1.0607860088348389, + 0.5398488640785217, + 0.6020828485488892, + -0.5065795183181763, + 0.02798180840909481, + 0.33880186080932617, + 0.320717990398407, + -3.3552045822143555, + -1.1435439586639404, + 0.19279444217681885, + -0.5245973467826843, + 0.30647164583206177, + 1.3726601600646973, + 0.3123217523097992, + -2.9675674438476562, + -1.7007204294204712 + ], + [ + -0.6432619094848633, + 0.79607754945755, + -0.49864423274993896, + -1.0710605382919312, + 1.0709084272384644, + -1.612175703048706, + 1.2555015087127686, + 0.2873077392578125, + -0.36858057975769043, + 0.05837755650281906, + 1.1025807857513428, + 0.7949575185775757, + -3.2468795776367188, + -2.556741237640381, + -0.6579093337059021, + -0.8874800801277161, + 0.7922345399856567, + 0.4745541512966156, + 0.07610362768173218, + -2.041800022125244, + -2.2069954872131348 + ], + [ + -0.9264093637466431, + 1.9828436374664307, + 0.7907096147537231, + -1.260971188545227, + 1.7548933029174805, + -2.2685132026672363, + 1.1821955442428589, + -0.49968039989471436, + 0.10982713848352432, + 0.2384604662656784, + 0.2075445055961609, + -0.7872775793075562, + -2.076058864593506, + -1.1888585090637207, + 0.7447717785835266, + 0.2355569452047348, + -0.3163020610809326, + 0.3282558023929596, + -0.4075327515602112, + -3.2451019287109375, + -2.4086263179779053 + ], + [ + -0.7190575003623962, + 1.3702023029327393, + 0.8307373523712158, + -0.7533494234085083, + 0.574251651763916, + -2.3376822471618652, + 0.5512753129005432, + -0.04391670227050781, + 0.2242431342601776, + -0.46594029664993286, + 0.5667901039123535, + -0.12832416594028473, + -2.9870448112487793, + -1.2032369375228882, + 0.17313259840011597, + 0.5932936668395996, + 0.7428228259086609, + 0.6570835113525391, + -0.1891915649175644, + -1.3703107833862305, + -1.7630409002304077 + ], + [ + 0.799586832523346, + 1.0164330005645752, + -0.49519193172454834, + -0.6983597278594971, + 1.7632484436035156, + -0.9710203409194946, + 1.5416830778121948, + 0.9653489589691162, + 0.06498706340789795, + 0.2132291942834854, + -0.1650814414024353, + 0.06786654889583588, + -1.8782389163970947, + -2.3576126098632812, + 0.9410911798477173, + -0.576595664024353, + -0.8031080365180969, + 0.7120062112808228, + 0.6479252576828003, + -2.3928208351135254, + -1.4952012300491333 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.11725310236215591, + 0.3854118585586548, + 0.06966311484575272, + -0.16570186614990234, + 0.06442362070083618, + -0.43052077293395996, + 0.0, + 0.11858775466680527, + 0.0573427751660347, + -0.14938217401504517, + -0.01657569408416748, + 0.060478124767541885, + -1.6284780502319336, + 0.0, + 0.016584808006882668, + 0.18205130100250244, + -0.13064442574977875, + 0.15621143579483032, + 0.056775350123643875, + -1.2762084007263184, + 0.0 + ] + ], + "predicted_scores": [ + 7.788398742675781, + 6.110999584197998, + 5.820715427398682, + 4.917744159698486, + 4.872241497039795, + 4.84718656539917, + 4.464103698730469, + 3.651381731033325 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "256770874/train2", + "selected_index": 0, + "state_hash": "3a7aa4ba27a5b34cdeb0f31ce9818588b1dd4ede96bf546f07302616c2e9b3a6", + "stored_base_utility": 0.8314920663833618, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20372186601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20372186601161957 + }, + "base_progress": 0.20372186601161957, + "base_success": false, + "base_utility": 0.20372186601161957, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1758389174938202, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1758389174938202 + }, + { + "contact_quality": null, + "progress": 0.18668952584266663, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18668952584266663 + }, + { + "contact_quality": null, + "progress": 0.2069244086742401, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2069244086742401 + }, + { + "contact_quality": null, + "progress": 0.19020694494247437, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19020694494247437 + }, + { + "contact_quality": null, + "progress": 0.22462211549282074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22462211549282074 + }, + { + "contact_quality": null, + "progress": 0.22120237350463867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22120237350463867 + }, + { + "contact_quality": null, + "progress": 0.5639582276344299, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5639582276344299 + }, + { + "contact_quality": null, + "progress": 0.22967948019504547, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22967948019504547 + } + ], + "candidate_progress": [ + 0.1758389174938202, + 0.18668952584266663, + 0.2069244086742401, + 0.19020694494247437, + 0.22462211549282074, + 0.22120237350463867, + 0.5639582276344299, + 0.22967948019504547 + ], + "candidate_source_chart_ids": [ + "ms-5bddb39f792f20e5e6d4", + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "generated_tangents": [ + [ + -0.11020086705684662, + -0.013108022511005402, + -0.1264011561870575, + 0.35455426573753357, + 0.23730328679084778, + -0.48435285687446594, + 0.07835516333580017, + 0.052899062633514404, + 0.04665840417146683, + 0.06724787503480911, + 0.0012043118476867676, + 0.2893897294998169, + -1.6423239707946777, + -0.009604468941688538, + 0.1118192970752716, + 0.16256406903266907, + 0.041011497378349304, + -0.21209131181240082, + -0.019342273473739624, + -1.4293444156646729, + -0.02278698980808258 + ], + [ + -0.026120781898498535, + 0.018893331289291382, + -0.14871594309806824, + 0.261665016412735, + 0.1527482122182846, + -0.435959130525589, + -0.11187863349914551, + 0.06613901257514954, + 0.03334052115678787, + 0.012848034501075745, + 0.029305411502718925, + 0.21651701629161835, + -1.7776308059692383, + 0.0027957111597061157, + 0.08954580128192902, + 0.12566040456295013, + 0.06295119225978851, + -0.25235462188720703, + 0.006839275360107422, + -1.576141119003296, + 0.08545006811618805 + ], + [ + -0.0192459374666214, + 0.06497050076723099, + -0.22957196831703186, + 0.22761625051498413, + 0.20351259410381317, + -0.4425376355648041, + -0.052713632583618164, + 0.06681163609027863, + 0.008113421499729156, + -0.0003314986824989319, + -0.07100499421358109, + 0.22354894876480103, + -1.7586311101913452, + 0.06674952805042267, + 0.06904176622629166, + 0.09685906767845154, + 0.05805554986000061, + -0.18940134346485138, + 0.027307987213134766, + -1.6631107330322266, + 0.06823643296957016 + ], + [ + 0.023560263216495514, + 0.009691145271062851, + -0.14869794249534607, + 0.30883529782295227, + 0.17246560752391815, + -0.6573750972747803, + -0.04758906364440918, + 0.052341386675834656, + 0.046923957765102386, + 0.024762749671936035, + 0.006820760667324066, + 0.10632195323705673, + -1.573324203491211, + -0.037993401288986206, + 0.07132565230131149, + 0.11828601360321045, + -0.0035343170166015625, + -0.21954290568828583, + 0.066749706864357, + -1.4483566284179688, + 0.05538436025381088 + ], + [ + -0.0713127851486206, + -0.08605244755744934, + -0.4092229902744293, + 0.3931208550930023, + 0.19602271914482117, + -0.673324704170227, + 0.12838059663772583, + 0.15265193581581116, + -0.02485211379826069, + -0.03920639306306839, + -0.12418157607316971, + 0.06869977712631226, + -1.7395849227905273, + -0.03127090632915497, + -0.02653767541050911, + 0.16294054687023163, + 0.16098234057426453, + -0.2561621069908142, + -0.15497267246246338, + -1.359914779663086, + -0.105194091796875 + ], + [ + -0.08782057464122772, + -0.08472000062465668, + -0.31380751729011536, + 0.5326024293899536, + 0.19600187242031097, + -0.5924414396286011, + -0.07720458507537842, + 0.189269557595253, + 0.03365449607372284, + -0.01644543558359146, + -0.12385163456201553, + 0.04005500674247742, + -2.000369071960449, + -0.11754660308361053, + 0.07276436686515808, + 0.21560324728488922, + 0.12088972330093384, + -0.3859550356864929, + -0.07491222023963928, + -1.3207519054412842, + 0.11474438011646271 + ], + [ + -0.1525118201971054, + 0.03449248522520065, + -0.14602509140968323, + 0.22883886098861694, + 0.17169666290283203, + -0.24276667833328247, + -0.21707427501678467, + 0.19013364613056183, + -0.031076611950993538, + -0.0734015628695488, + -0.2084907591342926, + 0.11446955054998398, + -1.7619656324386597, + -0.02972029149532318, + 0.14165401458740234, + 0.11926732957363129, + 0.02618981897830963, + -0.23360036313533783, + -0.09116426110267639, + -1.7336742877960205, + 0.12123022228479385 + ], + [ + -0.09768658876419067, + 0.016946300864219666, + -0.1720522940158844, + 0.2824391722679138, + 0.2329549938440323, + -0.23956191539764404, + -0.1392601728439331, + 0.22499649226665497, + -0.019799647852778435, + -0.02801615744829178, + 0.004370331764221191, + 0.08938870579004288, + -1.7242966890335083, + 0.11016328632831573, + -0.05263916775584221, + 0.12067274749279022, + 0.06463602185249329, + -0.23037971556186676, + -0.08607134222984314, + -1.4778838157653809, + 0.0240279883146286 + ] + ], + "generated_utilities": [ + 0.1758389174938202, + 0.18668952584266663, + 0.2069244086742401, + 0.19020694494247437, + 0.22462211549282074, + 0.22120237350463867, + 0.5639582276344299, + 0.22967948019504547 + ], + "group_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8363267779350281, + 2.0, + 2.0, + 0.16206762194633484, + 0.5390529036521912, + 0.08151453733444214, + 0.5261932611465454, + 0.5265869498252869, + 0.5459305047988892, + 0.18360589444637299, + 0.16689825057983398, + 0.15426510572433472, + 0.0028923004865646362, + 0.15516109764575958, + 0.0506085604429245, + 0.19578850269317627 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.0005862116813659668, + 0.18783891201019287, + -0.02734696865081787, + 0.09201061725616455, + 0.010171383619308472, + -0.4856022596359253, + 0.0, + 0.10349468141794205, + 0.4567214846611023, + -0.1893925815820694, + 0.12298360466957092, + 0.3369159996509552, + -1.6010875701904297, + 0.0, + 0.304210901260376, + -0.3049007058143616, + 0.3435661494731903, + 0.3158445656299591, + 0.2931552231311798, + -1.2709691524505615, + 0.0 + ], + [ + -0.726529061794281, + -0.8121610879898071, + -1.0273469686508179, + 1.0920106172561646, + 0.3926190733909607, + -1.4856022596359253, + 1.8979665040969849, + -0.029249273240566254, + 0.008005961775779724, + 0.013417234644293785, + -0.10860864818096161, + 0.43491366505622864, + -2.6010875701904297, + -1.493168592453003, + -0.01047949492931366, + -0.0018908085767179728, + 0.019280990585684776, + 0.20726272463798523, + 0.45750224590301514, + -2.2709691524505615, + -1.4645930528640747 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7959330081939697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.986337184906006, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9291861057281494 + ], + [ + -0.9019005298614502, + -0.5901641845703125, + -0.920559287071228, + 0.991828978061676, + 0.2407551407814026, + -1.2655587196350098, + 1.7652356624603271, + 0.03909382224082947, + 0.04142814502120018, + 0.12125641852617264, + -0.2822783887386322, + 0.4936577081680298, + -2.5754144191741943, + -1.5686157941818237, + -0.22672900557518005, + -0.08431098610162735, + 0.09805622696876526, + 0.2479260265827179, + 0.4450404644012451, + -2.1972453594207764, + -1.4420135021209717 + ], + [ + -0.8822115063667297, + -0.9738405346870422, + -0.7449414730072021, + 0.8784142732620239, + 0.4583987295627594, + -1.3989503383636475, + 1.8331178426742554, + -0.006558071821928024, + 0.30383312702178955, + 0.2117801308631897, + -0.2538969814777374, + 0.13478082418441772, + -2.584805727005005, + -1.325882911682129, + 0.23593956232070923, + -0.2618964612483978, + -0.30780693888664246, + 0.24083535373210907, + 0.430472195148468, + -1.946203589439392, + -1.5077860355377197 + ], + [ + -0.39491432905197144, + -0.8895903825759888, + -0.9160035848617554, + 0.7220491170883179, + 0.36837494373321533, + -1.4429160356521606, + 1.4078493118286133, + 0.03969770669937134, + 0.007204015273600817, + 0.056278251111507416, + -0.4780465364456177, + 0.9121701717376709, + -2.512721538543701, + -1.6225675344467163, + -0.42074650526046753, + 0.15076839923858643, + -0.24544812738895416, + 0.08152849972248077, + 0.1987975835800171, + -2.712414264678955, + -1.9573136568069458 + ], + [ + -0.6879937648773193, + -0.8859440088272095, + -1.1152405738830566, + 0.8750674724578857, + -0.05955067276954651, + -1.5194538831710815, + 1.876329779624939, + -0.2068542242050171, + -0.5498799085617065, + 0.46419233083724976, + 0.42569708824157715, + -0.02127784490585327, + -2.5905418395996094, + -1.411050796508789, + -0.21327723562717438, + 0.20716798305511475, + 0.20993508398532867, + 0.33476364612579346, + -0.09872657060623169, + -2.5958731174468994, + -1.6374839544296265 + ], + [ + -0.5723236799240112, + -0.28514552116394043, + -0.6157487630844116, + 0.6151646375656128, + 0.32639649510383606, + -1.7217803001403809, + 2.5557403564453125, + 0.532052218914032, + 0.5977330207824707, + -0.5707970261573792, + -0.21707558631896973, + 0.8829224109649658, + -3.1287553310394287, + -1.0988328456878662, + 0.38229066133499146, + 0.0032835237216204405, + -0.3569248616695404, + 0.0031215548515319824, + 1.1060712337493896, + -2.3536932468414307, + -1.383341670036316 + ], + [ + -0.7901747822761536, + -0.8977189660072327, + -1.4548863172531128, + 0.5700533986091614, + -0.13018959760665894, + -0.8057670593261719, + 2.4019269943237305, + 0.5368256568908691, + 0.5227565765380859, + 0.28199371695518494, + -0.1637011021375656, + -0.2898564040660858, + -2.9142980575561523, + -1.8492696285247803, + 0.7177971601486206, + 0.2360217124223709, + -0.13862104713916779, + 0.9262527227401733, + 0.39495596289634705, + -2.470649480819702, + -1.3260751962661743 + ], + [ + -0.6053685545921326, + -0.3819047510623932, + -0.258634090423584, + 0.19338607788085938, + 0.19874510169029236, + -1.3407407999038696, + 2.5238022804260254, + -0.6922282576560974, + 0.8137098550796509, + 0.39421388506889343, + 0.09789760410785675, + 0.12572914361953735, + -2.518153667449951, + -1.6892600059509277, + 0.503250002861023, + 0.88390052318573, + -0.7520112991333008, + -0.3356115520000458, + 0.15291047096252441, + -2.5788943767547607, + -0.9403089284896851 + ], + [ + -0.464814156293869, + -0.6034223437309265, + -1.6505372524261475, + 0.6883972883224487, + -0.3758969306945801, + -1.785051941871643, + 2.605452060699463, + 0.550828218460083, + 0.4742209315299988, + -0.19333000481128693, + -1.002321481704712, + 0.48782211542129517, + -1.8015108108520508, + -0.8254865407943726, + -0.7333722710609436, + -0.4948355555534363, + 0.6016485095024109, + 0.5632485151290894, + -0.24367755651474, + -2.201673984527588, + -0.5806979537010193 + ], + [ + -1.4628195762634277, + -0.4433446228504181, + -1.9926375150680542, + 0.33977705240249634, + 0.03379333019256592, + -1.6042804718017578, + 2.221484422683716, + 0.7110810279846191, + -0.41500210762023926, + 0.08095695078372955, + -0.23683495819568634, + -0.43207624554634094, + -3.080759286880493, + -0.9071659445762634, + 0.47082334756851196, + -0.9117310047149658, + -0.06435621529817581, + -0.6813669204711914, + -0.2741369605064392, + -2.3763599395751953, + -1.000203013420105 + ], + [ + -1.382751226425171, + -1.7026821374893188, + -0.11846715211868286, + 1.5398613214492798, + -0.5885447859764099, + -1.8825289011001587, + 1.5456308126449585, + -0.320659339427948, + -0.964028000831604, + -0.1812019646167755, + 0.14226128160953522, + 0.4527861177921295, + -2.714468002319336, + -1.9451607465744019, + 0.195057213306427, + 0.16905252635478973, + -0.9247990250587463, + 0.9112424850463867, + 0.79449862241745, + -2.714893341064453, + -1.6473028659820557 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.02016383409500122, + 0.02509528398513794, + -0.02734696865081787, + 0.09201061725616455, + -0.04331648349761963, + -0.4856022596359253, + 0.0, + -0.022040890529751778, + -0.003832761198282242, + -0.01408458687365055, + 0.06165061891078949, + 0.02554956078529358, + -1.6010875701904297, + 0.0, + 0.009550465270876884, + 0.03566932678222656, + 0.013732164166867733, + 0.017749279737472534, + 0.0699148178100586, + -1.2709691524505615, + 0.0 + ], + [ + 0.023195505142211914, + 0.029890060424804688, + -0.02734696865081787, + 0.09201061725616455, + 0.06108534336090088, + -0.4856022596359253, + 0.0, + -0.034551952034235, + -0.15152660012245178, + -0.21811580657958984, + -0.063543401658535, + 0.23667243123054504, + -1.6010875701904297, + 0.0, + -0.1614403873682022, + -0.17222382128238678, + -0.09706380218267441, + 0.3660350441932678, + 0.13013160228729248, + -1.2709691524505615, + 0.0 + ] + ], + "predicted_scores": [ + 6.827688694000244, + 6.575668811798096, + 6.572710990905762, + 6.568657398223877, + 6.127616882324219, + 5.754226207733154, + 5.634440898895264, + 5.44002628326416 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "957101489/train2", + "selected_index": 0, + "state_hash": "e8473553a2b17f5adf09e08e7271b5502d53a51225c17abac3d9f78906cf8c46", + "stored_base_utility": 0.8363267779350281, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8431589603424072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8431589603424072 + }, + "base_progress": 0.8431589603424072, + "base_success": false, + "base_utility": 0.8431589603424072, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.21672730147838593, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21672730147838593 + }, + { + "contact_quality": null, + "progress": 0.1849215030670166, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1849215030670166 + }, + { + "contact_quality": null, + "progress": 0.21649955213069916, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21649955213069916 + }, + { + "contact_quality": null, + "progress": 0.1958213746547699, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1958213746547699 + }, + { + "contact_quality": null, + "progress": 0.22092504799365997, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22092504799365997 + }, + { + "contact_quality": null, + "progress": 0.17803087830543518, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17803087830543518 + }, + { + "contact_quality": null, + "progress": 0.20323540270328522, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20323540270328522 + }, + { + "contact_quality": null, + "progress": 0.19053253531455994, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19053253531455994 + } + ], + "candidate_progress": [ + 0.21672730147838593, + 0.1849215030670166, + 0.21649955213069916, + 0.1958213746547699, + 0.22092504799365997, + 0.17803087830543518, + 0.20323540270328522, + 0.19053253531455994 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4", + "ms-ce9505086ea115f86649", + "ms-5bddb39f792f20e5e6d4", + "ms-9228f8295b9891cf011c", + "ms-9d12ad4ebdf37419994c", + "ms-9d12ad4ebdf37419994c" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9a14552b7db695dee92c", + "generated_tangents": [ + [ + 0.05048983544111252, + -0.06354224681854248, + -0.2508169710636139, + 0.3281157910823822, + 0.06176993250846863, + -1.0395859479904175, + -0.11872172355651855, + 0.04913552105426788, + 0.03442516177892685, + 0.06192578002810478, + 0.1901962161064148, + 0.15638986229896545, + -1.8924229145050049, + -0.1239984780550003, + -0.013814132660627365, + 0.08172887563705444, + 0.06453713774681091, + -0.039126038551330566, + 0.0215013325214386, + -1.6615486145019531, + -0.16162586212158203 + ], + [ + -0.061983928084373474, + -0.09459616243839264, + -0.2104627788066864, + 0.3599303662776947, + 0.05293440818786621, + -0.7814253568649292, + -0.02095434069633484, + 0.09287988394498825, + 0.02182001620531082, + 0.11593418568372726, + 0.1766892969608307, + 0.31205666065216064, + -1.836709976196289, + -0.14834998548030853, + -0.014053288847208023, + 0.13140583038330078, + 0.09235826134681702, + -0.08436663448810577, + -0.07792538404464722, + -1.6235504150390625, + -0.19722992181777954 + ], + [ + -0.06358054280281067, + -0.1707766354084015, + -0.4351976811885834, + 0.6192716360092163, + 0.17808198928833008, + -1.0362719297409058, + 0.02410733699798584, + 0.18320702016353607, + -0.057587020099163055, + -0.03287283331155777, + 0.014523003250360489, + 0.07170911133289337, + -2.2714781761169434, + -0.13795168697834015, + -0.00746951624751091, + 0.15431226789951324, + 0.24740000069141388, + -0.13378311693668365, + -0.14430412650108337, + -1.5733966827392578, + -0.14196494221687317 + ], + [ + -0.017947256565093994, + 0.0040266141295433044, + -0.13806882500648499, + 0.5576839447021484, + 0.055895641446113586, + -1.1047216653823853, + -0.23974275588989258, + 0.043097689747810364, + -0.030100451782345772, + 0.04504986107349396, + 0.06926173716783524, + 0.007569238543510437, + -2.192204236984253, + 0.08518990874290466, + 0.022515147924423218, + 0.07497668266296387, + 0.06646496057510376, + -0.03373345732688904, + -0.27405136823654175, + -1.905529260635376, + 0.02017819881439209 + ], + [ + -0.12511123716831207, + -0.14703741669654846, + -0.4999878704547882, + 0.4722348153591156, + 0.21037617325782776, + -1.1627404689788818, + 0.08314064145088196, + 0.15366435050964355, + -0.11278150230646133, + -0.03228030353784561, + -0.02198120951652527, + 0.04202157258987427, + -2.1234261989593506, + -0.05932125449180603, + -0.10859385132789612, + 0.11286456882953644, + 0.28593140840530396, + -0.034758344292640686, + -0.21397221088409424, + -1.6376261711120605, + -0.31162482500076294 + ], + [ + 0.09778532385826111, + -0.07935328036546707, + -0.1260325014591217, + 0.45217737555503845, + 0.0013529956340789795, + -1.891746163368225, + -0.10438776016235352, + 0.055621206760406494, + -0.03399578481912613, + 0.04440484941005707, + 0.11405788362026215, + 0.11137016862630844, + -1.7957154512405396, + -0.2559959292411804, + 0.01711001619696617, + -0.04273573309183121, + 0.1087232455611229, + -0.030084341764450073, + -0.1149483323097229, + -1.6477367877960205, + -0.058281928300857544 + ], + [ + 0.172541543841362, + -0.032881900668144226, + -0.22155597805976868, + 0.3285578191280365, + -0.09919175505638123, + -1.7492244243621826, + -0.034220099449157715, + 0.0644887387752533, + -0.04440489411354065, + 0.051439717411994934, + 0.07902070879936218, + 0.16825929284095764, + -1.7917665243148804, + -0.2121959775686264, + -0.049102213233709335, + -0.08361957967281342, + 0.12374039739370346, + 0.017558544874191284, + -0.11949673295021057, + -1.7161366939544678, + 0.07041347771883011 + ], + [ + 0.12216803431510925, + -0.0765799954533577, + -0.15727552771568298, + 0.3809538185596466, + 0.01112222671508789, + -1.882515788078308, + -0.05779761075973511, + 0.05188503861427307, + -0.041006267070770264, + 0.0496743842959404, + 0.17175006866455078, + 0.09753971546888351, + -1.8451467752456665, + -0.1975952535867691, + -0.0013269595801830292, + -0.07415686547756195, + 0.02080000936985016, + -0.04785656929016113, + -0.1824963390827179, + -1.6308767795562744, + 0.008575230836868286 + ] + ], + "generated_utilities": [ + 0.21672730147838593, + 0.1849215030670166, + 0.21649955213069916, + 0.1958213746547699, + 0.22092504799365997, + 0.17803087830543518, + 0.20323540270328522, + 0.19053253531455994 + ], + "group_id": "ms-9a14552b7db695dee92c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.838127613067627, + 2.0, + 0.15671375393867493, + 0.17482513189315796, + 0.5684618353843689, + 0.19147612154483795, + 0.5643423795700073, + 0.1653006672859192, + 0.11670731008052826, + 0.14578887820243835, + 0.16058449447155, + 0.10923929512500763, + 0.13408902287483215, + 0.06845971941947937, + 0.02954423427581787, + 0.08425161242485046 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.22581616044044495, + -0.32600927352905273, + -0.054001130163669586, + 1.2553811073303223, + -0.1383839249610901, + -1.9753923416137695, + 0.0, + 0.09536236524581909, + 0.017635054886341095, + -0.06946105509996414, + 0.06630897521972656, + -0.10150641947984695, + -1.9112331867218018, + 0.0, + -0.23092225193977356, + 0.04748937487602234, + 0.3013715445995331, + -0.22072738409042358, + 0.0733548104763031, + -1.6514825820922852, + 0.0 + ], + [ + 0.22913068532943726, + -0.32600927352905273, + 0.24692901968955994, + 1.2553811073303223, + 0.3704858124256134, + -1.9753923416137695, + 0.0, + 0.036500319838523865, + 0.12820330262184143, + -0.09674224257469177, + 0.2668613791465759, + -0.14372897148132324, + -1.9112331867218018, + 0.0, + -0.27894535660743713, + -0.3322452902793884, + 0.16300754249095917, + -0.24123603105545044, + -0.2359180599451065, + -1.6514825820922852, + 0.0 + ], + [ + 0.0198141410946846, + -1.3260092735290527, + 0.011403528973460197, + 2.2553811073303223, + 0.5258303284645081, + -2.9753923416137695, + 1.8956809043884277, + -0.028990529477596283, + 0.007858656346797943, + 0.03128144145011902, + 0.5457771420478821, + -0.14793801307678223, + -2.9112331867218018, + -1.5743238925933838, + -0.011861175298690796, + -0.002163209253922105, + 0.015663353726267815, + 0.7587639689445496, + -0.07801014184951782, + -2.651482582092285, + -1.5696079730987549 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7913618087768555, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1486477851867676, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1392159461975098 + ], + [ + 0.13539990782737732, + -1.2862521409988403, + 0.28441768884658813, + 2.3658432960510254, + 0.4678441286087036, + -2.8791494369506836, + 1.8408029079437256, + -0.11893181502819061, + -0.1259741485118866, + -0.20943574607372284, + 0.4765416979789734, + -0.40862777829170227, + -2.8372273445129395, + -1.4697245359420776, + -0.08606886118650436, + -0.06554404646158218, + -0.27424973249435425, + 0.907739520072937, + -0.10130079835653305, + -2.6951611042022705, + -1.807652235031128 + ], + [ + -0.15379676222801208, + -1.7091000080108643, + 0.31153741478919983, + 2.2179949283599854, + 0.859725832939148, + -2.6038026809692383, + 1.98358952999115, + 0.22934654355049133, + -0.16202330589294434, + 0.02916553243994713, + 0.7702362537384033, + -0.0907563641667366, + -2.6495683193206787, + -1.394291877746582, + 0.2534489035606384, + -0.07077484577894211, + 0.10660970956087112, + 0.7866172194480896, + -0.41323983669281006, + -2.4847500324249268, + -1.5881152153015137 + ], + [ + 0.14051103591918945, + -0.8684865236282349, + 0.45214012265205383, + 2.499347686767578, + 0.310441792011261, + -2.875823497772217, + 1.4898443222045898, + 0.3396439850330353, + -0.033532168716192245, + 0.20189781486988068, + 0.3506202697753906, + 0.04831346869468689, + -2.4730942249298096, + -1.4369138479232788, + -0.4840375483036041, + -0.17814871668815613, + -0.1863224357366562, + 0.47034892439842224, + -0.5315045714378357, + -3.019744396209717, + -1.2571722269058228 + ], + [ + 0.5503659844398499, + -1.517655372619629, + -0.20272362232208252, + 2.5588791370391846, + 0.7160627841949463, + -2.551017999649048, + 2.423107385635376, + 0.18953359127044678, + -0.4268995523452759, + 0.014377228915691376, + 0.969605028629303, + -0.2440275251865387, + -2.385143756866455, + -1.4786417484283447, + 0.44389280676841736, + -0.11543146520853043, + -0.5315021872520447, + 1.2627880573272705, + 0.24797001481056213, + -3.17976713180542, + -1.8759949207305908 + ], + [ + -0.07551594823598862, + -1.6101253032684326, + 0.3926098048686981, + 1.726672649383545, + 1.0158472061157227, + -3.367068290710449, + 1.4499058723449707, + -0.5616514086723328, + 0.002434568479657173, + -0.5033466815948486, + -0.12249338626861572, + -0.07335222512483597, + -3.5959439277648926, + -1.0662813186645508, + -0.034436360001564026, + 0.512275218963623, + -0.04822078347206116, + 0.6117934584617615, + -0.745093822479248, + -2.2700355052948, + -2.090803623199463 + ], + [ + 0.6751457452774048, + -0.7857800126075745, + 0.11759950965642929, + 2.7748122215270996, + 1.3096001148223877, + -2.3352341651916504, + 1.4677345752716064, + 0.7219796180725098, + 0.21788907051086426, + -0.3245394825935364, + 0.08372032642364502, + -0.6393120288848877, + -2.8002729415893555, + -2.157836437225342, + 0.11339983344078064, + -0.2834126651287079, + -0.3824191987514496, + 1.130112648010254, + -0.8028184175491333, + -2.3846678733825684, + -1.8259286880493164 + ], + [ + -0.01608770713210106, + -2.1245148181915283, + -0.175740584731102, + 2.754368782043457, + 1.0032098293304443, + -3.7700157165527344, + 1.3539870977401733, + -0.7906507849693298, + -0.8311541080474854, + 0.8180835247039795, + 0.10450628399848938, + 0.01135176420211792, + -3.396463394165039, + -0.6868918538093567, + 0.22346650063991547, + 0.693323016166687, + -0.47104692459106445, + 0.8188157081604004, + -0.6246469020843506, + -2.4006571769714355, + -2.431541919708252 + ], + [ + -0.7931793928146362, + -1.7645766735076904, + 0.23125068843364716, + 2.816321849822998, + 1.1080517768859863, + -3.8820090293884277, + 1.8420056104660034, + 0.4984684884548187, + -0.9149360656738281, + -0.028868291527032852, + 0.14621934294700623, + 0.546608567237854, + -2.7883894443511963, + -1.3650472164154053, + -0.055590901523828506, + 0.41940492391586304, + -0.7235082983970642, + 1.1734092235565186, + -0.6126058101654053, + -3.217271566390991, + -2.512387275695801 + ], + [ + -0.5335642695426941, + -2.139784336090088, + 0.9540140628814697, + 1.7781559228897095, + 1.2117886543273926, + -3.71681809425354, + 2.461766242980957, + 0.4546531140804291, + 0.09786976873874664, + -0.8281612396240234, + -0.19971781969070435, + -1.1042157411575317, + -3.047807216644287, + -1.0697336196899414, + -0.22256404161453247, + -0.3800235986709595, + 0.3603542447090149, + 0.7543611526489258, + -0.23184219002723694, + -3.5156431198120117, + -2.1170644760131836 + ], + [ + 0.27338743209838867, + -0.9654052257537842, + -0.2700650095939636, + 2.6957814693450928, + 0.0643424391746521, + -3.094449281692505, + 0.8981494307518005, + 0.19603312015533447, + 0.5969407558441162, + -0.45200687646865845, + 1.466981291770935, + -0.45456382632255554, + -2.997983455657959, + -1.1903986930847168, + 0.5722758769989014, + -0.9932370781898499, + 0.28074830770492554, + 0.6244409084320068, + 0.6411240696907043, + -3.2179243564605713, + -1.905179738998413 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.025789255276322365, + -0.32600927352905273, + -0.02888326905667782, + 1.2553811073303223, + 0.013092517852783203, + -1.9753923416137695, + 0.0, + 0.04008308798074722, + -0.05201057717204094, + -0.008613351732492447, + -0.06225985288619995, + -0.0397564172744751, + -1.9112331867218018, + 0.0, + -0.02836347185075283, + -0.0055325529538095, + 0.053034864366054535, + 0.009740114212036133, + -0.006228044629096985, + -1.6514825820922852, + 0.0 + ] + ], + "predicted_scores": [ + 6.210387706756592, + 6.031088352203369, + 5.818679332733154, + 5.786194324493408, + 5.449621200561523, + 5.4391560554504395, + 5.10698938369751, + 5.087178707122803 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264/train2", + "selected_index": 0, + "state_hash": "49d38eabb7404e0028c75dd98e1239274b5dc2f04b9db8653001e44369ca8f3a", + "stored_base_utility": 0.838127613067627, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8326728343963623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8326728343963623 + }, + "base_progress": 0.8326728343963623, + "base_success": false, + "base_utility": 0.8326728343963623, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.814136266708374, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.814136266708374 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.17079035937786102, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17079035937786102 + }, + { + "contact_quality": null, + "progress": 0.8617495894432068, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8617495894432068 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.814136266708374, + 1.0, + 1.0, + 1.0, + 0.17079035937786102, + 0.8617495894432068, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-eb7189a4f2b0f0a5d4bd", + "ms-19a3669e096e98232ec8", + "ms-5dfbb16e173e1a3e2bd6", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-c80e71e55d51d8595499", + "ms-c80e71e55d51d8595499" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + false, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-9f5d57fe2153fd267774", + "generated_tangents": [ + [ + -0.036842867732048035, + 0.048847898840904236, + -0.04570475220680237, + 0.2915644943714142, + -0.08123436570167542, + -1.4965012073516846, + -0.01906418800354004, + 0.013378605246543884, + -0.02631569467484951, + -0.012372992932796478, + 0.34770116209983826, + 0.19430376589298248, + -1.3123862743377686, + -0.05775482952594757, + -0.12261678278446198, + 0.06757453083992004, + 0.0445905476808548, + -0.034035637974739075, + -0.2077113389968872, + -0.8378490805625916, + 0.12341295182704926 + ], + [ + 0.11348025500774384, + 0.08447839319705963, + -0.16422703862190247, + 0.258289098739624, + -0.018433362245559692, + -1.6425235271453857, + 0.009867280721664429, + -0.030288204550743103, + -0.07243704795837402, + 0.01182042807340622, + 0.3166850805282593, + 0.15702839195728302, + -1.2816481590270996, + -0.15220917761325836, + -0.11737127602100372, + 0.03544020280241966, + 0.12793458998203278, + 0.006233096122741699, + -0.31687653064727783, + -1.0269509553909302, + 0.160850390791893 + ], + [ + -0.20835839211940765, + -0.10486369580030441, + 0.057728081941604614, + -0.004304111003875732, + 0.23344916105270386, + -1.4573490619659424, + 0.11878395080566406, + 0.019034922122955322, + -0.12262396514415741, + 0.06636328995227814, + 0.4257264733314514, + 0.2640498876571655, + -1.5653103590011597, + 0.10354546457529068, + -0.05625971779227257, + 0.0666455328464508, + 0.2862924635410309, + 0.009381785988807678, + -0.2783547639846802, + -0.9582595229148865, + 0.04166419804096222 + ], + [ + 0.1694633960723877, + 0.08095662295818329, + 0.10999870300292969, + 0.3663580119609833, + -0.18532249331474304, + -1.7100986242294312, + -0.04058218002319336, + 0.01620161533355713, + -0.015664301812648773, + -0.06205829232931137, + 0.31903040409088135, + 0.17578113079071045, + -1.196927547454834, + 0.045018360018730164, + -0.10143482685089111, + 0.03413206711411476, + 0.06631229817867279, + -0.008357241749763489, + -0.1705082654953003, + -0.7727985382080078, + 0.22886992990970612 + ], + [ + 0.17452169954776764, + 0.0699475109577179, + -0.07850918173789978, + 0.16264608502388, + -0.011320486664772034, + -1.5561927556991577, + -0.006917595863342285, + 0.09690427035093307, + -0.03578643500804901, + 0.02951199561357498, + 0.33576828241348267, + 0.07957399636507034, + -1.2511686086654663, + -0.11705876886844635, + -0.10165093839168549, + 0.07081210613250732, + 0.15367388725280762, + 0.00503203272819519, + -0.2053247094154358, + -0.9252009391784668, + 0.16531416773796082 + ], + [ + 0.10311972349882126, + 0.07893295586109161, + -0.0973103940486908, + 0.20181289315223694, + -0.019721180200576782, + -1.6984442472457886, + -0.08015614748001099, + 0.05988536775112152, + -0.06321536004543304, + 0.014984451234340668, + 0.3754003643989563, + 0.11885345727205276, + -1.2185344696044922, + -0.0361989289522171, + -0.07925519347190857, + 0.09786573052406311, + 0.0974055826663971, + 0.04563935101032257, + -0.16137880086898804, + -1.018756628036499, + 0.13988687098026276 + ], + [ + 0.11710644513368607, + 0.0685444176197052, + -0.05941978096961975, + 0.323681503534317, + -0.002460569143295288, + -1.6858038902282715, + -0.21159547567367554, + 0.03656338155269623, + -0.048801787197589874, + 0.06270995736122131, + 0.3533012866973877, + 0.07704123854637146, + -1.2540138959884644, + -0.050995469093322754, + -0.09173603355884552, + 0.12975971400737762, + 0.07895532250404358, + 0.028982236981391907, + -0.11575859785079956, + -0.963138222694397, + 0.07556076347827911 + ], + [ + 0.14770382642745972, + -0.0066012367606163025, + 0.4265309274196625, + 0.13929837942123413, + 0.2602374851703644, + -1.796496868133545, + -0.7083551287651062, + 0.03829829394817352, + -0.15409168601036072, + 0.21598482131958008, + 0.1338273286819458, + -0.04723362624645233, + -1.3035222291946411, + -0.02220889925956726, + 0.02892022579908371, + 0.19045981764793396, + 0.1990739405155182, + 0.17386750876903534, + -0.42094820737838745, + -0.9603549838066101, + 0.04604174196720123 + ] + ], + "generated_utilities": [ + 0.814136266708374, + 2.0, + 2.0, + 2.0, + 0.17079035937786102, + 0.8617495894432068, + 2.0, + 2.0 + ], + "group_id": "ms-9f5d57fe2153fd267774", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8544998168945312, + 2.0, + 0.14427810907363892, + 0.1566883623600006, + 0.7847082018852234, + 0.6161996126174927, + 0.5900160074234009, + 0.12891459465026855, + 0.5811972618103027, + 0.12705880403518677, + 0.12002001702785492, + 0.07508957386016846, + 0.1335657238960266, + 2.0, + 0.5254277586936951, + 0.1602712869644165 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.07957560569047928, + 0.1778574287891388, + -0.06101015955209732, + 0.08770006895065308, + -0.13665570318698883, + -1.1242008209228516, + 0.0, + 0.20265339314937592, + 0.03117954358458519, + -0.08283126354217529, + 0.051615357398986816, + 0.006725620478391647, + -0.7304134368896484, + 0.0, + -0.03756266087293625, + 0.051428791135549545, + 0.13959813117980957, + 0.040047526359558105, + -0.04084143042564392, + -0.31500017642974854, + 0.0 + ], + [ + 0.31541678309440613, + -0.16606420278549194, + 0.09380461275577545, + 0.031209349632263184, + -0.10083025693893433, + -1.1242008209228516, + 0.0, + 0.09663133323192596, + -0.095388263463974, + 0.4012776017189026, + 0.051615357398986816, + 0.167547807097435, + -0.7304134368896484, + 0.0, + -0.6302167177200317, + 0.05524101108312607, + -0.14198890328407288, + 0.040047526359558105, + 0.5201720595359802, + -0.31500017642974854, + 0.0 + ], + [ + 0.004154659807682037, + -0.0034282731357961893, + 0.005892834160476923, + 1.0312093496322632, + -0.15828494727611542, + -2.1242008209228516, + -1.4059314727783203, + 0.002088826149702072, + 0.0016306412871927023, + 0.005124530289322138, + 1.0516153573989868, + -0.05936579033732414, + -1.7304134368896484, + -1.2081255912780762, + 0.010161396116018295, + 0.0017865521367639303, + -0.006222852971404791, + 1.040047526359558, + 0.07346899807453156, + -1.3150001764297485, + -0.9752891659736633 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.8118629455566406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.4162511825561523, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9505783319473267 + ], + [ + 0.05903958901762962, + -0.05713164061307907, + 0.27611011266708374, + 0.9773679971694946, + -0.2122824341058731, + -2.198127031326294, + -1.1234419345855713, + 0.22609303891658783, + 0.14432841539382935, + 0.13455358147621155, + 0.8475942611694336, + 0.19007176160812378, + -1.4536203145980835, + -1.208316445350647, + -0.24895474314689636, + 0.08610261976718903, + 0.16216596961021423, + 1.2356702089309692, + 0.07108174264431, + -1.4601069688796997, + -1.0005295276641846 + ], + [ + -0.2862337827682495, + -0.34814661741256714, + -0.3317800462245941, + 1.383265495300293, + -0.3485583961009979, + -1.7799506187438965, + -1.6264256238937378, + -0.24356596171855927, + -0.18337340652942657, + 0.3351142108440399, + 1.36115562915802, + -0.4404119849205017, + -1.4858057498931885, + -0.9229021668434143, + 0.3078569769859314, + -0.18743832409381866, + -0.3414495885372162, + 1.0045359134674072, + -0.1438894122838974, + -1.5133963823318481, + -0.7331042885780334 + ], + [ + 0.14600041508674622, + -0.10555070638656616, + 0.06090353801846504, + 1.1715545654296875, + 0.14829085767269135, + -1.8377182483673096, + -1.2019046545028687, + -0.33082208037376404, + 0.27619102597236633, + 0.3936208188533783, + 1.4077775478363037, + -0.2165551781654358, + -2.094696283340454, + -0.7843748331069946, + -0.13239514827728271, + 0.208020880818367, + 0.17500364780426025, + 0.5608562231063843, + 0.5641188025474548, + -1.6192113161087036, + -1.1969176530838013 + ], + [ + -0.03659747168421745, + -0.2400680035352707, + 0.49718841910362244, + 1.0422266721725464, + 0.13088186085224152, + -2.068427324295044, + -1.1215412616729736, + 0.06697356700897217, + 0.19986751675605774, + -0.5827334523200989, + 0.9792195558547974, + -0.5810520648956299, + -1.7176907062530518, + -0.6760571002960205, + -0.08971267938613892, + 0.36716267466545105, + 0.4829793870449066, + 0.7045803070068359, + -0.3341261148452759, + -1.3750282526016235, + -1.086517095565796 + ], + [ + 0.09393255412578583, + 0.6277099251747131, + 0.653377115726471, + 0.5602235198020935, + -0.8483365178108215, + -1.7264833450317383, + -1.698084831237793, + -0.3303340971469879, + 0.43070027232170105, + -0.13159427046775818, + 1.0274523496627808, + 0.4760425388813019, + -2.1360630989074707, + -0.6506720185279846, + 0.3130929470062256, + 0.6801184415817261, + 0.6575731039047241, + 0.9053056240081787, + 0.18805859982967377, + -1.3225764036178589, + -1.0653810501098633 + ], + [ + 0.536294162273407, + -0.026106586679816246, + 0.11593607813119888, + 0.2775042653083801, + 0.1801595836877823, + -1.590940237045288, + -0.6846873164176941, + 0.5080465078353882, + -0.7678324580192566, + 0.24115782976150513, + 0.2675415277481079, + -0.3879689872264862, + -2.1869571208953857, + -0.7176487445831299, + -0.15474224090576172, + -0.5326727032661438, + -0.12406099587678909, + 0.411626398563385, + 0.3519209027290344, + -1.6924495697021484, + -1.4648362398147583 + ], + [ + 0.2688097357749939, + -0.7928301095962524, + -0.39608004689216614, + 1.3961963653564453, + -0.026196151971817017, + -2.263073682785034, + -2.044499635696411, + 0.055598195642232895, + -0.485431969165802, + -0.058985911309719086, + 1.781836986541748, + 0.7104123830795288, + -2.0781984329223633, + -0.6968748569488525, + -0.1437186300754547, + 0.8155919313430786, + 0.5188283920288086, + 0.23954623937606812, + -0.6596511602401733, + -1.3867920637130737, + -0.7448391318321228 + ], + [ + 0.11259422451257706, + 0.43877506256103516, + -0.585435152053833, + 0.12099790573120117, + 0.5070493817329407, + -2.101219654083252, + -1.3759371042251587, + -0.7516231536865234, + 0.7018498778343201, + 0.7899036407470703, + 0.9772143959999084, + -0.7032701373100281, + -1.4699928760528564, + -1.4471495151519775, + 0.4750489890575409, + 0.8803708553314209, + -0.25032687187194824, + 1.3429648876190186, + 0.7985087037086487, + -0.5786607265472412, + -1.563289761543274 + ], + [ + 0.4965847432613373, + -0.8452511429786682, + 0.21300016343593597, + 0.28797537088394165, + -1.0059970617294312, + -3.040114402770996, + -1.1050269603729248, + 0.7826927304267883, + 0.8645586371421814, + -0.9447845816612244, + 1.0121192932128906, + 0.5123239755630493, + -1.5470938682556152, + -0.6170141100883484, + 0.2538997530937195, + 0.912939727306366, + -0.8613666892051697, + 0.9517492055892944, + 0.7185115218162537, + -2.1534924507141113, + -0.6818757057189941 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025899745523929596, + -0.03433045744895935, + -0.044045113027095795, + 0.031209349632263184, + -0.014532476663589478, + -1.1242008209228516, + 0.0, + 0.10768167674541473, + -0.020356399938464165, + 0.09216367453336716, + 0.051615357398986816, + -0.011370409280061722, + -0.7304134368896484, + 0.0, + 0.04558054730296135, + 0.038496289402246475, + 0.02504396252334118, + 0.040047526359558105, + 0.01217309758067131, + -0.31500017642974854, + 0.0 + ], + [ + -0.5198070406913757, + -0.8798871040344238, + 0.04596509411931038, + 1.55931556224823, + 0.31027060747146606, + -1.5283797979354858, + -1.6053413152694702, + -0.2754230797290802, + -0.012456716038286686, + 0.7998818159103394, + 1.5184144973754883, + 0.05059843137860298, + -1.85283362865448, + -0.9533926248550415, + 0.9978112578392029, + 0.8525571227073669, + -0.9611027836799622, + 0.32179969549179077, + -0.448661744594574, + -1.9452965259552002, + -0.7132223844528198 + ] + ], + "predicted_scores": [ + 5.260086536407471, + 5.183110237121582, + 5.1446919441223145, + 4.91237735748291, + 4.682591438293457, + 4.640739917755127, + 4.59874963760376, + 3.972043991088867 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1607773860/train2", + "selected_index": 0, + "state_hash": "fdc7379ab4491c9f65860cadd3b2f3c9f2335f26aae91fb1c06d3a4edf858d6a", + "stored_base_utility": 0.8544998168945312, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8272150158882141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8272150158882141 + }, + "base_progress": 0.8272150158882141, + "base_success": false, + "base_utility": 0.8272150158882141, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.15089993178844452, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15089993178844452 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8353422284126282, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8353422284126282 + }, + { + "contact_quality": null, + "progress": 0.13203097879886627, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13203097879886627 + }, + { + "contact_quality": null, + "progress": 0.13766705989837646, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13766705989837646 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.19075119495391846, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19075119495391846 + } + ], + "candidate_progress": [ + 1.0, + 0.15089993178844452, + 1.0, + 0.8353422284126282, + 0.13203097879886627, + 0.13766705989837646, + 1.0, + 0.19075119495391846 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-20f3cfcf183c69853b1e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + true, + false, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b0fee26e1db967f90b81", + "generated_tangents": [ + [ + 0.08246950805187225, + 0.08750855922698975, + -0.0660984218120575, + -0.06540507078170776, + 0.1978612244129181, + -2.1107535362243652, + -0.23482418060302734, + -0.04672335088253021, + -0.0629846528172493, + 0.06864479184150696, + 0.22597938776016235, + 0.20545238256454468, + -1.647207260131836, + -0.30052053928375244, + 0.02503054589033127, + 0.005656566470861435, + 0.13866335153579712, + -0.0044862329959869385, + -0.06137773394584656, + -1.3930904865264893, + -0.24488231539726257 + ], + [ + 0.1859014928340912, + 0.07491914927959442, + 0.09269857406616211, + 0.003977775573730469, + 0.1321168690919876, + -2.0118014812469482, + -0.3092394471168518, + -0.023855969309806824, + -0.0012620426714420319, + 0.05011509358882904, + 0.2728317379951477, + 0.33122509717941284, + -1.6019604206085205, + -0.24240286648273468, + 0.014448247849941254, + -0.020438531413674355, + 0.09186503291130066, + 0.026910707354545593, + 0.07068610191345215, + -1.2765593528747559, + -0.21925383806228638 + ], + [ + -0.006058543920516968, + 0.06651297211647034, + 0.04328954219818115, + -0.06965845823287964, + 0.15326642990112305, + -1.8159501552581787, + -0.21902137994766235, + -0.007013410329818726, + -0.004383314400911331, + 0.07494658976793289, + 0.3194279372692108, + 0.28470128774642944, + -1.6724324226379395, + -0.16556812822818756, + -0.01442759856581688, + 0.02155320718884468, + 0.05166323482990265, + -0.026102766394615173, + -0.0037977099418640137, + -1.270317792892456, + -0.26662954688072205 + ], + [ + 0.0936608612537384, + 0.07955317199230194, + -0.11822423338890076, + -0.004888221621513367, + 0.23157638311386108, + -2.067514419555664, + -0.30729854106903076, + -0.0008642375469207764, + -0.05714868754148483, + 0.11828251928091049, + 0.244703471660614, + 0.11098344624042511, + -1.8146916627883911, + -0.19908224046230316, + 0.024550646543502808, + 0.004102073609828949, + 0.0627811849117279, + 0.007633984088897705, + -0.07723230123519897, + -1.5077738761901855, + -0.2014407217502594 + ], + [ + 0.14827285706996918, + 0.060327932238578796, + 0.03986567258834839, + -0.12716135382652283, + 0.23965947329998016, + -1.939994215965271, + -0.22944331169128418, + 0.10510952025651932, + -0.034818679094314575, + 0.09262728691101074, + 0.229600727558136, + 0.19778873026371002, + -1.5633316040039062, + -0.2708389163017273, + 0.017550159245729446, + 0.037912219762802124, + 0.19822438061237335, + -0.023131489753723145, + 0.037082210183143616, + -1.2809975147247314, + -0.22135639190673828 + ], + [ + -0.029288768768310547, + 0.03452843427658081, + 0.018285095691680908, + -0.0342509001493454, + 0.1954054832458496, + -1.9721804857254028, + -0.22642755508422852, + 0.1918209195137024, + -0.085311159491539, + 0.08068110048770905, + 0.0671565979719162, + 0.22854061424732208, + -1.9288711547851562, + -0.13851411640644073, + -0.023961801081895828, + 0.058769483119249344, + 0.22576399147510529, + -0.0660112053155899, + -0.08861735463142395, + -1.6937940120697021, + -0.22313708066940308 + ], + [ + -0.06338757276535034, + 0.062273234128952026, + 0.0781092643737793, + -0.04689791798591614, + 0.18885481357574463, + -2.0480566024780273, + -0.22808384895324707, + 0.23956647515296936, + -0.11650949716567993, + 0.10949895530939102, + 0.09395516663789749, + 0.1618867814540863, + -1.806831955909729, + 0.07582738995552063, + -0.018856864422559738, + 0.06067037954926491, + 0.20329849421977997, + 0.01653990149497986, + -0.04835501313209534, + -1.5202925205230713, + -0.15853941440582275 + ], + [ + 0.221816748380661, + -0.011786513030529022, + 0.03290960192680359, + -0.2630120515823364, + 0.23217853903770447, + -1.3343830108642578, + -0.6715918183326721, + -0.0034032315015792847, + -0.09411638230085373, + -0.048669032752513885, + 0.21319153904914856, + 0.1285199522972107, + -1.395736813545227, + -0.3864459991455078, + -0.0024505741894245148, + 0.1499291956424713, + 0.10702880471944809, + 0.00843365490436554, + -0.1614212691783905, + -1.4488356113433838, + -0.13157981634140015 + ] + ], + "generated_utilities": [ + 2.0, + 0.15089993178844452, + 2.0, + 0.8353422284126282, + 0.13203097879886627, + 0.13766705989837646, + 2.0, + 0.19075119495391846 + ], + "group_id": "ms-b0fee26e1db967f90b81", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8397466540336609, + 0.8594310283660889, + 0.14638642966747284, + 0.09418028593063354, + 0.7838175296783447, + 0.5982922315597534, + 0.5913184881210327, + 2.0, + 0.20996521413326263, + 0.17419946193695068, + 0.11757439374923706, + 0.1645697057247162, + 0.05191062390804291, + 0.08658662438392639, + 0.04989185929298401, + 0.05867189168930054 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1510201096534729, + -0.21724483370780945, + 0.14240525662899017, + 0.003108598291873932, + -0.11609727144241333, + -1.6516358852386475, + 0.0, + -0.09976008534431458, + 0.061503566801548004, + 0.3070257902145386, + -0.08051937818527222, + 0.05735237896442413, + -1.6075372695922852, + 0.0, + -0.19649454951286316, + -0.15815469622612, + 0.2621642053127289, + -0.17578476667404175, + -0.034231506288051605, + -1.3808586597442627, + 0.0 + ], + [ + 0.014505341649055481, + 0.0403006337583065, + 0.5529788732528687, + 0.0714028924703598, + -0.41683781147003174, + -1.6516358852386475, + 0.0, + 0.41446545720100403, + -0.16278114914894104, + -0.4971221685409546, + -0.3640303611755371, + 0.02483367919921875, + -1.6075372695922852, + 0.0, + 0.5711643695831299, + -0.5244585275650024, + -0.5069849491119385, + 0.31709733605384827, + -0.4254770278930664, + -1.3808586597442627, + 0.0 + ], + [ + -0.044587142765522, + 0.03136333078145981, + 0.014485275372862816, + 0.04301099479198456, + 0.47328460216522217, + -2.6516358852386475, + -0.32303211092948914, + -0.01855158433318138, + -0.003608003491535783, + 0.029763570055365562, + 0.6359696388244629, + -0.1304762363433838, + -2.607537269592285, + -1.7957698106765747, + -0.009414590895175934, + -0.000939749414101243, + 0.018994441255927086, + 0.7486392855644226, + -0.06401278078556061, + -2.3808586597442627, + -1.722450852394104 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.6460642218589783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.5915396213531494, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.444901704788208 + ], + [ + 0.18585285544395447, + 0.17557260394096375, + 0.08478302508592606, + -0.17644454538822174, + 0.7477058172225952, + -2.614673137664795, + -0.3018328547477722, + 0.2204062044620514, + -0.0497702918946743, + 0.1961878538131714, + 0.5304322242736816, + -0.15872131288051605, + -2.604066848754883, + -2.053436517715454, + 0.18444111943244934, + -0.0024187257513403893, + 0.0054171085357666016, + 1.0256288051605225, + 0.02724038064479828, + -2.426236391067505, + -1.6594898700714111 + ], + [ + -0.5335133075714111, + 0.12298225611448288, + -0.12197553366422653, + 0.0424814410507679, + 0.002416938543319702, + -2.8428826332092285, + -0.17827168107032776, + -0.3377644121646881, + 0.04258471727371216, + -0.15910103917121887, + 0.9169985055923462, + 0.003966793417930603, + -3.069180727005005, + -1.407257318496704, + -0.47542956471443176, + -0.2210284322500229, + -0.10021048784255981, + 0.7806655168533325, + 0.3228837251663208, + -2.058253049850464, + -1.556677222251892 + ], + [ + 0.20687156915664673, + -0.5511468052864075, + 0.3945963978767395, + -0.16104432940483093, + 0.4936041831970215, + -2.1271684169769287, + 0.02855244278907776, + 0.544975996017456, + -0.3336343765258789, + 0.029475202783942223, + 0.8120307326316833, + -0.3246060609817505, + -2.885145425796509, + -2.2030622959136963, + -0.28730231523513794, + 0.19937433302402496, + 0.19840951263904572, + 1.1192030906677246, + -0.3995746374130249, + -2.5594136714935303, + -1.1545796394348145 + ], + [ + -0.5497553944587708, + -0.39125651121139526, + 0.2725587487220764, + -0.27965104579925537, + 1.0584466457366943, + -2.1044363975524902, + -0.6898342370986938, + 0.36552220582962036, + -0.2289373278617859, + 0.24627351760864258, + 0.7968363165855408, + 0.14776766300201416, + -2.988917112350464, + -1.6111457347869873, + -0.5116392374038696, + 0.030796781182289124, + -0.08262427151203156, + 1.0816104412078857, + 0.336497962474823, + -1.684936285018921, + -2.290850877761841 + ], + [ + 0.18322327733039856, + 0.6938241720199585, + 0.5252001881599426, + 0.06770005077123642, + 1.120344877243042, + -1.8558356761932373, + 0.18930694460868835, + -0.7033196687698364, + 0.7861698865890503, + -0.0965423658490181, + 0.015094935894012451, + -0.5303559303283691, + -2.6036770343780518, + -2.589250087738037, + -0.468231737613678, + -0.5901691317558289, + -0.7109078764915466, + 0.986366331577301, + -0.7634903192520142, + -2.984590530395508, + -1.1651530265808105 + ], + [ + 0.7039692401885986, + -0.5346179008483887, + 0.1466517597436905, + -0.6284777522087097, + -0.07994896173477173, + -2.4640438556671143, + -1.102164626121521, + -0.26685386896133423, + -0.3165925443172455, + 0.8653677701950073, + 0.232782244682312, + -0.5054740905761719, + -3.400317668914795, + -1.4077497720718384, + -0.09987740218639374, + -0.27442091703414917, + 0.5428174138069153, + 1.5128605365753174, + 0.747156023979187, + -2.0367631912231445, + -2.464085578918457 + ], + [ + 0.4474923610687256, + -0.7824500203132629, + 0.5928975343704224, + 0.57558673620224, + -0.17630118131637573, + -2.176387310028076, + -0.3299017548561096, + -0.6172446608543396, + 0.5065999627113342, + 0.46063295006752014, + 0.5951727628707886, + -0.02366364747285843, + -3.254279136657715, + -0.8880517482757568, + 0.3714323043823242, + 0.12449074536561966, + 0.6677311658859253, + -0.20713549852371216, + 0.5062628984451294, + -3.2156763076782227, + -1.9652986526489258 + ], + [ + -0.437069296836853, + 0.615723192691803, + -0.7743021249771118, + 0.20228458940982819, + 0.4320293366909027, + -2.0403428077697754, + -0.32251936197280884, + -0.7280393838882446, + 0.10878247022628784, + 0.9395551681518555, + 0.6021312475204468, + -0.3748174011707306, + -2.0175063610076904, + -1.9166245460510254, + -0.10793138295412064, + 0.6512030959129333, + 0.3747653365135193, + 1.5698660612106323, + -0.7995363473892212, + -1.8378328084945679, + -1.0564453601837158 + ], + [ + -0.7214477062225342, + -0.5686705708503723, + 0.8819299936294556, + 0.7319017648696899, + 1.0695421695709229, + -2.0338172912597656, + 0.2973436415195465, + -0.19576850533485413, + 0.4506514370441437, + 0.9037411212921143, + 0.0996239185333252, + 0.5076434016227722, + -2.611851930618286, + -2.7078182697296143, + 0.022706281393766403, + -0.4812851846218109, + -0.40778082609176636, + 1.6509652137756348, + -0.5721896886825562, + -1.8604621887207031, + -1.2943172454833984 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.017527949064970016, + -0.13547566533088684, + -0.31396082043647766, + -0.06045208126306534, + 0.7718918323516846, + -2.996134042739868, + -0.0199548602104187, + 0.26194432377815247, + -0.303554505109787, + -0.09068091958761215, + 0.3007984459400177, + -0.39815279841423035, + -2.626121997833252, + -1.8776893615722656, + 0.25959014892578125, + 0.20619672536849976, + -0.25346171855926514, + 0.40209779143333435, + 0.1943899542093277, + -2.464694023132324, + -1.5338099002838135 + ] + ], + "predicted_scores": [ + 5.351569652557373, + 5.150033950805664, + 5.108182430267334, + 4.922300815582275, + 4.596928596496582, + 4.560333728790283, + 4.243762969970703, + 4.0818963050842285 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1358421325/train2", + "selected_index": 0, + "state_hash": "b8e04d64f31138fff8f83f1bb017f9e8515565a2ada2ba5787ad5f06b796589c", + "stored_base_utility": 0.8397466540336609, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8307622671127319, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8307622671127319 + }, + "base_progress": 0.8307622671127319, + "base_success": false, + "base_utility": 0.8307622671127319, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1599900722503662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1599900722503662 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.821384847164154, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.821384847164154 + }, + { + "contact_quality": null, + "progress": 0.15789160132408142, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15789160132408142 + }, + { + "contact_quality": null, + "progress": 0.15891481935977936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15891481935977936 + }, + { + "contact_quality": null, + "progress": 0.16026467084884644, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16026467084884644 + }, + { + "contact_quality": null, + "progress": 0.20434951782226562, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20434951782226562 + }, + { + "contact_quality": null, + "progress": 0.19699636101722717, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19699636101722717 + } + ], + "candidate_progress": [ + 0.1599900722503662, + 1.0, + 0.821384847164154, + 0.15789160132408142, + 0.15891481935977936, + 0.16026467084884644, + 0.20434951782226562, + 0.19699636101722717 + ], + "candidate_source_chart_ids": [ + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-852c671b0adeccfa4bf1", + "ms-5a32c809ed679663dcb1", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "generated_tangents": [ + [ + 0.16271671652793884, + 0.11835204064846039, + -0.012397438287734985, + -0.1449543535709381, + 0.6666361093521118, + -0.0610959529876709, + 0.05983695387840271, + -0.12314370274543762, + 0.05490633845329285, + 0.06102598458528519, + 0.011549975723028183, + 0.06348592042922974, + -1.7331287860870361, + -0.06759564578533173, + -0.01418175920844078, + 0.08610397577285767, + 0.02035468816757202, + -0.13417764008045197, + -0.08576458692550659, + -1.360227346420288, + 0.07081732898950577 + ], + [ + 0.05573435127735138, + 0.08733311295509338, + 0.0460299551486969, + -0.14449870586395264, + 0.6969571113586426, + 0.11252725124359131, + 0.14553365111351013, + -0.08895868062973022, + 0.08528045564889908, + 0.09448707103729248, + 0.019099678844213486, + 0.09892076998949051, + -1.745173692703247, + -0.02503219246864319, + 0.040615588426589966, + 0.15552745759487152, + 0.03870461881160736, + -0.1297275274991989, + -0.12249332666397095, + -1.3522653579711914, + -0.015355393290519714 + ], + [ + 0.05131131410598755, + 0.09055924415588379, + -0.004006803035736084, + -0.12880590558052063, + 0.5213919878005981, + 0.02825668454170227, + 0.0794881284236908, + 0.025297001004219055, + 0.07260261476039886, + 0.053302377462387085, + 0.018651578575372696, + 0.04935432970523834, + -1.71018648147583, + 0.11532102525234222, + 0.06491315364837646, + 0.03935879096388817, + 0.04305471479892731, + -0.029442936182022095, + -0.09051260352134705, + -1.34885835647583, + 0.13415014743804932 + ], + [ + 0.16039256751537323, + 0.10715499520301819, + -1.5407800674438477e-05, + -0.08977057039737701, + 0.6557028293609619, + -0.02549922466278076, + -0.05239427089691162, + -0.050411373376846313, + 0.03698960691690445, + 0.07752439379692078, + 0.015422597527503967, + -0.06018456816673279, + -1.77348792552948, + -0.09620897471904755, + -0.12304249405860901, + 0.08278532326221466, + -0.023905202746391296, + -0.11396853625774384, + 0.012908339500427246, + -1.378190279006958, + 0.15033148229122162 + ], + [ + 0.1748061180114746, + 0.12763012945652008, + -0.036204904317855835, + -0.22588416934013367, + 0.6186838150024414, + -0.1957300901412964, + -0.12827563285827637, + 0.015215441584587097, + -0.04022820293903351, + 0.07155010104179382, + 0.0013560131192207336, + 0.015266939997673035, + -1.793965458869934, + -0.012374207377433777, + -0.03334435448050499, + 0.11777634918689728, + 0.11445989459753036, + -0.07886438071727753, + -0.22546911239624023, + -1.4198358058929443, + 0.013972818851470947 + ], + [ + -0.04156471788883209, + 0.19573041796684265, + -0.17388007044792175, + -0.13879147171974182, + 0.6510802507400513, + 0.014952152967453003, + -0.06441938877105713, + 0.05719183385372162, + -0.057870566844940186, + 0.04107788950204849, + 0.045885320752859116, + -0.08660179376602173, + -1.7200990915298462, + -0.02635994553565979, + -0.08880971372127533, + 0.11992290616035461, + 0.2511446177959442, + -0.3118141293525696, + -0.258758008480072, + -1.4797537326812744, + 0.0482817143201828 + ], + [ + -0.0361819714307785, + -0.04947558790445328, + -0.37064018845558167, + -0.02005597949028015, + 0.7335584759712219, + -0.15239670872688293, + 0.5580345988273621, + -0.013057202100753784, + -0.06334485113620758, + -0.06316176801919937, + -0.050221458077430725, + -0.04865361750125885, + -1.955536961555481, + -0.15965412557125092, + -0.029360156506299973, + 0.17591671645641327, + 0.17993104457855225, + -0.18448616564273834, + -0.30352967977523804, + -1.5083882808685303, + -0.28336772322654724 + ], + [ + -0.061140745878219604, + -0.06449203193187714, + -0.28942009806632996, + 0.10007861256599426, + 0.7646355032920837, + 0.012566015124320984, + 0.4356676936149597, + 0.00030077993869781494, + 0.018040115013718605, + -0.05286494642496109, + -0.06451456248760223, + -0.1296241283416748, + -2.231977939605713, + -0.3048020601272583, + 0.020629890263080597, + 0.24416236579418182, + 0.22915402054786682, + -0.26925599575042725, + -0.2489616870880127, + -1.3599307537078857, + -0.06671296060085297 + ] + ], + "generated_utilities": [ + 0.1599900722503662, + 2.0, + 0.821384847164154, + 0.15789160132408142, + 0.15891481935977936, + 0.16026467084884644, + 0.20434951782226562, + 0.19699636101722717 + ], + "group_id": "ms-bc2d3b425d9163a62dbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8266608119010925, + 2.0, + 0.15905821323394775, + 0.13648143410682678, + 0.5621776580810547, + 0.18381908535957336, + 0.20885342359542847, + 0.18184903264045715, + 0.19253113865852356, + 0.5574395656585693, + 0.10362428426742554, + 0.0659959614276886, + 0.09997051954269409, + 0.2210000902414322, + 0.00010728836059570312, + 0.12723565101623535 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.059883326292037964, + -0.16710972785949707, + -0.04027438163757324, + -0.23492489755153656, + 0.34960758686065674, + 0.16551673412322998, + 0.0, + -0.27484941482543945, + 0.07304387539625168, + -0.38491085171699524, + 0.03056102991104126, + 0.16416949033737183, + -1.5270280838012695, + 0.0, + 0.01279987208545208, + -0.07740817964076996, + 0.07519636303186417, + 0.14440757036209106, + 0.0022157281637191772, + -1.3374013900756836, + 0.0 + ], + [ + -0.6122239828109741, + -0.0734642744064331, + -0.4064018726348877, + 0.11902926862239838, + 0.34960758686065674, + -0.07764682173728943, + 0.0, + 0.37407442927360535, + -0.08740249276161194, + -0.20959049463272095, + -0.37996697425842285, + -0.304497092962265, + -1.5270280838012695, + 0.0, + -0.9113190174102783, + 0.3964778184890747, + 0.4811243414878845, + 0.2385387420654297, + -0.11476142704486847, + -1.3374013900756836, + 0.0 + ], + [ + 0.3877759873867035, + 0.6924911141395569, + -1.0402743816375732, + 0.005685340613126755, + 1.3496075868606567, + -0.2760518193244934, + 1.2840001583099365, + -0.028063178062438965, + 0.0023769063409417868, + 0.03187502175569534, + 0.04654876887798309, + 0.3263346254825592, + -2.5270280838012695, + -1.7457892894744873, + -0.012772172689437866, + -0.008647367358207703, + 0.019657021388411522, + 0.3050904870033264, + 0.19037143886089325, + -2.3374013900756836, + -1.738369345664978 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.568000316619873, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.4915785789489746, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.476738691329956 + ], + [ + 0.1989155411720276, + 0.7169155478477478, + -0.9921301603317261, + 0.22762086987495422, + 1.516188144683838, + -0.23096826672554016, + 1.0414973497390747, + 0.0012522991746664047, + -0.0518387071788311, + -0.23540246486663818, + 0.08515512943267822, + 0.5060473084449768, + -2.5231378078460693, + -1.4593229293823242, + -0.09671962261199951, + 0.09134317189455032, + -0.07944301515817642, + 0.18403387069702148, + 0.2437569499015808, + -2.270580291748047, + -1.9640144109725952 + ], + [ + 0.256519615650177, + 0.7805030941963196, + -1.3583617210388184, + 0.3310847878456116, + 1.2735480070114136, + -0.4127388000488281, + 1.2665152549743652, + -0.19997301697731018, + -0.11129427701234818, + 0.1647017002105713, + 0.3767858147621155, + 0.4831879138946533, + -2.752143621444702, + -1.6790910959243774, + 0.2554209232330322, + 0.3251851797103882, + -0.12605378031730652, + 0.696884274482727, + 0.29535919427871704, + -2.726069688796997, + -1.704551100730896 + ], + [ + 0.49772778153419495, + 1.0026791095733643, + -0.5513590574264526, + 0.008379313163459301, + 1.7768750190734863, + -0.6102496981620789, + 1.7048192024230957, + -0.29564282298088074, + -0.20420490205287933, + -0.23934513330459595, + -0.34791648387908936, + 0.0690477192401886, + -2.602295398712158, + -1.8138929605484009, + -0.38315677642822266, + -0.16421082615852356, + 0.17502839863300323, + -0.13996875286102295, + -0.30929630994796753, + -2.544666290283203, + -1.2411268949508667 + ], + [ + 0.07701912522315979, + 0.9529868364334106, + -1.497549295425415, + -0.3016125559806824, + 1.8504154682159424, + -0.3136579394340515, + 1.6619524955749512, + -0.28860214352607727, + 0.20993559062480927, + 0.6072397232055664, + 0.04243222624063492, + -0.133962482213974, + -2.0948100090026855, + -2.1691184043884277, + 0.44940999150276184, + -0.4183289408683777, + 0.5662978291511536, + 0.5016863942146301, + 0.7054548263549805, + -2.282663345336914, + -1.8892805576324463 + ], + [ + -0.2981111705303192, + 0.038420140743255615, + -1.687281847000122, + -0.37934571504592896, + 1.1300733089447021, + -0.6208293437957764, + 1.8144867420196533, + 0.16701459884643555, + 0.5508663654327393, + 0.4797254800796509, + -0.09421360492706299, + -0.3499920070171356, + -2.5030596256256104, + -1.5871793031692505, + -0.33045580983161926, + -0.29071635007858276, + 0.5385350584983826, + 0.9102991223335266, + 0.3121117353439331, + -2.2699851989746094, + -2.081907272338867 + ], + [ + -0.28236815333366394, + 0.18060839176177979, + -0.32270634174346924, + 0.033799149096012115, + 1.128151774406433, + -0.7577929496765137, + 1.6937967538833618, + -0.813525378704071, + 0.48178502917289734, + -0.6699209809303284, + -0.03810972720384598, + 0.8273124694824219, + -2.2768704891204834, + -1.317199945449829, + 0.0046694036573171616, + 0.5900158286094666, + -0.12453534454107285, + 1.046316385269165, + 0.2770566940307617, + -2.0448076725006104, + -2.4180057048797607 + ], + [ + 0.26116815209388733, + 1.305420994758606, + -0.5357515811920166, + 0.6523149013519287, + 0.5250536203384399, + -0.6821175217628479, + 1.2918037176132202, + 0.3275931477546692, + -0.8845832943916321, + -0.6987056136131287, + 0.10023711621761322, + -0.2849085032939911, + -1.7303776741027832, + -1.0117536783218384, + -0.06847482174634933, + 0.6783800721168518, + -0.04633811116218567, + 0.0702061653137207, + -0.3159322142601013, + -3.004882335662842, + -2.0691566467285156 + ], + [ + 0.5418367385864258, + 1.1724635362625122, + -1.717900276184082, + 0.9217556118965149, + 2.316706657409668, + -0.2411489486694336, + 0.720655083656311, + -0.6330894231796265, + 0.03703225776553154, + 0.38938266038894653, + -0.8305367827415466, + -0.2152116596698761, + -2.614929437637329, + -2.045497179031372, + -0.9044661521911621, + 0.9391879439353943, + -0.2323634922504425, + 0.8392186164855957, + -0.6889055371284485, + -3.0831263065338135, + -1.0035808086395264 + ], + [ + 1.0110303163528442, + 1.0131109952926636, + -1.3200907707214355, + -0.433234840631485, + 1.9466626644134521, + 0.46447479724884033, + 1.528633952140808, + 0.40949776768684387, + 0.07322453707456589, + -0.12349443882703781, + 0.24514153599739075, + 0.005379408597946167, + -2.487516164779663, + -1.1890358924865723, + -0.026332437992095947, + -0.6102214455604553, + 0.6559669971466064, + 0.4371780753135681, + -0.6651540994644165, + -2.8161611557006836, + -1.4271454811096191 + ], + [ + 0.9640108346939087, + 1.0122193098068237, + -1.9809117317199707, + 0.40601760149002075, + 0.9296187162399292, + 0.601173460483551, + 2.229055643081665, + -0.4972975254058838, + 0.6548057794570923, + 0.6165774464607239, + -0.7309268116950989, + 0.697066068649292, + -3.0948128700256348, + -2.4828031063079834, + 0.16930773854255676, + -0.11851751059293747, + -0.8277174830436707, + 0.8252183794975281, + -0.4829612374305725, + -2.922985076904297, + -1.1981642246246338 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03792491555213928, + -0.040427446365356445, + -0.04027438163757324, + 0.02325565740466118, + 0.34960758686065674, + -0.05261008441448212, + 0.0, + 0.018791399896144867, + 0.05303370952606201, + 0.01128687709569931, + 0.01692008785903454, + 0.06139722466468811, + -1.5270280838012695, + 0.0, + -0.004229618236422539, + 0.017936373129487038, + 0.06901376694440842, + -0.04424762725830078, + -0.033029720187187195, + -1.3374013900756836, + 0.0 + ] + ], + "predicted_scores": [ + 7.299339294433594, + 7.067145824432373, + 7.013951301574707, + 6.319223880767822, + 5.728745937347412, + 4.369734287261963, + 4.050492286682129, + 3.6663787364959717 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007/train2", + "selected_index": 0, + "state_hash": "58b3d0c1cafc199ad58508912becbaef59038a655cef0caafd263ade5c9b53fa", + "stored_base_utility": 0.8266608119010925, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20032641291618347, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20032641291618347 + }, + "base_progress": 0.20032641291618347, + "base_success": false, + "base_utility": 0.20032641291618347, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.19381189346313477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19381189346313477 + }, + { + "contact_quality": null, + "progress": 0.23963995277881622, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23963995277881622 + }, + { + "contact_quality": null, + "progress": 0.21024136245250702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21024136245250702 + }, + { + "contact_quality": null, + "progress": 0.13467177748680115, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13467177748680115 + }, + { + "contact_quality": null, + "progress": 0.2017955482006073, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2017955482006073 + }, + { + "contact_quality": null, + "progress": 0.20696409046649933, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20696409046649933 + }, + { + "contact_quality": null, + "progress": 0.1936945617198944, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1936945617198944 + }, + { + "contact_quality": null, + "progress": 0.21807821094989777, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21807821094989777 + } + ], + "candidate_progress": [ + 0.19381189346313477, + 0.23963995277881622, + 0.21024136245250702, + 0.13467177748680115, + 0.2017955482006073, + 0.20696409046649933, + 0.1936945617198944, + 0.21807821094989777 + ], + "candidate_source_chart_ids": [ + "ms-5ac1bf2d7d319955d45d", + "ms-95028134c6e3670b89d5", + "ms-b8492a3a812095651d75", + "ms-ee2fcbc94d41385713d0", + "ms-95028134c6e3670b89d5", + "ms-9ae5c6c4132218ecf49e", + "ms-016c5de39ed3c111889a", + "ms-c8706d0aa14a930236de" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-d1960368ef76bbb10669", + "generated_tangents": [ + [ + 0.19640028476715088, + 0.2096945196390152, + 0.07385209202766418, + -0.256763756275177, + 0.7537413835525513, + 0.3147052526473999, + 0.5640119910240173, + -0.2542858123779297, + 0.06437408179044724, + 0.10259352624416351, + 0.2628327012062073, + 0.2347445785999298, + -1.871469497680664, + 0.571297287940979, + -0.01785074546933174, + 0.06142580881714821, + -0.3025362491607666, + 0.005423873662948608, + -0.4231855869293213, + -2.0950539112091064, + -0.12601861357688904 + ], + [ + 0.0627809390425682, + 0.1603173166513443, + -0.20445504784584045, + -0.22461488842964172, + 0.7926700115203857, + -0.1396835744380951, + 0.9558979272842407, + -0.03341652452945709, + -0.09778351336717606, + -0.06997280567884445, + 0.20971012115478516, + 0.08173273503780365, + -1.808558464050293, + 0.6446847915649414, + -0.10960032045841217, + 0.08301788568496704, + -0.08366130292415619, + 0.07870107889175415, + -0.4879615902900696, + -1.9494152069091797, + -0.4764796793460846 + ], + [ + 0.21831931173801422, + 0.11013790965080261, + 0.04802587628364563, + -0.05057603120803833, + 0.6370452046394348, + -0.25856971740722656, + 0.42446526885032654, + -0.18769535422325134, + -0.04787231981754303, + -0.048109270632267, + 0.1623188555240631, + 0.1620917171239853, + -1.741257905960083, + 0.6164520978927612, + -0.0207456536591053, + 0.08974412083625793, + -0.28634965419769287, + 0.023157447576522827, + -0.5793269276618958, + -1.7957277297973633, + -0.1764676868915558 + ], + [ + 0.3186380863189697, + 0.21159712970256805, + 0.00373879075050354, + -0.3227611184120178, + 0.6657334566116333, + 0.30726322531700134, + 0.5001685619354248, + -0.16107040643692017, + 0.03624623268842697, + 0.26211827993392944, + 0.3158920705318451, + 0.16389289498329163, + -1.5911295413970947, + 0.6357441544532776, + 0.020050078630447388, + 0.09652507305145264, + -0.1914803832769394, + 0.018958255648612976, + -0.2784596085548401, + -1.7731802463531494, + -0.1289038360118866 + ], + [ + 0.22214922308921814, + 0.22001208364963531, + -0.003042936325073242, + -0.19827944040298462, + 0.7902159094810486, + -0.2831520736217499, + 0.5995392799377441, + -0.1823556125164032, + -0.016991518437862396, + 0.09439627826213837, + 0.21597382426261902, + 0.2315910905599594, + -1.4815831184387207, + 0.5753792524337769, + 0.03296735882759094, + -0.0037064021453261375, + -0.23173846304416656, + 0.0810437873005867, + -0.42252302169799805, + -1.8860173225402832, + -0.34162893891334534 + ], + [ + 0.1852109283208847, + 0.20847393572330475, + -0.049645811319351196, + -0.20558671653270721, + 0.7042896747589111, + -0.2510814964771271, + 0.4876273572444916, + -0.12039870023727417, + -0.00768355093896389, + 0.11149121820926666, + 0.1688014715909958, + 0.15713095664978027, + -1.6489306688308716, + 0.6193579435348511, + 0.02712581679224968, + 0.024227909743785858, + -0.22401832044124603, + 0.08762368559837341, + -0.3901628255844116, + -1.922485113143921, + -0.23640349507331848 + ], + [ + 0.2828711271286011, + 0.2225273698568344, + -0.09926721453666687, + -0.31204161047935486, + 0.674272894859314, + -0.2707802653312683, + 0.26839739084243774, + -0.15598011016845703, + -0.02566390298306942, + 0.11355242878198624, + 0.2185007929801941, + 0.0926683098077774, + -1.4880703687667847, + 0.5231723189353943, + 0.03246282413601875, + 0.0991555005311966, + -0.09589745104312897, + -0.012649968266487122, + -0.3544027805328369, + -1.788517713546753, + -0.20161205530166626 + ], + [ + 0.15808746218681335, + 0.08072054386138916, + -0.12960389256477356, + -0.2147287279367447, + 0.63108229637146, + -0.02273125946521759, + 0.5305259823799133, + -0.21009698510169983, + 0.014720534905791283, + 0.16780930757522583, + 0.1487010419368744, + -0.1937374770641327, + -1.485355257987976, + 0.6777828335762024, + -0.031599801033735275, + 0.0647251307964325, + -0.3129230737686157, + 0.018576815724372864, + -0.6107969880104065, + -1.9060275554656982, + -0.17941057682037354 + ] + ], + "generated_utilities": [ + 0.19381189346313477, + 0.23963995277881622, + 0.21024136245250702, + 0.13467177748680115, + 0.2017955482006073, + 0.20696409046649933, + 0.1936945617198944, + 0.21807821094989777 + ], + "group_id": "ms-d1960368ef76bbb10669", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.842734158039093, + 2.0, + 0.1342049241065979, + 0.18085119128227234, + 0.5091985464096069, + 0.016112878918647766, + 0.5080842971801758, + 0.19944033026695251, + 0.5132148265838623, + 0.5265648365020752, + 0.11437654495239258, + 0.04028162360191345, + 0.04675377905368805, + 0.13603349030017853, + 0.12877747416496277, + 0.05102705955505371 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2770041823387146, + 0.08894228935241699, + 0.07033950090408325, + 0.12426833808422089, + 0.16520383954048157, + 0.023725181818008423, + 0.0, + -0.25569218397140503, + -0.057803913950920105, + 0.06703128665685654, + 0.1732010841369629, + 0.07705201953649521, + -1.0374658107757568, + 0.0, + 0.04586680978536606, + 0.028830496594309807, + -0.0374501496553421, + 0.10374182462692261, + -0.007100850343704224, + -1.7975506782531738, + 0.0 + ], + [ + -0.4268343448638916, + 0.13483470678329468, + -0.06075429916381836, + 0.19817759096622467, + 0.09922409057617188, + 0.6547578573226929, + 0.0, + 0.15026617050170898, + 0.002472013235092163, + -0.31349992752075195, + 0.5119487047195435, + 0.571077287197113, + -1.0374658107757568, + 0.0, + -0.5828977823257446, + -0.20166385173797607, + -0.7097158432006836, + 0.8187870979309082, + -0.0694284737110138, + -1.7975506782531738, + 0.0 + ], + [ + -0.23462048172950745, + 1.088942289352417, + -0.5585125088691711, + 0.10042184591293335, + 0.4514904320240021, + -0.12210102379322052, + 2.176525354385376, + -0.10348781198263168, + 0.19529865682125092, + 0.12132670730352402, + 0.7056414484977722, + 0.12544797360897064, + -2.037465810775757, + 0.6289246678352356, + -0.008351176977157593, + 0.0010435802396386862, + 0.017314812168478966, + 0.6334809064865112, + -0.3720659911632538, + -2.797550678253174, + -1.5395829677581787 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.353050708770752, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.2578493356704712, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0791659355163574 + ], + [ + -0.09280626475811005, + 1.1662368774414062, + -0.8228501081466675, + 0.24497312307357788, + 0.5648186802864075, + -0.03522863984107971, + 2.259786367416382, + 0.14498579502105713, + 0.2030920535326004, + 0.39727938175201416, + 0.8023015856742859, + 0.3385764956474304, + -1.8738088607788086, + 0.7300182580947876, + 0.20887598395347595, + 0.23107871413230896, + 0.3028872311115265, + 0.7558343410491943, + -0.5975744724273682, + -3.0329887866973877, + -1.8127071857452393 + ], + [ + -0.13783538341522217, + 0.9487118721008301, + -0.3492928743362427, + -0.2956767678260803, + 0.2793389856815338, + -0.16702969372272491, + 2.558225631713867, + 0.17572781443595886, + 0.570004940032959, + 0.27895650267601013, + 0.6778483986854553, + -0.27156615257263184, + -1.8747575283050537, + 1.0240070819854736, + 0.27303546667099, + 0.33387133479118347, + 0.1442956030368805, + 0.24661725759506226, + -0.5561860799789429, + -2.9813332557678223, + -1.833754062652588 + ], + [ + -0.18553051352500916, + 1.0909820795059204, + -0.27226096391677856, + 0.1659364104270935, + 0.7108361721038818, + -0.27768880128860474, + 2.239938974380493, + 0.019483201205730438, + 0.21263465285301208, + -0.09839647263288498, + 0.9676799178123474, + -0.36985379457473755, + -2.133316993713379, + 0.3758644461631775, + -0.47750571370124817, + 0.17470555007457733, + -0.17035381495952606, + 0.2707958221435547, + -0.5672953128814697, + -2.394361972808838, + -1.668647289276123 + ], + [ + -0.3557406961917877, + 0.9564932584762573, + -1.0014276504516602, + -0.38222843408584595, + 0.41349664330482483, + -0.05637756735086441, + 1.6011608839035034, + -0.31295540928840637, + -0.15234358608722687, + 0.4648304879665375, + 0.2744198739528656, + -0.44805359840393066, + -2.1544065475463867, + 1.166503667831421, + 0.014861157163977623, + -0.23438170552253723, + 0.1278134286403656, + 0.5812121033668518, + -0.7571046352386475, + -2.5030438899993896, + -2.100955009460449 + ], + [ + -0.16500982642173767, + 0.7556273937225342, + -0.988593339920044, + 0.6213532090187073, + 0.15840065479278564, + 0.5245341062545776, + 2.452624797821045, + -0.5693200826644897, + -0.32241708040237427, + 0.49458175897598267, + 1.3341540098190308, + 0.059507817029953, + -2.304706335067749, + 0.7334492206573486, + -0.20702646672725677, + 0.32367274165153503, + -0.5396079421043396, + 0.9809319972991943, + -0.6172815561294556, + -3.197409152984619, + -0.9358728528022766 + ], + [ + -0.07666900753974915, + 1.8442692756652832, + -0.6603910326957703, + -0.40882760286331177, + 0.22092406451702118, + -0.23551997542381287, + 1.6105436086654663, + 0.09144749492406845, + 0.8141599893569946, + 0.8528898358345032, + 1.0663814544677734, + 0.3283998966217041, + -1.7124712467193604, + 1.2278027534484863, + 0.7039424180984497, + 0.3645748496055603, + -0.2968631386756897, + 0.4998055696487427, + -0.9362069368362427, + -3.461055278778076, + -0.8184676766395569 + ], + [ + -0.1520652174949646, + 1.405963659286499, + -0.7959750294685364, + -0.39229461550712585, + 0.46590372920036316, + -0.8267906904220581, + 2.952258586883545, + 0.5923218727111816, + -0.6996292471885681, + 0.8352785706520081, + -0.15220212936401367, + -0.1307106763124466, + -2.849807024002075, + 0.4983477294445038, + 0.5902999639511108, + -0.5390225648880005, + 0.13897128403186798, + 0.21545273065567017, + -1.1186240911483765, + -3.1762759685516357, + -1.0118725299835205 + ], + [ + -0.564094066619873, + 0.4751617908477783, + -1.3977731466293335, + 0.8189605474472046, + 0.17298781871795654, + 0.2476775199174881, + 2.8234164714813232, + -0.4518245458602905, + -0.7904537916183472, + -0.39086827635765076, + -0.05634337663650513, + -0.09099806845188141, + -1.9690911769866943, + 1.1869969367980957, + -0.5615661144256592, + 0.32865890860557556, + -0.8041903376579285, + 1.3324832916259766, + -0.6190662384033203, + -2.4903852939605713, + -2.3098156452178955 + ], + [ + -0.2328718602657318, + 0.9724385142326355, + -1.5535115003585815, + 0.7688977718353271, + 0.4409410357475281, + -0.8374075889587402, + 1.2719230651855469, + 0.653096616268158, + -0.2887840270996094, + 0.4892227351665497, + 0.9943453669548035, + 0.5566720962524414, + -2.496413469314575, + 0.6299278140068054, + -0.08689186722040176, + 0.024782167747616768, + 0.5747005939483643, + 0.023068726062774658, + -1.111369252204895, + -2.187150001525879, + -1.2047184705734253 + ], + [ + -1.1679872274398804, + 1.9997228384017944, + -0.45194220542907715, + -0.8861345648765564, + -0.5369991064071655, + 0.46434593200683594, + 2.175499439239502, + 0.7599186301231384, + 0.38914406299591064, + 0.6877046823501587, + -0.004208564758300781, + -0.1321776658296585, + -1.3474420309066772, + 0.8496907353401184, + 0.4218248426914215, + -0.9354817271232605, + 0.945868730545044, + 1.60379958152771, + -1.195396065711975, + -3.597724676132202, + -1.9041136503219604 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04476936161518097, + 0.08894228935241699, + 0.06436485052108765, + -0.0775555968284607, + 0.04825553297996521, + -0.060782790184020996, + 0.0, + -0.07688627392053604, + -0.06308497488498688, + -0.03861912339925766, + 0.1235572099685669, + -0.0017572194337844849, + -1.0374658107757568, + 0.0, + 0.03638586029410362, + 0.0556478351354599, + 0.0387292355298996, + 0.044096171855926514, + 0.029627740383148193, + -1.7975506782531738, + 0.0 + ] + ], + "predicted_scores": [ + 4.789662837982178, + 4.267895698547363, + 4.223406791687012, + 3.989636182785034, + 3.9498660564422607, + 3.7960212230682373, + 3.673099994659424, + 2.7361390590667725 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "416183106/train2", + "selected_index": 0, + "state_hash": "cd187560c1c39db9e6bf7d9a611a8b7108dc88766de58fd62b4317f0bbcc1b50", + "stored_base_utility": 0.842734158039093, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8350887894630432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8350887894630432 + }, + "base_progress": 0.8350887894630432, + "base_success": false, + "base_utility": 0.8350887894630432, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.865778386592865, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.865778386592865 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1467595398426056, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1467595398426056 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.865778386592865, + 1.0, + 1.0, + 1.0, + 1.0, + 0.1467595398426056, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-5dfbb16e173e1a3e2bd6", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-efdc07c985935d871adf", + "generated_tangents": [ + [ + -0.05860698223114014, + 0.04369671642780304, + -0.0003642439842224121, + 0.16397789120674133, + -0.010445520281791687, + -1.7564449310302734, + -0.18920999765396118, + 0.04669620096683502, + 0.004963694605976343, + 0.08446834981441498, + 0.2735006511211395, + 0.3326651453971863, + -1.5656996965408325, + -0.11894083023071289, + 0.04678551107645035, + 0.05955914035439491, + 0.03427165746688843, + -0.050642624497413635, + 0.012973636388778687, + -1.1163395643234253, + -0.12019971013069153 + ], + [ + 0.13518911600112915, + 0.0794111043214798, + -0.020963460206985474, + -0.00895296037197113, + 0.027858629822731018, + -1.8182843923568726, + -0.10103672742843628, + 0.19927749037742615, + -0.049562275409698486, + 0.0963386595249176, + 0.22088587284088135, + 0.27336210012435913, + -1.3436905145645142, + -0.1528788059949875, + 0.01853266730904579, + 0.046957455575466156, + 0.17621290683746338, + 0.0900363102555275, + 0.1809384971857071, + -1.142998456954956, + -0.09620139002799988 + ], + [ + 0.10362868010997772, + 0.038398973643779755, + -0.010404139757156372, + 0.01275297999382019, + 0.04852853715419769, + -1.8492344617843628, + -0.14889788627624512, + 0.1451866328716278, + -0.021449225023388863, + 0.09165805578231812, + 0.17682726681232452, + 0.2057131826877594, + -1.4815155267715454, + -0.2417202740907669, + 0.043272290378808975, + 0.06020967289805412, + 0.17743179202079773, + -0.04293885827064514, + 0.056448474526405334, + -1.2009186744689941, + -0.08430451154708862 + ], + [ + 0.11029098182916641, + 0.014361292123794556, + -0.06367388367652893, + 0.09728631377220154, + 0.09020566940307617, + -1.7351406812667847, + -0.16312217712402344, + 0.14239948987960815, + 0.003564263693988323, + 0.10082831233739853, + 0.22786083817481995, + 0.27086347341537476, + -1.3380085229873657, + -0.2649409770965576, + 0.054218970239162445, + 0.09831023216247559, + 0.20629757642745972, + -0.03452478349208832, + 0.0876840353012085, + -1.101557970046997, + -0.23052719235420227 + ], + [ + -0.01963682472705841, + 0.11445944011211395, + -0.02500179409980774, + -0.0197308212518692, + 0.08328165113925934, + -1.2299985885620117, + -0.5541284680366516, + 0.4182572364807129, + 0.037195004522800446, + 0.15475031733512878, + 0.3062090575695038, + 0.23689785599708557, + -2.347663640975952, + -0.1468450278043747, + -0.16971290111541748, + 0.0723450630903244, + 0.2985181212425232, + -0.12749190628528595, + 0.26839175820350647, + -0.9119347333908081, + -0.5532509088516235 + ], + [ + -0.1897812932729721, + -0.06167051941156387, + 0.16612344980239868, + -0.05314287543296814, + 0.20351320505142212, + -1.6314184665679932, + 0.006596177816390991, + 0.17027737200260162, + -0.11343333125114441, + 0.06522493064403534, + 0.27704447507858276, + 0.2916208505630493, + -1.7608485221862793, + 0.031241923570632935, + 0.06306252628564835, + 0.05768124386668205, + 0.33894219994544983, + -0.054138943552970886, + -0.17477935552597046, + -1.081310749053955, + 0.053815118968486786 + ], + [ + 0.030201315879821777, + 0.14871327579021454, + -0.10771158337593079, + -0.0532061904668808, + 0.16459324955940247, + -1.67531156539917, + -0.8611230254173279, + 0.3317149877548218, + -0.18816863000392914, + -0.21422088146209717, + 0.3490787148475647, + 0.6242982149124146, + -1.4872329235076904, + -0.04529920220375061, + -0.030703160911798477, + 0.06164492294192314, + 0.11357485502958298, + 0.11759739369153976, + 0.3729802072048187, + -1.235586166381836, + -0.6288784742355347 + ], + [ + -0.04409302771091461, + -0.029062695801258087, + -0.11780580878257751, + 0.06878486275672913, + 0.07232259213924408, + -1.6242724657058716, + -0.3504095673561096, + 0.18393154442310333, + -0.0379820317029953, + 0.28007423877716064, + 0.4622671902179718, + 0.3136138319969177, + -1.6597620248794556, + -0.20851437747478485, + 0.033345915377140045, + 0.11675897240638733, + 0.45757216215133667, + -0.15742127597332, + 0.10456898808479309, + -1.3649070262908936, + -0.32653769850730896 + ] + ], + "generated_utilities": [ + 2.0, + 0.865778386592865, + 2.0, + 2.0, + 2.0, + 2.0, + 0.1467595398426056, + 2.0 + ], + "group_id": "ms-efdc07c985935d871adf", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8197344541549683, + 2.0, + 0.8125, + 0.8125142455101013, + 0.7847581505775452, + 0.7794750332832336, + 0.5920823216438293, + 0.5697357058525085, + 2.0, + 0.5847549438476562, + 0.09384849667549133, + 0.07791829109191895, + 0.011395931243896484, + 0.09286168217658997, + 0.10894183814525604, + 0.0616525262594223 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2827143669128418, + -0.12206736207008362, + 0.1543778032064438, + 0.27268126606941223, + -0.17288367450237274, + -2.5338973999023438, + -1.7615662813186646, + -0.19161763787269592, + 0.1886080503463745, + 0.31686434149742126, + 0.5112922191619873, + 0.14046527445316315, + -2.17134952545166, + -1.8021173477172852, + 0.11115366220474243, + 0.25490811467170715, + -0.025668112561106682, + 0.4787765145301819, + 0.46609628200531006, + -1.711870551109314, + -1.5312864780426025 + ], + [ + 0.03046044334769249, + 0.042622197419404984, + -0.12103251367807388, + 0.7814357876777649, + 0.5093902349472046, + -2.648094654083252, + -1.5764049291610718, + -0.16382060945034027, + 0.15227201581001282, + -0.29746392369270325, + 0.394713819026947, + 0.044284895062446594, + -2.150869131088257, + -1.853789210319519, + 0.24474957585334778, + 0.16074210405349731, + -0.30960795283317566, + 0.986261248588562, + 0.4512072801589966, + -1.9782602787017822, + -1.814382553100586 + ], + [ + -0.47424501180648804, + 0.4193490147590637, + 0.31025299429893494, + 0.436926007270813, + 0.07080172002315521, + -2.2750933170318604, + -2.139326572418213, + 0.19072748720645905, + -0.05066118389368057, + -0.38944289088249207, + 0.11913672089576721, + -0.4728909432888031, + -2.630631446838379, + -1.9070930480957031, + 0.5720394849777222, + 0.1898895502090454, + -0.0872373953461647, + 0.3081148862838745, + 0.6700773239135742, + -1.8990029096603394, + -1.655537486076355 + ], + [ + -0.112115278840065, + -0.5390006899833679, + -0.27749133110046387, + -0.08230990171432495, + 0.18304404616355896, + -2.5003390312194824, + -1.177594542503357, + -0.37152859568595886, + 0.1827065348625183, + -0.1805598884820938, + 0.8884490728378296, + 0.3141748607158661, + -2.7738473415374756, + -1.8585447072982788, + -0.6611859798431396, + 0.5365979075431824, + 0.024639926850795746, + 1.0166820287704468, + 0.6261310577392578, + -1.3820469379425049, + -1.1859028339385986 + ], + [ + -0.013153300620615482, + 0.6059781908988953, + -0.7563084363937378, + 0.6436030864715576, + 0.4828413128852844, + -1.8755569458007812, + -1.896214246749878, + -0.445474773645401, + -0.6578570604324341, + -0.21610267460346222, + 1.3274874687194824, + -0.6251860857009888, + -2.005418300628662, + -1.463393211364746, + -0.40788042545318604, + -0.5932772159576416, + -0.6501284837722778, + 0.1491221785545349, + 0.8418278694152832, + -2.578186511993408, + -1.628711462020874 + ], + [ + -0.051258739084005356, + 0.6121965050697327, + -0.4303005337715149, + 0.378423810005188, + -0.4187525510787964, + -2.0054633617401123, + -1.5839252471923828, + 0.38306933641433716, + -0.8548648953437805, + 0.5034631490707397, + 0.3682137727737427, + 0.867935061454773, + -2.5900886058807373, + -1.8283436298370361, + 0.3810080289840698, + -0.6684208512306213, + 0.5362142324447632, + -0.17851078510284424, + 1.083348035812378, + -1.9181195497512817, + -2.306365728378296 + ], + [ + 0.5487861037254333, + 0.6947635412216187, + -0.3694673776626587, + 1.3571128845214844, + -0.20053730905056, + -2.9099156856536865, + -1.493881106376648, + 0.9321863055229187, + -0.355974406003952, + 0.6561991572380066, + -0.29231804609298706, + 0.4913230240345001, + -1.8409770727157593, + -2.0504229068756104, + 0.14528872072696686, + 0.483799010515213, + -0.32654285430908203, + 1.6123273372650146, + 0.03987070918083191, + -0.8449068665504456, + -2.3494603633880615 + ], + [ + -0.3259732127189636, + -0.5385614633560181, + 0.7304226160049438, + -0.12314724922180176, + 0.03945803642272949, + -1.5919837951660156, + -1.907224416732788, + 0.3930681645870209, + -0.8835994005203247, + -0.3352096974849701, + 0.17923185229301453, + 0.17222784459590912, + -2.512176752090454, + -1.859424352645874, + 0.536429762840271, + 0.11454571038484573, + 0.3300909101963043, + 0.5491204261779785, + -0.04762554168701172, + -1.1976318359375, + -0.5251960158348083 + ], + [ + 0.33097749948501587, + -0.04192434251308441, + -0.34520047903060913, + -0.04110339283943176, + -0.5535940527915955, + -2.2057979106903076, + -2.5934860706329346, + -0.9023091793060303, + 0.5658158659934998, + -0.6125177145004272, + 1.095580816268921, + -0.05609257519245148, + -1.606997013092041, + -2.0733447074890137, + -0.5881220102310181, + 0.8953573703765869, + 0.9258373975753784, + 0.27281180024147034, + 0.7141706943511963, + -0.9142226576805115, + -1.3141751289367676 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05978316813707352, + 0.057473838329315186, + -0.09624473005533218, + -0.011721253395080566, + 0.028141841292381287, + -1.4311916828155518, + 0.0, + -0.024008113890886307, + 0.07049623876810074, + 0.03528204560279846, + -0.010002851486206055, + 0.010910652577877045, + -1.1452367305755615, + 0.0, + 0.020785929635167122, + -0.03374411538243294, + 0.11616832762956619, + 0.05772918462753296, + 0.01671847701072693, + -0.828272819519043, + 0.0 + ], + [ + -0.13237658143043518, + -0.3393171727657318, + 0.29588890075683594, + 0.8627858757972717, + -0.1666169911623001, + -2.520320177078247, + -1.6265922784805298, + -0.031582750380039215, + 0.11019003391265869, + -0.2699315845966339, + 0.9513242244720459, + 0.15523536503314972, + -1.7896702289581299, + -1.2776744365692139, + -0.3942914605140686, + -0.30590808391571045, + -0.3098926246166229, + 0.8258905410766602, + 0.29909753799438477, + -1.5711073875427246, + -1.5744072198867798 + ] + ], + "predicted_scores": [ + 4.8488054275512695, + 4.764893531799316, + 4.676024436950684, + 4.401856422424316, + 4.025679111480713, + 3.68040132522583, + 3.5672194957733154, + 2.0043821334838867 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "706479600/train2", + "selected_index": 0, + "state_hash": "b482f902b03ca480898798a8b5f19263d7cbb18c066305330b0796b607427fa7", + "stored_base_utility": 0.8197344541549683, + "task_id": "StackCube-v1", + "train_seed": "2" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03719887137413025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03719887137413025 + }, + "base_progress": 0.03719887137413025, + "base_success": false, + "base_utility": 0.03719887137413025, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.029491469264030457, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029491469264030457 + }, + { + "contact_quality": null, + "progress": 0.0387403666973114, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0387403666973114 + }, + { + "contact_quality": null, + "progress": 0.04824826121330261, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04824826121330261 + }, + { + "contact_quality": null, + "progress": 0.0322902649641037, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0322902649641037 + }, + { + "contact_quality": null, + "progress": 0.02138236165046692, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02138236165046692 + }, + { + "contact_quality": null, + "progress": 0.021242722868919373, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021242722868919373 + }, + { + "contact_quality": null, + "progress": 0.04859863221645355, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04859863221645355 + }, + { + "contact_quality": null, + "progress": 0.03606592118740082, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03606592118740082 + } + ], + "candidate_progress": [ + 0.029491469264030457, + 0.0387403666973114, + 0.04824826121330261, + 0.0322902649641037, + 0.02138236165046692, + 0.021242722868919373, + 0.04859863221645355, + 0.03606592118740082 + ], + "candidate_source_chart_ids": [ + "ms-bb299baa34e49eafb9fc", + "ms-ee2fcbc94d41385713d0", + "ms-0b1dd0e538b8df15a6e4", + "ms-07b479748a91a9c92b20", + "ms-d4c5d7eea19d94314e2e", + "ms-0a4d4909900fb0bc4a9a", + "ms-2fef2f43af8009bb88f1", + "ms-5ac1bf2d7d319955d45d" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank6", + "ctt_transport_rank7", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "generated_tangents": [ + [ + 0.0818156823515892, + 0.7376402616500854, + 0.260955810546875, + -1.8874306678771973, + 0.11456064134836197, + 0.6268888711929321, + -0.3406338095664978, + -0.10714408755302429, + -0.3122781217098236, + 0.12723010778427124, + 1.0230602025985718, + 0.26463550329208374, + -1.9134429693222046, + -0.24195240437984467, + -0.12432089447975159, + -0.005063745658844709, + 0.22826595604419708, + -0.13979198038578033, + -0.04610705375671387, + -1.7523059844970703, + -0.07397112250328064 + ], + [ + -0.018952831625938416, + 0.783510684967041, + 0.12759295105934143, + -1.827862024307251, + 0.27751418948173523, + 0.7717159986495972, + -0.12066727876663208, + -0.17187073826789856, + -0.33216238021850586, + 0.12718318402767181, + 1.0422388315200806, + 0.3275884985923767, + -1.8554681539535522, + -0.3043630123138428, + -0.14465337991714478, + -0.01591307297348976, + 0.21231937408447266, + -0.17680726945400238, + -0.012129753828048706, + -1.9596967697143555, + -0.0648861825466156 + ], + [ + 0.025759495794773102, + 0.799794614315033, + 0.22568532824516296, + -1.8786691427230835, + 0.3682592511177063, + 0.43615004420280457, + -0.12424099445343018, + -0.19355234503746033, + -0.3693811893463135, + 0.037883155047893524, + 1.0797016620635986, + 0.2897043824195862, + -1.912314772605896, + -0.3106462359428406, + -0.10004118084907532, + -0.014992143958806992, + 0.15777814388275146, + -0.15011344850063324, + -0.1049012541770935, + -1.7705650329589844, + -0.1936914622783661 + ], + [ + 0.12796276807785034, + 0.7324540019035339, + 0.12481063604354858, + -1.946326494216919, + 0.21629221737384796, + 0.688870906829834, + -0.07897979021072388, + -0.19743359088897705, + -0.3120637536048889, + 0.1287928968667984, + 1.02448308467865, + 0.41274213790893555, + -1.6761292219161987, + -0.2994609475135803, + -0.15494367480278015, + 0.00253965612500906, + 0.21550241112709045, + -0.15498657524585724, + -0.052129149436950684, + -1.7956414222717285, + -0.056675300002098083 + ], + [ + 0.21264928579330444, + 0.6281499266624451, + 0.005421280860900879, + -2.297286033630371, + 0.22898004949092865, + 0.9802001714706421, + 0.04851001501083374, + -0.22916251420974731, + -0.2755090594291687, + 0.2792716920375824, + 1.0580480098724365, + 0.11975810676813126, + -1.6677312850952148, + -0.07127648591995239, + -0.15530571341514587, + 0.06525619328022003, + 0.15787893533706665, + -0.04296962916851044, + 0.005149960517883301, + -1.6606080532073975, + -0.1715010702610016 + ], + [ + 0.17323878407478333, + 0.6328699588775635, + 0.22087153792381287, + -2.0211126804351807, + 0.09133979678153992, + 0.2522924542427063, + -0.2608356475830078, + -0.02745489776134491, + -0.29795435070991516, + 0.19968056678771973, + 1.178425908088684, + 0.3355984687805176, + -1.8080487251281738, + -0.035159483551979065, + -0.2646137773990631, + 0.11011427640914917, + 0.23902000486850739, + 0.05524830520153046, + -0.16241887211799622, + -1.4475040435791016, + 0.16269637644290924 + ], + [ + 0.09056825935840607, + 0.7814904451370239, + 0.25533759593963623, + -1.9061659574508667, + 0.1476134955883026, + 0.7096577882766724, + -0.0997546911239624, + -0.24902793765068054, + -0.3023238182067871, + 0.14217494428157806, + 1.0836024284362793, + 0.33638328313827515, + -1.959166407585144, + -0.30432748794555664, + -0.14443287253379822, + 0.03149983659386635, + 0.20935511589050293, + -0.16482730209827423, + -0.08840945363044739, + -1.9092836380004883, + -0.09851974248886108 + ], + [ + -0.09150759875774384, + 0.8264052867889404, + 0.12514424324035645, + -1.967693567276001, + 0.16516178846359253, + 0.7895872592926025, + -0.1622459888458252, + -0.29067984223365784, + -0.3645361065864563, + 0.04662332683801651, + 1.0962855815887451, + 0.21468932926654816, + -2.3187038898468018, + -0.401155948638916, + -0.17081262171268463, + 0.04716471955180168, + 0.27044185996055603, + -0.22255687415599823, + -0.18857508897781372, + -2.237642765045166, + -0.1585465371608734 + ] + ], + "generated_utilities": [ + 0.029491469264030457, + 0.0387403666973114, + 0.04824826121330261, + 0.0322902649641037, + 0.02138236165046692, + 0.021242722868919373, + 0.04859863221645355, + 0.03606592118740082 + ], + "group_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558897972106934, + 2.0, + 0.04248148202896118, + 0.05394089221954346, + 0.057141318917274475, + 0.03495943546295166, + 0.08633947372436523, + 0.046462297439575195, + 0.043458908796310425, + 0.023027777671813965, + 0.061922311782836914, + 0.034134700894355774, + 0.010651648044586182, + 0.01158277690410614, + 0.02622869610786438, + 0.012083128094673157 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.4298704266548157, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + 0.09995961934328079, + 1.162198781967163, + 0.0, + -0.00648990273475647, + -0.3028862476348877, + -0.0050805360078811646, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + 0.23185759782791138, + -0.11529958993196487, + -0.058992013335227966, + -0.010845988988876343, + 0.06423944234848022, + -2.0730173587799072, + 0.0 + ], + [ + -0.48041507601737976, + 0.6002404689788818, + 0.5398462414741516, + -2.198392152786255, + 0.1505453884601593, + 1.162198781967163, + 0.0, + 0.32028457522392273, + -0.264798641204834, + -0.27968189120292664, + 1.143568515777588, + 0.623285710811615, + -1.6758570671081543, + 0.0, + 0.3762620687484741, + 0.20924779772758484, + 0.033039893954992294, + -0.22763806581497192, + 0.8496624827384949, + -2.0730173587799072, + 0.0 + ], + [ + -0.4385460615158081, + 1.6002404689788818, + 1.1335551738739014, + -3.198392152786255, + 0.11579504609107971, + 2.162198781967163, + -1.8792641162872314, + 0.44874686002731323, + -1.264798641204834, + 0.08758684247732162, + 2.143568515777588, + 1.074564814567566, + -2.6758570671081543, + 1.4318934679031372, + -0.01953941211104393, + 0.012724362313747406, + 0.02134235017001629, + 0.39346790313720703, + -0.15033751726150513, + -3.0730173587799072, + -1.573193907737732 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.758528232574463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8637869358062744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.146387815475464 + ], + [ + -0.4582313299179077, + 1.465004801750183, + 0.9327377080917358, + -2.900977373123169, + -0.12230202555656433, + 2.23574161529541, + -2.1357157230377197, + 0.42294827103614807, + -1.506612777709961, + 0.29363107681274414, + 2.034971237182617, + 1.023054599761963, + -2.559215784072876, + 1.1742838621139526, + 0.026679620146751404, + -0.0811251625418663, + 0.16039283573627472, + 0.3527735471725464, + 0.041033580899238586, + -2.8834660053253174, + -1.2895927429199219 + ], + [ + -0.8261920809745789, + 1.8162921667099, + 1.1455696821212769, + -3.5306012630462646, + -0.15104833245277405, + 2.447983741760254, + -2.128190755844116, + 0.19818204641342163, + -1.146593689918518, + 0.03477605804800987, + 2.4299705028533936, + 0.8797258734703064, + -2.331338882446289, + 1.5516352653503418, + -0.19862954318523407, + -0.2736409604549408, + -0.21770769357681274, + 0.47925806045532227, + -0.15124429762363434, + -3.0052826404571533, + -1.2108670473098755 + ], + [ + -0.2760573625564575, + 1.6254181861877441, + 0.8188510537147522, + -3.0690624713897705, + -0.22635167837142944, + 2.051481246948242, + -1.417292833328247, + 0.026889145374298096, + -1.3347617387771606, + -0.15878626704216003, + 1.9778507947921753, + 0.7640928030014038, + -3.1165366172790527, + 1.862673282623291, + -0.451316773891449, + -0.15708383917808533, + 0.26080578565597534, + 0.6509285569190979, + 0.23991608619689941, + -3.50675892829895, + -1.297291874885559 + ], + [ + -0.8321853280067444, + 1.9553383588790894, + 1.5495907068252563, + -3.790785312652588, + 0.3716217279434204, + 2.5994434356689453, + -2.1380205154418945, + -0.10642725229263306, + -0.7562419772148132, + -0.442053884267807, + 2.3244006633758545, + 1.161316990852356, + -2.699451208114624, + 1.6399294137954712, + 0.31003186106681824, + 0.5301566123962402, + -0.16996291279792786, + 0.9281869530677795, + 0.3683878183364868, + -2.650294780731201, + -1.1098875999450684 + ], + [ + -0.6997023820877075, + 1.4820023775100708, + 1.3613333702087402, + -3.8556365966796875, + -0.5036565065383911, + 1.4896399974822998, + -2.322740077972412, + 0.03597790002822876, + -1.1858596801757812, + -0.018024958670139313, + 2.0489397048950195, + 0.8914362192153931, + -2.3395535945892334, + 1.1835731267929077, + 0.5559698343276978, + 0.5892979502677917, + 0.4496755301952362, + 0.6100684404373169, + 0.22263532876968384, + -3.0098915100097656, + -1.455836296081543 + ], + [ + -0.594323456287384, + 1.846432089805603, + 0.8105549812316895, + -2.6211562156677246, + 0.2629232406616211, + 1.4025108814239502, + -2.3828163146972656, + 0.530456006526947, + -1.3874722719192505, + 0.49274298548698425, + 1.5545053482055664, + 0.4927254915237427, + -3.1490466594696045, + 1.5643255710601807, + 0.5331913232803345, + -0.53307044506073, + 0.7433652281761169, + 0.6546258926391602, + -0.74479740858078, + -3.721141815185547, + -0.9948303699493408 + ], + [ + -0.4409070312976837, + 2.4138050079345703, + 1.3223544359207153, + -3.3717033863067627, + -0.03033369779586792, + 2.350304365158081, + -2.112061023712158, + -0.22452247142791748, + -1.420605182647705, + -0.026168949902057648, + 1.4026775360107422, + 0.9532885551452637, + -2.6276464462280273, + 2.1231069564819336, + -0.49286600947380066, + -0.3450928032398224, + 0.5427718758583069, + -0.03872644901275635, + -0.6895387172698975, + -2.357243061065674, + -2.3816022872924805 + ], + [ + 0.5156354308128357, + 2.2980997562408447, + 2.0945849418640137, + -3.690303087234497, + 1.0696191787719727, + 2.266188144683838, + -1.0399069786071777, + 0.012727499008178711, + -1.2163764238357544, + 0.5646419525146484, + 1.1664416790008545, + 1.563025951385498, + -3.425304412841797, + 0.5126914978027344, + -0.18885818123817444, + -0.5277795791625977, + -0.1741226613521576, + 0.2763793468475342, + 0.8411282896995544, + -3.2598915100097656, + -2.2270073890686035 + ], + [ + -1.2896747589111328, + 2.3828718662261963, + 1.191500186920166, + -3.9747776985168457, + 0.7604506015777588, + 2.298020839691162, + -1.0116384029388428, + 1.177309513092041, + -1.6719520092010498, + -0.4176469147205353, + 2.709542989730835, + 0.26378321647644043, + -2.119570255279541, + 0.7668893933296204, + 0.6772738099098206, + -0.5338667035102844, + -0.7434203028678894, + 0.2167283445596695, + 0.03269544243812561, + -3.895913600921631, + -0.9124676585197449 + ], + [ + -1.323930263519287, + 2.338547468185425, + 1.4313098192214966, + -2.960175037384033, + 1.1079072952270508, + 1.5746114253997803, + -1.8946435451507568, + -0.4213535189628601, + -1.922096848487854, + 0.6645004153251648, + 2.1448850631713867, + 0.2334994077682495, + -3.4603066444396973, + 1.5447293519973755, + 0.25094738602638245, + 0.7163193821907043, + -0.10547596216201782, + -0.42469942569732666, + -0.49689921736717224, + -2.8712027072906494, + -1.6389087438583374 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027082443237304688, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + -0.03311736881732941, + 1.162198781967163, + 0.0, + 0.043807268142700195, + -0.264798641204834, + 0.0069312527775764465, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + -0.01150114368647337, + -0.007074322551488876, + -0.018292216584086418, + -0.035625845193862915, + 0.05120308697223663, + -2.0730173587799072, + 0.0 + ] + ], + "predicted_scores": [ + 6.205906391143799, + 5.960236072540283, + 5.870675563812256, + 5.737954616546631, + 5.6567230224609375, + 5.656543254852295, + 5.328103542327881, + 4.695163726806641 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1919198188/train2", + "selected_index": 0, + "state_hash": "6b9e4c034c70f8e93116064dbcb48aa5fea37958e04fa30734f597c38aefea8b", + "stored_base_utility": 0.8558897972106934, + "task_id": "StackCube-v1", + "train_seed": "2" + } + ], + "run_dirs": [ + "runs/ctt_residual_base_context_obj_rollout_val69_seed0", + "runs/ctt_residual_base_context_obj_rollout_val69_seed1", + "runs/ctt_residual_base_context_obj_rollout_val69_seed2" + ], + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381", + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381", + "train_seeds": [ + "0", + "1", + "2" + ] +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/command.txt b/runs/ctt_base_context_obj_val_rollout_comparison/command.txt new file mode 100644 index 0000000000000000000000000000000000000000..f77b0a7a15b998355a6fb2e797e0a1f77f4208ff --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/command.txt @@ -0,0 +1 @@ +python scripts/build_ctt_rollout_comparison.py --run-glob runs/ctt_residual_base_context_obj_rollout_val69_seed* --out-dir runs/ctt_base_context_obj_val_rollout_comparison --k 8 diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/data_hash.txt b/runs/ctt_base_context_obj_val_rollout_comparison/data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..49b12b50537d95327f9c62ddd41910531a6c14a4 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/data_hash.txt @@ -0,0 +1 @@ +1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708 diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/eval.log b/runs/ctt_base_context_obj_val_rollout_comparison/eval.log new file mode 100644 index 0000000000000000000000000000000000000000..37c506302a057a3482cf2f7434b55ca6e30bb60c --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/eval.log @@ -0,0 +1,3 @@ +runs/ctt_residual_base_context_obj_rollout_val69_seed0: measured_candidates.json +runs/ctt_residual_base_context_obj_rollout_val69_seed1: measured_candidates.json +runs/ctt_residual_base_context_obj_rollout_val69_seed2: measured_candidates.json diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/git_hash.txt b/runs/ctt_base_context_obj_val_rollout_comparison/git_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..293c6ad4e775f2f937dca574f1b83a5f93f0bdb1 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/git_hash.txt @@ -0,0 +1 @@ +14e81292e2004925f81e0bb4810f6f144bfd87b7 diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics.json b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..0d23ee2398c2236dfbd66a3e800bd99c9340bf26 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics.json @@ -0,0 +1,9467 @@ +{ + "epsilon": 0.0, + "k": 8, + "mode": "measured", + "num_rows": 207, + "rows": [ + { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 1.2770342230796814, + "chart_id": "ms-484ea653bbca2de3a4a2", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.19966589920839098, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "seed": "1231780997/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7229657769203186, + "selected_utility_gain_over_base_at_8": -0.004040360450744629, + "selector_regret_at_8": 1.2770342230796814, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2770342230796814 + }, + { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 1.281606376171112, + "chart_id": "ms-4b4cd12d70c0615e0930", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.20050368294022264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.471626341342926, + "seed": "1025740212/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7183936238288879, + "selected_utility_gain_over_base_at_8": 0.19001996517181396, + "selector_regret_at_8": 1.281606376171112, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.281606376171112 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2853952050209045, + "chart_id": "ms-6e268d1e2915740816a1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.41171052461838353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1475440973/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7146047949790955, + "selected_utility_gain_over_base_at_8": -1.2853952050209045, + "selector_regret_at_8": 1.2853952050209045, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2853952050209045 + }, + { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2800043225288391, + "chart_id": "ms-8864796e31bbe93d7c5d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3791036296516339, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "seed": "551504635/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7199956774711609, + "selected_utility_gain_over_base_at_8": -0.003976881504058838, + "selector_regret_at_8": 1.2800043225288391, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2800043225288391 + }, + { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.11861386895179749, + "chart_id": "ms-b04b489f4fbad93b65ca", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3774241061869338, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5588552355766296, + "proposal_oracle_utility_gain_over_base_at_8": 0.13654270768165588, + "seed": "1113268940/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.44024136662483215, + "selected_utility_gain_over_base_at_8": 0.0179288387298584, + "selector_regret_at_8": 0.11861386895179749, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4411447644233704, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.5597586333751678 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c663f7e73e7e8376a05c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15436827657008492, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1528740417/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "seed": "1273247072/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "task_id": "PickCube-v1", + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + { + "base_success": 0.0, + "base_utility": 0.050487834960222244, + "branch_car_at_8": 0.16002291440963745, + "chart_id": "ms-0345d8ac530f88cefd61", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1798098385334015, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35253205571641333, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1601850390434265, + "proposal_oracle_utility_gain_over_base_at_8": 0.10969720408320427, + "seed": "141984615/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0001621246337890625, + "selected_utility_gain_over_base_at_8": -0.05032571032643318, + "selector_regret_at_8": 0.16002291440963745, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.019624799489974976, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17964771389961243 + }, + { + "base_success": 0.0, + "base_utility": 0.07743936777114868, + "branch_car_at_8": 0.0009531378746032715, + "chart_id": "ms-0819eb89e627595e7e1b", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.14572584629058838, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15574723256687859, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.009828686714172363, + "proposal_oracle_utility_gain_over_base_at_8": -0.06761068105697632, + "seed": "2015956957/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.008875548839569092, + "selected_utility_gain_over_base_at_8": -0.06856381893157959, + "selector_regret_at_8": 0.0009531378746032715, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13589715957641602, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1368502974510193 + }, + { + "base_success": 0.0, + "base_utility": 0.19400137662887573, + "branch_car_at_8": 0.05573397874832153, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3968084572232568, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0932278037071228, + "proposal_oracle_utility_gain_over_base_at_8": -0.10077357292175293, + "seed": "565369392/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03749382495880127, + "selected_utility_gain_over_base_at_8": -0.15650755167007446, + "selector_regret_at_8": 0.05573397874832153, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9067721962928772, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9625061750411987 + }, + { + "base_success": 0.0, + "base_utility": 0.04583283141255379, + "branch_car_at_8": 0.050342779606580734, + "chart_id": "ms-126eb8b36769f9ba4671", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1985912322998047, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1912988221177401, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1050744280219078, + "proposal_oracle_utility_gain_over_base_at_8": 0.05924159660935402, + "seed": "767747424/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05473164841532707, + "selected_utility_gain_over_base_at_8": 0.008898817002773285, + "selector_regret_at_8": 0.050342779606580734, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09351680427789688, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.14385958388447762 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-16661b2fd2c85f57a66c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3285899297359119, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008215903653763235, + "chart_id": "ms-20e1efbb745fd02abc95", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0996910344508803, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0015756288776174188, + "proposal_oracle_utility_gain_over_base_at_8": -1.9984243711223826, + "seed": "1375492786/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0007540385122410953, + "selected_utility_gain_over_base_at_8": -1.999245961487759, + "selector_regret_at_8": 0.0008215903653763235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24187534058000892, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24269693094538525 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-2dc5138251afd26a040d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565621826/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.08252689987421036, + "branch_car_at_8": 0.008593454957008362, + "chart_id": "ms-387e6771a55be98f1a59", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20744429528713226, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.22806100330619278, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10834071785211563, + "proposal_oracle_utility_gain_over_base_at_8": 0.025813817977905273, + "seed": "767747424/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09974726289510727, + "selected_utility_gain_over_base_at_8": 0.01722036302089691, + "selector_regret_at_8": 0.008593454957008362, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09910357743501663, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.107697032392025 + }, + { + "base_success": 0.0, + "base_utility": 0.03917040303349495, + "branch_car_at_8": 0.0, + "chart_id": "ms-38f7298184a2c5005e94", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1767096221446991, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.25037633004492843, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.12086125463247299, + "proposal_oracle_utility_gain_over_base_at_8": 0.08169085159897804, + "seed": "2015956957/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12086125463247299, + "selected_utility_gain_over_base_at_8": 0.08169085159897804, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.055848367512226105, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.055848367512226105 + }, + { + "base_success": 0.0, + "base_utility": 0.009124159812927246, + "branch_car_at_8": 0.002313693054020405, + "chart_id": "ms-482c5c8d208b5907a2c2", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1834145486354828, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09740165004185923, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.011097331531345844, + "proposal_oracle_utility_gain_over_base_at_8": 0.001973171718418598, + "seed": "626486813/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.00878363847732544, + "selected_utility_gain_over_base_at_8": -0.00034052133560180664, + "selector_regret_at_8": 0.002313693054020405, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.17231721710413694, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17463091015815735 + }, + { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.02001962810754776, + "chart_id": "ms-4da03300d92420e80ed0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11661420436902857, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1273784190416336, + "proposal_oracle_utility_gain_over_base_at_8": -0.019252806901931763, + "seed": "1747178030/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10735879093408585, + "selected_utility_gain_over_base_at_8": -0.03927243500947952, + "selector_regret_at_8": 0.02001962810754776, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8726215809583664, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8926412090659142 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-57be9ae181c077e7821f", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.17875755926385148, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.1544450968503952, + "branch_car_at_8": 1.937544982880354, + "chart_id": "ms-6388b5d2356c556c3e55", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07909133535710673, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8455549031496048, + "seed": "2015956957/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06245501711964607, + "selected_utility_gain_over_base_at_8": -0.09199007973074913, + "selector_regret_at_8": 1.937544982880354, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.937544982880354 + }, + { + "base_success": 0.0, + "base_utility": 0.06173701956868172, + "branch_car_at_8": 0.05367838591337204, + "chart_id": "ms-638d2ba1917eff4d09e0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.11176212877035141, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.29235072384279687, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.15528590977191925, + "proposal_oracle_utility_gain_over_base_at_8": 0.09354889020323753, + "seed": "767747424/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10160752385854721, + "selected_utility_gain_over_base_at_8": 0.039870504289865494, + "selector_regret_at_8": 0.05367838591337204, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0101546049118042 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9834009613841772, + "chart_id": "ms-652ad552ba98d2736620", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2367449700832367, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.40752231020077345, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "767747424/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.016599038615822792, + "selected_utility_gain_over_base_at_8": -1.9834009613841772, + "selector_regret_at_8": 1.9834009613841772, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2201459314674139 + }, + { + "base_success": 0.0, + "base_utility": 0.03784912824630737, + "branch_car_at_8": 0.0, + "chart_id": "ms-674fad0e7c042d23629e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.15573063492774963, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.18759978271166325, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10381317138671875, + "proposal_oracle_utility_gain_over_base_at_8": 0.06596404314041138, + "seed": "2015956957/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10381317138671875, + "selected_utility_gain_over_base_at_8": 0.06596404314041138, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.051917463541030884, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.051917463541030884 + }, + { + "base_success": 0.0, + "base_utility": 0.12791956961154938, + "branch_car_at_8": 0.0, + "chart_id": "ms-728b865677134d3788e3", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.22658847272396088, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2562046743306349, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.13394887745380402, + "proposal_oracle_utility_gain_over_base_at_8": 0.006029307842254639, + "seed": "750448296/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.13394887745380402, + "selected_utility_gain_over_base_at_8": 0.006029307842254639, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09263959527015686, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.09263959527015686 + }, + { + "base_success": 0.0, + "base_utility": 0.20529556274414062, + "branch_car_at_8": 0.09963546693325043, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.27989473938941956, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11890836225710137, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.20188765227794647, + "proposal_oracle_utility_gain_over_base_at_8": -0.003407910466194153, + "seed": "565369392/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10225218534469604, + "selected_utility_gain_over_base_at_8": -0.10304337739944458, + "selector_regret_at_8": 0.09963546693325043, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.07800708711147308, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1776425540447235 + }, + { + "base_success": 0.0, + "base_utility": 0.18213583528995514, + "branch_car_at_8": 0.000459273811429739, + "chart_id": "ms-7f57045c114ddad153b5", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.227622389793396, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09636414360460299, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0052528977394104, + "proposal_oracle_utility_gain_over_base_at_8": -0.17688293755054474, + "seed": "278008524/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.004793623927980661, + "selected_utility_gain_over_base_at_8": -0.17734221136197448, + "selector_regret_at_8": 0.000459273811429739, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2223694920539856, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22282876586541533 + }, + { + "base_success": 0.0, + "base_utility": 0.0056487321853637695, + "branch_car_at_8": 0.014695842750370502, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1631626933813095, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5338617369066084, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.025841474533081055, + "proposal_oracle_utility_gain_over_base_at_8": 0.020192742347717285, + "seed": "626486813/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011145631782710552, + "selected_utility_gain_over_base_at_8": 0.005496899597346783, + "selector_regret_at_8": 0.014695842750370502, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13732121884822845, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.15201706159859896 + }, + { + "base_success": 0.0, + "base_utility": 0.00698717450723052, + "branch_car_at_8": 0.08160265535116196, + "chart_id": "ms-8a72b73ac452bc190aeb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.10329639911651611, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.18502306719781839, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.14832614362239838, + "proposal_oracle_utility_gain_over_base_at_8": 0.14133896911516786, + "seed": "626486813/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06672348827123642, + "selected_utility_gain_over_base_at_8": 0.0597363137640059, + "selector_regret_at_8": 0.08160265535116196, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.036572910845279694 + }, + { + "base_success": 0.0, + "base_utility": 0.08285123109817505, + "branch_car_at_8": 0.011325618252158165, + "chart_id": "ms-a96bca1ef0b74f686018", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1572066992521286, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22880234105797714, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.03157500550150871, + "proposal_oracle_utility_gain_over_base_at_8": -0.051276225596666336, + "seed": "141984615/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.020249387249350548, + "selected_utility_gain_over_base_at_8": -0.0626018438488245, + "selector_regret_at_8": 0.011325618252158165, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1256316937506199, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13695731200277805 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-ac4fb33f3ab4db853995", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13163114441189208, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13897453248500824, + "proposal_oracle_utility_gain_over_base_at_8": -1.8610254675149918, + "seed": "565369392/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13897453248500824, + "selected_utility_gain_over_base_at_8": -1.8610254675149918, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8610254675149918, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8610254675149918 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "2054899126/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.037537217140197754, + "branch_car_at_8": 0.0031203627586364746, + "chart_id": "ms-c832439976a37f52d094", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2594742774963379, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.4058728111694587, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.007275819778442383, + "proposal_oracle_utility_gain_over_base_at_8": -0.03026139736175537, + "seed": "585904130/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.004155457019805908, + "selected_utility_gain_over_base_at_8": -0.033381760120391846, + "selector_regret_at_8": 0.0031203627586364746, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2521984577178955, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.255318820476532 + }, + { + "base_success": 0.0, + "base_utility": 0.09138330072164536, + "branch_car_at_8": 0.010135690681636333, + "chart_id": "ms-c96b415f083f60857334", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.26426148414611816, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2433265403791355, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0225119199603796, + "proposal_oracle_utility_gain_over_base_at_8": -0.06887138076126575, + "seed": "585904130/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.012376229278743267, + "selected_utility_gain_over_base_at_8": -0.07900707144290209, + "selector_regret_at_8": 0.010135690681636333, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24174956418573856, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2518852548673749 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18033001516380143, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "534717141/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.14809735119342804, + "branch_car_at_8": 0.0007792116375640035, + "chart_id": "ms-da142025733b53fcf7eb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2643269896507263, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3120068884872385, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0020139615517109632, + "proposal_oracle_utility_gain_over_base_at_8": -0.14608338964171708, + "seed": "141984615/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0012347499141469598, + "selected_utility_gain_over_base_at_8": -0.14686260127928108, + "selector_regret_at_8": 0.0007792116375640035, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.26231302809901536, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.26309223973657936 + }, + { + "base_success": 0.0, + "base_utility": 0.1362423449754715, + "branch_car_at_8": 0.0032713431864976883, + "chart_id": "ms-e2bad6e99e8a281207cc", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20825156569480896, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20447116827433545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.03361199423670769, + "proposal_oracle_utility_gain_over_base_at_8": -0.10263035073876381, + "seed": "1549324805/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03034065105021, + "selected_utility_gain_over_base_at_8": -0.1059016939252615, + "selector_regret_at_8": 0.0032713431864976883, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.17463957145810127, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17791091464459896 + }, + { + "base_success": 0.0, + "base_utility": 0.09695577621459961, + "branch_car_at_8": 0.038976866751909256, + "chart_id": "ms-e30fd80adc089e9f697a", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18920521438121796, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3409869835510214, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07222036272287369, + "proposal_oracle_utility_gain_over_base_at_8": -0.02473541349172592, + "seed": "585904130/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03324349597096443, + "selected_utility_gain_over_base_at_8": -0.06371228024363518, + "selector_regret_at_8": 0.038976866751909256, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11698485165834427, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.15596171841025352 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.036640286445617676, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.17097233660673458, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13109225034713745, + "proposal_oracle_utility_gain_over_base_at_8": -1.8689077496528625, + "seed": "626486813/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.09445196390151978, + "selected_utility_gain_over_base_at_8": -1.9055480360984802, + "selector_regret_at_8": 0.036640286445617676, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8689077496528625, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9055480360984802 + }, + { + "base_success": 0.0, + "base_utility": 0.09514755010604858, + "branch_car_at_8": 0.03174654766917229, + "chart_id": "ms-ec013396e4aff6170ee1", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24502670764923096, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5028853351950527, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08578010648488998, + "proposal_oracle_utility_gain_over_base_at_8": -0.0093674436211586, + "seed": "750448296/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0540335588157177, + "selected_utility_gain_over_base_at_8": -0.04111399129033089, + "selector_regret_at_8": 0.03174654766917229, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15924660116434097, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.19099314883351326 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0048203859478235245, + "chart_id": "ms-ee30ebaf98622759d888", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19331728269593437, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.024462858214974403, + "proposal_oracle_utility_gain_over_base_at_8": -1.9755371417850256, + "seed": "636680137/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.01964247226715088, + "selected_utility_gain_over_base_at_8": -1.9803575277328491, + "selector_regret_at_8": 0.0048203859478235245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755371417850256, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9803575277328491 + }, + { + "base_success": 0.0, + "base_utility": 0.026694953441619873, + "branch_car_at_8": 0.01985256001353264, + "chart_id": "ms-f3f1d07cfd59b823da47", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.17124073207378387, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4586111091834293, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07150205224752426, + "proposal_oracle_utility_gain_over_base_at_8": 0.04480709880590439, + "seed": "2015956957/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05164949223399162, + "selected_utility_gain_over_base_at_8": 0.02495453879237175, + "selector_regret_at_8": 0.01985256001353264, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09973867982625961, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11959123983979225 + }, + { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 0.008425161242485046, + "chart_id": "ms-081952a63087fc8f83cd", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07518554002658885, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08595690131187439, + "proposal_oracle_utility_gain_over_base_at_8": 0.01514965295791626, + "seed": "2141802816/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07753174006938934, + "selected_utility_gain_over_base_at_8": 0.006724491715431213, + "selector_regret_at_8": 0.008425161242485046, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9140430986881256, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9224682599306107 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-121bc2e6676c1ac10911", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "77147264/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-22f153f0d4fa656018e6", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1271718007/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.002386912703514099, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20266639463366093, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.027498364448547363, + "proposal_oracle_utility_gain_over_base_at_8": -0.004528626799583435, + "seed": "1881223807/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.025111451745033264, + "selected_utility_gain_over_base_at_8": -0.006915539503097534, + "selector_regret_at_8": 0.002386912703514099, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9725016355514526, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9748885482549667 + }, + { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "chart_id": "ms-2867deaca7e98582620b", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1340247769860469, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "seed": "1189392917/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 1.9452667087316513, + "chart_id": "ms-365d8c6de6de16f66744", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3641822422309461, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "seed": "1997090716/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.054733291268348694, + "selected_utility_gain_over_base_at_8": -0.08454149961471558, + "selector_regret_at_8": 1.9452667087316513, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9452667087316513 + }, + { + "base_success": 0.0, + "base_utility": 0.03280790150165558, + "branch_car_at_8": 0.0012761056423187256, + "chart_id": "ms-3c326d05fec66e1d6bea", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11595738961017374, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.021766602993011475, + "proposal_oracle_utility_gain_over_base_at_8": -0.011041298508644104, + "seed": "278008524/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02049049735069275, + "selected_utility_gain_over_base_at_8": -0.01231740415096283, + "selector_regret_at_8": 0.0012761056423187256, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9782333970069885, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9795095026493073 + }, + { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3405496231635778, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "seed": "794181626/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-58950fed04d203b97ab9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026017268580811406, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1113371261/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 1.8899840712547302, + "chart_id": "ms-651bc0395d78d2819158", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35034389349653194, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "seed": "1483645336/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11001592874526978, + "selected_utility_gain_over_base_at_8": 0.09359987080097198, + "selector_regret_at_8": 1.8899840712547302, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.8899840712547302 + }, + { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9300042241811752, + "chart_id": "ms-9fad82fd288e4296e354", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1246594058194818, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "seed": "1176943004/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06999577581882477, + "selected_utility_gain_over_base_at_8": -0.08461795747280121, + "selector_regret_at_8": 1.9300042241811752, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9300042241811752 + }, + { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0010883957147598267, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1991858618764328, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09276536107063293, + "proposal_oracle_utility_gain_over_base_at_8": 0.024948790669441223, + "seed": "998761162/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09167696535587311, + "selected_utility_gain_over_base_at_8": 0.023860394954681396, + "selector_regret_at_8": 0.0010883957147598267, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.907234638929367, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.908323034644127 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.017522692680358887, + "chart_id": "ms-fc8084d5cd7da937fc02", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09717880195623664, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13679209351539612, + "proposal_oracle_utility_gain_over_base_at_8": -1.8632079064846039, + "seed": "1991414747/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11926940083503723, + "selected_utility_gain_over_base_at_8": -1.8807305991649628, + "selector_regret_at_8": 0.017522692680358887, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8632079064846039, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.8807305991649628 + }, + { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.003223627805709839, + "chart_id": "ms-3afd8959363e917591d4", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.13808834192306887, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2283252328634262, + "proposal_oracle_utility_gain_over_base_at_8": 0.025573596358299255, + "seed": "75070641/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.22510160505771637, + "selected_utility_gain_over_base_at_8": 0.022349968552589417, + "selector_regret_at_8": 0.003223627805709839, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7716747671365738, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7748983949422836 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.828233242034912, + "chart_id": "ms-3ff1aedb72f65a182b58", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30857397468613657, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1055567618/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1717667579650879, + "selected_utility_gain_over_base_at_8": -1.828233242034912, + "selector_regret_at_8": 1.828233242034912, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.828233242034912 + }, + { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.4498589485883713, + "chart_id": "ms-4048f8f6e047422e3c79", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.43030250381223584, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5705926418304443, + "proposal_oracle_utility_gain_over_base_at_8": 0.37410856783390045, + "seed": "1055157354/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12073369324207306, + "selected_utility_gain_over_base_at_8": -0.07575038075447083, + "selector_regret_at_8": 0.4498589485883713, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4294073581695557, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.879266306757927 + }, + { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.030991941690444946, + "chart_id": "ms-5b26c33feed6992386b1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.31643520896589616, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21208032965660095, + "proposal_oracle_utility_gain_over_base_at_8": 0.01363721489906311, + "seed": "1522272686/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.181088387966156, + "selected_utility_gain_over_base_at_8": -0.017354726791381836, + "selector_regret_at_8": 0.030991941690444946, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.787919670343399, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.818911612033844 + }, + { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "chart_id": "ms-63ac245579b1188935ad", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14834095177221712, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8135772347450256, + "proposal_oracle_utility_gain_over_base_at_8": -0.0252571702003479, + "seed": "211245842/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8135772347450256, + "selected_utility_gain_over_base_at_8": -0.0252571702003479, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1864227652549744, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1864227652549744 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1665739511/train0", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.44993481040000916, + "chart_id": "ms-72777024349c2aacd5e0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18757436508611464, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.574478030204773, + "proposal_oracle_utility_gain_over_base_at_8": -1.425521969795227, + "seed": "256770874/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1245432198047638, + "selected_utility_gain_over_base_at_8": -1.8754567801952362, + "selector_regret_at_8": 0.44993481040000916, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.425521969795227, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8754567801952362 + }, + { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3607921451330185, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2493251178612577, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5799027681350708, + "proposal_oracle_utility_gain_over_base_at_8": 0.37618090212345123, + "seed": "957101489/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2191106230020523, + "selected_utility_gain_over_base_at_8": 0.01538875699043274, + "selector_regret_at_8": 0.3607921451330185, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4200972318649292, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7808893769979477 + }, + { + "base_success": 0.0, + "base_utility": 0.8431589603424072, + "branch_car_at_8": 0.026760950684547424, + "chart_id": "ms-9a14552b7db695dee92c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.4512356830429367, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22642898559570312, + "proposal_oracle_utility_gain_over_base_at_8": -0.6167299747467041, + "seed": "77147264/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1996680349111557, + "selected_utility_gain_over_base_at_8": -0.6434909254312515, + "selector_regret_at_8": 0.026760950684547424, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7735710144042969, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8003319650888443 + }, + { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.8556141257286072, + "chart_id": "ms-9f5d57fe2153fd267774", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3951037237909243, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "seed": "1607773860/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14438587427139282, + "selected_utility_gain_over_base_at_8": -0.6882869601249695, + "selector_regret_at_8": 1.8556141257286072, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8556141257286072 + }, + { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.6354914605617523, + "chart_id": "ms-b0fee26e1db967f90b81", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.28863435648886215, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8147144317626953, + "proposal_oracle_utility_gain_over_base_at_8": -0.012500584125518799, + "seed": "1358421325/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.179222971200943, + "selected_utility_gain_over_base_at_8": -0.6479920446872711, + "selector_regret_at_8": 0.6354914605617523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1852855682373047, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.820777028799057 + }, + { + "base_success": 0.0, + "base_utility": 0.8307622671127319, + "branch_car_at_8": 1.8165037035942078, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35878451162224945, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.169237732887268, + "seed": "1271718007/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.18349629640579224, + "selected_utility_gain_over_base_at_8": -0.6472659707069397, + "selector_regret_at_8": 1.8165037035942078, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8165037035942078 + }, + { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.3453921228647232, + "chart_id": "ms-d1960368ef76bbb10669", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1517874168346938, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.581784188747406, + "proposal_oracle_utility_gain_over_base_at_8": 0.38145777583122253, + "seed": "416183106/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2363920658826828, + "selected_utility_gain_over_base_at_8": 0.03606565296649933, + "selector_regret_at_8": 0.3453921228647232, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.418215811252594, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7636079341173172 + }, + { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.1455947458744049, + "chart_id": "ms-efdc07c985935d871adf", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2692917587355154, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21975377202033997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6153350174427032, + "seed": "706479600/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07415902614593506, + "selected_utility_gain_over_base_at_8": -0.7609297633171082, + "selector_regret_at_8": 0.1455947458744049, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.78024622797966, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.925840973854065 + }, + { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.021911710500717163, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09083822075370537, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09917314350605011, + "proposal_oracle_utility_gain_over_base_at_8": 0.06197427213191986, + "seed": "1919198188/train0", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07726143300533295, + "selected_utility_gain_over_base_at_8": 0.0400625616312027, + "selector_regret_at_8": 0.021911710500717163, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.90082685649395, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.922738566994667 + }, + { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "chart_id": "ms-484ea653bbca2de3a4a2", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07419277505792353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "seed": "1231780997/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.2729938626289368, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.0031174421310424805, + "chart_id": "ms-4b4cd12d70c0615e0930", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14400158741031796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2148684412240982, + "proposal_oracle_utility_gain_over_base_at_8": -0.31350521743297577, + "seed": "1025740212/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21175099909305573, + "selected_utility_gain_over_base_at_8": -0.31662265956401825, + "selector_regret_at_8": 0.0031174421310424805, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7851315587759018, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.7882490009069443 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-6e268d1e2915740816a1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0862090980213101, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7181311249732971, + "proposal_oracle_utility_gain_over_base_at_8": -1.2818688750267029, + "seed": "1475440973/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7181311249732971, + "selected_utility_gain_over_base_at_8": -1.2818688750267029, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2818688750267029, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2818688750267029 + }, + { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2779860496520996, + "chart_id": "ms-8864796e31bbe93d7c5d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.23627606487156766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "seed": "551504635/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7220139503479004, + "selected_utility_gain_over_base_at_8": -0.001958608627319336, + "selector_regret_at_8": 1.2779860496520996, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2779860496520996 + }, + { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.06326162815093994, + "chart_id": "ms-b04b489f4fbad93b65ca", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06911736513585261, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7256770730018616, + "proposal_oracle_utility_gain_over_base_at_8": 0.3033645451068878, + "seed": "1113268940/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6624154448509216, + "selected_utility_gain_over_base_at_8": 0.24010291695594788, + "selector_regret_at_8": 0.06326162815093994, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2743229269981384, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.3375845551490784 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.008421123027801514, + "chart_id": "ms-c663f7e73e7e8376a05c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026961937751403514, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7255622744560242, + "proposal_oracle_utility_gain_over_base_at_8": -1.2744377255439758, + "seed": "1528740417/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7171411514282227, + "selected_utility_gain_over_base_at_8": -1.2828588485717773, + "selector_regret_at_8": 0.008421123027801514, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2744377255439758, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2828588485717773 + }, + { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "seed": "1273247072/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "task_id": "PickCube-v1", + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + { + "base_success": 0.0, + "base_utility": 0.050487834960222244, + "branch_car_at_8": 0.09754210896790028, + "chart_id": "ms-0345d8ac530f88cefd61", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1798098385334015, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.27579031898477024, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.11701449006795883, + "proposal_oracle_utility_gain_over_base_at_8": 0.06652665510773659, + "seed": "141984615/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.019472381100058556, + "selected_utility_gain_over_base_at_8": -0.03101545386016369, + "selector_regret_at_8": 0.09754210896790028, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.06279534846544266, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.16033745743334293 + }, + { + "base_success": 0.0, + "base_utility": 0.07743936777114868, + "branch_car_at_8": 0.005068322643637657, + "chart_id": "ms-0819eb89e627595e7e1b", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.14572584629058838, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2986384171869234, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.013550917617976665, + "proposal_oracle_utility_gain_over_base_at_8": -0.06388845015317202, + "seed": "2015956957/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.008482594974339008, + "selected_utility_gain_over_base_at_8": -0.06895677279680967, + "selector_regret_at_8": 0.005068322643637657, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1321749286726117, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13724325131624937 + }, + { + "base_success": 0.0, + "base_utility": 0.19400137662887573, + "branch_car_at_8": 0.07661644369363785, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30484520189300646, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1163921132683754, + "proposal_oracle_utility_gain_over_base_at_8": -0.07760926336050034, + "seed": "565369392/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03977566957473755, + "selected_utility_gain_over_base_at_8": -0.15422570705413818, + "selector_regret_at_8": 0.07661644369363785, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8836078867316246, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9602243304252625 + }, + { + "base_success": 0.0, + "base_utility": 0.04583283141255379, + "branch_car_at_8": 0.0687156543135643, + "chart_id": "ms-126eb8b36769f9ba4671", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1985912322998047, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.22659482311753865, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.17300955951213837, + "proposal_oracle_utility_gain_over_base_at_8": 0.12717672809958458, + "seed": "767747424/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10429390519857407, + "selected_utility_gain_over_base_at_8": 0.05846107378602028, + "selector_regret_at_8": 0.0687156543135643, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.02558167278766632, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.09429732710123062 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-16661b2fd2c85f57a66c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.001000205724267289, + "chart_id": "ms-20e1efbb745fd02abc95", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16573502466028908, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.001480837701819837, + "proposal_oracle_utility_gain_over_base_at_8": -1.9985191622981802, + "seed": "1375492786/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.00048063197755254805, + "selected_utility_gain_over_base_at_8": -1.9995193680224475, + "selector_regret_at_8": 0.001000205724267289, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2419701317558065, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2429703374800738 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-2dc5138251afd26a040d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565621826/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.08252689987421036, + "branch_car_at_8": 0.08448734506964684, + "chart_id": "ms-387e6771a55be98f1a59", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20744429528713226, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06284898813868309, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.14589862525463104, + "proposal_oracle_utility_gain_over_base_at_8": 0.06337172538042068, + "seed": "767747424/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06141128018498421, + "selected_utility_gain_over_base_at_8": -0.02111561968922615, + "selector_regret_at_8": 0.08448734506964684, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.06154567003250122, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.14603301510214806 + }, + { + "base_success": 0.0, + "base_utility": 0.03917040303349495, + "branch_car_at_8": 0.0, + "chart_id": "ms-38f7298184a2c5005e94", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1767096221446991, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.11490267487254319, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.11718571186065674, + "proposal_oracle_utility_gain_over_base_at_8": 0.07801530882716179, + "seed": "2015956957/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11718571186065674, + "selected_utility_gain_over_base_at_8": 0.07801530882716179, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05952391028404236, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.05952391028404236 + }, + { + "base_success": 0.0, + "base_utility": 0.009124159812927246, + "branch_car_at_8": 0.01312979031354189, + "chart_id": "ms-482c5c8d208b5907a2c2", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1834145486354828, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24422123975177484, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.021352291107177734, + "proposal_oracle_utility_gain_over_base_at_8": 0.012228131294250488, + "seed": "626486813/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.008222500793635845, + "selected_utility_gain_over_base_at_8": -0.0009016590192914009, + "selector_regret_at_8": 0.01312979031354189, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.16206225752830505, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17519204784184694 + }, + { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 1.9076288938522339, + "chart_id": "ms-4da03300d92420e80ed0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1055876219626688, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "seed": "1747178030/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09237110614776611, + "selected_utility_gain_over_base_at_8": -0.054260119795799255, + "selector_regret_at_8": 1.9076288938522339, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9076288938522339 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.009623387828469276, + "chart_id": "ms-57be9ae181c077e7821f", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15538118252826771, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.01863894797861576, + "proposal_oracle_utility_gain_over_base_at_8": -1.9813610520213842, + "seed": "1549324805/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.009015560150146484, + "selected_utility_gain_over_base_at_8": -1.9909844398498535, + "selector_regret_at_8": 0.009623387828469276, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9813610520213842, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9909844398498535 + }, + { + "base_success": 0.0, + "base_utility": 0.1544450968503952, + "branch_car_at_8": 0.0, + "chart_id": "ms-6388b5d2356c556c3e55", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13157149394108555, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07292797416448593, + "proposal_oracle_utility_gain_over_base_at_8": -0.08151712268590927, + "seed": "2015956957/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07292797416448593, + "selected_utility_gain_over_base_at_8": -0.08151712268590927, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.927072025835514, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.927072025835514 + }, + { + "base_success": 0.0, + "base_utility": 0.06173701956868172, + "branch_car_at_8": 0.028241891413927078, + "chart_id": "ms-638d2ba1917eff4d09e0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.11176212877035141, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12990558981115807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08538073301315308, + "proposal_oracle_utility_gain_over_base_at_8": 0.02364371344447136, + "seed": "767747424/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.057138841599226, + "selected_utility_gain_over_base_at_8": -0.004598177969455719, + "selector_regret_at_8": 0.028241891413927078, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.026381395757198334, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.05462328717112541 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00025266362354159355, + "chart_id": "ms-652ad552ba98d2736620", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2367449700832367, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11085562899340784, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.007886429317295551, + "proposal_oracle_utility_gain_over_base_at_8": -1.9921135706827044, + "seed": "767747424/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.007633765693753958, + "selected_utility_gain_over_base_at_8": -1.992366234306246, + "selector_regret_at_8": 0.00025266362354159355, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.22885854076594114, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22911120438948274 + }, + { + "base_success": 0.0, + "base_utility": 0.03784912824630737, + "branch_car_at_8": 0.01684868335723877, + "chart_id": "ms-674fad0e7c042d23629e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.15573063492774963, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2935561879950145, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.057549476623535156, + "proposal_oracle_utility_gain_over_base_at_8": 0.019700348377227783, + "seed": "2015956957/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04070079326629639, + "selected_utility_gain_over_base_at_8": 0.0028516650199890137, + "selector_regret_at_8": 0.01684868335723877, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09818115830421448, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11502984166145325 + }, + { + "base_success": 0.0, + "base_utility": 0.12791956961154938, + "branch_car_at_8": 0.059114933013916016, + "chart_id": "ms-728b865677134d3788e3", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.22658847272396088, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24450025940895065, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.13260090351104736, + "proposal_oracle_utility_gain_over_base_at_8": 0.004681333899497986, + "seed": "750448296/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07348597049713135, + "selected_utility_gain_over_base_at_8": -0.05443359911441803, + "selector_regret_at_8": 0.059114933013916016, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09398756921291351, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.15310250222682953 + }, + { + "base_success": 0.0, + "base_utility": 0.20529556274414062, + "branch_car_at_8": 0.0617663636803627, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.27989473938941956, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.38411828214949667, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09449902921915054, + "proposal_oracle_utility_gain_over_base_at_8": -0.11079653352499008, + "seed": "565369392/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03273266553878784, + "selected_utility_gain_over_base_at_8": -0.17256289720535278, + "selector_regret_at_8": 0.0617663636803627, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.185395710170269, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2471620738506317 + }, + { + "base_success": 0.0, + "base_utility": 0.18213583528995514, + "branch_car_at_8": 0.0027207134990021586, + "chart_id": "ms-7f57045c114ddad153b5", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.227622389793396, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10807032783608049, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.003711601020768285, + "proposal_oracle_utility_gain_over_base_at_8": -0.17842423426918685, + "seed": "278008524/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0009908875217661262, + "selected_utility_gain_over_base_at_8": -0.181144947768189, + "selector_regret_at_8": 0.0027207134990021586, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2239107887726277, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22663150227162987 + }, + { + "base_success": 0.0, + "base_utility": 0.0056487321853637695, + "branch_car_at_8": 0.0, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1631626933813095, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2438992767980762, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.020783662796020508, + "proposal_oracle_utility_gain_over_base_at_8": 0.015134930610656738, + "seed": "626486813/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.020783662796020508, + "selected_utility_gain_over_base_at_8": 0.015134930610656738, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.142379030585289, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.142379030585289 + }, + { + "base_success": 0.0, + "base_utility": 0.00698717450723052, + "branch_car_at_8": 0.02401920035481453, + "chart_id": "ms-8a72b73ac452bc190aeb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.10329639911651611, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15532500254873144, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08651334047317505, + "proposal_oracle_utility_gain_over_base_at_8": 0.07952616596594453, + "seed": "626486813/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06249414011836052, + "selected_utility_gain_over_base_at_8": 0.05550696561113, + "selector_regret_at_8": 0.02401920035481453, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.016783058643341064, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.040802258998155594 + }, + { + "base_success": 0.0, + "base_utility": 0.08285123109817505, + "branch_car_at_8": 0.015108168125152588, + "chart_id": "ms-a96bca1ef0b74f686018", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1572066992521286, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.03601989971092138, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09714335203170776, + "proposal_oracle_utility_gain_over_base_at_8": 0.014292120933532715, + "seed": "141984615/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.08203518390655518, + "selected_utility_gain_over_base_at_8": -0.000816047191619873, + "selector_regret_at_8": 0.015108168125152588, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.06006334722042084, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.07517151534557343 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00563424825668335, + "chart_id": "ms-ac4fb33f3ab4db853995", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16883301005240778, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.14365969598293304, + "proposal_oracle_utility_gain_over_base_at_8": -1.856340304017067, + "seed": "565369392/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1380254477262497, + "selected_utility_gain_over_base_at_8": -1.8619745522737503, + "selector_regret_at_8": 0.00563424825668335, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.856340304017067, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8619745522737503 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "2054899126/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.037537217140197754, + "branch_car_at_8": 0.013502320274710655, + "chart_id": "ms-c832439976a37f52d094", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2594742774963379, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22009063480440055, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.025354644283652306, + "proposal_oracle_utility_gain_over_base_at_8": -0.012182572856545448, + "seed": "585904130/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.01185232400894165, + "selected_utility_gain_over_base_at_8": -0.025684893131256104, + "selector_regret_at_8": 0.013502320274710655, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.23411963321268559, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24762195348739624 + }, + { + "base_success": 0.0, + "base_utility": 0.09138330072164536, + "branch_car_at_8": 0.0, + "chart_id": "ms-c96b415f083f60857334", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.26426148414611816, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14208489100702013, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.011022846214473248, + "proposal_oracle_utility_gain_over_base_at_8": -0.08036045450717211, + "seed": "585904130/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011022846214473248, + "selected_utility_gain_over_base_at_8": -0.08036045450717211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2532386379316449, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2532386379316449 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.12736602361488303, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "534717141/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.14809735119342804, + "branch_car_at_8": 0.0008638898143544793, + "chart_id": "ms-da142025733b53fcf7eb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2643269896507263, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18818951741396497, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.001536289812065661, + "proposal_oracle_utility_gain_over_base_at_8": -0.14656106138136238, + "seed": "141984615/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0006723999977111816, + "selected_utility_gain_over_base_at_8": -0.14742495119571686, + "selector_regret_at_8": 0.0008638898143544793, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.26279069983866066, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.26365458965301514 + }, + { + "base_success": 0.0, + "base_utility": 0.1362423449754715, + "branch_car_at_8": 0.005128046497702599, + "chart_id": "ms-e2bad6e99e8a281207cc", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20825156569480896, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15502450434938486, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.018407067283988, + "proposal_oracle_utility_gain_over_base_at_8": -0.1178352776914835, + "seed": "1549324805/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0132790207862854, + "selected_utility_gain_over_base_at_8": -0.1229633241891861, + "selector_regret_at_8": 0.005128046497702599, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.18984449841082096, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.19497254490852356 + }, + { + "base_success": 0.0, + "base_utility": 0.09695577621459961, + "branch_car_at_8": 0.0, + "chart_id": "ms-e30fd80adc089e9f697a", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18920521438121796, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1335740334444898, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.023269018158316612, + "proposal_oracle_utility_gain_over_base_at_8": -0.073686758056283, + "seed": "585904130/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.023269018158316612, + "selected_utility_gain_over_base_at_8": -0.073686758056283, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.16593619622290134, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.16593619622290134 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07539165019989014, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18186684363707276, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.14950889348983765, + "proposal_oracle_utility_gain_over_base_at_8": -1.8504911065101624, + "seed": "626486813/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.07411724328994751, + "selected_utility_gain_over_base_at_8": -1.9258827567100525, + "selector_regret_at_8": 0.07539165019989014, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8504911065101624, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9258827567100525 + }, + { + "base_success": 0.0, + "base_utility": 0.09514755010604858, + "branch_car_at_8": 0.029236994683742523, + "chart_id": "ms-ec013396e4aff6170ee1", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24502670764923096, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.06092325242684257, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07131137698888779, + "proposal_oracle_utility_gain_over_base_at_8": -0.023836173117160797, + "seed": "750448296/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.042074382305145264, + "selected_utility_gain_over_base_at_8": -0.05307316780090332, + "selector_regret_at_8": 0.029236994683742523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.17371533066034317, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2029523253440857 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.004942953586578369, + "chart_id": "ms-ee30ebaf98622759d888", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.45862923715160336, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.015581388957798481, + "proposal_oracle_utility_gain_over_base_at_8": -1.9844186110422015, + "seed": "636680137/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.010638435371220112, + "selected_utility_gain_over_base_at_8": -1.9893615646287799, + "selector_regret_at_8": 0.004942953586578369, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9844186110422015, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9893615646287799 + }, + { + "base_success": 0.0, + "base_utility": 0.026694953441619873, + "branch_car_at_8": 0.004920683801174164, + "chart_id": "ms-f3f1d07cfd59b823da47", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.17124073207378387, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.30358571851076954, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07510259002447128, + "proposal_oracle_utility_gain_over_base_at_8": 0.04840763658285141, + "seed": "2015956957/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07018190622329712, + "selected_utility_gain_over_base_at_8": 0.043486952781677246, + "selector_regret_at_8": 0.004920683801174164, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09613814204931259, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.10105882585048676 + }, + { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.934090182185173, + "chart_id": "ms-081952a63087fc8f83cd", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.40989236974714055, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "seed": "2141802816/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06590981781482697, + "selected_utility_gain_over_base_at_8": -0.0048974305391311646, + "selector_regret_at_8": 1.934090182185173, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.934090182185173 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-121bc2e6676c1ac10911", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "77147264/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-22f153f0d4fa656018e6", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1271718007/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 1.964881807565689, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10190418918720931, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9679730087518692, + "seed": "1881223807/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03511819243431091, + "selected_utility_gain_over_base_at_8": 0.0030912011861801147, + "selector_regret_at_8": 1.964881807565689, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.964881807565689 + }, + { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 1.9264162927865982, + "chart_id": "ms-2867deaca7e98582620b", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10709427493066431, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "seed": "1189392917/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0735837072134018, + "selected_utility_gain_over_base_at_8": -0.03483493626117706, + "selector_regret_at_8": 1.9264162927865982, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9264162927865982 + }, + { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "chart_id": "ms-365d8c6de6de16f66744", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3553584127845171, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "seed": "1997090716/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.03280790150165558, + "branch_car_at_8": 0.012138530611991882, + "chart_id": "ms-3c326d05fec66e1d6bea", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10448389878723242, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0349019318819046, + "proposal_oracle_utility_gain_over_base_at_8": 0.0020940303802490234, + "seed": "278008524/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02276340126991272, + "selected_utility_gain_over_base_at_8": -0.010044500231742859, + "selector_regret_at_8": 0.012138530611991882, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9650980681180954, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9772365987300873 + }, + { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 1.9626595228910446, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3619001242329895, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "seed": "794181626/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03734047710895538, + "selected_utility_gain_over_base_at_8": -0.14361557364463806, + "selector_regret_at_8": 1.9626595228910446, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9626595228910446 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-58950fed04d203b97ab9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.015047211698777683, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1113371261/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "chart_id": "ms-651bc0395d78d2819158", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06079377324000063, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "seed": "1483645336/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.953145906329155, + "chart_id": "ms-9fad82fd288e4296e354", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15442455751472706, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "seed": "1176943004/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04685409367084503, + "selected_utility_gain_over_base_at_8": -0.10775963962078094, + "selector_regret_at_8": 1.953145906329155, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.953145906329155 + }, + { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24035198055524792, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "seed": "998761162/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9461991488933563, + "chart_id": "ms-fc8084d5cd7da937fc02", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.44645064197830664, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1991414747/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.05380085110664368, + "selected_utility_gain_over_base_at_8": -1.9461991488933563, + "selector_regret_at_8": 1.9461991488933563, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9461991488933563 + }, + { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.007853284478187561, + "chart_id": "ms-3afd8959363e917591d4", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.34224193319263285, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24087366461753845, + "proposal_oracle_utility_gain_over_base_at_8": 0.0381220281124115, + "seed": "75070641/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2330203801393509, + "selected_utility_gain_over_base_at_8": 0.030268743634223938, + "selector_regret_at_8": 0.007853284478187561, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7591263353824615, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.766979619860649 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-3ff1aedb72f65a182b58", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15812100047218913, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1055567618/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.6509099900722504, + "chart_id": "ms-4048f8f6e047422e3c79", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32609963351937027, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.812659502029419, + "proposal_oracle_utility_gain_over_base_at_8": 0.6161754280328751, + "seed": "1055157354/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.16174951195716858, + "selected_utility_gain_over_base_at_8": -0.034734562039375305, + "selector_regret_at_8": 0.6509099900722504, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.187340497970581, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8382504880428314 + }, + { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.03571280837059021, + "chart_id": "ms-5b26c33feed6992386b1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24182565394435904, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24029652774333954, + "proposal_oracle_utility_gain_over_base_at_8": 0.0418534129858017, + "seed": "1522272686/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20458371937274933, + "selected_utility_gain_over_base_at_8": 0.006140604615211487, + "selector_regret_at_8": 0.03571280837059021, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7597034722566605, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7954162806272507 + }, + { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 1.1816681623458862, + "chart_id": "ms-63ac245579b1188935ad", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.036206191308278124, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "seed": "211245842/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8183318376541138, + "selected_utility_gain_over_base_at_8": -0.020502567291259766, + "selector_regret_at_8": 1.1816681623458862, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1816681623458862 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.1815597414970398, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.05717674118582694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1665739511/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.8184402585029602, + "selected_utility_gain_over_base_at_8": -1.1815597414970398, + "selector_regret_at_8": 1.1815597414970398, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1815597414970398 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.6129319667816162, + "chart_id": "ms-72777024349c2aacd5e0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3193203145842822, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.8128684163093567, + "proposal_oracle_utility_gain_over_base_at_8": -1.1871315836906433, + "seed": "256770874/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.19993644952774048, + "selected_utility_gain_over_base_at_8": -1.8000635504722595, + "selector_regret_at_8": 0.6129319667816162, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1871315836906433, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8000635504722595 + }, + { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3404468297958374, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2708447347116597, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5798611640930176, + "proposal_oracle_utility_gain_over_base_at_8": 0.376139298081398, + "seed": "957101489/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.23941433429718018, + "selected_utility_gain_over_base_at_8": 0.03569246828556061, + "selector_regret_at_8": 0.3404468297958374, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4201388359069824, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7605856657028198 + }, + { + "base_success": 0.0, + "base_utility": 0.8431589603424072, + "branch_car_at_8": 1.7759137749671936, + "chart_id": "ms-9a14552b7db695dee92c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06758506233539574, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1568410396575928, + "seed": "77147264/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2240862250328064, + "selected_utility_gain_over_base_at_8": -0.6190727353096008, + "selector_regret_at_8": 1.7759137749671936, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7759137749671936 + }, + { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 0.0, + "chart_id": "ms-9f5d57fe2153fd267774", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2544258391757567, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "seed": "1607773860/train1", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1673271656036377, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.007964074611663818, + "chart_id": "ms-b0fee26e1db967f90b81", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.04701404433315851, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8245300054550171, + "proposal_oracle_utility_gain_over_base_at_8": -0.0026850104331970215, + "seed": "1358421325/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8165659308433533, + "selected_utility_gain_over_base_at_8": -0.01064908504486084, + "selector_regret_at_8": 0.007964074611663818, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.175469994544983, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1834340691566467 + }, + { + "base_success": 0.0, + "base_utility": 0.8307622671127319, + "branch_car_at_8": 1.7905852794647217, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21421799686366766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.169237732887268, + "seed": "1271718007/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20941472053527832, + "selected_utility_gain_over_base_at_8": -0.6213475465774536, + "selector_regret_at_8": 1.7905852794647217, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7905852794647217 + }, + { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.01654154062271118, + "chart_id": "ms-d1960368ef76bbb10669", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1925298309196861, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22490769624710083, + "proposal_oracle_utility_gain_over_base_at_8": 0.02458128333091736, + "seed": "416183106/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20836615562438965, + "selected_utility_gain_over_base_at_8": 0.008039742708206177, + "selector_regret_at_8": 0.01654154062271118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7750923037528992, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7916338443756104 + }, + { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 1.185575544834137, + "chart_id": "ms-efdc07c985935d871adf", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15129914778803955, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "seed": "706479600/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814424455165863, + "selected_utility_gain_over_base_at_8": -0.020664334297180176, + "selector_regret_at_8": 1.185575544834137, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.185575544834137 + }, + { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.18855391442775726, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2569381375337174, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2203868180513382, + "proposal_oracle_utility_gain_over_base_at_8": 0.18318794667720795, + "seed": "1919198188/train1", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03183290362358093, + "selected_utility_gain_over_base_at_8": -0.005365967750549316, + "selector_regret_at_8": 0.18855391442775726, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7796131819486618, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.968167096376419 + }, + { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "chart_id": "ms-484ea653bbca2de3a4a2", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16621555087386797, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.725460946559906, + "proposal_oracle_utility_gain_over_base_at_8": -0.0015451908111572266, + "seed": "1231780997/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.725460946559906, + "selected_utility_gain_over_base_at_8": -0.0015451908111572266, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.274539053440094, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.274539053440094 + }, + { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.051884353160858154, + "chart_id": "ms-4b4cd12d70c0615e0930", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12800533274597592, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7146708369255066, + "proposal_oracle_utility_gain_over_base_at_8": 0.18629717826843262, + "seed": "1025740212/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6627864837646484, + "selected_utility_gain_over_base_at_8": 0.13441282510757446, + "selector_regret_at_8": 0.051884353160858154, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2853291630744934, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.3372135162353516 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2959344387054443, + "chart_id": "ms-6e268d1e2915740816a1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15809903475948942, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1475440973/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7040655612945557, + "selected_utility_gain_over_base_at_8": -1.2959344387054443, + "selector_regret_at_8": 1.2959344387054443, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2959344387054443 + }, + { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.287484347820282, + "chart_id": "ms-8864796e31bbe93d7c5d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4327797462092032, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "seed": "551504635/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.712515652179718, + "selected_utility_gain_over_base_at_8": -0.011456906795501709, + "selector_regret_at_8": 1.287484347820282, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.287484347820282 + }, + { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.48447395861148834, + "chart_id": "ms-b04b489f4fbad93b65ca", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2534904144040469, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.6990174651145935, + "proposal_oracle_utility_gain_over_base_at_8": 0.27670493721961975, + "seed": "1113268940/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21454350650310516, + "selected_utility_gain_over_base_at_8": -0.2077690213918686, + "selector_regret_at_8": 0.48447395861148834, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.3009825348854065, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.7854564934968948 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c663f7e73e7e8376a05c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3242760158848682, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1528740417/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "seed": "1273247072/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "task_id": "PickCube-v1", + "total_car_to_hidden_at_8": 1.922220028936863 + }, + { + "base_success": 0.0, + "base_utility": 0.050487834960222244, + "branch_car_at_8": 0.06330369412899017, + "chart_id": "ms-0345d8ac530f88cefd61", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1798098385334015, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09873073474489925, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.18499283492565155, + "proposal_oracle_utility_gain_over_base_at_8": 0.1345049999654293, + "seed": "141984615/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12168914079666138, + "selected_utility_gain_over_base_at_8": 0.07120130583643913, + "selector_regret_at_8": 0.06330369412899017, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.05812069773674011 + }, + { + "base_success": 0.0, + "base_utility": 0.07743936777114868, + "branch_car_at_8": 0.006376962177455425, + "chart_id": "ms-0819eb89e627595e7e1b", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.14572584629058838, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1633033505229465, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.016623934730887413, + "proposal_oracle_utility_gain_over_base_at_8": -0.06081543304026127, + "seed": "2015956957/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.010246972553431988, + "selected_utility_gain_over_base_at_8": -0.0671923952177167, + "selector_regret_at_8": 0.006376962177455425, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12910191155970097, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1354788737371564 + }, + { + "base_success": 0.0, + "base_utility": 0.19400137662887573, + "branch_car_at_8": 0.11550857126712799, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22386163138867388, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.16342134773731232, + "proposal_oracle_utility_gain_over_base_at_8": -0.030580028891563416, + "seed": "565369392/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.047912776470184326, + "selected_utility_gain_over_base_at_8": -0.1460886001586914, + "selector_regret_at_8": 0.11550857126712799, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8365786522626877, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9520872235298157 + }, + { + "base_success": 0.0, + "base_utility": 0.04583283141255379, + "branch_car_at_8": 0.007768414914608002, + "chart_id": "ms-126eb8b36769f9ba4671", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1985912322998047, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32227493498212384, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07069015502929688, + "proposal_oracle_utility_gain_over_base_at_8": 0.024857323616743088, + "seed": "767747424/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06292174011468887, + "selected_utility_gain_over_base_at_8": 0.017088908702135086, + "selector_regret_at_8": 0.007768414914608002, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1279010772705078, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13566949218511581 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-16661b2fd2c85f57a66c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22254540334575248, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00036396586801856756, + "chart_id": "ms-20e1efbb745fd02abc95", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.29573093343728263, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0012095769634470344, + "proposal_oracle_utility_gain_over_base_at_8": -1.998790423036553, + "seed": "1375492786/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0008456110954284668, + "selected_utility_gain_over_base_at_8": -1.9991543889045715, + "selector_regret_at_8": 0.00036396586801856756, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2422413924941793, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24260535836219788 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-2dc5138251afd26a040d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565621826/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.08252689987421036, + "branch_car_at_8": 0.004420019686222076, + "chart_id": "ms-387e6771a55be98f1a59", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20744429528713226, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.312878719204232, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09707490354776382, + "proposal_oracle_utility_gain_over_base_at_8": 0.014548003673553467, + "seed": "767747424/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09265488386154175, + "selected_utility_gain_over_base_at_8": 0.01012798398733139, + "selector_regret_at_8": 0.004420019686222076, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11036939173936844, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11478941142559052 + }, + { + "base_success": 0.0, + "base_utility": 0.03917040303349495, + "branch_car_at_8": 0.04887852817773819, + "chart_id": "ms-38f7298184a2c5005e94", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1767096221446991, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12442156901905659, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.11467790603637695, + "proposal_oracle_utility_gain_over_base_at_8": 0.075507503002882, + "seed": "2015956957/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06579937785863876, + "selected_utility_gain_over_base_at_8": 0.026628974825143814, + "selector_regret_at_8": 0.04887852817773819, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.062031716108322144, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11091024428606033 + }, + { + "base_success": 0.0, + "base_utility": 0.009124159812927246, + "branch_car_at_8": 0.00928109884262085, + "chart_id": "ms-482c5c8d208b5907a2c2", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1834145486354828, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.18390629227998556, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.030993500724434853, + "proposal_oracle_utility_gain_over_base_at_8": 0.021869340911507607, + "seed": "626486813/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.021712401881814003, + "selected_utility_gain_over_base_at_8": 0.012588242068886757, + "selector_regret_at_8": 0.00928109884262085, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15242104791104794, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.16170214675366879 + }, + { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.0, + "chart_id": "ms-4da03300d92420e80ed0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33194730961537694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "seed": "1747178030/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8533687740564346, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008774008601903915, + "chart_id": "ms-57be9ae181c077e7821f", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2838898926846715, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.010276774875819683, + "proposal_oracle_utility_gain_over_base_at_8": -1.9897232251241803, + "seed": "1549324805/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.009399374015629292, + "selected_utility_gain_over_base_at_8": -1.9906006259843707, + "selector_regret_at_8": 0.0008774008601903915, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9897232251241803, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9906006259843707 + }, + { + "base_success": 0.0, + "base_utility": 0.1544450968503952, + "branch_car_at_8": 1.990578512661159, + "chart_id": "ms-6388b5d2356c556c3e55", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.28218050665295125, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8455549031496048, + "seed": "2015956957/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.009421487338840961, + "selected_utility_gain_over_base_at_8": -0.14502360951155424, + "selector_regret_at_8": 1.990578512661159, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.990578512661159 + }, + { + "base_success": 0.0, + "base_utility": 0.06173701956868172, + "branch_car_at_8": 0.039066337049007416, + "chart_id": "ms-638d2ba1917eff4d09e0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.11176212877035141, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10082628405067513, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.14829117059707642, + "proposal_oracle_utility_gain_over_base_at_8": 0.0865541510283947, + "seed": "767747424/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.109224833548069, + "selected_utility_gain_over_base_at_8": 0.04748781397938728, + "selector_regret_at_8": 0.039066337049007416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0025372952222824097 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.981676161289215, + "chart_id": "ms-652ad552ba98d2736620", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2367449700832367, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.08904583687943832, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "767747424/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.018323838710784912, + "selected_utility_gain_over_base_at_8": -1.981676161289215, + "selector_regret_at_8": 1.981676161289215, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.21842113137245178 + }, + { + "base_success": 0.0, + "base_utility": 0.03784912824630737, + "branch_car_at_8": 0.08733024075627327, + "chart_id": "ms-674fad0e7c042d23629e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.15573063492774963, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3185519049632928, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.12104523181915283, + "proposal_oracle_utility_gain_over_base_at_8": 0.08319610357284546, + "seed": "2015956957/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03371499106287956, + "selected_utility_gain_over_base_at_8": -0.004134137183427811, + "selector_regret_at_8": 0.08733024075627327, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0346854031085968, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.12201564386487007 + }, + { + "base_success": 0.0, + "base_utility": 0.12791956961154938, + "branch_car_at_8": 0.0684172473847866, + "chart_id": "ms-728b865677134d3788e3", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.22658847272396088, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.248063405084695, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10312310606241226, + "proposal_oracle_utility_gain_over_base_at_8": -0.024796463549137115, + "seed": "750448296/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.034705858677625656, + "selected_utility_gain_over_base_at_8": -0.09321371093392372, + "selector_regret_at_8": 0.0684172473847866, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12346536666154861, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.19188261404633522 + }, + { + "base_success": 0.0, + "base_utility": 0.20529556274414062, + "branch_car_at_8": 0.05776144564151764, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.27989473938941956, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10783128480272308, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.19780242443084717, + "proposal_oracle_utility_gain_over_base_at_8": -0.007493138313293457, + "seed": "565369392/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14004097878932953, + "selected_utility_gain_over_base_at_8": -0.0652545839548111, + "selector_regret_at_8": 0.05776144564151764, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.08209231495857239, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13985376060009003 + }, + { + "base_success": 0.0, + "base_utility": 0.18213583528995514, + "branch_car_at_8": 0.001691202400252223, + "chart_id": "ms-7f57045c114ddad153b5", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.227622389793396, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.43578303044475486, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.004366755485534668, + "proposal_oracle_utility_gain_over_base_at_8": -0.17776907980442047, + "seed": "278008524/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.002675553085282445, + "selected_utility_gain_over_base_at_8": -0.1794602822046727, + "selector_regret_at_8": 0.001691202400252223, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.22325563430786133, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22494683670811355 + }, + { + "base_success": 0.0, + "base_utility": 0.0056487321853637695, + "branch_car_at_8": 0.0014750361442565918, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1631626933813095, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.11431568558471643, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01331732701510191, + "proposal_oracle_utility_gain_over_base_at_8": 0.00766859482973814, + "seed": "626486813/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011842290870845318, + "selected_utility_gain_over_base_at_8": 0.006193558685481548, + "selector_regret_at_8": 0.0014750361442565918, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1498453663662076, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1513204025104642 + }, + { + "base_success": 0.0, + "base_utility": 0.00698717450723052, + "branch_car_at_8": 0.04227004759013653, + "chart_id": "ms-8a72b73ac452bc190aeb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.10329639911651611, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1973708126521778, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07134183496236801, + "proposal_oracle_utility_gain_over_base_at_8": 0.06435466045513749, + "seed": "626486813/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029071787372231483, + "selected_utility_gain_over_base_at_8": 0.022084612865000963, + "selector_regret_at_8": 0.04227004759013653, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0319545641541481, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.07422461174428463 + }, + { + "base_success": 0.0, + "base_utility": 0.08285123109817505, + "branch_car_at_8": 0.04249020293354988, + "chart_id": "ms-a96bca1ef0b74f686018", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1572066992521286, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.4035470095413562, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07769393920898438, + "proposal_oracle_utility_gain_over_base_at_8": -0.005157291889190674, + "seed": "141984615/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.035203736275434494, + "selected_utility_gain_over_base_at_8": -0.047647494822740555, + "selector_regret_at_8": 0.04249020293354988, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.07951276004314423, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.12200296297669411 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-ac4fb33f3ab4db853995", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.32123621367047994, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565369392/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "2054899126/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.037537217140197754, + "branch_car_at_8": 0.0038126707077026367, + "chart_id": "ms-c832439976a37f52d094", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2594742774963379, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.24086667034114959, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.029142936691641808, + "proposal_oracle_utility_gain_over_base_at_8": -0.008394280448555946, + "seed": "585904130/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02533026598393917, + "selected_utility_gain_over_base_at_8": -0.012206951156258583, + "selector_regret_at_8": 0.0038126707077026367, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.23033134080469608, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.23414401151239872 + }, + { + "base_success": 0.0, + "base_utility": 0.09138330072164536, + "branch_car_at_8": 0.001046299934387207, + "chart_id": "ms-c96b415f083f60857334", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.26426148414611816, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.17074054804518998, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01693703792989254, + "proposal_oracle_utility_gain_over_base_at_8": -0.07444626279175282, + "seed": "585904130/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.015890737995505333, + "selected_utility_gain_over_base_at_8": -0.07549256272614002, + "selector_regret_at_8": 0.001046299934387207, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24732444621622562, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24837074615061283 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15703220108978472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "534717141/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.14809735119342804, + "branch_car_at_8": 0.00038852286525070667, + "chart_id": "ms-da142025733b53fcf7eb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2643269896507263, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18128422623207796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0024978716392070055, + "proposal_oracle_utility_gain_over_base_at_8": -0.14559947955422103, + "seed": "141984615/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.002109348773956299, + "selected_utility_gain_over_base_at_8": -0.14598800241947174, + "selector_regret_at_8": 0.00038852286525070667, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2618291180115193, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.26221764087677 + }, + { + "base_success": 0.0, + "base_utility": 0.1362423449754715, + "branch_car_at_8": 0.030585486441850662, + "chart_id": "ms-e2bad6e99e8a281207cc", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20825156569480896, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.33910771964071507, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04993142560124397, + "proposal_oracle_utility_gain_over_base_at_8": -0.08631091937422752, + "seed": "1549324805/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.01934593915939331, + "selected_utility_gain_over_base_at_8": -0.11689640581607819, + "selector_regret_at_8": 0.030585486441850662, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.158320140093565, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.18890562653541565 + }, + { + "base_success": 0.0, + "base_utility": 0.09695577621459961, + "branch_car_at_8": 0.01554206945002079, + "chart_id": "ms-e30fd80adc089e9f697a", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18920521438121796, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.25602335450638297, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04263921454548836, + "proposal_oracle_utility_gain_over_base_at_8": -0.05431656166911125, + "seed": "585904130/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.027097145095467567, + "selected_utility_gain_over_base_at_8": -0.06985863111913204, + "selector_regret_at_8": 0.01554206945002079, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1465659998357296, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1621080692857504 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10228227769065046, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.1320568472146988, + "proposal_oracle_utility_gain_over_base_at_8": -1.8679431527853012, + "seed": "626486813/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1320568472146988, + "selected_utility_gain_over_base_at_8": -1.8679431527853012, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8679431527853012, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8679431527853012 + }, + { + "base_success": 0.0, + "base_utility": 0.09514755010604858, + "branch_car_at_8": 0.0030524134635925293, + "chart_id": "ms-ec013396e4aff6170ee1", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24502670764923096, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1325592590169343, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06178152561187744, + "proposal_oracle_utility_gain_over_base_at_8": -0.03336602449417114, + "seed": "750448296/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05872911214828491, + "selected_utility_gain_over_base_at_8": -0.03641843795776367, + "selector_regret_at_8": 0.0030524134635925293, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.18324518203735352, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.18629759550094604 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.028240702114999294, + "chart_id": "ms-ee30ebaf98622759d888", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.6813618274184183, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.036433976143598557, + "proposal_oracle_utility_gain_over_base_at_8": -1.9635660238564014, + "seed": "636680137/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.008193274028599262, + "selected_utility_gain_over_base_at_8": -1.9918067259714007, + "selector_regret_at_8": 0.028240702114999294, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9635660238564014, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9918067259714007 + }, + { + "base_success": 0.0, + "base_utility": 0.026694953441619873, + "branch_car_at_8": 0.023953834548592567, + "chart_id": "ms-f3f1d07cfd59b823da47", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.17124073207378387, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3481076775975991, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.044700879603624344, + "proposal_oracle_utility_gain_over_base_at_8": 0.01800592616200447, + "seed": "2015956957/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.020747045055031776, + "selected_utility_gain_over_base_at_8": -0.005947908386588097, + "selector_regret_at_8": 0.023953834548592567, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12653985247015953, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1504936870187521 + }, + { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.954741045832634, + "chart_id": "ms-081952a63087fc8f83cd", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3183279052482729, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "seed": "2141802816/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04525895416736603, + "selected_utility_gain_over_base_at_8": -0.025548294186592102, + "selector_regret_at_8": 1.954741045832634, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.954741045832634 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8714426755905151, + "chart_id": "ms-121bc2e6676c1ac10911", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.21826309519081116, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "77147264/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.12855732440948486, + "selected_utility_gain_over_base_at_8": -1.8714426755905151, + "selector_regret_at_8": 1.8714426755905151, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.8714426755905151 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-22f153f0d4fa656018e6", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1130050850209727, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1271718007/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.0, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1643392210528543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.040287479758262634, + "proposal_oracle_utility_gain_over_base_at_8": 0.008260488510131836, + "seed": "1881223807/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.040287479758262634, + "selected_utility_gain_over_base_at_8": 0.008260488510131836, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9597125202417374, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9597125202417374 + }, + { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "chart_id": "ms-2867deaca7e98582620b", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33283923875771465, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "seed": "1189392917/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "chart_id": "ms-365d8c6de6de16f66744", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21318958083944717, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "seed": "1997090716/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.03280790150165558, + "branch_car_at_8": 0.00636233389377594, + "chart_id": "ms-3c326d05fec66e1d6bea", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.38232847915297685, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.02442134916782379, + "proposal_oracle_utility_gain_over_base_at_8": -0.008386552333831787, + "seed": "278008524/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.01805901527404785, + "selected_utility_gain_over_base_at_8": -0.014748886227607727, + "selector_regret_at_8": 0.00636233389377594, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755786508321762, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9819409847259521 + }, + { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1880089150142482, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "seed": "794181626/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-58950fed04d203b97ab9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13380940221299786, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1113371261/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "chart_id": "ms-651bc0395d78d2819158", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06905173445346233, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "seed": "1483645336/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9587705433368683, + "chart_id": "ms-9fad82fd288e4296e354", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2664806376409974, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "seed": "1176943004/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.041229456663131714, + "selected_utility_gain_over_base_at_8": -0.11338427662849426, + "selector_regret_at_8": 1.9587705433368683, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9587705433368683 + }, + { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.14873333521535068, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "seed": "998761162/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9767832309007645, + "chart_id": "ms-fc8084d5cd7da937fc02", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.613109405537996, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1991414747/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.023216769099235535, + "selected_utility_gain_over_base_at_8": -1.9767832309007645, + "selector_regret_at_8": 1.9767832309007645, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9767832309007645 + }, + { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.048324257135391235, + "chart_id": "ms-3afd8959363e917591d4", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2643509727736545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21781498193740845, + "proposal_oracle_utility_gain_over_base_at_8": 0.015063345432281494, + "seed": "75070641/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1694907248020172, + "selected_utility_gain_over_base_at_8": -0.03326091170310974, + "selector_regret_at_8": 0.048324257135391235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7821850180625916, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8305092751979828 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8424437642097473, + "chart_id": "ms-3ff1aedb72f65a182b58", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22991638132379041, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1055567618/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.15755623579025269, + "selected_utility_gain_over_base_at_8": -1.8424437642097473, + "selector_regret_at_8": 1.8424437642097473, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8424437642097473 + }, + { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.0, + "chart_id": "ms-4048f8f6e047422e3c79", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06066548841424423, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8651304244995117, + "proposal_oracle_utility_gain_over_base_at_8": 0.6686463505029678, + "seed": "1055157354/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8651304244995117, + "selected_utility_gain_over_base_at_8": 0.6686463505029678, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1348695755004883, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1348695755004883 + }, + { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.056304022669792175, + "chart_id": "ms-5b26c33feed6992386b1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21028164676312622, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23198044300079346, + "proposal_oracle_utility_gain_over_base_at_8": 0.033537328243255615, + "seed": "1522272686/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.17567642033100128, + "selected_utility_gain_over_base_at_8": -0.02276669442653656, + "selector_regret_at_8": 0.056304022669792175, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7680195569992065, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8243235796689987 + }, + { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "chart_id": "ms-63ac245579b1188935ad", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.08971682945849302, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "seed": "211245842/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1611655950546265, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.868122011423111, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.26263826459603723, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1665739511/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13187798857688904, + "selected_utility_gain_over_base_at_8": -1.868122011423111, + "selector_regret_at_8": 1.868122011423111, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.868122011423111 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07907943427562714, + "chart_id": "ms-72777024349c2aacd5e0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5133629211325229, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.19852638244628906, + "proposal_oracle_utility_gain_over_base_at_8": -1.801473617553711, + "seed": "256770874/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11944694817066193, + "selected_utility_gain_over_base_at_8": -1.880553051829338, + "selector_regret_at_8": 0.07907943427562714, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.801473617553711, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.880553051829338 + }, + { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.38811931014060974, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5372965735602796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5639582276344299, + "proposal_oracle_utility_gain_over_base_at_8": 0.36023636162281036, + "seed": "957101489/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1758389174938202, + "selected_utility_gain_over_base_at_8": -0.027882948517799377, + "selector_regret_at_8": 0.38811931014060974, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.43604177236557, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8241610825061798 + }, + { + "base_success": 0.0, + "base_utility": 0.8431589603424072, + "branch_car_at_8": 0.004197746515274048, + "chart_id": "ms-9a14552b7db695dee92c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.054992714566980724, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22092504799365997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6222339123487473, + "seed": "77147264/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21672730147838593, + "selected_utility_gain_over_base_at_8": -0.6264316588640213, + "selector_regret_at_8": 0.004197746515274048, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.77907495200634, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.783272698521614 + }, + { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.185863733291626, + "chart_id": "ms-9f5d57fe2153fd267774", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2143457671242473, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "seed": "1607773860/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814136266708374, + "selected_utility_gain_over_base_at_8": -0.01853656768798828, + "selector_regret_at_8": 1.185863733291626, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.185863733291626 + }, + { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.0, + "chart_id": "ms-b0fee26e1db967f90b81", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09625642405798478, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1727849841117859, + "seed": "1358421325/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1727849841117859, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.8307622671127319, + "branch_car_at_8": 1.8400099277496338, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.30145083880557255, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.169237732887268, + "seed": "1271718007/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1599900722503662, + "selected_utility_gain_over_base_at_8": -0.6707721948623657, + "selector_regret_at_8": 1.8400099277496338, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8400099277496338 + }, + { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.04582805931568146, + "chart_id": "ms-d1960368ef76bbb10669", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21826032886833752, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23963995277881622, + "proposal_oracle_utility_gain_over_base_at_8": 0.03931353986263275, + "seed": "416183106/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.19381189346313477, + "selected_utility_gain_over_base_at_8": -0.006514519453048706, + "selector_regret_at_8": 0.04582805931568146, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7603600472211838, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8061881065368652 + }, + { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.0, + "chart_id": "ms-efdc07c985935d871adf", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15321122852132601, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "seed": "706479600/train2", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1649112105369568, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.019107162952423096, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21294576266807125, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04859863221645355, + "proposal_oracle_utility_gain_over_base_at_8": 0.011399760842323303, + "seed": "1919198188/train2", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029491469264030457, + "selected_utility_gain_over_base_at_8": -0.0077074021100997925, + "selector_regret_at_8": 0.019107162952423096, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9514013677835464, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.9705085307359695 + } + ], + "summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3788416075650118, + "low": 0.23817966903073282, + "mean": 0.3085106382978724, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 1.0, + "1055567618/train1": 1.0, + "1055567618/train2": 1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 0.0, + "1189392917/train1": 0.0, + "1189392917/train2": 0.0, + "1231780997/train0": 0.0, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 0.0, + "1375492786/train0": 1.0, + "1375492786/train1": 1.0, + "1375492786/train2": 1.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 1.0, + "1475440973/train2": 1.0, + "1483645336/train0": 0.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 1.0, + "1528740417/train1": 1.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.6666666666666666, + "1549324805/train2": 0.6666666666666666, + "1607773860/train0": 0.0, + "1607773860/train1": 0.0, + "1607773860/train2": 0.0, + "1665739511/train0": 1.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 0.0, + "1747178030/train2": 0.0, + "1881223807/train0": 0.0, + "1881223807/train1": 0.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 1.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 0.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.0, + "2015956957/train2": 0.0, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 0.0, + "211245842/train1": 0.0, + "211245842/train2": 0.0, + "2141802816/train0": 0.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 0.3333333333333333, + "565369392/train1": 0.3333333333333333, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 0.0, + "706479600/train1": 0.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.25, + "767747424/train1": 0.25, + "767747424/train2": 0.25, + "77147264/train0": 0.5, + "77147264/train1": 0.5, + "77147264/train2": 0.5, + "794181626/train0": 0.0, + "794181626/train1": 0.0, + "794181626/train2": 0.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3125188536953243, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.33816425120772947, + "low": 0.21739130434782608, + "mean": 0.2753623188405797, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9440655855692033, + "low": 0.6842917830640909, + "mean": 0.8163502178883103, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.8163502178883103, + "per_seed": { + "1025740212/train0": 0.528373658657074, + "1025740212/train1": 0.528373658657074, + "1025740212/train2": 0.528373658657074, + "1055157354/train0": 0.19648407399654388, + "1055157354/train1": 0.19648407399654388, + "1055157354/train2": 0.19648407399654388, + "1055567618/train0": 2.0, + "1055567618/train1": 2.0, + "1055567618/train2": 2.0, + "1113268940/train0": 0.42231252789497375, + "1113268940/train1": 0.42231252789497375, + "1113268940/train2": 0.42231252789497375, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 0.15461373329162598, + "1176943004/train1": 0.15461373329162598, + "1176943004/train2": 0.15461373329162598, + "1189392917/train0": 0.10841864347457886, + "1189392917/train1": 0.10841864347457886, + "1189392917/train2": 0.10841864347457886, + "1231780997/train0": 0.7270061373710632, + "1231780997/train1": 0.7270061373710632, + "1231780997/train2": 0.7270061373710632, + "1271718007/train0": 1.415381133556366, + "1271718007/train1": 1.415381133556366, + "1271718007/train2": 1.415381133556366, + "1273247072/train0": 0.15342965722084045, + "1273247072/train1": 0.15342965722084045, + "1273247072/train2": 0.15342965722084045, + "1358421325/train0": 0.8272150158882141, + "1358421325/train1": 0.8272150158882141, + "1358421325/train2": 0.8272150158882141, + "1375492786/train0": 2.0, + "1375492786/train1": 2.0, + "1375492786/train2": 2.0, + "141984615/train0": 0.09381213908394177, + "141984615/train1": 0.09381213908394177, + "141984615/train2": 0.09381213908394177, + "1475440973/train0": 2.0, + "1475440973/train1": 2.0, + "1475440973/train2": 2.0, + "1483645336/train0": 0.01641605794429779, + "1483645336/train1": 0.01641605794429779, + "1483645336/train2": 0.01641605794429779, + "1522272686/train0": 0.19844311475753784, + "1522272686/train1": 0.19844311475753784, + "1522272686/train2": 0.19844311475753784, + "1528740417/train0": 2.0, + "1528740417/train1": 2.0, + "1528740417/train2": 2.0, + "1549324805/train0": 1.3787474483251572, + "1549324805/train1": 1.3787474483251572, + "1549324805/train2": 1.3787474483251572, + "1607773860/train0": 0.8326728343963623, + "1607773860/train1": 0.8326728343963623, + "1607773860/train2": 0.8326728343963623, + "1665739511/train0": 2.0, + "1665739511/train1": 2.0, + "1665739511/train2": 2.0, + "1747178030/train0": 0.14663122594356537, + "1747178030/train1": 0.14663122594356537, + "1747178030/train2": 0.14663122594356537, + "1881223807/train0": 0.0320269912481308, + "1881223807/train1": 0.0320269912481308, + "1881223807/train2": 0.0320269912481308, + "1919198188/train0": 0.03719887137413025, + "1919198188/train1": 0.03719887137413025, + "1919198188/train2": 0.03719887137413025, + "1991414747/train0": 2.0, + "1991414747/train1": 2.0, + "1991414747/train2": 2.0, + "1997090716/train0": 0.13927479088306427, + "1997090716/train1": 0.13927479088306427, + "1997090716/train2": 0.13927479088306427, + "2015956957/train0": 0.06711978986859321, + "2015956957/train1": 0.06711978986859321, + "2015956957/train2": 0.06711978986859321, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 0.8388344049453735, + "211245842/train1": 0.8388344049453735, + "211245842/train2": 0.8388344049453735, + "2141802816/train0": 0.07080724835395813, + "2141802816/train1": 0.07080724835395813, + "2141802816/train2": 0.07080724835395813, + "256770874/train0": 2.0, + "256770874/train1": 2.0, + "256770874/train2": 2.0, + "278008524/train0": 0.10747186839580536, + "278008524/train1": 0.10747186839580536, + "278008524/train2": 0.10747186839580536, + "416183106/train0": 0.20032641291618347, + "416183106/train1": 0.20032641291618347, + "416183106/train2": 0.20032641291618347, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 0.7239725589752197, + "551504635/train1": 0.7239725589752197, + "551504635/train2": 0.7239725589752197, + "565369392/train0": 0.7997656464576721, + "565369392/train1": 0.7997656464576721, + "565369392/train2": 0.7997656464576721, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.07529209802548091, + "585904130/train1": 0.07529209802548091, + "585904130/train2": 0.07529209802548091, + "626486813/train0": 0.5054400166263804, + "626486813/train1": 0.5054400166263804, + "626486813/train2": 0.5054400166263804, + "636680137/train0": 2.0, + "636680137/train1": 2.0, + "636680137/train2": 2.0, + "706479600/train0": 0.8350887894630432, + "706479600/train1": 0.8350887894630432, + "706479600/train2": 0.8350887894630432, + "750448296/train0": 0.11153355985879898, + "750448296/train1": 0.11153355985879898, + "750448296/train2": 0.11153355985879898, + "75070641/train0": 0.20275163650512695, + "75070641/train1": 0.20275163650512695, + "75070641/train2": 0.20275163650512695, + "767747424/train0": 0.5475241877138615, + "767747424/train1": 0.5475241877138615, + "767747424/train2": 0.5475241877138615, + "77147264/train0": 1.4215794801712036, + "77147264/train1": 1.4215794801712036, + "77147264/train2": 1.4215794801712036, + "794181626/train0": 0.18095605075359344, + "794181626/train1": 0.18095605075359344, + "794181626/train2": 0.18095605075359344, + "957101489/train0": 0.20372186601161957, + "957101489/train1": 0.20372186601161957, + "957101489/train2": 0.20372186601161957, + "998761162/train0": 0.06781657040119171, + "998761162/train1": 0.06781657040119171, + "998761162/train2": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9086439887682596, + "low": 0.382927798086266, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.8321223414957452, + "low": 0.6101555917457034, + "mean": 0.7180729379567007, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.558614392092107, + "low": 0.3184796732449662, + "mean": 0.42802111370607065, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.42802111370607065, + "per_seed": { + "1025740212/train0": 1.281606376171112, + "1025740212/train1": 0.0031174421310424805, + "1025740212/train2": 0.051884353160858154, + "1055157354/train0": 0.4498589485883713, + "1055157354/train1": 0.6509099900722504, + "1055157354/train2": 0.0, + "1055567618/train0": 1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": 1.8424437642097473, + "1113268940/train0": 0.11861386895179749, + "1113268940/train1": 0.06326162815093994, + "1113268940/train2": 0.48447395861148834, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.9300042241811752, + "1176943004/train1": 1.953145906329155, + "1176943004/train2": 1.9587705433368683, + "1189392917/train0": 0.0, + "1189392917/train1": 1.9264162927865982, + "1189392917/train2": 0.0, + "1231780997/train0": 1.2770342230796814, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.9082518517971039, + "1271718007/train1": 0.8952926397323608, + "1271718007/train2": 0.9200049638748169, + "1273247072/train0": 0.42020074278116226, + "1273247072/train1": 0.42682039737701416, + "1273247072/train2": 0.39307598024606705, + "1358421325/train0": 0.6354914605617523, + "1358421325/train1": 0.007964074611663818, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0008215903653763235, + "1375492786/train1": 0.001000205724267289, + "1375492786/train2": 0.00036396586801856756, + "141984615/train0": 0.05737591476645321, + "141984615/train1": 0.03783805563580245, + "141984615/train2": 0.03539413997593025, + "1475440973/train0": 1.2853952050209045, + "1475440973/train1": 0.0, + "1475440973/train2": 1.2959344387054443, + "1483645336/train0": 1.8899840712547302, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.030991941690444946, + "1522272686/train1": 0.03571280837059021, + "1522272686/train2": 0.056304022669792175, + "1528740417/train0": 0.0, + "1528740417/train1": 0.008421123027801514, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0010904477288325627, + "1549324805/train1": 0.004917144775390625, + "1549324805/train2": 0.010487629100680351, + "1607773860/train0": 1.8556141257286072, + "1607773860/train1": 0.0, + "1607773860/train2": 1.185863733291626, + "1665739511/train0": 0.0, + "1665739511/train1": 1.1815597414970398, + "1665739511/train2": 1.868122011423111, + "1747178030/train0": 0.02001962810754776, + "1747178030/train1": 1.9076288938522339, + "1747178030/train2": 0.0, + "1881223807/train0": 0.002386912703514099, + "1881223807/train1": 1.964881807565689, + "1881223807/train2": 0.0, + "1919198188/train0": 0.021911710500717163, + "1919198188/train1": 0.18855391442775726, + "1919198188/train2": 0.019107162952423096, + "1991414747/train0": 0.017522692680358887, + "1991414747/train1": 1.9461991488933563, + "1991414747/train2": 1.9767832309007645, + "1997090716/train0": 1.9452667087316513, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.39167013615369795, + "2015956957/train1": 0.005367537960410118, + "2015956957/train2": 0.4314236156642437, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.1816681623458862, + "211245842/train2": 0.0, + "2141802816/train0": 0.008425161242485046, + "2141802816/train1": 1.934090182185173, + "2141802816/train2": 1.954741045832634, + "256770874/train0": 0.44993481040000916, + "256770874/train1": 0.6129319667816162, + "256770874/train2": 0.07907943427562714, + "278008524/train0": 0.0008676897268742323, + "278008524/train1": 0.0074296220554970205, + "278008524/train2": 0.0040267681470140815, + "416183106/train0": 0.3453921228647232, + "416183106/train1": 0.01654154062271118, + "416183106/train2": 0.04582805931568146, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2800043225288391, + "551504635/train1": 1.2779860496520996, + "551504635/train2": 1.287484347820282, + "565369392/train0": 0.05178981522719065, + "565369392/train1": 0.048005685210227966, + "565369392/train2": 0.057756672302881874, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.01741097339739402, + "585904130/train1": 0.004500773424903552, + "585904130/train2": 0.006800346697370212, + "626486813/train0": 0.033813119400292635, + "626486813/train1": 0.02813516021706164, + "626486813/train2": 0.013256545644253492, + "636680137/train0": 0.0048203859478235245, + "636680137/train1": 0.004942953586578369, + "636680137/train2": 0.028240702114999294, + "706479600/train0": 0.1455947458744049, + "706479600/train1": 1.185575544834137, + "706479600/train2": 0.0, + "750448296/train0": 0.015873273834586143, + "750448296/train1": 0.04417596384882927, + "750448296/train2": 0.03573483042418957, + "75070641/train0": 0.003223627805709839, + "75070641/train1": 0.007853284478187561, + "75070641/train2": 0.048324257135391235, + "767747424/train0": 0.5240038954652846, + "767747424/train1": 0.04542438860516995, + "767747424/train2": 0.5082327332347631, + "77147264/train0": 0.013380475342273712, + "77147264/train1": 0.8879568874835968, + "77147264/train2": 0.9378202110528946, + "794181626/train0": 0.0, + "794181626/train1": 1.9626595228910446, + "794181626/train2": 0.0, + "957101489/train0": 0.3607921451330185, + "957101489/train1": 0.3404468297958374, + "957101489/train2": 0.38811931014060974, + "998761162/train0": 0.0010883957147598267, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.5834326290625793, + "low": 0.26088621819349617, + "mean": 0.45165208199755824, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.45165208199755824, + "per_task": { + "LiftPegUpright-v1": 0.5397342965006828, + "PickCube-v1": 0.4133657068014145, + "PullCube-v1": 0.1168446104449115, + "PushCube-v1": 0.6477842434094503, + "StackCube-v1": 0.5405315528313319 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4417064196763646, + "low": 0.2639209286670835, + "mean": 0.3500529956192077, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.8887706855791963, + "low": 0.7754137115839246, + "mean": 0.8358156028368796, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.8358156028368796, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 1.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 1.0, + "1055567618/train1": 1.0, + "1055567618/train2": 1.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 1.0, + "1271718007/train0": 1.0, + "1271718007/train1": 1.0, + "1271718007/train2": 1.0, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 1.0, + "1358421325/train1": 1.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 1.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 1.0, + "1528740417/train1": 1.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.6666666666666666, + "1549324805/train2": 0.6666666666666666, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 1.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 1.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 1.0, + "1881223807/train1": 1.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 1.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.2, + "2015956957/train2": 0.2, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 1.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 1.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.5, + "278008524/train1": 0.5, + "278008524/train2": 0.5, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.6666666666666666, + "565369392/train1": 0.6666666666666666, + "565369392/train2": 0.6666666666666666, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 1.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 1.0, + "77147264/train1": 1.0, + "77147264/train2": 1.0, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.7246376811594203, + "low": 0.5990338164251208, + "mean": 0.6666666666666666, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.8010394367889166, + "low": 1.5995275395488624, + "mean": 1.706202811163896, + "n": 141, + "num_samples": 1000 + }, + "mean": 1.706202811163896, + "per_seed": { + "1025740212/train0": 2.0, + "1025740212/train1": 2.0, + "1025740212/train2": 2.0, + "1055157354/train0": 2.0, + "1055157354/train1": 2.0, + "1055157354/train2": 2.0, + "1055567618/train0": 2.0, + "1055567618/train1": 2.0, + "1055567618/train2": 2.0, + "1113268940/train0": 2.0, + "1113268940/train1": 2.0, + "1113268940/train2": 2.0, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 2.0, + "1176943004/train1": 2.0, + "1176943004/train2": 2.0, + "1189392917/train0": 2.0, + "1189392917/train1": 2.0, + "1189392917/train2": 2.0, + "1231780997/train0": 2.0, + "1231780997/train1": 2.0, + "1231780997/train2": 2.0, + "1271718007/train0": 2.0, + "1271718007/train1": 2.0, + "1271718007/train2": 2.0, + "1273247072/train0": 2.0, + "1273247072/train1": 2.0, + "1273247072/train2": 2.0, + "1358421325/train0": 2.0, + "1358421325/train1": 2.0, + "1358421325/train2": 2.0, + "1375492786/train0": 0.24345096945762634, + "1375492786/train1": 0.24345096945762634, + "1375492786/train2": 0.24345096945762634, + "141984615/train0": 0.20044784247875214, + "141984615/train1": 0.20044784247875214, + "141984615/train2": 0.20044784247875214, + "1475440973/train0": 2.0, + "1475440973/train1": 2.0, + "1475440973/train2": 2.0, + "1483645336/train0": 2.0, + "1483645336/train1": 2.0, + "1483645336/train2": 2.0, + "1522272686/train0": 2.0, + "1522272686/train1": 2.0, + "1522272686/train2": 2.0, + "1528740417/train0": 2.0, + "1528740417/train1": 2.0, + "1528740417/train2": 2.0, + "1549324805/train0": 1.4027505218982697, + "1549324805/train1": 1.4027505218982697, + "1549324805/train2": 1.4027505218982697, + "1607773860/train0": 2.0, + "1607773860/train1": 2.0, + "1607773860/train2": 2.0, + "1665739511/train0": 2.0, + "1665739511/train1": 2.0, + "1665739511/train2": 2.0, + "1747178030/train0": 2.0, + "1747178030/train1": 2.0, + "1747178030/train2": 2.0, + "1881223807/train0": 2.0, + "1881223807/train1": 2.0, + "1881223807/train2": 2.0, + "1919198188/train0": 2.0, + "1919198188/train1": 2.0, + "1919198188/train2": 2.0, + "1991414747/train0": 2.0, + "1991414747/train1": 2.0, + "1991414747/train2": 2.0, + "1997090716/train0": 2.0, + "1997090716/train1": 2.0, + "1997090716/train2": 2.0, + "2015956957/train0": 0.5298813670873642, + "2015956957/train1": 0.5298813670873642, + "2015956957/train2": 0.5298813670873642, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 2.0, + "211245842/train1": 2.0, + "211245842/train2": 2.0, + "2141802816/train0": 2.0, + "2141802816/train1": 2.0, + "2141802816/train2": 2.0, + "256770874/train0": 2.0, + "256770874/train1": 2.0, + "256770874/train2": 2.0, + "278008524/train0": 1.113811194896698, + "278008524/train1": 1.113811194896698, + "278008524/train2": 1.113811194896698, + "416183106/train0": 2.0, + "416183106/train1": 2.0, + "416183106/train2": 2.0, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 2.0, + "551504635/train1": 2.0, + "551504635/train2": 2.0, + "565369392/train0": 1.4266315797964733, + "565369392/train1": 1.4266315797964733, + "565369392/train2": 1.4266315797964733, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.23764699200789133, + "585904130/train1": 0.23764699200789133, + "585904130/train2": 0.23764699200789133, + "626486813/train0": 0.6124684102833271, + "626486813/train1": 0.6124684102833271, + "626486813/train2": 0.6124684102833271, + "636680137/train0": 2.0, + "636680137/train1": 2.0, + "636680137/train2": 2.0, + "706479600/train0": 2.0, + "706479600/train1": 2.0, + "706479600/train2": 2.0, + "750448296/train0": 0.23580759018659592, + "750448296/train1": 0.23580759018659592, + "750448296/train2": 0.23580759018659592, + "75070641/train0": 2.0, + "75070641/train1": 2.0, + "75070641/train2": 2.0, + "767747424/train0": 0.18863565661013126, + "767747424/train1": 0.18863565661013126, + "767747424/train2": 0.18863565661013126, + "77147264/train0": 2.0, + "77147264/train1": 2.0, + "77147264/train2": 2.0, + "794181626/train0": 2.0, + "794181626/train1": 2.0, + "794181626/train2": 2.0, + "957101489/train0": 2.0, + "957101489/train1": 2.0, + "957101489/train2": 2.0, + "998761162/train0": 2.0, + "998761162/train1": 2.0, + "998761162/train2": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 1000 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.5062634588583657, + "low": 1.2799855626507657, + "mean": 1.3999846732054932, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.610047281323877, + "low": 0.4546099290780141, + "mean": 0.5356973995271868, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.5356973995271868, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.3333333333333333, + "141984615/train1": 0.6666666666666666, + "141984615/train2": 0.3333333333333333, + "1475440973/train0": 0.0, + "1475440973/train1": 0.0, + "1475440973/train2": 0.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 0.0, + "1528740417/train1": 0.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.0, + "1549324805/train2": 0.0, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 0.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.8, + "2015956957/train1": 0.6, + "2015956957/train2": 0.8, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 1.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.5, + "278008524/train2": 0.0, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.75, + "626486813/train1": 0.75, + "626486813/train2": 0.75, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.5, + "750448296/train1": 0.5, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.75, + "767747424/train1": 0.75, + "767747424/train2": 0.75, + "77147264/train0": 0.0, + "77147264/train1": 0.5, + "77147264/train2": 0.0, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8444444444444444, + "low": 0.4740573152337858, + "mean": 0.6400703871292106, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.6400703871292106, + "per_task": { + "LiftPegUpright-v1": 0.5555555555555556, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3627450980392157, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.6666666666666666 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5652173913043478, + "low": 0.42995169082125606, + "mean": 0.5024154589371981, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.23027504332506904, + "low": 0.1884004262069988, + "mean": 0.20904812235529077, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.20904812235529077, + "per_seed": { + "1025740212/train0": 0.20050368294022264, + "1025740212/train1": 0.14400158741031796, + "1025740212/train2": 0.12800533274597592, + "1055157354/train0": 0.43030250381223584, + "1055157354/train1": 0.32609963351937027, + "1055157354/train2": 0.06066548841424423, + "1055567618/train0": 0.30857397468613657, + "1055567618/train1": 0.15812100047218913, + "1055567618/train2": 0.22991638132379041, + "1113268940/train0": 0.3774241061869338, + "1113268940/train1": 0.06911736513585261, + "1113268940/train2": 0.2534904144040469, + "1113371261/train0": 0.026017268580811406, + "1113371261/train1": 0.015047211698777683, + "1113371261/train2": 0.13380940221299786, + "1176943004/train0": 0.1246594058194818, + "1176943004/train1": 0.15442455751472706, + "1176943004/train2": 0.2664806376409974, + "1189392917/train0": 0.1340247769860469, + "1189392917/train1": 0.10709427493066431, + "1189392917/train2": 0.33283923875771465, + "1231780997/train0": 0.19966589920839098, + "1231780997/train1": 0.07419277505792353, + "1231780997/train2": 0.16621555087386797, + "1271718007/train0": 0.17939225581112472, + "1271718007/train1": 0.10710899843183383, + "1271718007/train2": 0.20722796191327264, + "1273247072/train0": 0.16962492308566807, + "1273247072/train1": 0.1410368025527166, + "1273247072/train2": 0.2134820953504291, + "1358421325/train0": 0.28863435648886215, + "1358421325/train1": 0.04701404433315851, + "1358421325/train2": 0.09625642405798478, + "1375492786/train0": 0.0996910344508803, + "1375492786/train1": 0.16573502466028908, + "1375492786/train2": 0.29573093343728263, + "141984615/train0": 0.297780428420543, + "141984615/train1": 0.16666657870321888, + "141984615/train2": 0.22785399017277783, + "1475440973/train0": 0.41171052461838353, + "1475440973/train1": 0.0862090980213101, + "1475440973/train2": 0.15809903475948942, + "1483645336/train0": 0.35034389349653194, + "1483645336/train1": 0.06079377324000063, + "1483645336/train2": 0.06905173445346233, + "1522272686/train0": 0.31643520896589616, + "1522272686/train1": 0.24182565394435904, + "1522272686/train2": 0.21028164676312622, + "1528740417/train0": 0.15436827657008492, + "1528740417/train1": 0.026961937751403514, + "1528740417/train2": 0.3242760158848682, + "1549324805/train0": 0.23727288575803293, + "1549324805/train1": 0.10346856229255086, + "1549324805/train2": 0.2818476718903797, + "1607773860/train0": 0.3951037237909243, + "1607773860/train1": 0.2544258391757567, + "1607773860/train2": 0.2143457671242473, + "1665739511/train0": 0.0, + "1665739511/train1": 0.05717674118582694, + "1665739511/train2": 0.26263826459603723, + "1747178030/train0": 0.11661420436902857, + "1747178030/train1": 0.1055876219626688, + "1747178030/train2": 0.33194730961537694, + "1881223807/train0": 0.20266639463366093, + "1881223807/train1": 0.10190418918720931, + "1881223807/train2": 0.1643392210528543, + "1919198188/train0": 0.09083822075370537, + "1919198188/train1": 0.2569381375337174, + "1919198188/train2": 0.21294576266807125, + "1991414747/train0": 0.09717880195623664, + "1991414747/train1": 0.44645064197830664, + "1991414747/train2": 0.613109405537996, + "1997090716/train0": 0.3641822422309461, + "1997090716/train1": 0.3553584127845171, + "1997090716/train2": 0.21318958083944717, + "2015956957/train0": 0.22628515797280127, + "2015956957/train1": 0.2284508985012672, + "2015956957/train2": 0.24731300175116924, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.14834095177221712, + "211245842/train1": 0.036206191308278124, + "211245842/train2": 0.08971682945849302, + "2141802816/train0": 0.07518554002658885, + "2141802816/train1": 0.40989236974714055, + "2141802816/train2": 0.3183279052482729, + "256770874/train0": 0.18757436508611464, + "256770874/train1": 0.3193203145842822, + "256770874/train2": 0.5133629211325229, + "278008524/train0": 0.10616076660738837, + "278008524/train1": 0.10627711331165646, + "278008524/train2": 0.40905575479886586, + "416183106/train0": 0.1517874168346938, + "416183106/train1": 0.1925298309196861, + "416183106/train2": 0.21826032886833752, + "534717141/train0": 0.18033001516380143, + "534717141/train1": 0.12736602361488303, + "534717141/train2": 0.15703220108978472, + "551504635/train0": 0.3791036296516339, + "551504635/train1": 0.23627606487156766, + "551504635/train2": 0.4327797462092032, + "565369392/train0": 0.21578265463075008, + "565369392/train1": 0.2859321646983037, + "565369392/train2": 0.2176430432872923, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.3300621116998719, + "585904130/train1": 0.1652498530853035, + "585904130/train2": 0.22254352429757418, + "626486813/train0": 0.24681469768825515, + "626486813/train1": 0.20632809068391378, + "626486813/train2": 0.14946876705188256, + "636680137/train0": 0.19331728269593437, + "636680137/train1": 0.45862923715160336, + "636680137/train2": 0.6813618274184183, + "706479600/train0": 0.2692917587355154, + "706479600/train1": 0.15129914778803955, + "706479600/train2": 0.15321122852132601, + "750448296/train0": 0.3795450047628438, + "750448296/train1": 0.1527117559178966, + "750448296/train2": 0.19031133205081463, + "75070641/train0": 0.13808834192306887, + "75070641/train1": 0.34224193319263285, + "75070641/train2": 0.2643509727736545, + "767747424/train0": 0.2798082148668758, + "767747424/train1": 0.13255125751519692, + "767747424/train2": 0.20625644377911734, + "77147264/train0": 0.22561784152146835, + "77147264/train1": 0.03379253116769787, + "77147264/train2": 0.13662790487889595, + "794181626/train0": 0.3405496231635778, + "794181626/train1": 0.3619001242329895, + "794181626/train2": 0.1880089150142482, + "957101489/train0": 0.2493251178612577, + "957101489/train1": 0.2708447347116597, + "957101489/train2": 0.5372965735602796, + "998761162/train0": 0.1991858618764328, + "998761162/train1": 0.24035198055524792, + "998761162/train2": 0.14873333521535068 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.21752558375317918, + "low": 0.18602643769659166, + "mean": 0.20364489391565144, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.20364489391565144, + "per_task": { + "LiftPegUpright-v1": 0.2123556134611932, + "PickCube-v1": 0.17471460699627128, + "PullCube-v1": 0.21225649007113187, + "PushCube-v1": 0.19356765816347193, + "StackCube-v1": 0.2253301008861889 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.22903802877401228, + "low": 0.19393562395778954, + "mean": 0.21104202926956342, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5638297872340425, + "low": 0.4011820330969268, + "mean": 0.4791962174940898, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.4791962174940898, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 1.0, + "1055567618/train1": 1.0, + "1055567618/train2": 1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 1.0, + "1271718007/train1": 1.0, + "1271718007/train2": 1.0, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 0.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 1.0, + "1528740417/train1": 0.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 1.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 0.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.0, + "2015956957/train2": 0.2, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.0, + "626486813/train1": 0.0, + "626486813/train2": 0.0, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.25, + "767747424/train1": 0.0, + "767747424/train2": 0.25, + "77147264/train0": 0.5, + "77147264/train1": 1.0, + "77147264/train2": 0.5, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6272498743086977, + "low": 0.1503267973856209, + "mean": 0.38930115635997986, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.38930115635997986, + "per_task": { + "LiftPegUpright-v1": 0.5555555555555556, + "PickCube-v1": 0.0, + "PullCube-v1": 0.19607843137254902, + "PushCube-v1": 0.7948717948717948, + "StackCube-v1": 0.4 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4444444444444444, + "low": 0.3188405797101449, + "mean": 0.38164251207729466, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.2685579196217494, + "low": 0.0768321513002364, + "mean": 0.1706855791962175, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.1706855791962175, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": -1.0, + "1375492786/train1": -1.0, + "1375492786/train2": -1.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 0.0, + "1475440973/train1": -1.0, + "1475440973/train2": 0.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 0.0, + "1528740417/train1": -1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": -0.3333333333333333, + "1549324805/train2": -0.3333333333333333, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": -1.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.0, + "2015956957/train2": 0.2, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": -1.0, + "256770874/train1": -1.0, + "256770874/train2": -1.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": -0.3333333333333333, + "565369392/train1": -0.3333333333333333, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": -0.25, + "626486813/train1": -0.25, + "626486813/train2": -0.25, + "636680137/train0": -1.0, + "636680137/train1": -1.0, + "636680137/train2": -1.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.0, + "767747424/train1": -0.25, + "767747424/train2": 0.0, + "77147264/train0": 0.0, + "77147264/train1": 0.5, + "77147264/train2": 0.0, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3367521367521367, + "low": -0.014379084967320262, + "mean": 0.16227249874308697, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.16227249874308697, + "per_task": { + "LiftPegUpright-v1": 0.2222222222222222, + "PickCube-v1": 0.0, + "PullCube-v1": -0.09803921568627451, + "PushCube-v1": 0.48717948717948717, + "StackCube-v1": 0.2 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.18357487922705315, + "low": 0.03864734299516908, + "mean": 0.10628019323671498, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.2493007679215036, + "low": 0.9614900063099141, + "mean": 1.1006925804823984, + "n": 141, + "num_samples": 1000 + }, + "mean": 1.1006925804823984, + "per_seed": { + "1025740212/train0": 2.0, + "1025740212/train1": 0.2148684412240982, + "1025740212/train2": 0.7146708369255066, + "1055157354/train0": 0.5705926418304443, + "1055157354/train1": 0.812659502029419, + "1055157354/train2": 0.8651304244995117, + "1055567618/train0": 2.0, + "1055567618/train1": 2.0, + "1055567618/train2": 2.0, + "1113268940/train0": 0.5588552355766296, + "1113268940/train1": 0.7256770730018616, + "1113268940/train2": 0.6990174651145935, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 2.0, + "1176943004/train1": 2.0, + "1176943004/train2": 2.0, + "1189392917/train0": 2.0, + "1189392917/train1": 2.0, + "1189392917/train2": 2.0, + "1231780997/train0": 2.0, + "1231780997/train1": 2.0, + "1231780997/train2": 0.725460946559906, + "1271718007/train0": 2.0, + "1271718007/train1": 2.0, + "1271718007/train2": 2.0, + "1273247072/train0": 0.4659159183502197, + "1273247072/train1": 0.46439141035079956, + "1273247072/train2": 0.4708559513092041, + "1358421325/train0": 0.8147144317626953, + "1358421325/train1": 0.8245300054550171, + "1358421325/train2": 2.0, + "1375492786/train0": 0.0015756288776174188, + "1375492786/train1": 0.001480837701819837, + "1375492786/train2": 0.0012095769634470344, + "141984615/train0": 0.06459133536554873, + "141984615/train1": 0.07189804397057742, + "141984615/train2": 0.08839488192461431, + "1475440973/train0": 2.0, + "1475440973/train1": 0.7181311249732971, + "1475440973/train2": 2.0, + "1483645336/train0": 2.0, + "1483645336/train1": 2.0, + "1483645336/train2": 2.0, + "1522272686/train0": 0.21208032965660095, + "1522272686/train1": 0.24029652774333954, + "1522272686/train2": 0.23198044300079346, + "1528740417/train0": 2.0, + "1528740417/train1": 0.7255622744560242, + "1528740417/train2": 2.0, + "1549324805/train0": 1.344537331412236, + "1549324805/train1": 0.6790153384208679, + "1549324805/train2": 0.6867360668256879, + "1607773860/train0": 2.0, + "1607773860/train1": 2.0, + "1607773860/train2": 2.0, + "1665739511/train0": 2.0, + "1665739511/train1": 2.0, + "1665739511/train2": 2.0, + "1747178030/train0": 0.1273784190416336, + "1747178030/train1": 2.0, + "1747178030/train2": 2.0, + "1881223807/train0": 0.027498364448547363, + "1881223807/train1": 2.0, + "1881223807/train2": 0.040287479758262634, + "1919198188/train0": 0.09917314350605011, + "1919198188/train1": 0.2203868180513382, + "1919198188/train2": 0.04859863221645355, + "1991414747/train0": 0.13679209351539612, + "1991414747/train1": 2.0, + "1991414747/train2": 2.0, + "1997090716/train0": 2.0, + "1997090716/train1": 2.0, + "1997090716/train2": 2.0, + "2015956957/train0": 0.46120103299617765, + "2015956957/train1": 0.06726333405822515, + "2015956957/train2": 0.45940959043800833, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 0.8135772347450256, + "211245842/train1": 2.0, + "211245842/train2": 2.0, + "2141802816/train0": 0.08595690131187439, + "2141802816/train1": 2.0, + "2141802816/train2": 2.0, + "256770874/train0": 0.574478030204773, + "256770874/train1": 0.8128684163093567, + "256770874/train2": 0.19852638244628906, + "278008524/train0": 0.013509750366210938, + "278008524/train1": 0.019306766451336443, + "278008524/train2": 0.01439405232667923, + "416183106/train0": 0.581784188747406, + "416183106/train1": 0.22490769624710083, + "416183106/train2": 0.23963995277881622, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 2.0, + "551504635/train1": 2.0, + "551504635/train2": 2.0, + "565369392/train0": 0.14469666282335916, + "565369392/train1": 0.11818361282348633, + "565369392/train2": 0.7870745907227198, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.034002700820565224, + "585904130/train1": 0.01988216955214739, + "585904130/train2": 0.029573063055674236, + "626486813/train0": 0.07908930000849068, + "626486813/train1": 0.06953954696655273, + "626486813/train2": 0.06192737747915089, + "636680137/train0": 0.024462858214974403, + "636680137/train1": 0.015581388957798481, + "636680137/train2": 0.036433976143598557, + "706479600/train0": 0.21975377202033997, + "706479600/train1": 2.0, + "706479600/train2": 2.0, + "750448296/train0": 0.109864491969347, + "750448296/train1": 0.10195614024996758, + "750448296/train2": 0.08245231583714485, + "75070641/train0": 0.2283252328634262, + "75070641/train1": 0.24087366461753845, + "75070641/train2": 0.21781498193740845, + "767747424/train0": 0.5921752639114857, + "767747424/train1": 0.10304383677430451, + "767747424/train2": 0.5790140572935343, + "77147264/train0": 1.1132144927978516, + "77147264/train1": 2.0, + "77147264/train2": 1.11046252399683, + "794181626/train0": 2.0, + "794181626/train1": 2.0, + "794181626/train2": 2.0, + "957101489/train0": 0.5799027681350708, + "957101489/train1": 0.5798611640930176, + "957101489/train2": 0.5639582276344299, + "998761162/train0": 0.09276536107063293, + "998761162/train1": 2.0, + "998761162/train2": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.374731707572937, + "low": 0.5777494399969655, + "mean": 0.9933715603032812, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.9933715603032812, + "per_task": { + "LiftPegUpright-v1": 1.3934579665462177, + "PickCube-v1": 0.46705442667007446, + "PullCube-v1": 0.44994396614321674, + "PushCube-v1": 1.6016510277986526, + "StackCube-v1": 1.054750414358245 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.9970273724183494, + "low": 0.7636547594911595, + "mean": 0.8807045891532306, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.4425467990562275, + "low": 0.12576461925014992, + "mean": 0.2843423625940884, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.2843423625940884, + "per_seed": { + "1025740212/train0": 1.471626341342926, + "1025740212/train1": -0.31350521743297577, + "1025740212/train2": 0.18629717826843262, + "1055157354/train0": 0.37410856783390045, + "1055157354/train1": 0.6161754280328751, + "1055157354/train2": 0.6686463505029678, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 0.13654270768165588, + "1113268940/train1": 0.3033645451068878, + "1113268940/train2": 0.27670493721961975, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.845386266708374, + "1176943004/train1": 1.845386266708374, + "1176943004/train2": 1.845386266708374, + "1189392917/train0": 1.8915813565254211, + "1189392917/train1": 1.8915813565254211, + "1189392917/train2": 1.8915813565254211, + "1231780997/train0": 1.2729938626289368, + "1231780997/train1": 1.2729938626289368, + "1231780997/train2": -0.0015451908111572266, + "1271718007/train0": 0.584618866443634, + "1271718007/train1": 0.584618866443634, + "1271718007/train2": 0.584618866443634, + "1273247072/train0": 0.3124862611293793, + "1273247072/train1": 0.3109617531299591, + "1273247072/train2": 0.31742629408836365, + "1358421325/train0": -0.012500584125518799, + "1358421325/train1": -0.0026850104331970215, + "1358421325/train2": 1.1727849841117859, + "1375492786/train0": -1.9984243711223826, + "1375492786/train1": -1.9985191622981802, + "1375492786/train2": -1.998790423036553, + "141984615/train0": -0.029220803718393046, + "141984615/train1": -0.021914095113364358, + "141984615/train2": -0.005417257159327467, + "1475440973/train0": 0.0, + "1475440973/train1": -1.2818688750267029, + "1475440973/train2": 0.0, + "1483645336/train0": 1.9835839420557022, + "1483645336/train1": 1.9835839420557022, + "1483645336/train2": 1.9835839420557022, + "1522272686/train0": 0.01363721489906311, + "1522272686/train1": 0.0418534129858017, + "1522272686/train2": 0.033537328243255615, + "1528740417/train0": 0.0, + "1528740417/train1": -1.2744377255439758, + "1528740417/train2": 0.0, + "1549324805/train0": -0.03421011691292127, + "1549324805/train1": -0.6997321099042892, + "1549324805/train2": -0.6920113814994693, + "1607773860/train0": 1.1673271656036377, + "1607773860/train1": 1.1673271656036377, + "1607773860/train2": 1.1673271656036377, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": -0.019252806901931763, + "1747178030/train1": 1.8533687740564346, + "1747178030/train2": 1.8533687740564346, + "1881223807/train0": -0.004528626799583435, + "1881223807/train1": 1.9679730087518692, + "1881223807/train2": 0.008260488510131836, + "1919198188/train0": 0.06197427213191986, + "1919198188/train1": 0.18318794667720795, + "1919198188/train2": 0.011399760842323303, + "1991414747/train0": -1.8632079064846039, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 1.8607252091169357, + "1997090716/train1": 1.8607252091169357, + "1997090716/train2": 1.8607252091169357, + "2015956957/train0": 0.39408124312758447, + "2015956957/train1": 0.00014354418963193893, + "2015956957/train2": 0.3922898005694151, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": -0.0252571702003479, + "211245842/train1": 1.1611655950546265, + "211245842/train2": 1.1611655950546265, + "2141802816/train0": 0.01514965295791626, + "2141802816/train1": 1.9291927516460419, + "2141802816/train2": 1.9291927516460419, + "256770874/train0": -1.425521969795227, + "256770874/train1": -1.1871315836906433, + "256770874/train2": -1.801473617553711, + "278008524/train0": -0.09396211802959442, + "278008524/train1": -0.08816510194446892, + "278008524/train2": -0.09307781606912613, + "416183106/train0": 0.38145777583122253, + "416183106/train1": 0.02458128333091736, + "416183106/train2": 0.03931353986263275, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2760274410247803, + "551504635/train1": 1.2760274410247803, + "551504635/train2": 1.2760274410247803, + "565369392/train0": -0.655068983634313, + "565369392/train1": -0.6815820336341858, + "565369392/train2": -0.012691055734952291, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": -0.04128939720491568, + "585904130/train1": -0.055409928473333515, + "585904130/train2": -0.04571903496980667, + "626486813/train0": -0.4263507166178897, + "626486813/train1": -0.43590046965982765, + "626486813/train2": -0.4435126391472295, + "636680137/train0": -1.9755371417850256, + "636680137/train1": -1.9844186110422015, + "636680137/train2": -1.9635660238564014, + "706479600/train0": -0.6153350174427032, + "706479600/train1": 1.1649112105369568, + "706479600/train2": 1.1649112105369568, + "750448296/train0": -0.0016690678894519806, + "750448296/train1": -0.009577419608831406, + "750448296/train2": -0.02908124402165413, + "75070641/train0": 0.025573596358299255, + "75070641/train1": 0.0381220281124115, + "75070641/train2": 0.015063345432281494, + "767747424/train0": 0.04465107619762421, + "767747424/train1": -0.44448035093955696, + "767747424/train2": 0.03148986957967281, + "77147264/train0": -0.30836498737335205, + "77147264/train1": 0.5784205198287964, + "77147264/train2": -0.3111169561743736, + "794181626/train0": 1.8190439492464066, + "794181626/train1": 1.8190439492464066, + "794181626/train2": 1.8190439492464066, + "957101489/train0": 0.37618090212345123, + "957101489/train1": 0.376139298081398, + "957101489/train2": 0.36023636162281036, + "998761162/train0": 0.024948790669441223, + "998761162/train1": 1.9321834295988083, + "998761162/train2": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6703226770982782, + "low": 0.004672606112709377, + "mean": 0.3229939035546056, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.3229939035546056, + "per_task": { + "LiftPegUpright-v1": 0.3265138193964958, + "PickCube-v1": 0.313624769449234, + "PullCube-v1": -0.20129550277830704, + "PushCube-v1": 0.9244865671946452, + "StackCube-v1": 0.25163986451096004 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.28991549138085476, + "low": 0.04696476409731406, + "mean": 0.16263165119652984, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3250591016548463, + "low": 0.1879432624113475, + "mean": 0.25650118203309696, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.25650118203309696, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 0.0, + "1055567618/train1": 1.0, + "1055567618/train2": 0.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 1.0, + "1189392917/train1": 0.0, + "1189392917/train2": 1.0, + "1231780997/train0": 0.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 0.0, + "1475440973/train1": 0.0, + "1475440973/train2": 0.0, + "1483645336/train0": 0.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 1.0, + "1528740417/train1": 0.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 0.0, + "1607773860/train1": 1.0, + "1607773860/train2": 0.0, + "1665739511/train0": 1.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 0.0, + "1747178030/train1": 0.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 0.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 0.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 0.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.0, + "2015956957/train2": 0.0, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 0.0, + "211245842/train1": 0.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.0, + "626486813/train1": 0.0, + "626486813/train2": 0.0, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 0.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 0.5, + "77147264/train1": 0.5, + "77147264/train2": 0.0, + "794181626/train0": 1.0, + "794181626/train1": 0.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.333001508295626, + "low": 0.06, + "mean": 0.18171945701357467, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.18171945701357467, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.13333333333333333 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2560386473429952, + "low": 0.1497584541062802, + "mean": 0.2028985507246377, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.033687943262411355, + "low": -0.14243498817966901, + "mean": -0.052009456264775406, + "n": 141, + "num_samples": 1000 + }, + "mean": -0.052009456264775406, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": -1.0, + "1055567618/train1": 0.0, + "1055567618/train2": -1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 1.0, + "1189392917/train1": 0.0, + "1189392917/train2": 1.0, + "1231780997/train0": 0.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.0, + "1271718007/train1": 0.0, + "1271718007/train2": 0.0, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": -1.0, + "1375492786/train1": -1.0, + "1375492786/train2": -1.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": -1.0, + "1475440973/train1": -1.0, + "1475440973/train2": -1.0, + "1483645336/train0": 0.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 0.0, + "1528740417/train1": -1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": -0.3333333333333333, + "1549324805/train2": -0.3333333333333333, + "1607773860/train0": 0.0, + "1607773860/train1": 1.0, + "1607773860/train2": 0.0, + "1665739511/train0": 0.0, + "1665739511/train1": -1.0, + "1665739511/train2": -1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 0.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 0.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": -1.0, + "1991414747/train1": -1.0, + "1991414747/train2": -1.0, + "1997090716/train0": 0.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.0, + "2015956957/train2": 0.0, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 0.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": -1.0, + "256770874/train1": -1.0, + "256770874/train2": -1.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": -0.3333333333333333, + "565369392/train1": -0.3333333333333333, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": -0.25, + "626486813/train1": -0.25, + "626486813/train2": -0.25, + "636680137/train0": -1.0, + "636680137/train1": -1.0, + "636680137/train2": -1.0, + "706479600/train0": 0.0, + "706479600/train1": 0.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": -0.25, + "767747424/train1": -0.25, + "767747424/train2": -0.25, + "77147264/train0": 0.0, + "77147264/train1": 0.0, + "77147264/train2": -0.5, + "794181626/train0": 1.0, + "794181626/train1": 0.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.05897435897435899, + "low": -0.1388235294117647, + "mean": -0.04530920060331824, + "n": 5, + "num_samples": 1000 + }, + "mean": -0.04530920060331824, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.14705882352941177, + "PushCube-v1": 0.15384615384615385, + "StackCube-v1": -0.06666666666666667 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.004830917874396135, + "low": -0.13526570048309178, + "mean": -0.07246376811594203, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.8042305547841185, + "low": 0.5404751630640456, + "mean": 0.6726714667763278, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.6726714667763278, + "per_seed": { + "1025740212/train0": 0.7183936238288879, + "1025740212/train1": 0.21175099909305573, + "1025740212/train2": 0.6627864837646484, + "1055157354/train0": 0.12073369324207306, + "1055157354/train1": 0.16174951195716858, + "1055157354/train2": 0.8651304244995117, + "1055567618/train0": 0.1717667579650879, + "1055567618/train1": 2.0, + "1055567618/train2": 0.15755623579025269, + "1113268940/train0": 0.44024136662483215, + "1113268940/train1": 0.6624154448509216, + "1113268940/train2": 0.21454350650310516, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 0.06999577581882477, + "1176943004/train1": 0.04685409367084503, + "1176943004/train2": 0.041229456663131714, + "1189392917/train0": 2.0, + "1189392917/train1": 0.0735837072134018, + "1189392917/train2": 2.0, + "1231780997/train0": 0.7229657769203186, + "1231780997/train1": 2.0, + "1231780997/train2": 0.725460946559906, + "1271718007/train0": 1.0917481482028961, + "1271718007/train1": 1.1047073602676392, + "1271718007/train2": 1.079995036125183, + "1273247072/train0": 0.045715175569057465, + "1273247072/train1": 0.0375710129737854, + "1273247072/train2": 0.07777997106313705, + "1358421325/train0": 0.179222971200943, + "1358421325/train1": 0.8165659308433533, + "1358421325/train2": 2.0, + "1375492786/train0": 0.0007540385122410953, + "1375492786/train1": 0.00048063197755254805, + "1375492786/train2": 0.0008456110954284668, + "141984615/train0": 0.007215420599095523, + "141984615/train1": 0.03405998833477497, + "141984615/train2": 0.05300074194868406, + "1475440973/train0": 0.7146047949790955, + "1475440973/train1": 0.7181311249732971, + "1475440973/train2": 0.7040655612945557, + "1483645336/train0": 0.11001592874526978, + "1483645336/train1": 2.0, + "1483645336/train2": 2.0, + "1522272686/train0": 0.181088387966156, + "1522272686/train1": 0.20458371937274933, + "1522272686/train2": 0.17567642033100128, + "1528740417/train0": 2.0, + "1528740417/train1": 0.7171411514282227, + "1528740417/train2": 2.0, + "1549324805/train0": 1.3434468836834033, + "1549324805/train1": 0.6740981936454773, + "1549324805/train2": 0.6762484377250075, + "1607773860/train0": 0.14438587427139282, + "1607773860/train1": 2.0, + "1607773860/train2": 0.814136266708374, + "1665739511/train0": 2.0, + "1665739511/train1": 0.8184402585029602, + "1665739511/train2": 0.13187798857688904, + "1747178030/train0": 0.10735879093408585, + "1747178030/train1": 0.09237110614776611, + "1747178030/train2": 2.0, + "1881223807/train0": 0.025111451745033264, + "1881223807/train1": 0.03511819243431091, + "1881223807/train2": 0.040287479758262634, + "1919198188/train0": 0.07726143300533295, + "1919198188/train1": 0.03183290362358093, + "1919198188/train2": 0.029491469264030457, + "1991414747/train0": 0.11926940083503723, + "1991414747/train1": 0.05380085110664368, + "1991414747/train2": 0.023216769099235535, + "1997090716/train0": 0.054733291268348694, + "1997090716/train1": 2.0, + "1997090716/train2": 2.0, + "2015956957/train0": 0.0695308968424797, + "2015956957/train1": 0.06189579609781504, + "2015956957/train2": 0.02798597477376461, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 0.8135772347450256, + "211245842/train1": 0.8183318376541138, + "211245842/train2": 2.0, + "2141802816/train0": 0.07753174006938934, + "2141802816/train1": 0.06590981781482697, + "2141802816/train2": 0.04525895416736603, + "256770874/train0": 0.1245432198047638, + "256770874/train1": 0.19993644952774048, + "256770874/train2": 0.11944694817066193, + "278008524/train0": 0.012642060639336705, + "278008524/train1": 0.011877144395839423, + "278008524/train2": 0.010367284179665148, + "416183106/train0": 0.2363920658826828, + "416183106/train1": 0.20836615562438965, + "416183106/train2": 0.19381189346313477, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 0.7199956774711609, + "551504635/train1": 0.7220139503479004, + "551504635/train2": 0.712515652179718, + "565369392/train0": 0.09290684759616852, + "565369392/train1": 0.07017792761325836, + "565369392/train2": 0.729317918419838, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.016591727423171203, + "585904130/train1": 0.015381396127243837, + "585904130/train2": 0.022772716358304024, + "626486813/train0": 0.04527618060819805, + "626486813/train1": 0.041404386749491096, + "626486813/train2": 0.0486708318348974, + "636680137/train0": 0.01964247226715088, + "636680137/train1": 0.010638435371220112, + "636680137/train2": 0.008193274028599262, + "706479600/train0": 0.07415902614593506, + "706479600/train1": 0.814424455165863, + "706479600/train2": 2.0, + "750448296/train0": 0.09399121813476086, + "750448296/train1": 0.057780176401138306, + "750448296/train2": 0.046717485412955284, + "75070641/train0": 0.22510160505771637, + "75070641/train1": 0.2330203801393509, + "75070641/train2": 0.1694907248020172, + "767747424/train0": 0.06817136844620109, + "767747424/train1": 0.05761944816913456, + "767747424/train2": 0.07078132405877113, + "77147264/train0": 1.0998340174555779, + "77147264/train1": 1.1120431125164032, + "77147264/train2": 0.1726423129439354, + "794181626/train0": 2.0, + "794181626/train1": 0.03734047710895538, + "794181626/train2": 2.0, + "957101489/train0": 0.2191106230020523, + "957101489/train1": 0.23941433429718018, + "957101489/train2": 0.1758389174938202, + "998761162/train0": 0.09167696535587311, + "998761162/train1": 2.0, + "998761162/train2": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.805851331210544, + "low": 0.23790077653196123, + "mean": 0.5417194783057231, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.5417194783057231, + "per_task": { + "LiftPegUpright-v1": 0.8537236700455347, + "PickCube-v1": 0.05368871986865997, + "PullCube-v1": 0.33309935569830523, + "PushCube-v1": 0.9538667843892024, + "StackCube-v1": 0.5142188615269131 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6390190568436522, + "low": 0.4293877342067702, + "mean": 0.5306515935340228, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.006899217451949011, + "low": -0.2966154331014432, + "mean": -0.14367875111198225, + "n": 141, + "num_samples": 1000 + }, + "mean": -0.14367875111198225, + "per_seed": { + "1025740212/train0": 0.19001996517181396, + "1025740212/train1": -0.31662265956401825, + "1025740212/train2": 0.13441282510757446, + "1055157354/train0": -0.07575038075447083, + "1055157354/train1": -0.034734562039375305, + "1055157354/train2": 0.6686463505029678, + "1055567618/train0": -1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": -1.8424437642097473, + "1113268940/train0": 0.0179288387298584, + "1113268940/train1": 0.24010291695594788, + "1113268940/train2": -0.2077690213918686, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": -0.08461795747280121, + "1176943004/train1": -0.10775963962078094, + "1176943004/train2": -0.11338427662849426, + "1189392917/train0": 1.8915813565254211, + "1189392917/train1": -0.03483493626117706, + "1189392917/train2": 1.8915813565254211, + "1231780997/train0": -0.004040360450744629, + "1231780997/train1": 1.2729938626289368, + "1231780997/train2": -0.0015451908111572266, + "1271718007/train0": -0.32363298535346985, + "1271718007/train1": -0.3106737732887268, + "1271718007/train2": -0.33538609743118286, + "1273247072/train0": -0.10771448165178299, + "1273247072/train1": -0.11585864424705505, + "1273247072/train2": -0.0756496861577034, + "1358421325/train0": -0.6479920446872711, + "1358421325/train1": -0.01064908504486084, + "1358421325/train2": 1.1727849841117859, + "1375492786/train0": -1.999245961487759, + "1375492786/train1": -1.9995193680224475, + "1375492786/train2": -1.9991543889045715, + "141984615/train0": -0.08659671848484625, + "141984615/train1": -0.05975215074916681, + "141984615/train2": -0.04081139713525772, + "1475440973/train0": -1.2853952050209045, + "1475440973/train1": -1.2818688750267029, + "1475440973/train2": -1.2959344387054443, + "1483645336/train0": 0.09359987080097198, + "1483645336/train1": 1.9835839420557022, + "1483645336/train2": 1.9835839420557022, + "1522272686/train0": -0.017354726791381836, + "1522272686/train1": 0.006140604615211487, + "1522272686/train2": -0.02276669442653656, + "1528740417/train0": 0.0, + "1528740417/train1": -1.2828588485717773, + "1528740417/train2": 0.0, + "1549324805/train0": -0.03530056464175383, + "1549324805/train1": -0.7046492546796799, + "1549324805/train2": -0.7024990106001496, + "1607773860/train0": -0.6882869601249695, + "1607773860/train1": 1.1673271656036377, + "1607773860/train2": -0.01853656768798828, + "1665739511/train0": 0.0, + "1665739511/train1": -1.1815597414970398, + "1665739511/train2": -1.868122011423111, + "1747178030/train0": -0.03927243500947952, + "1747178030/train1": -0.054260119795799255, + "1747178030/train2": 1.8533687740564346, + "1881223807/train0": -0.006915539503097534, + "1881223807/train1": 0.0030912011861801147, + "1881223807/train2": 0.008260488510131836, + "1919198188/train0": 0.0400625616312027, + "1919198188/train1": -0.005365967750549316, + "1919198188/train2": -0.0077074021100997925, + "1991414747/train0": -1.8807305991649628, + "1991414747/train1": -1.9461991488933563, + "1991414747/train2": -1.9767832309007645, + "1997090716/train0": -0.08454149961471558, + "1997090716/train1": 1.8607252091169357, + "1997090716/train2": 1.8607252091169357, + "2015956957/train0": 0.0024111069738864898, + "2015956957/train1": -0.005223993770778179, + "2015956957/train2": -0.039133815094828606, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": -0.0252571702003479, + "211245842/train1": -0.020502567291259766, + "211245842/train2": 1.1611655950546265, + "2141802816/train0": 0.006724491715431213, + "2141802816/train1": -0.0048974305391311646, + "2141802816/train2": -0.025548294186592102, + "256770874/train0": -1.8754567801952362, + "256770874/train1": -1.8000635504722595, + "256770874/train2": -1.880553051829338, + "278008524/train0": -0.09482980775646865, + "278008524/train1": -0.09559472399996594, + "278008524/train2": -0.09710458421614021, + "416183106/train0": 0.03606565296649933, + "416183106/train1": 0.008039742708206177, + "416183106/train2": -0.006514519453048706, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": -0.003976881504058838, + "551504635/train1": -0.001958608627319336, + "551504635/train2": -0.011456906795501709, + "565369392/train0": -0.7068587988615036, + "565369392/train1": -0.7295877188444138, + "565369392/train2": -0.07044772803783417, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": -0.058700370602309704, + "585904130/train1": -0.05991070189823707, + "585904130/train2": -0.05251938166717688, + "626486813/train0": -0.46016383601818234, + "626486813/train1": -0.4640356298768893, + "626486813/train2": -0.456769184791483, + "636680137/train0": -1.9803575277328491, + "636680137/train1": -1.9893615646287799, + "636680137/train2": -1.9918067259714007, + "706479600/train0": -0.7609297633171082, + "706479600/train1": -0.020664334297180176, + "706479600/train2": 1.1649112105369568, + "750448296/train0": -0.017542341724038124, + "750448296/train1": -0.053753383457660675, + "750448296/train2": -0.0648160744458437, + "75070641/train0": 0.022349968552589417, + "75070641/train1": 0.030268743634223938, + "75070641/train2": -0.03326091170310974, + "767747424/train0": -0.4793528192676604, + "767747424/train1": -0.4899047395447269, + "767747424/train2": -0.47674286365509033, + "77147264/train0": -0.32174546271562576, + "77147264/train1": -0.3095363676548004, + "77147264/train2": -1.2489371672272682, + "794181626/train0": 1.8190439492464066, + "794181626/train1": -0.14361557364463806, + "794181626/train2": 1.8190439492464066, + "957101489/train0": 0.01538875699043274, + "957101489/train1": 0.03569246828556061, + "957101489/train2": -0.027882948517799377, + "998761162/train0": 0.023860394954681396, + "998761162/train1": 1.9321834295988083, + "998761162/train2": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.0807332034294422, + "low": -0.2854568160382735, + "mean": -0.12865817844295258, + "n": 5, + "num_samples": 1000 + }, + "mean": -0.12865817844295258, + "per_task": { + "LiftPegUpright-v1": -0.213220477104187, + "PickCube-v1": -0.09974093735218048, + "PullCube-v1": -0.31814011322321856, + "PushCube-v1": 0.276702323785195, + "StackCube-v1": -0.2888916883203718 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.07080568554462496, + "low": -0.2967283767217811, + "mean": -0.1874213444226779, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.558614392092107, + "low": 0.3184796732449662, + "mean": 0.42802111370607065, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.42802111370607065, + "per_seed": { + "1025740212/train0": 1.281606376171112, + "1025740212/train1": 0.0031174421310424805, + "1025740212/train2": 0.051884353160858154, + "1055157354/train0": 0.4498589485883713, + "1055157354/train1": 0.6509099900722504, + "1055157354/train2": 0.0, + "1055567618/train0": 1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": 1.8424437642097473, + "1113268940/train0": 0.11861386895179749, + "1113268940/train1": 0.06326162815093994, + "1113268940/train2": 0.48447395861148834, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.9300042241811752, + "1176943004/train1": 1.953145906329155, + "1176943004/train2": 1.9587705433368683, + "1189392917/train0": 0.0, + "1189392917/train1": 1.9264162927865982, + "1189392917/train2": 0.0, + "1231780997/train0": 1.2770342230796814, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.9082518517971039, + "1271718007/train1": 0.8952926397323608, + "1271718007/train2": 0.9200049638748169, + "1273247072/train0": 0.42020074278116226, + "1273247072/train1": 0.42682039737701416, + "1273247072/train2": 0.39307598024606705, + "1358421325/train0": 0.6354914605617523, + "1358421325/train1": 0.007964074611663818, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0008215903653763235, + "1375492786/train1": 0.001000205724267289, + "1375492786/train2": 0.00036396586801856756, + "141984615/train0": 0.05737591476645321, + "141984615/train1": 0.03783805563580245, + "141984615/train2": 0.03539413997593025, + "1475440973/train0": 1.2853952050209045, + "1475440973/train1": 0.0, + "1475440973/train2": 1.2959344387054443, + "1483645336/train0": 1.8899840712547302, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.030991941690444946, + "1522272686/train1": 0.03571280837059021, + "1522272686/train2": 0.056304022669792175, + "1528740417/train0": 0.0, + "1528740417/train1": 0.008421123027801514, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0010904477288325627, + "1549324805/train1": 0.004917144775390625, + "1549324805/train2": 0.010487629100680351, + "1607773860/train0": 1.8556141257286072, + "1607773860/train1": 0.0, + "1607773860/train2": 1.185863733291626, + "1665739511/train0": 0.0, + "1665739511/train1": 1.1815597414970398, + "1665739511/train2": 1.868122011423111, + "1747178030/train0": 0.02001962810754776, + "1747178030/train1": 1.9076288938522339, + "1747178030/train2": 0.0, + "1881223807/train0": 0.002386912703514099, + "1881223807/train1": 1.964881807565689, + "1881223807/train2": 0.0, + "1919198188/train0": 0.021911710500717163, + "1919198188/train1": 0.18855391442775726, + "1919198188/train2": 0.019107162952423096, + "1991414747/train0": 0.017522692680358887, + "1991414747/train1": 1.9461991488933563, + "1991414747/train2": 1.9767832309007645, + "1997090716/train0": 1.9452667087316513, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.39167013615369795, + "2015956957/train1": 0.005367537960410118, + "2015956957/train2": 0.4314236156642437, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.1816681623458862, + "211245842/train2": 0.0, + "2141802816/train0": 0.008425161242485046, + "2141802816/train1": 1.934090182185173, + "2141802816/train2": 1.954741045832634, + "256770874/train0": 0.44993481040000916, + "256770874/train1": 0.6129319667816162, + "256770874/train2": 0.07907943427562714, + "278008524/train0": 0.0008676897268742323, + "278008524/train1": 0.0074296220554970205, + "278008524/train2": 0.0040267681470140815, + "416183106/train0": 0.3453921228647232, + "416183106/train1": 0.01654154062271118, + "416183106/train2": 0.04582805931568146, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2800043225288391, + "551504635/train1": 1.2779860496520996, + "551504635/train2": 1.287484347820282, + "565369392/train0": 0.05178981522719065, + "565369392/train1": 0.048005685210227966, + "565369392/train2": 0.057756672302881874, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.01741097339739402, + "585904130/train1": 0.004500773424903552, + "585904130/train2": 0.006800346697370212, + "626486813/train0": 0.033813119400292635, + "626486813/train1": 0.02813516021706164, + "626486813/train2": 0.013256545644253492, + "636680137/train0": 0.0048203859478235245, + "636680137/train1": 0.004942953586578369, + "636680137/train2": 0.028240702114999294, + "706479600/train0": 0.1455947458744049, + "706479600/train1": 1.185575544834137, + "706479600/train2": 0.0, + "750448296/train0": 0.015873273834586143, + "750448296/train1": 0.04417596384882927, + "750448296/train2": 0.03573483042418957, + "75070641/train0": 0.003223627805709839, + "75070641/train1": 0.007853284478187561, + "75070641/train2": 0.048324257135391235, + "767747424/train0": 0.5240038954652846, + "767747424/train1": 0.04542438860516995, + "767747424/train2": 0.5082327332347631, + "77147264/train0": 0.013380475342273712, + "77147264/train1": 0.8879568874835968, + "77147264/train2": 0.9378202110528946, + "794181626/train0": 0.0, + "794181626/train1": 1.9626595228910446, + "794181626/train2": 0.0, + "957101489/train0": 0.3607921451330185, + "957101489/train1": 0.3404468297958374, + "957101489/train2": 0.38811931014060974, + "998761162/train0": 0.0010883957147598267, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.5834326290625793, + "low": 0.26088621819349617, + "mean": 0.45165208199755824, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.45165208199755824, + "per_task": { + "LiftPegUpright-v1": 0.5397342965006828, + "PickCube-v1": 0.4133657068014145, + "PullCube-v1": 0.1168446104449115, + "PushCube-v1": 0.6477842434094503, + "StackCube-v1": 0.5405315528313319 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4417064196763646, + "low": 0.2639209286670835, + "mean": 0.3500529956192077, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.298936170212766, + "low": 0.15638297872340423, + "mean": 0.2226950354609929, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.2226950354609929, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 1.0, + "1055567618/train1": 0.0, + "1055567618/train2": 1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 0.0, + "1189392917/train1": 1.0, + "1189392917/train2": 0.0, + "1231780997/train0": 1.0, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 0.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 0.0, + "1528740417/train1": 0.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.0, + "1549324805/train2": 0.0, + "1607773860/train0": 1.0, + "1607773860/train1": 0.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 0.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 0.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.0, + "2015956957/train2": 0.2, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 0.0, + "2141802816/train0": 0.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.0, + "626486813/train1": 0.0, + "626486813/train2": 0.0, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.25, + "767747424/train1": 0.0, + "767747424/train2": 0.25, + "77147264/train0": 0.0, + "77147264/train1": 0.5, + "77147264/train2": 0.5, + "794181626/train0": 0.0, + "794181626/train1": 1.0, + "794181626/train2": 0.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3311111111111111, + "low": 0.07294117647058823, + "mean": 0.2075816993464052, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.2075816993464052, + "per_task": { + "LiftPegUpright-v1": 0.3888888888888889, + "PickCube-v1": 0.0, + "PullCube-v1": 0.049019607843137254, + "PushCube-v1": 0.3333333333333333, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.23671497584541062, + "low": 0.12560386473429952, + "mean": 0.178743961352657, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.43900709219858153, + "low": 0.28451536643026, + "mean": 0.3601654846335697, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.3601654846335697, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 1.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 0.0, + "1189392917/train1": 0.0, + "1189392917/train2": 0.0, + "1231780997/train0": 0.0, + "1231780997/train1": 0.0, + "1231780997/train2": 1.0, + "1271718007/train0": 0.0, + "1271718007/train1": 0.0, + "1271718007/train2": 0.0, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 1.0, + "1358421325/train1": 1.0, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 0.0, + "1475440973/train1": 1.0, + "1475440973/train2": 0.0, + "1483645336/train0": 0.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 0.0, + "1528740417/train1": 1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 0.0, + "1607773860/train1": 0.0, + "1607773860/train2": 0.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 1.0, + "1747178030/train1": 0.0, + "1747178030/train2": 0.0, + "1881223807/train0": 1.0, + "1881223807/train1": 0.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 1.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 0.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.2, + "2015956957/train2": 0.0, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.0, + "211245842/train1": 0.0, + "211245842/train2": 0.0, + "2141802816/train0": 1.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.5, + "278008524/train1": 0.5, + "278008524/train2": 0.5, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 0.6666666666666666, + "565369392/train1": 0.6666666666666666, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 1.0, + "706479600/train1": 0.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 0.5, + "77147264/train1": 0.0, + "77147264/train2": 0.5, + "794181626/train0": 0.0, + "794181626/train1": 0.0, + "794181626/train2": 0.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7610256410256411, + "low": 0.22976370035193563, + "mean": 0.47932629462041226, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.47932629462041226, + "per_task": { + "LiftPegUpright-v1": 0.4444444444444444, + "PickCube-v1": 1.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.20512820512820512, + "StackCube-v1": 0.6 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.36231884057971014, + "low": 0.2318840579710145, + "mean": 0.2946859903381642, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6560283687943264, + "low": 0.5061465721040188, + "mean": 0.5793144208037826, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.5793144208037826, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 1.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 1.0, + "1055567618/train1": 0.0, + "1055567618/train2": 1.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 0.0, + "1189392917/train1": 1.0, + "1189392917/train2": 0.0, + "1231780997/train0": 1.0, + "1231780997/train1": 0.0, + "1231780997/train2": 1.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 1.0, + "1358421325/train1": 1.0, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 1.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 0.0, + "1528740417/train1": 1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 1.0, + "1607773860/train1": 0.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 1.0, + "1747178030/train1": 1.0, + "1747178030/train2": 0.0, + "1881223807/train0": 1.0, + "1881223807/train1": 1.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 1.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.2, + "2015956957/train2": 0.2, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.0, + "211245842/train1": 1.0, + "211245842/train2": 0.0, + "2141802816/train0": 1.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.5, + "278008524/train1": 0.5, + "278008524/train2": 0.5, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.6666666666666666, + "565369392/train1": 0.6666666666666666, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 1.0, + "706479600/train1": 1.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 0.5, + "77147264/train1": 0.5, + "77147264/train2": 1.0, + "794181626/train0": 0.0, + "794181626/train1": 1.0, + "794181626/train2": 0.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9133333333333333, + "low": 0.3936651583710407, + "mean": 0.6829864253393665, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.6829864253393665, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.8666666666666667 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5314009661835749, + "low": 0.3961352657004831, + "mean": 0.463768115942029, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7356242316190471, + "low": 0.4883750475817041, + "mean": 0.6119969386407603, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.6119969386407603, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 1.7851315587759018, + "1025740212/train2": 1.2853291630744934, + "1055157354/train0": 1.4294073581695557, + "1055157354/train1": 1.187340497970581, + "1055157354/train2": 1.1348695755004883, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 1.4411447644233704, + "1113268940/train1": 1.2743229269981384, + "1113268940/train2": 1.3009825348854065, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 0.0, + "1189392917/train1": 0.0, + "1189392917/train2": 0.0, + "1231780997/train0": 0.0, + "1231780997/train1": 0.0, + "1231780997/train2": 1.274539053440094, + "1271718007/train0": 0.0, + "1271718007/train1": 0.0, + "1271718007/train2": 0.0, + "1273247072/train0": 1.5340840816497803, + "1273247072/train1": 1.5356085896492004, + "1273247072/train2": 1.529144048690796, + "1358421325/train0": 1.1852855682373047, + "1358421325/train1": 1.175469994544983, + "1358421325/train2": 0.0, + "1375492786/train0": 0.24187534058000892, + "1375492786/train1": 0.2419701317558065, + "1375492786/train2": 0.2422413924941793, + "141984615/train0": 0.1358565071132034, + "141984615/train1": 0.12854979850817472, + "141984615/train2": 0.11378062601822118, + "1475440973/train0": 0.0, + "1475440973/train1": 1.2818688750267029, + "1475440973/train2": 0.0, + "1483645336/train0": 0.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.787919670343399, + "1522272686/train1": 1.7597034722566605, + "1522272686/train2": 1.7680195569992065, + "1528740417/train0": 0.0, + "1528740417/train1": 1.2744377255439758, + "1528740417/train2": 0.0, + "1549324805/train0": 0.05821319048603376, + "1549324805/train1": 0.7237351834774017, + "1549324805/train2": 0.7160144550725818, + "1607773860/train0": 0.0, + "1607773860/train1": 0.0, + "1607773860/train2": 0.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 1.8726215809583664, + "1747178030/train1": 0.0, + "1747178030/train2": 0.0, + "1881223807/train0": 1.9725016355514526, + "1881223807/train1": 0.0, + "1881223807/train2": 1.9597125202417374, + "1919198188/train0": 1.90082685649395, + "1919198188/train1": 1.7796131819486618, + "1919198188/train2": 1.9514013677835464, + "1991414747/train0": 1.8632079064846039, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 0.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.06868033409118653, + "2015956957/train1": 0.46261803302913906, + "2015956957/train2": 0.07047177664935589, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.1864227652549744, + "211245842/train1": 0.0, + "211245842/train2": 0.0, + "2141802816/train0": 1.9140430986881256, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 1.425521969795227, + "256770874/train1": 1.1871315836906433, + "256770874/train2": 1.801473617553711, + "278008524/train0": 1.100301444530487, + "278008524/train1": 1.0945044284453616, + "278008524/train2": 1.0994171425700188, + "416183106/train0": 1.418215811252594, + "416183106/train1": 1.7750923037528992, + "416183106/train2": 1.7603600472211838, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 1.281934916973114, + "565369392/train1": 1.308447966972987, + "565369392/train2": 0.6395569890737534, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.2036442911873261, + "585904130/train1": 0.21776482245574394, + "585904130/train2": 0.2080739289522171, + "626486813/train0": 0.544636546401307, + "626486813/train1": 0.5429288633167744, + "626486813/train2": 0.5505410328041762, + "636680137/train0": 1.9755371417850256, + "636680137/train1": 1.9844186110422015, + "636680137/train2": 1.9635660238564014, + "706479600/train0": 1.78024622797966, + "706479600/train1": 0.0, + "706479600/train2": 0.0, + "750448296/train0": 0.12594309821724892, + "750448296/train1": 0.13385144993662834, + "750448296/train2": 0.15335527434945107, + "75070641/train0": 1.7716747671365738, + "75070641/train1": 1.7591263353824615, + "75070641/train2": 1.7821850180625916, + "767747424/train0": 0.04815509542822838, + "767747424/train1": 0.08559181983582675, + "767747424/train2": 0.05956761725246906, + "77147264/train0": 0.8867855072021484, + "77147264/train1": 0.0, + "77147264/train2": 0.88953747600317, + "794181626/train0": 0.0, + "794181626/train1": 0.0, + "794181626/train2": 0.0, + "957101489/train0": 1.4200972318649292, + "957101489/train1": 1.4201388359069824, + "957101489/train2": 1.43604177236557, + "998761162/train0": 1.907234638929367, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.1884870555665759, + "low": 0.4279394403967422, + "mean": 0.7702629553380855, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.7702629553380855, + "per_task": { + "LiftPegUpright-v1": 0.6065420334537824, + "PickCube-v1": 1.5329455733299255, + "PullCube-v1": 0.36822861206361696, + "PushCube-v1": 0.39834897220134735, + "StackCube-v1": 0.945249585641755 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6440332006670958, + "low": 0.437744075585953, + "mean": 0.5369456667747744, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.1599391866837798, + "low": 0.9069358115994052, + "mean": 1.0335313443875687, + "n": 141, + "num_samples": 1000 + }, + "mean": 1.0335313443875687, + "per_seed": { + "1025740212/train0": 1.281606376171112, + "1025740212/train1": 1.7882490009069443, + "1025740212/train2": 1.3372135162353516, + "1055157354/train0": 1.879266306757927, + "1055157354/train1": 1.8382504880428314, + "1055157354/train2": 1.1348695755004883, + "1055567618/train0": 1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": 1.8424437642097473, + "1113268940/train0": 1.5597586333751678, + "1113268940/train1": 1.3375845551490784, + "1113268940/train2": 1.7854564934968948, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.9300042241811752, + "1176943004/train1": 1.953145906329155, + "1176943004/train2": 1.9587705433368683, + "1189392917/train0": 0.0, + "1189392917/train1": 1.9264162927865982, + "1189392917/train2": 0.0, + "1231780997/train0": 1.2770342230796814, + "1231780997/train1": 0.0, + "1231780997/train2": 1.274539053440094, + "1271718007/train0": 0.9082518517971039, + "1271718007/train1": 0.8952926397323608, + "1271718007/train2": 0.9200049638748169, + "1273247072/train0": 1.9542848244309425, + "1273247072/train1": 1.9624289870262146, + "1273247072/train2": 1.922220028936863, + "1358421325/train0": 1.820777028799057, + "1358421325/train1": 1.1834340691566467, + "1358421325/train2": 0.0, + "1375492786/train0": 0.24269693094538525, + "1375492786/train1": 0.2429703374800738, + "1375492786/train2": 0.24260535836219788, + "141984615/train0": 0.1932324218796566, + "141984615/train1": 0.16638785414397717, + "141984615/train2": 0.14744710053006807, + "1475440973/train0": 1.2853952050209045, + "1475440973/train1": 1.2818688750267029, + "1475440973/train2": 1.2959344387054443, + "1483645336/train0": 1.8899840712547302, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.818911612033844, + "1522272686/train1": 1.7954162806272507, + "1522272686/train2": 1.8243235796689987, + "1528740417/train0": 0.0, + "1528740417/train1": 1.2828588485717773, + "1528740417/train2": 0.0, + "1549324805/train0": 0.05930363821486632, + "1549324805/train1": 0.7286523282527924, + "1549324805/train2": 0.7265020841732621, + "1607773860/train0": 1.8556141257286072, + "1607773860/train1": 0.0, + "1607773860/train2": 1.185863733291626, + "1665739511/train0": 0.0, + "1665739511/train1": 1.1815597414970398, + "1665739511/train2": 1.868122011423111, + "1747178030/train0": 1.8926412090659142, + "1747178030/train1": 1.9076288938522339, + "1747178030/train2": 0.0, + "1881223807/train0": 1.9748885482549667, + "1881223807/train1": 1.964881807565689, + "1881223807/train2": 1.9597125202417374, + "1919198188/train0": 1.922738566994667, + "1919198188/train1": 1.968167096376419, + "1919198188/train2": 1.9705085307359695, + "1991414747/train0": 1.8807305991649628, + "1991414747/train1": 1.9461991488933563, + "1991414747/train2": 1.9767832309007645, + "1997090716/train0": 1.9452667087316513, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.4603504702448845, + "2015956957/train1": 0.4679855709895492, + "2015956957/train2": 0.5018953923135996, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.1864227652549744, + "211245842/train1": 1.1816681623458862, + "211245842/train2": 0.0, + "2141802816/train0": 1.9224682599306107, + "2141802816/train1": 1.934090182185173, + "2141802816/train2": 1.954741045832634, + "256770874/train0": 1.8754567801952362, + "256770874/train1": 1.8000635504722595, + "256770874/train2": 1.880553051829338, + "278008524/train0": 1.1011691342573613, + "278008524/train1": 1.1019340505008586, + "278008524/train2": 1.1034439107170328, + "416183106/train0": 1.7636079341173172, + "416183106/train1": 1.7916338443756104, + "416183106/train2": 1.8061881065368652, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2800043225288391, + "551504635/train1": 1.2779860496520996, + "551504635/train2": 1.287484347820282, + "565369392/train0": 1.3337247322003047, + "565369392/train1": 1.356453652183215, + "565369392/train2": 0.6973136613766352, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.22105526458472013, + "585904130/train1": 0.2222655958806475, + "585904130/train2": 0.2148742756495873, + "626486813/train0": 0.5671922296751291, + "626486813/train1": 0.571064023533836, + "626486813/train2": 0.5637975784484297, + "636680137/train0": 1.9803575277328491, + "636680137/train1": 1.9893615646287799, + "636680137/train2": 1.9918067259714007, + "706479600/train0": 1.925840973854065, + "706479600/train1": 1.185575544834137, + "706479600/train2": 0.0, + "750448296/train0": 0.14181637205183506, + "750448296/train1": 0.1780274137854576, + "750448296/train2": 0.18909010477364063, + "75070641/train0": 1.7748983949422836, + "75070641/train1": 1.766979619860649, + "75070641/train2": 1.8305092751979828, + "767747424/train0": 0.12046428816393018, + "767747424/train1": 0.1310162084409967, + "767747424/train2": 0.11785433255136013, + "77147264/train0": 0.9001659825444221, + "77147264/train1": 0.8879568874835968, + "77147264/train2": 1.8273576870560646, + "794181626/train0": 0.0, + "794181626/train1": 1.9626595228910446, + "794181626/train2": 0.0, + "957101489/train0": 1.7808893769979477, + "957101489/train1": 1.7605856657028198, + "957101489/train2": 1.8241610825061798, + "998761162/train0": 1.908323034644127, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.674169638895581, + "low": 0.7759163594774237, + "mean": 1.2147448890541535, + "n": 5, + "num_samples": 1000 + }, + "mean": 1.2147448890541535, + "per_task": { + "LiftPegUpright-v1": 1.1462763299544652, + "PickCube-v1": 1.94631128013134, + "PullCube-v1": 0.44922248110107793, + "PushCube-v1": 1.0461332156107976, + "StackCube-v1": 1.4857811384730868 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.9811940932400909, + "low": 0.7554519398117963, + "mean": 0.8693330796714703, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + } + }, + "thresholds": [ + 0.2, + 0.4 + ] +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics_by_seed.json b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..77d86957efc60dc4988bf21a3a4d4cbd31a04b57 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics_by_seed.json @@ -0,0 +1,3245 @@ +{ + "1025740212/train0": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 1.281606376171112, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.20050368294022264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.471626341342926, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7183936238288879, + "selected_utility_gain_over_base_at_8": 0.19001996517181396, + "selector_regret_at_8": 1.281606376171112, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.281606376171112 + }, + "1025740212/train1": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.0031174421310424805, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14400158741031796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2148684412240982, + "proposal_oracle_utility_gain_over_base_at_8": -0.31350521743297577, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21175099909305573, + "selected_utility_gain_over_base_at_8": -0.31662265956401825, + "selector_regret_at_8": 0.0031174421310424805, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7851315587759018, + "total_car_to_hidden_at_8": 1.7882490009069443 + }, + "1025740212/train2": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.051884353160858154, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12800533274597592, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7146708369255066, + "proposal_oracle_utility_gain_over_base_at_8": 0.18629717826843262, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6627864837646484, + "selected_utility_gain_over_base_at_8": 0.13441282510757446, + "selector_regret_at_8": 0.051884353160858154, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2853291630744934, + "total_car_to_hidden_at_8": 1.3372135162353516 + }, + "1055157354/train0": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.4498589485883713, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.43030250381223584, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5705926418304443, + "proposal_oracle_utility_gain_over_base_at_8": 0.37410856783390045, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12073369324207306, + "selected_utility_gain_over_base_at_8": -0.07575038075447083, + "selector_regret_at_8": 0.4498589485883713, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4294073581695557, + "total_car_to_hidden_at_8": 1.879266306757927 + }, + "1055157354/train1": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.6509099900722504, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32609963351937027, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.812659502029419, + "proposal_oracle_utility_gain_over_base_at_8": 0.6161754280328751, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.16174951195716858, + "selected_utility_gain_over_base_at_8": -0.034734562039375305, + "selector_regret_at_8": 0.6509099900722504, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.187340497970581, + "total_car_to_hidden_at_8": 1.8382504880428314 + }, + "1055157354/train2": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06066548841424423, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8651304244995117, + "proposal_oracle_utility_gain_over_base_at_8": 0.6686463505029678, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8651304244995117, + "selected_utility_gain_over_base_at_8": 0.6686463505029678, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1348695755004883, + "total_car_to_hidden_at_8": 1.1348695755004883 + }, + "1055567618/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.828233242034912, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30857397468613657, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1717667579650879, + "selected_utility_gain_over_base_at_8": -1.828233242034912, + "selector_regret_at_8": 1.828233242034912, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.828233242034912 + }, + "1055567618/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15812100047218913, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1055567618/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8424437642097473, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22991638132379041, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.15755623579025269, + "selected_utility_gain_over_base_at_8": -1.8424437642097473, + "selector_regret_at_8": 1.8424437642097473, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8424437642097473 + }, + "1113268940/train0": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.11861386895179749, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3774241061869338, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5588552355766296, + "proposal_oracle_utility_gain_over_base_at_8": 0.13654270768165588, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.44024136662483215, + "selected_utility_gain_over_base_at_8": 0.0179288387298584, + "selector_regret_at_8": 0.11861386895179749, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4411447644233704, + "total_car_to_hidden_at_8": 1.5597586333751678 + }, + "1113268940/train1": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.06326162815093994, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06911736513585261, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7256770730018616, + "proposal_oracle_utility_gain_over_base_at_8": 0.3033645451068878, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6624154448509216, + "selected_utility_gain_over_base_at_8": 0.24010291695594788, + "selector_regret_at_8": 0.06326162815093994, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2743229269981384, + "total_car_to_hidden_at_8": 1.3375845551490784 + }, + "1113268940/train2": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.48447395861148834, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2534904144040469, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.6990174651145935, + "proposal_oracle_utility_gain_over_base_at_8": 0.27670493721961975, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21454350650310516, + "selected_utility_gain_over_base_at_8": -0.2077690213918686, + "selector_regret_at_8": 0.48447395861148834, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.3009825348854065, + "total_car_to_hidden_at_8": 1.7854564934968948 + }, + "1113371261/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026017268580811406, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1113371261/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.015047211698777683, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1113371261/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13380940221299786, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004/train0": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9300042241811752, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1246594058194818, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06999577581882477, + "selected_utility_gain_over_base_at_8": -0.08461795747280121, + "selector_regret_at_8": 1.9300042241811752, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9300042241811752 + }, + "1176943004/train1": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.953145906329155, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15442455751472706, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04685409367084503, + "selected_utility_gain_over_base_at_8": -0.10775963962078094, + "selector_regret_at_8": 1.953145906329155, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.953145906329155 + }, + "1176943004/train2": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9587705433368683, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2664806376409974, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.041229456663131714, + "selected_utility_gain_over_base_at_8": -0.11338427662849426, + "selector_regret_at_8": 1.9587705433368683, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9587705433368683 + }, + "1189392917/train0": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1340247769860469, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1189392917/train1": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 1.9264162927865982, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10709427493066431, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0735837072134018, + "selected_utility_gain_over_base_at_8": -0.03483493626117706, + "selector_regret_at_8": 1.9264162927865982, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9264162927865982 + }, + "1189392917/train2": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33283923875771465, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997/train0": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 1.2770342230796814, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.19966589920839098, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7229657769203186, + "selected_utility_gain_over_base_at_8": -0.004040360450744629, + "selector_regret_at_8": 1.2770342230796814, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2770342230796814 + }, + "1231780997/train1": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07419277505792353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.2729938626289368, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997/train2": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16621555087386797, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.725460946559906, + "proposal_oracle_utility_gain_over_base_at_8": -0.0015451908111572266, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.725460946559906, + "selected_utility_gain_over_base_at_8": -0.0015451908111572266, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.274539053440094, + "total_car_to_hidden_at_8": 1.274539053440094 + }, + "1271718007/train0": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9082518517971039, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.17939225581112472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0917481482028961, + "selected_utility_gain_over_base_at_8": -0.32363298535346985, + "selector_regret_at_8": 0.9082518517971039, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9082518517971039 + }, + "1271718007/train1": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.8952926397323608, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10710899843183383, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1047073602676392, + "selected_utility_gain_over_base_at_8": -0.3106737732887268, + "selector_regret_at_8": 0.8952926397323608, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8952926397323608 + }, + "1271718007/train2": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9200049638748169, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.20722796191327264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.079995036125183, + "selected_utility_gain_over_base_at_8": -0.33538609743118286, + "selector_regret_at_8": 0.9200049638748169, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9200049638748169 + }, + "1273247072/train0": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + "1273247072/train1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + "1273247072/train2": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "total_car_to_hidden_at_8": 1.922220028936863 + }, + "1358421325/train0": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.6354914605617523, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.28863435648886215, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8147144317626953, + "proposal_oracle_utility_gain_over_base_at_8": -0.012500584125518799, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.179222971200943, + "selected_utility_gain_over_base_at_8": -0.6479920446872711, + "selector_regret_at_8": 0.6354914605617523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1852855682373047, + "total_car_to_hidden_at_8": 1.820777028799057 + }, + "1358421325/train1": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.007964074611663818, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.04701404433315851, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8245300054550171, + "proposal_oracle_utility_gain_over_base_at_8": -0.0026850104331970215, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8165659308433533, + "selected_utility_gain_over_base_at_8": -0.01064908504486084, + "selector_regret_at_8": 0.007964074611663818, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.175469994544983, + "total_car_to_hidden_at_8": 1.1834340691566467 + }, + "1358421325/train2": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09625642405798478, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1727849841117859, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1727849841117859, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1375492786/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008215903653763235, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0996910344508803, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0015756288776174188, + "proposal_oracle_utility_gain_over_base_at_8": -1.9984243711223826, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0007540385122410953, + "selected_utility_gain_over_base_at_8": -1.999245961487759, + "selector_regret_at_8": 0.0008215903653763235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24187534058000892, + "total_car_to_hidden_at_8": 0.24269693094538525 + }, + "1375492786/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.001000205724267289, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16573502466028908, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.001480837701819837, + "proposal_oracle_utility_gain_over_base_at_8": -1.9985191622981802, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.00048063197755254805, + "selected_utility_gain_over_base_at_8": -1.9995193680224475, + "selector_regret_at_8": 0.001000205724267289, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2419701317558065, + "total_car_to_hidden_at_8": 0.2429703374800738 + }, + "1375492786/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00036396586801856756, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.29573093343728263, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0012095769634470344, + "proposal_oracle_utility_gain_over_base_at_8": -1.998790423036553, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0008456110954284668, + "selected_utility_gain_over_base_at_8": -1.9991543889045715, + "selector_regret_at_8": 0.00036396586801856756, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2422413924941793, + "total_car_to_hidden_at_8": 0.24260535836219788 + }, + "141984615/train0": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.05737591476645321, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.297780428420543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06459133536554873, + "proposal_oracle_utility_gain_over_base_at_8": -0.029220803718393046, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.007215420599095523, + "selected_utility_gain_over_base_at_8": -0.08659671848484625, + "selector_regret_at_8": 0.05737591476645321, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1358565071132034, + "total_car_to_hidden_at_8": 0.1932324218796566 + }, + "141984615/train1": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03783805563580245, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.16666657870321888, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07189804397057742, + "proposal_oracle_utility_gain_over_base_at_8": -0.021914095113364358, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03405998833477497, + "selected_utility_gain_over_base_at_8": -0.05975215074916681, + "selector_regret_at_8": 0.03783805563580245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12854979850817472, + "total_car_to_hidden_at_8": 0.16638785414397717 + }, + "141984615/train2": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03539413997593025, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.22785399017277783, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08839488192461431, + "proposal_oracle_utility_gain_over_base_at_8": -0.005417257159327467, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05300074194868406, + "selected_utility_gain_over_base_at_8": -0.04081139713525772, + "selector_regret_at_8": 0.03539413997593025, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11378062601822118, + "total_car_to_hidden_at_8": 0.14744710053006807 + }, + "1475440973/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2853952050209045, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.41171052461838353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7146047949790955, + "selected_utility_gain_over_base_at_8": -1.2853952050209045, + "selector_regret_at_8": 1.2853952050209045, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2853952050209045 + }, + "1475440973/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0862090980213101, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7181311249732971, + "proposal_oracle_utility_gain_over_base_at_8": -1.2818688750267029, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7181311249732971, + "selected_utility_gain_over_base_at_8": -1.2818688750267029, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2818688750267029, + "total_car_to_hidden_at_8": 1.2818688750267029 + }, + "1475440973/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2959344387054443, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15809903475948942, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7040655612945557, + "selected_utility_gain_over_base_at_8": -1.2959344387054443, + "selector_regret_at_8": 1.2959344387054443, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2959344387054443 + }, + "1483645336/train0": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 1.8899840712547302, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35034389349653194, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11001592874526978, + "selected_utility_gain_over_base_at_8": 0.09359987080097198, + "selector_regret_at_8": 1.8899840712547302, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8899840712547302 + }, + "1483645336/train1": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06079377324000063, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1483645336/train2": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06905173445346233, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1522272686/train0": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.030991941690444946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.31643520896589616, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21208032965660095, + "proposal_oracle_utility_gain_over_base_at_8": 0.01363721489906311, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.181088387966156, + "selected_utility_gain_over_base_at_8": -0.017354726791381836, + "selector_regret_at_8": 0.030991941690444946, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.787919670343399, + "total_car_to_hidden_at_8": 1.818911612033844 + }, + "1522272686/train1": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.03571280837059021, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24182565394435904, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24029652774333954, + "proposal_oracle_utility_gain_over_base_at_8": 0.0418534129858017, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20458371937274933, + "selected_utility_gain_over_base_at_8": 0.006140604615211487, + "selector_regret_at_8": 0.03571280837059021, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7597034722566605, + "total_car_to_hidden_at_8": 1.7954162806272507 + }, + "1522272686/train2": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.056304022669792175, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21028164676312622, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23198044300079346, + "proposal_oracle_utility_gain_over_base_at_8": 0.033537328243255615, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.17567642033100128, + "selected_utility_gain_over_base_at_8": -0.02276669442653656, + "selector_regret_at_8": 0.056304022669792175, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7680195569992065, + "total_car_to_hidden_at_8": 1.8243235796689987 + }, + "1528740417/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15436827657008492, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1528740417/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.008421123027801514, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026961937751403514, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7255622744560242, + "proposal_oracle_utility_gain_over_base_at_8": -1.2744377255439758, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7171411514282227, + "selected_utility_gain_over_base_at_8": -1.2828588485717773, + "selector_regret_at_8": 0.008421123027801514, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2744377255439758, + "total_car_to_hidden_at_8": 1.2828588485717773 + }, + "1528740417/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3242760158848682, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1549324805/train0": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.0010904477288325627, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.23727288575803293, + "proposal_oracle_success_at_8": 0.6666666666666666, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.344537331412236, + "proposal_oracle_utility_gain_over_base_at_8": -0.03421011691292127, + "selected_success_at_8": 0.6666666666666666, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.3434468836834033, + "selected_utility_gain_over_base_at_8": -0.03530056464175383, + "selector_regret_at_8": 0.0010904477288325627, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05821319048603376, + "total_car_to_hidden_at_8": 0.05930363821486632 + }, + "1549324805/train1": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.004917144775390625, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10346856229255086, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6790153384208679, + "proposal_oracle_utility_gain_over_base_at_8": -0.6997321099042892, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6740981936454773, + "selected_utility_gain_over_base_at_8": -0.7046492546796799, + "selector_regret_at_8": 0.004917144775390625, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7237351834774017, + "total_car_to_hidden_at_8": 0.7286523282527924 + }, + "1549324805/train2": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.010487629100680351, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2818476718903797, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6867360668256879, + "proposal_oracle_utility_gain_over_base_at_8": -0.6920113814994693, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6762484377250075, + "selected_utility_gain_over_base_at_8": -0.7024990106001496, + "selector_regret_at_8": 0.010487629100680351, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7160144550725818, + "total_car_to_hidden_at_8": 0.7265020841732621 + }, + "1607773860/train0": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.8556141257286072, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3951037237909243, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14438587427139282, + "selected_utility_gain_over_base_at_8": -0.6882869601249695, + "selector_regret_at_8": 1.8556141257286072, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8556141257286072 + }, + "1607773860/train1": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2544258391757567, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1673271656036377, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1607773860/train2": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.185863733291626, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2143457671242473, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814136266708374, + "selected_utility_gain_over_base_at_8": -0.01853656768798828, + "selector_regret_at_8": 1.185863733291626, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185863733291626 + }, + "1665739511/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1665739511/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.1815597414970398, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.05717674118582694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.8184402585029602, + "selected_utility_gain_over_base_at_8": -1.1815597414970398, + "selector_regret_at_8": 1.1815597414970398, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1815597414970398 + }, + "1665739511/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.868122011423111, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.26263826459603723, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13187798857688904, + "selected_utility_gain_over_base_at_8": -1.868122011423111, + "selector_regret_at_8": 1.868122011423111, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.868122011423111 + }, + "1747178030/train0": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.02001962810754776, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11661420436902857, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1273784190416336, + "proposal_oracle_utility_gain_over_base_at_8": -0.019252806901931763, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10735879093408585, + "selected_utility_gain_over_base_at_8": -0.03927243500947952, + "selector_regret_at_8": 0.02001962810754776, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8726215809583664, + "total_car_to_hidden_at_8": 1.8926412090659142 + }, + "1747178030/train1": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 1.9076288938522339, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1055876219626688, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09237110614776611, + "selected_utility_gain_over_base_at_8": -0.054260119795799255, + "selector_regret_at_8": 1.9076288938522339, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9076288938522339 + }, + "1747178030/train2": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33194730961537694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8533687740564346, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1881223807/train0": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.002386912703514099, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20266639463366093, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.027498364448547363, + "proposal_oracle_utility_gain_over_base_at_8": -0.004528626799583435, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.025111451745033264, + "selected_utility_gain_over_base_at_8": -0.006915539503097534, + "selector_regret_at_8": 0.002386912703514099, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9725016355514526, + "total_car_to_hidden_at_8": 1.9748885482549667 + }, + "1881223807/train1": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 1.964881807565689, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10190418918720931, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9679730087518692, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03511819243431091, + "selected_utility_gain_over_base_at_8": 0.0030912011861801147, + "selector_regret_at_8": 1.964881807565689, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.964881807565689 + }, + "1881223807/train2": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1643392210528543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.040287479758262634, + "proposal_oracle_utility_gain_over_base_at_8": 0.008260488510131836, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.040287479758262634, + "selected_utility_gain_over_base_at_8": 0.008260488510131836, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9597125202417374, + "total_car_to_hidden_at_8": 1.9597125202417374 + }, + "1919198188/train0": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.021911710500717163, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09083822075370537, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09917314350605011, + "proposal_oracle_utility_gain_over_base_at_8": 0.06197427213191986, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07726143300533295, + "selected_utility_gain_over_base_at_8": 0.0400625616312027, + "selector_regret_at_8": 0.021911710500717163, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.90082685649395, + "total_car_to_hidden_at_8": 1.922738566994667 + }, + "1919198188/train1": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.18855391442775726, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2569381375337174, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2203868180513382, + "proposal_oracle_utility_gain_over_base_at_8": 0.18318794667720795, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03183290362358093, + "selected_utility_gain_over_base_at_8": -0.005365967750549316, + "selector_regret_at_8": 0.18855391442775726, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7796131819486618, + "total_car_to_hidden_at_8": 1.968167096376419 + }, + "1919198188/train2": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.019107162952423096, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21294576266807125, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04859863221645355, + "proposal_oracle_utility_gain_over_base_at_8": 0.011399760842323303, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029491469264030457, + "selected_utility_gain_over_base_at_8": -0.0077074021100997925, + "selector_regret_at_8": 0.019107162952423096, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9514013677835464, + "total_car_to_hidden_at_8": 1.9705085307359695 + }, + "1991414747/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.017522692680358887, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09717880195623664, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13679209351539612, + "proposal_oracle_utility_gain_over_base_at_8": -1.8632079064846039, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11926940083503723, + "selected_utility_gain_over_base_at_8": -1.8807305991649628, + "selector_regret_at_8": 0.017522692680358887, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8632079064846039, + "total_car_to_hidden_at_8": 1.8807305991649628 + }, + "1991414747/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9461991488933563, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.44645064197830664, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.05380085110664368, + "selected_utility_gain_over_base_at_8": -1.9461991488933563, + "selector_regret_at_8": 1.9461991488933563, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9461991488933563 + }, + "1991414747/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9767832309007645, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.613109405537996, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.023216769099235535, + "selected_utility_gain_over_base_at_8": -1.9767832309007645, + "selector_regret_at_8": 1.9767832309007645, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9767832309007645 + }, + "1997090716/train0": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 1.9452667087316513, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3641822422309461, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.054733291268348694, + "selected_utility_gain_over_base_at_8": -0.08454149961471558, + "selector_regret_at_8": 1.9452667087316513, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9452667087316513 + }, + "1997090716/train1": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3553584127845171, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1997090716/train2": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21318958083944717, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2015956957/train0": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.39167013615369795, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.22628515797280127, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.46120103299617765, + "proposal_oracle_utility_gain_over_base_at_8": 0.39408124312758447, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0695308968424797, + "selected_utility_gain_over_base_at_8": 0.0024111069738864898, + "selector_regret_at_8": 0.39167013615369795, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.06868033409118653, + "total_car_to_hidden_at_8": 0.4603504702448845 + }, + "2015956957/train1": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.005367537960410118, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.6, + "pairwise_causal_calibration_ece": 0.2284508985012672, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06726333405822515, + "proposal_oracle_utility_gain_over_base_at_8": 0.00014354418963193893, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06189579609781504, + "selected_utility_gain_over_base_at_8": -0.005223993770778179, + "selector_regret_at_8": 0.005367537960410118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.2, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.46261803302913906, + "total_car_to_hidden_at_8": 0.4679855709895492 + }, + "2015956957/train2": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.4314236156642437, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.24731300175116924, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.45940959043800833, + "proposal_oracle_utility_gain_over_base_at_8": 0.3922898005694151, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02798597477376461, + "selected_utility_gain_over_base_at_8": -0.039133815094828606, + "selector_regret_at_8": 0.4314236156642437, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.07047177664935589, + "total_car_to_hidden_at_8": 0.5018953923135996 + }, + "2054899126/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2054899126/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2054899126/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842/train0": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14834095177221712, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8135772347450256, + "proposal_oracle_utility_gain_over_base_at_8": -0.0252571702003479, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8135772347450256, + "selected_utility_gain_over_base_at_8": -0.0252571702003479, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1864227652549744, + "total_car_to_hidden_at_8": 1.1864227652549744 + }, + "211245842/train1": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 1.1816681623458862, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.036206191308278124, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8183318376541138, + "selected_utility_gain_over_base_at_8": -0.020502567291259766, + "selector_regret_at_8": 1.1816681623458862, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1816681623458862 + }, + "211245842/train2": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.08971682945849302, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1611655950546265, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2141802816/train0": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 0.008425161242485046, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07518554002658885, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08595690131187439, + "proposal_oracle_utility_gain_over_base_at_8": 0.01514965295791626, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07753174006938934, + "selected_utility_gain_over_base_at_8": 0.006724491715431213, + "selector_regret_at_8": 0.008425161242485046, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9140430986881256, + "total_car_to_hidden_at_8": 1.9224682599306107 + }, + "2141802816/train1": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.934090182185173, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.40989236974714055, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06590981781482697, + "selected_utility_gain_over_base_at_8": -0.0048974305391311646, + "selector_regret_at_8": 1.934090182185173, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.934090182185173 + }, + "2141802816/train2": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.954741045832634, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3183279052482729, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04525895416736603, + "selected_utility_gain_over_base_at_8": -0.025548294186592102, + "selector_regret_at_8": 1.954741045832634, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.954741045832634 + }, + "256770874/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.44993481040000916, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18757436508611464, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.574478030204773, + "proposal_oracle_utility_gain_over_base_at_8": -1.425521969795227, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1245432198047638, + "selected_utility_gain_over_base_at_8": -1.8754567801952362, + "selector_regret_at_8": 0.44993481040000916, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.425521969795227, + "total_car_to_hidden_at_8": 1.8754567801952362 + }, + "256770874/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.6129319667816162, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3193203145842822, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.8128684163093567, + "proposal_oracle_utility_gain_over_base_at_8": -1.1871315836906433, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.19993644952774048, + "selected_utility_gain_over_base_at_8": -1.8000635504722595, + "selector_regret_at_8": 0.6129319667816162, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1871315836906433, + "total_car_to_hidden_at_8": 1.8000635504722595 + }, + "256770874/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07907943427562714, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5133629211325229, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.19852638244628906, + "proposal_oracle_utility_gain_over_base_at_8": -1.801473617553711, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11944694817066193, + "selected_utility_gain_over_base_at_8": -1.880553051829338, + "selector_regret_at_8": 0.07907943427562714, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.801473617553711, + "total_car_to_hidden_at_8": 1.880553051829338 + }, + "278008524/train0": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0008676897268742323, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10616076660738837, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.013509750366210938, + "proposal_oracle_utility_gain_over_base_at_8": -0.09396211802959442, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.012642060639336705, + "selected_utility_gain_over_base_at_8": -0.09482980775646865, + "selector_regret_at_8": 0.0008676897268742323, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.100301444530487, + "total_car_to_hidden_at_8": 1.1011691342573613 + }, + "278008524/train1": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0074296220554970205, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10627711331165646, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.019306766451336443, + "proposal_oracle_utility_gain_over_base_at_8": -0.08816510194446892, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011877144395839423, + "selected_utility_gain_over_base_at_8": -0.09559472399996594, + "selector_regret_at_8": 0.0074296220554970205, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0945044284453616, + "total_car_to_hidden_at_8": 1.1019340505008586 + }, + "278008524/train2": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0040267681470140815, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.40905575479886586, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01439405232667923, + "proposal_oracle_utility_gain_over_base_at_8": -0.09307781606912613, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.010367284179665148, + "selected_utility_gain_over_base_at_8": -0.09710458421614021, + "selector_regret_at_8": 0.0040267681470140815, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0994171425700188, + "total_car_to_hidden_at_8": 1.1034439107170328 + }, + "416183106/train0": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.3453921228647232, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1517874168346938, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.581784188747406, + "proposal_oracle_utility_gain_over_base_at_8": 0.38145777583122253, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2363920658826828, + "selected_utility_gain_over_base_at_8": 0.03606565296649933, + "selector_regret_at_8": 0.3453921228647232, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.418215811252594, + "total_car_to_hidden_at_8": 1.7636079341173172 + }, + "416183106/train1": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.01654154062271118, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1925298309196861, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22490769624710083, + "proposal_oracle_utility_gain_over_base_at_8": 0.02458128333091736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20836615562438965, + "selected_utility_gain_over_base_at_8": 0.008039742708206177, + "selector_regret_at_8": 0.01654154062271118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7750923037528992, + "total_car_to_hidden_at_8": 1.7916338443756104 + }, + "416183106/train2": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.04582805931568146, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21826032886833752, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23963995277881622, + "proposal_oracle_utility_gain_over_base_at_8": 0.03931353986263275, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.19381189346313477, + "selected_utility_gain_over_base_at_8": -0.006514519453048706, + "selector_regret_at_8": 0.04582805931568146, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7603600472211838, + "total_car_to_hidden_at_8": 1.8061881065368652 + }, + "534717141/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18033001516380143, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "534717141/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.12736602361488303, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "534717141/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15703220108978472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635/train0": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2800043225288391, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3791036296516339, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7199956774711609, + "selected_utility_gain_over_base_at_8": -0.003976881504058838, + "selector_regret_at_8": 1.2800043225288391, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2800043225288391 + }, + "551504635/train1": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2779860496520996, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.23627606487156766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7220139503479004, + "selected_utility_gain_over_base_at_8": -0.001958608627319336, + "selector_regret_at_8": 1.2779860496520996, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2779860496520996 + }, + "551504635/train2": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.287484347820282, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4327797462092032, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.712515652179718, + "selected_utility_gain_over_base_at_8": -0.011456906795501709, + "selector_regret_at_8": 1.287484347820282, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.287484347820282 + }, + "565369392/train0": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.05178981522719065, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.21578265463075008, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.14469666282335916, + "proposal_oracle_utility_gain_over_base_at_8": -0.655068983634313, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.09290684759616852, + "selected_utility_gain_over_base_at_8": -0.7068587988615036, + "selector_regret_at_8": 0.05178981522719065, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.281934916973114, + "total_car_to_hidden_at_8": 1.3337247322003047 + }, + "565369392/train1": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.048005685210227966, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2859321646983037, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.11818361282348633, + "proposal_oracle_utility_gain_over_base_at_8": -0.6815820336341858, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.07017792761325836, + "selected_utility_gain_over_base_at_8": -0.7295877188444138, + "selector_regret_at_8": 0.048005685210227966, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.308447966972987, + "total_car_to_hidden_at_8": 1.356453652183215 + }, + "565369392/train2": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.057756672302881874, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2176430432872923, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7870745907227198, + "proposal_oracle_utility_gain_over_base_at_8": -0.012691055734952291, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.729317918419838, + "selected_utility_gain_over_base_at_8": -0.07044772803783417, + "selector_regret_at_8": 0.057756672302881874, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.6395569890737534, + "total_car_to_hidden_at_8": 0.6973136613766352 + }, + "565621826/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "565621826/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "565621826/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130/train0": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.01741097339739402, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3300621116998719, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.034002700820565224, + "proposal_oracle_utility_gain_over_base_at_8": -0.04128939720491568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.016591727423171203, + "selected_utility_gain_over_base_at_8": -0.058700370602309704, + "selector_regret_at_8": 0.01741097339739402, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2036442911873261, + "total_car_to_hidden_at_8": 0.22105526458472013 + }, + "585904130/train1": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.004500773424903552, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1652498530853035, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01988216955214739, + "proposal_oracle_utility_gain_over_base_at_8": -0.055409928473333515, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.015381396127243837, + "selected_utility_gain_over_base_at_8": -0.05991070189823707, + "selector_regret_at_8": 0.004500773424903552, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.21776482245574394, + "total_car_to_hidden_at_8": 0.2222655958806475 + }, + "585904130/train2": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.006800346697370212, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22254352429757418, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.029573063055674236, + "proposal_oracle_utility_gain_over_base_at_8": -0.04571903496980667, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.022772716358304024, + "selected_utility_gain_over_base_at_8": -0.05251938166717688, + "selector_regret_at_8": 0.006800346697370212, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2080739289522171, + "total_car_to_hidden_at_8": 0.2148742756495873 + }, + "626486813/train0": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.033813119400292635, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.24681469768825515, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.07908930000849068, + "proposal_oracle_utility_gain_over_base_at_8": -0.4263507166178897, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.04527618060819805, + "selected_utility_gain_over_base_at_8": -0.46016383601818234, + "selector_regret_at_8": 0.033813119400292635, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.544636546401307, + "total_car_to_hidden_at_8": 0.5671922296751291 + }, + "626486813/train1": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.02813516021706164, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20632809068391378, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06953954696655273, + "proposal_oracle_utility_gain_over_base_at_8": -0.43590046965982765, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.041404386749491096, + "selected_utility_gain_over_base_at_8": -0.4640356298768893, + "selector_regret_at_8": 0.02813516021706164, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5429288633167744, + "total_car_to_hidden_at_8": 0.571064023533836 + }, + "626486813/train2": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.013256545644253492, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.14946876705188256, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06192737747915089, + "proposal_oracle_utility_gain_over_base_at_8": -0.4435126391472295, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.0486708318348974, + "selected_utility_gain_over_base_at_8": -0.456769184791483, + "selector_regret_at_8": 0.013256545644253492, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5505410328041762, + "total_car_to_hidden_at_8": 0.5637975784484297 + }, + "636680137/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0048203859478235245, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19331728269593437, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.024462858214974403, + "proposal_oracle_utility_gain_over_base_at_8": -1.9755371417850256, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.01964247226715088, + "selected_utility_gain_over_base_at_8": -1.9803575277328491, + "selector_regret_at_8": 0.0048203859478235245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755371417850256, + "total_car_to_hidden_at_8": 1.9803575277328491 + }, + "636680137/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.004942953586578369, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.45862923715160336, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.015581388957798481, + "proposal_oracle_utility_gain_over_base_at_8": -1.9844186110422015, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.010638435371220112, + "selected_utility_gain_over_base_at_8": -1.9893615646287799, + "selector_regret_at_8": 0.004942953586578369, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9844186110422015, + "total_car_to_hidden_at_8": 1.9893615646287799 + }, + "636680137/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.028240702114999294, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.6813618274184183, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.036433976143598557, + "proposal_oracle_utility_gain_over_base_at_8": -1.9635660238564014, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.008193274028599262, + "selected_utility_gain_over_base_at_8": -1.9918067259714007, + "selector_regret_at_8": 0.028240702114999294, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9635660238564014, + "total_car_to_hidden_at_8": 1.9918067259714007 + }, + "706479600/train0": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.1455947458744049, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2692917587355154, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21975377202033997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6153350174427032, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07415902614593506, + "selected_utility_gain_over_base_at_8": -0.7609297633171082, + "selector_regret_at_8": 0.1455947458744049, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.78024622797966, + "total_car_to_hidden_at_8": 1.925840973854065 + }, + "706479600/train1": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 1.185575544834137, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15129914778803955, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814424455165863, + "selected_utility_gain_over_base_at_8": -0.020664334297180176, + "selector_regret_at_8": 1.185575544834137, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185575544834137 + }, + "706479600/train2": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15321122852132601, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1649112105369568, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "750448296/train0": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.015873273834586143, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.3795450047628438, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.109864491969347, + "proposal_oracle_utility_gain_over_base_at_8": -0.0016690678894519806, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09399121813476086, + "selected_utility_gain_over_base_at_8": -0.017542341724038124, + "selector_regret_at_8": 0.015873273834586143, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12594309821724892, + "total_car_to_hidden_at_8": 0.14181637205183506 + }, + "750448296/train1": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.04417596384882927, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.1527117559178966, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10195614024996758, + "proposal_oracle_utility_gain_over_base_at_8": -0.009577419608831406, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.057780176401138306, + "selected_utility_gain_over_base_at_8": -0.053753383457660675, + "selector_regret_at_8": 0.04417596384882927, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13385144993662834, + "total_car_to_hidden_at_8": 0.1780274137854576 + }, + "750448296/train2": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.03573483042418957, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19031133205081463, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08245231583714485, + "proposal_oracle_utility_gain_over_base_at_8": -0.02908124402165413, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.046717485412955284, + "selected_utility_gain_over_base_at_8": -0.0648160744458437, + "selector_regret_at_8": 0.03573483042418957, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15335527434945107, + "total_car_to_hidden_at_8": 0.18909010477364063 + }, + "75070641/train0": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.003223627805709839, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.13808834192306887, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2283252328634262, + "proposal_oracle_utility_gain_over_base_at_8": 0.025573596358299255, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.22510160505771637, + "selected_utility_gain_over_base_at_8": 0.022349968552589417, + "selector_regret_at_8": 0.003223627805709839, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7716747671365738, + "total_car_to_hidden_at_8": 1.7748983949422836 + }, + "75070641/train1": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.007853284478187561, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.34224193319263285, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24087366461753845, + "proposal_oracle_utility_gain_over_base_at_8": 0.0381220281124115, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2330203801393509, + "selected_utility_gain_over_base_at_8": 0.030268743634223938, + "selector_regret_at_8": 0.007853284478187561, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7591263353824615, + "total_car_to_hidden_at_8": 1.766979619860649 + }, + "75070641/train2": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.048324257135391235, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2643509727736545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21781498193740845, + "proposal_oracle_utility_gain_over_base_at_8": 0.015063345432281494, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1694907248020172, + "selected_utility_gain_over_base_at_8": -0.03326091170310974, + "selector_regret_at_8": 0.048324257135391235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7821850180625916, + "total_car_to_hidden_at_8": 1.8305092751979828 + }, + "767747424/train0": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5240038954652846, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.2798082148668758, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5921752639114857, + "proposal_oracle_utility_gain_over_base_at_8": 0.04465107619762421, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.06817136844620109, + "selected_utility_gain_over_base_at_8": -0.4793528192676604, + "selector_regret_at_8": 0.5240038954652846, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.04815509542822838, + "total_car_to_hidden_at_8": 0.12046428816393018 + }, + "767747424/train1": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.04542438860516995, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.13255125751519692, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.10304383677430451, + "proposal_oracle_utility_gain_over_base_at_8": -0.44448035093955696, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.05761944816913456, + "selected_utility_gain_over_base_at_8": -0.4899047395447269, + "selector_regret_at_8": 0.04542438860516995, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.08559181983582675, + "total_car_to_hidden_at_8": 0.1310162084409967 + }, + "767747424/train2": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5082327332347631, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20625644377911734, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5790140572935343, + "proposal_oracle_utility_gain_over_base_at_8": 0.03148986957967281, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.07078132405877113, + "selected_utility_gain_over_base_at_8": -0.47674286365509033, + "selector_regret_at_8": 0.5082327332347631, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05956761725246906, + "total_car_to_hidden_at_8": 0.11785433255136013 + }, + "77147264/train0": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.013380475342273712, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22561784152146835, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.1132144927978516, + "proposal_oracle_utility_gain_over_base_at_8": -0.30836498737335205, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0998340174555779, + "selected_utility_gain_over_base_at_8": -0.32174546271562576, + "selector_regret_at_8": 0.013380475342273712, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.8867855072021484, + "total_car_to_hidden_at_8": 0.9001659825444221 + }, + "77147264/train1": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.8879568874835968, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.03379253116769787, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.5784205198287964, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1120431125164032, + "selected_utility_gain_over_base_at_8": -0.3095363676548004, + "selector_regret_at_8": 0.8879568874835968, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8879568874835968 + }, + "77147264/train2": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.9378202110528946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13662790487889595, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.11046252399683, + "proposal_oracle_utility_gain_over_base_at_8": -0.3111169561743736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.5, + "selected_utility_at_8": 0.1726423129439354, + "selected_utility_gain_over_base_at_8": -1.2489371672272682, + "selector_regret_at_8": 0.9378202110528946, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.88953747600317, + "total_car_to_hidden_at_8": 1.8273576870560646 + }, + "794181626/train0": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3405496231635778, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "794181626/train1": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 1.9626595228910446, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3619001242329895, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03734047710895538, + "selected_utility_gain_over_base_at_8": -0.14361557364463806, + "selector_regret_at_8": 1.9626595228910446, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9626595228910446 + }, + "794181626/train2": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1880089150142482, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "957101489/train0": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3607921451330185, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2493251178612577, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5799027681350708, + "proposal_oracle_utility_gain_over_base_at_8": 0.37618090212345123, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2191106230020523, + "selected_utility_gain_over_base_at_8": 0.01538875699043274, + "selector_regret_at_8": 0.3607921451330185, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4200972318649292, + "total_car_to_hidden_at_8": 1.7808893769979477 + }, + "957101489/train1": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3404468297958374, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2708447347116597, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5798611640930176, + "proposal_oracle_utility_gain_over_base_at_8": 0.376139298081398, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.23941433429718018, + "selected_utility_gain_over_base_at_8": 0.03569246828556061, + "selector_regret_at_8": 0.3404468297958374, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4201388359069824, + "total_car_to_hidden_at_8": 1.7605856657028198 + }, + "957101489/train2": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.38811931014060974, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5372965735602796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5639582276344299, + "proposal_oracle_utility_gain_over_base_at_8": 0.36023636162281036, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1758389174938202, + "selected_utility_gain_over_base_at_8": -0.027882948517799377, + "selector_regret_at_8": 0.38811931014060974, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.43604177236557, + "total_car_to_hidden_at_8": 1.8241610825061798 + }, + "998761162/train0": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0010883957147598267, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1991858618764328, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09276536107063293, + "proposal_oracle_utility_gain_over_base_at_8": 0.024948790669441223, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09167696535587311, + "selected_utility_gain_over_base_at_8": 0.023860394954681396, + "selector_regret_at_8": 0.0010883957147598267, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.907234638929367, + "total_car_to_hidden_at_8": 1.908323034644127 + }, + "998761162/train1": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24035198055524792, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "998761162/train2": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.14873333521535068, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + } +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics_by_task.json b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..e69284aae964c2bfef7c2a847db553b4ad3d9363 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.5397342965006828, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5555555555555556, + "pairwise_causal_calibration_ece": 0.2123556134611932, + "proposal_oracle_success_at_8": 0.5555555555555556, + "proposal_oracle_success_gain_over_base_at_8": 0.2222222222222222, + "proposal_oracle_utility_at_8": 1.3934579665462177, + "proposal_oracle_utility_gain_over_base_at_8": 0.3265138193964958, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8537236700455347, + "selected_utility_gain_over_base_at_8": -0.213220477104187, + "selector_regret_at_8": 0.5397342965006828, + "success_selector_gap_at_8": 0.3888888888888889, + "success_support_gap_at_8": 0.4444444444444444, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.6065420334537824, + "total_car_to_hidden_at_8": 1.1462763299544652 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.4133657068014145, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.17471460699627128, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46705442667007446, + "proposal_oracle_utility_gain_over_base_at_8": 0.313624769449234, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05368871986865997, + "selected_utility_gain_over_base_at_8": -0.09974093735218048, + "selector_regret_at_8": 0.4133657068014145, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5329455733299255, + "total_car_to_hidden_at_8": 1.94631128013134 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.1168446104449115, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.3627450980392157, + "pairwise_causal_calibration_ece": 0.21225649007113187, + "proposal_oracle_success_at_8": 0.19607843137254902, + "proposal_oracle_success_gain_over_base_at_8": -0.09803921568627451, + "proposal_oracle_utility_at_8": 0.44994396614321674, + "proposal_oracle_utility_gain_over_base_at_8": -0.20129550277830704, + "selected_success_at_8": 0.14705882352941177, + "selected_success_gain_over_base_at_8": -0.14705882352941177, + "selected_utility_at_8": 0.33309935569830523, + "selected_utility_gain_over_base_at_8": -0.31814011322321856, + "selector_regret_at_8": 0.1168446104449115, + "success_selector_gap_at_8": 0.049019607843137254, + "success_support_gap_at_8": 0.14705882352941177, + "success_total_car_to_hidden_at_8": 0.17647058823529413, + "support_gap_at_8": 0.36822861206361696, + "total_car_to_hidden_at_8": 0.44922248110107793 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.6477842434094503, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6153846153846154, + "pairwise_causal_calibration_ece": 0.19356765816347193, + "proposal_oracle_success_at_8": 0.7948717948717948, + "proposal_oracle_success_gain_over_base_at_8": 0.48717948717948717, + "proposal_oracle_utility_at_8": 1.6016510277986526, + "proposal_oracle_utility_gain_over_base_at_8": 0.9244865671946452, + "selected_success_at_8": 0.46153846153846156, + "selected_success_gain_over_base_at_8": 0.15384615384615385, + "selected_utility_at_8": 0.9538667843892024, + "selected_utility_gain_over_base_at_8": 0.276702323785195, + "selector_regret_at_8": 0.6477842434094503, + "success_selector_gap_at_8": 0.3333333333333333, + "success_support_gap_at_8": 0.20512820512820512, + "success_total_car_to_hidden_at_8": 0.5384615384615384, + "support_gap_at_8": 0.39834897220134735, + "total_car_to_hidden_at_8": 1.0461332156107976 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.5405315528313319, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.2253301008861889, + "proposal_oracle_success_at_8": 0.4, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 1.054750414358245, + "proposal_oracle_utility_gain_over_base_at_8": 0.25163986451096004, + "selected_success_at_8": 0.13333333333333333, + "selected_success_gain_over_base_at_8": -0.06666666666666667, + "selected_utility_at_8": 0.5142188615269131, + "selected_utility_gain_over_base_at_8": -0.2888916883203718, + "selector_regret_at_8": 0.5405315528313319, + "success_selector_gap_at_8": 0.26666666666666666, + "success_support_gap_at_8": 0.6, + "success_total_car_to_hidden_at_8": 0.8666666666666667, + "support_gap_at_8": 0.945249585641755, + "total_car_to_hidden_at_8": 1.4857811384730868 + } +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/report.md b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/report.md new file mode 100644 index 0000000000000000000000000000000000000000..a2e331cf5bda838a248b339d4e61d6f6f394fc14 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/report.md @@ -0,0 +1,27 @@ +# Metric Evaluation (measured) + +K: `8` + +| Metric | N | Micro mean | 95% CI | Task macro | Seed macro | +| --- | ---: | ---: | ---: | ---: | ---: | +| base_success | 207 | 0.2754 | [0.2174, 0.3382] | 0.2270 | 0.3085 | +| base_utility | 207 | 0.7181 | [0.6102, 0.8321] | 0.6704 | 0.8164 | +| branch_car_at_8 | 207 | 0.3501 | [0.2639, 0.4417] | 0.4517 | 0.4280 | +| hidden_chart_oracle_success_at_8 | 207 | 0.6667 | [0.5990, 0.7246] | 0.8647 | 0.8358 | +| hidden_chart_oracle_utility_at_8 | 207 | 1.4000 | [1.2800, 1.5063] | 1.7565 | 1.7062 | +| outcome_ptr_at_8 | 207 | 0.5024 | [0.4300, 0.5652] | 0.6401 | 0.5357 | +| pairwise_causal_calibration_ece | 207 | 0.2110 | [0.1939, 0.2290] | 0.2036 | 0.2090 | +| proposal_oracle_success_at_8 | 207 | 0.3816 | [0.3188, 0.4444] | 0.3893 | 0.4792 | +| proposal_oracle_success_gain_over_base_at_8 | 207 | 0.1063 | [0.0386, 0.1836] | 0.1623 | 0.1707 | +| proposal_oracle_utility_at_8 | 207 | 0.8807 | [0.7637, 0.9970] | 0.9934 | 1.1007 | +| proposal_oracle_utility_gain_over_base_at_8 | 207 | 0.1626 | [0.0470, 0.2899] | 0.3230 | 0.2843 | +| selected_success_at_8 | 207 | 0.2029 | [0.1498, 0.2560] | 0.1817 | 0.2565 | +| selected_success_gain_over_base_at_8 | 207 | -0.0725 | [-0.1353, -0.0048] | -0.0453 | -0.0520 | +| selected_utility_at_8 | 207 | 0.5307 | [0.4294, 0.6390] | 0.5417 | 0.6727 | +| selected_utility_gain_over_base_at_8 | 207 | -0.1874 | [-0.2967, -0.0708] | -0.1287 | -0.1437 | +| selector_regret_at_8 | 207 | 0.3501 | [0.2639, 0.4417] | 0.4517 | 0.4280 | +| success_selector_gap_at_8 | 207 | 0.1787 | [0.1256, 0.2367] | 0.2076 | 0.2227 | +| success_support_gap_at_8 | 207 | 0.2947 | [0.2319, 0.3623] | 0.4793 | 0.3602 | +| success_total_car_to_hidden_at_8 | 207 | 0.4638 | [0.3961, 0.5314] | 0.6830 | 0.5793 | +| support_gap_at_8 | 207 | 0.5369 | [0.4377, 0.6440] | 0.7703 | 0.6120 | +| total_car_to_hidden_at_8 | 207 | 0.8693 | [0.7555, 0.9812] | 1.2147 | 1.0335 | diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/table.tex b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..89cdf5a6446bd9583423d7dcc4dd5c2869ffb75c --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/measured_metrics/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 207 & 0.2754 & 0.2174 & 0.3382 \\ +base\_utility & 207 & 0.7181 & 0.6102 & 0.8321 \\ +branch\_car\_at\_8 & 207 & 0.3501 & 0.2639 & 0.4417 \\ +hidden\_chart\_oracle\_success\_at\_8 & 207 & 0.6667 & 0.5990 & 0.7246 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 207 & 1.4000 & 1.2800 & 1.5063 \\ +outcome\_ptr\_at\_8 & 207 & 0.5024 & 0.4300 & 0.5652 \\ +pairwise\_causal\_calibration\_ece & 207 & 0.2110 & 0.1939 & 0.2290 \\ +proposal\_oracle\_success\_at\_8 & 207 & 0.3816 & 0.3188 & 0.4444 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 207 & 0.1063 & 0.0386 & 0.1836 \\ +proposal\_oracle\_utility\_at\_8 & 207 & 0.8807 & 0.7637 & 0.9970 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 207 & 0.1626 & 0.0470 & 0.2899 \\ +selected\_success\_at\_8 & 207 & 0.2029 & 0.1498 & 0.2560 \\ +selected\_success\_gain\_over\_base\_at\_8 & 207 & -0.0725 & -0.1353 & -0.0048 \\ +selected\_utility\_at\_8 & 207 & 0.5307 & 0.4294 & 0.6390 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 207 & -0.1874 & -0.2967 & -0.0708 \\ +selector\_regret\_at\_8 & 207 & 0.3501 & 0.2639 & 0.4417 \\ +success\_selector\_gap\_at\_8 & 207 & 0.1787 & 0.1256 & 0.2367 \\ +success\_support\_gap\_at\_8 & 207 & 0.2947 & 0.2319 & 0.3623 \\ +success\_total\_car\_to\_hidden\_at\_8 & 207 & 0.4638 & 0.3961 & 0.5314 \\ +support\_gap\_at\_8 & 207 & 0.5369 & 0.4377 & 0.6440 \\ +total\_car\_to\_hidden\_at\_8 & 207 & 0.8693 & 0.7555 & 0.9812 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/metrics.json b/runs/ctt_base_context_obj_val_rollout_comparison/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..eda2b5f1a566a91e1587ad8290c9bab563b48260 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/metrics.json @@ -0,0 +1,3899 @@ +{ + "k": 8, + "num_rows": 207, + "report_type": "ctt_measured_rollout_comparison", + "run_dirs": [ + "runs/ctt_residual_base_context_obj_rollout_val69_seed0", + "runs/ctt_residual_base_context_obj_rollout_val69_seed1", + "runs/ctt_residual_base_context_obj_rollout_val69_seed2" + ], + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "success_summary": { + "base_success_rate": 0.2753623188405797, + "base_utility_mean": 0.7180729379567007, + "hidden_chart_oracle_success_rate": 0.6666666666666666, + "hidden_chart_oracle_utility_mean": 1.3999846732054932, + "max_restore_error": 1.4901161193847656e-08, + "proposal_oracle_success_gain_over_base": 0.10628019323671498, + "proposal_oracle_success_rate": 0.38164251207729466, + "proposal_oracle_utility_mean": 0.8807045891532306, + "selected_success_gain_over_base": -0.07246376811594203, + "selected_success_rate": 0.2028985507246377, + "selected_utility_mean": 0.5306515935340228, + "success_selector_gap": 0.178743961352657, + "success_support_gap": 0.2946859903381642 + }, + "summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3788416075650118, + "low": 0.23817966903073282, + "mean": 0.3085106382978724, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 1.0, + "1055567618/train1": 1.0, + "1055567618/train2": 1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 0.0, + "1189392917/train1": 0.0, + "1189392917/train2": 0.0, + "1231780997/train0": 0.0, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 0.0, + "1375492786/train0": 1.0, + "1375492786/train1": 1.0, + "1375492786/train2": 1.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 1.0, + "1475440973/train2": 1.0, + "1483645336/train0": 0.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 1.0, + "1528740417/train1": 1.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.6666666666666666, + "1549324805/train2": 0.6666666666666666, + "1607773860/train0": 0.0, + "1607773860/train1": 0.0, + "1607773860/train2": 0.0, + "1665739511/train0": 1.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 0.0, + "1747178030/train2": 0.0, + "1881223807/train0": 0.0, + "1881223807/train1": 0.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 1.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 0.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.0, + "2015956957/train2": 0.0, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 0.0, + "211245842/train1": 0.0, + "211245842/train2": 0.0, + "2141802816/train0": 0.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 0.3333333333333333, + "565369392/train1": 0.3333333333333333, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 0.0, + "706479600/train1": 0.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.25, + "767747424/train1": 0.25, + "767747424/train2": 0.25, + "77147264/train0": 0.5, + "77147264/train1": 0.5, + "77147264/train2": 0.5, + "794181626/train0": 0.0, + "794181626/train1": 0.0, + "794181626/train2": 0.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3125188536953243, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.33816425120772947, + "low": 0.21739130434782608, + "mean": 0.2753623188405797, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9440655855692033, + "low": 0.6842917830640909, + "mean": 0.8163502178883103, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.8163502178883103, + "per_seed": { + "1025740212/train0": 0.528373658657074, + "1025740212/train1": 0.528373658657074, + "1025740212/train2": 0.528373658657074, + "1055157354/train0": 0.19648407399654388, + "1055157354/train1": 0.19648407399654388, + "1055157354/train2": 0.19648407399654388, + "1055567618/train0": 2.0, + "1055567618/train1": 2.0, + "1055567618/train2": 2.0, + "1113268940/train0": 0.42231252789497375, + "1113268940/train1": 0.42231252789497375, + "1113268940/train2": 0.42231252789497375, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 0.15461373329162598, + "1176943004/train1": 0.15461373329162598, + "1176943004/train2": 0.15461373329162598, + "1189392917/train0": 0.10841864347457886, + "1189392917/train1": 0.10841864347457886, + "1189392917/train2": 0.10841864347457886, + "1231780997/train0": 0.7270061373710632, + "1231780997/train1": 0.7270061373710632, + "1231780997/train2": 0.7270061373710632, + "1271718007/train0": 1.415381133556366, + "1271718007/train1": 1.415381133556366, + "1271718007/train2": 1.415381133556366, + "1273247072/train0": 0.15342965722084045, + "1273247072/train1": 0.15342965722084045, + "1273247072/train2": 0.15342965722084045, + "1358421325/train0": 0.8272150158882141, + "1358421325/train1": 0.8272150158882141, + "1358421325/train2": 0.8272150158882141, + "1375492786/train0": 2.0, + "1375492786/train1": 2.0, + "1375492786/train2": 2.0, + "141984615/train0": 0.09381213908394177, + "141984615/train1": 0.09381213908394177, + "141984615/train2": 0.09381213908394177, + "1475440973/train0": 2.0, + "1475440973/train1": 2.0, + "1475440973/train2": 2.0, + "1483645336/train0": 0.01641605794429779, + "1483645336/train1": 0.01641605794429779, + "1483645336/train2": 0.01641605794429779, + "1522272686/train0": 0.19844311475753784, + "1522272686/train1": 0.19844311475753784, + "1522272686/train2": 0.19844311475753784, + "1528740417/train0": 2.0, + "1528740417/train1": 2.0, + "1528740417/train2": 2.0, + "1549324805/train0": 1.3787474483251572, + "1549324805/train1": 1.3787474483251572, + "1549324805/train2": 1.3787474483251572, + "1607773860/train0": 0.8326728343963623, + "1607773860/train1": 0.8326728343963623, + "1607773860/train2": 0.8326728343963623, + "1665739511/train0": 2.0, + "1665739511/train1": 2.0, + "1665739511/train2": 2.0, + "1747178030/train0": 0.14663122594356537, + "1747178030/train1": 0.14663122594356537, + "1747178030/train2": 0.14663122594356537, + "1881223807/train0": 0.0320269912481308, + "1881223807/train1": 0.0320269912481308, + "1881223807/train2": 0.0320269912481308, + "1919198188/train0": 0.03719887137413025, + "1919198188/train1": 0.03719887137413025, + "1919198188/train2": 0.03719887137413025, + "1991414747/train0": 2.0, + "1991414747/train1": 2.0, + "1991414747/train2": 2.0, + "1997090716/train0": 0.13927479088306427, + "1997090716/train1": 0.13927479088306427, + "1997090716/train2": 0.13927479088306427, + "2015956957/train0": 0.06711978986859321, + "2015956957/train1": 0.06711978986859321, + "2015956957/train2": 0.06711978986859321, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 0.8388344049453735, + "211245842/train1": 0.8388344049453735, + "211245842/train2": 0.8388344049453735, + "2141802816/train0": 0.07080724835395813, + "2141802816/train1": 0.07080724835395813, + "2141802816/train2": 0.07080724835395813, + "256770874/train0": 2.0, + "256770874/train1": 2.0, + "256770874/train2": 2.0, + "278008524/train0": 0.10747186839580536, + "278008524/train1": 0.10747186839580536, + "278008524/train2": 0.10747186839580536, + "416183106/train0": 0.20032641291618347, + "416183106/train1": 0.20032641291618347, + "416183106/train2": 0.20032641291618347, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 0.7239725589752197, + "551504635/train1": 0.7239725589752197, + "551504635/train2": 0.7239725589752197, + "565369392/train0": 0.7997656464576721, + "565369392/train1": 0.7997656464576721, + "565369392/train2": 0.7997656464576721, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.07529209802548091, + "585904130/train1": 0.07529209802548091, + "585904130/train2": 0.07529209802548091, + "626486813/train0": 0.5054400166263804, + "626486813/train1": 0.5054400166263804, + "626486813/train2": 0.5054400166263804, + "636680137/train0": 2.0, + "636680137/train1": 2.0, + "636680137/train2": 2.0, + "706479600/train0": 0.8350887894630432, + "706479600/train1": 0.8350887894630432, + "706479600/train2": 0.8350887894630432, + "750448296/train0": 0.11153355985879898, + "750448296/train1": 0.11153355985879898, + "750448296/train2": 0.11153355985879898, + "75070641/train0": 0.20275163650512695, + "75070641/train1": 0.20275163650512695, + "75070641/train2": 0.20275163650512695, + "767747424/train0": 0.5475241877138615, + "767747424/train1": 0.5475241877138615, + "767747424/train2": 0.5475241877138615, + "77147264/train0": 1.4215794801712036, + "77147264/train1": 1.4215794801712036, + "77147264/train2": 1.4215794801712036, + "794181626/train0": 0.18095605075359344, + "794181626/train1": 0.18095605075359344, + "794181626/train2": 0.18095605075359344, + "957101489/train0": 0.20372186601161957, + "957101489/train1": 0.20372186601161957, + "957101489/train2": 0.20372186601161957, + "998761162/train0": 0.06781657040119171, + "998761162/train1": 0.06781657040119171, + "998761162/train2": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9086439887682596, + "low": 0.382927798086266, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.8321223414957452, + "low": 0.6101555917457034, + "mean": 0.7180729379567007, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.558614392092107, + "low": 0.3184796732449662, + "mean": 0.42802111370607065, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.42802111370607065, + "per_seed": { + "1025740212/train0": 1.281606376171112, + "1025740212/train1": 0.0031174421310424805, + "1025740212/train2": 0.051884353160858154, + "1055157354/train0": 0.4498589485883713, + "1055157354/train1": 0.6509099900722504, + "1055157354/train2": 0.0, + "1055567618/train0": 1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": 1.8424437642097473, + "1113268940/train0": 0.11861386895179749, + "1113268940/train1": 0.06326162815093994, + "1113268940/train2": 0.48447395861148834, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.9300042241811752, + "1176943004/train1": 1.953145906329155, + "1176943004/train2": 1.9587705433368683, + "1189392917/train0": 0.0, + "1189392917/train1": 1.9264162927865982, + "1189392917/train2": 0.0, + "1231780997/train0": 1.2770342230796814, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.9082518517971039, + "1271718007/train1": 0.8952926397323608, + "1271718007/train2": 0.9200049638748169, + "1273247072/train0": 0.42020074278116226, + "1273247072/train1": 0.42682039737701416, + "1273247072/train2": 0.39307598024606705, + "1358421325/train0": 0.6354914605617523, + "1358421325/train1": 0.007964074611663818, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0008215903653763235, + "1375492786/train1": 0.001000205724267289, + "1375492786/train2": 0.00036396586801856756, + "141984615/train0": 0.05737591476645321, + "141984615/train1": 0.03783805563580245, + "141984615/train2": 0.03539413997593025, + "1475440973/train0": 1.2853952050209045, + "1475440973/train1": 0.0, + "1475440973/train2": 1.2959344387054443, + "1483645336/train0": 1.8899840712547302, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.030991941690444946, + "1522272686/train1": 0.03571280837059021, + "1522272686/train2": 0.056304022669792175, + "1528740417/train0": 0.0, + "1528740417/train1": 0.008421123027801514, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0010904477288325627, + "1549324805/train1": 0.004917144775390625, + "1549324805/train2": 0.010487629100680351, + "1607773860/train0": 1.8556141257286072, + "1607773860/train1": 0.0, + "1607773860/train2": 1.185863733291626, + "1665739511/train0": 0.0, + "1665739511/train1": 1.1815597414970398, + "1665739511/train2": 1.868122011423111, + "1747178030/train0": 0.02001962810754776, + "1747178030/train1": 1.9076288938522339, + "1747178030/train2": 0.0, + "1881223807/train0": 0.002386912703514099, + "1881223807/train1": 1.964881807565689, + "1881223807/train2": 0.0, + "1919198188/train0": 0.021911710500717163, + "1919198188/train1": 0.18855391442775726, + "1919198188/train2": 0.019107162952423096, + "1991414747/train0": 0.017522692680358887, + "1991414747/train1": 1.9461991488933563, + "1991414747/train2": 1.9767832309007645, + "1997090716/train0": 1.9452667087316513, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.39167013615369795, + "2015956957/train1": 0.005367537960410118, + "2015956957/train2": 0.4314236156642437, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.1816681623458862, + "211245842/train2": 0.0, + "2141802816/train0": 0.008425161242485046, + "2141802816/train1": 1.934090182185173, + "2141802816/train2": 1.954741045832634, + "256770874/train0": 0.44993481040000916, + "256770874/train1": 0.6129319667816162, + "256770874/train2": 0.07907943427562714, + "278008524/train0": 0.0008676897268742323, + "278008524/train1": 0.0074296220554970205, + "278008524/train2": 0.0040267681470140815, + "416183106/train0": 0.3453921228647232, + "416183106/train1": 0.01654154062271118, + "416183106/train2": 0.04582805931568146, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2800043225288391, + "551504635/train1": 1.2779860496520996, + "551504635/train2": 1.287484347820282, + "565369392/train0": 0.05178981522719065, + "565369392/train1": 0.048005685210227966, + "565369392/train2": 0.057756672302881874, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.01741097339739402, + "585904130/train1": 0.004500773424903552, + "585904130/train2": 0.006800346697370212, + "626486813/train0": 0.033813119400292635, + "626486813/train1": 0.02813516021706164, + "626486813/train2": 0.013256545644253492, + "636680137/train0": 0.0048203859478235245, + "636680137/train1": 0.004942953586578369, + "636680137/train2": 0.028240702114999294, + "706479600/train0": 0.1455947458744049, + "706479600/train1": 1.185575544834137, + "706479600/train2": 0.0, + "750448296/train0": 0.015873273834586143, + "750448296/train1": 0.04417596384882927, + "750448296/train2": 0.03573483042418957, + "75070641/train0": 0.003223627805709839, + "75070641/train1": 0.007853284478187561, + "75070641/train2": 0.048324257135391235, + "767747424/train0": 0.5240038954652846, + "767747424/train1": 0.04542438860516995, + "767747424/train2": 0.5082327332347631, + "77147264/train0": 0.013380475342273712, + "77147264/train1": 0.8879568874835968, + "77147264/train2": 0.9378202110528946, + "794181626/train0": 0.0, + "794181626/train1": 1.9626595228910446, + "794181626/train2": 0.0, + "957101489/train0": 0.3607921451330185, + "957101489/train1": 0.3404468297958374, + "957101489/train2": 0.38811931014060974, + "998761162/train0": 0.0010883957147598267, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.5834326290625793, + "low": 0.26088621819349617, + "mean": 0.45165208199755824, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.45165208199755824, + "per_task": { + "LiftPegUpright-v1": 0.5397342965006828, + "PickCube-v1": 0.4133657068014145, + "PullCube-v1": 0.1168446104449115, + "PushCube-v1": 0.6477842434094503, + "StackCube-v1": 0.5405315528313319 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4417064196763646, + "low": 0.2639209286670835, + "mean": 0.3500529956192077, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.8887706855791963, + "low": 0.7754137115839246, + "mean": 0.8358156028368796, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.8358156028368796, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 1.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 1.0, + "1055567618/train1": 1.0, + "1055567618/train2": 1.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 1.0, + "1271718007/train0": 1.0, + "1271718007/train1": 1.0, + "1271718007/train2": 1.0, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 1.0, + "1358421325/train1": 1.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 1.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 1.0, + "1528740417/train1": 1.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.6666666666666666, + "1549324805/train2": 0.6666666666666666, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 1.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 1.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 1.0, + "1881223807/train1": 1.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 1.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.2, + "2015956957/train2": 0.2, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 1.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 1.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.5, + "278008524/train1": 0.5, + "278008524/train2": 0.5, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.6666666666666666, + "565369392/train1": 0.6666666666666666, + "565369392/train2": 0.6666666666666666, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 1.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 1.0, + "77147264/train1": 1.0, + "77147264/train2": 1.0, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.7246376811594203, + "low": 0.5990338164251208, + "mean": 0.6666666666666666, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.8010394367889166, + "low": 1.5995275395488624, + "mean": 1.706202811163896, + "n": 141, + "num_samples": 1000 + }, + "mean": 1.706202811163896, + "per_seed": { + "1025740212/train0": 2.0, + "1025740212/train1": 2.0, + "1025740212/train2": 2.0, + "1055157354/train0": 2.0, + "1055157354/train1": 2.0, + "1055157354/train2": 2.0, + "1055567618/train0": 2.0, + "1055567618/train1": 2.0, + "1055567618/train2": 2.0, + "1113268940/train0": 2.0, + "1113268940/train1": 2.0, + "1113268940/train2": 2.0, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 2.0, + "1176943004/train1": 2.0, + "1176943004/train2": 2.0, + "1189392917/train0": 2.0, + "1189392917/train1": 2.0, + "1189392917/train2": 2.0, + "1231780997/train0": 2.0, + "1231780997/train1": 2.0, + "1231780997/train2": 2.0, + "1271718007/train0": 2.0, + "1271718007/train1": 2.0, + "1271718007/train2": 2.0, + "1273247072/train0": 2.0, + "1273247072/train1": 2.0, + "1273247072/train2": 2.0, + "1358421325/train0": 2.0, + "1358421325/train1": 2.0, + "1358421325/train2": 2.0, + "1375492786/train0": 0.24345096945762634, + "1375492786/train1": 0.24345096945762634, + "1375492786/train2": 0.24345096945762634, + "141984615/train0": 0.20044784247875214, + "141984615/train1": 0.20044784247875214, + "141984615/train2": 0.20044784247875214, + "1475440973/train0": 2.0, + "1475440973/train1": 2.0, + "1475440973/train2": 2.0, + "1483645336/train0": 2.0, + "1483645336/train1": 2.0, + "1483645336/train2": 2.0, + "1522272686/train0": 2.0, + "1522272686/train1": 2.0, + "1522272686/train2": 2.0, + "1528740417/train0": 2.0, + "1528740417/train1": 2.0, + "1528740417/train2": 2.0, + "1549324805/train0": 1.4027505218982697, + "1549324805/train1": 1.4027505218982697, + "1549324805/train2": 1.4027505218982697, + "1607773860/train0": 2.0, + "1607773860/train1": 2.0, + "1607773860/train2": 2.0, + "1665739511/train0": 2.0, + "1665739511/train1": 2.0, + "1665739511/train2": 2.0, + "1747178030/train0": 2.0, + "1747178030/train1": 2.0, + "1747178030/train2": 2.0, + "1881223807/train0": 2.0, + "1881223807/train1": 2.0, + "1881223807/train2": 2.0, + "1919198188/train0": 2.0, + "1919198188/train1": 2.0, + "1919198188/train2": 2.0, + "1991414747/train0": 2.0, + "1991414747/train1": 2.0, + "1991414747/train2": 2.0, + "1997090716/train0": 2.0, + "1997090716/train1": 2.0, + "1997090716/train2": 2.0, + "2015956957/train0": 0.5298813670873642, + "2015956957/train1": 0.5298813670873642, + "2015956957/train2": 0.5298813670873642, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 2.0, + "211245842/train1": 2.0, + "211245842/train2": 2.0, + "2141802816/train0": 2.0, + "2141802816/train1": 2.0, + "2141802816/train2": 2.0, + "256770874/train0": 2.0, + "256770874/train1": 2.0, + "256770874/train2": 2.0, + "278008524/train0": 1.113811194896698, + "278008524/train1": 1.113811194896698, + "278008524/train2": 1.113811194896698, + "416183106/train0": 2.0, + "416183106/train1": 2.0, + "416183106/train2": 2.0, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 2.0, + "551504635/train1": 2.0, + "551504635/train2": 2.0, + "565369392/train0": 1.4266315797964733, + "565369392/train1": 1.4266315797964733, + "565369392/train2": 1.4266315797964733, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.23764699200789133, + "585904130/train1": 0.23764699200789133, + "585904130/train2": 0.23764699200789133, + "626486813/train0": 0.6124684102833271, + "626486813/train1": 0.6124684102833271, + "626486813/train2": 0.6124684102833271, + "636680137/train0": 2.0, + "636680137/train1": 2.0, + "636680137/train2": 2.0, + "706479600/train0": 2.0, + "706479600/train1": 2.0, + "706479600/train2": 2.0, + "750448296/train0": 0.23580759018659592, + "750448296/train1": 0.23580759018659592, + "750448296/train2": 0.23580759018659592, + "75070641/train0": 2.0, + "75070641/train1": 2.0, + "75070641/train2": 2.0, + "767747424/train0": 0.18863565661013126, + "767747424/train1": 0.18863565661013126, + "767747424/train2": 0.18863565661013126, + "77147264/train0": 2.0, + "77147264/train1": 2.0, + "77147264/train2": 2.0, + "794181626/train0": 2.0, + "794181626/train1": 2.0, + "794181626/train2": 2.0, + "957101489/train0": 2.0, + "957101489/train1": 2.0, + "957101489/train2": 2.0, + "998761162/train0": 2.0, + "998761162/train1": 2.0, + "998761162/train2": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 1000 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.5062634588583657, + "low": 1.2799855626507657, + "mean": 1.3999846732054932, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.610047281323877, + "low": 0.4546099290780141, + "mean": 0.5356973995271868, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.5356973995271868, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.3333333333333333, + "141984615/train1": 0.6666666666666666, + "141984615/train2": 0.3333333333333333, + "1475440973/train0": 0.0, + "1475440973/train1": 0.0, + "1475440973/train2": 0.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 0.0, + "1528740417/train1": 0.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.0, + "1549324805/train2": 0.0, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 0.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.8, + "2015956957/train1": 0.6, + "2015956957/train2": 0.8, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 1.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.5, + "278008524/train2": 0.0, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.75, + "626486813/train1": 0.75, + "626486813/train2": 0.75, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.5, + "750448296/train1": 0.5, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.75, + "767747424/train1": 0.75, + "767747424/train2": 0.75, + "77147264/train0": 0.0, + "77147264/train1": 0.5, + "77147264/train2": 0.0, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8444444444444444, + "low": 0.4740573152337858, + "mean": 0.6400703871292106, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.6400703871292106, + "per_task": { + "LiftPegUpright-v1": 0.5555555555555556, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3627450980392157, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.6666666666666666 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5652173913043478, + "low": 0.42995169082125606, + "mean": 0.5024154589371981, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.23027504332506904, + "low": 0.1884004262069988, + "mean": 0.20904812235529077, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.20904812235529077, + "per_seed": { + "1025740212/train0": 0.20050368294022264, + "1025740212/train1": 0.14400158741031796, + "1025740212/train2": 0.12800533274597592, + "1055157354/train0": 0.43030250381223584, + "1055157354/train1": 0.32609963351937027, + "1055157354/train2": 0.06066548841424423, + "1055567618/train0": 0.30857397468613657, + "1055567618/train1": 0.15812100047218913, + "1055567618/train2": 0.22991638132379041, + "1113268940/train0": 0.3774241061869338, + "1113268940/train1": 0.06911736513585261, + "1113268940/train2": 0.2534904144040469, + "1113371261/train0": 0.026017268580811406, + "1113371261/train1": 0.015047211698777683, + "1113371261/train2": 0.13380940221299786, + "1176943004/train0": 0.1246594058194818, + "1176943004/train1": 0.15442455751472706, + "1176943004/train2": 0.2664806376409974, + "1189392917/train0": 0.1340247769860469, + "1189392917/train1": 0.10709427493066431, + "1189392917/train2": 0.33283923875771465, + "1231780997/train0": 0.19966589920839098, + "1231780997/train1": 0.07419277505792353, + "1231780997/train2": 0.16621555087386797, + "1271718007/train0": 0.17939225581112472, + "1271718007/train1": 0.10710899843183383, + "1271718007/train2": 0.20722796191327264, + "1273247072/train0": 0.16962492308566807, + "1273247072/train1": 0.1410368025527166, + "1273247072/train2": 0.2134820953504291, + "1358421325/train0": 0.28863435648886215, + "1358421325/train1": 0.04701404433315851, + "1358421325/train2": 0.09625642405798478, + "1375492786/train0": 0.0996910344508803, + "1375492786/train1": 0.16573502466028908, + "1375492786/train2": 0.29573093343728263, + "141984615/train0": 0.297780428420543, + "141984615/train1": 0.16666657870321888, + "141984615/train2": 0.22785399017277783, + "1475440973/train0": 0.41171052461838353, + "1475440973/train1": 0.0862090980213101, + "1475440973/train2": 0.15809903475948942, + "1483645336/train0": 0.35034389349653194, + "1483645336/train1": 0.06079377324000063, + "1483645336/train2": 0.06905173445346233, + "1522272686/train0": 0.31643520896589616, + "1522272686/train1": 0.24182565394435904, + "1522272686/train2": 0.21028164676312622, + "1528740417/train0": 0.15436827657008492, + "1528740417/train1": 0.026961937751403514, + "1528740417/train2": 0.3242760158848682, + "1549324805/train0": 0.23727288575803293, + "1549324805/train1": 0.10346856229255086, + "1549324805/train2": 0.2818476718903797, + "1607773860/train0": 0.3951037237909243, + "1607773860/train1": 0.2544258391757567, + "1607773860/train2": 0.2143457671242473, + "1665739511/train0": 0.0, + "1665739511/train1": 0.05717674118582694, + "1665739511/train2": 0.26263826459603723, + "1747178030/train0": 0.11661420436902857, + "1747178030/train1": 0.1055876219626688, + "1747178030/train2": 0.33194730961537694, + "1881223807/train0": 0.20266639463366093, + "1881223807/train1": 0.10190418918720931, + "1881223807/train2": 0.1643392210528543, + "1919198188/train0": 0.09083822075370537, + "1919198188/train1": 0.2569381375337174, + "1919198188/train2": 0.21294576266807125, + "1991414747/train0": 0.09717880195623664, + "1991414747/train1": 0.44645064197830664, + "1991414747/train2": 0.613109405537996, + "1997090716/train0": 0.3641822422309461, + "1997090716/train1": 0.3553584127845171, + "1997090716/train2": 0.21318958083944717, + "2015956957/train0": 0.22628515797280127, + "2015956957/train1": 0.2284508985012672, + "2015956957/train2": 0.24731300175116924, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.14834095177221712, + "211245842/train1": 0.036206191308278124, + "211245842/train2": 0.08971682945849302, + "2141802816/train0": 0.07518554002658885, + "2141802816/train1": 0.40989236974714055, + "2141802816/train2": 0.3183279052482729, + "256770874/train0": 0.18757436508611464, + "256770874/train1": 0.3193203145842822, + "256770874/train2": 0.5133629211325229, + "278008524/train0": 0.10616076660738837, + "278008524/train1": 0.10627711331165646, + "278008524/train2": 0.40905575479886586, + "416183106/train0": 0.1517874168346938, + "416183106/train1": 0.1925298309196861, + "416183106/train2": 0.21826032886833752, + "534717141/train0": 0.18033001516380143, + "534717141/train1": 0.12736602361488303, + "534717141/train2": 0.15703220108978472, + "551504635/train0": 0.3791036296516339, + "551504635/train1": 0.23627606487156766, + "551504635/train2": 0.4327797462092032, + "565369392/train0": 0.21578265463075008, + "565369392/train1": 0.2859321646983037, + "565369392/train2": 0.2176430432872923, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.3300621116998719, + "585904130/train1": 0.1652498530853035, + "585904130/train2": 0.22254352429757418, + "626486813/train0": 0.24681469768825515, + "626486813/train1": 0.20632809068391378, + "626486813/train2": 0.14946876705188256, + "636680137/train0": 0.19331728269593437, + "636680137/train1": 0.45862923715160336, + "636680137/train2": 0.6813618274184183, + "706479600/train0": 0.2692917587355154, + "706479600/train1": 0.15129914778803955, + "706479600/train2": 0.15321122852132601, + "750448296/train0": 0.3795450047628438, + "750448296/train1": 0.1527117559178966, + "750448296/train2": 0.19031133205081463, + "75070641/train0": 0.13808834192306887, + "75070641/train1": 0.34224193319263285, + "75070641/train2": 0.2643509727736545, + "767747424/train0": 0.2798082148668758, + "767747424/train1": 0.13255125751519692, + "767747424/train2": 0.20625644377911734, + "77147264/train0": 0.22561784152146835, + "77147264/train1": 0.03379253116769787, + "77147264/train2": 0.13662790487889595, + "794181626/train0": 0.3405496231635778, + "794181626/train1": 0.3619001242329895, + "794181626/train2": 0.1880089150142482, + "957101489/train0": 0.2493251178612577, + "957101489/train1": 0.2708447347116597, + "957101489/train2": 0.5372965735602796, + "998761162/train0": 0.1991858618764328, + "998761162/train1": 0.24035198055524792, + "998761162/train2": 0.14873333521535068 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.21752558375317918, + "low": 0.18602643769659166, + "mean": 0.20364489391565144, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.20364489391565144, + "per_task": { + "LiftPegUpright-v1": 0.2123556134611932, + "PickCube-v1": 0.17471460699627128, + "PullCube-v1": 0.21225649007113187, + "PushCube-v1": 0.19356765816347193, + "StackCube-v1": 0.2253301008861889 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.22903802877401228, + "low": 0.19393562395778954, + "mean": 0.21104202926956342, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5638297872340425, + "low": 0.4011820330969268, + "mean": 0.4791962174940898, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.4791962174940898, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 1.0, + "1055567618/train1": 1.0, + "1055567618/train2": 1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 1.0, + "1271718007/train1": 1.0, + "1271718007/train2": 1.0, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 0.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 1.0, + "1528740417/train1": 0.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 1.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 0.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.0, + "2015956957/train2": 0.2, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.0, + "626486813/train1": 0.0, + "626486813/train2": 0.0, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.25, + "767747424/train1": 0.0, + "767747424/train2": 0.25, + "77147264/train0": 0.5, + "77147264/train1": 1.0, + "77147264/train2": 0.5, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6272498743086977, + "low": 0.1503267973856209, + "mean": 0.38930115635997986, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.38930115635997986, + "per_task": { + "LiftPegUpright-v1": 0.5555555555555556, + "PickCube-v1": 0.0, + "PullCube-v1": 0.19607843137254902, + "PushCube-v1": 0.7948717948717948, + "StackCube-v1": 0.4 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4444444444444444, + "low": 0.3188405797101449, + "mean": 0.38164251207729466, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.2685579196217494, + "low": 0.0768321513002364, + "mean": 0.1706855791962175, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.1706855791962175, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 1.0, + "1189392917/train1": 1.0, + "1189392917/train2": 1.0, + "1231780997/train0": 1.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": -1.0, + "1375492786/train1": -1.0, + "1375492786/train2": -1.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 0.0, + "1475440973/train1": -1.0, + "1475440973/train2": 0.0, + "1483645336/train0": 1.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 0.0, + "1528740417/train1": -1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": -0.3333333333333333, + "1549324805/train2": -0.3333333333333333, + "1607773860/train0": 1.0, + "1607773860/train1": 1.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": -1.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 1.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.0, + "2015956957/train2": 0.2, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": -1.0, + "256770874/train1": -1.0, + "256770874/train2": -1.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": -0.3333333333333333, + "565369392/train1": -0.3333333333333333, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": -0.25, + "626486813/train1": -0.25, + "626486813/train2": -0.25, + "636680137/train0": -1.0, + "636680137/train1": -1.0, + "636680137/train2": -1.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.0, + "767747424/train1": -0.25, + "767747424/train2": 0.0, + "77147264/train0": 0.0, + "77147264/train1": 0.5, + "77147264/train2": 0.0, + "794181626/train0": 1.0, + "794181626/train1": 1.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3367521367521367, + "low": -0.014379084967320262, + "mean": 0.16227249874308697, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.16227249874308697, + "per_task": { + "LiftPegUpright-v1": 0.2222222222222222, + "PickCube-v1": 0.0, + "PullCube-v1": -0.09803921568627451, + "PushCube-v1": 0.48717948717948717, + "StackCube-v1": 0.2 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.18357487922705315, + "low": 0.03864734299516908, + "mean": 0.10628019323671498, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.2493007679215036, + "low": 0.9614900063099141, + "mean": 1.1006925804823984, + "n": 141, + "num_samples": 1000 + }, + "mean": 1.1006925804823984, + "per_seed": { + "1025740212/train0": 2.0, + "1025740212/train1": 0.2148684412240982, + "1025740212/train2": 0.7146708369255066, + "1055157354/train0": 0.5705926418304443, + "1055157354/train1": 0.812659502029419, + "1055157354/train2": 0.8651304244995117, + "1055567618/train0": 2.0, + "1055567618/train1": 2.0, + "1055567618/train2": 2.0, + "1113268940/train0": 0.5588552355766296, + "1113268940/train1": 0.7256770730018616, + "1113268940/train2": 0.6990174651145935, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 2.0, + "1176943004/train1": 2.0, + "1176943004/train2": 2.0, + "1189392917/train0": 2.0, + "1189392917/train1": 2.0, + "1189392917/train2": 2.0, + "1231780997/train0": 2.0, + "1231780997/train1": 2.0, + "1231780997/train2": 0.725460946559906, + "1271718007/train0": 2.0, + "1271718007/train1": 2.0, + "1271718007/train2": 2.0, + "1273247072/train0": 0.4659159183502197, + "1273247072/train1": 0.46439141035079956, + "1273247072/train2": 0.4708559513092041, + "1358421325/train0": 0.8147144317626953, + "1358421325/train1": 0.8245300054550171, + "1358421325/train2": 2.0, + "1375492786/train0": 0.0015756288776174188, + "1375492786/train1": 0.001480837701819837, + "1375492786/train2": 0.0012095769634470344, + "141984615/train0": 0.06459133536554873, + "141984615/train1": 0.07189804397057742, + "141984615/train2": 0.08839488192461431, + "1475440973/train0": 2.0, + "1475440973/train1": 0.7181311249732971, + "1475440973/train2": 2.0, + "1483645336/train0": 2.0, + "1483645336/train1": 2.0, + "1483645336/train2": 2.0, + "1522272686/train0": 0.21208032965660095, + "1522272686/train1": 0.24029652774333954, + "1522272686/train2": 0.23198044300079346, + "1528740417/train0": 2.0, + "1528740417/train1": 0.7255622744560242, + "1528740417/train2": 2.0, + "1549324805/train0": 1.344537331412236, + "1549324805/train1": 0.6790153384208679, + "1549324805/train2": 0.6867360668256879, + "1607773860/train0": 2.0, + "1607773860/train1": 2.0, + "1607773860/train2": 2.0, + "1665739511/train0": 2.0, + "1665739511/train1": 2.0, + "1665739511/train2": 2.0, + "1747178030/train0": 0.1273784190416336, + "1747178030/train1": 2.0, + "1747178030/train2": 2.0, + "1881223807/train0": 0.027498364448547363, + "1881223807/train1": 2.0, + "1881223807/train2": 0.040287479758262634, + "1919198188/train0": 0.09917314350605011, + "1919198188/train1": 0.2203868180513382, + "1919198188/train2": 0.04859863221645355, + "1991414747/train0": 0.13679209351539612, + "1991414747/train1": 2.0, + "1991414747/train2": 2.0, + "1997090716/train0": 2.0, + "1997090716/train1": 2.0, + "1997090716/train2": 2.0, + "2015956957/train0": 0.46120103299617765, + "2015956957/train1": 0.06726333405822515, + "2015956957/train2": 0.45940959043800833, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 0.8135772347450256, + "211245842/train1": 2.0, + "211245842/train2": 2.0, + "2141802816/train0": 0.08595690131187439, + "2141802816/train1": 2.0, + "2141802816/train2": 2.0, + "256770874/train0": 0.574478030204773, + "256770874/train1": 0.8128684163093567, + "256770874/train2": 0.19852638244628906, + "278008524/train0": 0.013509750366210938, + "278008524/train1": 0.019306766451336443, + "278008524/train2": 0.01439405232667923, + "416183106/train0": 0.581784188747406, + "416183106/train1": 0.22490769624710083, + "416183106/train2": 0.23963995277881622, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 2.0, + "551504635/train1": 2.0, + "551504635/train2": 2.0, + "565369392/train0": 0.14469666282335916, + "565369392/train1": 0.11818361282348633, + "565369392/train2": 0.7870745907227198, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.034002700820565224, + "585904130/train1": 0.01988216955214739, + "585904130/train2": 0.029573063055674236, + "626486813/train0": 0.07908930000849068, + "626486813/train1": 0.06953954696655273, + "626486813/train2": 0.06192737747915089, + "636680137/train0": 0.024462858214974403, + "636680137/train1": 0.015581388957798481, + "636680137/train2": 0.036433976143598557, + "706479600/train0": 0.21975377202033997, + "706479600/train1": 2.0, + "706479600/train2": 2.0, + "750448296/train0": 0.109864491969347, + "750448296/train1": 0.10195614024996758, + "750448296/train2": 0.08245231583714485, + "75070641/train0": 0.2283252328634262, + "75070641/train1": 0.24087366461753845, + "75070641/train2": 0.21781498193740845, + "767747424/train0": 0.5921752639114857, + "767747424/train1": 0.10304383677430451, + "767747424/train2": 0.5790140572935343, + "77147264/train0": 1.1132144927978516, + "77147264/train1": 2.0, + "77147264/train2": 1.11046252399683, + "794181626/train0": 2.0, + "794181626/train1": 2.0, + "794181626/train2": 2.0, + "957101489/train0": 0.5799027681350708, + "957101489/train1": 0.5798611640930176, + "957101489/train2": 0.5639582276344299, + "998761162/train0": 0.09276536107063293, + "998761162/train1": 2.0, + "998761162/train2": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.374731707572937, + "low": 0.5777494399969655, + "mean": 0.9933715603032812, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.9933715603032812, + "per_task": { + "LiftPegUpright-v1": 1.3934579665462177, + "PickCube-v1": 0.46705442667007446, + "PullCube-v1": 0.44994396614321674, + "PushCube-v1": 1.6016510277986526, + "StackCube-v1": 1.054750414358245 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.9970273724183494, + "low": 0.7636547594911595, + "mean": 0.8807045891532306, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.4425467990562275, + "low": 0.12576461925014992, + "mean": 0.2843423625940884, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.2843423625940884, + "per_seed": { + "1025740212/train0": 1.471626341342926, + "1025740212/train1": -0.31350521743297577, + "1025740212/train2": 0.18629717826843262, + "1055157354/train0": 0.37410856783390045, + "1055157354/train1": 0.6161754280328751, + "1055157354/train2": 0.6686463505029678, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 0.13654270768165588, + "1113268940/train1": 0.3033645451068878, + "1113268940/train2": 0.27670493721961975, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.845386266708374, + "1176943004/train1": 1.845386266708374, + "1176943004/train2": 1.845386266708374, + "1189392917/train0": 1.8915813565254211, + "1189392917/train1": 1.8915813565254211, + "1189392917/train2": 1.8915813565254211, + "1231780997/train0": 1.2729938626289368, + "1231780997/train1": 1.2729938626289368, + "1231780997/train2": -0.0015451908111572266, + "1271718007/train0": 0.584618866443634, + "1271718007/train1": 0.584618866443634, + "1271718007/train2": 0.584618866443634, + "1273247072/train0": 0.3124862611293793, + "1273247072/train1": 0.3109617531299591, + "1273247072/train2": 0.31742629408836365, + "1358421325/train0": -0.012500584125518799, + "1358421325/train1": -0.0026850104331970215, + "1358421325/train2": 1.1727849841117859, + "1375492786/train0": -1.9984243711223826, + "1375492786/train1": -1.9985191622981802, + "1375492786/train2": -1.998790423036553, + "141984615/train0": -0.029220803718393046, + "141984615/train1": -0.021914095113364358, + "141984615/train2": -0.005417257159327467, + "1475440973/train0": 0.0, + "1475440973/train1": -1.2818688750267029, + "1475440973/train2": 0.0, + "1483645336/train0": 1.9835839420557022, + "1483645336/train1": 1.9835839420557022, + "1483645336/train2": 1.9835839420557022, + "1522272686/train0": 0.01363721489906311, + "1522272686/train1": 0.0418534129858017, + "1522272686/train2": 0.033537328243255615, + "1528740417/train0": 0.0, + "1528740417/train1": -1.2744377255439758, + "1528740417/train2": 0.0, + "1549324805/train0": -0.03421011691292127, + "1549324805/train1": -0.6997321099042892, + "1549324805/train2": -0.6920113814994693, + "1607773860/train0": 1.1673271656036377, + "1607773860/train1": 1.1673271656036377, + "1607773860/train2": 1.1673271656036377, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": -0.019252806901931763, + "1747178030/train1": 1.8533687740564346, + "1747178030/train2": 1.8533687740564346, + "1881223807/train0": -0.004528626799583435, + "1881223807/train1": 1.9679730087518692, + "1881223807/train2": 0.008260488510131836, + "1919198188/train0": 0.06197427213191986, + "1919198188/train1": 0.18318794667720795, + "1919198188/train2": 0.011399760842323303, + "1991414747/train0": -1.8632079064846039, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 1.8607252091169357, + "1997090716/train1": 1.8607252091169357, + "1997090716/train2": 1.8607252091169357, + "2015956957/train0": 0.39408124312758447, + "2015956957/train1": 0.00014354418963193893, + "2015956957/train2": 0.3922898005694151, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": -0.0252571702003479, + "211245842/train1": 1.1611655950546265, + "211245842/train2": 1.1611655950546265, + "2141802816/train0": 0.01514965295791626, + "2141802816/train1": 1.9291927516460419, + "2141802816/train2": 1.9291927516460419, + "256770874/train0": -1.425521969795227, + "256770874/train1": -1.1871315836906433, + "256770874/train2": -1.801473617553711, + "278008524/train0": -0.09396211802959442, + "278008524/train1": -0.08816510194446892, + "278008524/train2": -0.09307781606912613, + "416183106/train0": 0.38145777583122253, + "416183106/train1": 0.02458128333091736, + "416183106/train2": 0.03931353986263275, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2760274410247803, + "551504635/train1": 1.2760274410247803, + "551504635/train2": 1.2760274410247803, + "565369392/train0": -0.655068983634313, + "565369392/train1": -0.6815820336341858, + "565369392/train2": -0.012691055734952291, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": -0.04128939720491568, + "585904130/train1": -0.055409928473333515, + "585904130/train2": -0.04571903496980667, + "626486813/train0": -0.4263507166178897, + "626486813/train1": -0.43590046965982765, + "626486813/train2": -0.4435126391472295, + "636680137/train0": -1.9755371417850256, + "636680137/train1": -1.9844186110422015, + "636680137/train2": -1.9635660238564014, + "706479600/train0": -0.6153350174427032, + "706479600/train1": 1.1649112105369568, + "706479600/train2": 1.1649112105369568, + "750448296/train0": -0.0016690678894519806, + "750448296/train1": -0.009577419608831406, + "750448296/train2": -0.02908124402165413, + "75070641/train0": 0.025573596358299255, + "75070641/train1": 0.0381220281124115, + "75070641/train2": 0.015063345432281494, + "767747424/train0": 0.04465107619762421, + "767747424/train1": -0.44448035093955696, + "767747424/train2": 0.03148986957967281, + "77147264/train0": -0.30836498737335205, + "77147264/train1": 0.5784205198287964, + "77147264/train2": -0.3111169561743736, + "794181626/train0": 1.8190439492464066, + "794181626/train1": 1.8190439492464066, + "794181626/train2": 1.8190439492464066, + "957101489/train0": 0.37618090212345123, + "957101489/train1": 0.376139298081398, + "957101489/train2": 0.36023636162281036, + "998761162/train0": 0.024948790669441223, + "998761162/train1": 1.9321834295988083, + "998761162/train2": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6703226770982782, + "low": 0.004672606112709377, + "mean": 0.3229939035546056, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.3229939035546056, + "per_task": { + "LiftPegUpright-v1": 0.3265138193964958, + "PickCube-v1": 0.313624769449234, + "PullCube-v1": -0.20129550277830704, + "PushCube-v1": 0.9244865671946452, + "StackCube-v1": 0.25163986451096004 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.28991549138085476, + "low": 0.04696476409731406, + "mean": 0.16263165119652984, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3250591016548463, + "low": 0.1879432624113475, + "mean": 0.25650118203309696, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.25650118203309696, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 0.0, + "1055567618/train1": 1.0, + "1055567618/train2": 0.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 1.0, + "1113371261/train1": 1.0, + "1113371261/train2": 1.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 1.0, + "1189392917/train1": 0.0, + "1189392917/train2": 1.0, + "1231780997/train0": 0.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 0.0, + "1475440973/train1": 0.0, + "1475440973/train2": 0.0, + "1483645336/train0": 0.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 1.0, + "1528740417/train1": 0.0, + "1528740417/train2": 1.0, + "1549324805/train0": 0.6666666666666666, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 0.0, + "1607773860/train1": 1.0, + "1607773860/train2": 0.0, + "1665739511/train0": 1.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 0.0, + "1747178030/train1": 0.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 0.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 0.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 0.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.0, + "2015956957/train2": 0.0, + "2054899126/train0": 1.0, + "2054899126/train1": 1.0, + "2054899126/train2": 1.0, + "211245842/train0": 0.0, + "211245842/train1": 0.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 1.0, + "534717141/train1": 1.0, + "534717141/train2": 1.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 1.0, + "565621826/train1": 1.0, + "565621826/train2": 1.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.0, + "626486813/train1": 0.0, + "626486813/train2": 0.0, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 0.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 0.5, + "77147264/train1": 0.5, + "77147264/train2": 0.0, + "794181626/train0": 1.0, + "794181626/train1": 0.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.333001508295626, + "low": 0.06, + "mean": 0.18171945701357467, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.18171945701357467, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.13333333333333333 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2560386473429952, + "low": 0.1497584541062802, + "mean": 0.2028985507246377, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.033687943262411355, + "low": -0.14243498817966901, + "mean": -0.052009456264775406, + "n": 141, + "num_samples": 1000 + }, + "mean": -0.052009456264775406, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": -1.0, + "1055567618/train1": 0.0, + "1055567618/train2": -1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 1.0, + "1189392917/train1": 0.0, + "1189392917/train2": 1.0, + "1231780997/train0": 0.0, + "1231780997/train1": 1.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.0, + "1271718007/train1": 0.0, + "1271718007/train2": 0.0, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 1.0, + "1375492786/train0": -1.0, + "1375492786/train1": -1.0, + "1375492786/train2": -1.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": -1.0, + "1475440973/train1": -1.0, + "1475440973/train2": -1.0, + "1483645336/train0": 0.0, + "1483645336/train1": 1.0, + "1483645336/train2": 1.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 0.0, + "1528740417/train1": -1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": -0.3333333333333333, + "1549324805/train2": -0.3333333333333333, + "1607773860/train0": 0.0, + "1607773860/train1": 1.0, + "1607773860/train2": 0.0, + "1665739511/train0": 0.0, + "1665739511/train1": -1.0, + "1665739511/train2": -1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 0.0, + "1747178030/train2": 1.0, + "1881223807/train0": 0.0, + "1881223807/train1": 0.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": -1.0, + "1991414747/train1": -1.0, + "1991414747/train2": -1.0, + "1997090716/train0": 0.0, + "1997090716/train1": 1.0, + "1997090716/train2": 1.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.0, + "2015956957/train2": 0.0, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 0.0, + "211245842/train2": 1.0, + "2141802816/train0": 0.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": -1.0, + "256770874/train1": -1.0, + "256770874/train2": -1.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": -0.3333333333333333, + "565369392/train1": -0.3333333333333333, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": -0.25, + "626486813/train1": -0.25, + "626486813/train2": -0.25, + "636680137/train0": -1.0, + "636680137/train1": -1.0, + "636680137/train2": -1.0, + "706479600/train0": 0.0, + "706479600/train1": 0.0, + "706479600/train2": 1.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": -0.25, + "767747424/train1": -0.25, + "767747424/train2": -0.25, + "77147264/train0": 0.0, + "77147264/train1": 0.0, + "77147264/train2": -0.5, + "794181626/train0": 1.0, + "794181626/train1": 0.0, + "794181626/train2": 1.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 1.0, + "998761162/train2": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.05897435897435899, + "low": -0.1388235294117647, + "mean": -0.04530920060331824, + "n": 5, + "num_samples": 1000 + }, + "mean": -0.04530920060331824, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.14705882352941177, + "PushCube-v1": 0.15384615384615385, + "StackCube-v1": -0.06666666666666667 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.004830917874396135, + "low": -0.13526570048309178, + "mean": -0.07246376811594203, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.8042305547841185, + "low": 0.5404751630640456, + "mean": 0.6726714667763278, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.6726714667763278, + "per_seed": { + "1025740212/train0": 0.7183936238288879, + "1025740212/train1": 0.21175099909305573, + "1025740212/train2": 0.6627864837646484, + "1055157354/train0": 0.12073369324207306, + "1055157354/train1": 0.16174951195716858, + "1055157354/train2": 0.8651304244995117, + "1055567618/train0": 0.1717667579650879, + "1055567618/train1": 2.0, + "1055567618/train2": 0.15755623579025269, + "1113268940/train0": 0.44024136662483215, + "1113268940/train1": 0.6624154448509216, + "1113268940/train2": 0.21454350650310516, + "1113371261/train0": 2.0, + "1113371261/train1": 2.0, + "1113371261/train2": 2.0, + "1176943004/train0": 0.06999577581882477, + "1176943004/train1": 0.04685409367084503, + "1176943004/train2": 0.041229456663131714, + "1189392917/train0": 2.0, + "1189392917/train1": 0.0735837072134018, + "1189392917/train2": 2.0, + "1231780997/train0": 0.7229657769203186, + "1231780997/train1": 2.0, + "1231780997/train2": 0.725460946559906, + "1271718007/train0": 1.0917481482028961, + "1271718007/train1": 1.1047073602676392, + "1271718007/train2": 1.079995036125183, + "1273247072/train0": 0.045715175569057465, + "1273247072/train1": 0.0375710129737854, + "1273247072/train2": 0.07777997106313705, + "1358421325/train0": 0.179222971200943, + "1358421325/train1": 0.8165659308433533, + "1358421325/train2": 2.0, + "1375492786/train0": 0.0007540385122410953, + "1375492786/train1": 0.00048063197755254805, + "1375492786/train2": 0.0008456110954284668, + "141984615/train0": 0.007215420599095523, + "141984615/train1": 0.03405998833477497, + "141984615/train2": 0.05300074194868406, + "1475440973/train0": 0.7146047949790955, + "1475440973/train1": 0.7181311249732971, + "1475440973/train2": 0.7040655612945557, + "1483645336/train0": 0.11001592874526978, + "1483645336/train1": 2.0, + "1483645336/train2": 2.0, + "1522272686/train0": 0.181088387966156, + "1522272686/train1": 0.20458371937274933, + "1522272686/train2": 0.17567642033100128, + "1528740417/train0": 2.0, + "1528740417/train1": 0.7171411514282227, + "1528740417/train2": 2.0, + "1549324805/train0": 1.3434468836834033, + "1549324805/train1": 0.6740981936454773, + "1549324805/train2": 0.6762484377250075, + "1607773860/train0": 0.14438587427139282, + "1607773860/train1": 2.0, + "1607773860/train2": 0.814136266708374, + "1665739511/train0": 2.0, + "1665739511/train1": 0.8184402585029602, + "1665739511/train2": 0.13187798857688904, + "1747178030/train0": 0.10735879093408585, + "1747178030/train1": 0.09237110614776611, + "1747178030/train2": 2.0, + "1881223807/train0": 0.025111451745033264, + "1881223807/train1": 0.03511819243431091, + "1881223807/train2": 0.040287479758262634, + "1919198188/train0": 0.07726143300533295, + "1919198188/train1": 0.03183290362358093, + "1919198188/train2": 0.029491469264030457, + "1991414747/train0": 0.11926940083503723, + "1991414747/train1": 0.05380085110664368, + "1991414747/train2": 0.023216769099235535, + "1997090716/train0": 0.054733291268348694, + "1997090716/train1": 2.0, + "1997090716/train2": 2.0, + "2015956957/train0": 0.0695308968424797, + "2015956957/train1": 0.06189579609781504, + "2015956957/train2": 0.02798597477376461, + "2054899126/train0": 2.0, + "2054899126/train1": 2.0, + "2054899126/train2": 2.0, + "211245842/train0": 0.8135772347450256, + "211245842/train1": 0.8183318376541138, + "211245842/train2": 2.0, + "2141802816/train0": 0.07753174006938934, + "2141802816/train1": 0.06590981781482697, + "2141802816/train2": 0.04525895416736603, + "256770874/train0": 0.1245432198047638, + "256770874/train1": 0.19993644952774048, + "256770874/train2": 0.11944694817066193, + "278008524/train0": 0.012642060639336705, + "278008524/train1": 0.011877144395839423, + "278008524/train2": 0.010367284179665148, + "416183106/train0": 0.2363920658826828, + "416183106/train1": 0.20836615562438965, + "416183106/train2": 0.19381189346313477, + "534717141/train0": 2.0, + "534717141/train1": 2.0, + "534717141/train2": 2.0, + "551504635/train0": 0.7199956774711609, + "551504635/train1": 0.7220139503479004, + "551504635/train2": 0.712515652179718, + "565369392/train0": 0.09290684759616852, + "565369392/train1": 0.07017792761325836, + "565369392/train2": 0.729317918419838, + "565621826/train0": 2.0, + "565621826/train1": 2.0, + "565621826/train2": 2.0, + "585904130/train0": 0.016591727423171203, + "585904130/train1": 0.015381396127243837, + "585904130/train2": 0.022772716358304024, + "626486813/train0": 0.04527618060819805, + "626486813/train1": 0.041404386749491096, + "626486813/train2": 0.0486708318348974, + "636680137/train0": 0.01964247226715088, + "636680137/train1": 0.010638435371220112, + "636680137/train2": 0.008193274028599262, + "706479600/train0": 0.07415902614593506, + "706479600/train1": 0.814424455165863, + "706479600/train2": 2.0, + "750448296/train0": 0.09399121813476086, + "750448296/train1": 0.057780176401138306, + "750448296/train2": 0.046717485412955284, + "75070641/train0": 0.22510160505771637, + "75070641/train1": 0.2330203801393509, + "75070641/train2": 0.1694907248020172, + "767747424/train0": 0.06817136844620109, + "767747424/train1": 0.05761944816913456, + "767747424/train2": 0.07078132405877113, + "77147264/train0": 1.0998340174555779, + "77147264/train1": 1.1120431125164032, + "77147264/train2": 0.1726423129439354, + "794181626/train0": 2.0, + "794181626/train1": 0.03734047710895538, + "794181626/train2": 2.0, + "957101489/train0": 0.2191106230020523, + "957101489/train1": 0.23941433429718018, + "957101489/train2": 0.1758389174938202, + "998761162/train0": 0.09167696535587311, + "998761162/train1": 2.0, + "998761162/train2": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.805851331210544, + "low": 0.23790077653196123, + "mean": 0.5417194783057231, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.5417194783057231, + "per_task": { + "LiftPegUpright-v1": 0.8537236700455347, + "PickCube-v1": 0.05368871986865997, + "PullCube-v1": 0.33309935569830523, + "PushCube-v1": 0.9538667843892024, + "StackCube-v1": 0.5142188615269131 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6390190568436522, + "low": 0.4293877342067702, + "mean": 0.5306515935340228, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.006899217451949011, + "low": -0.2966154331014432, + "mean": -0.14367875111198225, + "n": 141, + "num_samples": 1000 + }, + "mean": -0.14367875111198225, + "per_seed": { + "1025740212/train0": 0.19001996517181396, + "1025740212/train1": -0.31662265956401825, + "1025740212/train2": 0.13441282510757446, + "1055157354/train0": -0.07575038075447083, + "1055157354/train1": -0.034734562039375305, + "1055157354/train2": 0.6686463505029678, + "1055567618/train0": -1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": -1.8424437642097473, + "1113268940/train0": 0.0179288387298584, + "1113268940/train1": 0.24010291695594788, + "1113268940/train2": -0.2077690213918686, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": -0.08461795747280121, + "1176943004/train1": -0.10775963962078094, + "1176943004/train2": -0.11338427662849426, + "1189392917/train0": 1.8915813565254211, + "1189392917/train1": -0.03483493626117706, + "1189392917/train2": 1.8915813565254211, + "1231780997/train0": -0.004040360450744629, + "1231780997/train1": 1.2729938626289368, + "1231780997/train2": -0.0015451908111572266, + "1271718007/train0": -0.32363298535346985, + "1271718007/train1": -0.3106737732887268, + "1271718007/train2": -0.33538609743118286, + "1273247072/train0": -0.10771448165178299, + "1273247072/train1": -0.11585864424705505, + "1273247072/train2": -0.0756496861577034, + "1358421325/train0": -0.6479920446872711, + "1358421325/train1": -0.01064908504486084, + "1358421325/train2": 1.1727849841117859, + "1375492786/train0": -1.999245961487759, + "1375492786/train1": -1.9995193680224475, + "1375492786/train2": -1.9991543889045715, + "141984615/train0": -0.08659671848484625, + "141984615/train1": -0.05975215074916681, + "141984615/train2": -0.04081139713525772, + "1475440973/train0": -1.2853952050209045, + "1475440973/train1": -1.2818688750267029, + "1475440973/train2": -1.2959344387054443, + "1483645336/train0": 0.09359987080097198, + "1483645336/train1": 1.9835839420557022, + "1483645336/train2": 1.9835839420557022, + "1522272686/train0": -0.017354726791381836, + "1522272686/train1": 0.006140604615211487, + "1522272686/train2": -0.02276669442653656, + "1528740417/train0": 0.0, + "1528740417/train1": -1.2828588485717773, + "1528740417/train2": 0.0, + "1549324805/train0": -0.03530056464175383, + "1549324805/train1": -0.7046492546796799, + "1549324805/train2": -0.7024990106001496, + "1607773860/train0": -0.6882869601249695, + "1607773860/train1": 1.1673271656036377, + "1607773860/train2": -0.01853656768798828, + "1665739511/train0": 0.0, + "1665739511/train1": -1.1815597414970398, + "1665739511/train2": -1.868122011423111, + "1747178030/train0": -0.03927243500947952, + "1747178030/train1": -0.054260119795799255, + "1747178030/train2": 1.8533687740564346, + "1881223807/train0": -0.006915539503097534, + "1881223807/train1": 0.0030912011861801147, + "1881223807/train2": 0.008260488510131836, + "1919198188/train0": 0.0400625616312027, + "1919198188/train1": -0.005365967750549316, + "1919198188/train2": -0.0077074021100997925, + "1991414747/train0": -1.8807305991649628, + "1991414747/train1": -1.9461991488933563, + "1991414747/train2": -1.9767832309007645, + "1997090716/train0": -0.08454149961471558, + "1997090716/train1": 1.8607252091169357, + "1997090716/train2": 1.8607252091169357, + "2015956957/train0": 0.0024111069738864898, + "2015956957/train1": -0.005223993770778179, + "2015956957/train2": -0.039133815094828606, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": -0.0252571702003479, + "211245842/train1": -0.020502567291259766, + "211245842/train2": 1.1611655950546265, + "2141802816/train0": 0.006724491715431213, + "2141802816/train1": -0.0048974305391311646, + "2141802816/train2": -0.025548294186592102, + "256770874/train0": -1.8754567801952362, + "256770874/train1": -1.8000635504722595, + "256770874/train2": -1.880553051829338, + "278008524/train0": -0.09482980775646865, + "278008524/train1": -0.09559472399996594, + "278008524/train2": -0.09710458421614021, + "416183106/train0": 0.03606565296649933, + "416183106/train1": 0.008039742708206177, + "416183106/train2": -0.006514519453048706, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": -0.003976881504058838, + "551504635/train1": -0.001958608627319336, + "551504635/train2": -0.011456906795501709, + "565369392/train0": -0.7068587988615036, + "565369392/train1": -0.7295877188444138, + "565369392/train2": -0.07044772803783417, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": -0.058700370602309704, + "585904130/train1": -0.05991070189823707, + "585904130/train2": -0.05251938166717688, + "626486813/train0": -0.46016383601818234, + "626486813/train1": -0.4640356298768893, + "626486813/train2": -0.456769184791483, + "636680137/train0": -1.9803575277328491, + "636680137/train1": -1.9893615646287799, + "636680137/train2": -1.9918067259714007, + "706479600/train0": -0.7609297633171082, + "706479600/train1": -0.020664334297180176, + "706479600/train2": 1.1649112105369568, + "750448296/train0": -0.017542341724038124, + "750448296/train1": -0.053753383457660675, + "750448296/train2": -0.0648160744458437, + "75070641/train0": 0.022349968552589417, + "75070641/train1": 0.030268743634223938, + "75070641/train2": -0.03326091170310974, + "767747424/train0": -0.4793528192676604, + "767747424/train1": -0.4899047395447269, + "767747424/train2": -0.47674286365509033, + "77147264/train0": -0.32174546271562576, + "77147264/train1": -0.3095363676548004, + "77147264/train2": -1.2489371672272682, + "794181626/train0": 1.8190439492464066, + "794181626/train1": -0.14361557364463806, + "794181626/train2": 1.8190439492464066, + "957101489/train0": 0.01538875699043274, + "957101489/train1": 0.03569246828556061, + "957101489/train2": -0.027882948517799377, + "998761162/train0": 0.023860394954681396, + "998761162/train1": 1.9321834295988083, + "998761162/train2": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.0807332034294422, + "low": -0.2854568160382735, + "mean": -0.12865817844295258, + "n": 5, + "num_samples": 1000 + }, + "mean": -0.12865817844295258, + "per_task": { + "LiftPegUpright-v1": -0.213220477104187, + "PickCube-v1": -0.09974093735218048, + "PullCube-v1": -0.31814011322321856, + "PushCube-v1": 0.276702323785195, + "StackCube-v1": -0.2888916883203718 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.07080568554462496, + "low": -0.2967283767217811, + "mean": -0.1874213444226779, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.558614392092107, + "low": 0.3184796732449662, + "mean": 0.42802111370607065, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.42802111370607065, + "per_seed": { + "1025740212/train0": 1.281606376171112, + "1025740212/train1": 0.0031174421310424805, + "1025740212/train2": 0.051884353160858154, + "1055157354/train0": 0.4498589485883713, + "1055157354/train1": 0.6509099900722504, + "1055157354/train2": 0.0, + "1055567618/train0": 1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": 1.8424437642097473, + "1113268940/train0": 0.11861386895179749, + "1113268940/train1": 0.06326162815093994, + "1113268940/train2": 0.48447395861148834, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.9300042241811752, + "1176943004/train1": 1.953145906329155, + "1176943004/train2": 1.9587705433368683, + "1189392917/train0": 0.0, + "1189392917/train1": 1.9264162927865982, + "1189392917/train2": 0.0, + "1231780997/train0": 1.2770342230796814, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.9082518517971039, + "1271718007/train1": 0.8952926397323608, + "1271718007/train2": 0.9200049638748169, + "1273247072/train0": 0.42020074278116226, + "1273247072/train1": 0.42682039737701416, + "1273247072/train2": 0.39307598024606705, + "1358421325/train0": 0.6354914605617523, + "1358421325/train1": 0.007964074611663818, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0008215903653763235, + "1375492786/train1": 0.001000205724267289, + "1375492786/train2": 0.00036396586801856756, + "141984615/train0": 0.05737591476645321, + "141984615/train1": 0.03783805563580245, + "141984615/train2": 0.03539413997593025, + "1475440973/train0": 1.2853952050209045, + "1475440973/train1": 0.0, + "1475440973/train2": 1.2959344387054443, + "1483645336/train0": 1.8899840712547302, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.030991941690444946, + "1522272686/train1": 0.03571280837059021, + "1522272686/train2": 0.056304022669792175, + "1528740417/train0": 0.0, + "1528740417/train1": 0.008421123027801514, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0010904477288325627, + "1549324805/train1": 0.004917144775390625, + "1549324805/train2": 0.010487629100680351, + "1607773860/train0": 1.8556141257286072, + "1607773860/train1": 0.0, + "1607773860/train2": 1.185863733291626, + "1665739511/train0": 0.0, + "1665739511/train1": 1.1815597414970398, + "1665739511/train2": 1.868122011423111, + "1747178030/train0": 0.02001962810754776, + "1747178030/train1": 1.9076288938522339, + "1747178030/train2": 0.0, + "1881223807/train0": 0.002386912703514099, + "1881223807/train1": 1.964881807565689, + "1881223807/train2": 0.0, + "1919198188/train0": 0.021911710500717163, + "1919198188/train1": 0.18855391442775726, + "1919198188/train2": 0.019107162952423096, + "1991414747/train0": 0.017522692680358887, + "1991414747/train1": 1.9461991488933563, + "1991414747/train2": 1.9767832309007645, + "1997090716/train0": 1.9452667087316513, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.39167013615369795, + "2015956957/train1": 0.005367537960410118, + "2015956957/train2": 0.4314236156642437, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.1816681623458862, + "211245842/train2": 0.0, + "2141802816/train0": 0.008425161242485046, + "2141802816/train1": 1.934090182185173, + "2141802816/train2": 1.954741045832634, + "256770874/train0": 0.44993481040000916, + "256770874/train1": 0.6129319667816162, + "256770874/train2": 0.07907943427562714, + "278008524/train0": 0.0008676897268742323, + "278008524/train1": 0.0074296220554970205, + "278008524/train2": 0.0040267681470140815, + "416183106/train0": 0.3453921228647232, + "416183106/train1": 0.01654154062271118, + "416183106/train2": 0.04582805931568146, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2800043225288391, + "551504635/train1": 1.2779860496520996, + "551504635/train2": 1.287484347820282, + "565369392/train0": 0.05178981522719065, + "565369392/train1": 0.048005685210227966, + "565369392/train2": 0.057756672302881874, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.01741097339739402, + "585904130/train1": 0.004500773424903552, + "585904130/train2": 0.006800346697370212, + "626486813/train0": 0.033813119400292635, + "626486813/train1": 0.02813516021706164, + "626486813/train2": 0.013256545644253492, + "636680137/train0": 0.0048203859478235245, + "636680137/train1": 0.004942953586578369, + "636680137/train2": 0.028240702114999294, + "706479600/train0": 0.1455947458744049, + "706479600/train1": 1.185575544834137, + "706479600/train2": 0.0, + "750448296/train0": 0.015873273834586143, + "750448296/train1": 0.04417596384882927, + "750448296/train2": 0.03573483042418957, + "75070641/train0": 0.003223627805709839, + "75070641/train1": 0.007853284478187561, + "75070641/train2": 0.048324257135391235, + "767747424/train0": 0.5240038954652846, + "767747424/train1": 0.04542438860516995, + "767747424/train2": 0.5082327332347631, + "77147264/train0": 0.013380475342273712, + "77147264/train1": 0.8879568874835968, + "77147264/train2": 0.9378202110528946, + "794181626/train0": 0.0, + "794181626/train1": 1.9626595228910446, + "794181626/train2": 0.0, + "957101489/train0": 0.3607921451330185, + "957101489/train1": 0.3404468297958374, + "957101489/train2": 0.38811931014060974, + "998761162/train0": 0.0010883957147598267, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.5834326290625793, + "low": 0.26088621819349617, + "mean": 0.45165208199755824, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.45165208199755824, + "per_task": { + "LiftPegUpright-v1": 0.5397342965006828, + "PickCube-v1": 0.4133657068014145, + "PullCube-v1": 0.1168446104449115, + "PushCube-v1": 0.6477842434094503, + "StackCube-v1": 0.5405315528313319 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4417064196763646, + "low": 0.2639209286670835, + "mean": 0.3500529956192077, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.298936170212766, + "low": 0.15638297872340423, + "mean": 0.2226950354609929, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.2226950354609929, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 0.0, + "1025740212/train2": 0.0, + "1055157354/train0": 0.0, + "1055157354/train1": 0.0, + "1055157354/train2": 0.0, + "1055567618/train0": 1.0, + "1055567618/train1": 0.0, + "1055567618/train2": 1.0, + "1113268940/train0": 0.0, + "1113268940/train1": 0.0, + "1113268940/train2": 0.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 0.0, + "1189392917/train1": 1.0, + "1189392917/train2": 0.0, + "1231780997/train0": 1.0, + "1231780997/train1": 0.0, + "1231780997/train2": 0.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 0.0, + "1273247072/train1": 0.0, + "1273247072/train2": 0.0, + "1358421325/train0": 0.0, + "1358421325/train1": 0.0, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 0.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 0.0, + "1522272686/train1": 0.0, + "1522272686/train2": 0.0, + "1528740417/train0": 0.0, + "1528740417/train1": 0.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.0, + "1549324805/train2": 0.0, + "1607773860/train0": 1.0, + "1607773860/train1": 0.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 0.0, + "1747178030/train1": 1.0, + "1747178030/train2": 0.0, + "1881223807/train0": 0.0, + "1881223807/train1": 1.0, + "1881223807/train2": 0.0, + "1919198188/train0": 0.0, + "1919198188/train1": 0.0, + "1919198188/train2": 0.0, + "1991414747/train0": 0.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.0, + "2015956957/train2": 0.2, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 0.0, + "211245842/train1": 1.0, + "211245842/train2": 0.0, + "2141802816/train0": 0.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 0.0, + "256770874/train1": 0.0, + "256770874/train2": 0.0, + "278008524/train0": 0.0, + "278008524/train1": 0.0, + "278008524/train2": 0.0, + "416183106/train0": 0.0, + "416183106/train1": 0.0, + "416183106/train2": 0.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.0, + "565369392/train1": 0.0, + "565369392/train2": 0.0, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.0, + "626486813/train1": 0.0, + "626486813/train2": 0.0, + "636680137/train0": 0.0, + "636680137/train1": 0.0, + "636680137/train2": 0.0, + "706479600/train0": 0.0, + "706479600/train1": 1.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 0.0, + "75070641/train1": 0.0, + "75070641/train2": 0.0, + "767747424/train0": 0.25, + "767747424/train1": 0.0, + "767747424/train2": 0.25, + "77147264/train0": 0.0, + "77147264/train1": 0.5, + "77147264/train2": 0.5, + "794181626/train0": 0.0, + "794181626/train1": 1.0, + "794181626/train2": 0.0, + "957101489/train0": 0.0, + "957101489/train1": 0.0, + "957101489/train2": 0.0, + "998761162/train0": 0.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3311111111111111, + "low": 0.07294117647058823, + "mean": 0.2075816993464052, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.2075816993464052, + "per_task": { + "LiftPegUpright-v1": 0.3888888888888889, + "PickCube-v1": 0.0, + "PullCube-v1": 0.049019607843137254, + "PushCube-v1": 0.3333333333333333, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.23671497584541062, + "low": 0.12560386473429952, + "mean": 0.178743961352657, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.43900709219858153, + "low": 0.28451536643026, + "mean": 0.3601654846335697, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.3601654846335697, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 1.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 0.0, + "1189392917/train1": 0.0, + "1189392917/train2": 0.0, + "1231780997/train0": 0.0, + "1231780997/train1": 0.0, + "1231780997/train2": 1.0, + "1271718007/train0": 0.0, + "1271718007/train1": 0.0, + "1271718007/train2": 0.0, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 1.0, + "1358421325/train1": 1.0, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 0.0, + "1475440973/train1": 1.0, + "1475440973/train2": 0.0, + "1483645336/train0": 0.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 0.0, + "1528740417/train1": 1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 0.0, + "1607773860/train1": 0.0, + "1607773860/train2": 0.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 1.0, + "1747178030/train1": 0.0, + "1747178030/train2": 0.0, + "1881223807/train0": 1.0, + "1881223807/train1": 0.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 1.0, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 0.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.0, + "2015956957/train1": 0.2, + "2015956957/train2": 0.0, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.0, + "211245842/train1": 0.0, + "211245842/train2": 0.0, + "2141802816/train0": 1.0, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.5, + "278008524/train1": 0.5, + "278008524/train2": 0.5, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 0.6666666666666666, + "565369392/train1": 0.6666666666666666, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 1.0, + "706479600/train1": 0.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 0.5, + "77147264/train1": 0.0, + "77147264/train2": 0.5, + "794181626/train0": 0.0, + "794181626/train1": 0.0, + "794181626/train2": 0.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7610256410256411, + "low": 0.22976370035193563, + "mean": 0.47932629462041226, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.47932629462041226, + "per_task": { + "LiftPegUpright-v1": 0.4444444444444444, + "PickCube-v1": 1.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.20512820512820512, + "StackCube-v1": 0.6 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.36231884057971014, + "low": 0.2318840579710145, + "mean": 0.2946859903381642, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6560283687943264, + "low": 0.5061465721040188, + "mean": 0.5793144208037826, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.5793144208037826, + "per_seed": { + "1025740212/train0": 1.0, + "1025740212/train1": 1.0, + "1025740212/train2": 1.0, + "1055157354/train0": 1.0, + "1055157354/train1": 1.0, + "1055157354/train2": 1.0, + "1055567618/train0": 1.0, + "1055567618/train1": 0.0, + "1055567618/train2": 1.0, + "1113268940/train0": 1.0, + "1113268940/train1": 1.0, + "1113268940/train2": 1.0, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.0, + "1176943004/train1": 1.0, + "1176943004/train2": 1.0, + "1189392917/train0": 0.0, + "1189392917/train1": 1.0, + "1189392917/train2": 0.0, + "1231780997/train0": 1.0, + "1231780997/train1": 0.0, + "1231780997/train2": 1.0, + "1271718007/train0": 0.5, + "1271718007/train1": 0.5, + "1271718007/train2": 0.5, + "1273247072/train0": 1.0, + "1273247072/train1": 1.0, + "1273247072/train2": 1.0, + "1358421325/train0": 1.0, + "1358421325/train1": 1.0, + "1358421325/train2": 0.0, + "1375492786/train0": 0.0, + "1375492786/train1": 0.0, + "1375492786/train2": 0.0, + "141984615/train0": 0.0, + "141984615/train1": 0.0, + "141984615/train2": 0.0, + "1475440973/train0": 1.0, + "1475440973/train1": 1.0, + "1475440973/train2": 1.0, + "1483645336/train0": 1.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.0, + "1522272686/train1": 1.0, + "1522272686/train2": 1.0, + "1528740417/train0": 0.0, + "1528740417/train1": 1.0, + "1528740417/train2": 0.0, + "1549324805/train0": 0.0, + "1549324805/train1": 0.3333333333333333, + "1549324805/train2": 0.3333333333333333, + "1607773860/train0": 1.0, + "1607773860/train1": 0.0, + "1607773860/train2": 1.0, + "1665739511/train0": 0.0, + "1665739511/train1": 1.0, + "1665739511/train2": 1.0, + "1747178030/train0": 1.0, + "1747178030/train1": 1.0, + "1747178030/train2": 0.0, + "1881223807/train0": 1.0, + "1881223807/train1": 1.0, + "1881223807/train2": 1.0, + "1919198188/train0": 1.0, + "1919198188/train1": 1.0, + "1919198188/train2": 1.0, + "1991414747/train0": 1.0, + "1991414747/train1": 1.0, + "1991414747/train2": 1.0, + "1997090716/train0": 1.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.2, + "2015956957/train1": 0.2, + "2015956957/train2": 0.2, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.0, + "211245842/train1": 1.0, + "211245842/train2": 0.0, + "2141802816/train0": 1.0, + "2141802816/train1": 1.0, + "2141802816/train2": 1.0, + "256770874/train0": 1.0, + "256770874/train1": 1.0, + "256770874/train2": 1.0, + "278008524/train0": 0.5, + "278008524/train1": 0.5, + "278008524/train2": 0.5, + "416183106/train0": 1.0, + "416183106/train1": 1.0, + "416183106/train2": 1.0, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.0, + "551504635/train1": 1.0, + "551504635/train2": 1.0, + "565369392/train0": 0.6666666666666666, + "565369392/train1": 0.6666666666666666, + "565369392/train2": 0.3333333333333333, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.0, + "585904130/train1": 0.0, + "585904130/train2": 0.0, + "626486813/train0": 0.25, + "626486813/train1": 0.25, + "626486813/train2": 0.25, + "636680137/train0": 1.0, + "636680137/train1": 1.0, + "636680137/train2": 1.0, + "706479600/train0": 1.0, + "706479600/train1": 1.0, + "706479600/train2": 0.0, + "750448296/train0": 0.0, + "750448296/train1": 0.0, + "750448296/train2": 0.0, + "75070641/train0": 1.0, + "75070641/train1": 1.0, + "75070641/train2": 1.0, + "767747424/train0": 0.0, + "767747424/train1": 0.0, + "767747424/train2": 0.0, + "77147264/train0": 0.5, + "77147264/train1": 0.5, + "77147264/train2": 1.0, + "794181626/train0": 0.0, + "794181626/train1": 1.0, + "794181626/train2": 0.0, + "957101489/train0": 1.0, + "957101489/train1": 1.0, + "957101489/train2": 1.0, + "998761162/train0": 1.0, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9133333333333333, + "low": 0.3936651583710407, + "mean": 0.6829864253393665, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.6829864253393665, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.8666666666666667 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5314009661835749, + "low": 0.3961352657004831, + "mean": 0.463768115942029, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7356242316190471, + "low": 0.4883750475817041, + "mean": 0.6119969386407603, + "n": 141, + "num_samples": 1000 + }, + "mean": 0.6119969386407603, + "per_seed": { + "1025740212/train0": 0.0, + "1025740212/train1": 1.7851315587759018, + "1025740212/train2": 1.2853291630744934, + "1055157354/train0": 1.4294073581695557, + "1055157354/train1": 1.187340497970581, + "1055157354/train2": 1.1348695755004883, + "1055567618/train0": 0.0, + "1055567618/train1": 0.0, + "1055567618/train2": 0.0, + "1113268940/train0": 1.4411447644233704, + "1113268940/train1": 1.2743229269981384, + "1113268940/train2": 1.3009825348854065, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 0.0, + "1176943004/train1": 0.0, + "1176943004/train2": 0.0, + "1189392917/train0": 0.0, + "1189392917/train1": 0.0, + "1189392917/train2": 0.0, + "1231780997/train0": 0.0, + "1231780997/train1": 0.0, + "1231780997/train2": 1.274539053440094, + "1271718007/train0": 0.0, + "1271718007/train1": 0.0, + "1271718007/train2": 0.0, + "1273247072/train0": 1.5340840816497803, + "1273247072/train1": 1.5356085896492004, + "1273247072/train2": 1.529144048690796, + "1358421325/train0": 1.1852855682373047, + "1358421325/train1": 1.175469994544983, + "1358421325/train2": 0.0, + "1375492786/train0": 0.24187534058000892, + "1375492786/train1": 0.2419701317558065, + "1375492786/train2": 0.2422413924941793, + "141984615/train0": 0.1358565071132034, + "141984615/train1": 0.12854979850817472, + "141984615/train2": 0.11378062601822118, + "1475440973/train0": 0.0, + "1475440973/train1": 1.2818688750267029, + "1475440973/train2": 0.0, + "1483645336/train0": 0.0, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.787919670343399, + "1522272686/train1": 1.7597034722566605, + "1522272686/train2": 1.7680195569992065, + "1528740417/train0": 0.0, + "1528740417/train1": 1.2744377255439758, + "1528740417/train2": 0.0, + "1549324805/train0": 0.05821319048603376, + "1549324805/train1": 0.7237351834774017, + "1549324805/train2": 0.7160144550725818, + "1607773860/train0": 0.0, + "1607773860/train1": 0.0, + "1607773860/train2": 0.0, + "1665739511/train0": 0.0, + "1665739511/train1": 0.0, + "1665739511/train2": 0.0, + "1747178030/train0": 1.8726215809583664, + "1747178030/train1": 0.0, + "1747178030/train2": 0.0, + "1881223807/train0": 1.9725016355514526, + "1881223807/train1": 0.0, + "1881223807/train2": 1.9597125202417374, + "1919198188/train0": 1.90082685649395, + "1919198188/train1": 1.7796131819486618, + "1919198188/train2": 1.9514013677835464, + "1991414747/train0": 1.8632079064846039, + "1991414747/train1": 0.0, + "1991414747/train2": 0.0, + "1997090716/train0": 0.0, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.06868033409118653, + "2015956957/train1": 0.46261803302913906, + "2015956957/train2": 0.07047177664935589, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.1864227652549744, + "211245842/train1": 0.0, + "211245842/train2": 0.0, + "2141802816/train0": 1.9140430986881256, + "2141802816/train1": 0.0, + "2141802816/train2": 0.0, + "256770874/train0": 1.425521969795227, + "256770874/train1": 1.1871315836906433, + "256770874/train2": 1.801473617553711, + "278008524/train0": 1.100301444530487, + "278008524/train1": 1.0945044284453616, + "278008524/train2": 1.0994171425700188, + "416183106/train0": 1.418215811252594, + "416183106/train1": 1.7750923037528992, + "416183106/train2": 1.7603600472211838, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 0.0, + "551504635/train1": 0.0, + "551504635/train2": 0.0, + "565369392/train0": 1.281934916973114, + "565369392/train1": 1.308447966972987, + "565369392/train2": 0.6395569890737534, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.2036442911873261, + "585904130/train1": 0.21776482245574394, + "585904130/train2": 0.2080739289522171, + "626486813/train0": 0.544636546401307, + "626486813/train1": 0.5429288633167744, + "626486813/train2": 0.5505410328041762, + "636680137/train0": 1.9755371417850256, + "636680137/train1": 1.9844186110422015, + "636680137/train2": 1.9635660238564014, + "706479600/train0": 1.78024622797966, + "706479600/train1": 0.0, + "706479600/train2": 0.0, + "750448296/train0": 0.12594309821724892, + "750448296/train1": 0.13385144993662834, + "750448296/train2": 0.15335527434945107, + "75070641/train0": 1.7716747671365738, + "75070641/train1": 1.7591263353824615, + "75070641/train2": 1.7821850180625916, + "767747424/train0": 0.04815509542822838, + "767747424/train1": 0.08559181983582675, + "767747424/train2": 0.05956761725246906, + "77147264/train0": 0.8867855072021484, + "77147264/train1": 0.0, + "77147264/train2": 0.88953747600317, + "794181626/train0": 0.0, + "794181626/train1": 0.0, + "794181626/train2": 0.0, + "957101489/train0": 1.4200972318649292, + "957101489/train1": 1.4201388359069824, + "957101489/train2": 1.43604177236557, + "998761162/train0": 1.907234638929367, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.1884870555665759, + "low": 0.4279394403967422, + "mean": 0.7702629553380855, + "n": 5, + "num_samples": 1000 + }, + "mean": 0.7702629553380855, + "per_task": { + "LiftPegUpright-v1": 0.6065420334537824, + "PickCube-v1": 1.5329455733299255, + "PullCube-v1": 0.36822861206361696, + "PushCube-v1": 0.39834897220134735, + "StackCube-v1": 0.945249585641755 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6440332006670958, + "low": 0.437744075585953, + "mean": 0.5369456667747744, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.1599391866837798, + "low": 0.9069358115994052, + "mean": 1.0335313443875687, + "n": 141, + "num_samples": 1000 + }, + "mean": 1.0335313443875687, + "per_seed": { + "1025740212/train0": 1.281606376171112, + "1025740212/train1": 1.7882490009069443, + "1025740212/train2": 1.3372135162353516, + "1055157354/train0": 1.879266306757927, + "1055157354/train1": 1.8382504880428314, + "1055157354/train2": 1.1348695755004883, + "1055567618/train0": 1.828233242034912, + "1055567618/train1": 0.0, + "1055567618/train2": 1.8424437642097473, + "1113268940/train0": 1.5597586333751678, + "1113268940/train1": 1.3375845551490784, + "1113268940/train2": 1.7854564934968948, + "1113371261/train0": 0.0, + "1113371261/train1": 0.0, + "1113371261/train2": 0.0, + "1176943004/train0": 1.9300042241811752, + "1176943004/train1": 1.953145906329155, + "1176943004/train2": 1.9587705433368683, + "1189392917/train0": 0.0, + "1189392917/train1": 1.9264162927865982, + "1189392917/train2": 0.0, + "1231780997/train0": 1.2770342230796814, + "1231780997/train1": 0.0, + "1231780997/train2": 1.274539053440094, + "1271718007/train0": 0.9082518517971039, + "1271718007/train1": 0.8952926397323608, + "1271718007/train2": 0.9200049638748169, + "1273247072/train0": 1.9542848244309425, + "1273247072/train1": 1.9624289870262146, + "1273247072/train2": 1.922220028936863, + "1358421325/train0": 1.820777028799057, + "1358421325/train1": 1.1834340691566467, + "1358421325/train2": 0.0, + "1375492786/train0": 0.24269693094538525, + "1375492786/train1": 0.2429703374800738, + "1375492786/train2": 0.24260535836219788, + "141984615/train0": 0.1932324218796566, + "141984615/train1": 0.16638785414397717, + "141984615/train2": 0.14744710053006807, + "1475440973/train0": 1.2853952050209045, + "1475440973/train1": 1.2818688750267029, + "1475440973/train2": 1.2959344387054443, + "1483645336/train0": 1.8899840712547302, + "1483645336/train1": 0.0, + "1483645336/train2": 0.0, + "1522272686/train0": 1.818911612033844, + "1522272686/train1": 1.7954162806272507, + "1522272686/train2": 1.8243235796689987, + "1528740417/train0": 0.0, + "1528740417/train1": 1.2828588485717773, + "1528740417/train2": 0.0, + "1549324805/train0": 0.05930363821486632, + "1549324805/train1": 0.7286523282527924, + "1549324805/train2": 0.7265020841732621, + "1607773860/train0": 1.8556141257286072, + "1607773860/train1": 0.0, + "1607773860/train2": 1.185863733291626, + "1665739511/train0": 0.0, + "1665739511/train1": 1.1815597414970398, + "1665739511/train2": 1.868122011423111, + "1747178030/train0": 1.8926412090659142, + "1747178030/train1": 1.9076288938522339, + "1747178030/train2": 0.0, + "1881223807/train0": 1.9748885482549667, + "1881223807/train1": 1.964881807565689, + "1881223807/train2": 1.9597125202417374, + "1919198188/train0": 1.922738566994667, + "1919198188/train1": 1.968167096376419, + "1919198188/train2": 1.9705085307359695, + "1991414747/train0": 1.8807305991649628, + "1991414747/train1": 1.9461991488933563, + "1991414747/train2": 1.9767832309007645, + "1997090716/train0": 1.9452667087316513, + "1997090716/train1": 0.0, + "1997090716/train2": 0.0, + "2015956957/train0": 0.4603504702448845, + "2015956957/train1": 0.4679855709895492, + "2015956957/train2": 0.5018953923135996, + "2054899126/train0": 0.0, + "2054899126/train1": 0.0, + "2054899126/train2": 0.0, + "211245842/train0": 1.1864227652549744, + "211245842/train1": 1.1816681623458862, + "211245842/train2": 0.0, + "2141802816/train0": 1.9224682599306107, + "2141802816/train1": 1.934090182185173, + "2141802816/train2": 1.954741045832634, + "256770874/train0": 1.8754567801952362, + "256770874/train1": 1.8000635504722595, + "256770874/train2": 1.880553051829338, + "278008524/train0": 1.1011691342573613, + "278008524/train1": 1.1019340505008586, + "278008524/train2": 1.1034439107170328, + "416183106/train0": 1.7636079341173172, + "416183106/train1": 1.7916338443756104, + "416183106/train2": 1.8061881065368652, + "534717141/train0": 0.0, + "534717141/train1": 0.0, + "534717141/train2": 0.0, + "551504635/train0": 1.2800043225288391, + "551504635/train1": 1.2779860496520996, + "551504635/train2": 1.287484347820282, + "565369392/train0": 1.3337247322003047, + "565369392/train1": 1.356453652183215, + "565369392/train2": 0.6973136613766352, + "565621826/train0": 0.0, + "565621826/train1": 0.0, + "565621826/train2": 0.0, + "585904130/train0": 0.22105526458472013, + "585904130/train1": 0.2222655958806475, + "585904130/train2": 0.2148742756495873, + "626486813/train0": 0.5671922296751291, + "626486813/train1": 0.571064023533836, + "626486813/train2": 0.5637975784484297, + "636680137/train0": 1.9803575277328491, + "636680137/train1": 1.9893615646287799, + "636680137/train2": 1.9918067259714007, + "706479600/train0": 1.925840973854065, + "706479600/train1": 1.185575544834137, + "706479600/train2": 0.0, + "750448296/train0": 0.14181637205183506, + "750448296/train1": 0.1780274137854576, + "750448296/train2": 0.18909010477364063, + "75070641/train0": 1.7748983949422836, + "75070641/train1": 1.766979619860649, + "75070641/train2": 1.8305092751979828, + "767747424/train0": 0.12046428816393018, + "767747424/train1": 0.1310162084409967, + "767747424/train2": 0.11785433255136013, + "77147264/train0": 0.9001659825444221, + "77147264/train1": 0.8879568874835968, + "77147264/train2": 1.8273576870560646, + "794181626/train0": 0.0, + "794181626/train1": 1.9626595228910446, + "794181626/train2": 0.0, + "957101489/train0": 1.7808893769979477, + "957101489/train1": 1.7605856657028198, + "957101489/train2": 1.8241610825061798, + "998761162/train0": 1.908323034644127, + "998761162/train1": 0.0, + "998761162/train2": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.674169638895581, + "low": 0.7759163594774237, + "mean": 1.2147448890541535, + "n": 5, + "num_samples": 1000 + }, + "mean": 1.2147448890541535, + "per_task": { + "LiftPegUpright-v1": 1.1462763299544652, + "PickCube-v1": 1.94631128013134, + "PullCube-v1": 0.44922248110107793, + "PushCube-v1": 1.0461332156107976, + "StackCube-v1": 1.4857811384730868 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.9811940932400909, + "low": 0.7554519398117963, + "mean": 0.8693330796714703, + "n": 207, + "num_samples": 1000 + }, + "n": 207 + } + }, + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381", + "train_seeds": [ + "0", + "1", + "2" + ] +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/metrics_by_seed.json b/runs/ctt_base_context_obj_val_rollout_comparison/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..77d86957efc60dc4988bf21a3a4d4cbd31a04b57 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/metrics_by_seed.json @@ -0,0 +1,3245 @@ +{ + "1025740212/train0": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 1.281606376171112, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.20050368294022264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.471626341342926, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7183936238288879, + "selected_utility_gain_over_base_at_8": 0.19001996517181396, + "selector_regret_at_8": 1.281606376171112, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.281606376171112 + }, + "1025740212/train1": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.0031174421310424805, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14400158741031796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2148684412240982, + "proposal_oracle_utility_gain_over_base_at_8": -0.31350521743297577, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21175099909305573, + "selected_utility_gain_over_base_at_8": -0.31662265956401825, + "selector_regret_at_8": 0.0031174421310424805, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7851315587759018, + "total_car_to_hidden_at_8": 1.7882490009069443 + }, + "1025740212/train2": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.051884353160858154, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12800533274597592, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7146708369255066, + "proposal_oracle_utility_gain_over_base_at_8": 0.18629717826843262, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6627864837646484, + "selected_utility_gain_over_base_at_8": 0.13441282510757446, + "selector_regret_at_8": 0.051884353160858154, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2853291630744934, + "total_car_to_hidden_at_8": 1.3372135162353516 + }, + "1055157354/train0": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.4498589485883713, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.43030250381223584, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5705926418304443, + "proposal_oracle_utility_gain_over_base_at_8": 0.37410856783390045, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12073369324207306, + "selected_utility_gain_over_base_at_8": -0.07575038075447083, + "selector_regret_at_8": 0.4498589485883713, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4294073581695557, + "total_car_to_hidden_at_8": 1.879266306757927 + }, + "1055157354/train1": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.6509099900722504, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32609963351937027, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.812659502029419, + "proposal_oracle_utility_gain_over_base_at_8": 0.6161754280328751, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.16174951195716858, + "selected_utility_gain_over_base_at_8": -0.034734562039375305, + "selector_regret_at_8": 0.6509099900722504, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.187340497970581, + "total_car_to_hidden_at_8": 1.8382504880428314 + }, + "1055157354/train2": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06066548841424423, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8651304244995117, + "proposal_oracle_utility_gain_over_base_at_8": 0.6686463505029678, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8651304244995117, + "selected_utility_gain_over_base_at_8": 0.6686463505029678, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1348695755004883, + "total_car_to_hidden_at_8": 1.1348695755004883 + }, + "1055567618/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.828233242034912, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30857397468613657, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1717667579650879, + "selected_utility_gain_over_base_at_8": -1.828233242034912, + "selector_regret_at_8": 1.828233242034912, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.828233242034912 + }, + "1055567618/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15812100047218913, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1055567618/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8424437642097473, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22991638132379041, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.15755623579025269, + "selected_utility_gain_over_base_at_8": -1.8424437642097473, + "selector_regret_at_8": 1.8424437642097473, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8424437642097473 + }, + "1113268940/train0": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.11861386895179749, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3774241061869338, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5588552355766296, + "proposal_oracle_utility_gain_over_base_at_8": 0.13654270768165588, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.44024136662483215, + "selected_utility_gain_over_base_at_8": 0.0179288387298584, + "selector_regret_at_8": 0.11861386895179749, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4411447644233704, + "total_car_to_hidden_at_8": 1.5597586333751678 + }, + "1113268940/train1": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.06326162815093994, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06911736513585261, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7256770730018616, + "proposal_oracle_utility_gain_over_base_at_8": 0.3033645451068878, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6624154448509216, + "selected_utility_gain_over_base_at_8": 0.24010291695594788, + "selector_regret_at_8": 0.06326162815093994, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2743229269981384, + "total_car_to_hidden_at_8": 1.3375845551490784 + }, + "1113268940/train2": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.48447395861148834, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2534904144040469, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.6990174651145935, + "proposal_oracle_utility_gain_over_base_at_8": 0.27670493721961975, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21454350650310516, + "selected_utility_gain_over_base_at_8": -0.2077690213918686, + "selector_regret_at_8": 0.48447395861148834, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.3009825348854065, + "total_car_to_hidden_at_8": 1.7854564934968948 + }, + "1113371261/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026017268580811406, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1113371261/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.015047211698777683, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1113371261/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13380940221299786, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004/train0": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9300042241811752, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1246594058194818, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06999577581882477, + "selected_utility_gain_over_base_at_8": -0.08461795747280121, + "selector_regret_at_8": 1.9300042241811752, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9300042241811752 + }, + "1176943004/train1": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.953145906329155, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15442455751472706, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04685409367084503, + "selected_utility_gain_over_base_at_8": -0.10775963962078094, + "selector_regret_at_8": 1.953145906329155, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.953145906329155 + }, + "1176943004/train2": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9587705433368683, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2664806376409974, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.041229456663131714, + "selected_utility_gain_over_base_at_8": -0.11338427662849426, + "selector_regret_at_8": 1.9587705433368683, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9587705433368683 + }, + "1189392917/train0": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1340247769860469, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1189392917/train1": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 1.9264162927865982, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10709427493066431, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0735837072134018, + "selected_utility_gain_over_base_at_8": -0.03483493626117706, + "selector_regret_at_8": 1.9264162927865982, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9264162927865982 + }, + "1189392917/train2": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33283923875771465, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997/train0": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 1.2770342230796814, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.19966589920839098, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7229657769203186, + "selected_utility_gain_over_base_at_8": -0.004040360450744629, + "selector_regret_at_8": 1.2770342230796814, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2770342230796814 + }, + "1231780997/train1": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07419277505792353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.2729938626289368, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997/train2": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16621555087386797, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.725460946559906, + "proposal_oracle_utility_gain_over_base_at_8": -0.0015451908111572266, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.725460946559906, + "selected_utility_gain_over_base_at_8": -0.0015451908111572266, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.274539053440094, + "total_car_to_hidden_at_8": 1.274539053440094 + }, + "1271718007/train0": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9082518517971039, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.17939225581112472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0917481482028961, + "selected_utility_gain_over_base_at_8": -0.32363298535346985, + "selector_regret_at_8": 0.9082518517971039, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9082518517971039 + }, + "1271718007/train1": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.8952926397323608, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10710899843183383, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1047073602676392, + "selected_utility_gain_over_base_at_8": -0.3106737732887268, + "selector_regret_at_8": 0.8952926397323608, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8952926397323608 + }, + "1271718007/train2": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9200049638748169, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.20722796191327264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.079995036125183, + "selected_utility_gain_over_base_at_8": -0.33538609743118286, + "selector_regret_at_8": 0.9200049638748169, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9200049638748169 + }, + "1273247072/train0": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + "1273247072/train1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + "1273247072/train2": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "total_car_to_hidden_at_8": 1.922220028936863 + }, + "1358421325/train0": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.6354914605617523, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.28863435648886215, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8147144317626953, + "proposal_oracle_utility_gain_over_base_at_8": -0.012500584125518799, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.179222971200943, + "selected_utility_gain_over_base_at_8": -0.6479920446872711, + "selector_regret_at_8": 0.6354914605617523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1852855682373047, + "total_car_to_hidden_at_8": 1.820777028799057 + }, + "1358421325/train1": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.007964074611663818, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.04701404433315851, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8245300054550171, + "proposal_oracle_utility_gain_over_base_at_8": -0.0026850104331970215, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8165659308433533, + "selected_utility_gain_over_base_at_8": -0.01064908504486084, + "selector_regret_at_8": 0.007964074611663818, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.175469994544983, + "total_car_to_hidden_at_8": 1.1834340691566467 + }, + "1358421325/train2": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09625642405798478, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1727849841117859, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1727849841117859, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1375492786/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008215903653763235, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0996910344508803, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0015756288776174188, + "proposal_oracle_utility_gain_over_base_at_8": -1.9984243711223826, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0007540385122410953, + "selected_utility_gain_over_base_at_8": -1.999245961487759, + "selector_regret_at_8": 0.0008215903653763235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24187534058000892, + "total_car_to_hidden_at_8": 0.24269693094538525 + }, + "1375492786/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.001000205724267289, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16573502466028908, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.001480837701819837, + "proposal_oracle_utility_gain_over_base_at_8": -1.9985191622981802, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.00048063197755254805, + "selected_utility_gain_over_base_at_8": -1.9995193680224475, + "selector_regret_at_8": 0.001000205724267289, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2419701317558065, + "total_car_to_hidden_at_8": 0.2429703374800738 + }, + "1375492786/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00036396586801856756, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.29573093343728263, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0012095769634470344, + "proposal_oracle_utility_gain_over_base_at_8": -1.998790423036553, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0008456110954284668, + "selected_utility_gain_over_base_at_8": -1.9991543889045715, + "selector_regret_at_8": 0.00036396586801856756, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2422413924941793, + "total_car_to_hidden_at_8": 0.24260535836219788 + }, + "141984615/train0": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.05737591476645321, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.297780428420543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06459133536554873, + "proposal_oracle_utility_gain_over_base_at_8": -0.029220803718393046, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.007215420599095523, + "selected_utility_gain_over_base_at_8": -0.08659671848484625, + "selector_regret_at_8": 0.05737591476645321, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1358565071132034, + "total_car_to_hidden_at_8": 0.1932324218796566 + }, + "141984615/train1": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03783805563580245, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.16666657870321888, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07189804397057742, + "proposal_oracle_utility_gain_over_base_at_8": -0.021914095113364358, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03405998833477497, + "selected_utility_gain_over_base_at_8": -0.05975215074916681, + "selector_regret_at_8": 0.03783805563580245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12854979850817472, + "total_car_to_hidden_at_8": 0.16638785414397717 + }, + "141984615/train2": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03539413997593025, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.22785399017277783, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08839488192461431, + "proposal_oracle_utility_gain_over_base_at_8": -0.005417257159327467, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05300074194868406, + "selected_utility_gain_over_base_at_8": -0.04081139713525772, + "selector_regret_at_8": 0.03539413997593025, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11378062601822118, + "total_car_to_hidden_at_8": 0.14744710053006807 + }, + "1475440973/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2853952050209045, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.41171052461838353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7146047949790955, + "selected_utility_gain_over_base_at_8": -1.2853952050209045, + "selector_regret_at_8": 1.2853952050209045, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2853952050209045 + }, + "1475440973/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0862090980213101, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7181311249732971, + "proposal_oracle_utility_gain_over_base_at_8": -1.2818688750267029, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7181311249732971, + "selected_utility_gain_over_base_at_8": -1.2818688750267029, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2818688750267029, + "total_car_to_hidden_at_8": 1.2818688750267029 + }, + "1475440973/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2959344387054443, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15809903475948942, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7040655612945557, + "selected_utility_gain_over_base_at_8": -1.2959344387054443, + "selector_regret_at_8": 1.2959344387054443, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2959344387054443 + }, + "1483645336/train0": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 1.8899840712547302, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35034389349653194, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11001592874526978, + "selected_utility_gain_over_base_at_8": 0.09359987080097198, + "selector_regret_at_8": 1.8899840712547302, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8899840712547302 + }, + "1483645336/train1": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06079377324000063, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1483645336/train2": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06905173445346233, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1522272686/train0": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.030991941690444946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.31643520896589616, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21208032965660095, + "proposal_oracle_utility_gain_over_base_at_8": 0.01363721489906311, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.181088387966156, + "selected_utility_gain_over_base_at_8": -0.017354726791381836, + "selector_regret_at_8": 0.030991941690444946, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.787919670343399, + "total_car_to_hidden_at_8": 1.818911612033844 + }, + "1522272686/train1": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.03571280837059021, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24182565394435904, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24029652774333954, + "proposal_oracle_utility_gain_over_base_at_8": 0.0418534129858017, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20458371937274933, + "selected_utility_gain_over_base_at_8": 0.006140604615211487, + "selector_regret_at_8": 0.03571280837059021, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7597034722566605, + "total_car_to_hidden_at_8": 1.7954162806272507 + }, + "1522272686/train2": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.056304022669792175, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21028164676312622, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23198044300079346, + "proposal_oracle_utility_gain_over_base_at_8": 0.033537328243255615, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.17567642033100128, + "selected_utility_gain_over_base_at_8": -0.02276669442653656, + "selector_regret_at_8": 0.056304022669792175, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7680195569992065, + "total_car_to_hidden_at_8": 1.8243235796689987 + }, + "1528740417/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15436827657008492, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1528740417/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.008421123027801514, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026961937751403514, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7255622744560242, + "proposal_oracle_utility_gain_over_base_at_8": -1.2744377255439758, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7171411514282227, + "selected_utility_gain_over_base_at_8": -1.2828588485717773, + "selector_regret_at_8": 0.008421123027801514, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2744377255439758, + "total_car_to_hidden_at_8": 1.2828588485717773 + }, + "1528740417/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3242760158848682, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1549324805/train0": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.0010904477288325627, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.23727288575803293, + "proposal_oracle_success_at_8": 0.6666666666666666, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.344537331412236, + "proposal_oracle_utility_gain_over_base_at_8": -0.03421011691292127, + "selected_success_at_8": 0.6666666666666666, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.3434468836834033, + "selected_utility_gain_over_base_at_8": -0.03530056464175383, + "selector_regret_at_8": 0.0010904477288325627, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05821319048603376, + "total_car_to_hidden_at_8": 0.05930363821486632 + }, + "1549324805/train1": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.004917144775390625, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10346856229255086, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6790153384208679, + "proposal_oracle_utility_gain_over_base_at_8": -0.6997321099042892, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6740981936454773, + "selected_utility_gain_over_base_at_8": -0.7046492546796799, + "selector_regret_at_8": 0.004917144775390625, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7237351834774017, + "total_car_to_hidden_at_8": 0.7286523282527924 + }, + "1549324805/train2": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.010487629100680351, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2818476718903797, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6867360668256879, + "proposal_oracle_utility_gain_over_base_at_8": -0.6920113814994693, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6762484377250075, + "selected_utility_gain_over_base_at_8": -0.7024990106001496, + "selector_regret_at_8": 0.010487629100680351, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7160144550725818, + "total_car_to_hidden_at_8": 0.7265020841732621 + }, + "1607773860/train0": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.8556141257286072, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3951037237909243, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14438587427139282, + "selected_utility_gain_over_base_at_8": -0.6882869601249695, + "selector_regret_at_8": 1.8556141257286072, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8556141257286072 + }, + "1607773860/train1": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2544258391757567, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1673271656036377, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1607773860/train2": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.185863733291626, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2143457671242473, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814136266708374, + "selected_utility_gain_over_base_at_8": -0.01853656768798828, + "selector_regret_at_8": 1.185863733291626, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185863733291626 + }, + "1665739511/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1665739511/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.1815597414970398, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.05717674118582694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.8184402585029602, + "selected_utility_gain_over_base_at_8": -1.1815597414970398, + "selector_regret_at_8": 1.1815597414970398, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1815597414970398 + }, + "1665739511/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.868122011423111, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.26263826459603723, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13187798857688904, + "selected_utility_gain_over_base_at_8": -1.868122011423111, + "selector_regret_at_8": 1.868122011423111, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.868122011423111 + }, + "1747178030/train0": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.02001962810754776, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11661420436902857, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1273784190416336, + "proposal_oracle_utility_gain_over_base_at_8": -0.019252806901931763, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10735879093408585, + "selected_utility_gain_over_base_at_8": -0.03927243500947952, + "selector_regret_at_8": 0.02001962810754776, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8726215809583664, + "total_car_to_hidden_at_8": 1.8926412090659142 + }, + "1747178030/train1": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 1.9076288938522339, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1055876219626688, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09237110614776611, + "selected_utility_gain_over_base_at_8": -0.054260119795799255, + "selector_regret_at_8": 1.9076288938522339, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9076288938522339 + }, + "1747178030/train2": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33194730961537694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8533687740564346, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1881223807/train0": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.002386912703514099, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20266639463366093, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.027498364448547363, + "proposal_oracle_utility_gain_over_base_at_8": -0.004528626799583435, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.025111451745033264, + "selected_utility_gain_over_base_at_8": -0.006915539503097534, + "selector_regret_at_8": 0.002386912703514099, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9725016355514526, + "total_car_to_hidden_at_8": 1.9748885482549667 + }, + "1881223807/train1": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 1.964881807565689, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10190418918720931, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9679730087518692, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03511819243431091, + "selected_utility_gain_over_base_at_8": 0.0030912011861801147, + "selector_regret_at_8": 1.964881807565689, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.964881807565689 + }, + "1881223807/train2": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1643392210528543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.040287479758262634, + "proposal_oracle_utility_gain_over_base_at_8": 0.008260488510131836, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.040287479758262634, + "selected_utility_gain_over_base_at_8": 0.008260488510131836, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9597125202417374, + "total_car_to_hidden_at_8": 1.9597125202417374 + }, + "1919198188/train0": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.021911710500717163, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09083822075370537, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09917314350605011, + "proposal_oracle_utility_gain_over_base_at_8": 0.06197427213191986, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07726143300533295, + "selected_utility_gain_over_base_at_8": 0.0400625616312027, + "selector_regret_at_8": 0.021911710500717163, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.90082685649395, + "total_car_to_hidden_at_8": 1.922738566994667 + }, + "1919198188/train1": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.18855391442775726, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2569381375337174, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2203868180513382, + "proposal_oracle_utility_gain_over_base_at_8": 0.18318794667720795, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03183290362358093, + "selected_utility_gain_over_base_at_8": -0.005365967750549316, + "selector_regret_at_8": 0.18855391442775726, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7796131819486618, + "total_car_to_hidden_at_8": 1.968167096376419 + }, + "1919198188/train2": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.019107162952423096, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21294576266807125, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04859863221645355, + "proposal_oracle_utility_gain_over_base_at_8": 0.011399760842323303, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029491469264030457, + "selected_utility_gain_over_base_at_8": -0.0077074021100997925, + "selector_regret_at_8": 0.019107162952423096, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9514013677835464, + "total_car_to_hidden_at_8": 1.9705085307359695 + }, + "1991414747/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.017522692680358887, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09717880195623664, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13679209351539612, + "proposal_oracle_utility_gain_over_base_at_8": -1.8632079064846039, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11926940083503723, + "selected_utility_gain_over_base_at_8": -1.8807305991649628, + "selector_regret_at_8": 0.017522692680358887, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8632079064846039, + "total_car_to_hidden_at_8": 1.8807305991649628 + }, + "1991414747/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9461991488933563, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.44645064197830664, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.05380085110664368, + "selected_utility_gain_over_base_at_8": -1.9461991488933563, + "selector_regret_at_8": 1.9461991488933563, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9461991488933563 + }, + "1991414747/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9767832309007645, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.613109405537996, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.023216769099235535, + "selected_utility_gain_over_base_at_8": -1.9767832309007645, + "selector_regret_at_8": 1.9767832309007645, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9767832309007645 + }, + "1997090716/train0": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 1.9452667087316513, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3641822422309461, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.054733291268348694, + "selected_utility_gain_over_base_at_8": -0.08454149961471558, + "selector_regret_at_8": 1.9452667087316513, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9452667087316513 + }, + "1997090716/train1": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3553584127845171, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1997090716/train2": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21318958083944717, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2015956957/train0": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.39167013615369795, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.22628515797280127, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.46120103299617765, + "proposal_oracle_utility_gain_over_base_at_8": 0.39408124312758447, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0695308968424797, + "selected_utility_gain_over_base_at_8": 0.0024111069738864898, + "selector_regret_at_8": 0.39167013615369795, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.06868033409118653, + "total_car_to_hidden_at_8": 0.4603504702448845 + }, + "2015956957/train1": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.005367537960410118, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.6, + "pairwise_causal_calibration_ece": 0.2284508985012672, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06726333405822515, + "proposal_oracle_utility_gain_over_base_at_8": 0.00014354418963193893, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06189579609781504, + "selected_utility_gain_over_base_at_8": -0.005223993770778179, + "selector_regret_at_8": 0.005367537960410118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.2, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.46261803302913906, + "total_car_to_hidden_at_8": 0.4679855709895492 + }, + "2015956957/train2": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.4314236156642437, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.24731300175116924, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.45940959043800833, + "proposal_oracle_utility_gain_over_base_at_8": 0.3922898005694151, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02798597477376461, + "selected_utility_gain_over_base_at_8": -0.039133815094828606, + "selector_regret_at_8": 0.4314236156642437, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.07047177664935589, + "total_car_to_hidden_at_8": 0.5018953923135996 + }, + "2054899126/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2054899126/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2054899126/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842/train0": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14834095177221712, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8135772347450256, + "proposal_oracle_utility_gain_over_base_at_8": -0.0252571702003479, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8135772347450256, + "selected_utility_gain_over_base_at_8": -0.0252571702003479, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1864227652549744, + "total_car_to_hidden_at_8": 1.1864227652549744 + }, + "211245842/train1": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 1.1816681623458862, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.036206191308278124, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8183318376541138, + "selected_utility_gain_over_base_at_8": -0.020502567291259766, + "selector_regret_at_8": 1.1816681623458862, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1816681623458862 + }, + "211245842/train2": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.08971682945849302, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1611655950546265, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2141802816/train0": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 0.008425161242485046, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07518554002658885, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08595690131187439, + "proposal_oracle_utility_gain_over_base_at_8": 0.01514965295791626, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07753174006938934, + "selected_utility_gain_over_base_at_8": 0.006724491715431213, + "selector_regret_at_8": 0.008425161242485046, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9140430986881256, + "total_car_to_hidden_at_8": 1.9224682599306107 + }, + "2141802816/train1": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.934090182185173, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.40989236974714055, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06590981781482697, + "selected_utility_gain_over_base_at_8": -0.0048974305391311646, + "selector_regret_at_8": 1.934090182185173, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.934090182185173 + }, + "2141802816/train2": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.954741045832634, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3183279052482729, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04525895416736603, + "selected_utility_gain_over_base_at_8": -0.025548294186592102, + "selector_regret_at_8": 1.954741045832634, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.954741045832634 + }, + "256770874/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.44993481040000916, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18757436508611464, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.574478030204773, + "proposal_oracle_utility_gain_over_base_at_8": -1.425521969795227, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1245432198047638, + "selected_utility_gain_over_base_at_8": -1.8754567801952362, + "selector_regret_at_8": 0.44993481040000916, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.425521969795227, + "total_car_to_hidden_at_8": 1.8754567801952362 + }, + "256770874/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.6129319667816162, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3193203145842822, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.8128684163093567, + "proposal_oracle_utility_gain_over_base_at_8": -1.1871315836906433, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.19993644952774048, + "selected_utility_gain_over_base_at_8": -1.8000635504722595, + "selector_regret_at_8": 0.6129319667816162, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1871315836906433, + "total_car_to_hidden_at_8": 1.8000635504722595 + }, + "256770874/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07907943427562714, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5133629211325229, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.19852638244628906, + "proposal_oracle_utility_gain_over_base_at_8": -1.801473617553711, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11944694817066193, + "selected_utility_gain_over_base_at_8": -1.880553051829338, + "selector_regret_at_8": 0.07907943427562714, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.801473617553711, + "total_car_to_hidden_at_8": 1.880553051829338 + }, + "278008524/train0": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0008676897268742323, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10616076660738837, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.013509750366210938, + "proposal_oracle_utility_gain_over_base_at_8": -0.09396211802959442, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.012642060639336705, + "selected_utility_gain_over_base_at_8": -0.09482980775646865, + "selector_regret_at_8": 0.0008676897268742323, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.100301444530487, + "total_car_to_hidden_at_8": 1.1011691342573613 + }, + "278008524/train1": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0074296220554970205, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10627711331165646, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.019306766451336443, + "proposal_oracle_utility_gain_over_base_at_8": -0.08816510194446892, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011877144395839423, + "selected_utility_gain_over_base_at_8": -0.09559472399996594, + "selector_regret_at_8": 0.0074296220554970205, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0945044284453616, + "total_car_to_hidden_at_8": 1.1019340505008586 + }, + "278008524/train2": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0040267681470140815, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.40905575479886586, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01439405232667923, + "proposal_oracle_utility_gain_over_base_at_8": -0.09307781606912613, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.010367284179665148, + "selected_utility_gain_over_base_at_8": -0.09710458421614021, + "selector_regret_at_8": 0.0040267681470140815, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0994171425700188, + "total_car_to_hidden_at_8": 1.1034439107170328 + }, + "416183106/train0": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.3453921228647232, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1517874168346938, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.581784188747406, + "proposal_oracle_utility_gain_over_base_at_8": 0.38145777583122253, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2363920658826828, + "selected_utility_gain_over_base_at_8": 0.03606565296649933, + "selector_regret_at_8": 0.3453921228647232, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.418215811252594, + "total_car_to_hidden_at_8": 1.7636079341173172 + }, + "416183106/train1": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.01654154062271118, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1925298309196861, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22490769624710083, + "proposal_oracle_utility_gain_over_base_at_8": 0.02458128333091736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20836615562438965, + "selected_utility_gain_over_base_at_8": 0.008039742708206177, + "selector_regret_at_8": 0.01654154062271118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7750923037528992, + "total_car_to_hidden_at_8": 1.7916338443756104 + }, + "416183106/train2": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.04582805931568146, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21826032886833752, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23963995277881622, + "proposal_oracle_utility_gain_over_base_at_8": 0.03931353986263275, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.19381189346313477, + "selected_utility_gain_over_base_at_8": -0.006514519453048706, + "selector_regret_at_8": 0.04582805931568146, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7603600472211838, + "total_car_to_hidden_at_8": 1.8061881065368652 + }, + "534717141/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18033001516380143, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "534717141/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.12736602361488303, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "534717141/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15703220108978472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635/train0": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2800043225288391, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3791036296516339, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7199956774711609, + "selected_utility_gain_over_base_at_8": -0.003976881504058838, + "selector_regret_at_8": 1.2800043225288391, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2800043225288391 + }, + "551504635/train1": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2779860496520996, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.23627606487156766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7220139503479004, + "selected_utility_gain_over_base_at_8": -0.001958608627319336, + "selector_regret_at_8": 1.2779860496520996, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2779860496520996 + }, + "551504635/train2": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.287484347820282, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4327797462092032, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.712515652179718, + "selected_utility_gain_over_base_at_8": -0.011456906795501709, + "selector_regret_at_8": 1.287484347820282, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.287484347820282 + }, + "565369392/train0": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.05178981522719065, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.21578265463075008, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.14469666282335916, + "proposal_oracle_utility_gain_over_base_at_8": -0.655068983634313, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.09290684759616852, + "selected_utility_gain_over_base_at_8": -0.7068587988615036, + "selector_regret_at_8": 0.05178981522719065, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.281934916973114, + "total_car_to_hidden_at_8": 1.3337247322003047 + }, + "565369392/train1": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.048005685210227966, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2859321646983037, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.11818361282348633, + "proposal_oracle_utility_gain_over_base_at_8": -0.6815820336341858, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.07017792761325836, + "selected_utility_gain_over_base_at_8": -0.7295877188444138, + "selector_regret_at_8": 0.048005685210227966, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.308447966972987, + "total_car_to_hidden_at_8": 1.356453652183215 + }, + "565369392/train2": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.057756672302881874, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2176430432872923, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7870745907227198, + "proposal_oracle_utility_gain_over_base_at_8": -0.012691055734952291, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.729317918419838, + "selected_utility_gain_over_base_at_8": -0.07044772803783417, + "selector_regret_at_8": 0.057756672302881874, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.6395569890737534, + "total_car_to_hidden_at_8": 0.6973136613766352 + }, + "565621826/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "565621826/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "565621826/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130/train0": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.01741097339739402, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3300621116998719, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.034002700820565224, + "proposal_oracle_utility_gain_over_base_at_8": -0.04128939720491568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.016591727423171203, + "selected_utility_gain_over_base_at_8": -0.058700370602309704, + "selector_regret_at_8": 0.01741097339739402, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2036442911873261, + "total_car_to_hidden_at_8": 0.22105526458472013 + }, + "585904130/train1": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.004500773424903552, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1652498530853035, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01988216955214739, + "proposal_oracle_utility_gain_over_base_at_8": -0.055409928473333515, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.015381396127243837, + "selected_utility_gain_over_base_at_8": -0.05991070189823707, + "selector_regret_at_8": 0.004500773424903552, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.21776482245574394, + "total_car_to_hidden_at_8": 0.2222655958806475 + }, + "585904130/train2": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.006800346697370212, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22254352429757418, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.029573063055674236, + "proposal_oracle_utility_gain_over_base_at_8": -0.04571903496980667, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.022772716358304024, + "selected_utility_gain_over_base_at_8": -0.05251938166717688, + "selector_regret_at_8": 0.006800346697370212, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2080739289522171, + "total_car_to_hidden_at_8": 0.2148742756495873 + }, + "626486813/train0": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.033813119400292635, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.24681469768825515, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.07908930000849068, + "proposal_oracle_utility_gain_over_base_at_8": -0.4263507166178897, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.04527618060819805, + "selected_utility_gain_over_base_at_8": -0.46016383601818234, + "selector_regret_at_8": 0.033813119400292635, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.544636546401307, + "total_car_to_hidden_at_8": 0.5671922296751291 + }, + "626486813/train1": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.02813516021706164, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20632809068391378, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06953954696655273, + "proposal_oracle_utility_gain_over_base_at_8": -0.43590046965982765, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.041404386749491096, + "selected_utility_gain_over_base_at_8": -0.4640356298768893, + "selector_regret_at_8": 0.02813516021706164, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5429288633167744, + "total_car_to_hidden_at_8": 0.571064023533836 + }, + "626486813/train2": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.013256545644253492, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.14946876705188256, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06192737747915089, + "proposal_oracle_utility_gain_over_base_at_8": -0.4435126391472295, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.0486708318348974, + "selected_utility_gain_over_base_at_8": -0.456769184791483, + "selector_regret_at_8": 0.013256545644253492, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5505410328041762, + "total_car_to_hidden_at_8": 0.5637975784484297 + }, + "636680137/train0": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0048203859478235245, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19331728269593437, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.024462858214974403, + "proposal_oracle_utility_gain_over_base_at_8": -1.9755371417850256, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.01964247226715088, + "selected_utility_gain_over_base_at_8": -1.9803575277328491, + "selector_regret_at_8": 0.0048203859478235245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755371417850256, + "total_car_to_hidden_at_8": 1.9803575277328491 + }, + "636680137/train1": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.004942953586578369, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.45862923715160336, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.015581388957798481, + "proposal_oracle_utility_gain_over_base_at_8": -1.9844186110422015, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.010638435371220112, + "selected_utility_gain_over_base_at_8": -1.9893615646287799, + "selector_regret_at_8": 0.004942953586578369, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9844186110422015, + "total_car_to_hidden_at_8": 1.9893615646287799 + }, + "636680137/train2": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.028240702114999294, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.6813618274184183, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.036433976143598557, + "proposal_oracle_utility_gain_over_base_at_8": -1.9635660238564014, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.008193274028599262, + "selected_utility_gain_over_base_at_8": -1.9918067259714007, + "selector_regret_at_8": 0.028240702114999294, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9635660238564014, + "total_car_to_hidden_at_8": 1.9918067259714007 + }, + "706479600/train0": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.1455947458744049, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2692917587355154, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21975377202033997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6153350174427032, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07415902614593506, + "selected_utility_gain_over_base_at_8": -0.7609297633171082, + "selector_regret_at_8": 0.1455947458744049, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.78024622797966, + "total_car_to_hidden_at_8": 1.925840973854065 + }, + "706479600/train1": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 1.185575544834137, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15129914778803955, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814424455165863, + "selected_utility_gain_over_base_at_8": -0.020664334297180176, + "selector_regret_at_8": 1.185575544834137, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185575544834137 + }, + "706479600/train2": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15321122852132601, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1649112105369568, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "750448296/train0": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.015873273834586143, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.3795450047628438, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.109864491969347, + "proposal_oracle_utility_gain_over_base_at_8": -0.0016690678894519806, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09399121813476086, + "selected_utility_gain_over_base_at_8": -0.017542341724038124, + "selector_regret_at_8": 0.015873273834586143, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12594309821724892, + "total_car_to_hidden_at_8": 0.14181637205183506 + }, + "750448296/train1": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.04417596384882927, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.1527117559178966, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10195614024996758, + "proposal_oracle_utility_gain_over_base_at_8": -0.009577419608831406, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.057780176401138306, + "selected_utility_gain_over_base_at_8": -0.053753383457660675, + "selector_regret_at_8": 0.04417596384882927, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13385144993662834, + "total_car_to_hidden_at_8": 0.1780274137854576 + }, + "750448296/train2": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.03573483042418957, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19031133205081463, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08245231583714485, + "proposal_oracle_utility_gain_over_base_at_8": -0.02908124402165413, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.046717485412955284, + "selected_utility_gain_over_base_at_8": -0.0648160744458437, + "selector_regret_at_8": 0.03573483042418957, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15335527434945107, + "total_car_to_hidden_at_8": 0.18909010477364063 + }, + "75070641/train0": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.003223627805709839, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.13808834192306887, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2283252328634262, + "proposal_oracle_utility_gain_over_base_at_8": 0.025573596358299255, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.22510160505771637, + "selected_utility_gain_over_base_at_8": 0.022349968552589417, + "selector_regret_at_8": 0.003223627805709839, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7716747671365738, + "total_car_to_hidden_at_8": 1.7748983949422836 + }, + "75070641/train1": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.007853284478187561, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.34224193319263285, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24087366461753845, + "proposal_oracle_utility_gain_over_base_at_8": 0.0381220281124115, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2330203801393509, + "selected_utility_gain_over_base_at_8": 0.030268743634223938, + "selector_regret_at_8": 0.007853284478187561, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7591263353824615, + "total_car_to_hidden_at_8": 1.766979619860649 + }, + "75070641/train2": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.048324257135391235, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2643509727736545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21781498193740845, + "proposal_oracle_utility_gain_over_base_at_8": 0.015063345432281494, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1694907248020172, + "selected_utility_gain_over_base_at_8": -0.03326091170310974, + "selector_regret_at_8": 0.048324257135391235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7821850180625916, + "total_car_to_hidden_at_8": 1.8305092751979828 + }, + "767747424/train0": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5240038954652846, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.2798082148668758, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5921752639114857, + "proposal_oracle_utility_gain_over_base_at_8": 0.04465107619762421, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.06817136844620109, + "selected_utility_gain_over_base_at_8": -0.4793528192676604, + "selector_regret_at_8": 0.5240038954652846, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.04815509542822838, + "total_car_to_hidden_at_8": 0.12046428816393018 + }, + "767747424/train1": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.04542438860516995, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.13255125751519692, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.10304383677430451, + "proposal_oracle_utility_gain_over_base_at_8": -0.44448035093955696, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.05761944816913456, + "selected_utility_gain_over_base_at_8": -0.4899047395447269, + "selector_regret_at_8": 0.04542438860516995, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.08559181983582675, + "total_car_to_hidden_at_8": 0.1310162084409967 + }, + "767747424/train2": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5082327332347631, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20625644377911734, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5790140572935343, + "proposal_oracle_utility_gain_over_base_at_8": 0.03148986957967281, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.07078132405877113, + "selected_utility_gain_over_base_at_8": -0.47674286365509033, + "selector_regret_at_8": 0.5082327332347631, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05956761725246906, + "total_car_to_hidden_at_8": 0.11785433255136013 + }, + "77147264/train0": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.013380475342273712, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22561784152146835, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.1132144927978516, + "proposal_oracle_utility_gain_over_base_at_8": -0.30836498737335205, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0998340174555779, + "selected_utility_gain_over_base_at_8": -0.32174546271562576, + "selector_regret_at_8": 0.013380475342273712, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.8867855072021484, + "total_car_to_hidden_at_8": 0.9001659825444221 + }, + "77147264/train1": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.8879568874835968, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.03379253116769787, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.5784205198287964, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1120431125164032, + "selected_utility_gain_over_base_at_8": -0.3095363676548004, + "selector_regret_at_8": 0.8879568874835968, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8879568874835968 + }, + "77147264/train2": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.9378202110528946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13662790487889595, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.11046252399683, + "proposal_oracle_utility_gain_over_base_at_8": -0.3111169561743736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.5, + "selected_utility_at_8": 0.1726423129439354, + "selected_utility_gain_over_base_at_8": -1.2489371672272682, + "selector_regret_at_8": 0.9378202110528946, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.88953747600317, + "total_car_to_hidden_at_8": 1.8273576870560646 + }, + "794181626/train0": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3405496231635778, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "794181626/train1": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 1.9626595228910446, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3619001242329895, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03734047710895538, + "selected_utility_gain_over_base_at_8": -0.14361557364463806, + "selector_regret_at_8": 1.9626595228910446, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9626595228910446 + }, + "794181626/train2": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1880089150142482, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "957101489/train0": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3607921451330185, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2493251178612577, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5799027681350708, + "proposal_oracle_utility_gain_over_base_at_8": 0.37618090212345123, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2191106230020523, + "selected_utility_gain_over_base_at_8": 0.01538875699043274, + "selector_regret_at_8": 0.3607921451330185, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4200972318649292, + "total_car_to_hidden_at_8": 1.7808893769979477 + }, + "957101489/train1": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3404468297958374, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2708447347116597, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5798611640930176, + "proposal_oracle_utility_gain_over_base_at_8": 0.376139298081398, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.23941433429718018, + "selected_utility_gain_over_base_at_8": 0.03569246828556061, + "selector_regret_at_8": 0.3404468297958374, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4201388359069824, + "total_car_to_hidden_at_8": 1.7605856657028198 + }, + "957101489/train2": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.38811931014060974, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5372965735602796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5639582276344299, + "proposal_oracle_utility_gain_over_base_at_8": 0.36023636162281036, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1758389174938202, + "selected_utility_gain_over_base_at_8": -0.027882948517799377, + "selector_regret_at_8": 0.38811931014060974, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.43604177236557, + "total_car_to_hidden_at_8": 1.8241610825061798 + }, + "998761162/train0": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0010883957147598267, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1991858618764328, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09276536107063293, + "proposal_oracle_utility_gain_over_base_at_8": 0.024948790669441223, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09167696535587311, + "selected_utility_gain_over_base_at_8": 0.023860394954681396, + "selector_regret_at_8": 0.0010883957147598267, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.907234638929367, + "total_car_to_hidden_at_8": 1.908323034644127 + }, + "998761162/train1": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24035198055524792, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "998761162/train2": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.14873333521535068, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + } +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/metrics_by_task.json b/runs/ctt_base_context_obj_val_rollout_comparison/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..e69284aae964c2bfef7c2a847db553b4ad3d9363 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.5397342965006828, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5555555555555556, + "pairwise_causal_calibration_ece": 0.2123556134611932, + "proposal_oracle_success_at_8": 0.5555555555555556, + "proposal_oracle_success_gain_over_base_at_8": 0.2222222222222222, + "proposal_oracle_utility_at_8": 1.3934579665462177, + "proposal_oracle_utility_gain_over_base_at_8": 0.3265138193964958, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8537236700455347, + "selected_utility_gain_over_base_at_8": -0.213220477104187, + "selector_regret_at_8": 0.5397342965006828, + "success_selector_gap_at_8": 0.3888888888888889, + "success_support_gap_at_8": 0.4444444444444444, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.6065420334537824, + "total_car_to_hidden_at_8": 1.1462763299544652 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.4133657068014145, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.17471460699627128, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46705442667007446, + "proposal_oracle_utility_gain_over_base_at_8": 0.313624769449234, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05368871986865997, + "selected_utility_gain_over_base_at_8": -0.09974093735218048, + "selector_regret_at_8": 0.4133657068014145, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5329455733299255, + "total_car_to_hidden_at_8": 1.94631128013134 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.1168446104449115, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.3627450980392157, + "pairwise_causal_calibration_ece": 0.21225649007113187, + "proposal_oracle_success_at_8": 0.19607843137254902, + "proposal_oracle_success_gain_over_base_at_8": -0.09803921568627451, + "proposal_oracle_utility_at_8": 0.44994396614321674, + "proposal_oracle_utility_gain_over_base_at_8": -0.20129550277830704, + "selected_success_at_8": 0.14705882352941177, + "selected_success_gain_over_base_at_8": -0.14705882352941177, + "selected_utility_at_8": 0.33309935569830523, + "selected_utility_gain_over_base_at_8": -0.31814011322321856, + "selector_regret_at_8": 0.1168446104449115, + "success_selector_gap_at_8": 0.049019607843137254, + "success_support_gap_at_8": 0.14705882352941177, + "success_total_car_to_hidden_at_8": 0.17647058823529413, + "support_gap_at_8": 0.36822861206361696, + "total_car_to_hidden_at_8": 0.44922248110107793 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.6477842434094503, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6153846153846154, + "pairwise_causal_calibration_ece": 0.19356765816347193, + "proposal_oracle_success_at_8": 0.7948717948717948, + "proposal_oracle_success_gain_over_base_at_8": 0.48717948717948717, + "proposal_oracle_utility_at_8": 1.6016510277986526, + "proposal_oracle_utility_gain_over_base_at_8": 0.9244865671946452, + "selected_success_at_8": 0.46153846153846156, + "selected_success_gain_over_base_at_8": 0.15384615384615385, + "selected_utility_at_8": 0.9538667843892024, + "selected_utility_gain_over_base_at_8": 0.276702323785195, + "selector_regret_at_8": 0.6477842434094503, + "success_selector_gap_at_8": 0.3333333333333333, + "success_support_gap_at_8": 0.20512820512820512, + "success_total_car_to_hidden_at_8": 0.5384615384615384, + "support_gap_at_8": 0.39834897220134735, + "total_car_to_hidden_at_8": 1.0461332156107976 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.5405315528313319, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.2253301008861889, + "proposal_oracle_success_at_8": 0.4, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 1.054750414358245, + "proposal_oracle_utility_gain_over_base_at_8": 0.25163986451096004, + "selected_success_at_8": 0.13333333333333333, + "selected_success_gain_over_base_at_8": -0.06666666666666667, + "selected_utility_at_8": 0.5142188615269131, + "selected_utility_gain_over_base_at_8": -0.2888916883203718, + "selector_regret_at_8": 0.5405315528313319, + "success_selector_gap_at_8": 0.26666666666666666, + "success_support_gap_at_8": 0.6, + "success_total_car_to_hidden_at_8": 0.8666666666666667, + "support_gap_at_8": 0.945249585641755, + "total_car_to_hidden_at_8": 1.4857811384730868 + } +} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/report.md b/runs/ctt_base_context_obj_val_rollout_comparison/report.md new file mode 100644 index 0000000000000000000000000000000000000000..d42812fe59980b3a3d788d5de2d49bcde08c59e4 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/report.md @@ -0,0 +1,51 @@ +# CTT Validation Measured Rollout Comparison + +Runs: `3` +Rows: `207` +K: `8` + +| Metric | N | Micro mean | 95% CI | +| --- | ---: | ---: | ---: | +| base_success | 207 | 0.2754 | [0.2174, 0.3382] | +| base_utility | 207 | 0.7181 | [0.6102, 0.8321] | +| branch_car_at_8 | 207 | 0.3501 | [0.2639, 0.4417] | +| hidden_chart_oracle_success_at_8 | 207 | 0.6667 | [0.5990, 0.7246] | +| hidden_chart_oracle_utility_at_8 | 207 | 1.4000 | [1.2800, 1.5063] | +| outcome_ptr_at_8 | 207 | 0.5024 | [0.4300, 0.5652] | +| pairwise_causal_calibration_ece | 207 | 0.2110 | [0.1939, 0.2290] | +| proposal_oracle_success_at_8 | 207 | 0.3816 | [0.3188, 0.4444] | +| proposal_oracle_success_gain_over_base_at_8 | 207 | 0.1063 | [0.0386, 0.1836] | +| proposal_oracle_utility_at_8 | 207 | 0.8807 | [0.7637, 0.9970] | +| proposal_oracle_utility_gain_over_base_at_8 | 207 | 0.1626 | [0.0470, 0.2899] | +| selected_success_at_8 | 207 | 0.2029 | [0.1498, 0.2560] | +| selected_success_gain_over_base_at_8 | 207 | -0.0725 | [-0.1353, -0.0048] | +| selected_utility_at_8 | 207 | 0.5307 | [0.4294, 0.6390] | +| selected_utility_gain_over_base_at_8 | 207 | -0.1874 | [-0.2967, -0.0708] | +| selector_regret_at_8 | 207 | 0.3501 | [0.2639, 0.4417] | +| success_selector_gap_at_8 | 207 | 0.1787 | [0.1256, 0.2367] | +| success_support_gap_at_8 | 207 | 0.2947 | [0.2319, 0.3623] | +| success_total_car_to_hidden_at_8 | 207 | 0.4638 | [0.3961, 0.5314] | +| support_gap_at_8 | 207 | 0.5369 | [0.4377, 0.6440] | +| total_car_to_hidden_at_8 | 207 | 0.8693 | [0.7555, 0.9812] | + +| Success/Utility | Mean | +| --- | ---: | +| base_success_rate | 0.2754 | +| selected_success_rate | 0.2029 | +| proposal_oracle_success_rate | 0.3816 | +| hidden_chart_oracle_success_rate | 0.6667 | +| selected_success_gain_over_base | -0.0725 | +| proposal_oracle_success_gain_over_base | 0.1063 | +| success_support_gap | 0.2947 | +| success_selector_gap | 0.1787 | +| base_utility_mean | 0.7181 | +| selected_utility_mean | 0.5307 | +| proposal_oracle_utility_mean | 0.8807 | +| hidden_chart_oracle_utility_mean | 1.4000 | + +These are measured generated-candidate rollouts, not PPTC proxies. + +Run dirs: +- `runs/ctt_residual_base_context_obj_rollout_val69_seed0` +- `runs/ctt_residual_base_context_obj_rollout_val69_seed1` +- `runs/ctt_residual_base_context_obj_rollout_val69_seed2` diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/source_data_hash.txt b/runs/ctt_base_context_obj_val_rollout_comparison/source_data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..1ef49748c0932e9ccb1c83a065667e7a4032f7db --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/source_data_hash.txt @@ -0,0 +1 @@ +67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42 diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/split_hash.txt b/runs/ctt_base_context_obj_val_rollout_comparison/split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8533e80f235606cef4ba0119a3ac3806d63a0983 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/split_hash.txt @@ -0,0 +1 @@ +9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381 diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/table.tex b/runs/ctt_base_context_obj_val_rollout_comparison/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..89cdf5a6446bd9583423d7dcc4dd5c2869ffb75c --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 207 & 0.2754 & 0.2174 & 0.3382 \\ +base\_utility & 207 & 0.7181 & 0.6102 & 0.8321 \\ +branch\_car\_at\_8 & 207 & 0.3501 & 0.2639 & 0.4417 \\ +hidden\_chart\_oracle\_success\_at\_8 & 207 & 0.6667 & 0.5990 & 0.7246 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 207 & 1.4000 & 1.2800 & 1.5063 \\ +outcome\_ptr\_at\_8 & 207 & 0.5024 & 0.4300 & 0.5652 \\ +pairwise\_causal\_calibration\_ece & 207 & 0.2110 & 0.1939 & 0.2290 \\ +proposal\_oracle\_success\_at\_8 & 207 & 0.3816 & 0.3188 & 0.4444 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 207 & 0.1063 & 0.0386 & 0.1836 \\ +proposal\_oracle\_utility\_at\_8 & 207 & 0.8807 & 0.7637 & 0.9970 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 207 & 0.1626 & 0.0470 & 0.2899 \\ +selected\_success\_at\_8 & 207 & 0.2029 & 0.1498 & 0.2560 \\ +selected\_success\_gain\_over\_base\_at\_8 & 207 & -0.0725 & -0.1353 & -0.0048 \\ +selected\_utility\_at\_8 & 207 & 0.5307 & 0.4294 & 0.6390 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 207 & -0.1874 & -0.2967 & -0.0708 \\ +selector\_regret\_at\_8 & 207 & 0.3501 & 0.2639 & 0.4417 \\ +success\_selector\_gap\_at\_8 & 207 & 0.1787 & 0.1256 & 0.2367 \\ +success\_support\_gap\_at\_8 & 207 & 0.2947 & 0.2319 & 0.3623 \\ +success\_total\_car\_to\_hidden\_at\_8 & 207 & 0.4638 & 0.3961 & 0.5314 \\ +support\_gap\_at\_8 & 207 & 0.5369 & 0.4377 & 0.6440 \\ +total\_car\_to\_hidden\_at\_8 & 207 & 0.8693 & 0.7555 & 0.9812 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_base_context_obj_val_rollout_comparison/train.log b/runs/ctt_base_context_obj_val_rollout_comparison/train.log new file mode 100644 index 0000000000000000000000000000000000000000..518e1c04f4cca9e59f377237f5a12c0cf039c704 --- /dev/null +++ b/runs/ctt_base_context_obj_val_rollout_comparison/train.log @@ -0,0 +1 @@ +comparison artifact; source runs trained/evaluated separately diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/command.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/command.txt new file mode 100644 index 0000000000000000000000000000000000000000..d940c739bd2dee84946a73e55be24904d2191065 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/command.txt @@ -0,0 +1 @@ +python scripts/eval_ctt_generated_rollout.py --checkpoint runs/ctt_residual_base_context_obj_seed0/model.pt --source-index data/cil_charts_rgb_refs/train/index.json --target-index data/cil_charts_rgb_refs/val/index.json --out-dir runs/ctt_residual_base_context_obj_rollout_val69_seed0 --k 8 --pool-size 0 --neighbors 8 --max-target-charts 69 --group-batch-size 1 --device cpu --sim-backend physx_cpu --render-backend cpu --restore-tolerance 1e-5 --delta-scale 1.0 --bootstrap-samples 200 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/config.yaml b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/config.yaml new file mode 100644 index 0000000000000000000000000000000000000000..12000ec6f43f9b6d7a60fc04d71e546e2160db20 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/config.yaml @@ -0,0 +1,18 @@ +bootstrap_samples: 200 +checkpoint: runs/ctt_residual_base_context_obj_seed0/model.pt +delta_scale: 1.0 +device: cpu +exclude_self_source: False +group_batch_size: 1 +include_targets_without_positives: False +k: 8 +max_target_charts: 69 +neighbors: 8 +out_dir: runs/ctt_residual_base_context_obj_rollout_val69_seed0 +pool_size: 0 +render_backend: cpu +restore_tolerance: 1e-05 +sim_backend: physx_cpu +skip_metrics: False +source_index: data/cil_charts_rgb_refs/train/index.json +target_index: data/cil_charts_rgb_refs/val/index.json diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/data_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..49b12b50537d95327f9c62ddd41910531a6c14a4 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/data_hash.txt @@ -0,0 +1 @@ +1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/eval.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/eval.log new file mode 100644 index 0000000000000000000000000000000000000000..b6ac7d222a5ee4b675092aa04f64c0b12161e33d --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/eval.log @@ -0,0 +1,766 @@ +start +importing gymnasium/mani_skill +imported gymnasium/mani_skill +loaded checkpoint=runs/ctt_residual_base_context_obj_seed0/model.pt +loaded charts source=301 target=69 +generated proposals rows=69 total=552 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=PickCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PickCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +rollout complete rows=69 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/git_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/git_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8b137891791fe96927ad78e64b0aad7bded08bdc --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/git_hash.txt @@ -0,0 +1 @@ + diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_candidates.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_candidates.json new file mode 100644 index 0000000000000000000000000000000000000000..573a957ea316fb7cf366512464defb5306db053e --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_candidates.json @@ -0,0 +1,44442 @@ +{ + "candidates_evaluated": true, + "checkpoint": "runs/ctt_residual_base_context_obj_seed0/model.pt", + "decoder": { + "delta_scale": 1.0, + "lossless": false, + "name": "linear_keyframe_decode", + "source_code": "spline_tangent_code stores start/mid/end residual keyframes" + }, + "exclude_self_source": false, + "k": 8, + "neighbors": 8, + "pool_size": 8, + "report_type": "ctt_generated_measured_rollout", + "rows": [ + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7270061373710632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7270061373710632 + }, + "base_progress": 0.7270061373710632, + "base_success": false, + "base_utility": 0.7270061373710632, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7229657769203186, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7229657769203186 + }, + { + "contact_quality": null, + "progress": 0.7291424870491028, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7291424870491028 + }, + { + "contact_quality": null, + "progress": 0.7071980834007263, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7071980834007263 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7273857593536377, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7273857593536377 + }, + { + "contact_quality": null, + "progress": 0.7259960770606995, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7259960770606995 + }, + { + "contact_quality": null, + "progress": 0.727085530757904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.727085530757904 + }, + { + "contact_quality": null, + "progress": 0.7209001183509827, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7209001183509827 + } + ], + "candidate_progress": [ + 0.7229657769203186, + 0.7291424870491028, + 0.7071980834007263, + 1.0, + 0.7273857593536377, + 0.7259960770606995, + 0.727085530757904, + 0.7209001183509827 + ], + "candidate_source_chart_ids": [ + "ms-efbde8652e75274fbd69", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-df961fe24a43e56d1716", + "ms-9ced92cccacfe4d157b0", + "ms-efbde8652e75274fbd69", + "ms-dd32ce9859c365c535b4", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-484ea653bbca2de3a4a2", + "generated_tangents": [ + [ + -0.07231272757053375, + -0.05361426621675491, + 0.20084986090660095, + 0.2549743354320526, + 1.2756636142730713, + 0.26493221521377563, + -0.2770671248435974, + 0.1440885066986084, + -0.12119890749454498, + 0.18552835285663605, + 0.1064738929271698, + 0.9770814776420593, + 0.11324295401573181, + 0.2581104040145874, + 0.37570127844810486, + 0.14162081480026245, + 0.015085473656654358, + -0.25780385732650757, + 1.0848740339279175, + -0.438931405544281, + -0.16297724843025208 + ], + [ + -0.058384791016578674, + -0.007196493446826935, + 0.2863960862159729, + 0.4616982042789459, + 0.8279897570610046, + 0.026855826377868652, + -0.2453579306602478, + 0.108298659324646, + -0.010689212009310722, + 0.24004188179969788, + -0.11884095519781113, + 1.309671401977539, + 0.21858593821525574, + 0.2140667587518692, + 0.10542087256908417, + 0.04384743049740791, + -0.2249954491853714, + 0.033864542841911316, + 1.0120162963867188, + -0.18448886275291443, + -0.001940220594406128 + ], + [ + 0.0398905873298645, + 0.03704178333282471, + 0.46594569087028503, + 0.5894558429718018, + 0.9248438477516174, + 0.15800584852695465, + -0.2163742184638977, + -0.031178757548332214, + -0.059490740299224854, + 0.3896148204803467, + 0.03863520175218582, + 1.3416554927825928, + 0.23786357045173645, + 0.2686057686805725, + 0.308365136384964, + 0.09169572591781616, + -0.10002933442592621, + -0.12801562249660492, + 1.0708489418029785, + -0.11242121458053589, + -0.06980758905410767 + ], + [ + 0.04414123296737671, + 0.05196486413478851, + 0.31238871812820435, + 0.5328384637832642, + 1.097949504852295, + -0.24277085065841675, + -0.4461241364479065, + 0.03629066050052643, + 0.051987677812576294, + 0.116251140832901, + -0.11456320434808731, + 1.1187355518341064, + 0.14071431756019592, + -0.06319041550159454, + 0.18655209243297577, + 0.11288036406040192, + -0.1382099837064743, + -0.07332096993923187, + 1.2357680797576904, + -0.11672963201999664, + -0.23744389414787292 + ], + [ + -0.04112638533115387, + 0.045152448117733, + 0.25177115201950073, + 0.41037455201148987, + 0.9143605828285217, + 0.062146469950675964, + -0.29822736978530884, + 0.003190666437149048, + -0.017644867300987244, + 0.17557278275489807, + 0.030187007039785385, + 1.3407450914382935, + 0.12058094143867493, + 0.18465670943260193, + 0.14753586053848267, + 0.02446366474032402, + -0.15492106974124908, + 0.05071836709976196, + 1.0075147151947021, + -0.0926724225282669, + -0.005339428782463074 + ], + [ + -0.07725875079631805, + -0.03352033719420433, + 0.3147411346435547, + 0.3762994110584259, + 0.9385537505149841, + 0.1675524264574051, + -0.6023285984992981, + -0.0003843158483505249, + -0.04391191899776459, + 0.2910858988761902, + 0.17489781975746155, + 1.2902905941009521, + 0.03016170859336853, + 0.02391386777162552, + 0.19830992817878723, + 0.04218254238367081, + -0.17453138530254364, + 0.14636392891407013, + 1.2725138664245605, + -0.411970317363739, + 0.10102862864732742 + ], + [ + 0.0015261024236679077, + 0.04789748787879944, + 0.2689938545227051, + 0.38422873616218567, + 0.9711422324180603, + 0.04219160974025726, + -0.2555607557296753, + -0.0010332614183425903, + -0.018974101170897484, + 0.21855208277702332, + 0.002623647451400757, + 1.3563590049743652, + 0.016860365867614746, + 0.10275750607252121, + 0.2021571546792984, + 0.05061839148402214, + -0.09986038506031036, + 0.09534766525030136, + 1.0070284605026245, + -0.07725197076797485, + -0.054655373096466064 + ], + [ + -0.19963373243808746, + 0.025442376732826233, + 0.1647450029850006, + 0.462211936712265, + 0.8526129126548767, + -0.09105783700942993, + -0.21945291757583618, + 0.11289386451244354, + -0.03213582932949066, + 0.1903569996356964, + -0.06764494627714157, + 1.3399022817611694, + -0.006895154714584351, + 0.030383944511413574, + 0.21221129596233368, + 0.08718833327293396, + -0.1539117842912674, + 0.04677437245845795, + 0.9803423881530762, + -0.09331116080284119, + 0.07617859542369843 + ] + ], + "generated_utilities": [ + 0.7229657769203186, + 0.7291424870491028, + 0.7071980834007263, + 2.0, + 0.7273857593536377, + 0.7259960770606995, + 0.727085530757904, + 0.7209001183509827 + ], + "group_id": "ms-484ea653bbca2de3a4a2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7242962718009949, + 0.7209752798080444, + 2.0, + 0.19684189558029175, + 0.21780464053153992, + 0.19326990842819214, + 0.22509369254112244, + 0.20856516063213348, + 0.21041584014892578, + 0.19898515939712524, + 0.20721960067749023, + 0.20830264687538147, + 0.19554394483566284, + 0.22549311816692352, + 0.19481703639030457, + 0.2172507494688034 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.4380870461463928, + 0.4186934530735016, + 0.6294705867767334, + -0.08003801107406616, + 0.5882754325866699, + 0.006411761045455933, + 0.0, + 0.27439412474632263, + -0.43702489137649536, + -0.26345914602279663, + -0.3102925717830658, + 1.1419475078582764, + -0.05133828520774841, + 0.0, + 0.11964668333530426, + -0.04283810034394264, + 0.021698474884033203, + -0.3213534653186798, + 1.444929838180542, + 0.058529436588287354, + 0.0 + ], + [ + 0.18277202546596527, + -0.31313952803611755, + 1.5374140739440918, + 0.9199619889259338, + 1.58827543258667, + -0.42588773369789124, + -1.307411789894104, + -0.1648276150226593, + 0.17999683320522308, + 0.7365408539772034, + -0.2825929522514343, + 2.1419475078582764, + -0.44205278158187866, + 0.3557547330856323, + 0.11787930130958557, + 0.04347073659300804, + 0.5306029915809631, + -0.6201959848403931, + 2.444929838180542, + -0.7078790068626404, + 0.7410526275634766 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.614823579788208, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7115094661712646, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4821052551269531 + ], + [ + 0.2415536344051361, + -0.2574644386768341, + 1.3600168228149414, + 0.8914169669151306, + 1.311052918434143, + -0.13483268022537231, + -1.4109176397323608, + -0.007070749998092651, + 0.30199557542800903, + 0.867316484451294, + -0.0111103355884552, + 2.228789806365967, + -0.26305240392684937, + 0.4046628475189209, + 0.25534987449645996, + 0.29946526885032654, + 0.777583122253418, + -0.535997748374939, + 2.2955121994018555, + -0.6670190095901489, + 0.8460665941238403 + ], + [ + -0.212661013007164, + -0.6653347015380859, + 1.8750245571136475, + 0.7373600006103516, + 1.2695744037628174, + -0.7584145069122314, + -1.695502758026123, + 0.05721297860145569, + -0.04310476779937744, + 0.4696756601333618, + -0.2954420745372772, + 1.9897286891937256, + -0.6554491519927979, + 0.5805841088294983, + 0.17026324570178986, + 0.3433569669723511, + 0.3734014630317688, + -0.8102566003799438, + 2.792973279953003, + -0.8463170528411865, + 1.0759955644607544 + ], + [ + -0.039009034633636475, + -0.7773740291595459, + 1.064732313156128, + 0.712108850479126, + 1.270837426185608, + -0.5491334199905396, + -1.7919384241104126, + -0.5782418251037598, + 0.4058297872543335, + 0.5231752395629883, + -0.521438479423523, + 2.2066850662231445, + -0.5258626341819763, + -0.09100466966629028, + 0.24944378435611725, + -0.2271452248096466, + 0.9009795784950256, + -0.8140718340873718, + 2.6494152545928955, + -0.5153427124023438, + 0.671539306640625 + ], + [ + -0.3804699182510376, + -0.5808351039886475, + 1.8389031887054443, + 0.7141245603561401, + 1.0997881889343262, + -0.1678517758846283, + -1.884540319442749, + -0.3411717414855957, + 0.4830858111381531, + 0.8356180787086487, + -0.8238832950592041, + 1.743321418762207, + -0.3751307427883148, + 0.5275082588195801, + -0.08622422814369202, + 0.23666565120220184, + 0.802552342414856, + -0.8389871120452881, + 1.9817423820495605, + -0.4874984323978424, + 0.4390762150287628 + ], + [ + 0.4497724771499634, + -0.7880887389183044, + 1.4436085224151611, + 1.083756446838379, + 2.2298290729522705, + -0.03008332848548889, + -1.0250177383422852, + -0.6266056299209595, + 0.3751387298107147, + 1.1566712856292725, + -0.6914000511169434, + 1.953734278678894, + 0.08129626512527466, + 0.9812837839126587, + -0.4981238543987274, + 0.6697847247123718, + -0.05150604248046875, + -0.06941354274749756, + 1.9621039628982544, + -1.345557451248169, + 0.43397778272628784 + ], + [ + 0.03950639069080353, + -1.037135124206543, + 1.1041697263717651, + 1.109439730644226, + 1.144017219543457, + -0.07355684041976929, + -0.8942177295684814, + -0.16419151425361633, + -0.12125559151172638, + 0.36935141682624817, + -0.7298048734664917, + 1.4954023361206055, + -0.018926143646240234, + 0.4787553548812866, + -0.3127076327800751, + 0.6351315379142761, + 0.5993886590003967, + -0.5143972635269165, + 2.429173469543457, + -0.6674217581748962, + 0.5048977732658386 + ], + [ + 0.9763806462287903, + -1.1549197435379028, + 2.0378684997558594, + 1.2418556213378906, + 2.427785634994507, + -1.1326193809509277, + -1.5491721630096436, + -0.8937685489654541, + 0.2583051919937134, + -0.10687285661697388, + 0.2628750801086426, + 1.6517728567123413, + -0.21639958024024963, + 0.7581590414047241, + -0.29044294357299805, + -0.08927425742149353, + 1.3009071350097656, + -0.9239004850387573, + 2.5070056915283203, + -1.3246500492095947, + 0.24751555919647217 + ], + [ + 0.9781984090805054, + -0.13759100437164307, + 1.8698725700378418, + 1.4942535161972046, + 2.5587658882141113, + -1.1335766315460205, + -0.6910277605056763, + -0.1434302181005478, + 0.4021325707435608, + -0.12275522947311401, + 0.6571680307388306, + 1.9964357614517212, + -0.1442476212978363, + 1.3201656341552734, + 0.2586786448955536, + -0.38222962617874146, + 0.9950553774833679, + -0.8371140956878662, + 1.8977141380310059, + -0.795723021030426, + 1.505234956741333 + ], + [ + -0.6250126361846924, + 0.4231366813182831, + 1.701646089553833, + 1.6233667135238647, + 1.0861127376556396, + -0.3387698233127594, + -0.9825164079666138, + -1.1511213779449463, + 0.11523068696260452, + 1.5823121070861816, + 0.5840664505958557, + 2.145885705947876, + -1.2145874500274658, + 1.1558246612548828, + -0.5131542682647705, + -0.4715161919593811, + 1.0824741125106812, + -0.7667124271392822, + 1.8457963466644287, + -1.228658676147461, + 1.2112116813659668 + ], + [ + 1.1640653610229492, + -1.1697030067443848, + 1.6035369634628296, + 0.2161661982536316, + 1.4819366931915283, + 0.2569611370563507, + -1.3529595136642456, + 0.41208329796791077, + 0.03121529519557953, + 1.731370210647583, + -0.8131548166275024, + 2.996675729751587, + -1.1173620223999023, + 0.7474988102912903, + 0.21459203958511353, + -0.2133086621761322, + -0.22307389974594116, + -0.6306187510490417, + 2.0490896701812744, + -0.49808698892593384, + 0.8432145714759827 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.014628380537033081, + -0.18207009136676788, + 0.5374140739440918, + -0.04374772310256958, + 0.5882754325866699, + 0.25251278281211853, + 0.0, + -0.23516394197940826, + 0.12562739849090576, + 0.3381730020046234, + 0.033050984144210815, + 1.1419475078582764, + -0.3290551006793976, + 0.0, + 0.04625481367111206, + -0.0026901662349700928, + 0.0754578709602356, + -0.1481514573097229, + 1.444929838180542, + 0.049897611141204834, + 0.0 + ] + ], + "predicted_scores": [ + 8.008492469787598, + 7.5172505378723145, + 7.382919788360596, + 7.347182750701904, + 7.306814670562744, + 7.287628650665283, + 7.275352954864502, + 7.248529434204102 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1231780997", + "selected_index": 0, + "state_hash": "771fb752a186163f05a49a9b404284bde55d89838769a764fca1d6d1a9419652", + "stored_base_utility": 0.7242962718009949, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.528373658657074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.528373658657074 + }, + "base_progress": 0.528373658657074, + "base_success": false, + "base_utility": 0.528373658657074, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7183936238288879, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7183936238288879 + }, + { + "contact_quality": null, + "progress": 0.40290379524230957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.40290379524230957 + }, + { + "contact_quality": null, + "progress": 0.70465487241745, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.70465487241745 + }, + { + "contact_quality": null, + "progress": 0.21204102039337158, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21204102039337158 + }, + { + "contact_quality": null, + "progress": 0.7051377892494202, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7051377892494202 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.5513363480567932, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5513363480567932 + }, + { + "contact_quality": null, + "progress": 0.6998324394226074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6998324394226074 + } + ], + "candidate_progress": [ + 0.7183936238288879, + 0.40290379524230957, + 0.70465487241745, + 0.21204102039337158, + 0.7051377892494202, + 1.0, + 0.5513363480567932, + 0.6998324394226074 + ], + "candidate_source_chart_ids": [ + "ms-0660d5f65df5fc4b6132", + "ms-c090bce47703a4c4bb0b", + "ms-13d8bae55d3c3710138f", + "ms-e1b6bcb41532a800d3cb", + "ms-df961fe24a43e56d1716", + "ms-e1b6bcb41532a800d3cb", + "ms-8f3d4e1d07bb5dcc7c57", + "ms-502dba1003c09ec60ba0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank6" + ], + "candidates_evaluated": true, + "chart_id": "ms-4b4cd12d70c0615e0930", + "generated_tangents": [ + [ + -0.08855167031288147, + -0.01947638764977455, + 0.8525065183639526, + 0.7275866270065308, + -0.02662205696105957, + -0.24836260080337524, + 0.06696748733520508, + 0.12216038256883621, + 0.01841357909142971, + -0.20160210132598877, + -0.02135186642408371, + 1.4949530363082886, + -0.39647719264030457, + -0.21711276471614838, + 0.25206345319747925, + -0.0022895345464348793, + -0.1862376183271408, + -0.2952665686607361, + 1.3065345287322998, + -0.4216145873069763, + -0.30772337317466736 + ], + [ + -0.2863571047782898, + -0.015488618984818459, + 0.7250853776931763, + 0.6399716138839722, + 0.04953143000602722, + -0.3244479298591614, + -0.16923648118972778, + 0.1838855892419815, + 0.011750123463571072, + -0.20457276701927185, + 0.18300533294677734, + 1.4162662029266357, + -0.42234745621681213, + -0.1418713480234146, + 0.3295390009880066, + -0.06663493067026138, + -0.1425269991159439, + -0.1821397989988327, + 1.3395352363586426, + -0.3949070870876312, + -0.30942055583000183 + ], + [ + 0.013350412249565125, + -0.023595791310071945, + 0.8012206554412842, + 0.7013665437698364, + -0.060644835233688354, + -0.07592709362506866, + 0.015901386737823486, + 0.09888153523206711, + 0.05620475858449936, + -0.11867684870958328, + 0.0716228112578392, + 1.5111439228057861, + -0.2909482717514038, + -0.14769954979419708, + 0.14648599922657013, + -0.0014249207451939583, + -0.14039750397205353, + -0.24590559303760529, + 1.3765792846679688, + -0.50099778175354, + -0.3407852351665497 + ], + [ + -0.003335699439048767, + -0.10702508687973022, + 1.04110586643219, + 0.7626993656158447, + -0.07068541646003723, + -0.0499863475561142, + -0.03780931234359741, + 0.08456481993198395, + 0.10477641224861145, + -0.08769210427999496, + 0.0016717985272407532, + 1.5022451877593994, + -0.3843003809452057, + 0.15376827120780945, + 0.17219732701778412, + 0.022844955325126648, + -0.21136970818042755, + -0.2430574744939804, + 1.319983959197998, + -0.46171414852142334, + 0.028106391429901123 + ], + [ + -0.10632289946079254, + -0.0019309855997562408, + 0.8705449104309082, + 0.5842269659042358, + -0.2546175718307495, + -0.026848211884498596, + -0.247003436088562, + 0.1544450968503952, + 0.017757084220647812, + -0.2705927789211273, + 0.09732816368341446, + 1.4460221529006958, + -0.2708267271518707, + -0.2734214663505554, + 0.2506394684314728, + 0.0013196561485528946, + -0.14811690151691437, + -0.2630149722099304, + 1.469775915145874, + -0.46460628509521484, + -0.08990907669067383 + ], + [ + -0.07995054125785828, + -0.04254212602972984, + 0.9654157161712646, + 0.7605307102203369, + -0.2271745800971985, + -0.2645329535007477, + -0.1342710256576538, + 0.23652783036231995, + 0.1368769258260727, + -0.16730135679244995, + 0.1140194833278656, + 1.2983299493789673, + -0.17730361223220825, + -0.1553676277399063, + 0.2944020628929138, + 0.0955798327922821, + -0.015879377722740173, + -0.3163635730743408, + 1.5000483989715576, + -0.12904201447963715, + -0.20126837491989136 + ], + [ + -0.006187573075294495, + -0.006004199385643005, + 0.6518256068229675, + 0.7949789762496948, + -0.162237286567688, + 0.09742814302444458, + -0.24809986352920532, + 0.004151567816734314, + 0.052042752504348755, + -0.004879884421825409, + 0.3146045506000519, + 1.6640344858169556, + -0.2553577423095703, + -0.1807306855916977, + 0.18132221698760986, + -0.12746618688106537, + -0.09342710673809052, + -0.18865381181240082, + 1.5411925315856934, + -0.37166672945022583, + -0.12060332298278809 + ], + [ + -0.02540813386440277, + 0.031248025596141815, + 0.5566630363464355, + 0.8019369840621948, + -0.2968810200691223, + 0.25957709550857544, + -0.4576603174209595, + 0.07570387423038483, + -0.055670879781246185, + -0.03128836303949356, + 0.28585225343704224, + 1.640716552734375, + -0.3310239315032959, + 0.05364423990249634, + 0.15118665993213654, + -0.08055293560028076, + -0.011531352996826172, + -0.21709395945072174, + 1.5603160858154297, + -0.372607558965683, + 0.26071345806121826 + ] + ], + "generated_utilities": [ + 0.7183936238288879, + 0.40290379524230957, + 0.70465487241745, + 0.21204102039337158, + 0.7051377892494202, + 2.0, + 0.5513363480567932, + 0.6998324394226074 + ], + "group_id": "ms-4b4cd12d70c0615e0930", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7225138545036316, + 2.0, + 0.6951258182525635, + 0.18915215134620667, + 0.20867900550365448, + 0.20041285455226898, + 0.20335882902145386, + 0.20815956592559814, + 0.2024221420288086, + 0.1940193623304367, + 0.20689067244529724, + 0.19995209574699402, + 0.21901433169841766, + 0.19443893432617188, + 0.20502614974975586, + 0.1959807276725769 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.09018868207931519, + 0.3819495439529419, + 0.9586174488067627, + 1.0247726440429688, + -0.179018035531044, + 0.125870943069458, + 0.0, + -0.1166897565126419, + 0.07711493968963623, + -0.24461698532104492, + -0.005436837673187256, + 1.737370491027832, + -0.4425608515739441, + 0.0, + 0.24725346267223358, + -0.03395397961139679, + 0.643236517906189, + 0.05422210693359375, + 1.2738127708435059, + 0.18164724111557007, + 0.0 + ], + [ + 0.6095315217971802, + 0.35729360580444336, + 1.9586174488067627, + 2.0247726440429688, + -0.2820541560649872, + 1.044631004333496, + -2.239265203475952, + -0.14250506460666656, + -0.4363628029823303, + -1.244616985321045, + -0.32413747906684875, + 2.737370491027832, + 0.24581307172775269, + -0.053983256220817566, + 0.2877395749092102, + 0.18538890779018402, + 0.45247530937194824, + -0.9457778930664062, + 2.273812770843506, + -0.8183527588844299, + 0.5866778492927551 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -4.478530406951904, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.10796651244163513, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1733556985855103 + ], + [ + 0.8415398597717285, + 0.17323988676071167, + 1.9669679403305054, + 2.188286304473877, + -0.386857271194458, + 1.3415166139602661, + -2.06968355178833, + -0.38555508852005005, + -0.6971946954727173, + -1.0110756158828735, + -0.19943451881408691, + 3.0229673385620117, + 0.03575621545314789, + -0.26961493492126465, + 0.5202459692955017, + 0.0921676978468895, + 0.7439858913421631, + -0.724668562412262, + 2.116668939590454, + -1.0585715770721436, + 0.7851569652557373 + ], + [ + 0.6909375190734863, + -0.018118590116500854, + 1.677350401878357, + 1.9505902528762817, + 0.11725521087646484, + 1.2839276790618896, + -2.0543243885040283, + -0.38843441009521484, + -0.6865814328193665, + -1.5103397369384766, + -0.2952713072299957, + 2.384809970855713, + -0.10558199882507324, + 0.29018354415893555, + 0.33942633867263794, + -0.19196103513240814, + 0.35257238149642944, + -0.8657878637313843, + 2.5445473194122314, + -0.706183910369873, + 0.7214455008506775 + ], + [ + 0.8977934122085571, + 0.7311913371086121, + 2.203964948654175, + 1.686698079109192, + -0.338996022939682, + 0.631161630153656, + -2.3877103328704834, + 0.07195566594600677, + -0.6892462968826294, + -0.9634823799133301, + -0.5621621012687683, + 2.479537010192871, + 0.675940752029419, + -0.047649919986724854, + 0.14082925021648407, + 0.19517391920089722, + 0.15626585483551025, + -1.3974462747573853, + 2.759931802749634, + -1.166107416152954, + 0.18018442392349243 + ], + [ + 0.7796773910522461, + 0.06440207362174988, + 1.5159449577331543, + 1.6007665395736694, + -0.12459242343902588, + 1.5781673192977905, + -2.5707991123199463, + 0.034798189997673035, + -0.44772639870643616, + -1.3837177753448486, + -0.5199699401855469, + 3.1566030979156494, + -0.22805768251419067, + -0.5881848931312561, + 0.49021050333976746, + -0.08376060426235199, + 0.5395568609237671, + -1.3038995265960693, + 2.37483811378479, + -0.615712583065033, + 0.7884383201599121 + ], + [ + 0.4830939471721649, + -0.12824741005897522, + 2.133068561553955, + 2.42085337638855, + -0.1641906201839447, + 1.6598048210144043, + -2.5325045585632324, + 0.06623834371566772, + 0.14292877912521362, + -1.85994553565979, + 0.01158285140991211, + 3.2622194290161133, + -0.017848491668701172, + -0.08653604984283447, + 0.2199593186378479, + -0.0463244765996933, + -0.04547742009162903, + -0.5160569548606873, + 2.1955723762512207, + -0.364714115858078, + 1.0323584079742432 + ], + [ + 0.7637265920639038, + -0.4020084738731384, + 2.4714059829711914, + 1.33060622215271, + 0.1464400589466095, + 0.7656213045120239, + -1.9972842931747437, + 0.42333751916885376, + -1.118815302848816, + -1.5639967918395996, + -0.32944348454475403, + 2.0334420204162598, + 0.37165334820747375, + -0.2861292064189911, + 0.6376298666000366, + 0.8521975874900818, + 0.6845490336418152, + -1.515796422958374, + 1.5925071239471436, + -0.4112323224544525, + 1.1011840105056763 + ], + [ + 1.456933617591858, + 1.0030901432037354, + 1.1997551918029785, + 1.7652757167816162, + 0.03292572498321533, + 1.302201271057129, + -2.748408317565918, + 0.7148768305778503, + -0.5089412331581116, + -0.5701506733894348, + -0.40410247445106506, + 3.586395502090454, + -0.37424683570861816, + -0.6850852966308594, + 0.9945629835128784, + 0.5821471810340881, + 0.8666700124740601, + -1.1607441902160645, + 2.1460094451904297, + -0.7912135720252991, + 0.6301633715629578 + ], + [ + -0.09475094079971313, + -0.28330767154693604, + 1.8353716135025024, + 2.3373594284057617, + 0.7110240459442139, + 1.9665697813034058, + -1.8502212762832642, + -0.40577346086502075, + -0.03506562113761902, + -0.7167142033576965, + -0.39153769612312317, + 2.3472883701324463, + 0.08497899770736694, + -0.9910531640052795, + -0.32254326343536377, + -0.11388416588306427, + -0.4873260259628296, + -0.8018304109573364, + 2.295731782913208, + -1.325164794921875, + 1.2513983249664307 + ], + [ + -0.32657402753829956, + 0.8198520541191101, + 1.211392879486084, + 1.974273443222046, + -0.11595214903354645, + 1.954041838645935, + -1.5547080039978027, + 0.06433123350143433, + -0.079216867685318, + -0.5887003540992737, + -0.6015424728393555, + 2.756274938583374, + 0.641853928565979, + -0.8642687201499939, + 0.07965350151062012, + 0.11391039192676544, + 1.10663902759552, + -1.506749153137207, + 3.222418785095215, + -1.0799858570098877, + -0.12158209085464478 + ], + [ + 0.9412136077880859, + -0.5255860090255737, + 1.9096710681915283, + 2.113077402114868, + -1.0082788467407227, + 0.1505374312400818, + -3.0413341522216797, + -0.24721917510032654, + -0.20225940644741058, + -1.6247544288635254, + -0.33320704102516174, + 1.8443769216537476, + 0.7569764256477356, + -0.5270920395851135, + 1.1813421249389648, + 1.0764858722686768, + 0.01533237099647522, + -1.4125463962554932, + 1.82404363155365, + 0.054037630558013916, + 0.45250433683395386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.056152522563934326, + 0.005679905414581299, + 0.9586174488067627, + 1.0247726440429688, + -0.054595887660980225, + 0.044631004333496094, + 0.0, + -0.09190673381090164, + 0.09592300653457642, + -0.24461698532104492, + -0.007907181978225708, + 1.737370491027832, + -0.029382914304733276, + 0.0, + 0.06305502355098724, + 0.03764393925666809, + 0.0785532295703888, + 0.005711078643798828, + 1.2738127708435059, + -0.04058480262756348, + 0.0 + ] + ], + "predicted_scores": [ + 8.606647491455078, + 8.40678596496582, + 8.361595153808594, + 8.183815002441406, + 8.076087951660156, + 7.391698837280273, + 6.736213684082031, + 6.477442264556885 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1025740212", + "selected_index": 0, + "state_hash": "5b394fca1712c0667d126198527a4f0ba81893d536a5fcbf7b90f5b13fa703dc", + "stored_base_utility": 0.7225138545036316, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7146047949790955, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7146047949790955 + }, + { + "contact_quality": null, + "progress": 0.6888599991798401, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6888599991798401 + }, + { + "contact_quality": null, + "progress": 0.20082925260066986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20082925260066986 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.4248526990413666, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4248526990413666 + } + ], + "candidate_progress": [ + 0.7146047949790955, + 0.6888599991798401, + 0.20082925260066986, + 1.0, + 1.0, + 1.0, + 1.0, + 0.4248526990413666 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dfc38c43e9a87f99ef7f", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + true, + true, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6e268d1e2915740816a1", + "generated_tangents": [ + [ + 0.05382830649614334, + -0.11331828683614731, + 0.24708518385887146, + 0.2622750401496887, + 1.526719570159912, + 0.1897905468940735, + 0.06158256530761719, + 0.012340977787971497, + -0.06714219599962234, + 0.2525322735309601, + 0.18585392832756042, + 1.2123268842697144, + 0.10725459456443787, + 0.2626100778579712, + 0.20533233880996704, + 0.10720622539520264, + 0.10231655836105347, + -0.23213641345500946, + 0.8290656805038452, + -0.16684581339359283, + 0.006923690438270569 + ], + [ + -0.20426492393016815, + -0.0987476259469986, + -0.040768831968307495, + 0.24197199940681458, + 1.2935476303100586, + -0.34130075573921204, + 0.22074918448925018, + 0.1377646028995514, + -0.07693739235401154, + 0.2259937971830368, + 0.41069290041923523, + 1.3074543476104736, + -0.01693519949913025, + 0.33140867948532104, + 0.2790125012397766, + 0.057907070964574814, + 0.3529486656188965, + 0.031113892793655396, + 1.1380252838134766, + -0.3711043894290924, + 0.3389366567134857 + ], + [ + 0.021486014127731323, + 0.031080424785614014, + -0.009821504354476929, + 0.17265313863754272, + 1.284395694732666, + -0.1241624653339386, + 0.21560007333755493, + 0.08408765494823456, + -0.020719414576888084, + -0.051108650863170624, + -0.03287827968597412, + 0.9338744878768921, + 0.000622481107711792, + 0.016557902097702026, + 0.09422267973423004, + 0.0624970979988575, + -0.04071719944477081, + -0.21155504882335663, + 0.6939430236816406, + -0.09980952739715576, + -5.316734313964844e-05 + ], + [ + 0.06665319204330444, + -0.01946987211704254, + -0.016627758741378784, + 0.00996515154838562, + 1.5666582584381104, + -0.030515894293785095, + 0.039327532052993774, + 0.008302077651023865, + -0.026523491367697716, + 0.0132223442196846, + 0.05194508284330368, + 1.0613441467285156, + -0.14745602011680603, + 0.040985725820064545, + 0.00977376475930214, + 0.0787392109632492, + 0.04062876105308533, + -0.07444895803928375, + 0.7646706104278564, + -0.11978107690811157, + 0.04210434854030609 + ], + [ + 0.09597141295671463, + -0.05036396533250809, + -0.01115688681602478, + 0.19940954446792603, + 1.1502326726913452, + -0.20421072840690613, + 0.0500774085521698, + -0.059695541858673096, + -0.03573247790336609, + 0.12813347578048706, + 0.1269264817237854, + 1.308370590209961, + -0.0026264190673828125, + 0.29808908700942993, + 0.1656384915113449, + 0.026575077325105667, + 0.1393168419599533, + -0.07339702546596527, + 1.13913893699646, + -0.23282338678836823, + 0.20437416434288025 + ], + [ + -0.001097455620765686, + -0.030729271471500397, + -0.031170636415481567, + 0.047467440366744995, + 1.507990837097168, + -0.00832030177116394, + -0.05092889070510864, + -0.009620800614356995, + -0.02732613869011402, + 0.013773016631603241, + 0.0974811390042305, + 1.0921239852905273, + -0.03999364376068115, + 0.10736331343650818, + -0.041851554065942764, + 0.057456668466329575, + 0.005798563361167908, + -0.09395335614681244, + 0.8034914135932922, + -0.11294423043727875, + 0.06748195737600327 + ], + [ + 0.011999949812889099, + -0.0961015596985817, + 0.012530922889709473, + 0.0672605037689209, + 1.4114291667938232, + 0.0021196305751800537, + 0.06693747639656067, + 0.05514101684093475, + 0.009224820882081985, + 0.07333384454250336, + 0.021902766078710556, + 1.0683848857879639, + 0.07747206091880798, + 0.11161360144615173, + -0.08942191302776337, + 0.08119836449623108, + -0.04012192785739899, + -0.026301532983779907, + 0.791610836982727, + -0.1961107701063156, + 0.13662290573120117 + ], + [ + -0.25270652770996094, + -0.13033172488212585, + 0.03448626399040222, + -0.04329735040664673, + 1.2990400791168213, + -0.2328546941280365, + 0.02604636549949646, + -0.17190861701965332, + -0.08365319669246674, + -0.012697093188762665, + 0.27850812673568726, + 1.6041350364685059, + 0.07613518834114075, + 0.6118152737617493, + 0.07233655452728271, + -0.06824109703302383, + 0.12395923584699631, + -0.07512913644313812, + 1.2368290424346924, + -0.3936001658439636, + 0.5614538192749023 + ] + ], + "generated_utilities": [ + 0.7146047949790955, + 0.6888599991798401, + 0.20082925260066986, + 2.0, + 2.0, + 2.0, + 2.0, + 0.4248526990413666 + ], + "group_id": "ms-6e268d1e2915740816a1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7257392406463623, + 0.7165254950523376, + 2.0, + 0.72465980052948, + 0.7087006568908691, + 0.2306610643863678, + 0.7143489122390747, + 0.7045385837554932, + 0.5905889272689819, + 0.1919487714767456, + 0.1914421170949936, + 0.2230607271194458, + 0.20312398672103882, + 0.19757536053657532, + 0.19724789261817932, + 0.19653666019439697 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.44810131192207336, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0264720916748047, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.9882198572158813 + ], + [ + -0.4707597494125366, + 0.09937268495559692, + 0.6395220160484314, + -0.40935325622558594, + 2.4514670372009277, + -0.02397046983242035, + -0.21874001622200012, + 0.16799744963645935, + 0.0360843725502491, + -0.09719255566596985, + -0.8619975447654724, + 2.2499821186065674, + -0.4933375120162964, + 0.35259634256362915, + -0.34735018014907837, + -0.517992377281189, + 0.6015943288803101, + -0.7677568793296814, + 1.5267367362976074, + -0.16888000071048737, + 0.3771851658821106 + ], + [ + -0.8530820608139038, + -0.6707327961921692, + 0.7970488667488098, + 0.34902632236480713, + 2.1393990516662598, + 0.5651285648345947, + 0.113044872879982, + 0.2987540364265442, + 0.4139185845851898, + 0.41691792011260986, + -0.4906405806541443, + 1.5687955617904663, + -0.8899085521697998, + -0.01883232593536377, + 0.24473188817501068, + 0.554391622543335, + 0.23424841463565826, + -0.10042615234851837, + 2.1214542388916016, + -0.15780393779277802, + 0.5807479023933411 + ], + [ + -0.6562768220901489, + -0.059059903025627136, + 0.24766850471496582, + 0.06101328134536743, + 2.5052053928375244, + 0.5204365849494934, + 0.1595638245344162, + 0.47176462411880493, + 0.3036114573478699, + 0.13197286427021027, + -0.2551935017108917, + 2.185537099838257, + -1.0984127521514893, + 0.3213496804237366, + 0.49875718355178833, + 0.1223822832107544, + 0.2575722932815552, + -0.7017092704772949, + 1.7495591640472412, + -0.19501343369483948, + 1.1214004755020142 + ], + [ + -0.45427173376083374, + -0.041353583335876465, + 0.8143320083618164, + -0.39680930972099304, + 2.4542014598846436, + 0.382559597492218, + -0.9993512630462646, + 0.5202391147613525, + 0.5282149314880371, + 0.4165073335170746, + 0.43153735995292664, + 2.5625123977661133, + -0.721943199634552, + 0.6765530109405518, + -0.6697012186050415, + -0.0637422502040863, + 0.6919896602630615, + -1.033813714981079, + 2.317282199859619, + -0.13503669202327728, + -0.07009828090667725 + ], + [ + -0.548885703086853, + -0.609938383102417, + 0.7181543707847595, + 0.11203774809837341, + 2.478057861328125, + -0.6170704960823059, + -1.0383654832839966, + 0.3524467349052429, + 0.5748242139816284, + 0.42987847328186035, + 0.013119548559188843, + 1.3954555988311768, + -0.6403790712356567, + 0.025414258241653442, + -0.7780460119247437, + -0.23124903440475464, + 0.48347827792167664, + 0.05266502499580383, + 1.692553997039795, + 0.362108051776886, + 0.6330842971801758 + ], + [ + -1.3070502281188965, + -0.947432816028595, + 0.4198402762413025, + -1.1768442392349243, + 1.7843327522277832, + 0.7587301731109619, + 0.7447960376739502, + 0.9411643147468567, + 0.3013244867324829, + -0.3862171769142151, + -1.0263384580612183, + 2.97957181930542, + 0.1849566102027893, + 0.08799406886100769, + 0.1984272301197052, + 0.5277765393257141, + 0.2931878864765167, + -1.1385071277618408, + 1.884846568107605, + 0.23083388805389404, + 0.6359583139419556 + ], + [ + -0.5375385880470276, + 0.16159841418266296, + 1.260543942451477, + -0.6744770407676697, + 2.1646244525909424, + 0.6190755367279053, + -0.746461808681488, + -0.38674360513687134, + 0.8897944092750549, + -0.594306468963623, + -0.05647379159927368, + 1.7390328645706177, + -0.037059009075164795, + 0.470071941614151, + 0.010003047063946724, + 0.08421775698661804, + -0.4613104462623596, + -0.22854508459568024, + 1.772307276725769, + -0.29165884852409363, + 0.6179300546646118 + ], + [ + -0.6489957571029663, + 0.4526708126068115, + 0.5115790367126465, + 0.31459903717041016, + 2.9655239582061768, + 0.07586031407117844, + -1.1667817831039429, + -0.7146965861320496, + 1.0295621156692505, + 0.1932714432477951, + -0.2703063488006592, + 2.9046523571014404, + -1.5882594585418701, + 0.3030351996421814, + 0.6543201208114624, + -0.448961466550827, + 0.4115087389945984, + -0.4087216854095459, + 2.6432881355285645, + -0.6797424554824829, + 0.2669999599456787 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.22510427236557007, + -0.26737168431282043, + 0.10375632345676422, + 0.12666502594947815, + 1.379244327545166, + -0.01088210940361023, + 0.0, + 0.23721180856227875, + -0.10981421172618866, + -0.021816372871398926, + -0.071512371301651, + 1.0360438823699951, + 0.09019589424133301, + 0.0, + 0.03181244805455208, + -0.06466590613126755, + 0.2068846970796585, + 0.14229199290275574, + 0.7610219717025757, + -0.03211131691932678, + 0.0 + ] + ], + "predicted_scores": [ + 8.914471626281738, + 8.706720352172852, + 8.530256271362305, + 8.064072608947754, + 7.93787145614624, + 7.906517505645752, + 7.663044452667236, + 6.954155921936035 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1475440973", + "selected_index": 0, + "state_hash": "38ff5a264c3c39ff1ec8756d84e622963a11ff1ac61568ba57fbea0c114c3140", + "stored_base_utility": 0.7257392406463623, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7239725589752197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7239725589752197 + }, + "base_progress": 0.7239725589752197, + "base_success": false, + "base_utility": 0.7239725589752197, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7199956774711609, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7199956774711609 + }, + { + "contact_quality": null, + "progress": 0.7059324383735657, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7059324383735657 + }, + { + "contact_quality": null, + "progress": 0.7095306515693665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7095306515693665 + }, + { + "contact_quality": null, + "progress": 0.7091171145439148, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7091171145439148 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7119956612586975, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7119956612586975 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7287879586219788, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7287879586219788 + } + ], + "candidate_progress": [ + 0.7199956774711609, + 0.7059324383735657, + 0.7095306515693665, + 0.7091171145439148, + 1.0, + 0.7119956612586975, + 1.0, + 0.7287879586219788 + ], + "candidate_source_chart_ids": [ + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + true, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8864796e31bbe93d7c5d", + "generated_tangents": [ + [ + -0.24948729574680328, + -0.10154072195291519, + 0.18723857402801514, + 0.04775512218475342, + 1.304319143295288, + -0.6922988891601562, + 0.09477657079696655, + 0.2288207709789276, + -0.19800102710723877, + 0.09405873715877533, + 0.33343005180358887, + 1.656205654144287, + 0.13534113764762878, + 0.5131504535675049, + 0.2962728440761566, + 0.026287253946065903, + 0.38559427857398987, + -0.07534115016460419, + 1.5071253776550293, + -0.5716783404350281, + 0.48471352458000183 + ], + [ + -0.051453426480293274, + -0.20163246989250183, + 0.09931141138076782, + -0.28834080696105957, + 1.3524904251098633, + -0.6391878128051758, + 0.03127303719520569, + 0.20816756784915924, + -0.1451248824596405, + -0.04293700307607651, + -0.0452580600976944, + 1.5044714212417603, + 0.10335925221443176, + 0.3435777425765991, + 0.19738075137138367, + -0.001113283447921276, + 0.00808076560497284, + -0.3290145993232727, + 1.2862768173217773, + -0.49025091528892517, + 0.16652131080627441 + ], + [ + -0.05811721086502075, + -0.1174122765660286, + 0.07382097840309143, + -0.27939334511756897, + 1.4479877948760986, + -0.5250906944274902, + 0.023662209510803223, + 0.12426549196243286, + -0.13466525077819824, + -0.07124709337949753, + 0.036029260605573654, + 1.5989625453948975, + -0.13268673419952393, + 0.21213477849960327, + 0.29586273431777954, + 0.03312433138489723, + 0.08937002718448639, + -0.30316615104675293, + 1.2356297969818115, + -0.28782886266708374, + 0.12316034734249115 + ], + [ + -0.07423686981201172, + -0.11393339186906815, + 0.05387812852859497, + -0.2970837950706482, + 1.4041261672973633, + -0.5529136657714844, + -0.010064423084259033, + 0.12289397418498993, + -0.12582704424858093, + -0.10877882689237595, + 0.0802784413099289, + 1.5274192094802856, + -0.05774989724159241, + 0.2809354364871979, + 0.24325363337993622, + 0.008402958512306213, + 0.0143260657787323, + -0.2726101875305176, + 1.2107031345367432, + -0.3309725224971771, + 0.17144231498241425 + ], + [ + -0.1317937821149826, + -0.042197875678539276, + 0.0962325930595398, + -0.07744404673576355, + 1.1796233654022217, + -0.5120006799697876, + 0.20142537355422974, + -0.01813267171382904, + -0.13524067401885986, + 0.16094869375228882, + 0.12391580641269684, + 1.3178013563156128, + -0.09236329793930054, + 0.26182490587234497, + 0.24179939925670624, + -0.05918389558792114, + 0.18928943574428558, + -0.299518346786499, + 1.5256214141845703, + -0.4084209203720093, + 0.23592253029346466 + ], + [ + -0.16534243524074554, + -0.1010637879371643, + 0.1295507848262787, + -0.16955994069576263, + 1.2102713584899902, + -0.4370811879634857, + 0.3040257692337036, + 0.15275846421718597, + -0.13499030470848083, + -0.1539989411830902, + -0.0651867613196373, + 1.4934074878692627, + -0.1477290689945221, + 0.1023603081703186, + 0.1667497307062149, + -0.018498580902814865, + 0.019455790519714355, + -0.4738162159919739, + 1.3704724311828613, + -0.3399258852005005, + 0.2120511531829834 + ], + [ + 0.0197494775056839, + -0.09809909760951996, + 0.10868239402770996, + 0.004440367221832275, + 1.1123642921447754, + -0.536103367805481, + 0.0898403525352478, + -0.012524142861366272, + -0.14132468402385712, + 0.1149870827794075, + 0.07034085690975189, + 1.5279051065444946, + 0.0010775923728942871, + 0.503241777420044, + 0.17414912581443787, + -0.01966852694749832, + 0.21650609374046326, + -0.28958094120025635, + 1.553039312362671, + -0.3947843909263611, + 0.1815936416387558 + ], + [ + -0.30940502882003784, + -0.20311163365840912, + -0.015687376260757446, + -0.24670451879501343, + 1.1989598274230957, + -0.7257016897201538, + -0.05698734521865845, + -0.1187492311000824, + -0.13922591507434845, + -0.039650849997997284, + 0.20960965752601624, + 1.7681872844696045, + 0.11312928795814514, + 0.6690563559532166, + 0.21477000415325165, + -0.10057109594345093, + 0.19097304344177246, + -0.2768039107322693, + 1.617624044418335, + -0.6416243314743042, + 0.503345251083374 + ] + ], + "generated_utilities": [ + 0.7199956774711609, + 0.7059324383735657, + 0.7095306515693665, + 0.7091171145439148, + 2.0, + 0.7119956612586975, + 2.0, + 0.7287879586219788 + ], + "group_id": "ms-8864796e31bbe93d7c5d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.728268027305603, + 0.7257949113845825, + 2.0, + 0.7287554740905762, + 0.72548508644104, + 0.7011849880218506, + 0.7089968919754028, + 0.689879298210144, + 0.722068727016449, + 0.7046384811401367, + 0.1933039277791977, + 0.629345178604126, + 0.18965840339660645, + 0.18740439414978027, + 0.2032846063375473, + 0.20379847288131714 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.3859826922416687, + 0.1119421124458313, + 0.6443181037902832, + 0.11836245656013489, + 1.863237738609314, + -1.166780948638916, + 0.8025261163711548, + -0.5342166423797607, + -0.2057022899389267, + 0.8987585306167603, + -0.525823712348938, + 1.6212486028671265, + 0.25674968957901, + 0.4037417471408844, + -0.4561460018157959, + -0.18429945409297943, + -0.1778680682182312, + -0.22491087019443512, + 2.095893144607544, + -0.3484030067920685, + 0.47628670930862427 + ], + [ + -0.446367084980011, + -0.19229409098625183, + -0.03271818161010742, + 0.39123907685279846, + 1.1786221265792847, + -0.16883885860443115, + -0.3337137699127197, + 0.7789723873138428, + -0.12953178584575653, + -0.028340458869934082, + -0.871502161026001, + 0.7702060341835022, + -0.645424485206604, + 1.0858956575393677, + -0.05413459241390228, + -0.14592209458351135, + -0.03685358166694641, + -0.686826229095459, + 2.084512710571289, + 0.11898508667945862, + 0.20845119655132294 + ], + [ + 0.09377387166023254, + -0.03508002683520317, + -0.29283326864242554, + -0.9559528827667236, + 2.0994086265563965, + 0.02743762731552124, + 0.6819101572036743, + 0.607413113117218, + 0.4378519356250763, + 0.6497513055801392, + -0.5086554288864136, + 0.9556822180747986, + -0.8518581986427307, + 0.31636691093444824, + -0.4196639060974121, + 0.6866374015808105, + 0.5175505876541138, + -0.572692334651947, + 1.651871681213379, + -0.7227057814598083, + -0.33861052989959717 + ], + [ + -0.42610836029052734, + 0.10144555568695068, + 0.7105271220207214, + -1.291037678718567, + 2.2307684421539307, + 0.17725014686584473, + -0.08436906337738037, + 0.9681323766708374, + 0.5482062697410583, + 1.368379831314087, + -1.021532416343689, + 2.5197110176086426, + -0.3508763909339905, + 0.7584415078163147, + 1.0978648662567139, + 0.2626119554042816, + 0.6115584373474121, + -0.7777479290962219, + 1.7278802394866943, + -0.15240566432476044, + 0.4239119291305542 + ], + [ + 1.0207308530807495, + 0.8557561039924622, + 0.4489574134349823, + -1.0117379426956177, + 0.8262714147567749, + -0.630092442035675, + 0.09812727570533752, + 0.21663640439510345, + 0.23771141469478607, + -0.47456812858581543, + -1.3189021348953247, + 2.269665002822876, + -0.7066116333007812, + 1.1826255321502686, + 1.0058563947677612, + 0.1694127470254898, + -0.5649446845054626, + -0.32043421268463135, + 0.46698564291000366, + -0.9052557945251465, + -0.12436902523040771 + ], + [ + -0.5202209949493408, + 0.30583277344703674, + 1.2163622379302979, + 0.4305659234523773, + 1.7276779413223267, + -0.1535550355911255, + -0.5289345383644104, + -0.632349967956543, + 0.9393473267555237, + 0.38391292095184326, + -1.1998018026351929, + 1.9739618301391602, + -1.3405330181121826, + 0.048939138650894165, + 0.6369897127151489, + 0.629721999168396, + 1.388192892074585, + -1.065996766090393, + 1.9119879007339478, + -1.0432264804840088, + 0.9627590179443359 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.13860583305358887, + -0.06957411766052246, + 0.1504676342010498, + 0.1671028435230255, + 0.5675687789916992, + 0.16587841510772705, + 0.0, + -0.21137550473213196, + -0.21542736887931824, + 0.2565074861049652, + -0.00923699140548706, + 0.5251069068908691, + -0.06964048743247986, + 0.0, + -0.19547803699970245, + 0.10278619825839996, + -0.19206219911575317, + -0.1487000733613968, + 0.44575393199920654, + 0.0725468099117279, + 0.0 + ] + ], + "predicted_scores": [ + 9.099912643432617, + 8.674956321716309, + 8.618236541748047, + 8.410686492919922, + 8.1096830368042, + 7.981508731842041, + 7.959592342376709, + 6.850729465484619 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "551504635", + "selected_index": 0, + "state_hash": "ab6587e0b2a3b5f21f23c56daf8894249774912a3b20794bd49dc1b78ec736a7", + "stored_base_utility": 0.728268027305603, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.42231252789497375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.42231252789497375 + }, + "base_progress": 0.42231252789497375, + "base_success": false, + "base_utility": 0.42231252789497375, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.44024136662483215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.44024136662483215 + }, + { + "contact_quality": null, + "progress": 0.3434007465839386, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.3434007465839386 + }, + { + "contact_quality": null, + "progress": 0.2951448857784271, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2951448857784271 + }, + { + "contact_quality": null, + "progress": 0.30091413855552673, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.30091413855552673 + }, + { + "contact_quality": null, + "progress": 0.21940119564533234, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21940119564533234 + }, + { + "contact_quality": null, + "progress": 0.46552690863609314, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.46552690863609314 + }, + { + "contact_quality": null, + "progress": 0.3839980363845825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.3839980363845825 + }, + { + "contact_quality": null, + "progress": 0.5588552355766296, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5588552355766296 + } + ], + "candidate_progress": [ + 0.44024136662483215, + 0.3434007465839386, + 0.2951448857784271, + 0.30091413855552673, + 0.21940119564533234, + 0.46552690863609314, + 0.3839980363845825, + 0.5588552355766296 + ], + "candidate_source_chart_ids": [ + "ms-e1b6bcb41532a800d3cb", + "ms-0660d5f65df5fc4b6132", + "ms-df961fe24a43e56d1716", + "ms-c090bce47703a4c4bb0b", + "ms-8bc7aa419d17368d81b5", + "ms-8bc7aa419d17368d81b5", + "ms-13d8bae55d3c3710138f", + "ms-e1b6bcb41532a800d3cb" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b04b489f4fbad93b65ca", + "generated_tangents": [ + [ + -0.09565496444702148, + -0.13263177871704102, + 0.6378915309906006, + 0.6397560834884644, + 0.19761790335178375, + -0.47166624665260315, + -0.24765628576278687, + 0.12111947685480118, + 0.2253347933292389, + -0.019841589033603668, + 0.003625147044658661, + 1.1748700141906738, + -0.07906889915466309, + -0.18107132613658905, + 0.39330577850341797, + 0.15019100904464722, + -0.11007456481456757, + -0.17186932265758514, + 1.6526024341583252, + -0.09600405395030975, + -0.1769442856311798 + ], + [ + -0.10873229801654816, + -0.1422015279531479, + 0.545904815196991, + 0.7196091413497925, + 0.4515576958656311, + -0.5527061223983765, + -0.19033455848693848, + 0.042164817452430725, + 0.1231064647436142, + -0.04291204363107681, + -0.04714444279670715, + 1.3284800052642822, + -0.32049399614334106, + -0.24472157657146454, + 0.40207263827323914, + 0.038899146020412445, + -0.2567472457885742, + -0.26330113410949707, + 1.5134329795837402, + -0.5167853236198425, + -0.31856706738471985 + ], + [ + -0.13590170443058014, + -0.11261137574911118, + 0.43233710527420044, + 0.6035963296890259, + 0.43890368938446045, + -0.3353692591190338, + -0.4279569387435913, + 0.07998514175415039, + 0.10984005779027939, + -0.11447293311357498, + 0.0253337100148201, + 1.31846284866333, + -0.23573988676071167, + -0.30657511949539185, + 0.3927244544029236, + 0.03903277590870857, + -0.29468464851379395, + -0.21908621490001678, + 1.6010043621063232, + -0.48466551303863525, + -0.29354098439216614 + ], + [ + -0.2986154556274414, + -0.1027606651186943, + 0.2969779968261719, + 0.5915371179580688, + 0.4253876209259033, + -0.5407867431640625, + -0.24466967582702637, + 0.09115974605083466, + 0.12147343903779984, + -0.12931665778160095, + 0.04664324223995209, + 1.3543797731399536, + -0.5234538316726685, + -0.24561326205730438, + 0.4230315685272217, + -0.0007578907534480095, + -0.23283682763576508, + -0.2190060168504715, + 1.5609934329986572, + -0.4632875919342041, + -0.4699924886226654 + ], + [ + -0.39335954189300537, + -0.2351718693971634, + 0.549156129360199, + 0.9912879467010498, + 0.6062486171722412, + 0.06946003437042236, + -0.6914623379707336, + 0.2573244571685791, + 0.0023165950551629066, + -0.07012902945280075, + 0.2582142651081085, + 1.4619543552398682, + 0.18062707781791687, + -0.20514728128910065, + 0.37644141912460327, + 0.097018763422966, + -0.3484461307525635, + -0.03962719440460205, + 1.5388891696929932, + -0.38853707909584045, + 0.2924542725086212 + ], + [ + -0.17095820605754852, + -0.16430068016052246, + 0.40511637926101685, + 0.8542691469192505, + 0.044514626264572144, + 0.3263012766838074, + -0.7373185753822327, + 0.12780943512916565, + 0.0031920671463012695, + 0.06597979366779327, + 0.3259868621826172, + 1.5937522649765015, + -0.053085148334503174, + 0.025669053196907043, + 0.327912300825119, + -0.008785219863057137, + -0.177174374461174, + -0.20275263488292694, + 1.6584527492523193, + -0.24582432210445404, + -0.01783326268196106 + ], + [ + 0.03756118565797806, + -0.13389122486114502, + 0.41211017966270447, + 0.7176592350006104, + 0.4354831278324127, + -0.40109220147132874, + -0.2534736394882202, + 0.018677935004234314, + 0.1709856390953064, + 0.0442802831530571, + 0.04523874446749687, + 1.3362438678741455, + -0.28743869066238403, + -0.23050443828105927, + 0.27787747979164124, + 0.032572779804468155, + -0.23757053911685944, + -0.1685778945684433, + 1.602299451828003, + -0.500733494758606, + -0.3583735525608063 + ], + [ + -1.6555190086364746e-05, + -0.12617585062980652, + 0.6307705044746399, + 0.7687804698944092, + 0.34772413969039917, + -0.4515925943851471, + -0.24265265464782715, + -0.013622060418128967, + 0.18106137216091156, + 0.02512286603450775, + -0.0892622247338295, + 1.3640633821487427, + -0.4228178858757019, + -0.02813822031021118, + 0.24052219092845917, + 0.07421454787254333, + -0.19758112728595734, + -0.12108777463436127, + 1.4839928150177002, + -0.5218106508255005, + -0.02068018913269043 + ] + ], + "generated_utilities": [ + 0.44024136662483215, + 0.3434007465839386, + 0.2951448857784271, + 0.30091413855552673, + 0.21940119564533234, + 0.46552690863609314, + 0.3839980363845825, + 0.5588552355766296 + ], + "group_id": "ms-b04b489f4fbad93b65ca", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7198216915130615, + 0.7247374057769775, + 2.0, + 0.22730140388011932, + 0.21130704879760742, + 0.2076549530029297, + 0.2006116509437561, + 0.21285679936408997, + 0.19614991545677185, + 0.20440611243247986, + 0.2015489637851715, + 0.19511334598064423, + 0.20091164112091064, + 0.19681888818740845, + 0.19675713777542114, + 0.20853900909423828 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.30706143379211426, + 0.2797066569328308, + 0.4565920829772949, + 0.7342244386672974, + -0.28428468108177185, + 0.012420237064361572, + 0.0, + 0.19500797986984253, + -0.1289721131324768, + -0.3524337112903595, + 0.07460063695907593, + 1.7684156894683838, + 0.11335399746894836, + 0.0, + -0.5014548301696777, + 0.4537479281425476, + 0.1963198333978653, + -0.025568604469299316, + 1.6286182403564453, + -0.0963512659072876, + 0.0 + ], + [ + 1.3070614337921143, + -0.7202933430671692, + 1.456592082977295, + 1.7342244386672974, + -0.15130555629730225, + 0.9519147872924805, + -1.4541058540344238, + -0.6432010531425476, + -0.26959508657455444, + -0.3406839668750763, + -0.4419664740562439, + 2.768415689468384, + -0.3271861970424652, + -0.13685251772403717, + 0.23842881619930267, + 0.20406432449817657, + 0.3414191007614136, + -1.0255686044692993, + 2.6286182403564453, + -0.9051820039749146, + 0.6850019693374634 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9082117080688477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.27370503544807434, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.3700039386749268 + ], + [ + 1.0634682178497314, + -0.8590207099914551, + 1.6810569763183594, + 1.8525642156600952, + 0.06716938316822052, + 1.0774534940719604, + -1.177553415298462, + -0.7412493228912354, + -0.3193308115005493, + -0.4253617525100708, + -0.4890587329864502, + 2.624048948287964, + -0.109145388007164, + -0.10380380600690842, + 0.20137116312980652, + 0.20110605657100677, + 0.06929725408554077, + -0.9908366203308105, + 2.5783159732818604, + -0.7248227000236511, + 0.919882595539093 + ], + [ + 1.2083125114440918, + -0.9112091064453125, + 1.1255995035171509, + 1.9678500890731812, + -0.3424606919288635, + 1.350451946258545, + -1.5799542665481567, + -1.014689326286316, + -0.3427739441394806, + -0.24094250798225403, + -0.4992668032646179, + 2.403038740158081, + -0.29080134630203247, + -0.1569681167602539, + 0.4877905249595642, + -0.16464389860630035, + 0.41957730054855347, + -0.6997033357620239, + 2.80957293510437, + -1.0509651899337769, + 0.8937826156616211 + ], + [ + 1.2133954763412476, + -1.0171141624450684, + 1.1572445631027222, + 1.8050156831741333, + -0.29570114612579346, + 0.7672240734100342, + -0.976697564125061, + -0.8219587802886963, + -0.746895432472229, + -0.346658855676651, + -0.7942779064178467, + 3.0403196811676025, + -0.6172577738761902, + 0.3615778088569641, + 0.546725332736969, + -0.06415437161922455, + 0.43834370374679565, + -1.332592248916626, + 3.029940605163574, + -0.5720348358154297, + 1.1660562753677368 + ], + [ + 0.7937494516372681, + -0.45201122760772705, + 2.0367026329040527, + 2.1590728759765625, + 0.38935917615890503, + 0.5424834489822388, + -1.391097903251648, + -1.0657228231430054, + -0.44411367177963257, + -0.3797546923160553, + -0.03612968325614929, + 2.867069721221924, + -0.26322364807128906, + -0.32477179169654846, + 0.007963955402374268, + 0.053216055035591125, + 0.36466139554977417, + -0.7477234601974487, + 2.170649766921997, + -0.46266868710517883, + 0.7669195532798767 + ], + [ + 1.0920023918151855, + -1.0678586959838867, + 1.4391454458236694, + 2.155820608139038, + -0.7347199320793152, + 0.6208673715591431, + -1.6380563974380493, + -0.6589961051940918, + -0.8267081379890442, + 0.011062979698181152, + -0.1169244647026062, + 2.743579626083374, + -0.41342735290527344, + -0.5980290770530701, + 0.49618369340896606, + -0.09863747656345367, + 0.2179805338382721, + -0.6006474494934082, + 2.2965025901794434, + -0.97348952293396, + 1.2447597980499268 + ], + [ + 1.8286218643188477, + -1.486896276473999, + 1.8610713481903076, + 1.792364478111267, + -0.06209900975227356, + 0.7138283252716064, + -1.0023046731948853, + -0.8588749170303345, + -0.30936315655708313, + 0.33444294333457947, + 0.2771298885345459, + 3.0452492237091064, + -0.6618144512176514, + -0.747363805770874, + 0.19540274143218994, + -0.05051566660404205, + 0.5057645440101624, + -0.3770519495010376, + 3.028423309326172, + -0.19366812705993652, + 1.1766406297683716 + ], + [ + 1.057653784751892, + -0.15160071849822998, + 1.1646175384521484, + 2.517653226852417, + 0.037323400378227234, + 1.4844036102294922, + -1.2659661769866943, + -0.9694168567657471, + -0.792635440826416, + 0.15484732389450073, + -0.6592323780059814, + 2.426063060760498, + -0.8063763976097107, + 0.37280529737472534, + -0.1984816938638687, + 0.3488052487373352, + 0.36773377656936646, + -0.9600323438644409, + 1.801730990409851, + -1.1618295907974243, + 0.526383101940155 + ], + [ + 0.8403785228729248, + -1.7131881713867188, + 1.7484346628189087, + 1.3915244340896606, + 0.038706690073013306, + 1.7859117984771729, + -0.6996253132820129, + -0.12596702575683594, + -0.07181461155414581, + -0.8556877374649048, + -0.6567700505256653, + 2.935109853744507, + 0.3808797299861908, + 0.1733608990907669, + 0.9097857475280762, + 0.7001722455024719, + 0.14087507128715515, + -0.32659411430358887, + 2.278188943862915, + -1.87721586227417, + 1.438266634941101 + ], + [ + 1.259635329246521, + -0.5244767665863037, + 2.201894998550415, + 2.2337045669555664, + -0.333431601524353, + 0.7420244216918945, + -2.0896377563476562, + 0.00931471586227417, + 0.6261725425720215, + -0.49785688519477844, + -0.9404669404029846, + 3.756554365158081, + 0.5252982378005981, + -0.3158212900161743, + 0.4053819179534912, + 0.8204222917556763, + 0.5041900277137756, + -0.43809741735458374, + 3.519717216491699, + -1.0316743850708008, + 0.3069072663784027 + ], + [ + 0.9530733823776245, + -0.011007547378540039, + 2.396453380584717, + 2.6382436752319336, + 0.29263433814048767, + 1.6715352535247803, + -0.819398045539856, + -0.5916767120361328, + 0.006437480449676514, + -0.3942071795463562, + -0.5187541842460632, + 3.677276849746704, + -0.4376804232597351, + 0.3331298828125, + -0.3693196773529053, + -0.3432707190513611, + 1.104022741317749, + -1.0780245065689087, + 3.0859882831573486, + -0.26956111192703247, + 0.8377041816711426 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.30706143379211426, + 0.024911105632781982, + 0.4565920829772949, + 0.7342244386672974, + 0.01698923110961914, + -0.04808521270751953, + 0.0, + 0.10375595092773438, + 0.23247671127319336, + -0.12021404504776001, + -0.19280612468719482, + 1.7684156894683838, + -0.23127278685569763, + 0.0, + -0.1010780781507492, + 0.00952194631099701, + -0.16307592391967773, + -0.14496707916259766, + 1.6286182403564453, + -0.28656232357025146, + 0.0 + ] + ], + "predicted_scores": [ + 6.63995361328125, + 6.638190746307373, + 6.4909772872924805, + 6.348557949066162, + 6.141026020050049, + 5.725180625915527, + 5.716888427734375, + 5.514687538146973 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113268940", + "selected_index": 0, + "state_hash": "76e9a962970055fa7565c007db1b35e003cd352908be4cf573066d8b58e2d443", + "stored_base_utility": 0.7198216915130615, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7235286831855774, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7235286831855774 + }, + { + "contact_quality": null, + "progress": 0.7263574600219727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7263574600219727 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7248995304107666, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7248995304107666 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7258209586143494, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7258209586143494 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.7235286831855774, + 0.7263574600219727, + 1.0, + 0.7248995304107666, + 1.0, + 0.7258209586143494, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-efbde8652e75274fbd69", + "ms-b483cac145e16ccdf42a", + "ms-efbde8652e75274fbd69", + "ms-062addb9da32a0c2ea97", + "ms-baa36ee75ec2de550b9b", + "ms-b483cac145e16ccdf42a", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + false, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-c663f7e73e7e8376a05c", + "generated_tangents": [ + [ + 0.11944491416215897, + 0.031826525926589966, + -0.33816173672676086, + 0.029209166765213013, + 1.7001824378967285, + 0.2834256887435913, + 0.1120532751083374, + 0.13245084881782532, + -0.11714623868465424, + 0.06882806867361069, + -0.08093375712633133, + 0.7392621636390686, + 0.10305061936378479, + -0.002112284302711487, + 0.19480200111865997, + 0.14463014900684357, + 0.0032646209001541138, + -0.31107252836227417, + 0.9268627166748047, + -0.20665471255779266, + -0.23001036047935486 + ], + [ + 0.08528398722410202, + 0.0176863893866539, + -0.38725516200065613, + 0.15819844603538513, + 1.5603866577148438, + 0.01915358006954193, + -0.05864226818084717, + 0.021228879690170288, + -0.03750317543745041, + 0.14633160829544067, + -0.018779106438159943, + 0.9331331849098206, + 0.14669212698936462, + 0.11888884752988815, + 0.1058644950389862, + 0.05184245482087135, + -0.10833440721035004, + -0.0656033605337143, + 1.0013415813446045, + -0.06810598075389862, + -0.027165308594703674 + ], + [ + 0.05952262133359909, + -0.013512890785932541, + -0.3565492331981659, + 0.16039448976516724, + 1.6706795692443848, + 0.020124569535255432, + -0.22814667224884033, + -0.025210320949554443, + -0.06667977571487427, + 0.17737813293933868, + 0.043758049607276917, + 0.9179978370666504, + 0.07474192976951599, + 0.09498706459999084, + 0.09281895309686661, + 0.03722773492336273, + -0.17794738709926605, + 0.022280260920524597, + 0.9304789304733276, + -0.0854514092206955, + -0.01712864637374878 + ], + [ + 0.09514965116977692, + -0.0005120635032653809, + -0.37256380915641785, + 0.15310999751091003, + 1.5711615085601807, + 0.013629376888275146, + -0.0076225996017456055, + 0.03125244379043579, + -0.04634927213191986, + 0.09063060581684113, + -0.050507135689258575, + 1.019006371498108, + 0.12158665060997009, + 0.07587262988090515, + 0.13865254819393158, + 0.03530486300587654, + -0.06182853877544403, + -0.08614863455295563, + 0.9436187744140625, + -0.07899908721446991, + 0.005535438656806946 + ], + [ + 0.07381761819124222, + 0.04191262274980545, + -0.36779484152793884, + 0.14787515997886658, + 1.6664400100708008, + -0.10343590378761292, + -0.03951066732406616, + -0.0016361922025680542, + -0.015768131241202354, + 0.12629695236682892, + -0.0589388832449913, + 0.9795559644699097, + 0.1183227002620697, + 0.03436177968978882, + 0.12721997499465942, + 0.04293711483478546, + -0.10107798874378204, + -0.02046598494052887, + 0.844606876373291, + -0.005056291818618774, + -0.05871495604515076 + ], + [ + 0.20549790561199188, + 0.06547529250383377, + -0.3145895302295685, + 0.036741018295288086, + 1.6055974960327148, + -0.26423338055610657, + -0.0729488730430603, + 0.034103766083717346, + 0.026352986693382263, + 0.12014830112457275, + 0.04075225442647934, + 0.8376508355140686, + 0.22271093726158142, + 0.052514754235744476, + 0.16109326481819153, + 0.061621490865945816, + 0.001104786992073059, + -0.05881549417972565, + 1.1016579866409302, + 0.029139339923858643, + 0.0015471577644348145 + ], + [ + 0.07018432766199112, + 0.06617987155914307, + -0.270182341337204, + 0.2668461501598358, + 1.4160938262939453, + -0.21099865436553955, + 0.026192456483840942, + 0.05559873580932617, + -0.033873192965984344, + 0.09704998880624771, + -0.11236021667718887, + 0.9734514951705933, + -0.11001318693161011, + 0.1813044548034668, + 0.172516331076622, + 0.04784749075770378, + -0.16102685034275055, + -0.04399195313453674, + 0.7409241795539856, + 0.07622691988945007, + -0.12989309430122375 + ], + [ + 0.09469524025917053, + 0.051173701882362366, + -0.27179792523384094, + 0.250503808259964, + 1.4205782413482666, + -0.22312268614768982, + 0.07304289937019348, + 0.08305234462022781, + -0.04132872074842453, + 0.07534755766391754, + -0.12687236070632935, + 0.9700124859809875, + -0.05576625466346741, + 0.16492176055908203, + 0.14910683035850525, + 0.05023576319217682, + -0.14562980830669403, + -0.07328973710536957, + 0.7361587285995483, + 0.0778588354587555, + -0.012201830744743347 + ] + ], + "generated_utilities": [ + 2.0, + 0.7235286831855774, + 0.7263574600219727, + 2.0, + 0.7248995304107666, + 2.0, + 0.7258209586143494, + 2.0 + ], + "group_id": "ms-c663f7e73e7e8376a05c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7233090400695801, + 0.7198824882507324, + 2.0, + 0.7120869159698486, + 0.27145418524742126, + 0.23090600967407227, + 0.2526220679283142, + 0.2656436562538147, + 0.264312207698822, + 0.27572086453437805, + 0.296668142080307, + 0.2128777951002121, + 0.22965002059936523, + 0.2196153998374939, + 0.22027263045310974, + 0.19939398765563965 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.038699887692928314, + -0.3685438632965088, + -1.3628361225128174, + 0.0675731897354126, + 2.4906270503997803, + 0.8971554040908813, + 0.06863358616828918, + 0.14689946174621582, + 0.0500560887157917, + 0.5852231383323669, + -0.5345680713653564, + 1.7433128356933594, + -0.5520187020301819, + 0.36285334825515747, + 0.06837204098701477, + 0.04091557487845421, + 0.26646411418914795, + -0.3870337903499603, + 1.7901486158370972, + -0.46956348419189453, + 0.5412080883979797 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.13726717233657837, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7257066965103149, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0824161767959595 + ], + [ + -0.09956050664186478, + -0.6439464092254639, + -1.5331499576568604, + -0.2275703251361847, + 2.2570624351501465, + 1.145668387413025, + -0.19933581352233887, + 0.15889883041381836, + -0.07524599134922028, + 0.6580387353897095, + -0.5754004716873169, + 1.6105921268463135, + -0.25637149810791016, + 0.37256744503974915, + 0.17931225895881653, + 0.286465048789978, + 0.07884949445724487, + -0.35266172885894775, + 1.548264980316162, + -0.2917717695236206, + 0.3307930827140808 + ], + [ + 0.13072630763053894, + -0.5927069783210754, + -1.4328954219818115, + 0.1252535879611969, + 2.5439467430114746, + 1.119347095489502, + -0.12134809792041779, + -0.09485669434070587, + -0.040146272629499435, + 0.4683093726634979, + -0.6791773438453674, + 2.1314187049865723, + -0.4368498623371124, + 0.5876930952072144, + 0.27027207612991333, + -0.1808907389640808, + 0.037005022168159485, + -0.6130290627479553, + 1.60811185836792, + -0.8427046537399292, + 0.5648807883262634 + ], + [ + -0.019807856529951096, + -0.40178078413009644, + -1.5710430145263672, + 0.3568149507045746, + 2.8693275451660156, + 1.2080440521240234, + 0.45903119444847107, + -0.07479061186313629, + 0.2587376832962036, + 0.37667450308799744, + -0.5607666373252869, + 1.9825769662857056, + -0.7874732613563538, + 0.48894837498664856, + -0.04161303490400314, + 0.2237108200788498, + 0.12043289840221405, + -0.8541218638420105, + 1.3131444454193115, + -0.8318716287612915, + 0.7653946876525879 + ], + [ + -0.039324238896369934, + 0.18316024541854858, + -1.2052042484283447, + -0.020045489072799683, + 2.441659927368164, + 1.3116326332092285, + 0.04364071041345596, + 0.617156982421875, + -0.5366188287734985, + 0.5837841629981995, + -0.6157102584838867, + 1.946446180343628, + -0.44197604060173035, + -0.12030342221260071, + 0.2635759711265564, + -0.34203875064849854, + 0.3788418769836426, + -0.01673591136932373, + 1.7612254619598389, + -0.8283271193504333, + 0.44542378187179565 + ], + [ + 0.020479684695601463, + -0.3728349208831787, + -1.2566280364990234, + -0.4903675317764282, + 2.7149407863616943, + 1.1410136222839355, + 0.18438707292079926, + 0.15751159191131592, + 0.5994289517402649, + 0.7441410422325134, + -0.5757432579994202, + 2.025078058242798, + -1.1044142246246338, + -0.24153441190719604, + 0.7623711824417114, + -0.07102327048778534, + 0.7491155862808228, + -0.853058397769928, + 1.1346031427383423, + -0.4604499638080597, + 0.3380756974220276 + ], + [ + -0.0438276082277298, + 0.21775567531585693, + -0.9180121421813965, + 0.2164689004421234, + 1.7669851779937744, + 0.7701845765113831, + -0.4171431064605713, + -0.44690096378326416, + -0.6473215818405151, + 0.5592721700668335, + 0.1537061333656311, + 1.2201651334762573, + -0.02862715721130371, + 0.759445309638977, + -0.08159622550010681, + -0.038715001195669174, + 0.8136870265007019, + -1.129607081413269, + 1.5099153518676758, + -1.2117259502410889, + 1.2496943473815918 + ], + [ + -0.6884663105010986, + -0.7670764923095703, + -1.3375780582427979, + 0.42161574959754944, + 2.4925031661987305, + 0.18111640214920044, + -0.4373469054698944, + 0.6228322386741638, + -0.3739543557167053, + 1.3811917304992676, + 0.05884122848510742, + 1.0582032203674316, + -0.38174134492874146, + 0.7282494902610779, + 0.9550073146820068, + -0.6478474736213684, + -0.467768132686615, + -0.22179017961025238, + 2.1456754207611084, + -0.39938730001449585, + 1.08339524269104 + ], + [ + 0.7042056322097778, + -0.4828507900238037, + -1.914047122001648, + -0.7084186673164368, + 1.87782621383667, + 0.42005008459091187, + 0.511604905128479, + -0.2043081521987915, + 0.9001545906066895, + 1.083567500114441, + -0.9796027541160583, + 1.3277345895767212, + -0.5564201474189758, + 0.6932777166366577, + -0.5934457778930664, + -0.296670138835907, + 0.14519014954566956, + -0.8612968325614929, + 0.8968887329101562, + -0.5584146976470947, + -0.434501588344574 + ], + [ + 0.19210004806518555, + 0.5024477243423462, + -0.4993329644203186, + -0.826042115688324, + 1.6430644989013672, + 0.9877098798751831, + 0.49057987332344055, + -0.13611945509910583, + 0.7228546738624573, + 1.4937405586242676, + 0.18012672662734985, + 2.188062906265259, + -0.6084107160568237, + 1.0874407291412354, + 0.5744171142578125, + 0.9375161528587341, + -0.43115919828414917, + 0.19787153601646423, + 2.652085781097412, + -0.529538631439209, + 0.03626328706741333 + ], + [ + -0.014053341001272202, + -1.351341724395752, + -0.7293765544891357, + 0.9410406947135925, + 3.139275074005127, + 1.3463419675827026, + -0.728021502494812, + -0.6353646516799927, + -0.19685329496860504, + 1.5315749645233154, + -1.2550475597381592, + 1.7411813735961914, + 0.14418578147888184, + 0.7038239240646362, + 0.1004495620727539, + 0.7590892910957336, + 0.9287852048873901, + -0.23726919293403625, + 0.8381044864654541, + -1.2906644344329834, + 0.9584261178970337 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03644278272986412, + -0.12137487530708313, + -0.3628361225128174, + -0.027813076972961426, + 1.4906270503997803, + -0.10284459590911865, + 0.0, + 0.15174832940101624, + 0.015595663338899612, + -0.10680150985717773, + -0.14723241329193115, + 0.7433128356933594, + 0.25961190462112427, + 0.0, + -0.23801246285438538, + -0.10784421861171722, + 0.09500086307525635, + -0.06586915254592896, + 0.7901486158370972, + -0.10978007316589355, + 0.0 + ] + ], + "predicted_scores": [ + 6.860178470611572, + 6.529614448547363, + 6.503523826599121, + 6.478070259094238, + 6.436008453369141, + 6.433952808380127, + 6.285012722015381, + 6.118747234344482 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1528740417", + "selected_index": 0, + "state_hash": "c02fdfdd2339e4215d60b2221be925e066a7763f9e3bba3ff33a15f20e286f5a", + "stored_base_utility": 0.7233090400695801, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15342965722084045, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15342965722084045 + }, + "base_progress": 0.15342965722084045, + "base_success": false, + "base_utility": 0.15342965722084045, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.045715175569057465, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.045715175569057465 + }, + { + "contact_quality": null, + "progress": 0.05852393060922623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05852393060922623 + }, + { + "contact_quality": null, + "progress": 0.04664717987179756, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04664717987179756 + }, + { + "contact_quality": null, + "progress": 0.03780760616064072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03780760616064072 + }, + { + "contact_quality": null, + "progress": 0.4659159183502197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4659159183502197 + }, + { + "contact_quality": null, + "progress": 0.037175167351961136, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037175167351961136 + }, + { + "contact_quality": null, + "progress": 0.03381476551294327, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03381476551294327 + }, + { + "contact_quality": null, + "progress": 0.030572224408388138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030572224408388138 + } + ], + "candidate_progress": [ + 0.045715175569057465, + 0.05852393060922623, + 0.04664717987179756, + 0.03780760616064072, + 0.4659159183502197, + 0.037175167351961136, + 0.03381476551294327, + 0.030572224408388138 + ], + "candidate_source_chart_ids": [ + "ms-a509a572b3871e591d26", + "ms-b7e7fe66a6c86898eec2", + "ms-b7e7fe66a6c86898eec2", + "ms-4015c296239564108da2", + "ms-0541a144afe57b5a7ae6", + "ms-23a355317920e192147f", + "ms-307f9067d5b66f55b3a6", + "ms-5df54a8baaceb8c73c8a" + ], + "candidate_source_task_ids": [ + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "generated_tangents": [ + [ + -0.09744466841220856, + 0.1615401655435562, + 0.9181876182556152, + 0.28216755390167236, + -0.1380867063999176, + 0.039674438536167145, + 0.2712133526802063, + 0.0829060822725296, + -0.09420070052146912, + 0.19250546395778656, + 0.3357296586036682, + 0.1237170398235321, + 0.3213690221309662, + 1.6887140274047852, + -0.16327401995658875, + -0.020726287737488747, + 0.18956893682479858, + -0.3159496784210205, + -0.06023472547531128, + 0.05663737654685974, + 1.5617576837539673 + ], + [ + -0.38515937328338623, + 0.2430860847234726, + 0.5041823387145996, + 0.06428265571594238, + -0.2101939618587494, + -0.4898875653743744, + 0.08606752753257751, + 0.12442474067211151, + -0.08885777741670609, + 0.05913913995027542, + 0.6192091703414917, + 0.26889485120773315, + 0.7972333431243896, + 1.415450096130371, + 0.13711148500442505, + 0.030369684100151062, + 0.20247438549995422, + -0.16939528286457062, + 0.00013431906700134277, + 0.08190920948982239, + 1.030199646949768 + ], + [ + -0.4897669553756714, + 0.2485741823911667, + 0.7056241035461426, + 0.06932103633880615, + -0.1584480106830597, + -0.4563254415988922, + 0.1472950577735901, + 0.12362554669380188, + -0.07727830857038498, + -0.13273054361343384, + 0.633302628993988, + 0.3257061243057251, + 0.8085421323776245, + 1.2082523107528687, + 0.10343445837497711, + 0.1136644035577774, + 0.2867079973220825, + -0.35651665925979614, + 0.0778837502002716, + 0.22079920768737793, + 1.1263872385025024 + ], + [ + -0.3609389662742615, + -0.11747902631759644, + 1.1539137363433838, + -0.038696080446243286, + -0.13913333415985107, + 0.057155631482601166, + -0.043315112590789795, + 0.027488023042678833, + -0.12646590173244476, + 0.13201157748699188, + 0.3032550513744354, + 0.06060728430747986, + 0.6882154941558838, + 1.3331786394119263, + 0.08885516226291656, + -0.07271460443735123, + -0.029094234108924866, + -0.2840335965156555, + -0.14702251553535461, + 0.25668275356292725, + 1.2032619714736938 + ], + [ + -0.12941421568393707, + -0.04688846319913864, + 0.8783526420593262, + 0.02738828957080841, + -0.018859386444091797, + 0.4211578369140625, + -0.09172028303146362, + 0.1925777941942215, + -0.18124040961265564, + 0.15872693061828613, + 0.5672739744186401, + 0.41217777132987976, + 0.6639254093170166, + 1.3441522121429443, + 0.06883271038532257, + -0.0027641523629426956, + -0.07542546093463898, + -0.3838971257209778, + -0.18540611863136292, + 0.10263770818710327, + 1.2445440292358398 + ], + [ + -0.336281418800354, + -0.06685802340507507, + 1.120572805404663, + -0.13340243697166443, + -0.2491285502910614, + 0.11167621612548828, + 0.23930606245994568, + 0.05797678232192993, + -0.17424003779888153, + 0.1641796976327896, + 0.2597396671772003, + 0.1383146047592163, + 0.8167866468429565, + 1.2172259092330933, + 0.029573053121566772, + -0.06002848967909813, + -0.00965861976146698, + -0.29838794469833374, + -0.22452014684677124, + 0.2784825563430786, + 1.1556915044784546 + ], + [ + -0.40881097316741943, + -0.015014121308922768, + 1.1260370016098022, + -0.05635581910610199, + -0.17688068747520447, + 0.13964474201202393, + 0.06273573637008667, + 0.15282008051872253, + -0.12866325676441193, + 0.15834760665893555, + 0.42304787039756775, + 0.16547508537769318, + 0.7014168500900269, + 1.255672574043274, + 0.018889322876930237, + -0.04071212559938431, + 0.0684061199426651, + -0.2770296335220337, + -0.2071492075920105, + 0.1232648491859436, + 1.2043311595916748 + ], + [ + -0.03652302920818329, + -0.013153169304132462, + 0.9712647199630737, + 0.05203777551651001, + -0.2379392385482788, + 0.33988142013549805, + -0.003175705671310425, + 0.2110832929611206, + -0.18604235351085663, + 0.17053517699241638, + 0.5016647577285767, + 0.2015095353126526, + 0.6059832572937012, + 1.3119239807128906, + 0.06690624356269836, + 0.014003142714500427, + -0.04031161963939667, + -0.37628525495529175, + -0.19961538910865784, + 0.2973998785018921, + 1.0768033266067505 + ] + ], + "generated_utilities": [ + 0.045715175569057465, + 0.05852393060922623, + 0.04664717987179756, + 0.03780760616064072, + 0.4659159183502197, + 0.037175167351961136, + 0.03381476551294327, + 0.030572224408388138 + ], + "group_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.6040759086608887, + 2.0, + 0.14582239091396332, + 0.5723002552986145, + 0.10770338028669357, + 0.10128920525312424, + 0.08499570190906525, + 0.14736615121364594, + 0.07181884348392487, + 0.11773071438074112, + 0.06549974530935287, + 0.06568789482116699, + 0.10681070387363434, + 0.05432777479290962, + 0.0625491738319397, + 0.06991603225469589 + ], + "instruction": "Pick up the cube and move it to the goal position.", + "negative_tangents": [ + [ + -0.072060227394104, + 0.2043294906616211, + 0.8692872524261475, + 0.020746350288391113, + -0.004742175340652466, + 0.05779099464416504, + 0.0, + -0.0498790442943573, + -0.20218299329280853, + 0.41556206345558167, + -0.2306707501411438, + 0.32090064883232117, + -0.2089032530784607, + 0.0, + 0.05318737402558327, + -0.04637153446674347, + 0.04412045329809189, + 0.08670947700738907, + -0.10918038338422775, + -0.059010982513427734, + 0.0 + ], + [ + -0.5282122492790222, + 1.204329490661621, + 1.8692872524261475, + -0.8456907868385315, + 0.42858073115348816, + -0.3134727478027344, + -2.5308754444122314, + 0.1655428111553192, + -0.12024828046560287, + 0.0010203085839748383, + 0.2592625319957733, + 0.2679504156112671, + 0.25922709703445435, + 2.851924419403076, + 0.0039860643446445465, + -0.014717570506036282, + 0.01173020526766777, + 0.10578173398971558, + 0.05138798803091049, + 0.3216937780380249, + 2.9243662357330322 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.061750888824463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.703848838806152, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.8487324714660645 + ], + [ + -0.6558206081390381, + 1.3553937673568726, + 2.057647466659546, + -1.0405915975570679, + 0.2629950940608978, + -0.28233209252357483, + -2.589219093322754, + -0.06961745023727417, + 0.02406928688287735, + 0.2341437190771103, + 0.49766814708709717, + 0.4930538535118103, + 0.370309442281723, + 3.111422061920166, + 0.06151486933231354, + -0.2028706818819046, + 0.12007936835289001, + 0.2164750099182129, + 0.16723006963729858, + 0.26430171728134155, + 3.0643396377563477 + ], + [ + -0.18174728751182556, + 0.8890945315361023, + 1.5388343334197998, + -0.4532086253166199, + 0.3864310085773468, + -0.45410382747650146, + -2.623199462890625, + 0.5360933542251587, + -0.04706605523824692, + 0.11160072684288025, + 0.5460608005523682, + 0.39992716908454895, + 0.17028246819972992, + 2.8760411739349365, + 0.29450932145118713, + -0.3028962016105652, + 0.06744809448719025, + 0.1687500774860382, + 0.42097336053848267, + 0.3965923488140106, + 3.28550386428833 + ], + [ + -0.4313088357448578, + 0.9752410054206848, + 2.0068275928497314, + -0.5783756375312805, + 0.5761726498603821, + -0.5289677381515503, + -2.321643590927124, + 0.0023015737533569336, + -0.07372284680604935, + 0.05664224550127983, + 0.03747282922267914, + 0.4900178611278534, + 0.31588616967201233, + 3.018609046936035, + 0.12529893219470978, + -0.2872275412082672, + -0.2500917911529541, + 0.0006694421172142029, + -0.39133700728416443, + 0.6382719874382019, + 2.810041904449463 + ], + [ + -0.5560051202774048, + 1.4042110443115234, + 1.662226915359497, + -1.4059122800827026, + 0.8242433071136475, + -0.10998553037643433, + -2.7556281089782715, + -0.26875895261764526, + -0.4201813042163849, + -0.371376633644104, + 0.7718684673309326, + 0.6452833414077759, + 0.027421817183494568, + 2.3879575729370117, + -0.47588983178138733, + -0.045049820095300674, + 0.0838315486907959, + 0.10257662832736969, + 0.4181908369064331, + 0.2650004029273987, + 3.006909132003784 + ], + [ + -0.4896228015422821, + 1.0570447444915771, + 1.408416509628296, + -0.7269805669784546, + 1.023213267326355, + -0.22378163039684296, + -2.1348509788513184, + -0.4423682987689972, + 0.26834720373153687, + -0.5835619568824768, + 0.3675343692302704, + 0.3307996988296509, + -0.01537257432937622, + 3.4422543048858643, + 0.3622148334980011, + -0.26806333661079407, + 0.4238347113132477, + 0.2583734393119812, + 0.10431534051895142, + 0.2077663242816925, + 2.446356773376465 + ], + [ + -1.1077957153320312, + 0.4289678931236267, + 1.9882663488388062, + -0.3021383285522461, + -0.006195276975631714, + -0.22295325994491577, + -1.8697566986083984, + 0.9590625762939453, + -0.0024877265095710754, + -0.3088582456111908, + 0.6517632007598877, + 0.4271944761276245, + 0.34532302618026733, + 2.704721450805664, + 0.7557594776153564, + -0.8073300719261169, + -0.7479258179664612, + -0.19068273901939392, + 0.14240311086177826, + 0.2233867347240448, + 2.1774275302886963 + ], + [ + -0.9354162812232971, + 1.5971527099609375, + 1.0168101787567139, + -0.7164653539657593, + 0.519247829914093, + -0.7477964162826538, + -3.338376045227051, + 0.7212944030761719, + -0.6477807760238647, + 0.38193029165267944, + -0.1568622589111328, + 0.47457069158554077, + -0.30742186307907104, + 2.6848907470703125, + -0.4251534044742584, + 0.6623334884643555, + 0.03348249942064285, + -0.2996619939804077, + -0.8319172263145447, + 0.7883892059326172, + 3.5870752334594727 + ], + [ + -0.44698429107666016, + 0.3201255202293396, + 1.5405346155166626, + -0.9956957101821899, + 0.8944320678710938, + -0.4196317791938782, + -2.010188102722168, + 0.7199084758758545, + -0.49276289343833923, + 0.4512734115123749, + 0.48276829719543457, + 0.22657281160354614, + 0.8205416798591614, + 3.6653342247009277, + 0.6818231344223022, + 0.4660317897796631, + -0.14916014671325684, + 0.5328372716903687, + -0.5979922413825989, + 0.3124805986881256, + 3.110726833343506 + ], + [ + -0.5630171298980713, + 1.934722900390625, + 1.4954721927642822, + -1.684873342514038, + 0.133146733045578, + -0.4359630346298218, + -2.681238889694214, + 0.8904907703399658, + -0.48221057653427124, + -0.7635119557380676, + -0.23804441094398499, + 0.5691602230072021, + -0.23042798042297363, + 3.760842800140381, + 0.3684825003147125, + -0.20475278794765472, + 0.04838913306593895, + 0.17439094185829163, + 0.07042267918586731, + -0.1653938889503479, + 3.844882011413574 + ], + [ + -0.3705383241176605, + 1.5082495212554932, + 0.9002141356468201, + -1.3291171789169312, + 0.15676641464233398, + -1.2286558151245117, + -1.7006758451461792, + 0.34438908100128174, + 0.32962289452552795, + -0.6556114554405212, + 0.3502740263938904, + 0.7518025636672974, + -0.49080890417099, + 2.10355806350708, + 0.36426910758018494, + -0.4822288453578949, + 0.7496572732925415, + 0.43558254837989807, + 0.34650692343711853, + 0.9977220296859741, + 1.9853770732879639 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.005563676357269287, + 0.2043294906616211, + 0.8692872524261475, + 0.08163481950759888, + 0.05995750427246094, + 0.012857973575592041, + 0.0, + 0.01583634316921234, + 0.04724548012018204, + 0.018815450370311737, + 0.02742627263069153, + -0.03381502628326416, + 0.007930099964141846, + 0.0, + 0.01774722896516323, + 0.08064591884613037, + -0.06814983487129211, + -0.002621002495288849, + 0.05759798362851143, + 0.05164051055908203, + 0.0 + ] + ], + "predicted_scores": [ + 3.5102407932281494, + 3.406304121017456, + 2.9733526706695557, + 2.7440531253814697, + 2.741786003112793, + 2.675002098083496, + 2.5794975757598877, + 2.283445358276367 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1273247072", + "selected_index": 0, + "state_hash": "237fafab6bfdb561d66c862026f803fbd19d4007f6313bdb3d79b6dd74f69327", + "stored_base_utility": 0.6040759086608887, + "task_id": "PickCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.050487834960222244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.050487834960222244 + }, + "base_progress": 0.050487834960222244, + "base_success": false, + "base_utility": 0.050487834960222244, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0001621246337890625, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0001621246337890625 + }, + { + "contact_quality": null, + "progress": 6.29425048828125e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 6.29425048828125e-05 + }, + { + "contact_quality": null, + "progress": 0.10238292068243027, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10238292068243027 + }, + { + "contact_quality": null, + "progress": 0.019885918125510216, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019885918125510216 + }, + { + "contact_quality": null, + "progress": 0.0074851117096841335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0074851117096841335 + }, + { + "contact_quality": null, + "progress": 0.0004499355854932219, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0004499355854932219 + }, + { + "contact_quality": null, + "progress": 0.0001078049317584373, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0001078049317584373 + }, + { + "contact_quality": null, + "progress": 0.1601850390434265, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1601850390434265 + } + ], + "candidate_progress": [ + 0.0001621246337890625, + 6.29425048828125e-05, + 0.10238292068243027, + 0.019885918125510216, + 0.0074851117096841335, + 0.0004499355854932219, + 0.0001078049317584373, + 0.1601850390434265 + ], + "candidate_source_chart_ids": [ + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0345d8ac530f88cefd61", + "generated_tangents": [ + [ + 0.477573037147522, + -0.08181564509868622, + -0.2302217185497284, + 0.2649448812007904, + 0.41885775327682495, + 0.2656289339065552, + 0.6136137247085571, + 0.04889358580112457, + 0.3357800543308258, + 0.522717297077179, + -0.8441930413246155, + -0.06290313601493835, + -0.47927236557006836, + -0.05249950289726257, + -0.43283581733703613, + 0.1604544073343277, + 0.19137689471244812, + -0.25409168004989624, + -1.2012383937835693, + 0.6015713810920715, + 0.19387075304985046 + ], + [ + 0.546461284160614, + -0.1376083493232727, + -0.19663384556770325, + 0.12129625678062439, + 0.6050100326538086, + 0.44333183765411377, + 0.3731582760810852, + 0.0072501301765441895, + 0.3092706501483917, + 0.6923475861549377, + -0.7566579580307007, + -0.2266210913658142, + -0.4289129078388214, + -0.2520459294319153, + -0.5079135894775391, + 0.3172563314437866, + 0.28310734033584595, + -0.18701614439487457, + -1.145477056503296, + 0.4701024889945984, + 0.06293369829654694 + ], + [ + 0.2231089174747467, + -0.053958710283041, + -0.13760390877723694, + -0.0734052062034607, + 0.2534986138343811, + 0.212349072098732, + 0.2112533450126648, + 0.20914889872074127, + 0.3499075174331665, + 0.6650373935699463, + -0.7289736270904541, + -0.5831161141395569, + -0.6725144386291504, + -0.4102931022644043, + -0.525953471660614, + 0.22492794692516327, + 0.29086461663246155, + -0.22250987589359283, + -1.1890788078308105, + 0.7573410868644714, + -0.09697198867797852 + ], + [ + 0.4393550753593445, + -0.09524684399366379, + -0.21939662098884583, + 0.29503098130226135, + 0.26314741373062134, + 0.1890282928943634, + 0.6083811521530151, + 0.03921709954738617, + 0.2885315418243408, + 0.44384944438934326, + -0.8744981288909912, + -0.04786786437034607, + -0.24640974402427673, + -0.032241448760032654, + -0.4413410425186157, + 0.21452073752880096, + 0.24054858088493347, + -0.11529876291751862, + -1.2987656593322754, + 0.48039329051971436, + 0.1959494650363922 + ], + [ + 0.3388286828994751, + 0.0007252991199493408, + 0.025149255990982056, + 0.20357713103294373, + 0.326875776052475, + 0.32344821095466614, + 0.050565630197525024, + -0.12077262997627258, + 0.42205339670181274, + 0.5342826843261719, + -0.8682466745376587, + -0.059065788984298706, + 0.040771037340164185, + -0.3112889528274536, + -0.37888073921203613, + 0.17731750011444092, + -0.2201598435640335, + -0.07172225415706635, + -1.107750415802002, + 0.061236560344696045, + -0.14528051018714905 + ], + [ + 0.47605276107788086, + -0.05462750792503357, + -0.3020217716693878, + 0.39696410298347473, + 0.5762397050857544, + 0.39844024181365967, + 0.731613278388977, + -0.08021789789199829, + 0.29469701647758484, + 0.6287778615951538, + -1.0296783447265625, + 0.015520885586738586, + -0.45798254013061523, + 0.346148818731308, + -0.4485040307044983, + 0.14196041226387024, + 0.12641339004039764, + -0.2951399087905884, + -1.1279394626617432, + 0.3661571741104126, + 0.20907360315322876 + ], + [ + 0.39464622735977173, + -0.020881781354546547, + 0.015634357929229736, + 0.3644595444202423, + 0.3951271176338196, + 0.0026221275329589844, + 0.37702199816703796, + -0.153532475233078, + 0.2507711350917816, + 0.4081285297870636, + -1.0615930557250977, + -0.07481244206428528, + -0.24236957728862762, + 0.34386101365089417, + -0.4009522497653961, + 0.17217549681663513, + 0.013853684067726135, + -0.14843608438968658, + -1.3126630783081055, + 0.2959710955619812, + 0.19673213362693787 + ], + [ + 0.44646137952804565, + -0.12067033350467682, + 0.08336853981018066, + 0.5232672691345215, + 0.3084275722503662, + 0.2186690866947174, + 0.5163891315460205, + -0.17833521962165833, + 0.25861865282058716, + 0.39861077070236206, + -1.1110869646072388, + -0.07697618007659912, + -0.2048243284225464, + 0.24569550156593323, + -0.435116171836853, + 0.05934559181332588, + -0.07593880593776703, + -0.29134851694107056, + -1.3166427612304688, + 0.047914475202560425, + 0.12264642119407654 + ] + ], + "generated_utilities": [ + 0.0001621246337890625, + 6.29425048828125e-05, + 0.10238292068243027, + 0.019885918125510216, + 0.0074851117096841335, + 0.0004499355854932219, + 0.0001078049317584373, + 0.1601850390434265 + ], + "group_id": "ms-0345d8ac530f88cefd61", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.007898053154349327, + 0.0058197579346597195, + 0.052657388150691986, + 0.016109248623251915, + 0.1698339879512787, + 0.008177022449672222, + 0.12333428859710693, + 0.1150994747877121, + 0.15141737461090088, + 0.08515926450490952, + 0.1523766964673996, + 0.04073335975408554, + 0.027175527065992355, + 0.1798098385334015, + 0.08658158779144287, + 0.0618208646774292 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2244938164949417, + -0.708204448223114, + 1.4058912992477417, + 0.19480958580970764, + 1.1751772165298462, + -1.2265020608901978, + -0.8636159896850586, + -0.07021944224834442, + 1.2708500623703003, + 1.0246398448944092, + -1.7615737915039062, + -0.44495686888694763, + -1.2927910089492798, + 0.27074551582336426, + -1.6349679231643677, + 0.9748592376708984, + 1.0210484266281128, + 0.36906859278678894, + -2.528050661087036, + 0.7326674461364746, + 1.553327202796936 + ], + [ + 0.35595428943634033, + -0.8224175572395325, + 1.1192705631256104, + 0.4673205614089966, + 1.1115962266921997, + -1.4676826000213623, + -1.057816505432129, + -0.008461203426122665, + 0.9729399085044861, + 0.8019976615905762, + -1.4944581985473633, + -0.21817980706691742, + -1.3722625970840454, + 0.3557305932044983, + -1.8202308416366577, + 1.2552388906478882, + 0.7855625748634338, + 0.5709130167961121, + -2.4140331745147705, + 0.9063631296157837, + 1.282220721244812 + ], + [ + 0.19927366077899933, + -0.7607958912849426, + 1.4928451776504517, + 0.38265493512153625, + 1.2970150709152222, + -1.2964622974395752, + -0.5547049045562744, + -0.2585906684398651, + 1.3020356893539429, + 1.0737547874450684, + -1.5593467950820923, + -0.1617565155029297, + -1.4868489503860474, + 0.15243440866470337, + -1.4100472927093506, + 0.7991768717765808, + 1.0339381694793701, + 0.6228882074356079, + -2.8132376670837402, + 0.7685560584068298, + 1.8755284547805786 + ], + [ + 0.6393168568611145, + -0.2793915271759033, + 1.4690577983856201, + 0.4745514690876007, + 0.9989874362945557, + -0.7359552979469299, + -1.2171094417572021, + -0.39386802911758423, + 1.1646839380264282, + 0.6661151051521301, + -1.9162347316741943, + -0.43809595704078674, + -1.3973355293273926, + 0.7664861679077148, + -1.553026795387268, + 1.1995127201080322, + 1.285676121711731, + 0.822955846786499, + -2.574894666671753, + 0.3542059361934662, + 1.5986168384552002 + ], + [ + 0.26509445905685425, + -0.5443360805511475, + 1.2502005100250244, + 0.6112543940544128, + 0.924185037612915, + -1.0554168224334717, + -0.7927505373954773, + -0.07152043282985687, + 0.9228929281234741, + 1.545881986618042, + -1.6973015069961548, + -0.496639221906662, + -0.8727940320968628, + 0.5596445202827454, + -2.1939897537231445, + 1.239556074142456, + 1.4699029922485352, + -0.182618647813797, + -2.7263355255126953, + 0.976334810256958, + 1.0402796268463135 + ], + [ + 0.1353418529033661, + -1.1921336650848389, + 1.6913124322891235, + -0.43279919028282166, + 1.1331026554107666, + -1.8503069877624512, + -0.46699997782707214, + 0.5785945653915405, + 1.5516390800476074, + 0.38185298442840576, + -2.192314386367798, + -0.6304612159729004, + -1.6237419843673706, + -0.06383657455444336, + -1.2674099206924438, + 0.7967972755432129, + 1.2453874349594116, + 0.01532512903213501, + -2.6341590881347656, + 0.8366512656211853, + 1.9056174755096436 + ], + [ + 1.0916199684143066, + 0.23665857315063477, + 1.7737994194030762, + -0.8046588897705078, + 1.944507360458374, + -1.3606563806533813, + -0.022103488445281982, + 0.5769228935241699, + 1.0605480670928955, + 1.3100652694702148, + -1.9243974685668945, + 0.43040379881858826, + -1.9046732187271118, + -0.49132758378982544, + -1.9095335006713867, + 1.8850300312042236, + 1.366606593132019, + 0.37651723623275757, + -2.86037540435791, + 0.7753443717956543, + 2.072996139526367 + ], + [ + 0.7277938723564148, + 0.23822063207626343, + 0.4208102226257324, + -0.18717050552368164, + 1.1951282024383545, + -1.1622966527938843, + -0.6617333292961121, + -0.6339871883392334, + 0.8901646137237549, + 0.9977425932884216, + -1.8568902015686035, + 0.3291373550891876, + -0.9377886056900024, + 1.0705609321594238, + -0.90934818983078, + 1.7013394832611084, + 1.5952999591827393, + -0.05327540636062622, + -2.546459197998047, + 0.15792781114578247, + 1.2277774810791016 + ], + [ + 0.9283657073974609, + -1.421413779258728, + 1.6497737169265747, + -0.1984737515449524, + 0.9705085158348083, + -0.8917888402938843, + 0.026184558868408203, + 0.09181642532348633, + 0.6335604786872864, + 1.6531281471252441, + -1.4918432235717773, + -0.01735416054725647, + -0.5425058603286743, + 0.7866935133934021, + -0.7011347413063049, + 0.9797741770744324, + 1.6596866846084595, + -0.13051137328147888, + -2.692556858062744, + -0.05617690086364746, + 0.9771407842636108 + ] + ], + "predicted_scores": [ + 8.901857376098633, + 8.735929489135742, + 8.728458404541016, + 8.656224250793457, + 7.982328414916992, + 7.478723049163818, + 6.688599109649658, + 6.352318286895752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "1ca7c651b2dc4638488078318fe8aa293aa6f6da78672d440752daa9f2d4e8a3", + "stored_base_utility": 0.007898053154349327, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07743936777114868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07743936777114868 + }, + "base_progress": 0.07743936777114868, + "base_success": false, + "base_utility": 0.07743936777114868, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008875548839569092, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008875548839569092 + }, + { + "contact_quality": null, + "progress": 0.009828686714172363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009828686714172363 + }, + { + "contact_quality": null, + "progress": 0.0055730342864990234, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0055730342864990234 + }, + { + "contact_quality": null, + "progress": 0.008734583854675293, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008734583854675293 + }, + { + "contact_quality": null, + "progress": 0.008340597152709961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008340597152709961 + }, + { + "contact_quality": null, + "progress": 0.006452083587646484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006452083587646484 + }, + { + "contact_quality": null, + "progress": 0.007816612720489502, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007816612720489502 + }, + { + "contact_quality": null, + "progress": 0.007453759666532278, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007453759666532278 + } + ], + "candidate_progress": [ + 0.008875548839569092, + 0.009828686714172363, + 0.0055730342864990234, + 0.008734583854675293, + 0.008340597152709961, + 0.006452083587646484, + 0.007816612720489502, + 0.007453759666532278 + ], + "candidate_source_chart_ids": [ + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-02311c7783bc67e188b4", + "ms-02311c7783bc67e188b4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-0819eb89e627595e7e1b", + "generated_tangents": [ + [ + -0.17152051627635956, + -0.038122884929180145, + 1.340081810951233, + 0.47131815552711487, + 0.9815719127655029, + 1.6158288717269897, + 1.3484032154083252, + -0.5256087779998779, + -0.18259528279304504, + 1.419359803199768, + 0.17800158262252808, + 0.598058819770813, + 2.6912803649902344, + 1.049239158630371, + -0.5156311392784119, + -0.058238785713911057, + 0.9955028891563416, + 0.6501286029815674, + 0.5224418640136719, + 1.1980977058410645, + 0.792129635810852 + ], + [ + -0.18255217373371124, + -0.05061272904276848, + 1.2194602489471436, + 0.4751015603542328, + 0.9917681217193604, + 1.566107153892517, + 1.3416451215744019, + -0.4890477955341339, + -0.17969372868537903, + 1.2580264806747437, + 0.10395483672618866, + 0.5031521916389465, + 2.7228047847747803, + 0.972133457660675, + -0.5049965381622314, + -0.07441367208957672, + 0.9628568291664124, + 0.6687726974487305, + 0.40544599294662476, + 1.200563669204712, + 0.7176809310913086 + ], + [ + -0.45789438486099243, + -0.09919900447130203, + 0.6399000883102417, + 0.706047773361206, + 0.4280887842178345, + 1.7801594734191895, + 1.6320436000823975, + -0.38427188992500305, + -0.41299012303352356, + 1.3940125703811646, + 0.6254867911338806, + 0.8550894856452942, + 3.0050437450408936, + 2.065139055252075, + -0.4582861065864563, + -0.22844375669956207, + 0.9927664399147034, + 0.47996097803115845, + 0.6306631565093994, + 1.4493701457977295, + 0.7380000352859497 + ], + [ + -0.35566914081573486, + 0.007236521691083908, + 1.0994839668273926, + 0.7574254274368286, + 1.263378381729126, + 1.4886584281921387, + 1.5016376972198486, + -0.5011583566665649, + -0.3243262767791748, + 1.4724236726760864, + 0.18251696228981018, + 0.6980839967727661, + 2.178234338760376, + 1.044251799583435, + -0.28006064891815186, + -0.06012527272105217, + 0.7044966816902161, + 0.8577057719230652, + 0.32886219024658203, + 0.8952638506889343, + 0.3278191089630127 + ], + [ + -0.3057391047477722, + 0.004349537193775177, + 1.0532543659210205, + 0.7294998168945312, + 1.3418724536895752, + 1.5626895427703857, + 1.5369725227355957, + -0.6195144653320312, + -0.22753973305225372, + 1.4388295412063599, + 0.13561950623989105, + 0.769778311252594, + 2.3072755336761475, + 1.0169832706451416, + -0.24158892035484314, + -0.18682266771793365, + 0.6608469486236572, + 0.7943220734596252, + 0.401975154876709, + 0.9778363108634949, + 0.5728996992111206 + ], + [ + -0.26647835969924927, + -0.11081273853778839, + 0.4576881229877472, + 0.583040714263916, + 1.4537286758422852, + 1.577552080154419, + 1.454306960105896, + -0.5423310995101929, + -0.32421642541885376, + 1.5886645317077637, + -0.06395097821950912, + 1.2373117208480835, + 2.02956223487854, + 0.7510688304901123, + -0.2381059229373932, + -0.17396648228168488, + 0.7958939075469971, + 0.6796910762786865, + 0.395303875207901, + 1.1233546733856201, + 0.04672019183635712 + ], + [ + -0.20528943836688995, + 0.10780379176139832, + 1.2915796041488647, + 0.19357836246490479, + 0.6346437931060791, + 1.8733876943588257, + 0.9660515785217285, + -0.44898679852485657, + -0.28768423199653625, + 1.721643090248108, + 0.4985387921333313, + 0.7298404574394226, + 2.0187413692474365, + 0.9921810030937195, + 0.08743637055158615, + -0.28891173005104065, + 0.9795303344726562, + 0.334699809551239, + 0.758746862411499, + 0.8109932541847229, + 0.305303156375885 + ], + [ + -0.3197581171989441, + 0.12134270370006561, + 1.3069734573364258, + 0.22892767190933228, + 0.7294712662696838, + 1.7926363945007324, + 0.8312811255455017, + -0.5016231536865234, + -0.31257396936416626, + 1.6537065505981445, + 0.5000704526901245, + 0.818077802658081, + 1.9235714673995972, + 1.0741196870803833, + 0.11200298368930817, + -0.30520081520080566, + 0.9781718850135803, + 0.3617241382598877, + 0.7511255741119385, + 0.8701092600822449, + 0.3813852071762085 + ] + ], + "generated_utilities": [ + 0.008875548839569092, + 0.009828686714172363, + 0.0055730342864990234, + 0.008734583854675293, + 0.008340597152709961, + 0.006452083587646484, + 0.007816612720489502, + 0.007453759666532278 + ], + "group_id": "ms-0819eb89e627595e7e1b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07638250291347504, + 0.11176031827926636, + 0.139996737241745, + 0.06282445043325424, + 0.08466696739196777, + 0.0763862133026123, + 0.05953681468963623, + 0.06854937970638275, + 0.06834080070257187, + 0.09309522807598114, + 0.14572584629058838, + 0.07182137668132782, + 0.020047685131430626, + 0.05106111615896225, + 0.03756682202219963, + 0.061605654656887054 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.04522436857223511, + -0.3834587037563324, + 2.32004976272583, + 1.6386330127716064, + 1.508955717086792, + 1.681371808052063, + 2.9060468673706055, + -0.5984516143798828, + 0.36428770422935486, + 0.7442611455917358, + 1.163742184638977, + 0.4921322464942932, + 1.9586588144302368, + 3.335318088531494, + -0.02929070219397545, + 0.5442408919334412, + 2.4529502391815186, + 0.17471881210803986, + 1.804263710975647, + 1.5498225688934326, + 1.9392809867858887 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.19627264142036438, + 0.015629824250936508, + 0.723259687423706, + 0.0313839316368103, + 0.02149224281311035, + 0.7142634391784668, + 0.0, + -0.29767167568206787, + -0.10304851830005646, + 0.6168417930603027, + 0.0271989107131958, + 0.369077205657959, + 1.5924487113952637, + 0.0, + -0.11617732048034668, + -0.1419914811849594, + 0.9400725364685059, + -0.04139697551727295, + 0.7166774272918701, + 1.092864990234375, + 0.0 + ], + [ + -0.18197131156921387, + 0.30914804339408875, + 1.4557228088378906, + 1.4375734329223633, + 0.9600651264190674, + 1.7448713779449463, + 3.031123399734497, + -0.6073551177978516, + -0.4749099016189575, + 1.3016932010650635, + 0.32392996549606323, + 0.6961254477500916, + 2.1961381435394287, + 3.730100154876709, + -0.6154190897941589, + -0.3908892273902893, + 2.0994153022766113, + 0.18937554955482483, + 1.5752670764923096, + 2.2457275390625, + 1.5629570484161377 + ] + ], + "predicted_scores": [ + 7.262422561645508, + 7.140332221984863, + 6.653880596160889, + 6.33031702041626, + 5.919139862060547, + 5.892663955688477, + 4.470665454864502, + 4.338894367218018 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "582daf047d78a1b01018d54f5798143c4be6129fd7b6cbc025fc85818e519f3c", + "stored_base_utility": 0.07638250291347504, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19400137662887573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19400137662887573 + }, + "base_progress": 0.19400137662887573, + "base_success": false, + "base_utility": 0.19400137662887573, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03749382495880127, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03749382495880127 + }, + { + "contact_quality": null, + "progress": 0.04604196548461914, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04604196548461914 + }, + { + "contact_quality": null, + "progress": 0.03564453125, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03564453125 + }, + { + "contact_quality": null, + "progress": 0.0471922941505909, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0471922941505909 + }, + { + "contact_quality": null, + "progress": 0.033758122473955154, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.033758122473955154 + }, + { + "contact_quality": null, + "progress": 0.04913504794239998, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04913504794239998 + }, + { + "contact_quality": null, + "progress": 0.04759601876139641, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04759601876139641 + }, + { + "contact_quality": null, + "progress": 0.0932278037071228, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0932278037071228 + } + ], + "candidate_progress": [ + 0.03749382495880127, + 0.04604196548461914, + 0.03564453125, + 0.0471922941505909, + 0.033758122473955154, + 0.04913504794239998, + 0.04759601876139641, + 0.0932278037071228 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "generated_tangents": [ + [ + 1.8905562162399292, + 0.3944854736328125, + -1.7180052995681763, + -0.1349945217370987, + -0.5599319934844971, + -4.130958080291748, + -2.674062490463257, + -1.320008635520935, + 0.0017402367666363716, + 1.5784907341003418, + 1.1209584474563599, + -0.13940736651420593, + 0.5197409391403198, + -0.3171689510345459, + -0.6695423722267151, + -0.19212783873081207, + 0.45296603441238403, + 1.6345059871673584, + 0.11391966044902802, + 0.6570355892181396, + 0.7228900194168091 + ], + [ + 1.8707126379013062, + 0.41409242153167725, + -1.839680790901184, + 0.03306730091571808, + -0.2336004674434662, + -4.06272554397583, + -2.651236057281494, + -1.1379244327545166, + -0.06307099014520645, + 1.6282907724380493, + 1.0120092630386353, + 0.0491805374622345, + 0.14716574549674988, + -0.8551839590072632, + -0.5584245324134827, + -0.11800333857536316, + 0.24090628325939178, + 1.3936920166015625, + 0.4446020722389221, + 0.44069159030914307, + 0.3575472831726074 + ], + [ + 2.0141654014587402, + 0.5335431098937988, + -1.2890888452529907, + -0.20137447118759155, + -0.5691367983818054, + -4.134076118469238, + -3.181198835372925, + -1.1823093891143799, + 0.06391821801662445, + 1.7136962413787842, + 1.2516483068466187, + 0.49389082193374634, + 0.5434608459472656, + -0.962183952331543, + -0.812558650970459, + -0.17930343747138977, + 0.3569537401199341, + 1.2693164348602295, + 0.5134433507919312, + 1.1229405403137207, + 0.39776551723480225 + ], + [ + 1.9669517278671265, + 0.518299400806427, + -1.7549928426742554, + -0.16333818435668945, + -0.5614626407623291, + -3.7372679710388184, + -3.6689260005950928, + -1.1913681030273438, + 0.04750648885965347, + 1.492919683456421, + 1.0730613470077515, + 0.5416345596313477, + 0.21158185601234436, + -0.44773614406585693, + -0.7684616446495056, + -0.14286024868488312, + 0.25122955441474915, + 1.5649583339691162, + 0.3667457103729248, + 0.665382444858551, + 0.47018304467201233 + ], + [ + 1.797508955001831, + 0.5287662148475647, + -1.608214259147644, + -0.18494614958763123, + -0.8209599852561951, + -3.972693920135498, + -3.305769205093384, + -1.0902042388916016, + -0.05148264020681381, + 1.5328097343444824, + 1.3380097150802612, + 0.44910597801208496, + 0.16804906725883484, + -0.22458527982234955, + -0.8841680288314819, + -0.2944192886352539, + 0.6147395372390747, + 1.554239273071289, + 0.5591936707496643, + 0.4726758599281311, + 0.06745950877666473 + ], + [ + 1.8798527717590332, + 0.47933393716812134, + -1.692487120628357, + -0.22272565960884094, + -0.6054636240005493, + -3.8631744384765625, + -3.3329451084136963, + -1.049681544303894, + 0.03719450533390045, + 1.4770680665969849, + 1.1387784481048584, + 0.5392275452613831, + 0.28753021359443665, + -0.5957773923873901, + -0.7329325079917908, + -0.2532622516155243, + 0.3489782512187958, + 1.4910532236099243, + 0.4014784097671509, + 0.6709848642349243, + 0.2531224489212036 + ], + [ + 1.7510665655136108, + 0.48223114013671875, + -1.705487847328186, + -0.32500597834587097, + -0.6420989036560059, + -3.8617329597473145, + -3.319277048110962, + -1.0553152561187744, + 0.03045062907040119, + 1.4631993770599365, + 1.137038230895996, + 0.5703677535057068, + 0.2730879485607147, + -0.5021155476570129, + -0.7537959814071655, + -0.2491789013147354, + 0.31574124097824097, + 1.545933485031128, + 0.5048801302909851, + 0.7393592000007629, + 0.5303146839141846 + ], + [ + 1.8198521137237549, + 0.3624112010002136, + -1.5549098253250122, + -0.5984248518943787, + -0.322936475276947, + -3.43082857131958, + -3.303149700164795, + -1.0558255910873413, + 0.124747633934021, + 1.0588371753692627, + 1.1908138990402222, + 0.7646531462669373, + 0.4460482895374298, + -0.20368830859661102, + -0.6139333248138428, + -0.40287846326828003, + 0.13066786527633667, + 1.6527705192565918, + 0.13463273644447327, + 0.49561959505081177, + 0.5370721817016602 + ] + ], + "generated_utilities": [ + 0.03749382495880127, + 0.04604196548461914, + 0.03564453125, + 0.0471922941505909, + 0.033758122473955154, + 0.04913504794239998, + 0.04759601876139641, + 0.0932278037071228 + ], + "group_id": "ms-0dee962a5901fb8b5bbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23161475360393524, + 0.23611454665660858, + 0.1380448043346405, + 2.0, + 0.25444933772087097, + 0.25831863284111023, + 0.22753827273845673, + 0.2483263909816742, + 0.21622240543365479, + 0.17534737288951874, + 0.23816657066345215, + 0.09714888036251068, + 0.05844185873866081, + 0.12679120898246765, + 0.0472835898399353, + 0.03277532383799553 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7612967491149902, + 0.192471444606781, + -0.2699308395385742, + 0.017076104879379272, + -0.14400714635849, + -1.9032442569732666, + 0.0, + -0.28746497631073, + -0.2857179641723633, + -0.09024286270141602, + -0.03538453206419945, + 0.06432074308395386, + -0.2535901665687561, + 0.0, + 0.06180626153945923, + 0.16256888210773468, + -0.031329914927482605, + 1.563896656036377, + 0.04795600473880768, + 0.02485179901123047, + 0.0 + ], + [ + 1.2132649421691895, + 0.536715567111969, + -1.4273607730865479, + -1.0538015365600586, + -0.5170869827270508, + -3.4203505516052246, + -2.3889083862304688, + -0.7717514634132385, + -1.1465628147125244, + 0.8393621444702148, + -0.4419952630996704, + -0.23069629073143005, + -0.0637764036655426, + -1.5193647146224976, + -1.0273863077163696, + 0.39812424778938293, + -0.02874515950679779, + 2.7962026596069336, + 0.2370568960905075, + 1.5884850025177002, + 0.9987253546714783 + ], + [ + 2.1451404094696045, + 0.09629477560520172, + -1.8500688076019287, + 0.012462526559829712, + 0.13087880611419678, + -2.4062256813049316, + -3.303264856338501, + -1.3855981826782227, + -0.0292893648147583, + -0.17000514268875122, + 0.0005041323602199554, + -0.7487195134162903, + 0.06868582963943481, + -2.107095718383789, + -0.5890907645225525, + 0.1777246743440628, + -0.5576009750366211, + 2.767899990081787, + -0.3420543074607849, + 1.1357430219650269, + 1.8840694427490234 + ], + [ + 2.032343626022339, + 0.4649246335029602, + -0.5597869753837585, + 0.37935349345207214, + 0.14823919534683228, + -3.389478921890259, + -1.972822904586792, + -0.0014220476150512695, + -0.6147069334983826, + 0.9483420252799988, + 0.641323983669281, + -0.7033213973045349, + 0.4629128575325012, + -1.6774024963378906, + -1.1420938968658447, + 0.5891244411468506, + -0.712251603603363, + 2.1924612522125244, + 0.18694229423999786, + 0.2795591354370117, + 2.4723148345947266 + ], + [ + 2.299381732940674, + -0.6742787957191467, + -0.886760950088501, + -0.1637115478515625, + 0.32663029432296753, + -1.9267126321792603, + -1.9424545764923096, + -0.3586595058441162, + -1.1000827550888062, + 0.45532336831092834, + -0.4314403235912323, + -0.12869995832443237, + -0.032513588666915894, + -2.046614170074463, + -1.2942235469818115, + 0.3962012231349945, + 0.797533392906189, + 2.202794313430786, + 0.4938191771507263, + 0.15863806009292603, + 1.745660662651062 + ], + [ + 2.107815742492676, + 0.963295042514801, + -1.2979395389556885, + -1.0134333372116089, + -1.319197177886963, + -3.176518440246582, + -2.4782180786132812, + -1.3465216159820557, + -1.0473096370697021, + -0.22631770372390747, + 0.7881518006324768, + -1.4033093452453613, + 0.3855947256088257, + -1.3816941976547241, + -1.1257284879684448, + -0.7653472423553467, + 0.6845007538795471, + 2.9796371459960938, + 0.1967305988073349, + 1.4200057983398438, + 1.2843091487884521 + ], + [ + 1.7303649187088013, + 1.0728741884231567, + -1.1676844358444214, + -0.20355769991874695, + -0.8773125410079956, + -3.2701919078826904, + -2.1479928493499756, + -0.044036924839019775, + -1.252698302268982, + -0.01393243670463562, + 0.6447213292121887, + -0.7088868618011475, + 0.11592492461204529, + -2.1431162357330322, + -0.8284967541694641, + 0.4642922580242157, + -0.5207862854003906, + 1.8997660875320435, + -0.6910213828086853, + 0.2906065583229065, + 1.4081878662109375 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7612967491149902, + -0.5331565737724304, + -0.2699308395385742, + -0.20421931147575378, + -0.22613269090652466, + -1.9032442569732666, + 0.0, + 0.09110367298126221, + 0.009229481220245361, + 0.004811525344848633, + 0.656868577003479, + -0.28485387563705444, + -0.31901538372039795, + 0.0, + 0.08918565511703491, + 0.3355104923248291, + 0.026827186346054077, + 1.563896656036377, + 0.13480885326862335, + 0.02485179901123047, + 0.0 + ] + ], + "predicted_scores": [ + 9.025599479675293, + 8.796256065368652, + 8.275415420532227, + 8.15134048461914, + 7.997213840484619, + 7.892197608947754, + 7.640031814575195, + 7.517190456390381 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "73f2826502af62b561f13c32614947588918c3deb631165739e66d787008fa98", + "stored_base_utility": 0.23161475360393524, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.04583283141255379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04583283141255379 + }, + "base_progress": 0.04583283141255379, + "base_success": false, + "base_utility": 0.04583283141255379, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05473164841532707, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05473164841532707 + }, + { + "contact_quality": null, + "progress": 0.0898062214255333, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0898062214255333 + }, + { + "contact_quality": null, + "progress": 0.09243235737085342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09243235737085342 + }, + { + "contact_quality": null, + "progress": 0.04779624938964844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04779624938964844 + }, + { + "contact_quality": null, + "progress": 0.1050744280219078, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1050744280219078 + }, + { + "contact_quality": null, + "progress": 0.08136358112096786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08136358112096786 + }, + { + "contact_quality": null, + "progress": 0.10417115688323975, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10417115688323975 + }, + { + "contact_quality": null, + "progress": 0.05421211197972298, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05421211197972298 + } + ], + "candidate_progress": [ + 0.05473164841532707, + 0.0898062214255333, + 0.09243235737085342, + 0.04779624938964844, + 0.1050744280219078, + 0.08136358112096786, + 0.10417115688323975, + 0.05421211197972298 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-7a044cb6c7bb03f3ea10", + "ms-38dff128761905bb1507" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-126eb8b36769f9ba4671", + "generated_tangents": [ + [ + 1.6391998529434204, + -0.10492207109928131, + -1.2173104286193848, + 0.06312951445579529, + 0.034627094864845276, + -3.809929609298706, + -2.655278205871582, + 1.55155611038208, + -0.049403004348278046, + 0.14256125688552856, + 0.07096967101097107, + 1.5670653581619263, + -0.8856337070465088, + -0.3646290898323059, + -0.3188478946685791, + 0.23871849477291107, + 0.5090757608413696, + 0.578620970249176, + 0.6506721377372742, + 1.3572771549224854, + 1.5281459093093872 + ], + [ + 1.632312297821045, + -0.073216512799263, + -1.279280185699463, + 0.04200807213783264, + 0.13810274004936218, + -3.7365379333496094, + -2.5355067253112793, + 1.5218451023101807, + -0.11356581747531891, + 0.07684358209371567, + 0.02117886021733284, + 1.4602035284042358, + -0.9178345203399658, + -0.33722400665283203, + -0.41049888730049133, + 0.2655564248561859, + 0.510861873626709, + 0.48537325859069824, + 0.6985797882080078, + 1.3627231121063232, + 1.4742491245269775 + ], + [ + 1.3266669511795044, + -0.21584570407867432, + -1.0917428731918335, + 0.5911462306976318, + -0.091316819190979, + -3.1719870567321777, + -2.0689079761505127, + 1.423898458480835, + -0.051188841462135315, + 0.0882360190153122, + 0.01707397773861885, + 1.9493176937103271, + -0.2595904469490051, + -0.4032917022705078, + -0.5435537099838257, + 0.18033447861671448, + 0.6712300777435303, + 0.5846545696258545, + 0.4349537491798401, + 1.231971025466919, + 0.9769790172576904 + ], + [ + 1.688482642173767, + -0.14923816919326782, + -1.429479718208313, + 0.03564286231994629, + 0.049825385212898254, + -3.9505109786987305, + -2.792681932449341, + 1.6349338293075562, + -0.0660700872540474, + 0.06310125440359116, + 0.04371916875243187, + 1.4827817678451538, + -0.9865245819091797, + -0.6827672719955444, + -0.2880074679851532, + 0.3062531352043152, + 0.49833357334136963, + 0.47552281618118286, + 0.5298202037811279, + 1.146613597869873, + 1.6703341007232666 + ], + [ + 1.4574108123779297, + 0.0944143533706665, + -0.8780643939971924, + 0.31943944096565247, + 0.07821029424667358, + -3.033165454864502, + -1.998957633972168, + 1.0657954216003418, + -0.2797210216522217, + -0.008155487477779388, + -0.14730030298233032, + 1.6937888860702515, + -0.2869578003883362, + -0.6444816589355469, + -0.5542390942573547, + 0.1780693531036377, + 0.6108785271644592, + -0.19697339832782745, + 0.4930592179298401, + 0.9760110974311829, + 1.555848240852356 + ], + [ + 1.4546087980270386, + 0.03867151588201523, + -0.8583587408065796, + 0.4685031473636627, + 0.006796449422836304, + -3.1022281646728516, + -1.964062213897705, + 1.0652985572814941, + -0.2551897168159485, + -0.042359210550785065, + -0.25985828042030334, + 1.5183671712875366, + -0.3688545525074005, + -0.6041058897972107, + -0.6142784357070923, + 0.20553471148014069, + 0.5365957617759705, + -0.24196593463420868, + 0.5062716007232666, + 0.869109570980072, + 1.450219988822937 + ], + [ + 1.4247417449951172, + -0.21659106016159058, + -1.1823608875274658, + 0.8322272300720215, + -0.0005451738834381104, + -3.0540988445281982, + -2.1361265182495117, + 1.3151968717575073, + -0.03238707035779953, + 0.10789983719587326, + 0.1868794858455658, + 2.102938175201416, + -0.4955365061759949, + -0.32517802715301514, + -0.41326475143432617, + 0.08100670576095581, + 0.5753326416015625, + 0.6134408712387085, + 0.4230189919471741, + 1.4242136478424072, + 0.9708864688873291 + ], + [ + 1.3358324766159058, + -0.11553329974412918, + -1.3399983644485474, + 0.6663671731948853, + 0.23060394823551178, + -2.7645184993743896, + -2.443643808364868, + 2.059368848800659, + -0.024196317419409752, + -0.31755587458610535, + 0.39442718029022217, + 2.483910083770752, + -1.1303515434265137, + -0.4766498804092407, + -0.5132682919502258, + 0.19200073182582855, + 0.46852320432662964, + 0.2817460298538208, + 0.6593990325927734, + 1.3848536014556885, + 0.8981823921203613 + ] + ], + "generated_utilities": [ + 0.05473164841532707, + 0.0898062214255333, + 0.09243235737085342, + 0.04779624938964844, + 0.1050744280219078, + 0.08136358112096786, + 0.10417115688323975, + 0.05421211197972298 + ], + "group_id": "ms-126eb8b36769f9ba4671", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05304628610610962, + 0.14190539717674255, + 0.1985912322998047, + 0.1210482120513916, + 0.18570420145988464, + 0.05259696766734123, + 0.1827520728111267, + 0.15291261672973633, + 0.07772710174322128, + 0.15551692247390747, + 0.08802877366542816, + 0.07544025033712387, + 0.08058679103851318, + 0.10843702405691147, + 0.023751219734549522, + 0.00727109145373106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7799564599990845, + -0.0863567665219307, + -0.04370701313018799, + -0.04603701829910278, + -0.003975361585617065, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.02617853879928589, + 0.009054642170667648, + -0.050068795680999756, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + -0.007249712944030762, + 0.0760425329208374, + 1.0404269695281982, + -0.04351449012756348, + -0.010379016399383545, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + 0.005863763391971588, + -0.04370701313018799, + 0.010433673858642578, + -0.05825299024581909, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.22243881225585938, + -0.007246788591146469, + 0.19961372017860413, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.11177664995193481, + -0.1665668487548828, + 1.0404269695281982, + -0.09226274490356445, + -0.18212640285491943, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + -0.7482962012290955, + -0.04370701313018799, + -0.2370874285697937, + 0.3763168752193451, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + -0.441777765750885, + -0.44057023525238037, + -0.6443575024604797, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.008161306381225586, + 0.22021520137786865, + 1.0404269695281982, + -0.09226274490356445, + -0.35497862100601196, + 1.0522968769073486, + 0.0 + ], + [ + 1.7799564599990845, + 0.04087676852941513, + -1.043707013130188, + -0.8482787609100342, + 0.3640388548374176, + -3.3430333137512207, + -2.8715221881866455, + 1.8326367139816284, + -0.4197058081626892, + -0.040397901087999344, + -0.37471577525138855, + 1.7451434135437012, + -1.6306462287902832, + -0.7785218954086304, + -0.5121169090270996, + -0.7797847986221313, + 2.0404269695281982, + 0.9077372550964355, + 0.645021378993988, + 2.0522968769073486, + 1.2749258279800415 + ], + [ + 1.6519862413406372, + 0.1747584044933319, + -0.9129671454429626, + -0.6869257092475891, + 0.12357090413570404, + -3.2319328784942627, + -2.704270839691162, + 1.7347972393035889, + -0.40612348914146423, + 0.23568139970302582, + -0.3912501931190491, + 1.8630532026290894, + -1.8548930883407593, + -0.9794526100158691, + -0.25177696347236633, + -0.6884964108467102, + 2.237109422683716, + 1.1246912479400635, + 0.7490227222442627, + 2.320941686630249, + 1.2039268016815186 + ], + [ + 1.7142105102539062, + 0.3941279351711273, + -1.021533489227295, + -1.1548089981079102, + 0.5052449107170105, + -3.714951992034912, + -3.04028058052063, + 1.529634952545166, + -0.22215746343135834, + 0.19690367579460144, + -0.12262976169586182, + 1.4847477674484253, + -1.490212082862854, + -0.7427945137023926, + -0.7506002187728882, + -0.9221256375312805, + 1.9211885929107666, + 0.5114957094192505, + 0.30887648463249207, + 2.4437777996063232, + 1.3625030517578125 + ], + [ + 1.3140592575073242, + -0.1129835918545723, + -0.9130957722663879, + -1.1089756488800049, + 0.7560886144638062, + -3.3889732360839844, + -3.1182656288146973, + 1.7436503171920776, + -0.9962512254714966, + 0.43109408020973206, + -0.8453629016876221, + 1.2057833671569824, + -2.194112777709961, + -0.22479140758514404, + -0.7258806824684143, + -0.9702881574630737, + 1.8160195350646973, + 0.649415135383606, + 0.6846981048583984, + 2.1997463703155518, + 1.7655411958694458 + ], + [ + 2.6430320739746094, + -0.7696453332901001, + -1.765415906906128, + -0.8281408548355103, + -0.1440155804157257, + -3.3335886001586914, + -3.398141860961914, + 2.4138498306274414, + 0.47537559270858765, + -0.9514593482017517, + 0.2674855887889862, + 2.6307411193847656, + -0.7876039743423462, + -0.35013434290885925, + -0.7511636018753052, + -0.04397457838058472, + 2.123702049255371, + 1.1604151725769043, + 0.7323322892189026, + 1.88338303565979, + 1.5078264474868774 + ] + ], + "predicted_scores": [ + 6.030942440032959, + 5.877603054046631, + 5.745731830596924, + 5.655466556549072, + 5.458425521850586, + 5.389834403991699, + 5.023163318634033, + 4.013148307800293 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "2390fc23ce8b50bdcdcbb1ab5f0583d2dad2b94fb871db73b1a8401265f59987", + "stored_base_utility": 0.05304628610610962, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.027269184589385986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027269184589385986 + }, + { + "contact_quality": null, + "progress": 0.04800540208816528, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04800540208816528 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.030054310336709023, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030054310336709023 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.027269184589385986, + 0.04800540208816528, + 1.0, + 1.0, + 1.0, + 0.030054310336709023, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-1f19fd22562f87bdb776", + "ms-c1620854ba52f09e24d5", + "ms-067cac2e4dbeddb12239", + "ms-067cac2e4dbeddb12239", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-cad6645f8175e38ad155", + "ms-babaeb20129c46551281" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-16661b2fd2c85f57a66c", + "generated_tangents": [ + [ + 0.07182317972183228, + -0.25195980072021484, + 0.8839491009712219, + -0.1986687183380127, + 0.3855177164077759, + 0.746080756187439, + 2.4408936500549316, + 1.2690387964248657, + -0.36878761649131775, + -0.40345215797424316, + 0.591017484664917, + -0.168995201587677, + 1.0220140218734741, + 1.8888201713562012, + 0.7933868169784546, + -0.24187451601028442, + -1.1002981662750244, + -0.26305556297302246, + -0.1259298026561737, + -1.0836772918701172, + -2.1015565395355225 + ], + [ + 0.10315769910812378, + -0.20259320735931396, + 0.34571170806884766, + -0.8588720560073853, + 0.36758148670196533, + 1.6789014339447021, + 2.649001121520996, + 1.201626181602478, + -0.19718889892101288, + -0.2105022370815277, + 0.14505070447921753, + 0.018547385931015015, + 1.5460213422775269, + 1.587507724761963, + 0.8956091403961182, + -0.18251945078372955, + -0.8965319991111755, + -0.22472302615642548, + 0.241672083735466, + -1.4455063343048096, + -1.3432704210281372 + ], + [ + 0.10031376034021378, + -0.11849141120910645, + 0.7268736958503723, + -0.7146854400634766, + 0.5790175199508667, + 0.8996216058731079, + 2.2201876640319824, + 1.3257726430892944, + -0.1206386536359787, + -0.713047206401825, + 0.5406262874603271, + -0.8258600234985352, + 1.211934208869934, + 1.4730538129806519, + 0.9376559257507324, + -0.0715533196926117, + -1.210080862045288, + -0.12432421743869781, + -0.035953670740127563, + -0.8264876008033752, + -1.9952523708343506 + ], + [ + 0.24770130217075348, + -0.0796661376953125, + 0.21971911191940308, + -1.67794930934906, + -0.008098766207695007, + 1.618289828300476, + 2.0531442165374756, + 1.5219308137893677, + -0.09871240705251694, + -0.47520357370376587, + 0.4357303977012634, + -0.5788902044296265, + 1.6422539949417114, + 1.752223253250122, + 0.9891992211341858, + -0.20013757050037384, + -1.398559808731079, + -0.343084454536438, + 0.010496914386749268, + -1.246861457824707, + -2.188300371170044 + ], + [ + -0.07818293571472168, + -0.19883371889591217, + 0.436402291059494, + -0.05419538915157318, + 0.4581429958343506, + 1.3210529088974, + 2.8119122982025146, + 1.2695547342300415, + -0.18630556762218475, + -0.5067341327667236, + 0.2933301031589508, + -0.00015692412853240967, + 1.197396993637085, + 1.4097285270690918, + 1.2938770055770874, + -0.1203649491071701, + -0.7299903035163879, + -0.07005701959133148, + 0.002072364091873169, + -1.2534825801849365, + -1.885866641998291 + ], + [ + -0.08446246385574341, + -0.21925637125968933, + 0.6055530905723572, + -0.021726801991462708, + 0.5052362084388733, + 1.4888149499893188, + 2.800708055496216, + 1.181645154953003, + -0.1087164506316185, + -0.4942442774772644, + 0.2532975375652313, + 0.01925988495349884, + 1.042837381362915, + 1.2664003372192383, + 1.3407131433486938, + -0.10998737812042236, + -0.6873182654380798, + -0.14442871510982513, + -0.03526574373245239, + -1.0848643779754639, + -1.9860200881958008 + ], + [ + 0.08374780416488647, + -0.22458630800247192, + 0.4597207307815552, + -0.24619752168655396, + 0.026582494378089905, + 1.2279653549194336, + 2.9190549850463867, + 1.5459320545196533, + -0.16730205714702606, + -0.5966821908950806, + 0.24363699555397034, + -0.008007138967514038, + 1.217457890510559, + 1.8039884567260742, + 1.1827164888381958, + -0.12527476251125336, + -0.7073969841003418, + 0.08288136124610901, + -0.025503098964691162, + -1.0367083549499512, + -2.144688129425049 + ], + [ + -0.0046021342277526855, + -0.0868464931845665, + 0.8236863613128662, + -0.37719231843948364, + 0.7548291087150574, + 1.6137198209762573, + 2.9738709926605225, + 1.1011310815811157, + -0.1957755833864212, + -0.6226890087127686, + 0.012403644621372223, + 0.06340701878070831, + 1.4640778303146362, + 1.4870837926864624, + 1.3600130081176758, + 0.010157547891139984, + -1.0214133262634277, + -0.11970458924770355, + -0.16698560118675232, + -1.4716496467590332, + -2.4429054260253906 + ] + ], + "generated_utilities": [ + 2.0, + 0.027269184589385986, + 0.04800540208816528, + 2.0, + 2.0, + 2.0, + 0.030054310336709023, + 2.0 + ], + "group_id": "ms-16661b2fd2c85f57a66c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23635749518871307, + 0.1864171028137207, + 0.18280422687530518, + 0.16576561331748962, + 0.2967458665370941, + 0.2833873927593231, + 0.2587948143482208, + 0.2695174813270569, + 0.2239263951778412, + 0.15686756372451782, + 0.2505500912666321, + 0.12250886857509613, + 0.16453635692596436, + 2.0, + 0.13450095057487488, + 0.11914549767971039 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.12099093198776245, + 0.06886856257915497, + 0.2952747344970703, + -0.1764048933982849, + -0.12411242723464966, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + -0.046127498149871826, + 0.12581980228424072, + 0.042072445154190063, + -0.06942430138587952, + 0.7517287731170654, + 0.0, + 0.690804123878479, + -0.06023237854242325, + 0.049633681774139404, + 0.01943105459213257, + 0.013327360153198242, + -0.8525059223175049, + 0.0 + ], + [ + 0.3599843978881836, + -0.34296315908432007, + 0.2952747344970703, + -0.3797188997268677, + -0.41771262884140015, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + 0.06634362041950226, + 0.138022780418396, + -0.2342309057712555, + -0.36017337441444397, + 0.7517287731170654, + 0.0, + 0.690804123878479, + 0.10550708323717117, + -0.35431206226348877, + 0.16382712125778198, + 0.06674730777740479, + -0.8525059223175049, + 0.0 + ], + [ + 0.567882239818573, + -0.4729353189468384, + 1.6256722211837769, + -1.5372763872146606, + 1.0487556457519531, + 1.5327277183532715, + 2.2847697734832764, + 2.6703484058380127, + 0.4013517200946808, + -1.3629915714263916, + 0.6385155916213989, + 0.2664203345775604, + 1.2691267728805542, + 1.2942520380020142, + 2.2084813117980957, + 0.6982719302177429, + -0.8422143459320068, + 1.070263385772705, + -0.9558742046356201, + -1.3996182680130005, + -1.4242117404937744 + ], + [ + 0.6830785274505615, + -0.5584439635276794, + 0.841848611831665, + -0.5293771624565125, + -0.14354348182678223, + 2.097932815551758, + 2.9733188152313232, + 2.603226661682129, + -0.1639159619808197, + -0.918163537979126, + 1.002030849456787, + 0.40132570266723633, + 1.1751198768615723, + 1.455662727355957, + 2.3202755451202393, + -0.18584474921226501, + -1.3483099937438965, + 0.0660635232925415, + 0.4330971837043762, + -1.1989169120788574, + -2.0896859169006348 + ], + [ + 0.44418102502822876, + 0.07469291985034943, + 1.6960139274597168, + -1.329486608505249, + 1.3067576885223389, + 2.2694292068481445, + 2.0299551486968994, + 1.6317954063415527, + 0.7472026348114014, + -0.12518703937530518, + 0.3035789430141449, + -0.5889836549758911, + 2.345867395401001, + 1.9470158815383911, + 0.8398547172546387, + -0.6090301871299744, + -1.5060018301010132, + 0.5695335865020752, + -0.6025367975234985, + -1.8703374862670898, + -2.728482246398926 + ], + [ + 0.31904691457748413, + 0.04995492100715637, + 1.3823881149291992, + -0.5106721520423889, + -0.2236303687095642, + 0.9921484589576721, + 1.5933295488357544, + 2.741320848464966, + -0.21808990836143494, + -0.7775652408599854, + 1.224307894706726, + -0.08168366551399231, + 2.692474842071533, + 2.269430160522461, + 2.3270838260650635, + -0.2901480495929718, + -1.2098140716552734, + -0.07570821046829224, + -1.1340655088424683, + -1.9999847412109375, + -1.6802332401275635 + ], + [ + -0.6257733106613159, + 0.3840647339820862, + 0.9012271165847778, + -1.248098611831665, + -0.1650761365890503, + 2.7645747661590576, + 2.7588071823120117, + 3.0172548294067383, + 0.4120163321495056, + -1.6110656261444092, + -0.10423526167869568, + 0.5334746241569519, + 1.7809263467788696, + 0.5690644383430481, + 1.5881801843643188, + 0.8860509991645813, + -1.1502506732940674, + 0.7821243405342102, + -0.549727201461792, + -1.326378345489502, + -1.5100535154342651 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2773973345756531, + -0.13796104490756989, + 1.2952747344970703, + -1.0323810577392578, + 0.5605664849281311, + 1.8606208562850952, + 2.439758777618408, + 2.353257894515991, + 0.24642756581306458, + -0.861977219581604, + 0.4799116551876068, + 0.264199823141098, + 1.7517287731170654, + 1.4282559156417847, + 1.690804123878479, + 0.1215878501534462, + -0.9322434067726135, + 0.8536378145217896, + -0.35758620500564575, + -1.8525059223175049, + -1.910990595817566 + ], + [ + -0.4916617274284363, + -0.5934253931045532, + 2.074575424194336, + -0.9512085318565369, + 0.3705800175666809, + 1.691048264503479, + 2.2907681465148926, + 1.532329797744751, + 0.8991785049438477, + -0.7733803987503052, + 1.3577321767807007, + 0.9844847917556763, + 1.7379451990127563, + 2.050845146179199, + 0.7804086804389954, + -0.8400509357452393, + -1.6339564323425293, + 0.9922879934310913, + 0.6006971597671509, + -1.0612030029296875, + -1.3134464025497437 + ] + ], + "predicted_scores": [ + 7.757726192474365, + 6.17840051651001, + 5.942082405090332, + 5.575725078582764, + 5.345473766326904, + 4.9476094245910645, + 4.759594440460205, + 4.737806797027588 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "74b116fd9ceddd5b2261edd96a14934a4b4a78a40191a654d6743661b819f407", + "stored_base_utility": 0.23635749518871307, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0007540385122410953, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007540385122410953 + }, + { + "contact_quality": null, + "progress": 0.0013887485256418586, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013887485256418586 + }, + { + "contact_quality": null, + "progress": 0.0012941956520080566, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0012941956520080566 + }, + { + "contact_quality": null, + "progress": 0.0005388259887695312, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0005388259887695312 + }, + { + "contact_quality": null, + "progress": 0.0009148319368250668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009148319368250668 + }, + { + "contact_quality": null, + "progress": 0.0015756288776174188, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0015756288776174188 + }, + { + "contact_quality": null, + "progress": 0.0008049209718592465, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008049209718592465 + }, + { + "contact_quality": null, + "progress": 0.0010410547256469727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0010410547256469727 + } + ], + "candidate_progress": [ + 0.0007540385122410953, + 0.0013887485256418586, + 0.0012941956520080566, + 0.0005388259887695312, + 0.0009148319368250668, + 0.0015756288776174188, + 0.0008049209718592465, + 0.0010410547256469727 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-8e12f7ad446482627dd3", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-20e1efbb745fd02abc95", + "generated_tangents": [ + [ + 0.06185590475797653, + -0.04598106071352959, + 1.7466535568237305, + -0.7043129205703735, + 0.90647292137146, + 2.1465933322906494, + 1.7426588535308838, + -0.2489314079284668, + -0.25054457783699036, + 1.6796447038650513, + 0.8467791676521301, + 0.7874410152435303, + 2.371890068054199, + 2.07037615776062, + -1.2214183807373047, + -0.4488571584224701, + 2.096590280532837, + 0.8521057963371277, + 0.6450879573822021, + 2.701753854751587, + 2.2453272342681885 + ], + [ + 0.17104472219944, + -0.05031730607151985, + 1.577580213546753, + -0.3243550658226013, + 1.1014646291732788, + 1.9892752170562744, + 1.8683958053588867, + -0.10321146249771118, + -0.12209179997444153, + 1.6653223037719727, + 0.5897968411445618, + 0.64275062084198, + 2.425130605697632, + 1.7522447109222412, + -1.1056369543075562, + -0.2550979554653168, + 1.9297103881835938, + 0.4884070158004761, + 0.8509853482246399, + 2.699551820755005, + 1.9898948669433594 + ], + [ + 0.17789174616336823, + 0.03985103964805603, + 1.661165475845337, + -0.8181127309799194, + 1.0224950313568115, + 2.2704620361328125, + 2.1650853157043457, + -0.26494675874710083, + 0.035614825785160065, + 1.4089508056640625, + 0.49506404995918274, + 1.2877193689346313, + 2.4452085494995117, + 2.5399973392486572, + -1.1478750705718994, + -0.31928303837776184, + 2.1709225177764893, + 0.49177998304367065, + 0.7644610404968262, + 3.309995412826538, + 2.784111261367798 + ], + [ + 0.17100991308689117, + -0.07769951224327087, + 1.689501166343689, + -0.553665041923523, + 0.8678314089775085, + 2.2808072566986084, + 2.4742748737335205, + -0.5780261754989624, + -0.1790362298488617, + 1.832779049873352, + 0.9880365133285522, + 0.8067461252212524, + 2.8785808086395264, + 2.3283064365386963, + -1.0634372234344482, + -0.3440331816673279, + 2.48679780960083, + 0.5058993101119995, + 0.7634018063545227, + 2.664320468902588, + 2.1168289184570312 + ], + [ + -0.012141048908233643, + 0.14199289679527283, + 1.4682300090789795, + -0.6070286631584167, + 1.2793245315551758, + 2.4998717308044434, + 1.9116482734680176, + -0.3240754306316376, + -0.16107217967510223, + 1.746077537536621, + 0.7436660528182983, + 0.9374769330024719, + 2.0339906215667725, + 2.403305768966675, + -1.0634084939956665, + -0.41096529364585876, + 2.193126916885376, + 0.6560408473014832, + 0.9145709872245789, + 2.9004108905792236, + 2.422254800796509 + ], + [ + -0.13088126480579376, + -0.16616514325141907, + 1.3518954515457153, + -0.06645262241363525, + 0.8044505715370178, + 2.339695692062378, + 2.1612751483917236, + -0.21809199452400208, + -0.21274007856845856, + 1.3352855443954468, + 0.20809608697891235, + 0.7528249621391296, + 1.6302942037582397, + 3.068659543991089, + -0.8531628847122192, + -0.3069373071193695, + 1.8509960174560547, + 0.5853466987609863, + 0.6205123662948608, + 2.9361536502838135, + 2.5021796226501465 + ], + [ + 0.10634994506835938, + -0.06957791745662689, + 1.5537424087524414, + -0.7663921117782593, + 1.0942169427871704, + 2.330355405807495, + 2.1368660926818848, + -0.3144095838069916, + -0.1361977607011795, + 1.6854244470596313, + 0.7905531525611877, + 0.9736430048942566, + 2.2341573238372803, + 2.2617242336273193, + -1.1853855848312378, + -0.3789823651313782, + 2.3444769382476807, + 0.656904935836792, + 0.6562376022338867, + 3.087292194366455, + 2.7287371158599854 + ], + [ + 0.03744655102491379, + -0.05327264592051506, + 1.592834711074829, + -0.5894061326980591, + 1.4490220546722412, + 2.5584278106689453, + 2.2257516384124756, + -0.5216046571731567, + -0.07101935148239136, + 1.5448328256607056, + 0.4931126832962036, + 0.8571008443832397, + 2.0394937992095947, + 2.248305320739746, + -0.9961563944816589, + -0.3045949339866638, + 1.9616707563400269, + 0.6377894878387451, + 0.9306171536445618, + 3.1057169437408447, + 2.726101875305176 + ] + ], + "generated_utilities": [ + 0.0007540385122410953, + 0.0013887485256418586, + 0.0012941956520080566, + 0.0005388259887695312, + 0.0009148319368250668, + 0.0015756288776174188, + 0.0008049209718592465, + 0.0010410547256469727 + ], + "group_id": "ms-20e1efbb745fd02abc95", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13665315508842468, + 0.1696699559688568, + 0.049180448055267334, + 0.05942712351679802, + 0.22736775875091553, + 0.0799722671508789, + 0.17261533439159393, + 0.21202586591243744, + 0.18980810046195984, + 0.24345096945762634, + 0.08349347114562988, + 0.11624568700790405, + 0.17245106399059296, + 0.07060472667217255, + 0.041495781391859055, + 0.028585990890860558 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.08645056188106537, + -0.060760222375392914, + 0.6493666172027588, + -0.016124725341796875, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + -0.2971414625644684, + 0.023367643356323242, + 0.6914786100387573, + -0.1027369350194931, + 0.1733318567276001, + 1.2948026657104492, + 0.0, + -0.30476057529449463, + -0.10394972562789917, + 0.9822354316711426, + 0.055961012840270996, + 0.004969358444213867, + 2.122485637664795, + 0.0 + ], + [ + 0.046016186475753784, + -0.24098780751228333, + 0.6493666172027588, + 0.05737954378128052, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + 0.22233793139457703, + 0.3896067142486572, + 0.6914786100387573, + -0.061455681920051575, + 0.17170286178588867, + 1.2948026657104492, + 0.0, + -0.4717581272125244, + -0.5722612738609314, + 0.9822354316711426, + 0.3690842390060425, + 0.33333829045295715, + 2.122485637664795, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.590132236480713, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.670657634735107, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.27303409576416 + ], + [ + -0.4654892086982727, + -0.6659128069877625, + 2.053840160369873, + -0.17420679330825806, + 1.1238669157028198, + 2.773953914642334, + 2.9706993103027344, + -0.9652835726737976, + -0.6603021621704102, + 1.957632303237915, + 0.30636119842529297, + 1.0932379961013794, + 2.4988603591918945, + 2.1735024452209473, + -0.7229697108268738, + -0.6565687656402588, + 2.1097376346588135, + 0.79427170753479, + 1.1421128511428833, + 3.200641393661499, + 3.3294520378112793 + ], + [ + 0.8783296346664429, + -0.6790766716003418, + 1.3789844512939453, + -1.578566551208496, + 1.2084856033325195, + 1.906846046447754, + 2.7278928756713867, + -0.8570022583007812, + -0.3114888072013855, + 2.660332202911377, + 0.4615887999534607, + -0.16199523210525513, + 1.9830265045166016, + 2.4224376678466797, + -0.9260885715484619, + 0.5555675029754639, + 2.5474588871002197, + 0.0070996880531311035, + 1.627825379371643, + 3.9487287998199463, + 3.03330135345459 + ], + [ + -0.1274794638156891, + -0.26015421748161316, + 2.5414626598358154, + -1.0220305919647217, + 1.4708960056304932, + 2.0982167720794678, + 3.1457836627960205, + -1.0992621183395386, + -0.9075721502304077, + 0.8330968022346497, + 0.15631377696990967, + 1.3095502853393555, + 1.8582203388214111, + 2.101317882537842, + -0.9184049367904663, + 0.4676415026187897, + 2.784534454345703, + 0.4529607892036438, + 0.6403366327285767, + 3.1566896438598633, + 2.5268735885620117 + ], + [ + 0.9289348125457764, + 0.1545984447002411, + 1.6792887449264526, + -0.6400681138038635, + 0.5956984758377075, + 2.5183756351470947, + 1.6261346340179443, + -0.019961506128311157, + -0.2638699412345886, + 1.310987114906311, + 0.4996453523635864, + -0.06606829166412354, + 1.9830868244171143, + 2.392476797103882, + -0.3871336579322815, + -0.7797489166259766, + 1.927225947380066, + 0.4438522458076477, + 1.0257132053375244, + 3.211881160736084, + 3.374328851699829 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05862678587436676, + -0.042836256325244904, + 1.6493666172027588, + -0.7465556859970093, + 1.1865841150283813, + 2.873413562774658, + 2.2950661182403564, + -0.48127153515815735, + -0.11381363868713379, + 1.6914786100387573, + 0.06965510547161102, + 0.8288453221321106, + 2.294802665710449, + 2.3353288173675537, + -1.3047605752944946, + -0.39009055495262146, + 1.9822354316711426, + 0.9993376731872559, + 0.81731116771698, + 3.122485637664795, + 2.63651704788208 + ], + [ + -0.045653510838747025, + 0.00397084653377533, + 1.3591042757034302, + -0.6414353847503662, + 0.8377937078475952, + 2.8277816772460938, + 2.203674554824829, + -0.1951642632484436, + 0.12272481620311737, + 1.7411555051803589, + -0.004818864166736603, + 0.8769108057022095, + 2.243906021118164, + 2.4328396320343018, + -1.6330946683883667, + -0.4807296395301819, + 2.2988247871398926, + 0.8021057844161987, + 0.6970767378807068, + 2.8728103637695312, + 2.288278102874756 + ], + [ + -0.4273069500923157, + 0.16743427515029907, + 1.538269281387329, + -0.7002415657043457, + 0.7622614502906799, + 2.805328369140625, + 2.267277479171753, + -0.6541028022766113, + -0.2980780303478241, + 1.456940770149231, + 0.5105806589126587, + 1.040317177772522, + 2.4321720600128174, + 2.696331024169922, + -1.1555191278457642, + -0.025772064924240112, + 1.6384800672531128, + 1.4557092189788818, + 0.6594157814979553, + 3.1446406841278076, + 2.249678134918213 + ], + [ + -0.6431475877761841, + -0.24224388599395752, + 1.3122833967208862, + -0.16711556911468506, + 0.9232780337333679, + 2.850522994995117, + 2.645284414291382, + -0.379825621843338, + -0.6750738620758057, + 1.1236441135406494, + 0.4938291311264038, + 1.1075668334960938, + 2.203859806060791, + 2.799032688140869, + -1.359815239906311, + -0.5104091763496399, + 2.0970702171325684, + 1.057171106338501, + 1.1167995929718018, + 2.8190994262695312, + 2.2184524536132812 + ] + ], + "predicted_scores": [ + 8.597485542297363, + 8.596193313598633, + 8.534893035888672, + 8.443713188171387, + 8.348173141479492, + 8.27610969543457, + 8.26349925994873, + 8.097298622131348 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1375492786", + "selected_index": 0, + "state_hash": "56acb365a02ebac0ac4893531e168df3c430b486416156feeea40eeb178bda3e", + "stored_base_utility": 0.13665315508842468, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-2dc5138251afd26a040d", + "generated_tangents": [ + [ + 2.229734420776367, + 0.07782519608736038, + -1.911167025566101, + -0.29360246658325195, + 0.03879709541797638, + -0.060477957129478455, + -1.1201860904693604, + 0.7442946434020996, + 0.0975741595029831, + -0.6522977352142334, + 0.061974864453077316, + 0.5730195045471191, + -2.8622052669525146, + -3.0645809173583984, + 0.8219016194343567, + 0.2688923478126526, + 0.6182039976119995, + -1.8234680891036987, + 0.9948716759681702, + -2.5656540393829346, + -1.6501619815826416 + ], + [ + 2.120349884033203, + 0.07830886542797089, + -1.718955636024475, + -0.5642669200897217, + -0.06578114628791809, + 0.1020921990275383, + -1.2869176864624023, + 0.9663091897964478, + 0.017531784251332283, + -0.5773118734359741, + -0.30360743403434753, + 0.4511907994747162, + -3.376307487487793, + -3.2012767791748047, + 0.9626085758209229, + 0.39912110567092896, + 0.6345493197441101, + -1.8809996843338013, + 0.9704519510269165, + -2.526883125305176, + -2.046485424041748 + ], + [ + 2.1305999755859375, + 0.07375235855579376, + -1.5923258066177368, + -0.04012773931026459, + -0.32000255584716797, + -0.30043599009513855, + -0.6495680212974548, + 0.5238656401634216, + 0.3066084682941437, + -0.3996679186820984, + -0.16557320952415466, + 0.20371773838996887, + -3.1474225521087646, + -2.543048620223999, + 0.7771709561347961, + 0.11174799501895905, + 0.6515008211135864, + -1.7105644941329956, + 0.8088074922561646, + -2.6919498443603516, + -1.465714454650879 + ], + [ + 2.3269028663635254, + -0.24510174989700317, + -1.5488661527633667, + 0.07778221368789673, + -0.4977880120277405, + -0.5684682130813599, + -0.7216100096702576, + 0.6022121906280518, + 0.2655333876609802, + -0.38561928272247314, + -0.35218241810798645, + 0.4150071144104004, + -3.3866188526153564, + -2.8475182056427, + 0.894115149974823, + 0.22672773897647858, + 0.2564984858036041, + -1.740992784500122, + 1.0877726078033447, + -2.64404559135437, + -2.1503942012786865 + ], + [ + 2.1344754695892334, + 0.0311201810836792, + -1.6761525869369507, + -0.45513665676116943, + -0.02005326747894287, + -0.06954048573970795, + -1.3852035999298096, + 0.8574519157409668, + 0.012595122680068016, + -0.597812831401825, + -0.12699559330940247, + 0.3659045100212097, + -3.115100622177124, + -3.1035373210906982, + 0.8429934978485107, + 0.3820308446884155, + 0.6470757126808167, + -1.9184659719467163, + 0.9036051034927368, + -2.6084611415863037, + -1.7757612466812134 + ], + [ + 2.328768014907837, + -0.07092507183551788, + -1.7646962404251099, + -0.5830026268959045, + -0.03958779573440552, + 0.29470857977867126, + -1.1875858306884766, + 0.997925877571106, + 0.0465409979224205, + -0.5099322199821472, + -0.14093852043151855, + 0.44292837381362915, + -3.160677671432495, + -3.035654306411743, + 0.8148537278175354, + 0.3543292284011841, + 0.6301447153091431, + -2.0400350093841553, + 0.8489795327186584, + -3.1253316402435303, + -1.534785270690918 + ], + [ + 2.012805700302124, + -0.30843496322631836, + -1.6121374368667603, + 0.1371026337146759, + 0.11304327845573425, + -0.41266801953315735, + -1.0327105522155762, + 0.8589532375335693, + 0.09012092649936676, + -0.31097614765167236, + -0.16042950749397278, + 0.6064033508300781, + -3.2289366722106934, + -2.1153907775878906, + 0.7666473388671875, + 0.3098098039627075, + 0.8585698008537292, + -1.6206648349761963, + 1.3534557819366455, + -2.3459465503692627, + -1.2446222305297852 + ], + [ + 2.11568021774292, + -0.004121083766222, + -1.6251686811447144, + -0.4688686430454254, + 0.09955087304115295, + -0.3957272469997406, + -1.2614753246307373, + 1.0185052156448364, + -0.04080372303724289, + -0.5344919562339783, + -0.24649104475975037, + 0.1748218536376953, + -3.2859137058258057, + -2.9421751499176025, + 0.7877935767173767, + 0.46348288655281067, + 0.7609414458274841, + -1.991679310798645, + 0.8857476115226746, + -2.7842953205108643, + -1.7650737762451172 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-2dc5138251afd26a040d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1373138427734375, + 0.1398601531982422, + 0.11154793202877045, + 0.07755142450332642, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7500379085540771, + -0.32870030403137207, + -0.3281364440917969, + 0.020404458045959473, + 0.03884005546569824, + 0.26651138067245483, + 0.0, + 0.2614295184612274, + 0.02006034180521965, + 0.6256309151649475, + 0.10119366645812988, + 0.0365104079246521, + -1.9807136058807373, + 0.0, + -0.4040032625198364, + -0.3390212059020996, + 0.444367915391922, + -1.3744113445281982, + 0.3090496063232422, + -1.6372854709625244, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7500379085540771, + 0.1347455233335495, + -1.3281364440917969, + -0.3984496593475342, + -0.6640693545341492, + -0.7334886193275452, + -1.63351309299469, + 0.4696621894836426, + -0.03721961751580238, + 0.0896599143743515, + -0.8988063335418701, + -0.047963276505470276, + -2.9807136058807373, + -2.542311906814575, + 0.5875393152236938, + 0.41857796907424927, + 0.32660895586013794, + -2.3744113445281982, + 0.9575246572494507, + -2.6372854709625244, + -1.9586280584335327 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.26702618598938, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.08462381362915, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.9172561168670654 + ], + [ + 1.7928142547607422, + -0.046942055225372314, + -1.3733142614364624, + -0.4212188422679901, + -0.5566414594650269, + -0.9966021180152893, + -1.7926092147827148, + 0.37284746766090393, + 0.18181614577770233, + -0.056899696588516235, + -0.8793746829032898, + -0.06206277757883072, + -3.2539288997650146, + -2.3651227951049805, + 0.543649435043335, + 0.4969414472579956, + 0.3953157365322113, + -2.6153955459594727, + 1.239845633506775, + -2.9289329051971436, + -2.1138057708740234 + ], + [ + 1.4313925504684448, + 0.3287010192871094, + -1.4217545986175537, + -0.11546039581298828, + -0.38684383034706116, + -0.3866080045700073, + -1.4945378303527832, + 0.4180656969547272, + 0.08673545718193054, + -0.30675017833709717, + -0.8370212316513062, + -0.3124745488166809, + -3.2861690521240234, + -2.5144338607788086, + 0.7133253216743469, + 0.5272455215454102, + 0.6288780570030212, + -2.7024707794189453, + 1.0730262994766235, + -2.603350877761841, + -1.8104162216186523 + ], + [ + 2.0841901302337646, + 0.18348857760429382, + -1.4159170389175415, + -0.5868262052536011, + -0.3290694057941437, + -0.8189557790756226, + -1.811798334121704, + 0.8927561044692993, + -0.4510294198989868, + -0.1373635083436966, + -0.46978816390037537, + -0.2017078548669815, + -3.1228857040405273, + -2.619961977005005, + 0.7154320478439331, + 0.4623824954032898, + 0.8127838373184204, + -2.5683043003082275, + 0.8921399116516113, + -2.799823522567749, + -1.6220401525497437 + ], + [ + 1.5681514739990234, + -0.11813957989215851, + -1.066882848739624, + -0.4835556447505951, + -0.4833664894104004, + -0.1345365047454834, + -1.872452735900879, + 1.059093952178955, + -0.3537854254245758, + 0.5130321979522705, + -0.7769420146942139, + -0.4462882876396179, + -3.249842882156372, + -2.2144932746887207, + 0.6907999515533447, + 0.14545467495918274, + 0.8816275596618652, + -2.4357430934906006, + 1.424092411994934, + -2.656961441040039, + -2.3529839515686035 + ], + [ + 2.0935373306274414, + 0.20671290159225464, + -1.0638595819473267, + -0.37751641869544983, + -0.5913090705871582, + -0.3702518939971924, + -1.3611384630203247, + 0.8432458639144897, + 0.4395832121372223, + -0.19578908383846283, + -1.2028591632843018, + -0.07535962760448456, + -3.2909016609191895, + -2.9216225147247314, + 1.1581575870513916, + 0.9284459352493286, + -0.3347962498664856, + -1.9277539253234863, + 0.8971017003059387, + -3.245394229888916, + -2.1499102115631104 + ], + [ + 2.5108895301818848, + -0.2977973222732544, + -1.458537220954895, + -0.5705423355102539, + -1.4453105926513672, + -0.4854125380516052, + -2.383378028869629, + 0.39735814929008484, + -0.3523370325565338, + 0.4586251974105835, + -1.2388068437576294, + 0.1082226037979126, + -3.5858566761016846, + -1.8809643983840942, + -0.18688076734542847, + 0.0871247947216034, + 0.0062812864780426025, + -3.012958526611328, + 1.0266033411026, + -3.1649136543273926, + -2.6460556983947754 + ], + [ + 1.2948591709136963, + 0.21422013640403748, + -1.3904463052749634, + -0.13111025094985962, + -0.1661812961101532, + -0.4983506500720978, + -2.0006139278411865, + -0.4048727750778198, + 0.10130663216114044, + 0.1955089420080185, + -0.27839744091033936, + 0.8026529550552368, + -3.2792744636535645, + -1.6839708089828491, + 0.6468846201896667, + -0.2772523760795593, + 0.45975640416145325, + -2.612403631210327, + 1.6571701765060425, + -1.8971503973007202, + -1.2593979835510254 + ], + [ + 0.9154124855995178, + -0.2073056846857071, + -2.1218466758728027, + -0.4230979084968567, + -1.2478010654449463, + -0.5439814925193787, + -1.185287594795227, + -0.06685006618499756, + -0.7776275873184204, + -0.015365853905677795, + -1.3256038427352905, + 0.46163398027420044, + -2.8853814601898193, + -2.387786865234375, + 1.2300693988800049, + 0.9541484117507935, + 0.29864931106567383, + -2.931433916091919, + 1.3643410205841064, + -2.001633882522583, + -2.6548502445220947 + ], + [ + 2.035454750061035, + 0.03935282677412033, + -1.9080866575241089, + -0.17657285928726196, + -1.4843900203704834, + -0.4207392632961273, + -1.554489016532898, + 0.1909814178943634, + -0.5061874389648438, + -0.420978844165802, + -0.7634562849998474, + -0.7818551063537598, + -3.374903678894043, + -2.5324442386627197, + 1.2213574647903442, + 0.9676649570465088, + 0.5052346587181091, + -2.0974624156951904, + 0.9588823914527893, + -3.6367201805114746, + -1.953636646270752 + ], + [ + 1.4856661558151245, + 0.3259274363517761, + -2.1635096073150635, + -0.8793277740478516, + 0.2686290740966797, + -0.340938925743103, + -0.8516338467597961, + -0.3830271363258362, + 0.5376158356666565, + -0.2721732258796692, + -1.2276334762573242, + -0.38624608516693115, + -3.346294403076172, + -3.093522071838379, + 0.6735774278640747, + 0.5564011931419373, + -0.5450864434242249, + -1.5815953016281128, + 1.4068235158920288, + -2.7531042098999023, + -2.515698194503784 + ] + ], + "predicted_scores": [ + 9.033143997192383, + 8.816910743713379, + 8.61434555053711, + 8.342315673828125, + 8.196283340454102, + 7.688064098358154, + 7.435515880584717, + 7.300971031188965 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565621826", + "selected_index": 0, + "state_hash": "f6edd849b0b51788d7be25e402fb514cb777ccf220797faaf769000814df02cb", + "stored_base_utility": 0.1373138427734375, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08252689987421036, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08252689987421036 + }, + "base_progress": 0.08252689987421036, + "base_success": false, + "base_utility": 0.08252689987421036, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09974726289510727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09974726289510727 + }, + { + "contact_quality": null, + "progress": 0.06912392377853394, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06912392377853394 + }, + { + "contact_quality": null, + "progress": 0.04023398831486702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04023398831486702 + }, + { + "contact_quality": null, + "progress": 0.07608189433813095, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07608189433813095 + }, + { + "contact_quality": null, + "progress": 0.10834071785211563, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10834071785211563 + }, + { + "contact_quality": null, + "progress": 0.059768736362457275, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.059768736362457275 + }, + { + "contact_quality": null, + "progress": 0.06140579655766487, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06140579655766487 + }, + { + "contact_quality": null, + "progress": 0.06661012023687363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06661012023687363 + } + ], + "candidate_progress": [ + 0.09974726289510727, + 0.06912392377853394, + 0.04023398831486702, + 0.07608189433813095, + 0.10834071785211563, + 0.059768736362457275, + 0.06140579655766487, + 0.06661012023687363 + ], + "candidate_source_chart_ids": [ + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-206099bab6bfe7c84fce", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-387e6771a55be98f1a59", + "generated_tangents": [ + [ + 0.31287968158721924, + -0.04098396748304367, + 0.6680215001106262, + 0.6064445972442627, + 1.366408109664917, + -1.0435786247253418, + -0.5852083563804626, + -0.37832167744636536, + 0.049432963132858276, + -0.18769046664237976, + 0.7405737042427063, + 0.09098008275032043, + 0.30926015973091125, + -0.6268219947814941, + 0.13203081488609314, + 0.1299392133951187, + 0.13280877470970154, + -0.4919320344924927, + -0.3845571279525757, + -0.08100757002830505, + -0.7122464179992676 + ], + [ + 0.053921133279800415, + -0.11941242218017578, + 0.07218372821807861, + 0.366675466299057, + 1.1164110898971558, + -0.686102032661438, + -0.5308035016059875, + -0.21884965896606445, + 0.00896742008626461, + -0.19040140509605408, + 0.6187528371810913, + 0.06797173619270325, + -0.15170753002166748, + -0.44486385583877563, + 0.18360966444015503, + 0.04875386506319046, + 0.027315065264701843, + -0.30808204412460327, + -0.34212684631347656, + -0.3659634590148926, + -0.08947956562042236 + ], + [ + 0.009828701615333557, + -0.06292109191417694, + 0.21970674395561218, + 0.32178470492362976, + 1.2790746688842773, + -0.6256959438323975, + -0.1923474669456482, + -0.31916186213493347, + 0.08678598701953888, + 0.012465961277484894, + 0.4704698622226715, + 0.1250826120376587, + -0.3155871629714966, + -0.24395690858364105, + 0.22821149230003357, + 0.05037960037589073, + 0.056975677609443665, + -0.43167275190353394, + -0.4937831163406372, + -0.20799744129180908, + -0.05310693383216858 + ], + [ + -0.0005264431238174438, + -0.04468899592757225, + 0.024656325578689575, + 0.31170758605003357, + 1.4589426517486572, + 0.13515056669712067, + -0.47113358974456787, + -0.059382736682891846, + 0.02040584199130535, + -0.14974522590637207, + 0.5188410878181458, + 0.2623038589954376, + -0.12810590863227844, + -0.5048943758010864, + 0.00869831070303917, + 0.1569286435842514, + 0.10765936225652695, + -0.27352994680404663, + -0.46486592292785645, + 0.20116907358169556, + 0.2253880500793457 + ], + [ + 0.08948926627635956, + -0.24587205052375793, + 0.561453104019165, + -0.09249259531497955, + 0.8570199012756348, + -0.9929342269897461, + -0.33456337451934814, + 0.22664164006710052, + 0.05681823939085007, + 0.11038579791784286, + 0.7123820781707764, + -0.02159436047077179, + -0.24912689626216888, + -0.9577621221542358, + 0.5930779576301575, + -0.02983243763446808, + -0.29516899585723877, + -0.28703922033309937, + -0.3999032974243164, + -0.13385555148124695, + -0.524699330329895 + ], + [ + 0.09799520671367645, + -0.22077828645706177, + 0.20052370429039001, + 0.021120905876159668, + 0.22559405863285065, + -1.1440191268920898, + -0.09133774042129517, + 0.21286557614803314, + -0.010845828801393509, + -0.026369161903858185, + 0.7046958208084106, + -0.4672173857688904, + -0.5708131194114685, + -0.10577349364757538, + 0.16442500054836273, + 0.022730842232704163, + 0.1690899133682251, + -0.20347245037555695, + -0.3841298222541809, + 0.2875376343727112, + 0.3768569529056549 + ], + [ + 0.12132717669010162, + -0.24124819040298462, + 0.19241198897361755, + 0.03107506036758423, + 0.39432281255722046, + -1.046919345855713, + -0.16484200954437256, + 0.23667003214359283, + -0.018077827990055084, + -0.016010873019695282, + 0.7737922072410583, + -0.4034934639930725, + -0.6087322235107422, + -0.25388848781585693, + 0.163319393992424, + -0.011344634927809238, + 0.0313936322927475, + -0.08741562068462372, + -0.3662693500518799, + 0.2606653571128845, + 0.39731377363204956 + ], + [ + 0.09076237678527832, + -0.22154167294502258, + 0.11351713538169861, + 0.11486765742301941, + 0.2227010577917099, + -1.207038402557373, + -0.16412675380706787, + 0.250341534614563, + 0.08164624869823456, + 0.048383429646492004, + 0.8020948171615601, + -0.34508824348449707, + -0.6843000650405884, + -0.226591095328331, + 0.1049489825963974, + 0.028447404503822327, + 0.11986105144023895, + -0.16639776527881622, + -0.350197434425354, + 0.3151845932006836, + 0.3616069257259369 + ] + ], + "generated_utilities": [ + 0.09974726289510727, + 0.06912392377853394, + 0.04023398831486702, + 0.07608189433813095, + 0.10834071785211563, + 0.059768736362457275, + 0.06140579655766487, + 0.06661012023687363 + ], + "group_id": "ms-387e6771a55be98f1a59", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.03822460025548935, + 0.18199902772903442, + 0.20744429528713226, + 0.05679210275411606, + 0.07646497339010239, + 0.038853906095027924, + 0.0824488028883934, + 0.05751838535070419, + 0.02475176379084587, + 0.040490686893463135, + 0.027167975902557373, + 0.06515108048915863, + 0.04604649543762207, + 0.014139334671199322, + 0.0258652176707983, + 0.034187138080596924 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0643576979637146, + 0.02293895184993744, + 0.03272289037704468, + 0.02799379825592041, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + 0.05611860752105713, + -0.010578691959381104, + 0.03396539017558098, + 0.754948616027832, + -0.12030506134033203, + 0.05740344524383545, + 0.0, + 0.01674991101026535, + 0.0017660856246948242, + 0.1110122799873352, + -0.29033517837524414, + -0.015073239803314209, + -0.062492311000823975, + 0.0 + ], + [ + 0.03256732225418091, + 0.029803231358528137, + -0.11524808406829834, + -0.03975573182106018, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + -0.11635321378707886, + -0.12916040420532227, + -0.10934318602085114, + 0.754948616027832, + -0.10127544403076172, + 0.10553282499313354, + 0.0, + 0.2124219685792923, + -0.13985675573349, + 0.20122885704040527, + -0.29033517837524414, + -0.09104716777801514, + -0.33053046464920044, + 0.0 + ] + ], + "predicted_scores": [ + 11.25488567352295, + 8.949102401733398, + 8.357065200805664, + 7.5187668800354, + 7.139684200286865, + 6.768582820892334, + 6.739411354064941, + 6.115901947021484 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "6b753a027f9378ff308946331f6bb658d452d8e77220ab0cd6fc813be0ac75cc", + "stored_base_utility": 0.03822460025548935, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03917040303349495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03917040303349495 + }, + "base_progress": 0.03917040303349495, + "base_success": false, + "base_utility": 0.03917040303349495, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12086125463247299, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12086125463247299 + }, + { + "contact_quality": null, + "progress": 0.0998828187584877, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0998828187584877 + }, + { + "contact_quality": null, + "progress": 0.03178328275680542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03178328275680542 + }, + { + "contact_quality": null, + "progress": 0.05774128437042236, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05774128437042236 + }, + { + "contact_quality": null, + "progress": 0.09802126884460449, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09802126884460449 + }, + { + "contact_quality": null, + "progress": 0.0678229108452797, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0678229108452797 + }, + { + "contact_quality": null, + "progress": 0.04722684621810913, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04722684621810913 + }, + { + "contact_quality": null, + "progress": 0.0725700631737709, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0725700631737709 + } + ], + "candidate_progress": [ + 0.12086125463247299, + 0.0998828187584877, + 0.03178328275680542, + 0.05774128437042236, + 0.09802126884460449, + 0.0678229108452797, + 0.04722684621810913, + 0.0725700631737709 + ], + "candidate_source_chart_ids": [ + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-7dedf60ad41ea2d92ee5", + "ms-98eb81debb5186ebbe38", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-be219977a9db8dbb1e58" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-38f7298184a2c5005e94", + "generated_tangents": [ + [ + 0.5549951791763306, + -0.011571042239665985, + -0.2968709170818329, + -0.005292847752571106, + 0.23439687490463257, + -2.1923131942749023, + 0.06370526552200317, + 0.4421412944793701, + -0.026232538744807243, + 0.12058635801076889, + -0.8667294383049011, + 1.0839509963989258, + -2.001406192779541, + -0.4074585437774658, + 0.7603763341903687, + 0.44663211703300476, + -0.5391051769256592, + 0.19690975546836853, + 0.6973133087158203, + -1.2703843116760254, + -1.0618451833724976 + ], + [ + 0.39722830057144165, + -0.1376485377550125, + -0.1670621931552887, + -0.06577108800411224, + 0.7057873010635376, + -1.2404489517211914, + 0.1869962215423584, + 0.3036459982395172, + 0.05380883812904358, + 0.5606752634048462, + -0.44692346453666687, + 1.2536674737930298, + -2.245656967163086, + -0.0013831406831741333, + 0.8167431950569153, + 0.4715690314769745, + -0.32873111963272095, + 0.38793396949768066, + 0.5243843793869019, + -1.7466576099395752, + -0.9168370962142944 + ], + [ + -0.08484864234924316, + -0.34610819816589355, + -0.5074306726455688, + 0.17315590381622314, + 0.8245617747306824, + -0.8755991458892822, + 0.5550235509872437, + 0.3240965008735657, + -0.08381851017475128, + 0.44868627190589905, + -0.3857881724834442, + 1.4537556171417236, + -0.32488834857940674, + 0.812469482421875, + 0.5597079992294312, + 0.34746792912483215, + -0.11295543611049652, + 0.10157401114702225, + 0.5262054204940796, + -2.596778631210327, + -0.43700775504112244 + ], + [ + 0.289387583732605, + -0.23139259219169617, + 0.6175347566604614, + -0.005333542823791504, + 0.717567503452301, + -1.312751293182373, + 0.08327877521514893, + 0.3430246114730835, + 0.019609035924077034, + 0.5139692425727844, + -0.43625301122665405, + 1.0173360109329224, + -1.7371784448623657, + -0.0010738074779510498, + 0.3718268871307373, + 0.4355454444885254, + -0.1140870600938797, + 0.027144625782966614, + 0.4506351351737976, + -2.010155439376831, + -1.7357187271118164 + ], + [ + 0.6307879686355591, + -0.02272743359208107, + -0.17862418293952942, + 0.12746986746788025, + 0.792668342590332, + -1.7350504398345947, + 0.04134935140609741, + 0.1101926937699318, + 0.06904052942991257, + 0.26937684416770935, + -0.5020616054534912, + 1.1872003078460693, + -1.7481523752212524, + -0.26263177394866943, + 0.5534759163856506, + 0.44994625449180603, + -0.40802204608917236, + 0.36935731768608093, + 0.6418775320053101, + -1.568533182144165, + -1.0431292057037354 + ], + [ + 0.5947062373161316, + -0.04412045329809189, + -0.1653849184513092, + 0.14315366744995117, + 0.8885510563850403, + -1.604752540588379, + 0.026499778032302856, + 0.02529023587703705, + 0.07011409848928452, + 0.2430332601070404, + -0.46852627396583557, + 1.2573213577270508, + -1.7811534404754639, + -0.24917252361774445, + 0.4690021574497223, + 0.40950876474380493, + -0.45773476362228394, + 0.2865621745586395, + 0.7381479740142822, + -1.572784662246704, + -0.9362776279449463 + ], + [ + 0.29183512926101685, + -0.1997244954109192, + 0.10648375749588013, + -0.16376827657222748, + 0.8216965198516846, + -1.3862097263336182, + 0.352537602186203, + 0.21297241747379303, + 0.059821225702762604, + 0.3932165801525116, + -0.3990858495235443, + 1.1140875816345215, + -1.1900757551193237, + -0.06085972487926483, + 0.05613749101758003, + 0.48587721586227417, + 0.07300464808940887, + 0.05014313757419586, + 0.40683120489120483, + -2.082291841506958, + -1.4638994932174683 + ], + [ + 0.5243908762931824, + 0.08422103524208069, + 0.1465301513671875, + -0.20865270495414734, + 0.6889839172363281, + -1.152407169342041, + 0.11039990186691284, + 0.22387364506721497, + 0.020331615582108498, + 0.8254491686820984, + -0.5028030872344971, + 0.660714864730835, + -0.8017787337303162, + 0.3961716294288635, + 0.4127199649810791, + 0.31803300976753235, + 0.273790568113327, + -0.3687719702720642, + 0.6621459722518921, + -1.9532408714294434, + -0.9297870397567749 + ] + ], + "generated_utilities": [ + 0.12086125463247299, + 0.0998828187584877, + 0.03178328275680542, + 0.05774128437042236, + 0.09802126884460449, + 0.0678229108452797, + 0.04722684621810913, + 0.0725700631737709 + ], + "group_id": "ms-38f7298184a2c5005e94", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1194220781326294, + 0.1767096221446991, + 0.0505807027220726, + 0.026383202522993088, + 0.09035396575927734, + 0.11899666488170624, + 0.059947751462459564, + 0.08427222818136215, + 0.041255753487348557, + 0.05034176632761955, + 0.02779809758067131, + 0.011037330143153667, + 0.020422041416168213, + 0.049474477767944336, + 0.03517866134643555, + 0.02796018123626709 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.08923250436782837, + 0.05675226449966431, + -0.1250607967376709, + -0.07566308975219727, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.001595139503479004, + 0.026048749685287476, + 0.1681501865386963, + -0.11568224430084229, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.18507611751556396, + -0.15127414464950562, + 0.07637521624565125, + -0.15284772217273712, + 0.0320390909910202, + -1.8774652481079102, + 0.0 + ], + [ + -0.2749255299568176, + 0.2578675150871277, + -0.46759605407714844, + -0.5041444897651672, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.13614141941070557, + 0.11120480298995972, + 0.15478569269180298, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.01827770471572876, + -0.38639169931411743, + -0.18168723583221436, + 0.3532869219779968, + -0.5437159538269043, + -1.8774652481079102, + 0.0 + ], + [ + 0.4466920495033264, + 0.07532927393913269, + 0.8145812749862671, + -0.963495135307312, + 1.3846542835235596, + -3.0133423805236816, + -1.0476078987121582, + 0.6759306788444519, + 0.5485053658485413, + 0.634345293045044, + -0.7682492733001709, + 2.2281322479248047, + -2.7100830078125, + -0.8660445213317871, + 0.7648183703422546, + 0.607989490032196, + -0.6848516464233398, + -0.06598936021327972, + 0.2030458152294159, + -2.9672060012817383, + -1.4293488264083862 + ], + [ + 1.0058056116104126, + 0.3426892161369324, + 0.6375484466552734, + -0.5828341841697693, + 1.7963712215423584, + -3.3760194778442383, + -0.9086123704910278, + 0.20509907603263855, + 0.10472571849822998, + 1.1158980131149292, + -1.0710231065750122, + 2.203950881958008, + -2.4466300010681152, + -0.3184358477592468, + 0.43084120750427246, + 0.32133060693740845, + -0.6005205512046814, + 0.07880236953496933, + 0.52874755859375, + -2.7805917263031006, + -1.6972103118896484 + ], + [ + 0.48786461353302, + -0.007949680089950562, + 0.41629794239997864, + -0.2277727723121643, + 1.0953291654586792, + -2.7825636863708496, + -1.7178279161453247, + 0.8412191271781921, + 0.6389099359512329, + 0.7100303769111633, + -1.1701807975769043, + 2.8546230792999268, + -2.570579767227173, + -0.2383948564529419, + 0.4282584488391876, + 0.27555936574935913, + -0.4073549509048462, + 0.49247825145721436, + 0.6483681797981262, + -3.3716540336608887, + -1.891394853591919 + ], + [ + 0.17139381170272827, + 0.831203818321228, + 0.1966383457183838, + -1.5011540651321411, + 1.811257004737854, + -2.6416001319885254, + -1.8791863918304443, + 0.03218984603881836, + 1.0182924270629883, + 1.5643268823623657, + -1.2954871654510498, + 2.249680757522583, + -1.9383416175842285, + -1.4303685426712036, + 0.48937559127807617, + 0.2848057746887207, + -0.6113857626914978, + 0.6453920006752014, + -0.13643476366996765, + -2.514732599258423, + -1.8046306371688843 + ], + [ + 0.1380026936531067, + 0.5165722370147705, + 0.9639596939086914, + -0.3682268261909485, + 2.2811193466186523, + -3.5902557373046875, + -1.047336220741272, + 1.3064022064208984, + 0.7264143228530884, + 1.3356540203094482, + -0.41242116689682007, + 1.8184869289398193, + -3.016441822052002, + -0.8501555919647217, + 1.0147737264633179, + 1.0417438745498657, + -0.24422766268253326, + 0.3441629707813263, + 1.075059175491333, + -3.178764820098877, + -1.7388081550598145 + ], + [ + 1.0805304050445557, + -0.5906903743743896, + -0.32329434156417847, + -1.7024246454238892, + 1.3012806177139282, + -3.3655173778533936, + -2.0100932121276855, + 0.28595322370529175, + -0.42309290170669556, + 0.9014679193496704, + -0.11985284090042114, + 3.033172607421875, + -1.890408992767334, + -0.049424707889556885, + 0.8836197853088379, + 0.07151538133621216, + -0.6898007392883301, + 0.6614744663238525, + 0.7956362962722778, + -3.344180107116699, + -1.4918344020843506 + ], + [ + 1.4192681312561035, + -0.5825785994529724, + 0.11329725384712219, + -0.08465814590454102, + 1.0504729747772217, + -2.438842535018921, + -0.9858216047286987, + -0.35852980613708496, + 1.4301284551620483, + 0.37676453590393066, + -1.3725404739379883, + 3.114079475402832, + -2.0786290168762207, + -0.5835205912590027, + 0.1985606551170349, + 0.260464072227478, + 0.41686397790908813, + -0.104665607213974, + 0.6382055282592773, + -2.7474825382232666, + -1.663126826286316 + ], + [ + 1.0832164287567139, + 0.11555392295122147, + 1.2108526229858398, + -1.5446858406066895, + 0.9112894535064697, + -3.3027408123016357, + -1.2419135570526123, + -0.057880282402038574, + -0.07349961996078491, + 0.026286423206329346, + -0.897603452205658, + 3.2857306003570557, + -2.232046365737915, + -1.2509676218032837, + 1.1900405883789062, + 0.8446143269538879, + -1.339659333229065, + -0.07882180064916611, + 0.1166829764842987, + -2.662961721420288, + -1.0079293251037598 + ], + [ + 0.43863287568092346, + -0.2024492472410202, + 0.0567094087600708, + -0.3029903173446655, + 1.612892746925354, + -3.26739764213562, + -1.7898839712142944, + -0.1649491786956787, + 0.2927369475364685, + 1.020188808441162, + -0.10171866416931152, + 2.3230679035186768, + -2.615206003189087, + -0.08625847101211548, + 0.9057100415229797, + 1.4201092720031738, + -0.3173455595970154, + 0.09508261829614639, + 0.9565507173538208, + -2.2954390048980713, + -1.2229193449020386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.03360384702682495, + -0.028338372707366943, + -0.12341004610061646, + -0.04655510187149048, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + 0.032517433166503906, + -0.02922782301902771, + -0.033792078495025635, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + -0.06465893983840942, + -0.026740312576293945, + 0.046701133251190186, + -0.06953387707471848, + 0.02321198582649231, + -1.8774652481079102, + 0.0 + ] + ], + "predicted_scores": [ + 10.316132545471191, + 7.8692193031311035, + 7.442691326141357, + 7.317057132720947, + 7.183282375335693, + 7.134940147399902, + 5.849112510681152, + 4.547755718231201 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "dee046e5a3c9b5e78be27f1df1f4f235344ec818a1a7439afe5f4cb30139cf34", + "stored_base_utility": 0.1194220781326294, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.009124159812927246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009124159812927246 + }, + "base_progress": 0.009124159812927246, + "base_success": false, + "base_utility": 0.009124159812927246, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.00878363847732544, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00878363847732544 + }, + { + "contact_quality": null, + "progress": 0.005450725555419922, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005450725555419922 + }, + { + "contact_quality": null, + "progress": 0.008500377647578716, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008500377647578716 + }, + { + "contact_quality": null, + "progress": 0.011097331531345844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011097331531345844 + }, + { + "contact_quality": null, + "progress": 0.0051685175858438015, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0051685175858438015 + }, + { + "contact_quality": null, + "progress": 0.0060965814627707005, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0060965814627707005 + }, + { + "contact_quality": null, + "progress": 0.007386326789855957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007386326789855957 + }, + { + "contact_quality": null, + "progress": 0.0030460357666015625, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0030460357666015625 + } + ], + "candidate_progress": [ + 0.00878363847732544, + 0.005450725555419922, + 0.008500377647578716, + 0.011097331531345844, + 0.0051685175858438015, + 0.0060965814627707005, + 0.007386326789855957, + 0.0030460357666015625 + ], + "candidate_source_chart_ids": [ + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-397e502908d16b7804b0", + "ms-6758deb06644a14aa722", + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-6758deb06644a14aa722" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-482c5c8d208b5907a2c2", + "generated_tangents": [ + [ + -0.049815550446510315, + -0.04656240716576576, + 0.705135703086853, + -0.0781581848859787, + 2.057030200958252, + 0.6591335535049438, + 0.360661119222641, + 0.22798390686511993, + 0.03546068072319031, + 1.0741779804229736, + -0.45267316699028015, + 1.567286729812622, + 1.4588344097137451, + 0.44813960790634155, + 0.10778860002756119, + -0.0587795116007328, + 0.148574560880661, + 0.2691079378128052, + 0.3142581582069397, + 0.273465096950531, + -1.0474905967712402 + ], + [ + -0.23655660450458527, + -0.041145652532577515, + 0.6016224026679993, + 0.012054666876792908, + 2.1290836334228516, + 0.7756989002227783, + 0.4296463429927826, + 0.1317601501941681, + 0.051725782454013824, + 1.3776813745498657, + -0.4998546540737152, + 1.4107518196105957, + 1.6778227090835571, + 0.28998851776123047, + 0.08754293620586395, + -0.10128724575042725, + 0.36676013469696045, + 0.2597346305847168, + 0.49081090092658997, + 0.30386966466903687, + -0.8922455310821533 + ], + [ + -0.5773165822029114, + 0.25543367862701416, + 0.9771597385406494, + -0.27743569016456604, + 1.397900104522705, + 0.6213514804840088, + 0.20530939102172852, + 0.18989403545856476, + 0.10289112478494644, + 1.0490386486053467, + -0.5493795871734619, + 1.824069857597351, + 1.4883545637130737, + 0.08942125737667084, + 0.04887757450342178, + -0.17101269960403442, + -0.18515239655971527, + 0.5316981077194214, + 0.5965712070465088, + -0.3456134796142578, + -0.9148825407028198 + ], + [ + -0.06939999759197235, + -0.048543453216552734, + 0.7926576137542725, + 0.0693444311618805, + 1.4298961162567139, + 0.38798975944519043, + 0.5715478658676147, + 0.19409309327602386, + 0.18216733634471893, + 1.0031983852386475, + -0.7306927442550659, + 1.1852830648422241, + 0.12281766533851624, + 0.47779446840286255, + -0.027130860835313797, + -0.14800427854061127, + -0.04451192915439606, + 0.6661083698272705, + 0.2828511893749237, + 0.4495900273323059, + -0.5427080392837524 + ], + [ + -0.29909998178482056, + 0.14783556759357452, + 0.8953452110290527, + 0.042054951190948486, + 2.0431716442108154, + 0.6359003782272339, + 0.04129081964492798, + 0.04279421269893646, + 0.13073933124542236, + 1.0399121046066284, + -0.43709564208984375, + 1.3877813816070557, + 1.1673877239227295, + 0.07621181011199951, + 0.08724887669086456, + -0.23650838434696198, + 0.2798236310482025, + 0.48398661613464355, + 0.5118979215621948, + 0.1778949499130249, + -0.7096110582351685 + ], + [ + -0.31867051124572754, + 0.160978764295578, + 0.771454930305481, + -0.025081157684326172, + 1.8512072563171387, + 0.6680611371994019, + -0.13499385118484497, + 0.07672888785600662, + 0.0857548788189888, + 1.0777310132980347, + -0.5266196727752686, + 1.4448391199111938, + 0.9337642192840576, + 0.005745984613895416, + 0.07911935448646545, + -0.2396399974822998, + 0.23839005827903748, + 0.5045702457427979, + 0.43266230821609497, + 0.1670060157775879, + -0.67417311668396 + ], + [ + -0.16751603782176971, + -0.0066785626113414764, + 0.857994556427002, + -0.015456080436706543, + 2.1056907176971436, + 0.6720499992370605, + 0.4198402464389801, + 0.139546737074852, + 0.14446470141410828, + 1.180782675743103, + -0.46742770075798035, + 1.5358421802520752, + 1.3812111616134644, + 0.4399641156196594, + 0.1323907971382141, + -0.026886098086833954, + 0.13452789187431335, + 0.1520424336194992, + 0.42508020997047424, + 0.2228962779045105, + -1.1466140747070312 + ], + [ + -0.06298300623893738, + -0.12911100685596466, + 0.8522658348083496, + 0.73267662525177, + 1.7848231792449951, + 0.3261961340904236, + 0.15888312458992004, + -0.08184540271759033, + 0.037027791142463684, + 1.0633642673492432, + -0.5972768068313599, + 1.5755444765090942, + 1.2024102210998535, + 0.6883900165557861, + 0.26560884714126587, + -0.06333936750888824, + 0.11718880385160446, + 0.2505548894405365, + 0.793315052986145, + 0.9461155533790588, + -0.7721607685089111 + ] + ], + "generated_utilities": [ + 0.00878363847732544, + 0.005450725555419922, + 0.008500377647578716, + 0.011097331531345844, + 0.0051685175858438015, + 0.0060965814627707005, + 0.007386326789855957, + 0.0030460357666015625 + ], + "group_id": "ms-482c5c8d208b5907a2c2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0359329991042614, + 0.1834145486354828, + 0.04271410033106804, + 0.03202331066131592, + 0.12089761346578598, + 0.03575414419174194, + 0.07773216813802719, + 0.1062050461769104, + 0.10849495977163315, + 0.08369352668523788, + 0.14167599380016327, + 0.008830646984279156, + 0.014355997554957867, + 0.02518286369740963, + 0.07873731851577759, + 0.015250584110617638 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.007817894220352173, + 0.01831631362438202, + 0.14601469039916992, + 0.10826835036277771, + 1.6591682434082031, + -0.005238652229309082, + 0.0, + -0.02753499150276184, + -0.009624123573303223, + 0.5762790441513062, + -0.3171614408493042, + 1.4374253749847412, + 0.7921857833862305, + 0.0, + -0.037753358483314514, + -0.025671005249023438, + 0.027724146842956543, + 0.4680846929550171, + -0.08415359258651733, + 0.033902376890182495, + 0.0 + ], + [ + 0.4543180465698242, + 0.20962828397750854, + 1.14601469039917, + -0.33832865953445435, + 2.659168243408203, + -0.7141264081001282, + 0.3707960247993469, + 0.21857939660549164, + 0.6796972155570984, + 1.5762790441513062, + -1.3171614408493042, + 2.437425374984741, + 1.7921857833862305, + 1.7551628351211548, + -0.19566796720027924, + 0.6842412948608398, + 0.11985036730766296, + 1.468084692955017, + 0.29799214005470276, + 0.3715361952781677, + 0.2624160647392273 + ], + [ + 0.2115812450647354, + -0.09577503800392151, + 0.7509078979492188, + -0.6239187121391296, + 2.4163239002227783, + -0.5409671664237976, + 0.16154196858406067, + 0.5615253448486328, + 1.0765573978424072, + 1.4198161363601685, + -1.430187463760376, + 2.5212466716766357, + 1.9029841423034668, + 1.5550287961959839, + -0.5099657773971558, + 1.0103299617767334, + -0.21028786897659302, + 1.4441221952438354, + 0.5605546236038208, + 0.6799722909927368, + 0.5554409027099609 + ], + [ + 0.46792101860046387, + 0.08014003932476044, + 1.5437835454940796, + 0.08789932727813721, + 2.3046298027038574, + -1.1965231895446777, + 0.8527317047119141, + 0.22510167956352234, + 1.0473172664642334, + 1.3720369338989258, + -1.2376489639282227, + 2.066061019897461, + 2.033992052078247, + 2.166865825653076, + -0.46199846267700195, + 1.1675353050231934, + -0.22657811641693115, + 1.7576167583465576, + 0.766140878200531, + 0.1856600046157837, + 0.008163660764694214 + ], + [ + 0.9431964755058289, + 0.6587716341018677, + 0.8381389379501343, + -0.8407833576202393, + 2.3666741847991943, + -1.05813729763031, + 0.1429125964641571, + 0.10640819370746613, + 1.0277029275894165, + 1.193630576133728, + -0.7695048451423645, + 2.9484994411468506, + 1.1866973638534546, + 1.5341145992279053, + -0.1225738674402237, + 0.9121334552764893, + 0.7116963863372803, + 0.8198207020759583, + 0.6562460660934448, + 0.1118304431438446, + -0.2320157289505005 + ] + ], + "predicted_scores": [ + 9.43040657043457, + 9.323161125183105, + 8.993653297424316, + 8.677950859069824, + 8.614483833312988, + 8.41909408569336, + 8.413841247558594, + 6.8669514656066895 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "903f78d5baa541d73612cac5e063bb553fba566c0d4adcb9603aa055d710fb15", + "stored_base_utility": 0.0359329991042614, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14663122594356537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14663122594356537 + }, + "base_progress": 0.14663122594356537, + "base_success": false, + "base_utility": 0.14663122594356537, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10735879093408585, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10735879093408585 + }, + { + "contact_quality": null, + "progress": 0.08596404641866684, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08596404641866684 + }, + { + "contact_quality": null, + "progress": 0.1273784190416336, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1273784190416336 + }, + { + "contact_quality": null, + "progress": 0.10580883175134659, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10580883175134659 + }, + { + "contact_quality": null, + "progress": 0.10845587402582169, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10845587402582169 + }, + { + "contact_quality": null, + "progress": 0.1056336760520935, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1056336760520935 + }, + { + "contact_quality": null, + "progress": 0.09012359380722046, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09012359380722046 + }, + { + "contact_quality": null, + "progress": 0.06587376445531845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06587376445531845 + } + ], + "candidate_progress": [ + 0.10735879093408585, + 0.08596404641866684, + 0.1273784190416336, + 0.10580883175134659, + 0.10845587402582169, + 0.1056336760520935, + 0.09012359380722046, + 0.06587376445531845 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-19cb10d10516bd0216a6", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-4da03300d92420e80ed0", + "generated_tangents": [ + [ + 2.634641170501709, + -0.3039514422416687, + -0.7532751560211182, + 0.6672413349151611, + -0.5675072073936462, + 1.9859477281570435, + 1.4871575832366943, + 2.147381067276001, + 0.20933063328266144, + -1.4684480428695679, + -0.5884499549865723, + -0.9568467140197754, + -3.2579658031463623, + -3.3139076232910156, + 1.7043448686599731, + -0.08272968232631683, + -1.1290931701660156, + 0.9141604900360107, + -0.08389303088188171, + -0.5761035680770874, + -0.5719590187072754 + ], + [ + 2.464007616043091, + -0.18512862920761108, + -0.8321398496627808, + 0.5767568349838257, + -0.30320900678634644, + 1.859495997428894, + 0.9342386722564697, + 1.911637306213379, + 0.153948575258255, + -1.4111170768737793, + -0.4420132339000702, + -0.8328026533126831, + -2.989961862564087, + -2.6137354373931885, + 1.9184253215789795, + -0.2443116456270218, + -1.2287453413009644, + 0.5091448426246643, + 0.23109352588653564, + -0.7274672985076904, + -0.08955064415931702 + ], + [ + 2.5132815837860107, + -0.15570500493049622, + -0.9391363859176636, + 0.5854440927505493, + -0.6245869398117065, + 1.7375870943069458, + 1.9102296829223633, + 2.0940334796905518, + 0.2903728485107422, + -1.451993703842163, + -0.693550169467926, + -0.9917333126068115, + -3.2797203063964844, + -3.627514123916626, + 2.05561900138855, + -0.2094503939151764, + -1.0858231782913208, + 0.8155262470245361, + -0.11378800868988037, + -0.6239851117134094, + -0.5379022359848022 + ], + [ + 2.5011184215545654, + -0.1403145045042038, + -0.8085476160049438, + 0.4468512237071991, + -0.660551905632019, + 1.8433722257614136, + 1.7551867961883545, + 2.2682418823242188, + 0.2240917980670929, + -1.4834834337234497, + -0.6019911766052246, + -1.0474611520767212, + -3.151282787322998, + -3.324010133743286, + 1.8510380983352661, + -0.21391025185585022, + -0.9325986504554749, + 1.0144681930541992, + -0.12915343046188354, + -0.7882421016693115, + -0.492827445268631 + ], + [ + 2.663836717605591, + -0.25584420561790466, + -1.0287872552871704, + 0.4594871699810028, + -0.5309082865715027, + 1.634170651435852, + 1.7936627864837646, + 1.9982304573059082, + 0.27105197310447693, + -1.551898717880249, + -0.5929011106491089, + -1.1161805391311646, + -3.272610664367676, + -3.361637830734253, + 1.9080802202224731, + -0.2649810016155243, + -1.1685264110565186, + 0.7430374026298523, + -0.1481437385082245, + -0.6536601781845093, + -0.8733140230178833 + ], + [ + 2.6133060455322266, + -0.1696094274520874, + -0.9842472076416016, + 0.4757023751735687, + -0.5343642830848694, + 1.8002469539642334, + 1.8543155193328857, + 2.150381326675415, + 0.24019616842269897, + -1.530577301979065, + -0.6877753138542175, + -0.9864175319671631, + -3.030122756958008, + -3.4371330738067627, + 2.08855938911438, + -0.29169246554374695, + -1.1221535205841064, + 0.8243769407272339, + -0.1287095844745636, + -0.7423315048217773, + -0.5376543998718262 + ], + [ + 2.6695475578308105, + -0.1514185220003128, + -0.8053789138793945, + 0.07773524522781372, + -0.48225104808807373, + 1.7529746294021606, + 1.6568171977996826, + 1.9272019863128662, + 0.28966644406318665, + -1.7189970016479492, + -0.49786052107810974, + -0.9074803590774536, + -2.829857587814331, + -3.054576873779297, + 1.805141568183899, + -0.4113864004611969, + -1.086623191833496, + 0.9642729163169861, + 0.018310487270355225, + -0.8757671117782593, + -0.7149254083633423 + ], + [ + 2.584070920944214, + -0.17364898324012756, + -0.796271562576294, + 0.599783182144165, + -0.2156740427017212, + 1.8570375442504883, + 0.8505010604858398, + 2.0467300415039062, + 0.22562220692634583, + -1.358185887336731, + -0.1811409592628479, + -0.8399637937545776, + -2.6548526287078857, + -2.437251329421997, + 1.983381986618042, + -0.22362051904201508, + -0.9771478772163391, + 0.6202883720397949, + 0.6414510011672974, + -0.07619726657867432, + -1.3737967014312744 + ] + ], + "generated_utilities": [ + 0.10735879093408585, + 0.08596404641866684, + 0.1273784190416336, + 0.10580883175134659, + 0.10845587402582169, + 0.1056336760520935, + 0.09012359380722046, + 0.06587376445531845 + ], + "group_id": "ms-4da03300d92420e80ed0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18184101581573486, + 0.1803790181875229, + 0.09926676750183105, + 0.20791122317314148, + 0.26338279247283936, + 0.2025860995054245, + 0.2575649917125702, + 0.23894403874874115, + 0.2174234390258789, + 2.0, + 0.1918075978755951, + 0.10365168750286102, + 2.0, + 2.0, + 2.0, + 0.15933696925640106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.1168947219848633, + 0.13667210936546326, + -0.051466986536979675, + 0.03360617160797119, + -0.19173425436019897, + 1.7646396160125732, + 0.0, + 0.9886866807937622, + 0.19114632904529572, + -0.3927978277206421, + -0.21108940243721008, + -0.031822919845581055, + -1.6683034896850586, + 0.0, + 0.9908627271652222, + -0.05301591753959656, + 0.08875280618667603, + 0.04224884510040283, + -0.03718748688697815, + -0.06952071189880371, + 0.0 + ], + [ + 2.1713812351226807, + -0.41443270444869995, + -0.23275701701641083, + 0.6524178981781006, + -0.25828492641448975, + 3.331055164337158, + 2.47529673576355, + 1.4984867572784424, + -0.396874338388443, + -0.725897490978241, + 0.1870076060295105, + -0.4014764428138733, + -2.239036798477173, + -2.218061923980713, + 1.6540865898132324, + 0.013689309358596802, + -0.22782719135284424, + 1.2475574016571045, + 0.7231295704841614, + -0.281164288520813, + -1.4593302011489868 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 2.1168947219848633, + 0.045049574226140976, + -0.25911271572113037, + 0.6112266778945923, + -0.5530642867088318, + 2.7646396160125732, + 1.9622430801391602, + 1.9886866807937622, + 0.28067120909690857, + -1.392797827720642, + -0.5793421268463135, + -1.031822919845581, + -2.6683034896850586, + -2.812993049621582, + 1.9908627271652222, + -0.28380459547042847, + -0.7305920720100403, + 0.7978808879852295, + 0.1574108749628067, + -1.0695207118988037, + -0.9408642053604126 + ], + [ + 2.228377103805542, + -0.19656506180763245, + -0.23844876885414124, + 0.6676552295684814, + -0.7639804482460022, + 2.937652111053467, + 1.882341980934143, + 1.8207013607025146, + 0.27702635526657104, + -1.549763798713684, + -0.5955427289009094, + -1.185685157775879, + -2.8421716690063477, + -3.054234266281128, + 1.691805362701416, + -0.37051287293434143, + -0.43653741478919983, + 0.514695405960083, + -0.0132378488779068, + -0.899186372756958, + -0.8349848985671997 + ], + [ + 2.425011157989502, + -0.03216857090592384, + -0.010275468230247498, + 0.4265761971473694, + -0.7737585306167603, + 3.1123812198638916, + 1.5686779022216797, + 2.112421989440918, + 0.3475198745727539, + -1.3452436923980713, + -0.6067132949829102, + -1.1150503158569336, + -2.461210250854492, + -2.8643558025360107, + 2.185579299926758, + -0.6223857402801514, + -0.8183724880218506, + 0.8074819445610046, + 0.536148726940155, + -0.7826522588729858, + -0.6682058572769165 + ], + [ + 1.6831411123275757, + -0.19312052428722382, + 0.010244876146316528, + 1.1032377481460571, + -0.4994945824146271, + 2.9000508785247803, + 2.187715768814087, + 2.200220823287964, + 0.4744641184806824, + -1.6715843677520752, + -0.493664413690567, + -1.459894061088562, + -2.90458345413208, + -2.3254852294921875, + 1.4726771116256714, + -0.21338894963264465, + -1.293912649154663, + 1.3146823644638062, + -0.09375743567943573, + -0.5040683150291443, + -0.9596792459487915 + ], + [ + 1.6738543510437012, + 0.6350066661834717, + 0.19669359922409058, + -0.1396545171737671, + -0.9625160694122314, + 2.207883596420288, + 1.7678128480911255, + 1.319273829460144, + -0.267609566450119, + -0.5747214555740356, + 0.19376236200332642, + -1.4490230083465576, + -2.6397883892059326, + -3.2567262649536133, + 1.2646806240081787, + -0.14523190259933472, + -0.050948500633239746, + 0.6943628787994385, + -0.1503342241048813, + -0.7313232421875, + -1.7930638790130615 + ], + [ + 2.139432668685913, + 0.34125545620918274, + 0.6370852589607239, + 0.8591548204421997, + 0.14890235662460327, + 2.6835381984710693, + 2.324920892715454, + 2.0091233253479004, + 0.13480046391487122, + -0.47407835721969604, + -1.102479100227356, + -1.0710523128509521, + -2.3226847648620605, + -3.6534740924835205, + 1.8767409324645996, + -0.01776304841041565, + -1.167153000831604, + 0.693484902381897, + 0.563054084777832, + -0.17544519901275635, + -0.8315125703811646 + ], + [ + 1.1254388093948364, + -0.03027276322245598, + -0.8753920793533325, + 0.3578681945800781, + -0.4256642460823059, + 1.7686536312103271, + 2.253190517425537, + 2.8274970054626465, + 0.3938461244106293, + -1.4084759950637817, + -0.7252857685089111, + -0.07288885116577148, + -1.7749395370483398, + -2.1030378341674805, + 1.92497980594635, + 0.20398634672164917, + -0.03521168231964111, + 1.4702520370483398, + -0.36331063508987427, + -1.5291211605072021, + -0.8369957804679871 + ] + ], + "predicted_scores": [ + 7.433258056640625, + 6.800236225128174, + 6.594235420227051, + 6.578792095184326, + 6.326344966888428, + 6.24533224105835, + 5.853445529937744, + 4.849090099334717 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1747178030", + "selected_index": 0, + "state_hash": "87c74ddf97c53ddeeaae98467b4a13b1e0e499bd94eddb617759d39164cab542", + "stored_base_utility": 0.18184101581573486, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.021834811195731163, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021834811195731163 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.019551733508706093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019551733508706093 + }, + { + "contact_quality": null, + "progress": 0.024202564731240273, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024202564731240273 + }, + { + "contact_quality": null, + "progress": 0.02782593108713627, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02782593108713627 + }, + { + "contact_quality": null, + "progress": 0.005327919963747263, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005327919963747263 + }, + { + "contact_quality": null, + "progress": 0.00631558895111084, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00631558895111084 + } + ], + "candidate_progress": [ + 1.0, + 0.021834811195731163, + 1.0, + 0.019551733508706093, + 0.024202564731240273, + 0.02782593108713627, + 0.005327919963747263, + 0.00631558895111084 + ], + "candidate_source_chart_ids": [ + "ms-6a3add21affa4796f9be", + "ms-dc26b448cce4164a23e6", + "ms-0e592951fa32139b6dc9", + "ms-e84163d9ef2e7014daca", + "ms-f85a653af630821f2572", + "ms-6a3add21affa4796f9be", + "ms-e84163d9ef2e7014daca", + "ms-7eaea428dfdef7255a66" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + false, + true, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank3", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-57be9ae181c077e7821f", + "generated_tangents": [ + [ + -0.2797073721885681, + 0.13484108448028564, + 0.6566584706306458, + -0.4192536473274231, + -0.010051384568214417, + 1.6424551010131836, + 0.4706100821495056, + -0.018184825778007507, + -0.07831493020057678, + 0.726234495639801, + 0.32589784264564514, + 0.461823970079422, + 1.713438868522644, + 0.613983690738678, + 0.8218561410903931, + -0.05769515037536621, + -0.039725109934806824, + -0.0921669751405716, + -0.014564752578735352, + 0.924396812915802, + 0.1459539532661438 + ], + [ + -0.309101939201355, + 0.009493179619312286, + 0.9876956939697266, + -0.30612802505493164, + 0.5839683413505554, + 1.243936538696289, + 1.1610774993896484, + 0.04859870672225952, + -0.13851861655712128, + 0.8417831063270569, + -0.06353219598531723, + 0.38287553191185, + 1.6576346158981323, + 1.1466187238693237, + 0.6121523976325989, + -0.11832182109355927, + 0.20552831888198853, + 0.08858387917280197, + 0.24941027164459229, + 0.9831594824790955, + 0.9868859052658081 + ], + [ + -0.19697992503643036, + 0.09599083662033081, + 0.9275882244110107, + -0.28163638710975647, + 0.3541561961174011, + 1.3389673233032227, + 1.0469138622283936, + -0.22703340649604797, + -0.09654742479324341, + 1.0090891122817993, + 0.039891310036182404, + 0.791820228099823, + 1.5588668584823608, + 0.6951661109924316, + 0.7266017198562622, + -0.14599080383777618, + -0.07227890193462372, + -0.008913800120353699, + 0.1740609109401703, + 1.2036690711975098, + 0.5470603108406067 + ], + [ + -0.34583592414855957, + 0.03694339841604233, + 1.0050984621047974, + -0.4203651249408722, + 0.4847651720046997, + 1.7126538753509521, + 0.7576368451118469, + 0.04789559543132782, + -0.19675417244434357, + 0.7332325577735901, + -0.050296053290367126, + 0.5248405933380127, + 1.6081269979476929, + 1.103381872177124, + 0.6356422305107117, + -0.12914817035198212, + 0.1542941927909851, + 0.08850230276584625, + 0.04196876287460327, + 1.2002403736114502, + 0.8037097454071045 + ], + [ + -0.42469507455825806, + -0.04173311963677406, + 0.7264179587364197, + 0.06337344646453857, + -0.3319808840751648, + 1.5662562847137451, + 1.011260986328125, + 0.10568541288375854, + -0.10509131103754044, + 0.9041814208030701, + -0.06494040042161942, + 0.21946558356285095, + 1.254018783569336, + 1.038018822669983, + 0.5477806925773621, + -0.043720562011003494, + 0.12014467269182205, + 0.09301504492759705, + 0.11733841896057129, + 1.2624869346618652, + 0.3243844509124756 + ], + [ + -0.33015257120132446, + 0.029943212866783142, + 0.9176104068756104, + -0.43329212069511414, + 0.2955731749534607, + 1.3744759559631348, + 0.6465149521827698, + 0.1046447902917862, + -0.18162041902542114, + 0.6467805504798889, + 0.006262581795454025, + 0.562364935874939, + 1.4995815753936768, + 0.8783308267593384, + 0.7870968580245972, + -0.08096165955066681, + 0.16088604927062988, + -0.006748855113983154, + 0.23232635855674744, + 1.063927173614502, + 0.4299710988998413 + ], + [ + -0.4944083094596863, + -0.08790145069360733, + 1.29244863986969, + -0.3121066093444824, + 0.26694127917289734, + 1.671442985534668, + 0.7521165609359741, + -0.0812029242515564, + -0.29114994406700134, + 0.8142672181129456, + -0.13427919149398804, + 0.5399618148803711, + 1.3985484838485718, + 1.0996084213256836, + 0.47243985533714294, + -0.12033167481422424, + 0.17510870099067688, + 0.14865869283676147, + -0.12706416845321655, + 1.2753329277038574, + 0.5404187440872192 + ], + [ + -0.37128323316574097, + -0.014755124226212502, + 1.141469955444336, + -0.08029656112194061, + 0.2596544623374939, + 1.8179931640625, + 0.9341777563095093, + 0.044848546385765076, + -0.2405034601688385, + 0.8614826202392578, + 0.06876950711011887, + 0.22597625851631165, + 1.7494922876358032, + 1.202664852142334, + 0.2998929023742676, + -0.22034409642219543, + 0.2735995948314667, + 0.3597015142440796, + 0.49169713258743286, + 1.3961365222930908, + 0.8222110271453857 + ] + ], + "generated_utilities": [ + 2.0, + 0.021834811195731163, + 2.0, + 0.019551733508706093, + 0.024202564731240273, + 0.02782593108713627, + 0.005327919963747263, + 0.00631558895111084 + ], + "group_id": "ms-57be9ae181c077e7821f", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.28031715750694275, + 0.19804009795188904, + 0.15024888515472412, + 0.23104076087474823, + 0.21580509841442108, + 0.013450047001242638, + 0.20318475365638733, + 0.22562944889068604, + 0.14178292453289032, + 0.15763869881629944, + 0.27735981345176697, + 0.1909358948469162, + 0.06666654348373413, + 2.0, + 0.060366373509168625, + 0.05085722729563713 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.06739383935928345, + 0.06319573521614075, + 0.8779754638671875, + -0.03682136535644531, + -0.07093041390180588, + 1.3382306098937988, + 0.0, + 0.07168813049793243, + 0.01971740648150444, + 0.4866671562194824, + 0.07402461767196655, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + 0.03574584424495697, + 0.021435141563415527, + 0.0021790266036987305, + -0.06565424799919128, + 1.1393508911132812, + 0.0 + ], + [ + -0.17397063970565796, + -0.08753994852304459, + 0.8779754638671875, + -0.09028738737106323, + -0.17196941375732422, + 1.3382306098937988, + 0.0, + 0.022144019603729248, + 0.1154828891158104, + 0.4866671562194824, + -0.09212043881416321, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + -0.2021142691373825, + -0.022606968879699707, + 0.0021790266036987305, + 0.03272634744644165, + 1.1393508911132812, + 0.0 + ], + [ + -1.0144902467727661, + -0.023572498932480812, + 1.8779754638671875, + -1.0368213653564453, + -0.03799626976251602, + 2.338230609893799, + 2.239948272705078, + 0.16168542206287384, + 0.07221447676420212, + 1.4866671562194824, + -0.3557073175907135, + 1.3622210025787354, + 1.896626591682434, + 2.0382778644561768, + 1.4886415004730225, + 0.1997920423746109, + -0.4386618137359619, + 1.0021790266036987, + -0.3174663782119751, + 2.1393508911132812, + 1.9118322134017944 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.479896545410156, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0765557289123535, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.823664426803589 + ], + [ + -1.0723698139190674, + -0.05779706686735153, + 2.0670831203460693, + -0.830497682094574, + -0.17302533984184265, + 2.2277159690856934, + 2.0435242652893066, + 0.08191673457622528, + 0.08711377531290054, + 1.416495680809021, + -0.5633984804153442, + 1.3216067552566528, + 1.8660295009613037, + 2.2198705673217773, + 1.6887608766555786, + 0.06658947467803955, + -0.4010601043701172, + 1.1803714036941528, + -0.15144284069538116, + 2.308220386505127, + 1.8460347652435303 + ], + [ + -0.8901733756065369, + 0.3378032147884369, + 2.2360360622406006, + -1.207388162612915, + -0.34048303961753845, + 2.460798978805542, + 2.100795030593872, + -0.1847335249185562, + -0.06362440437078476, + 1.2987924814224243, + -0.11972835659980774, + 1.170393943786621, + 2.0620977878570557, + 2.4285032749176025, + 1.3666276931762695, + -0.01825574040412903, + -0.4620283842086792, + 1.1138218641281128, + -0.5787107944488525, + 1.78761887550354, + 1.740113615989685 + ], + [ + -1.1796774864196777, + -0.12369386106729507, + 1.546891689300537, + -1.0500564575195312, + -0.026454759761691093, + 2.177122116088867, + 2.0003931522369385, + 0.13697585463523865, + 0.37715619802474976, + 1.9734747409820557, + -0.16338464617729187, + 1.3775254487991333, + 2.3054146766662598, + 1.5546050071716309, + 1.2645983695983887, + 0.3812609910964966, + 0.02191823720932007, + 1.0944219827651978, + 0.09844094514846802, + 1.8145865201950073, + 2.0624771118164062 + ], + [ + -0.5514534711837769, + -0.4441075921058655, + 1.8971614837646484, + -1.6112515926361084, + 0.1306135058403015, + 2.736365556716919, + 2.270737886428833, + -0.35948747396469116, + 0.03968408331274986, + 1.061530351638794, + -0.29366230964660645, + 1.5066968202590942, + 2.34443998336792, + 2.3445637226104736, + 1.2286460399627686, + 0.13919776678085327, + -0.21014492213726044, + 0.9723342061042786, + -0.5822539329528809, + 1.8932968378067017, + 2.159864664077759 + ], + [ + -0.9470903873443604, + -0.7433359622955322, + 1.798203945159912, + -0.5850675106048584, + -0.7611241340637207, + 2.2761964797973633, + 2.0114798545837402, + 0.7724847793579102, + -0.23453289270401, + 0.7384847402572632, + -1.022956132888794, + 1.0059813261032104, + 1.652310848236084, + 2.6376113891601562, + 2.148331642150879, + 0.9844839572906494, + 0.18531352281570435, + 1.6280735731124878, + -0.06812705099582672, + 2.2684829235076904, + 1.1641287803649902 + ], + [ + -1.729402780532837, + 0.15382106602191925, + 1.0736894607543945, + -0.5141006708145142, + -0.6476962566375732, + 2.0120980739593506, + 2.8703343868255615, + 0.5578619837760925, + -0.8154823184013367, + 1.4325131177902222, + -1.0468122959136963, + 0.9670607447624207, + 1.973939061164856, + 2.7364890575408936, + 1.8303502798080444, + 0.8089333772659302, + -0.41138362884521484, + 1.8146085739135742, + -0.11676108837127686, + 2.3541221618652344, + 2.3875224590301514 + ], + [ + -1.8409323692321777, + -0.2118261456489563, + 1.5354409217834473, + -0.3873547911643982, + 0.7043789029121399, + 2.2068612575531006, + 2.5864486694335938, + 0.8632195591926575, + -0.10360705107450485, + 1.6340630054473877, + -0.017105937004089355, + 1.90720796585083, + 2.608898639678955, + 2.3387808799743652, + 2.3744726181030273, + 0.13692857325077057, + -0.7716406583786011, + 1.3477450609207153, + -0.9811362028121948, + 2.15514874458313, + 2.298004150390625 + ], + [ + -1.2484530210494995, + 0.5977066159248352, + 2.8256869316101074, + -0.3418053388595581, + -1.031638503074646, + 1.7023751735687256, + 1.4487309455871582, + -0.3013373017311096, + -0.3682725131511688, + 2.4404053688049316, + -0.8263542652130127, + 2.1105899810791016, + 2.5769248008728027, + 1.6751525402069092, + 0.8367876410484314, + -0.1554209142923355, + 0.17985862493515015, + 1.0017495155334473, + -0.3176954984664917, + 2.6737403869628906, + 1.7722500562667847 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -1.0015538930892944, + 0.43126925826072693, + 1.2786648273468018, + -0.7182899713516235, + 0.06109866499900818, + 2.323270797729492, + 1.3882524967193604, + -0.2783380150794983, + 0.35085979104042053, + 0.5870303511619568, + -1.0134913921356201, + 0.4675567150115967, + 0.9185858964920044, + 1.9441449642181396, + 1.3693488836288452, + 0.05759717524051666, + 0.19058758020401, + 1.9360744953155518, + -1.0018632411956787, + 2.043489456176758, + 1.1226990222930908 + ] + ], + "predicted_scores": [ + 6.654320240020752, + 6.55549430847168, + 6.52157735824585, + 6.20645809173584, + 5.943201065063477, + 5.8155646324157715, + 5.276596546173096, + 5.020887851715088 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "9c4de49d50d74c8c571620759fff1d43be23093a5f66e946f7ae6d75cb26ade5", + "stored_base_utility": 0.28031715750694275, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1544450968503952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1544450968503952 + }, + "base_progress": 0.1544450968503952, + "base_success": false, + "base_utility": 0.1544450968503952, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06245501711964607, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06245501711964607 + }, + { + "contact_quality": null, + "progress": 0.057290513068437576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057290513068437576 + }, + { + "contact_quality": null, + "progress": 0.05731761455535889, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05731761455535889 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06114928051829338, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06114928051829338 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.056892890483140945, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.056892890483140945 + }, + { + "contact_quality": null, + "progress": 0.052340131253004074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.052340131253004074 + } + ], + "candidate_progress": [ + 0.06245501711964607, + 0.057290513068437576, + 0.05731761455535889, + 1.0, + 0.06114928051829338, + 1.0, + 0.056892890483140945, + 0.052340131253004074 + ], + "candidate_source_chart_ids": [ + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-6388b5d2356c556c3e55", + "generated_tangents": [ + [ + 0.14085455238819122, + 0.06595045328140259, + 1.8042998313903809, + -0.10692273825407028, + 0.7195324301719666, + 2.251777410507202, + 2.3358867168426514, + 2.566293716430664, + -0.19326137006282806, + -0.9937949180603027, + 2.420006036758423, + -0.030497834086418152, + 1.7948921918869019, + 1.7487417459487915, + -0.09461890161037445, + -0.10772505402565002, + 1.1486867666244507, + -0.1017664223909378, + 1.311736822128296, + 3.0273165702819824, + 3.01809024810791 + ], + [ + 0.16229595243930817, + 0.08912916481494904, + 1.6193690299987793, + -0.1410122513771057, + 0.8347203135490417, + 2.2059223651885986, + 2.0994341373443604, + 2.6510233879089355, + -0.291916161775589, + -0.9467397928237915, + 2.579068183898926, + -0.11217635869979858, + 1.8465524911880493, + 1.8256187438964844, + -0.054164987057447433, + -0.18806178867816925, + 1.0668989419937134, + -0.00449015200138092, + 1.264209508895874, + 2.879173755645752, + 3.214197874069214 + ], + [ + 0.03580384701490402, + 0.22809405624866486, + 2.0692615509033203, + -0.4423205256462097, + 0.892505943775177, + 2.2922213077545166, + 2.545778274536133, + 2.5572688579559326, + -0.26238247752189636, + -0.6881310939788818, + 2.270177125930786, + 0.20732051134109497, + 2.214596748352051, + 2.104609489440918, + 0.03223583102226257, + 0.19487497210502625, + 0.9772140383720398, + 0.009927839040756226, + 1.5587480068206787, + 2.3638627529144287, + 3.3562300205230713 + ], + [ + 0.0926257073879242, + 0.04837973415851593, + 1.2499054670333862, + -0.3065544068813324, + 0.8986547589302063, + 2.4435501098632812, + 2.1432111263275146, + 2.4063634872436523, + -0.3533810079097748, + -0.6136487722396851, + 2.9670586585998535, + -0.06668078899383545, + 2.0088655948638916, + 2.0828006267547607, + -0.3177120089530945, + -0.014724710024893284, + 0.974626362323761, + -0.026060760021209717, + 1.354006052017212, + 2.240135669708252, + 2.896519660949707 + ], + [ + 0.07820472866296768, + 0.25062239170074463, + 1.8994182348251343, + -0.5110874176025391, + 0.8189709782600403, + 2.1727852821350098, + 2.6295180320739746, + 2.643580198287964, + -0.29673078656196594, + -0.6997730731964111, + 2.2748427391052246, + 0.0454077273607254, + 2.230524778366089, + 2.1575260162353516, + 0.14411985874176025, + 0.2165374904870987, + 0.8992494940757751, + -0.006085202097892761, + 1.5451223850250244, + 2.374079704284668, + 2.979363441467285 + ], + [ + 0.3065474331378937, + -0.1955948770046234, + 1.202223539352417, + 0.05112254619598389, + 0.8425878882408142, + 2.120248317718506, + 1.3745603561401367, + 2.951903820037842, + -0.5075027942657471, + -0.490806519985199, + 2.6812856197357178, + 0.1408054232597351, + 1.8529664278030396, + 2.296215295791626, + 0.020097725093364716, + -0.019534600898623466, + 1.205973744392395, + 0.013922587037086487, + 1.3845617771148682, + 2.048712968826294, + 3.2170848846435547 + ], + [ + 0.1498587429523468, + 0.24786995351314545, + 1.7572414875030518, + -0.5540152788162231, + 0.8134774565696716, + 1.9461852312088013, + 2.737715721130371, + 2.755290985107422, + -0.3167344927787781, + -0.7857706546783447, + 2.2916619777679443, + 0.014517530798912048, + 2.2775135040283203, + 2.3802807331085205, + 0.15740102529525757, + 0.17527072131633759, + 0.9125971794128418, + 0.030883222818374634, + 1.5133967399597168, + 2.335270404815674, + 2.915964365005493 + ], + [ + 0.5088602900505066, + 0.10750138759613037, + 1.8345903158187866, + -0.16440215706825256, + 0.8437427878379822, + 2.507436752319336, + 1.8617327213287354, + 2.5663468837738037, + -0.08433163166046143, + -0.8705015778541565, + 2.830353260040283, + 0.49730610847473145, + 1.8448563814163208, + 1.9258184432983398, + 0.015072338283061981, + -0.058465342968702316, + 1.3149197101593018, + 0.10777977854013443, + 1.3854131698608398, + 3.1543467044830322, + 3.2663450241088867 + ] + ], + "generated_utilities": [ + 0.06245501711964607, + 0.057290513068437576, + 0.05731761455535889, + 2.0, + 0.06114928051829338, + 2.0, + 0.056892890483140945, + 0.052340131253004074 + ], + "group_id": "ms-6388b5d2356c556c3e55", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08186312764883041, + 0.11667267978191376, + 0.09751813113689423, + 2.0, + 0.27431026101112366, + 0.11760994046926498, + 0.20386962592601776, + 0.19832774996757507, + 0.22735612094402313, + 0.1356993168592453, + 0.1713169515132904, + 0.1792232096195221, + 0.06848804652690887, + 0.1094626635313034, + 0.11555596441030502, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0018978863954544067, + -0.4448504149913788, + 0.7108399868011475, + -0.33157044649124146, + -0.062200844287872314, + 1.4758033752441406, + 0.0, + 1.4153923988342285, + 0.14116239547729492, + 0.00777280330657959, + 0.5541017055511475, + 0.07029426097869873, + 0.26837193965911865, + 0.0, + -0.015542566776275635, + 0.3573532998561859, + 0.6168417930603027, + 0.3612897992134094, + 0.369077205657959, + 1.5924487113952637, + 0.0 + ], + [ + -0.09549225866794586, + -0.20593886077404022, + 1.7108399868011475, + -0.1554051786661148, + 0.9377991557121277, + 2.4758033752441406, + 2.4156031608581543, + 2.4153923988342285, + -0.2459496259689331, + -0.9922271966934204, + 1.5541017055511475, + 0.4461512267589569, + 1.2501436471939087, + 1.7566912174224854, + -0.8014360666275024, + 0.04551727697253227, + 1.6168417930603027, + 0.5393297672271729, + 1.369077205657959, + 2.5924487113952637, + 3.1126015186309814 + ], + [ + -0.11610143631696701, + -0.3961976170539856, + 1.9081053733825684, + -0.14575888216495514, + 1.1381605863571167, + 2.7434439659118652, + 2.5136890411376953, + 2.6401467323303223, + -0.45453745126724243, + -0.9589858651161194, + 1.4083189964294434, + 0.5800129175186157, + 0.9940754175186157, + 1.7450767755508423, + -1.0145976543426514, + 0.3312625586986542, + 1.4729199409484863, + 0.2520747482776642, + 1.3623151779174805, + 2.5670394897460938, + 3.3589630126953125 + ], + [ + -0.2318952977657318, + 0.024853751063346863, + 1.829875111579895, + 0.09877045452594757, + 1.1515556573867798, + 2.2239503860473633, + 2.153250217437744, + 2.8057901859283447, + -0.1800437569618225, + -1.2225371599197388, + 1.421208143234253, + 0.6627710461616516, + 1.3895055055618286, + 1.856442928314209, + -0.5928603410720825, + 0.4364123046398163, + 1.7852745056152344, + 0.3345714807510376, + 1.1167365312576294, + 2.7676005363464355, + 3.041402578353882 + ], + [ + -0.2227625697851181, + -0.012646213173866272, + 1.7988159656524658, + 0.21457992494106293, + 0.8877373933792114, + 2.9257700443267822, + 2.2101094722747803, + 2.2874886989593506, + -0.09066365659236908, + -0.49909964203834534, + 2.042389392852783, + 0.6287553906440735, + 1.074569582939148, + 1.7805378437042236, + -1.1840007305145264, + 0.3728531002998352, + 1.165173053741455, + 0.3054528832435608, + 1.4924153089523315, + 2.5540966987609863, + 2.664964199066162 + ], + [ + -0.6864494681358337, + 0.119636669754982, + 1.2460956573486328, + -0.6367122530937195, + 1.4080986976623535, + 2.5262017250061035, + 2.508194923400879, + 2.1323020458221436, + -0.43302130699157715, + -1.2678899765014648, + 1.0870052576065063, + 0.46296897530555725, + 0.765267550945282, + 2.3093044757843018, + -0.46595194935798645, + -0.43870145082473755, + 2.028986930847168, + 0.26284897327423096, + 1.5542359352111816, + 2.530392646789551, + 2.7435970306396484 + ], + [ + 0.18175412714481354, + -0.478390634059906, + 1.286731481552124, + -0.8066962361335754, + 1.4787592887878418, + 2.1033878326416016, + 1.750453233718872, + 2.047861099243164, + 0.3163147568702698, + -0.30588728189468384, + 1.0766479969024658, + 0.5822455286979675, + 1.673675775527954, + 2.039386510848999, + -0.62891685962677, + 0.615638792514801, + 2.018416404724121, + 0.2530737817287445, + 1.34544837474823, + 3.2287912368774414, + 2.9459850788116455 + ], + [ + 0.5059412121772766, + 0.17952178418636322, + 1.3319568634033203, + -0.8850483894348145, + 0.6336387395858765, + 1.775376796722412, + 2.3280270099639893, + 1.7305657863616943, + -0.02617873251438141, + -0.7963138818740845, + 1.9142115116119385, + 0.6635866165161133, + 1.3493314981460571, + 1.9258644580841064, + -0.3795176148414612, + 0.7861210107803345, + 1.0032029151916504, + 0.8491684794425964, + 0.5918128490447998, + 2.399027109146118, + 2.3932981491088867 + ], + [ + -0.34448128938674927, + -0.2996852695941925, + 0.9233196377754211, + -0.22890916466712952, + 1.6932814121246338, + 3.465762138366699, + 2.2005038261413574, + 2.6472361087799072, + -0.8981531858444214, + -0.2658765912055969, + 1.8755900859832764, + 0.2962568402290344, + 1.8239326477050781, + 1.0176585912704468, + -0.7912051677703857, + 0.6727061867713928, + 2.199978828430176, + 1.015731692314148, + 0.43087130784988403, + 3.5876548290252686, + 3.893789052963257 + ] + ], + "predicted_scores": [ + 3.226083755493164, + 2.7768771648406982, + 2.7206974029541016, + 2.6579699516296387, + 1.9610471725463867, + 1.883847713470459, + 1.7879574298858643, + 1.453969955444336 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "38d539e46140304e3f157cbc207aa0d9489e5470633c216a0379906af445a1ec", + "stored_base_utility": 0.08186312764883041, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06173701956868172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06173701956868172 + }, + "base_progress": 0.06173701956868172, + "base_success": false, + "base_utility": 0.06173701956868172, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10160752385854721, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10160752385854721 + }, + { + "contact_quality": null, + "progress": 0.01461742352694273, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01461742352694273 + }, + { + "contact_quality": null, + "progress": 0.046429235488176346, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.046429235488176346 + }, + { + "contact_quality": null, + "progress": 0.029767373576760292, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029767373576760292 + }, + { + "contact_quality": null, + "progress": 0.10250582545995712, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10250582545995712 + }, + { + "contact_quality": null, + "progress": 0.15528590977191925, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15528590977191925 + }, + { + "contact_quality": null, + "progress": 0.03336955979466438, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03336955979466438 + }, + { + "contact_quality": null, + "progress": 0.06291665881872177, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06291665881872177 + } + ], + "candidate_progress": [ + 0.10160752385854721, + 0.01461742352694273, + 0.046429235488176346, + 0.029767373576760292, + 0.10250582545995712, + 0.15528590977191925, + 0.03336955979466438, + 0.06291665881872177 + ], + "candidate_source_chart_ids": [ + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-d535b38d768633f6d922", + "ms-a2518826fa0a816cc46f", + "ms-3a4e2d080c9c4b279895", + "ms-7e88428497a6b13936c4", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-638d2ba1917eff4d09e0", + "generated_tangents": [ + [ + 0.20012439787387848, + -0.3785173296928406, + 0.28743481636047363, + 0.057181090116500854, + 1.5917069911956787, + -0.11141654849052429, + 0.5379866361618042, + 0.11633799970149994, + -0.04841218888759613, + 0.24334482848644257, + 0.07627934217453003, + 0.6672567129135132, + 0.03587189316749573, + 0.08851487934589386, + 0.7830518484115601, + 0.1023007184267044, + -0.3814665675163269, + -0.2820456027984619, + 0.9330448508262634, + -1.4249541759490967, + -0.9199477434158325 + ], + [ + 0.03092768043279648, + -0.4322233200073242, + 0.2368573248386383, + 0.17836228013038635, + 1.8263330459594727, + 0.304324746131897, + 0.5671171545982361, + -0.05702170729637146, + -0.057950958609580994, + 0.2906959056854248, + 0.27702105045318604, + 0.8885995149612427, + 0.09528079628944397, + -0.31255626678466797, + 0.4552042782306671, + -0.030130892992019653, + -0.19965483248233795, + -0.4369654059410095, + 0.798297643661499, + -1.5233819484710693, + -0.9924530982971191 + ], + [ + 0.13268131017684937, + -0.38998734951019287, + 0.4595482051372528, + 0.13375499844551086, + 1.5339815616607666, + 0.18641261756420135, + 0.593211829662323, + 0.1785918027162552, + -0.05231153219938278, + 0.09126681834459305, + 0.027456795796751976, + 0.6521029472351074, + -0.07697564363479614, + -0.16427795588970184, + 0.6907652020454407, + 0.1520497351884842, + -0.4432736039161682, + -0.23638613522052765, + 0.711456298828125, + -1.4344983100891113, + -0.8440983295440674 + ], + [ + 0.01169373095035553, + -0.46332138776779175, + 0.2159072756767273, + -0.0831880271434784, + 1.592785120010376, + 0.417820006608963, + 1.3587965965270996, + 0.2749141454696655, + -0.261660635471344, + 0.23481294512748718, + -0.15995407104492188, + 0.545703649520874, + 0.33227142691612244, + 0.0875142440199852, + 0.6457905769348145, + 0.003443394787609577, + -0.46850788593292236, + -0.08103956282138824, + 0.658976137638092, + -0.8811950087547302, + -0.45714953541755676 + ], + [ + 0.12743258476257324, + -0.4842522144317627, + 0.1821211576461792, + -0.11842761188745499, + 1.7030308246612549, + 0.09627389162778854, + 0.49623584747314453, + 0.20606066286563873, + -0.14327944815158844, + 0.20020434260368347, + 0.1686018705368042, + 0.9053546190261841, + 0.15055426955223083, + -0.40241777896881104, + 0.6978731155395508, + 0.048628415912389755, + -0.3393581509590149, + -0.20582269132137299, + 0.8972986936569214, + -1.3415148258209229, + -0.7074575424194336 + ], + [ + 0.3362257480621338, + -0.2868293821811676, + 0.11786612868309021, + 0.3090389668941498, + 1.3401665687561035, + 0.017566069960594177, + 0.4646468758583069, + 0.39307647943496704, + 0.0022079935297369957, + 0.13397003710269928, + 0.20744341611862183, + 0.8147593140602112, + 0.8614964485168457, + 0.2619123160839081, + 0.529552161693573, + 0.09387147426605225, + -0.2555035948753357, + -0.018945425748825073, + 0.5246317386627197, + -0.565754771232605, + -0.4137805998325348 + ], + [ + -0.029415994882583618, + -0.34208476543426514, + 0.0194321870803833, + -0.3278515338897705, + 1.7799267768859863, + 0.023271575570106506, + 0.6620983481407166, + 0.10545758903026581, + -0.175527885556221, + 0.08659183979034424, + 0.2616179585456848, + 0.9107527732849121, + 0.19200876355171204, + -0.20335085690021515, + 0.7550808191299438, + 0.04693594202399254, + -0.3627760410308838, + -0.36395782232284546, + 0.8388381004333496, + -1.1179988384246826, + -0.8995155096054077 + ], + [ + 0.20678108930587769, + -0.5316930413246155, + 0.5591402649879456, + -0.1520969569683075, + 1.5175111293792725, + 0.5175721645355225, + 0.4183806777000427, + 0.3324818015098572, + -0.12351128458976746, + 0.34945082664489746, + 0.09516577422618866, + 1.0129802227020264, + 0.2763718068599701, + 0.2244502753019333, + 0.7656605839729309, + 0.03992048278450966, + -0.4802477955818176, + -0.2962467670440674, + 0.9774383306503296, + -1.3668091297149658, + -0.4972158968448639 + ] + ], + "generated_utilities": [ + 0.10160752385854721, + 0.01461742352694273, + 0.046429235488176346, + 0.029767373576760292, + 0.10250582545995712, + 0.15528590977191925, + 0.03336955979466438, + 0.06291665881872177 + ], + "group_id": "ms-638d2ba1917eff4d09e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.028925320133566856, + 0.11176212877035141, + 0.057868462055921555, + 0.09960909932851791, + 0.07349967956542969, + 0.029011588543653488, + 0.06443297863006592, + 0.05166193097829819, + 0.025834282860159874, + 0.056667208671569824, + 0.04477008432149887, + 0.019685089588165283, + 0.018628716468811035, + 0.08587563037872314, + 0.02232462167739868, + 0.010265171527862549 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04120570421218872, + 0.038339823484420776, + 0.5979268550872803, + -0.05306033045053482, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.3407015800476074, + -0.21580028533935547, + -0.036036908626556396, + -0.004389047622680664, + 0.5364620685577393, + 0.026709645986557007, + 0.0, + 1.0126078128814697, + -0.10034149885177612, + -0.16565871238708496, + 0.07125610113143921, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + -0.5276790261268616, + -0.19392354786396027, + 0.5979268550872803, + 0.05016906559467316, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.6852816939353943, + -0.21580028533935547, + 0.04531368613243103, + -0.09276384115219116, + 0.5364620685577393, + 0.021915271878242493, + 0.0, + 1.0126078128814697, + 0.29255303740501404, + -0.16565871238708496, + 0.14600259065628052, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + 0.5910290479660034, + -0.038985610008239746, + 1.5555552244186401, + -1.028314232826233, + 2.3103959560394287, + 0.8163888454437256, + 2.064336061477661, + 1.168483018875122, + -0.21718227863311768, + -0.19770365953445435, + 0.11310696601867676, + 0.6850457191467285, + -0.06804794818162918, + 0.33076921105384827, + 1.7529149055480957, + -0.562712550163269, + -1.6352436542510986, + 0.1224215030670166, + 3.354771852493286, + -1.1317338943481445, + -1.7348618507385254 + ] + ], + "predicted_scores": [ + 9.201016426086426, + 9.197566986083984, + 9.097320556640625, + 8.973604202270508, + 8.596654891967773, + 8.47115421295166, + 8.250267028808594, + 7.663734436035156 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "5d6f4227fab3b4bb36756c9a15c32b6fe6c9a0ae4b07b3f20a2bd65c2594df13", + "stored_base_utility": 0.028925320133566856, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.016599038615822792, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016599038615822792 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.005813241004943848, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005813241004943848 + }, + { + "contact_quality": null, + "progress": 0.011173486709594727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011173486709594727 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.016599038615822792, + 1.0, + 0.005813241004943848, + 0.011173486709594727, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-a6a6de64c39abad45f86", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-652ad552ba98d2736620", + "generated_tangents": [ + [ + 0.7725136876106262, + -0.03431672602891922, + 0.8048340678215027, + -0.6374152898788452, + 1.5430598258972168, + 1.4735641479492188, + 0.8320761919021606, + 0.06031659245491028, + -0.11114482581615448, + 0.6126077771186829, + 0.6522234678268433, + 1.0537309646606445, + 0.13850536942481995, + -0.1393839567899704, + -0.18330305814743042, + -0.2873992323875427, + 1.2469110488891602, + -0.3705272078514099, + 1.253023386001587, + 2.37744402885437, + 1.3933371305465698 + ], + [ + 0.3059130311012268, + 0.1567693054676056, + 1.0042239427566528, + -0.9915871620178223, + 2.010279417037964, + 1.189953327178955, + 1.4461641311645508, + 0.05451717972755432, + -0.032694846391677856, + 0.6890350580215454, + 0.657055675983429, + 1.0749082565307617, + 0.8170125484466553, + 0.39462989568710327, + -0.14280761778354645, + -0.170671746134758, + 1.371072769165039, + -0.31456679105758667, + 1.4619224071502686, + 2.193998098373413, + 1.4680534601211548 + ], + [ + 0.21431779861450195, + 0.112717404961586, + 1.1256744861602783, + -1.0663964748382568, + 1.601625680923462, + 1.118420958518982, + 0.9243471622467041, + -0.02183324098587036, + -0.036620594561100006, + 0.9164050221443176, + 0.8578189015388489, + 1.2168374061584473, + 1.1195783615112305, + 0.2532307505607605, + 0.044171541929244995, + -0.03379511088132858, + 1.4963239431381226, + -0.1239987462759018, + 1.2028415203094482, + 1.8915939331054688, + 2.001439332962036 + ], + [ + 0.3998737335205078, + 0.1277637630701065, + 1.0006946325302124, + -0.688167929649353, + 1.4692552089691162, + 0.8067119121551514, + 1.0645239353179932, + -0.04656617343425751, + -0.11209604144096375, + 0.7425906658172607, + 0.7275300025939941, + 1.2272968292236328, + 0.40083661675453186, + 0.3038826584815979, + -0.1416773945093155, + -0.09133654832839966, + 1.551317811012268, + -0.325903058052063, + 1.350942850112915, + 2.2170968055725098, + 1.2916264533996582 + ], + [ + 0.3216557204723358, + 0.20781610906124115, + 0.8242918252944946, + -1.1267178058624268, + 2.052201509475708, + 1.4753129482269287, + 1.241973638534546, + 0.19720833003520966, + -0.13343657553195953, + 0.603374183177948, + 0.6644425392150879, + 1.220246434211731, + 0.5619497299194336, + 0.6616710424423218, + -0.11985982954502106, + -0.12091799080371857, + 1.4248565435409546, + -0.46000170707702637, + 1.15903639793396, + 1.783252477645874, + 1.544547438621521 + ], + [ + 0.2983326315879822, + 0.18560101091861725, + 0.9183748960494995, + -0.9748181104660034, + 1.922440767288208, + 1.3594715595245361, + 1.3505611419677734, + 0.055494919419288635, + -0.06414700299501419, + 0.6860648989677429, + 0.6104075312614441, + 1.1296989917755127, + 0.5256576538085938, + 0.320169061422348, + -0.09568566083908081, + -0.15992745757102966, + 1.4739841222763062, + -0.22081340849399567, + 1.463674545288086, + 2.2050154209136963, + 1.76393723487854 + ], + [ + 0.3417985439300537, + 0.002032928168773651, + 0.7763372659683228, + -1.1412901878356934, + 2.0572874546051025, + 2.0098888874053955, + 1.3594467639923096, + 0.29879310727119446, + -0.24184048175811768, + 0.858045756816864, + 0.8992239832878113, + 0.896336555480957, + 0.3268645107746124, + 0.7245426774024963, + -0.16418415307998657, + -0.1624162793159485, + 1.325331211090088, + -0.4718431830406189, + 1.2613296508789062, + 2.447932243347168, + 2.0713791847229004 + ], + [ + 0.21297046542167664, + 0.046242982149124146, + 1.0241634845733643, + -0.5505623817443848, + 1.6444051265716553, + 1.4813787937164307, + 0.8535062074661255, + 0.1877230852842331, + -0.09278794378042221, + 0.409674733877182, + 0.397246390581131, + 0.8508703112602234, + 0.20856890082359314, + 0.3168705701828003, + -0.07951639592647552, + -0.22509704530239105, + 1.1620877981185913, + -0.0015839487314224243, + 0.9055641889572144, + 2.1325931549072266, + 1.0961734056472778 + ] + ], + "generated_utilities": [ + 0.016599038615822792, + 2.0, + 0.005813241004943848, + 0.011173486709594727, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-652ad552ba98d2736620", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08378833532333374, + 0.10458396375179291, + 0.19086095690727234, + 0.04039820283651352, + 0.15980230271816254, + 0.0428888201713562, + 0.2367449700832367, + 0.18185830116271973, + 0.17593276500701904, + 0.16103637218475342, + 0.15250858664512634, + 0.05226989835500717, + 0.1701633185148239, + 0.11590366065502167, + 0.05217729136347771, + 0.03575938940048218 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.09009888768196106, + 0.08798432350158691, + 0.718157172203064, + -0.6351244449615479, + 1.405301570892334, + 0.6596384048461914, + 0.0, + 0.060225605964660645, + -0.0894029438495636, + -0.10323715209960938, + 0.052752673625946045, + 0.5354657173156738, + -0.013395614922046661, + 0.0, + -0.3450838029384613, + -0.11172826588153839, + 0.7912200689315796, + 0.33702391386032104, + 0.5586241483688354, + 1.8093180656433105, + 0.0 + ], + [ + 0.557296633720398, + 1.087984323501587, + 1.718157172203064, + -1.6351244449615479, + 2.405301570892334, + 1.6596384048461914, + 1.7430553436279297, + 0.3360595703125, + -0.2521040737628937, + 0.8967628479003906, + 0.26587578654289246, + 1.5354657173156738, + -0.12294083833694458, + -0.022678710520267487, + -0.416314959526062, + -0.26016199588775635, + 1.7912200689315796, + -0.16825227439403534, + 1.5586241483688354, + 2.8093180656433105, + 2.2589924335479736 + ], + [ + 0.2635665237903595, + 1.0005124807357788, + 1.973160743713379, + -1.7548720836639404, + 2.176497459411621, + 1.5416926145553589, + 1.9556171894073486, + 0.2215275913476944, + -0.5432531833648682, + 0.8434366583824158, + 0.3559061288833618, + 1.6657240390777588, + -0.3025258779525757, + 0.04401832073926926, + -0.17062894999980927, + -0.1509227454662323, + 1.7901219129562378, + 0.0010261386632919312, + 1.726505994796753, + 2.9778714179992676, + 2.0530292987823486 + ], + [ + 0.23987123370170593, + 0.8904718160629272, + 1.467047929763794, + -1.990342378616333, + 2.7685346603393555, + 2.0486786365509033, + 2.013002395629883, + 0.006436288356781006, + -0.25436922907829285, + 1.1101897954940796, + 0.4448293447494507, + 1.3105798959732056, + 0.07367905974388123, + 0.23779401183128357, + -0.34580594301223755, + -0.35948312282562256, + 1.4435756206512451, + -0.35472720861434937, + 1.801603078842163, + 3.0683224201202393, + 2.2330009937286377 + ], + [ + 0.6738064289093018, + 1.3973212242126465, + 2.0960373878479004, + -1.735988736152649, + 2.5216798782348633, + 1.1975955963134766, + 2.234865665435791, + 0.4602587819099426, + -0.7169049382209778, + 0.9412583708763123, + 0.7081141471862793, + 1.1730585098266602, + -0.016052812337875366, + -0.10474976152181625, + -0.03458032011985779, + 0.044317394495010376, + 1.6339548826217651, + -0.5292693376541138, + 1.5412750244140625, + 3.018833875656128, + 2.6051149368286133 + ], + [ + 0.6305910348892212, + 1.540043592453003, + 1.626009464263916, + -1.8823078870773315, + 2.2431485652923584, + 2.1866872310638428, + 2.3003249168395996, + 0.25838330388069153, + -0.2936764061450958, + 1.2413102388381958, + -0.12014040350914001, + 1.743309497833252, + 0.3618801534175873, + -0.37178292870521545, + 0.11553955078125, + 0.04856392741203308, + 1.2029554843902588, + 0.04960852861404419, + 2.0190422534942627, + 2.5883281230926514, + 2.7636377811431885 + ], + [ + 0.5004369616508484, + 1.4795310497283936, + 2.325191020965576, + -1.7464677095413208, + 1.9271103143692017, + 1.2483245134353638, + 1.9852193593978882, + 0.24291110038757324, + 0.4463852345943451, + 0.5267151594161987, + 0.32179588079452515, + 1.2356117963790894, + 0.03201058506965637, + 0.2881847023963928, + -1.0345205068588257, + -0.4171831011772156, + 1.5080864429473877, + 0.11479203402996063, + 1.9481834173202515, + 2.840816020965576, + 2.427428960800171 + ], + [ + -0.3133848309516907, + 0.43174320459365845, + 2.305823802947998, + -1.657350778579712, + 2.010749578475952, + 1.9222235679626465, + 1.3648147583007812, + 0.1938892900943756, + -0.6084526777267456, + 0.7855725288391113, + -0.07715931534767151, + 0.9140809178352356, + -0.9242425560951233, + -0.4681842029094696, + -0.07827043533325195, + -0.23733587563037872, + 0.9328996539115906, + -0.6088451147079468, + 0.8040639162063599, + 2.620466947555542, + 2.5148611068725586 + ] + ], + "predicted_scores": [ + 6.483285427093506, + 6.3353800773620605, + 6.225844383239746, + 6.091470718383789, + 5.966473579406738, + 5.65541410446167, + 5.353701114654541, + 5.223989009857178 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "8c1ab306d1e82f81085196c60b3c846d2e2eb20ebc538597af359a6252623e7e", + "stored_base_utility": 0.08378833532333374, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03784912824630737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03784912824630737 + }, + "base_progress": 0.03784912824630737, + "base_success": false, + "base_utility": 0.03784912824630737, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10381317138671875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10381317138671875 + }, + { + "contact_quality": null, + "progress": 0.034510593861341476, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.034510593861341476 + }, + { + "contact_quality": null, + "progress": 0.03771030902862549, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03771030902862549 + }, + { + "contact_quality": null, + "progress": 0.04082806780934334, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04082806780934334 + }, + { + "contact_quality": null, + "progress": 0.051552534103393555, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.051552534103393555 + }, + { + "contact_quality": null, + "progress": 0.04706305265426636, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04706305265426636 + }, + { + "contact_quality": null, + "progress": 0.033499401062726974, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.033499401062726974 + }, + { + "contact_quality": null, + "progress": 0.036169927567243576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.036169927567243576 + } + ], + "candidate_progress": [ + 0.10381317138671875, + 0.034510593861341476, + 0.03771030902862549, + 0.04082806780934334, + 0.051552534103393555, + 0.04706305265426636, + 0.033499401062726974, + 0.036169927567243576 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-48178f52a96de3d1ead0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-674fad0e7c042d23629e", + "generated_tangents": [ + [ + 1.2150087356567383, + 0.09120199084281921, + -1.1203824281692505, + -0.20854628086090088, + -0.32002097368240356, + -1.7052526473999023, + -0.3327398896217346, + 0.4944519102573395, + 0.21956883370876312, + -0.5045254826545715, + -0.34264978766441345, + -0.2553563714027405, + -0.9397702217102051, + -0.7462403178215027, + -0.001640181988477707, + 0.1612023413181305, + -0.07497359812259674, + -1.3687654733657837, + -0.3035293221473694, + -0.4836989641189575, + -0.18741783499717712 + ], + [ + 0.9599210619926453, + 0.05319547653198242, + -0.886374831199646, + 0.03741931915283203, + -0.28284740447998047, + -2.136850357055664, + -0.46446549892425537, + 0.5796945095062256, + 0.1916380375623703, + -0.5130059123039246, + -0.7689229846000671, + -0.499808132648468, + -1.1670085191726685, + -0.8926345109939575, + 0.008976638317108154, + 0.13501359522342682, + 0.09401464462280273, + -1.3207203149795532, + -0.32666802406311035, + -0.2381674349308014, + 0.0005655288696289062 + ], + [ + 1.250658631324768, + 0.06865726411342621, + -0.7989288568496704, + -0.05752471089363098, + -0.21533417701721191, + -2.410933017730713, + -0.4800800085067749, + 0.6189042329788208, + 0.19408512115478516, + -0.551322877407074, + -0.679336667060852, + -0.30612629652023315, + -0.9502570629119873, + -1.1006320714950562, + 0.22709600627422333, + 0.11564396321773529, + 0.013436108827590942, + -1.2888011932373047, + -0.2702549695968628, + -0.49332308769226074, + -0.01012738049030304 + ], + [ + 1.2414168119430542, + -0.02945125102996826, + -0.6219155788421631, + 0.16242310404777527, + -0.009782344102859497, + -1.6693488359451294, + -0.7143471837043762, + 0.3483055531978607, + 0.19092519581317902, + -0.39566612243652344, + -0.4636634290218353, + -0.19324156641960144, + -1.0706819295883179, + -0.8795170783996582, + 0.15468624234199524, + -0.02661825716495514, + 0.33147311210632324, + -1.3698220252990723, + -0.3245302438735962, + -0.5012667179107666, + -0.028592616319656372 + ], + [ + 1.181836724281311, + -0.022661203518509865, + -0.7083592414855957, + 0.07817289233207703, + -0.1884171962738037, + -1.9427238702774048, + -0.8347828984260559, + 0.35041749477386475, + 0.1952485591173172, + -0.45024192333221436, + -0.49669674038887024, + -0.16414275765419006, + -0.7385827302932739, + -0.7952350378036499, + 0.11043927073478699, + -0.08943100273609161, + 0.10465411096811295, + -1.1381027698516846, + -0.5002793669700623, + -0.4082665741443634, + 0.019734010100364685 + ], + [ + 1.087027907371521, + 0.09609384834766388, + -0.7468534708023071, + 0.030354589223861694, + -0.18484348058700562, + -2.2053050994873047, + -0.5580365061759949, + 0.5209965705871582, + 0.21578319370746613, + -0.513771116733551, + -0.7245553731918335, + -0.3617209196090698, + -1.0672869682312012, + -0.8714480400085449, + 0.07010158151388168, + 0.12505894899368286, + 0.07052204012870789, + -1.2415176630020142, + -0.3515787124633789, + -0.26459744572639465, + -0.14684909582138062 + ], + [ + 1.285050392150879, + -0.05704961344599724, + -0.6177409887313843, + 0.1098192036151886, + 0.018284335732460022, + -1.8350772857666016, + -0.923826277256012, + 0.40002524852752686, + 0.12818598747253418, + -0.36871203780174255, + -0.510184109210968, + -0.18622660636901855, + -0.7467116117477417, + -0.9038758277893066, + 0.07761846482753754, + 0.01538304053246975, + 0.2775520086288452, + -1.2692476511001587, + -0.38015347719192505, + -0.4504619836807251, + 0.0455203652381897 + ], + [ + 1.344892144203186, + 0.05414992570877075, + -0.5603504180908203, + -0.09251777082681656, + -0.17817610502243042, + -2.0477712154388428, + -0.0023575127124786377, + 0.6702426075935364, + 0.2437015026807785, + -0.49742692708969116, + -0.8353731036186218, + 0.2615560293197632, + -1.0196740627288818, + -1.1268343925476074, + 0.15198864042758942, + 0.14432001113891602, + 0.1254526972770691, + -1.67019522190094, + -0.4600372314453125, + -0.6491325497627258, + -0.47356680035591125 + ] + ], + "generated_utilities": [ + 0.10381317138671875, + 0.034510593861341476, + 0.03771030902862549, + 0.04082806780934334, + 0.051552534103393555, + 0.04706305265426636, + 0.033499401062726974, + 0.036169927567243576 + ], + "group_id": "ms-674fad0e7c042d23629e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07829316705465317, + 0.080319844186306, + 0.11335399001836777, + 0.15264075994491577, + 0.14950451254844666, + 0.07767044007778168, + 0.12988285720348358, + 0.10878288745880127, + 0.15573063492774963, + 0.10951457917690277, + 0.06582474708557129, + 0.02477661892771721, + 0.07745532691478729, + 0.03348777815699577, + 0.10050909221172333, + 0.033770203590393066 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.5533217191696167, + 0.5758762359619141, + -0.6480545997619629, + -0.10892429947853088, + 0.7943102717399597, + -2.7399702072143555, + -1.8395006656646729, + 1.447904109954834, + 1.4495785236358643, + -0.055675506591796875, + -1.3537942171096802, + 0.38944748044013977, + -1.6196211576461792, + -1.2710431814193726, + -0.9744676947593689, + 0.057668983936309814, + 1.8350472450256348, + -1.2618876695632935, + -1.2385435104370117, + -1.7976703643798828, + -1.9491409063339233 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9072847366333008, + 0.06923278421163559, + -0.14729690551757812, + 0.15558937191963196, + 0.40412187576293945, + -2.342223644256592, + 0.0, + 0.2589508295059204, + 0.35182610154151917, + 0.10176032781600952, + -0.3489869236946106, + 0.13150231540203094, + -1.3123579025268555, + 0.0, + 0.10376164317131042, + 0.0909266471862793, + 0.1919306516647339, + 0.28928786516189575, + -0.4625590741634369, + -0.30475670099258423, + 0.0 + ], + [ + 1.9072847366333008, + 0.05373973771929741, + -1.1472969055175781, + -0.47107377648353577, + 0.3914180397987366, + -3.342223644256592, + -2.4980990886688232, + 1.2589508295059204, + 0.8455543518066406, + -0.7174267172813416, + -1.0694091320037842, + 0.33346641063690186, + -2.3123579025268555, + -1.1108808517456055, + -0.33852219581604004, + -0.7232105731964111, + 1.145590901374817, + -0.7107121348381042, + -0.5049349069595337, + -1.112295150756836, + -1.244770884513855 + ], + [ + 1.808228611946106, + -0.18174676597118378, + -1.2057973146438599, + -0.6115378737449646, + 0.11874309182167053, + -3.3499772548675537, + -2.3861043453216553, + 1.2540870904922485, + 1.1210845708847046, + -0.953934371471405, + -1.3015953302383423, + 0.6089286804199219, + -2.1655566692352295, + -0.8375923037528992, + -0.26793593168258667, + -0.998324990272522, + 1.2481704950332642, + -0.8320856690406799, + -0.376935750246048, + -1.0882012844085693, + -0.9857641458511353 + ], + [ + 2.291844606399536, + 0.05574151128530502, + -0.9760321974754333, + -0.5710926055908203, + 0.8478215336799622, + -3.22356915473938, + -2.591245651245117, + 1.2184754610061646, + 0.855300784111023, + -1.0267595052719116, + -1.3318730592727661, + 0.4197496771812439, + -2.0368778705596924, + -1.3685163259506226, + -0.582176685333252, + -0.8082455992698669, + 1.3435251712799072, + -0.44022125005722046, + -0.06775334477424622, + -0.7692400217056274, + -0.8863754272460938 + ] + ], + "predicted_scores": [ + 7.748844146728516, + 6.1985087394714355, + 6.143899440765381, + 6.135042667388916, + 5.7854323387146, + 5.741323471069336, + 5.381580829620361, + 5.288109302520752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "6802b2d1fb8a5afa54b7321e2bb938176de48d74f28e9b2a9c741c23547f0ef1", + "stored_base_utility": 0.07829316705465317, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.12791956961154938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12791956961154938 + }, + "base_progress": 0.12791956961154938, + "base_success": false, + "base_utility": 0.12791956961154938, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.13394887745380402, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13394887745380402 + }, + { + "contact_quality": null, + "progress": 0.06245779991149902, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06245779991149902 + }, + { + "contact_quality": null, + "progress": 0.06475379317998886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06475379317998886 + }, + { + "contact_quality": null, + "progress": 0.07846011966466904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07846011966466904 + }, + { + "contact_quality": null, + "progress": 0.06508652120828629, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06508652120828629 + }, + { + "contact_quality": null, + "progress": 0.051506221294403076, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.051506221294403076 + }, + { + "contact_quality": null, + "progress": 0.07821440696716309, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07821440696716309 + }, + { + "contact_quality": null, + "progress": 0.08463313430547714, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08463313430547714 + } + ], + "candidate_progress": [ + 0.13394887745380402, + 0.06245779991149902, + 0.06475379317998886, + 0.07846011966466904, + 0.06508652120828629, + 0.051506221294403076, + 0.07821440696716309, + 0.08463313430547714 + ], + "candidate_source_chart_ids": [ + "ms-1ba37e30b90a3b28e62f", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-728b865677134d3788e3", + "generated_tangents": [ + [ + -0.5147793292999268, + 0.01635870337486267, + 0.5936979055404663, + 0.22692140936851501, + 0.806601345539093, + -0.31535428762435913, + 0.9833135008811951, + 0.49836429953575134, + 0.06978798657655716, + -0.24665281176567078, + -0.4312334358692169, + 0.40476545691490173, + -0.7772910594940186, + -0.9794929027557373, + -1.0157660245895386, + 0.5901522040367126, + 0.9773255586624146, + -0.36914050579071045, + -0.2945755124092102, + -1.3986175060272217, + -0.3224315345287323 + ], + [ + -0.4392404556274414, + -0.28786003589630127, + 0.3327646851539612, + 0.6239948272705078, + 1.2620210647583008, + -0.15100327134132385, + -0.5116145014762878, + 0.5213421583175659, + -0.06871151924133301, + 0.1391371637582779, + -0.2599793076515198, + 0.5955996513366699, + -0.9769980907440186, + -0.04995851218700409, + -0.9729883074760437, + 0.6893180012702942, + 0.6582030057907104, + -0.4735340476036072, + -0.9789217114448547, + -0.4554268717765808, + -0.7393642663955688 + ], + [ + -0.346935510635376, + -0.2151426374912262, + 0.2607693076133728, + 0.4880162179470062, + 0.9172625541687012, + 0.4352869391441345, + -0.16314536333084106, + 0.7385636568069458, + 0.04108767956495285, + 0.35693731904029846, + -0.46944478154182434, + 0.1872413158416748, + -1.5915976762771606, + -0.5122110843658447, + -1.081807017326355, + 0.6284721493721008, + 0.7538945078849792, + -0.45732825994491577, + -1.1423842906951904, + -0.5460995435714722, + -0.994105339050293 + ], + [ + -0.47438687086105347, + -0.2941962480545044, + 0.08930006623268127, + 0.47087201476097107, + 1.0732719898223877, + 0.7297742366790771, + -0.37707096338272095, + 0.6198458671569824, + -0.01236293837428093, + 0.19176959991455078, + -0.3815520703792572, + 0.4481086730957031, + -1.3105427026748657, + -0.6763743758201599, + -0.928192138671875, + 0.6959629654884338, + 0.8909534811973572, + -0.2225792557001114, + -1.0983164310455322, + 0.0018378496170043945, + -0.6349530220031738 + ], + [ + -0.29603272676467896, + -0.21692737936973572, + 0.34010499715805054, + 0.5396671295166016, + 1.0621541738510132, + 0.47140786051750183, + -0.16609090566635132, + 0.6417175531387329, + 0.02341892570257187, + 0.2948392331600189, + -0.5026776790618896, + 0.1336493194103241, + -1.3191566467285156, + -0.436301052570343, + -1.0181485414505005, + 0.6648247838020325, + 0.8199318051338196, + -0.44266223907470703, + -1.0431838035583496, + -0.4796689748764038, + -0.9188610315322876 + ], + [ + 0.00443209707736969, + -0.2032320648431778, + 0.6682220697402954, + 0.32137826085090637, + 0.6583718061447144, + 0.18535318970680237, + 0.11783808469772339, + 0.5542389154434204, + -0.036951079964637756, + 0.23047593235969543, + -0.48136118054389954, + -0.26839587092399597, + -1.2748585939407349, + -0.48537975549697876, + -0.9735562205314636, + 0.598412275314331, + 1.0493358373641968, + -0.3935312032699585, + -1.1261112689971924, + -0.5450727939605713, + -0.6077508926391602 + ], + [ + -0.3254464864730835, + -0.2069281041622162, + 0.5057176351547241, + 0.6721740961074829, + 0.6974209547042847, + 0.6202588081359863, + 0.06076979637145996, + 0.4048321545124054, + 0.07321679592132568, + 0.29294872283935547, + -0.5859494209289551, + 0.32686787843704224, + -1.3482577800750732, + -0.1401742547750473, + -1.0666649341583252, + 0.7469267249107361, + 0.6608222723007202, + -0.6406729221343994, + -1.144120454788208, + -0.49726879596710205, + -1.376233458518982 + ], + [ + -0.1317814141511917, + -0.08274295926094055, + 0.14742302894592285, + 0.12061131000518799, + 0.7764797806739807, + 0.5823787450790405, + -0.5051177144050598, + 0.5014482736587524, + 0.04729006439447403, + 0.3133741617202759, + -0.712706446647644, + 0.03383323550224304, + -1.275682806968689, + -0.030220016837120056, + -0.9907340407371521, + 0.704752504825592, + 0.8170828223228455, + -0.7511892318725586, + -1.035818338394165, + -0.5890134572982788, + -1.2898623943328857 + ] + ], + "generated_utilities": [ + 0.13394887745380402, + 0.06245779991149902, + 0.06475379317998886, + 0.07846011966466904, + 0.06508652120828629, + 0.051506221294403076, + 0.07821440696716309, + 0.08463313430547714 + ], + "group_id": "ms-728b865677134d3788e3", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05257302522659302, + 0.05559448525309563, + 0.11835493892431259, + 0.04238405078649521, + 0.16897141933441162, + 0.02001136541366577, + 0.1609041690826416, + 0.14248299598693848, + 0.14091376960277557, + 0.17822234332561493, + 0.22658847272396088, + 0.1736571490764618, + 0.18659374117851257, + 0.12753915786743164, + 0.07164359092712402, + 0.0063618822023272514 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04065108299255371, + 0.05943702906370163, + 0.0039327144622802734, + 0.10760968923568726, + -0.3465921878814697, + -0.6843758821487427, + 0.0, + 0.3034271001815796, + -0.38217779994010925, + 0.019724853336811066, + -1.1125431060791016, + -0.016437113285064697, + 0.09826856851577759, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + 0.055904269218444824, + -0.2721228003501892, + 0.07830148935317993, + 0.0 + ], + [ + -0.9075424671173096, + 0.06243014335632324, + 1.0039327144622803, + -0.6845037937164307, + 0.050797268748283386, + -1.6843758821487427, + -1.3340954780578613, + 1.3034271001815796, + -0.29618290066719055, + -0.035879287868738174, + -2.1125431060791016, + 0.9835628867149353, + -0.9017314314842224, + -1.01132333278656, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233 + ], + [ + -1.117501139640808, + 0.23503510653972626, + 0.9669153094291687, + -0.8317459225654602, + 0.06331060081720352, + -1.5474717617034912, + -1.3840656280517578, + 1.240980863571167, + -0.053158730268478394, + 0.10333722829818726, + -2.2084054946899414, + 1.1802411079406738, + -0.8987206816673279, + -1.136216402053833, + -1.9180536270141602, + 1.4563629627227783, + 2.625094413757324, + -0.1296394169330597, + 0.3340006470680237, + -0.7212131023406982, + 0.9384214878082275 + ], + [ + -1.2640053033828735, + 0.2415706366300583, + 0.9424604177474976, + -0.8472410440444946, + 0.35185110569000244, + -1.8022713661193848, + -1.2391949892044067, + 0.9333417415618896, + 0.03309231996536255, + 0.22933605313301086, + -1.7521884441375732, + 1.0360857248306274, + -0.6365935802459717, + -0.7614871859550476, + -2.103987216949463, + 1.5381183624267578, + 2.11069917678833, + -0.5904175639152527, + -0.07874824106693268, + -0.3884509205818176, + 1.0602728128433228 + ], + [ + -0.7705078721046448, + 0.18350359797477722, + 0.579454779624939, + -0.5434049963951111, + 0.2274666726589203, + -1.7208446264266968, + -1.5971848964691162, + 1.6145182847976685, + -0.6915329694747925, + 0.20122811198234558, + -1.8584421873092651, + 0.9026989340782166, + -0.5741626024246216, + -0.5299967527389526, + -1.544856309890747, + 1.7236231565475464, + 2.1288580894470215, + -0.7592856287956238, + 0.3358893394470215, + -0.2929669916629791, + 1.67848801612854 + ], + [ + -0.3686429262161255, + 0.3043383061885834, + 0.7294905781745911, + -0.8736650347709656, + 0.3122289776802063, + -1.5051836967468262, + -1.8593363761901855, + 0.8389579653739929, + -0.8785254955291748, + -0.5520821213722229, + -1.8517990112304688, + 1.3417646884918213, + -0.32390326261520386, + -0.42145586013793945, + -1.6530441045761108, + 1.7202519178390503, + 2.180105209350586, + -0.4262692928314209, + -0.1161729246377945, + -0.08550512790679932, + 1.1130309104919434 + ], + [ + -0.9971213936805725, + -0.4085577130317688, + 0.9404422044754028, + -0.9977414011955261, + 0.4963066577911377, + -1.4445728063583374, + -1.8557989597320557, + 1.3813822269439697, + -0.9350934028625488, + 0.6622016429901123, + -2.0982701778411865, + 0.5712680220603943, + -1.3476169109344482, + -0.8056140542030334, + -2.2595674991607666, + 1.7601569890975952, + 2.5158329010009766, + -0.8607662916183472, + -0.010314732789993286, + -0.9802933931350708, + 0.587103545665741 + ], + [ + -0.538765549659729, + -0.2632336914539337, + 0.574018120765686, + -1.3315669298171997, + 0.23096206784248352, + -1.4029772281646729, + -1.2746275663375854, + 1.674896478652954, + -0.6276504993438721, + -0.06763898581266403, + -1.44939124584198, + 1.1256837844848633, + -0.6540690064430237, + -0.7850072979927063, + -2.472240686416626, + 1.201231837272644, + 2.056978702545166, + 0.4725829064846039, + 0.7320148944854736, + -0.47291314601898193, + 1.2244172096252441 + ], + [ + -0.5474913120269775, + 0.3537777066230774, + 0.1717698574066162, + -0.9464929103851318, + -0.24061335623264313, + -2.5145106315612793, + -1.9605224132537842, + 2.171875476837158, + 0.5462924242019653, + 0.2324821650981903, + -2.4616305828094482, + 1.3455290794372559, + -0.703314483165741, + -0.6289631128311157, + -1.848944902420044, + 1.755210518836975, + 2.649005174636841, + -0.931267499923706, + -0.3109248876571655, + 0.21283608675003052, + 0.6994417309761047 + ], + [ + -1.4273285865783691, + 0.48591291904449463, + 1.4724129438400269, + -1.1116547584533691, + -0.04224221408367157, + -1.1567587852478027, + -1.0686161518096924, + 1.7429871559143066, + -1.1814582347869873, + 0.07730802893638611, + -2.1004433631896973, + 0.45253968238830566, + -0.7293854355812073, + -0.4477439522743225, + -1.986203908920288, + 1.1158549785614014, + 2.0316240787506104, + 0.5763667821884155, + -0.48857587575912476, + -1.07709801197052, + 0.5611822605133057 + ] + ], + "predicted_scores": [ + 6.5777764320373535, + 5.295867443084717, + 5.214710712432861, + 5.058055400848389, + 4.8819661140441895, + 4.6562299728393555, + 4.570862293243408, + 4.170901298522949 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296", + "selected_index": 0, + "state_hash": "62e00728e448102460275bc2c35150dbf429ab0d5670a6706fa41256273bac3c", + "stored_base_utility": 0.05257302522659302, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20529556274414062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20529556274414062 + }, + "base_progress": 0.20529556274414062, + "base_success": false, + "base_utility": 0.20529556274414062, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10225218534469604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10225218534469604 + }, + { + "contact_quality": null, + "progress": 0.20188765227794647, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20188765227794647 + }, + { + "contact_quality": null, + "progress": 0.08769619464874268, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08769619464874268 + }, + { + "contact_quality": null, + "progress": 0.08251267671585083, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08251267671585083 + }, + { + "contact_quality": null, + "progress": 0.11142279952764511, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11142279952764511 + }, + { + "contact_quality": null, + "progress": 0.11381959915161133, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11381959915161133 + }, + { + "contact_quality": null, + "progress": 0.10222844034433365, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10222844034433365 + }, + { + "contact_quality": null, + "progress": 0.06529740244150162, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06529740244150162 + } + ], + "candidate_progress": [ + 0.10225218534469604, + 0.20188765227794647, + 0.08769619464874268, + 0.08251267671585083, + 0.11142279952764511, + 0.11381959915161133, + 0.10222844034433365, + 0.06529740244150162 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-6ddecc7f074b9e267409", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "generated_tangents": [ + [ + 0.6351838707923889, + 0.0554143562912941, + -0.6360185146331787, + 0.26119446754455566, + 0.40959879755973816, + -3.0019710063934326, + -1.2204558849334717, + -0.9697473049163818, + -0.07487937062978745, + 1.4689207077026367, + 0.8908620476722717, + 0.1178353801369667, + 1.4230598211288452, + 0.7022956013679504, + 0.10874947905540466, + 0.0357222706079483, + -0.4160614609718323, + 0.6562888622283936, + 0.3235812187194824, + -0.5519699454307556, + -0.43149104714393616 + ], + [ + 0.7582695484161377, + 0.02489960938692093, + -0.4969500005245209, + -0.16363608837127686, + 0.28399404883384705, + -3.015408754348755, + -2.1042070388793945, + -1.028049349784851, + 0.1981009840965271, + 0.9594929814338684, + 0.9217855334281921, + 0.3362846374511719, + 1.3983911275863647, + 1.0891989469528198, + 0.1956273913383484, + -0.2859035134315491, + -0.5257833003997803, + 0.8497936725616455, + -0.03918731212615967, + -0.18944242596626282, + 0.14280645549297333 + ], + [ + 0.8326467871665955, + 0.0583222359418869, + -0.6408113241195679, + 0.2002996802330017, + -0.02010539174079895, + -2.884190082550049, + -1.7671189308166504, + -1.2717655897140503, + -0.002875702455639839, + 1.2529200315475464, + 0.8150051236152649, + 0.26369452476501465, + 1.3973784446716309, + 0.9947073459625244, + 0.09533832967281342, + -0.07589471340179443, + -0.36194247007369995, + 0.7034773826599121, + 0.39536020159721375, + -0.37164270877838135, + -0.13375824689865112 + ], + [ + 0.8077759146690369, + 0.042051851749420166, + -0.0677703320980072, + 0.09784567356109619, + 0.2418215423822403, + -3.357654094696045, + -1.4908311367034912, + -1.162305235862732, + 0.06381805986166, + 1.28940749168396, + 1.0562357902526855, + 0.1521969735622406, + 1.536420226097107, + 0.3797754645347595, + -0.06844878196716309, + -0.1321399062871933, + -0.40969645977020264, + 0.5953527688980103, + 0.522000789642334, + -0.021678835153579712, + -0.348192423582077 + ], + [ + 0.7868461608886719, + -0.029668498784303665, + -0.7807492017745972, + -0.25344526767730713, + -0.43694227933883667, + -3.8253989219665527, + -2.1909995079040527, + -0.9644989967346191, + 0.08103717863559723, + 1.0811434984207153, + 1.4036166667938232, + 0.7007911205291748, + 1.2752180099487305, + 0.7197125554084778, + -0.0727132111787796, + -0.04798858240246773, + -0.02189616858959198, + 0.4078947603702545, + 0.31187865138053894, + 0.25585711002349854, + -0.39843496680259705 + ], + [ + 0.6747683882713318, + 0.009402386844158173, + -0.4406537711620331, + 0.04984545707702637, + 0.15408065915107727, + -3.1775104999542236, + -1.4603171348571777, + -1.0818332433700562, + 0.08093071728944778, + 1.2560621500015259, + 0.8789746165275574, + 0.21762767434120178, + 1.328750491142273, + 1.098083734512329, + 0.11343250423669815, + -0.1459447741508484, + -0.3442935347557068, + 0.622115433216095, + 0.3984382152557373, + -0.4171231985092163, + -0.29798778891563416 + ], + [ + 0.5690445303916931, + 0.00414787232875824, + -0.5417822599411011, + -0.05685913562774658, + 0.09590134024620056, + -3.223590135574341, + -1.3231456279754639, + -1.07421875, + 0.07517006993293762, + 1.3157005310058594, + 0.9222729206085205, + 0.3335229158401489, + 1.3532094955444336, + 1.1141421794891357, + 0.05757361277937889, + -0.15622928738594055, + -0.37984025478363037, + 0.6170745491981506, + 0.4994942247867584, + -0.30145910382270813, + -0.11340078711509705 + ], + [ + 0.5306358933448792, + 0.07138641923666, + -0.3610648810863495, + 0.08101078867912292, + -0.20862016081809998, + -3.3606131076812744, + -1.4476242065429688, + -1.1502933502197266, + -0.018826277926564217, + 1.3719052076339722, + 1.0567415952682495, + 0.14625899493694305, + 1.2521876096725464, + 1.2068270444869995, + -0.07887710630893707, + -0.20102551579475403, + -0.17374534904956818, + 0.6486114263534546, + 0.6353914737701416, + -0.6090413331985474, + -0.6406717300415039 + ] + ], + "generated_utilities": [ + 0.10225218534469604, + 0.20188765227794647, + 0.08769619464874268, + 0.08251267671585083, + 0.11142279952764511, + 0.11381959915161133, + 0.10222844034433365, + 0.06529740244150162 + ], + "group_id": "ms-7ea98b8f4d2220347ee0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18600693345069885, + 0.07475636899471283, + 0.27452313899993896, + 0.20920448005199432, + 0.18411576747894287, + 0.27989473938941956, + 0.19416475296020508, + 0.10948207229375839, + 0.123555026948452, + 0.1468026340007782, + 0.12693151831626892, + 0.11466284841299057, + 0.07585643231868744, + 0.12908481061458588, + 0.22501108050346375, + 0.06431269645690918 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.06974434852600098, + -0.02351950854063034, + 0.04567508399486542, + -0.07042121887207031, + -0.038941264152526855, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.08617860078811646, + 0.9112892150878906, + 0.7374391555786133, + 0.06539928913116455, + 0.03787863254547119, + 0.0, + -0.021528124809265137, + -0.04566385596990585, + -0.03331831097602844, + 1.6466763019561768, + 0.04444807767868042, + -0.27494049072265625, + 0.0 + ], + [ + 1.1160619258880615, + -0.10018755495548248, + 0.19327251613140106, + -1.3990073204040527, + 0.672177255153656, + -3.303851842880249, + -2.832740545272827, + -1.902563214302063, + 0.39503830671310425, + 1.542332410812378, + 1.6850805282592773, + 0.6842766404151917, + 1.1400834321975708, + 1.670598030090332, + 0.631821870803833, + -0.26601675152778625, + -0.2725638151168823, + 3.003178119659424, + 0.23009130358695984, + -1.0388810634613037, + -0.19119825959205627 + ], + [ + 1.031497836112976, + -0.2943698763847351, + -0.11846622079610825, + -1.3361037969589233, + 0.6225074529647827, + -3.545761823654175, + -2.5078811645507812, + -1.4024183750152588, + 0.6736462116241455, + 2.093475580215454, + 1.9168713092803955, + 0.6377919316291809, + 0.5105260014533997, + 1.8172619342803955, + 1.1300766468048096, + 0.1616329848766327, + 0.03408849239349365, + 2.2435190677642822, + 0.735376238822937, + -1.5760226249694824, + -0.7514586448669434 + ], + [ + 1.7542359828948975, + -0.7351009249687195, + -0.854544997215271, + -1.223644495010376, + 0.07327532768249512, + -3.150663137435913, + -2.5486767292022705, + -2.110039472579956, + 1.0206904411315918, + 2.1492340564727783, + 2.187993288040161, + 1.2547765970230103, + 0.8607165217399597, + 1.1150517463684082, + 0.9828771948814392, + 0.5030426383018494, + -0.4691219627857208, + 2.2485060691833496, + -0.14985057711601257, + -1.9480454921722412, + -0.05157876014709473 + ], + [ + 0.6040669679641724, + 0.5665496587753296, + 0.24913550913333893, + -1.6600215435028076, + -0.10813313722610474, + -3.8031911849975586, + -3.226392984390259, + -1.5098689794540405, + 1.0684192180633545, + 2.1557064056396484, + 1.545849084854126, + 1.011389970779419, + 1.113816499710083, + 0.9937302470207214, + 1.372757911682129, + -0.4055754244327545, + -0.23403912782669067, + 3.0650274753570557, + 0.1330145299434662, + -0.7907523512840271, + -0.6428619623184204 + ], + [ + 0.30788952112197876, + 0.6807127594947815, + -0.035938769578933716, + -0.8788743615150452, + -0.30191969871520996, + -4.111589431762695, + -2.432647705078125, + -1.7479791641235352, + -0.321926474571228, + 1.9823087453842163, + 1.6617395877838135, + 1.3319180011749268, + 1.5937504768371582, + 2.0502309799194336, + 0.63725745677948, + 0.7232187390327454, + -0.23885302245616913, + 1.8472312688827515, + 0.3300834596157074, + -0.43588966131210327, + -1.2869892120361328 + ], + [ + 0.3636443018913269, + -0.4284004867076874, + 0.2675324082374573, + -0.4861376881599426, + 0.2239738255739212, + -2.813422918319702, + -2.4891929626464844, + -1.165071725845337, + 0.6442201733589172, + 1.1830427646636963, + 1.177597999572754, + 0.33577069640159607, + 0.6362329721450806, + 1.5409071445465088, + 0.29718804359436035, + -0.5603961944580078, + -0.8089720010757446, + 2.6271419525146484, + -0.5456377267837524, + -0.2824941873550415, + -0.7960987091064453 + ], + [ + 1.1001086235046387, + -0.9762114882469177, + -0.00277864933013916, + -1.704659104347229, + 1.2373628616333008, + -3.385840654373169, + -1.6910454034805298, + -1.470743179321289, + 0.9354362487792969, + 1.9378679990768433, + 1.515415906906128, + -0.3002503514289856, + 0.21796274185180664, + 1.542279839515686, + 1.7353785037994385, + 0.23501873016357422, + -0.16601301729679108, + 2.1989498138427734, + 0.8671687841415405, + -1.6252647638320923, + -1.2141560316085815 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06974434852600098, + -0.15391214191913605, + -0.22159501910209656, + -0.07042121887207031, + -0.09706336259841919, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.1400253176689148, + 0.9112892150878906, + 0.7374391555786133, + 0.13415807485580444, + -0.10778343677520752, + 0.0, + -0.08470147848129272, + 0.1231045201420784, + 0.08352774381637573, + 1.6466763019561768, + -0.005736887454986572, + -0.27494049072265625, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.125826835632324, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.152101516723633, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.8942345976829529 + ] + ], + "predicted_scores": [ + 6.957677364349365, + 6.9082770347595215, + 6.669839859008789, + 6.668172359466553, + 6.608238697052002, + 6.531609058380127, + 5.921152114868164, + 5.370590686798096 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "a2b9f3b3ea6458c6734081cea861cf54026796a0a594bf9be3657d28f12e9b3c", + "stored_base_utility": 0.18600693345069885, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18213583528995514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18213583528995514 + }, + "base_progress": 0.18213583528995514, + "base_success": false, + "base_utility": 0.18213583528995514, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.004793623927980661, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004793623927980661 + }, + { + "contact_quality": null, + "progress": 0.0028257768135517836, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0028257768135517836 + }, + { + "contact_quality": null, + "progress": 0.0050675272941589355, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0050675272941589355 + }, + { + "contact_quality": null, + "progress": 0.0031685233116149902, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0031685233116149902 + }, + { + "contact_quality": null, + "progress": 0.0052528977394104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0052528977394104 + }, + { + "contact_quality": null, + "progress": 0.0036421220283955336, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0036421220283955336 + }, + { + "contact_quality": null, + "progress": 0.0021342237014323473, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0021342237014323473 + }, + { + "contact_quality": null, + "progress": 0.0036080479621887207, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0036080479621887207 + } + ], + "candidate_progress": [ + 0.004793623927980661, + 0.0028257768135517836, + 0.0050675272941589355, + 0.0031685233116149902, + 0.0052528977394104, + 0.0036421220283955336, + 0.0021342237014323473, + 0.0036080479621887207 + ], + "candidate_source_chart_ids": [ + "ms-8e12f7ad446482627dd3", + "ms-02311c7783bc67e188b4", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7f57045c114ddad153b5", + "generated_tangents": [ + [ + 0.19242781400680542, + -0.23486019670963287, + 1.0551730394363403, + -0.21431955695152283, + 1.8361363410949707, + 1.890921711921692, + 1.3092901706695557, + 0.3047295808792114, + -0.029983485117554665, + 0.7014550566673279, + 0.7062481045722961, + 1.2387549877166748, + 0.944677472114563, + 2.0297346115112305, + -0.12552250921726227, + -0.09496954083442688, + 1.6345525979995728, + -0.637722373008728, + 1.8834655284881592, + 1.8923728466033936, + 1.549077033996582 + ], + [ + 0.16655893623828888, + -0.2928306460380554, + 1.2990028858184814, + -0.5302565097808838, + 1.5151221752166748, + 1.952709674835205, + 1.515507698059082, + 0.39710211753845215, + -0.132785364985466, + 0.9167823195457458, + 0.584041953086853, + 1.1585752964019775, + 1.3343281745910645, + 1.6751090288162231, + -0.021682340651750565, + -0.006324931979179382, + 1.7948086261749268, + -0.5942758321762085, + 1.473301887512207, + 1.2825021743774414, + 1.409175992012024 + ], + [ + 0.1552114635705948, + -0.22567999362945557, + 1.1855015754699707, + -0.5588181614875793, + 1.2776870727539062, + 2.3382818698883057, + 1.4986867904663086, + 0.4236941337585449, + -0.12306109070777893, + 0.7310000061988831, + 0.6485676765441895, + 1.2998638153076172, + 1.7941218614578247, + 1.2643907070159912, + 0.12577852606773376, + 0.020068857818841934, + 1.6570929288864136, + -0.4710620641708374, + 1.4371609687805176, + 1.2294025421142578, + 1.42363703250885 + ], + [ + 0.2973508834838867, + -0.2790142893791199, + 1.1926442384719849, + -0.5703457593917847, + 1.4720335006713867, + 1.7663577795028687, + 1.6142139434814453, + 0.4225847125053406, + -0.10414119064807892, + 0.7925950288772583, + 0.8482772707939148, + 1.0342340469360352, + 1.5343438386917114, + 1.5455867052078247, + -0.214212566614151, + -0.04831160604953766, + 1.9689501523971558, + -0.5318179130554199, + 1.4848265647888184, + 1.660297155380249, + 1.5958086252212524 + ], + [ + 0.21230991184711456, + -0.18754854798316956, + 1.5562338829040527, + -0.1213027760386467, + 1.1427690982818604, + 1.6968116760253906, + 0.987946093082428, + 0.5736876726150513, + -0.13050565123558044, + 0.7572095990180969, + 0.550136387348175, + 0.46257203817367554, + 1.6667393445968628, + 1.0320311784744263, + -0.0623105950653553, + 0.08181780576705933, + 1.3406203985214233, + -0.35682857036590576, + 1.4835853576660156, + 1.5075814723968506, + 1.0305763483047485 + ], + [ + 0.22347886860370636, + -0.2839810252189636, + 1.3782404661178589, + -0.5439313054084778, + 1.3498327732086182, + 2.2578561305999756, + 1.5557303428649902, + 0.5230444669723511, + -0.17672592401504517, + 0.7562742233276367, + 0.5604004859924316, + 1.095818042755127, + 1.5878697633743286, + 1.8663661479949951, + -0.08795695006847382, + 0.01589171588420868, + 1.7301238775253296, + -0.5995655655860901, + 1.4616014957427979, + 1.4391319751739502, + 1.274280309677124 + ], + [ + 0.04314441233873367, + -0.31074202060699463, + 1.23866605758667, + -0.3395167291164398, + 1.4657118320465088, + 1.9613734483718872, + 1.8112022876739502, + 0.4000732898712158, + -0.178915336728096, + 0.9016785025596619, + 0.8109214901924133, + 0.8285111784934998, + 1.2619868516921997, + 1.8626943826675415, + -0.071990966796875, + -0.10383464395999908, + 2.030254364013672, + -0.37101006507873535, + 1.51361083984375, + 1.507206678390503, + 1.6656185388565063 + ], + [ + 0.15120479464530945, + -0.2534239888191223, + 1.0571345090866089, + -0.6449457406997681, + 1.4762277603149414, + 2.024937152862549, + 1.728184461593628, + 0.5318474173545837, + -0.19190508127212524, + 0.733979344367981, + 0.8016237020492554, + 0.957295298576355, + 1.5044066905975342, + 1.654699683189392, + -0.20007109642028809, + -0.04618840292096138, + 1.9570878744125366, + -0.4365650415420532, + 1.5185191631317139, + 1.5623745918273926, + 1.5185866355895996 + ] + ], + "generated_utilities": [ + 0.004793623927980661, + 0.0028257768135517836, + 0.0050675272941589355, + 0.0031685233116149902, + 0.0052528977394104, + 0.0036421220283955336, + 0.0021342237014323473, + 0.0036080479621887207 + ], + "group_id": "ms-7f57045c114ddad153b5", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.11277659982442856, + 0.1479199379682541, + 0.07541074603796005, + 0.042439721524715424, + 0.15517297387123108, + 0.03584214299917221, + 0.1660110056400299, + 0.12549421191215515, + 0.227622389793396, + 0.09924932569265366, + 0.18198737502098083, + 0.11130931228399277, + 0.10854844748973846, + 0.01617596670985222, + 0.08437560498714447, + 0.04182587191462517 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.013166621327400208, + -0.12399804592132568, + 0.48011910915374756, + 0.4925293028354645, + 0.6256281137466431, + 2.213102102279663, + 0.0, + -0.4044480323791504, + 0.1722363382577896, + 0.3389366865158081, + -0.08761294186115265, + 0.21791136264801025, + 0.9396321773529053, + 0.0, + 0.06717276573181152, + -0.5456919074058533, + 0.722014307975769, + -0.051606684923172, + 0.7784175276756287, + 0.5338386297225952, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.9143741130828857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.455221176147461, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.263336420059204 + ], + [ + -0.47406601905822754, + -0.37954479455947876, + 1.123671293258667, + -0.8903511762619019, + 1.5515481233596802, + 3.7488584518432617, + 0.9699987173080444, + -0.23581115901470184, + -0.2149372398853302, + 0.8883978128433228, + 0.5803861021995544, + 1.7047908306121826, + 2.149672746658325, + 2.2741973400115967, + 0.2513730227947235, + 0.6982267498970032, + 2.1354174613952637, + -1.151021122932434, + 1.9550246000289917, + 0.5981679558753967, + 1.338004469871521 + ], + [ + -0.1836935132741928, + -0.42300617694854736, + 1.7664438486099243, + -0.5576877593994141, + 1.0626983642578125, + 2.5276265144348145, + 2.3329849243164062, + -0.4834023118019104, + -0.25722599029541016, + 0.5209662914276123, + 0.24240870773792267, + 0.8533134460449219, + 1.5531291961669922, + 2.304234504699707, + 0.7609332799911499, + 1.0344312191009521, + 2.270921230316162, + -0.926997184753418, + 0.9014018177986145, + 2.259265422821045, + 2.172292470932007 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3910996615886688, + -0.9467402100563049, + 1.4744372367858887, + -0.2748112082481384, + 1.753147840499878, + 3.3321292400360107, + 1.7170560359954834, + 0.09920699894428253, + -0.11250317096710205, + 1.5213767290115356, + 0.0891318991780281, + 1.0193004608154297, + 1.7396382093429565, + 2.202378988265991, + 0.1176290512084961, + 0.5433868765830994, + 1.6113250255584717, + -0.2701912522315979, + 1.7789868116378784, + 1.246854305267334, + 1.6763029098510742 + ], + [ + 0.3669140636920929, + -0.8853410482406616, + 1.1054010391235352, + -0.008406088687479496, + 1.683157205581665, + 2.8090970516204834, + 2.1524839401245117, + -0.1140800267457962, + -0.16068093478679657, + 1.3338439464569092, + -0.34659242630004883, + 1.4494925737380981, + 2.0485284328460693, + 2.244007110595703, + 0.4862309694290161, + 0.6211890578269958, + 1.5708913803100586, + -0.22552455961704254, + 1.7964838743209839, + 1.386763095855713, + 1.3543281555175781 + ], + [ + 0.7718826532363892, + -0.5710152983665466, + 2.151604175567627, + -0.5648711323738098, + 2.1273765563964844, + 3.657623291015625, + 1.9845168590545654, + -0.1599704772233963, + -0.6097310781478882, + 1.135812520980835, + -0.09587362408638, + 1.555975079536438, + 2.3309824466705322, + 1.9832960367202759, + 0.46482378244400024, + 0.6467846035957336, + 1.7182108163833618, + -0.0750286877155304, + 1.9485927820205688, + 0.959426760673523, + 0.9474185109138489 + ] + ], + "predicted_scores": [ + 6.556987762451172, + 6.543363094329834, + 6.536247253417969, + 6.463170051574707, + 6.38189697265625, + 6.139730453491211, + 6.116417407989502, + 5.926776885986328 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524", + "selected_index": 0, + "state_hash": "4885f9acf2601b3d2c201bc8e00ef617845ad2a6d08f96d585249345f457be89", + "stored_base_utility": 0.11277659982442856, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0056487321853637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056487321853637695 + }, + "base_progress": 0.0056487321853637695, + "base_success": false, + "base_utility": 0.0056487321853637695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.011145631782710552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011145631782710552 + }, + { + "contact_quality": null, + "progress": 0.005635479930788279, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005635479930788279 + }, + { + "contact_quality": null, + "progress": 0.0070413947105407715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0070413947105407715 + }, + { + "contact_quality": null, + "progress": 0.007375796791166067, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007375796791166067 + }, + { + "contact_quality": null, + "progress": 0.014856874942779541, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014856874942779541 + }, + { + "contact_quality": null, + "progress": 0.01037945318967104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01037945318967104 + }, + { + "contact_quality": null, + "progress": 0.025841474533081055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025841474533081055 + }, + { + "contact_quality": null, + "progress": 0.012394865043461323, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012394865043461323 + } + ], + "candidate_progress": [ + 0.011145631782710552, + 0.005635479930788279, + 0.0070413947105407715, + 0.007375796791166067, + 0.014856874942779541, + 0.01037945318967104, + 0.025841474533081055, + 0.012394865043461323 + ], + "candidate_source_chart_ids": [ + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "generated_tangents": [ + [ + -0.2132212072610855, + -0.35612571239471436, + 0.3206631541252136, + -0.6110020875930786, + 0.6720107793807983, + 1.8810362815856934, + 3.4679126739501953, + 0.42357224225997925, + -0.43441298604011536, + 0.38837340474128723, + -0.8442613482475281, + 1.170778512954712, + 1.1464968919754028, + 1.0374091863632202, + 0.8390651345252991, + -0.85950767993927, + -1.3317396640777588, + 0.4298861622810364, + 0.6741939783096313, + -1.2672675848007202, + -1.9145469665527344 + ], + [ + -0.15743465721607208, + -0.3636484146118164, + 0.842019259929657, + -0.7178770303726196, + 0.8974410891532898, + 2.18573260307312, + 2.904831886291504, + 0.42274922132492065, + -0.4765997529029846, + -0.02085917443037033, + -1.1092324256896973, + 1.50281822681427, + 1.4443620443344116, + 1.0851465463638306, + 1.1450011730194092, + -0.9535958170890808, + -1.3039427995681763, + 0.2522483170032501, + 0.7284320592880249, + -1.1985808610916138, + -1.8808517456054688 + ], + [ + -0.16789843142032623, + -0.38044852018356323, + 0.34564560651779175, + -0.7870309352874756, + 1.1070396900177002, + 1.9073704481124878, + 3.3052501678466797, + 0.5817533731460571, + -0.5266355872154236, + 0.40520206093788147, + -0.8947142362594604, + 1.0438597202301025, + 1.6923879384994507, + 0.6507884860038757, + 1.108149766921997, + -0.8551848530769348, + -1.1152418851852417, + 0.6647673845291138, + 0.48249322175979614, + -1.077775478363037, + -2.070512056350708 + ], + [ + -0.26104307174682617, + -0.2625615894794464, + 0.2304205298423767, + -0.793540358543396, + 1.4105303287506104, + 1.854358434677124, + 3.293538808822632, + 0.5218722820281982, + -0.5116921067237854, + 0.3137792646884918, + -0.7852323651313782, + 1.2465404272079468, + 1.62022864818573, + 0.8447396159172058, + 1.2068796157836914, + -0.9205024838447571, + -1.1353341341018677, + 0.5821287035942078, + 0.2939908802509308, + -0.8216809630393982, + -2.2633392810821533 + ], + [ + -0.6575313806533813, + -0.23661266267299652, + 0.48808079957962036, + -0.8921552896499634, + 1.520573377609253, + 1.6683564186096191, + 2.1068243980407715, + 0.3933829069137573, + -0.33618858456611633, + 0.18094909191131592, + -1.3109817504882812, + 0.3120250701904297, + 1.5246118307113647, + 1.3523991107940674, + 0.9605004787445068, + -0.4774652421474457, + -1.7408987283706665, + 0.3444976806640625, + 0.009613960981369019, + -1.4281773567199707, + -1.9245011806488037 + ], + [ + -0.598523736000061, + -0.30479729175567627, + 0.5191957354545593, + -0.8284745216369629, + 1.6283166408538818, + 1.6675724983215332, + 2.249469518661499, + 0.5003149509429932, + -0.3607867360115051, + 0.1619831770658493, + -1.466862678527832, + 0.3162725269794464, + 1.5596030950546265, + 1.3193833827972412, + 1.085457682609558, + -0.3821507692337036, + -1.578502893447876, + 0.38166701793670654, + 0.4683684706687927, + -1.3051135540008545, + -1.7824459075927734 + ], + [ + -0.7587069869041443, + -0.46214473247528076, + 0.35767853260040283, + -0.9201654195785522, + 1.1327975988388062, + 1.5855541229248047, + 2.024463176727295, + 0.25509265065193176, + -0.22467383742332458, + 0.2761390209197998, + -1.2066025733947754, + 0.3324081301689148, + 1.442461609840393, + 1.7368476390838623, + 0.9266427159309387, + -0.4226803183555603, + -1.7133150100708008, + 0.2403825968503952, + 0.1079477071762085, + -1.9669630527496338, + -1.821255087852478 + ], + [ + -0.3811972737312317, + -0.5896664261817932, + 0.9327340126037598, + -1.2426563501358032, + 1.617842197418213, + 1.8171278238296509, + 2.357351064682007, + 0.24668310582637787, + -0.2678268849849701, + 0.17105263471603394, + -1.330452799797058, + 0.31128764152526855, + 0.7893147468566895, + 1.9798027276992798, + 0.8406444787979126, + -0.400086909532547, + -1.5437424182891846, + 0.1732153743505478, + 0.13039374351501465, + -1.1935398578643799, + -2.0147836208343506 + ] + ], + "generated_utilities": [ + 0.011145631782710552, + 0.005635479930788279, + 0.0070413947105407715, + 0.007375796791166067, + 0.014856874942779541, + 0.01037945318967104, + 0.025841474533081055, + 0.012394865043461323 + ], + "group_id": "ms-81f4e8858b2ba90aea2e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.01361372135579586, + 0.007307033054530621, + 0.021326780319213867, + 0.04093001410365105, + 0.15059307217597961, + 0.013598322868347168, + 0.09240063279867172, + 0.1631626933813095, + 0.05027441307902336, + 0.05712141841650009, + 0.1538398563861847, + 0.06023097038269043, + 0.020811915397644043, + 0.019836824387311935, + 0.1318017691373825, + 0.051341354846954346 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06500570476055145, + 0.046472612768411636, + 1.2439939975738525, + -1.446747064590454, + 1.7077040672302246, + 2.291205883026123, + 2.1016674041748047, + -0.6354803442955017, + -0.4465000331401825, + 0.9666666984558105, + -1.1514010429382324, + 0.45639365911483765, + 1.613925814628601, + 1.0568119287490845, + 1.3792463541030884, + -0.7852597832679749, + -1.576931118965149, + 1.1258286237716675, + 0.9614027738571167, + -0.9620904922485352, + -1.7528356313705444 + ], + [ + -0.23400194942951202, + -0.21453839540481567, + 1.458961844444275, + -1.701543927192688, + 1.4158955812454224, + 2.2648162841796875, + 2.2270467281341553, + -0.5239131450653076, + -0.44378146529197693, + 1.0904332399368286, + -1.223331093788147, + 0.45784294605255127, + 1.6057316064834595, + 0.9251953363418579, + 1.308751106262207, + -0.6351475715637207, + -1.5950077772140503, + 1.0428481101989746, + 0.6781296730041504, + -0.9007717370986938, + -1.880659818649292 + ], + [ + 0.2324782907962799, + 0.05129373073577881, + 1.1239540576934814, + -1.1853233575820923, + 1.3290393352508545, + 1.933064341545105, + 2.192976951599121, + -0.23718085885047913, + -0.279358446598053, + 1.3028839826583862, + -1.4676742553710938, + 0.16475489735603333, + 1.67984938621521, + 0.6600161790847778, + 1.5654886960983276, + -1.1545368432998657, + -1.5593464374542236, + 1.1976646184921265, + 0.5715436935424805, + -0.9047394394874573, + -1.9614591598510742 + ], + [ + 0.7452290058135986, + 0.614987313747406, + 0.7383754849433899, + -1.5856380462646484, + 1.264894723892212, + 1.7337512969970703, + 2.502504825592041, + 0.040616750717163086, + -1.1382783651351929, + 0.7095704674720764, + -1.014717698097229, + 0.5126742720603943, + 1.3401073217391968, + 1.4275505542755127, + 1.4739655256271362, + -0.7308655381202698, + -0.9416123032569885, + 0.661049485206604, + 0.9453439712524414, + -1.5968079566955566, + -1.715583324432373 + ], + [ + 0.15219944715499878, + 0.5165310502052307, + 1.0654370784759521, + -0.5756344199180603, + 1.2834129333496094, + 1.6370232105255127, + 2.414426326751709, + -1.1256691217422485, + -1.1171015501022339, + 0.5337101817131042, + -0.9850769639015198, + -0.35428309440612793, + 2.4350547790527344, + 1.8563504219055176, + 2.1929399967193604, + -0.1821107268333435, + -1.4600040912628174, + 1.5430631637573242, + 1.4937412738800049, + -0.8152608275413513, + -2.020042896270752 + ] + ], + "predicted_scores": [ + 13.856819152832031, + 12.800796508789062, + 12.187582015991211, + 12.16637897491455, + 11.499547004699707, + 11.33340072631836, + 10.854905128479004, + 10.459314346313477 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "562e8bc3804d1d310e0eaf612dbc110b8efc94514458dacf551ef33f4f0b8ee7", + "stored_base_utility": 0.01361372135579586, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.00698717450723052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00698717450723052 + }, + "base_progress": 0.00698717450723052, + "base_success": false, + "base_utility": 0.00698717450723052, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06672348827123642, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06672348827123642 + }, + { + "contact_quality": null, + "progress": 0.029613912105560303, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029613912105560303 + }, + { + "contact_quality": null, + "progress": 0.14832614362239838, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14832614362239838 + }, + { + "contact_quality": null, + "progress": 0.1094479188323021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1094479188323021 + }, + { + "contact_quality": null, + "progress": 0.10840656608343124, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10840656608343124 + }, + { + "contact_quality": null, + "progress": 0.09923364967107773, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09923364967107773 + }, + { + "contact_quality": null, + "progress": 0.055296678096055984, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.055296678096055984 + }, + { + "contact_quality": null, + "progress": 0.09003522247076035, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09003522247076035 + } + ], + "candidate_progress": [ + 0.06672348827123642, + 0.029613912105560303, + 0.14832614362239838, + 0.1094479188323021, + 0.10840656608343124, + 0.09923364967107773, + 0.055296678096055984, + 0.09003522247076035 + ], + "candidate_source_chart_ids": [ + "ms-45fbd749d5ed3fe37c13", + "ms-48178f52a96de3d1ead0", + "ms-25ab20bb3da8ba34be95", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-9229ce141dd41a11fac1", + "ms-9229ce141dd41a11fac1" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8a72b73ac452bc190aeb", + "generated_tangents": [ + [ + 0.5802809000015259, + -0.17232182621955872, + -0.13074412941932678, + -0.9402573108673096, + 0.24096477031707764, + -2.593315362930298, + -1.6888880729675293, + 0.38430607318878174, + -0.14453405141830444, + -0.09521184116601944, + 0.7020482420921326, + 0.7879524230957031, + 0.5401550531387329, + -0.31366825103759766, + 0.40505534410476685, + -0.15511159598827362, + -0.3023306727409363, + 0.30714017152786255, + 0.24821415543556213, + -0.5041369199752808, + -0.7858303785324097 + ], + [ + 0.585083544254303, + 0.010379355400800705, + -0.24248263239860535, + -0.8754599094390869, + 0.4871060848236084, + -3.1324737071990967, + -0.9886792302131653, + 0.08241399377584457, + -0.1512652486562729, + -0.04688064008951187, + 0.7735022306442261, + 0.4781321883201599, + 0.3961280286312103, + -0.13697953522205353, + 0.14559334516525269, + -0.17399317026138306, + -0.23298145830631256, + 0.3033895492553711, + 0.06670184433460236, + -0.3057488799095154, + -0.6404585838317871 + ], + [ + 0.41040998697280884, + -0.16142237186431885, + -0.21280953288078308, + -0.6524162292480469, + 0.4905272424221039, + -2.5773022174835205, + -1.0271387100219727, + 0.2494909167289734, + -0.1529950648546219, + -0.03776315599679947, + 0.550376296043396, + 0.3485774099826813, + 0.14609310030937195, + -0.1448907107114792, + -0.09965980052947998, + -0.11929327249526978, + 0.058319732546806335, + 0.5182379484176636, + 0.07967442274093628, + -0.2018723487854004, + -0.012257322669029236 + ], + [ + 0.544282078742981, + -0.1067652553319931, + -0.37422457337379456, + -0.8048790693283081, + 0.10579922795295715, + -2.541926622390747, + -1.207503080368042, + 0.35123443603515625, + -0.1142362654209137, + -0.0673704519867897, + 0.7051560282707214, + 0.46379974484443665, + -0.07719600200653076, + 0.3612074851989746, + -4.211440682411194e-05, + -0.1668364554643631, + 0.2595708966255188, + 0.6407525539398193, + 0.3123566806316376, + -0.25552186369895935, + -0.0617387592792511 + ], + [ + 0.4974521994590759, + -0.12853306531906128, + -0.4152775704860687, + -0.7610821723937988, + 0.13901180028915405, + -2.373908519744873, + -1.3535418510437012, + 0.3698514997959137, + -0.14082284271717072, + -0.04558420926332474, + 0.7580047845840454, + 0.5020407438278198, + -0.051934778690338135, + 0.4010813236236572, + 0.002136319875717163, + -0.17584328353405, + 0.19179822504520416, + 0.7005574107170105, + 0.3955182433128357, + -0.19895638525485992, + -0.027681156992912292 + ], + [ + 0.532270073890686, + -0.057570524513721466, + -0.3229869306087494, + -0.8626034259796143, + 0.03135751187801361, + -2.48866605758667, + -1.1516649723052979, + 0.3311689496040344, + -0.10964304953813553, + 0.00415164977312088, + 0.6936989426612854, + 0.5873826146125793, + -0.2168811857700348, + 0.25383713841438293, + -0.00653226301074028, + -0.16879411041736603, + 0.17690184712409973, + 0.6384727954864502, + 0.4651601016521454, + -0.18900656700134277, + -0.113162100315094 + ], + [ + 0.2758582532405853, + -0.11883850395679474, + -0.0904715359210968, + -0.8463842868804932, + 0.4136640429496765, + -2.2815024852752686, + -0.79213947057724, + 0.09477245807647705, + -0.22156566381454468, + -0.08652854710817337, + 0.6423437595367432, + 0.5776209831237793, + -0.06339520215988159, + 0.28895413875579834, + 0.08238177746534348, + -0.19808807969093323, + 0.13033677637577057, + 0.6563591957092285, + 0.33592918515205383, + 0.0385357141494751, + -0.17127865552902222 + ], + [ + 0.2607884705066681, + -0.12364064902067184, + -0.3031565845012665, + -0.7547322511672974, + 0.5611181259155273, + -2.331019163131714, + -0.8667411208152771, + 0.09982959181070328, + -0.23611301183700562, + -0.10993482917547226, + 0.659524142742157, + 0.5154794454574585, + -0.25269031524658203, + 0.3010188937187195, + 0.05662054196000099, + -0.20263463258743286, + 0.09702354669570923, + 0.7584511041641235, + 0.25624024868011475, + -0.11499883234500885, + -0.2106596827507019 + ] + ], + "generated_utilities": [ + 0.06672348827123642, + 0.029613912105560303, + 0.14832614362239838, + 0.1094479188323021, + 0.10840656608343124, + 0.09923364967107773, + 0.055296678096055984, + 0.09003522247076035 + ], + "group_id": "ms-8a72b73ac452bc190aeb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.02298194169998169, + 0.01763375662267208, + 0.009361605159938335, + 0.02526181936264038, + 0.09214292466640472, + 0.022978028282523155, + 0.07704488933086395, + 0.056830525398254395, + 0.045355141162872314, + 0.06095697730779648, + 0.10329639911651611, + 0.05954315513372421, + 0.03003152273595333, + 0.008136491291224957, + 0.05599620193243027, + 0.011116823181509972 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.0913797616958618, + 0.3305599093437195, + -0.0778055489063263, + -2.223818778991699, + 0.9291321635246277, + -2.905466318130493, + -2.4575929641723633, + 1.1732350587844849, + 0.22933529317378998, + -0.8654288053512573, + 2.1122353076934814, + 1.5987088680267334, + 1.144788146018982, + 0.40604642033576965, + 0.10343833267688751, + -0.7513935565948486, + -1.054006814956665, + 1.396103024482727, + -0.16642586886882782, + -0.04196041822433472, + -1.2630513906478882 + ], + [ + 1.3690472841262817, + 0.2738865315914154, + -0.27599766850471497, + -2.431976556777954, + 0.7025187015533447, + -3.155372142791748, + -2.6898910999298096, + 1.1788215637207031, + 0.19074484705924988, + -0.7378551959991455, + 2.3089826107025146, + 1.3478807210922241, + 1.3458092212677002, + 0.3417958617210388, + 0.24155153334140778, + -1.004831314086914, + -1.3499912023544312, + 1.311987042427063, + -0.25646817684173584, + -0.3180008828639984, + -1.3290650844573975 + ], + [ + 0.9032705426216125, + -0.3011789321899414, + -0.21911907196044922, + -2.884286403656006, + 0.319735586643219, + -2.613290548324585, + -2.175532817840576, + 1.5185720920562744, + -0.1741684526205063, + -0.4879068434238434, + 2.721827507019043, + 2.1753995418548584, + 0.8214534521102905, + 0.7874165177345276, + 0.013686276972293854, + -0.7485443949699402, + -0.764183521270752, + 0.9452257752418518, + 0.07210072875022888, + 0.2788098156452179, + -1.8349425792694092 + ] + ], + "predicted_scores": [ + 9.848700523376465, + 9.15388298034668, + 8.921833038330078, + 8.639586448669434, + 8.411186218261719, + 8.158957481384277, + 7.247722625732422, + 7.04017972946167 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "722240dc5e14c5a30badc5fbec9f15ecf388cf6402fb77013c8f9dbdcd100846", + "stored_base_utility": 0.02298194169998169, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08285123109817505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08285123109817505 + }, + "base_progress": 0.08285123109817505, + "base_success": false, + "base_utility": 0.08285123109817505, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.020249387249350548, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020249387249350548 + }, + { + "contact_quality": null, + "progress": 0.023661911487579346, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023661911487579346 + }, + { + "contact_quality": null, + "progress": 0.022281387820839882, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022281387820839882 + }, + { + "contact_quality": null, + "progress": 0.016953786835074425, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016953786835074425 + }, + { + "contact_quality": null, + "progress": 0.03129972890019417, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03129972890019417 + }, + { + "contact_quality": null, + "progress": 0.016959169879555702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016959169879555702 + }, + { + "contact_quality": null, + "progress": 0.03157500550150871, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03157500550150871 + }, + { + "contact_quality": null, + "progress": 0.018817603588104248, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018817603588104248 + } + ], + "candidate_progress": [ + 0.020249387249350548, + 0.023661911487579346, + 0.022281387820839882, + 0.016953786835074425, + 0.03129972890019417, + 0.016959169879555702, + 0.03157500550150871, + 0.018817603588104248 + ], + "candidate_source_chart_ids": [ + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-a96bca1ef0b74f686018", + "generated_tangents": [ + [ + 0.3499390184879303, + 0.028515934944152832, + 0.48948806524276733, + -0.19309116899967194, + 0.6901988983154297, + 0.7414987087249756, + 1.3991317749023438, + 0.3446234166622162, + -0.09190987050533295, + 0.4723714292049408, + -1.2144007682800293, + 0.027294978499412537, + 0.2527715861797333, + -0.9874541759490967, + 0.16269424557685852, + 0.018625933676958084, + 0.18118378520011902, + -0.828308641910553, + -0.5594342350959778, + -0.08648960292339325, + 0.18008112907409668 + ], + [ + -0.09269364178180695, + -0.1040714681148529, + 0.22524771094322205, + -0.3401437997817993, + 0.35260310769081116, + -0.31100767850875854, + 0.9163292646408081, + 0.5302736759185791, + -0.2567659616470337, + 0.14966259896755219, + -1.0596189498901367, + 0.646892786026001, + -0.529009222984314, + -0.9970532655715942, + -0.02623092755675316, + 0.17372286319732666, + 0.23263417184352875, + -0.3831707835197449, + -0.5076351165771484, + -0.2827019691467285, + -1.0047863721847534 + ], + [ + 0.27982860803604126, + 0.038833484053611755, + 0.5253910422325134, + -0.2953806221485138, + 0.7850337028503418, + 0.04751972109079361, + 0.6080531477928162, + 0.255418062210083, + -0.08356447517871857, + 0.4131060838699341, + -1.0247589349746704, + 0.26699909567832947, + -0.08355796337127686, + -0.8738237619400024, + 0.292209655046463, + 0.10542076826095581, + 0.22872647643089294, + -0.5168390870094299, + -0.3613443970680237, + -0.38312795758247375, + -0.16131848096847534 + ], + [ + 0.24409164488315582, + -0.07521600276231766, + 0.4027271270751953, + -0.21762821078300476, + 0.7221702337265015, + 0.15511786937713623, + 0.7601394653320312, + 0.31568828225135803, + -0.17028523981571198, + 0.398857057094574, + -1.0321813821792603, + 0.28403183817863464, + -0.012781113386154175, + -0.808844804763794, + 0.3304174244403839, + 0.10524564981460571, + 0.17134501039981842, + -0.5901852250099182, + -0.24532389640808105, + -0.3594452440738678, + -0.03938356041908264 + ], + [ + 0.221520334482193, + 0.13385054469108582, + 0.5074416399002075, + -0.5112059116363525, + 0.6898107528686523, + 0.50984787940979, + 0.28407323360443115, + 0.2582123875617981, + 0.007201352156698704, + 0.3026183843612671, + -0.8207933902740479, + 0.3518885374069214, + -0.16800397634506226, + 0.003198191523551941, + 0.29176315665245056, + 0.11074809730052948, + 0.12544113397598267, + -0.5459253191947937, + -0.35079002380371094, + -0.3331269919872284, + -0.412781685590744 + ], + [ + 0.3424416184425354, + -0.03370369225740433, + 0.48118382692337036, + -0.23526175320148468, + 0.7294629812240601, + 0.1193767637014389, + 0.536425769329071, + 0.2792583405971527, + -0.16809089481830597, + 0.3494786322116852, + -1.0050755739212036, + 0.27076736092567444, + 0.1457333266735077, + -0.9353020191192627, + 0.36139115691185, + 0.05636360123753548, + 0.17889519035816193, + -0.6878611445426941, + -0.2541653513908386, + -0.47968965768814087, + 0.10619336366653442 + ], + [ + 0.35013851523399353, + 0.19337916374206543, + 0.6391496658325195, + -0.44786036014556885, + 0.8085934519767761, + 0.7319002151489258, + 0.5004884004592896, + 0.22929589450359344, + -0.0033745383843779564, + 0.08201625198125839, + -0.8485991358757019, + 0.2280895859003067, + -0.3726976215839386, + -0.11025431752204895, + 0.4019944965839386, + 0.133483424782753, + 0.10760033875703812, + -0.5622149109840393, + -0.2597987651824951, + 0.11388179659843445, + -0.44191405177116394 + ], + [ + 0.2736077308654785, + -0.04517064243555069, + 0.520946204662323, + -0.3296894133090973, + 1.0444467067718506, + 0.8971071243286133, + 0.7990139722824097, + 0.08666207641363144, + -0.24233770370483398, + 0.5963983535766602, + -0.9442305564880371, + -0.13999488949775696, + -0.15416544675827026, + -0.3024454116821289, + 0.16026315093040466, + 0.14706969261169434, + 0.12341497838497162, + -0.4079155921936035, + -0.12926006317138672, + -0.03957882523536682, + 0.3990253806114197 + ] + ], + "generated_utilities": [ + 0.020249387249350548, + 0.023661911487579346, + 0.022281387820839882, + 0.016953786835074425, + 0.03129972890019417, + 0.016959169879555702, + 0.03157500550150871, + 0.018817603588104248 + ], + "group_id": "ms-a96bca1ef0b74f686018", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07923875749111176, + 0.1507580727338791, + 0.09708476066589355, + 0.1572066992521286, + 0.09626718610525131, + 0.0781460627913475, + 0.07252912223339081, + 0.04244875907897949, + 0.026137471199035645, + 0.03892441838979721, + 0.04886416718363762, + 0.10530619323253632, + 0.0477486252784729, + 0.0898323506116867, + 0.01968318223953247, + 0.0289560966193676 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.48996758460998535, + 0.9053895473480225, + 1.8985445499420166, + -1.3594893217086792, + 1.6097791194915771, + 1.132408618927002, + 1.8319320678710938, + 0.46463528275489807, + 0.13208985328674316, + 0.8792686462402344, + -1.868321418762207, + 1.1490275859832764, + -2.0193891525268555, + -2.0637927055358887, + 0.11448347568511963, + 1.2957178354263306, + 1.293219804763794, + -1.7789109945297241, + -0.4312591254711151, + -1.7441130876541138, + -0.06536102294921875 + ], + [ + 0.03654795140028, + 1.7676925659179688, + 1.048339605331421, + -1.366391658782959, + 2.5093846321105957, + 2.160305976867676, + 2.8675124645233154, + -0.309950590133667, + -0.5841768980026245, + 1.3030188083648682, + -1.2228018045425415, + 1.1373907327651978, + -2.831526041030884, + -2.2111129760742188, + -0.14044174551963806, + 1.660491704940796, + 1.5215414762496948, + -1.0137104988098145, + -0.40579986572265625, + -1.9571893215179443, + 1.241256833076477 + ], + [ + -0.2826732099056244, + 2.2520718574523926, + 2.418309450149536, + -1.7572941780090332, + 1.5832278728485107, + 0.6361246705055237, + 2.2009639739990234, + 0.9109495878219604, + 0.22027403116226196, + 0.7075455188751221, + -1.2762477397918701, + 0.8630691766738892, + -1.2986351251602173, + -2.567821979522705, + -0.4690782427787781, + 2.1363346576690674, + 0.1853562593460083, + -1.541505217552185, + -0.033681899309158325, + -1.0161254405975342, + 0.06877446174621582 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03534998744726181, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.3504420518875122, + 0.0, + -0.013733148574829102, + 0.021970629692077637, + 0.014389097690582275, + -0.7432765960693359, + 0.006671249866485596, + -0.9755387306213379, + 0.0, + -0.027152668684720993, + 0.2708500623703003, + 0.052558183670043945, + -0.7615737915039062, + 0.020878344774246216, + -0.2927910089492798, + 0.0 + ], + [ + -0.1400495320558548, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.5714048147201538, + 0.0, + -0.3271818161010742, + -0.04413123428821564, + 0.16598933935165405, + -0.7432765960693359, + 0.2282017469406128, + -0.9755387306213379, + 0.0, + -0.4382812976837158, + 0.2708500623703003, + 0.2094714641571045, + -0.7615737915039062, + 0.23577484488487244, + -0.2927910089492798, + 0.0 + ] + ], + "predicted_scores": [ + 8.650808334350586, + 8.515298843383789, + 8.338103294372559, + 7.722647190093994, + 7.502362251281738, + 7.491379261016846, + 7.030318260192871, + 6.6235456466674805 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "61714505b9ac6cbb1c6deff698bb3e96fd54c7cdb8f0d4307bfdd236634525c0", + "stored_base_utility": 0.07923875749111176, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.13897453248500824, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13897453248500824 + }, + { + "contact_quality": null, + "progress": 0.10920015722513199, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10920015722513199 + }, + { + "contact_quality": null, + "progress": 0.1372818946838379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1372818946838379 + }, + { + "contact_quality": null, + "progress": 0.11862999200820923, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11862999200820923 + }, + { + "contact_quality": null, + "progress": 0.11477562040090561, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11477562040090561 + }, + { + "contact_quality": null, + "progress": 0.13643063604831696, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13643063604831696 + }, + { + "contact_quality": null, + "progress": 0.128423273563385, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.128423273563385 + }, + { + "contact_quality": null, + "progress": 0.09418871253728867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09418871253728867 + } + ], + "candidate_progress": [ + 0.13897453248500824, + 0.10920015722513199, + 0.1372818946838379, + 0.11862999200820923, + 0.11477562040090561, + 0.13643063604831696, + 0.128423273563385, + 0.09418871253728867 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-ac4fb33f3ab4db853995", + "generated_tangents": [ + [ + 2.3757364749908447, + -0.27223077416419983, + -0.3891852796077728, + 0.7745223045349121, + -0.446671724319458, + 2.279881000518799, + 1.8678135871887207, + 2.2902791500091553, + 0.25990647077560425, + -1.510072112083435, + -0.7205798625946045, + -0.9597678184509277, + -2.9762816429138184, + -3.139012098312378, + 1.6004019975662231, + 0.10748055577278137, + -1.2820088863372803, + 0.7429741024971008, + -0.2779664397239685, + -0.9638658165931702, + -1.136738657951355 + ], + [ + 2.2156591415405273, + -0.1741964966058731, + -0.6328035593032837, + 0.6409015655517578, + -0.20161858201026917, + 2.043597459793091, + 1.1954050064086914, + 2.1038784980773926, + 0.17956028878688812, + -1.4242427349090576, + -0.5126789808273315, + -0.8243266344070435, + -2.6796391010284424, + -2.430363893508911, + 1.8180451393127441, + -0.07958535850048065, + -1.4335682392120361, + 0.4636613130569458, + 0.040396884083747864, + -1.315406322479248, + -0.4311096966266632 + ], + [ + 2.419240951538086, + -0.23622797429561615, + -0.6210044622421265, + 0.5424371957778931, + -0.3453226089477539, + 2.029294490814209, + 2.1774356365203857, + 2.1821088790893555, + 0.301272988319397, + -1.5728907585144043, + -0.7277917265892029, + -1.073206901550293, + -3.0157980918884277, + -3.2045888900756836, + 1.7884563207626343, + -0.04911618307232857, + -1.4222990274429321, + 0.6680842638015747, + -0.36462271213531494, + -1.1607216596603394, + -1.462888479232788 + ], + [ + 2.3462376594543457, + -0.21311748027801514, + -0.5850708484649658, + 0.58466637134552, + -0.3197994828224182, + 2.0855517387390137, + 2.11960768699646, + 2.3327271938323975, + 0.25269460678100586, + -1.5906559228897095, + -0.6323912739753723, + -0.9704151153564453, + -2.539240598678589, + -3.236276865005493, + 1.8979101181030273, + -0.004533700179308653, + -1.3840817213058472, + 0.7861436009407043, + -0.29279518127441406, + -1.1931861639022827, + -1.2252819538116455 + ], + [ + 2.338474750518799, + -0.13967642188072205, + -0.39760419726371765, + 0.5495784282684326, + -0.6413520574569702, + 2.1784238815307617, + 1.938929557800293, + 2.422785758972168, + 0.26703953742980957, + -1.5116997957229614, + -0.6223490834236145, + -1.0662301778793335, + -2.746263265609741, + -3.1816065311431885, + 1.769331932067871, + -0.002677437849342823, + -1.1240551471710205, + 0.8676563501358032, + -0.2712363004684448, + -1.125828742980957, + -1.083319902420044 + ], + [ + 2.255201578140259, + -0.12751361727714539, + -0.5567959547042847, + 0.6307381391525269, + -0.4799419045448303, + 2.024019479751587, + 2.1195852756500244, + 2.303117513656616, + 0.2746199369430542, + -1.4792096614837646, + -0.6680079102516174, + -1.0119272470474243, + -2.908134937286377, + -3.2704591751098633, + 1.9480485916137695, + -0.011402618139982224, + -1.2913742065429688, + 0.7679071426391602, + -0.230330228805542, + -1.0150169134140015, + -1.1405054330825806 + ], + [ + 2.3860251903533936, + -0.22043639421463013, + -0.4959961473941803, + 0.20974993705749512, + -0.1611173152923584, + 1.9938403367996216, + 2.0718514919281006, + 2.077143430709839, + 0.31236532330513, + -1.8010988235473633, + -0.5873530507087708, + -0.979231595993042, + -2.47955584526062, + -2.967294216156006, + 1.6589266061782837, + -0.19949805736541748, + -1.3049830198287964, + 0.8772557973861694, + -0.18165990710258484, + -1.5418906211853027, + -1.3559668064117432 + ], + [ + 2.299116373062134, + -0.1917165219783783, + -0.4935736358165741, + 0.6249113082885742, + -0.025500982999801636, + 1.851459264755249, + 1.139762043952942, + 2.1488285064697266, + 0.2613762617111206, + -1.504454255104065, + -0.25766506791114807, + -0.9192250967025757, + -2.4594264030456543, + -2.520716905593872, + 1.9626132249832153, + -0.038190580904483795, + -1.1951498985290527, + 0.6068487763404846, + 0.4906315207481384, + -0.607388973236084, + -1.844285488128662 + ] + ], + "generated_utilities": [ + 0.13897453248500824, + 0.10920015722513199, + 0.1372818946838379, + 0.11862999200820923, + 0.11477562040090561, + 0.13643063604831696, + 0.128423273563385, + 0.09418871253728867 + ], + "group_id": "ms-ac4fb33f3ab4db853995", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.2651830017566681, + 0.22676818072795868, + 0.22454051673412323, + 2.0, + 0.31214088201522827, + 0.21535459160804749, + 0.18962346017360687, + 2.0, + 0.1830430030822754, + 0.15030252933502197, + 0.12050267308950424, + 0.17022761702537537, + 0.11389490216970444, + 0.07058519124984741, + 0.07076358795166016, + 0.18658748269081116 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 2.0962133407592773, + 0.057441189885139465, + -0.2733909487724304, + 0.4170292317867279, + -0.7772165536880493, + 2.858414888381958, + 2.1098427772521973, + 1.8366810083389282, + 0.017507493495941162, + -1.6127345561981201, + -0.39152991771698, + -1.017486333847046, + -2.815659999847412, + -2.787520408630371, + 1.6748408079147339, + 0.2986273169517517, + -0.6001855134963989, + 0.5475705862045288, + -0.28012242913246155, + -1.5382447242736816, + -1.6652274131774902 + ], + [ + 2.1338062286376953, + -0.07656055688858032, + -0.7531881332397461, + 0.48830646276474, + -0.36628106236457825, + 2.381558418273926, + 1.7315943241119385, + 1.838928461074829, + -0.050897032022476196, + -1.171897292137146, + -0.8350631594657898, + -0.8261303901672363, + -2.2635834217071533, + -3.2190167903900146, + 1.3416705131530762, + 0.44279617071151733, + -0.7299901247024536, + 1.2929750680923462, + -0.0032969415187835693, + -1.5000369548797607, + -1.6060972213745117 + ], + [ + 1.5521352291107178, + 0.03659695386886597, + -0.44382959604263306, + 0.21355122327804565, + -1.0471938848495483, + 3.194620370864868, + 1.9892739057540894, + 2.5084128379821777, + 0.531404972076416, + -0.9168282747268677, + -0.49052953720092773, + -1.1588059663772583, + -3.156189441680908, + -3.278008460998535, + 0.921509325504303, + 0.49181532859802246, + -1.0228296518325806, + 0.5866082310676575, + -0.4882396459579468, + -0.7946261763572693, + -2.119061231613159 + ], + [ + 2.7138938903808594, + -0.3406146764755249, + -0.32174432277679443, + 0.9962238669395447, + -0.558576226234436, + 2.049264907836914, + 1.8703808784484863, + 2.113438367843628, + 0.9615883827209473, + -1.9615708589553833, + -0.0062177181243896484, + -0.5808464288711548, + -2.1758205890655518, + -2.219890594482422, + 1.022050142288208, + 0.7201904654502869, + -1.3177021741867065, + 0.4089192748069763, + -0.0315951406955719, + -0.7215378880500793, + -2.069262981414795 + ], + [ + 2.5701913833618164, + 0.3137266933917999, + -0.5592924356460571, + 0.0026398897171020508, + -0.19283491373062134, + 2.2903807163238525, + 2.637770652770996, + 2.5494186878204346, + 0.9258015155792236, + -0.7146614193916321, + -0.686233639717102, + -0.7985042929649353, + -3.431988000869751, + -3.2178471088409424, + 1.319100260734558, + -0.15109702944755554, + -0.2694774866104126, + 1.0912322998046875, + -0.6380462050437927, + -2.0780839920043945, + -2.1209774017333984 + ], + [ + 2.6195321083068848, + -0.2646539509296417, + 0.15159690380096436, + 0.8749752044677734, + -1.1848887205123901, + 2.8789353370666504, + 1.6037814617156982, + 1.3343384265899658, + -0.2254078984260559, + -2.041776657104492, + -0.9811336994171143, + -1.3441768884658813, + -2.2502636909484863, + -1.9091612100601196, + 0.9959704279899597, + 0.42473238706588745, + -0.5330828428268433, + 1.6945891380310059, + 0.4069111943244934, + -0.5568764805793762, + -1.472819447517395 + ], + [ + 2.9485361576080322, + -0.6907311677932739, + -0.057036399841308594, + 1.2457408905029297, + 0.09488356113433838, + 2.6509125232696533, + 2.0378005504608154, + 2.278454303741455, + 1.1734973192214966, + -1.853393316268921, + 0.09614759683609009, + -0.35674989223480225, + -2.1597909927368164, + -1.9672894477844238, + 1.9760816097259521, + -0.47744297981262207, + 0.021782994270324707, + 1.6328153610229492, + -0.8569514751434326, + -1.0814869403839111, + -1.7780723571777344 + ], + [ + 2.1897926330566406, + -0.3883178234100342, + -0.6492674946784973, + 1.3223958015441895, + 0.30954307317733765, + 2.031817674636841, + 1.878535270690918, + 2.5030624866485596, + 0.6644855737686157, + -1.7382179498672485, + -1.1862120628356934, + -0.7967990636825562, + -2.8425841331481934, + -2.3635308742523193, + 2.0061349868774414, + -0.6015456318855286, + -1.429253339767456, + 1.2773160934448242, + 0.04122643172740936, + -2.0497689247131348, + -1.1791472434997559 + ], + [ + 1.586853265762329, + 0.06395213305950165, + -0.8944701552391052, + 0.5118889808654785, + -1.1310638189315796, + 2.865907669067383, + 1.3295278549194336, + 1.0862370729446411, + 0.6303088665008545, + -0.3754369616508484, + -0.04129624366760254, + -0.7044278979301453, + -2.923245429992676, + -1.7837438583374023, + 1.5932514667510986, + 1.2467985153198242, + 0.13401156663894653, + 1.6706414222717285, + -0.816297709941864, + -1.2893165349960327, + -1.0985345840454102 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.1383795738220215, + -0.1483764797449112, + -0.27178800106048584, + 0.26738643646240234, + -0.12845781445503235, + 1.6495440006256104, + 0.0, + 0.9713280200958252, + -0.050046056509017944, + -0.36543774604797363, + -0.022124826908111572, + 0.08494043350219727, + -1.671510934829712, + 0.0, + 0.5271386504173279, + 0.3898613452911377, + 0.21280205249786377, + -0.2029280662536621, + -0.01802508533000946, + -0.35994601249694824, + 0.0 + ], + [ + 1.7570452690124512, + 0.07137972116470337, + 0.02318662405014038, + 0.2907020151615143, + -0.2308352291584015, + 2.89892578125, + 2.537220001220703, + 2.238605499267578, + 0.16207315027713776, + -1.6996564865112305, + -0.8050198554992676, + -1.2014062404632568, + -2.5764517784118652, + -2.393839120864868, + 1.1276613473892212, + 0.31928181648254395, + -1.036102533340454, + 1.1266506910324097, + -0.1944657564163208, + -1.5998979806900024, + -2.151465892791748 + ] + ], + "predicted_scores": [ + 7.231818675994873, + 6.486433506011963, + 6.4527907371521, + 6.395447731018066, + 6.298037052154541, + 6.243171215057373, + 5.761966705322266, + 4.903831958770752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "0b62c42c3beda0eb959692d55792514223681b522edd0a5e73247682cf0bfe95", + "stored_base_utility": 0.2651830017566681, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "generated_tangents": [ + [ + 1.9789299964904785, + 0.18482418358325958, + -1.461500644683838, + -0.10560663789510727, + -0.481938898563385, + -0.6769506931304932, + -0.8734853863716125, + 0.45116496086120605, + 0.2704058289527893, + -0.25655752420425415, + -0.3241361677646637, + 0.23260793089866638, + -3.059511661529541, + -2.303499698638916, + 0.5132331848144531, + 0.14198794960975647, + 0.6234413385391235, + -1.2580815553665161, + 0.6944096684455872, + -2.3318138122558594, + -1.224258542060852 + ], + [ + 2.0985231399536133, + -0.11934754997491837, + -1.388933777809143, + 0.053231775760650635, + -0.6105248928070068, + -0.7982304096221924, + -0.7380546927452087, + 0.559706449508667, + 0.21137775480747223, + -0.22078683972358704, + -0.41030898690223694, + 0.47239795327186584, + -3.256456136703491, + -2.5135691165924072, + 0.6467328071594238, + 0.25664636492729187, + 0.3035256266593933, + -1.3714854717254639, + 0.9082530736923218, + -2.3454537391662598, + -1.850858449935913 + ], + [ + 1.9810668230056763, + 0.16683797538280487, + -1.8061074018478394, + -0.3732788860797882, + -0.0841829776763916, + -0.29402515292167664, + -1.1140425205230713, + 0.6322353482246399, + -0.005932627245783806, + -0.46816450357437134, + -0.08492376655340195, + 0.6054893136024475, + -2.676499366760254, + -2.7999207973480225, + 0.6312494277954102, + 0.30878767371177673, + 0.614536464214325, + -1.368439793586731, + 0.8445847630500793, + -2.334414482116699, + -1.2858355045318604 + ], + [ + 1.9530315399169922, + 0.15125535428524017, + -1.6527477502822876, + -0.6363152861595154, + -0.15198585391044617, + -0.07052426040172577, + -1.2695701122283936, + 0.8097606897354126, + -0.09663358330726624, + -0.3970586061477661, + -0.47795388102531433, + 0.5740829706192017, + -3.153839349746704, + -2.915522813796997, + 0.7115260362625122, + 0.4020631015300751, + 0.5967467427253723, + -1.4221256971359253, + 0.8709081411361694, + -2.372349739074707, + -1.5803728103637695 + ], + [ + 1.9482799768447876, + 0.11157537996768951, + -1.57022225856781, + -0.5205692052841187, + -0.11835727095603943, + -0.28709328174591064, + -1.3624851703643799, + 0.7494869232177734, + -0.05077909678220749, + -0.41678833961486816, + -0.28302374482154846, + 0.44418662786483765, + -2.9330546855926514, + -2.786648750305176, + 0.6355831623077393, + 0.4074035882949829, + 0.6077445149421692, + -1.4902455806732178, + 0.7996195554733276, + -2.435551881790161, + -1.4435468912124634 + ], + [ + 1.7593673467636108, + -0.1820324957370758, + -1.3708407878875732, + -0.11217817664146423, + 0.006536126136779785, + -0.5965646505355835, + -1.046247959136963, + 0.7461029291152954, + 0.024881847202777863, + -0.20207908749580383, + -0.24686360359191895, + 0.6631032228469849, + -3.025989294052124, + -1.7664202451705933, + 0.5276257991790771, + 0.3083820939064026, + 0.9129725098609924, + -1.3154927492141724, + 1.2958452701568604, + -2.1096575260162354, + -0.78299880027771 + ], + [ + 2.1856632232666016, + 0.0016049854457378387, + -1.6249693632125854, + -0.5716937780380249, + -0.16711804270744324, + 0.08690698444843292, + -1.261176347732544, + 0.9117177724838257, + 0.002620358020067215, + -0.3168095648288727, + -0.3148176670074463, + 0.5934818983078003, + -2.957610845565796, + -2.7233242988586426, + 0.7135811448097229, + 0.35289466381073, + 0.5558337569236755, + -1.6773006916046143, + 0.8219923973083496, + -2.949918508529663, + -1.2962119579315186 + ], + [ + 1.944191575050354, + 0.09757903218269348, + -1.4973571300506592, + -0.5029289722442627, + -0.022460639476776123, + -0.5843757390975952, + -1.260545015335083, + 0.872179388999939, + -0.11007992923259735, + -0.3647612929344177, + -0.3585472106933594, + 0.22520257532596588, + -3.1110293865203857, + -2.633798599243164, + 0.6089034676551819, + 0.46097666025161743, + 0.7376998662948608, + -1.6006757020950317, + 0.7458087205886841, + -2.6057984828948975, + -1.4574354887008667 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.149173304438591, + 0.06221481412649155, + 0.1385849118232727, + 0.06656398624181747, + 2.0, + 0.10397814214229584, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7398372888565063, + -0.02836926281452179, + -0.4292973279953003, + 0.028730571269989014, + 0.009982943534851074, + -0.3464317321777344, + 0.0, + 0.009800434112548828, + -0.06816069781780243, + 0.030653484165668488, + -0.00554502010345459, + -0.06216755136847496, + -1.9802618026733398, + 0.0, + 0.01717478036880493, + -0.05601567029953003, + 0.010390102863311768, + -0.8072932958602905, + -0.05775028467178345, + -1.9342114925384521, + 0.0 + ], + [ + 0.7398372888565063, + -0.1961192637681961, + -0.4292973279953003, + -0.0017886161804199219, + 0.06461763381958008, + -0.3464317321777344, + 0.0, + 0.42972221970558167, + -0.5797654986381531, + 0.19436834752559662, + -0.23844504356384277, + 0.1733923852443695, + -1.9802618026733398, + 0.0, + -0.27269166707992554, + 0.04145494103431702, + 0.12319430708885193, + -0.8072932958602905, + 0.4949256181716919, + -1.9342114925384521, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7398372888565063, + 0.2242223471403122, + -1.4292973279953003, + -0.537040650844574, + -0.9353823661804199, + -1.3464317321777344, + -2.159137487411499, + 0.489885151386261, + -0.006211067084223032, + 0.09238272905349731, + -1.0055450201034546, + -0.04778099060058594, + -2.98026180267334, + -2.576570987701416, + 0.2992324233055115, + 0.34874510765075684, + 0.5165960192680359, + -1.8072932958602905, + 0.834364652633667, + -2.934211492538452, + -2.0790040493011475 + ], + [ + 1.9740948677062988, + 0.4187958538532257, + -1.2225801944732666, + -0.6112513542175293, + -1.0962133407592773, + -1.4875872135162354, + -2.1883857250213623, + 0.6603437662124634, + -0.20142723619937897, + 0.1697872281074524, + -0.8377580046653748, + -0.16444604098796844, + -2.87237286567688, + -2.4153482913970947, + 0.03898555040359497, + 0.5175161957740784, + 0.7254846096038818, + -1.9598647356033325, + 1.1053451299667358, + -3.172945976257324, + -1.9334450960159302 + ], + [ + 1.5938913822174072, + -0.018363401293754578, + -1.1851862668991089, + -0.20344951748847961, + -0.9226735234260559, + -1.3553601503372192, + -1.932102084159851, + 0.6446930170059204, + 0.30503973364830017, + 0.3522501289844513, + -0.9527953267097473, + -0.15980565547943115, + -2.9574379920959473, + -2.9056527614593506, + 0.6069583892822266, + 0.3722725808620453, + 0.7255810499191284, + -1.8503519296646118, + 0.5613387823104858, + -3.095244884490967, + -2.023747444152832 + ], + [ + 1.6626678705215454, + -0.02834661304950714, + -1.0125243663787842, + -0.5629516839981079, + -1.230878233909607, + -1.414373755455017, + -2.42856764793396, + 0.8520264625549316, + -0.1296539008617401, + -0.22410845756530762, + -1.0539721250534058, + -0.060277245938777924, + -2.740527391433716, + -2.1194698810577393, + 0.6826534867286682, + 0.7535779476165771, + 0.35573869943618774, + -1.9124289751052856, + 0.7101565003395081, + -3.326658010482788, + -1.8195855617523193 + ], + [ + 2.2881288528442383, + 0.6951426863670349, + -1.0978585481643677, + -0.008003771305084229, + -1.0754073858261108, + -1.7734596729278564, + -2.139570474624634, + 0.9205271601676941, + 0.29560548067092896, + 0.49158430099487305, + -1.2606573104858398, + 0.43956148624420166, + -2.831833600997925, + -2.960129976272583, + 0.6832801103591919, + -0.03894898295402527, + 1.036036491394043, + -2.127277135848999, + 0.6759494543075562, + -3.1690073013305664, + -1.9895020723342896 + ], + [ + 1.9020261764526367, + 0.6575945019721985, + -1.3994193077087402, + -0.5040701031684875, + -0.6857678890228271, + -0.7056208252906799, + -1.5344107151031494, + 0.7615087032318115, + 0.6105715036392212, + 0.19082021713256836, + -0.6303516626358032, + -0.4614318609237671, + -2.3375303745269775, + -2.188478469848633, + 0.6886799335479736, + -0.03237125277519226, + 0.483882874250412, + -1.3982734680175781, + 0.23996800184249878, + -3.3969180583953857, + -1.5116117000579834 + ], + [ + 1.666227102279663, + 0.6915197968482971, + -1.6392340660095215, + -0.9650112390518188, + -0.2591104507446289, + -1.5417896509170532, + -1.695828914642334, + 1.130251169204712, + -0.555443525314331, + -0.4872768521308899, + -1.0994479656219482, + 0.5934816002845764, + -3.5218958854675293, + -1.8239338397979736, + 0.7150859832763672, + 0.8003170490264893, + 1.0113730430603027, + -2.1905770301818848, + 0.8595032095909119, + -3.039506673812866, + -2.3675880432128906 + ], + [ + 1.59856116771698, + -0.036071136593818665, + -1.700652837753296, + 0.27602267265319824, + -1.608276128768921, + -0.875938355922699, + -1.6728798151016235, + 0.8851535320281982, + 0.7461313605308533, + -0.7497223019599915, + -1.0928030014038086, + -0.4735495150089264, + -2.223254680633545, + -2.5322465896606445, + 0.044906944036483765, + 0.6160990595817566, + 1.3697642087936401, + -1.6367510557174683, + 0.086475670337677, + -3.323115825653076, + -2.058189630508423 + ], + [ + 2.285855531692505, + 0.6314746141433716, + -1.2119637727737427, + 0.039707839488983154, + -0.9751565456390381, + -1.7796330451965332, + -1.5261995792388916, + 0.5450726747512817, + -0.8933275938034058, + 0.7903657555580139, + -1.5466060638427734, + 0.5994364023208618, + -3.370340585708618, + -3.1641077995300293, + 0.3225497603416443, + -0.39524197578430176, + 1.152822732925415, + -1.508494257926941, + 1.1221402883529663, + -3.361729860305786, + -1.9251339435577393 + ], + [ + 2.2599079608917236, + -0.005553290247917175, + -2.4053664207458496, + 0.39961689710617065, + -0.40980857610702515, + -0.9905421733856201, + -1.7244036197662354, + -0.06417369842529297, + 0.6376181840896606, + 0.5168956518173218, + -1.6740490198135376, + -0.1833474189043045, + -3.9368975162506104, + -1.7146861553192139, + 0.9366075992584229, + 0.701168417930603, + -0.23131346702575684, + -2.740941286087036, + 0.908729076385498, + -3.6231396198272705, + -1.3161351680755615 + ], + [ + 1.819408655166626, + -0.5504165291786194, + -1.302573800086975, + -0.6927691698074341, + -0.7548186779022217, + -2.1824703216552734, + -2.4921669960021973, + 0.6337187886238098, + 0.1691494882106781, + 0.7547447681427002, + -0.4371253252029419, + -0.6562469601631165, + -2.313655376434326, + -2.4974589347839355, + 0.05105690658092499, + 0.3473445773124695, + -0.4244192838668823, + -1.8233305215835571, + -0.1157217025756836, + -2.0408761501312256, + -2.2010600566864014 + ] + ], + "predicted_scores": [ + 7.137040615081787, + 6.98091983795166, + 6.582064628601074, + 5.941701412200928, + 5.787785053253174, + 5.627976894378662, + 5.44954252243042, + 5.1042938232421875 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2054899126", + "selected_index": 0, + "state_hash": "b26c5f64bc7e877098323281f2b58e7850275b30405bc411ed7ffc7cf69f4e26", + "stored_base_utility": 0.149173304438591, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.037537217140197754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037537217140197754 + }, + "base_progress": 0.037537217140197754, + "base_success": false, + "base_utility": 0.037537217140197754, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.004155457019805908, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004155457019805908 + }, + { + "contact_quality": null, + "progress": 0.003685553791001439, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003685553791001439 + }, + { + "contact_quality": null, + "progress": 0.004423340316861868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004423340316861868 + }, + { + "contact_quality": null, + "progress": 0.005550503730773926, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005550503730773926 + }, + { + "contact_quality": null, + "progress": 0.005441188812255859, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005441188812255859 + }, + { + "contact_quality": null, + "progress": 0.00635818624868989, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00635818624868989 + }, + { + "contact_quality": null, + "progress": 0.007275819778442383, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007275819778442383 + }, + { + "contact_quality": null, + "progress": 0.004515886306762695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004515886306762695 + } + ], + "candidate_progress": [ + 0.004155457019805908, + 0.003685553791001439, + 0.004423340316861868, + 0.005550503730773926, + 0.005441188812255859, + 0.00635818624868989, + 0.007275819778442383, + 0.004515886306762695 + ], + "candidate_source_chart_ids": [ + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-6f26118aa94bacd82250" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c832439976a37f52d094", + "generated_tangents": [ + [ + -0.29347485303878784, + -0.007099565118551254, + 0.12988796830177307, + 0.19824010133743286, + -0.018882334232330322, + 0.2551068961620331, + -0.6858754754066467, + 1.5179896354675293, + -0.3161171078681946, + -0.8263975977897644, + -1.5724046230316162, + 1.4986058473587036, + -1.329538106918335, + -2.121022939682007, + 0.14946617186069489, + 0.22442542016506195, + 0.04264265298843384, + 0.0784381628036499, + 0.5420059561729431, + 2.1648428440093994, + 0.6930601596832275 + ], + [ + -0.0771825760602951, + 0.1640229970216751, + 0.20471450686454773, + 0.5147385597229004, + -0.1493327021598816, + -0.21187573671340942, + -0.5549275279045105, + 1.415863037109375, + -0.30152058601379395, + -0.8002134561538696, + -1.7488105297088623, + 1.2112573385238647, + -1.4759513139724731, + -1.9330061674118042, + 0.21658742427825928, + 0.1840268075466156, + -0.12504936754703522, + -0.024263814091682434, + 0.45800065994262695, + 1.1132078170776367, + 0.2909638583660126 + ], + [ + -0.12291647493839264, + 0.16723252832889557, + 0.301577091217041, + 0.6293084621429443, + 0.07272940874099731, + -0.11016964912414551, + -0.4484826326370239, + 1.4451688528060913, + -0.3433241546154022, + -0.7893127799034119, + -1.6048904657363892, + 1.4750607013702393, + -1.3179724216461182, + -1.5939264297485352, + 0.16794894635677338, + 0.19196021556854248, + -0.14914895594120026, + -0.0010900944471359253, + 0.3348276615142822, + 1.2292795181274414, + 0.34833234548568726 + ], + [ + -0.2835384011268616, + 0.05340753495693207, + -0.14205411076545715, + 0.43108847737312317, + 0.12497077882289886, + 0.537225604057312, + 0.06485310196876526, + 1.2107707262039185, + -0.4080606997013092, + -0.5053415894508362, + -1.3102506399154663, + 1.2501064538955688, + -1.1867755651474, + -1.7842912673950195, + 0.3090161085128784, + 0.10927611589431763, + 0.10385721921920776, + -0.042656123638153076, + 0.3441688120365143, + 1.5987038612365723, + 0.43927308917045593 + ], + [ + -0.46174389123916626, + 0.07585708796977997, + -0.09058967232704163, + 0.45497867465019226, + 0.07016071677207947, + 0.3548803925514221, + -0.4941389560699463, + 1.3059459924697876, + -0.500140368938446, + -0.5840087532997131, + -1.5642633438110352, + 1.3629822731018066, + -1.434400200843811, + -2.0762150287628174, + 0.2914178669452667, + 0.13851869106292725, + 0.0022342801094055176, + -0.01099364459514618, + 0.5860754251480103, + 1.1163792610168457, + 0.6413156986236572 + ], + [ + -0.4503415822982788, + 0.11110465228557587, + -0.03892597556114197, + 0.3356201946735382, + 0.12225022912025452, + 0.2972986698150635, + -0.30001384019851685, + 1.2160446643829346, + -0.461029589176178, + -0.46410125494003296, + -1.5512192249298096, + 1.3206120729446411, + -1.3483890295028687, + -1.9572304487228394, + 0.35477444529533386, + 0.1161455512046814, + 0.19328224658966064, + -0.057489290833473206, + 0.43606698513031006, + 1.4491631984710693, + 0.534327507019043 + ], + [ + -0.5631589889526367, + 0.02370622009038925, + 0.020105421543121338, + 0.538189172744751, + -0.03366956114768982, + 0.1482982486486435, + -0.5607455372810364, + 1.1600546836853027, + -0.3802933692932129, + -0.6177942156791687, + -1.509847640991211, + 1.4257028102874756, + -1.2602781057357788, + -1.923789620399475, + 0.2732735872268677, + 0.08432956039905548, + 0.00864499807357788, + -0.051631078124046326, + 0.490532249212265, + 1.2050237655639648, + 0.6564277410507202 + ], + [ + 0.09293289482593536, + 0.042575716972351074, + -0.11180070042610168, + 0.5683645009994507, + -0.07158035039901733, + -0.35383954644203186, + 0.19918754696846008, + 1.123085618019104, + -0.18470652401447296, + -0.3706522583961487, + -1.3804121017456055, + 0.8163624405860901, + -1.9296634197235107, + -1.5383012294769287, + 0.4167742431163788, + -0.036186885088682175, + -0.03164835274219513, + 0.019610613584518433, + 0.2125152349472046, + 1.6182937622070312, + 0.4322349429130554 + ] + ], + "generated_utilities": [ + 0.004155457019805908, + 0.003685553791001439, + 0.004423340316861868, + 0.005550503730773926, + 0.005441188812255859, + 0.00635818624868989, + 0.007275819778442383, + 0.004515886306762695 + ], + "group_id": "ms-c832439976a37f52d094", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0808674544095993, + 0.13129448890686035, + 0.11199682950973511, + 0.2594742774963379, + 0.17684690654277802, + 0.11511725187301636, + 0.17100229859352112, + 0.12906593084335327, + 0.12404869496822357, + 0.10653237998485565, + 0.15492436289787292, + 0.05800905078649521, + 0.04767048358917236, + 0.09533219039440155, + 0.03300365060567856, + 0.07704520225524902 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025296777486801147, + 0.11629299819469452, + -0.024434387683868408, + 1.0812468528747559, + 0.04022188484668732, + -0.01547396183013916, + 0.0, + 0.05167186260223389, + 0.07629208266735077, + -0.00870610773563385, + -0.5777420997619629, + 0.07687681913375854, + -0.6336902379989624, + 0.0, + -0.043449461460113525, + -0.029798656702041626, + 0.18944299221038818, + 0.03022998571395874, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + 0.05685463547706604, + -0.3914306163787842, + 0.4578048586845398, + 1.0812468528747559, + 0.10990457236766815, + -0.18159747123718262, + 0.0, + 0.05167186260223389, + -0.5002377033233643, + 0.04299694299697876, + -0.5777420997619629, + 0.02747189998626709, + -0.6336902379989624, + 0.0, + -0.08638033270835876, + 0.17561247944831848, + 0.37304359674453735, + 0.2538231611251831, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + -0.3902576267719269, + 0.03852551057934761, + 0.16114424169063568, + 2.081246852874756, + 0.22516529262065887, + 0.8184025287628174, + 0.3388257324695587, + 1.0516718626022339, + -0.061541248112916946, + 0.22279328107833862, + -1.577742099761963, + 1.027471899986267, + -1.6336902379989624, + -1.6284003257751465, + -0.24836044013500214, + -0.3427097499370575, + 1.1894429922103882, + -0.15580247342586517, + 1.7312084436416626, + 2.8623785972595215, + 1.0248357057571411 + ], + [ + -0.5903857946395874, + -0.10628272593021393, + -0.07594148814678192, + 2.194340944290161, + 0.11632551997900009, + 0.942036509513855, + 0.06351110339164734, + 0.7966359853744507, + 0.1702205240726471, + 0.1549067199230194, + -1.439386248588562, + 1.3076335191726685, + -1.3458164930343628, + -1.90281081199646, + -0.2967699468135834, + -0.27783676981925964, + 1.0556988716125488, + 0.06510581076145172, + 1.6170521974563599, + 3.027111530303955, + 1.154527187347412 + ], + [ + -0.7559880614280701, + 0.694898247718811, + -0.3551136255264282, + 1.8615511655807495, + 0.9249891042709351, + 0.969862699508667, + 0.4452719986438751, + 1.5801937580108643, + 0.13303470611572266, + -0.13816028833389282, + -1.2399333715438843, + 1.5769362449645996, + -1.5071980953216553, + -2.0578060150146484, + -0.8449342846870422, + 0.3327537477016449, + 1.6781622171401978, + 0.4016150236129761, + 1.8172147274017334, + 3.1609320640563965, + 1.4916398525238037 + ] + ], + "predicted_scores": [ + 7.05134916305542, + 6.958633899688721, + 6.76113748550415, + 6.500876426696777, + 6.25551700592041, + 6.220318794250488, + 6.049957752227783, + 5.959582805633545 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "0f2810f22383259329f6e94fdc8384176c664f2ed476924e59db7cda4f4a4863", + "stored_base_utility": 0.0808674544095993, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09138330072164536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09138330072164536 + }, + "base_progress": 0.09138330072164536, + "base_success": false, + "base_utility": 0.09138330072164536, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.012376229278743267, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012376229278743267 + }, + { + "contact_quality": null, + "progress": 0.006582856178283691, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006582856178283691 + }, + { + "contact_quality": null, + "progress": 0.005475322250276804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005475322250276804 + }, + { + "contact_quality": null, + "progress": 0.015102227218449116, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015102227218449116 + }, + { + "contact_quality": null, + "progress": 0.007560948375612497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007560948375612497 + }, + { + "contact_quality": null, + "progress": 0.010635594837367535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010635594837367535 + }, + { + "contact_quality": null, + "progress": 0.0225119199603796, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0225119199603796 + }, + { + "contact_quality": null, + "progress": 0.003913839813321829, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003913839813321829 + } + ], + "candidate_progress": [ + 0.012376229278743267, + 0.006582856178283691, + 0.005475322250276804, + 0.015102227218449116, + 0.007560948375612497, + 0.010635594837367535, + 0.0225119199603796, + 0.003913839813321829 + ], + "candidate_source_chart_ids": [ + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c96b415f083f60857334", + "generated_tangents": [ + [ + -0.2556810975074768, + 0.1454085111618042, + 0.9927430152893066, + 0.615959882736206, + 1.1354053020477295, + 1.9658490419387817, + 1.965430498123169, + -0.10682886838912964, + -0.3494552969932556, + 0.8687063455581665, + 0.34482988715171814, + 0.40220969915390015, + 1.392223834991455, + 1.1267098188400269, + -0.06866274774074554, + -0.06443829834461212, + 0.03723445534706116, + 1.259427785873413, + -0.4557071924209595, + 0.15480095148086548, + -0.4620818793773651 + ], + [ + -0.5475700497627258, + 0.12537738680839539, + 1.3051954507827759, + 0.2073497474193573, + 1.4401142597198486, + 1.6757891178131104, + 1.8697736263275146, + -0.32841405272483826, + -0.39485877752304077, + 0.9930245280265808, + -0.22751465439796448, + 0.3758634924888611, + 2.1365277767181396, + 1.4237288236618042, + -0.37164661288261414, + -0.445719838142395, + -0.14464490115642548, + 1.3406617641448975, + 0.010874032974243164, + 0.2400699257850647, + -0.7034790515899658 + ], + [ + -0.3907206058502197, + 0.0502595528960228, + 0.9871405959129333, + 0.31128743290901184, + 1.4287807941436768, + 1.6949974298477173, + 1.7582051753997803, + -0.25582271814346313, + -0.34462422132492065, + 1.0377522706985474, + -0.09380679577589035, + 0.47910740971565247, + 1.8251665830612183, + 1.2638260126113892, + -0.3799847364425659, + -0.4535689055919647, + 0.02676314115524292, + 1.3922412395477295, + 0.07411304116249084, + 0.33839088678359985, + -0.6817395687103271 + ], + [ + -0.24493928253650665, + 0.19693492352962494, + 0.9518611431121826, + 0.5513356924057007, + 1.1608537435531616, + 2.050548553466797, + 1.889594554901123, + -0.17655092477798462, + -0.27446961402893066, + 0.7984658479690552, + 0.33503103256225586, + 0.4188429117202759, + 1.4712934494018555, + 1.1284092664718628, + -0.00196128711104393, + -0.17179828882217407, + -0.049263134598731995, + 1.2499834299087524, + -0.42109644412994385, + 0.1783105432987213, + -0.18764570355415344 + ], + [ + -0.3054867386817932, + 0.13826729357242584, + 1.0732299089431763, + 0.330486923456192, + 1.3794970512390137, + 1.8489264249801636, + 1.951803207397461, + -0.13306602835655212, + -0.3248480558395386, + 1.088460922241211, + 0.05854111164808273, + 0.5211734771728516, + 1.756280541419983, + 1.2685545682907104, + -0.2445927858352661, + -0.32804498076438904, + 0.06367179751396179, + 1.267756700515747, + 0.09229117631912231, + 0.22304201126098633, + -0.7972757816314697 + ], + [ + -0.3916054368019104, + 0.13326752185821533, + 1.1780461072921753, + 0.06310266256332397, + 1.5708105564117432, + 1.890318512916565, + 2.242936849594116, + -0.04418173432350159, + -0.3064570426940918, + 1.042787790298462, + 0.12111161649227142, + 0.6410789489746094, + 1.734998106956482, + 1.1888160705566406, + -0.15793560445308685, + -0.39206135272979736, + -0.12200386822223663, + 1.140578269958496, + 0.08651520311832428, + 0.050670623779296875, + -0.5860601663589478 + ], + [ + -0.5884166359901428, + 0.06495284289121628, + 0.22037458419799805, + -0.3189915716648102, + 0.9872580766677856, + 1.8330312967300415, + 1.990811824798584, + 0.1546839475631714, + -0.24982738494873047, + 1.0657873153686523, + 0.23279926180839539, + 0.9398138523101807, + 1.384837031364441, + 1.351023554801941, + 0.02624328061938286, + -0.3241584300994873, + -0.5798515677452087, + 1.010702133178711, + 0.10710541903972626, + 0.38128888607025146, + -0.37607601284980774 + ], + [ + -0.749691903591156, + -0.2506883442401886, + 0.6185764074325562, + -0.19299718737602234, + 1.258925199508667, + 2.0286459922790527, + 1.6187307834625244, + -0.08473244309425354, + -0.4074743092060089, + 0.8556436896324158, + 0.5780456066131592, + 0.38366544246673584, + 1.8112229108810425, + 2.0794122219085693, + -0.5160185098648071, + -0.6219602227210999, + -0.17286916077136993, + 1.4655016660690308, + -0.12483066320419312, + 0.32146573066711426, + -0.16379857063293457 + ] + ], + "generated_utilities": [ + 0.012376229278743267, + 0.006582856178283691, + 0.005475322250276804, + 0.015102227218449116, + 0.007560948375612497, + 0.010635594837367535, + 0.0225119199603796, + 0.003913839813321829 + ], + "group_id": "ms-c96b415f083f60857334", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15946325659751892, + 0.16313008964061737, + 0.2226802557706833, + 0.18744397163391113, + 0.21545825898647308, + 0.15985511243343353, + 0.16218391060829163, + 0.26426148414611816, + 0.184181809425354, + 0.24352896213531494, + 0.14179718494415283, + 0.1026589497923851, + 0.1274908483028412, + 0.05275428295135498, + 0.08544860780239105, + 0.07143914699554443 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.16191565990448, + 0.4561867117881775, + 1.1241415739059448, + 0.08947351574897766, + 0.32140541076660156, + 3.6895532608032227, + 3.6284518241882324, + -1.8666775226593018, + -0.22328677773475647, + 1.48662531375885, + 1.4737720489501953, + 1.35750150680542, + 2.507124423980713, + 1.9315069913864136, + -0.8371121883392334, + 0.5339290499687195, + 1.0061794519424438, + 2.149812698364258, + 0.08911214768886566, + 0.17410686612129211, + 0.3549031913280487 + ], + [ + -0.14804881811141968, + -0.5333454608917236, + 0.7481307983398438, + 0.22411420941352844, + 1.7204210758209229, + 3.908766746520996, + 3.3698034286499023, + -1.6747719049453735, + 0.2769082486629486, + 2.042428970336914, + 0.44748249650001526, + 1.9492043256759644, + 2.918121576309204, + 3.2413060665130615, + -0.6994609832763672, + -0.3854917883872986, + -0.6702244281768799, + 2.408869504928589, + -0.5672509670257568, + 0.5941196084022522, + 0.46011871099472046 + ], + [ + -0.46204906702041626, + -0.4236203730106354, + 1.8582539558410645, + 0.8416532278060913, + 1.2386386394500732, + 3.7320151329040527, + 2.242549180984497, + -1.8652616739273071, + -1.0259326696395874, + 1.3383660316467285, + 1.7068204879760742, + 0.7941814661026001, + 3.3389508724212646, + 2.7797679901123047, + -0.9363516569137573, + 0.17812880873680115, + 0.6111975908279419, + 2.066016435623169, + -0.28941091895103455, + 1.501695990562439, + -0.35273995995521545 + ], + [ + -0.7471081614494324, + -0.029650092124938965, + 1.019482135772705, + -0.30564722418785095, + 1.380143642425537, + 3.247985601425171, + 3.0548973083496094, + -0.5582134127616882, + -0.920264482498169, + 2.3859832286834717, + 1.1080355644226074, + 1.5716153383255005, + 3.089156150817871, + 2.043484687805176, + 0.1913408637046814, + -0.7557333707809448, + -0.6297664642333984, + 1.7829738855361938, + -0.6967166662216187, + -0.10757219791412354, + 0.229112908244133 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027880847454071045, + -0.2692106068134308, + 0.12925803661346436, + -0.1745016872882843, + -0.1231425404548645, + 1.9967715740203857, + 0.0, + -0.1433020830154419, + 0.2667914927005768, + 0.8867976665496826, + -0.057856738567352295, + 0.5789351463317871, + 1.7831475734710693, + 0.0, + -0.35565105080604553, + 0.293251097202301, + 0.012226536870002747, + 1.121243953704834, + -0.15060651302337646, + 0.03409510850906372, + 0.0 + ], + [ + -0.7738120555877686, + 0.08056297898292542, + 1.1292580366134644, + 0.4734484851360321, + 0.8768574595451355, + 2.9967715740203857, + 3.0509116649627686, + -1.143302083015442, + -0.1310054361820221, + 1.8867976665496826, + 0.9421432614326477, + 1.578935146331787, + 2.7831475734710693, + 2.6037895679473877, + -0.5597631931304932, + 0.11406822502613068, + 0.2593899667263031, + 2.121243953704834, + -0.1791832000017166, + 0.5773167610168457, + 0.41381677985191345 + ], + [ + -0.6024693250656128, + 0.29007619619369507, + 1.3939473628997803, + 0.3474765419960022, + 1.1848478317260742, + 3.127882719039917, + 3.3329083919525146, + -1.2855982780456543, + 0.20972535014152527, + 1.8329110145568848, + 1.267270803451538, + 1.709198236465454, + 2.927372455596924, + 2.9659509658813477, + -0.7555317282676697, + 0.015305742621421814, + 0.6009905338287354, + 2.188507556915283, + 0.07961152493953705, + 0.4547080397605896, + 0.20378917455673218 + ], + [ + -1.2702982425689697, + 0.06162632256746292, + 0.9447290897369385, + 0.837199330329895, + 0.5848468542098999, + 3.4801836013793945, + 2.8138537406921387, + -1.5083554983139038, + -0.45768383145332336, + 2.08561110496521, + 1.4951050281524658, + 1.8484808206558228, + 2.5617129802703857, + 2.942486047744751, + -0.562404215335846, + 0.17441512644290924, + 0.7880480289459229, + 1.8807158470153809, + -0.043379828333854675, + 0.39090824127197266, + 0.43096962571144104 + ] + ], + "predicted_scores": [ + 9.991924285888672, + 9.819470405578613, + 9.650274276733398, + 9.505208969116211, + 9.463309288024902, + 9.436361312866211, + 7.990633487701416, + 7.58080530166626 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "bf24d506cf320effed2cd7faf94d8618b0626bf8d5c3abc1f837513597c18231", + "stored_base_utility": 0.15946325659751892, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05853639170527458, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05853639170527458 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06729111820459366, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06729111820459366 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.05853639170527458, + 1.0, + 1.0, + 0.06729111820459366, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ebacae0135af2832dfbb", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683", + "ms-3d090ac2459c4b891dd1", + "ms-003a2225c0892dcda683", + "ms-ebacae0135af2832dfbb", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "generated_tangents": [ + [ + 0.19821582734584808, + 0.11862018704414368, + 0.2940768003463745, + -0.2675343453884125, + 1.5529329776763916, + 0.03267388045787811, + -0.3338521122932434, + 0.20622263848781586, + -0.16410942375659943, + -0.0012958496809005737, + 0.2084309458732605, + 0.06816083192825317, + 0.009602665901184082, + -0.1524500995874405, + -0.5379070043563843, + -0.0378229059278965, + 0.7765757441520691, + 0.12415192276239395, + -0.17680013179779053, + 1.699528694152832, + -0.1770472228527069 + ], + [ + 0.030832655727863312, + 0.0030104704201221466, + 0.40686947107315063, + -0.09046083688735962, + 1.7328708171844482, + 0.3114061951637268, + -0.18760895729064941, + 0.008012473583221436, + -0.06498051434755325, + 0.23737448453903198, + 0.14674559235572815, + 0.2794705033302307, + 0.2682911455631256, + -0.07735355198383331, + -0.6674419641494751, + -0.00031372811645269394, + 0.8229855895042419, + 0.08752962201833725, + -0.184705913066864, + 1.746021032333374, + 0.003982633352279663 + ], + [ + 0.11717712879180908, + 0.018371693789958954, + 0.31803399324417114, + -0.17400242388248444, + 1.637786865234375, + 0.34610849618911743, + -0.2596966028213501, + -0.007424697279930115, + -0.16548901796340942, + 0.20763692259788513, + 0.25298720598220825, + 0.33889806270599365, + 0.35871079564094543, + 0.051082104444503784, + -0.6170861124992371, + -0.022225230932235718, + 0.9731844067573547, + -0.06910558044910431, + -0.19000715017318726, + 1.9738883972167969, + 0.01560114324092865 + ], + [ + -0.011447951197624207, + 0.018537312746047974, + 0.40751609206199646, + -0.21178218722343445, + 1.707357406616211, + 0.25973063707351685, + 0.005991578102111816, + 0.0486777126789093, + -0.12137611210346222, + 0.22774119675159454, + 0.11506661772727966, + 0.22778189182281494, + 0.2834572494029999, + -0.07247871160507202, + -0.5803357362747192, + -0.030795760452747345, + 0.7830451726913452, + -0.07372553646564484, + -0.2007940709590912, + 1.7946207523345947, + 0.06326296180486679 + ], + [ + 0.06426630169153214, + -0.01832268014550209, + 0.32810738682746887, + -0.19499707221984863, + 1.594602108001709, + 0.31487834453582764, + -0.17571765184402466, + 0.09489920735359192, + -0.11910350620746613, + 0.20192202925682068, + 0.24325668811798096, + 0.3401080071926117, + 0.34818747639656067, + -0.018125757575035095, + -0.6066502928733826, + 0.04364753141999245, + 0.9730539321899414, + -0.05469249188899994, + -0.10362094640731812, + 2.006929636001587, + 0.05612626671791077 + ], + [ + -0.039808452129364014, + -0.030687691643834114, + 0.3820580244064331, + -0.102969229221344, + 1.6634316444396973, + 0.43563738465309143, + -0.13620656728744507, + 0.06918269395828247, + -0.11970779299736023, + 0.2843729257583618, + 0.15523378551006317, + 0.2955305278301239, + 0.30110087990760803, + -0.13235266506671906, + -0.690915048122406, + 0.03766452521085739, + 0.9249383211135864, + -0.03850726783275604, + -0.30874335765838623, + 1.7154533863067627, + 0.04161655902862549 + ], + [ + -0.08949635922908783, + 0.003914676606655121, + 0.4041188359260559, + -0.09749069064855576, + 1.6887085437774658, + 0.41461846232414246, + -0.20469671487808228, + -0.08192518353462219, + -0.13876117765903473, + 0.1715545803308487, + 0.14327865839004517, + 0.33035677671432495, + 0.4296446144580841, + -0.12914808094501495, + -0.6596945524215698, + 0.04474303126335144, + 0.8887852430343628, + 0.03434893488883972, + -0.23880088329315186, + 1.7352321147918701, + 0.08018404245376587 + ], + [ + -0.06557545065879822, + 0.1724398285150528, + 0.388532429933548, + -0.22448638081550598, + 1.5847713947296143, + 0.27204227447509766, + -0.37074482440948486, + 0.08504746854305267, + -0.22743508219718933, + 0.23558951914310455, + 0.22090449929237366, + 0.41367703676223755, + 0.24430879950523376, + 0.18120113015174866, + -0.6481141448020935, + -0.07451341301202774, + 0.8715323805809021, + -0.0502660870552063, + -0.13620072603225708, + 1.8175432682037354, + 0.1621093600988388 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.05853639170527458, + 2.0, + 2.0, + 0.06729111820459366, + 2.0, + 2.0 + ], + "group_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20931994915008545, + 2.0, + 2.0, + 0.2008315920829773, + 0.09476469457149506, + 0.05857282876968384, + 0.10009106248617172, + 0.07648500055074692, + 0.05889680236577988, + 0.014406224712729454, + 0.09262995421886444, + 0.15997587144374847, + 0.04682888835668564, + 0.09751574695110321, + 0.06459556519985199, + 0.20335909724235535 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7537296414375305, + 0.4158197343349457, + 1.4326180219650269, + -1.4107877016067505, + 2.9579341411590576, + 0.9669660925865173, + 1.4339183568954468, + 0.5987398624420166, + -0.2018759399652481, + 0.7105501890182495, + -0.03802938759326935, + 1.3799947500228882, + -0.6568997502326965, + -0.37473973631858826, + -1.857657790184021, + 0.038770925253629684, + 1.931185007095337, + 0.6317195296287537, + 0.1380508691072464, + 2.954265832901001, + 2.9641003608703613 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8678367137908936, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.7494794726371765, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.928200721740723 + ], + [ + 0.6226351857185364, + 0.34334811568260193, + 1.492361068725586, + -1.5917733907699585, + 3.1334922313690186, + 1.0313001871109009, + 1.2075668573379517, + 0.3046378493309021, + -0.11550238728523254, + 0.4499495327472687, + 0.16646838188171387, + 1.5877203941345215, + -0.6488803625106812, + -0.5760968327522278, + -1.8349132537841797, + 0.14188142120838165, + 2.0770444869995117, + 0.5806879997253418, + 0.23913151025772095, + 2.8127381801605225, + 3.0649962425231934 + ], + [ + 0.5975998044013977, + 0.722756028175354, + 1.7548110485076904, + -1.166775107383728, + 2.6060047149658203, + 0.6675758361816406, + 1.125187635421753, + 0.3373170793056488, + -0.1842741072177887, + 0.8186168670654297, + -0.31481224298477173, + 1.2082113027572632, + -0.9804694652557373, + -0.14131735265254974, + -1.700500249862671, + 0.07075637578964233, + 1.5488730669021606, + 0.549576461315155, + 0.034750498831272125, + 3.227431297302246, + 2.9464566707611084 + ], + [ + 0.844982922077179, + 0.7132244110107422, + 1.4430484771728516, + -1.3331272602081299, + 3.1703803539276123, + 1.1851513385772705, + 1.355688452720642, + 0.6115275025367737, + 0.2616361379623413, + 1.2030918598175049, + 0.05625858157873154, + 1.6050300598144531, + -0.7155078053474426, + -0.021488159894943237, + -2.073806047439575, + 0.5387547016143799, + 1.6223118305206299, + 0.7140727043151855, + -0.06027664244174957, + 2.7503156661987305, + 2.6344075202941895 + ], + [ + 0.5821706056594849, + 0.78206866979599, + 1.8119354248046875, + -0.8252442479133606, + 3.038909435272217, + 1.3900940418243408, + 1.7462186813354492, + 0.005083680152893066, + -0.27262064814567566, + 0.867202877998352, + -0.5555376410484314, + 1.172202706336975, + -1.2016760110855103, + -0.06355220079421997, + -2.2925045490264893, + 0.6078628301620483, + 1.7895314693450928, + 0.8720078468322754, + 0.472573459148407, + 3.377012014389038, + 2.6039178371429443 + ], + [ + 0.3789674639701843, + 0.3270051181316376, + 1.056584358215332, + -2.0788497924804688, + 2.948819875717163, + 1.6022355556488037, + 0.8599810004234314, + 0.9570736885070801, + -0.6380544900894165, + 0.3103100061416626, + -0.42896610498428345, + 1.7150276899337769, + -0.8975871205329895, + -0.278381884098053, + -1.305256724357605, + 0.3637259304523468, + 1.8004443645477295, + 0.16446983814239502, + 0.19097819924354553, + 3.43778657913208, + 3.538135528564453 + ], + [ + 0.6903814673423767, + 1.1327197551727295, + 1.8523926734924316, + -2.0428731441497803, + 3.1870431900024414, + 0.5352929830551147, + 0.6004904508590698, + 1.2120530605316162, + 0.16280107200145721, + 1.119707465171814, + 0.7612010836601257, + 0.7890661954879761, + -1.3447637557983398, + -0.1703537553548813, + -1.1010353565216064, + 0.32831913232803345, + 1.5534476041793823, + -0.2119135856628418, + -0.562728762626648, + 2.4702467918395996, + 2.5528204441070557 + ], + [ + 0.9177048206329346, + -0.04751536250114441, + 1.4819984436035156, + -2.1542677879333496, + 3.522390842437744, + 1.8177759647369385, + 2.2213237285614014, + 0.8223017454147339, + -0.1973809152841568, + -0.053272128105163574, + -0.5862784385681152, + 0.6421516537666321, + -0.8713911771774292, + -0.9495017528533936, + -1.6454768180847168, + -0.38891172409057617, + 1.648432970046997, + 1.4951133728027344, + 0.5893608331680298, + 2.449934482574463, + 3.419659376144409 + ], + [ + 1.2956775426864624, + -0.3119873106479645, + 1.5930066108703613, + -1.7682520151138306, + 2.4385876655578613, + 1.4679458141326904, + 1.4536705017089844, + 0.31068816781044006, + 0.660710871219635, + 0.8104690909385681, + 0.483506441116333, + 2.1112046241760254, + -0.22927385568618774, + 0.17858651280403137, + -2.485126495361328, + -0.9187790155410767, + 1.2810213565826416, + 1.3727718591690063, + 0.44044917821884155, + 3.4018898010253906, + 3.39054799079895 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0012856721878051758, + 0.00795695185661316, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + 0.11055737733840942, + 0.0, + 0.02143383026123047, + 0.0025591105222702026, + -0.030314385890960693, + -0.0125969760119915, + 0.3799947500228882, + 0.0076233744621276855, + 0.0, + -0.857657790184021, + 0.013764806091785431, + 0.9311850070953369, + 0.07110422849655151, + 0.03423507511615753, + 1.954265832901001, + 0.0 + ], + [ + -0.24627035856246948, + -0.16095152497291565, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + -0.023127973079681396, + 0.0, + -0.24312204122543335, + 0.1389055997133255, + 0.11457735300064087, + 0.06618259102106094, + 0.3799947500228882, + 0.07616078853607178, + 0.0, + -0.857657790184021, + 0.09948430210351944, + 0.9311850070953369, + 0.006249547004699707, + -0.33439844846725464, + 1.954265832901001, + 0.0 + ] + ], + "predicted_scores": [ + 8.333680152893066, + 8.00536060333252, + 7.794225215911865, + 7.692945957183838, + 7.54518985748291, + 7.450100898742676, + 7.431529998779297, + 7.38453483581543 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "534717141", + "selected_index": 0, + "state_hash": "5318995ba859471b8678d6594274c7c0565a1ccfd83ea302378cc2884dc5cc32", + "stored_base_utility": 0.20931994915008545, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14809735119342804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14809735119342804 + }, + "base_progress": 0.14809735119342804, + "base_success": false, + "base_utility": 0.14809735119342804, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0012347499141469598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0012347499141469598 + }, + { + "contact_quality": null, + "progress": 0.0006952881813049316, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006952881813049316 + }, + { + "contact_quality": null, + "progress": 0.0009832581272348762, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009832581272348762 + }, + { + "contact_quality": null, + "progress": 0.001222034334205091, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001222034334205091 + }, + { + "contact_quality": null, + "progress": 0.001184225082397461, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001184225082397461 + }, + { + "contact_quality": null, + "progress": 0.0017026265850290656, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0017026265850290656 + }, + { + "contact_quality": null, + "progress": 0.0010568499565124512, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0010568499565124512 + }, + { + "contact_quality": null, + "progress": 0.0020139615517109632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0020139615517109632 + } + ], + "candidate_progress": [ + 0.0012347499141469598, + 0.0006952881813049316, + 0.0009832581272348762, + 0.001222034334205091, + 0.001184225082397461, + 0.0017026265850290656, + 0.0010568499565124512, + 0.0020139615517109632 + ], + "candidate_source_chart_ids": [ + "ms-f2b2b2a1aa2863c3676f", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-da142025733b53fcf7eb", + "generated_tangents": [ + [ + -0.14792807400226593, + -0.12037685513496399, + 1.2677156925201416, + 0.22903186082839966, + 0.7840882539749146, + 2.7241084575653076, + 2.1215412616729736, + -0.26216986775398254, + -0.20857413113117218, + 1.357494592666626, + 0.36197763681411743, + 0.7453258633613586, + 1.7902954816818237, + 2.072998046875, + -1.1923516988754272, + -0.32508280873298645, + 1.9591964483261108, + 0.8809031248092651, + 0.35911834239959717, + 2.363125801086426, + 1.8979345560073853 + ], + [ + -0.008208125829696655, + -0.15411368012428284, + 1.5386511087417603, + -0.11599639058113098, + 0.3467293083667755, + 2.42981219291687, + 2.20508074760437, + -0.571887731552124, + -0.09749794006347656, + 1.5345336198806763, + 0.8130164742469788, + 0.654485285282135, + 2.4730300903320312, + 1.6559977531433105, + -1.1437333822250366, + -0.29220739006996155, + 2.4106664657592773, + 0.5348500609397888, + 0.42562544345855713, + 1.9994966983795166, + 1.5532678365707397 + ], + [ + -0.12392519414424896, + -0.10915147513151169, + 1.6301875114440918, + -0.11622365564107895, + 0.41216981410980225, + 2.134881019592285, + 1.5720794200897217, + -0.2339523434638977, + -0.1566099375486374, + 1.3242512941360474, + 0.5845993161201477, + 0.8133550882339478, + 1.8213011026382446, + 1.6081379652023315, + -1.277977705001831, + -0.3984118402004242, + 2.031397581100464, + 0.8786622285842896, + 0.3703434467315674, + 2.047316312789917, + 1.8929274082183838 + ], + [ + -0.25291842222213745, + -0.001621447503566742, + 1.231136679649353, + -0.13570603728294373, + 0.867662250995636, + 2.5823974609375, + 1.6352968215942383, + -0.27264800667762756, + -0.07834309339523315, + 1.3511162996292114, + 0.5456430315971375, + 0.8499860167503357, + 1.3987655639648438, + 1.8022953271865845, + -1.127785086631775, + -0.399728924036026, + 2.1605467796325684, + 0.685965359210968, + 0.6784347295761108, + 2.2681829929351807, + 1.9904145002365112 + ], + [ + -0.047950804233551025, + -0.09295839071273804, + 1.4757448434829712, + -0.2742197513580322, + 1.0798732042312622, + 2.7047958374023438, + 2.0619595050811768, + -0.4945589601993561, + -0.03208528459072113, + 1.337076187133789, + 0.49107930064201355, + 0.7432037591934204, + 1.6977874040603638, + 1.8670111894607544, + -1.0429173707962036, + -0.26384344696998596, + 1.962184190750122, + 0.5686109066009521, + 0.6349022388458252, + 2.673661470413208, + 2.3525197505950928 + ], + [ + -0.007411673665046692, + -0.08534751832485199, + 1.3961024284362793, + -0.2842656970024109, + 0.445515900850296, + 2.27243971824646, + 1.8400790691375732, + -0.19332990050315857, + 0.022110791876912117, + 1.1534346342086792, + 0.37642133235931396, + 1.1431910991668701, + 1.9179803133010864, + 2.2775497436523438, + -1.1335296630859375, + -0.35102662444114685, + 2.029925584793091, + 0.6594119668006897, + 0.5136226415634155, + 2.668724536895752, + 2.2608978748321533 + ], + [ + -0.11100061237812042, + -0.1824159175157547, + 1.3723037242889404, + -0.2519010305404663, + 0.6622046232223511, + 2.399218797683716, + 1.8872203826904297, + -0.22966846823692322, + -0.07654914259910583, + 1.373837947845459, + 0.5841163396835327, + 0.9163534045219421, + 1.6412230730056763, + 1.7161020040512085, + -1.2128006219863892, + -0.34733760356903076, + 2.2784645557403564, + 0.7162875533103943, + 0.46822768449783325, + 2.420989513397217, + 2.2971346378326416 + ], + [ + -0.3248453736305237, + -0.3080900311470032, + 1.159929871559143, + 0.49731698632240295, + 0.577373743057251, + 2.29689884185791, + 1.7428398132324219, + -0.19394347071647644, + -0.18259301781654358, + 1.1071598529815674, + 0.11351655423641205, + 0.7244806885719299, + 1.2099586725234985, + 2.5740606784820557, + -0.7791820168495178, + -0.2937033474445343, + 1.731724739074707, + 0.6920046806335449, + 0.42418861389160156, + 2.5110080242156982, + 1.9522656202316284 + ] + ], + "generated_utilities": [ + 0.0012347499141469598, + 0.0006952881813049316, + 0.0009832581272348762, + 0.001222034334205091, + 0.001184225082397461, + 0.0017026265850290656, + 0.0010568499565124512, + 0.0020139615517109632 + ], + "group_id": "ms-da142025733b53fcf7eb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.14843207597732544, + 0.1717694252729416, + 0.09356185048818588, + 0.23231086134910583, + 0.19341367483139038, + 0.04822012037038803, + 0.23024770617485046, + 0.16561973094940186, + 0.2643269896507263, + 0.12403462827205658, + 0.12689702212810516, + 0.16619956493377686, + 0.13481859862804413, + 0.07537531852722168, + 0.11867459863424301, + 0.04740651696920395 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.04848463833332062, + -0.18164527416229248, + 0.6558789014816284, + 0.038211286067962646, + 0.08492374420166016, + 2.265157699584961, + 0.0, + -0.09011825919151306, + 0.0018594712018966675, + 0.6884076595306396, + 0.027596401050686836, + 0.06117302179336548, + 1.278116226196289, + 0.0, + -0.3951960802078247, + -0.08061015605926514, + 1.07906174659729, + 0.3229694366455078, + -0.21076995134353638, + 1.8564467430114746, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0903544425964355, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.570276260375977, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.736253261566162 + ], + [ + 0.08095964789390564, + -0.7926032543182373, + 1.571103811264038, + 0.07894467562437057, + 0.2701656222343445, + 3.806621789932251, + 2.203669786453247, + -0.042464032769203186, + 0.7168105244636536, + 1.8671371936798096, + -0.30838707089424133, + 1.7242767810821533, + 1.8847527503967285, + 1.305934190750122, + -0.42665380239486694, + -0.2973874807357788, + 1.1881210803985596, + 1.6921244859695435, + 1.070060133934021, + 2.067784309387207, + 1.916501760482788 + ], + [ + 0.7620636224746704, + -2.1605899333953857, + 2.1443262100219727, + 0.17886832356452942, + 1.6259945631027222, + 3.787412166595459, + 1.3663169145584106, + 0.49799659848213196, + -0.6301277279853821, + 2.275850534439087, + -0.6735645532608032, + 1.7504758834838867, + 2.7512195110321045, + 1.7462939023971558, + -0.6835377216339111, + 0.36623528599739075, + 1.405979871749878, + 0.6472548842430115, + 1.4791045188903809, + 2.7838869094848633, + 3.0856707096099854 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.29792410135269165, + -0.30587875843048096, + 0.6558789014816284, + -0.4171115756034851, + -0.1733894944190979, + 2.265157699584961, + 0.0, + 0.14607194066047668, + 0.05651267617940903, + 0.6884076595306396, + -0.0433836504817009, + 0.07522618770599365, + 1.278116226196289, + 0.0, + -0.41462820768356323, + 0.26706579327583313, + 1.07906174659729, + 0.44956159591674805, + -0.37072819471359253, + 1.8564467430114746, + 0.0 + ], + [ + 0.18432100117206573, + -0.9473621249198914, + 1.5389783382415771, + -0.041053857654333115, + 0.6282865405082703, + 3.276425838470459, + 2.1839330196380615, + -0.3612370491027832, + 0.23081421852111816, + 1.8094429969787598, + 0.10581490397453308, + 0.8865737915039062, + 2.5146288871765137, + 2.2134878635406494, + -1.6751070022583008, + -0.08268362283706665, + 2.303063154220581, + 1.3061904907226562, + 0.4770132899284363, + 3.109205722808838, + 2.509927272796631 + ], + [ + -0.5288646817207336, + -1.5067847967147827, + 1.376499056816101, + 0.13231416046619415, + 1.048660159111023, + 3.5048489570617676, + 1.9678237438201904, + -0.14848726987838745, + 0.05475667119026184, + 1.5529061555862427, + 0.1597936749458313, + 0.7318928241729736, + 2.17749285697937, + 2.0782582759857178, + -1.5473355054855347, + -0.6715520620346069, + 1.7043644189834595, + 1.1190917491912842, + 0.4214729070663452, + 3.3400330543518066, + 2.3147284984588623 + ] + ], + "predicted_scores": [ + 9.4291353225708, + 9.417654037475586, + 8.99586009979248, + 8.808168411254883, + 8.785951614379883, + 8.686368942260742, + 8.476127624511719, + 7.9679741859436035 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "0026cc5b3762b520baadbda7bf138a2d24fffa5b2306f245ddbc1d1556cccc22", + "stored_base_utility": 0.14843207597732544, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1362423449754715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1362423449754715 + }, + "base_progress": 0.1362423449754715, + "base_success": false, + "base_utility": 0.1362423449754715, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03034065105021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03034065105021 + }, + { + "contact_quality": null, + "progress": 0.03361199423670769, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03361199423670769 + }, + { + "contact_quality": null, + "progress": 0.033086616545915604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.033086616545915604 + }, + { + "contact_quality": null, + "progress": 0.019350707530975342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019350707530975342 + }, + { + "contact_quality": null, + "progress": 0.0312865786254406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0312865786254406 + }, + { + "contact_quality": null, + "progress": 0.008349676616489887, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008349676616489887 + }, + { + "contact_quality": null, + "progress": 0.007631381507962942, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007631381507962942 + }, + { + "contact_quality": null, + "progress": 0.00866107176989317, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00866107176989317 + } + ], + "candidate_progress": [ + 0.03034065105021, + 0.03361199423670769, + 0.033086616545915604, + 0.019350707530975342, + 0.0312865786254406, + 0.008349676616489887, + 0.007631381507962942, + 0.00866107176989317 + ], + "candidate_source_chart_ids": [ + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-cf4b63d109a88a4c42bf", + "ms-7e88428497a6b13936c4", + "ms-76f27d9a935b25e15c38", + "ms-a2518826fa0a816cc46f", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-e2bad6e99e8a281207cc", + "generated_tangents": [ + [ + -0.16096971929073334, + -0.31177908182144165, + 1.854225516319275, + -0.2686333656311035, + 1.4799003601074219, + 2.5605580806732178, + 2.5912859439849854, + 1.0004174709320068, + -0.15160799026489258, + -0.3394080698490143, + 1.0288563966751099, + 1.3309565782546997, + 0.2021808922290802, + -0.01970010995864868, + 0.9773865938186646, + 0.24064843356609344, + -0.39815592765808105, + -0.9252974390983582, + 1.032639503479004, + -2.2695326805114746, + -1.7332607507705688 + ], + [ + -0.2856951951980591, + -0.32508283853530884, + 1.8239376544952393, + -0.4597397446632385, + 1.4449050426483154, + 2.6679069995880127, + 2.6012749671936035, + 0.9416310787200928, + -0.10728119313716888, + -0.2472185492515564, + 0.9183798432350159, + 1.1879003047943115, + 0.1981048882007599, + -0.10409484803676605, + 1.0273622274398804, + 0.24549944698810577, + -0.45119577646255493, + -0.8537749648094177, + 0.9723955988883972, + -2.472135543823242, + -1.6466033458709717 + ], + [ + -0.2856389880180359, + -0.3050289750099182, + 1.602755069732666, + -0.5033504366874695, + 1.32881498336792, + 2.7149498462677, + 2.5599634647369385, + 1.0255323648452759, + -0.12451672554016113, + -0.2931959927082062, + 0.8575699329376221, + 1.1229257583618164, + 0.0978078544139862, + -0.051552996039390564, + 1.0300428867340088, + 0.1655631810426712, + -0.6067094802856445, + -0.9393851161003113, + 0.9195767641067505, + -2.4180305004119873, + -1.631742238998413 + ], + [ + -1.2150877714157104, + -0.5264142155647278, + 2.4768450260162354, + -0.3646775186061859, + 1.4079499244689941, + 2.2131717205047607, + 2.17423152923584, + 1.2163509130477905, + -0.19565480947494507, + -0.2968308925628662, + 1.1425195932388306, + 1.0473694801330566, + -0.047198206186294556, + 0.05088570713996887, + 1.2641568183898926, + -0.25054869055747986, + -0.3730945587158203, + -1.5274989604949951, + 0.6246023178100586, + -3.6558520793914795, + -2.540768623352051 + ], + [ + -0.36524295806884766, + -0.2680980861186981, + 1.5086112022399902, + -0.3838243782520294, + 1.519239902496338, + 3.145357370376587, + 3.230030059814453, + 0.8950217962265015, + -0.11261885613203049, + -0.11335598677396774, + 0.5106876492500305, + 1.5056214332580566, + 0.21764037013053894, + 0.12477201968431473, + 1.3788115978240967, + 0.09976649284362793, + -0.7163516879081726, + -0.819511353969574, + 1.0259342193603516, + -2.596590518951416, + -1.5088063478469849 + ], + [ + -1.3984849452972412, + -0.4602620005607605, + 2.099067211151123, + -0.43182337284088135, + 2.5447330474853516, + 1.9244742393493652, + 2.7048635482788086, + 1.1009728908538818, + -0.5510379076004028, + 0.26409685611724854, + 1.122795820236206, + 0.7084417343139648, + -1.1655607223510742, + -0.22580210864543915, + 1.222090482711792, + -0.14433996379375458, + -0.9677732586860657, + -1.102054238319397, + 0.7394117712974548, + -2.6706671714782715, + -2.4400110244750977 + ], + [ + -0.7646927833557129, + -0.5394712090492249, + 2.0804500579833984, + -0.38889607787132263, + 2.2072486877441406, + 2.602515935897827, + 2.5818116664886475, + 1.3543554544448853, + -0.5414590835571289, + -0.05968841165304184, + 1.278780221939087, + 1.177679181098938, + -1.1409351825714111, + -0.7847484350204468, + 1.2067512273788452, + -0.13696825504302979, + -0.7112036943435669, + -0.5763885974884033, + 1.5668368339538574, + -2.2055201530456543, + -1.8194376230239868 + ], + [ + -1.420310616493225, + -0.4101560115814209, + 2.1167399883270264, + -0.42431551218032837, + 2.489821434020996, + 2.1338396072387695, + 2.5821707248687744, + 1.0836492776870728, + -0.551445484161377, + 0.18411391973495483, + 1.121557593345642, + 0.8325760364532471, + -1.1308687925338745, + -0.350838303565979, + 1.4645041227340698, + -0.18100634217262268, + -1.0054962635040283, + -1.1382744312286377, + 0.761302649974823, + -2.7134416103363037, + -2.494117498397827 + ] + ], + "generated_utilities": [ + 0.03034065105021, + 0.03361199423670769, + 0.033086616545915604, + 0.019350707530975342, + 0.0312865786254406, + 0.008349676616489887, + 0.007631381507962942, + 0.00866107176989317 + ], + "group_id": "ms-e2bad6e99e8a281207cc", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13993772864341736, + 0.16482847929000854, + 0.20825156569480896, + 0.16188287734985352, + 0.13454246520996094, + 0.14753298461437225, + 0.14504572749137878, + 0.06676968187093735, + 0.09772863239049911, + 0.0585557222366333, + 0.12863430380821228, + 0.10292482376098633, + 0.07993738353252411, + 0.03641533851623535, + 0.04985213279724121, + 0.1516878604888916 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.3249012231826782, + -0.9610954523086548, + 2.054325819015503, + 1.018629789352417, + 2.393625497817993, + 2.2159433364868164, + 2.098443031311035, + -0.49140021204948425, + 0.044472046196460724, + 0.8045110702514648, + 1.4511574506759644, + -0.3077920377254486, + -0.052587300539016724, + 0.036380887031555176, + 0.7875679731369019, + -0.3928021192550659, + -0.09456783533096313, + -1.4901847839355469, + 1.1711406707763672, + -2.1349995136260986, + -2.076193332672119 + ], + [ + -1.4271517992019653, + -1.114187479019165, + 1.618905782699585, + 1.0621737241744995, + 1.6005314588546753, + 3.017716407775879, + 2.3444156646728516, + -1.0368298292160034, + 0.6155713796615601, + 1.0049463510513306, + 1.3708372116088867, + -0.27396243810653687, + -0.19098177552223206, + -0.5735776424407959, + 0.863328218460083, + -0.4340485632419586, + 0.09502258896827698, + -1.8856427669525146, + 0.9514620304107666, + -2.3912711143493652, + -2.6571128368377686 + ], + [ + -1.0088777542114258, + -1.803802251815796, + 2.6037826538085938, + 0.763943612575531, + 1.7241847515106201, + 2.433204174041748, + 2.273603916168213, + 0.06359493732452393, + -0.17274422943592072, + -0.23299545049667358, + 1.7126038074493408, + -0.3797045052051544, + 0.7567765712738037, + -0.9981052875518799, + 1.0371832847595215, + -0.5120593309402466, + -0.059397101402282715, + -1.9757622480392456, + 0.7378945350646973, + -2.6195693016052246, + -2.178041458129883 + ], + [ + -0.09093856811523438, + -0.14735746383666992, + 1.350311517715454, + 1.477137565612793, + 2.134960174560547, + 1.7331886291503906, + 2.24615740776062, + -1.7148966789245605, + -0.3949584364891052, + 0.4433448612689972, + 0.9034961462020874, + -0.23445925116539001, + -0.6371403932571411, + -0.038229942321777344, + 2.010648250579834, + 0.14472149312496185, + -0.26303571462631226, + -2.3003270626068115, + 0.565727949142456, + -2.586710214614868, + -2.098479747772217 + ], + [ + -1.8506121635437012, + -1.4700038433074951, + 2.4313056468963623, + 1.6264371871948242, + 1.1548577547073364, + 2.9678268432617188, + 2.990251302719116, + -0.25485146045684814, + -0.36889538168907166, + -0.14847010374069214, + 0.6971698999404907, + -0.6752342581748962, + 0.44545215368270874, + 0.663374125957489, + 0.6998850107192993, + -0.6500043869018555, + -0.04876291751861572, + -2.446995258331299, + 0.7760334014892578, + -1.38918137550354, + -1.5372083187103271 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.053252458572387695, + -0.006605625152587891, + 1.0103552341461182, + 0.25736117362976074, + 1.010573148727417, + 1.4690883159637451, + 0.0, + -0.0840916633605957, + -0.24044206738471985, + -0.06801849603652954, + 0.061444878578186035, + 0.11697901785373688, + 0.03979811072349548, + 0.0, + 0.054485201835632324, + -0.11758939921855927, + 0.31161248683929443, + -0.602575421333313, + -0.021049559116363525, + -1.3257782459259033, + 0.0 + ] + ], + "predicted_scores": [ + 6.693980693817139, + 6.22277307510376, + 5.9340410232543945, + 5.360879898071289, + 5.2152886390686035, + 4.9879469871521, + 4.837738513946533, + 4.346799850463867 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "f054ce13f25dcda8808ebde2bee98ca84bfbdc3fc328a0eb831e21faada3fa4c", + "stored_base_utility": 0.13993772864341736, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09695577621459961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09695577621459961 + }, + "base_progress": 0.09695577621459961, + "base_success": false, + "base_utility": 0.09695577621459961, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03324349597096443, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03324349597096443 + }, + { + "contact_quality": null, + "progress": 0.06069042161107063, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06069042161107063 + }, + { + "contact_quality": null, + "progress": 0.05841739848256111, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05841739848256111 + }, + { + "contact_quality": null, + "progress": 0.05722795054316521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05722795054316521 + }, + { + "contact_quality": null, + "progress": 0.07222036272287369, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07222036272287369 + }, + { + "contact_quality": null, + "progress": 0.05981389805674553, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05981389805674553 + }, + { + "contact_quality": null, + "progress": 0.07067686319351196, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07067686319351196 + }, + { + "contact_quality": null, + "progress": 0.04897226765751839, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04897226765751839 + } + ], + "candidate_progress": [ + 0.03324349597096443, + 0.06069042161107063, + 0.05841739848256111, + 0.05722795054316521, + 0.07222036272287369, + 0.05981389805674553, + 0.07067686319351196, + 0.04897226765751839 + ], + "candidate_source_chart_ids": [ + "ms-6e27d292a9cbea108fc9", + "ms-206099bab6bfe7c84fce", + "ms-6e27d292a9cbea108fc9", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e30fd80adc089e9f697a", + "generated_tangents": [ + [ + 0.14059625566005707, + 0.28916049003601074, + -0.16963151097297668, + 1.314062237739563, + 0.7050162553787231, + -1.0451172590255737, + -1.3139665126800537, + -0.2140854001045227, + -0.011177374050021172, + -0.08635316044092178, + 0.8264796733856201, + 0.22506049275398254, + 0.16762247681617737, + -0.4966709017753601, + -0.38111889362335205, + 0.11698208749294281, + 0.43732184171676636, + -0.2739487290382385, + -0.5380127429962158, + 0.09203675389289856, + -0.018765494227409363 + ], + [ + 0.12499076873064041, + -0.05133494734764099, + -0.36809441447257996, + 1.2461128234863281, + 0.8098480105400085, + -0.6893341541290283, + -1.1495845317840576, + -0.07863315939903259, + -0.0462343692779541, + -0.10261634737253189, + 0.7539368271827698, + 0.17526593804359436, + 0.05268070101737976, + -0.49184125661849976, + -0.6165304183959961, + 0.31208768486976624, + 0.39507371187210083, + -0.14809341728687286, + -0.5398120880126953, + 0.4846477508544922, + 0.326396644115448 + ], + [ + 0.14577935636043549, + 0.1370626538991928, + -0.2611781656742096, + 0.999873161315918, + 0.6963958740234375, + -1.1857969760894775, + -1.25225830078125, + -0.125323086977005, + -0.03071071393787861, + 0.06095661222934723, + 0.7633594870567322, + 0.15987130999565125, + 0.07322022318840027, + -0.6642807722091675, + -0.5136773586273193, + 0.22974568605422974, + 0.375954806804657, + -0.2856103777885437, + -0.35792022943496704, + 0.3997807502746582, + 0.006541267037391663 + ], + [ + 0.17108535766601562, + -0.1494685709476471, + -0.061604052782058716, + 1.040161371231079, + 0.242130309343338, + -0.7885425090789795, + -1.5219964981079102, + 0.2713053226470947, + 0.18858802318572998, + 0.17530256509780884, + 1.0080199241638184, + 0.807894766330719, + -0.001270681619644165, + -0.5104037523269653, + -0.9724703431129456, + 0.2597637176513672, + 0.7744123339653015, + 0.22221793234348297, + -0.28697168827056885, + 0.4465206265449524, + -0.16993442177772522 + ], + [ + 0.13455934822559357, + -0.0938536748290062, + -0.007500439882278442, + 0.8863233327865601, + 0.19334590435028076, + -1.0735853910446167, + -1.4584605693817139, + 0.25584304332733154, + 0.12632793188095093, + 0.09572681039571762, + 1.2192953824996948, + 0.4742032289505005, + 0.02632009983062744, + -0.622262716293335, + -0.9880751967430115, + 0.2867356836795807, + 0.5601792931556702, + 0.06965525448322296, + -0.4905439019203186, + 0.6875616312026978, + -0.03526487946510315 + ], + [ + 0.24791967868804932, + -0.19294780492782593, + 0.014462172985076904, + 0.935484766960144, + 0.2310054898262024, + -0.996497392654419, + -1.5055625438690186, + 0.15239405632019043, + 0.21127238869667053, + 0.16762784123420715, + 0.9673843383789062, + 0.7749419212341309, + -0.05129283666610718, + -0.4622701406478882, + -0.9616836905479431, + 0.25189459323883057, + 0.6646955609321594, + 0.2064647078514099, + -0.24755346775054932, + 0.5304515957832336, + -0.11553612351417542 + ], + [ + 0.3456129729747772, + -0.24611318111419678, + 0.0590285062789917, + 0.9294030666351318, + 0.21586112678050995, + -0.8951681852340698, + -1.4217758178710938, + -0.010192766785621643, + 0.24937164783477783, + 0.1815493106842041, + 0.7579646110534668, + 0.8004189133644104, + 0.09421303868293762, + -0.26927292346954346, + -1.0741379261016846, + 0.19147054851055145, + 0.5296486616134644, + 0.02319309115409851, + -0.23959577083587646, + 0.5275599956512451, + 0.00764957070350647 + ], + [ + 0.32306885719299316, + -0.24005533754825592, + -0.03500804305076599, + 1.013843059539795, + 0.17810168862342834, + -1.1318376064300537, + -1.4078500270843506, + 0.17959968745708466, + 0.24683411419391632, + 0.1291821449995041, + 0.8833457231521606, + 0.9837989807128906, + 0.22038337588310242, + -0.5237548351287842, + -0.9737306237220764, + 0.28015780448913574, + 0.5724561214447021, + 0.2557889521121979, + -0.0892578661441803, + 0.626552164554596, + -0.23747605085372925 + ] + ], + "generated_utilities": [ + 0.03324349597096443, + 0.06069042161107063, + 0.05841739848256111, + 0.05722795054316521, + 0.07222036272287369, + 0.05981389805674553, + 0.07067686319351196, + 0.04897226765751839 + ], + "group_id": "ms-e30fd80adc089e9f697a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.12557005882263184, + 0.11941087245941162, + 0.10894306749105453, + 0.060310762375593185, + 0.18920521438121796, + 0.11678676307201385, + 0.14083251357078552, + 0.12988364696502686, + 0.18350017070770264, + 0.1286114901304245, + 0.06744732707738876, + 0.04428015276789665, + 0.08399941772222519, + 0.0742691159248352, + 0.05598282814025879, + 0.09703926742076874 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.044197678565979004, + -0.12298870831727982, + -0.37176433205604553, + 0.5503368377685547, + 0.5411782264709473, + 0.6186751127243042, + 0.0, + 0.28674131631851196, + 0.08053566515445709, + 0.3295105993747711, + 0.8157306909561157, + 0.1715114712715149, + -0.43090736865997314, + 0.0, + -0.5639103651046753, + -0.48903688788414, + 0.7485904693603516, + -0.4474331736564636, + -0.18787053227424622, + 1.2062294483184814, + 0.0 + ], + [ + 1.3153167963027954, + 0.7455203533172607, + -1.0113484859466553, + 2.097999334335327, + 0.7455009818077087, + 0.3843202590942383, + -0.16713692247867584, + -0.8400754928588867, + -0.3845864534378052, + 1.065396785736084, + 1.712794542312622, + 0.803345263004303, + 0.6529266238212585, + 0.6022698879241943, + -1.1222561597824097, + 0.06260442733764648, + 1.101211428642273, + -0.133493572473526, + -0.8626402616500854, + 1.8406728506088257, + 0.8011860847473145 + ], + [ + 0.23532289266586304, + -0.516789972782135, + -0.3119599223136902, + 1.501338005065918, + 1.4930639266967773, + -0.419913113117218, + -0.6910054683685303, + 0.30246037244796753, + 0.041916340589523315, + 1.2685117721557617, + 2.5912232398986816, + 1.2864888906478882, + 0.3821069598197937, + 0.4677737355232239, + -1.84663724899292, + -0.8953573703765869, + 2.3256053924560547, + 0.18948839604854584, + -0.5580936670303345, + 1.528401494026184, + 1.4914395809173584 + ], + [ + 0.6903802752494812, + 0.3962534964084625, + -0.8207939863204956, + 0.5973623991012573, + 1.310564398765564, + 0.6411572694778442, + -0.2032768577337265, + 0.25504088401794434, + 0.6581551432609558, + 0.3938753306865692, + 1.0127826929092407, + 1.6796927452087402, + 0.4867250621318817, + 0.05769670009613037, + -2.026149034500122, + -0.17722094058990479, + 0.9414948225021362, + 0.6044259667396545, + 0.03290170431137085, + 2.9423327445983887, + -0.060306549072265625 + ], + [ + 0.47530579566955566, + -0.6085019111633301, + 0.12691906094551086, + 0.5673632025718689, + 0.08076304197311401, + 0.8255819082260132, + -0.9781221151351929, + -0.796129584312439, + -0.6745100021362305, + 0.7784937024116516, + 1.7613102197647095, + 1.4703174829483032, + -0.36175650358200073, + 0.16939619183540344, + -0.9399533271789551, + -1.2696547508239746, + 1.3314080238342285, + -0.3690394163131714, + -1.0323668718338013, + 2.305772304534912, + 0.27392566204071045 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.6770331859588623, + 0.09077263623476028, + -0.3141186833381653, + 1.5503368377685547, + 1.0133107900619507, + 0.12400172650814056, + -0.2399108111858368, + -0.4914711117744446, + -0.11917559802532196, + 0.7716305255889893, + 1.8157306909561157, + 0.7167991995811462, + 0.5690926313400269, + 0.6154549717903137, + -1.5639103651046753, + -0.5166093111038208, + 1.7485904693603516, + -0.15158627927303314, + -0.46638885140419006, + 2.2062294483184814, + 0.7902159690856934 + ], + [ + 0.9456368684768677, + -0.3964601159095764, + -0.5845282077789307, + 1.2133667469024658, + 0.5843623876571655, + -0.02685314416885376, + -0.006928771734237671, + -0.2159697711467743, + -0.3017333149909973, + 0.40500110387802124, + 1.9734654426574707, + 0.46303731203079224, + 0.33380845189094543, + 0.7758987545967102, + -2.060479164123535, + -0.955610990524292, + 1.942940592765808, + -0.23636659979820251, + -0.10507094860076904, + 1.8367226123809814, + 1.2234702110290527 + ] + ], + "predicted_scores": [ + 5.362038612365723, + 4.596473217010498, + 4.504575252532959, + 4.01229190826416, + 3.903841018676758, + 3.577493667602539, + 3.082627534866333, + 2.9932708740234375 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "ca8873454d7cf9723b1e743355bf255813544ae3212d80abc954fd347d7adf27", + "stored_base_utility": 0.12557005882263184, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09445196390151978, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09445196390151978 + }, + { + "contact_quality": null, + "progress": 0.13109225034713745, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13109225034713745 + }, + { + "contact_quality": null, + "progress": 0.07720557600259781, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07720557600259781 + }, + { + "contact_quality": null, + "progress": 0.0657956600189209, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0657956600189209 + }, + { + "contact_quality": null, + "progress": 0.08409891277551651, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08409891277551651 + }, + { + "contact_quality": null, + "progress": 0.10617973655462265, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10617973655462265 + }, + { + "contact_quality": null, + "progress": 0.11467918008565903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11467918008565903 + }, + { + "contact_quality": null, + "progress": 0.07619664818048477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07619664818048477 + } + ], + "candidate_progress": [ + 0.09445196390151978, + 0.13109225034713745, + 0.07720557600259781, + 0.0657956600189209, + 0.08409891277551651, + 0.10617973655462265, + 0.11467918008565903, + 0.07619664818048477 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "generated_tangents": [ + [ + 1.8835853338241577, + -0.36436957120895386, + -0.36908021569252014, + 0.19512054324150085, + -0.06663826107978821, + 2.007988452911377, + 1.8292548656463623, + 2.070711374282837, + 0.27917805314064026, + -1.3850232362747192, + -0.705543577671051, + -0.9560496807098389, + -2.571524143218994, + -3.2129359245300293, + 1.9666616916656494, + -0.006010702811181545, + -1.6693131923675537, + -0.13773630559444427, + -0.44948458671569824, + -1.9241769313812256, + -1.8779592514038086 + ], + [ + 1.8852283954620361, + -0.376431941986084, + -0.11510768532752991, + 0.44733771681785583, + -0.21439245343208313, + 2.242299795150757, + 1.5470595359802246, + 2.1995959281921387, + 0.26516249775886536, + -1.354644536972046, + -0.7016180157661438, + -0.7497810125350952, + -2.5319321155548096, + -3.1401779651641846, + 1.7917345762252808, + 0.1281343400478363, + -1.514054298400879, + -0.0010046660900115967, + -0.3372483253479004, + -1.6821763515472412, + -1.5676567554473877 + ], + [ + 1.737519383430481, + -0.26957106590270996, + -0.2971702516078949, + 0.2566094398498535, + -0.15461108088493347, + 1.9364793300628662, + 1.6857423782348633, + 2.138697624206543, + 0.27545449137687683, + -1.2138079404830933, + -0.6317903995513916, + -0.8632838726043701, + -2.3770337104797363, + -3.1577882766723633, + 2.170635938644409, + 0.02896834909915924, + -1.6298086643218994, + 0.026990994811058044, + -0.24416643381118774, + -1.7977702617645264, + -1.639520287513733 + ], + [ + 1.613640546798706, + -0.29559779167175293, + -0.3531942665576935, + 0.19700193405151367, + 0.005303964018821716, + 2.067160129547119, + 1.7746155261993408, + 1.940363883972168, + 0.12679347395896912, + -1.5333608388900757, + -0.6188918352127075, + -0.9786030054092407, + -2.2097017765045166, + -3.2174675464630127, + 1.716732382774353, + -0.04426439106464386, + -1.3874294757843018, + -0.03497208654880524, + -0.45412445068359375, + -2.0235114097595215, + -2.2255663871765137 + ], + [ + 1.8246638774871826, + -0.31953758001327515, + -0.6649669408798218, + 0.42153432965278625, + 0.14494477212429047, + 1.8097628355026245, + 0.9051036834716797, + 2.0791258811950684, + 0.19699890911579132, + -1.3843746185302734, + -0.5807974338531494, + -0.6608120203018188, + -2.4870944023132324, + -2.3387491703033447, + 1.909354567527771, + -0.06960231810808182, + -1.6148953437805176, + -0.17316053807735443, + -0.06122326850891113, + -1.9254603385925293, + -0.9899587631225586 + ], + [ + 1.8433128595352173, + -0.33298271894454956, + -0.41172775626182556, + 0.10377022624015808, + -0.052587538957595825, + 2.0646111965179443, + 1.7742080688476562, + 2.2596335411071777, + 0.23512783646583557, + -1.2986104488372803, + -0.6844664812088013, + -0.8772629499435425, + -2.056953191757202, + -3.096824884414673, + 2.1217925548553467, + 0.004722908139228821, + -1.7100458145141602, + -0.0009026676416397095, + -0.29141080379486084, + -1.7385342121124268, + -1.6228255033493042 + ], + [ + 1.9682713747024536, + -0.3308408856391907, + -0.4873868525028229, + 0.002664342522621155, + 0.182377427816391, + 2.071789026260376, + 1.8561325073242188, + 2.1711413860321045, + 0.3807043731212616, + -1.516637921333313, + -0.4644528925418854, + -0.6258189082145691, + -2.9982683658599854, + -2.980710506439209, + 1.9127426147460938, + 0.03430766612291336, + -1.6087192296981812, + 0.046004101634025574, + -0.03771457076072693, + -1.7101502418518066, + -2.1152431964874268 + ], + [ + 1.793805480003357, + -0.188386932015419, + -0.12450191378593445, + 0.22051435708999634, + 0.3403468728065491, + 1.8198211193084717, + 0.9342815279960632, + 2.1495068073272705, + 0.3071441054344177, + -1.4927021265029907, + -0.364870548248291, + -0.696351170539856, + -2.0207245349884033, + -2.5684070587158203, + 2.123406171798706, + -0.06222609058022499, + -1.3718527555465698, + -0.1457296460866928, + 0.38743728399276733, + -1.2874568700790405, + -2.3363471031188965 + ] + ], + "generated_utilities": [ + 0.09445196390151978, + 0.13109225034713745, + 0.07720557600259781, + 0.0657956600189209, + 0.08409891277551651, + 0.10617973655462265, + 0.11467918008565903, + 0.07619664818048477 + ], + "group_id": "ms-e9ed5d488afdfd6449d1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.22621813416481018, + 0.26591962575912476, + 0.22478708624839783, + 0.074407659471035, + 0.3134628236293793, + 0.19173070788383484, + 0.2438776195049286, + 0.26689577102661133, + 0.24477621912956238, + 0.24051737785339355, + 0.2738616168498993, + 0.04947046563029289, + 2.0, + 0.04723116010427475, + 0.08354459702968597, + 0.06540697813034058 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.6524851322174072, + -0.023243099451065063, + -0.05794792249798775, + 0.1783304661512375, + -0.3767814338207245, + 1.4893875122070312, + 0.0, + 1.024657964706421, + 0.3163401186466217, + -0.49971193075180054, + 0.05114656686782837, + -0.13273626565933228, + -1.9796202182769775, + 0.0, + 0.818771481513977, + -0.13705189526081085, + -0.21289467811584473, + 0.3914462625980377, + -0.6477798819541931, + -1.0717217922210693, + 0.0 + ], + [ + 2.363477945327759, + -1.042197585105896, + -0.8369066119194031, + 0.17341986298561096, + 0.6040215492248535, + 1.9536304473876953, + 2.2788424491882324, + 1.5015904903411865, + -0.07916292548179626, + -0.6816228628158569, + -1.3893373012542725, + -0.2375274896621704, + -2.5662975311279297, + -2.7615673542022705, + 1.151218295097351, + 0.29418760538101196, + -0.4583754539489746, + -0.4555256962776184, + -0.6620294451713562, + -2.534846067428589, + -1.916935682296753 + ], + [ + 2.0104916095733643, + -0.539351224899292, + 0.13553066551685333, + 0.9507300853729248, + 0.6636125445365906, + 1.5649433135986328, + 2.0495975017547607, + 2.832026958465576, + 1.1830073595046997, + -1.6508865356445312, + -0.26773035526275635, + -0.3988863527774811, + -2.8235108852386475, + -2.2897520065307617, + 1.4448039531707764, + -0.5226297378540039, + -2.1615242958068848, + 0.8214395046234131, + -0.755327045917511, + -2.571093797683716, + -1.7457375526428223 + ], + [ + 2.166736602783203, + -0.3969564735889435, + 0.5171664357185364, + 0.12789201736450195, + -0.9176173806190491, + 1.8307013511657715, + 1.834255576133728, + 1.3594005107879639, + 0.13751044869422913, + -0.9923363327980042, + -1.136932611465454, + -1.2923126220703125, + -2.445897102355957, + -2.4779889583587646, + 2.6843042373657227, + 0.10015779733657837, + -2.0766196250915527, + 0.1269409954547882, + 0.16698843240737915, + -1.2709240913391113, + -2.7113101482391357 + ], + [ + 1.0476312637329102, + -1.2484432458877563, + -0.8652145862579346, + 0.8706620335578918, + 0.6091254353523254, + 1.9634168148040771, + 2.105522394180298, + 1.2906999588012695, + -0.3275601267814636, + -1.0110361576080322, + -1.6514918804168701, + -1.376278281211853, + -2.345550060272217, + -2.4931728839874268, + 1.00270676612854, + 0.04506785422563553, + -0.9420130252838135, + -0.6248484253883362, + -1.469360589981079, + -3.059006929397583, + -3.495709180831909 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.6524851322174072, + -0.3197403848171234, + -0.07008056342601776, + 0.38692954182624817, + -0.013657404109835625, + 2.4893875122070312, + 2.3340368270874023, + 2.024657964706421, + 0.35216206312179565, + -1.4125897884368896, + -0.8262880444526672, + -0.49936625361442566, + -2.9796202182769775, + -2.9234721660614014, + 1.818771481513977, + 0.09626653790473938, + -1.2128946781158447, + 0.10123217105865479, + -0.7020833492279053, + -2.0717217922210693, + -2.5944693088531494 + ], + [ + 0.9812780022621155, + 0.2612687647342682, + 0.8081930875778198, + 0.3689844310283661, + -0.16453352570533752, + 2.8411240577697754, + 2.2401933670043945, + 1.7904633283615112, + -0.04837304353713989, + -0.6139949560165405, + -0.8549875020980835, + 0.3574656546115875, + -2.8409454822540283, + -2.1011672019958496, + 1.382948875427246, + 0.6656897068023682, + -1.5090553760528564, + 0.4825518727302551, + -0.3345044255256653, + -2.26375150680542, + -2.561906576156616 + ] + ], + "predicted_scores": [ + 10.056475639343262, + 10.005084991455078, + 9.567598342895508, + 9.539158821105957, + 9.329384803771973, + 9.105973243713379, + 9.076784133911133, + 6.996925354003906 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "5608eea5b7e99b5fed6b316f742ecd34044df87286b2338705d8472043fc4375", + "stored_base_utility": 0.22621813416481018, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09514755010604858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09514755010604858 + }, + "base_progress": 0.09514755010604858, + "base_success": false, + "base_utility": 0.09514755010604858, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0540335588157177, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0540335588157177 + }, + { + "contact_quality": null, + "progress": 0.054324448108673096, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054324448108673096 + }, + { + "contact_quality": null, + "progress": 0.03629179671406746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03629179671406746 + }, + { + "contact_quality": null, + "progress": 0.05553414300084114, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05553414300084114 + }, + { + "contact_quality": null, + "progress": 0.05642610788345337, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05642610788345337 + }, + { + "contact_quality": null, + "progress": 0.02399025298655033, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02399025298655033 + }, + { + "contact_quality": null, + "progress": 0.08578010648488998, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08578010648488998 + }, + { + "contact_quality": null, + "progress": 0.07784923166036606, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07784923166036606 + } + ], + "candidate_progress": [ + 0.0540335588157177, + 0.054324448108673096, + 0.03629179671406746, + 0.05553414300084114, + 0.05642610788345337, + 0.02399025298655033, + 0.08578010648488998, + 0.07784923166036606 + ], + "candidate_source_chart_ids": [ + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-8c0266b35fc59963795c" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-ec013396e4aff6170ee1", + "generated_tangents": [ + [ + 0.07290751487016678, + 0.042255647480487823, + 0.1707490086555481, + 0.8329348564147949, + 0.44402647018432617, + -0.3983820378780365, + 0.10629785060882568, + -0.803307294845581, + 0.4631201922893524, + 1.6467640399932861, + -0.6992282271385193, + -0.4703778624534607, + 0.10239848494529724, + 0.43653029203414917, + -0.37150177359580994, + 0.6536993980407715, + 0.33850470185279846, + 0.36333948373794556, + -0.3031330704689026, + -1.2828834056854248, + -0.7913951873779297 + ], + [ + 0.03804558515548706, + -0.0011482909321784973, + 0.08555513620376587, + 0.824670672416687, + 0.472704142332077, + -0.2663976550102234, + 0.04659953713417053, + -0.7887725830078125, + 0.4096556603908539, + 1.7044203281402588, + -0.6283047199249268, + -0.45713889598846436, + 0.042866021394729614, + 0.6094751954078674, + -0.3879110813140869, + 0.6427829265594482, + 0.2444669008255005, + 0.48182326555252075, + -0.3695583939552307, + -1.1493947505950928, + -0.7599753141403198 + ], + [ + 0.1189035177230835, + 0.07704579830169678, + 0.19802147150039673, + 0.5909813642501831, + 0.32087916135787964, + -0.533021092414856, + -0.17082196474075317, + -0.7932863235473633, + 0.4692364037036896, + 1.8881500959396362, + -0.40812453627586365, + -0.1450972855091095, + -0.4662216901779175, + 0.2754039764404297, + -0.5360662341117859, + 0.6342980861663818, + 0.4219111204147339, + 0.6539356112480164, + -0.1759810447692871, + -1.173879861831665, + -0.9458827972412109 + ], + [ + -0.17840464413166046, + 0.0069093964993953705, + 0.010284781455993652, + 0.6991860866546631, + 0.7778811454772949, + -0.2496197521686554, + 0.17770043015480042, + -0.6878794431686401, + 0.43920040130615234, + 1.8311333656311035, + -0.6852989196777344, + -0.4559307098388672, + 0.09635785222053528, + 0.3610568642616272, + -0.2797762453556061, + 0.717698872089386, + 0.10359212756156921, + 0.29468992352485657, + -0.49036896228790283, + -1.2893831729888916, + -0.7048418521881104 + ], + [ + 0.026275962591171265, + 0.012835856527090073, + 0.0007028877735137939, + 0.6091057062149048, + 0.5208897590637207, + -0.22868439555168152, + 0.04974102973937988, + -0.7718795537948608, + 0.4654478132724762, + 1.7504278421401978, + -0.4801095426082611, + -0.44256842136383057, + 0.2232321798801422, + 0.30360251665115356, + -0.33298662304878235, + 0.6408830285072327, + 0.16710898280143738, + 0.42723631858825684, + -0.24493813514709473, + -1.30322265625, + -0.6780710220336914 + ], + [ + 0.023552551865577698, + -0.07760243117809296, + 0.7463976740837097, + 1.2157301902770996, + 0.1827164590358734, + -0.3384844660758972, + -0.036058902740478516, + -0.7126957178115845, + 0.3476243019104004, + 1.75017511844635, + -0.7299929857254028, + 0.0388372540473938, + -0.007054537534713745, + 1.4830682277679443, + -0.1733592450618744, + 0.5351212024688721, + 0.08154581487178802, + 0.39984363317489624, + -0.071033775806427, + -0.722948431968689, + -0.041042447090148926 + ], + [ + -0.12444590032100677, + 0.030472688376903534, + -0.06746813654899597, + 0.7309269905090332, + -0.10249662399291992, + 0.11368206143379211, + 0.2779630422592163, + -1.0385422706604004, + 0.4566822946071625, + 1.4930098056793213, + -0.2374260127544403, + 0.09677603840827942, + 0.3349209725856781, + 0.48507100343704224, + -0.8404542207717896, + 0.6343052387237549, + 0.17045344412326813, + 0.49400824308395386, + -0.5750795006752014, + -1.7462642192840576, + 0.06725037097930908 + ], + [ + -0.27741050720214844, + -0.14250724017620087, + -0.1027999222278595, + 0.7692158222198486, + -0.019599080085754395, + 0.22784535586833954, + 0.2978934049606323, + -0.9854612350463867, + 0.47614023089408875, + 1.4697589874267578, + -0.4946310222148895, + 0.36601242423057556, + 0.36599960923194885, + 0.4570406675338745, + -0.7772973775863647, + 0.4872051477432251, + 0.004006624221801758, + 0.7636597156524658, + -0.7082494497299194, + -1.9388642311096191, + 0.7660597562789917 + ] + ], + "generated_utilities": [ + 0.0540335588157177, + 0.054324448108673096, + 0.03629179671406746, + 0.05553414300084114, + 0.05642610788345337, + 0.02399025298655033, + 0.08578010648488998, + 0.07784923166036606 + ], + "group_id": "ms-ec013396e4aff6170ee1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.010581731796264648, + 0.049839019775390625, + 0.06402494758367538, + 0.012959301471710205, + 0.11614581197500229, + 0.012509982101619244, + 0.09607907384634018, + 0.1605721116065979, + 0.1708059310913086, + 0.055588722229003906, + 0.2052522599697113, + 0.0690404623746872, + 0.11059975624084473, + 0.03047112748026848, + 0.24502670764923096, + 0.22897732257843018 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.16510963439941406, + -0.011389851570129395, + -0.16074393689632416, + 0.8547754287719727, + -0.09704409539699554, + 0.09471678733825684, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + -0.29076793789863586, + 0.0122823566198349, + 0.059903502464294434, + 0.0, + 0.13894814252853394, + 0.2961620092391968, + 0.025543689727783203, + 0.01881265640258789, + 0.052319109439849854, + -1.1815040111541748, + 0.0 + ], + [ + -0.6906574964523315, + 0.5966935753822327, + 0.1873793751001358, + 1.8547754287719727, + 0.16492335498332977, + 0.4949548840522766, + 0.624785304069519, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233, + -0.44174641370773315, + 0.866999626159668, + 1.0255436897277832, + 1.018812656402588, + 0.6033775210380554, + -2.181504011154175, + -0.4328075349330902 + ], + [ + -0.68684321641922, + 0.35380983352661133, + 0.41641929745674133, + 2.0589241981506348, + 0.20610027015209198, + 0.2780776619911194, + 0.6000017523765564, + -1.8711607456207275, + 1.71079421043396, + 2.484286069869995, + -0.12494567036628723, + 0.3767848014831543, + -0.59480881690979, + 1.3215627670288086, + -0.18650269508361816, + 0.5971667766571045, + 1.00047767162323, + 1.2905888557434082, + 0.7682754993438721, + -2.280778408050537, + -0.26339399814605713 + ], + [ + -1.0669870376586914, + 0.5963312387466431, + 0.24800772964954376, + 1.5324974060058594, + 0.049914538860321045, + 0.33535104990005493, + 0.9774697422981262, + -2.318913698196411, + 1.14731764793396, + 2.1673643589019775, + -0.3369846045970917, + -0.13161931931972504, + -0.7527784705162048, + 1.2647147178649902, + -0.5229569673538208, + 0.6999921798706055, + 0.654599666595459, + 0.9745514988899231, + 0.5995839238166809, + -2.3844704627990723, + -0.7404599189758301 + ], + [ + -0.660759449005127, + 0.560627818107605, + -0.19398994743824005, + 1.7401045560836792, + 0.006708189845085144, + 0.2270365059375763, + 0.9201371073722839, + -1.6917380094528198, + 1.8382139205932617, + 2.184737205505371, + -0.07137502729892731, + 0.13004565238952637, + -0.8376798629760742, + 1.3597201108932495, + -0.8258764147758484, + 0.521450400352478, + 1.4799309968948364, + 0.7742871046066284, + 1.007819414138794, + -2.398472785949707, + -0.7120757102966309 + ], + [ + -0.3511212170124054, + 0.6072589755058289, + 0.6547060012817383, + 2.306784152984619, + 0.03337647020816803, + 1.1676279306411743, + 0.48420602083206177, + -1.7057392597198486, + 1.7822117805480957, + 2.741852283477783, + -0.970782995223999, + 0.3188708424568176, + -0.1370975375175476, + 1.740339994430542, + 0.18519049882888794, + 0.2771400809288025, + 0.8003557920455933, + 0.8813821077346802, + 0.9829010963439941, + -1.9104408025741577, + -0.7000454664230347 + ], + [ + -1.1815872192382812, + 1.0524637699127197, + 0.5610368251800537, + 2.407832622528076, + -0.2576238512992859, + 0.1467430293560028, + 0.27881303429603577, + -1.6810011863708496, + 1.5921515226364136, + 2.929525375366211, + 0.19559642672538757, + 0.6552202105522156, + -1.3889155387878418, + 1.1560360193252563, + -0.4104887843132019, + 0.8887493014335632, + 1.283969759941101, + 0.7811828851699829, + -0.10630232095718384, + -1.4329519271850586, + -0.5646982192993164 + ], + [ + -1.4926873445510864, + 0.11535564064979553, + -0.6240940690040588, + 1.631784200668335, + 0.7930782437324524, + -0.35493338108062744, + 0.20104485750198364, + -1.5626122951507568, + 1.3110854625701904, + 3.0686309337615967, + -0.8419785499572754, + -0.0010939836502075195, + -0.49860918521881104, + 1.5949748754501343, + -0.938766598701477, + 1.762406587600708, + 0.625543475151062, + 0.20789378881454468, + 1.1513795852661133, + -2.3980860710144043, + -0.36970049142837524 + ], + [ + -1.0271471738815308, + 0.6133835911750793, + -0.3870598077774048, + 1.9299802780151367, + 0.9606215357780457, + -0.19876420497894287, + 0.29684266448020935, + -1.6164112091064453, + 1.3925114870071411, + 1.9474635124206543, + -1.2669438123703003, + -0.6821421980857849, + -0.7869906425476074, + 1.3602250814437866, + -0.14778050780296326, + 0.9938437938690186, + 0.6618988513946533, + 0.1858283281326294, + 0.8312947750091553, + -1.7043554782867432, + -0.23048581182956696 + ], + [ + -0.409736305475235, + 1.5572996139526367, + 1.0024399757385254, + 1.6385810375213623, + 0.83731609582901, + -0.1629825234413147, + -0.05952054262161255, + -1.0968945026397705, + 0.728961706161499, + 3.1370975971221924, + -0.44591856002807617, + 1.149456262588501, + -1.3439809083938599, + 1.738822340965271, + 0.4063607454299927, + 0.4372418224811554, + 1.276847004890442, + 1.4612212181091309, + 0.6917726993560791, + -1.914740800857544, + 0.5415292978286743 + ] + ], + "predicted_scores": [ + 4.424034118652344, + 3.8042120933532715, + 3.8025412559509277, + 3.6810898780822754, + 3.4984779357910156, + 3.059438467025757, + 1.8871726989746094, + -0.2703617811203003 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296", + "selected_index": 0, + "state_hash": "f5a19cf07e6a6cc71c95ce01e976d34d1eeae68c8a93ecc985fd0a78cb0c1a5e", + "stored_base_utility": 0.010581731796264648, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01964247226715088, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01964247226715088 + }, + { + "contact_quality": null, + "progress": 0.015831351280212402, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015831351280212402 + }, + { + "contact_quality": null, + "progress": 0.020106056705117226, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020106056705117226 + }, + { + "contact_quality": null, + "progress": 0.01859968900680542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01859968900680542 + }, + { + "contact_quality": null, + "progress": 0.014677624218165874, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014677624218165874 + }, + { + "contact_quality": null, + "progress": 0.013308187015354633, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013308187015354633 + }, + { + "contact_quality": null, + "progress": 0.024462858214974403, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024462858214974403 + }, + { + "contact_quality": null, + "progress": 0.017958322539925575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017958322539925575 + } + ], + "candidate_progress": [ + 0.01964247226715088, + 0.015831351280212402, + 0.020106056705117226, + 0.01859968900680542, + 0.014677624218165874, + 0.013308187015354633, + 0.024462858214974403, + 0.017958322539925575 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-003a2225c0892dcda683", + "ms-0285f5815c210ac00bec", + "ms-0285f5815c210ac00bec", + "ms-be219977a9db8dbb1e58", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-ee30ebaf98622759d888", + "generated_tangents": [ + [ + 0.3891523480415344, + -0.5476866364479065, + 0.6235164999961853, + -0.20499247312545776, + 2.1759257316589355, + 1.303849220275879, + 1.1280488967895508, + 0.45156192779541016, + -0.09827959537506104, + 0.2792431116104126, + 0.09855307638645172, + 0.8370361328125, + -0.6810653805732727, + -0.23990671336650848, + 0.3469979465007782, + 0.31861793994903564, + 0.7256045937538147, + -0.5493374466896057, + 1.4329445362091064, + -1.4406800270080566, + -0.9786754846572876 + ], + [ + 0.37029337882995605, + -0.5844452977180481, + 0.6414795517921448, + -0.19069625437259674, + 2.104177951812744, + 1.303071141242981, + 1.1138598918914795, + 0.34920334815979004, + -0.10691405832767487, + 0.3129993677139282, + 0.07368563115596771, + 0.7885751128196716, + -0.6418795585632324, + -0.11100578308105469, + 0.35918915271759033, + 0.2706502676010132, + 0.6997917294502258, + -0.6293377876281738, + 1.474531650543213, + -1.5463147163391113, + -0.9765516519546509 + ], + [ + 0.4721396565437317, + -0.5913329720497131, + 0.8011988401412964, + -0.13740664720535278, + 2.0072481632232666, + 1.1481791734695435, + 0.8831367492675781, + 0.5639805793762207, + -0.036464571952819824, + 0.3808593451976776, + 0.015503205358982086, + 0.7085211277008057, + -1.0928772687911987, + -0.18293340504169464, + 0.3239908814430237, + 0.3095884323120117, + 0.6343111395835876, + -0.6279930472373962, + 1.5116267204284668, + -1.6963310241699219, + -1.1867793798446655 + ], + [ + 0.46050024032592773, + -0.5885443091392517, + 0.8316137194633484, + -0.1667107492685318, + 2.0395278930664062, + 1.1687372922897339, + 0.8199740648269653, + 0.5970370769500732, + -0.054191239178180695, + 0.44421935081481934, + -0.016800306737422943, + 0.6808668971061707, + -1.1212115287780762, + -0.24237792193889618, + 0.3289034068584442, + 0.3153565526008606, + 0.6198508739471436, + -0.5979838371276855, + 1.4770114421844482, + -1.6674690246582031, + -1.1689096689224243 + ], + [ + 0.7132302522659302, + -0.17021259665489197, + 0.49647775292396545, + -0.16039976477622986, + 1.8878815174102783, + 1.0434235334396362, + 0.8708289265632629, + 0.3052845001220703, + -0.09951447695493698, + 0.7682023048400879, + -0.0017570219933986664, + 0.37346988916397095, + -0.7110070586204529, + 0.2721971869468689, + 0.21601971983909607, + 0.2050550878047943, + 0.8422085642814636, + -0.7900719046592712, + 1.510375738143921, + -1.7179946899414062, + -0.6792943477630615 + ], + [ + 0.3379596769809723, + -0.6248862743377686, + 0.14679178595542908, + 0.23979803919792175, + 1.7175281047821045, + 1.069279432296753, + 0.9246268272399902, + 0.3281165063381195, + -0.1648385375738144, + 0.5740443468093872, + 0.18547499179840088, + 0.8467764854431152, + -0.5968696475028992, + 0.3580775260925293, + 0.23001572489738464, + 0.3481334149837494, + 0.6544145345687866, + -0.5044842958450317, + 1.3106539249420166, + -2.1478288173675537, + -0.2725996971130371 + ], + [ + 0.5897027850151062, + -0.43777740001678467, + 0.2121739387512207, + -0.2832771837711334, + 1.2150468826293945, + 0.8416855335235596, + 1.674008846282959, + 0.576818585395813, + -0.07556045800447464, + 0.7981460690498352, + -0.025754310190677643, + 0.9362323880195618, + -0.21349698305130005, + -0.30837517976760864, + 0.49233758449554443, + 0.3927343189716339, + 0.4829665422439575, + -1.0015685558319092, + 1.53029465675354, + -1.8476758003234863, + -0.7793848514556885 + ], + [ + 0.6190139055252075, + -0.6827624440193176, + 0.6732935905456543, + -0.034232839941978455, + 1.2853405475616455, + 0.5646194219589233, + 0.8531442880630493, + 0.18110041320323944, + -0.19192703068256378, + 0.3013923168182373, + 0.270673930644989, + 0.7177053689956665, + 0.03218114376068115, + 0.22037963569164276, + 0.6184743046760559, + 0.307494580745697, + 0.5671831965446472, + -1.006978154182434, + 1.4893624782562256, + -1.7233738899230957, + -0.5448800325393677 + ] + ], + "generated_utilities": [ + 0.01964247226715088, + 0.015831351280212402, + 0.020106056705117226, + 0.01859968900680542, + 0.014677624218165874, + 0.013308187015354633, + 0.024462858214974403, + 0.017958322539925575 + ], + "group_id": "ms-ee30ebaf98622759d888", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08405601978302002, + 0.09846439212560654, + 0.04595305770635605, + 2.0, + 0.08787640184164047, + 0.032481253147125244, + 0.08755072206258774, + 0.058936577290296555, + 0.10809408128261566, + 0.06723630428314209, + 0.0625009536743164, + 0.1415095031261444, + 0.08326160907745361, + 0.10944461822509766, + 0.0673408955335617, + 0.033725105226039886 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.4581542015075684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9406367540359497, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.470216989517212 + ], + [ + 1.6340367794036865, + -0.30366095900535583, + 0.2830277681350708, + -1.1627482175827026, + 3.58767032623291, + 0.6565636396408081, + 1.4321975708007812, + 0.8704822659492493, + 0.14327874779701233, + 1.0190284252166748, + 0.3078446090221405, + 0.7927982807159424, + -1.9305998086929321, + -1.7108646631240845, + 0.10729888081550598, + 0.9427294731140137, + 0.41770225763320923, + -1.7673753499984741, + 1.3209179639816284, + -2.8053033351898193, + -1.5330681800842285 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.00628507137298584, + 0.0595628023147583, + 0.39481037855148315, + -0.006641983985900879, + 1.8433213233947754, + 0.14830267429351807, + 0.0, + -0.24425631761550903, + 0.42130333185195923, + 0.41399094462394714, + 0.478841632604599, + -0.007461488246917725, + -0.5437495708465576, + 0.0, + 0.255215585231781, + -0.07331699132919312, + -0.1736500859260559, + -0.285358190536499, + 0.567956805229187, + -1.7103407382965088, + 0.0 + ], + [ + 1.0371580123901367, + -0.33467406034469604, + 0.9327473640441895, + -0.5396467447280884, + 2.1487879753112793, + 0.6294846534729004, + 0.6252679228782654, + 0.5140941143035889, + -0.20645928382873535, + -0.04528486728668213, + 0.14780870079994202, + 0.430325984954834, + -0.9293508529663086, + -1.5147639513015747, + 1.1104007959365845, + -0.05476301908493042, + 0.1755150556564331, + -1.3470268249511719, + 1.675830364227295, + -3.2944674491882324, + -1.9277915954589844 + ] + ], + "predicted_scores": [ + 8.547076225280762, + 8.304204940795898, + 7.853271484375, + 7.627488136291504, + 7.04011344909668, + 6.585521221160889, + 6.538170337677002, + 5.8520026206970215 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "636680137", + "selected_index": 0, + "state_hash": "2de607a9c69005130e51ff2b551c0eeedc85c58fef796682e6527bb5f719ff7b", + "stored_base_utility": 0.08405601978302002, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.026694953441619873, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026694953441619873 + }, + "base_progress": 0.026694953441619873, + "base_success": false, + "base_utility": 0.026694953441619873, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05164949223399162, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05164949223399162 + }, + { + "contact_quality": null, + "progress": 0.058844465762376785, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058844465762376785 + }, + { + "contact_quality": null, + "progress": 0.06657024472951889, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06657024472951889 + }, + { + "contact_quality": null, + "progress": 0.0681784376502037, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0681784376502037 + }, + { + "contact_quality": null, + "progress": 0.06253000348806381, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06253000348806381 + }, + { + "contact_quality": null, + "progress": 0.07150205224752426, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07150205224752426 + }, + { + "contact_quality": null, + "progress": 0.03520200774073601, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03520200774073601 + }, + { + "contact_quality": null, + "progress": 0.06939387321472168, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06939387321472168 + } + ], + "candidate_progress": [ + 0.05164949223399162, + 0.058844465762376785, + 0.06657024472951889, + 0.0681784376502037, + 0.06253000348806381, + 0.07150205224752426, + 0.03520200774073601, + 0.06939387321472168 + ], + "candidate_source_chart_ids": [ + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-53a86042af3c0887e556", + "ms-02fd0338e80ef19ceec1", + "ms-1a4979e969aa331fb126", + "ms-02fd0338e80ef19ceec1" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-f3f1d07cfd59b823da47", + "generated_tangents": [ + [ + 1.675511360168457, + 0.26294803619384766, + -1.4619871377944946, + 0.09748843312263489, + 0.3256249725818634, + -1.1040034294128418, + -0.6247721314430237, + 0.3977947533130646, + 0.4779442548751831, + -0.24419444799423218, + -1.0073573589324951, + 0.9466355443000793, + -0.7505614757537842, + 0.035916902124881744, + 0.5673705339431763, + -0.12787234783172607, + 0.2507711946964264, + -1.643969178199768, + 0.35970455408096313, + 0.5917714834213257, + 1.1986117362976074 + ], + [ + 1.560312032699585, + 0.2568896412849426, + -1.2574419975280762, + 0.07654470205307007, + 0.06049570441246033, + -1.1585447788238525, + -0.3848813772201538, + 0.48328763246536255, + 0.4009150564670563, + -0.10568415373563766, + -1.0240111351013184, + 0.8088661432266235, + -0.5029674172401428, + -0.18004967272281647, + 0.36323776841163635, + -0.11226554214954376, + 0.45954424142837524, + -1.7849628925323486, + 0.16309618949890137, + 0.4507681131362915, + 1.1250041723251343 + ], + [ + 1.6391892433166504, + 0.21951808035373688, + -1.2775074243545532, + 0.15611949563026428, + -0.13830643892288208, + -1.5244075059890747, + -0.721285879611969, + 0.47953909635543823, + 0.4167535603046417, + -0.0974070206284523, + -0.986589789390564, + 0.8505012392997742, + -0.30110645294189453, + -0.1401648372411728, + 0.38734349608421326, + -0.07386188209056854, + 0.30531561374664307, + -1.8018094301223755, + 0.0659090131521225, + 0.31123441457748413, + 1.0316061973571777 + ], + [ + 1.7891749143600464, + 0.1960882842540741, + -1.3303453922271729, + 0.2006845474243164, + -0.07155710458755493, + -1.420583724975586, + -0.4426800608634949, + 0.5741879940032959, + 0.346225768327713, + -0.13232919573783875, + -0.9523600339889526, + 0.5992389917373657, + -0.2720524072647095, + 0.03333738446235657, + 0.4209025204181671, + -0.07059133797883987, + 0.2101317048072815, + -1.7509722709655762, + 0.14263051748275757, + 0.2756538987159729, + 0.9637851715087891 + ], + [ + 1.5337421894073486, + 0.10512694716453552, + -1.608415961265564, + 0.09830436110496521, + -0.13416674733161926, + -1.8996893167495728, + -0.7178241610527039, + 0.5179807543754578, + 0.07927774637937546, + -0.0958782508969307, + -0.7517118453979492, + 0.5402031540870667, + -0.8901242017745972, + -0.42635107040405273, + 0.5358250737190247, + -0.18648803234100342, + 0.4157024919986725, + -1.4566999673843384, + 0.3473930060863495, + 0.42524397373199463, + 0.6094003915786743 + ], + [ + 1.5719108581542969, + 0.3079289197921753, + -1.0445280075073242, + 0.3755629360675812, + 0.2826114296913147, + -1.1220356225967407, + -0.13396495580673218, + 0.476203054189682, + 0.10016392916440964, + -0.13018935918807983, + -1.5082701444625854, + 0.618819534778595, + -0.4119489789009094, + -0.49988651275634766, + 0.2877805829048157, + -0.03060813993215561, + 0.5619557499885559, + -1.9755491018295288, + 0.0077082812786102295, + -0.4956511855125427, + 0.6007959246635437 + ], + [ + 1.664137840270996, + -0.013941869139671326, + -1.6251887083053589, + 0.08530622720718384, + -0.3102415204048157, + -1.9219149351119995, + -0.34388530254364014, + 0.5052124261856079, + 0.14826075732707977, + 0.05399750918149948, + -0.842513918876648, + 0.7535050511360168, + -0.6396822333335876, + -0.2530655264854431, + 0.3765254616737366, + -0.25213462114334106, + 0.14109429717063904, + -1.3795732259750366, + 0.3979029655456543, + 0.18450382351875305, + 0.26160329580307007 + ], + [ + 1.533952236175537, + 0.2404658943414688, + -0.9675089120864868, + 0.45229610800743103, + 0.14575034379959106, + -1.1170729398727417, + -0.10040819644927979, + 0.28900712728500366, + 0.07777386903762817, + -0.16432088613510132, + -1.539223074913025, + 0.5006495714187622, + -0.47292834520339966, + -0.4586854577064514, + 0.12763994932174683, + 0.029766645282506943, + 0.513634204864502, + -1.9874070882797241, + -0.11878997087478638, + -0.5150457620620728, + 0.5731385946273804 + ] + ], + "generated_utilities": [ + 0.05164949223399162, + 0.058844465762376785, + 0.06657024472951889, + 0.0681784376502037, + 0.06253000348806381, + 0.07150205224752426, + 0.03520200774073601, + 0.06939387321472168 + ], + "group_id": "ms-f3f1d07cfd59b823da47", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07767709344625473, + 0.07153411954641342, + 0.08480638265609741, + 0.06087514013051987, + 0.15252648293972015, + 0.0776997059583664, + 0.1476559340953827, + 0.17124073207378387, + 0.12327007949352264, + 0.12037092447280884, + 0.08562581241130829, + 0.09917601197957993, + 0.07137986272573471, + 0.039922356605529785, + 0.04496842622756958, + 0.05561637878417969 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.9664052724838257, + 0.20301294326782227, + -1.1196715831756592, + -0.17260856926441193, + -0.5341777801513672, + -2.678128242492676, + -2.431448459625244, + 1.5603530406951904, + 0.03585217520594597, + -0.3520800471305847, + -2.473386287689209, + 1.0162299871444702, + -0.7868502736091614, + -0.9102090001106262, + 0.9442894458770752, + -0.41473180055618286, + 0.7349075675010681, + -0.8863289952278137, + 0.8796874284744263, + 0.028114937245845795, + 0.24920660257339478 + ], + [ + 1.900488257408142, + -0.00838443636894226, + -1.2311820983886719, + -0.361179918050766, + -0.3987593352794647, + -2.551008462905884, + -2.701385021209717, + 1.7117550373077393, + -0.20423506200313568, + -0.3429180979728699, + -2.6584055423736572, + 0.8018259406089783, + -0.856230616569519, + -0.901038408279419, + 1.2131764888763428, + -0.6892215609550476, + 0.6726197600364685, + -0.7798839211463928, + 0.6868253946304321, + 0.07731680572032928, + 0.30107203125953674 + ], + [ + 1.6257587671279907, + 0.275219202041626, + -0.7838636636734009, + -0.5118123292922974, + -0.3429456353187561, + -2.342111349105835, + -2.7158243656158447, + 1.7038744688034058, + -0.1860528439283371, + -0.11617843806743622, + -2.1894750595092773, + 1.2306283712387085, + -0.8203369379043579, + -0.5157363414764404, + 0.7951346635818481, + -0.464976966381073, + 0.8823708295822144, + -0.7671404480934143, + 0.9700374007225037, + 0.21184596419334412, + -0.11244213581085205 + ] + ], + "predicted_scores": [ + 2.532451629638672, + 2.415501594543457, + 1.4083845615386963, + 0.6119946241378784, + 0.362302303314209, + -0.1249924898147583, + -0.4573264718055725, + -0.7109801173210144 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "3f477a29208b02558b99bd0f3305d0f1845c277d32e0b5afdafa71012673383b", + "stored_base_utility": 0.07767709344625473, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07080724835395813, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07080724835395813 + }, + "base_progress": 0.07080724835395813, + "base_success": false, + "base_utility": 0.07080724835395813, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07753174006938934, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07753174006938934 + }, + { + "contact_quality": null, + "progress": 0.08595690131187439, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08595690131187439 + }, + { + "contact_quality": null, + "progress": 0.07273110747337341, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07273110747337341 + }, + { + "contact_quality": null, + "progress": 0.05202282965183258, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05202282965183258 + }, + { + "contact_quality": null, + "progress": 0.07089017331600189, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07089017331600189 + }, + { + "contact_quality": null, + "progress": 0.07804974913597107, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07804974913597107 + }, + { + "contact_quality": null, + "progress": 0.05986259877681732, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05986259877681732 + }, + { + "contact_quality": null, + "progress": 0.07560504972934723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07560504972934723 + } + ], + "candidate_progress": [ + 0.07753174006938934, + 0.08595690131187439, + 0.07273110747337341, + 0.05202282965183258, + 0.07089017331600189, + 0.07804974913597107, + 0.05986259877681732, + 0.07560504972934723 + ], + "candidate_source_chart_ids": [ + "ms-bfe867e8d518b9f8333a", + "ms-bfe867e8d518b9f8333a", + "ms-8278abbc47a6efcb5b68", + "ms-dab80e1bb7987577e33d", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-081952a63087fc8f83cd", + "generated_tangents": [ + [ + -0.08413964509963989, + 0.03917499631643295, + 1.2771568298339844, + -0.5833404660224915, + -1.2979159355163574, + -0.041353851556777954, + -0.3111611008644104, + -0.05563884973526001, + -0.12758560478687286, + 0.067821204662323, + 0.06210175156593323, + -1.336356520652771, + 0.2173217236995697, + -0.1255672425031662, + -0.012791294604539871, + -0.04858876392245293, + -0.0979764312505722, + -0.056227996945381165, + -0.8472711443901062, + -0.14808081090450287, + -0.3216269612312317 + ], + [ + -0.07111789286136627, + 0.03896534442901611, + 1.2926208972930908, + -0.5241761207580566, + -1.3137810230255127, + -0.09370563924312592, + -0.23570245504379272, + -0.03557869791984558, + -0.12659049034118652, + 0.10744508355855942, + -0.004305452108383179, + -1.3119266033172607, + 0.11784610152244568, + -0.10802482068538666, + 0.010881159454584122, + -0.04123491793870926, + -0.06381000578403473, + 0.006138816475868225, + -0.8246253132820129, + -0.1516846865415573, + -0.24148374795913696 + ], + [ + -0.053984493017196655, + -0.00562024861574173, + 1.1701840162277222, + -0.7135963439941406, + -1.5532948970794678, + -0.23730385303497314, + -0.3853493332862854, + 0.05915568768978119, + -0.2242712825536728, + 0.07229231297969818, + -0.012854605913162231, + -1.1027201414108276, + 0.18021747469902039, + -0.16772504150867462, + 0.03970526158809662, + -0.08681954443454742, + -0.11705993115901947, + 0.08885939419269562, + -0.647961437702179, + -0.021091073751449585, + -0.2371947467327118 + ], + [ + -0.04518130421638489, + 0.0816914290189743, + 1.1488096714019775, + -0.7166935205459595, + -1.2783079147338867, + -0.350945383310318, + -0.22076040506362915, + 0.030124112963676453, + -0.17097710072994232, + 0.1264670193195343, + 0.0489087849855423, + -1.3729066848754883, + 0.2075645625591278, + -0.14103834331035614, + 0.06115962937474251, + -0.09586217999458313, + -0.17797888815402985, + -0.0452701598405838, + -0.8797468543052673, + -0.10645312070846558, + -0.3185126483440399 + ], + [ + 0.11026055365800858, + -0.0033897459506988525, + 1.1837600469589233, + -0.6096736192703247, + -1.4959893226623535, + -0.28977492451667786, + -0.4396909475326538, + 0.0747135728597641, + -0.1966264396905899, + 0.12885478138923645, + -0.011177584528923035, + -1.1425373554229736, + 0.4872763454914093, + -0.2498861402273178, + 0.08118508011102676, + -0.09010106325149536, + -0.13109330832958221, + 0.04437825083732605, + -0.6183353662490845, + -0.03854522109031677, + -0.34984156489372253 + ], + [ + -0.1432105451822281, + -0.02981628105044365, + 1.0101110935211182, + -0.5665549039840698, + -1.189751148223877, + -0.12700220942497253, + -0.34022319316864014, + 0.07882219552993774, + -0.311112642288208, + 0.10574111342430115, + 0.031146898865699768, + -1.334547996520996, + 0.5379418134689331, + -0.01745852828025818, + 0.055465541779994965, + -0.2213810831308365, + -0.4261760711669922, + -0.035221174359321594, + -0.9912137389183044, + -0.1770738661289215, + -0.30926287174224854 + ], + [ + 0.15350456535816193, + -0.10617871582508087, + 0.9539602994918823, + -0.47117069363594055, + -1.4828500747680664, + -0.4168761074542999, + -0.36822497844696045, + 0.04427656531333923, + -0.3682369291782379, + 0.10710634291172028, + 0.09130032360553741, + -1.3806426525115967, + 0.5532610416412354, + -0.3403542637825012, + 0.12044143676757812, + -0.17413827776908875, + -0.4426705837249756, + -0.04571153223514557, + -0.885425865650177, + -0.11885681748390198, + -0.21574485301971436 + ], + [ + 0.10382343828678131, + 0.007059935480356216, + 0.9621686935424805, + -0.6107028126716614, + -1.3704118728637695, + -0.447879821062088, + -0.5862154364585876, + 0.26559633016586304, + -0.3464195430278778, + 0.02941250056028366, + 0.20671603083610535, + -1.2919561862945557, + 0.472435861825943, + -0.25946474075317383, + 0.14030812680721283, + -0.051595766097307205, + -0.31563740968704224, + 0.01749001443386078, + -0.9021003842353821, + -0.042577385902404785, + -0.33679935336112976 + ] + ], + "generated_utilities": [ + 0.07753174006938934, + 0.08595690131187439, + 0.07273110747337341, + 0.05202282965183258, + 0.07089017331600189, + 0.07804974913597107, + 0.05986259877681732, + 0.07560504972934723 + ], + "group_id": "ms-081952a63087fc8f83cd", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0006054937839508057, + 2.0, + 2.0, + 2.0, + 0.09041815996170044, + 0.03755016624927521, + 0.10023769736289978, + 0.08501018583774567, + 0.10316208004951477, + 0.13556814193725586, + 0.0687420666217804, + 0.16420090198516846, + 0.07440806925296783, + 0.03177754580974579, + 0.05512912571430206, + 0.05822136998176575 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05530714988708496, + -0.0015276968479156494, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.028481781482696533, + 0.0, + 0.07634823024272919, + 0.10118435323238373, + 0.01472395658493042, + -0.021184463053941727, + -1.3624117374420166, + 0.06702810525894165, + 0.0, + 0.064424529671669, + -0.13110561668872833, + -0.04677841067314148, + -0.0666094422340393, + -0.8611308336257935, + -0.02982822060585022, + 0.0 + ], + [ + -0.13999104499816895, + -0.16139763593673706, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.17549771070480347, + 0.0, + -0.15019088983535767, + 0.2207401990890503, + -0.06357991695404053, + 0.10177183151245117, + -1.3624117374420166, + -0.05549207329750061, + 0.0, + -0.1271420419216156, + -0.057942673563957214, + -0.279840350151062, + -0.11312717199325562, + -0.8611308336257935, + 0.1586582064628601, + 0.0 + ], + [ + 0.2317524552345276, + -0.2474955916404724, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.2997032403945923, + 0.0, + -0.3180278241634369, + 0.1991673707962036, + 0.26833486557006836, + 0.005899060517549515, + -1.3624117374420166, + 0.032782405614852905, + 0.0, + 0.17653466761112213, + 0.5550464987754822, + 0.13015127182006836, + -0.11312717199325562, + -0.8611308336257935, + 0.020897403359413147, + 0.0 + ], + [ + -0.7016198635101318, + 0.312343955039978, + 2.582733154296875, + -1.5283455848693848, + -2.530975103378296, + 0.7002967596054077, + 2.0224196910858154, + -0.031223494559526443, + -0.07945594191551208, + 0.7471928000450134, + -0.05505431815981865, + -2.3624117374420166, + 0.4337144196033478, + 1.385794997215271, + -0.18244768679141998, + -0.22281835973262787, + -0.5003347992897034, + 0.8868728280067444, + -1.8611308336257935, + 0.19736243784427643, + 0.2646070420742035 + ], + [ + -0.41503679752349854, + 0.3199126422405243, + 2.3341803550720215, + -1.5432449579238892, + -2.6036009788513184, + 0.8186125755310059, + 1.889835238456726, + -0.20609386265277863, + 0.0827421247959137, + 0.8381537795066833, + -0.0944519191980362, + -2.4805798530578613, + 0.3916063606739044, + 1.2392640113830566, + -0.2266693413257599, + -0.33516526222229004, + -0.3536234498023987, + 0.9749420881271362, + -2.1439476013183594, + 0.3261662721633911, + 0.13869160413742065 + ], + [ + -0.6778035759925842, + 0.3674989640712738, + 2.3216195106506348, + -1.6129508018493652, + -2.3233790397644043, + 0.37060582637786865, + 2.3460428714752197, + 0.2269098311662674, + -0.269074022769928, + 0.9810001254081726, + 0.023842941969633102, + -2.478595733642578, + 0.46739575266838074, + 1.1671457290649414, + -0.41542381048202515, + 0.030661001801490784, + -0.48762595653533936, + 0.7803086638450623, + -1.7778995037078857, + 0.10219369828701019, + 0.08611755073070526 + ], + [ + -1.0292376279830933, + 0.05653715133666992, + 3.017642021179199, + -1.2965388298034668, + -2.1434991359710693, + 0.8112732172012329, + 2.3553943634033203, + 0.14745590090751648, + -0.42473679780960083, + 0.8983602523803711, + -0.5389888286590576, + -1.9595260620117188, + 0.8665074110031128, + 1.7057592868804932, + -0.36306142807006836, + -0.6694610714912415, + -0.3699086308479309, + 0.7089512944221497, + -2.2496888637542725, + 0.1759258657693863, + 0.13106243312358856 + ], + [ + -0.2738665044307709, + 0.13772070407867432, + 2.3146424293518066, + -1.9326081275939941, + -2.760838508605957, + 0.10491085052490234, + 1.9823331832885742, + 0.3494338393211365, + 0.3550114631652832, + 0.7471076846122742, + -0.06264704465866089, + -2.2210469245910645, + 0.5386381149291992, + 1.431454062461853, + 0.1620958298444748, + -0.00858105719089508, + -0.8883554935455322, + 1.1764272451400757, + -1.8280467987060547, + 0.30690017342567444, + -0.32479730248451233 + ], + [ + -0.8548018932342529, + -0.2078946828842163, + 2.740286111831665, + -1.6552648544311523, + -2.470123291015625, + 1.0607044696807861, + 2.4267592430114746, + -0.16535070538520813, + -0.2640931308269501, + 0.2447068691253662, + 0.5616287589073181, + -2.808340311050415, + 1.0820176601409912, + 1.2134929895401, + 0.2653927206993103, + -0.454708456993103, + 0.03163468837738037, + 0.8172736763954163, + -1.8861355781555176, + 0.45078086853027344, + 0.29641321301460266 + ], + [ + -0.7025277018547058, + 0.5556451678276062, + 2.3374006748199463, + -1.9256110191345215, + -1.7495018243789673, + 0.9721168279647827, + 1.8326293230056763, + 0.5355191826820374, + -0.5788277983665466, + 0.19061005115509033, + -0.6322742700576782, + -3.031383514404297, + -0.1651831567287445, + 0.8584551215171814, + 0.2280312329530716, + 0.27221786975860596, + -0.3728860020637512, + 1.102063775062561, + -2.5995376110076904, + 0.7578867673873901, + -0.3784477412700653 + ], + [ + -7.212162017822266e-06, + 0.552659273147583, + 2.0397679805755615, + -1.8501746654510498, + -2.5457160472869873, + 1.0514975786209106, + 2.5270917415618896, + -0.7367458343505859, + 0.7676748037338257, + 0.7428221702575684, + -0.538831353187561, + -2.7423243522644043, + 0.07969537377357483, + 1.0069079399108887, + -0.8100140690803528, + -0.9174093008041382, + -0.9411231875419617, + 1.0430617332458496, + -1.4683198928833008, + 1.0334573984146118, + 0.4721405506134033 + ], + [ + -0.41159626841545105, + 0.41878798604011536, + 1.6363914012908936, + -2.4960880279541016, + -3.049954891204834, + -0.09567022323608398, + 2.3269245624542236, + 0.11159682273864746, + -0.7885257005691528, + 0.44582536816596985, + -0.47802892327308655, + -1.5574463605880737, + 1.0153859853744507, + 2.258706569671631, + -0.36591836810112, + 0.49115514755249023, + -0.9523675441741943, + 0.5892280340194702, + -2.1512646675109863, + -0.7744256854057312, + 0.592117428779602 + ], + [ + 0.1343594193458557, + 0.45838433504104614, + 2.6735801696777344, + -2.005676746368408, + -3.340463161468506, + 1.0152086019515991, + 1.429718255996704, + 0.7929191589355469, + -0.2598758339881897, + 1.6483545303344727, + -0.7411167025566101, + -2.8978958129882812, + 1.3759692907333374, + 1.4385837316513062, + 0.2961248755455017, + -0.042098239064216614, + -0.6380948424339294, + 1.0523303747177124, + -2.1783251762390137, + -0.045405834913253784, + 0.7960561513900757 + ] + ], + "predicted_scores": [ + 9.857619285583496, + 9.777491569519043, + 9.474225997924805, + 9.17353343963623, + 9.032880783081055, + 8.81174087524414, + 8.667104721069336, + 8.220312118530273 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2141802816", + "selected_index": 0, + "state_hash": "d1cf06bba4675ad6a6942dc4f6fbade890fbf21b5c3068f8bbe30cfc60c54d9b", + "stored_base_utility": 0.0006054937839508057, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-2ba02dcefa15a9371706" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-121bc2e6676c1ac10911", + "generated_tangents": [ + [ + -0.318436861038208, + -0.24442674219608307, + -0.6643575429916382, + -0.1913587898015976, + -1.2045369148254395, + -0.887955904006958, + 1.3046760559082031, + 0.26366913318634033, + 0.2270069718360901, + -0.6376742720603943, + 0.27735474705696106, + 0.12393875420093536, + -0.2746342122554779, + 0.2749885618686676, + 0.07337480783462524, + -0.08053277432918549, + -0.23778875172138214, + 0.6927629709243774, + -0.4625341296195984, + -0.5742101669311523, + 0.22084379196166992 + ], + [ + 0.13696816563606262, + -0.0004087015986442566, + -0.24295303225517273, + -0.5239930152893066, + -0.5989259481430054, + -0.3572954833507538, + 1.5271000862121582, + 0.2594227194786072, + 0.17775587737560272, + -0.2258957028388977, + 0.21136316657066345, + -0.05476680397987366, + -0.3642699420452118, + 1.1860625743865967, + 0.05777456611394882, + -0.024060523137450218, + -0.20670513808727264, + 0.5988383889198303, + -0.0570034384727478, + -0.08882161974906921, + 0.37320005893707275 + ], + [ + -0.25128161907196045, + -0.3132287859916687, + -0.6979422569274902, + -0.3270675241947174, + -1.327434778213501, + -0.15659767389297485, + 1.2446153163909912, + 0.5223498940467834, + 0.14893178641796112, + -0.5016502737998962, + 0.32958194613456726, + -0.14257335662841797, + -0.07503217458724976, + 0.3861290216445923, + -0.01677720621228218, + -0.13776861131191254, + -0.3512023687362671, + 0.7592557072639465, + 0.08203499019145966, + -0.45473146438598633, + -0.15495941042900085 + ], + [ + -0.2094193547964096, + -0.2954549193382263, + -0.8341259956359863, + -0.7916555404663086, + -0.8491094708442688, + -0.5231232643127441, + 1.3542044162750244, + 0.42853909730911255, + 0.12180132418870926, + -0.44825518131256104, + 0.2387983500957489, + -0.12857186794281006, + 0.1692427694797516, + 0.27743467688560486, + 0.019608058035373688, + 0.19956675171852112, + -0.10563810169696808, + 0.5816810727119446, + 0.1299280971288681, + -0.6717466711997986, + 1.3120108842849731 + ], + [ + -0.24599553644657135, + -0.23191988468170166, + -0.6010323762893677, + -0.30621418356895447, + -1.1158783435821533, + -0.5426607131958008, + 1.2219572067260742, + 0.40301400423049927, + 0.0556047260761261, + -0.5179594159126282, + 0.34780120849609375, + -0.04690465331077576, + -0.40648210048675537, + 0.38549935817718506, + -0.05463051423430443, + -0.006608236115425825, + -0.24081142246723175, + 0.6482144594192505, + 0.11671988666057587, + -0.17064699530601501, + 0.6720317006111145 + ], + [ + -0.31878381967544556, + -0.12913677096366882, + -0.6215265989303589, + -0.24297234416007996, + -0.9468169808387756, + -0.5165524482727051, + 1.264082431793213, + 0.34228554368019104, + 0.0017401427030563354, + -0.599934458732605, + 0.3516547977924347, + 0.22312335669994354, + -0.43553489446640015, + 0.4596540331840515, + -0.016192656010389328, + -0.06471820175647736, + -0.2596778869628906, + 0.7153933048248291, + -0.12586182355880737, + -0.1682382971048355, + 0.8540645837783813 + ], + [ + -0.21072883903980255, + -0.20984023809432983, + -0.7151176929473877, + -0.422762393951416, + -0.9866161942481995, + -0.8116936683654785, + 1.283542275428772, + 0.33878257870674133, + 0.17407064139842987, + -0.4530002474784851, + 0.3152390122413635, + 0.13520950078964233, + -0.5315513610839844, + 0.12147153913974762, + -0.008829396218061447, + -0.08355154097080231, + -0.3370814323425293, + 0.738246500492096, + 0.08903366327285767, + -0.27276211977005005, + 0.1756531000137329 + ], + [ + -0.1948457807302475, + -0.2748200595378876, + -0.7478799819946289, + -0.5438199043273926, + -1.2477014064788818, + -0.44240716099739075, + 1.1769423484802246, + 0.2894083261489868, + -0.015147244557738304, + -0.15536320209503174, + 0.3076595366001129, + -0.4715762138366699, + -0.42085719108581543, + 0.38623708486557007, + -0.3163565695285797, + -0.100701242685318, + -0.5614150166511536, + 0.5407496094703674, + 0.21292541921138763, + 0.2552551031112671, + 0.5606482028961182 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-121bc2e6676c1ac10911", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15086030960083008, + 0.08354458212852478, + 2.0, + 0.1453719437122345, + 0.19646552205085754, + 0.10343040525913239, + 0.1443832516670227, + 0.13976123929023743, + 0.17431864142417908, + 2.0, + 0.12949708104133606, + 2.0, + 2.0, + 0.11397694051265717, + 0.1271887570619583, + 0.025320395827293396 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.39665424823760986, + 0.08244594186544418, + -0.026901081204414368, + -0.959688663482666, + -1.2648048400878906, + 0.05568990111351013, + 0.0, + -0.048488348722457886, + 0.03576555848121643, + -0.012020409107208252, + 0.08576416969299316, + -0.25812816619873047, + 0.03589746356010437, + 0.0, + 0.0016258209943771362, + -0.04025750607252121, + 0.010670006275177002, + 0.5142596960067749, + 0.06220144033432007, + 0.07906517386436462, + 0.0 + ], + [ + -2.15720796585083, + 0.13025060296058655, + 0.41441529989242554, + -1.4498568773269653, + -2.6386165618896484, + 0.5572659969329834, + 0.7093658447265625, + -1.0835764408111572, + -0.896172285079956, + 0.07744038105010986, + 1.1224509477615356, + -1.6321502923965454, + 0.877888560295105, + 0.4978562295436859, + 0.5774499177932739, + 0.8093571662902832, + -0.8521628379821777, + 2.054164171218872, + -0.6021197438240051, + 1.0350128412246704, + 0.823830246925354 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.39665424823760986, + -0.2284793257713318, + -0.027931194752454758, + -0.959688663482666, + -1.2648048400878906, + -0.17197783291339874, + 0.0, + -0.06679429113864899, + 0.1646350920200348, + -0.11601293087005615, + 0.08576416969299316, + -0.25812816619873047, + 0.1479925513267517, + 0.0, + -0.04833713173866272, + 0.22201494872570038, + -0.16177666187286377, + 0.5142596960067749, + -0.10178101062774658, + 0.21631890535354614, + 0.0 + ], + [ + -0.8824495077133179, + 0.0034109503030776978, + -0.5904110074043274, + -1.400491714477539, + -2.4447474479675293, + -0.00789790228009224, + 1.6208159923553467, + 0.26819390058517456, + -0.23575741052627563, + -0.7184416651725769, + 0.9072303771972656, + -1.2077033519744873, + -0.21496516466140747, + 0.8258882761001587, + -0.4575563669204712, + -0.3655202388763428, + -0.5908474922180176, + 2.0210399627685547, + -1.2132787704467773, + 0.9719676971435547, + 0.24625355005264282 + ], + [ + -0.6953714489936829, + 0.4026735723018646, + 0.6648998856544495, + -1.356613278388977, + -2.953742742538452, + 0.037713564932346344, + 1.53464937210083, + 0.027994811534881592, + 0.03435474634170532, + -1.1939163208007812, + 0.9877840280532837, + -1.180513620376587, + 0.4064505994319916, + 0.5743880867958069, + 0.1558205783367157, + 0.3796072006225586, + -0.6676555275917053, + 1.4665511846542358, + 0.04170459508895874, + 0.17645099759101868, + 0.2388516366481781 + ], + [ + -1.4095232486724854, + -0.06358731538057327, + -0.7774888873100281, + -2.806776285171509, + -2.44041109085083, + -0.8258867859840393, + 2.383592128753662, + -0.093393474817276, + -0.24701429903507233, + -0.5286431312561035, + 0.7741218209266663, + -2.111828565597534, + -0.31807881593704224, + 1.3003538846969604, + -0.9306488037109375, + -0.8708829283714294, + -0.3694953918457031, + 2.169797420501709, + -0.7153271436691284, + 0.029519766569137573, + -0.5629553198814392 + ] + ], + "predicted_scores": [ + 4.049079895019531, + 3.6869189739227295, + 3.434694528579712, + 2.4521331787109375, + 2.2561216354370117, + 2.2208027839660645, + 2.131687641143799, + 1.2841386795043945 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264", + "selected_index": 0, + "state_hash": "d1280a9e9362f8da3290de84fdd0f0b78475d470cd0230e590ccaf8f5428548a", + "stored_base_utility": 0.15086030960083008, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-22f153f0d4fa656018e6", + "generated_tangents": [ + [ + -0.5488069653511047, + -0.12914320826530457, + -0.6730979681015015, + -1.359136700630188, + -1.8728837966918945, + -0.8446304798126221, + -0.013707607984542847, + 0.10725326836109161, + -0.11694855988025665, + -0.07521618157625198, + 0.20123475790023804, + -1.3166704177856445, + -0.21968255937099457, + 0.005218483507633209, + -0.2585197985172272, + 0.1388433575630188, + -0.07499019801616669, + -0.01642654836177826, + -0.47000378370285034, + 0.2271711230278015, + 0.0815325528383255 + ], + [ + -0.5242918729782104, + -0.12676024436950684, + -0.3443840444087982, + -1.167953610420227, + -1.8117244243621826, + -0.3880108892917633, + -0.34209388494491577, + -0.05148926377296448, + -0.014074873179197311, + -0.0524558499455452, + 0.10325033217668533, + -1.1297495365142822, + 0.027836233377456665, + 0.0595313236117363, + -0.1211487203836441, + -0.029480289667844772, + -0.35129958391189575, + 0.005054488778114319, + -0.5940192341804504, + -0.3171018362045288, + -0.09436988830566406 + ], + [ + -0.6183275580406189, + -0.14431670308113098, + -0.2307959496974945, + -1.1094334125518799, + -1.853135108947754, + -0.6286224126815796, + -0.3655887842178345, + -0.057424843311309814, + -0.05028947442770004, + -0.09524308890104294, + 0.04739289730787277, + -1.1648361682891846, + -0.06320396065711975, + 0.024961866438388824, + 0.05146493762731552, + 0.038867440074682236, + -0.3965842127799988, + -0.01534724235534668, + -0.5904173254966736, + -0.24465589225292206, + -0.3948509991168976 + ], + [ + -0.4657665491104126, + -0.07714395225048065, + -0.24930688738822937, + -1.0198901891708374, + -1.9631257057189941, + -0.4246378242969513, + -0.30768728256225586, + 0.022464826703071594, + -0.05418386310338974, + -0.07536391168832779, + 0.06656061857938766, + -1.1389085054397583, + -0.043238699436187744, + -0.1611955612897873, + -0.07273463904857635, + 0.018092108890414238, + -0.3483579158782959, + -0.1254226416349411, + -0.5164749026298523, + -0.14282383024692535, + -0.2232438325881958 + ], + [ + -0.36254262924194336, + -0.07197297364473343, + -0.2658368647098541, + -1.0821423530578613, + -1.8539245128631592, + -0.5465748310089111, + -0.18021124601364136, + 0.070470429956913, + -0.06224729120731354, + -0.11129259318113327, + 0.1406990885734558, + -1.0396536588668823, + -0.13670280575752258, + -0.17410142719745636, + -0.1736098825931549, + 0.02032379060983658, + -0.2861585021018982, + 0.04271472990512848, + -0.49530476331710815, + 0.033352822065353394, + -0.09253549575805664 + ], + [ + -0.4510018229484558, + -0.0446300283074379, + -0.2147749960422516, + -1.1680705547332764, + -1.8639743328094482, + -0.5043494701385498, + -0.3151446580886841, + 0.017371058464050293, + -0.04363667964935303, + -0.15871462225914001, + 0.13338208198547363, + -0.9481333494186401, + 0.10634538531303406, + -0.1777450293302536, + 0.031976766884326935, + -0.009491289034485817, + -0.3257819414138794, + -0.037767812609672546, + -0.535429835319519, + -0.11300502717494965, + -0.41360774636268616 + ], + [ + -0.6611120104789734, + 0.1196136474609375, + 0.00514674186706543, + -1.0458996295928955, + -1.8806705474853516, + -0.6966314315795898, + -0.07055884599685669, + -0.022101834416389465, + -0.000422540120780468, + -0.00888977199792862, + 0.07196106761693954, + -0.9307974576950073, + 0.33626559376716614, + -0.3816136121749878, + 0.09100475162267685, + 0.09832784533500671, + -0.38475584983825684, + -0.11374054849147797, + -0.5637039542198181, + -0.1436951756477356, + 0.009382277727127075 + ], + [ + -0.2370283454656601, + 0.04556134343147278, + 0.2720962464809418, + -0.6675891876220703, + -1.6301751136779785, + -0.39291802048683167, + -0.0955129861831665, + 0.04320399463176727, + -0.013827163726091385, + 0.011234015226364136, + -0.015867426991462708, + -1.0916461944580078, + 0.17198309302330017, + 0.15019218623638153, + 0.005726959556341171, + 0.042269546538591385, + -0.19741766154766083, + -0.07113371789455414, + -0.46631568670272827, + 0.133294939994812, + -0.4270465672016144 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-22f153f0d4fa656018e6", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 2.60770320892334e-06, + 3.129243850708008e-07, + 6.109476089477539e-07, + 2.0, + 0.20781123638153076, + 2.0, + 0.16136802732944489, + 0.15920132398605347, + 0.16996727883815765, + 0.14405608177185059, + 0.15874098241329193, + 0.08526891469955444, + 0.0828208178281784, + 2.0, + 0.034265756607055664, + 2.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.3937166929244995, + 0.17073306441307068, + -0.5225132703781128, + -0.8479964733123779, + -1.713785171508789, + -0.19215916097164154, + 0.0, + 0.8004336953163147, + 0.4925512969493866, + 0.3161412179470062, + 0.10414126515388489, + -0.9857386350631714, + -0.4383167624473572, + 0.0, + 0.05412057042121887, + 0.021832358092069626, + 0.5262433290481567, + -0.24631118774414062, + -0.20008409023284912, + -0.04257836565375328, + 0.0 + ], + [ + -1.3937166929244995, + 0.2384568303823471, + 0.3933030962944031, + -1.847996473312378, + -2.713785171508789, + 0.20405007898807526, + 1.2454378604888916, + 0.0006654821336269379, + -0.07783810049295425, + -0.002305123955011368, + 0.18527865409851074, + -1.9857386350631714, + -0.009703370742499828, + 0.7078995108604431, + -0.08121174573898315, + -0.031056255102157593, + -0.40927961468696594, + 0.5954965949058533, + -0.8039777874946594, + 0.05187072232365608, + 0.2283477634191513 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.490875720977783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4157990217208862, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4566955268383026 + ], + [ + -1.5219146013259888, + 0.23501667380332947, + 0.5663670301437378, + -1.892876386642456, + -2.744781494140625, + -0.04844887554645538, + 1.3053655624389648, + -0.11823110282421112, + -0.02477371320128441, + -0.17061075568199158, + 0.22949494421482086, + -1.854048252105713, + -0.025548618286848068, + 0.7958164215087891, + 0.12320645153522491, + -0.23218455910682678, + -0.6995364427566528, + 0.8081986308097839, + -0.558222234249115, + -0.18823161721229553, + 0.04578563570976257 + ], + [ + -1.608424186706543, + 0.022089049220085144, + 0.15639334917068481, + -1.982436180114746, + -2.6509876251220703, + -0.01820056140422821, + 0.95000159740448, + -0.08617040514945984, + -0.46914470195770264, + 0.3453706204891205, + -0.020491018891334534, + -1.738605260848999, + -0.0859340950846672, + 0.443394273519516, + 0.09907735884189606, + -0.38721394538879395, + -0.3415156602859497, + 0.524202287197113, + -0.7343614101409912, + -0.22556106746196747, + 0.4535057842731476 + ], + [ + -1.7957028150558472, + 0.593864381313324, + 0.06846150755882263, + -2.0563552379608154, + -2.2595136165618896, + -0.12188448011875153, + 0.9899688959121704, + -0.14783401787281036, + -0.22327497601509094, + -0.32116562128067017, + -0.07331573963165283, + -1.910165786743164, + 0.264303058385849, + 1.0567166805267334, + 0.016188345849514008, + 0.159999281167984, + -0.22885318100452423, + 0.705758273601532, + -0.8778964877128601, + 0.4381847381591797, + 0.19897118210792542 + ], + [ + -1.499897837638855, + -0.2043466418981552, + 0.42094048857688904, + -1.430327296257019, + -3.1506710052490234, + -0.08664844930171967, + 1.4003938436508179, + 0.24064289033412933, + -0.35471564531326294, + -0.37839531898498535, + 0.47059693932533264, + -1.7703200578689575, + 0.101543128490448, + 0.8470064401626587, + 0.16171279549598694, + -0.0019259154796600342, + -0.6455138325691223, + 0.5694639682769775, + -0.9997703433036804, + 0.5366826057434082, + -0.11106891930103302 + ], + [ + -1.8847153186798096, + 0.21543154120445251, + 0.9309400320053101, + -2.1095237731933594, + -2.136512517929077, + 0.6948637366294861, + 1.233637809753418, + 0.577111542224884, + -0.5727867484092712, + -0.6055393815040588, + 0.8738593459129333, + -2.6654083728790283, + -0.171565979719162, + 0.8934992551803589, + 0.17394188046455383, + 0.6209473609924316, + -0.1149752140045166, + 0.6160256266593933, + -0.5519447326660156, + 0.7195507287979126, + 0.5175096392631531 + ], + [ + -2.138639450073242, + 0.36738795042037964, + -0.33481496572494507, + -1.9255597591400146, + -2.2420146465301514, + 0.31966179609298706, + 1.3172208070755005, + -0.48551350831985474, + 0.27687638998031616, + -0.41806119680404663, + 0.05362778902053833, + -2.3665032386779785, + -0.47635504603385925, + 0.081889808177948, + 0.542697548866272, + -0.7215265035629272, + -0.906980037689209, + 0.6977691054344177, + -1.4041197299957275, + -0.11211369931697845, + 0.5149455070495605 + ], + [ + -1.4869698286056519, + -0.30130720138549805, + 1.28191077709198, + -0.9839465618133545, + -2.2501890659332275, + -0.20459605753421783, + 1.3607591390609741, + 0.6349532008171082, + -0.8627415895462036, + -0.623334527015686, + 0.21189503371715546, + -2.2093186378479004, + -0.4655338525772095, + 0.7016672492027283, + 0.6916180849075317, + -0.0807163268327713, + 0.1465948522090912, + 0.05083376169204712, + -0.8381134271621704, + 0.015291888266801834, + 0.05565965175628662 + ], + [ + -2.365525245666504, + 0.45377761125564575, + 0.2984481453895569, + -1.3690096139907837, + -2.843494176864624, + 0.32462257146835327, + 0.5082458257675171, + 0.7033615112304688, + 0.31817564368247986, + -0.4890395700931549, + 0.09036337584257126, + -2.904804229736328, + 0.560937762260437, + 1.649811029434204, + -0.9795587062835693, + -0.46419358253479004, + -0.4739357829093933, + 0.34800490736961365, + -1.0712809562683105, + 0.9167320132255554, + -0.48252081871032715 + ], + [ + -1.2987136840820312, + 0.7370830774307251, + -0.49668365716934204, + -1.201794147491455, + -3.552318572998047, + -0.5406968593597412, + 2.1693286895751953, + 0.5925421714782715, + 0.7903083562850952, + 0.991620659828186, + 0.7936639785766602, + -2.3832802772521973, + -0.621701180934906, + 0.43542394042015076, + -0.14415577054023743, + 0.15921397507190704, + -0.395956814289093, + -0.2778439521789551, + -1.069305419921875, + -0.5347616076469421, + -0.26581722497940063 + ] + ], + "predicted_scores": [ + 5.7841668128967285, + 5.764790058135986, + 5.701019763946533, + 5.555482387542725, + 5.275122165679932, + 4.941834449768066, + 4.859184265136719, + 3.589162588119507 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007", + "selected_index": 0, + "state_hash": "3139b3ac90f6fb958f14d875d082c95c8569f039086e0789e6c9e079d602b945", + "stored_base_utility": 2.60770320892334e-06, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0320269912481308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0320269912481308 + }, + "base_progress": 0.0320269912481308, + "base_success": false, + "base_utility": 0.0320269912481308, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.025111451745033264, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025111451745033264 + }, + { + "contact_quality": null, + "progress": 0.01827371120452881, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01827371120452881 + }, + { + "contact_quality": null, + "progress": 0.027498364448547363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027498364448547363 + }, + { + "contact_quality": null, + "progress": 0.026322290301322937, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026322290301322937 + }, + { + "contact_quality": null, + "progress": 0.020748361945152283, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020748361945152283 + }, + { + "contact_quality": null, + "progress": 0.02072523534297943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02072523534297943 + }, + { + "contact_quality": null, + "progress": 0.02157720923423767, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02157720923423767 + }, + { + "contact_quality": null, + "progress": 0.017048463225364685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017048463225364685 + } + ], + "candidate_progress": [ + 0.025111451745033264, + 0.01827371120452881, + 0.027498364448547363, + 0.026322290301322937, + 0.020748361945152283, + 0.02072523534297943, + 0.02157720923423767, + 0.017048463225364685 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-ad5107846b45eae5a0ab", + "ms-543339410c02ce2269d1", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-2423891634e7b469eed8", + "ms-940cce8479900a244d54" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "generated_tangents": [ + [ + 0.21391832828521729, + 0.27911072969436646, + 0.6514626741409302, + -1.236355185508728, + 0.026780158281326294, + 0.04066044092178345, + 0.1448931097984314, + -0.11620920896530151, + 0.0004434185102581978, + 0.02462116628885269, + -0.18335825204849243, + -0.42883485555648804, + 0.3130630552768707, + 0.34273093938827515, + 0.15518979728221893, + -0.06447125971317291, + 0.24042530357837677, + -0.5651291608810425, + -0.5243905782699585, + 0.4151548147201538, + 0.3093748390674591 + ], + [ + 0.031997427344322205, + 0.22568179666996002, + 0.7923535108566284, + -1.2483689785003662, + -0.08416551351547241, + -0.019817635416984558, + 0.09127455949783325, + -0.055856361985206604, + 0.009183604270219803, + 0.14164593815803528, + 0.02516769990324974, + -0.5547645092010498, + 0.5003151893615723, + 0.17507660388946533, + -0.05569937452673912, + -0.14774176478385925, + -0.0396450012922287, + -0.5300953984260559, + -0.5317016243934631, + 0.03582993149757385, + 0.18680451810359955 + ], + [ + 0.19381044805049896, + 0.19411520659923553, + 0.6819138526916504, + -1.2794206142425537, + -0.10723105072975159, + 0.007015615701675415, + 0.22282150387763977, + -0.058224886655807495, + 0.014266949146986008, + -0.022819705307483673, + -0.1932016909122467, + -0.6837509870529175, + 0.32226911187171936, + 0.16902875900268555, + -0.017442185431718826, + -0.15719394385814667, + 0.005850657820701599, + -0.49608033895492554, + -0.5501047372817993, + 0.3775603175163269, + 0.2457629293203354 + ], + [ + 0.18482279777526855, + 0.22926713526248932, + 0.7575491666793823, + -1.2488725185394287, + 0.06903427839279175, + 0.16657881438732147, + 0.21184700727462769, + -0.09736546874046326, + 0.0339469239115715, + 0.00885976105928421, + 0.04673461616039276, + -0.44935286045074463, + 0.20288899540901184, + 0.3733958303928375, + -0.0769776701927185, + -0.15453627705574036, + 0.03576599061489105, + -0.5208722352981567, + -0.46502888202667236, + 0.19200679659843445, + 0.49625730514526367 + ], + [ + 0.45117801427841187, + 0.21957920491695404, + 0.66351318359375, + -1.3271337747573853, + 0.09396044909954071, + -0.03816880285739899, + 0.38407742977142334, + -0.026697292923927307, + 0.06369594484567642, + -0.009219236671924591, + -0.3157995641231537, + -0.448066771030426, + 0.13001969456672668, + 0.3111880421638489, + -0.016908634454011917, + -0.12007813155651093, + -0.03340102732181549, + -0.6338077783584595, + -0.6817090511322021, + 0.06826531887054443, + 0.36431747674942017 + ], + [ + 0.23480692505836487, + 0.15920279920101166, + 0.6638433933258057, + -1.2215197086334229, + -0.009544029831886292, + 0.18046826124191284, + 0.3554733097553253, + 0.00799146294593811, + 0.03908376395702362, + 0.05289533734321594, + -0.08747530728578568, + -0.5114601254463196, + 0.37424519658088684, + 0.15484236180782318, + -0.010930489748716354, + -0.1264394074678421, + 0.05480580031871796, + -0.5123270750045776, + -0.5571259260177612, + 0.4353986978530884, + 0.24013832211494446 + ], + [ + 0.305020272731781, + 0.2628641128540039, + 0.6076853275299072, + -1.1824610233306885, + 0.018159329891204834, + 0.12448372691869736, + 0.07640895247459412, + 0.010020792484283447, + 0.07619402557611465, + 0.07756601274013519, + -0.05289169400930405, + -0.3794039487838745, + 0.886386513710022, + 0.27065667510032654, + 0.15791872143745422, + -0.06339281052350998, + 0.013592511415481567, + -0.4755313992500305, + -0.43516629934310913, + 0.4843834638595581, + 0.27058181166648865 + ], + [ + 0.2745092511177063, + 0.1170053482055664, + 0.6184108853340149, + -1.2551156282424927, + -0.1045483648777008, + 0.14739400148391724, + 0.24699805676937103, + 0.011442571878433228, + 0.0034699542447924614, + -0.038183487951755524, + -0.07970982044935226, + -0.551439106464386, + 0.3451023995876312, + 0.22736720740795135, + -0.10189378261566162, + -0.12885276973247528, + 0.02862030267715454, + -0.5456748604774475, + -0.5757142305374146, + 0.15437385439872742, + 0.2642448842525482 + ] + ], + "generated_utilities": [ + 0.025111451745033264, + 0.01827371120452881, + 0.027498364448547363, + 0.026322290301322937, + 0.020748361945152283, + 0.02072523534297943, + 0.02157720923423767, + 0.017048463225364685 + ], + "group_id": "ms-27c9f70e97b8850cc3ab", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.09440004825592041, + 0.14095592498779297, + 2.0, + 0.09219714999198914, + 0.05765572190284729, + 0.02530825138092041, + 0.07676126062870026, + 0.06643831729888916, + 0.047928065061569214, + 0.07107676565647125, + 0.05561619997024536, + 0.02124820649623871, + 0.08948668837547302, + 0.10203869640827179, + 0.06119762361049652, + 0.04547174274921417 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.984761953353882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.7230467796325684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.554086923599243 + ], + [ + 1.0343247652053833, + 1.287368655204773, + 1.867376446723938, + -2.4465248584747314, + 1.036996841430664, + -1.2664310932159424, + 1.4399157762527466, + -0.8265262246131897, + -0.28319495916366577, + 0.16331201791763306, + -1.5917816162109375, + -1.5816115140914917, + -0.1664504110813141, + 0.7195659279823303, + 0.29144588112831116, + -0.18517816066741943, + -0.08826595544815063, + -0.8266173601150513, + -0.7933371067047119, + -0.938305139541626, + 0.5359267592430115 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3968086242675781, + 0.5359851121902466, + 0.7176156044006348, + -1.52370023727417, + -0.1481943428516388, + 0.10887271165847778, + 0.0, + -0.12331700325012207, + 0.14385993778705597, + -0.041522324085235596, + 0.16089874505996704, + -0.6189663410186768, + -0.01699581742286682, + 0.0, + -0.07216246426105499, + -0.15073662996292114, + -0.134138286113739, + -0.06049776077270508, + -0.26776671409606934, + -0.019470810890197754, + 0.0 + ] + ], + "predicted_scores": [ + 4.651674270629883, + 4.646581649780273, + 4.335030555725098, + 4.235018253326416, + 4.019102096557617, + 4.008563041687012, + 3.998105525970459, + 3.9744935035705566 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1881223807", + "selected_index": 0, + "state_hash": "0f58a11c36c331c9abab00795189dccbc7e0aebf985ca9df67bf9da756f154e3", + "stored_base_utility": 0.09440004825592041, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.10841864347457886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10841864347457886 + }, + "base_progress": 0.10841864347457886, + "base_success": false, + "base_utility": 0.10841864347457886, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.09370604157447815, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09370604157447815 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.09923991560935974, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09923991560935974 + }, + { + "contact_quality": null, + "progress": 0.08881917595863342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08881917595863342 + }, + { + "contact_quality": null, + "progress": 0.047322407364845276, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.047322407364845276 + }, + { + "contact_quality": null, + "progress": 0.0901307463645935, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0901307463645935 + } + ], + "candidate_progress": [ + 1.0, + 0.09370604157447815, + 1.0, + 1.0, + 0.09923991560935974, + 0.08881917595863342, + 0.047322407364845276, + 0.0901307463645935 + ], + "candidate_source_chart_ids": [ + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + true, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2867deaca7e98582620b", + "generated_tangents": [ + [ + -0.04763971269130707, + -0.0678432285785675, + 0.6449398398399353, + -0.1981954127550125, + -1.3741230964660645, + -0.33557868003845215, + -0.6917322278022766, + -0.09107854962348938, + -0.041970714926719666, + 0.18134674429893494, + 0.4880267083644867, + -1.5248818397521973, + 0.36955246329307556, + -0.1101178228855133, + 0.01678931713104248, + -0.13375253975391388, + 0.24641911685466766, + 0.37347614765167236, + -0.6676339507102966, + -0.26728612184524536, + -0.6003302335739136 + ], + [ + -0.09627243876457214, + -0.11876752227544785, + 0.6641285419464111, + -0.16966316103935242, + -1.4288198947906494, + -0.3930072486400604, + -0.6900435090065002, + -0.22106891870498657, + -0.09698735922574997, + 0.1734238713979721, + 0.4727734327316284, + -1.5060700178146362, + 0.40777507424354553, + -0.12340553104877472, + -0.036167170852422714, + -0.16675320267677307, + 0.3040286898612976, + 0.36621904373168945, + -0.6314959526062012, + -0.21443834900856018, + -0.4289957582950592 + ], + [ + -0.039548009634017944, + -0.022421829402446747, + 0.8354246616363525, + -0.4206569790840149, + -1.8109169006347656, + -0.6381659507751465, + -0.4683982729911804, + -0.12251394987106323, + -0.14813877642154694, + 0.02744618058204651, + 0.4390793442726135, + -1.274978756904602, + 0.11389526724815369, + -0.6276320219039917, + 0.017472438514232635, + -0.12346456944942474, + 0.14443257451057434, + 0.6089066863059998, + -0.755329430103302, + -0.400123655796051, + -0.5216860771179199 + ], + [ + -0.14656580984592438, + -0.01067439466714859, + 0.7990288734436035, + -0.5086795687675476, + -1.7642951011657715, + -0.646909236907959, + -0.4217536449432373, + -0.1634492576122284, + -0.09018047899007797, + 0.12375391274690628, + 0.3884236812591553, + -1.5418099164962769, + 0.0873769223690033, + -0.6337696313858032, + 0.018779784440994263, + -0.11734679341316223, + 0.20474588871002197, + 0.5171203017234802, + -0.7829434275627136, + -0.39981991052627563, + -0.6596001386642456 + ], + [ + -0.0746893435716629, + -0.045699138194322586, + 0.5160688757896423, + -0.42363274097442627, + -1.7745115756988525, + -0.2991913855075836, + -0.48841381072998047, + -0.1964462697505951, + -0.11526210606098175, + -0.048754192888736725, + 0.3773079514503479, + -1.0535447597503662, + 0.0609075129032135, + -0.16449986398220062, + -0.0647135078907013, + -0.07983292639255524, + 0.2888284921646118, + 0.6147249937057495, + -0.48144179582595825, + -0.773140013217926, + -0.5666354894638062 + ], + [ + -0.09176909923553467, + -0.03295668959617615, + 0.7303811311721802, + -0.6170456409454346, + -1.4957702159881592, + -0.1792428195476532, + -0.6505621075630188, + -0.24939551949501038, + -0.07375060021877289, + 0.23282092809677124, + 0.4616556167602539, + -1.710983157157898, + -0.19623568654060364, + -0.3865475654602051, + 0.05085122957825661, + -0.10729581117630005, + 0.29688090085983276, + 0.5926147103309631, + -0.7100726366043091, + -0.46041321754455566, + -0.404010146856308 + ], + [ + -0.10214368999004364, + -0.1263304054737091, + 0.9959338903427124, + -0.5758398771286011, + -1.518005132675171, + -0.41378673911094666, + -0.7111075520515442, + -0.5181982517242432, + -0.08770646899938583, + 0.15437161922454834, + 0.2556975781917572, + -1.6821696758270264, + 0.002704530954360962, + -0.261150598526001, + -0.14594191312789917, + -0.026637643575668335, + 0.29601413011550903, + 0.4495137929916382, + -0.9120617508888245, + -0.4980655908584595, + -0.44152769446372986 + ], + [ + -0.10134515166282654, + -0.14118391275405884, + 0.9698335528373718, + -0.6442772150039673, + -1.2969911098480225, + -0.9057308435440063, + -0.7732080817222595, + -0.26239848136901855, + -0.11714011430740356, + -0.04360116273164749, + 0.43906840682029724, + -1.540845274925232, + 0.11133697628974915, + -0.5487735271453857, + -0.061068709939718246, + 0.02590206265449524, + 0.2362658828496933, + 0.4810160994529724, + -0.7179185152053833, + -0.2162087857723236, + -0.5954456329345703 + ] + ], + "generated_utilities": [ + 2.0, + 0.09370604157447815, + 2.0, + 2.0, + 0.09923991560935974, + 0.08881917595863342, + 0.047322407364845276, + 0.0901307463645935 + ], + "group_id": "ms-2867deaca7e98582620b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0924256443977356, + 0.05646410584449768, + 2.0, + 2.0, + 0.183049738407135, + 2.0, + 0.16824382543563843, + 0.1424575299024582, + 0.08230158686637878, + 0.15871968865394592, + 0.16382521390914917, + 0.05336849391460419, + 0.07674244046211243, + 0.10727545619010925, + 0.06594061851501465, + 0.045542001724243164 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.44511327147483826, + 0.05827039107680321, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.2619132101535797, + 0.0, + -0.37528133392333984, + -0.07968637347221375, + -0.07514983415603638, + 0.19317352771759033, + 0.03868463635444641, + -0.06793206930160522, + 0.0, + 0.13187909126281738, + -0.20905672013759613, + 0.030073702335357666, + 0.21597933769226074, + -0.0761370062828064, + -0.11662125587463379, + 0.0 + ], + [ + -0.15486939251422882, + 0.24427275359630585, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.44790399074554443, + 0.0, + -0.37528133392333984, + -0.29508399963378906, + 0.7311322093009949, + 0.19317352771759033, + 0.09925723075866699, + 0.1767415851354599, + 0.0, + -0.5768744349479675, + 0.1300974041223526, + 0.21151533722877502, + 0.21597933769226074, + 0.04431194067001343, + -0.040631115436553955, + 0.0 + ], + [ + -0.29166290163993835, + 0.09681949019432068, + 2.2608249187469482, + -1.3577959537506104, + -2.195330858230591, + -0.13884210586547852, + 2.5595173835754395, + -1.3752813339233398, + 0.390796959400177, + 0.673005223274231, + 1.1931735277175903, + -0.41860872507095337, + -0.24474714696407318, + 0.3015621602535248, + -0.2400326132774353, + 0.14207373559474945, + -0.26512715220451355, + 1.2159793376922607, + -0.3237651586532593, + -0.7466862201690674, + 0.30001604557037354 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.119034767150879, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6031243205070496, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6000320911407471 + ], + [ + -0.3580354154109955, + 0.03320832550525665, + 2.5546932220458984, + -1.138798713684082, + -2.260084629058838, + -0.04964215308427811, + 2.8354909420013428, + -1.6732244491577148, + 0.21277914941310883, + 0.5097208023071289, + 1.387328028678894, + -0.691602349281311, + -0.28483790159225464, + 0.35576167702674866, + -0.0605899840593338, + 0.3764685094356537, + -0.28031954169273376, + 1.063202977180481, + -0.03841540217399597, + -0.5410391092300415, + 0.5580100417137146 + ], + [ + -0.5578174591064453, + 0.3727092146873474, + 2.1929497718811035, + -1.577579140663147, + -1.8962993621826172, + -0.2802738845348358, + 2.1670117378234863, + -1.0752147436141968, + 0.16372516751289368, + 0.6087679862976074, + 0.8854888081550598, + -0.18080148100852966, + -0.18821784853935242, + 0.010591715574264526, + -0.12560874223709106, + 0.15850795805454254, + -0.1898641288280487, + 1.1823316812515259, + 0.06655246019363403, + -0.4014361798763275, + 0.4376043379306793 + ], + [ + -0.43790310621261597, + -0.12778332829475403, + 2.6857643127441406, + -1.6293541193008423, + -1.9850949048995972, + -0.7167636752128601, + 2.465301513671875, + -1.4102312326431274, + 0.2071499079465866, + 0.5742072463035583, + 0.9525313377380371, + -0.34116798639297485, + -0.04760183393955231, + -0.05569759011268616, + -0.4777999520301819, + 0.17186680436134338, + 0.0841393768787384, + 0.7708102464675903, + -0.2784478962421417, + -1.2458651065826416, + 0.46140795946121216 + ], + [ + 0.06170770525932312, + 0.45555436611175537, + 1.6506444215774536, + -0.9394457340240479, + -1.9117670059204102, + -0.6910014152526855, + 2.40145206451416, + -1.7531975507736206, + 0.10217079520225525, + 1.2514894008636475, + 1.3236911296844482, + -0.6504120826721191, + -0.6943472027778625, + 0.7029284238815308, + -0.7387814521789551, + -0.33777421712875366, + -0.8187699317932129, + 1.8448829650878906, + 0.00849190354347229, + -1.394152045249939, + 0.918946385383606 + ] + ], + "predicted_scores": [ + 9.423457145690918, + 9.199984550476074, + 9.018994331359863, + 8.800616264343262, + 8.53002643585205, + 8.281440734863281, + 8.193784713745117, + 7.748565196990967 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1189392917", + "selected_index": 0, + "state_hash": "e8cf9b95f09f32d8e7942b114792725bd89c392a97ff7e6468ebf09df1bb0d21", + "stored_base_utility": 0.0924256443977356, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.13927479088306427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13927479088306427 + }, + "base_progress": 0.13927479088306427, + "base_success": false, + "base_utility": 0.13927479088306427, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.054733291268348694, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054733291268348694 + }, + { + "contact_quality": null, + "progress": 0.130090594291687, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.130090594291687 + }, + { + "contact_quality": null, + "progress": 0.1239127367734909, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1239127367734909 + }, + { + "contact_quality": null, + "progress": 0.11601470410823822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11601470410823822 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1307433694601059, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1307433694601059 + } + ], + "candidate_progress": [ + 0.054733291268348694, + 0.130090594291687, + 0.1239127367734909, + 0.11601470410823822, + 1.0, + 1.0, + 1.0, + 0.1307433694601059 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + true, + true, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-365d8c6de6de16f66744", + "generated_tangents": [ + [ + -0.3216032385826111, + -0.09330794960260391, + 1.3372516632080078, + -0.4772129952907562, + -1.5004887580871582, + 0.016235902905464172, + 0.21235880255699158, + -0.044765010476112366, + -0.183041512966156, + 0.28999531269073486, + -0.13156822323799133, + -1.3798290491104126, + -0.18255624175071716, + 0.23481377959251404, + -0.014678005129098892, + -0.056500520557165146, + 0.03761991858482361, + -0.21641232073307037, + -0.2754674553871155, + -0.09078086912631989, + 0.05808679014444351 + ], + [ + -0.1144808977842331, + -0.05292811617255211, + 1.2491137981414795, + -0.39868515729904175, + -1.4802651405334473, + -0.3096119463443756, + -0.09811574220657349, + -0.005206048488616943, + -0.11761322617530823, + 0.13060137629508972, + 0.05133647844195366, + -1.5365304946899414, + 0.04028269648551941, + -0.007292404770851135, + 0.1082511842250824, + -0.07130888104438782, + 0.08780141174793243, + -0.014397680759429932, + -0.2765340805053711, + -0.244420126080513, + -0.20084506273269653 + ], + [ + -0.06980770826339722, + -0.10473823547363281, + 1.195068359375, + -0.4298571050167084, + -1.5672507286071777, + -0.20873722434043884, + -0.15355783700942993, + -0.05104941129684448, + -0.12815922498703003, + 0.15062212944030762, + -0.02957005798816681, + -1.336713194847107, + 0.015305101871490479, + -0.006348222494125366, + 0.13029927015304565, + -0.07177798449993134, + 0.04681609570980072, + 0.0243389755487442, + -0.32356107234954834, + -0.18493227660655975, + -0.22214433550834656 + ], + [ + -0.042171478271484375, + -0.12031754106283188, + 1.2885713577270508, + -0.46994948387145996, + -1.515977144241333, + -0.39441218972206116, + 0.03862711787223816, + -0.1264740526676178, + -0.11714942753314972, + 0.13802669942378998, + -0.046181827783584595, + -1.3958179950714111, + -0.008493602275848389, + 0.1297491043806076, + 0.07320552319288254, + -0.09979991614818573, + -0.034600868821144104, + 0.0075378865003585815, + -0.3996841311454773, + -0.1345905214548111, + -0.22771862149238586 + ], + [ + 0.011613085865974426, + -0.08167794346809387, + 1.2190431356430054, + -0.40916967391967773, + -1.4880385398864746, + -0.39875528216362, + -0.14453530311584473, + 0.0459790974855423, + -0.12014222145080566, + 0.18404307961463928, + 0.07235030829906464, + -1.5115779638290405, + 0.03647473454475403, + -0.054496511816978455, + 0.24218566715717316, + -0.04489048942923546, + 0.09471248090267181, + -0.05882169306278229, + -0.3326870799064636, + -0.15918555855751038, + -0.1406424641609192 + ], + [ + 0.01318301260471344, + -0.1407550722360611, + 1.2396866083145142, + -0.3165585398674011, + -1.4180755615234375, + -0.3175802528858185, + 0.043541669845581055, + 0.03394855558872223, + -0.1642056405544281, + 0.16651961207389832, + -0.27376940846443176, + -1.378136396408081, + 0.5216007232666016, + -0.04210418462753296, + 0.11567295342683792, + -0.030030319467186928, + -0.10715444386005402, + -0.13276152312755585, + -0.4976702928543091, + -0.30539098381996155, + -0.48064926266670227 + ], + [ + -0.0824265331029892, + -0.06618291139602661, + 1.276452660560608, + -0.3603707253932953, + -1.496096134185791, + -0.4063102900981903, + -0.4149729609489441, + -0.15529516339302063, + -0.08880845457315445, + 0.10781368613243103, + 0.021841958165168762, + -1.2306064367294312, + 0.34022465348243713, + -0.04823920130729675, + -0.19550997018814087, + -0.03683430328965187, + -0.20409326255321503, + -0.011105716228485107, + -0.28989434242248535, + -0.10742746293544769, + -0.3438708484172821 + ], + [ + -0.00818982720375061, + -0.11088667809963226, + 1.400978446006775, + -0.3424351215362549, + -1.455199956893921, + -0.3057328462600708, + -0.11656457185745239, + 0.08311361819505692, + -0.1791704148054123, + 0.1343432366847992, + -0.2693428099155426, + -1.2886607646942139, + 0.6036311388015747, + -0.4333891272544861, + -0.0030657686293125153, + 0.08980372548103333, + -0.06903629004955292, + -0.12000139057636261, + -0.4740470051765442, + -0.32972341775894165, + -0.5390377044677734 + ] + ], + "generated_utilities": [ + 0.054733291268348694, + 0.130090594291687, + 0.1239127367734909, + 0.11601470410823822, + 2.0, + 2.0, + 2.0, + 0.1307433694601059 + ], + "group_id": "ms-365d8c6de6de16f66744", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20231953263282776, + 0.12319359183311462, + 2.0, + 2.0, + 0.1590207815170288, + 0.019343823194503784, + 0.1322992593050003, + 0.09479346871376038, + 0.1768283247947693, + 0.1054161787033081, + 0.16278749704360962, + 0.08510464429855347, + 0.09564858675003052, + 0.07681193947792053, + 0.1334480196237564, + 0.08148902654647827 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.09058855473995209, + -0.022760659456253052, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.07837581634521484, + 0.0, + -0.05228453874588013, + -0.0679071694612503, + 0.026890814304351807, + 0.03908693790435791, + -1.4526653289794922, + 0.04786187410354614, + 0.0, + 0.019858457148075104, + -0.05534793436527252, + 0.17386651039123535, + -0.024165034294128418, + -0.05880159139633179, + 0.044484615325927734, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.063174724578857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6561365127563477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0078073740005493 + ], + [ + 0.05168074369430542, + 0.27186232805252075, + 2.0911388397216797, + -1.6812132596969604, + -2.3280625343322754, + 0.38023167848587036, + 2.5090978145599365, + -0.2604735791683197, + 0.04725922644138336, + 0.8121852874755859, + -0.6659450531005859, + -2.2462568283081055, + 0.49988994002342224, + 1.9786770343780518, + 0.02324119210243225, + -0.45050501823425293, + 1.1065142154693604, + 0.5724118947982788, + -0.6320602893829346, + -0.7826599478721619, + 0.47463753819465637 + ], + [ + 0.46382713317871094, + -0.06331050395965576, + 2.657292366027832, + -1.1560972929000854, + -2.765941619873047, + 0.7805119752883911, + 2.3117406368255615, + -0.3644295036792755, + -0.25628525018692017, + 0.7623568773269653, + -0.4176616668701172, + -2.646364688873291, + 0.40904325246810913, + 1.7654002904891968, + -0.23315654695034027, + 0.026929572224617004, + 0.8236149549484253, + 0.8322737812995911, + -0.7612741589546204, + -0.4725078046321869, + 0.5895571708679199 + ], + [ + -0.26226741075515747, + 0.3922410011291504, + 2.450082302093506, + -0.9020822048187256, + -3.028742551803589, + 1.0746917724609375, + 3.104015350341797, + 0.3752234876155853, + -0.009036362171173096, + 0.3652568459510803, + -0.7608495950698853, + -2.337242603302002, + -0.03944078087806702, + 1.972575306892395, + -0.32381975650787354, + -0.45855021476745605, + 1.6927499771118164, + 0.8358478546142578, + -0.8610806465148926, + -0.6741821765899658, + 0.4013170003890991 + ], + [ + 0.5790615677833557, + 0.8308188915252686, + 3.141986846923828, + -1.218785285949707, + -2.0300495624542236, + 0.4300342798233032, + 3.0737316608428955, + -0.09167198091745377, + 0.4237035810947418, + 0.7421274185180664, + -0.637946367263794, + -2.1245827674865723, + 0.4184945523738861, + 1.2377147674560547, + -0.4068492650985718, + -0.03615438938140869, + 1.5385099649429321, + 0.51850825548172, + -0.5737811923027039, + -1.472043752670288, + 0.45800793170928955 + ], + [ + 0.38714414834976196, + 0.25513744354248047, + 2.9400248527526855, + -1.8116681575775146, + -1.7500431537628174, + 1.403240442276001, + 1.9709672927856445, + 0.6755956411361694, + 0.49445489048957825, + 0.19490671157836914, + -1.1990764141082764, + -2.803361177444458, + 0.499578058719635, + 1.164682149887085, + 0.9588776230812073, + 0.5154257416725159, + 1.0328537225723267, + 1.1309876441955566, + -0.9365650415420532, + -0.9663279056549072, + -0.032733798027038574 + ], + [ + 0.7417005896568298, + -0.12406697869300842, + 3.0325844287872314, + -0.5656485557556152, + -2.5084800720214844, + 0.27374324202537537, + 2.2886011600494385, + 0.5997821092605591, + 0.9623891115188599, + 0.38451504707336426, + -1.4501185417175293, + -1.6908833980560303, + 0.6743149757385254, + 1.8593690395355225, + -0.018024832010269165, + -0.20444278419017792, + 1.364742636680603, + 0.34230074286460876, + -1.4427728652954102, + -1.5577378273010254, + 0.5589802265167236 + ], + [ + 0.3448556661605835, + -0.5947309732437134, + 2.3888885974884033, + -1.0376405715942383, + -2.218167781829834, + 1.568857192993164, + 3.0995821952819824, + -0.4682580530643463, + 0.3072889745235443, + 0.14532917737960815, + -0.567415177822113, + -3.085655689239502, + -0.6656016111373901, + 1.6447856426239014, + 0.8135664463043213, + 0.3758860230445862, + 1.399398922920227, + 0.11528515815734863, + -0.531513512134552, + -0.6104787588119507, + 1.4474445581436157 + ], + [ + -0.18966330587863922, + -0.7208725214004517, + 2.586766481399536, + -0.6973344087600708, + -3.2187445163726807, + 1.4117884635925293, + 3.0797414779663086, + -0.29149553179740906, + 0.09620378166437149, + 0.874613881111145, + -0.8004729151725769, + -2.0231733322143555, + -0.5948910713195801, + 1.5080900192260742, + -0.750367283821106, + 0.8155628442764282, + 1.6562600135803223, + 0.8091130256652832, + -1.1569771766662598, + -1.4121384620666504, + 0.3738836348056793 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06428352743387222, + -0.015524566173553467, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + -0.1607142686843872, + 0.0, + 0.27928173542022705, + 0.05664636939764023, + -0.03914147615432739, + 0.1700872778892517, + -1.4526653289794922, + 0.05555404722690582, + 0.0, + 0.18240894377231598, + -0.05239863693714142, + 0.17386651039123535, + -0.21921080350875854, + -0.3728579580783844, + 0.11250120401382446, + 0.0 + ], + [ + 0.008915487676858902, + 0.540733277797699, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.22902190685272217, + 0.0, + -0.3544532060623169, + -0.2348114550113678, + 0.04456782341003418, + 0.2907915711402893, + -1.4526653289794922, + -0.1788860559463501, + 0.0, + 0.07762694358825684, + -0.07040916383266449, + 0.17386651039123535, + 0.44059625267982483, + -0.19159233570098877, + 0.09149748086929321, + 0.0 + ] + ], + "predicted_scores": [ + 10.039008140563965, + 9.910670280456543, + 9.840947151184082, + 9.763011932373047, + 9.72486400604248, + 9.312314987182617, + 9.310540199279785, + 9.040696144104004 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1997090716", + "selected_index": 0, + "state_hash": "98870eab9f394b355af9b8a6d689b3bcd574af94db5d1a9139d0aed0b633b959", + "stored_base_utility": 0.20231953263282776, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03280790150165558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03280790150165558 + }, + "base_progress": 0.03280790150165558, + "base_success": false, + "base_utility": 0.03280790150165558, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.02049049735069275, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02049049735069275 + }, + { + "contact_quality": null, + "progress": 0.019123628735542297, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019123628735542297 + }, + { + "contact_quality": null, + "progress": 0.02138802409172058, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02138802409172058 + }, + { + "contact_quality": null, + "progress": 0.021766602993011475, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021766602993011475 + }, + { + "contact_quality": null, + "progress": 0.020316720008850098, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020316720008850098 + }, + { + "contact_quality": null, + "progress": 0.020232409238815308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020232409238815308 + }, + { + "contact_quality": null, + "progress": 0.020674332976341248, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020674332976341248 + }, + { + "contact_quality": null, + "progress": 0.01947517693042755, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01947517693042755 + } + ], + "candidate_progress": [ + 0.02049049735069275, + 0.019123628735542297, + 0.02138802409172058, + 0.021766602993011475, + 0.020316720008850098, + 0.020232409238815308, + 0.020674332976341248, + 0.01947517693042755 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1", + "ms-fe38b164397dd669f391", + "ms-2423891634e7b469eed8", + "ms-2cbe83da02847008bf24", + "ms-2cbe83da02847008bf24", + "ms-4e67cfd30a257b4bad0c", + "ms-3f6cd47af5e9e6781b73" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3c326d05fec66e1d6bea", + "generated_tangents": [ + [ + 0.16476602852344513, + -0.7746200561523438, + 0.4733996093273163, + 0.6301577091217041, + 0.4205824136734009, + -0.45246633887290955, + 0.2243681252002716, + 0.14780843257904053, + 0.4026601314544678, + -0.151661217212677, + -0.2815152704715729, + -0.6137023568153381, + 0.01335868239402771, + -0.3578207492828369, + 0.22738154232501984, + 0.0726105272769928, + 0.0457034707069397, + -0.21299202740192413, + 0.11014659702777863, + -0.14157848060131073, + 0.07258753478527069 + ], + [ + 0.276533842086792, + -0.7341556549072266, + 0.526631772518158, + 0.6573271751403809, + 0.22801227867603302, + -0.5764880180358887, + -0.07588177919387817, + 0.1517592966556549, + 0.42064860463142395, + -0.07303262501955032, + -0.2933375835418701, + -0.4886290431022644, + -0.2643055319786072, + -0.2738226056098938, + -0.0169597826898098, + -0.11798045039176941, + -0.03258328139781952, + -0.17629940807819366, + 0.20040032267570496, + -0.1956155002117157, + 0.2750420570373535 + ], + [ + 0.31270211935043335, + -0.7637926340103149, + 0.6518735289573669, + 0.5016700029373169, + 0.306433767080307, + -0.3301326632499695, + -0.14478343725204468, + 0.2674139142036438, + 0.4751649796962738, + -0.22880062460899353, + -0.274884432554245, + -0.4472060203552246, + -0.07112160325050354, + -0.24198009073734283, + 0.20803046226501465, + -0.1601962298154831, + -0.10024450719356537, + -0.12129254639148712, + 0.16944289207458496, + -0.19254659116268158, + 0.4723520576953888 + ], + [ + 0.3514142334461212, + -0.7639657258987427, + 0.5289261937141418, + 0.4677295386791229, + 0.3823462724685669, + -0.20860344171524048, + -0.7424694895744324, + 0.30742573738098145, + 0.44662776589393616, + -0.07738064974546432, + -0.2882902920246124, + -0.007942810654640198, + 0.06950792670249939, + -0.36652618646621704, + 0.23742523789405823, + -0.18107864260673523, + -0.2582334876060486, + -0.3388598561286926, + 0.25595366954803467, + -0.12364216148853302, + 0.2660079598426819 + ], + [ + 0.3825204372406006, + -0.8296485543251038, + 0.69476717710495, + 0.4517246186733246, + 0.327655166387558, + -0.5421425104141235, + -0.344096839427948, + 0.1634248048067093, + 0.45052796602249146, + -0.201113760471344, + -0.31148332357406616, + -0.5693178772926331, + -0.15159597992897034, + -0.08663521707057953, + 0.10822102427482605, + -0.07047496736049652, + -0.11448802053928375, + -0.17071433365345, + 0.26362794637680054, + 0.14572039246559143, + 0.38151073455810547 + ], + [ + 0.38860803842544556, + -0.8245835304260254, + 0.6431694030761719, + 0.53558349609375, + 0.3753895163536072, + -0.5509328842163086, + -0.4268704056739807, + 0.12425629794597626, + 0.3904174566268921, + -0.17857274413108826, + -0.34679609537124634, + -0.5098536014556885, + -0.019853979349136353, + -0.16961641609668732, + -0.018545765429735184, + -0.1217147707939148, + -0.16610486805438995, + -0.20802204310894012, + 0.12054277956485748, + -0.0583534836769104, + 0.3944220542907715 + ], + [ + 0.09971822798252106, + -0.8076961040496826, + 0.6701511144638062, + 0.656691312789917, + 0.4004058241844177, + -0.4149051010608673, + -0.007472395896911621, + 0.16103240847587585, + 0.37219664454460144, + -0.1118859276175499, + -0.2798059582710266, + -0.6645269393920898, + 0.07491078972816467, + -0.3090340495109558, + -0.12858299911022186, + -0.06796237826347351, + -0.04526151716709137, + -0.18992768228054047, + 0.1250699907541275, + -0.4557264745235443, + 0.210085928440094 + ], + [ + 0.5095956325531006, + -0.5202342867851257, + 0.839331865310669, + 0.5186622142791748, + 0.4385504126548767, + -0.6590684652328491, + 0.07915988564491272, + 0.1053347960114479, + 0.5073033571243286, + -0.2813633978366852, + -0.19366136193275452, + -0.6410282850265503, + -0.19180193543434143, + -0.14046071469783783, + 0.2305084615945816, + -0.10779756307601929, + -0.37554121017456055, + -0.2032083123922348, + 0.08785460889339447, + -0.053603753447532654, + 0.3816084563732147 + ] + ], + "generated_utilities": [ + 0.02049049735069275, + 0.019123628735542297, + 0.02138802409172058, + 0.021766602993011475, + 0.020316720008850098, + 0.020232409238815308, + 0.020674332976341248, + 0.01947517693042755 + ], + "group_id": "ms-3c326d05fec66e1d6bea", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18246883153915405, + 2.0, + 2.0, + 0.11388173699378967, + 0.035728439688682556, + 0.05159728229045868, + 0.043176501989364624, + 0.04580867290496826, + 0.02321702241897583, + 0.040450215339660645, + 0.04055213928222656, + 0.024657413363456726, + 0.017660364508628845, + 0.044850513339042664, + 0.031234994530677795, + 0.06195336580276489 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.018136948347091675, + 0.10074883699417114, + 0.11574286222457886, + 0.0, + 0.1410461664199829, + 0.38714247941970825, + 0.2381301373243332, + 0.06927800178527832, + -0.6138286590576172, + -0.33484986424446106, + 0.0, + 0.012610852718353271, + 0.5407634973526001, + 0.07784794270992279, + 0.3537199795246124, + -0.1602279543876648, + 0.002093076705932617, + 0.0 + ], + [ + 1.9327397346496582, + -1.7429183721542358, + 1.8877019882202148, + -0.33001917600631714, + 0.49746543169021606, + 0.47390231490135193, + 1.7802670001983643, + -0.8589538335800171, + 0.37976425886154175, + -0.18104670941829681, + -0.8859321475028992, + -1.6138286590576172, + 0.34242650866508484, + 1.2023069858551025, + 0.555707573890686, + -0.35764527320861816, + -0.10027600824832916, + 0.4702257812023163, + -0.6294202208518982, + 0.1926015019416809, + 0.5916687846183777 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.5605340003967285, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.404613971710205, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1833375692367554 + ], + [ + 1.7389304637908936, + -1.4791877269744873, + 2.158487319946289, + -0.3266654312610626, + 0.5907376408576965, + 0.43260636925697327, + 1.9136874675750732, + -0.5760036110877991, + 0.3071907162666321, + -0.4073694348335266, + -0.8863769769668579, + -1.8163083791732788, + 0.5969879627227783, + 1.196010947227478, + 0.7674136161804199, + -0.2368052750825882, + -0.12123071402311325, + 0.36782756447792053, + -0.8187789916992188, + 0.15793970227241516, + 0.43327897787094116 + ], + [ + 1.9560860395431519, + -2.008063793182373, + 1.6910814046859741, + -0.2576735019683838, + 0.647503137588501, + 0.5322431921958923, + 1.9268598556518555, + -1.2501428127288818, + 0.775363564491272, + -0.0965004414319992, + -0.5331029891967773, + -1.639766812324524, + 0.44450435042381287, + 1.5980249643325806, + 0.9402694702148438, + -0.1454993337392807, + -0.2541998028755188, + 0.6709543466567993, + -0.44338259100914, + -0.043697476387023926, + 0.7131086587905884 + ], + [ + 1.4609386920928955, + -1.6896755695343018, + 2.174467086791992, + -0.34207087755203247, + 0.962078332901001, + 0.28904104232788086, + 2.0044631958007812, + -1.184704303741455, + 0.8424319624900818, + -0.6237084865570068, + -0.9171007871627808, + -1.5616872310638428, + 0.3556978702545166, + 1.5233795642852783, + 0.7210954427719116, + 0.028062552213668823, + 0.21759141981601715, + 0.2823620140552521, + -0.588395357131958, + -0.12581703066825867, + 0.7275858521461487 + ], + [ + 2.087728500366211, + -2.0203187465667725, + 2.4627456665039062, + 0.09322953224182129, + 0.4771752953529358, + 0.2439241260290146, + 2.3431313037872314, + -0.7176190614700317, + 0.8298519849777222, + -0.47738587856292725, + -1.4635096788406372, + -1.2598087787628174, + 0.008061617612838745, + 0.9835944175720215, + 0.10656380653381348, + -0.1548040360212326, + 0.20115382969379425, + 0.4581896960735321, + -1.197885274887085, + -0.28882738947868347, + 0.17229679226875305 + ], + [ + 2.535038471221924, + -2.2647459506988525, + 1.280318260192871, + -0.8243153095245361, + 0.36149507761001587, + 0.01678559184074402, + 1.259200096130371, + -0.5383518934249878, + 1.0661393404006958, + -0.1134992390871048, + -1.0009663105010986, + -1.8629896640777588, + 0.5329263806343079, + 1.1952301263809204, + 0.5380017757415771, + -0.21264730393886566, + 0.33232176303863525, + 0.9111320972442627, + -1.0945085287094116, + 0.7546109557151794, + 0.8692140579223633 + ], + [ + 2.3966352939605713, + -2.3338406085968018, + 2.0661728382110596, + -0.1174439787864685, + -0.28071242570877075, + 0.8010929822921753, + 2.485050678253174, + -0.5360689163208008, + 0.9712552428245544, + 0.5487139821052551, + -0.46108368039131165, + -2.0631186962127686, + 0.6046813726425171, + 0.9991111159324646, + 1.299267292022705, + 0.3742367625236511, + -0.7156072854995728, + 0.7805436849594116, + -1.3091955184936523, + -0.5538330674171448, + -0.20684683322906494 + ], + [ + 2.614837884902954, + -2.24919056892395, + 2.2911505699157715, + -0.2939787209033966, + 1.1876089572906494, + 0.15644106268882751, + 1.7348854541778564, + -0.7635170221328735, + 0.5141214728355408, + -0.17918013036251068, + -0.8421157598495483, + -1.5084272623062134, + -0.18822845816612244, + 1.8389434814453125, + 0.24652135372161865, + 0.38509100675582886, + 0.5761538147926331, + -0.15904095768928528, + 0.0013433098793029785, + 0.6120359897613525, + 1.3977106809616089 + ], + [ + 1.5386989116668701, + -2.0177698135375977, + 1.8581953048706055, + -1.2861346006393433, + 1.0945217609405518, + 0.8363389372825623, + 2.708517074584961, + -0.9201098084449768, + 0.8546208739280701, + -0.058566831052303314, + -1.5932974815368652, + -2.015472412109375, + 0.2565937638282776, + 1.988462209701538, + -0.04372602701187134, + -0.8754178881645203, + 0.27551382780075073, + -0.29693683981895447, + -0.2824576497077942, + 0.40926772356033325, + 0.7658611536026001 + ], + [ + 2.8735361099243164, + -2.675261974334717, + 1.459324836730957, + 0.380817174911499, + 0.6231281757354736, + 0.03267902135848999, + 2.296022415161133, + -1.587423324584961, + 0.7833051681518555, + 0.6618785262107849, + -1.7110093832015991, + -1.436572551727295, + -0.14784568548202515, + 1.525026559829712, + 1.4909214973449707, + -1.0601696968078613, + -0.5454426407814026, + -0.09859517216682434, + -1.4344520568847656, + -0.058256953954696655, + -0.3444262146949768 + ], + [ + 2.0854334831237793, + -1.3952012062072754, + 2.651411771774292, + -0.7663257122039795, + -0.05179548263549805, + 0.8797686100006104, + 2.0904030799865723, + -0.8943268656730652, + 1.0378066301345825, + 0.08841629326343536, + -0.36119532585144043, + -1.236596703529358, + 1.0752983093261719, + 0.349973201751709, + -0.19527214765548706, + -0.8867040872573853, + 0.18460334837436676, + 1.454195261001587, + -0.6220338940620422, + 0.9919598698616028, + 1.229588508605957 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.009577363729476929, + -0.06759434938430786, + 0.05406871438026428, + 0.0, + 0.003663182258605957, + 0.0492824912071228, + -0.038241639733314514, + -0.02580159902572632, + -0.6138286590576172, + -0.056222498416900635, + 0.0, + -0.026217222213745117, + -0.051223963499069214, + -0.031640589237213135, + -0.010870158672332764, + 0.08728909492492676, + -0.020528942346572876, + 0.0 + ], + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.06735062599182129, + -0.023807168006896973, + 0.1439460813999176, + 0.0, + -0.19209599494934082, + -0.14847975969314575, + -0.22449488937854767, + 0.09710657596588135, + -0.6138286590576172, + -0.1854294240474701, + 0.0, + -0.17072844505310059, + 0.09961000084877014, + -0.017846792936325073, + 0.22415585815906525, + -0.18760734796524048, + -0.060083091259002686, + 0.0 + ] + ], + "predicted_scores": [ + 8.38741397857666, + 8.032026290893555, + 7.941018581390381, + 7.737374305725098, + 7.561871528625488, + 7.263994216918945, + 7.038136959075928, + 6.148074626922607 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524", + "selected_index": 0, + "state_hash": "6333fcf80f76890398820f27d1a99efe459cdb6ff795201d73e3bcedc966664f", + "stored_base_utility": 0.18246883153915405, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18095605075359344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18095605075359344 + }, + "base_progress": 0.18095605075359344, + "base_success": false, + "base_utility": 0.18095605075359344, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.061786800622940063, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061786800622940063 + }, + { + "contact_quality": null, + "progress": 0.07495661079883575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07495661079883575 + }, + { + "contact_quality": null, + "progress": 0.061137303709983826, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061137303709983826 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.061786800622940063, + 0.07495661079883575, + 0.061137303709983826, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "generated_tangents": [ + [ + -0.7440127730369568, + 0.014998644590377808, + -0.5712763071060181, + -1.8051837682724, + -0.9082604050636292, + -1.1683580875396729, + -0.6399858593940735, + -0.12223121523857117, + -0.2638425827026367, + -0.1335526704788208, + 0.1557423621416092, + 0.018410593271255493, + 0.4193209707736969, + -0.3744836449623108, + -0.024524342268705368, + -0.5091266632080078, + 0.1175609827041626, + 0.5742997527122498, + 0.20372354984283447, + -0.4122728705406189, + -0.4277847707271576 + ], + [ + -0.3522629737854004, + -0.06639464199542999, + -0.26701971888542175, + -2.1856768131256104, + -0.9222970604896545, + -0.8079099655151367, + -0.6796756386756897, + -0.2296118438243866, + -0.12299269437789917, + 0.09488151222467422, + 0.08475707471370697, + -0.17584702372550964, + 0.007751137018203735, + -0.2910386919975281, + -0.10719436407089233, + -0.4741843640804291, + 0.18213149905204773, + 0.27579787373542786, + 0.28895407915115356, + -0.6325862407684326, + -0.62404465675354 + ], + [ + -0.598300039768219, + 0.04896893352270126, + -0.20403441786766052, + -2.139519214630127, + -1.08447265625, + -0.4189446270465851, + -0.6346327662467957, + -0.14519792795181274, + -0.1999586522579193, + 0.005803830921649933, + 0.30325591564178467, + -0.2296145260334015, + -0.08870476484298706, + -0.24513129889965057, + -0.06902675330638885, + -0.4448564350605011, + 0.0938514918088913, + 0.297969788312912, + 0.36479681730270386, + -0.5445393919944763, + -0.7320643663406372 + ], + [ + -0.4095943570137024, + -0.0890439972281456, + -0.2352524697780609, + -2.2584950923919678, + -0.8821102976799011, + -0.8211241960525513, + -0.7584159970283508, + -0.21534979343414307, + -0.08541803807020187, + 0.11774807423353195, + 0.1859683394432068, + -0.11890029907226562, + 0.007693499326705933, + -0.2932388186454773, + -0.06781680881977081, + -0.5304293036460876, + 0.2986355423927307, + 0.17837464809417725, + 0.29122135043144226, + -0.6381885409355164, + -0.6429785490036011 + ], + [ + -0.6171810626983643, + -0.15433329343795776, + -0.5142687559127808, + -2.111546039581299, + -1.4152092933654785, + -0.9933813810348511, + -1.0423638820648193, + -0.05100378394126892, + -0.2275928407907486, + 0.14460963010787964, + 0.304403156042099, + -0.596270740032196, + 0.14442339539527893, + -0.6126604676246643, + 0.026041459292173386, + -0.433432936668396, + 0.12705498933792114, + 0.3901751637458801, + 0.37796664237976074, + -0.5136858224868774, + -0.45762547850608826 + ], + [ + -0.5448775291442871, + -0.18337921798229218, + -0.5216808319091797, + -1.9630666971206665, + -1.3005261421203613, + -0.9385174512863159, + -0.9818581938743591, + -0.07276719808578491, + -0.2240539789199829, + 0.14343151450157166, + 0.40550243854522705, + -0.5454055070877075, + 0.028278887271881104, + -0.5270435214042664, + 0.012806683778762817, + -0.42532992362976074, + 0.23145246505737305, + 0.3342704772949219, + 0.35551169514656067, + -0.5486299395561218, + -0.4504261910915375 + ], + [ + -0.6717805862426758, + -0.19726186990737915, + -0.6103730201721191, + -2.224104642868042, + -1.4604973793029785, + -0.4195825755596161, + -1.0958538055419922, + -0.11987295746803284, + -0.228374645113945, + 0.08262364566326141, + -0.03332985192537308, + -0.440670907497406, + -0.14603298902511597, + -0.7307149767875671, + -0.030686799436807632, + -0.4650437533855438, + 0.13531675934791565, + 0.2419317364692688, + 0.48018038272857666, + -0.787667989730835, + -0.016932129859924316 + ], + [ + -0.6525436043739319, + -0.12835577130317688, + -0.7270898818969727, + -1.7835642099380493, + -1.3232734203338623, + -1.0632609128952026, + -0.8576175570487976, + -0.10045084357261658, + -0.16924014687538147, + 0.00046965479850769043, + 0.3871758282184601, + -0.3993864059448242, + -0.0893002450466156, + -0.5050527453422546, + -0.16586345434188843, + -0.44161608815193176, + 0.38248682022094727, + 0.2313019037246704, + 0.18844608962535858, + -0.2711566388607025, + -0.5559765100479126 + ] + ], + "generated_utilities": [ + 2.0, + 0.061786800622940063, + 0.07495661079883575, + 0.061137303709983826, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.17512565851211548, + 2.0, + 2.0, + 2.0, + 0.10078734159469604, + 2.0, + 2.0, + 0.06518048048019409, + 2.0, + 2.0, + 0.05268773436546326, + 0.056524619460105896, + 0.07100173830986023, + 0.16304254531860352, + 2.0, + 0.07095202803611755 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -1.7685134410858154, + 0.4561518430709839, + 0.8554723262786865, + -2.9925076961517334, + -1.652122139930725, + -0.9696183204650879, + 1.440989375114441, + -0.1663060486316681, + 0.005464707035571337, + -0.08078178763389587, + 0.5969918370246887, + -1.7046172618865967, + 0.3472420275211334, + 0.7052499651908875, + -0.026682782918214798, + 0.013620201498270035, + 0.002864290028810501, + 1.5629743337631226, + 0.06437517702579498, + -0.20232005417346954, + 0.06808048486709595 + ], + [ + -1.571894645690918, + 0.49223509430885315, + 1.2358425855636597, + -2.7074813842773438, + -1.6053731441497803, + -0.9737302660942078, + 1.2407286167144775, + -0.41521748900413513, + 0.27269765734672546, + -0.08917579054832458, + 0.4081580936908722, + -1.3627166748046875, + 0.07880738377571106, + 0.7477485537528992, + -0.2180848866701126, + -0.13109977543354034, + -0.3875364661216736, + 1.8194886445999146, + -0.06430798768997192, + 0.07869948446750641, + -0.12621839344501495 + ], + [ + -1.7744590044021606, + 0.367733359336853, + 1.3591725826263428, + -3.3725903034210205, + -1.8434977531433105, + -0.8745664954185486, + 1.6967558860778809, + -0.44676536321640015, + -0.6177129149436951, + 0.03244173526763916, + 0.4171296954154968, + -1.4895254373550415, + 0.9296896457672119, + 0.15846145153045654, + -0.006281482055783272, + 0.6454329490661621, + 0.6225314140319824, + 1.6578954458236694, + 0.3061527609825134, + -0.41300463676452637, + -0.2527916431427002 + ], + [ + -1.7233139276504517, + 0.020191192626953125, + 1.0188184976577759, + -2.873849868774414, + -2.027033567428589, + -1.1650824546813965, + 2.0792155265808105, + -0.8799198865890503, + 0.6924232840538025, + -0.7650182247161865, + -0.04520058631896973, + -2.468881130218506, + -0.13257521390914917, + 0.6857031583786011, + -0.10177457332611084, + -0.7609032988548279, + -0.08249430358409882, + 1.7503252029418945, + 0.1621951013803482, + -0.22467197477817535, + 0.43942898511886597 + ], + [ + -2.3507251739501953, + 0.16114166378974915, + 1.5912489891052246, + -2.1838369369506836, + -2.061230182647705, + -1.7918524742126465, + 0.8407515287399292, + -0.3446531295776367, + -0.7727420926094055, + -0.2617931067943573, + 0.6622774004936218, + -1.05295991897583, + 0.13840271532535553, + 1.2008695602416992, + -0.509428858757019, + 0.7052024006843567, + -0.6614989042282104, + 2.0029079914093018, + -0.82997065782547, + -0.9390356540679932, + 0.7515151500701904 + ], + [ + -1.7504347562789917, + -0.13171464204788208, + 0.7620843648910522, + -3.4511537551879883, + -2.5023341178894043, + -0.8595067858695984, + 1.6361833810806274, + -0.4263937175273895, + 0.005345506593585014, + -0.8147202730178833, + 0.4142894744873047, + -0.8735606670379639, + -0.4809738099575043, + 0.9412479996681213, + -0.6222366690635681, + -0.8843234777450562, + -0.8842994570732117, + 2.4228873252868652, + 0.6301752328872681, + 0.727215051651001, + 0.5050253868103027 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.7685134410858154, + -0.04630154371261597, + -0.002802610397338867, + -1.9925076961517334, + -0.6521221399307251, + -0.013848721981048584, + 0.0, + 0.014648675918579102, + -0.0837627500295639, + -0.04761720821261406, + -0.024705946445465088, + -0.7046172618865967, + 0.0914837121963501, + 0.0, + 0.04683725908398628, + 0.048148274421691895, + 0.03868643194437027, + 0.5629743337631226, + -0.043637245893478394, + -0.08927876502275467, + 0.0 + ], + [ + -0.7685134410858154, + 0.1268729865550995, + -0.0878140926361084, + -1.9925076961517334, + -0.6521221399307251, + -0.08497345447540283, + 0.0, + -0.23278282582759857, + 0.1505891978740692, + -0.0029508620500564575, + -0.03223156929016113, + -0.7046172618865967, + -0.18528565764427185, + 0.0, + 0.009110454469919205, + 0.006383728235960007, + -0.012130538001656532, + 0.5629743337631226, + 0.23608635365962982, + -0.06009632349014282, + 0.0 + ], + [ + -0.7685134410858154, + 0.6076252460479736, + 0.2745842933654785, + -1.9925076961517334, + -0.6521221399307251, + 0.03038167953491211, + 0.0, + 0.33581337332725525, + 0.35011863708496094, + -0.06728862971067429, + -0.4030081629753113, + -0.7046172618865967, + -0.44776007533073425, + 0.0, + -0.01595483534038067, + -0.040477462112903595, + 0.28290560841560364, + 0.5629743337631226, + -0.42285358905792236, + -0.11823103576898575, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.881978750228882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.410499930381775, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.1361609697341919 + ], + [ + -1.5711963176727295, + 0.5461994409561157, + 0.8384940028190613, + -2.960416078567505, + -1.9055238962173462, + -0.7765143513679504, + 1.4308912754058838, + 0.10734349489212036, + 0.011199884116649628, + -0.028424743562936783, + 0.5893871784210205, + -1.7020739316940308, + 0.48640453815460205, + 0.9950923323631287, + 0.20675025880336761, + 0.11047534644603729, + 0.09110014140605927, + 1.3974875211715698, + -0.21079741418361664, + -0.3005509674549103, + 0.09192514419555664 + ], + [ + -2.12906813621521, + 0.3435881733894348, + 0.7164700627326965, + -2.750021457672119, + -1.3681434392929077, + -0.8937081694602966, + 1.6475064754486084, + -0.1447279453277588, + -0.44103574752807617, + -0.5038806200027466, + 0.9006609320640564, + -1.8256494998931885, + 0.433420330286026, + 0.3506351411342621, + 0.4619242548942566, + -0.3905109167098999, + -0.01803053542971611, + 1.9505069255828857, + -0.339522123336792, + -0.20715753734111786, + -0.410531610250473 + ], + [ + -1.194382667541504, + 0.9652969837188721, + 0.41722428798675537, + -2.6203644275665283, + -1.139541506767273, + -1.5412108898162842, + 1.667648434638977, + -0.5256404876708984, + 0.17716272175312042, + 0.30056339502334595, + 0.09983009099960327, + -1.8219844102859497, + -0.23339274525642395, + 0.5763673782348633, + -0.4285883605480194, + 0.1476086974143982, + -0.3515554666519165, + 1.2458031177520752, + 0.3135184645652771, + -0.33711761236190796, + -0.09264840185642242 + ], + [ + -2.0614774227142334, + -0.4080510139465332, + 0.5476135015487671, + -3.0662269592285156, + -1.497420310974121, + -0.11606889963150024, + 1.03195321559906, + -0.3406473994255066, + 0.12844708561897278, + -0.4317864775657654, + 1.035879135131836, + -0.7729326486587524, + 1.0862895250320435, + 0.16315871477127075, + -0.14380013942718506, + -0.8498450517654419, + 0.9780882000923157, + 1.0358695983886719, + -0.5399289727210999, + -0.10386796295642853, + 0.8144621849060059 + ] + ], + "predicted_scores": [ + 8.302830696105957, + 8.272442817687988, + 8.218302726745605, + 8.129446029663086, + 7.833746433258057, + 7.824558734893799, + 7.328679084777832, + 7.000413417816162 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "794181626", + "selected_index": 0, + "state_hash": "4043b3c43ca4f115e1ea4fbcd98c7d08f797df2efbfe6b67bcffa5592a8aa36d", + "stored_base_utility": 0.17512565851211548, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.054018765687942505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054018765687942505 + }, + { + "contact_quality": null, + "progress": 5.517899990081787e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 5.517899990081787e-05 + }, + { + "contact_quality": null, + "progress": 0.058512330055236816, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058512330055236816 + }, + { + "contact_quality": null, + "progress": 0.00018827617168426514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00018827617168426514 + }, + { + "contact_quality": null, + "progress": 0.00025878846645355225, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00025878846645355225 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 7.644295692443848e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 7.644295692443848e-05 + } + ], + "candidate_progress": [ + 1.0, + 0.054018765687942505, + 5.517899990081787e-05, + 0.058512330055236816, + 0.00018827617168426514, + 0.00025878846645355225, + 1.0, + 7.644295692443848e-05 + ], + "candidate_source_chart_ids": [ + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-58950fed04d203b97ab9", + "generated_tangents": [ + [ + -0.1778576821088791, + 0.0036302395164966583, + 1.1676632165908813, + -0.7546417713165283, + -1.697277307510376, + -0.2011466920375824, + 0.298480749130249, + -0.07482165098190308, + -0.12784148752689362, + 0.13857874274253845, + -0.21941524744033813, + -1.5108556747436523, + 0.477994829416275, + 0.13235871493816376, + 0.2661297619342804, + 0.06012414023280144, + -0.3290677070617676, + -0.36622101068496704, + -0.9605446457862854, + -0.30346766114234924, + -0.2351449429988861 + ], + [ + -0.28704923391342163, + 0.09078311920166016, + 1.114987850189209, + -0.7498209476470947, + -1.8012349605560303, + -0.28389838337898254, + 0.18818894028663635, + -0.037162914872169495, + -0.0017223749309778214, + 0.14139193296432495, + -0.08193399757146835, + -1.5852880477905273, + 0.19839653372764587, + 0.2623569071292877, + 0.2278982251882553, + 0.01609227992594242, + -0.08104647696018219, + -0.13260115683078766, + -0.7025322318077087, + -0.2369544357061386, + -0.14419075846672058 + ], + [ + -0.2206813246011734, + 0.026308827102184296, + 1.2314008474349976, + -0.7984839677810669, + -1.924837350845337, + -0.368415504693985, + 0.1726488173007965, + -0.1132161021232605, + -0.023435836657881737, + 0.05976184457540512, + -0.1202930435538292, + -1.4768600463867188, + 0.1491192877292633, + 0.24533262848854065, + 0.22463300824165344, + -0.023225639015436172, + -0.18822930753231049, + -0.13598276674747467, + -0.7937994599342346, + -0.10444749891757965, + -0.28017333149909973 + ], + [ + -0.4899563789367676, + 0.043576642870903015, + 1.2521061897277832, + -0.92316734790802, + -1.7282285690307617, + -0.0303565114736557, + 0.5548646450042725, + -0.08239397406578064, + -0.07094340771436691, + 0.15895822644233704, + -0.1991833746433258, + -1.516670823097229, + -0.07032427191734314, + 0.3291274309158325, + 0.23166821897029877, + 0.09363971650600433, + -0.19648267328739166, + -0.3459843397140503, + -0.6647997498512268, + -0.04420539736747742, + 0.18072429299354553 + ], + [ + -0.1396852284669876, + 0.001727461814880371, + 1.2981458902359009, + -0.7096518278121948, + -1.8154969215393066, + -0.21825021505355835, + 0.14054328203201294, + -0.07004091143608093, + -0.13817119598388672, + 0.07811032235622406, + -0.22212743759155273, + -1.3904515504837036, + 0.5396687984466553, + -0.12960778176784515, + 0.19688141345977783, + 0.14898556470870972, + -0.3176721930503845, + -0.3955593705177307, + -0.9047334790229797, + -0.3793342411518097, + -0.38841238617897034 + ], + [ + -0.18346960842609406, + 0.04333057999610901, + 1.108190655708313, + -0.7183141708374023, + -1.9171998500823975, + -0.12311094999313354, + 0.10123905539512634, + -0.02744145691394806, + -0.0041778963059186935, + 0.109732985496521, + -0.13936829566955566, + -1.401587963104248, + 0.2145768702030182, + 0.16982248425483704, + 0.22343409061431885, + 0.0016652271151542664, + -0.1294211596250534, + -0.15836508572101593, + -0.742404043674469, + -0.14056962728500366, + -0.22674208879470825 + ], + [ + -0.15506576001644135, + 0.031700484454631805, + 1.0520659685134888, + -0.7341622114181519, + -1.780832052230835, + -0.44813862442970276, + 0.11466643214225769, + -0.002584487199783325, + -0.00038519036024808884, + 0.15540660917758942, + -0.10375405102968216, + -1.5776569843292236, + 0.21429434418678284, + 0.1920042335987091, + 0.33942121267318726, + 0.002302626147866249, + -0.13589848577976227, + -0.14653809368610382, + -0.8399917483329773, + -0.07796430587768555, + -0.07958358526229858 + ], + [ + -0.20411308109760284, + 0.02717725932598114, + 1.1939626932144165, + -0.866493821144104, + -1.8202178478240967, + -0.44825616478919983, + -0.0890854001045227, + -0.22147828340530396, + -0.02458440326154232, + 0.012295819818973541, + -8.651614189147949e-05, + -1.3786108493804932, + 0.38345596194267273, + -0.01583552360534668, + -0.014947537332773209, + 0.07425405085086823, + -0.43137431144714355, + -0.19424985349178314, + -0.73204106092453, + 0.02016991376876831, + -0.2934549152851105 + ] + ], + "generated_utilities": [ + 2.0, + 0.054018765687942505, + 5.517899990081787e-05, + 0.058512330055236816, + 0.00018827617168426514, + 0.00025878846645355225, + 2.0, + 7.644295692443848e-05 + ], + "group_id": "ms-58950fed04d203b97ab9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20183078944683075, + 0.18959736824035645, + 2.0, + 0.0385412722826004, + 0.15877258777618408, + 0.03365510702133179, + 0.17151890695095062, + 0.13845136761665344, + 0.11539244651794434, + 0.1388089954853058, + 0.15365298092365265, + 0.0967668741941452, + 0.0908290445804596, + 0.06094485521316528, + 0.0377749502658844, + 0.028643101453781128 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.028361663222312927, + -0.489355206489563, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.16713106632232666, + 0.0, + 0.5545767545700073, + -0.18274790048599243, + 0.2772042155265808, + -0.45601847767829895, + -1.3811633586883545, + -0.1571006327867508, + 0.0, + -0.004826217889785767, + 0.2916078567504883, + 0.34807145595550537, + -0.37882840633392334, + 0.08759653568267822, + -0.07658697664737701, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.797813892364502, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.654926061630249, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4572639465332031 + ], + [ + 0.17793144285678864, + 0.008656054735183716, + 2.343496322631836, + -1.9455183744430542, + -2.4697093963623047, + 1.0036901235580444, + 2.5124316215515137, + -0.29308056831359863, + -0.02035735547542572, + -0.059014320373535156, + -0.5863867402076721, + -2.254472255706787, + 0.3354390263557434, + 1.381989598274231, + -0.27729421854019165, + 0.5023907423019409, + -0.8654648661613464, + 0.39588332176208496, + -0.5682056546211243, + 0.20400357246398926, + 0.48529666662216187 + ], + [ + -0.5227808952331543, + 0.12448178231716156, + 2.4855635166168213, + -1.6602094173431396, + -2.3857760429382324, + 0.323077529668808, + 1.9053025245666504, + -0.698577344417572, + 0.619890034198761, + 0.3500756621360779, + -0.1476593315601349, + -2.8024404048919678, + -0.21084021031856537, + 1.0928113460540771, + -0.3938775062561035, + 0.4228009581565857, + -0.39060449600219727, + -0.31665533781051636, + -0.9205541610717773, + 0.03774743899703026, + 0.3333621621131897 + ], + [ + 0.10275222361087799, + 0.08925776183605194, + 2.0934102535247803, + -1.379586935043335, + -3.3245811462402344, + 0.6494860649108887, + 1.990661382675171, + -0.021198302507400513, + 0.4400750398635864, + 0.39176517724990845, + -0.8639137148857117, + -1.981496810913086, + 0.01414397917687893, + 1.2019504308700562, + -0.7839368581771851, + 0.108054518699646, + -0.17177742719650269, + -0.36239588260650635, + -0.5078206062316895, + 0.4236416518688202, + 0.25084057450294495 + ], + [ + -0.8110343217849731, + -0.11216431856155396, + 2.8897132873535156, + -1.319732427597046, + -3.1141648292541504, + 0.416613906621933, + 2.3244988918304443, + -0.6579729914665222, + -0.39938226342201233, + -0.08646327257156372, + -0.3429824113845825, + -2.4165658950805664, + -0.6488300561904907, + 1.4793217182159424, + -0.8628695011138916, + 0.6845848560333252, + -0.7794754505157471, + -0.014068186283111572, + -0.27444708347320557, + 0.8030790686607361, + 0.08975183963775635 + ], + [ + -0.23147638142108917, + -0.26693814992904663, + 2.8227345943450928, + -2.3573813438415527, + -3.3905396461486816, + 0.38961973786354065, + 1.9831907749176025, + -1.2505497932434082, + 0.00624750554561615, + 0.9689428210258484, + 0.11981219053268433, + -1.5865073204040527, + 0.4901374876499176, + 0.6158997416496277, + 0.3767610788345337, + -0.025944411754608154, + -1.4681568145751953, + 0.48217374086380005, + -1.5973502397537231, + -0.7199738621711731, + 0.5119494795799255 + ], + [ + -0.19429966807365417, + 0.7059361934661865, + 3.2694599628448486, + -2.656050682067871, + -2.5502617359161377, + -0.23801833391189575, + 1.628230333328247, + 0.36624830961227417, + -0.08827590942382812, + -0.2098810374736786, + -1.1745270490646362, + -2.2196781635284424, + 0.44475334882736206, + 0.48411858081817627, + -0.25864526629447937, + 0.2009485512971878, + -0.9008907079696655, + -0.3571908473968506, + -0.16894888877868652, + -0.6256314516067505, + -0.4078131914138794 + ], + [ + 0.22205571830272675, + 0.6026778221130371, + 1.8579424619674683, + -2.404672384262085, + -3.728808641433716, + -0.19806396961212158, + 2.8292319774627686, + -0.2435564398765564, + 0.7903643846511841, + -0.376501202583313, + -1.0294175148010254, + -1.7672805786132812, + 0.5992995500564575, + 2.044847249984741, + 0.0011147260665893555, + 0.18739356100559235, + -0.952850341796875, + 0.8702673316001892, + 0.038269996643066406, + -0.4372258484363556, + -0.37835830450057983 + ], + [ + -0.3783646821975708, + 0.08899453282356262, + 2.721517324447632, + -2.3160932064056396, + -2.0679585933685303, + 0.549463152885437, + 3.141833543777466, + 0.09782099723815918, + 0.5830706357955933, + 0.7134941816329956, + 0.2986952066421509, + -2.9980335235595703, + -0.6391466856002808, + 1.075132131576538, + -0.7696593403816223, + 0.4343666732311249, + -0.911625862121582, + -0.31454378366470337, + -1.5935450792312622, + -0.8993679881095886, + -0.3811922073364258 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.1209426149725914, + 0.10675475001335144, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.0875239372253418, + 0.0, + -0.0486987829208374, + 0.06580346822738647, + 0.24258655309677124, + -0.20229026675224304, + -1.3811633586883545, + 0.4185009002685547, + 0.0, + -0.1431063860654831, + 0.15154658257961273, + 0.17190229892730713, + 0.10405468940734863, + 0.036865830421447754, + -0.1988307237625122, + 0.0 + ] + ], + "predicted_scores": [ + 8.322562217712402, + 8.284337043762207, + 8.195613861083984, + 8.095207214355469, + 8.043414115905762, + 7.969996929168701, + 7.789015293121338, + 7.55466365814209 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113371261", + "selected_index": 0, + "state_hash": "1b9018834a77531260bf8dfa74a9832916fe215bda78b5367b061bb4e0ce2333", + "stored_base_utility": 0.20183078944683075, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.01641605794429779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01641605794429779 + }, + "base_progress": 0.01641605794429779, + "base_success": false, + "base_utility": 0.01641605794429779, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11001592874526978, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11001592874526978 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 7.614493370056152e-06, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 7.614493370056152e-06 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.11001592874526978, + 1.0, + 1.0, + 1.0, + 7.614493370056152e-06, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-9f61e09c05b4c8f74366", + "ms-9f61e09c05b4c8f74366", + "ms-74704688fb63757619e1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + false, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-651bc0395d78d2819158", + "generated_tangents": [ + [ + -0.28055286407470703, + -0.12427914142608643, + 0.7458057403564453, + -0.8203845024108887, + -1.9704015254974365, + -0.5069563388824463, + 0.5400291681289673, + 0.056122586131095886, + -0.19652147591114044, + -0.3618527948856354, + 0.003855101764202118, + -1.6388840675354004, + 0.09804847836494446, + -0.4239356517791748, + -0.07491517066955566, + -0.09474967420101166, + 0.18653123080730438, + 0.7591179013252258, + -0.27929621934890747, + -0.4226912260055542, + 0.41058146953582764 + ], + [ + -0.25913751125335693, + -0.15451189875602722, + 0.857841968536377, + -0.7576884031295776, + -1.992818832397461, + -0.5041130781173706, + 0.3823484778404236, + 0.11783941835165024, + -0.09992609173059464, + -0.3692428767681122, + -0.042962513864040375, + -1.6736167669296265, + 0.15392783284187317, + -0.5406212210655212, + -0.04191606119275093, + -0.1567104160785675, + 0.16465872526168823, + 0.7287470102310181, + -0.266432523727417, + -0.49043866991996765, + 0.2968205213546753 + ], + [ + -0.14534591138362885, + -0.1459084153175354, + 1.0194928646087646, + -0.7109029293060303, + -1.8421368598937988, + 0.12687039375305176, + 0.4298076629638672, + -0.02768939733505249, + -0.0852721706032753, + -0.32993319630622864, + 0.12060986459255219, + -1.4413669109344482, + -0.022756069898605347, + -0.3273512125015259, + -0.018921706825494766, + -0.2069738656282425, + 0.13648608326911926, + 0.6646034717559814, + -0.16806504130363464, + -0.3461107611656189, + 0.047406673431396484 + ], + [ + -0.19669298827648163, + -0.16399496793746948, + 0.7689091563224792, + -0.4982009828090668, + -1.6042232513427734, + -0.23108428716659546, + 0.8815572261810303, + 0.11060038208961487, + -0.0574159249663353, + -0.46911656856536865, + 0.22009345889091492, + -1.443450689315796, + 0.21278473734855652, + -0.31276553869247437, + 0.10376255214214325, + -0.2275824397802353, + 0.2820473313331604, + 0.47658276557922363, + -0.14072933793067932, + -0.17545510828495026, + 0.009726181626319885 + ], + [ + -0.14645330607891083, + -0.15107746422290802, + 0.8798984289169312, + -0.7744108438491821, + -1.9667823314666748, + -0.6426529884338379, + 0.3072587549686432, + 0.010424420237541199, + -0.11162305623292923, + -0.3278885781764984, + -0.09279123693704605, + -1.683987021446228, + 0.20445826649665833, + -0.5105334520339966, + -0.033252593129873276, + -0.09740671515464783, + 0.2381344586610794, + 0.7486972212791443, + -0.14820578694343567, + -0.28766438364982605, + 0.19942304491996765 + ], + [ + -0.2849956750869751, + -0.12620314955711365, + 0.9757925271987915, + -0.6153578758239746, + -1.86106538772583, + 0.03962574899196625, + 0.4314451217651367, + 0.059229254722595215, + -0.020434828475117683, + -0.47481411695480347, + 0.016673069447278976, + -1.5006448030471802, + -0.29565802216529846, + -0.21682687103748322, + 0.12796173989772797, + -0.09710533916950226, + 0.11769160628318787, + 0.6002511382102966, + -0.2779631018638611, + -0.4966781735420227, + 0.055744290351867676 + ], + [ + -0.32628291845321655, + -0.16823628544807434, + 0.9088062644004822, + -0.6385703682899475, + -1.7459666728973389, + -3.316998481750488e-05, + 0.257204532623291, + 0.09736340492963791, + 0.07916215062141418, + -0.42485809326171875, + 0.018571551889181137, + -1.442797064781189, + -0.1283630132675171, + -0.3043360114097595, + 0.022733233869075775, + 0.0018946034833788872, + 0.27441564202308655, + 0.6587740778923035, + -0.2499735951423645, + -0.7256455421447754, + 0.12833906710147858 + ], + [ + -0.20989848673343658, + -0.09311650693416595, + 0.9274475574493408, + -0.9091185331344604, + -2.068138837814331, + -0.6280347108840942, + 0.29668983817100525, + 0.04231889545917511, + -0.07097811251878738, + -0.27199476957321167, + 0.0860631912946701, + -1.5460503101348877, + 0.16375216841697693, + -0.5114734172821045, + 0.004433806985616684, + -0.11124520003795624, + 0.3245386481285095, + 0.8809368014335632, + -0.11035174131393433, + -0.3040013313293457, + 0.12515383958816528 + ] + ], + "generated_utilities": [ + 0.11001592874526978, + 2.0, + 2.0, + 2.0, + 7.614493370056152e-06, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-651bc0395d78d2819158", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 7.152557373046875e-07, + 2.0, + 0.0, + 0.0, + 0.18490922451019287, + 0.0, + 0.18801423907279968, + 2.0, + 0.18292954564094543, + 0.06743006408214569, + 2.0, + 0.08718416094779968, + 0.0016145259141921997, + 2.0, + 0.0, + 0.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.13822472095489502, + 0.052123308181762695, + 1.2542681694030762, + -0.5262794494628906, + -1.515254020690918, + -0.022995710372924805, + 0.0, + 0.07829460501670837, + 0.056280869990587234, + -0.006563127040863037, + 0.00594717264175415, + -1.0464823246002197, + 0.028334707021713257, + 0.0, + 0.04214516654610634, + -0.0032968670129776, + 0.025241270661354065, + 1.0898823738098145, + -0.09987176954746246, + 0.009208589792251587, + 0.0 + ], + [ + -1.138224720954895, + -0.12241673469543457, + 2.254268169403076, + -1.5262794494628906, + -2.515254020690918, + -0.4036164879798889, + 2.5164785385131836, + -0.48240265250205994, + 0.09632497280836105, + -1.0002868175506592, + 0.6668505668640137, + -2.0464823246002197, + 0.5158075094223022, + 0.48990970849990845, + 0.02257649227976799, + -0.23748205602169037, + -0.2213379293680191, + 2.0898823738098145, + 0.1738651841878891, + -0.37470543384552, + -0.06145218014717102 + ], + [ + -1.0695267915725708, + -0.1775755137205124, + 2.1683619022369385, + -1.4577077627182007, + -2.7917678356170654, + -0.632304310798645, + 2.3911960124969482, + -0.4637458324432373, + 0.35316434502601624, + -0.8419650793075562, + 0.8253071308135986, + -1.9648017883300781, + 0.2602742314338684, + 0.4741981625556946, + 0.17542533576488495, + -0.15374207496643066, + -0.4857789874076843, + 2.119079828262329, + 0.44966256618499756, + -0.3760189712047577, + 0.18533895909786224 + ], + [ + -0.7955029010772705, + 0.23637625575065613, + 1.9978463649749756, + -1.8726613521575928, + -2.2562904357910156, + -0.3048732876777649, + 2.261690378189087, + -0.6620026230812073, + 0.06043636053800583, + -1.091504693031311, + 1.038455605506897, + -1.9805728197097778, + 0.7370907068252563, + 0.5352527499198914, + 0.26298987865448, + -0.2739749252796173, + 0.14720074832439423, + 2.0173659324645996, + -0.16336800158023834, + -0.5858949422836304, + -0.1647021770477295 + ], + [ + -1.6335437297821045, + -0.08274295926094055, + 1.833235502243042, + -1.758880853652954, + -2.0250279903411865, + -0.8149738311767578, + 2.4238271713256836, + -0.33182430267333984, + -0.16191768646240234, + -1.4929053783416748, + 0.8467401266098022, + -2.4791038036346436, + 0.2757382392883301, + 0.16471439599990845, + 0.18122689425945282, + -0.2995648682117462, + -0.34926944971084595, + 2.063732147216797, + 0.3726923167705536, + -0.259366512298584, + 0.36020517349243164 + ], + [ + -0.9912663698196411, + -0.19695568084716797, + 2.342681646347046, + -2.1077044010162354, + -3.031508684158325, + -0.6591198444366455, + 2.5516788959503174, + -0.8631293773651123, + 0.2481168806552887, + -1.107275366783142, + 0.5796921253204346, + -2.479954719543457, + 1.0449233055114746, + 0.40612131357192993, + -0.12985309958457947, + -0.18829946219921112, + -0.6196849346160889, + 1.6295442581176758, + 0.4197757840156555, + 0.02911210060119629, + 0.5378711223602295 + ], + [ + -0.6689683198928833, + -0.5626275539398193, + 1.738625407218933, + -1.919468879699707, + -2.6463799476623535, + -0.32590651512145996, + 2.287154197692871, + -0.9975759983062744, + 0.7224722504615784, + -1.427150845527649, + 0.572014331817627, + -2.2473490238189697, + 0.8538937568664551, + 0.0030456185340881348, + -0.1059482991695404, + -0.003734409809112549, + -0.2650519907474518, + 2.554786205291748, + -0.24461163580417633, + -0.6881535649299622, + 0.5301691293716431 + ], + [ + -1.5373890399932861, + 0.42636638879776, + 2.9180266857147217, + -2.126905679702759, + -1.825490117073059, + -0.6280819773674011, + 2.593716621398926, + -0.6465503573417664, + 0.5798410177230835, + -0.5376753807067871, + 0.5435326099395752, + -2.610151767730713, + 1.0700397491455078, + 0.7953996658325195, + 0.5072687864303589, + -0.02665230631828308, + -0.6953800320625305, + 2.0521392822265625, + 0.751386821269989, + -0.7565212249755859, + -0.5263289213180542 + ], + [ + -0.7768699526786804, + -0.18900753557682037, + 1.7192409038543701, + -2.2835047245025635, + -1.713951826095581, + 0.004747450351715088, + 3.2709672451019287, + -1.1866822242736816, + 0.38759592175483704, + -0.7857802510261536, + 1.539168119430542, + -1.3250088691711426, + 0.8453556299209595, + 0.5691420435905457, + -0.701140820980072, + -0.6115058660507202, + -0.8829987645149231, + 1.5742011070251465, + -0.3939879536628723, + 0.45414644479751587, + -0.23744438588619232 + ] + ], + "predicted_scores": [ + 9.421090126037598, + 9.321776390075684, + 8.89632797241211, + 8.840374946594238, + 8.587821960449219, + 8.185254096984863, + 8.08885669708252, + 7.968245983123779 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1483645336", + "selected_index": 0, + "state_hash": "857c28f61d13d7ee5a59956a87579f6c17f6c601c1680e574691e095e373d439", + "stored_base_utility": 7.152557373046875e-07, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15461373329162598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15461373329162598 + }, + "base_progress": 0.15461373329162598, + "base_success": false, + "base_utility": 0.15461373329162598, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06999577581882477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06999577581882477 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03132614493370056, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03132614493370056 + }, + { + "contact_quality": null, + "progress": 0.023272648453712463, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023272648453712463 + }, + { + "contact_quality": null, + "progress": 0.030094027519226074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030094027519226074 + }, + { + "contact_quality": null, + "progress": 0.04187875986099243, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04187875986099243 + }, + { + "contact_quality": null, + "progress": 0.0486045777797699, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0486045777797699 + } + ], + "candidate_progress": [ + 0.06999577581882477, + 1.0, + 1.0, + 0.03132614493370056, + 0.023272648453712463, + 0.030094027519226074, + 0.04187875986099243, + 0.0486045777797699 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9fad82fd288e4296e354", + "generated_tangents": [ + [ + -0.1998446136713028, + -0.13613846898078918, + 1.1954774856567383, + -0.19622287154197693, + -1.2181291580200195, + -0.09228421747684479, + -0.5131031274795532, + -0.09343284368515015, + 0.034097351133823395, + 0.5444074869155884, + -0.06611961126327515, + -1.2418341636657715, + -0.07458993792533875, + 0.09996648132801056, + -0.08733539283275604, + 0.02461063489317894, + 0.11653249710798264, + -0.5559351444244385, + -0.22178617119789124, + -0.018946915864944458, + 0.09585287421941757 + ], + [ + 0.10768678784370422, + -0.14886173605918884, + 1.122004747390747, + -0.30997174978256226, + -1.2846577167510986, + -0.4357328712940216, + -0.6459855437278748, + 0.028352245688438416, + 0.13504968583583832, + 0.46079155802726746, + 0.015527933835983276, + -1.360116720199585, + 0.17759987711906433, + -0.19781215488910675, + 0.034844718873500824, + 0.016100937500596046, + 0.08097121119499207, + -0.40727776288986206, + -0.3540729284286499, + -0.2883093059062958, + -0.13371184468269348 + ], + [ + 0.018745824694633484, + -0.2062220275402069, + 1.1598033905029297, + -0.1009679064154625, + -1.332383632659912, + -0.4028060734272003, + -0.9239940047264099, + 0.0017495453357696533, + 0.06034529209136963, + 0.5863930583000183, + -0.13289305567741394, + -1.2339847087860107, + 0.06810715794563293, + -0.2841215133666992, + -0.10978451371192932, + 0.14976097643375397, + 0.16245397925376892, + -0.4142678380012512, + -0.37755507230758667, + 0.04395031929016113, + 0.09613826125860214 + ], + [ + 0.1592612862586975, + -0.1608738750219345, + 1.1876754760742188, + -0.33564597368240356, + -1.2400803565979004, + -0.4422418773174286, + -0.6213474869728088, + 0.00048807263374328613, + 0.0908799022436142, + 0.4733821749687195, + 0.018075991421937943, + -1.3230524063110352, + 0.12521758675575256, + -0.26577723026275635, + 0.10867087543010712, + 0.09838718175888062, + 0.14024819433689117, + -0.38509494066238403, + -0.3966064453125, + -0.11265681684017181, + 0.008435025811195374 + ], + [ + 0.09822267293930054, + -0.14454904198646545, + 1.1871142387390137, + -0.0997898131608963, + -1.2865939140319824, + -0.485890656709671, + -0.9018321633338928, + -0.009793668985366821, + 0.08223867416381836, + 0.5608381032943726, + -0.08273427933454514, + -1.2424618005752563, + 0.03088495135307312, + -0.25332319736480713, + -0.0384976752102375, + 0.13081185519695282, + 0.16568070650100708, + -0.4293673038482666, + -0.37687379121780396, + 0.0026108622550964355, + 0.11234897375106812 + ], + [ + 0.10180012881755829, + -0.1550700068473816, + 1.188444972038269, + -0.25300154089927673, + -1.3772516250610352, + -0.4051682651042938, + -0.8206649422645569, + 0.031919628381729126, + 0.06656257808208466, + 0.5998708009719849, + 0.17621514201164246, + -1.5509395599365234, + -0.40087807178497314, + -0.2322075217962265, + -0.05952772870659828, + 0.1748443990945816, + 0.08963941037654877, + -0.37823981046676636, + -0.3356948494911194, + 0.21281248331069946, + 0.004816889762878418 + ], + [ + 0.2217009961605072, + -0.25093889236450195, + 1.0543618202209473, + -0.1766597479581833, + -1.1321866512298584, + -0.5494941473007202, + -0.55526202917099, + -0.0037053674459457397, + -0.013644784688949585, + 0.5054594874382019, + -0.29368942975997925, + -1.4012982845306396, + 0.36760565638542175, + -0.32580989599227905, + 0.1359534114599228, + 0.25731244683265686, + 0.052984341979026794, + -0.7190414667129517, + -0.3065682649612427, + -0.07599075138568878, + -0.3095443546772003 + ], + [ + 0.2995772361755371, + -0.19727033376693726, + 1.2040073871612549, + -0.15190288424491882, + -1.2154841423034668, + -0.327819287776947, + -0.9828824400901794, + 0.12254729866981506, + -0.12098874151706696, + 0.6633365750312805, + -0.18690431118011475, + -1.6477895975112915, + 0.1402067244052887, + 0.2056141048669815, + 0.04854009672999382, + 0.2965231239795685, + 0.13651099801063538, + -0.5691934823989868, + -0.36577552556991577, + 0.3864631652832031, + 0.024560481309890747 + ] + ], + "generated_utilities": [ + 0.06999577581882477, + 2.0, + 2.0, + 0.03132614493370056, + 0.023272648453712463, + 0.030094027519226074, + 0.04187875986099243, + 0.0486045777797699 + ], + "group_id": "ms-9fad82fd288e4296e354", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.19493231177330017, + 0.06623455882072449, + 2.0, + 0.15396031737327576, + 0.16184639930725098, + 0.025735855102539062, + 0.134642094373703, + 0.15606965124607086, + 0.12982505559921265, + 0.14038166403770447, + 0.09718823432922363, + 0.015769004821777344, + 0.09577034413814545, + 0.057760089635849, + 0.047366172075271606, + 0.035309627652168274 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.0622546412050724, + -0.09277644753456116, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + -0.005287081003189087, + 0.0, + -0.005681931972503662, + 0.03343720734119415, + 1.0536105632781982, + -0.00917971134185791, + -1.333775520324707, + 0.03716959059238434, + 0.0, + 0.019381746649742126, + 0.026990801095962524, + 0.21505475044250488, + -0.029777705669403076, + -0.14875531196594238, + -0.048092663288116455, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.959449768066406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.103323459625244, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.591951370239258 + ], + [ + -0.13909977674484253, + -0.26507115364074707, + 2.099381446838379, + -1.432984709739685, + -2.215683698654175, + 0.18431386351585388, + 2.42065691947937, + 0.3679453730583191, + -0.24726781249046326, + 2.055530071258545, + -0.4911385774612427, + -2.0511183738708496, + 0.04109602048993111, + 2.784976005554199, + 0.005537793040275574, + 0.05149966478347778, + 1.124292016029358, + -0.6457813382148743, + -1.226535677909851, + -0.28760284185409546, + 1.0270601511001587 + ], + [ + 0.30531278252601624, + -0.4336698055267334, + 2.6217315196990967, + -1.5083816051483154, + -2.5322670936584473, + 0.06784650683403015, + 2.3866119384765625, + -0.09851980209350586, + 0.2774229049682617, + 2.157179355621338, + -0.9796892404556274, + -2.8201849460601807, + 0.08135517686605453, + 2.818830728530884, + -0.6848310828208923, + 0.37913888692855835, + 0.7420558929443359, + -0.22611916065216064, + -1.5158166885375977, + 0.0614812970161438, + 1.3468891382217407 + ], + [ + 0.27444392442703247, + -0.22534464299678802, + 2.616544246673584, + -1.2606401443481445, + -2.0848917961120605, + -0.23496872186660767, + 2.1798665523529053, + 0.7968885898590088, + 0.26037585735321045, + 1.5473439693450928, + -0.233917236328125, + -1.867544412612915, + 0.48288947343826294, + 1.9728566408157349, + -0.4071806073188782, + 0.5528238415718079, + 1.6528295278549194, + -1.0936471223831177, + -1.062675952911377, + -0.5311015248298645, + 0.9778987765312195 + ], + [ + 0.5435068607330322, + 0.24469684064388275, + 2.6045663356781006, + -1.284223198890686, + -2.6095967292785645, + 0.9732104539871216, + 2.741159200668335, + 0.6031707525253296, + -0.5612202882766724, + 2.4774889945983887, + 0.1548280119895935, + -2.5516507625579834, + -0.0651187151670456, + 2.3864798545837402, + 0.130750373005867, + -0.1318313330411911, + 1.6559979915618896, + -1.0673086643218994, + -0.5477704405784607, + 0.18809780478477478, + 0.7076419591903687 + ], + [ + 0.31294435262680054, + 0.0263526514172554, + 3.0396268367767334, + -1.3165022134780884, + -1.8981198072433472, + 0.5049136281013489, + 2.31935715675354, + -0.4664327800273895, + -0.56757652759552, + 2.7719295024871826, + 0.21425002813339233, + -2.3925602436065674, + -0.26789456605911255, + 2.008218288421631, + -0.1879691481590271, + -0.34721577167510986, + 1.6638673543930054, + -0.30173182487487793, + -1.8719977140426636, + -0.8951039910316467, + 2.094865083694458 + ], + [ + 0.533157467842102, + 0.308005690574646, + 1.7770951986312866, + -0.6794775724411011, + -3.2953453063964844, + -0.15337547659873962, + 2.5854313373565674, + 0.07320006191730499, + -0.24033141136169434, + 2.5622174739837646, + -0.9168426990509033, + -1.5317132472991943, + -0.5942863821983337, + 2.2703769207000732, + -1.0709311962127686, + 0.6969255208969116, + 1.5304101705551147, + -0.7959219217300415, + -0.9165043830871582, + -0.8046965003013611, + 0.638719916343689 + ], + [ + 0.7393907904624939, + -0.13299384713172913, + 2.4281864166259766, + -1.861674189567566, + -3.3330533504486084, + -0.18516618013381958, + 1.6492295265197754, + -0.6932566165924072, + 0.06020718812942505, + 1.1755363941192627, + -0.016285300254821777, + -3.0671546459198, + -0.6052039861679077, + 2.6829426288604736, + 0.7021514177322388, + -0.477348268032074, + 0.466697633266449, + -0.9403301477432251, + -0.8690205812454224, + -1.1888904571533203, + 0.5360406041145325 + ], + [ + -0.0713023692369461, + -0.9776724576950073, + 1.2895376682281494, + -1.6489125490188599, + -1.4475497007369995, + -0.6822370886802673, + 3.14093017578125, + -0.5356632471084595, + -0.7523763179779053, + 2.766268253326416, + -0.8215572834014893, + -2.753042697906494, + 0.10929550230503082, + 2.570046901702881, + -0.34730663895606995, + 0.8763631582260132, + 1.3356308937072754, + -0.14330342411994934, + -1.1863114833831787, + -0.30006271600723267, + 0.7264611721038818 + ], + [ + 0.35943418741226196, + 0.22802312672138214, + 1.7158434391021729, + -1.4605827331542969, + -3.3450541496276855, + 0.23581582307815552, + 1.759531855583191, + 0.7012510299682617, + 0.0034519433975219727, + 1.4857903718948364, + -1.2866599559783936, + -2.1787426471710205, + 0.8807055354118347, + 1.9066205024719238, + -0.9918697476387024, + -0.32221919298171997, + 1.1986194849014282, + -1.187239646911621, + -0.5752912163734436, + -0.015687286853790283, + 1.775393009185791 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16282694041728973, + -0.1309112012386322, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + 0.15446226298809052, + 0.0, + -0.15669959783554077, + -0.08878482133150101, + 1.0536105632781982, + -0.09701520204544067, + -1.333775520324707, + 0.10988402366638184, + 0.0, + -0.06573517620563507, + -0.2108776718378067, + 0.21505475044250488, + -0.16805344820022583, + -0.14875531196594238, + 0.12529203295707703, + 0.0 + ] + ], + "predicted_scores": [ + 9.313835144042969, + 9.259336471557617, + 8.602334022521973, + 8.550065040588379, + 8.497767448425293, + 7.882334232330322, + 7.811857223510742, + 6.602944374084473 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1176943004", + "selected_index": 0, + "state_hash": "0d923438bf2b227e3521cb937d011f5bddf9b553a882853098b702ec31090614", + "stored_base_utility": 0.19493231177330017, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06781657040119171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06781657040119171 + }, + "base_progress": 0.06781657040119171, + "base_success": false, + "base_utility": 0.06781657040119171, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09167696535587311, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09167696535587311 + }, + { + "contact_quality": null, + "progress": 0.00022964179515838623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00022964179515838623 + }, + { + "contact_quality": null, + "progress": 0.0006267279386520386, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006267279386520386 + }, + { + "contact_quality": null, + "progress": 6.0498714447021484e-06, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 6.0498714447021484e-06 + }, + { + "contact_quality": null, + "progress": 0.00010013580322265625, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00010013580322265625 + }, + { + "contact_quality": null, + "progress": 1.5795230865478516e-06, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.5795230865478516e-06 + }, + { + "contact_quality": null, + "progress": 0.002237573266029358, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.002237573266029358 + }, + { + "contact_quality": null, + "progress": 0.09276536107063293, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09276536107063293 + } + ], + "candidate_progress": [ + 0.09167696535587311, + 0.00022964179515838623, + 0.0006267279386520386, + 6.0498714447021484e-06, + 0.00010013580322265625, + 1.5795230865478516e-06, + 0.002237573266029358, + 0.09276536107063293 + ], + "candidate_source_chart_ids": [ + "ms-dec13f2e40baaf5c8d40", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "generated_tangents": [ + [ + -0.07401131093502045, + -0.04983069375157356, + 1.0861337184906006, + -0.7663307189941406, + -1.9048068523406982, + -0.47873374819755554, + 0.08673620223999023, + -0.015595436096191406, + 0.059554338455200195, + 0.11692804843187332, + -0.01721273362636566, + -1.3845481872558594, + 0.2936886250972748, + -0.06758728623390198, + -0.11758345365524292, + 0.41535380482673645, + -0.004899993538856506, + -0.6369214057922363, + -0.5080356597900391, + -0.43489518761634827, + 0.021485134959220886 + ], + [ + -0.11963720619678497, + 0.023982718586921692, + 1.0234575271606445, + -0.956607460975647, + -1.6449265480041504, + -0.47221317887306213, + -0.10534799098968506, + 0.1610494703054428, + 0.13252530992031097, + -0.015598975121974945, + 0.043594684451818466, + -1.275907039642334, + 0.17958441376686096, + 0.04892846941947937, + -0.0540248341858387, + 0.3318483233451843, + 0.08719268441200256, + -0.8543687462806702, + -0.43198680877685547, + -0.03569546341896057, + -0.11463424563407898 + ], + [ + -0.1281290501356125, + 0.07537854462862015, + 1.1528681516647339, + -0.9410957098007202, + -1.7254095077514648, + -0.3934711515903473, + -0.19175148010253906, + 0.12348711490631104, + 0.08994673937559128, + -0.037045590579509735, + -0.03958366811275482, + -1.3818869590759277, + 0.09270164370536804, + -0.035149917006492615, + -0.04040056839585304, + 0.38400712609291077, + 0.028642013669013977, + -0.8570398688316345, + -0.5078204274177551, + -0.09519390761852264, + -0.0698556900024414 + ], + [ + -0.10154861211776733, + -0.05857856199145317, + 1.125746250152588, + -0.6748766899108887, + -1.605238676071167, + -0.7067629098892212, + -0.12017327547073364, + 0.0111217200756073, + 0.07350906729698181, + -0.033876873552799225, + -0.07184863835573196, + -1.2392839193344116, + 0.20886299014091492, + 0.2580723762512207, + 0.03831881284713745, + 0.2756733000278473, + 0.2141614854335785, + -0.6981217861175537, + -0.5911544561386108, + -0.3402017652988434, + -0.33051034808158875 + ], + [ + -0.1350162774324417, + 0.04171042889356613, + 0.7024990320205688, + -0.6363195776939392, + -1.4881393909454346, + -0.32491403818130493, + 0.13575655221939087, + 0.06681013107299805, + 0.05388261377811432, + -0.00012364238500595093, + 0.05719592794775963, + -1.1104090213775635, + 0.5736833810806274, + 0.1972261369228363, + -0.16631761193275452, + 0.17855453491210938, + 0.019907936453819275, + -0.5646957159042358, + 0.04457584023475647, + -0.1348414570093155, + 0.7787601947784424 + ], + [ + -0.09109653532505035, + -0.10604146867990494, + 1.1458743810653687, + -0.6595280170440674, + -1.6466548442840576, + -0.6165168285369873, + -0.24602818489074707, + -0.02194201946258545, + 0.030948543921113014, + -0.13117703795433044, + -0.23636949062347412, + -1.1983355283737183, + 0.08925655484199524, + 0.24316507577896118, + 0.10543453693389893, + 0.33899813890457153, + 0.18528787791728973, + -0.680165708065033, + -0.6733214259147644, + -0.20355458557605743, + -0.26392191648483276 + ], + [ + -0.14924006164073944, + 0.0018317140638828278, + 1.1730269193649292, + -0.9112308025360107, + -1.6417372226715088, + -0.4951991140842438, + -0.2560999393463135, + 0.004106879234313965, + 0.1028987318277359, + -0.035963691771030426, + 0.01761562004685402, + -1.3713557720184326, + 0.4157550036907196, + 0.2217365801334381, + 0.0577625073492527, + 0.36985331773757935, + 0.23366516828536987, + -0.949657142162323, + -0.6014523506164551, + 0.19395634531974792, + 0.06467265635728836 + ], + [ + -0.06714653968811035, + 0.0016189664602279663, + 1.1881128549575806, + -0.6453325748443604, + -1.7248032093048096, + -0.7292300462722778, + -0.04763680696487427, + 0.030370160937309265, + 0.06427609175443649, + 0.07249383628368378, + -0.13684776425361633, + -1.2338018417358398, + 0.1924196183681488, + 0.21586701273918152, + 0.034250739961862564, + 0.3291316628456116, + 0.20251800119876862, + -0.7360188364982605, + -0.508668839931488, + -0.14231330156326294, + -0.2979564368724823 + ] + ], + "generated_utilities": [ + 0.09167696535587311, + 0.00022964179515838623, + 0.0006267279386520386, + 6.0498714447021484e-06, + 0.00010013580322265625, + 1.5795230865478516e-06, + 0.002237573266029358, + 0.09276536107063293 + ], + "group_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0793772041797638, + 2.0, + 2.0, + 2.0, + 0.1144598126411438, + 0.04584880173206329, + 0.08543911576271057, + 0.1307348757982254, + 0.10944780707359314, + 0.07284757494926453, + 0.05719476938247681, + 0.09351089596748352, + 0.13337698578834534, + 0.039893895387649536, + 0.09353269636631012, + 0.08224260807037354 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.016482055187225342, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.0903974175453186, + 0.0, + 0.012217208743095398, + 0.010632793419063091, + 0.03624187409877777, + 0.04524834081530571, + -1.3283450603485107, + -0.00880381464958191, + 0.0, + -0.014520794153213501, + 0.5632922649383545, + 0.003262042999267578, + -0.11586570739746094, + 0.0014255642890930176, + 0.015446990728378296, + 0.0 + ], + [ + 0.04007059335708618, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.12278485298156738, + 0.0, + -0.0962398424744606, + -0.08766138553619385, + 0.29553186893463135, + 0.10519622266292572, + -1.3283450603485107, + 0.00032788515090942383, + 0.0, + -0.13836456835269928, + 0.5632922649383545, + -0.13551044464111328, + -0.11586570739746094, + 0.07183873653411865, + -0.03684847429394722, + 0.0 + ], + [ + 0.08037775754928589, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.33122140169143677, + 0.0, + 0.20786377787590027, + -0.42098182439804077, + -0.17684945464134216, + 0.1062583178281784, + -1.3283450603485107, + 0.28051087260246277, + 0.0, + 0.7470210194587708, + 0.5632922649383545, + 0.03247350454330444, + -0.11586570739746094, + 0.4809897243976593, + -0.1744043231010437, + 0.0 + ], + [ + -0.4189682602882385, + 1.1229554414749146, + 2.8264856338500977, + -2.2389113903045654, + -2.4226768016815186, + 1.2001769542694092, + 2.1568119525909424, + -0.016906127333641052, + -0.24083517491817474, + -0.23574936389923096, + -0.13933053612709045, + -2.458132028579712, + 0.40251442790031433, + 0.870932936668396, + -0.28115370869636536, + 1.4463695287704468, + 0.9248121380805969, + -0.9938455820083618, + -0.680131196975708, + -0.051702484488487244, + -0.4663417339324951 + ], + [ + -0.6373537182807922, + 1.3288449048995972, + 3.1669085025787354, + -1.4171695709228516, + -2.242492198944092, + 1.6900161504745483, + 2.778874158859253, + 0.47899991273880005, + 0.3402186334133148, + 0.4507899582386017, + 0.8237879276275635, + -1.7865383625030518, + 1.1732149124145508, + 1.5826115608215332, + 0.4416889548301697, + 2.222722053527832, + 0.6443713307380676, + -1.8598005771636963, + -0.9690301418304443, + 0.3504742383956909, + -1.1371408700942993 + ] + ], + "predicted_scores": [ + 9.256302833557129, + 8.243694305419922, + 8.192558288574219, + 8.030529022216797, + 7.7356977462768555, + 7.631898880004883, + 7.5903639793396, + 7.43136739730835 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "998761162", + "selected_index": 0, + "state_hash": "78cc63bdeba9f4d6d6da2efd23c0f7f6631321321e917a70299a7c8025e71d47", + "stored_base_utility": 0.0793772041797638, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11926940083503723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11926940083503723 + }, + { + "contact_quality": null, + "progress": 0.12008294463157654, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12008294463157654 + }, + { + "contact_quality": null, + "progress": 0.0918070524930954, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0918070524930954 + }, + { + "contact_quality": null, + "progress": 0.12258778512477875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12258778512477875 + }, + { + "contact_quality": null, + "progress": 0.04670017957687378, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04670017957687378 + }, + { + "contact_quality": null, + "progress": 0.10160571336746216, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10160571336746216 + }, + { + "contact_quality": null, + "progress": 0.085929274559021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.085929274559021 + }, + { + "contact_quality": null, + "progress": 0.13679209351539612, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13679209351539612 + } + ], + "candidate_progress": [ + 0.11926940083503723, + 0.12008294463157654, + 0.0918070524930954, + 0.12258778512477875, + 0.04670017957687378, + 0.10160571336746216, + 0.085929274559021, + 0.13679209351539612 + ], + "candidate_source_chart_ids": [ + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-fc8084d5cd7da937fc02", + "generated_tangents": [ + [ + 0.03669255971908569, + -0.056140247732400894, + 1.241540551185608, + -0.5289525985717773, + -1.5482816696166992, + -0.28787001967430115, + -0.15741264820098877, + 0.08146894723176956, + -0.14238183200359344, + 0.09228740632534027, + 0.047836124897003174, + -1.544621229171753, + 0.6215853691101074, + -0.5542036890983582, + 0.12395864725112915, + -0.016038352623581886, + -0.002518489956855774, + -0.09731416404247284, + -0.4493175745010376, + -0.407426655292511, + -0.2785944640636444 + ], + [ + 0.013331606984138489, + -0.046189721673727036, + 1.1529415845870972, + -0.5309498310089111, + -1.622788667678833, + -0.27052074670791626, + -0.1664489507675171, + 0.09679500013589859, + -0.16496385633945465, + 0.12681393325328827, + 0.046891264617443085, + -1.5165820121765137, + 0.4988780915737152, + -0.4027971029281616, + 0.15612731873989105, + 0.02184528112411499, + 0.08176298439502716, + -0.08559386432170868, + -0.5042839646339417, + -0.25376665592193604, + -0.10230544209480286 + ], + [ + -0.06215904653072357, + -0.07553570717573166, + 1.1406681537628174, + -0.6841037273406982, + -1.4110968112945557, + -0.3699903190135956, + -0.28840702772140503, + 0.026922211050987244, + -0.039488911628723145, + 0.11670631170272827, + 0.1159554272890091, + -1.5273253917694092, + 0.06421884894371033, + -0.15283460915088654, + 0.18330299854278564, + -0.0704086571931839, + -0.029702797532081604, + -0.1403348594903946, + -0.21635115146636963, + -0.15562571585178375, + -0.23336240649223328 + ], + [ + 0.018672600388526917, + -0.08738701045513153, + 1.168372392654419, + -0.6658843755722046, + -1.414001226425171, + -0.34462401270866394, + -0.29870617389678955, + 0.07341969758272171, + -0.07476446777582169, + 0.08924899995326996, + -0.03867051750421524, + -1.5284569263458252, + -0.03962814807891846, + -0.18334807455539703, + 0.23609218001365662, + -0.03537214547395706, + -0.06439022719860077, + -0.12337668240070343, + -0.32380378246307373, + -0.20941540598869324, + -0.1552450954914093 + ], + [ + 0.07217048108577728, + -0.0567023865878582, + 1.09761381149292, + -0.5650160312652588, + -1.452897071838379, + -0.3664596974849701, + -0.2504051923751831, + 0.018995091319084167, + -0.05408535897731781, + 0.08457290381193161, + 0.004011616110801697, + -1.353523850440979, + 0.0910172164440155, + -0.14501617848873138, + 0.1858120709657669, + -0.05699671059846878, + -0.03672941029071808, + -0.05039876699447632, + -0.26932191848754883, + -0.1295747309923172, + -0.24405178427696228 + ], + [ + 0.0012893974781036377, + -0.030742498114705086, + 1.1018292903900146, + -0.6289820671081543, + -1.3554205894470215, + -0.4472218453884125, + -0.3116702437400818, + 0.022877201437950134, + -0.042790912091732025, + 0.08544452488422394, + -0.017446249723434448, + -1.4039801359176636, + 0.2083112895488739, + -0.14769314229488373, + 0.15416696667671204, + -0.0833895355463028, + -0.07448841631412506, + -0.10906194150447845, + -0.2639598250389099, + -0.24095386266708374, + -0.2905418872833252 + ], + [ + -0.019429758191108704, + -0.06427917629480362, + 1.1811469793319702, + -0.6200525760650635, + -1.438051700592041, + -0.448493629693985, + -0.38918638229370117, + 0.04172700643539429, + -0.09341886639595032, + 0.15284757316112518, + 0.014197561889886856, + -1.3975614309310913, + 0.1681613028049469, + -0.11636488139629364, + 0.2413283884525299, + -0.02814667858183384, + 0.039856329560279846, + -0.15920402109622955, + -0.34145277738571167, + 0.05902218818664551, + -0.23194381594657898 + ], + [ + -0.004085913300514221, + 0.04330699145793915, + 0.8578941822052002, + -0.21346747875213623, + -1.3380348682403564, + -0.4132215082645416, + -0.14985018968582153, + -0.006819739937782288, + -0.15147656202316284, + 0.03852641582489014, + 0.12215299904346466, + -1.341388463973999, + 0.7584611177444458, + -0.15727286040782928, + 0.0740395188331604, + -0.14490386843681335, + -0.09812299907207489, + -0.028176352381706238, + -0.22412532567977905, + 0.20315155386924744, + -0.004033252596855164 + ] + ], + "generated_utilities": [ + 0.11926940083503723, + 0.12008294463157654, + 0.0918070524930954, + 0.12258778512477875, + 0.04670017957687378, + 0.10160571336746216, + 0.085929274559021, + 0.13679209351539612 + ], + "group_id": "ms-fc8084d5cd7da937fc02", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.120394766330719, + 0.1662210375070572, + 2.0, + 0.0270967036485672, + 0.16474007070064545, + 0.014024153351783752, + 0.13986149430274963, + 0.17214345932006836, + 0.19527268409729004, + 0.08535367250442505, + 0.1354738473892212, + 0.11415943503379822, + 0.05034071207046509, + 0.10602898895740509, + 0.01804502308368683, + 0.09680917859077454 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.4742112457752228, + -0.6877930760383606, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.0012008249759674072, + 0.0, + 0.04633669555187225, + -0.307714581489563, + -0.22357922792434692, + -0.35245198011398315, + -1.486854076385498, + 0.15573161840438843, + 0.0, + 0.5157145261764526, + 0.243257537484169, + 0.2085912823677063, + 0.48896148800849915, + -0.06008961796760559, + 0.4603208899497986, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.883587837219238, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6201469898223877, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.5597648620605469 + ], + [ + -0.8389496207237244, + 0.5873794555664062, + 2.327263116836548, + -2.175445795059204, + -1.844232439994812, + 0.15723112225532532, + 2.920079231262207, + -0.5335484743118286, + -0.6761462688446045, + 0.48151493072509766, + 0.2968217730522156, + -2.99812650680542, + -0.27805766463279724, + 2.0262699127197266, + 0.10151885449886322, + -0.7244771122932434, + 1.0176090002059937, + 1.278473973274231, + -0.2260074019432068, + -1.0207180976867676, + 0.7682583928108215 + ], + [ + 0.2503487765789032, + 0.20628705620765686, + 2.289511203765869, + -0.8669976592063904, + -1.9807379245758057, + 1.1820356845855713, + 2.8384368419647217, + 0.43699324131011963, + -0.688888669013977, + 1.0812339782714844, + -0.18395745754241943, + -1.6906647682189941, + -0.040969282388687134, + 1.254612684249878, + 0.24077603220939636, + -1.0562812089920044, + 1.2436273097991943, + 0.2388666272163391, + -0.5016235709190369, + 0.4570234417915344, + 1.355008840560913 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.10285450518131256, + -0.2634444832801819, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.12851905822753906, + 0.0, + 0.06062778830528259, + 0.006933145225048065, + -0.08638572692871094, + 0.32057398557662964, + -1.486854076385498, + -0.1531560719013214, + 0.0, + 0.14251592755317688, + 0.07826228439807892, + -0.24055713415145874, + -0.11116695404052734, + 0.200389564037323, + -0.07051187753677368, + 0.0 + ], + [ + -0.07637833058834076, + 0.6696901321411133, + 2.130587100982666, + -1.9006763696670532, + -2.6654670238494873, + 0.06729818880558014, + 2.65816593170166, + -0.0612010583281517, + -0.1226595938205719, + 0.9200000166893005, + -0.4352770149707794, + -2.266615390777588, + 0.48697614669799805, + 1.5162358283996582, + -0.014945849776268005, + -0.4459138512611389, + 1.1448073387145996, + 0.8031747937202454, + -0.6817312240600586, + -0.20262596011161804, + 1.1690938472747803 + ], + [ + 0.519248902797699, + 0.27624276280403137, + 1.911887764930725, + -1.8784401416778564, + -2.9212372303009033, + 0.7762739658355713, + 2.547428846359253, + 0.08362685143947601, + -0.1671719253063202, + 0.8596282601356506, + -0.17510351538658142, + -2.220954656600952, + 0.4918195605278015, + 1.357650876045227, + 0.3462192416191101, + 0.2476332038640976, + 0.6965686082839966, + 1.0915483236312866, + -0.6818809509277344, + -0.7465203404426575, + 0.724744439125061 + ] + ], + "predicted_scores": [ + 9.291539192199707, + 9.234026908874512, + 9.123018264770508, + 8.854728698730469, + 8.799765586853027, + 8.680414199829102, + 8.657164573669434, + 8.29908561706543 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1991414747", + "selected_index": 0, + "state_hash": "127cf975da326f174c815484811a6acc4855bf07460a2aa5b0d728cb2a1f41c7", + "stored_base_utility": 0.120394766330719, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20275163650512695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20275163650512695 + }, + "base_progress": 0.20275163650512695, + "base_success": false, + "base_utility": 0.20275163650512695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.22510160505771637, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22510160505771637 + }, + { + "contact_quality": null, + "progress": 0.20951656997203827, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20951656997203827 + }, + { + "contact_quality": null, + "progress": 0.22272604703903198, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22272604703903198 + }, + { + "contact_quality": null, + "progress": 0.22567233443260193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22567233443260193 + }, + { + "contact_quality": null, + "progress": 0.2283252328634262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2283252328634262 + }, + { + "contact_quality": null, + "progress": 0.22638562321662903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22638562321662903 + }, + { + "contact_quality": null, + "progress": 0.1809503734111786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1809503734111786 + }, + { + "contact_quality": null, + "progress": 0.12205810844898224, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12205810844898224 + } + ], + "candidate_progress": [ + 0.22510160505771637, + 0.20951656997203827, + 0.22272604703903198, + 0.22567233443260193, + 0.2283252328634262, + 0.22638562321662903, + 0.1809503734111786, + 0.12205810844898224 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3afd8959363e917591d4", + "generated_tangents": [ + [ + 0.21758851408958435, + 0.45839619636535645, + -0.36980870366096497, + -1.0366790294647217, + 0.02991729974746704, + -0.614129900932312, + 0.618423581123352, + -0.06544056534767151, + -0.12036630511283875, + -0.11159460991621017, + 0.43743187189102173, + -0.1392863690853119, + -1.8090202808380127, + -0.5089260935783386, + -0.16651156544685364, + -0.015876447781920433, + 0.00580163300037384, + 0.04044273495674133, + -0.34339046478271484, + -1.698594093322754, + -0.002629324793815613 + ], + [ + 0.26656264066696167, + 0.43753552436828613, + -0.4475541412830353, + -1.1809070110321045, + 0.05887360870838165, + -0.204149067401886, + 0.40684348344802856, + -0.0066803693771362305, + -0.15968716144561768, + -0.183632493019104, + 0.1343637853860855, + -0.37479788064956665, + -1.9812617301940918, + -0.5234700441360474, + -0.22862210869789124, + -0.0699257180094719, + 0.1735115647315979, + -0.015770673751831055, + -0.42854249477386475, + -1.456526756286621, + 0.2175431102514267 + ], + [ + 0.09575708955526352, + 0.4790728688240051, + -0.3521564304828644, + -0.9997414350509644, + 0.20344986021518707, + -0.42155060172080994, + 0.46642380952835083, + -0.022058755159378052, + -0.14270851016044617, + -0.16610124707221985, + 0.1806793063879013, + -0.2857516407966614, + -1.8942155838012695, + -0.30111968517303467, + -0.21404579281806946, + -0.029661960899829865, + 0.2486855685710907, + 0.01654106378555298, + -0.2938593626022339, + -1.6678509712219238, + 0.15769878029823303 + ], + [ + 0.026058882474899292, + 0.3004976511001587, + -0.12490341067314148, + -0.9865210056304932, + 0.4065275490283966, + -0.15159425139427185, + 0.33865076303482056, + 0.023164525628089905, + -0.05884639173746109, + -0.09854967147111893, + -0.0044835880398750305, + -0.25652483105659485, + -1.9442698955535889, + 0.14995869994163513, + 0.015987791121006012, + -0.07364068925380707, + 0.22663886845111847, + 0.040101274847984314, + 0.10516948997974396, + -1.4604427814483643, + 0.30864012241363525 + ], + [ + 0.08715064823627472, + 0.3317517638206482, + -0.08255305886268616, + -1.070832371711731, + 0.43441343307495117, + -0.0382179319858551, + 0.03654316067695618, + 0.0064200758934021, + -0.17202529311180115, + -0.14261490106582642, + -0.034951142966747284, + -0.10838693380355835, + -1.9902634620666504, + 0.012336239218711853, + -0.006129477173089981, + -0.04287237673997879, + 0.15029937028884888, + -0.08352373540401459, + -0.09005069732666016, + -1.4715790748596191, + 0.23539552092552185 + ], + [ + 0.2370307594537735, + 0.4244530200958252, + -0.28319671750068665, + -1.0386545658111572, + 0.18457840383052826, + -0.17398586869239807, + 0.5261819362640381, + -0.016083568334579468, + -0.07158034294843674, + 0.018106073141098022, + 0.00092262402176857, + -0.11308425664901733, + -1.7991769313812256, + -0.10645650327205658, + 0.026809033006429672, + -0.016156738623976707, + 0.17475959658622742, + -0.07639913260936737, + -0.19715937972068787, + -1.747654914855957, + 0.29403311014175415 + ], + [ + 0.43924206495285034, + 0.44182294607162476, + -0.085472971200943, + -0.9050775766372681, + 0.6168959140777588, + -0.44606825709342957, + 0.03122866153717041, + 0.01404985785484314, + -0.03282729536294937, + -0.04009737819433212, + -0.03906179219484329, + -0.3012073338031769, + -2.002332925796509, + -0.370714008808136, + -0.17952914535999298, + 0.08427910506725311, + 0.3502427935600281, + 0.162503182888031, + -0.3028104305267334, + -1.8390069007873535, + 0.01095251739025116 + ], + [ + 0.23200273513793945, + 0.23491232097148895, + 0.3147900104522705, + -1.0627516508102417, + 0.6707007884979248, + -0.14352652430534363, + 0.6275544166564941, + 0.1423129439353943, + -0.057731397449970245, + -0.14694270491600037, + -0.03461664170026779, + -0.12505418062210083, + -0.9721747636795044, + -0.5255884528160095, + 0.08521084487438202, + -0.13859756290912628, + 0.4756892919540405, + -0.2449180632829666, + -0.1088409423828125, + -0.9841295480728149, + -0.07133238017559052 + ] + ], + "generated_utilities": [ + 0.22510160505771637, + 0.20951656997203827, + 0.22272604703903198, + 0.22567233443260193, + 0.2283252328634262, + 0.22638562321662903, + 0.1809503734111786, + 0.12205810844898224 + ], + "group_id": "ms-3afd8959363e917591d4", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8582140207290649, + 2.0, + 0.18031787872314453, + 0.07887399196624756, + 0.5568947792053223, + 0.16715207695960999, + 0.5635193586349487, + 0.18913890421390533, + 0.5682768225669861, + 0.10102970898151398, + 0.09163376688957214, + 0.12909311056137085, + 0.12299326062202454, + 0.09030237793922424, + 0.13531166315078735, + 0.06874948740005493 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.15699657797813416, + 0.6247961521148682, + 0.107974112033844, + -0.3284463882446289, + 0.22832846641540527, + -0.12728703022003174, + 0.0, + -0.2739208936691284, + -0.051639966666698456, + -0.2660524249076843, + -0.2539072632789612, + 0.09800386428833008, + -1.84090256690979, + 0.0, + -0.017676634714007378, + -0.018830953165888786, + 0.1583891659975052, + -0.07512116432189941, + -0.01770937442779541, + -1.5237011909484863, + 0.0 + ], + [ + 0.4675478935241699, + 0.6247961521148682, + 0.3345896601676941, + -0.5404787063598633, + -0.06834304332733154, + -0.12728703022003174, + 0.0, + -0.14999517798423767, + -0.2730908691883087, + 0.08837036788463593, + 0.45600926876068115, + 0.410434365272522, + -1.84090256690979, + 0.0, + 0.4969322681427002, + 0.22034725546836853, + 0.2921968400478363, + 0.5872196555137634, + 0.19443652033805847, + -1.5237011909484863, + 0.0 + ], + [ + 0.41189882159233093, + 1.6247961521148682, + 0.12157746404409409, + -1.328446388244629, + 0.9316569566726685, + -1.1272870302200317, + 1.276085376739502, + -0.029595039784908295, + -0.014941371977329254, + 0.004283940885215998, + 0.6188332438468933, + -0.49464237689971924, + -2.84090256690979, + -1.8046140670776367, + 0.0007966384291648865, + -0.006994657218456268, + 0.014263113960623741, + 0.6869955658912659, + -0.2918644845485687, + -2.5237011909484863, + -1.7481608390808105 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.552170753479004, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.6092281341552734, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.496321678161621 + ], + [ + 0.44720953702926636, + 1.4417266845703125, + 0.4182493984699249, + -1.547792911529541, + 0.990667462348938, + -0.9114025831222534, + 1.535681962966919, + 0.21791213750839233, + -0.30513259768486023, + 0.013669610023498535, + 0.7976462841033936, + -0.6395556926727295, + -2.5809314250946045, + -1.571488618850708, + 0.038753002882003784, + 0.04824339970946312, + -0.11262937635183334, + 0.9187417030334473, + -0.1308273822069168, + -2.59043288230896, + -1.9625922441482544 + ], + [ + 0.5162960290908813, + 1.2868248224258423, + 0.2058304250240326, + -1.4812198877334595, + 1.085485816001892, + -1.05157470703125, + 1.1480523347854614, + 0.2796477973461151, + -0.1532609462738037, + 0.12988577783107758, + 0.8581418395042419, + -0.6760798692703247, + -2.769819736480713, + -2.192568778991699, + 0.10523252189159393, + 0.05066283047199249, + -0.08061028271913528, + 0.7438116669654846, + -0.07841359078884125, + -2.3079848289489746, + -1.4200246334075928 + ], + [ + 0.4112767279148102, + 1.8745536804199219, + 0.5301340222358704, + -0.9917724132537842, + 0.8905197978019714, + -0.8391705751419067, + 1.3120794296264648, + 0.3723032474517822, + -0.19477176666259766, + -0.35689839720726013, + 0.2843710780143738, + -0.45066511631011963, + -2.812030792236328, + -1.588818073272705, + -0.09205310046672821, + -0.5067525506019592, + -0.41988417506217957, + 1.0423402786254883, + 0.1759319305419922, + -2.945967197418213, + -2.05134916305542 + ], + [ + 0.46365559101104736, + 1.105017900466919, + 0.466808021068573, + -1.5382812023162842, + 0.8287478089332581, + -1.4617033004760742, + 0.8807560205459595, + 0.2107943296432495, + 0.2954367995262146, + 0.45928943157196045, + 0.6575880646705627, + -0.5414218902587891, + -2.711836099624634, + -1.6827114820480347, + 0.3175691068172455, + -0.38371264934539795, + 0.34041139483451843, + 0.34132614731788635, + -0.8783347606658936, + -2.133782148361206, + -1.3974260091781616 + ], + [ + 0.13586324453353882, + 1.6158640384674072, + 0.24489957094192505, + -1.4942185878753662, + 1.1602565050125122, + -1.648895502090454, + 1.1836220026016235, + 0.17831489443778992, + -0.5400142669677734, + -0.42907604575157166, + 0.5951722264289856, + -0.7885013818740845, + -3.2272911071777344, + -2.208972692489624, + -0.4857475459575653, + 0.5623884201049805, + 0.02566027268767357, + 0.590702474117279, + -0.6581307649612427, + -2.7863473892211914, + -2.031818151473999 + ], + [ + 0.21801596879959106, + 1.0595136880874634, + 0.9040011763572693, + -1.1076470613479614, + 1.2744677066802979, + -0.7150566577911377, + 1.0042598247528076, + 0.5841938257217407, + -0.2832033336162567, + -0.45343834161758423, + -0.17037296295166016, + -0.5152367353439331, + -2.1963798999786377, + -1.5652427673339844, + 0.03223085403442383, + 0.07496664673089981, + -0.1515023559331894, + 1.233866572380066, + 0.4301607310771942, + -2.489328384399414, + -1.8620245456695557 + ], + [ + -0.15804454684257507, + 2.2168471813201904, + 0.9374236464500427, + -1.8158314228057861, + 1.114590048789978, + -0.9279311895370483, + 1.890993356704712, + -0.8653054237365723, + -0.5551189184188843, + 0.4295695126056671, + 1.1614964008331299, + -0.5840407609939575, + -2.900725841522217, + -2.579883575439453, + -0.1900041103363037, + 0.42368441820144653, + 0.38376376032829285, + 0.9654008150100708, + 0.10608989000320435, + -3.332181453704834, + -1.8879828453063965 + ], + [ + 1.0901068449020386, + 2.576328754425049, + 0.5122992396354675, + -1.9627916812896729, + 0.026246964931488037, + -0.3622438311576843, + 0.9286589622497559, + 0.644849956035614, + 0.9211093187332153, + 0.3528344929218292, + 0.3661169707775116, + 0.20941221714019775, + -2.582620143890381, + -1.6867890357971191, + 0.09869468957185745, + -0.8795469403266907, + -0.2239246666431427, + 0.5206944942474365, + -0.20716872811317444, + -2.8072235584259033, + -2.3385443687438965 + ], + [ + -0.0005125105381011963, + 1.8920408487319946, + 1.0018010139465332, + -2.2556979656219482, + 0.7338991165161133, + -0.4559611678123474, + 1.027335524559021, + -1.0165901184082031, + -0.14792552590370178, + 0.11212217807769775, + 0.8827763795852661, + 0.1683982014656067, + -3.2136244773864746, + -2.4300284385681152, + -0.9183067083358765, + 0.10752613097429276, + -0.2955099046230316, + 0.31867098808288574, + 0.06515705585479736, + -3.416110038757324, + -1.5137704610824585 + ], + [ + -0.39508435130119324, + 2.55599308013916, + 0.128604456782341, + -1.8821169137954712, + 0.6769157648086548, + -0.7477259635925293, + 1.6593644618988037, + -0.005859369412064552, + 0.22320973873138428, + -0.2906110882759094, + 0.22343486547470093, + -1.488371729850769, + -2.81829833984375, + -0.939697265625, + 0.9867495894432068, + 0.2955887019634247, + -0.11908379942178726, + 0.473803848028183, + -0.11059392988681793, + -3.471217155456543, + -2.3480191230773926 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.018576741218566895, + 0.6247961521148682, + -0.017397306859493256, + -0.3284463882446289, + 0.044771432876586914, + -0.12728703022003174, + 0.0, + -0.005512619391083717, + 0.03525955602526665, + -0.005881692748516798, + 0.04493492841720581, + 0.056808531284332275, + -1.84090256690979, + 0.0, + 0.11075251549482346, + -0.05448189005255699, + -0.041690655052661896, + 0.010008454322814941, + 0.056893616914749146, + -1.5237011909484863, + 0.0 + ] + ], + "predicted_scores": [ + 5.160774230957031, + 4.728915691375732, + 4.570479393005371, + 4.122161865234375, + 4.062873840332031, + 4.021884441375732, + 3.392411708831787, + 3.2204978466033936 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "75070641", + "selected_index": 0, + "state_hash": "bd1bbc057450972b9fa6e08a54ebd8c6d82d09997dad16cecf9584632b705b7b", + "stored_base_utility": 0.8582140207290649, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1717667579650879, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1717667579650879 + }, + { + "contact_quality": null, + "progress": 0.18597783148288727, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18597783148288727 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.19216947257518768, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19216947257518768 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.16492807865142822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16492807865142822 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.1717667579650879, + 0.18597783148288727, + 1.0, + 1.0, + 0.19216947257518768, + 1.0, + 0.16492807865142822, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919", + "ms-0949097d19e597fc7f63", + "ms-4cab292bc65fa59274ef" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + true, + true, + false, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-3ff1aedb72f65a182b58", + "generated_tangents": [ + [ + 0.09392966330051422, + -0.073333740234375, + -0.02382710576057434, + -0.1307896375656128, + -0.09160763025283813, + -2.3890573978424072, + -0.2402210235595703, + 0.03725676238536835, + 0.06001857668161392, + 0.05920320004224777, + 0.06732793897390366, + -0.04652625322341919, + -1.9502712488174438, + -0.13314096629619598, + 0.018359988927841187, + 0.0077648162841796875, + 0.06510114669799805, + -0.1600084751844406, + -0.006948888301849365, + -1.5971033573150635, + -0.03555750846862793 + ], + [ + 0.056459277868270874, + -0.026371078565716743, + -0.008917242288589478, + -0.21969135105609894, + -0.04966193437576294, + -2.2545814514160156, + -0.24377870559692383, + 0.07215207815170288, + 0.03315239027142525, + 0.011191889643669128, + 0.05912366136908531, + -0.011779963970184326, + -1.8740211725234985, + -0.15244542062282562, + 0.03680367395281792, + 0.031709812581539154, + 0.06003528833389282, + -0.15856648981571198, + 0.009031146764755249, + -1.582228660583496, + -0.04679661989212036 + ], + [ + 0.12764057517051697, + -0.056162089109420776, + 0.11235526204109192, + -0.1247410848736763, + 0.15366747975349426, + -2.1857078075408936, + -0.08992600440979004, + 0.07578996568918228, + 0.0003266548737883568, + 0.10295122861862183, + 0.04417681694030762, + 0.13179272413253784, + -1.7686823606491089, + 0.03846485912799835, + -0.04094785079360008, + -0.006326349917799234, + 0.008371815085411072, + -0.135220006108284, + 0.06420838832855225, + -1.5203099250793457, + -0.06928402185440063 + ], + [ + -0.08496171236038208, + -0.09299589693546295, + 0.08713719248771667, + -0.14648938179016113, + -0.06924819946289062, + -2.3240087032318115, + -0.17085039615631104, + 0.018144264817237854, + -0.017813703045248985, + -0.0343199297785759, + -0.07212626188993454, + 0.02971920371055603, + -1.9930564165115356, + -0.24653057754039764, + -0.007997963577508926, + 0.02391284704208374, + -0.07883070409297943, + -0.27574771642684937, + -0.19631525874137878, + -1.6504130363464355, + -0.043965622782707214 + ], + [ + -0.20934848487377167, + -0.1266692578792572, + 0.10706019401550293, + 0.0272170752286911, + -0.21940195560455322, + -2.659048318862915, + -0.4746842384338379, + 0.2459060251712799, + 0.05170027166604996, + -0.23570626974105835, + 0.12484186142683029, + -0.28754469752311707, + -1.6175321340560913, + -0.2846195697784424, + 0.20102715492248535, + 0.09623122215270996, + -0.05053211748600006, + -0.2426890879869461, + -0.06533008813858032, + -2.0795533657073975, + -0.16265299916267395 + ], + [ + 0.0054766833782196045, + -0.12461771816015244, + 0.09641531109809875, + -0.12070398777723312, + -0.6008101105690002, + -2.37394642829895, + -0.8811764121055603, + -0.03970348834991455, + -0.08527133613824844, + 0.08193513005971909, + -0.07997668534517288, + 0.16318264603614807, + -2.3331503868103027, + 0.425695538520813, + 0.0248819999396801, + 0.20716316998004913, + 0.12654536962509155, + -0.11810736358165741, + -0.4127483367919922, + -1.7783329486846924, + -0.47662726044654846 + ], + [ + 0.15129965543746948, + -0.05064266920089722, + 0.1171606183052063, + -0.08955720067024231, + -0.023857414722442627, + -2.461094856262207, + -0.45744675397872925, + 0.013203874230384827, + 0.017858093604445457, + -0.03413312882184982, + 0.24308684468269348, + 0.05985631048679352, + -1.8113385438919067, + -0.14572568237781525, + -0.019261468201875687, + 0.057120610028505325, + 0.11785639822483063, + -0.22399063408374786, + 0.053573429584503174, + -1.7720108032226562, + 0.08494687080383301 + ], + [ + -0.10472635924816132, + 0.09618918597698212, + 0.13700050115585327, + -0.026022180914878845, + -0.195050448179245, + -2.350626230239868, + -0.5292054414749146, + 0.12431493401527405, + 0.09990887343883514, + -0.0625324472784996, + 0.29243823885917664, + -0.2240816056728363, + -1.6519718170166016, + -0.2509976029396057, + 0.013827253133058548, + 0.18157435953617096, + 0.2608470320701599, + -0.015192806720733643, + 0.14551664888858795, + -2.0923616886138916, + 0.056387029588222504 + ] + ], + "generated_utilities": [ + 0.1717667579650879, + 0.18597783148288727, + 2.0, + 2.0, + 0.19216947257518768, + 2.0, + 0.16492807865142822, + 2.0 + ], + "group_id": "ms-3ff1aedb72f65a182b58", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.851725161075592, + 2.0, + 0.8126396536827087, + 2.0, + 0.7855525612831116, + 0.7821746468544006, + 0.6115735173225403, + 0.5732660889625549, + 0.7798006534576416, + 0.09109212458133698, + 0.8456447124481201, + 0.16636842489242554, + 0.0955686867237091, + 0.0822453498840332, + 0.0018809139728546143, + 0.08579078316688538 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.09319563210010529, + 0.18433167040348053, + 0.03857926279306412, + 0.20899039506912231, + -0.19732166826725006, + -2.7716264724731445, + -0.9089346528053284, + -0.012062903493642807, + -0.001710914308205247, + 0.02447224222123623, + 0.6588084697723389, + -0.43759873509407043, + -2.7224881649017334, + -1.560755968093872, + -0.002114389091730118, + -0.004199527204036713, + 0.015975257381796837, + 0.7779304385185242, + -0.31706690788269043, + -2.4826252460479736, + -1.5341289043426514 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.8178693056106567, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.121511936187744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0682578086853027 + ], + [ + -0.10748189687728882, + 0.29221633076667786, + 0.12325333058834076, + 0.21938443183898926, + 0.017790749669075012, + -2.485055446624756, + -1.1110891103744507, + 0.062413301318883896, + -0.2567306160926819, + -0.15453597903251648, + 0.549060583114624, + -0.18322163820266724, + -2.5428390502929688, + -1.8537542819976807, + -0.07324761152267456, + 0.28765758872032166, + 0.16924789547920227, + 0.6205734610557556, + -0.4268453121185303, + -2.6385459899902344, + -1.5760513544082642 + ], + [ + 0.44683152437210083, + 0.3713763356208801, + 0.1084144338965416, + 0.2590586543083191, + 0.1997859627008438, + -2.787759780883789, + -1.1921976804733276, + 0.33878061175346375, + 0.019987009465694427, + 0.021556159481406212, + 0.36814823746681213, + -0.2135106772184372, + -2.416769027709961, + -1.3776189088821411, + -0.3500843346118927, + -0.26303720474243164, + 0.10322260856628418, + 0.9827795624732971, + -0.3747403025627136, + -2.6419029235839844, + -1.4116339683532715 + ], + [ + -0.2883707880973816, + 0.02873578667640686, + -0.11175497621297836, + -0.106661856174469, + 0.03059367835521698, + -2.3899149894714355, + -1.0789995193481445, + 0.2610729932785034, + 0.4800889790058136, + -0.45478329062461853, + 1.0867195129394531, + -0.7417076826095581, + -3.09529185295105, + -1.730720043182373, + -0.33508142828941345, + 0.10841497033834457, + -0.38902348279953003, + 0.44348275661468506, + 0.0854053795337677, + -2.3334603309631348, + -1.7851121425628662 + ], + [ + 0.16367590427398682, + 0.05515189468860626, + -0.3197028636932373, + 0.5501840114593506, + -0.3621746301651001, + -3.0582237243652344, + -1.135798454284668, + -0.15054848790168762, + -0.5578444004058838, + 0.3691062331199646, + 0.9388304352760315, + -0.19103051722049713, + -2.7536559104919434, + -2.093773603439331, + -0.037153881043195724, + 0.19014233350753784, + -0.36205345392227173, + 0.24469107389450073, + -0.5916419625282288, + -2.7417092323303223, + -1.637229323387146 + ], + [ + -0.3646707534790039, + 0.6696967482566833, + -0.44303250312805176, + -0.4559982419013977, + -0.6542845368385315, + -2.9630653858184814, + -1.4721050262451172, + 0.5010843873023987, + 0.5162685513496399, + 0.42702627182006836, + 0.6571387648582458, + -0.08207055926322937, + -3.278073310852051, + -1.7742136716842651, + -0.6448181867599487, + -0.45361432433128357, + 0.3905653953552246, + 0.7770582437515259, + -0.19482973217964172, + -2.094736337661743, + -1.083003282546997 + ], + [ + 0.9472348093986511, + 0.6904511451721191, + -0.2936761677265167, + -0.34556156396865845, + -0.7593099474906921, + -1.9670193195343018, + -0.7599589824676514, + 0.04467743635177612, + 0.6557444334030151, + 0.2765021026134491, + 0.042136549949645996, + -0.37541040778160095, + -3.161308765411377, + -0.8803613185882568, + 0.38967010378837585, + 0.4423010051250458, + 0.019825218245387077, + 0.9605399370193481, + -0.874176561832428, + -3.0791730880737305, + -1.3198856115341187 + ], + [ + 0.8605663180351257, + 0.04459291696548462, + -0.8886744976043701, + -0.38333380222320557, + -1.0364235639572144, + -3.0104973316192627, + -0.889693021774292, + -0.08674328029155731, + 0.20159615576267242, + 0.1069621816277504, + 1.4727201461791992, + 0.09883639216423035, + -3.129849433898926, + -0.8773520588874817, + -0.2628254294395447, + -0.3415844738483429, + -0.35313117504119873, + 0.3155616521835327, + -0.556646466255188, + -2.0992307662963867, + -0.7895610928535461 + ], + [ + 0.39962780475616455, + 0.3406272530555725, + 0.05363208055496216, + -0.06018832325935364, + -0.5703235268592834, + -2.5974044799804688, + -0.6783060431480408, + 0.7815009355545044, + 0.35641810297966003, + -0.00838596560060978, + 0.6228271126747131, + 0.25608590245246887, + -2.5914058685302734, + -0.8502807021141052, + 0.726007878780365, + -0.6774719953536987, + 0.6261520981788635, + 1.149846076965332, + 0.6782730221748352, + -3.391972541809082, + -0.7993651032447815 + ], + [ + -0.29702335596084595, + 0.6166410446166992, + 0.6319320201873779, + -0.6542804837226868, + -0.8892693519592285, + -1.8599720001220703, + -1.3999924659729004, + -0.3017314672470093, + 0.9357973337173462, + -0.5596981048583984, + 1.3143360614776611, + -0.30643409490585327, + -2.028008460998535, + -1.3395459651947021, + 0.6212120056152344, + 0.6739580631256104, + -0.5840877294540405, + -0.06664055585861206, + -0.23446696996688843, + -2.0225906372070312, + -2.438906669616699 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0649428591132164, + 0.09363124519586563, + -0.014398600906133652, + -0.012044146656990051, + 0.03117428719997406, + -1.7716264724731445, + 0.0, + 0.05836518481373787, + 0.012665675021708012, + -0.013142907992005348, + -0.06252282857894897, + -0.06281104683876038, + -1.7224881649017334, + 0.0, + -0.06448972970247269, + -0.03696601092815399, + 0.030633779242634773, + -0.0031400322914123535, + 0.0327644944190979, + -1.4826252460479736, + 0.0 + ], + [ + -0.04005520045757294, + 0.2738151252269745, + -0.05346817523241043, + 0.20275527238845825, + 0.10101960599422455, + -1.7716264724731445, + 0.0, + -0.36066195368766785, + -0.21821273863315582, + 0.22022001445293427, + -0.0596311092376709, + -0.17951729893684387, + -1.7224881649017334, + 0.0, + 0.3205486238002777, + 0.25526633858680725, + 0.17392843961715698, + 0.5170043706893921, + 0.43821561336517334, + -1.4826252460479736, + 0.0 + ] + ], + "predicted_scores": [ + 3.9676198959350586, + 3.801675796508789, + 3.7893948554992676, + 3.2359251976013184, + 3.0078318119049072, + 2.753028392791748, + 2.7207984924316406, + 1.9320051670074463 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055567618", + "selected_index": 0, + "state_hash": "027cc529c99c00db47a01ecfc12ab432557fbd911e8ff3c8a56adf2d3404549f", + "stored_base_utility": 0.851725161075592, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19648407399654388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19648407399654388 + }, + "base_progress": 0.19648407399654388, + "base_success": false, + "base_utility": 0.19648407399654388, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12073369324207306, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12073369324207306 + }, + { + "contact_quality": null, + "progress": 0.1906120628118515, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1906120628118515 + }, + { + "contact_quality": null, + "progress": 0.21016791462898254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21016791462898254 + }, + { + "contact_quality": null, + "progress": 0.2101641744375229, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2101641744375229 + }, + { + "contact_quality": null, + "progress": 0.1435815393924713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1435815393924713 + }, + { + "contact_quality": null, + "progress": 0.5575173497200012, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5575173497200012 + }, + { + "contact_quality": null, + "progress": 0.19436763226985931, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19436763226985931 + }, + { + "contact_quality": null, + "progress": 0.5705926418304443, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5705926418304443 + } + ], + "candidate_progress": [ + 0.12073369324207306, + 0.1906120628118515, + 0.21016791462898254, + 0.2101641744375229, + 0.1435815393924713, + 0.5575173497200012, + 0.19436763226985931, + 0.5705926418304443 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4048f8f6e047422e3c79", + "generated_tangents": [ + [ + 0.2171844244003296, + 0.18487316370010376, + 0.28260764479637146, + -0.36783677339553833, + 0.22836995124816895, + -0.14998310804367065, + 0.3650948703289032, + 0.11603033542633057, + -0.016029154881834984, + -0.09612519294023514, + 0.059937648475170135, + -0.03008727729320526, + -0.9651647806167603, + -0.7220410108566284, + 0.07607698440551758, + -0.14111168682575226, + 0.2967831492424011, + -0.19809038937091827, + -0.22854703664779663, + -1.0838415622711182, + -0.5010473728179932 + ], + [ + -0.03858329355716705, + 0.1825340986251831, + 0.020320653915405273, + -0.2719494700431824, + 0.17369604110717773, + -0.19225314259529114, + 0.12003076076507568, + 0.10427724570035934, + 0.02255987748503685, + -0.041600100696086884, + 0.13357362151145935, + -0.21711459755897522, + -1.9956270456314087, + -0.045915767550468445, + 0.04842653125524521, + -0.12164908647537231, + 0.08355529606342316, + 0.05983982980251312, + 0.05341607332229614, + -1.5519073009490967, + 0.013342559337615967 + ], + [ + 0.002469167113304138, + 0.1645757257938385, + 0.030765265226364136, + -0.31909507513046265, + 0.1670299768447876, + 0.10972249507904053, + -0.14881157875061035, + 0.034435123205184937, + -0.09460742026567459, + -0.06192999333143234, + 0.09626126289367676, + -0.17766708135604858, + -1.998361349105835, + -0.03360585868358612, + 0.023616105318069458, + -0.0715348869562149, + 0.06650449335575104, + -0.1089114099740982, + -0.19733011722564697, + -1.566237211227417, + -0.005705744028091431 + ], + [ + 0.24950110912322998, + 0.21605072915554047, + -0.08998242020606995, + -0.1326679289340973, + -0.04281589388847351, + -0.35822418332099915, + -0.1337767243385315, + -0.1644410490989685, + -0.05146455764770508, + -0.14123421907424927, + 0.5631891489028931, + -0.36045610904693604, + -1.8442262411117554, + -0.4992050528526306, + 0.09674850106239319, + -0.09498344361782074, + -0.24699492752552032, + 0.025555402040481567, + -0.3607638478279114, + -1.7779898643493652, + -0.07849517464637756 + ], + [ + 0.2732873558998108, + 0.36256277561187744, + -0.02408924698829651, + -0.1519061177968979, + 0.2774462103843689, + -0.5360255241394043, + -0.05277550220489502, + 0.00856037437915802, + 0.11335992813110352, + 0.07843641936779022, + 0.11148088425397873, + -0.47118204832077026, + -1.9787909984588623, + -0.653018057346344, + -0.12463939189910889, + 0.0651550441980362, + 0.1642739474773407, + 0.1699945628643036, + -0.45995843410491943, + -1.791959524154663, + -0.14612862467765808 + ], + [ + 0.30226635932922363, + 0.18178465962409973, + -0.22026470303535461, + -0.3560119867324829, + -0.13338452577590942, + -0.07905139029026031, + -0.1558377742767334, + -0.06049463152885437, + -0.06358654797077179, + -0.20067763328552246, + 0.2741665542125702, + -0.6450058221817017, + -1.9496437311172485, + -0.498136043548584, + -0.020649496465921402, + -0.04878104850649834, + -0.13140223920345306, + -0.036294564604759216, + -0.45630335807800293, + -1.4932804107666016, + 0.04894767701625824 + ], + [ + 0.30841732025146484, + 0.26397138833999634, + -0.2059001624584198, + -0.300531804561615, + -0.06339272856712341, + -0.21772345900535583, + 0.10650348663330078, + 0.004072010517120361, + -0.0054857488721609116, + 0.10548824071884155, + 0.08915750682353973, + -0.18550986051559448, + -1.8369694948196411, + -0.32808464765548706, + 0.05239670351147652, + -0.019515590742230415, + -0.003979310393333435, + -0.09749935567378998, + -0.2767573595046997, + -1.8588802814483643, + -0.06719134747982025 + ], + [ + 0.09183335304260254, + 0.2539528012275696, + -0.1101885735988617, + -0.16674894094467163, + -0.07158789038658142, + -0.3414709270000458, + -0.062735915184021, + -0.062052249908447266, + -0.025212986394762993, + -0.1355210840702057, + 0.2859673798084259, + -0.4291049838066101, + -1.8351331949234009, + -0.4200328588485718, + -0.026011284440755844, + -0.05623798072338104, + -0.03581221401691437, + 0.0038288086652755737, + -0.42838382720947266, + -1.7482473850250244, + -0.0438215434551239 + ] + ], + "generated_utilities": [ + 0.12073369324207306, + 0.1906120628118515, + 0.21016791462898254, + 0.2101641744375229, + 0.1435815393924713, + 0.5575173497200012, + 0.19436763226985931, + 0.5705926418304443 + ], + "group_id": "ms-4048f8f6e047422e3c79", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558006286621094, + 2.0, + 0.1986379623413086, + 0.03769078850746155, + 0.5361635684967041, + 0.18274834752082825, + 0.17300468683242798, + 0.12688571214675903, + 0.13836368918418884, + 0.07454231381416321, + 0.0899321436882019, + 0.10402637720108032, + 0.1711854636669159, + 0.17255032062530518, + 0.039613887667655945, + 0.10826331377029419 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1422826200723648, + 0.21033799648284912, + 0.1978604793548584, + 0.059162259101867676, + -0.2529524564743042, + -0.15161612629890442, + 0.0, + -0.2264404594898224, + -0.2209387719631195, + -0.025619791820645332, + 0.06467145681381226, + 0.07422274351119995, + -1.7048513889312744, + 0.0, + 0.15056702494621277, + -0.06247536092996597, + -0.1033044084906578, + 0.1334916353225708, + 0.2784581184387207, + -1.4811434745788574, + 0.0 + ], + [ + -0.44999396800994873, + 0.271686851978302, + -0.4267011880874634, + 0.2730966806411743, + -0.01798880100250244, + -0.04680868983268738, + 0.0, + -0.09472770243883133, + -0.06993720680475235, + -0.06752754002809525, + -0.13884133100509644, + 0.24737784266471863, + -1.7048513889312744, + 0.0, + -0.09056920558214188, + 0.4132460653781891, + 0.04097161442041397, + -0.25072288513183594, + 0.49742037057876587, + -1.4811434745788574, + 0.0 + ], + [ + 0.17392832040786743, + 1.2103379964828491, + -0.3708428144454956, + -0.36093437671661377, + 0.744896650314331, + 0.23929360508918762, + 1.4388959407806396, + -0.04149632900953293, + 0.015302501618862152, + 0.023577755317091942, + 0.5954978466033936, + -0.3274606168270111, + -2.7048513889312744, + -1.646976351737976, + 0.0006552413105964661, + -0.008917607367038727, + 0.022643378004431725, + 0.7492771148681641, + -0.26030659675598145, + -2.4811434745788574, + -1.684959888458252 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8777918815612793, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.293952703475952, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.369919776916504 + ], + [ + 0.1792769730091095, + 0.9774190187454224, + -0.5587559342384338, + -0.5885956883430481, + 0.9904338717460632, + 0.41647347807884216, + 1.4140102863311768, + -0.2462461292743683, + -0.1271664798259735, + -0.2736872732639313, + 0.8877567052841187, + -0.6213628053665161, + -2.937077283859253, + -1.3732452392578125, + -0.0015218837652355433, + -0.27707284688949585, + -0.16698139905929565, + 0.4978373646736145, + -0.479316771030426, + -2.7234396934509277, + -1.6759265661239624 + ], + [ + 0.1658722460269928, + 1.3070863485336304, + -0.4893108606338501, + -0.04209986329078674, + 0.7449368238449097, + 0.3420809805393219, + 1.4472918510437012, + 0.08132945746183395, + 0.13181470334529877, + 0.22494959831237793, + 0.22593030333518982, + -0.49233078956604004, + -2.7544617652893066, + -1.7597702741622925, + -0.365922212600708, + -0.17204320430755615, + -0.1821347028017044, + 0.6334611177444458, + -0.33006516098976135, + -2.2188992500305176, + -1.5452594757080078 + ], + [ + 0.18608316779136658, + 1.23563551902771, + -0.5727517604827881, + 0.10049733519554138, + 0.9073957204818726, + 0.5650919675827026, + 1.8946568965911865, + -0.17719107866287231, + 0.34025683999061584, + 0.44525983929634094, + 0.6176816821098328, + -0.2832425534725189, + -3.1802117824554443, + -1.8703548908233643, + 0.07433810085058212, + 0.1340292990207672, + 0.497944176197052, + 1.0071699619293213, + -0.44033437967300415, + -2.7992849349975586, + -1.8935935497283936 + ], + [ + 0.40270811319351196, + 1.5102664232254028, + -0.8070739507675171, + -0.16390679776668549, + 0.3215266466140747, + -0.16522419452667236, + 1.0900052785873413, + 0.07780961692333221, + 0.5627153515815735, + 0.5386761426925659, + 0.2949375510215759, + 0.059314221143722534, + -3.2417359352111816, + -1.899662971496582, + 0.4160631597042084, + 0.44336366653442383, + 0.47819244861602783, + 1.162381649017334, + -0.7831735014915466, + -2.625349521636963, + -1.2278814315795898 + ], + [ + 0.8529313802719116, + 1.5106499195098877, + 0.19478559494018555, + -0.6338159441947937, + 0.7529481053352356, + -0.24222904443740845, + 0.9723314046859741, + -0.14078867435455322, + -0.5563283562660217, + 0.5440018177032471, + 0.004017293453216553, + 0.019366413354873657, + -3.1521430015563965, + -2.0841636657714844, + -0.2760852575302124, + -0.5531485676765442, + 0.22684869170188904, + 1.100138545036316, + 0.046634942293167114, + -2.429711103439331, + -2.280839443206787 + ], + [ + -0.4760962128639221, + 1.8579951524734497, + 0.2322864532470703, + 0.19953304529190063, + 0.48002323508262634, + 0.7484984397888184, + 2.073361396789551, + -0.04567806422710419, + 0.04004429653286934, + -0.5635936260223389, + 0.5135918259620667, + 0.3908633887767792, + -1.9335949420928955, + -2.442507743835449, + -0.6620182394981384, + 0.4175874888896942, + 0.7711085677146912, + 1.2790143489837646, + -0.5464571714401245, + -3.2030673027038574, + -2.3768434524536133 + ], + [ + 1.0231709480285645, + 1.1804075241088867, + 0.11513572931289673, + -1.23623526096344, + 1.4848686456680298, + 1.1120561361312866, + 0.9504602551460266, + 0.8234015703201294, + -0.04407603666186333, + -0.37310001254081726, + 0.24512693285942078, + 0.4242170751094818, + -2.4985997676849365, + -2.1206297874450684, + 0.5509956479072571, + 0.8686066269874573, + -0.5043656229972839, + 1.3957195281982422, + -0.07662208378314972, + -2.371795654296875, + -2.2404346466064453 + ], + [ + 0.49735015630722046, + 0.3600291609764099, + 0.3103446364402771, + 0.6051294803619385, + -0.15457379817962646, + -0.5570405721664429, + 1.2383109331130981, + 0.9059417247772217, + 0.7879754900932312, + -0.31492647528648376, + 1.5871094465255737, + -0.8906484842300415, + -1.8698136806488037, + -2.401073694229126, + 0.3560212254524231, + -0.6580788493156433, + 0.9037240147590637, + 1.614617109298706, + 0.3788090944290161, + -1.6909503936767578, + -0.8765208125114441 + ], + [ + 0.10883396118879318, + 1.8993752002716064, + -0.7673929333686829, + -0.0936923623085022, + -0.15508884191513062, + -0.1517968773841858, + 0.465425968170166, + 0.43258219957351685, + 0.46641960740089417, + -0.3528187572956085, + 1.09894597530365, + -0.6569450497627258, + -2.025486946105957, + -1.2820806503295898, + -0.21141839027404785, + -0.9515489339828491, + 0.7230286002159119, + -0.21062928438186646, + -0.32822558283805847, + -3.289642333984375, + -1.7125959396362305 + ], + [ + 0.37358880043029785, + 2.052347183227539, + -1.212618112564087, + -0.6321570873260498, + 0.7031511068344116, + -0.3477393090724945, + 1.2687753438949585, + 0.8023800253868103, + -0.031013749539852142, + 0.35525181889533997, + 0.5262036323547363, + -0.9288194179534912, + -1.7686026096343994, + -2.128396987915039, + -0.7001423239707947, + -0.12872236967086792, + -0.012955078855156898, + 1.0237239599227905, + 0.03636348247528076, + -2.0397119522094727, + -0.9761244058609009 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.060203760862350464, + 0.21033799648284912, + 0.0356404185295105, + -0.03044089674949646, + 0.024639129638671875, + -0.032671064138412476, + 0.0, + 0.016544528305530548, + 0.04633430764079094, + 0.005854343995451927, + -0.014876246452331543, + 0.0028293728828430176, + -1.7048513889312744, + 0.0, + -0.05953434109687805, + -0.04181046783924103, + -0.0018185675144195557, + -0.09216523170471191, + 0.019338399171829224, + -1.4811434745788574, + 0.0 + ] + ], + "predicted_scores": [ + 6.748522758483887, + 6.704971790313721, + 5.85119104385376, + 5.7100701332092285, + 5.581671714782715, + 5.393603324890137, + 4.9805073738098145, + 4.962723255157471 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055157354", + "selected_index": 0, + "state_hash": "ae21d2b71722381a530d3a9259d02ea6728eb8a4a84fafc40ccd653d62ad3968", + "stored_base_utility": 0.8558006286621094, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19844311475753784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844311475753784 + }, + "base_progress": 0.19844311475753784, + "base_success": false, + "base_utility": 0.19844311475753784, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.181088387966156, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.181088387966156 + }, + { + "contact_quality": null, + "progress": 0.19361381232738495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19361381232738495 + }, + { + "contact_quality": null, + "progress": 0.1885397732257843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1885397732257843 + }, + { + "contact_quality": null, + "progress": 0.19568920135498047, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19568920135498047 + }, + { + "contact_quality": null, + "progress": 0.1507587730884552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1507587730884552 + }, + { + "contact_quality": null, + "progress": 0.15693092346191406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15693092346191406 + }, + { + "contact_quality": null, + "progress": 0.21208032965660095, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21208032965660095 + }, + { + "contact_quality": null, + "progress": 0.19735300540924072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19735300540924072 + } + ], + "candidate_progress": [ + 0.181088387966156, + 0.19361381232738495, + 0.1885397732257843, + 0.19568920135498047, + 0.1507587730884552, + 0.15693092346191406, + 0.21208032965660095, + 0.19735300540924072 + ], + "candidate_source_chart_ids": [ + "ms-5bddb39f792f20e5e6d4", + "ms-626ac2ad9d0dd95f1a2e", + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-5b26c33feed6992386b1", + "generated_tangents": [ + [ + -0.2862510681152344, + -0.13250859081745148, + -0.16434618830680847, + 0.282477468252182, + -0.2499389946460724, + -0.7596268653869629, + -0.1956716775894165, + -0.005874142050743103, + -0.031128281727433205, + 0.052492037415504456, + 0.08428576588630676, + 0.20952987670898438, + -1.914440393447876, + -0.3057788610458374, + 0.23437592387199402, + 0.031218454241752625, + 0.03350205719470978, + 0.01368558406829834, + 0.12202651798725128, + -1.9188876152038574, + -0.2509079575538635 + ], + [ + -0.0856473296880722, + -0.14153450727462769, + -0.13054463267326355, + 0.28511396050453186, + -0.16349881887435913, + -1.1399821043014526, + -0.2611961364746094, + -0.00705265998840332, + 0.03750584274530411, + 0.03383596986532211, + 0.18552809953689575, + 0.008926361799240112, + -2.0112624168395996, + -0.29566138982772827, + 0.1691342443227768, + 0.012138821184635162, + -0.016451910138130188, + -0.007437646389007568, + 0.08463667333126068, + -1.8638122081756592, + -0.2581925094127655 + ], + [ + -0.13828109204769135, + -0.10704736411571503, + -0.11944708228111267, + 0.29184237122535706, + -0.19101271033287048, + -0.832897424697876, + -0.25124406814575195, + 0.008945763111114502, + -0.003039782866835594, + 0.04226904362440109, + 0.13229313492774963, + 0.08428864926099777, + -2.043454885482788, + -0.2798768877983093, + 0.18739157915115356, + -0.03497983515262604, + -0.051990047097206116, + -0.052511975169181824, + 0.030673429369926453, + -2.039942741394043, + -0.20155519247055054 + ], + [ + -0.06138770282268524, + -0.17644724249839783, + -0.15302732586860657, + 0.3200128972530365, + -0.1305902898311615, + -0.7829512357711792, + -0.25700312852859497, + -0.039596185088157654, + -0.033647067844867706, + 0.06793227791786194, + 0.04071666672825813, + 0.14053404331207275, + -2.0391485691070557, + -0.16965924203395844, + 0.17837156355381012, + -0.038668420165777206, + -0.02522064745426178, + -0.060609444975852966, + 0.06050428748130798, + -1.9583921432495117, + -0.07141707837581635 + ], + [ + -0.09505705535411835, + -0.15290483832359314, + -0.048787087202072144, + 0.4441513121128082, + 0.2212580144405365, + -0.4604974687099457, + -0.1289454698562622, + -0.04884445667266846, + -0.027616044506430626, + 0.17209866642951965, + 0.015292763710021973, + 0.21872572600841522, + -1.8226814270019531, + 0.11185229569673538, + 0.1978069543838501, + -0.13484697043895721, + -0.02377013862133026, + 0.019368737936019897, + 0.07270364463329315, + -2.320669412612915, + -0.161236971616745 + ], + [ + 0.020276054739952087, + -0.10329604893922806, + -0.088380366563797, + 0.3229021728038788, + 0.04564817249774933, + -0.4610654413700104, + -0.3161851763725281, + -0.1608552634716034, + -0.053467437624931335, + 0.11612476408481598, + -0.058851636946201324, + -0.006638064980506897, + -2.034417152404785, + -0.3028770089149475, + 0.3163405656814575, + -0.013032857328653336, + 0.11392523348331451, + 0.09095031023025513, + 0.041872262954711914, + -2.2858188152313232, + -0.2532116174697876 + ], + [ + -0.1948942095041275, + -0.26143571734428406, + -0.41711851954460144, + 0.43631741404533386, + 0.027200281620025635, + -0.8053562641143799, + -0.6398534178733826, + 0.09320799261331558, + -0.03754061460494995, + 0.052125826478004456, + 0.018615175038576126, + 0.5453782081604004, + -1.841666340827942, + -0.027788981795310974, + 0.14161910116672516, + -0.034317318350076675, + 0.01090770959854126, + -0.06977881491184235, + 0.18669134378433228, + -2.060138463973999, + -0.02414742112159729 + ], + [ + -0.35576683282852173, + -0.22342397272586823, + -0.12707075476646423, + 0.6335762739181519, + 0.29755714535713196, + -0.39805683493614197, + -0.1325763463973999, + -0.044789984822273254, + -0.07913310080766678, + 0.017936237156391144, + 0.02942764386534691, + 0.22497786581516266, + -1.5915617942810059, + 0.07978734374046326, + 0.10442276298999786, + -0.21446844935417175, + -0.12771053612232208, + -0.05680327117443085, + 0.15677303075790405, + -2.2263312339782715, + 0.07280028611421585 + ] + ], + "generated_utilities": [ + 0.181088387966156, + 0.19361381232738495, + 0.1885397732257843, + 0.19568920135498047, + 0.1507587730884552, + 0.15693092346191406, + 0.21208032965660095, + 0.19735300540924072 + ], + "group_id": "ms-5b26c33feed6992386b1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8326650857925415, + 2.0, + 0.15131878852844238, + 0.15007886290550232, + 0.5293327569961548, + 0.06490398943424225, + 0.5425760746002197, + 0.20163774490356445, + 0.15259261429309845, + 0.23924465477466583, + 0.12766429781913757, + 0.08033241331577301, + 0.07788355648517609, + 0.13886235654354095, + 0.09716498851776123, + 0.11682072281837463 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.06662726402282715, + -0.14569896459579468, + 0.04974943399429321, + 0.539892315864563, + -0.19018621742725372, + -0.5884796380996704, + 0.0, + 0.12748029828071594, + 0.11038697510957718, + -0.03454907238483429, + 0.08700031787157059, + -0.13105249404907227, + -1.9082190990447998, + 0.0, + -0.0033895689994096756, + -0.15730009973049164, + 0.15846982598304749, + 0.11786291003227234, + -0.02387581765651703, + -1.5966379642486572, + 0.0 + ], + [ + -0.06662726402282715, + 0.059319257736206055, + -0.19934123754501343, + 0.539892315864563, + 0.07902508974075317, + -0.5884796380996704, + 0.0, + -0.3428957760334015, + 0.08558038622140884, + 0.28825634717941284, + -0.32335156202316284, + -0.3631777763366699, + -1.9082190990447998, + 0.0, + -0.3765835762023926, + -0.5554376840591431, + 0.3468284010887146, + 0.875933825969696, + 0.11397302150726318, + -1.5966379642486572, + 0.0 + ], + [ + -1.0666272640228271, + -0.6187554597854614, + -0.8818012475967407, + 1.539892315864563, + -0.428971529006958, + -1.5884796380996704, + 2.0647666454315186, + -0.03600159287452698, + 0.032059766352176666, + 0.030456876382231712, + 0.14377687871456146, + 0.20562076568603516, + -2.9082190990448, + -1.332404375076294, + -0.014788046479225159, + -0.0006958844605833292, + 0.023885300382971764, + 0.5519534349441528, + 0.1840958297252655, + -2.5966379642486572, + -1.4879226684570312 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.129533290863037, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.664808750152588, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9758453369140625 + ], + [ + -1.2971861362457275, + -0.7044790983200073, + -1.134492039680481, + 1.5821952819824219, + -0.7159937620162964, + -1.7615629434585571, + 2.031708002090454, + -0.25998222827911377, + 0.10180196911096573, + -0.24491730332374573, + 0.2074550986289978, + 0.2793378233909607, + -2.752415657043457, + -1.3360638618469238, + 0.03524862602353096, + -0.1854405552148819, + -0.1696440726518631, + 0.7172544598579407, + 0.0028989464044570923, + -2.680950164794922, + -1.2071388959884644 + ], + [ + -1.465846300125122, + -0.34887754917144775, + -1.046374797821045, + 1.703241229057312, + -0.19744430482387543, + -1.2604204416275024, + 1.9822474718093872, + 0.18854136765003204, + -0.146264910697937, + 0.059789031744003296, + 0.39534950256347656, + 0.29636046290397644, + -2.6612584590911865, + -1.5028973817825317, + 0.059081435203552246, + -0.11711112409830093, + -0.07238245755434036, + 0.25233787298202515, + 0.12625277042388916, + -2.2362401485443115, + -1.7053974866867065 + ], + [ + -1.1892943382263184, + -0.20831987261772156, + -0.8013461828231812, + 1.7197901010513306, + -0.39134201407432556, + -1.5160430669784546, + 2.4594008922576904, + -0.1669330894947052, + -0.337530255317688, + 0.15848031640052795, + 0.05738166719675064, + 0.41757503151893616, + -3.1393940448760986, + -0.9257140755653381, + -0.5068305134773254, + 0.1864723265171051, + 0.45369866490364075, + 0.6036283373832703, + -0.018199875950813293, + -2.8028440475463867, + -1.3847744464874268 + ], + [ + -1.1111165285110474, + -1.191490888595581, + -1.361661434173584, + 1.197826623916626, + -0.29219919443130493, + -1.7355475425720215, + 2.0579750537872314, + -0.08019472658634186, + 0.5140348672866821, + -0.5474883317947388, + -0.03311571478843689, + -0.060686588287353516, + -2.3427765369415283, + -1.8552747964859009, + -0.36303937435150146, + 0.01966947503387928, + -0.5651369094848633, + 0.22617560625076294, + -0.16322728991508484, + -2.421419143676758, + -0.921785831451416 + ], + [ + -1.012363076210022, + -0.6483663320541382, + -0.7951939702033997, + 1.2563079595565796, + -0.5323562622070312, + -1.5913673639297485, + 1.406038761138916, + -0.39036238193511963, + -0.12943506240844727, + -0.4373972415924072, + -0.20462383329868317, + 0.12806488573551178, + -3.570098400115967, + -1.8897531032562256, + 0.6369156837463379, + 0.2483828067779541, + 0.6809477210044861, + 0.8138958215713501, + 0.6604747176170349, + -2.717491865158081, + -1.9953765869140625 + ], + [ + -0.378162145614624, + -0.23438718914985657, + -1.2589956521987915, + 1.844052791595459, + -0.8936926126480103, + -1.230797290802002, + 2.5580525398254395, + -0.21314577758312225, + -0.05935201793909073, + 0.46811148524284363, + 0.38594305515289307, + 1.0049593448638916, + -2.491497278213501, + -1.3096134662628174, + 0.15605805814266205, + 0.7682130932807922, + -0.4010901153087616, + 0.8638691902160645, + 0.5755372047424316, + -3.324826240539551, + -1.6860322952270508 + ], + [ + -0.6920164227485657, + -0.5060986280441284, + -0.5146759748458862, + 2.3206489086151123, + 0.26810598373413086, + -1.7696658372879028, + 1.4922571182250977, + -0.6155052185058594, + -0.5581206679344177, + 0.4068271219730377, + 0.5812678337097168, + -0.2056950330734253, + -2.227752923965454, + -0.7079744935035706, + 0.3153572678565979, + 0.019053615629673004, + -0.10691303759813309, + -0.18750637769699097, + 0.5490041971206665, + -2.7611911296844482, + -1.0861625671386719 + ], + [ + -0.9623568654060364, + -0.821997880935669, + -0.0017323493957519531, + 1.8667385578155518, + 0.39793866872787476, + -1.3673040866851807, + 2.8636348247528076, + -0.739641547203064, + -0.4186078906059265, + -0.21357670426368713, + 0.6820193529129028, + 0.09538085013628006, + -3.242215633392334, + -1.023385763168335, + -0.2367737591266632, + 0.9248462915420532, + 0.04536557197570801, + 0.2294020652770996, + 0.15174128115177155, + -2.509904146194458, + -0.5901334881782532 + ], + [ + -0.565070390701294, + 0.37745314836502075, + -1.7464253902435303, + 1.4869437217712402, + -1.187619924545288, + -2.3802757263183594, + 2.5987753868103027, + 0.5777139663696289, + -0.04772312939167023, + -0.49332448840141296, + -0.515097975730896, + 0.751485288143158, + -2.7030012607574463, + -1.5739474296569824, + 0.8266244530677795, + -0.9454089403152466, + 0.7678747177124023, + 1.0557427406311035, + 0.0584852397441864, + -2.8590199947357178, + -0.8677505850791931 + ], + [ + -0.2763708233833313, + 0.28772038221359253, + -0.7008944153785706, + 1.3402774333953857, + -0.78607177734375, + -2.1210508346557617, + 1.4908761978149414, + -0.4155166447162628, + 0.40930652618408203, + -0.1769891083240509, + -0.20092366635799408, + 0.5321645736694336, + -3.2144649028778076, + -1.2440845966339111, + -0.247700035572052, + 0.7174559235572815, + 0.11071103811264038, + -0.023232340812683105, + 0.6976174116134644, + -2.9463140964508057, + -0.5927516222000122 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.06662726402282715, + -0.05619156360626221, + -0.02257448434829712, + 0.539892315864563, + 0.053197890520095825, + -0.5884796380996704, + 0.0, + -0.04219311475753784, + 0.010949447751045227, + 0.027476036921143532, + -0.02041013538837433, + -0.01791653037071228, + -1.9082190990447998, + 0.0, + 0.01256529800593853, + 0.014024228788912296, + 0.03545428439974785, + 0.019800782203674316, + -0.022738635540008545, + -1.5966379642486572, + 0.0 + ] + ], + "predicted_scores": [ + 6.4563469886779785, + 6.449648857116699, + 6.180249214172363, + 6.115482330322266, + 5.748321533203125, + 5.519894599914551, + 5.004046440124512, + 4.754415035247803 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1522272686", + "selected_index": 0, + "state_hash": "10034ca87ce4b5c491c0aa23d2a30389ba8265fa695d65198bf719c54565cf2a", + "stored_base_utility": 0.8326650857925415, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8388344049453735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8388344049453735 + }, + "base_progress": 0.8388344049453735, + "base_success": false, + "base_utility": 0.8388344049453735, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8135772347450256, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8135772347450256 + }, + { + "contact_quality": null, + "progress": 0.1985817849636078, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1985817849636078 + }, + { + "contact_quality": null, + "progress": 0.20784005522727966, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20784005522727966 + }, + { + "contact_quality": null, + "progress": 0.20173269510269165, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20173269510269165 + }, + { + "contact_quality": null, + "progress": 0.2171078473329544, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2171078473329544 + }, + { + "contact_quality": null, + "progress": 0.18976537883281708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18976537883281708 + }, + { + "contact_quality": null, + "progress": 0.20651468634605408, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20651468634605408 + }, + { + "contact_quality": null, + "progress": 0.22675341367721558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22675341367721558 + } + ], + "candidate_progress": [ + 0.8135772347450256, + 0.1985817849636078, + 0.20784005522727966, + 0.20173269510269165, + 0.2171078473329544, + 0.18976537883281708, + 0.20651468634605408, + 0.22675341367721558 + ], + "candidate_source_chart_ids": [ + "ms-eb7189a4f2b0f0a5d4bd", + "ms-5a456ddf97d294548cf1", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-0b226017db7f07c5307e", + "ms-5dfbb16e173e1a3e2bd6" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-63ac245579b1188935ad", + "generated_tangents": [ + [ + 0.024005480110645294, + -0.03487171605229378, + -0.08501949906349182, + 0.1531437635421753, + 0.059081971645355225, + -1.9487231969833374, + -0.12814170122146606, + -0.027122944593429565, + -0.06875290721654892, + -0.05074097961187363, + 0.21672600507736206, + 0.17542648315429688, + -1.233362078666687, + -0.028380826115608215, + 0.049982257187366486, + 0.07889854907989502, + 0.02847789227962494, + -0.18392501771450043, + 0.09474422037601471, + -1.1280198097229004, + -0.10618850588798523 + ], + [ + 0.0801134705543518, + -0.04132566228508949, + -0.07826349139213562, + 0.05996397137641907, + -0.05156397819519043, + -1.947234034538269, + -0.08109259605407715, + 0.011223122477531433, + -0.07242417335510254, + 0.03295724838972092, + 0.09064805507659912, + 0.08040205389261246, + -1.288273811340332, + -0.036930009722709656, + -0.016599636524915695, + 0.052263420075178146, + 0.09989877045154572, + -0.20034591853618622, + 0.056101441383361816, + -1.1894292831420898, + -0.04864327609539032 + ], + [ + 0.11546728014945984, + -0.01809103786945343, + -0.11312094330787659, + 0.02970258891582489, + -0.06853586435317993, + -2.0808277130126953, + -0.22655653953552246, + 0.04426082968711853, + -0.09063653647899628, + -0.07344181090593338, + 0.1791953146457672, + -0.04234561324119568, + -1.3234821557998657, + -0.15130607783794403, + 0.024940162897109985, + 0.07820416986942291, + 0.10006025433540344, + -0.10336704552173615, + 0.03928545117378235, + -1.260929822921753, + -0.13187727332115173 + ], + [ + 0.08999529480934143, + -0.017735548317432404, + -0.07061442732810974, + 0.00416797399520874, + -0.1770777702331543, + -1.8116111755371094, + -0.2593742609024048, + 0.06652607023715973, + -0.12665021419525146, + -0.08371938019990921, + 0.11404116451740265, + 0.046173423528671265, + -1.135612964630127, + -0.18963705003261566, + 0.07155746966600418, + 0.08745545148849487, + 0.1271779090166092, + -0.10870565474033356, + -0.06397175788879395, + -1.120802640914917, + -0.18013185262680054 + ], + [ + 0.04854058474302292, + -0.03224082291126251, + -0.04689648747444153, + -0.010309815406799316, + -0.11356344819068909, + -1.9703437089920044, + -0.23080027103424072, + 0.06358575820922852, + -0.11792464554309845, + -0.03149311989545822, + 0.08329933136701584, + 0.0213300883769989, + -1.308495283126831, + -0.12958507239818573, + 0.044057831168174744, + 0.06347276270389557, + 0.07978248596191406, + -0.1787070482969284, + -0.034421712160110474, + -1.2446329593658447, + -0.1634177267551422 + ], + [ + 0.1383705884218216, + 0.010324206203222275, + -0.07752159237861633, + -0.0030374079942703247, + -0.01030801236629486, + -1.7836098670959473, + -0.18338048458099365, + 0.1067870482802391, + -0.16640116274356842, + 0.031050056219100952, + 0.22548094391822815, + 0.028106689453125, + -1.0555660724639893, + -0.08256937563419342, + -0.003275398164987564, + 0.034911006689071655, + 0.05868661403656006, + -0.023772746324539185, + -0.0549464225769043, + -1.021216630935669, + -0.07599213719367981 + ], + [ + 0.03586271405220032, + -0.052382975816726685, + -0.016012579202651978, + -0.00796923041343689, + -0.16297471523284912, + -2.0362632274627686, + -0.3756687641143799, + 0.04673069715499878, + -0.10398165136575699, + -0.06536772102117538, + 0.16496363282203674, + 0.08488186448812485, + -1.3018834590911865, + -0.12473703920841217, + 0.0551728680729866, + 0.14191022515296936, + 0.16451120376586914, + -0.09062518179416656, + -0.04602450132369995, + -1.2958133220672607, + -0.1575165092945099 + ], + [ + 0.0409599244594574, + -0.10623904317617416, + -0.14421579241752625, + -0.2416801005601883, + -0.1338302195072174, + -2.4233953952789307, + -0.45108890533447266, + 0.2016845941543579, + -0.12244339287281036, + -0.1424170434474945, + 0.3580399751663208, + -0.14196431636810303, + -1.1082514524459839, + -0.19893445074558258, + -0.08443394303321838, + 0.1523725390434265, + 0.13660648465156555, + -0.1062401682138443, + -0.09151360392570496, + -1.2155197858810425, + -0.29695650935173035 + ] + ], + "generated_utilities": [ + 0.8135772347450256, + 0.1985817849636078, + 0.20784005522727966, + 0.20173269510269165, + 0.2171078473329544, + 0.18976537883281708, + 0.20651468634605408, + 0.22675341367721558 + ], + "group_id": "ms-63ac245579b1188935ad", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.855198860168457, + 0.8138954639434814, + 0.20312510430812836, + 2.0, + 0.7862945795059204, + 0.7843606472015381, + 0.6004383563995361, + 0.15506848692893982, + 0.15924105048179626, + 0.132452130317688, + 0.05381816625595093, + 0.09561765193939209, + 0.07140937447547913, + 0.10806754231452942, + 0.023168057203292847, + 0.07957816123962402 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.007633227854967117, + 0.02389371022582054, + 0.2181050330400467, + 0.026324495673179626, + 0.1209479570388794, + -1.3937149047851562, + 0.0, + -0.2493116855621338, + -0.17256586253643036, + -0.06812826544046402, + 0.019445210695266724, + -0.2657848000526428, + -1.0657541751861572, + 0.0, + -0.04306507110595703, + -0.07170329242944717, + 0.035314902663230896, + 0.06211268901824951, + 0.18158195912837982, + -0.7849693298339844, + 0.0 + ], + [ + -0.011555496603250504, + 0.0016381663735955954, + 0.019193550571799278, + 0.26171422004699707, + 0.08122420310974121, + -2.3937149047851562, + -1.53977632522583, + -0.00580335408449173, + -0.0020052569452673197, + 0.01787502132356167, + 0.4615197777748108, + 0.1938573122024536, + -2.0657541751861572, + -1.4660639762878418, + -0.00267898291349411, + -0.0038005562964826822, + 0.0179074015468359, + 0.5644202828407288, + 0.2690354883670807, + -1.7849693298339844, + -1.3308851718902588 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.07955265045166, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9321279525756836, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.6617703437805176 + ], + [ + -0.16036809980869293, + 0.11436319351196289, + -0.14857570827007294, + 0.09414434432983398, + 0.02274160459637642, + -2.426297903060913, + -1.7686878442764282, + -0.26862773299217224, + 0.16241224110126495, + 0.24208402633666992, + 0.4166663587093353, + 0.07841069251298904, + -2.2318098545074463, + -1.2228643894195557, + -0.08888694643974304, + 0.20051127672195435, + -0.14729085564613342, + 0.5799955129623413, + 0.33039355278015137, + -2.00492262840271, + -1.4948352575302124 + ], + [ + -0.3820815682411194, + -0.3679526746273041, + 0.18845602869987488, + 0.4168931245803833, + -0.007936321198940277, + -2.700572967529297, + -1.795093059539795, + 0.1203194335103035, + 0.19610334932804108, + 0.2749679982662201, + 0.8331882357597351, + -0.06589585542678833, + -2.333136558532715, + -1.1008005142211914, + -0.02170141413807869, + 0.24186654388904572, + -0.3734443485736847, + 0.5144574046134949, + 0.3134346902370453, + -1.5088386535644531, + -1.1978046894073486 + ], + [ + 0.13625949621200562, + 0.47836095094680786, + -0.05790846049785614, + -0.09614765644073486, + -0.14281350374221802, + -2.522264003753662, + -1.0473889112472534, + 0.4792231023311615, + -0.4869813621044159, + 0.17644789814949036, + 0.8425257205963135, + -0.28899794816970825, + -2.5508456230163574, + -0.9762126207351685, + -0.16381770372390747, + 0.1586131900548935, + -0.3460317850112915, + 0.6056831479072571, + 0.7327044010162354, + -1.494289755821228, + -1.4671605825424194 + ], + [ + -0.40018847584724426, + -0.028111951425671577, + 0.31324630975723267, + -0.22690656781196594, + 0.2902344763278961, + -2.9841673374176025, + -1.4654184579849243, + -0.1155615746974945, + -0.4125809073448181, + 0.6098153591156006, + 0.8544195890426636, + 0.22702345252037048, + -1.7853796482086182, + -1.3474400043487549, + -0.454882949590683, + -0.3122820556163788, + 0.22399993240833282, + 0.05299121141433716, + 0.07802119851112366, + -1.3896000385284424, + -1.894580602645874 + ], + [ + -0.6587541699409485, + -0.6905457973480225, + 0.10385943204164505, + -0.2785833477973938, + 0.2321375161409378, + -2.565701723098755, + -1.1605603694915771, + -0.6775721907615662, + -0.30887091159820557, + 0.1186944767832756, + 1.0924456119537354, + 0.5009081363677979, + -1.8667830228805542, + -1.2491121292114258, + -0.11290804296731949, + -0.03346216678619385, + -0.28778207302093506, + 1.2255009412765503, + 0.6856998801231384, + -1.1723419427871704, + -1.1220875978469849 + ], + [ + -0.5019480586051941, + 0.07311037182807922, + 0.5101989507675171, + 0.7159117460250854, + 0.005071915686130524, + -2.6556949615478516, + -2.321233034133911, + 0.6035679578781128, + -0.13785676658153534, + -0.4309522807598114, + 0.9664275050163269, + -0.020235419273376465, + -1.6894025802612305, + -0.7511875033378601, + 0.5713916420936584, + 0.24962951242923737, + 0.40067923069000244, + 0.12209588289260864, + 0.03222101926803589, + -2.27744460105896, + -1.9113109111785889 + ], + [ + 0.7552750110626221, + -0.8085685968399048, + 0.20896124839782715, + -0.6043995022773743, + 0.733860969543457, + -1.610467791557312, + -1.0368051528930664, + -0.02292630262672901, + 0.6320546269416809, + 0.14528857171535492, + 0.8372507095336914, + 0.41523146629333496, + -2.339751958847046, + -0.6185142993927002, + -0.5750091075897217, + 0.12324154376983643, + 0.2804809510707855, + 1.0503896474838257, + -0.15392154455184937, + -0.9555137753486633, + -1.517809510231018 + ], + [ + 0.8853163123130798, + 0.09475164860486984, + -0.13681088387966156, + 0.08429434895515442, + 0.13555681705474854, + -1.5859344005584717, + -2.1957859992980957, + 0.9337366819381714, + -1.0016963481903076, + 0.11365362256765366, + -0.31175100803375244, + 0.7412776350975037, + -1.0797133445739746, + -0.8527018427848816, + 0.6110302209854126, + 0.006565961055457592, + 0.38363590836524963, + 0.5695288181304932, + -0.2547180950641632, + -2.6268043518066406, + -1.748514175415039 + ], + [ + 0.513801097869873, + 0.7078304886817932, + 0.7309613823890686, + -0.32746708393096924, + -0.36263394355773926, + -3.3305888175964355, + -1.2108653783798218, + 0.5104080438613892, + -0.26447102427482605, + 0.6361017823219299, + 0.5986918210983276, + -0.48598653078079224, + -2.4424209594726562, + -1.4694169759750366, + -0.5649402737617493, + -0.22020848095417023, + 0.6405193209648132, + -0.0698668360710144, + 0.7155715227127075, + -2.352860450744629, + -0.7502654194831848 + ], + [ + -0.9911022782325745, + 0.9677192568778992, + 0.5195661187171936, + 0.6733791828155518, + 0.7589308023452759, + -3.3654959201812744, + -0.6360958814620972, + 0.5926312208175659, + -0.06773044914007187, + 0.3785405457019806, + 0.5934506058692932, + 0.6434512138366699, + -2.891094923019409, + -0.8924227356910706, + 0.8036434650421143, + 0.6109268665313721, + -0.6607317924499512, + 1.0173940658569336, + 0.2546094059944153, + -1.4653143882751465, + -2.0709643363952637 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16686837375164032, + 0.1354454904794693, + -0.11150697618722916, + 0.7247087359428406, + 0.2886054813861847, + -1.3937149047851562, + 0.0, + 0.6050093770027161, + 0.20291854441165924, + 0.5827622413635254, + -0.2687305808067322, + -0.25171223282814026, + -1.0657541751861572, + 0.0, + 0.29065752029418945, + -0.023413484916090965, + -0.24587245285511017, + 0.4600236713886261, + -0.4278547465801239, + -0.7849693298339844, + 0.0 + ] + ], + "predicted_scores": [ + 3.353055715560913, + 3.3517229557037354, + 3.3192176818847656, + 3.2793989181518555, + 3.27476167678833, + 2.8849124908447266, + 2.8019933700561523, + 2.0151007175445557 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "211245842", + "selected_index": 0, + "state_hash": "fb52e84b6addb8bdddf02a80d5c8567e0fa27195729e8440271b710441061b7f", + "stored_base_utility": 0.855198860168457, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-875a4668671c02054a3c", + "ms-20f3cfcf183c69853b1e", + "ms-0949097d19e597fc7f63", + "ms-875a4668671c02054a3c", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "generated_tangents": [ + [ + 0.006176412105560303, + 0.015031468123197556, + -0.04347166419029236, + -0.2196698635816574, + -0.20213130116462708, + -2.1178388595581055, + -0.2735292911529541, + 0.053335875272750854, + 0.023975348100066185, + 0.04375610500574112, + 0.06376722455024719, + 0.05684523284435272, + -1.8778741359710693, + -0.11852577328681946, + 0.09389108419418335, + 0.05454123392701149, + 0.007951110601425171, + -0.1374041885137558, + 0.05853183567523956, + -1.7561705112457275, + -0.08032986521720886 + ], + [ + 0.05350024253129959, + -0.008277975022792816, + -0.011172980070114136, + -0.19443228840827942, + -0.06973081827163696, + -2.4749886989593506, + -0.21836239099502563, + -0.0009081363677978516, + 0.04159802198410034, + 0.03741426020860672, + 0.06233856454491615, + -0.03803148865699768, + -1.9127192497253418, + -0.11715845763683319, + 0.03506483882665634, + 0.05396481230854988, + 0.05986599624156952, + -0.1762569695711136, + 0.054152652621269226, + -1.5897595882415771, + -0.030290141701698303 + ], + [ + 0.10290880501270294, + -0.007280625402927399, + 0.07842746376991272, + -0.1832699030637741, + 0.05195482075214386, + -2.537594795227051, + -0.46784335374832153, + -0.02841256558895111, + 0.006892894394695759, + -0.015205807983875275, + 0.2386656403541565, + 0.1220480352640152, + -1.7291165590286255, + -0.07446907460689545, + -0.008053001016378403, + 0.07738210260868073, + 0.08646567165851593, + -0.17380063235759735, + 0.03588852286338806, + -1.7457711696624756, + 0.07436027377843857 + ], + [ + 0.02357722818851471, + 0.01157565787434578, + -0.058383435010910034, + -0.20330168306827545, + -0.13086026906967163, + -2.556213140487671, + -0.6527280211448669, + 0.12240959703922272, + 0.010187995620071888, + -0.015915729105472565, + 0.1939363181591034, + 0.03682531416416168, + -1.8583874702453613, + -0.553052544593811, + 0.03224678337574005, + 0.11126010119915009, + 0.0709986686706543, + -0.048767030239105225, + 0.21647703647613525, + -1.8499352931976318, + -0.08911240100860596 + ], + [ + -0.0158156156539917, + -0.025778014212846756, + 0.046597957611083984, + -0.2615222632884979, + 0.09938192367553711, + -2.2407608032226562, + -0.43458765745162964, + 0.21323370933532715, + 0.059951357543468475, + 0.04161942005157471, + 0.023588914424180984, + 0.14518174529075623, + -1.6217304468154907, + -0.10320907831192017, + 0.19596292078495026, + 0.11118914186954498, + 0.06349106132984161, + -0.1608113795518875, + 0.2863720953464508, + -1.8274946212768555, + -0.01357463002204895 + ], + [ + 0.011767670512199402, + -0.06779482215642929, + 0.2283744513988495, + -0.4103007912635803, + -0.03333577513694763, + -2.293379545211792, + -0.6917316317558289, + 0.25185275077819824, + -0.027487674728035927, + 0.011037394404411316, + 0.22671815752983093, + 0.16355301439762115, + -1.3806061744689941, + -0.060945481061935425, + 0.20913317799568176, + 0.03424268215894699, + 0.07699966430664062, + -0.22638840973377228, + 0.2093367576599121, + -1.8172316551208496, + -0.06600260734558105 + ], + [ + 0.043054208159446716, + 0.006995994597673416, + 0.040226101875305176, + -0.20659546554088593, + 0.016365110874176025, + -2.440964698791504, + -0.5692809224128723, + 0.08967659622430801, + -0.005274813622236252, + -0.007912807166576385, + 0.11555744707584381, + 0.13973061740398407, + -1.6668994426727295, + -0.20925067365169525, + 0.01616767793893814, + 0.10505478084087372, + 0.05943283438682556, + -0.22194214165210724, + 0.15125006437301636, + -1.7992901802062988, + 0.15188738703727722 + ], + [ + 0.12475170940160751, + -0.069279246032238, + -0.30603304505348206, + -0.3186453878879547, + 0.4094564914703369, + -1.6150081157684326, + -0.6829964518547058, + 0.06369796395301819, + 0.06468655169010162, + 0.0865817442536354, + -0.09368977695703506, + -0.0324195921421051, + -0.980171799659729, + -0.3847846984863281, + 0.31843870878219604, + 0.08126239478588104, + 0.1735626757144928, + -0.2221849113702774, + 0.3698451817035675, + -1.811368465423584, + 0.22155342996120453 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-6c8d06bace98a28a8e4e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8583254218101501, + 2.0, + 0.8125, + 0.12893107533454895, + 0.8582976460456848, + 0.15346482396125793, + 0.18961156904697418, + 0.13901273906230927, + 0.10113435983657837, + 0.11161506175994873, + 0.09923169016838074, + 0.16662254929542542, + 0.04211682081222534, + 0.024710848927497864, + 0.05722464621067047, + 0.8584086298942566 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.10696407407522202, + 0.3957799971103668, + 0.15132969617843628, + -0.32772913575172424, + 0.005261177197098732, + -2.2650365829467773, + 0.0, + 0.3958877623081207, + -0.5120954513549805, + -0.4317946135997772, + -0.3970779776573181, + 0.5317854285240173, + -1.8659565448760986, + 0.0, + 0.1198807880282402, + -0.11760073900222778, + -0.6048114895820618, + 0.15180528163909912, + -0.532695472240448, + -1.6115882396697998, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.120756149291992, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.070481538772583, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0495758056640625 + ], + [ + 0.30105674266815186, + 0.4776385426521301, + -0.22267001867294312, + -0.13300693035125732, + -0.11095686256885529, + -3.138023853302002, + -0.793931245803833, + 0.16969743371009827, + -0.16339385509490967, + 0.02508675307035446, + 0.654399573802948, + -0.22830897569656372, + -3.1443004608154297, + -1.4291013479232788, + 0.005021120421588421, + 0.005560946650803089, + -0.1901366114616394, + 1.0211858749389648, + -0.09353887289762497, + -2.3457469940185547, + -1.406853199005127 + ], + [ + 0.4160586893558502, + 0.13316196203231812, + -0.21056225895881653, + -0.25006812810897827, + 0.060951750725507736, + -2.8735690116882324, + -0.9062713980674744, + -0.28707224130630493, + -0.2396206259727478, + -0.19859735667705536, + 0.29039445519447327, + -0.24077445268630981, + -2.644789934158325, + -1.8330551385879517, + -0.23438936471939087, + 0.04620177671313286, + 0.1472887098789215, + 0.9629149436950684, + -0.020509399473667145, + -2.7985036373138428, + -1.2463163137435913 + ], + [ + 0.2486386001110077, + -0.022333532571792603, + 0.14628465473651886, + -0.8381847143173218, + 0.21282373368740082, + -3.585387945175171, + -1.2156991958618164, + -0.08670558780431747, + -0.3178575336933136, + 0.4322021007537842, + 0.6421236395835876, + -0.33922863006591797, + -3.004714012145996, + -1.19637131690979, + -0.015640778467059135, + 0.10077434033155441, + -0.449941486120224, + 1.0431067943572998, + 0.2899783253669739, + -2.500624179840088, + -1.6148788928985596 + ], + [ + 0.46770766377449036, + 0.03902164101600647, + 0.2556130588054657, + -0.45893940329551697, + 0.08939866721630096, + -3.163339853286743, + -0.7599828243255615, + 0.3327450156211853, + 0.3252364993095398, + 0.3067358434200287, + 1.1394517421722412, + 0.472858726978302, + -3.1504058837890625, + -1.9780522584915161, + -0.26712900400161743, + -0.262444406747818, + 0.20269782841205597, + 1.336982250213623, + -0.06957009434700012, + -2.0651021003723145, + -1.8484716415405273 + ], + [ + 0.48348987102508545, + 0.19676931202411652, + -0.20689666271209717, + -0.21521174907684326, + -0.5048230886459351, + -3.491732358932495, + -0.7390058636665344, + 0.18321916460990906, + -0.1509065330028534, + 0.27166980504989624, + 1.004181981086731, + -0.5531666874885559, + -2.531594753265381, + -2.0243897438049316, + -0.44593361020088196, + 0.49786314368247986, + -0.006632372736930847, + 0.24423635005950928, + -0.6331305503845215, + -2.7707719802856445, + -2.1364808082580566 + ], + [ + -0.5684993863105774, + 0.0623701810836792, + -0.10449839383363724, + -0.7858190536499023, + -0.057080868631601334, + -3.981018304824829, + -0.6721940636634827, + 0.10191962122917175, + -0.7216448783874512, + 0.3870577812194824, + 0.5508396625518799, + -0.6811783313751221, + -3.6131227016448975, + -1.7484180927276611, + 0.34220433235168457, + -0.16193144023418427, + 0.16876991093158722, + 0.6509957909584045, + 0.2548854351043701, + -2.0779964923858643, + -1.5782216787338257 + ], + [ + -0.1224658265709877, + 0.22953394055366516, + -0.41326335072517395, + 0.19802969694137573, + 0.6866055130958557, + -2.6415462493896484, + -1.049781322479248, + -0.6788480281829834, + -0.8187242746353149, + 0.6141759157180786, + 1.3649951219558716, + -0.36378854513168335, + -2.2142889499664307, + -1.3759111166000366, + 0.583359956741333, + 0.6860965490341187, + 0.8285356760025024, + 1.4575884342193604, + -0.4101440906524658, + -2.218319892883301, + -1.5390111207962036 + ], + [ + 0.3368571698665619, + 0.4528505206108093, + -0.2985517382621765, + 0.37766069173812866, + 0.2064778208732605, + -2.8070714473724365, + -1.5598305463790894, + -0.9032005071640015, + -0.48420336842536926, + 0.639835000038147, + -0.3801422119140625, + -0.1376074254512787, + -3.369077205657959, + -1.9231767654418945, + -0.16211053729057312, + 0.53387051820755, + 0.6583098769187927, + 0.7611914873123169, + -0.46404945850372314, + -2.7554476261138916, + -1.4307246208190918 + ], + [ + 0.6002402901649475, + -0.14213827252388, + 0.30183425545692444, + -0.20118463039398193, + -0.16981711983680725, + -3.4212875366210938, + -0.2252577543258667, + 0.34123149514198303, + 0.003176456317305565, + 0.7433648109436035, + 1.1403348445892334, + 0.5709706544876099, + -1.8894051313400269, + -0.7200812101364136, + -0.7235198020935059, + 0.2147149294614792, + -0.28744715452194214, + 1.0323235988616943, + 0.5815067291259766, + -2.582993984222412, + -1.2160475254058838 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.019069351255893707, + 0.012574762105941772, + 0.07413499802350998, + -0.04097828269004822, + -0.03991875424981117, + -2.2650365829467773, + 0.0, + -0.06082560494542122, + 0.05821063742041588, + 0.07896332442760468, + -0.037061095237731934, + -0.08879614621400833, + -1.8659565448760986, + 0.0, + -0.022465867921710014, + -0.026843247935175896, + 0.04789978265762329, + -0.019972383975982666, + -0.04040021449327469, + -1.6115882396697998, + 0.0 + ] + ], + "predicted_scores": [ + 3.6132442951202393, + 3.3841521739959717, + 2.2444205284118652, + 2.1902849674224854, + 2.1477720737457275, + 2.0814521312713623, + 1.9497334957122803, + 0.9199833869934082 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1665739511", + "selected_index": 0, + "state_hash": "0f77663511063f17b0962024501b863c62b881513459d8bfa3ac00bbaed67de4", + "stored_base_utility": 0.8583254218101501, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1245432198047638, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1245432198047638 + }, + { + "contact_quality": null, + "progress": 0.574478030204773, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.574478030204773 + }, + { + "contact_quality": null, + "progress": 0.23411184549331665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23411184549331665 + }, + { + "contact_quality": null, + "progress": 0.23169663548469543, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23169663548469543 + }, + { + "contact_quality": null, + "progress": 0.1756683886051178, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1756683886051178 + }, + { + "contact_quality": null, + "progress": 0.1700567752122879, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1700567752122879 + }, + { + "contact_quality": null, + "progress": 0.20359350740909576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20359350740909576 + }, + { + "contact_quality": null, + "progress": 0.1943017691373825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1943017691373825 + } + ], + "candidate_progress": [ + 0.1245432198047638, + 0.574478030204773, + 0.23411184549331665, + 0.23169663548469543, + 0.1756683886051178, + 0.1700567752122879, + 0.20359350740909576, + 0.1943017691373825 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-72777024349c2aacd5e0", + "generated_tangents": [ + [ + 0.15045036375522614, + 0.1372452974319458, + 0.29213792085647583, + -0.8182337284088135, + 0.47608843445777893, + -0.38249924778938293, + 0.5203595161437988, + 0.14830099046230316, + -0.013280622661113739, + -0.10061783343553543, + -0.06293529272079468, + 0.06389458477497101, + -0.6957372426986694, + -0.5106804966926575, + 0.1213463544845581, + -0.08478164672851562, + 0.32867300510406494, + -0.3360060453414917, + -0.11113515496253967, + -1.101579189300537, + -0.2569182813167572 + ], + [ + 0.17874856293201447, + 0.2526323199272156, + -0.24080786108970642, + -0.6450339555740356, + -0.06338244676589966, + -0.6732321977615356, + 0.23784828186035156, + -0.06323722004890442, + -0.06856551021337509, + -0.13422897458076477, + 0.34170064330101013, + -0.10431668162345886, + -1.5888500213623047, + -0.36079859733581543, + -0.029090922325849533, + -0.0446627214550972, + -0.056386008858680725, + -0.07992379367351532, + -0.31195753812789917, + -1.8153178691864014, + -0.0052912235260009766 + ], + [ + 0.20602259039878845, + 0.1785765141248703, + -0.4026523530483246, + -0.8586504459381104, + -0.08055484294891357, + -0.41225311160087585, + 0.023905158042907715, + -0.006460845470428467, + -0.09959809482097626, + -0.2161354124546051, + 0.09675844013690948, + -0.2766232490539551, + -1.6789705753326416, + -0.3824644088745117, + -0.10450944304466248, + -0.02792646549642086, + -0.011458426713943481, + -0.1315893679857254, + -0.37362349033355713, + -1.5534007549285889, + 0.09506411105394363 + ], + [ + 0.03876104950904846, + 0.2657865285873413, + -0.26557520031929016, + -0.6566753387451172, + -0.04386025667190552, + -0.6304527521133423, + 0.1770125925540924, + -0.00988796353340149, + -0.043510593473911285, + -0.11370385438203812, + 0.1605052351951599, + -0.1115913987159729, + -1.4697049856185913, + -0.2920401692390442, + -0.09245873987674713, + -0.010158668272197247, + 0.12769964337348938, + -0.08878220617771149, + -0.31268852949142456, + -1.7868881225585938, + 0.08572258055210114 + ], + [ + -0.08376432955265045, + 0.18322238326072693, + -0.050839394330978394, + -0.6801820993423462, + 0.17961299419403076, + -0.4552444517612457, + 0.05471804738044739, + 0.07211660593748093, + 0.003387349657714367, + -0.038424305617809296, + -0.04805950075387955, + -0.15914759039878845, + -1.7790008783340454, + 0.07511967420578003, + -0.0040402524173259735, + -0.07099927216768265, + 0.16272948682308197, + -0.023879870772361755, + 0.09710381925106049, + -1.4573769569396973, + 0.11534436047077179 + ], + [ + 0.025695331394672394, + 0.19272837042808533, + -0.015701264142990112, + -0.692416787147522, + 0.17331527173519135, + -0.2513943612575531, + -0.05424690246582031, + -0.001001805067062378, + -0.012060513719916344, + -0.058356873691082, + -0.06549026072025299, + -0.09043657779693604, + -1.683553695678711, + 0.008678898215293884, + 0.0041976794600486755, + 0.012104487046599388, + 0.1945856362581253, + -0.135456845164299, + -0.016347736120224, + -1.4026732444763184, + 0.06931431591510773 + ], + [ + 0.07290415465831757, + 0.17387908697128296, + -0.07721421122550964, + -0.7396707534790039, + 0.13897329568862915, + -0.23482796549797058, + -0.13062971830368042, + 0.02382473647594452, + -0.02543010748922825, + -0.015188314020633698, + 0.0024499744176864624, + -0.19603782892227173, + -1.6690139770507812, + 0.0981595516204834, + -0.09673325717449188, + -0.021517161279916763, + 0.07098709046840668, + -0.11696939170360565, + -0.00607028603553772, + -1.488534927368164, + 0.052709437906742096 + ], + [ + 0.18209514021873474, + 0.2496199756860733, + -0.23308709263801575, + -0.7942858934402466, + -0.0037521719932556152, + -0.45543625950813293, + 0.12795934081077576, + -0.000754043459892273, + 0.0014778561890125275, + 0.055383406579494476, + -0.03958898037672043, + 0.01392875611782074, + -1.5706225633621216, + -0.0915948748588562, + 0.05942995101213455, + 0.00810825452208519, + 0.08427196741104126, + -0.14305253326892853, + -0.24507790803909302, + -1.9048233032226562, + 0.16264042258262634 + ] + ], + "generated_utilities": [ + 0.1245432198047638, + 0.574478030204773, + 0.23411184549331665, + 0.23169663548469543, + 0.1756683886051178, + 0.1700567752122879, + 0.20359350740909576, + 0.1943017691373825 + ], + "group_id": "ms-72777024349c2aacd5e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8314920663833618, + 0.1595238745212555, + 2.0, + 0.16971935331821442, + 0.5892879366874695, + 0.1617850512266159, + 0.1877371072769165, + 0.5782577395439148, + 0.12074722349643707, + 0.11237785220146179, + 0.08792310953140259, + 0.1209893673658371, + 0.13739405572414398, + 0.0773405134677887, + 0.09497407078742981, + 0.047447383403778076 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.002835560590028763, + 0.3854118585586548, + -0.06168194115161896, + -0.16570186614990234, + -0.05051702260971069, + -0.43052077293395996, + 0.0, + 0.024141468107700348, + -0.011737043969333172, + -0.05502588301897049, + 0.013536334037780762, + -0.000911470502614975, + -1.6284780502319336, + 0.0, + 0.030459173023700714, + -0.05656937509775162, + 0.03393230587244034, + 0.06894856691360474, + 0.00450655072927475, + -1.2762084007263184, + 0.0 + ], + [ + 0.2191334217786789, + 0.3854118585586548, + 0.5048428177833557, + -0.16570186614990234, + -0.20026534795761108, + -0.43052077293395996, + 0.0, + -0.026082046329975128, + 0.03163425251841545, + -0.04950854927301407, + 0.1969367265701294, + -0.05762401595711708, + -1.6284780502319336, + 0.0, + 0.002695085946470499, + 0.34764212369918823, + 0.6051180362701416, + 0.9295560717582703, + 0.38285934925079346, + -1.2762084007263184, + 0.0 + ], + [ + -0.04624907672405243, + 1.3854118585586548, + 0.05864633619785309, + -1.1657018661499023, + 0.7997346520423889, + -1.43052077293396, + 1.1980153322219849, + -0.023269198834896088, + -0.026228107511997223, + -0.009256521239876747, + 0.6858966946601868, + 0.03280947729945183, + -2.6284780502319336, + -1.7920862436294556, + -0.0004090145230293274, + -0.00536443293094635, + 0.007614931557327509, + 0.7401981949806213, + 0.11622340977191925, + -2.2762084007263184, + -1.6813035011291504 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.3960306644439697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.584172487258911, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.362607002258301 + ], + [ + -0.13329119980335236, + 1.1171538829803467, + -0.19378040730953217, + -0.9341158866882324, + 0.8714147806167603, + -1.659934163093567, + 1.1658833026885986, + 0.23766514658927917, + 0.21810689568519592, + -0.05730185657739639, + 0.4946695566177368, + 0.17369553446769714, + -2.559515953063965, + -1.5605767965316772, + 0.08475297689437866, + 0.22485606372356415, + -0.2591644823551178, + 0.9637633562088013, + -0.08890970051288605, + -2.1015453338623047, + -1.5709789991378784 + ], + [ + -0.3507236838340759, + 1.1737966537475586, + 0.10341192036867142, + -0.7761039137840271, + 0.47493064403533936, + -1.2297213077545166, + 1.4181653261184692, + -0.1981543004512787, + -0.18248644471168518, + 0.17768341302871704, + 0.843591034412384, + -0.07443197071552277, + -2.6062240600585938, + -1.750958800315857, + -0.15503963828086853, + 0.18218432366847992, + 0.002181168645620346, + 0.8641887903213501, + -0.026101917028427124, + -1.9398812055587769, + -1.8561092615127563 + ], + [ + -0.22363319993019104, + 1.533349871635437, + 0.4454386830329895, + -1.5271000862121582, + 0.8945828080177307, + -1.105146884918213, + 0.7171626687049866, + -0.20207461714744568, + 0.31458696722984314, + 0.20214562118053436, + 1.0593571662902832, + -0.4655032455921173, + -2.8809642791748047, + -1.580764651298523, + -0.22098299860954285, + -0.08731096982955933, + -0.15165574848651886, + 0.89030921459198, + 0.611029326915741, + -2.2792654037475586, + -1.798964262008667 + ], + [ + -0.3702698349952698, + 0.9771865606307983, + 0.27564477920532227, + -1.7643829584121704, + 0.7298657894134521, + -1.875503420829773, + 1.4598875045776367, + 0.47600361704826355, + 0.13439032435417175, + -0.23905222117900848, + 1.2192013263702393, + -0.44705769419670105, + -2.603621482849121, + -1.3898506164550781, + 0.16782721877098083, + -0.2559890151023865, + 0.2965841293334961, + 0.8750308752059937, + 0.1521819680929184, + -2.3417282104492188, + -2.2663588523864746 + ], + [ + -0.31705379486083984, + 0.866703450679779, + -0.09125347435474396, + -1.5959886312484741, + 0.8041162490844727, + -1.7037310600280762, + 1.6304850578308105, + 0.11580469459295273, + 0.2654762864112854, + 0.12200001627206802, + 0.6904033422470093, + 0.19859254360198975, + -2.7615463733673096, + -2.059410333633423, + 0.5591834783554077, + -0.4801292419433594, + -0.3343847393989563, + 0.8787070512771606, + -0.14748375117778778, + -2.0334713459014893, + -1.1611082553863525 + ], + [ + -0.11324095726013184, + 0.8777624368667603, + 0.8215147256851196, + -1.5343353748321533, + 0.881126344203949, + -1.0607860088348389, + 0.5398488640785217, + 0.6020828485488892, + -0.5065795183181763, + 0.02798180840909481, + 0.33880186080932617, + 0.320717990398407, + -3.3552045822143555, + -1.1435439586639404, + 0.19279444217681885, + -0.5245973467826843, + 0.30647164583206177, + 1.3726601600646973, + 0.3123217523097992, + -2.9675674438476562, + -1.7007204294204712 + ], + [ + -0.6432619094848633, + 0.79607754945755, + -0.49864423274993896, + -1.0710605382919312, + 1.0709084272384644, + -1.612175703048706, + 1.2555015087127686, + 0.2873077392578125, + -0.36858057975769043, + 0.05837755650281906, + 1.1025807857513428, + 0.7949575185775757, + -3.2468795776367188, + -2.556741237640381, + -0.6579093337059021, + -0.8874800801277161, + 0.7922345399856567, + 0.4745541512966156, + 0.07610362768173218, + -2.041800022125244, + -2.2069954872131348 + ], + [ + -0.9264093637466431, + 1.9828436374664307, + 0.7907096147537231, + -1.260971188545227, + 1.7548933029174805, + -2.2685132026672363, + 1.1821955442428589, + -0.49968039989471436, + 0.10982713848352432, + 0.2384604662656784, + 0.2075445055961609, + -0.7872775793075562, + -2.076058864593506, + -1.1888585090637207, + 0.7447717785835266, + 0.2355569452047348, + -0.3163020610809326, + 0.3282558023929596, + -0.4075327515602112, + -3.2451019287109375, + -2.4086263179779053 + ], + [ + -0.7190575003623962, + 1.3702023029327393, + 0.8307373523712158, + -0.7533494234085083, + 0.574251651763916, + -2.3376822471618652, + 0.5512753129005432, + -0.04391670227050781, + 0.2242431342601776, + -0.46594029664993286, + 0.5667901039123535, + -0.12832416594028473, + -2.9870448112487793, + -1.2032369375228882, + 0.17313259840011597, + 0.5932936668395996, + 0.7428228259086609, + 0.6570835113525391, + -0.1891915649175644, + -1.3703107833862305, + -1.7630409002304077 + ], + [ + 0.799586832523346, + 1.0164330005645752, + -0.49519193172454834, + -0.6983597278594971, + 1.7632484436035156, + -0.9710203409194946, + 1.5416830778121948, + 0.9653489589691162, + 0.06498706340789795, + 0.2132291942834854, + -0.1650814414024353, + 0.06786654889583588, + -1.8782389163970947, + -2.3576126098632812, + 0.9410911798477173, + -0.576595664024353, + -0.8031080365180969, + 0.7120062112808228, + 0.6479252576828003, + -2.3928208351135254, + -1.4952012300491333 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.11725310236215591, + 0.3854118585586548, + 0.06966311484575272, + -0.16570186614990234, + 0.06442362070083618, + -0.43052077293395996, + 0.0, + 0.11858775466680527, + 0.0573427751660347, + -0.14938217401504517, + -0.01657569408416748, + 0.060478124767541885, + -1.6284780502319336, + 0.0, + 0.016584808006882668, + 0.18205130100250244, + -0.13064442574977875, + 0.15621143579483032, + 0.056775350123643875, + -1.2762084007263184, + 0.0 + ] + ], + "predicted_scores": [ + 5.09656286239624, + 4.6442461013793945, + 4.363131999969482, + 4.087330341339111, + 4.079741477966309, + 3.85636568069458, + 3.712235927581787, + 3.2982747554779053 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "256770874", + "selected_index": 0, + "state_hash": "3a7aa4ba27a5b34cdeb0f31ce9818588b1dd4ede96bf546f07302616c2e9b3a6", + "stored_base_utility": 0.8314920663833618, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20372186601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20372186601161957 + }, + "base_progress": 0.20372186601161957, + "base_success": false, + "base_utility": 0.20372186601161957, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2191106230020523, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2191106230020523 + }, + { + "contact_quality": null, + "progress": 0.5739269256591797, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5739269256591797 + }, + { + "contact_quality": null, + "progress": 0.5799027681350708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5799027681350708 + }, + { + "contact_quality": null, + "progress": 0.2002597153186798, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2002597153186798 + }, + { + "contact_quality": null, + "progress": 0.1847754418849945, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1847754418849945 + }, + { + "contact_quality": null, + "progress": 0.5602454543113708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5602454543113708 + }, + { + "contact_quality": null, + "progress": 0.17806151509284973, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17806151509284973 + }, + { + "contact_quality": null, + "progress": 0.5666472911834717, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5666472911834717 + } + ], + "candidate_progress": [ + 0.2191106230020523, + 0.5739269256591797, + 0.5799027681350708, + 0.2002597153186798, + 0.1847754418849945, + 0.5602454543113708, + 0.17806151509284973, + 0.5666472911834717 + ], + "candidate_source_chart_ids": [ + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-626ac2ad9d0dd95f1a2e", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "generated_tangents": [ + [ + -0.021352455019950867, + -0.06388239562511444, + -0.19936755299568176, + 0.2723495066165924, + 0.05314575135707855, + -0.6790894269943237, + -0.08313643932342529, + 0.03967951238155365, + 0.04698224365711212, + 0.0017345771193504333, + -0.04005324840545654, + -0.02530290186405182, + -1.682046890258789, + -0.021227702498435974, + 0.09034234285354614, + 0.06362509727478027, + -0.06847070157527924, + -0.17661263048648834, + 0.03995142877101898, + -1.524829387664795, + 0.0024154335260391235 + ], + [ + 0.016905948519706726, + -0.12380242347717285, + -0.20470955967903137, + 0.24490293860435486, + 0.10680311918258667, + -0.6448712348937988, + -0.013684302568435669, + -0.006054997444152832, + 0.0004276838153600693, + 0.008075475692749023, + -0.13646629452705383, + 0.03656740486621857, + -1.6498140096664429, + 0.05820601433515549, + 0.09693363308906555, + 0.051125843077898026, + -0.0516250878572464, + -0.17521561682224274, + 0.04456280171871185, + -1.4436814785003662, + 0.041915059089660645 + ], + [ + -0.1459028571844101, + -0.09840792417526245, + -0.2813153564929962, + 0.24227577447891235, + -0.024093836545944214, + -0.5552691221237183, + -0.1000216007232666, + 0.054895058274269104, + -0.002643140032887459, + 0.021926842629909515, + -0.10481435805559158, + 0.16555088758468628, + -1.4554897546768188, + -0.04020188748836517, + 0.14717628061771393, + 0.10132670402526855, + -0.011192530393600464, + -0.18386848270893097, + 0.07136976718902588, + -1.3921089172363281, + -0.03706677258014679 + ], + [ + 0.01372237503528595, + -0.09023064374923706, + -0.19493785500526428, + 0.2167317271232605, + 0.05171874165534973, + -0.9290481805801392, + -0.1885535717010498, + 0.0333847850561142, + 0.06037026643753052, + 0.02035164088010788, + -0.0007323399186134338, + -0.08101320266723633, + -1.5808125734329224, + -0.04795512557029724, + 0.05824476107954979, + 0.08652494847774506, + -0.056111469864845276, + -0.12156517803668976, + 0.06414395570755005, + -1.3584651947021484, + -0.030695393681526184 + ], + [ + -0.014965668320655823, + -0.13042062520980835, + -0.13815221190452576, + 0.30335959792137146, + 0.18823647499084473, + -0.1961337924003601, + -0.11898910999298096, + -0.12108290195465088, + -0.07559175789356232, + 0.10593551397323608, + -0.17041346430778503, + -0.02239827811717987, + -1.5221431255340576, + -0.010524049401283264, + 0.2232595533132553, + 0.06306511163711548, + 0.048533231019973755, + -0.010736271739006042, + 0.04079015552997589, + -1.7418251037597656, + -0.013632670044898987 + ], + [ + -0.0961618423461914, + -0.15962864458560944, + -0.370579332113266, + 0.5263097286224365, + 0.09191268682479858, + -0.6222783327102661, + -0.43542051315307617, + 0.11238838732242584, + -0.036678895354270935, + -0.06750737875699997, + -0.037780873477458954, + 0.33894994854927063, + -1.5700141191482544, + 0.05773831158876419, + 0.15092140436172485, + 0.14408482611179352, + 0.003930628299713135, + -0.1570245772600174, + 0.24228744208812714, + -1.5104854106903076, + 0.13777191936969757 + ], + [ + -0.09765744209289551, + -0.14905546605587006, + -0.08675655722618103, + 0.4377281963825226, + 0.31750836968421936, + -0.19685977697372437, + -0.016438215970993042, + 0.027669772505760193, + -0.043940894305706024, + 0.11480231583118439, + -0.07276531308889389, + 0.16765084862709045, + -1.4150536060333252, + 0.29519912600517273, + 0.0946052148938179, + -0.056313324719667435, + -0.06864838302135468, + -0.07631354033946991, + 0.06261126697063446, + -1.8008344173431396, + 0.0017832964658737183 + ], + [ + -0.15826822817325592, + -0.24093644320964813, + -0.45203807950019836, + 0.39025309681892395, + 0.23050133883953094, + -0.5757313966751099, + -0.4597776532173157, + 0.11452986299991608, + -0.1091751903295517, + 0.022243544459342957, + -0.1284382939338684, + 0.5329670906066895, + -1.376935601234436, + 0.16788709163665771, + 0.05883866921067238, + 0.07033094763755798, + -0.11124368011951447, + -0.1769326776266098, + 0.1647515594959259, + -1.5973732471466064, + 0.18438316881656647 + ] + ], + "generated_utilities": [ + 0.2191106230020523, + 0.5739269256591797, + 0.5799027681350708, + 0.2002597153186798, + 0.1847754418849945, + 0.5602454543113708, + 0.17806151509284973, + 0.5666472911834717 + ], + "group_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8363267779350281, + 2.0, + 2.0, + 0.16206762194633484, + 0.5390529036521912, + 0.08151453733444214, + 0.5261932611465454, + 0.5265869498252869, + 0.5459305047988892, + 0.18360589444637299, + 0.16689825057983398, + 0.15426510572433472, + 0.0028923004865646362, + 0.15516109764575958, + 0.0506085604429245, + 0.19578850269317627 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.0005862116813659668, + 0.18783891201019287, + -0.02734696865081787, + 0.09201061725616455, + 0.010171383619308472, + -0.4856022596359253, + 0.0, + 0.10349468141794205, + 0.4567214846611023, + -0.1893925815820694, + 0.12298360466957092, + 0.3369159996509552, + -1.6010875701904297, + 0.0, + 0.304210901260376, + -0.3049007058143616, + 0.3435661494731903, + 0.3158445656299591, + 0.2931552231311798, + -1.2709691524505615, + 0.0 + ], + [ + -0.726529061794281, + -0.8121610879898071, + -1.0273469686508179, + 1.0920106172561646, + 0.3926190733909607, + -1.4856022596359253, + 1.8979665040969849, + -0.029249273240566254, + 0.008005961775779724, + 0.013417234644293785, + -0.10860864818096161, + 0.43491366505622864, + -2.6010875701904297, + -1.493168592453003, + -0.01047949492931366, + -0.0018908085767179728, + 0.019280990585684776, + 0.20726272463798523, + 0.45750224590301514, + -2.2709691524505615, + -1.4645930528640747 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7959330081939697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.986337184906006, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9291861057281494 + ], + [ + -0.9019005298614502, + -0.5901641845703125, + -0.920559287071228, + 0.991828978061676, + 0.2407551407814026, + -1.2655587196350098, + 1.7652356624603271, + 0.03909382224082947, + 0.04142814502120018, + 0.12125641852617264, + -0.2822783887386322, + 0.4936577081680298, + -2.5754144191741943, + -1.5686157941818237, + -0.22672900557518005, + -0.08431098610162735, + 0.09805622696876526, + 0.2479260265827179, + 0.4450404644012451, + -2.1972453594207764, + -1.4420135021209717 + ], + [ + -0.8822115063667297, + -0.9738405346870422, + -0.7449414730072021, + 0.8784142732620239, + 0.4583987295627594, + -1.3989503383636475, + 1.8331178426742554, + -0.006558071821928024, + 0.30383312702178955, + 0.2117801308631897, + -0.2538969814777374, + 0.13478082418441772, + -2.584805727005005, + -1.325882911682129, + 0.23593956232070923, + -0.2618964612483978, + -0.30780693888664246, + 0.24083535373210907, + 0.430472195148468, + -1.946203589439392, + -1.5077860355377197 + ], + [ + -0.39491432905197144, + -0.8895903825759888, + -0.9160035848617554, + 0.7220491170883179, + 0.36837494373321533, + -1.4429160356521606, + 1.4078493118286133, + 0.03969770669937134, + 0.007204015273600817, + 0.056278251111507416, + -0.4780465364456177, + 0.9121701717376709, + -2.512721538543701, + -1.6225675344467163, + -0.42074650526046753, + 0.15076839923858643, + -0.24544812738895416, + 0.08152849972248077, + 0.1987975835800171, + -2.712414264678955, + -1.9573136568069458 + ], + [ + -0.6879937648773193, + -0.8859440088272095, + -1.1152405738830566, + 0.8750674724578857, + -0.05955067276954651, + -1.5194538831710815, + 1.876329779624939, + -0.2068542242050171, + -0.5498799085617065, + 0.46419233083724976, + 0.42569708824157715, + -0.02127784490585327, + -2.5905418395996094, + -1.411050796508789, + -0.21327723562717438, + 0.20716798305511475, + 0.20993508398532867, + 0.33476364612579346, + -0.09872657060623169, + -2.5958731174468994, + -1.6374839544296265 + ], + [ + -0.5723236799240112, + -0.28514552116394043, + -0.6157487630844116, + 0.6151646375656128, + 0.32639649510383606, + -1.7217803001403809, + 2.5557403564453125, + 0.532052218914032, + 0.5977330207824707, + -0.5707970261573792, + -0.21707558631896973, + 0.8829224109649658, + -3.1287553310394287, + -1.0988328456878662, + 0.38229066133499146, + 0.0032835237216204405, + -0.3569248616695404, + 0.0031215548515319824, + 1.1060712337493896, + -2.3536932468414307, + -1.383341670036316 + ], + [ + -0.7901747822761536, + -0.8977189660072327, + -1.4548863172531128, + 0.5700533986091614, + -0.13018959760665894, + -0.8057670593261719, + 2.4019269943237305, + 0.5368256568908691, + 0.5227565765380859, + 0.28199371695518494, + -0.1637011021375656, + -0.2898564040660858, + -2.9142980575561523, + -1.8492696285247803, + 0.7177971601486206, + 0.2360217124223709, + -0.13862104713916779, + 0.9262527227401733, + 0.39495596289634705, + -2.470649480819702, + -1.3260751962661743 + ], + [ + -0.6053685545921326, + -0.3819047510623932, + -0.258634090423584, + 0.19338607788085938, + 0.19874510169029236, + -1.3407407999038696, + 2.5238022804260254, + -0.6922282576560974, + 0.8137098550796509, + 0.39421388506889343, + 0.09789760410785675, + 0.12572914361953735, + -2.518153667449951, + -1.6892600059509277, + 0.503250002861023, + 0.88390052318573, + -0.7520112991333008, + -0.3356115520000458, + 0.15291047096252441, + -2.5788943767547607, + -0.9403089284896851 + ], + [ + -0.464814156293869, + -0.6034223437309265, + -1.6505372524261475, + 0.6883972883224487, + -0.3758969306945801, + -1.785051941871643, + 2.605452060699463, + 0.550828218460083, + 0.4742209315299988, + -0.19333000481128693, + -1.002321481704712, + 0.48782211542129517, + -1.8015108108520508, + -0.8254865407943726, + -0.7333722710609436, + -0.4948355555534363, + 0.6016485095024109, + 0.5632485151290894, + -0.24367755651474, + -2.201673984527588, + -0.5806979537010193 + ], + [ + -1.4628195762634277, + -0.4433446228504181, + -1.9926375150680542, + 0.33977705240249634, + 0.03379333019256592, + -1.6042804718017578, + 2.221484422683716, + 0.7110810279846191, + -0.41500210762023926, + 0.08095695078372955, + -0.23683495819568634, + -0.43207624554634094, + -3.080759286880493, + -0.9071659445762634, + 0.47082334756851196, + -0.9117310047149658, + -0.06435621529817581, + -0.6813669204711914, + -0.2741369605064392, + -2.3763599395751953, + -1.000203013420105 + ], + [ + -1.382751226425171, + -1.7026821374893188, + -0.11846715211868286, + 1.5398613214492798, + -0.5885447859764099, + -1.8825289011001587, + 1.5456308126449585, + -0.320659339427948, + -0.964028000831604, + -0.1812019646167755, + 0.14226128160953522, + 0.4527861177921295, + -2.714468002319336, + -1.9451607465744019, + 0.195057213306427, + 0.16905252635478973, + -0.9247990250587463, + 0.9112424850463867, + 0.79449862241745, + -2.714893341064453, + -1.6473028659820557 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.02016383409500122, + 0.02509528398513794, + -0.02734696865081787, + 0.09201061725616455, + -0.04331648349761963, + -0.4856022596359253, + 0.0, + -0.022040890529751778, + -0.003832761198282242, + -0.01408458687365055, + 0.06165061891078949, + 0.02554956078529358, + -1.6010875701904297, + 0.0, + 0.009550465270876884, + 0.03566932678222656, + 0.013732164166867733, + 0.017749279737472534, + 0.0699148178100586, + -1.2709691524505615, + 0.0 + ], + [ + 0.023195505142211914, + 0.029890060424804688, + -0.02734696865081787, + 0.09201061725616455, + 0.06108534336090088, + -0.4856022596359253, + 0.0, + -0.034551952034235, + -0.15152660012245178, + -0.21811580657958984, + -0.063543401658535, + 0.23667243123054504, + -1.6010875701904297, + 0.0, + -0.1614403873682022, + -0.17222382128238678, + -0.09706380218267441, + 0.3660350441932678, + 0.13013160228729248, + -1.2709691524505615, + 0.0 + ] + ], + "predicted_scores": [ + 6.202249526977539, + 6.037703990936279, + 5.914402008056641, + 5.794589042663574, + 5.336828708648682, + 5.235112190246582, + 5.21646785736084, + 3.7788166999816895 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "957101489", + "selected_index": 0, + "state_hash": "e8473553a2b17f5adf09e08e7271b5502d53a51225c17abac3d9f78906cf8c46", + "stored_base_utility": 0.8363267779350281, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8431589603424072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8431589603424072 + }, + "base_progress": 0.8431589603424072, + "base_success": false, + "base_utility": 0.8431589603424072, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1996680349111557, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1996680349111557 + }, + { + "contact_quality": null, + "progress": 0.22532063722610474, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22532063722610474 + }, + { + "contact_quality": null, + "progress": 0.2148892730474472, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2148892730474472 + }, + { + "contact_quality": null, + "progress": 0.21345362067222595, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21345362067222595 + }, + { + "contact_quality": null, + "progress": 0.20482183992862701, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20482183992862701 + }, + { + "contact_quality": null, + "progress": 0.18579784035682678, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18579784035682678 + }, + { + "contact_quality": null, + "progress": 0.20295663177967072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20295663177967072 + }, + { + "contact_quality": null, + "progress": 0.22642898559570312, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22642898559570312 + } + ], + "candidate_progress": [ + 0.1996680349111557, + 0.22532063722610474, + 0.2148892730474472, + 0.21345362067222595, + 0.20482183992862701, + 0.18579784035682678, + 0.20295663177967072, + 0.22642898559570312 + ], + "candidate_source_chart_ids": [ + "ms-9d12ad4ebdf37419994c", + "ms-9228f8295b9891cf011c", + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-9d12ad4ebdf37419994c", + "ms-5bddb39f792f20e5e6d4", + "ms-ce9505086ea115f86649", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-9a14552b7db695dee92c", + "generated_tangents": [ + [ + 0.2624432146549225, + -0.2312541902065277, + -0.01603803038597107, + 0.716766357421875, + -0.08071556687355042, + -1.8082318305969238, + -0.09603911638259888, + 0.05549733340740204, + -0.05502775311470032, + -0.0594683513045311, + 0.20835864543914795, + -0.016604065895080566, + -1.7020565271377563, + -0.13113079965114594, + 0.038984961807727814, + 0.049232371151447296, + 0.011512726545333862, + 0.029467463493347168, + 0.15997520089149475, + -1.5683104991912842, + -0.21798065304756165 + ], + [ + 0.12213128805160522, + -0.21399933099746704, + -0.08174243569374084, + 0.6623992919921875, + 0.045203059911727905, + -1.9158293008804321, + -0.1745433807373047, + 0.04145631194114685, + -0.002330976538360119, + -0.0541447177529335, + 0.22507616877555847, + -0.08192059397697449, + -1.7410130500793457, + -0.28673940896987915, + 0.08530785143375397, + 0.0473250113427639, + -0.011274099349975586, + 0.001980200409889221, + -0.016880542039871216, + -1.586552619934082, + -0.27615809440612793 + ], + [ + -0.02724890410900116, + -0.1699056327342987, + -0.24820461869239807, + 0.4899890720844269, + -0.13699546456336975, + -1.2355618476867676, + -0.2498578429222107, + 0.06585212051868439, + -0.08921295404434204, + 0.006838686764240265, + 0.2097664773464203, + -0.07441318035125732, + -1.7632848024368286, + -0.23597757518291473, + 0.06457287818193436, + 0.047926463186740875, + -0.003697410225868225, + -0.00036013126373291016, + 0.01808789372444153, + -1.5946176052093506, + -0.18573492765426636 + ], + [ + -0.08218064904212952, + -0.1922997385263443, + -0.46739181876182556, + 0.7826329469680786, + -0.13812682032585144, + -0.9043611288070679, + -0.5052298903465271, + 0.08566302806138992, + -0.05732395499944687, + -0.00764002650976181, + 0.18206840753555298, + 0.30042794346809387, + -1.774909496307373, + -0.04345497488975525, + 0.23592454195022583, + 0.09190230071544647, + 0.1109326034784317, + -0.06977240741252899, + 0.32663580775260925, + -1.7537684440612793, + -0.058904170989990234 + ], + [ + 0.18238164484500885, + -0.23363518714904785, + -0.058075159788131714, + 0.6516423225402832, + 0.05666317045688629, + -2.0269320011138916, + -0.1774006485939026, + 0.012360140681266785, + -0.031836606562137604, + -0.08709903806447983, + 0.277927041053772, + -0.093505859375, + -1.7426353693008423, + -0.20086981356143951, + 0.10358453541994095, + 0.04527905955910683, + -0.07806704938411713, + -0.02976386249065399, + -0.0159989595413208, + -1.6452622413635254, + -0.277252197265625 + ], + [ + -0.15696273744106293, + -0.16912096738815308, + -0.3014816343784332, + 0.5433331727981567, + -0.2681531310081482, + -0.8654876947402954, + -0.15608221292495728, + 0.06168109178543091, + -0.14395484328269958, + 0.05676373839378357, + 0.12987977266311646, + 0.142566978931427, + -1.6192100048065186, + -0.20651580393314362, + 0.12550833821296692, + 0.07648462057113647, + 0.05763666331768036, + 0.0032028555870056152, + 0.030257001519203186, + -1.6643896102905273, + -0.10184425115585327 + ], + [ + -0.03403349220752716, + -0.11452681571245193, + -0.13719698786735535, + 0.7563047409057617, + -0.036255210638046265, + -1.2274441719055176, + -0.2351142168045044, + -0.03501904010772705, + -0.0992419496178627, + 0.02351989597082138, + 0.035086922347545624, + -0.05853426456451416, + -2.0476763248443604, + 0.052276626229286194, + 0.043744999915361404, + 0.03687640279531479, + -0.0037261992692947388, + -0.014840468764305115, + -0.06056523323059082, + -1.8509268760681152, + -0.05957487225532532 + ], + [ + -0.12679947912693024, + -0.23132160305976868, + -0.5529159307479858, + 0.6806193590164185, + -0.028687238693237305, + -0.8835886716842651, + -0.6139217019081116, + 0.14418193697929382, + -0.1342320442199707, + 0.0951058641076088, + 0.0733126774430275, + 0.4877215325832367, + -1.648722767829895, + 0.0299447700381279, + 0.11717789620161057, + 0.023823797702789307, + -0.003589138388633728, + -0.08864752948284149, + 0.26730063557624817, + -1.8048813343048096, + 0.03818659484386444 + ] + ], + "generated_utilities": [ + 0.1996680349111557, + 0.22532063722610474, + 0.2148892730474472, + 0.21345362067222595, + 0.20482183992862701, + 0.18579784035682678, + 0.20295663177967072, + 0.22642898559570312 + ], + "group_id": "ms-9a14552b7db695dee92c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.838127613067627, + 2.0, + 0.15671375393867493, + 0.17482513189315796, + 0.5684618353843689, + 0.19147612154483795, + 0.5643423795700073, + 0.1653006672859192, + 0.11670731008052826, + 0.14578887820243835, + 0.16058449447155, + 0.10923929512500763, + 0.13408902287483215, + 0.06845971941947937, + 0.02954423427581787, + 0.08425161242485046 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.22581616044044495, + -0.32600927352905273, + -0.054001130163669586, + 1.2553811073303223, + -0.1383839249610901, + -1.9753923416137695, + 0.0, + 0.09536236524581909, + 0.017635054886341095, + -0.06946105509996414, + 0.06630897521972656, + -0.10150641947984695, + -1.9112331867218018, + 0.0, + -0.23092225193977356, + 0.04748937487602234, + 0.3013715445995331, + -0.22072738409042358, + 0.0733548104763031, + -1.6514825820922852, + 0.0 + ], + [ + 0.22913068532943726, + -0.32600927352905273, + 0.24692901968955994, + 1.2553811073303223, + 0.3704858124256134, + -1.9753923416137695, + 0.0, + 0.036500319838523865, + 0.12820330262184143, + -0.09674224257469177, + 0.2668613791465759, + -0.14372897148132324, + -1.9112331867218018, + 0.0, + -0.27894535660743713, + -0.3322452902793884, + 0.16300754249095917, + -0.24123603105545044, + -0.2359180599451065, + -1.6514825820922852, + 0.0 + ], + [ + 0.0198141410946846, + -1.3260092735290527, + 0.011403528973460197, + 2.2553811073303223, + 0.5258303284645081, + -2.9753923416137695, + 1.8956809043884277, + -0.028990529477596283, + 0.007858656346797943, + 0.03128144145011902, + 0.5457771420478821, + -0.14793801307678223, + -2.9112331867218018, + -1.5743238925933838, + -0.011861175298690796, + -0.002163209253922105, + 0.015663353726267815, + 0.7587639689445496, + -0.07801014184951782, + -2.651482582092285, + -1.5696079730987549 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7913618087768555, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1486477851867676, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1392159461975098 + ], + [ + 0.13539990782737732, + -1.2862521409988403, + 0.28441768884658813, + 2.3658432960510254, + 0.4678441286087036, + -2.8791494369506836, + 1.8408029079437256, + -0.11893181502819061, + -0.1259741485118866, + -0.20943574607372284, + 0.4765416979789734, + -0.40862777829170227, + -2.8372273445129395, + -1.4697245359420776, + -0.08606886118650436, + -0.06554404646158218, + -0.27424973249435425, + 0.907739520072937, + -0.10130079835653305, + -2.6951611042022705, + -1.807652235031128 + ], + [ + -0.15379676222801208, + -1.7091000080108643, + 0.31153741478919983, + 2.2179949283599854, + 0.859725832939148, + -2.6038026809692383, + 1.98358952999115, + 0.22934654355049133, + -0.16202330589294434, + 0.02916553243994713, + 0.7702362537384033, + -0.0907563641667366, + -2.6495683193206787, + -1.394291877746582, + 0.2534489035606384, + -0.07077484577894211, + 0.10660970956087112, + 0.7866172194480896, + -0.41323983669281006, + -2.4847500324249268, + -1.5881152153015137 + ], + [ + 0.14051103591918945, + -0.8684865236282349, + 0.45214012265205383, + 2.499347686767578, + 0.310441792011261, + -2.875823497772217, + 1.4898443222045898, + 0.3396439850330353, + -0.033532168716192245, + 0.20189781486988068, + 0.3506202697753906, + 0.04831346869468689, + -2.4730942249298096, + -1.4369138479232788, + -0.4840375483036041, + -0.17814871668815613, + -0.1863224357366562, + 0.47034892439842224, + -0.5315045714378357, + -3.019744396209717, + -1.2571722269058228 + ], + [ + 0.5503659844398499, + -1.517655372619629, + -0.20272362232208252, + 2.5588791370391846, + 0.7160627841949463, + -2.551017999649048, + 2.423107385635376, + 0.18953359127044678, + -0.4268995523452759, + 0.014377228915691376, + 0.969605028629303, + -0.2440275251865387, + -2.385143756866455, + -1.4786417484283447, + 0.44389280676841736, + -0.11543146520853043, + -0.5315021872520447, + 1.2627880573272705, + 0.24797001481056213, + -3.17976713180542, + -1.8759949207305908 + ], + [ + -0.07551594823598862, + -1.6101253032684326, + 0.3926098048686981, + 1.726672649383545, + 1.0158472061157227, + -3.367068290710449, + 1.4499058723449707, + -0.5616514086723328, + 0.002434568479657173, + -0.5033466815948486, + -0.12249338626861572, + -0.07335222512483597, + -3.5959439277648926, + -1.0662813186645508, + -0.034436360001564026, + 0.512275218963623, + -0.04822078347206116, + 0.6117934584617615, + -0.745093822479248, + -2.2700355052948, + -2.090803623199463 + ], + [ + 0.6751457452774048, + -0.7857800126075745, + 0.11759950965642929, + 2.7748122215270996, + 1.3096001148223877, + -2.3352341651916504, + 1.4677345752716064, + 0.7219796180725098, + 0.21788907051086426, + -0.3245394825935364, + 0.08372032642364502, + -0.6393120288848877, + -2.8002729415893555, + -2.157836437225342, + 0.11339983344078064, + -0.2834126651287079, + -0.3824191987514496, + 1.130112648010254, + -0.8028184175491333, + -2.3846678733825684, + -1.8259286880493164 + ], + [ + -0.01608770713210106, + -2.1245148181915283, + -0.175740584731102, + 2.754368782043457, + 1.0032098293304443, + -3.7700157165527344, + 1.3539870977401733, + -0.7906507849693298, + -0.8311541080474854, + 0.8180835247039795, + 0.10450628399848938, + 0.01135176420211792, + -3.396463394165039, + -0.6868918538093567, + 0.22346650063991547, + 0.693323016166687, + -0.47104692459106445, + 0.8188157081604004, + -0.6246469020843506, + -2.4006571769714355, + -2.431541919708252 + ], + [ + -0.7931793928146362, + -1.7645766735076904, + 0.23125068843364716, + 2.816321849822998, + 1.1080517768859863, + -3.8820090293884277, + 1.8420056104660034, + 0.4984684884548187, + -0.9149360656738281, + -0.028868291527032852, + 0.14621934294700623, + 0.546608567237854, + -2.7883894443511963, + -1.3650472164154053, + -0.055590901523828506, + 0.41940492391586304, + -0.7235082983970642, + 1.1734092235565186, + -0.6126058101654053, + -3.217271566390991, + -2.512387275695801 + ], + [ + -0.5335642695426941, + -2.139784336090088, + 0.9540140628814697, + 1.7781559228897095, + 1.2117886543273926, + -3.71681809425354, + 2.461766242980957, + 0.4546531140804291, + 0.09786976873874664, + -0.8281612396240234, + -0.19971781969070435, + -1.1042157411575317, + -3.047807216644287, + -1.0697336196899414, + -0.22256404161453247, + -0.3800235986709595, + 0.3603542447090149, + 0.7543611526489258, + -0.23184219002723694, + -3.5156431198120117, + -2.1170644760131836 + ], + [ + 0.27338743209838867, + -0.9654052257537842, + -0.2700650095939636, + 2.6957814693450928, + 0.0643424391746521, + -3.094449281692505, + 0.8981494307518005, + 0.19603312015533447, + 0.5969407558441162, + -0.45200687646865845, + 1.466981291770935, + -0.45456382632255554, + -2.997983455657959, + -1.1903986930847168, + 0.5722758769989014, + -0.9932370781898499, + 0.28074830770492554, + 0.6244409084320068, + 0.6411240696907043, + -3.2179243564605713, + -1.905179738998413 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.025789255276322365, + -0.32600927352905273, + -0.02888326905667782, + 1.2553811073303223, + 0.013092517852783203, + -1.9753923416137695, + 0.0, + 0.04008308798074722, + -0.05201057717204094, + -0.008613351732492447, + -0.06225985288619995, + -0.0397564172744751, + -1.9112331867218018, + 0.0, + -0.02836347185075283, + -0.0055325529538095, + 0.053034864366054535, + 0.009740114212036133, + -0.006228044629096985, + -1.6514825820922852, + 0.0 + ] + ], + "predicted_scores": [ + 4.801419258117676, + 4.317750453948975, + 4.280590534210205, + 4.22405481338501, + 3.9708096981048584, + 3.954923629760742, + 3.847562074661255, + 2.8120131492614746 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264", + "selected_index": 0, + "state_hash": "49d38eabb7404e0028c75dd98e1239274b5dc2f04b9db8653001e44369ca8f3a", + "stored_base_utility": 0.838127613067627, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8326728343963623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8326728343963623 + }, + "base_progress": 0.8326728343963623, + "base_success": false, + "base_utility": 0.8326728343963623, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.14438587427139282, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14438587427139282 + }, + { + "contact_quality": null, + "progress": 0.21023166179656982, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21023166179656982 + }, + { + "contact_quality": null, + "progress": 0.19636064767837524, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19636064767837524 + }, + { + "contact_quality": null, + "progress": 0.2200891673564911, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2200891673564911 + }, + { + "contact_quality": null, + "progress": 0.21172496676445007, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21172496676445007 + }, + { + "contact_quality": null, + "progress": 0.17683273553848267, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17683273553848267 + }, + { + "contact_quality": null, + "progress": 0.2003195583820343, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2003195583820343 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.14438587427139282, + 0.21023166179656982, + 0.19636064767837524, + 0.2200891673564911, + 0.21172496676445007, + 0.17683273553848267, + 0.2003195583820343, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-5dfbb16e173e1a3e2bd6", + "ms-c80e71e55d51d8595499" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-9f5d57fe2153fd267774", + "generated_tangents": [ + [ + 0.2602660059928894, + -0.03273323178291321, + 0.07987892627716064, + 0.5466212034225464, + -0.20089450478553772, + -1.8387469053268433, + -0.059253692626953125, + -0.018537238240242004, + -0.1549907624721527, + -0.029300756752490997, + 0.41915199160575867, + 0.014187842607498169, + -1.1461056470870972, + -0.13178829848766327, + 0.06668057292699814, + -0.007875459268689156, + 0.03469371795654297, + 0.0015375465154647827, + 0.1495690643787384, + -1.0548312664031982, + -0.08297193050384521 + ], + [ + 0.09643995761871338, + -0.01801915094256401, + -0.08633384108543396, + 0.3083895742893219, + -0.22981655597686768, + -1.6308103799819946, + -0.17506033182144165, + 0.1317121833562851, + -0.1870492845773697, + -0.08348088711500168, + 0.4701962172985077, + -0.0415484756231308, + -1.1528005599975586, + -0.17862139642238617, + 0.09915521740913391, + 0.003914322704076767, + 0.1445608139038086, + -0.07952277362346649, + -0.09453406929969788, + -1.1416213512420654, + -0.1206238865852356 + ], + [ + 0.07473111152648926, + 0.019286468625068665, + -0.06443783640861511, + 0.3161158263683319, + -0.17376276850700378, + -1.7811124324798584, + -0.18677401542663574, + 0.06483151018619537, + -0.21161282062530518, + -0.0615890696644783, + 0.5126396417617798, + -0.005297720432281494, + -1.2125425338745117, + -0.16410191357135773, + 0.05861067771911621, + 0.011137725785374641, + 0.1303870677947998, + 0.02194388210773468, + -0.011279016733169556, + -1.0515601634979248, + -0.0638728141784668 + ], + [ + 0.09147374331951141, + -0.01692088134586811, + -0.07399740815162659, + 0.431353896856308, + -0.18794980645179749, + -1.7875381708145142, + -0.19772344827651978, + 0.035040900111198425, + -0.20439089834690094, + -0.12228672951459885, + 0.4379032850265503, + -0.014443904161453247, + -1.1933002471923828, + -0.1691025346517563, + 0.07060874998569489, + 0.015172602608799934, + 0.16315990686416626, + -0.08178214728832245, + -0.09501567482948303, + -1.133695363998413, + -0.08170938491821289 + ], + [ + 0.06649884581565857, + -0.021549489349126816, + -0.1452367603778839, + 0.49567893147468567, + -0.09795928001403809, + -1.7211921215057373, + -0.14746838808059692, + 0.02174447476863861, + -0.15541866421699524, + -0.12002352625131607, + 0.5594443678855896, + 0.09868992865085602, + -1.1431653499603271, + -0.1553684026002884, + 0.03961511701345444, + 0.009050954133272171, + 0.09205007553100586, + -0.07196493446826935, + 0.009062260389328003, + -1.06088125705719, + -0.035056307911872864 + ], + [ + 0.14043836295604706, + 0.01007155328989029, + -0.0518205463886261, + 0.4260527789592743, + -0.07622635364532471, + -1.8494645357131958, + -0.15775960683822632, + 0.014868944883346558, + -0.1826332062482834, + -0.05402038246393204, + 0.5112128257751465, + 0.04463024437427521, + -1.2181001901626587, + -0.13149280846118927, + -0.0059579648077487946, + 0.03604526072740555, + 0.031237870454788208, + 0.05012226104736328, + -0.027166098356246948, + -1.0655229091644287, + -0.06652700901031494 + ], + [ + 0.09830405563116074, + -0.07416731864213943, + -0.19190940260887146, + 0.050100743770599365, + -0.1889578104019165, + -2.2374842166900635, + -0.3995264172554016, + 0.20491744577884674, + -0.14394374191761017, + -0.1517687439918518, + 0.7002518773078918, + -0.04169680178165436, + -1.1539170742034912, + -0.3284742832183838, + -0.026428338140249252, + 0.12713925540447235, + 0.17761936783790588, + 0.0869329571723938, + -0.015213251113891602, + -1.076972484588623, + -0.31907981634140015 + ], + [ + 0.1832084208726883, + 0.04683810472488403, + -0.03905823826789856, + 0.5987484455108643, + -0.1558729112148285, + -1.8848572969436646, + -0.8485514521598816, + -0.0488336980342865, + -0.14652352035045624, + 0.34101128578186035, + 0.5387577414512634, + -0.03178811073303223, + -1.0788077116012573, + -0.6728173494338989, + 0.27982887625694275, + 0.18009895086288452, + 0.21637894213199615, + 0.3476164937019348, + 0.16768789291381836, + -1.624605417251587, + 0.2797800898551941 + ] + ], + "generated_utilities": [ + 0.14438587427139282, + 0.21023166179656982, + 0.19636064767837524, + 0.2200891673564911, + 0.21172496676445007, + 0.17683273553848267, + 0.2003195583820343, + 2.0 + ], + "group_id": "ms-9f5d57fe2153fd267774", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8544998168945312, + 2.0, + 0.14427810907363892, + 0.1566883623600006, + 0.7847082018852234, + 0.6161996126174927, + 0.5900160074234009, + 0.12891459465026855, + 0.5811972618103027, + 0.12705880403518677, + 0.12002001702785492, + 0.07508957386016846, + 0.1335657238960266, + 2.0, + 0.5254277586936951, + 0.1602712869644165 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.07957560569047928, + 0.1778574287891388, + -0.06101015955209732, + 0.08770006895065308, + -0.13665570318698883, + -1.1242008209228516, + 0.0, + 0.20265339314937592, + 0.03117954358458519, + -0.08283126354217529, + 0.051615357398986816, + 0.006725620478391647, + -0.7304134368896484, + 0.0, + -0.03756266087293625, + 0.051428791135549545, + 0.13959813117980957, + 0.040047526359558105, + -0.04084143042564392, + -0.31500017642974854, + 0.0 + ], + [ + 0.31541678309440613, + -0.16606420278549194, + 0.09380461275577545, + 0.031209349632263184, + -0.10083025693893433, + -1.1242008209228516, + 0.0, + 0.09663133323192596, + -0.095388263463974, + 0.4012776017189026, + 0.051615357398986816, + 0.167547807097435, + -0.7304134368896484, + 0.0, + -0.6302167177200317, + 0.05524101108312607, + -0.14198890328407288, + 0.040047526359558105, + 0.5201720595359802, + -0.31500017642974854, + 0.0 + ], + [ + 0.004154659807682037, + -0.0034282731357961893, + 0.005892834160476923, + 1.0312093496322632, + -0.15828494727611542, + -2.1242008209228516, + -1.4059314727783203, + 0.002088826149702072, + 0.0016306412871927023, + 0.005124530289322138, + 1.0516153573989868, + -0.05936579033732414, + -1.7304134368896484, + -1.2081255912780762, + 0.010161396116018295, + 0.0017865521367639303, + -0.006222852971404791, + 1.040047526359558, + 0.07346899807453156, + -1.3150001764297485, + -0.9752891659736633 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.8118629455566406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.4162511825561523, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9505783319473267 + ], + [ + 0.05903958901762962, + -0.05713164061307907, + 0.27611011266708374, + 0.9773679971694946, + -0.2122824341058731, + -2.198127031326294, + -1.1234419345855713, + 0.22609303891658783, + 0.14432841539382935, + 0.13455358147621155, + 0.8475942611694336, + 0.19007176160812378, + -1.4536203145980835, + -1.208316445350647, + -0.24895474314689636, + 0.08610261976718903, + 0.16216596961021423, + 1.2356702089309692, + 0.07108174264431, + -1.4601069688796997, + -1.0005295276641846 + ], + [ + -0.2862337827682495, + -0.34814661741256714, + -0.3317800462245941, + 1.383265495300293, + -0.3485583961009979, + -1.7799506187438965, + -1.6264256238937378, + -0.24356596171855927, + -0.18337340652942657, + 0.3351142108440399, + 1.36115562915802, + -0.4404119849205017, + -1.4858057498931885, + -0.9229021668434143, + 0.3078569769859314, + -0.18743832409381866, + -0.3414495885372162, + 1.0045359134674072, + -0.1438894122838974, + -1.5133963823318481, + -0.7331042885780334 + ], + [ + 0.14600041508674622, + -0.10555070638656616, + 0.06090353801846504, + 1.1715545654296875, + 0.14829085767269135, + -1.8377182483673096, + -1.2019046545028687, + -0.33082208037376404, + 0.27619102597236633, + 0.3936208188533783, + 1.4077775478363037, + -0.2165551781654358, + -2.094696283340454, + -0.7843748331069946, + -0.13239514827728271, + 0.208020880818367, + 0.17500364780426025, + 0.5608562231063843, + 0.5641188025474548, + -1.6192113161087036, + -1.1969176530838013 + ], + [ + -0.03659747168421745, + -0.2400680035352707, + 0.49718841910362244, + 1.0422266721725464, + 0.13088186085224152, + -2.068427324295044, + -1.1215412616729736, + 0.06697356700897217, + 0.19986751675605774, + -0.5827334523200989, + 0.9792195558547974, + -0.5810520648956299, + -1.7176907062530518, + -0.6760571002960205, + -0.08971267938613892, + 0.36716267466545105, + 0.4829793870449066, + 0.7045803070068359, + -0.3341261148452759, + -1.3750282526016235, + -1.086517095565796 + ], + [ + 0.09393255412578583, + 0.6277099251747131, + 0.653377115726471, + 0.5602235198020935, + -0.8483365178108215, + -1.7264833450317383, + -1.698084831237793, + -0.3303340971469879, + 0.43070027232170105, + -0.13159427046775818, + 1.0274523496627808, + 0.4760425388813019, + -2.1360630989074707, + -0.6506720185279846, + 0.3130929470062256, + 0.6801184415817261, + 0.6575731039047241, + 0.9053056240081787, + 0.18805859982967377, + -1.3225764036178589, + -1.0653810501098633 + ], + [ + 0.536294162273407, + -0.026106586679816246, + 0.11593607813119888, + 0.2775042653083801, + 0.1801595836877823, + -1.590940237045288, + -0.6846873164176941, + 0.5080465078353882, + -0.7678324580192566, + 0.24115782976150513, + 0.2675415277481079, + -0.3879689872264862, + -2.1869571208953857, + -0.7176487445831299, + -0.15474224090576172, + -0.5326727032661438, + -0.12406099587678909, + 0.411626398563385, + 0.3519209027290344, + -1.6924495697021484, + -1.4648362398147583 + ], + [ + 0.2688097357749939, + -0.7928301095962524, + -0.39608004689216614, + 1.3961963653564453, + -0.026196151971817017, + -2.263073682785034, + -2.044499635696411, + 0.055598195642232895, + -0.485431969165802, + -0.058985911309719086, + 1.781836986541748, + 0.7104123830795288, + -2.0781984329223633, + -0.6968748569488525, + -0.1437186300754547, + 0.8155919313430786, + 0.5188283920288086, + 0.23954623937606812, + -0.6596511602401733, + -1.3867920637130737, + -0.7448391318321228 + ], + [ + 0.11259422451257706, + 0.43877506256103516, + -0.585435152053833, + 0.12099790573120117, + 0.5070493817329407, + -2.101219654083252, + -1.3759371042251587, + -0.7516231536865234, + 0.7018498778343201, + 0.7899036407470703, + 0.9772143959999084, + -0.7032701373100281, + -1.4699928760528564, + -1.4471495151519775, + 0.4750489890575409, + 0.8803708553314209, + -0.25032687187194824, + 1.3429648876190186, + 0.7985087037086487, + -0.5786607265472412, + -1.563289761543274 + ], + [ + 0.4965847432613373, + -0.8452511429786682, + 0.21300016343593597, + 0.28797537088394165, + -1.0059970617294312, + -3.040114402770996, + -1.1050269603729248, + 0.7826927304267883, + 0.8645586371421814, + -0.9447845816612244, + 1.0121192932128906, + 0.5123239755630493, + -1.5470938682556152, + -0.6170141100883484, + 0.2538997530937195, + 0.912939727306366, + -0.8613666892051697, + 0.9517492055892944, + 0.7185115218162537, + -2.1534924507141113, + -0.6818757057189941 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025899745523929596, + -0.03433045744895935, + -0.044045113027095795, + 0.031209349632263184, + -0.014532476663589478, + -1.1242008209228516, + 0.0, + 0.10768167674541473, + -0.020356399938464165, + 0.09216367453336716, + 0.051615357398986816, + -0.011370409280061722, + -0.7304134368896484, + 0.0, + 0.04558054730296135, + 0.038496289402246475, + 0.02504396252334118, + 0.040047526359558105, + 0.01217309758067131, + -0.31500017642974854, + 0.0 + ], + [ + -0.5198070406913757, + -0.8798871040344238, + 0.04596509411931038, + 1.55931556224823, + 0.31027060747146606, + -1.5283797979354858, + -1.6053413152694702, + -0.2754230797290802, + -0.012456716038286686, + 0.7998818159103394, + 1.5184144973754883, + 0.05059843137860298, + -1.85283362865448, + -0.9533926248550415, + 0.9978112578392029, + 0.8525571227073669, + -0.9611027836799622, + 0.32179969549179077, + -0.448661744594574, + -1.9452965259552002, + -0.7132223844528198 + ] + ], + "predicted_scores": [ + 3.1715168952941895, + 2.7965338230133057, + 2.589705467224121, + 2.4722023010253906, + 2.296574831008911, + 2.2875022888183594, + 1.6957511901855469, + -1.3588550090789795 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1607773860", + "selected_index": 0, + "state_hash": "fdc7379ab4491c9f65860cadd3b2f3c9f2335f26aae91fb1c06d3a4edf858d6a", + "stored_base_utility": 0.8544998168945312, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8272150158882141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8272150158882141 + }, + "base_progress": 0.8272150158882141, + "base_success": false, + "base_utility": 0.8272150158882141, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.179222971200943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.179222971200943 + }, + { + "contact_quality": null, + "progress": 0.22463779151439667, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22463779151439667 + }, + { + "contact_quality": null, + "progress": 0.8147144317626953, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8147144317626953 + }, + { + "contact_quality": null, + "progress": 0.19045117497444153, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19045117497444153 + }, + { + "contact_quality": null, + "progress": 0.18127290904521942, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18127290904521942 + }, + { + "contact_quality": null, + "progress": 0.18905499577522278, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18905499577522278 + }, + { + "contact_quality": null, + "progress": 0.19813549518585205, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19813549518585205 + }, + { + "contact_quality": null, + "progress": 0.16648955643177032, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16648955643177032 + } + ], + "candidate_progress": [ + 0.179222971200943, + 0.22463779151439667, + 0.8147144317626953, + 0.19045117497444153, + 0.18127290904521942, + 0.18905499577522278, + 0.19813549518585205, + 0.16648955643177032 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-20f3cfcf183c69853b1e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b0fee26e1db967f90b81", + "generated_tangents": [ + [ + 0.22818443179130554, + -0.03238482400774956, + 0.18031135201454163, + -0.1053600162267685, + 0.07715210318565369, + -2.2039506435394287, + 0.03612273931503296, + 0.0057865530252456665, + -0.013363216072320938, + -0.04724108427762985, + 0.07863190770149231, + 0.00023005902767181396, + -1.5760533809661865, + -0.1736038774251938, + 0.04172223061323166, + 0.0436229482293129, + 0.0007335245609283447, + -0.022069111466407776, + 0.2518843412399292, + -1.3511199951171875, + -0.3136128783226013 + ], + [ + 0.06199656426906586, + -0.003600809723138809, + -0.03485104441642761, + -0.19633445143699646, + 0.04201178252696991, + -2.1112866401672363, + -0.12216496467590332, + -0.014988809823989868, + -0.08607646077871323, + -0.06515321880578995, + 0.08061842620372772, + 0.008123427629470825, + -1.598246693611145, + -0.21602292358875275, + 0.04981159791350365, + 0.057903606444597244, + 0.1086626797914505, + -0.11851386725902557, + 0.005406826734542847, + -1.420619249343872, + -0.20315900444984436 + ], + [ + 0.03715181350708008, + 0.0040266066789627075, + -0.055608779191970825, + -0.17337001860141754, + 0.104716956615448, + -1.9267939329147339, + -0.06935179233551025, + 0.01847061514854431, + -0.03166525810956955, + -0.03275337070226669, + 0.22659337520599365, + 0.13304156064987183, + -1.5220907926559448, + -0.21739737689495087, + 0.09191690385341644, + 0.08474767208099365, + 0.027780652046203613, + -0.054191216826438904, + 0.17941749095916748, + -1.3500516414642334, + -0.213168203830719 + ], + [ + 0.1229734867811203, + 0.010669179260730743, + 0.0094451904296875, + -0.22407174110412598, + 0.003824472427368164, + -2.3158624172210693, + -0.1022217869758606, + 0.012917712330818176, + -0.08186297118663788, + -0.03311104327440262, + 0.15838569402694702, + -0.09856075048446655, + -1.838870644569397, + -0.16266612708568573, + -0.018731724470853806, + 0.07653182744979858, + 0.08485358953475952, + -0.12221826612949371, + 0.0437079519033432, + -1.5114665031433105, + -0.1361807882785797 + ], + [ + 0.08229903131723404, + 0.012069184333086014, + 0.0042723119258880615, + -0.2510314881801605, + -0.11223533749580383, + -1.9687868356704712, + -0.24648147821426392, + 0.09049665182828903, + -0.07600291073322296, + 0.009642921388149261, + 0.1038266196846962, + -0.015335500240325928, + -1.5055532455444336, + -0.3190021514892578, + 0.08912461251020432, + 0.06337188184261322, + 0.0903693288564682, + -0.10074125230312347, + 0.0035130679607391357, + -1.4228956699371338, + -0.2583085894584656 + ], + [ + 0.15582038462162018, + 0.054556794464588165, + 0.06342050433158875, + -0.21441645920276642, + 0.06938904523849487, + -2.3516125679016113, + -0.31208133697509766, + -0.015061333775520325, + -0.13063524663448334, + -0.06840226799249649, + 0.30069854855537415, + 0.1009671688079834, + -1.6926590204238892, + -0.16687323153018951, + -0.026025038212537766, + 0.08407551050186157, + 0.08806981146335602, + -0.06830160319805145, + 0.11908502876758575, + -1.464280605316162, + -0.05905193090438843 + ], + [ + 0.08990472555160522, + 0.05539984256029129, + 0.03419211506843567, + -0.2913701832294464, + 0.10797059535980225, + -2.244950532913208, + -0.382861852645874, + 0.08609195798635483, + -0.1347731053829193, + -0.070514015853405, + 0.14696359634399414, + 0.05582806468009949, + -1.6561474800109863, + -0.2793393135070801, + 0.02065729722380638, + 0.13093791902065277, + 0.09945663809776306, + -0.10855282843112946, + 0.20223209261894226, + -1.6661033630371094, + 0.05775855481624603 + ], + [ + 0.16831718385219574, + -0.0490262508392334, + -0.2795744240283966, + -0.38708996772766113, + 0.5014309883117676, + -1.5272479057312012, + -0.5076436996459961, + 0.09016288071870804, + -0.03740059584379196, + -0.01181013137102127, + -0.016439422965049744, + -0.0940324068069458, + -0.9069281816482544, + -0.48866915702819824, + 0.26338136196136475, + 0.103096604347229, + 0.16522401571273804, + -0.2173214703798294, + 0.3400283455848694, + -1.6648454666137695, + 0.0634608268737793 + ] + ], + "generated_utilities": [ + 0.179222971200943, + 0.22463779151439667, + 0.8147144317626953, + 0.19045117497444153, + 0.18127290904521942, + 0.18905499577522278, + 0.19813549518585205, + 0.16648955643177032 + ], + "group_id": "ms-b0fee26e1db967f90b81", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8397466540336609, + 0.8594310283660889, + 0.14638642966747284, + 0.09418028593063354, + 0.7838175296783447, + 0.5982922315597534, + 0.5913184881210327, + 2.0, + 0.20996521413326263, + 0.17419946193695068, + 0.11757439374923706, + 0.1645697057247162, + 0.05191062390804291, + 0.08658662438392639, + 0.04989185929298401, + 0.05867189168930054 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1510201096534729, + -0.21724483370780945, + 0.14240525662899017, + 0.003108598291873932, + -0.11609727144241333, + -1.6516358852386475, + 0.0, + -0.09976008534431458, + 0.061503566801548004, + 0.3070257902145386, + -0.08051937818527222, + 0.05735237896442413, + -1.6075372695922852, + 0.0, + -0.19649454951286316, + -0.15815469622612, + 0.2621642053127289, + -0.17578476667404175, + -0.034231506288051605, + -1.3808586597442627, + 0.0 + ], + [ + 0.014505341649055481, + 0.0403006337583065, + 0.5529788732528687, + 0.0714028924703598, + -0.41683781147003174, + -1.6516358852386475, + 0.0, + 0.41446545720100403, + -0.16278114914894104, + -0.4971221685409546, + -0.3640303611755371, + 0.02483367919921875, + -1.6075372695922852, + 0.0, + 0.5711643695831299, + -0.5244585275650024, + -0.5069849491119385, + 0.31709733605384827, + -0.4254770278930664, + -1.3808586597442627, + 0.0 + ], + [ + -0.044587142765522, + 0.03136333078145981, + 0.014485275372862816, + 0.04301099479198456, + 0.47328460216522217, + -2.6516358852386475, + -0.32303211092948914, + -0.01855158433318138, + -0.003608003491535783, + 0.029763570055365562, + 0.6359696388244629, + -0.1304762363433838, + -2.607537269592285, + -1.7957698106765747, + -0.009414590895175934, + -0.000939749414101243, + 0.018994441255927086, + 0.7486392855644226, + -0.06401278078556061, + -2.3808586597442627, + -1.722450852394104 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.6460642218589783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.5915396213531494, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.444901704788208 + ], + [ + 0.18585285544395447, + 0.17557260394096375, + 0.08478302508592606, + -0.17644454538822174, + 0.7477058172225952, + -2.614673137664795, + -0.3018328547477722, + 0.2204062044620514, + -0.0497702918946743, + 0.1961878538131714, + 0.5304322242736816, + -0.15872131288051605, + -2.604066848754883, + -2.053436517715454, + 0.18444111943244934, + -0.0024187257513403893, + 0.0054171085357666016, + 1.0256288051605225, + 0.02724038064479828, + -2.426236391067505, + -1.6594898700714111 + ], + [ + -0.5335133075714111, + 0.12298225611448288, + -0.12197553366422653, + 0.0424814410507679, + 0.002416938543319702, + -2.8428826332092285, + -0.17827168107032776, + -0.3377644121646881, + 0.04258471727371216, + -0.15910103917121887, + 0.9169985055923462, + 0.003966793417930603, + -3.069180727005005, + -1.407257318496704, + -0.47542956471443176, + -0.2210284322500229, + -0.10021048784255981, + 0.7806655168533325, + 0.3228837251663208, + -2.058253049850464, + -1.556677222251892 + ], + [ + 0.20687156915664673, + -0.5511468052864075, + 0.3945963978767395, + -0.16104432940483093, + 0.4936041831970215, + -2.1271684169769287, + 0.02855244278907776, + 0.544975996017456, + -0.3336343765258789, + 0.029475202783942223, + 0.8120307326316833, + -0.3246060609817505, + -2.885145425796509, + -2.2030622959136963, + -0.28730231523513794, + 0.19937433302402496, + 0.19840951263904572, + 1.1192030906677246, + -0.3995746374130249, + -2.5594136714935303, + -1.1545796394348145 + ], + [ + -0.5497553944587708, + -0.39125651121139526, + 0.2725587487220764, + -0.27965104579925537, + 1.0584466457366943, + -2.1044363975524902, + -0.6898342370986938, + 0.36552220582962036, + -0.2289373278617859, + 0.24627351760864258, + 0.7968363165855408, + 0.14776766300201416, + -2.988917112350464, + -1.6111457347869873, + -0.5116392374038696, + 0.030796781182289124, + -0.08262427151203156, + 1.0816104412078857, + 0.336497962474823, + -1.684936285018921, + -2.290850877761841 + ], + [ + 0.18322327733039856, + 0.6938241720199585, + 0.5252001881599426, + 0.06770005077123642, + 1.120344877243042, + -1.8558356761932373, + 0.18930694460868835, + -0.7033196687698364, + 0.7861698865890503, + -0.0965423658490181, + 0.015094935894012451, + -0.5303559303283691, + -2.6036770343780518, + -2.589250087738037, + -0.468231737613678, + -0.5901691317558289, + -0.7109078764915466, + 0.986366331577301, + -0.7634903192520142, + -2.984590530395508, + -1.1651530265808105 + ], + [ + 0.7039692401885986, + -0.5346179008483887, + 0.1466517597436905, + -0.6284777522087097, + -0.07994896173477173, + -2.4640438556671143, + -1.102164626121521, + -0.26685386896133423, + -0.3165925443172455, + 0.8653677701950073, + 0.232782244682312, + -0.5054740905761719, + -3.400317668914795, + -1.4077497720718384, + -0.09987740218639374, + -0.27442091703414917, + 0.5428174138069153, + 1.5128605365753174, + 0.747156023979187, + -2.0367631912231445, + -2.464085578918457 + ], + [ + 0.4474923610687256, + -0.7824500203132629, + 0.5928975343704224, + 0.57558673620224, + -0.17630118131637573, + -2.176387310028076, + -0.3299017548561096, + -0.6172446608543396, + 0.5065999627113342, + 0.46063295006752014, + 0.5951727628707886, + -0.02366364747285843, + -3.254279136657715, + -0.8880517482757568, + 0.3714323043823242, + 0.12449074536561966, + 0.6677311658859253, + -0.20713549852371216, + 0.5062628984451294, + -3.2156763076782227, + -1.9652986526489258 + ], + [ + -0.437069296836853, + 0.615723192691803, + -0.7743021249771118, + 0.20228458940982819, + 0.4320293366909027, + -2.0403428077697754, + -0.32251936197280884, + -0.7280393838882446, + 0.10878247022628784, + 0.9395551681518555, + 0.6021312475204468, + -0.3748174011707306, + -2.0175063610076904, + -1.9166245460510254, + -0.10793138295412064, + 0.6512030959129333, + 0.3747653365135193, + 1.5698660612106323, + -0.7995363473892212, + -1.8378328084945679, + -1.0564453601837158 + ], + [ + -0.7214477062225342, + -0.5686705708503723, + 0.8819299936294556, + 0.7319017648696899, + 1.0695421695709229, + -2.0338172912597656, + 0.2973436415195465, + -0.19576850533485413, + 0.4506514370441437, + 0.9037411212921143, + 0.0996239185333252, + 0.5076434016227722, + -2.611851930618286, + -2.7078182697296143, + 0.022706281393766403, + -0.4812851846218109, + -0.40778082609176636, + 1.6509652137756348, + -0.5721896886825562, + -1.8604621887207031, + -1.2943172454833984 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.017527949064970016, + -0.13547566533088684, + -0.31396082043647766, + -0.06045208126306534, + 0.7718918323516846, + -2.996134042739868, + -0.0199548602104187, + 0.26194432377815247, + -0.303554505109787, + -0.09068091958761215, + 0.3007984459400177, + -0.39815279841423035, + -2.626121997833252, + -1.8776893615722656, + 0.25959014892578125, + 0.20619672536849976, + -0.25346171855926514, + 0.40209779143333435, + 0.1943899542093277, + -2.464694023132324, + -1.5338099002838135 + ] + ], + "predicted_scores": [ + 4.6016645431518555, + 3.48230242729187, + 3.2867255210876465, + 3.1944432258605957, + 3.0854947566986084, + 2.9843802452087402, + 2.575968027114868, + 0.7800743579864502 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1358421325", + "selected_index": 0, + "state_hash": "b8e04d64f31138fff8f83f1bb017f9e8515565a2ada2ba5787ad5f06b796589c", + "stored_base_utility": 0.8397466540336609, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8307622671127319, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8307622671127319 + }, + "base_progress": 0.8307622671127319, + "base_success": false, + "base_utility": 0.8307622671127319, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.18349629640579224, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18349629640579224 + }, + { + "contact_quality": null, + "progress": 0.18907088041305542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18907088041305542 + }, + { + "contact_quality": null, + "progress": 0.1815730631351471, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1815730631351471 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.5584076046943665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5584076046943665 + }, + { + "contact_quality": null, + "progress": 0.18897385895252228, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18897385895252228 + }, + { + "contact_quality": null, + "progress": 0.19263410568237305, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19263410568237305 + }, + { + "contact_quality": null, + "progress": 0.5586480498313904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5586480498313904 + } + ], + "candidate_progress": [ + 0.18349629640579224, + 0.18907088041305542, + 0.1815730631351471, + 1.0, + 0.5584076046943665, + 0.18897385895252228, + 0.19263410568237305, + 0.5586480498313904 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-852c671b0adeccfa4bf1", + "ms-5bddb39f792f20e5e6d4", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "generated_tangents": [ + [ + -0.020635083317756653, + 0.0959254801273346, + -0.061238616704940796, + -0.08016014099121094, + 0.44623851776123047, + -0.3494376242160797, + -0.14195102453231812, + 0.13645543158054352, + 0.08305901288986206, + 0.052140943706035614, + -0.12193156033754349, + -0.10969516634941101, + -1.6263598203659058, + 0.05815311521291733, + -0.0007215999066829681, + -0.03338552638888359, + -0.050850823521614075, + 0.02344834804534912, + 0.04952099919319153, + -1.2113022804260254, + 0.039293646812438965 + ], + [ + 0.028915219008922577, + 0.09982654452323914, + -0.12753590941429138, + -0.01064649224281311, + 0.43138062953948975, + -0.23808756470680237, + -0.06415492296218872, + 0.014027386903762817, + 0.03954826295375824, + 0.07355596125125885, + -0.12208562344312668, + -0.07834076881408691, + -1.5096139907836914, + -0.004875048995018005, + -0.022102344781160355, + 0.055252332240343094, + -0.03557021915912628, + -0.104021355509758, + -0.08897733688354492, + -1.2019795179367065, + -0.006463631987571716 + ], + [ + -0.041307464241981506, + 0.12543150782585144, + -0.1795656383037567, + -0.04052796959877014, + 0.36637312173843384, + -0.09606052935123444, + -0.13462674617767334, + 0.10015849024057388, + -0.032246269285678864, + 0.08517506718635559, + -0.21736830472946167, + 0.10787816345691681, + -1.371263861656189, + 0.057300083339214325, + 0.02319614589214325, + 0.1050652414560318, + 0.0813579112291336, + -0.12381233274936676, + -0.05572065711021423, + -1.125626802444458, + 0.01024523377418518 + ], + [ + 0.04413406550884247, + 0.08551359176635742, + 0.035018593072891235, + -0.02496318519115448, + 0.7133876085281372, + -0.4000389873981476, + -0.07011795043945312, + 0.04560220241546631, + -0.05485475808382034, + 0.07572954893112183, + -0.09889846295118332, + -0.09824007749557495, + -1.4547662734985352, + -0.0967516154050827, + 0.0005850046873092651, + 0.06128402426838875, + -0.010074526071548462, + -0.1729082614183426, + -0.21908676624298096, + -1.2497552633285522, + -0.03272366523742676 + ], + [ + -0.09880837798118591, + 0.005370769649744034, + -0.3337397277355194, + 0.2283633053302765, + 0.4611598253250122, + -0.1822780966758728, + -0.2319657802581787, + 0.00840727984905243, + -0.07833203673362732, + -0.1108061745762825, + -0.11645384877920151, + 0.1898232251405716, + -1.433670163154602, + 0.13133937120437622, + 0.028921213001012802, + 0.1270018070936203, + 0.0030322521924972534, + -0.13117165863513947, + -0.04420021176338196, + -1.1167993545532227, + 0.14613139629364014 + ], + [ + 0.11469144374132156, + 0.08521609008312225, + -0.20930549502372742, + -0.0015750080347061157, + 0.3572162389755249, + -0.14677658677101135, + -0.17239797115325928, + 0.06843464076519012, + 0.003756226971745491, + 0.09487643092870712, + -0.029724866151809692, + -0.15236535668373108, + -1.5197081565856934, + 0.03441619127988815, + -0.061104562133550644, + 0.03566605970263481, + -0.14348895847797394, + -0.09357191622257233, + -0.12039792537689209, + -1.241206169128418, + 0.01101510226726532 + ], + [ + 0.1544441431760788, + 0.14982743561267853, + -0.14339640736579895, + 0.1214568018913269, + 0.17490744590759277, + -0.32321736216545105, + -0.1192970871925354, + -0.06034785509109497, + -0.029587386175990105, + 0.00627436488866806, + -0.1050943061709404, + -0.3003830909729004, + -1.4825079441070557, + -0.22405023872852325, + 0.017843198031187057, + 0.03236152231693268, + -0.18885256350040436, + 0.053827688097953796, + -0.4133561849594116, + -1.4220597743988037, + -0.03589838743209839 + ], + [ + -0.10528199374675751, + -0.07495167851448059, + -0.4740738570690155, + 0.14678460359573364, + 0.6004019975662231, + -0.06772100925445557, + -0.30438971519470215, + 0.03796494007110596, + -0.1418292373418808, + -0.009805478155612946, + -0.18407884240150452, + 0.3589443564414978, + -1.2774512767791748, + 0.1350625902414322, + -0.04852801188826561, + 0.07202960550785065, + -0.11233554780483246, + -0.1494644433259964, + -0.13059982657432556, + -1.2425318956375122, + 0.13258422911167145 + ] + ], + "generated_utilities": [ + 0.18349629640579224, + 0.18907088041305542, + 0.1815730631351471, + 2.0, + 0.5584076046943665, + 0.18897385895252228, + 0.19263410568237305, + 0.5586480498313904 + ], + "group_id": "ms-bc2d3b425d9163a62dbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8266608119010925, + 2.0, + 0.15905821323394775, + 0.13648143410682678, + 0.5621776580810547, + 0.18381908535957336, + 0.20885342359542847, + 0.18184903264045715, + 0.19253113865852356, + 0.5574395656585693, + 0.10362428426742554, + 0.0659959614276886, + 0.09997051954269409, + 0.2210000902414322, + 0.00010728836059570312, + 0.12723565101623535 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.059883326292037964, + -0.16710972785949707, + -0.04027438163757324, + -0.23492489755153656, + 0.34960758686065674, + 0.16551673412322998, + 0.0, + -0.27484941482543945, + 0.07304387539625168, + -0.38491085171699524, + 0.03056102991104126, + 0.16416949033737183, + -1.5270280838012695, + 0.0, + 0.01279987208545208, + -0.07740817964076996, + 0.07519636303186417, + 0.14440757036209106, + 0.0022157281637191772, + -1.3374013900756836, + 0.0 + ], + [ + -0.6122239828109741, + -0.0734642744064331, + -0.4064018726348877, + 0.11902926862239838, + 0.34960758686065674, + -0.07764682173728943, + 0.0, + 0.37407442927360535, + -0.08740249276161194, + -0.20959049463272095, + -0.37996697425842285, + -0.304497092962265, + -1.5270280838012695, + 0.0, + -0.9113190174102783, + 0.3964778184890747, + 0.4811243414878845, + 0.2385387420654297, + -0.11476142704486847, + -1.3374013900756836, + 0.0 + ], + [ + 0.3877759873867035, + 0.6924911141395569, + -1.0402743816375732, + 0.005685340613126755, + 1.3496075868606567, + -0.2760518193244934, + 1.2840001583099365, + -0.028063178062438965, + 0.0023769063409417868, + 0.03187502175569534, + 0.04654876887798309, + 0.3263346254825592, + -2.5270280838012695, + -1.7457892894744873, + -0.012772172689437866, + -0.008647367358207703, + 0.019657021388411522, + 0.3050904870033264, + 0.19037143886089325, + -2.3374013900756836, + -1.738369345664978 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.568000316619873, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.4915785789489746, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.476738691329956 + ], + [ + 0.1989155411720276, + 0.7169155478477478, + -0.9921301603317261, + 0.22762086987495422, + 1.516188144683838, + -0.23096826672554016, + 1.0414973497390747, + 0.0012522991746664047, + -0.0518387071788311, + -0.23540246486663818, + 0.08515512943267822, + 0.5060473084449768, + -2.5231378078460693, + -1.4593229293823242, + -0.09671962261199951, + 0.09134317189455032, + -0.07944301515817642, + 0.18403387069702148, + 0.2437569499015808, + -2.270580291748047, + -1.9640144109725952 + ], + [ + 0.256519615650177, + 0.7805030941963196, + -1.3583617210388184, + 0.3310847878456116, + 1.2735480070114136, + -0.4127388000488281, + 1.2665152549743652, + -0.19997301697731018, + -0.11129427701234818, + 0.1647017002105713, + 0.3767858147621155, + 0.4831879138946533, + -2.752143621444702, + -1.6790910959243774, + 0.2554209232330322, + 0.3251851797103882, + -0.12605378031730652, + 0.696884274482727, + 0.29535919427871704, + -2.726069688796997, + -1.704551100730896 + ], + [ + 0.49772778153419495, + 1.0026791095733643, + -0.5513590574264526, + 0.008379313163459301, + 1.7768750190734863, + -0.6102496981620789, + 1.7048192024230957, + -0.29564282298088074, + -0.20420490205287933, + -0.23934513330459595, + -0.34791648387908936, + 0.0690477192401886, + -2.602295398712158, + -1.8138929605484009, + -0.38315677642822266, + -0.16421082615852356, + 0.17502839863300323, + -0.13996875286102295, + -0.30929630994796753, + -2.544666290283203, + -1.2411268949508667 + ], + [ + 0.07701912522315979, + 0.9529868364334106, + -1.497549295425415, + -0.3016125559806824, + 1.8504154682159424, + -0.3136579394340515, + 1.6619524955749512, + -0.28860214352607727, + 0.20993559062480927, + 0.6072397232055664, + 0.04243222624063492, + -0.133962482213974, + -2.0948100090026855, + -2.1691184043884277, + 0.44940999150276184, + -0.4183289408683777, + 0.5662978291511536, + 0.5016863942146301, + 0.7054548263549805, + -2.282663345336914, + -1.8892805576324463 + ], + [ + -0.2981111705303192, + 0.038420140743255615, + -1.687281847000122, + -0.37934571504592896, + 1.1300733089447021, + -0.6208293437957764, + 1.8144867420196533, + 0.16701459884643555, + 0.5508663654327393, + 0.4797254800796509, + -0.09421360492706299, + -0.3499920070171356, + -2.5030596256256104, + -1.5871793031692505, + -0.33045580983161926, + -0.29071635007858276, + 0.5385350584983826, + 0.9102991223335266, + 0.3121117353439331, + -2.2699851989746094, + -2.081907272338867 + ], + [ + -0.28236815333366394, + 0.18060839176177979, + -0.32270634174346924, + 0.033799149096012115, + 1.128151774406433, + -0.7577929496765137, + 1.6937967538833618, + -0.813525378704071, + 0.48178502917289734, + -0.6699209809303284, + -0.03810972720384598, + 0.8273124694824219, + -2.2768704891204834, + -1.317199945449829, + 0.0046694036573171616, + 0.5900158286094666, + -0.12453534454107285, + 1.046316385269165, + 0.2770566940307617, + -2.0448076725006104, + -2.4180057048797607 + ], + [ + 0.26116815209388733, + 1.305420994758606, + -0.5357515811920166, + 0.6523149013519287, + 0.5250536203384399, + -0.6821175217628479, + 1.2918037176132202, + 0.3275931477546692, + -0.8845832943916321, + -0.6987056136131287, + 0.10023711621761322, + -0.2849085032939911, + -1.7303776741027832, + -1.0117536783218384, + -0.06847482174634933, + 0.6783800721168518, + -0.04633811116218567, + 0.0702061653137207, + -0.3159322142601013, + -3.004882335662842, + -2.0691566467285156 + ], + [ + 0.5418367385864258, + 1.1724635362625122, + -1.717900276184082, + 0.9217556118965149, + 2.316706657409668, + -0.2411489486694336, + 0.720655083656311, + -0.6330894231796265, + 0.03703225776553154, + 0.38938266038894653, + -0.8305367827415466, + -0.2152116596698761, + -2.614929437637329, + -2.045497179031372, + -0.9044661521911621, + 0.9391879439353943, + -0.2323634922504425, + 0.8392186164855957, + -0.6889055371284485, + -3.0831263065338135, + -1.0035808086395264 + ], + [ + 1.0110303163528442, + 1.0131109952926636, + -1.3200907707214355, + -0.433234840631485, + 1.9466626644134521, + 0.46447479724884033, + 1.528633952140808, + 0.40949776768684387, + 0.07322453707456589, + -0.12349443882703781, + 0.24514153599739075, + 0.005379408597946167, + -2.487516164779663, + -1.1890358924865723, + -0.026332437992095947, + -0.6102214455604553, + 0.6559669971466064, + 0.4371780753135681, + -0.6651540994644165, + -2.8161611557006836, + -1.4271454811096191 + ], + [ + 0.9640108346939087, + 1.0122193098068237, + -1.9809117317199707, + 0.40601760149002075, + 0.9296187162399292, + 0.601173460483551, + 2.229055643081665, + -0.4972975254058838, + 0.6548057794570923, + 0.6165774464607239, + -0.7309268116950989, + 0.697066068649292, + -3.0948128700256348, + -2.4828031063079834, + 0.16930773854255676, + -0.11851751059293747, + -0.8277174830436707, + 0.8252183794975281, + -0.4829612374305725, + -2.922985076904297, + -1.1981642246246338 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03792491555213928, + -0.040427446365356445, + -0.04027438163757324, + 0.02325565740466118, + 0.34960758686065674, + -0.05261008441448212, + 0.0, + 0.018791399896144867, + 0.05303370952606201, + 0.01128687709569931, + 0.01692008785903454, + 0.06139722466468811, + -1.5270280838012695, + 0.0, + -0.004229618236422539, + 0.017936373129487038, + 0.06901376694440842, + -0.04424762725830078, + -0.033029720187187195, + -1.3374013900756836, + 0.0 + ] + ], + "predicted_scores": [ + 5.32202672958374, + 5.083075046539307, + 5.033701419830322, + 4.686803340911865, + 4.6638054847717285, + 4.2977118492126465, + 3.8724560737609863, + 3.265869617462158 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007", + "selected_index": 0, + "state_hash": "58b3d0c1cafc199ad58508912becbaef59038a655cef0caafd263ade5c9b53fa", + "stored_base_utility": 0.8266608119010925, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20032641291618347, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20032641291618347 + }, + "base_progress": 0.20032641291618347, + "base_success": false, + "base_utility": 0.20032641291618347, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2363920658826828, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2363920658826828 + }, + { + "contact_quality": null, + "progress": 0.5597574710845947, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5597574710845947 + }, + { + "contact_quality": null, + "progress": 0.581784188747406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.581784188747406 + }, + { + "contact_quality": null, + "progress": 0.21587182581424713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21587182581424713 + }, + { + "contact_quality": null, + "progress": 0.23852916061878204, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23852916061878204 + }, + { + "contact_quality": null, + "progress": 0.21416914463043213, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21416914463043213 + }, + { + "contact_quality": null, + "progress": 0.22261354327201843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22261354327201843 + }, + { + "contact_quality": null, + "progress": 0.23042941093444824, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23042941093444824 + } + ], + "candidate_progress": [ + 0.2363920658826828, + 0.5597574710845947, + 0.581784188747406, + 0.21587182581424713, + 0.23852916061878204, + 0.21416914463043213, + 0.22261354327201843, + 0.23042941093444824 + ], + "candidate_source_chart_ids": [ + "ms-9ae5c6c4132218ecf49e", + "ms-95028134c6e3670b89d5", + "ms-5ac1bf2d7d319955d45d", + "ms-b8492a3a812095651d75", + "ms-ee2fcbc94d41385713d0", + "ms-95028134c6e3670b89d5", + "ms-c8706d0aa14a930236de", + "ms-016c5de39ed3c111889a" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank6", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank5", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-d1960368ef76bbb10669", + "generated_tangents": [ + [ + -0.009002715349197388, + 0.08356291055679321, + -0.20533940196037292, + -0.1263590306043625, + 0.2406829297542572, + 0.06444787234067917, + 0.6394894123077393, + -0.08873367309570312, + 0.027290765196084976, + -0.16187569499015808, + -0.01088329404592514, + -0.02162127196788788, + -1.5415910482406616, + 0.676408052444458, + 0.19243642687797546, + -0.07440962642431259, + -0.11381717026233673, + -0.1056600958108902, + -0.2868006229400635, + -1.6187689304351807, + 0.2273474633693695 + ], + [ + -0.021199747920036316, + 0.12840774655342102, + -0.16464844346046448, + -0.15847478806972504, + 0.1140025183558464, + 0.10073689371347427, + 0.7886719703674316, + -0.13427844643592834, + 0.012372835539281368, + -0.11800023168325424, + -0.007419198751449585, + 0.04618501663208008, + -1.3953291177749634, + 0.6757482886314392, + 0.2294318526983261, + -0.032656021416187286, + -0.06413744390010834, + -0.05787228047847748, + -0.19111958146095276, + -1.6142559051513672, + 0.09649721533060074 + ], + [ + 0.04666370153427124, + 0.17349736392498016, + -0.10926619172096252, + -0.2744497060775757, + -0.05658543109893799, + 0.648232102394104, + 0.8430483341217041, + -0.20604443550109863, + 0.0634075179696083, + -0.14573881030082703, + -0.19670140743255615, + -0.007543712854385376, + -1.698667049407959, + 0.7503986358642578, + 0.0491400808095932, + 0.022863559424877167, + -0.14207951724529266, + -0.10934339463710785, + -0.24875205755233765, + -1.8660626411437988, + 0.21983829140663147 + ], + [ + 0.020145952701568604, + 0.20961762964725494, + -0.17780622839927673, + -0.07486733794212341, + 0.26312339305877686, + 0.17808207869529724, + 0.5103111863136292, + -0.2040085792541504, + -0.02692084200680256, + -0.09121080487966537, + 0.04213942587375641, + 0.19140036404132843, + -1.3332122564315796, + 0.5214415192604065, + 0.2798238694667816, + 0.008504919707775116, + -0.1379687637090683, + -0.09148384630680084, + -0.18972328305244446, + -1.5710396766662598, + -0.033500656485557556 + ], + [ + -0.004896685481071472, + 0.20563440024852753, + -0.2885300815105438, + -0.1926400363445282, + 0.05656532943248749, + 0.5185076594352722, + 0.8056787252426147, + -0.11853313446044922, + 0.03304257243871689, + -0.008739106357097626, + -0.08999045938253403, + -0.03273311257362366, + -1.4083976745605469, + 0.622584342956543, + 0.06208796426653862, + -0.019653087481856346, + -0.040728285908699036, + -0.08429069817066193, + -0.19704973697662354, + -1.7740581035614014, + 0.3122518062591553 + ], + [ + -0.12031464278697968, + 0.06919682770967484, + 0.00935259461402893, + -0.1544983685016632, + 0.37343141436576843, + 0.3457547724246979, + 0.8786454200744629, + -0.12150397896766663, + -0.06065675616264343, + -0.09790251404047012, + 0.06326153874397278, + 0.30927544832229614, + -1.5503307580947876, + 0.6292473077774048, + -0.06414148211479187, + -0.1344526708126068, + -0.08821617066860199, + 0.03610348701477051, + -0.2107485830783844, + -1.6647887229919434, + 0.1340775191783905 + ], + [ + -0.19174478948116302, + 0.18334776163101196, + -0.1974191963672638, + -0.20941951870918274, + 0.10067513585090637, + 0.5494449734687805, + 0.6163052916526794, + -0.17247524857521057, + 0.03027583658695221, + 0.05010638386011124, + -0.07675803452730179, + -0.14911431074142456, + -1.1016347408294678, + 0.7832385301589966, + 0.23437924683094025, + -0.093446746468544, + -0.009180739521980286, + -0.04516807198524475, + -0.08957105875015259, + -1.637145757675171, + 0.07488099485635757 + ], + [ + -0.009413033723831177, + 0.18980033695697784, + -0.1679915487766266, + -0.07231934368610382, + 0.4116837978363037, + 0.036274492740631104, + 0.6768482327461243, + -0.10792264342308044, + -0.021929940208792686, + -0.07781047374010086, + 0.11434338986873627, + 0.0351085364818573, + -1.3289271593093872, + 0.6333006620407104, + 0.10635165125131607, + -0.07997269928455353, + 0.007293254137039185, + -0.19972731173038483, + -0.2883608937263489, + -1.587573766708374, + 0.32571089267730713 + ] + ], + "generated_utilities": [ + 0.2363920658826828, + 0.5597574710845947, + 0.581784188747406, + 0.21587182581424713, + 0.23852916061878204, + 0.21416914463043213, + 0.22261354327201843, + 0.23042941093444824 + ], + "group_id": "ms-d1960368ef76bbb10669", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.842734158039093, + 2.0, + 0.1342049241065979, + 0.18085119128227234, + 0.5091985464096069, + 0.016112878918647766, + 0.5080842971801758, + 0.19944033026695251, + 0.5132148265838623, + 0.5265648365020752, + 0.11437654495239258, + 0.04028162360191345, + 0.04675377905368805, + 0.13603349030017853, + 0.12877747416496277, + 0.05102705955505371 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2770041823387146, + 0.08894228935241699, + 0.07033950090408325, + 0.12426833808422089, + 0.16520383954048157, + 0.023725181818008423, + 0.0, + -0.25569218397140503, + -0.057803913950920105, + 0.06703128665685654, + 0.1732010841369629, + 0.07705201953649521, + -1.0374658107757568, + 0.0, + 0.04586680978536606, + 0.028830496594309807, + -0.0374501496553421, + 0.10374182462692261, + -0.007100850343704224, + -1.7975506782531738, + 0.0 + ], + [ + -0.4268343448638916, + 0.13483470678329468, + -0.06075429916381836, + 0.19817759096622467, + 0.09922409057617188, + 0.6547578573226929, + 0.0, + 0.15026617050170898, + 0.002472013235092163, + -0.31349992752075195, + 0.5119487047195435, + 0.571077287197113, + -1.0374658107757568, + 0.0, + -0.5828977823257446, + -0.20166385173797607, + -0.7097158432006836, + 0.8187870979309082, + -0.0694284737110138, + -1.7975506782531738, + 0.0 + ], + [ + -0.23462048172950745, + 1.088942289352417, + -0.5585125088691711, + 0.10042184591293335, + 0.4514904320240021, + -0.12210102379322052, + 2.176525354385376, + -0.10348781198263168, + 0.19529865682125092, + 0.12132670730352402, + 0.7056414484977722, + 0.12544797360897064, + -2.037465810775757, + 0.6289246678352356, + -0.008351176977157593, + 0.0010435802396386862, + 0.017314812168478966, + 0.6334809064865112, + -0.3720659911632538, + -2.797550678253174, + -1.5395829677581787 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.353050708770752, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.2578493356704712, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0791659355163574 + ], + [ + -0.09280626475811005, + 1.1662368774414062, + -0.8228501081466675, + 0.24497312307357788, + 0.5648186802864075, + -0.03522863984107971, + 2.259786367416382, + 0.14498579502105713, + 0.2030920535326004, + 0.39727938175201416, + 0.8023015856742859, + 0.3385764956474304, + -1.8738088607788086, + 0.7300182580947876, + 0.20887598395347595, + 0.23107871413230896, + 0.3028872311115265, + 0.7558343410491943, + -0.5975744724273682, + -3.0329887866973877, + -1.8127071857452393 + ], + [ + -0.13783538341522217, + 0.9487118721008301, + -0.3492928743362427, + -0.2956767678260803, + 0.2793389856815338, + -0.16702969372272491, + 2.558225631713867, + 0.17572781443595886, + 0.570004940032959, + 0.27895650267601013, + 0.6778483986854553, + -0.27156615257263184, + -1.8747575283050537, + 1.0240070819854736, + 0.27303546667099, + 0.33387133479118347, + 0.1442956030368805, + 0.24661725759506226, + -0.5561860799789429, + -2.9813332557678223, + -1.833754062652588 + ], + [ + -0.18553051352500916, + 1.0909820795059204, + -0.27226096391677856, + 0.1659364104270935, + 0.7108361721038818, + -0.27768880128860474, + 2.239938974380493, + 0.019483201205730438, + 0.21263465285301208, + -0.09839647263288498, + 0.9676799178123474, + -0.36985379457473755, + -2.133316993713379, + 0.3758644461631775, + -0.47750571370124817, + 0.17470555007457733, + -0.17035381495952606, + 0.2707958221435547, + -0.5672953128814697, + -2.394361972808838, + -1.668647289276123 + ], + [ + -0.3557406961917877, + 0.9564932584762573, + -1.0014276504516602, + -0.38222843408584595, + 0.41349664330482483, + -0.05637756735086441, + 1.6011608839035034, + -0.31295540928840637, + -0.15234358608722687, + 0.4648304879665375, + 0.2744198739528656, + -0.44805359840393066, + -2.1544065475463867, + 1.166503667831421, + 0.014861157163977623, + -0.23438170552253723, + 0.1278134286403656, + 0.5812121033668518, + -0.7571046352386475, + -2.5030438899993896, + -2.100955009460449 + ], + [ + -0.16500982642173767, + 0.7556273937225342, + -0.988593339920044, + 0.6213532090187073, + 0.15840065479278564, + 0.5245341062545776, + 2.452624797821045, + -0.5693200826644897, + -0.32241708040237427, + 0.49458175897598267, + 1.3341540098190308, + 0.059507817029953, + -2.304706335067749, + 0.7334492206573486, + -0.20702646672725677, + 0.32367274165153503, + -0.5396079421043396, + 0.9809319972991943, + -0.6172815561294556, + -3.197409152984619, + -0.9358728528022766 + ], + [ + -0.07666900753974915, + 1.8442692756652832, + -0.6603910326957703, + -0.40882760286331177, + 0.22092406451702118, + -0.23551997542381287, + 1.6105436086654663, + 0.09144749492406845, + 0.8141599893569946, + 0.8528898358345032, + 1.0663814544677734, + 0.3283998966217041, + -1.7124712467193604, + 1.2278027534484863, + 0.7039424180984497, + 0.3645748496055603, + -0.2968631386756897, + 0.4998055696487427, + -0.9362069368362427, + -3.461055278778076, + -0.8184676766395569 + ], + [ + -0.1520652174949646, + 1.405963659286499, + -0.7959750294685364, + -0.39229461550712585, + 0.46590372920036316, + -0.8267906904220581, + 2.952258586883545, + 0.5923218727111816, + -0.6996292471885681, + 0.8352785706520081, + -0.15220212936401367, + -0.1307106763124466, + -2.849807024002075, + 0.4983477294445038, + 0.5902999639511108, + -0.5390225648880005, + 0.13897128403186798, + 0.21545273065567017, + -1.1186240911483765, + -3.1762759685516357, + -1.0118725299835205 + ], + [ + -0.564094066619873, + 0.4751617908477783, + -1.3977731466293335, + 0.8189605474472046, + 0.17298781871795654, + 0.2476775199174881, + 2.8234164714813232, + -0.4518245458602905, + -0.7904537916183472, + -0.39086827635765076, + -0.05634337663650513, + -0.09099806845188141, + -1.9690911769866943, + 1.1869969367980957, + -0.5615661144256592, + 0.32865890860557556, + -0.8041903376579285, + 1.3324832916259766, + -0.6190662384033203, + -2.4903852939605713, + -2.3098156452178955 + ], + [ + -0.2328718602657318, + 0.9724385142326355, + -1.5535115003585815, + 0.7688977718353271, + 0.4409410357475281, + -0.8374075889587402, + 1.2719230651855469, + 0.653096616268158, + -0.2887840270996094, + 0.4892227351665497, + 0.9943453669548035, + 0.5566720962524414, + -2.496413469314575, + 0.6299278140068054, + -0.08689186722040176, + 0.024782167747616768, + 0.5747005939483643, + 0.023068726062774658, + -1.111369252204895, + -2.187150001525879, + -1.2047184705734253 + ], + [ + -1.1679872274398804, + 1.9997228384017944, + -0.45194220542907715, + -0.8861345648765564, + -0.5369991064071655, + 0.46434593200683594, + 2.175499439239502, + 0.7599186301231384, + 0.38914406299591064, + 0.6877046823501587, + -0.004208564758300781, + -0.1321776658296585, + -1.3474420309066772, + 0.8496907353401184, + 0.4218248426914215, + -0.9354817271232605, + 0.945868730545044, + 1.60379958152771, + -1.195396065711975, + -3.597724676132202, + -1.9041136503219604 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04476936161518097, + 0.08894228935241699, + 0.06436485052108765, + -0.0775555968284607, + 0.04825553297996521, + -0.060782790184020996, + 0.0, + -0.07688627392053604, + -0.06308497488498688, + -0.03861912339925766, + 0.1235572099685669, + -0.0017572194337844849, + -1.0374658107757568, + 0.0, + 0.03638586029410362, + 0.0556478351354599, + 0.0387292355298996, + 0.044096171855926514, + 0.029627740383148193, + -1.7975506782531738, + 0.0 + ] + ], + "predicted_scores": [ + 8.316713333129883, + 8.251832962036133, + 8.109463691711426, + 7.959565162658691, + 7.918623924255371, + 7.686976909637451, + 7.613646030426025, + 7.346455097198486 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "416183106", + "selected_index": 0, + "state_hash": "cd187560c1c39db9e6bf7d9a611a8b7108dc88766de58fd62b4317f0bbcc1b50", + "stored_base_utility": 0.842734158039093, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8350887894630432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8350887894630432 + }, + "base_progress": 0.8350887894630432, + "base_success": false, + "base_utility": 0.8350887894630432, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07415902614593506, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07415902614593506 + }, + { + "contact_quality": null, + "progress": 0.21383549273014069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21383549273014069 + }, + { + "contact_quality": null, + "progress": 0.17038175463676453, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17038175463676453 + }, + { + "contact_quality": null, + "progress": 0.12335775792598724, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12335775792598724 + }, + { + "contact_quality": null, + "progress": 0.1851280927658081, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1851280927658081 + }, + { + "contact_quality": null, + "progress": 0.15958069264888763, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15958069264888763 + }, + { + "contact_quality": null, + "progress": 0.12082469463348389, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12082469463348389 + }, + { + "contact_quality": null, + "progress": 0.21975377202033997, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21975377202033997 + } + ], + "candidate_progress": [ + 0.07415902614593506, + 0.21383549273014069, + 0.17038175463676453, + 0.12335775792598724, + 0.1851280927658081, + 0.15958069264888763, + 0.12082469463348389, + 0.21975377202033997 + ], + "candidate_source_chart_ids": [ + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-5dfbb16e173e1a3e2bd6" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-efdc07c985935d871adf", + "generated_tangents": [ + [ + 0.3686366677284241, + 0.043556421995162964, + -0.15216979384422302, + -0.013488903641700745, + 0.309867262840271, + -2.2373733520507812, + -0.4430053234100342, + 0.35782021284103394, + 0.1588248312473297, + -0.13483312726020813, + 0.5717215538024902, + 0.402296245098114, + -1.5094940662384033, + -0.0051206499338150024, + 0.2405378669500351, + 0.049752142280340195, + 0.14417493343353271, + -0.08577902615070343, + 0.8010730743408203, + -0.8206570744514465, + -0.3179226517677307 + ], + [ + 0.012061640620231628, + -0.04155142232775688, + -0.06605860590934753, + 0.1513940393924713, + 0.011064141988754272, + -1.8977504968643188, + -0.11028212308883667, + 0.06433641910552979, + 0.014126881957054138, + -0.03595695644617081, + 0.21009963750839233, + 0.18688516318798065, + -1.3424303531646729, + -0.04337623715400696, + 0.05805295705795288, + 0.09326297044754028, + 0.014324337244033813, + -0.1732342392206192, + 0.16666626930236816, + -1.0746729373931885, + -0.015538901090621948 + ], + [ + 0.04373123496770859, + -0.026497136801481247, + 0.0027818381786346436, + -0.020371943712234497, + -0.13714423775672913, + -1.9406591653823853, + -0.2620019316673279, + 0.15213267505168915, + -0.04027959704399109, + 0.01317993551492691, + 0.06469055265188217, + 0.05747249722480774, + -1.3779916763305664, + -0.11872003972530365, + 0.05823354423046112, + 0.07354122400283813, + 0.08978244662284851, + -0.1656661480665207, + 0.028914421796798706, + -1.2013511657714844, + -0.08568954467773438 + ], + [ + 0.07770948857069016, + 0.09926199913024902, + -0.08355829119682312, + -0.23897604644298553, + -0.11167275905609131, + -2.064149856567383, + -0.6689459681510925, + 0.2126466929912567, + -0.07760970294475555, + -0.264264851808548, + 0.410615473985672, + 0.2747187912464142, + -1.1980723142623901, + -0.3064813017845154, + 0.07732196152210236, + 0.11608351767063141, + -0.06674216687679291, + -0.051100775599479675, + 0.43780913949012756, + -1.3022537231445312, + -0.2767101228237152 + ], + [ + 0.08911243081092834, + -0.01669311337172985, + -0.05099359154701233, + -0.04101960361003876, + -0.19033390283584595, + -1.7867720127105713, + -0.2750092148780823, + 0.15249690413475037, + -0.04276160150766373, + -0.028678081929683685, + 0.10094863921403885, + 0.11168846487998962, + -1.1930760145187378, + -0.21459124982357025, + 0.07177359610795975, + 0.10467469692230225, + 0.11782080680131912, + -0.10453735291957855, + 0.0208817720413208, + -1.0487130880355835, + -0.07126760482788086 + ], + [ + 0.11916574835777283, + -0.019409148022532463, + 0.004142403602600098, + -0.025014802813529968, + -0.031183749437332153, + -1.8143374919891357, + -0.254702627658844, + 0.18722927570343018, + -0.0933188870549202, + 0.07084082067012787, + 0.20232877135276794, + 0.10192329436540604, + -1.0879679918289185, + -0.05684955418109894, + -0.0026487447321414948, + 0.06035435572266579, + 0.06609466671943665, + -0.03150962293148041, + 0.03563234210014343, + -1.0091068744659424, + 0.0016781985759735107 + ], + [ + 0.3439967632293701, + 0.10091355443000793, + -0.22781983017921448, + -0.0903288871049881, + -0.08182793855667114, + -2.4931893348693848, + -0.6921324133872986, + 0.34496474266052246, + 0.0554824024438858, + 0.018603794276714325, + 0.31898757815361023, + 0.1792585402727127, + -1.3971823453903198, + -0.6852246522903442, + 0.13927975296974182, + 0.12209755182266235, + 0.29126888513565063, + -0.08204825222492218, + 0.39308786392211914, + -1.060652732849121, + -0.22201550006866455 + ], + [ + -0.02788279950618744, + -0.12240372598171234, + -0.09374865889549255, + -0.18003249168395996, + -0.21576184034347534, + -2.4772815704345703, + -0.6488150954246521, + 0.2751406133174896, + -0.0911252498626709, + -0.12399519234895706, + 0.4060862958431244, + -0.11341851949691772, + -1.222923994064331, + -0.26216763257980347, + -0.05457804724574089, + 0.15642470121383667, + 0.1643677055835724, + -0.03815515339374542, + 0.02476915717124939, + -1.194394588470459, + -0.12704795598983765 + ] + ], + "generated_utilities": [ + 0.07415902614593506, + 0.21383549273014069, + 0.17038175463676453, + 0.12335775792598724, + 0.1851280927658081, + 0.15958069264888763, + 0.12082469463348389, + 0.21975377202033997 + ], + "group_id": "ms-efdc07c985935d871adf", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8197344541549683, + 2.0, + 0.8125, + 0.8125142455101013, + 0.7847581505775452, + 0.7794750332832336, + 0.5920823216438293, + 0.5697357058525085, + 2.0, + 0.5847549438476562, + 0.09384849667549133, + 0.07791829109191895, + 0.011395931243896484, + 0.09286168217658997, + 0.10894183814525604, + 0.0616525262594223 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2827143669128418, + -0.12206736207008362, + 0.1543778032064438, + 0.27268126606941223, + -0.17288367450237274, + -2.5338973999023438, + -1.7615662813186646, + -0.19161763787269592, + 0.1886080503463745, + 0.31686434149742126, + 0.5112922191619873, + 0.14046527445316315, + -2.17134952545166, + -1.8021173477172852, + 0.11115366220474243, + 0.25490811467170715, + -0.025668112561106682, + 0.4787765145301819, + 0.46609628200531006, + -1.711870551109314, + -1.5312864780426025 + ], + [ + 0.03046044334769249, + 0.042622197419404984, + -0.12103251367807388, + 0.7814357876777649, + 0.5093902349472046, + -2.648094654083252, + -1.5764049291610718, + -0.16382060945034027, + 0.15227201581001282, + -0.29746392369270325, + 0.394713819026947, + 0.044284895062446594, + -2.150869131088257, + -1.853789210319519, + 0.24474957585334778, + 0.16074210405349731, + -0.30960795283317566, + 0.986261248588562, + 0.4512072801589966, + -1.9782602787017822, + -1.814382553100586 + ], + [ + -0.47424501180648804, + 0.4193490147590637, + 0.31025299429893494, + 0.436926007270813, + 0.07080172002315521, + -2.2750933170318604, + -2.139326572418213, + 0.19072748720645905, + -0.05066118389368057, + -0.38944289088249207, + 0.11913672089576721, + -0.4728909432888031, + -2.630631446838379, + -1.9070930480957031, + 0.5720394849777222, + 0.1898895502090454, + -0.0872373953461647, + 0.3081148862838745, + 0.6700773239135742, + -1.8990029096603394, + -1.655537486076355 + ], + [ + -0.112115278840065, + -0.5390006899833679, + -0.27749133110046387, + -0.08230990171432495, + 0.18304404616355896, + -2.5003390312194824, + -1.177594542503357, + -0.37152859568595886, + 0.1827065348625183, + -0.1805598884820938, + 0.8884490728378296, + 0.3141748607158661, + -2.7738473415374756, + -1.8585447072982788, + -0.6611859798431396, + 0.5365979075431824, + 0.024639926850795746, + 1.0166820287704468, + 0.6261310577392578, + -1.3820469379425049, + -1.1859028339385986 + ], + [ + -0.013153300620615482, + 0.6059781908988953, + -0.7563084363937378, + 0.6436030864715576, + 0.4828413128852844, + -1.8755569458007812, + -1.896214246749878, + -0.445474773645401, + -0.6578570604324341, + -0.21610267460346222, + 1.3274874687194824, + -0.6251860857009888, + -2.005418300628662, + -1.463393211364746, + -0.40788042545318604, + -0.5932772159576416, + -0.6501284837722778, + 0.1491221785545349, + 0.8418278694152832, + -2.578186511993408, + -1.628711462020874 + ], + [ + -0.051258739084005356, + 0.6121965050697327, + -0.4303005337715149, + 0.378423810005188, + -0.4187525510787964, + -2.0054633617401123, + -1.5839252471923828, + 0.38306933641433716, + -0.8548648953437805, + 0.5034631490707397, + 0.3682137727737427, + 0.867935061454773, + -2.5900886058807373, + -1.8283436298370361, + 0.3810080289840698, + -0.6684208512306213, + 0.5362142324447632, + -0.17851078510284424, + 1.083348035812378, + -1.9181195497512817, + -2.306365728378296 + ], + [ + 0.5487861037254333, + 0.6947635412216187, + -0.3694673776626587, + 1.3571128845214844, + -0.20053730905056, + -2.9099156856536865, + -1.493881106376648, + 0.9321863055229187, + -0.355974406003952, + 0.6561991572380066, + -0.29231804609298706, + 0.4913230240345001, + -1.8409770727157593, + -2.0504229068756104, + 0.14528872072696686, + 0.483799010515213, + -0.32654285430908203, + 1.6123273372650146, + 0.03987070918083191, + -0.8449068665504456, + -2.3494603633880615 + ], + [ + -0.3259732127189636, + -0.5385614633560181, + 0.7304226160049438, + -0.12314724922180176, + 0.03945803642272949, + -1.5919837951660156, + -1.907224416732788, + 0.3930681645870209, + -0.8835994005203247, + -0.3352096974849701, + 0.17923185229301453, + 0.17222784459590912, + -2.512176752090454, + -1.859424352645874, + 0.536429762840271, + 0.11454571038484573, + 0.3300909101963043, + 0.5491204261779785, + -0.04762554168701172, + -1.1976318359375, + -0.5251960158348083 + ], + [ + 0.33097749948501587, + -0.04192434251308441, + -0.34520047903060913, + -0.04110339283943176, + -0.5535940527915955, + -2.2057979106903076, + -2.5934860706329346, + -0.9023091793060303, + 0.5658158659934998, + -0.6125177145004272, + 1.095580816268921, + -0.05609257519245148, + -1.606997013092041, + -2.0733447074890137, + -0.5881220102310181, + 0.8953573703765869, + 0.9258373975753784, + 0.27281180024147034, + 0.7141706943511963, + -0.9142226576805115, + -1.3141751289367676 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05978316813707352, + 0.057473838329315186, + -0.09624473005533218, + -0.011721253395080566, + 0.028141841292381287, + -1.4311916828155518, + 0.0, + -0.024008113890886307, + 0.07049623876810074, + 0.03528204560279846, + -0.010002851486206055, + 0.010910652577877045, + -1.1452367305755615, + 0.0, + 0.020785929635167122, + -0.03374411538243294, + 0.11616832762956619, + 0.05772918462753296, + 0.01671847701072693, + -0.828272819519043, + 0.0 + ], + [ + -0.13237658143043518, + -0.3393171727657318, + 0.29588890075683594, + 0.8627858757972717, + -0.1666169911623001, + -2.520320177078247, + -1.6265922784805298, + -0.031582750380039215, + 0.11019003391265869, + -0.2699315845966339, + 0.9513242244720459, + 0.15523536503314972, + -1.7896702289581299, + -1.2776744365692139, + -0.3942914605140686, + -0.30590808391571045, + -0.3098926246166229, + 0.8258905410766602, + 0.29909753799438477, + -1.5711073875427246, + -1.5744072198867798 + ] + ], + "predicted_scores": [ + 3.255740165710449, + 2.8353159427642822, + 2.6269359588623047, + 2.6108710765838623, + 2.6059200763702393, + 2.0757269859313965, + 1.4323501586914062, + 1.3334894180297852 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "706479600", + "selected_index": 0, + "state_hash": "b482f902b03ca480898798a8b5f19263d7cbb18c066305330b0796b607427fa7", + "stored_base_utility": 0.8197344541549683, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03719887137413025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03719887137413025 + }, + "base_progress": 0.03719887137413025, + "base_success": false, + "base_utility": 0.03719887137413025, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07726143300533295, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07726143300533295 + }, + { + "contact_quality": null, + "progress": 0.06953388452529907, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06953388452529907 + }, + { + "contact_quality": null, + "progress": 0.046124204993247986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.046124204993247986 + }, + { + "contact_quality": null, + "progress": 0.09917314350605011, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09917314350605011 + }, + { + "contact_quality": null, + "progress": 0.06149482727050781, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06149482727050781 + }, + { + "contact_quality": null, + "progress": 0.04581362009048462, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04581362009048462 + }, + { + "contact_quality": null, + "progress": 0.0635814368724823, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0635814368724823 + }, + { + "contact_quality": null, + "progress": 0.05730763077735901, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05730763077735901 + } + ], + "candidate_progress": [ + 0.07726143300533295, + 0.06953388452529907, + 0.046124204993247986, + 0.09917314350605011, + 0.06149482727050781, + 0.04581362009048462, + 0.0635814368724823, + 0.05730763077735901 + ], + "candidate_source_chart_ids": [ + "ms-d4c5d7eea19d94314e2e", + "ms-0a4d4909900fb0bc4a9a", + "ms-5ac1bf2d7d319955d45d", + "ms-ee2fcbc94d41385713d0", + "ms-07b479748a91a9c92b20", + "ms-2fef2f43af8009bb88f1", + "ms-bb299baa34e49eafb9fc", + "ms-0b1dd0e538b8df15a6e4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank7" + ], + "candidates_evaluated": true, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "generated_tangents": [ + [ + -0.06495420634746552, + 0.3671957850456238, + 0.11139822006225586, + -1.7162952423095703, + -0.44263845682144165, + 0.37888675928115845, + 0.06493577361106873, + 0.20768825709819794, + -0.4021793603897095, + -0.16209673881530762, + 1.0889604091644287, + 0.1378367841243744, + -1.6365530490875244, + -0.23112384974956512, + -0.035061415284872055, + 0.15449780225753784, + 0.05566084384918213, + -0.3404231667518616, + -0.08311712741851807, + -1.6378886699676514, + -0.43263307213783264 + ], + [ + -0.09358946979045868, + 0.6348069310188293, + 0.17672815918922424, + -1.7474277019500732, + -0.270841121673584, + 0.7005739212036133, + -0.39553868770599365, + -0.0020561963319778442, + -0.38483887910842896, + -0.09940668195486069, + 1.285354733467102, + 0.31022900342941284, + -1.4367177486419678, + 0.034482650458812714, + -0.06608331203460693, + -0.06098440662026405, + 0.09633795917034149, + -0.2390567809343338, + -0.12952619791030884, + -1.5066163539886475, + -0.15631580352783203 + ], + [ + 0.01595573127269745, + 0.6981759667396545, + 0.25221866369247437, + -1.8958021402359009, + -0.30718374252319336, + 0.6601626873016357, + -0.33606773614883423, + -0.16047650575637817, + -0.518339991569519, + -0.08485382050275803, + 1.1284399032592773, + 0.06395082175731659, + -1.854662299156189, + -0.09885132312774658, + 0.08961603790521622, + 0.1571999043226242, + 0.1967727392911911, + -0.3777223229408264, + -0.04145368933677673, + -1.6753199100494385, + 0.10013195127248764 + ], + [ + -0.023770973086357117, + 0.6930935978889465, + 0.18124186992645264, + -1.7630515098571777, + -0.09737259149551392, + 0.8673893213272095, + -0.19814473390579224, + -0.0702391266822815, + -0.6056856513023376, + -0.07049527019262314, + 1.1719478368759155, + 0.19473861157894135, + -1.3805370330810547, + 0.011462561786174774, + 0.03564780205488205, + 0.13740402460098267, + 0.17615480720996857, + -0.35321688652038574, + -0.03179416060447693, + -1.5796303749084473, + 0.09967153519392014 + ], + [ + -0.0340493768453598, + 0.5901545286178589, + 0.1768604815006256, + -1.8761394023895264, + -0.2016609013080597, + 0.9064714908599854, + -0.05415022373199463, + -0.011286810040473938, + -0.5596218109130859, + -0.11609595268964767, + 1.1898362636566162, + 0.3228369355201721, + -1.4279693365097046, + -0.09022350609302521, + 0.08991342782974243, + 0.09004552662372589, + 0.16198056936264038, + -0.35726600885391235, + 0.043157368898391724, + -1.5128161907196045, + 0.0026268362998962402 + ], + [ + 0.021356597542762756, + 0.6034295558929443, + 0.36402377486228943, + -1.8414827585220337, + -0.3594658374786377, + 0.7170552015304565, + -0.1328151822090149, + -0.12948063015937805, + -0.48487037420272827, + -0.10393033176660538, + 1.1427063941955566, + 0.17752020061016083, + -1.6741441488265991, + -0.18471203744411469, + 0.09598825871944427, + 0.12697991728782654, + 0.17526964843273163, + -0.3313748240470886, + 0.02222144603729248, + -1.4967906475067139, + 0.025683149695396423 + ], + [ + 0.0121297687292099, + 0.5627127885818481, + 0.2642773985862732, + -2.0473899841308594, + -0.398726224899292, + 0.7902873754501343, + -0.04045999050140381, + -0.06379231810569763, + -0.5675560832023621, + -0.11296277493238449, + 1.1654249429702759, + 0.12139623612165451, + -1.8295652866363525, + -0.07097220420837402, + 0.10972034931182861, + 0.06632433831691742, + 0.103777214884758, + -0.2688276767730713, + 0.0077446699142456055, + -1.3927452564239502, + -0.1138572096824646 + ], + [ + 0.07004734128713608, + 0.6754263639450073, + 0.25278669595718384, + -1.6248010396957397, + -0.00012731552124023438, + 0.3310000002384186, + -0.10137492418289185, + -0.09762006998062134, + -0.623462975025177, + -0.09329239279031754, + 1.2168601751327515, + 0.25074687600135803, + -1.4885960817337036, + 0.014419980347156525, + 0.04936787486076355, + 0.1198219507932663, + 0.22938913106918335, + -0.3071914315223694, + -0.03162345290184021, + -1.460853099822998, + 0.0734221562743187 + ] + ], + "generated_utilities": [ + 0.07726143300533295, + 0.06953388452529907, + 0.046124204993247986, + 0.09917314350605011, + 0.06149482727050781, + 0.04581362009048462, + 0.0635814368724823, + 0.05730763077735901 + ], + "group_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558897972106934, + 2.0, + 0.04248148202896118, + 0.05394089221954346, + 0.057141318917274475, + 0.03495943546295166, + 0.08633947372436523, + 0.046462297439575195, + 0.043458908796310425, + 0.023027777671813965, + 0.061922311782836914, + 0.034134700894355774, + 0.010651648044586182, + 0.01158277690410614, + 0.02622869610786438, + 0.012083128094673157 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.4298704266548157, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + 0.09995961934328079, + 1.162198781967163, + 0.0, + -0.00648990273475647, + -0.3028862476348877, + -0.0050805360078811646, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + 0.23185759782791138, + -0.11529958993196487, + -0.058992013335227966, + -0.010845988988876343, + 0.06423944234848022, + -2.0730173587799072, + 0.0 + ], + [ + -0.48041507601737976, + 0.6002404689788818, + 0.5398462414741516, + -2.198392152786255, + 0.1505453884601593, + 1.162198781967163, + 0.0, + 0.32028457522392273, + -0.264798641204834, + -0.27968189120292664, + 1.143568515777588, + 0.623285710811615, + -1.6758570671081543, + 0.0, + 0.3762620687484741, + 0.20924779772758484, + 0.033039893954992294, + -0.22763806581497192, + 0.8496624827384949, + -2.0730173587799072, + 0.0 + ], + [ + -0.4385460615158081, + 1.6002404689788818, + 1.1335551738739014, + -3.198392152786255, + 0.11579504609107971, + 2.162198781967163, + -1.8792641162872314, + 0.44874686002731323, + -1.264798641204834, + 0.08758684247732162, + 2.143568515777588, + 1.074564814567566, + -2.6758570671081543, + 1.4318934679031372, + -0.01953941211104393, + 0.012724362313747406, + 0.02134235017001629, + 0.39346790313720703, + -0.15033751726150513, + -3.0730173587799072, + -1.573193907737732 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.758528232574463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8637869358062744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.146387815475464 + ], + [ + -0.4582313299179077, + 1.465004801750183, + 0.9327377080917358, + -2.900977373123169, + -0.12230202555656433, + 2.23574161529541, + -2.1357157230377197, + 0.42294827103614807, + -1.506612777709961, + 0.29363107681274414, + 2.034971237182617, + 1.023054599761963, + -2.559215784072876, + 1.1742838621139526, + 0.026679620146751404, + -0.0811251625418663, + 0.16039283573627472, + 0.3527735471725464, + 0.041033580899238586, + -2.8834660053253174, + -1.2895927429199219 + ], + [ + -0.8261920809745789, + 1.8162921667099, + 1.1455696821212769, + -3.5306012630462646, + -0.15104833245277405, + 2.447983741760254, + -2.128190755844116, + 0.19818204641342163, + -1.146593689918518, + 0.03477605804800987, + 2.4299705028533936, + 0.8797258734703064, + -2.331338882446289, + 1.5516352653503418, + -0.19862954318523407, + -0.2736409604549408, + -0.21770769357681274, + 0.47925806045532227, + -0.15124429762363434, + -3.0052826404571533, + -1.2108670473098755 + ], + [ + -0.2760573625564575, + 1.6254181861877441, + 0.8188510537147522, + -3.0690624713897705, + -0.22635167837142944, + 2.051481246948242, + -1.417292833328247, + 0.026889145374298096, + -1.3347617387771606, + -0.15878626704216003, + 1.9778507947921753, + 0.7640928030014038, + -3.1165366172790527, + 1.862673282623291, + -0.451316773891449, + -0.15708383917808533, + 0.26080578565597534, + 0.6509285569190979, + 0.23991608619689941, + -3.50675892829895, + -1.297291874885559 + ], + [ + -0.8321853280067444, + 1.9553383588790894, + 1.5495907068252563, + -3.790785312652588, + 0.3716217279434204, + 2.5994434356689453, + -2.1380205154418945, + -0.10642725229263306, + -0.7562419772148132, + -0.442053884267807, + 2.3244006633758545, + 1.161316990852356, + -2.699451208114624, + 1.6399294137954712, + 0.31003186106681824, + 0.5301566123962402, + -0.16996291279792786, + 0.9281869530677795, + 0.3683878183364868, + -2.650294780731201, + -1.1098875999450684 + ], + [ + -0.6997023820877075, + 1.4820023775100708, + 1.3613333702087402, + -3.8556365966796875, + -0.5036565065383911, + 1.4896399974822998, + -2.322740077972412, + 0.03597790002822876, + -1.1858596801757812, + -0.018024958670139313, + 2.0489397048950195, + 0.8914362192153931, + -2.3395535945892334, + 1.1835731267929077, + 0.5559698343276978, + 0.5892979502677917, + 0.4496755301952362, + 0.6100684404373169, + 0.22263532876968384, + -3.0098915100097656, + -1.455836296081543 + ], + [ + -0.594323456287384, + 1.846432089805603, + 0.8105549812316895, + -2.6211562156677246, + 0.2629232406616211, + 1.4025108814239502, + -2.3828163146972656, + 0.530456006526947, + -1.3874722719192505, + 0.49274298548698425, + 1.5545053482055664, + 0.4927254915237427, + -3.1490466594696045, + 1.5643255710601807, + 0.5331913232803345, + -0.53307044506073, + 0.7433652281761169, + 0.6546258926391602, + -0.74479740858078, + -3.721141815185547, + -0.9948303699493408 + ], + [ + -0.4409070312976837, + 2.4138050079345703, + 1.3223544359207153, + -3.3717033863067627, + -0.03033369779586792, + 2.350304365158081, + -2.112061023712158, + -0.22452247142791748, + -1.420605182647705, + -0.026168949902057648, + 1.4026775360107422, + 0.9532885551452637, + -2.6276464462280273, + 2.1231069564819336, + -0.49286600947380066, + -0.3450928032398224, + 0.5427718758583069, + -0.03872644901275635, + -0.6895387172698975, + -2.357243061065674, + -2.3816022872924805 + ], + [ + 0.5156354308128357, + 2.2980997562408447, + 2.0945849418640137, + -3.690303087234497, + 1.0696191787719727, + 2.266188144683838, + -1.0399069786071777, + 0.012727499008178711, + -1.2163764238357544, + 0.5646419525146484, + 1.1664416790008545, + 1.563025951385498, + -3.425304412841797, + 0.5126914978027344, + -0.18885818123817444, + -0.5277795791625977, + -0.1741226613521576, + 0.2763793468475342, + 0.8411282896995544, + -3.2598915100097656, + -2.2270073890686035 + ], + [ + -1.2896747589111328, + 2.3828718662261963, + 1.191500186920166, + -3.9747776985168457, + 0.7604506015777588, + 2.298020839691162, + -1.0116384029388428, + 1.177309513092041, + -1.6719520092010498, + -0.4176469147205353, + 2.709542989730835, + 0.26378321647644043, + -2.119570255279541, + 0.7668893933296204, + 0.6772738099098206, + -0.5338667035102844, + -0.7434203028678894, + 0.2167283445596695, + 0.03269544243812561, + -3.895913600921631, + -0.9124676585197449 + ], + [ + -1.323930263519287, + 2.338547468185425, + 1.4313098192214966, + -2.960175037384033, + 1.1079072952270508, + 1.5746114253997803, + -1.8946435451507568, + -0.4213535189628601, + -1.922096848487854, + 0.6645004153251648, + 2.1448850631713867, + 0.2334994077682495, + -3.4603066444396973, + 1.5447293519973755, + 0.25094738602638245, + 0.7163193821907043, + -0.10547596216201782, + -0.42469942569732666, + -0.49689921736717224, + -2.8712027072906494, + -1.6389087438583374 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027082443237304688, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + -0.03311736881732941, + 1.162198781967163, + 0.0, + 0.043807268142700195, + -0.264798641204834, + 0.0069312527775764465, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + -0.01150114368647337, + -0.007074322551488876, + -0.018292216584086418, + -0.035625845193862915, + 0.05120308697223663, + -2.0730173587799072, + 0.0 + ] + ], + "predicted_scores": [ + 6.755773067474365, + 5.860231876373291, + 5.797102928161621, + 5.742587089538574, + 5.7348408699035645, + 5.577351093292236, + 5.468424320220947, + 5.338929653167725 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1919198188", + "selected_index": 0, + "state_hash": "6b9e4c034c70f8e93116064dbcb48aa5fea37958e04fa30734f597c38aefea8b", + "stored_base_utility": 0.8558897972106934, + "task_id": "StackCube-v1" + } + ], + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "source_index": "data/cil_charts_rgb_refs/train/index.json", + "source_split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_index": "data/cil_charts_rgb_refs/val/index.json", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381" +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..9a0ab644c64eada09def42e8c9e235db26e29b80 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics.json @@ -0,0 +1,3767 @@ +{ + "epsilon": 0.0, + "k": 8, + "mode": "measured", + "num_rows": 69, + "rows": [ + { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 1.2770342230796814, + "chart_id": "ms-484ea653bbca2de3a4a2", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.19966589920839098, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "seed": "1231780997", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7229657769203186, + "selected_utility_gain_over_base_at_8": -0.004040360450744629, + "selector_regret_at_8": 1.2770342230796814, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2770342230796814 + }, + { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 1.281606376171112, + "chart_id": "ms-4b4cd12d70c0615e0930", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.20050368294022264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.471626341342926, + "seed": "1025740212", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7183936238288879, + "selected_utility_gain_over_base_at_8": 0.19001996517181396, + "selector_regret_at_8": 1.281606376171112, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.281606376171112 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2853952050209045, + "chart_id": "ms-6e268d1e2915740816a1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.41171052461838353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1475440973", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7146047949790955, + "selected_utility_gain_over_base_at_8": -1.2853952050209045, + "selector_regret_at_8": 1.2853952050209045, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2853952050209045 + }, + { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2800043225288391, + "chart_id": "ms-8864796e31bbe93d7c5d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3791036296516339, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "seed": "551504635", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7199956774711609, + "selected_utility_gain_over_base_at_8": -0.003976881504058838, + "selector_regret_at_8": 1.2800043225288391, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2800043225288391 + }, + { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.11861386895179749, + "chart_id": "ms-b04b489f4fbad93b65ca", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3774241061869338, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5588552355766296, + "proposal_oracle_utility_gain_over_base_at_8": 0.13654270768165588, + "seed": "1113268940", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.44024136662483215, + "selected_utility_gain_over_base_at_8": 0.0179288387298584, + "selector_regret_at_8": 0.11861386895179749, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4411447644233704, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.5597586333751678 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c663f7e73e7e8376a05c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15436827657008492, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1528740417", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "seed": "1273247072", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "task_id": "PickCube-v1", + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + { + "base_success": 0.0, + "base_utility": 0.050487834960222244, + "branch_car_at_8": 0.16002291440963745, + "chart_id": "ms-0345d8ac530f88cefd61", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1798098385334015, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35253205571641333, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1601850390434265, + "proposal_oracle_utility_gain_over_base_at_8": 0.10969720408320427, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0001621246337890625, + "selected_utility_gain_over_base_at_8": -0.05032571032643318, + "selector_regret_at_8": 0.16002291440963745, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.019624799489974976, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17964771389961243 + }, + { + "base_success": 0.0, + "base_utility": 0.07743936777114868, + "branch_car_at_8": 0.0009531378746032715, + "chart_id": "ms-0819eb89e627595e7e1b", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.14572584629058838, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15574723256687859, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.009828686714172363, + "proposal_oracle_utility_gain_over_base_at_8": -0.06761068105697632, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.008875548839569092, + "selected_utility_gain_over_base_at_8": -0.06856381893157959, + "selector_regret_at_8": 0.0009531378746032715, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13589715957641602, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1368502974510193 + }, + { + "base_success": 0.0, + "base_utility": 0.19400137662887573, + "branch_car_at_8": 0.05573397874832153, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3968084572232568, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0932278037071228, + "proposal_oracle_utility_gain_over_base_at_8": -0.10077357292175293, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03749382495880127, + "selected_utility_gain_over_base_at_8": -0.15650755167007446, + "selector_regret_at_8": 0.05573397874832153, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9067721962928772, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9625061750411987 + }, + { + "base_success": 0.0, + "base_utility": 0.04583283141255379, + "branch_car_at_8": 0.050342779606580734, + "chart_id": "ms-126eb8b36769f9ba4671", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1985912322998047, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1912988221177401, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1050744280219078, + "proposal_oracle_utility_gain_over_base_at_8": 0.05924159660935402, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05473164841532707, + "selected_utility_gain_over_base_at_8": 0.008898817002773285, + "selector_regret_at_8": 0.050342779606580734, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09351680427789688, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.14385958388447762 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-16661b2fd2c85f57a66c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3285899297359119, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008215903653763235, + "chart_id": "ms-20e1efbb745fd02abc95", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0996910344508803, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0015756288776174188, + "proposal_oracle_utility_gain_over_base_at_8": -1.9984243711223826, + "seed": "1375492786", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0007540385122410953, + "selected_utility_gain_over_base_at_8": -1.999245961487759, + "selector_regret_at_8": 0.0008215903653763235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24187534058000892, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24269693094538525 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-2dc5138251afd26a040d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565621826", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.08252689987421036, + "branch_car_at_8": 0.008593454957008362, + "chart_id": "ms-387e6771a55be98f1a59", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20744429528713226, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.22806100330619278, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10834071785211563, + "proposal_oracle_utility_gain_over_base_at_8": 0.025813817977905273, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09974726289510727, + "selected_utility_gain_over_base_at_8": 0.01722036302089691, + "selector_regret_at_8": 0.008593454957008362, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09910357743501663, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.107697032392025 + }, + { + "base_success": 0.0, + "base_utility": 0.03917040303349495, + "branch_car_at_8": 0.0, + "chart_id": "ms-38f7298184a2c5005e94", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1767096221446991, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.25037633004492843, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.12086125463247299, + "proposal_oracle_utility_gain_over_base_at_8": 0.08169085159897804, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12086125463247299, + "selected_utility_gain_over_base_at_8": 0.08169085159897804, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.055848367512226105, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.055848367512226105 + }, + { + "base_success": 0.0, + "base_utility": 0.009124159812927246, + "branch_car_at_8": 0.002313693054020405, + "chart_id": "ms-482c5c8d208b5907a2c2", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1834145486354828, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09740165004185923, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.011097331531345844, + "proposal_oracle_utility_gain_over_base_at_8": 0.001973171718418598, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.00878363847732544, + "selected_utility_gain_over_base_at_8": -0.00034052133560180664, + "selector_regret_at_8": 0.002313693054020405, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.17231721710413694, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17463091015815735 + }, + { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.02001962810754776, + "chart_id": "ms-4da03300d92420e80ed0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11661420436902857, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1273784190416336, + "proposal_oracle_utility_gain_over_base_at_8": -0.019252806901931763, + "seed": "1747178030", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10735879093408585, + "selected_utility_gain_over_base_at_8": -0.03927243500947952, + "selector_regret_at_8": 0.02001962810754776, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8726215809583664, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8926412090659142 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-57be9ae181c077e7821f", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.17875755926385148, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.1544450968503952, + "branch_car_at_8": 1.937544982880354, + "chart_id": "ms-6388b5d2356c556c3e55", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07909133535710673, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8455549031496048, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06245501711964607, + "selected_utility_gain_over_base_at_8": -0.09199007973074913, + "selector_regret_at_8": 1.937544982880354, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.937544982880354 + }, + { + "base_success": 0.0, + "base_utility": 0.06173701956868172, + "branch_car_at_8": 0.05367838591337204, + "chart_id": "ms-638d2ba1917eff4d09e0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.11176212877035141, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.29235072384279687, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.15528590977191925, + "proposal_oracle_utility_gain_over_base_at_8": 0.09354889020323753, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10160752385854721, + "selected_utility_gain_over_base_at_8": 0.039870504289865494, + "selector_regret_at_8": 0.05367838591337204, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0101546049118042 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9834009613841772, + "chart_id": "ms-652ad552ba98d2736620", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2367449700832367, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.40752231020077345, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.016599038615822792, + "selected_utility_gain_over_base_at_8": -1.9834009613841772, + "selector_regret_at_8": 1.9834009613841772, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2201459314674139 + }, + { + "base_success": 0.0, + "base_utility": 0.03784912824630737, + "branch_car_at_8": 0.0, + "chart_id": "ms-674fad0e7c042d23629e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.15573063492774963, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.18759978271166325, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10381317138671875, + "proposal_oracle_utility_gain_over_base_at_8": 0.06596404314041138, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10381317138671875, + "selected_utility_gain_over_base_at_8": 0.06596404314041138, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.051917463541030884, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.051917463541030884 + }, + { + "base_success": 0.0, + "base_utility": 0.12791956961154938, + "branch_car_at_8": 0.0, + "chart_id": "ms-728b865677134d3788e3", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.22658847272396088, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2562046743306349, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.13394887745380402, + "proposal_oracle_utility_gain_over_base_at_8": 0.006029307842254639, + "seed": "750448296", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.13394887745380402, + "selected_utility_gain_over_base_at_8": 0.006029307842254639, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09263959527015686, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.09263959527015686 + }, + { + "base_success": 0.0, + "base_utility": 0.20529556274414062, + "branch_car_at_8": 0.09963546693325043, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.27989473938941956, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11890836225710137, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.20188765227794647, + "proposal_oracle_utility_gain_over_base_at_8": -0.003407910466194153, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10225218534469604, + "selected_utility_gain_over_base_at_8": -0.10304337739944458, + "selector_regret_at_8": 0.09963546693325043, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.07800708711147308, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1776425540447235 + }, + { + "base_success": 0.0, + "base_utility": 0.18213583528995514, + "branch_car_at_8": 0.000459273811429739, + "chart_id": "ms-7f57045c114ddad153b5", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.227622389793396, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09636414360460299, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0052528977394104, + "proposal_oracle_utility_gain_over_base_at_8": -0.17688293755054474, + "seed": "278008524", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.004793623927980661, + "selected_utility_gain_over_base_at_8": -0.17734221136197448, + "selector_regret_at_8": 0.000459273811429739, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2223694920539856, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22282876586541533 + }, + { + "base_success": 0.0, + "base_utility": 0.0056487321853637695, + "branch_car_at_8": 0.014695842750370502, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1631626933813095, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5338617369066084, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.025841474533081055, + "proposal_oracle_utility_gain_over_base_at_8": 0.020192742347717285, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011145631782710552, + "selected_utility_gain_over_base_at_8": 0.005496899597346783, + "selector_regret_at_8": 0.014695842750370502, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13732121884822845, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.15201706159859896 + }, + { + "base_success": 0.0, + "base_utility": 0.00698717450723052, + "branch_car_at_8": 0.08160265535116196, + "chart_id": "ms-8a72b73ac452bc190aeb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.10329639911651611, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.18502306719781839, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.14832614362239838, + "proposal_oracle_utility_gain_over_base_at_8": 0.14133896911516786, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06672348827123642, + "selected_utility_gain_over_base_at_8": 0.0597363137640059, + "selector_regret_at_8": 0.08160265535116196, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.036572910845279694 + }, + { + "base_success": 0.0, + "base_utility": 0.08285123109817505, + "branch_car_at_8": 0.011325618252158165, + "chart_id": "ms-a96bca1ef0b74f686018", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1572066992521286, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22880234105797714, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.03157500550150871, + "proposal_oracle_utility_gain_over_base_at_8": -0.051276225596666336, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.020249387249350548, + "selected_utility_gain_over_base_at_8": -0.0626018438488245, + "selector_regret_at_8": 0.011325618252158165, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1256316937506199, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13695731200277805 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-ac4fb33f3ab4db853995", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13163114441189208, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13897453248500824, + "proposal_oracle_utility_gain_over_base_at_8": -1.8610254675149918, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13897453248500824, + "selected_utility_gain_over_base_at_8": -1.8610254675149918, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8610254675149918, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8610254675149918 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "2054899126", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.037537217140197754, + "branch_car_at_8": 0.0031203627586364746, + "chart_id": "ms-c832439976a37f52d094", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2594742774963379, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.4058728111694587, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.007275819778442383, + "proposal_oracle_utility_gain_over_base_at_8": -0.03026139736175537, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.004155457019805908, + "selected_utility_gain_over_base_at_8": -0.033381760120391846, + "selector_regret_at_8": 0.0031203627586364746, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2521984577178955, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.255318820476532 + }, + { + "base_success": 0.0, + "base_utility": 0.09138330072164536, + "branch_car_at_8": 0.010135690681636333, + "chart_id": "ms-c96b415f083f60857334", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.26426148414611816, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2433265403791355, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0225119199603796, + "proposal_oracle_utility_gain_over_base_at_8": -0.06887138076126575, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.012376229278743267, + "selected_utility_gain_over_base_at_8": -0.07900707144290209, + "selector_regret_at_8": 0.010135690681636333, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24174956418573856, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2518852548673749 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18033001516380143, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "534717141", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.14809735119342804, + "branch_car_at_8": 0.0007792116375640035, + "chart_id": "ms-da142025733b53fcf7eb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2643269896507263, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3120068884872385, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0020139615517109632, + "proposal_oracle_utility_gain_over_base_at_8": -0.14608338964171708, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0012347499141469598, + "selected_utility_gain_over_base_at_8": -0.14686260127928108, + "selector_regret_at_8": 0.0007792116375640035, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.26231302809901536, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.26309223973657936 + }, + { + "base_success": 0.0, + "base_utility": 0.1362423449754715, + "branch_car_at_8": 0.0032713431864976883, + "chart_id": "ms-e2bad6e99e8a281207cc", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20825156569480896, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20447116827433545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.03361199423670769, + "proposal_oracle_utility_gain_over_base_at_8": -0.10263035073876381, + "seed": "1549324805", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03034065105021, + "selected_utility_gain_over_base_at_8": -0.1059016939252615, + "selector_regret_at_8": 0.0032713431864976883, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.17463957145810127, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17791091464459896 + }, + { + "base_success": 0.0, + "base_utility": 0.09695577621459961, + "branch_car_at_8": 0.038976866751909256, + "chart_id": "ms-e30fd80adc089e9f697a", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18920521438121796, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3409869835510214, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07222036272287369, + "proposal_oracle_utility_gain_over_base_at_8": -0.02473541349172592, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03324349597096443, + "selected_utility_gain_over_base_at_8": -0.06371228024363518, + "selector_regret_at_8": 0.038976866751909256, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11698485165834427, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.15596171841025352 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.036640286445617676, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.17097233660673458, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13109225034713745, + "proposal_oracle_utility_gain_over_base_at_8": -1.8689077496528625, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.09445196390151978, + "selected_utility_gain_over_base_at_8": -1.9055480360984802, + "selector_regret_at_8": 0.036640286445617676, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8689077496528625, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9055480360984802 + }, + { + "base_success": 0.0, + "base_utility": 0.09514755010604858, + "branch_car_at_8": 0.03174654766917229, + "chart_id": "ms-ec013396e4aff6170ee1", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24502670764923096, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5028853351950527, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08578010648488998, + "proposal_oracle_utility_gain_over_base_at_8": -0.0093674436211586, + "seed": "750448296", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0540335588157177, + "selected_utility_gain_over_base_at_8": -0.04111399129033089, + "selector_regret_at_8": 0.03174654766917229, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15924660116434097, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.19099314883351326 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0048203859478235245, + "chart_id": "ms-ee30ebaf98622759d888", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19331728269593437, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.024462858214974403, + "proposal_oracle_utility_gain_over_base_at_8": -1.9755371417850256, + "seed": "636680137", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.01964247226715088, + "selected_utility_gain_over_base_at_8": -1.9803575277328491, + "selector_regret_at_8": 0.0048203859478235245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755371417850256, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9803575277328491 + }, + { + "base_success": 0.0, + "base_utility": 0.026694953441619873, + "branch_car_at_8": 0.01985256001353264, + "chart_id": "ms-f3f1d07cfd59b823da47", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.17124073207378387, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4586111091834293, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07150205224752426, + "proposal_oracle_utility_gain_over_base_at_8": 0.04480709880590439, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05164949223399162, + "selected_utility_gain_over_base_at_8": 0.02495453879237175, + "selector_regret_at_8": 0.01985256001353264, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09973867982625961, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11959123983979225 + }, + { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 0.008425161242485046, + "chart_id": "ms-081952a63087fc8f83cd", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07518554002658885, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08595690131187439, + "proposal_oracle_utility_gain_over_base_at_8": 0.01514965295791626, + "seed": "2141802816", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07753174006938934, + "selected_utility_gain_over_base_at_8": 0.006724491715431213, + "selector_regret_at_8": 0.008425161242485046, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9140430986881256, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9224682599306107 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-121bc2e6676c1ac10911", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "77147264", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-22f153f0d4fa656018e6", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1271718007", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.002386912703514099, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20266639463366093, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.027498364448547363, + "proposal_oracle_utility_gain_over_base_at_8": -0.004528626799583435, + "seed": "1881223807", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.025111451745033264, + "selected_utility_gain_over_base_at_8": -0.006915539503097534, + "selector_regret_at_8": 0.002386912703514099, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9725016355514526, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9748885482549667 + }, + { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "chart_id": "ms-2867deaca7e98582620b", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1340247769860469, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "seed": "1189392917", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 1.9452667087316513, + "chart_id": "ms-365d8c6de6de16f66744", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3641822422309461, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "seed": "1997090716", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.054733291268348694, + "selected_utility_gain_over_base_at_8": -0.08454149961471558, + "selector_regret_at_8": 1.9452667087316513, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9452667087316513 + }, + { + "base_success": 0.0, + "base_utility": 0.03280790150165558, + "branch_car_at_8": 0.0012761056423187256, + "chart_id": "ms-3c326d05fec66e1d6bea", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11595738961017374, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.021766602993011475, + "proposal_oracle_utility_gain_over_base_at_8": -0.011041298508644104, + "seed": "278008524", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02049049735069275, + "selected_utility_gain_over_base_at_8": -0.01231740415096283, + "selector_regret_at_8": 0.0012761056423187256, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9782333970069885, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9795095026493073 + }, + { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3405496231635778, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "seed": "794181626", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-58950fed04d203b97ab9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026017268580811406, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1113371261", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 1.8899840712547302, + "chart_id": "ms-651bc0395d78d2819158", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35034389349653194, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "seed": "1483645336", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11001592874526978, + "selected_utility_gain_over_base_at_8": 0.09359987080097198, + "selector_regret_at_8": 1.8899840712547302, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.8899840712547302 + }, + { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9300042241811752, + "chart_id": "ms-9fad82fd288e4296e354", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1246594058194818, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "seed": "1176943004", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06999577581882477, + "selected_utility_gain_over_base_at_8": -0.08461795747280121, + "selector_regret_at_8": 1.9300042241811752, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9300042241811752 + }, + { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0010883957147598267, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1991858618764328, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09276536107063293, + "proposal_oracle_utility_gain_over_base_at_8": 0.024948790669441223, + "seed": "998761162", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09167696535587311, + "selected_utility_gain_over_base_at_8": 0.023860394954681396, + "selector_regret_at_8": 0.0010883957147598267, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.907234638929367, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.908323034644127 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.017522692680358887, + "chart_id": "ms-fc8084d5cd7da937fc02", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09717880195623664, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13679209351539612, + "proposal_oracle_utility_gain_over_base_at_8": -1.8632079064846039, + "seed": "1991414747", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11926940083503723, + "selected_utility_gain_over_base_at_8": -1.8807305991649628, + "selector_regret_at_8": 0.017522692680358887, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8632079064846039, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.8807305991649628 + }, + { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.003223627805709839, + "chart_id": "ms-3afd8959363e917591d4", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.13808834192306887, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2283252328634262, + "proposal_oracle_utility_gain_over_base_at_8": 0.025573596358299255, + "seed": "75070641", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.22510160505771637, + "selected_utility_gain_over_base_at_8": 0.022349968552589417, + "selector_regret_at_8": 0.003223627805709839, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7716747671365738, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7748983949422836 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.828233242034912, + "chart_id": "ms-3ff1aedb72f65a182b58", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30857397468613657, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1055567618", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1717667579650879, + "selected_utility_gain_over_base_at_8": -1.828233242034912, + "selector_regret_at_8": 1.828233242034912, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.828233242034912 + }, + { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.4498589485883713, + "chart_id": "ms-4048f8f6e047422e3c79", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.43030250381223584, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5705926418304443, + "proposal_oracle_utility_gain_over_base_at_8": 0.37410856783390045, + "seed": "1055157354", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12073369324207306, + "selected_utility_gain_over_base_at_8": -0.07575038075447083, + "selector_regret_at_8": 0.4498589485883713, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4294073581695557, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.879266306757927 + }, + { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.030991941690444946, + "chart_id": "ms-5b26c33feed6992386b1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.31643520896589616, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21208032965660095, + "proposal_oracle_utility_gain_over_base_at_8": 0.01363721489906311, + "seed": "1522272686", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.181088387966156, + "selected_utility_gain_over_base_at_8": -0.017354726791381836, + "selector_regret_at_8": 0.030991941690444946, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.787919670343399, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.818911612033844 + }, + { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "chart_id": "ms-63ac245579b1188935ad", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14834095177221712, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8135772347450256, + "proposal_oracle_utility_gain_over_base_at_8": -0.0252571702003479, + "seed": "211245842", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8135772347450256, + "selected_utility_gain_over_base_at_8": -0.0252571702003479, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1864227652549744, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1864227652549744 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1665739511", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.44993481040000916, + "chart_id": "ms-72777024349c2aacd5e0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18757436508611464, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.574478030204773, + "proposal_oracle_utility_gain_over_base_at_8": -1.425521969795227, + "seed": "256770874", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1245432198047638, + "selected_utility_gain_over_base_at_8": -1.8754567801952362, + "selector_regret_at_8": 0.44993481040000916, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.425521969795227, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8754567801952362 + }, + { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3607921451330185, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2493251178612577, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5799027681350708, + "proposal_oracle_utility_gain_over_base_at_8": 0.37618090212345123, + "seed": "957101489", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2191106230020523, + "selected_utility_gain_over_base_at_8": 0.01538875699043274, + "selector_regret_at_8": 0.3607921451330185, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4200972318649292, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7808893769979477 + }, + { + "base_success": 0.0, + "base_utility": 0.8431589603424072, + "branch_car_at_8": 0.026760950684547424, + "chart_id": "ms-9a14552b7db695dee92c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.4512356830429367, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22642898559570312, + "proposal_oracle_utility_gain_over_base_at_8": -0.6167299747467041, + "seed": "77147264", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1996680349111557, + "selected_utility_gain_over_base_at_8": -0.6434909254312515, + "selector_regret_at_8": 0.026760950684547424, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7735710144042969, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8003319650888443 + }, + { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.8556141257286072, + "chart_id": "ms-9f5d57fe2153fd267774", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3951037237909243, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "seed": "1607773860", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14438587427139282, + "selected_utility_gain_over_base_at_8": -0.6882869601249695, + "selector_regret_at_8": 1.8556141257286072, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8556141257286072 + }, + { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.6354914605617523, + "chart_id": "ms-b0fee26e1db967f90b81", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.28863435648886215, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8147144317626953, + "proposal_oracle_utility_gain_over_base_at_8": -0.012500584125518799, + "seed": "1358421325", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.179222971200943, + "selected_utility_gain_over_base_at_8": -0.6479920446872711, + "selector_regret_at_8": 0.6354914605617523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1852855682373047, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.820777028799057 + }, + { + "base_success": 0.0, + "base_utility": 0.8307622671127319, + "branch_car_at_8": 1.8165037035942078, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35878451162224945, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.169237732887268, + "seed": "1271718007", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.18349629640579224, + "selected_utility_gain_over_base_at_8": -0.6472659707069397, + "selector_regret_at_8": 1.8165037035942078, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8165037035942078 + }, + { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.3453921228647232, + "chart_id": "ms-d1960368ef76bbb10669", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1517874168346938, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.581784188747406, + "proposal_oracle_utility_gain_over_base_at_8": 0.38145777583122253, + "seed": "416183106", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2363920658826828, + "selected_utility_gain_over_base_at_8": 0.03606565296649933, + "selector_regret_at_8": 0.3453921228647232, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.418215811252594, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7636079341173172 + }, + { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.1455947458744049, + "chart_id": "ms-efdc07c985935d871adf", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2692917587355154, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21975377202033997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6153350174427032, + "seed": "706479600", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07415902614593506, + "selected_utility_gain_over_base_at_8": -0.7609297633171082, + "selector_regret_at_8": 0.1455947458744049, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.78024622797966, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.925840973854065 + }, + { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.021911710500717163, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09083822075370537, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09917314350605011, + "proposal_oracle_utility_gain_over_base_at_8": 0.06197427213191986, + "seed": "1919198188", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07726143300533295, + "selected_utility_gain_over_base_at_8": 0.0400625616312027, + "selector_regret_at_8": 0.021911710500717163, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.90082685649395, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.922738566994667 + } + ], + "summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.46099290780141844, + "low": 0.19680851063829788, + "mean": 0.3085106382978724, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 1.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3176470588235294, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.4057971014492754, + "low": 0.18840579710144928, + "mean": 0.2753623188405797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.0808477015729916, + "low": 0.6013251688004412, + "mean": 0.8163502178883101, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8163502178883101, + "per_seed": { + "1025740212": 0.528373658657074, + "1055157354": 0.19648407399654388, + "1055567618": 2.0, + "1113268940": 0.42231252789497375, + "1113371261": 2.0, + "1176943004": 0.15461373329162598, + "1189392917": 0.10841864347457886, + "1231780997": 0.7270061373710632, + "1271718007": 1.415381133556366, + "1273247072": 0.15342965722084045, + "1358421325": 0.8272150158882141, + "1375492786": 2.0, + "141984615": 0.09381213908394177, + "1475440973": 2.0, + "1483645336": 0.01641605794429779, + "1522272686": 0.19844311475753784, + "1528740417": 2.0, + "1549324805": 1.3787474483251572, + "1607773860": 0.8326728343963623, + "1665739511": 2.0, + "1747178030": 0.14663122594356537, + "1881223807": 0.0320269912481308, + "1919198188": 0.03719887137413025, + "1991414747": 2.0, + "1997090716": 0.13927479088306427, + "2015956957": 0.06711978986859321, + "2054899126": 2.0, + "211245842": 0.8388344049453735, + "2141802816": 0.07080724835395813, + "256770874": 2.0, + "278008524": 0.10747186839580536, + "416183106": 0.20032641291618347, + "534717141": 2.0, + "551504635": 0.7239725589752197, + "565369392": 0.7997656464576721, + "565621826": 2.0, + "585904130": 0.07529209802548091, + "626486813": 0.5054400166263804, + "636680137": 2.0, + "706479600": 0.8350887894630432, + "750448296": 0.11153355985879898, + "75070641": 0.20275163650512695, + "767747424": 0.5475241877138615, + "77147264": 1.4215794801712036, + "794181626": 0.18095605075359344, + "957101489": 0.20372186601161957, + "998761162": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9058472741949395, + "low": 0.4133020142714182, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.923898630168127, + "low": 0.5448560892550302, + "mean": 0.7180729379567007, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6087407858949815, + "low": 0.243578380005804, + "mean": 0.4175693826273526, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4175693826273526, + "per_seed": { + "1025740212": 1.281606376171112, + "1055157354": 0.4498589485883713, + "1055567618": 1.828233242034912, + "1113268940": 0.11861386895179749, + "1113371261": 0.0, + "1176943004": 1.9300042241811752, + "1189392917": 0.0, + "1231780997": 1.2770342230796814, + "1271718007": 0.9082518517971039, + "1273247072": 0.42020074278116226, + "1358421325": 0.6354914605617523, + "1375492786": 0.0008215903653763235, + "141984615": 0.05737591476645321, + "1475440973": 1.2853952050209045, + "1483645336": 1.8899840712547302, + "1522272686": 0.030991941690444946, + "1528740417": 0.0, + "1549324805": 0.0010904477288325627, + "1607773860": 1.8556141257286072, + "1665739511": 0.0, + "1747178030": 0.02001962810754776, + "1881223807": 0.002386912703514099, + "1919198188": 0.021911710500717163, + "1991414747": 0.017522692680358887, + "1997090716": 1.9452667087316513, + "2015956957": 0.39167013615369795, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.008425161242485046, + "256770874": 0.44993481040000916, + "278008524": 0.0008676897268742323, + "416183106": 0.3453921228647232, + "534717141": 0.0, + "551504635": 1.2800043225288391, + "565369392": 0.05178981522719065, + "565621826": 0.0, + "585904130": 0.01741097339739402, + "626486813": 0.033813119400292635, + "636680137": 0.0048203859478235245, + "706479600": 0.1455947458744049, + "750448296": 0.015873273834586143, + "75070641": 0.003223627805709839, + "767747424": 0.5240038954652846, + "77147264": 0.013380475342273712, + "794181626": 0.0, + "957101489": 0.3607921451330185, + "998761162": 0.0010883957147598267 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6577742758489845, + "low": 0.27715372847280956, + "mean": 0.4814726852855894, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4814726852855894, + "per_task": { + "LiftPegUpright-v1": 0.8737756659587225, + "PickCube-v1": 0.42020074278116226, + "PullCube-v1": 0.13619081233799293, + "PushCube-v1": 0.4458426363193072, + "StackCube-v1": 0.5313535690307617 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4942535392575614, + "low": 0.19425320601178522, + "mean": 0.3486898574730098, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9148936170212765, + "low": 0.7432624113475178, + "mean": 0.8358156028368794, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8358156028368794, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.782608695652174, + "low": 0.5507246376811594, + "mean": 0.6666666666666666, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.850041654515774, + "low": 1.5397213380149701, + "mean": 1.7062028111638965, + "n": 47, + "num_samples": 200 + }, + "mean": 1.7062028111638965, + "per_seed": { + "1025740212": 2.0, + "1055157354": 2.0, + "1055567618": 2.0, + "1113268940": 2.0, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 2.0, + "1358421325": 2.0, + "1375492786": 0.24345096945762634, + "141984615": 0.20044784247875214, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 2.0, + "1528740417": 2.0, + "1549324805": 1.4027505218982697, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 2.0, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.5298813670873642, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 2.0, + "278008524": 1.113811194896698, + "416183106": 2.0, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 1.4266315797964733, + "565621826": 2.0, + "585904130": 0.23764699200789133, + "626486813": 0.6124684102833271, + "636680137": 2.0, + "706479600": 2.0, + "750448296": 0.23580759018659592, + "75070641": 2.0, + "767747424": 0.18863565661013126, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 2.0, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 200 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.6080652458080347, + "low": 1.1926357444861662, + "mean": 1.3999846732054932, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6170212765957447, + "low": 0.3319148936170213, + "mean": 0.48156028368794324, + "n": 47, + "num_samples": 200 + }, + "mean": 0.48156028368794324, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.3333333333333333, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.8, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.75, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.5, + "75070641": 1.0, + "767747424": 0.75, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8143589743589743, + "low": 0.45176470588235296, + "mean": 0.6182805429864253, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6182805429864253, + "per_task": { + "LiftPegUpright-v1": 0.6666666666666666, + "PickCube-v1": 1.0, + "PullCube-v1": 0.35294117647058826, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.5333333333333333 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5797101449275363, + "low": 0.34782608695652173, + "mean": 0.463768115942029, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.24376288810786303, + "low": 0.18434974480601804, + "mean": 0.21479075208855025, + "n": 47, + "num_samples": 200 + }, + "mean": 0.21479075208855025, + "per_seed": { + "1025740212": 0.20050368294022264, + "1055157354": 0.43030250381223584, + "1055567618": 0.30857397468613657, + "1113268940": 0.3774241061869338, + "1113371261": 0.026017268580811406, + "1176943004": 0.1246594058194818, + "1189392917": 0.1340247769860469, + "1231780997": 0.19966589920839098, + "1271718007": 0.17939225581112472, + "1273247072": 0.16962492308566807, + "1358421325": 0.28863435648886215, + "1375492786": 0.0996910344508803, + "141984615": 0.297780428420543, + "1475440973": 0.41171052461838353, + "1483645336": 0.35034389349653194, + "1522272686": 0.31643520896589616, + "1528740417": 0.15436827657008492, + "1549324805": 0.23727288575803293, + "1607773860": 0.3951037237909243, + "1665739511": 0.0, + "1747178030": 0.11661420436902857, + "1881223807": 0.20266639463366093, + "1919198188": 0.09083822075370537, + "1991414747": 0.09717880195623664, + "1997090716": 0.3641822422309461, + "2015956957": 0.22628515797280127, + "2054899126": 0.0, + "211245842": 0.14834095177221712, + "2141802816": 0.07518554002658885, + "256770874": 0.18757436508611464, + "278008524": 0.10616076660738837, + "416183106": 0.1517874168346938, + "534717141": 0.18033001516380143, + "551504635": 0.3791036296516339, + "565369392": 0.21578265463075008, + "565621826": 0.0, + "585904130": 0.3300621116998719, + "626486813": 0.24681469768825515, + "636680137": 0.19331728269593437, + "706479600": 0.2692917587355154, + "750448296": 0.3795450047628438, + "75070641": 0.13808834192306887, + "767747424": 0.2798082148668758, + "77147264": 0.22561784152146835, + "794181626": 0.3405496231635778, + "957101489": 0.2493251178612577, + "998761162": 0.1991858618764328 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.26552488687547154, + "low": 0.18076755457919055, + "mean": 0.21966205964357566, + "n": 5, + "num_samples": 200 + }, + "mean": 0.21966205964357566, + "per_task": { + "LiftPegUpright-v1": 0.28712935319594163, + "PickCube-v1": 0.16962492308566807, + "PullCube-v1": 0.23311818739476642, + "PushCube-v1": 0.15615009218311454, + "StackCube-v1": 0.25228774235838763 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.2555647418249429, + "low": 0.19558072798702378, + "mean": 0.22656067749912578, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5304964539007092, + "low": 0.2815602836879432, + "mean": 0.4173758865248227, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4173758865248227, + "per_seed": { + "1025740212": 1.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6230769230769232, + "low": 0.10666666666666666, + "mean": 0.38425339366515837, + "n": 5, + "num_samples": 200 + }, + "mean": 0.38425339366515837, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 0.0, + "PullCube-v1": 0.20588235294117646, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4927536231884058, + "low": 0.21739130434782608, + "mean": 0.34782608695652173, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.23829787234042552, + "low": -0.04539007092198581, + "mean": 0.10886524822695036, + "n": 47, + "num_samples": 200 + }, + "mean": 0.10886524822695036, + "per_seed": { + "1025740212": 1.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3438914027149321, + "low": -0.00862745098039216, + "mean": 0.15722473604826545, + "n": 5, + "num_samples": 200 + }, + "mean": 0.15722473604826545, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 0.0, + "PullCube-v1": -0.08823529411764706, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.06666666666666667 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.17391304347826086, + "low": -0.07246376811594203, + "mean": 0.07246376811594203, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.1794423699735643, + "low": 0.7291675245113577, + "mean": 0.9823928705606516, + "n": 47, + "num_samples": 200 + }, + "mean": 0.9823928705606516, + "per_seed": { + "1025740212": 2.0, + "1055157354": 0.5705926418304443, + "1055567618": 2.0, + "1113268940": 0.5588552355766296, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 0.4659159183502197, + "1358421325": 0.8147144317626953, + "1375492786": 0.0015756288776174188, + "141984615": 0.06459133536554873, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 0.21208032965660095, + "1528740417": 2.0, + "1549324805": 1.344537331412236, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 0.1273784190416336, + "1881223807": 0.027498364448547363, + "1919198188": 0.09917314350605011, + "1991414747": 0.13679209351539612, + "1997090716": 2.0, + "2015956957": 0.46120103299617765, + "2054899126": 2.0, + "211245842": 0.8135772347450256, + "2141802816": 0.08595690131187439, + "256770874": 0.574478030204773, + "278008524": 0.013509750366210938, + "416183106": 0.581784188747406, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 0.14469666282335916, + "565621826": 2.0, + "585904130": 0.034002700820565224, + "626486813": 0.07908930000849068, + "636680137": 0.024462858214974403, + "706479600": 0.21975377202033997, + "750448296": 0.109864491969347, + "75070641": 0.2283252328634262, + "767747424": 0.5921752639114857, + "77147264": 1.1132144927978516, + "794181626": 2.0, + "957101489": 0.5799027681350708, + "998761162": 0.09276536107063293 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.400834542971391, + "low": 0.5501597128059331, + "mean": 0.9640880215032903, + "n": 5, + "num_samples": 200 + }, + "mean": 0.9640880215032903, + "per_task": { + "LiftPegUpright-v1": 1.7598092059294383, + "PickCube-v1": 0.4659159183502197, + "PullCube-v1": 0.47449842058053676, + "PushCube-v1": 1.2588291787184203, + "StackCube-v1": 0.8613873839378356 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.0680866769491357, + "low": 0.5915389626904431, + "mean": 0.8180189497981464, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.4120927946805518, + "low": -0.10385253924549498, + "mean": 0.16604265267234164, + "n": 47, + "num_samples": 200 + }, + "mean": 0.16604265267234164, + "per_seed": { + "1025740212": 1.471626341342926, + "1055157354": 0.37410856783390045, + "1055567618": 0.0, + "1113268940": 0.13654270768165588, + "1113371261": 0.0, + "1176943004": 1.845386266708374, + "1189392917": 1.8915813565254211, + "1231780997": 1.2729938626289368, + "1271718007": 0.584618866443634, + "1273247072": 0.3124862611293793, + "1358421325": -0.012500584125518799, + "1375492786": -1.9984243711223826, + "141984615": -0.029220803718393046, + "1475440973": 0.0, + "1483645336": 1.9835839420557022, + "1522272686": 0.01363721489906311, + "1528740417": 0.0, + "1549324805": -0.03421011691292127, + "1607773860": 1.1673271656036377, + "1665739511": 0.0, + "1747178030": -0.019252806901931763, + "1881223807": -0.004528626799583435, + "1919198188": 0.06197427213191986, + "1991414747": -1.8632079064846039, + "1997090716": 1.8607252091169357, + "2015956957": 0.39408124312758447, + "2054899126": 0.0, + "211245842": -0.0252571702003479, + "2141802816": 0.01514965295791626, + "256770874": -1.425521969795227, + "278008524": -0.09396211802959442, + "416183106": 0.38145777583122253, + "534717141": 0.0, + "551504635": 1.2760274410247803, + "565369392": -0.655068983634313, + "565621826": 0.0, + "585904130": -0.04128939720491568, + "626486813": -0.4263507166178897, + "636680137": -1.9755371417850256, + "706479600": -0.6153350174427032, + "750448296": -0.0016690678894519806, + "75070641": 0.025573596358299255, + "767747424": 0.04465107619762421, + "77147264": -0.30836498737335205, + "794181626": 1.8190439492464066, + "957101489": 0.37618090212345123, + "998761162": 0.024948790669441223 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.5184734715865209, + "low": 0.015111566525701359, + "mean": 0.2937103647546145, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2937103647546145, + "per_task": { + "LiftPegUpright-v1": 0.6928650587797165, + "PickCube-v1": 0.3124862611293793, + "PullCube-v1": -0.17674104834098697, + "PushCube-v1": 0.5816647181144128, + "StackCube-v1": 0.05827683409055074 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2862129747003749, + "low": -0.1435656816482652, + "mean": 0.09994601184144562, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.30851063829787234, + "low": 0.10992907801418439, + "mean": 0.20567375886524825, + "n": 47, + "num_samples": 200 + }, + "mean": 0.20567375886524825, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.2774358974358974, + "low": 0.04666666666666667, + "mean": 0.15300150829562592, + "n": 5, + "num_samples": 200 + }, + "mean": 0.15300150829562592, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.38461538461538464, + "StackCube-v1": 0.06666666666666667 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2898550724637681, + "low": 0.10144927536231885, + "mean": 0.17391304347826086, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.0017730496453900741, + "low": -0.2127659574468085, + "mean": -0.10283687943262411, + "n": 47, + "num_samples": 200 + }, + "mean": -0.10283687943262411, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": -1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.001357466063348417, + "low": -0.156078431372549, + "mean": -0.07402714932126697, + "n": 5, + "num_samples": 200 + }, + "mean": -0.07402714932126697, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.14705882352941177, + "PushCube-v1": 0.07692307692307693, + "StackCube-v1": -0.13333333333333333 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.028985507246376812, + "low": -0.21739130434782608, + "mean": -0.10144927536231885, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.742602269810456, + "low": 0.37777582296799433, + "mean": 0.5648234879332992, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5648234879332992, + "per_seed": { + "1025740212": 0.7183936238288879, + "1055157354": 0.12073369324207306, + "1055567618": 0.1717667579650879, + "1113268940": 0.44024136662483215, + "1113371261": 2.0, + "1176943004": 0.06999577581882477, + "1189392917": 2.0, + "1231780997": 0.7229657769203186, + "1271718007": 1.0917481482028961, + "1273247072": 0.045715175569057465, + "1358421325": 0.179222971200943, + "1375492786": 0.0007540385122410953, + "141984615": 0.007215420599095523, + "1475440973": 0.7146047949790955, + "1483645336": 0.11001592874526978, + "1522272686": 0.181088387966156, + "1528740417": 2.0, + "1549324805": 1.3434468836834033, + "1607773860": 0.14438587427139282, + "1665739511": 2.0, + "1747178030": 0.10735879093408585, + "1881223807": 0.025111451745033264, + "1919198188": 0.07726143300533295, + "1991414747": 0.11926940083503723, + "1997090716": 0.054733291268348694, + "2015956957": 0.0695308968424797, + "2054899126": 2.0, + "211245842": 0.8135772347450256, + "2141802816": 0.07753174006938934, + "256770874": 0.1245432198047638, + "278008524": 0.012642060639336705, + "416183106": 0.2363920658826828, + "534717141": 2.0, + "551504635": 0.7199956774711609, + "565369392": 0.09290684759616852, + "565621826": 2.0, + "585904130": 0.016591727423171203, + "626486813": 0.04527618060819805, + "636680137": 0.01964247226715088, + "706479600": 0.07415902614593506, + "750448296": 0.09399121813476086, + "75070641": 0.22510160505771637, + "767747424": 0.06817136844620109, + "77147264": 1.0998340174555779, + "794181626": 2.0, + "957101489": 0.2191106230020523, + "998761162": 0.09167696535587311 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7326601550821198, + "low": 0.16275214863845203, + "mean": 0.4826153362177009, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4826153362177009, + "per_task": { + "LiftPegUpright-v1": 0.8860335399707159, + "PickCube-v1": 0.045715175569057465, + "PullCube-v1": 0.33830760824254386, + "PushCube-v1": 0.8129865423991129, + "StackCube-v1": 0.33003381490707395 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6777843068475309, + "low": 0.3036627637912465, + "mean": 0.46932909232513653, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": -0.0441184930471994, + "low": -0.4570727952216018, + "mean": -0.251526729955011, + "n": 47, + "num_samples": 200 + }, + "mean": -0.251526729955011, + "per_seed": { + "1025740212": 0.19001996517181396, + "1055157354": -0.07575038075447083, + "1055567618": -1.828233242034912, + "1113268940": 0.0179288387298584, + "1113371261": 0.0, + "1176943004": -0.08461795747280121, + "1189392917": 1.8915813565254211, + "1231780997": -0.004040360450744629, + "1271718007": -0.32363298535346985, + "1273247072": -0.10771448165178299, + "1358421325": -0.6479920446872711, + "1375492786": -1.999245961487759, + "141984615": -0.08659671848484625, + "1475440973": -1.2853952050209045, + "1483645336": 0.09359987080097198, + "1522272686": -0.017354726791381836, + "1528740417": 0.0, + "1549324805": -0.03530056464175383, + "1607773860": -0.6882869601249695, + "1665739511": 0.0, + "1747178030": -0.03927243500947952, + "1881223807": -0.006915539503097534, + "1919198188": 0.0400625616312027, + "1991414747": -1.8807305991649628, + "1997090716": -0.08454149961471558, + "2015956957": 0.0024111069738864898, + "2054899126": 0.0, + "211245842": -0.0252571702003479, + "2141802816": 0.006724491715431213, + "256770874": -1.8754567801952362, + "278008524": -0.09482980775646865, + "416183106": 0.03606565296649933, + "534717141": 0.0, + "551504635": -0.003976881504058838, + "565369392": -0.7068587988615036, + "565621826": 0.0, + "585904130": -0.058700370602309704, + "626486813": -0.46016383601818234, + "636680137": -1.9803575277328491, + "706479600": -0.7609297633171082, + "750448296": -0.017542341724038124, + "75070641": 0.022349968552589417, + "767747424": -0.4793528192676604, + "77147264": -0.32174546271562576, + "794181626": 1.8190439492464066, + "957101489": 0.01538875699043274, + "998761162": 0.023860394954681396 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": -0.017275244494533833, + "low": -0.34496083553122614, + "mean": -0.18776232053097486, + "n": 5, + "num_samples": 200 + }, + "mean": -0.18776232053097486, + "per_task": { + "LiftPegUpright-v1": -0.18091060717900595, + "PickCube-v1": -0.10771448165178299, + "PullCube-v1": -0.31293186067897993, + "PushCube-v1": 0.13582208179510558, + "StackCube-v1": -0.47307673494021096 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.08520316913642961, + "low": -0.4636818296832802, + "mean": -0.24874384563156418, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6087407858949815, + "low": 0.243578380005804, + "mean": 0.4175693826273526, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4175693826273526, + "per_seed": { + "1025740212": 1.281606376171112, + "1055157354": 0.4498589485883713, + "1055567618": 1.828233242034912, + "1113268940": 0.11861386895179749, + "1113371261": 0.0, + "1176943004": 1.9300042241811752, + "1189392917": 0.0, + "1231780997": 1.2770342230796814, + "1271718007": 0.9082518517971039, + "1273247072": 0.42020074278116226, + "1358421325": 0.6354914605617523, + "1375492786": 0.0008215903653763235, + "141984615": 0.05737591476645321, + "1475440973": 1.2853952050209045, + "1483645336": 1.8899840712547302, + "1522272686": 0.030991941690444946, + "1528740417": 0.0, + "1549324805": 0.0010904477288325627, + "1607773860": 1.8556141257286072, + "1665739511": 0.0, + "1747178030": 0.02001962810754776, + "1881223807": 0.002386912703514099, + "1919198188": 0.021911710500717163, + "1991414747": 0.017522692680358887, + "1997090716": 1.9452667087316513, + "2015956957": 0.39167013615369795, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.008425161242485046, + "256770874": 0.44993481040000916, + "278008524": 0.0008676897268742323, + "416183106": 0.3453921228647232, + "534717141": 0.0, + "551504635": 1.2800043225288391, + "565369392": 0.05178981522719065, + "565621826": 0.0, + "585904130": 0.01741097339739402, + "626486813": 0.033813119400292635, + "636680137": 0.0048203859478235245, + "706479600": 0.1455947458744049, + "750448296": 0.015873273834586143, + "75070641": 0.003223627805709839, + "767747424": 0.5240038954652846, + "77147264": 0.013380475342273712, + "794181626": 0.0, + "957101489": 0.3607921451330185, + "998761162": 0.0010883957147598267 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6577742758489845, + "low": 0.27715372847280956, + "mean": 0.4814726852855894, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4814726852855894, + "per_task": { + "LiftPegUpright-v1": 0.8737756659587225, + "PickCube-v1": 0.42020074278116226, + "PullCube-v1": 0.13619081233799293, + "PushCube-v1": 0.4458426363193072, + "StackCube-v1": 0.5313535690307617 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4942535392575614, + "low": 0.19425320601178522, + "mean": 0.3486898574730098, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3382978723404255, + "low": 0.10638297872340426, + "mean": 0.21170212765957444, + "n": 47, + "num_samples": 200 + }, + "mean": 0.21170212765957444, + "per_seed": { + "1025740212": 1.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.45176470588235296, + "low": 0.07529411764705883, + "mean": 0.2312518853695324, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2312518853695324, + "per_task": { + "LiftPegUpright-v1": 0.6666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.058823529411764705, + "PushCube-v1": 0.23076923076923078, + "StackCube-v1": 0.2 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2753623188405797, + "low": 0.08695652173913043, + "mean": 0.17391304347826086, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5656028368794326, + "low": 0.299645390070922, + "mean": 0.4237588652482269, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4237588652482269, + "per_seed": { + "1025740212": 0.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7760784313725491, + "low": 0.2024132730015083, + "mean": 0.48633484162895924, + "n": 5, + "num_samples": 200 + }, + "mean": 0.48633484162895924, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 1.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.38461538461538464, + "StackCube-v1": 0.7333333333333333 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.43478260869565216, + "low": 0.21739130434782608, + "mean": 0.3333333333333333, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7411347517730495, + "low": 0.48191489361702133, + "mean": 0.6301418439716312, + "n": 47, + "num_samples": 200 + }, + "mean": 0.6301418439716312, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9266666666666667, + "low": 0.42895927601809963, + "mean": 0.7117043740573152, + "n": 5, + "num_samples": 200 + }, + "mean": 0.7117043740573152, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.9333333333333333 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6231884057971014, + "low": 0.36231884057971014, + "mean": 0.4927536231884058, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.960610361603016, + "low": 0.5171997663569934, + "mean": 0.7336599861108202, + "n": 47, + "num_samples": 200 + }, + "mean": 0.7336599861108202, + "per_seed": { + "1025740212": 0.0, + "1055157354": 1.4294073581695557, + "1055567618": 0.0, + "1113268940": 1.4411447644233704, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.5340840816497803, + "1358421325": 1.1852855682373047, + "1375492786": 0.24187534058000892, + "141984615": 0.1358565071132034, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.787919670343399, + "1528740417": 0.0, + "1549324805": 0.05821319048603376, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 1.8726215809583664, + "1881223807": 1.9725016355514526, + "1919198188": 1.90082685649395, + "1991414747": 1.8632079064846039, + "1997090716": 0.0, + "2015956957": 0.06868033409118653, + "2054899126": 0.0, + "211245842": 1.1864227652549744, + "2141802816": 1.9140430986881256, + "256770874": 1.425521969795227, + "278008524": 1.100301444530487, + "416183106": 1.418215811252594, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 1.281934916973114, + "565621826": 0.0, + "585904130": 0.2036442911873261, + "626486813": 0.544636546401307, + "636680137": 1.9755371417850256, + "706479600": 1.78024622797966, + "750448296": 0.12594309821724892, + "75070641": 1.7716747671365738, + "767747424": 0.04815509542822838, + "77147264": 0.8867855072021484, + "794181626": 0.0, + "957101489": 1.4200972318649292, + "998761162": 1.907234638929367 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.2206306469485657, + "low": 0.36480631544491765, + "mean": 0.8032693373590819, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8032693373590819, + "per_task": { + "LiftPegUpright-v1": 0.24019079407056174, + "PickCube-v1": 1.5340840816497803, + "PullCube-v1": 0.3622883737313232, + "PushCube-v1": 0.7411708212815799, + "StackCube-v1": 1.1386126160621644 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.7914590279021692, + "low": 0.41281149387224647, + "mean": 0.6088035285584223, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.3427126641637903, + "low": 0.9024325011565566, + "mean": 1.141379323230597, + "n": 47, + "num_samples": 200 + }, + "mean": 1.141379323230597, + "per_seed": { + "1025740212": 1.281606376171112, + "1055157354": 1.879266306757927, + "1055567618": 1.828233242034912, + "1113268940": 1.5597586333751678, + "1113371261": 0.0, + "1176943004": 1.9300042241811752, + "1189392917": 0.0, + "1231780997": 1.2770342230796814, + "1271718007": 0.9082518517971039, + "1273247072": 1.9542848244309425, + "1358421325": 1.820777028799057, + "1375492786": 0.24269693094538525, + "141984615": 0.1932324218796566, + "1475440973": 1.2853952050209045, + "1483645336": 1.8899840712547302, + "1522272686": 1.818911612033844, + "1528740417": 0.0, + "1549324805": 0.05930363821486632, + "1607773860": 1.8556141257286072, + "1665739511": 0.0, + "1747178030": 1.8926412090659142, + "1881223807": 1.9748885482549667, + "1919198188": 1.922738566994667, + "1991414747": 1.8807305991649628, + "1997090716": 1.9452667087316513, + "2015956957": 0.4603504702448845, + "2054899126": 0.0, + "211245842": 1.1864227652549744, + "2141802816": 1.9224682599306107, + "256770874": 1.8754567801952362, + "278008524": 1.1011691342573613, + "416183106": 1.7636079341173172, + "534717141": 0.0, + "551504635": 1.2800043225288391, + "565369392": 1.3337247322003047, + "565621826": 0.0, + "585904130": 0.22105526458472013, + "626486813": 0.5671922296751291, + "636680137": 1.9803575277328491, + "706479600": 1.925840973854065, + "750448296": 0.14181637205183506, + "75070641": 1.7748983949422836, + "767747424": 0.12046428816393018, + "77147264": 0.9001659825444221, + "794181626": 0.0, + "957101489": 1.7808893769979477, + "998761162": 1.908323034644127 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.687103095329725, + "low": 0.8378044656728661, + "mean": 1.273849031142176, + "n": 5, + "num_samples": 200 + }, + "mean": 1.273849031142176, + "per_task": { + "LiftPegUpright-v1": 1.1139664600292842, + "PickCube-v1": 1.9542848244309425, + "PullCube-v1": 0.4440142285568393, + "PushCube-v1": 1.187013457600887, + "StackCube-v1": 1.669966185092926 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 1.1326504737118501, + "low": 0.71822580749142, + "mean": 0.9306555808803566, + "n": 69, + "num_samples": 200 + }, + "n": 69 + } + }, + "thresholds": [ + 0.2, + 0.4 + ] +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics_by_seed.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..971b465d20fdcad4e441a5b853373c0a428f8142 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics_by_seed.json @@ -0,0 +1,1083 @@ +{ + "1025740212": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 1.281606376171112, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.20050368294022264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.471626341342926, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7183936238288879, + "selected_utility_gain_over_base_at_8": 0.19001996517181396, + "selector_regret_at_8": 1.281606376171112, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.281606376171112 + }, + "1055157354": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.4498589485883713, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.43030250381223584, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5705926418304443, + "proposal_oracle_utility_gain_over_base_at_8": 0.37410856783390045, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12073369324207306, + "selected_utility_gain_over_base_at_8": -0.07575038075447083, + "selector_regret_at_8": 0.4498589485883713, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4294073581695557, + "total_car_to_hidden_at_8": 1.879266306757927 + }, + "1055567618": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.828233242034912, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30857397468613657, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1717667579650879, + "selected_utility_gain_over_base_at_8": -1.828233242034912, + "selector_regret_at_8": 1.828233242034912, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.828233242034912 + }, + "1113268940": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.11861386895179749, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3774241061869338, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5588552355766296, + "proposal_oracle_utility_gain_over_base_at_8": 0.13654270768165588, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.44024136662483215, + "selected_utility_gain_over_base_at_8": 0.0179288387298584, + "selector_regret_at_8": 0.11861386895179749, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4411447644233704, + "total_car_to_hidden_at_8": 1.5597586333751678 + }, + "1113371261": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026017268580811406, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9300042241811752, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1246594058194818, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06999577581882477, + "selected_utility_gain_over_base_at_8": -0.08461795747280121, + "selector_regret_at_8": 1.9300042241811752, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9300042241811752 + }, + "1189392917": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1340247769860469, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 1.2770342230796814, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.19966589920839098, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7229657769203186, + "selected_utility_gain_over_base_at_8": -0.004040360450744629, + "selector_regret_at_8": 1.2770342230796814, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2770342230796814 + }, + "1271718007": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9082518517971039, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.17939225581112472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0917481482028961, + "selected_utility_gain_over_base_at_8": -0.32363298535346985, + "selector_regret_at_8": 0.9082518517971039, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9082518517971039 + }, + "1273247072": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + "1358421325": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.6354914605617523, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.28863435648886215, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8147144317626953, + "proposal_oracle_utility_gain_over_base_at_8": -0.012500584125518799, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.179222971200943, + "selected_utility_gain_over_base_at_8": -0.6479920446872711, + "selector_regret_at_8": 0.6354914605617523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1852855682373047, + "total_car_to_hidden_at_8": 1.820777028799057 + }, + "1375492786": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008215903653763235, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0996910344508803, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0015756288776174188, + "proposal_oracle_utility_gain_over_base_at_8": -1.9984243711223826, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0007540385122410953, + "selected_utility_gain_over_base_at_8": -1.999245961487759, + "selector_regret_at_8": 0.0008215903653763235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24187534058000892, + "total_car_to_hidden_at_8": 0.24269693094538525 + }, + "141984615": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.05737591476645321, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.297780428420543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06459133536554873, + "proposal_oracle_utility_gain_over_base_at_8": -0.029220803718393046, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.007215420599095523, + "selected_utility_gain_over_base_at_8": -0.08659671848484625, + "selector_regret_at_8": 0.05737591476645321, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1358565071132034, + "total_car_to_hidden_at_8": 0.1932324218796566 + }, + "1475440973": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2853952050209045, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.41171052461838353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7146047949790955, + "selected_utility_gain_over_base_at_8": -1.2853952050209045, + "selector_regret_at_8": 1.2853952050209045, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2853952050209045 + }, + "1483645336": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 1.8899840712547302, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35034389349653194, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11001592874526978, + "selected_utility_gain_over_base_at_8": 0.09359987080097198, + "selector_regret_at_8": 1.8899840712547302, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8899840712547302 + }, + "1522272686": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.030991941690444946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.31643520896589616, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21208032965660095, + "proposal_oracle_utility_gain_over_base_at_8": 0.01363721489906311, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.181088387966156, + "selected_utility_gain_over_base_at_8": -0.017354726791381836, + "selector_regret_at_8": 0.030991941690444946, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.787919670343399, + "total_car_to_hidden_at_8": 1.818911612033844 + }, + "1528740417": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15436827657008492, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1549324805": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.0010904477288325627, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.23727288575803293, + "proposal_oracle_success_at_8": 0.6666666666666666, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.344537331412236, + "proposal_oracle_utility_gain_over_base_at_8": -0.03421011691292127, + "selected_success_at_8": 0.6666666666666666, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.3434468836834033, + "selected_utility_gain_over_base_at_8": -0.03530056464175383, + "selector_regret_at_8": 0.0010904477288325627, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05821319048603376, + "total_car_to_hidden_at_8": 0.05930363821486632 + }, + "1607773860": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.8556141257286072, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3951037237909243, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14438587427139282, + "selected_utility_gain_over_base_at_8": -0.6882869601249695, + "selector_regret_at_8": 1.8556141257286072, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8556141257286072 + }, + "1665739511": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1747178030": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.02001962810754776, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11661420436902857, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1273784190416336, + "proposal_oracle_utility_gain_over_base_at_8": -0.019252806901931763, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10735879093408585, + "selected_utility_gain_over_base_at_8": -0.03927243500947952, + "selector_regret_at_8": 0.02001962810754776, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8726215809583664, + "total_car_to_hidden_at_8": 1.8926412090659142 + }, + "1881223807": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.002386912703514099, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20266639463366093, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.027498364448547363, + "proposal_oracle_utility_gain_over_base_at_8": -0.004528626799583435, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.025111451745033264, + "selected_utility_gain_over_base_at_8": -0.006915539503097534, + "selector_regret_at_8": 0.002386912703514099, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9725016355514526, + "total_car_to_hidden_at_8": 1.9748885482549667 + }, + "1919198188": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.021911710500717163, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09083822075370537, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09917314350605011, + "proposal_oracle_utility_gain_over_base_at_8": 0.06197427213191986, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07726143300533295, + "selected_utility_gain_over_base_at_8": 0.0400625616312027, + "selector_regret_at_8": 0.021911710500717163, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.90082685649395, + "total_car_to_hidden_at_8": 1.922738566994667 + }, + "1991414747": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.017522692680358887, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09717880195623664, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13679209351539612, + "proposal_oracle_utility_gain_over_base_at_8": -1.8632079064846039, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11926940083503723, + "selected_utility_gain_over_base_at_8": -1.8807305991649628, + "selector_regret_at_8": 0.017522692680358887, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8632079064846039, + "total_car_to_hidden_at_8": 1.8807305991649628 + }, + "1997090716": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 1.9452667087316513, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3641822422309461, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.054733291268348694, + "selected_utility_gain_over_base_at_8": -0.08454149961471558, + "selector_regret_at_8": 1.9452667087316513, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9452667087316513 + }, + "2015956957": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.39167013615369795, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.22628515797280127, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.46120103299617765, + "proposal_oracle_utility_gain_over_base_at_8": 0.39408124312758447, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0695308968424797, + "selected_utility_gain_over_base_at_8": 0.0024111069738864898, + "selector_regret_at_8": 0.39167013615369795, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.06868033409118653, + "total_car_to_hidden_at_8": 0.4603504702448845 + }, + "2054899126": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14834095177221712, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8135772347450256, + "proposal_oracle_utility_gain_over_base_at_8": -0.0252571702003479, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8135772347450256, + "selected_utility_gain_over_base_at_8": -0.0252571702003479, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1864227652549744, + "total_car_to_hidden_at_8": 1.1864227652549744 + }, + "2141802816": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 0.008425161242485046, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07518554002658885, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08595690131187439, + "proposal_oracle_utility_gain_over_base_at_8": 0.01514965295791626, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07753174006938934, + "selected_utility_gain_over_base_at_8": 0.006724491715431213, + "selector_regret_at_8": 0.008425161242485046, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9140430986881256, + "total_car_to_hidden_at_8": 1.9224682599306107 + }, + "256770874": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.44993481040000916, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18757436508611464, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.574478030204773, + "proposal_oracle_utility_gain_over_base_at_8": -1.425521969795227, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1245432198047638, + "selected_utility_gain_over_base_at_8": -1.8754567801952362, + "selector_regret_at_8": 0.44993481040000916, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.425521969795227, + "total_car_to_hidden_at_8": 1.8754567801952362 + }, + "278008524": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0008676897268742323, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10616076660738837, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.013509750366210938, + "proposal_oracle_utility_gain_over_base_at_8": -0.09396211802959442, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.012642060639336705, + "selected_utility_gain_over_base_at_8": -0.09482980775646865, + "selector_regret_at_8": 0.0008676897268742323, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.100301444530487, + "total_car_to_hidden_at_8": 1.1011691342573613 + }, + "416183106": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.3453921228647232, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1517874168346938, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.581784188747406, + "proposal_oracle_utility_gain_over_base_at_8": 0.38145777583122253, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2363920658826828, + "selected_utility_gain_over_base_at_8": 0.03606565296649933, + "selector_regret_at_8": 0.3453921228647232, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.418215811252594, + "total_car_to_hidden_at_8": 1.7636079341173172 + }, + "534717141": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18033001516380143, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2800043225288391, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3791036296516339, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7199956774711609, + "selected_utility_gain_over_base_at_8": -0.003976881504058838, + "selector_regret_at_8": 1.2800043225288391, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2800043225288391 + }, + "565369392": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.05178981522719065, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.21578265463075008, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.14469666282335916, + "proposal_oracle_utility_gain_over_base_at_8": -0.655068983634313, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.09290684759616852, + "selected_utility_gain_over_base_at_8": -0.7068587988615036, + "selector_regret_at_8": 0.05178981522719065, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.281934916973114, + "total_car_to_hidden_at_8": 1.3337247322003047 + }, + "565621826": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.01741097339739402, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3300621116998719, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.034002700820565224, + "proposal_oracle_utility_gain_over_base_at_8": -0.04128939720491568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.016591727423171203, + "selected_utility_gain_over_base_at_8": -0.058700370602309704, + "selector_regret_at_8": 0.01741097339739402, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2036442911873261, + "total_car_to_hidden_at_8": 0.22105526458472013 + }, + "626486813": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.033813119400292635, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.24681469768825515, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.07908930000849068, + "proposal_oracle_utility_gain_over_base_at_8": -0.4263507166178897, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.04527618060819805, + "selected_utility_gain_over_base_at_8": -0.46016383601818234, + "selector_regret_at_8": 0.033813119400292635, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.544636546401307, + "total_car_to_hidden_at_8": 0.5671922296751291 + }, + "636680137": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0048203859478235245, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19331728269593437, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.024462858214974403, + "proposal_oracle_utility_gain_over_base_at_8": -1.9755371417850256, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.01964247226715088, + "selected_utility_gain_over_base_at_8": -1.9803575277328491, + "selector_regret_at_8": 0.0048203859478235245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755371417850256, + "total_car_to_hidden_at_8": 1.9803575277328491 + }, + "706479600": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.1455947458744049, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2692917587355154, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21975377202033997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6153350174427032, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07415902614593506, + "selected_utility_gain_over_base_at_8": -0.7609297633171082, + "selector_regret_at_8": 0.1455947458744049, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.78024622797966, + "total_car_to_hidden_at_8": 1.925840973854065 + }, + "750448296": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.015873273834586143, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.3795450047628438, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.109864491969347, + "proposal_oracle_utility_gain_over_base_at_8": -0.0016690678894519806, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09399121813476086, + "selected_utility_gain_over_base_at_8": -0.017542341724038124, + "selector_regret_at_8": 0.015873273834586143, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12594309821724892, + "total_car_to_hidden_at_8": 0.14181637205183506 + }, + "75070641": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.003223627805709839, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.13808834192306887, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2283252328634262, + "proposal_oracle_utility_gain_over_base_at_8": 0.025573596358299255, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.22510160505771637, + "selected_utility_gain_over_base_at_8": 0.022349968552589417, + "selector_regret_at_8": 0.003223627805709839, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7716747671365738, + "total_car_to_hidden_at_8": 1.7748983949422836 + }, + "767747424": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5240038954652846, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.2798082148668758, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5921752639114857, + "proposal_oracle_utility_gain_over_base_at_8": 0.04465107619762421, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.06817136844620109, + "selected_utility_gain_over_base_at_8": -0.4793528192676604, + "selector_regret_at_8": 0.5240038954652846, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.04815509542822838, + "total_car_to_hidden_at_8": 0.12046428816393018 + }, + "77147264": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.013380475342273712, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22561784152146835, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.1132144927978516, + "proposal_oracle_utility_gain_over_base_at_8": -0.30836498737335205, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0998340174555779, + "selected_utility_gain_over_base_at_8": -0.32174546271562576, + "selector_regret_at_8": 0.013380475342273712, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.8867855072021484, + "total_car_to_hidden_at_8": 0.9001659825444221 + }, + "794181626": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3405496231635778, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "957101489": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3607921451330185, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2493251178612577, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5799027681350708, + "proposal_oracle_utility_gain_over_base_at_8": 0.37618090212345123, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2191106230020523, + "selected_utility_gain_over_base_at_8": 0.01538875699043274, + "selector_regret_at_8": 0.3607921451330185, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4200972318649292, + "total_car_to_hidden_at_8": 1.7808893769979477 + }, + "998761162": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0010883957147598267, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1991858618764328, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09276536107063293, + "proposal_oracle_utility_gain_over_base_at_8": 0.024948790669441223, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09167696535587311, + "selected_utility_gain_over_base_at_8": 0.023860394954681396, + "selector_regret_at_8": 0.0010883957147598267, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.907234638929367, + "total_car_to_hidden_at_8": 1.908323034644127 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics_by_task.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..fdd9b6c5779514d2243a0d5cb14be8b9df426f12 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.8737756659587225, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.28712935319594163, + "proposal_oracle_success_at_8": 0.8333333333333334, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 1.7598092059294383, + "proposal_oracle_utility_gain_over_base_at_8": 0.6928650587797165, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8860335399707159, + "selected_utility_gain_over_base_at_8": -0.18091060717900595, + "selector_regret_at_8": 0.8737756659587225, + "success_selector_gap_at_8": 0.6666666666666666, + "success_support_gap_at_8": 0.16666666666666666, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.24019079407056174, + "total_car_to_hidden_at_8": 1.1139664600292842 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.13619081233799293, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.35294117647058826, + "pairwise_causal_calibration_ece": 0.23311818739476642, + "proposal_oracle_success_at_8": 0.20588235294117646, + "proposal_oracle_success_gain_over_base_at_8": -0.08823529411764706, + "proposal_oracle_utility_at_8": 0.47449842058053676, + "proposal_oracle_utility_gain_over_base_at_8": -0.17674104834098697, + "selected_success_at_8": 0.14705882352941177, + "selected_success_gain_over_base_at_8": -0.14705882352941177, + "selected_utility_at_8": 0.33830760824254386, + "selected_utility_gain_over_base_at_8": -0.31293186067897993, + "selector_regret_at_8": 0.13619081233799293, + "success_selector_gap_at_8": 0.058823529411764705, + "success_support_gap_at_8": 0.14705882352941177, + "success_total_car_to_hidden_at_8": 0.17647058823529413, + "support_gap_at_8": 0.3622883737313232, + "total_car_to_hidden_at_8": 0.4440142285568393 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.4458426363193072, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5384615384615384, + "pairwise_causal_calibration_ece": 0.15615009218311454, + "proposal_oracle_success_at_8": 0.6153846153846154, + "proposal_oracle_success_gain_over_base_at_8": 0.3076923076923077, + "proposal_oracle_utility_at_8": 1.2588291787184203, + "proposal_oracle_utility_gain_over_base_at_8": 0.5816647181144128, + "selected_success_at_8": 0.38461538461538464, + "selected_success_gain_over_base_at_8": 0.07692307692307693, + "selected_utility_at_8": 0.8129865423991129, + "selected_utility_gain_over_base_at_8": 0.13582208179510558, + "selector_regret_at_8": 0.4458426363193072, + "success_selector_gap_at_8": 0.23076923076923078, + "success_support_gap_at_8": 0.38461538461538464, + "success_total_car_to_hidden_at_8": 0.6153846153846154, + "support_gap_at_8": 0.7411708212815799, + "total_car_to_hidden_at_8": 1.187013457600887 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.5313535690307617, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5333333333333333, + "pairwise_causal_calibration_ece": 0.25228774235838763, + "proposal_oracle_success_at_8": 0.26666666666666666, + "proposal_oracle_success_gain_over_base_at_8": 0.06666666666666667, + "proposal_oracle_utility_at_8": 0.8613873839378356, + "proposal_oracle_utility_gain_over_base_at_8": 0.05827683409055074, + "selected_success_at_8": 0.06666666666666667, + "selected_success_gain_over_base_at_8": -0.13333333333333333, + "selected_utility_at_8": 0.33003381490707395, + "selected_utility_gain_over_base_at_8": -0.47307673494021096, + "selector_regret_at_8": 0.5313535690307617, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.7333333333333333, + "success_total_car_to_hidden_at_8": 0.9333333333333333, + "support_gap_at_8": 1.1386126160621644, + "total_car_to_hidden_at_8": 1.669966185092926 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/report.md b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/report.md new file mode 100644 index 0000000000000000000000000000000000000000..70f68f1f7a26b981878c9037191803e78cd558b1 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/report.md @@ -0,0 +1,27 @@ +# Metric Evaluation (measured) + +K: `8` + +| Metric | N | Micro mean | 95% CI | Task macro | Seed macro | +| --- | ---: | ---: | ---: | ---: | ---: | +| base_success | 69 | 0.2754 | [0.1884, 0.4058] | 0.2270 | 0.3085 | +| base_utility | 69 | 0.7181 | [0.5449, 0.9239] | 0.6704 | 0.8164 | +| branch_car_at_8 | 69 | 0.3487 | [0.1943, 0.4943] | 0.4815 | 0.4176 | +| hidden_chart_oracle_success_at_8 | 69 | 0.6667 | [0.5507, 0.7826] | 0.8647 | 0.8358 | +| hidden_chart_oracle_utility_at_8 | 69 | 1.4000 | [1.1926, 1.6081] | 1.7565 | 1.7062 | +| outcome_ptr_at_8 | 69 | 0.4638 | [0.3478, 0.5797] | 0.6183 | 0.4816 | +| pairwise_causal_calibration_ece | 69 | 0.2266 | [0.1956, 0.2556] | 0.2197 | 0.2148 | +| proposal_oracle_success_at_8 | 69 | 0.3478 | [0.2174, 0.4928] | 0.3843 | 0.4174 | +| proposal_oracle_success_gain_over_base_at_8 | 69 | 0.0725 | [-0.0725, 0.1739] | 0.1572 | 0.1089 | +| proposal_oracle_utility_at_8 | 69 | 0.8180 | [0.5915, 1.0681] | 0.9641 | 0.9824 | +| proposal_oracle_utility_gain_over_base_at_8 | 69 | 0.0999 | [-0.1436, 0.2862] | 0.2937 | 0.1660 | +| selected_success_at_8 | 69 | 0.1739 | [0.1014, 0.2899] | 0.1530 | 0.2057 | +| selected_success_gain_over_base_at_8 | 69 | -0.1014 | [-0.2174, -0.0290] | -0.0740 | -0.1028 | +| selected_utility_at_8 | 69 | 0.4693 | [0.3037, 0.6778] | 0.4826 | 0.5648 | +| selected_utility_gain_over_base_at_8 | 69 | -0.2487 | [-0.4637, -0.0852] | -0.1878 | -0.2515 | +| selector_regret_at_8 | 69 | 0.3487 | [0.1943, 0.4943] | 0.4815 | 0.4176 | +| success_selector_gap_at_8 | 69 | 0.1739 | [0.0870, 0.2754] | 0.2313 | 0.2117 | +| success_support_gap_at_8 | 69 | 0.3333 | [0.2174, 0.4348] | 0.4863 | 0.4238 | +| success_total_car_to_hidden_at_8 | 69 | 0.4928 | [0.3623, 0.6232] | 0.7117 | 0.6301 | +| support_gap_at_8 | 69 | 0.6088 | [0.4128, 0.7915] | 0.8033 | 0.7337 | +| total_car_to_hidden_at_8 | 69 | 0.9307 | [0.7182, 1.1327] | 1.2738 | 1.1414 | diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/table.tex b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..e8f238faca54e2beddb03b32a4ed0f1db12dd669 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/measured_metrics/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 69 & 0.2754 & 0.1884 & 0.4058 \\ +base\_utility & 69 & 0.7181 & 0.5449 & 0.9239 \\ +branch\_car\_at\_8 & 69 & 0.3487 & 0.1943 & 0.4943 \\ +hidden\_chart\_oracle\_success\_at\_8 & 69 & 0.6667 & 0.5507 & 0.7826 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 69 & 1.4000 & 1.1926 & 1.6081 \\ +outcome\_ptr\_at\_8 & 69 & 0.4638 & 0.3478 & 0.5797 \\ +pairwise\_causal\_calibration\_ece & 69 & 0.2266 & 0.1956 & 0.2556 \\ +proposal\_oracle\_success\_at\_8 & 69 & 0.3478 & 0.2174 & 0.4928 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 69 & 0.0725 & -0.0725 & 0.1739 \\ +proposal\_oracle\_utility\_at\_8 & 69 & 0.8180 & 0.5915 & 1.0681 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 69 & 0.0999 & -0.1436 & 0.2862 \\ +selected\_success\_at\_8 & 69 & 0.1739 & 0.1014 & 0.2899 \\ +selected\_success\_gain\_over\_base\_at\_8 & 69 & -0.1014 & -0.2174 & -0.0290 \\ +selected\_utility\_at\_8 & 69 & 0.4693 & 0.3037 & 0.6778 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 69 & -0.2487 & -0.4637 & -0.0852 \\ +selector\_regret\_at\_8 & 69 & 0.3487 & 0.1943 & 0.4943 \\ +success\_selector\_gap\_at\_8 & 69 & 0.1739 & 0.0870 & 0.2754 \\ +success\_support\_gap\_at\_8 & 69 & 0.3333 & 0.2174 & 0.4348 \\ +success\_total\_car\_to\_hidden\_at\_8 & 69 & 0.4928 & 0.3623 & 0.6232 \\ +support\_gap\_at\_8 & 69 & 0.6088 & 0.4128 & 0.7915 \\ +total\_car\_to\_hidden\_at\_8 & 69 & 0.9307 & 0.7182 & 1.1327 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..413b6b707a63ec5c78ca593a23fb71649e4b970a --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics.json @@ -0,0 +1,1918 @@ +{ + "checkpoint": "runs/ctt_residual_base_context_obj_seed0/model.pt", + "k": 8, + "measured_metric_summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.46099290780141844, + "low": 0.19680851063829788, + "mean": 0.3085106382978724, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 1.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3176470588235294, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.4057971014492754, + "low": 0.18840579710144928, + "mean": 0.2753623188405797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.0808477015729916, + "low": 0.6013251688004412, + "mean": 0.8163502178883101, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8163502178883101, + "per_seed": { + "1025740212": 0.528373658657074, + "1055157354": 0.19648407399654388, + "1055567618": 2.0, + "1113268940": 0.42231252789497375, + "1113371261": 2.0, + "1176943004": 0.15461373329162598, + "1189392917": 0.10841864347457886, + "1231780997": 0.7270061373710632, + "1271718007": 1.415381133556366, + "1273247072": 0.15342965722084045, + "1358421325": 0.8272150158882141, + "1375492786": 2.0, + "141984615": 0.09381213908394177, + "1475440973": 2.0, + "1483645336": 0.01641605794429779, + "1522272686": 0.19844311475753784, + "1528740417": 2.0, + "1549324805": 1.3787474483251572, + "1607773860": 0.8326728343963623, + "1665739511": 2.0, + "1747178030": 0.14663122594356537, + "1881223807": 0.0320269912481308, + "1919198188": 0.03719887137413025, + "1991414747": 2.0, + "1997090716": 0.13927479088306427, + "2015956957": 0.06711978986859321, + "2054899126": 2.0, + "211245842": 0.8388344049453735, + "2141802816": 0.07080724835395813, + "256770874": 2.0, + "278008524": 0.10747186839580536, + "416183106": 0.20032641291618347, + "534717141": 2.0, + "551504635": 0.7239725589752197, + "565369392": 0.7997656464576721, + "565621826": 2.0, + "585904130": 0.07529209802548091, + "626486813": 0.5054400166263804, + "636680137": 2.0, + "706479600": 0.8350887894630432, + "750448296": 0.11153355985879898, + "75070641": 0.20275163650512695, + "767747424": 0.5475241877138615, + "77147264": 1.4215794801712036, + "794181626": 0.18095605075359344, + "957101489": 0.20372186601161957, + "998761162": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9058472741949395, + "low": 0.4133020142714182, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.923898630168127, + "low": 0.5448560892550302, + "mean": 0.7180729379567007, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6087407858949815, + "low": 0.243578380005804, + "mean": 0.4175693826273526, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4175693826273526, + "per_seed": { + "1025740212": 1.281606376171112, + "1055157354": 0.4498589485883713, + "1055567618": 1.828233242034912, + "1113268940": 0.11861386895179749, + "1113371261": 0.0, + "1176943004": 1.9300042241811752, + "1189392917": 0.0, + "1231780997": 1.2770342230796814, + "1271718007": 0.9082518517971039, + "1273247072": 0.42020074278116226, + "1358421325": 0.6354914605617523, + "1375492786": 0.0008215903653763235, + "141984615": 0.05737591476645321, + "1475440973": 1.2853952050209045, + "1483645336": 1.8899840712547302, + "1522272686": 0.030991941690444946, + "1528740417": 0.0, + "1549324805": 0.0010904477288325627, + "1607773860": 1.8556141257286072, + "1665739511": 0.0, + "1747178030": 0.02001962810754776, + "1881223807": 0.002386912703514099, + "1919198188": 0.021911710500717163, + "1991414747": 0.017522692680358887, + "1997090716": 1.9452667087316513, + "2015956957": 0.39167013615369795, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.008425161242485046, + "256770874": 0.44993481040000916, + "278008524": 0.0008676897268742323, + "416183106": 0.3453921228647232, + "534717141": 0.0, + "551504635": 1.2800043225288391, + "565369392": 0.05178981522719065, + "565621826": 0.0, + "585904130": 0.01741097339739402, + "626486813": 0.033813119400292635, + "636680137": 0.0048203859478235245, + "706479600": 0.1455947458744049, + "750448296": 0.015873273834586143, + "75070641": 0.003223627805709839, + "767747424": 0.5240038954652846, + "77147264": 0.013380475342273712, + "794181626": 0.0, + "957101489": 0.3607921451330185, + "998761162": 0.0010883957147598267 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6577742758489845, + "low": 0.27715372847280956, + "mean": 0.4814726852855894, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4814726852855894, + "per_task": { + "LiftPegUpright-v1": 0.8737756659587225, + "PickCube-v1": 0.42020074278116226, + "PullCube-v1": 0.13619081233799293, + "PushCube-v1": 0.4458426363193072, + "StackCube-v1": 0.5313535690307617 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4942535392575614, + "low": 0.19425320601178522, + "mean": 0.3486898574730098, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9148936170212765, + "low": 0.7432624113475178, + "mean": 0.8358156028368794, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8358156028368794, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.782608695652174, + "low": 0.5507246376811594, + "mean": 0.6666666666666666, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.850041654515774, + "low": 1.5397213380149701, + "mean": 1.7062028111638965, + "n": 47, + "num_samples": 200 + }, + "mean": 1.7062028111638965, + "per_seed": { + "1025740212": 2.0, + "1055157354": 2.0, + "1055567618": 2.0, + "1113268940": 2.0, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 2.0, + "1358421325": 2.0, + "1375492786": 0.24345096945762634, + "141984615": 0.20044784247875214, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 2.0, + "1528740417": 2.0, + "1549324805": 1.4027505218982697, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 2.0, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.5298813670873642, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 2.0, + "278008524": 1.113811194896698, + "416183106": 2.0, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 1.4266315797964733, + "565621826": 2.0, + "585904130": 0.23764699200789133, + "626486813": 0.6124684102833271, + "636680137": 2.0, + "706479600": 2.0, + "750448296": 0.23580759018659592, + "75070641": 2.0, + "767747424": 0.18863565661013126, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 2.0, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 200 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.6080652458080347, + "low": 1.1926357444861662, + "mean": 1.3999846732054932, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6170212765957447, + "low": 0.3319148936170213, + "mean": 0.48156028368794324, + "n": 47, + "num_samples": 200 + }, + "mean": 0.48156028368794324, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.3333333333333333, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.8, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.75, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.5, + "75070641": 1.0, + "767747424": 0.75, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8143589743589743, + "low": 0.45176470588235296, + "mean": 0.6182805429864253, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6182805429864253, + "per_task": { + "LiftPegUpright-v1": 0.6666666666666666, + "PickCube-v1": 1.0, + "PullCube-v1": 0.35294117647058826, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.5333333333333333 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5797101449275363, + "low": 0.34782608695652173, + "mean": 0.463768115942029, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.24376288810786303, + "low": 0.18434974480601804, + "mean": 0.21479075208855025, + "n": 47, + "num_samples": 200 + }, + "mean": 0.21479075208855025, + "per_seed": { + "1025740212": 0.20050368294022264, + "1055157354": 0.43030250381223584, + "1055567618": 0.30857397468613657, + "1113268940": 0.3774241061869338, + "1113371261": 0.026017268580811406, + "1176943004": 0.1246594058194818, + "1189392917": 0.1340247769860469, + "1231780997": 0.19966589920839098, + "1271718007": 0.17939225581112472, + "1273247072": 0.16962492308566807, + "1358421325": 0.28863435648886215, + "1375492786": 0.0996910344508803, + "141984615": 0.297780428420543, + "1475440973": 0.41171052461838353, + "1483645336": 0.35034389349653194, + "1522272686": 0.31643520896589616, + "1528740417": 0.15436827657008492, + "1549324805": 0.23727288575803293, + "1607773860": 0.3951037237909243, + "1665739511": 0.0, + "1747178030": 0.11661420436902857, + "1881223807": 0.20266639463366093, + "1919198188": 0.09083822075370537, + "1991414747": 0.09717880195623664, + "1997090716": 0.3641822422309461, + "2015956957": 0.22628515797280127, + "2054899126": 0.0, + "211245842": 0.14834095177221712, + "2141802816": 0.07518554002658885, + "256770874": 0.18757436508611464, + "278008524": 0.10616076660738837, + "416183106": 0.1517874168346938, + "534717141": 0.18033001516380143, + "551504635": 0.3791036296516339, + "565369392": 0.21578265463075008, + "565621826": 0.0, + "585904130": 0.3300621116998719, + "626486813": 0.24681469768825515, + "636680137": 0.19331728269593437, + "706479600": 0.2692917587355154, + "750448296": 0.3795450047628438, + "75070641": 0.13808834192306887, + "767747424": 0.2798082148668758, + "77147264": 0.22561784152146835, + "794181626": 0.3405496231635778, + "957101489": 0.2493251178612577, + "998761162": 0.1991858618764328 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.26552488687547154, + "low": 0.18076755457919055, + "mean": 0.21966205964357566, + "n": 5, + "num_samples": 200 + }, + "mean": 0.21966205964357566, + "per_task": { + "LiftPegUpright-v1": 0.28712935319594163, + "PickCube-v1": 0.16962492308566807, + "PullCube-v1": 0.23311818739476642, + "PushCube-v1": 0.15615009218311454, + "StackCube-v1": 0.25228774235838763 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.2555647418249429, + "low": 0.19558072798702378, + "mean": 0.22656067749912578, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5304964539007092, + "low": 0.2815602836879432, + "mean": 0.4173758865248227, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4173758865248227, + "per_seed": { + "1025740212": 1.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6230769230769232, + "low": 0.10666666666666666, + "mean": 0.38425339366515837, + "n": 5, + "num_samples": 200 + }, + "mean": 0.38425339366515837, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 0.0, + "PullCube-v1": 0.20588235294117646, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4927536231884058, + "low": 0.21739130434782608, + "mean": 0.34782608695652173, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.23829787234042552, + "low": -0.04539007092198581, + "mean": 0.10886524822695036, + "n": 47, + "num_samples": 200 + }, + "mean": 0.10886524822695036, + "per_seed": { + "1025740212": 1.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3438914027149321, + "low": -0.00862745098039216, + "mean": 0.15722473604826545, + "n": 5, + "num_samples": 200 + }, + "mean": 0.15722473604826545, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 0.0, + "PullCube-v1": -0.08823529411764706, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.06666666666666667 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.17391304347826086, + "low": -0.07246376811594203, + "mean": 0.07246376811594203, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.1794423699735643, + "low": 0.7291675245113577, + "mean": 0.9823928705606516, + "n": 47, + "num_samples": 200 + }, + "mean": 0.9823928705606516, + "per_seed": { + "1025740212": 2.0, + "1055157354": 0.5705926418304443, + "1055567618": 2.0, + "1113268940": 0.5588552355766296, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 0.4659159183502197, + "1358421325": 0.8147144317626953, + "1375492786": 0.0015756288776174188, + "141984615": 0.06459133536554873, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 0.21208032965660095, + "1528740417": 2.0, + "1549324805": 1.344537331412236, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 0.1273784190416336, + "1881223807": 0.027498364448547363, + "1919198188": 0.09917314350605011, + "1991414747": 0.13679209351539612, + "1997090716": 2.0, + "2015956957": 0.46120103299617765, + "2054899126": 2.0, + "211245842": 0.8135772347450256, + "2141802816": 0.08595690131187439, + "256770874": 0.574478030204773, + "278008524": 0.013509750366210938, + "416183106": 0.581784188747406, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 0.14469666282335916, + "565621826": 2.0, + "585904130": 0.034002700820565224, + "626486813": 0.07908930000849068, + "636680137": 0.024462858214974403, + "706479600": 0.21975377202033997, + "750448296": 0.109864491969347, + "75070641": 0.2283252328634262, + "767747424": 0.5921752639114857, + "77147264": 1.1132144927978516, + "794181626": 2.0, + "957101489": 0.5799027681350708, + "998761162": 0.09276536107063293 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.400834542971391, + "low": 0.5501597128059331, + "mean": 0.9640880215032903, + "n": 5, + "num_samples": 200 + }, + "mean": 0.9640880215032903, + "per_task": { + "LiftPegUpright-v1": 1.7598092059294383, + "PickCube-v1": 0.4659159183502197, + "PullCube-v1": 0.47449842058053676, + "PushCube-v1": 1.2588291787184203, + "StackCube-v1": 0.8613873839378356 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.0680866769491357, + "low": 0.5915389626904431, + "mean": 0.8180189497981464, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.4120927946805518, + "low": -0.10385253924549498, + "mean": 0.16604265267234164, + "n": 47, + "num_samples": 200 + }, + "mean": 0.16604265267234164, + "per_seed": { + "1025740212": 1.471626341342926, + "1055157354": 0.37410856783390045, + "1055567618": 0.0, + "1113268940": 0.13654270768165588, + "1113371261": 0.0, + "1176943004": 1.845386266708374, + "1189392917": 1.8915813565254211, + "1231780997": 1.2729938626289368, + "1271718007": 0.584618866443634, + "1273247072": 0.3124862611293793, + "1358421325": -0.012500584125518799, + "1375492786": -1.9984243711223826, + "141984615": -0.029220803718393046, + "1475440973": 0.0, + "1483645336": 1.9835839420557022, + "1522272686": 0.01363721489906311, + "1528740417": 0.0, + "1549324805": -0.03421011691292127, + "1607773860": 1.1673271656036377, + "1665739511": 0.0, + "1747178030": -0.019252806901931763, + "1881223807": -0.004528626799583435, + "1919198188": 0.06197427213191986, + "1991414747": -1.8632079064846039, + "1997090716": 1.8607252091169357, + "2015956957": 0.39408124312758447, + "2054899126": 0.0, + "211245842": -0.0252571702003479, + "2141802816": 0.01514965295791626, + "256770874": -1.425521969795227, + "278008524": -0.09396211802959442, + "416183106": 0.38145777583122253, + "534717141": 0.0, + "551504635": 1.2760274410247803, + "565369392": -0.655068983634313, + "565621826": 0.0, + "585904130": -0.04128939720491568, + "626486813": -0.4263507166178897, + "636680137": -1.9755371417850256, + "706479600": -0.6153350174427032, + "750448296": -0.0016690678894519806, + "75070641": 0.025573596358299255, + "767747424": 0.04465107619762421, + "77147264": -0.30836498737335205, + "794181626": 1.8190439492464066, + "957101489": 0.37618090212345123, + "998761162": 0.024948790669441223 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.5184734715865209, + "low": 0.015111566525701359, + "mean": 0.2937103647546145, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2937103647546145, + "per_task": { + "LiftPegUpright-v1": 0.6928650587797165, + "PickCube-v1": 0.3124862611293793, + "PullCube-v1": -0.17674104834098697, + "PushCube-v1": 0.5816647181144128, + "StackCube-v1": 0.05827683409055074 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2862129747003749, + "low": -0.1435656816482652, + "mean": 0.09994601184144562, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.30851063829787234, + "low": 0.10992907801418439, + "mean": 0.20567375886524825, + "n": 47, + "num_samples": 200 + }, + "mean": 0.20567375886524825, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.2774358974358974, + "low": 0.04666666666666667, + "mean": 0.15300150829562592, + "n": 5, + "num_samples": 200 + }, + "mean": 0.15300150829562592, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.38461538461538464, + "StackCube-v1": 0.06666666666666667 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2898550724637681, + "low": 0.10144927536231885, + "mean": 0.17391304347826086, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.0017730496453900741, + "low": -0.2127659574468085, + "mean": -0.10283687943262411, + "n": 47, + "num_samples": 200 + }, + "mean": -0.10283687943262411, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": -1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.001357466063348417, + "low": -0.156078431372549, + "mean": -0.07402714932126697, + "n": 5, + "num_samples": 200 + }, + "mean": -0.07402714932126697, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.14705882352941177, + "PushCube-v1": 0.07692307692307693, + "StackCube-v1": -0.13333333333333333 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.028985507246376812, + "low": -0.21739130434782608, + "mean": -0.10144927536231885, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.742602269810456, + "low": 0.37777582296799433, + "mean": 0.5648234879332992, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5648234879332992, + "per_seed": { + "1025740212": 0.7183936238288879, + "1055157354": 0.12073369324207306, + "1055567618": 0.1717667579650879, + "1113268940": 0.44024136662483215, + "1113371261": 2.0, + "1176943004": 0.06999577581882477, + "1189392917": 2.0, + "1231780997": 0.7229657769203186, + "1271718007": 1.0917481482028961, + "1273247072": 0.045715175569057465, + "1358421325": 0.179222971200943, + "1375492786": 0.0007540385122410953, + "141984615": 0.007215420599095523, + "1475440973": 0.7146047949790955, + "1483645336": 0.11001592874526978, + "1522272686": 0.181088387966156, + "1528740417": 2.0, + "1549324805": 1.3434468836834033, + "1607773860": 0.14438587427139282, + "1665739511": 2.0, + "1747178030": 0.10735879093408585, + "1881223807": 0.025111451745033264, + "1919198188": 0.07726143300533295, + "1991414747": 0.11926940083503723, + "1997090716": 0.054733291268348694, + "2015956957": 0.0695308968424797, + "2054899126": 2.0, + "211245842": 0.8135772347450256, + "2141802816": 0.07753174006938934, + "256770874": 0.1245432198047638, + "278008524": 0.012642060639336705, + "416183106": 0.2363920658826828, + "534717141": 2.0, + "551504635": 0.7199956774711609, + "565369392": 0.09290684759616852, + "565621826": 2.0, + "585904130": 0.016591727423171203, + "626486813": 0.04527618060819805, + "636680137": 0.01964247226715088, + "706479600": 0.07415902614593506, + "750448296": 0.09399121813476086, + "75070641": 0.22510160505771637, + "767747424": 0.06817136844620109, + "77147264": 1.0998340174555779, + "794181626": 2.0, + "957101489": 0.2191106230020523, + "998761162": 0.09167696535587311 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7326601550821198, + "low": 0.16275214863845203, + "mean": 0.4826153362177009, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4826153362177009, + "per_task": { + "LiftPegUpright-v1": 0.8860335399707159, + "PickCube-v1": 0.045715175569057465, + "PullCube-v1": 0.33830760824254386, + "PushCube-v1": 0.8129865423991129, + "StackCube-v1": 0.33003381490707395 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6777843068475309, + "low": 0.3036627637912465, + "mean": 0.46932909232513653, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": -0.0441184930471994, + "low": -0.4570727952216018, + "mean": -0.251526729955011, + "n": 47, + "num_samples": 200 + }, + "mean": -0.251526729955011, + "per_seed": { + "1025740212": 0.19001996517181396, + "1055157354": -0.07575038075447083, + "1055567618": -1.828233242034912, + "1113268940": 0.0179288387298584, + "1113371261": 0.0, + "1176943004": -0.08461795747280121, + "1189392917": 1.8915813565254211, + "1231780997": -0.004040360450744629, + "1271718007": -0.32363298535346985, + "1273247072": -0.10771448165178299, + "1358421325": -0.6479920446872711, + "1375492786": -1.999245961487759, + "141984615": -0.08659671848484625, + "1475440973": -1.2853952050209045, + "1483645336": 0.09359987080097198, + "1522272686": -0.017354726791381836, + "1528740417": 0.0, + "1549324805": -0.03530056464175383, + "1607773860": -0.6882869601249695, + "1665739511": 0.0, + "1747178030": -0.03927243500947952, + "1881223807": -0.006915539503097534, + "1919198188": 0.0400625616312027, + "1991414747": -1.8807305991649628, + "1997090716": -0.08454149961471558, + "2015956957": 0.0024111069738864898, + "2054899126": 0.0, + "211245842": -0.0252571702003479, + "2141802816": 0.006724491715431213, + "256770874": -1.8754567801952362, + "278008524": -0.09482980775646865, + "416183106": 0.03606565296649933, + "534717141": 0.0, + "551504635": -0.003976881504058838, + "565369392": -0.7068587988615036, + "565621826": 0.0, + "585904130": -0.058700370602309704, + "626486813": -0.46016383601818234, + "636680137": -1.9803575277328491, + "706479600": -0.7609297633171082, + "750448296": -0.017542341724038124, + "75070641": 0.022349968552589417, + "767747424": -0.4793528192676604, + "77147264": -0.32174546271562576, + "794181626": 1.8190439492464066, + "957101489": 0.01538875699043274, + "998761162": 0.023860394954681396 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": -0.017275244494533833, + "low": -0.34496083553122614, + "mean": -0.18776232053097486, + "n": 5, + "num_samples": 200 + }, + "mean": -0.18776232053097486, + "per_task": { + "LiftPegUpright-v1": -0.18091060717900595, + "PickCube-v1": -0.10771448165178299, + "PullCube-v1": -0.31293186067897993, + "PushCube-v1": 0.13582208179510558, + "StackCube-v1": -0.47307673494021096 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.08520316913642961, + "low": -0.4636818296832802, + "mean": -0.24874384563156418, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6087407858949815, + "low": 0.243578380005804, + "mean": 0.4175693826273526, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4175693826273526, + "per_seed": { + "1025740212": 1.281606376171112, + "1055157354": 0.4498589485883713, + "1055567618": 1.828233242034912, + "1113268940": 0.11861386895179749, + "1113371261": 0.0, + "1176943004": 1.9300042241811752, + "1189392917": 0.0, + "1231780997": 1.2770342230796814, + "1271718007": 0.9082518517971039, + "1273247072": 0.42020074278116226, + "1358421325": 0.6354914605617523, + "1375492786": 0.0008215903653763235, + "141984615": 0.05737591476645321, + "1475440973": 1.2853952050209045, + "1483645336": 1.8899840712547302, + "1522272686": 0.030991941690444946, + "1528740417": 0.0, + "1549324805": 0.0010904477288325627, + "1607773860": 1.8556141257286072, + "1665739511": 0.0, + "1747178030": 0.02001962810754776, + "1881223807": 0.002386912703514099, + "1919198188": 0.021911710500717163, + "1991414747": 0.017522692680358887, + "1997090716": 1.9452667087316513, + "2015956957": 0.39167013615369795, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.008425161242485046, + "256770874": 0.44993481040000916, + "278008524": 0.0008676897268742323, + "416183106": 0.3453921228647232, + "534717141": 0.0, + "551504635": 1.2800043225288391, + "565369392": 0.05178981522719065, + "565621826": 0.0, + "585904130": 0.01741097339739402, + "626486813": 0.033813119400292635, + "636680137": 0.0048203859478235245, + "706479600": 0.1455947458744049, + "750448296": 0.015873273834586143, + "75070641": 0.003223627805709839, + "767747424": 0.5240038954652846, + "77147264": 0.013380475342273712, + "794181626": 0.0, + "957101489": 0.3607921451330185, + "998761162": 0.0010883957147598267 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6577742758489845, + "low": 0.27715372847280956, + "mean": 0.4814726852855894, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4814726852855894, + "per_task": { + "LiftPegUpright-v1": 0.8737756659587225, + "PickCube-v1": 0.42020074278116226, + "PullCube-v1": 0.13619081233799293, + "PushCube-v1": 0.4458426363193072, + "StackCube-v1": 0.5313535690307617 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4942535392575614, + "low": 0.19425320601178522, + "mean": 0.3486898574730098, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3382978723404255, + "low": 0.10638297872340426, + "mean": 0.21170212765957444, + "n": 47, + "num_samples": 200 + }, + "mean": 0.21170212765957444, + "per_seed": { + "1025740212": 1.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.45176470588235296, + "low": 0.07529411764705883, + "mean": 0.2312518853695324, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2312518853695324, + "per_task": { + "LiftPegUpright-v1": 0.6666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.058823529411764705, + "PushCube-v1": 0.23076923076923078, + "StackCube-v1": 0.2 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2753623188405797, + "low": 0.08695652173913043, + "mean": 0.17391304347826086, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5656028368794326, + "low": 0.299645390070922, + "mean": 0.4237588652482269, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4237588652482269, + "per_seed": { + "1025740212": 0.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7760784313725491, + "low": 0.2024132730015083, + "mean": 0.48633484162895924, + "n": 5, + "num_samples": 200 + }, + "mean": 0.48633484162895924, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 1.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.38461538461538464, + "StackCube-v1": 0.7333333333333333 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.43478260869565216, + "low": 0.21739130434782608, + "mean": 0.3333333333333333, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7411347517730495, + "low": 0.48191489361702133, + "mean": 0.6301418439716312, + "n": 47, + "num_samples": 200 + }, + "mean": 0.6301418439716312, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9266666666666667, + "low": 0.42895927601809963, + "mean": 0.7117043740573152, + "n": 5, + "num_samples": 200 + }, + "mean": 0.7117043740573152, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.9333333333333333 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6231884057971014, + "low": 0.36231884057971014, + "mean": 0.4927536231884058, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.960610361603016, + "low": 0.5171997663569934, + "mean": 0.7336599861108202, + "n": 47, + "num_samples": 200 + }, + "mean": 0.7336599861108202, + "per_seed": { + "1025740212": 0.0, + "1055157354": 1.4294073581695557, + "1055567618": 0.0, + "1113268940": 1.4411447644233704, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.5340840816497803, + "1358421325": 1.1852855682373047, + "1375492786": 0.24187534058000892, + "141984615": 0.1358565071132034, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.787919670343399, + "1528740417": 0.0, + "1549324805": 0.05821319048603376, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 1.8726215809583664, + "1881223807": 1.9725016355514526, + "1919198188": 1.90082685649395, + "1991414747": 1.8632079064846039, + "1997090716": 0.0, + "2015956957": 0.06868033409118653, + "2054899126": 0.0, + "211245842": 1.1864227652549744, + "2141802816": 1.9140430986881256, + "256770874": 1.425521969795227, + "278008524": 1.100301444530487, + "416183106": 1.418215811252594, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 1.281934916973114, + "565621826": 0.0, + "585904130": 0.2036442911873261, + "626486813": 0.544636546401307, + "636680137": 1.9755371417850256, + "706479600": 1.78024622797966, + "750448296": 0.12594309821724892, + "75070641": 1.7716747671365738, + "767747424": 0.04815509542822838, + "77147264": 0.8867855072021484, + "794181626": 0.0, + "957101489": 1.4200972318649292, + "998761162": 1.907234638929367 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.2206306469485657, + "low": 0.36480631544491765, + "mean": 0.8032693373590819, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8032693373590819, + "per_task": { + "LiftPegUpright-v1": 0.24019079407056174, + "PickCube-v1": 1.5340840816497803, + "PullCube-v1": 0.3622883737313232, + "PushCube-v1": 0.7411708212815799, + "StackCube-v1": 1.1386126160621644 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.7914590279021692, + "low": 0.41281149387224647, + "mean": 0.6088035285584223, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.3427126641637903, + "low": 0.9024325011565566, + "mean": 1.141379323230597, + "n": 47, + "num_samples": 200 + }, + "mean": 1.141379323230597, + "per_seed": { + "1025740212": 1.281606376171112, + "1055157354": 1.879266306757927, + "1055567618": 1.828233242034912, + "1113268940": 1.5597586333751678, + "1113371261": 0.0, + "1176943004": 1.9300042241811752, + "1189392917": 0.0, + "1231780997": 1.2770342230796814, + "1271718007": 0.9082518517971039, + "1273247072": 1.9542848244309425, + "1358421325": 1.820777028799057, + "1375492786": 0.24269693094538525, + "141984615": 0.1932324218796566, + "1475440973": 1.2853952050209045, + "1483645336": 1.8899840712547302, + "1522272686": 1.818911612033844, + "1528740417": 0.0, + "1549324805": 0.05930363821486632, + "1607773860": 1.8556141257286072, + "1665739511": 0.0, + "1747178030": 1.8926412090659142, + "1881223807": 1.9748885482549667, + "1919198188": 1.922738566994667, + "1991414747": 1.8807305991649628, + "1997090716": 1.9452667087316513, + "2015956957": 0.4603504702448845, + "2054899126": 0.0, + "211245842": 1.1864227652549744, + "2141802816": 1.9224682599306107, + "256770874": 1.8754567801952362, + "278008524": 1.1011691342573613, + "416183106": 1.7636079341173172, + "534717141": 0.0, + "551504635": 1.2800043225288391, + "565369392": 1.3337247322003047, + "565621826": 0.0, + "585904130": 0.22105526458472013, + "626486813": 0.5671922296751291, + "636680137": 1.9803575277328491, + "706479600": 1.925840973854065, + "750448296": 0.14181637205183506, + "75070641": 1.7748983949422836, + "767747424": 0.12046428816393018, + "77147264": 0.9001659825444221, + "794181626": 0.0, + "957101489": 1.7808893769979477, + "998761162": 1.908323034644127 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.687103095329725, + "low": 0.8378044656728661, + "mean": 1.273849031142176, + "n": 5, + "num_samples": 200 + }, + "mean": 1.273849031142176, + "per_task": { + "LiftPegUpright-v1": 1.1139664600292842, + "PickCube-v1": 1.9542848244309425, + "PullCube-v1": 0.4440142285568393, + "PushCube-v1": 1.187013457600887, + "StackCube-v1": 1.669966185092926 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 1.1326504737118501, + "low": 0.71822580749142, + "mean": 0.9306555808803566, + "n": 69, + "num_samples": 200 + }, + "n": 69 + } + }, + "num_rows": 69, + "report_type": "ctt_generated_measured_rollout", + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "source_split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "success_summary": { + "base_success_rate": 0.2753623188405797, + "base_utility_mean": 0.7180729379567007, + "hidden_chart_oracle_success_rate": 0.6666666666666666, + "hidden_chart_oracle_utility_mean": 1.3999846732054932, + "max_restore_error": 1.4901161193847656e-08, + "proposal_oracle_success_gain_over_base": 0.07246376811594203, + "proposal_oracle_success_rate": 0.34782608695652173, + "proposal_oracle_utility_mean": 0.8180189497981464, + "selected_success_gain_over_base": -0.10144927536231885, + "selected_success_rate": 0.17391304347826086, + "selected_utility_mean": 0.46932909232513653, + "success_selector_gap": 0.17391304347826086, + "success_support_gap": 0.3333333333333333 + }, + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381" +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics_by_seed.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..971b465d20fdcad4e441a5b853373c0a428f8142 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics_by_seed.json @@ -0,0 +1,1083 @@ +{ + "1025740212": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 1.281606376171112, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.20050368294022264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.471626341342926, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7183936238288879, + "selected_utility_gain_over_base_at_8": 0.19001996517181396, + "selector_regret_at_8": 1.281606376171112, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.281606376171112 + }, + "1055157354": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.4498589485883713, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.43030250381223584, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5705926418304443, + "proposal_oracle_utility_gain_over_base_at_8": 0.37410856783390045, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12073369324207306, + "selected_utility_gain_over_base_at_8": -0.07575038075447083, + "selector_regret_at_8": 0.4498589485883713, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4294073581695557, + "total_car_to_hidden_at_8": 1.879266306757927 + }, + "1055567618": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.828233242034912, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30857397468613657, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1717667579650879, + "selected_utility_gain_over_base_at_8": -1.828233242034912, + "selector_regret_at_8": 1.828233242034912, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.828233242034912 + }, + "1113268940": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.11861386895179749, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3774241061869338, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5588552355766296, + "proposal_oracle_utility_gain_over_base_at_8": 0.13654270768165588, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.44024136662483215, + "selected_utility_gain_over_base_at_8": 0.0179288387298584, + "selector_regret_at_8": 0.11861386895179749, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4411447644233704, + "total_car_to_hidden_at_8": 1.5597586333751678 + }, + "1113371261": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026017268580811406, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9300042241811752, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1246594058194818, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06999577581882477, + "selected_utility_gain_over_base_at_8": -0.08461795747280121, + "selector_regret_at_8": 1.9300042241811752, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9300042241811752 + }, + "1189392917": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1340247769860469, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 1.2770342230796814, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.19966589920839098, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7229657769203186, + "selected_utility_gain_over_base_at_8": -0.004040360450744629, + "selector_regret_at_8": 1.2770342230796814, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2770342230796814 + }, + "1271718007": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9082518517971039, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.17939225581112472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0917481482028961, + "selected_utility_gain_over_base_at_8": -0.32363298535346985, + "selector_regret_at_8": 0.9082518517971039, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9082518517971039 + }, + "1273247072": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + "1358421325": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.6354914605617523, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.28863435648886215, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8147144317626953, + "proposal_oracle_utility_gain_over_base_at_8": -0.012500584125518799, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.179222971200943, + "selected_utility_gain_over_base_at_8": -0.6479920446872711, + "selector_regret_at_8": 0.6354914605617523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1852855682373047, + "total_car_to_hidden_at_8": 1.820777028799057 + }, + "1375492786": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008215903653763235, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0996910344508803, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0015756288776174188, + "proposal_oracle_utility_gain_over_base_at_8": -1.9984243711223826, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0007540385122410953, + "selected_utility_gain_over_base_at_8": -1.999245961487759, + "selector_regret_at_8": 0.0008215903653763235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24187534058000892, + "total_car_to_hidden_at_8": 0.24269693094538525 + }, + "141984615": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.05737591476645321, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.297780428420543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06459133536554873, + "proposal_oracle_utility_gain_over_base_at_8": -0.029220803718393046, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.007215420599095523, + "selected_utility_gain_over_base_at_8": -0.08659671848484625, + "selector_regret_at_8": 0.05737591476645321, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1358565071132034, + "total_car_to_hidden_at_8": 0.1932324218796566 + }, + "1475440973": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2853952050209045, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.41171052461838353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7146047949790955, + "selected_utility_gain_over_base_at_8": -1.2853952050209045, + "selector_regret_at_8": 1.2853952050209045, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2853952050209045 + }, + "1483645336": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 1.8899840712547302, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.35034389349653194, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11001592874526978, + "selected_utility_gain_over_base_at_8": 0.09359987080097198, + "selector_regret_at_8": 1.8899840712547302, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8899840712547302 + }, + "1522272686": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.030991941690444946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.31643520896589616, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21208032965660095, + "proposal_oracle_utility_gain_over_base_at_8": 0.01363721489906311, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.181088387966156, + "selected_utility_gain_over_base_at_8": -0.017354726791381836, + "selector_regret_at_8": 0.030991941690444946, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.787919670343399, + "total_car_to_hidden_at_8": 1.818911612033844 + }, + "1528740417": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15436827657008492, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1549324805": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.0010904477288325627, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.23727288575803293, + "proposal_oracle_success_at_8": 0.6666666666666666, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.344537331412236, + "proposal_oracle_utility_gain_over_base_at_8": -0.03421011691292127, + "selected_success_at_8": 0.6666666666666666, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.3434468836834033, + "selected_utility_gain_over_base_at_8": -0.03530056464175383, + "selector_regret_at_8": 0.0010904477288325627, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05821319048603376, + "total_car_to_hidden_at_8": 0.05930363821486632 + }, + "1607773860": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.8556141257286072, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3951037237909243, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14438587427139282, + "selected_utility_gain_over_base_at_8": -0.6882869601249695, + "selector_regret_at_8": 1.8556141257286072, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8556141257286072 + }, + "1665739511": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1747178030": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.02001962810754776, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11661420436902857, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1273784190416336, + "proposal_oracle_utility_gain_over_base_at_8": -0.019252806901931763, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10735879093408585, + "selected_utility_gain_over_base_at_8": -0.03927243500947952, + "selector_regret_at_8": 0.02001962810754776, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8726215809583664, + "total_car_to_hidden_at_8": 1.8926412090659142 + }, + "1881223807": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.002386912703514099, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.20266639463366093, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.027498364448547363, + "proposal_oracle_utility_gain_over_base_at_8": -0.004528626799583435, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.025111451745033264, + "selected_utility_gain_over_base_at_8": -0.006915539503097534, + "selector_regret_at_8": 0.002386912703514099, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9725016355514526, + "total_car_to_hidden_at_8": 1.9748885482549667 + }, + "1919198188": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.021911710500717163, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09083822075370537, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09917314350605011, + "proposal_oracle_utility_gain_over_base_at_8": 0.06197427213191986, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07726143300533295, + "selected_utility_gain_over_base_at_8": 0.0400625616312027, + "selector_regret_at_8": 0.021911710500717163, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.90082685649395, + "total_car_to_hidden_at_8": 1.922738566994667 + }, + "1991414747": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.017522692680358887, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.09717880195623664, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.13679209351539612, + "proposal_oracle_utility_gain_over_base_at_8": -1.8632079064846039, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11926940083503723, + "selected_utility_gain_over_base_at_8": -1.8807305991649628, + "selector_regret_at_8": 0.017522692680358887, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8632079064846039, + "total_car_to_hidden_at_8": 1.8807305991649628 + }, + "1997090716": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 1.9452667087316513, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3641822422309461, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.054733291268348694, + "selected_utility_gain_over_base_at_8": -0.08454149961471558, + "selector_regret_at_8": 1.9452667087316513, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9452667087316513 + }, + "2015956957": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.39167013615369795, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.22628515797280127, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.46120103299617765, + "proposal_oracle_utility_gain_over_base_at_8": 0.39408124312758447, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0695308968424797, + "selected_utility_gain_over_base_at_8": 0.0024111069738864898, + "selector_regret_at_8": 0.39167013615369795, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.06868033409118653, + "total_car_to_hidden_at_8": 0.4603504702448845 + }, + "2054899126": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14834095177221712, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8135772347450256, + "proposal_oracle_utility_gain_over_base_at_8": -0.0252571702003479, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8135772347450256, + "selected_utility_gain_over_base_at_8": -0.0252571702003479, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1864227652549744, + "total_car_to_hidden_at_8": 1.1864227652549744 + }, + "2141802816": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 0.008425161242485046, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07518554002658885, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08595690131187439, + "proposal_oracle_utility_gain_over_base_at_8": 0.01514965295791626, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07753174006938934, + "selected_utility_gain_over_base_at_8": 0.006724491715431213, + "selector_regret_at_8": 0.008425161242485046, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9140430986881256, + "total_car_to_hidden_at_8": 1.9224682599306107 + }, + "256770874": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.44993481040000916, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18757436508611464, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.574478030204773, + "proposal_oracle_utility_gain_over_base_at_8": -1.425521969795227, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1245432198047638, + "selected_utility_gain_over_base_at_8": -1.8754567801952362, + "selector_regret_at_8": 0.44993481040000916, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.425521969795227, + "total_car_to_hidden_at_8": 1.8754567801952362 + }, + "278008524": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0008676897268742323, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10616076660738837, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.013509750366210938, + "proposal_oracle_utility_gain_over_base_at_8": -0.09396211802959442, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.012642060639336705, + "selected_utility_gain_over_base_at_8": -0.09482980775646865, + "selector_regret_at_8": 0.0008676897268742323, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.100301444530487, + "total_car_to_hidden_at_8": 1.1011691342573613 + }, + "416183106": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.3453921228647232, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1517874168346938, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.581784188747406, + "proposal_oracle_utility_gain_over_base_at_8": 0.38145777583122253, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2363920658826828, + "selected_utility_gain_over_base_at_8": 0.03606565296649933, + "selector_regret_at_8": 0.3453921228647232, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.418215811252594, + "total_car_to_hidden_at_8": 1.7636079341173172 + }, + "534717141": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18033001516380143, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2800043225288391, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3791036296516339, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7199956774711609, + "selected_utility_gain_over_base_at_8": -0.003976881504058838, + "selector_regret_at_8": 1.2800043225288391, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2800043225288391 + }, + "565369392": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.05178981522719065, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.21578265463075008, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.14469666282335916, + "proposal_oracle_utility_gain_over_base_at_8": -0.655068983634313, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.09290684759616852, + "selected_utility_gain_over_base_at_8": -0.7068587988615036, + "selector_regret_at_8": 0.05178981522719065, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.281934916973114, + "total_car_to_hidden_at_8": 1.3337247322003047 + }, + "565621826": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.01741097339739402, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3300621116998719, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.034002700820565224, + "proposal_oracle_utility_gain_over_base_at_8": -0.04128939720491568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.016591727423171203, + "selected_utility_gain_over_base_at_8": -0.058700370602309704, + "selector_regret_at_8": 0.01741097339739402, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2036442911873261, + "total_car_to_hidden_at_8": 0.22105526458472013 + }, + "626486813": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.033813119400292635, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.24681469768825515, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.07908930000849068, + "proposal_oracle_utility_gain_over_base_at_8": -0.4263507166178897, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.04527618060819805, + "selected_utility_gain_over_base_at_8": -0.46016383601818234, + "selector_regret_at_8": 0.033813119400292635, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.544636546401307, + "total_car_to_hidden_at_8": 0.5671922296751291 + }, + "636680137": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0048203859478235245, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19331728269593437, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.024462858214974403, + "proposal_oracle_utility_gain_over_base_at_8": -1.9755371417850256, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.01964247226715088, + "selected_utility_gain_over_base_at_8": -1.9803575277328491, + "selector_regret_at_8": 0.0048203859478235245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755371417850256, + "total_car_to_hidden_at_8": 1.9803575277328491 + }, + "706479600": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.1455947458744049, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2692917587355154, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21975377202033997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6153350174427032, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07415902614593506, + "selected_utility_gain_over_base_at_8": -0.7609297633171082, + "selector_regret_at_8": 0.1455947458744049, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.78024622797966, + "total_car_to_hidden_at_8": 1.925840973854065 + }, + "750448296": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.015873273834586143, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.3795450047628438, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.109864491969347, + "proposal_oracle_utility_gain_over_base_at_8": -0.0016690678894519806, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09399121813476086, + "selected_utility_gain_over_base_at_8": -0.017542341724038124, + "selector_regret_at_8": 0.015873273834586143, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12594309821724892, + "total_car_to_hidden_at_8": 0.14181637205183506 + }, + "75070641": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.003223627805709839, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.13808834192306887, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2283252328634262, + "proposal_oracle_utility_gain_over_base_at_8": 0.025573596358299255, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.22510160505771637, + "selected_utility_gain_over_base_at_8": 0.022349968552589417, + "selector_regret_at_8": 0.003223627805709839, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7716747671365738, + "total_car_to_hidden_at_8": 1.7748983949422836 + }, + "767747424": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5240038954652846, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.2798082148668758, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5921752639114857, + "proposal_oracle_utility_gain_over_base_at_8": 0.04465107619762421, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.06817136844620109, + "selected_utility_gain_over_base_at_8": -0.4793528192676604, + "selector_regret_at_8": 0.5240038954652846, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.04815509542822838, + "total_car_to_hidden_at_8": 0.12046428816393018 + }, + "77147264": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.013380475342273712, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22561784152146835, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.1132144927978516, + "proposal_oracle_utility_gain_over_base_at_8": -0.30836498737335205, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.0998340174555779, + "selected_utility_gain_over_base_at_8": -0.32174546271562576, + "selector_regret_at_8": 0.013380475342273712, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.8867855072021484, + "total_car_to_hidden_at_8": 0.9001659825444221 + }, + "794181626": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3405496231635778, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "957101489": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3607921451330185, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2493251178612577, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5799027681350708, + "proposal_oracle_utility_gain_over_base_at_8": 0.37618090212345123, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2191106230020523, + "selected_utility_gain_over_base_at_8": 0.01538875699043274, + "selector_regret_at_8": 0.3607921451330185, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4200972318649292, + "total_car_to_hidden_at_8": 1.7808893769979477 + }, + "998761162": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0010883957147598267, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1991858618764328, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09276536107063293, + "proposal_oracle_utility_gain_over_base_at_8": 0.024948790669441223, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09167696535587311, + "selected_utility_gain_over_base_at_8": 0.023860394954681396, + "selector_regret_at_8": 0.0010883957147598267, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.907234638929367, + "total_car_to_hidden_at_8": 1.908323034644127 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics_by_task.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..fdd9b6c5779514d2243a0d5cb14be8b9df426f12 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.8737756659587225, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.28712935319594163, + "proposal_oracle_success_at_8": 0.8333333333333334, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 1.7598092059294383, + "proposal_oracle_utility_gain_over_base_at_8": 0.6928650587797165, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8860335399707159, + "selected_utility_gain_over_base_at_8": -0.18091060717900595, + "selector_regret_at_8": 0.8737756659587225, + "success_selector_gap_at_8": 0.6666666666666666, + "success_support_gap_at_8": 0.16666666666666666, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.24019079407056174, + "total_car_to_hidden_at_8": 1.1139664600292842 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42020074278116226, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.16962492308566807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4659159183502197, + "proposal_oracle_utility_gain_over_base_at_8": 0.3124862611293793, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.045715175569057465, + "selected_utility_gain_over_base_at_8": -0.10771448165178299, + "selector_regret_at_8": 0.42020074278116226, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5340840816497803, + "total_car_to_hidden_at_8": 1.9542848244309425 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.13619081233799293, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.35294117647058826, + "pairwise_causal_calibration_ece": 0.23311818739476642, + "proposal_oracle_success_at_8": 0.20588235294117646, + "proposal_oracle_success_gain_over_base_at_8": -0.08823529411764706, + "proposal_oracle_utility_at_8": 0.47449842058053676, + "proposal_oracle_utility_gain_over_base_at_8": -0.17674104834098697, + "selected_success_at_8": 0.14705882352941177, + "selected_success_gain_over_base_at_8": -0.14705882352941177, + "selected_utility_at_8": 0.33830760824254386, + "selected_utility_gain_over_base_at_8": -0.31293186067897993, + "selector_regret_at_8": 0.13619081233799293, + "success_selector_gap_at_8": 0.058823529411764705, + "success_support_gap_at_8": 0.14705882352941177, + "success_total_car_to_hidden_at_8": 0.17647058823529413, + "support_gap_at_8": 0.3622883737313232, + "total_car_to_hidden_at_8": 0.4440142285568393 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.4458426363193072, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5384615384615384, + "pairwise_causal_calibration_ece": 0.15615009218311454, + "proposal_oracle_success_at_8": 0.6153846153846154, + "proposal_oracle_success_gain_over_base_at_8": 0.3076923076923077, + "proposal_oracle_utility_at_8": 1.2588291787184203, + "proposal_oracle_utility_gain_over_base_at_8": 0.5816647181144128, + "selected_success_at_8": 0.38461538461538464, + "selected_success_gain_over_base_at_8": 0.07692307692307693, + "selected_utility_at_8": 0.8129865423991129, + "selected_utility_gain_over_base_at_8": 0.13582208179510558, + "selector_regret_at_8": 0.4458426363193072, + "success_selector_gap_at_8": 0.23076923076923078, + "success_support_gap_at_8": 0.38461538461538464, + "success_total_car_to_hidden_at_8": 0.6153846153846154, + "support_gap_at_8": 0.7411708212815799, + "total_car_to_hidden_at_8": 1.187013457600887 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.5313535690307617, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5333333333333333, + "pairwise_causal_calibration_ece": 0.25228774235838763, + "proposal_oracle_success_at_8": 0.26666666666666666, + "proposal_oracle_success_gain_over_base_at_8": 0.06666666666666667, + "proposal_oracle_utility_at_8": 0.8613873839378356, + "proposal_oracle_utility_gain_over_base_at_8": 0.05827683409055074, + "selected_success_at_8": 0.06666666666666667, + "selected_success_gain_over_base_at_8": -0.13333333333333333, + "selected_utility_at_8": 0.33003381490707395, + "selected_utility_gain_over_base_at_8": -0.47307673494021096, + "selector_regret_at_8": 0.5313535690307617, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.7333333333333333, + "success_total_car_to_hidden_at_8": 0.9333333333333333, + "support_gap_at_8": 1.1386126160621644, + "total_car_to_hidden_at_8": 1.669966185092926 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/report.md b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/report.md new file mode 100644 index 0000000000000000000000000000000000000000..366becffec0299501fc9b4d832aca62aebe3ae71 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/report.md @@ -0,0 +1,13 @@ +# CTT Generated Measured Rollout + +Rows: `69` +K: `8` +Checkpoint: `runs/ctt_residual_base_context_obj_seed0/model.pt` + +| Metric | Mean | +| --- | ---: | +| OutcomePTR@K | 0.4638 | +| SelectorRegret@K | 0.3487 | +| SupportGap@K | 0.5820 | + +These are measured simulator rollouts of decoded CTT candidates, not PPTC proxies. diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/run.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/run.log new file mode 100644 index 0000000000000000000000000000000000000000..b6ac7d222a5ee4b675092aa04f64c0b12161e33d --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/run.log @@ -0,0 +1,766 @@ +start +importing gymnasium/mani_skill +imported gymnasium/mani_skill +loaded checkpoint=runs/ctt_residual_base_context_obj_seed0/model.pt +loaded charts source=301 target=69 +generated proposals rows=69 total=552 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=PickCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PickCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +rollout complete rows=69 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_data_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..1ef49748c0932e9ccb1c83a065667e7a4032f7db --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_data_hash.txt @@ -0,0 +1 @@ +67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_index.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_index.json new file mode 100644 index 0000000000000000000000000000000000000000..21f76a4f847a39cd7631c9675f06c81edf567240 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_index.json @@ -0,0 +1,4179 @@ +{ + "audience": "train_retrieval", + "base_candidate_types": [ + "policy", + "policy_residual", + "anchor", + "base", + "expert" + ], + "candidate_type_counts": { + "expert": 2044, + "near_miss": 4088, + "no_op": 2044, + "random_negative": 20440, + "wrong_direction": 2044, + "wrong_gripper": 2044 + }, + "content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "dataset": "/scratch/knguy52/dovla/experiments/six_task_h16_collection", + "deployment_candidate_excludes_expert": true, + "deployment_clean": true, + "embedding_content_hash": "00859bfac7590abb24a3e7cc4c26fbb17122ba52d9f63369b8bb71a6ad1bc302", + "epsilon": 0.05, + "format": "cil_charts_npz", + "group_hashes": [ + "001fcaa1974d30d8f6edbd3e9117a10a084af107360ce1b7899dba37d093d073", + "00526cba701d9c5d7399d0436d2718fe144013e2e0277c3e00f247c3f2b026e2", + "007da68284552fa257c5929a32c040913c93b9f31569bc4a9e0e5f469972c3f5", + "0081224c84c2faa89c82c842265c41240c850ffc35d5400b57531935df3d704d", + "00bbd346a4f8658a96b2167b8ed3b4a4275cd535f5b46ff6a2ba6fe55ab5af22", + "00c41e13d6af229bd31262983b05d48413360e477204b62ba89278a110632a3c", + "00ccbf9e23e7dd8cc6ea5d91197bab1ee5d7e26f1c2e9dff0db7c48332e262d7", + "00df496b285db96e866529484da8bed3b079234ab8ed99faf38a9b105ae136d5", + "01335787d62e2c1bfa20e6a53a53239d19087e703ff07e2bb0666862536259b8", + "014ef3543741d318e6b6f9a3427e4423b1594cc26a73fbfa8266a1dadffc051b", + "015b93c295196814c0cf417d49ecf1d82d00e68458e42d2a0b3a1179d6d4c135", + "01a286ab7524d1d907099802c01da4995ebb8381d71bba0460100499c3c622f9", + "01e693c8677801418926281fd44938d17c1d275ed367253f31d1208676259a5a", + "023bdae7b5ddc517f49ec900a0e48dc698bfe1ec6f1a7cc7edfac21b46f37333", + "025c271db3f4ffb8e484ec4ac3f7244b0380bbf904037af516580e8d7ebf605a", + "0267094d9bc010468e6f684ab19365e65b31b38da78897277def7201c91bac27", + "0294a908f74ea0b1e54e4fb6d547fe5ba986ca4c6881572213732a0c867699ba", + "02fc97ab4aa793222f215b66d0ffc5c856706ec647c7482ee66344e066456af2", + "03020ce876d6f83841e56c51882dc8e6be8bf33ec9c9c836ab42ab8e4a586538", + "030b87c34937f69c95bb61bf59d9773b5a365a86fa73a60f619ccd0256d2968b", + "036bd56b347b3ea03e19a9a1e1aa865021d7b541fbdca003409a10e34912b3c7", + "0388ca1306e31d2ebd07b30a8d5366063523b4cb18d426c93351e7497cdf2721", + "03b05cc0e617e7ab7149752551af4303fa838be521fe4b22a15389558dfc0d60", + "0414958360a2c4e04be7534d9347e289e132993e773f7518bc8ee75a02ede570", + "041af35d269bba29a9d94019ad4c8e433bd9b11e3d2c12af4922829649df041e", + "042cc293ccfe917dac796cd6101f1d2f4fd0fc5c67a050c391399bb3c89b83e9", + "043d323f81c1208341028b4176ed8c4c56e69abb8c08aec31ddf093056d9db95", + "044f47ab5be1af5d16cf8dc96c3201970c551380fc2e97dcce7ae2f5265658b3", + "04529d1ba49a38cd27f3ffca329e63ab856f5c1e9cadcf7599c074ea64d26acb", + "045cb769e1cc19f043df64a32a05dc520442e9dd62fe0dcaac93c048c8a53589", + "04a1476f8f2cbb1493afb3e1849003222903e15a7cc7e519dfabb48bd1bfb57a", + "04c63f785efcf0d3591dd2051455fa4878b473bc2cbeee626ec7e549bbf46a14", + "04d6d326619cb05f115813e476605dca0485b6bcbb256765822fd1be38def668", + "04efaf4eb209e619534ffaa3bbb08fdc7a05759f77c96ede295435e899557617", + "04f4ff1016421b38f73be280be75ca4fea5c372f3a8d1cda4da185de229da025", + "04faed0a7dfd3ebc89a4bd2e8e075c9a81c861e396b860cf1172c5ff89245a11", + "05240853bb100723195b299a8e993517d7636f79d04d39f3d60ebe025863c321", + "054f169f2c20ec618c2805cc410f661edd54db72ff4e124ef2e8e695f5c41483", + "057423d91c0d82d8f62f114440e3d5800ff93a36d789dee2cb1558189230b3f5", + "058273a8b1145d6860b07cd133fbfb6439dadbc9337d32c27c9b6e353b9938c6", + "05b7a5da0c35c1c6c9dd3e767ba83f3a5d8c65cdc688f7c7583b086d2f5e578f", + "05c0e14adca2fb1afd50c9c82effa52f4a3c036d865a6ef07cf43ab6cb7cf9c1", + "05c0fa198643aa9b11a16b11f508c0f3e4876d76adb60edfd63e5b22da4f6096", + "05d812f28f8fa0fc86ae7e47fee0b87e82a8c147cd2c20a18efca3e4d1716ed0", + "05eff262f0c51434f1a2717cd888ac132d5418a211f5be302c76204c694c9c22", + "05f242687ca564fcce8802518ff330054ba4aa6fd52dc16b739af6e2cd73265c", + "060071dc4d719b0172224dadd17bbcad4074d9a025bd9753dc25bcaa82a1b731", + "0612f705ce60a6e5719459f4ed2ae70cc12c6f610553af267bba19f1079e159d", + "062f09156bfa5ecc2c90411a5c30cf7ab44cfdab8fdf6bba8dbf0d0264398669", + "0637e894caaf7866508328256b86a787acb4b59279b4bc4d77fa920669620b0d", + "06646b414506091dc9562eec9f7ef5a47fc11f0d46fefd5f17118e104334ad20", + "069838f6f7f1d064c02323989485d308d36f8650e6062785a461e2b6cff0b0b0", + "06b2ec0364fcc116b85d61aa2b8f007fc319e48756acfc228765b7aa0e79db87", + "06f3fd02d9a8cbf4fa564d32546c328fc318d993b9d2d1ef685324463c69ddbe", + "0713a51c5892c5558579ce4dffdc60b974477b43e53c3fe0ae4345b45549ec17", + "07202bb06fd690678da08362e77ce653818eba36201fadb49d49d1e56ec9c228", + "0749514f298a42cfe924e24c240e25f5ff48f47a898f8586b200cfa0701c7a37", + "0768ebbabf013c2a4b7f1aad3a2099c8c13cc1b0b6a083b446a17336f2d17d04", + "07721a0faa4a8ca8a5af0f07c002538292f7cf23086a796ac71f17d92c68ccd5", + "077f12bea73da5c959c3558c431a5137f203c4468023cd9bb80ed7aa72c5cb15", + "0781ab28ea0865deb62cdf28ff45c8db767f6acac236094d1ac9671a621c0b10", + "079f2e4745cdf1e6fe28c5fd9c44ff1154b2317c9670dce3df3e46c5956b6266", + "07d1cf8fa045404915eaa0bc582d7c1f1ea3e5417ebd7d9b076c0106cf937b49", + "07d78d133b041c1b387a66dde85c2e8d619868aadbe25df65adf9a537733fbe2", + "07d981b9235d6f62b04fe940d3b60888c7324180a1eaf5258b8b7d28dc9e09e1", + "082bc6e23d9f8848668b5e2bcf5d214b05d642fe9247181ac7efd87c1152c4dc", + "0844b2faa86b75616ad9e61099a8546dbe2ce19734eb93507a1c294eed6904a7", + "084e99784f5d3144472f2bb4c6ad302124d7f238dae7d6cb5c48e30e490e2b0c", + "085f9fcd72c44e1ec1f4ae40b1d183e6854e606726d8358ac37b9f360c0358c3", + "0884cb0cd7bfeda804f3351dfee94fedc6468594260bbe98c0e2414ec08781f2", + "088693fbb45b889e85ff958da47452272c49d1720c61f8a24e4ad652bc1fb716", + "089bd7ee69056cfb1908280fe2f2b1bac9fe708220e479097c6dd18fb259a24a", + "08bd69953da3b8045ffd02deb5f986a11f2d120614be22b53347d8514e5658fe", + "08ee326bc033bfde4df8541283f59dede6da6662c31cd3826c8938218f7a90a4", + "0905b1f6e979661d55c733e49ea636ad4829039c2f8327ff1f261409b6f9c8e9", + "090d54a2871e4dfca2aefefbc2e9663d0a7b710e7957d20c00c361dc43a1d2e1", + "09c524d9908967bff722776dacc5e2dc1841dd82e85f2eac79be2986be4727c7", + "09c52593507b34845144191b4a8edc5cd9b4808dcdc1593b396fe166b29ac84b", + "09da6fc458cc6cf1e4be0ceef803120255238b8962070c2cbffe715551717085", + "09e9d938e278c8cc5f8322c610fcea1283b7b77bafe842eb6449215d49394e02", + "0a9312ec31c472903cdb28026d86b90d551127135683773d122e17b6e0cfb857", + "0a9bc7df156ecfad97d28ee1227663785ce40cc737168364932f1a2e30cb90b1", + "0aa1a4571153dba5c1748f018cab757e1ece943ee4ea64a0f26bdae4f936d3e6", + "0aa4cb3d3b2e041fa4253c1cb23a914b8a5c644b56967d3741516fb17d3631f9", + "0ab83ad36b03e69d64d2ae4ef8c556097be124061af0360d32848dc41bdfb7ea", + "0abf91fd3685960f7b6e0812b4d8ce52b6a31224b4e739a506304cb46cd29fea", + "0ae69b83b02e7dbc4eb264be79de0da7185863d38bf366a9be1ae5d745b9a7fb", + "0af0a212eca005fb212d32eff4d5c2ce71ca2de8e9970ee416416ebe5d936f06", + "0af0b65f74f5a6edff26999f485e115fb04396168b31c5e77ef731b8b413e366", + "0b1ff010af5f1418d0f838f4f21a71c1025a797e50f0dfce6fdb578b4bdac377", + "0b41ad1882d3407ba74fd9527ddba492ebdb1356a231d41d3b97bea671cc902c", + "0b660aea784794a286ecd4ad09d7536cde2eb65a19280c8ba81d9112849a05af", + "0b9d988ac32a9d5138800792db6436da932ecaba29485cb95676d018b8c4cc66", + "0bb16a236a2dbbbc53df7f8f514f71076a1408dbe89e9b64f56ff6308149a5aa", + "0bbb7041a0e42c9d69fb059687967f4f5cf4dcec5a48a233d3a1c1b6ae117304", + "0bc9c1dc25638927adbf62edb1f44c41fc6468564c0c0bd07f151256e70edcdf", + "0be11c9bfdda8d9d43bf40a9567a2a7acc8b09343e6411ddb55a6b0097406f58", + "0be26fe12af26d5427c1b23a13e9a0ab66560bd0bab91a5023987f9a483a5abf", + "0bfafacef2b0488487659ba6e9f07942ddf7a9734f49746e14e68e8b903a5e57", + "0bfffb14c5e767e6b6a0d073a8c38b271abb71ec60be03aa3c6676de8620d1e1", + "0c0015ce0828d91018c21a25c3a988ae23e9a116b77ccfee298c2d770fcbc1e4", + "0c0136f75efec62c8234e61675eb5d7cc92a005887e1daf40961dda217effdf3", + "0c26b22b6f73a920acd2c3eb853dd13714c7d1a15cb7f01189612964543e2bc1", + "0c5ebfeb8ce3384fdf5c7466d9febb871c0a55b3dce9196adbf45beeb3c3a8dc", + "0c729263a7865648102e96eb98f764199f7e3f9be15d9852bb832d72aaed3cfb", + "0c8ad825fb1b90dad15342b1a38a73230557030fc6cb483d074998a391017af5", + "0c8edffaf99aaa4628cfc8af78d3b08d78e69e645baeb1ed83bfd23d9be43a5d", + "0caf685769ea343e0714af61762ed28dd8a3619c999cf2d503f590ec404494d5", + "0cb26bcd10a801c61924e70d10b27e1c10e58ec84a1f7fd1ea0fd4691d2d577f", + "0cb8a1e195b091af4c1ab6264bdadcf5dea204e12810c9f9674a810717bccb54", + "0cd782853dfd7e50a17e062e2466a0c2834bb6ae09ccca30173972814b444d92", + "0cdbd13e6ddc48ca28d0018a7b2a880b226489f679aee1a6ac782581c3894de8", + "0ce23e2fa8a749821680c196c1169b01e7e30af9b335e1cf59b8b62c58e04fa6", + "0d54c732a07f0fe8a6fc5458c2fec6b0e153a40bb33677bf852c815add211b7c", + "0d70dcc638707ec9f6834ffbcb4630e07abdccb780a1609d713a970f3f0ac1cb", + "0dc1128ba4c2992987f1080cfbf80add951244c5512109367ac2ff6dd8481f04", + "0dc6b7748f769c6f4fb5b5e760d69c37a5c91fa538ed1dd31a3d5ba4cbc5d607", + "0e292618af8adc8bb13df6fdc700564538050caaf00643122064b7a3376a1d0f", + "0e321b35d16bc03a5586ec91b97acf8c107f0b79c2825450f90f35f4fa3eaa0a", + "0e54f1bcdc7d87ecfc6e74d6de3e4c43d13474e1bec0b95138e7167710390990", + "0e846791b18e5014d92e8a138b09f6aa1edfa36549c1ef12a8ca4dc7e1a7e90f", + "0e95a43b5c9a6e453d36ad6f7cadaa8af2df3f11f0ca06fc4ebaa43fec26445e", + "0e9cfc6bb4e6178112967c91b2e35b0a3b38e389cc2aac0aab443dbd63f6b02f", + "0ed2c0dfa1163e363e7e1d89889e7f49e18528658208c0a068ca52e82d3159b5", + "0f1330ac1eb5b8485604be1a622a26c53c19cb305a61c61fc348020b7eeeec9b", + "0f207722dc163c7f09f47a4b114cde2e4d675c7399a3b2941f8c22954ad5df5d", + "0f298d9056aff724d852e7d30dbb93cff797cf382b8fefb1a23884389ba72ae2", + "0f4ded9465d1adb2948c4ebf574aeacac709555b4cba78e2f7ab0514f3d1dc60", + "0f6e062f0fbb5aad354b50ca9695824a01b4b1a2b6650fe534dcd517aae3a3d7", + "0f97e4c29d2ee4474f3e91f9728b59c2ff6ecc3ba0fe4709bea263555e4eecdd", + "0fc1fff21404a7c4c3bf43d0a946b349cddd81fd91394830c215687fd29635c4", + "0fc42e7c404cbbb29ebc6b271746a16e9226ff4270cd13ef194d66d9385f98e4", + "10bafd54576f39b903e32333fa9de033d34630504b3ccfe756933c7cd139fb22", + "10bc4cc7a43e48b9fd9fd2b4dae3488801ea0e043959d8838a5b6f2a485f0756", + "10cb9fea43a85b51aa089d8184d5e7ead3b732a471283229eb053f86a7b89c64", + "10d3721833fbb982ec10f2e41634a06e94f3e11be89b76c449dda794bf4afb03", + "11012a866289284e636f856a871945d4089e1218b25714ddb7e14e3cab1c2d05", + "113f5145a36a62fa140f4eda8be653dc498c33132e8440ba8bf91149e5bc1abb", + "1148a49a8e54f0192c8e4ffb72c997dd10918c28363d1dff62fb9176225d3e5e", + "115434f9bc4c1998ab24ed4071f8513198a71ea8711b65bd911d2283855eeef3", + "1169947cec8d4834b14ef01a1b21b27715efabed8c282612bad37c32b70a4cec", + "11757f25e1971b1949f99f53e1aec63fe120316d0d794ff04c5e3bf3e1e4469e", + "1177abdc80e9dd30dd06522f2a39b4b22ae70c1ca17a29414e0cd1c9015f2340", + "11bb2f5d4191b2c5785176893b48e085f0bb8944d6d257b64647ef8831eaf362", + "11cba72504d167af81d43fb4f4aa447fc4dc1cb8a873cd123fc80c00a4bf605a", + "12044591849b00fdcaee3693c012680c8dbe3ffe120fe6e771ec86954622880e", + "1227a4aa8cd9896b3e02709120920bc948bef60870f206080881533008607752", + "124103b3e01c126d1d7c42ab8999b7069a649f26afe76483bbe75e0516fc9092", + "1272ddae5bbaa8cee4a5917fa083c82b10d9280d0a6dc8f85968f6df3b822eb4", + "129467e910b6d1d31fdf7405d1aee0adacf2b45bcec803f8e1ae3926c3ecc432", + "12a1492fbc2c5986df9c059a33a7a0b9fc5445555bdf672b5fae0a0f34da8a86", + "12bfa8c65887fb9fb257aab37ef0aca1a2f50dda0836afe9a3845fa66295a1d7", + "12cfb3f21f142c20b8b75fe51a500470a46848c69674b4592ceaf5aad9d976e6", + "12d2cda4fddfc7694bbf706182fd2c9ab83964cef1cc1c7b2e3abe84dc6d910d", + "12e49f621d373e186d124b525688c4d1219e8a2b7dbed956835d6e64aefeb8e9", + "12e61e6a086c54726212dd8ae40498e7bf40a4e6fd9ab5b677c950d02eb7cd52", + "12ee15c35b3f1356c065b1161d2a63d7bf8bf083bf7485034dbdc0b40f23c525", + "1311e81322d103c2c43a03f83aca27a945245bf2a02de6eabc767baa61aa2442", + "1313d324d3e798a357662678e05bcb4c9a763b4c397bcf7816611f318b2ea4ae", + "135b237070f72bb9bc691fa076a6343c512900564e119ad27281243d2af0e42a", + "139e442c52e6ca3ad180fe14f1acb72016fc4c630405d7dcd2b8f848edd43a9e", + "13c92688765abb9394668fe90e9843b50fe58a60036450bfc308769592840eac", + "13c9558f5a69f6784d34d046906d2759d6c77af2104091f964afaf7bce5b13ba", + "13ccedcc5114c00535afd5d9e128b3f7011a2926486054ea868bd4a7b6e4b46f", + "13d043c134ac85a633c31260192367eabc7cf1767299124779b9f01a21ba59d5", + "13ee615d8033d8a0e64e589fad80b9efa6d9939c3a95a006300263b5fc7bf19a", + "14b218a5081e90244f5425b45e7d278a74ce673dd2e49343e0a82f4874fecd56", + "14c0c9817f4f22d5f2251efcc78b8ffd67926fba73fb45e17293b59cdb3dd535", + "1501199487ec32e664d5f69df0b7da5186e71defb0c64229e946c7c5f0a0cb2f", + "1507748f87930160bae3bacbc4b87aeafb13ff0584c0b3457b0b15e33af846a5", + "150ecef476986ef9da9b97b6f1ffa83c74fc8ea3d8d57213831b3566d328b61a", + "15128cce41835053eda7e79d8a527734f51a5f94f0e71bc4f7c30c82bbd7381d", + "1590a2e329844ef537803416fa313ec171f54da63051be0a66528df467f45636", + "15c10dd97eff18ff9c11efeaac85032c3db028f8a024a25b5732401570b5de2b", + "15ce55e979b5b57f83c2c79f86f4ac3685778340aeb72db247ade562c3dc9fa4", + "15d91dbb5bb6cbcd2971a2c9005a7d1591a02de0780530cdd09301babf1d4295", + "15db89c2bd9b6360edcfd4126de0d04ab42d10be7e7c9eaba23f1184493696b2", + "15eafc044f81072b73a7e4978efb8a5053684d910f074bb9de0900cee1bde5ec", + "164faca0fe5fbd3466a4751b72dd20c69f8349f0f6b35367b75dcd27d79fde6e", + "165aed8e4463db7858a18e57d902eb84591607731c33d0799fe2e1dd9943a16a", + "166ad8c3cb7f8dd902042e4720f289b39d64cc34e240f516c37a552b98ddc204", + "1680fbae27ac70b8a104e58a59fa2ba47adba09377dcac5e76e68153151a1a24", + "16d7a66ecb74238e0d120dac82df1f13ff15fcd44325e4de897be2612471637a", + "16daf6296ef9184c97ba0e67164997e16876f7a16f3feb490c704ec7bb942f47", + "17242b256da38439da805284116ccd22b6dc0b4f1162e966c58d2699b88ed142", + "1728fffcde610b0ea864ffe387cb258be47affaa5fa2444d7271475cced07a00", + "1730561bf6306e61c4609c774a86ca11b4e49a8d1b01c3425f94ee9ebf718042", + "17794ad07b32b794fba99b4aa198cab80c73a617c80388cb3938f1bc448c33b4", + "179a8d46e8da7103776cc9d41954954f9a30cd4455377cc6af1bf76a0704580c", + "17c358ed8931a3e41bc39832f8024ab4cf2ecebd3070060890e8381a61c7b344", + "17c67143df844a780a50ec041cfb21740ed75142c30dd9a778cf56d46c5e2f5e", + "17e01942d367267f255ecec7a0d0756a47a18c02e6a868ccf4709c4bb7e5d75a", + "17f64daf3c523045868995b9b5893aa69c6406419d51d0d2d5d6dccc95768fb1", + "1828cbf1e501bd910192ca5f0118ad162ca6e99b3d11a588966650e20e5194f0", + "182b16a6d5297ed49a61456d81e924731b627fa701a33bebf113b5d1dbac34a0", + "182f7365b61944dec5bb0919a2d802ef89258842fe129664444ed82a58cfcca8", + "18657e4bb7bcc144077e48065c56afbfaffd0001a13ffdeaf8f6bb2326d8c483", + "18d4f30d97e86421c7a2f9bd716a3944dc4745651cfa54bbbca295c0ed56a933", + "19b08945d0d2787fc572f103b8f4bec38912b933f1489b4c73c41ea3b26e141d", + "19de7f92ee293142b40d8214a494dfd201964ef788d8663ece4c2ab5d3c8612c", + "19e0efc3b92b86a743d8a25d8228e7a35b8767c51244a2940bf325a900680e95", + "19fd5cb3bbb4d83eb86d436bf9786e970c339b1d854e91ea35e9a144e4155203", + "1a12204deff18038d5b27ab69ab157c6feaedb6adfe5560c46e31174eeff0f13", + "1a169940b1033a1daefd265afd89c6491a7a05d9c479667ea58efe6168771027", + "1a644d661b36094b489ea6e2ed67bc9fed3a3f5adbc01583591fef5eb9704bb1", + "1b261a0ce7ec407cba1565a8e4c24f3219233746509f716cab8c4747ee3537c3", + "1b2f9895a0fffaf04836d9581bb8bb6c2529fedf2283da978f875c2860d61f2b", + "1b42c5fb7e97213a533064bfe6bdad601e007166165a8f29f63e7e7999362c42", + "1b47810cd051a297cd61db10f6fb32c434e35242afbb75ebb728a1a9fca6c7de", + "1b644336b4b6654215dbcd04f4fff2f80f003d27531a1a7d5ae344a96223ec0c", + "1b779aa158ed431977a27764618e2a4e373f3ce3c5e20f75c8938781881d0e68", + "1b7dcea7adab5f8b915a848e6415ef988dcb9c01cf49cf61c573955f0845a4a4", + "1b8a6a6fa1b179d5d5437229af8793f416cd6ed3492b1fa08b9883757ba91bde", + "1b945f3f2ba81f701ae6363cb35f8805bef6fd083cb7f96fc28b60bd59b2a42e", + "1bf373e4138ede234792ca2a31f7cdf7d8000b81213d183c592c57810e5f860a", + "1c7c2472654cefecab0ce73cfd71bdce6ba3723a2d1b0e60db8c98aade0214c0", + "1c8a863bb173fb7383178f4791d07dab6d7e61deafe0533713f1dbe165714347", + "1ca29fa47c93e63575ec16caceca3cccb1433ddf46316e5a0e516a370aba3c98", + "1cb925d6fff01c6d651d84259d1a8ac66ef3e62bbe6edbb2dfbf8a32ba6e254b", + "1ce64787a72f626f6a88392499a9255a86c5383a749e004594bb8c1bd28ab033", + "1d31a051b2ec1fc2654e62a20c125ab46a8257c375a84eed7028b7e4e52d89c0", + "1d520b036adf88c23e2c7b29f0332b8b2e0145a89df01b7704c4dad4a4b7a26e", + "1d5844a8d16fd769c49e5ec5a2dbda1f912441f777969d005b10b7a54ec6f222", + "1d61e39cdd0ec95fd2b45881a6ae2dcabf25bc7b199d93850c9cea4689561fe2", + "1d71ce873b57f889c07cbca6e061d4c9a7e9b09f7a5080257809e5ccce1ef3f1", + "1d7833e75164b04c32bacecf88e1adbce107ef962491f630fa7e7995e7dfd250", + "1d7cc58ad59e6b600b4bd9ad4df955100e8d941433168c6cc049e26e843bde9f", + "1dbf59ec48199ba97c18205303d128b76b9c663595181fd12274398a2ca084e3", + "1dc1fd9b8eaa6883f2faca7699c9f508d1f6a8a77686902ac177131cf2899099", + "1e069441905d0a241eb511339e9dbb1fa46aeddbb8297488c2f610ee338cce43", + "1e43c66eca65528152bfe4159ecdb59354ee36c34b54eb62e985b212b25acd13", + "1e50848ce7dc42b4b42718f33d0c415ab9a46775c69ca5a4b82276908bb943eb", + "1e76599c93125e60681aa1bd6ae72f50d0a47a9c49cde9c6756909e98464d2ce", + "1e79355b42972c791c37b68852c99930c49365068ee1a5b7da7692d2aad48a32", + "1ecd0b4f77b2aa35440911addeb9c567c92d0871fc1fa9100e21b2be14139d18", + "1ecd4ac9f9621863890d5f786017185d6fef36d9a90002909562bd6a95eca2aa", + "1f4c8d73bf3a9ac57f9e2b43d427baf173b127d556762c817091febec2e02b83", + "1fa770f0d73df883caa5caf32e572447960a6f4a39cb919b7e5ef0ba5d8d4264", + "1fa94044a3bd20aff19e7c24bf541dfb9f85059bda026eee808d37fcf149bf33", + "1fd4988ce77729f824275c64a1736a759fd53bf3a9802c5fb4320f94889edac9", + "1fff8991748bca60d71af9cc9b2b3827ab3180ae705ebd62243c072721f8f094", + "203223582c0e7fca69776a87bd032ce7acdc2a2294698ff3a98d3b434f009347", + "20c786ab915d0509301693df90eae1ebf07cd0291a9238b3c107325d77b631f6", + "2106a3207e94eb944903c4fba0c66e02ab7f70389b6fd2655e6a934365c53a28", + "211894e79d6b7bc3345ad4e7706cd6c2e18970084a6b2e493b65bc8d59a5c727", + "2158667fab766e543795e7d22e30a62e6b02552493d9ae5d40764e724deb48ac", + "215af1c5f11ad99c5e43328eb8f930dd88cef637a51840abef90e51f91cbc371", + "215f4b47964817c247e28ca440642fd20f53f922013de4b009d90cde4c9d82cd", + "218195c1668cc7deb5f978b763d886317fec3a8ba0cc600b515d689330f959bc", + "21852b6057ee5fa7766c8f5498de18cb6a9b2ff18129ef78c35ccb43cb3a41d5", + "2193db36c24d0b68f9a562323f3455e60bac32e17e92c25108afeba03c0b8d83", + "21ae1c762c3fff7bc2ff0b1334fa52ec4290d14699b4ad762802e086427e6ea9", + "21bb9b4fcfff8d963d835e84fd8b0a3f0e3bb00da0c31b3ab99f7eda3b9d1879", + "21df15768401d6a240f92e69c17482210d8decca91922111b82544359047545e", + "2208bc75cce9607aa11acf9f64f8a37a4d2f1f8f9d3dcd59328f72e9f2f573c0", + "22243656978707a34f369f92c925fbe6e807c42c9200857afa1712beabec982c", + "223348e6289ba2185e679f0cda88bb6a8d7a8f5f81325fb7199f27c88ddd5192", + "2244cf8a582b801857e5074afda78acceda9fa1730380526087424fef042006a", + "2266cacc0b7ff3f0808e75c671cbfcaae5ac8f347d0c0c809779a3d3dbfa5fe0", + "226c86fbe1e36d2904f345f76cfaba39a6d71be0bd735335cb0c503e3b5ca3e5", + "227faa5ea2706a8fb5dc63022bd1338300485f9df8f7fdbf1733b170fc67397a", + "228107ee33b2c2e9e1e2aa7156e4cad7d4313d56cbcc7245c6b6f0b5ab6ee0df", + "229f12e885bb5e4ac5d830c3548c117169f2ddf5f11b730ccd852fb076c32ad6", + "22a71995d6c30b1889e0ca873a9de38aa25197eaba39464831d78c8132c2aebd", + "22c123126020665ad6ff2e62a8ac0395674089bcfd7a2a5e95879a0f3f42ba66", + "22f8bee7eb3aa2f15cc3a50fde768180b27cff6da04fd3ac602d0588349fccb9", + "2354b73626bcb881817c40eb612de6c984ceea635873d56767fd7d27cf72f337", + "2366155c840f1847f61784e44b578172919a4fb17e789c5e9993adff27f20ac1", + "2393fc9fe882416a98301134c8f1bc40959cbb0df4a788c18d6ba97227dab57f", + "239edc4db69e5c6c2ec386800ed2a16c1d14f40c5296dc5b474fe02f303e2932", + "23a19cdcdb11c480c19b5e826ffa9ae9a41f75356503c4c189b952e7e7229c10", + "23e7778305fd2fcf3d8f178ff8b0542c668ef7e7e7ef8c1a1719742ed4205899", + "23f90b26c36b6a00f084465c27149865452885e7b6741d29261be0963c5addc7", + "2405dad17d1d80f91da8f5654687782f23179182fa8ea95d0e7942f024d88ff2", + "240e6c1a1da07b1182d1f9c34e2585ca561d75e0c7d9c467e9f668638e3d4769", + "243ba124b326bfb985fc9db69b613aa893389059ec2b6d5083705fbf4996f0aa", + "244a86be38dfd9d932f23b9d27a9416b40b633a754cc528b81f7987672c3ccb0", + "2470bfaedd598e75adcd25e04758049ee6770a8f94a68950d5f90509fadc948f", + "2483f6c4d3c44d993895b6443f371d662e7e0d212307b47f3ab729db8d59a20a", + "248838a33696f03df582c2fdf2f6079a214d6ff05333caa5f95f138dc9af2b42", + "24ae887a86eb0636d75da5749f2d58725e00b258a152666689851b7ada661760", + "24aff802e6017c90e6244f7ded77aed1fcbb545953cb200d04972648eced96ea", + "24bde8b59e05b32bb2b477d9106f41f2782c9842a470e70bf08bb527d24132e3", + "25029d6aa49a43a7e8643cdbe81f4689c354b4913f90703da4256b6d93cbe73d", + "2584bdcca36c2871dd912988d49fe1f921aa8ad95ef0c60409c774bec286b379", + "259297aebc704d7d5569506071b5d478f9e3b879717b63b09ab00e4608e514fa", + "25938b7d578e370d915f65c0643c37264dc75c7c2c5a4e86211fc222ee78798f", + "25aaa260d7f61482d0dece1a6dc0ebddda72f625bd38bba89694a83b834d7a71", + "25d303833c68d194606fe985b8b39f09cdc60f97efec4e116be8787bb23f98fb", + "2605fcc76d52f90b98dfa94952e2a390477323091d8d0651c1227a952c2c191f", + "26273a075d3f2fa5551130d04d65fa7cdbec6f2c4770ba144eced4ae719cbffb", + "267564c04ba8e005d6b23639c895d35cb0591e0291a9e263cf5cbde05cfef1bc", + "267b0a39e499f0278a58eca2a6205c39a1503c08694d52754c5d8a487ce64c1d", + "26e4de7e4ae0c7104c8c6aedae934d81d97dfc89169ebf415a3b0219eb8fa9cf", + "26e71a515501d52e4c0fae7edfadaab7638fd7152ac373d88be078abeb2d001e", + "26fa8b7a8f9dbb96c9ea366cbbece69d993514b5ad6be8d8522c7f65a4fc2b4c", + "26fe2653ac2789f9ff3ce5dcf164b2c7f228b648e61805e3855ae9c8bf59526f", + "27427c19368afa942253ff0a473755aacfe1f98982af2f540963275b55a49d1d", + "2771ffdf18dfdfd65d02bf1b8bb36aa56570b790a173759cba7442958b7c8e59", + "27794ef63ff9fb7240d2bda2254462fb9094bdd72226a306977f6b7d82821b01", + "27c5d39a379b325588db6be3aae94fb88d53d4cb5159be340b12466159c224ad", + "27cebfb6abc8671e279fff311fab47dda886f886c26118a8a17132cc55ffd2d0", + "280c1a759d77de7a2eb914d964e2bbd88d32640e4fc9885ea5ad6e97b7926ecb", + "283124debec6334f87ecad97e22eb29412f665347953741d545a1362d50cf700", + "284292effe9290e071b49536e7676e25b1f2b51403501d067e35c4cfd930cb7d", + "285dc9617d8a64ad63c678ff0ff60dfd6c2c395a2084e2dd1b423511b99f452f", + "289e975996c5a4d8054e12f8cd26151ffa576b2fdcbc04ce39d0b8e3853ae395", + "28ce9db3376e8b1aadaf10eacf458f510bc957fcdcfedfd029bad2cb1b07bc73", + "28dca848a9bb9643b1b2d9a4a8f2f7a044c5a1994acb722baf9f46915260dcbd", + "28dd980e1372e6396aed2934d45aeeef28430910a7ed953e7c0152bf548e9c91", + "2921ea55f352623ff28dd31ac8b6126a8df490e5433d55abfbc4c3bc45d04047", + "2924d65dfedd8cff2e2d9db5f0f5f65c545597e7aef61c5eee9215374b1c97cd", + "292f2b9e562927a46924f407c14ef193343db63063425a32603740d594b02d7e", + "293f27e5ab477371a20a6fe6c65b647c259a6a98c3b813fd186b7a356bb63c54", + "29462899b56ca8dc69f31a04da863dfb0aa826f16e099bfc2d5978c938316a6b", + "2969c04005b15ee1f9eda734cb9eeaa861702af06deef784dcb9b8fd06f97282", + "2974f03b745944abd7ccd676a1f3750e8f16ae68c9c074bdc65e10b5623990f7", + "29a08c002de030c471f68e1e749966abc348ebb2ceb422612ad8c9ff3fdff258", + "29b3e33c27623bb4d5e4c858238a1d9fd530fd43dd87173f95456244872a5ec2", + "29b6050cc305b3bb5e1a40aba11155d89c2967f36eef823d4bf93474dc862356", + "29c8e5d1c664155ba776984de03294cdd6d46e2a2481599278cb12b9c042b93f", + "2a08bcf84f783530d696add19ea25a3fcc1226584bd2dc1ea0bd38a805fed8be", + "2a550c026e9ff65d75443586c017ace88635598a8faf177cb70fdec7b4761cb1", + "2a621087236e8404b3c0e45a264692011e1e30e8be64b0555a45885758aab52f", + "2a74d3e3f8eaea1675636298b1992c4b487c22866d70d483ea9d44873dc8f085", + "2a7965e1a56d537f2a2fc4bc7c9a7c4ef69a5cad08ad8411b497598b73fd9715", + "2a84af45206416f82ec91b76b8453b31a3ad892082d4ebaf0ba02647725db5d3", + "2ab5a2e4a8ad61ab47ed5ff6fa9b024652368e9fc35e4dfce3a004ba7169f077", + "2b2568383c501e9b66e98687135285a10376864a191834e8192a47db47ada0d0", + "2b2a0e3b2e9a54e95583639e8663921d0c0dc95ad6e1eab5776c4c85f3d3c519", + "2b3ebd6abe46a3e2d257c221c2521e4f2fb92b20c830f3370929106fdf462276", + "2b4a13516d94a909b64523740d2a7b42d960130275ed885b8476e5fe1a053ea8", + "2b519f4c71b588b9255394601a55032d4137587d9b4bc84ad87311564d4573e6", + "2b5fe3cbaa0557413f47e9904a66c0bb44c8e85dfb076eb68dcdc7195c587738", + "2b694c3919c53658c7115697287ec18707f91c178535e1b93a839ea7236674f6", + "2b6fc7dd39006b19ab1ad37d025d46563503e4d3b475dfc3944e4f30cc0edb9c", + "2b72a5c938c4ead2c17c0a0fa02caf08ffba9cd0c24c9f3dcc5af30e259af482", + "2b72e11390e9cb9502e0c03612bf3462bf0762abbc3c5ae2adfc7cee8e60ce57", + "2b738c4c7c9fb5a740202f324f24154482ea4a7b6f828354fa6daf638cd02e60", + "2b7a77bebff0ce005faa414e6e225e537967e9aca9e489090bd0c7c109c0c497", + "2bd3904c8bb216a19bed7b0a6f91e6fd12654312bad26d9b7e117d21a837f039", + "2bfdfb35553a281bbb7b5dbffa56a58f3d0c6c59d30d0597d4504feba177384e", + "2c69a32e259e132f0c383c6cc4674ab062850bbe9240405338f47b5392091a66", + "2cbb005c3d38bcd89685e0838a6145254e733833d7df151efeb9fce97f445636", + "2cbf2e5dbd2bff53014a66a4cbd93f69f92f886c5b4ee02353e4ba9c94e3826f", + "2cf03c960330a31b32f11d7bb56831aaf8e883f1e62b05cb3f4453fd80e40fbf", + "2d1adbbeeaa8811ef41d9bbd778507b51163ef3021fa808f767943b465b8ce16", + "2d2323f574328014c75e32fea3155f729fde3861ecf369ab6f337d310c90c1b1", + "2d2d9b13157d8f72554f0f880795745f9b6ff6461dd073d584e756caddc4cdf2", + "2d3a44eee75068d618b9bdd36b116e8e40ccbad1c16156b8bd0b06369ae81ee9", + "2da33335bb58839ce28b97fdecbf093fa0213f9d9d79801e050957efd4d4b0ce", + "2db74393909f5acfc2d5e00ffa646cc924b6ea3dabfabba9329b260a560dc354", + "2dc05339260685d20603763b5ca9b7ce8dfbe566b95e3c0979f5898dc39922b4", + "2dc306af4f727cee39b8ec93cdeede9dd9b08460753a2365f828c08333c8639b", + "2dd3adb43e3b61853b8a30448e12a24e24618eb60be5ac6a6e6310cd608eec63", + "2dddc0fa022e4dcd9424b09b16f89f0f61255a55b0748ab2a22890e87878a0bd", + "2defa16f507f490fb59bb70e071d85414ef07276dfa21b5317f8d06c4e3b1a73", + "2df21916ed241f489d69e97d370e4d6d6941211f36a925a3b86d7ed1c0fc1cec", + "2e053f8b5edaeb383ad6e73c2388a18a8f9a6d6869b237fac4325d5a2245a564", + "2e07bf8f9debd974bcd7d7f5721df3d08df6e4a9a3c891ed518cf783df0c9e3a", + "2e2ddfb1b4872831c5c5ee20abea9046923a7fddcc7462c975f79d71af030ad5", + "2e3166ed803467db2f1e7955b14d163daa016118a0240832622a3a89e31cc290", + "2e34ef045128d6d9abf7e7bbb3b7d655a355c932450e8c216830216a55f45bf6", + "2e4abffd42d065405d5c50a7e89b60b4635129858fd422bd8a5bc56debfa3d1f", + "2e7dc207e0ec7256ce878bee56d6eb7e89851dfb15739e987b5154450714819d", + "2e8b45b76496d343b60a3084f4c983916d31c5b7eed40d45909452d83f749012", + "2e992008ae3b92ac0e6c56a979e6e97376f6ab01ba1657b26eddc2f1b774eb4a", + "2e9e3a0dc28d9cc455e426bedca616fdfab0fcb4b07db71afc3a3cc55de065b7", + "2eb409f84a30521f4d270324a38a35757203f0aa3cbe84a002bb76d075c4ee95", + "2ecc79683b3b1ee11cf498155da25ab786ee469d99f7bc0348d55409a9e59f67", + "2f04d8c5bb5359ce1f9de574cc90c3d40d4e3623368871c574c3dc5419ce88d2", + "2f0a059d5a5edfb859aec1f0e51a86a767e10f8debc5af8b1d705bac6c820659", + "2f20d184ed355324c87d927888e73b46809427944c6c76926b221a1ff10fc4aa", + "2f74cccff6a3b3582d3d5c8b80bb2a826f8e8338d1baea38010dcd660f90b724", + "2fd6faf46ffe17059af06ec0c9ba806ced076ca905ac45a24879deed1c45afd2", + "301f4176d152f5ca05d3a3b212924e746ad4e088b3b8f3cb9b468e8c75c4eadf", + "304b4b93752f54c39c79148e2f4fe81ea04595e2ca127ee9162dba7a49c5be9b", + "306722a747e9b40790bb9dfc3277629b283df3204757190dfa3c9a77bb82b6c3", + "3067e3bef680d92ca05a1b10a1efaf2313dc1a700c292d07ac763b0717807e72", + "306ae438448f671a8b163290a896bea20caba868a7989a4115c607936c5a23db", + "3085ad2c20aa1751700c2a52e56a31582262e6d33490a2d863b49ad42eedf490", + "308d8a8b4f266866938940a106851d72fe3e6c09d9a246b1d339a7eaed04a3ad", + "308fd860e7b20a5003f598b1560f3d3d9ca6c1ed6ca172b94d89d8c0a2a81f2f", + "309094bdd0171c5d9013e448ee826e8683edc9caa3b05cdb830474e451b2967b", + "309ea352f1fa69f0db7582c33dae60a6e94af9b27e5b520c5d904f944fa68126", + "30a504f78e25c60f73e949b01851fa600f1748f14eb937da25d60e34f8914834", + "30ccbbd11c3360af33dc437ae8648c6f844ac6c6ad3d9a4bc066f5be76a23793", + "30e05421984dc87342b7a0c5e0f5ada02174ed62e5ddd94515fdcb0562a23a7c", + "30f332c1a28575705598386d0a16c103751abdfede5f2248d79bf7fc1b502292", + "30fe071a03f9190a7b9bf43e6848989931924b3ed57feac707a887d398225ace", + "31321c89060394789fa478b8e5e55158d305a2ecdc1d3e1736ad1fc2da960561", + "3155118baf245eb3638c6b00a0dac1335b2908925cc9ea929080acc31b1d4692", + "3156405fdd7823ec3d73009465ded9eacca77ce01ad522b2686e3f0a53afd15d", + "31904cf5060e463ae0df2858d4ffe8c5e6f88f1b4fc97583a9942f4e39461bc7", + "31c34949f3dd67ded3e6234af496cdc55c1601ffb2e2f9566e0901cbc42104a4", + "31f19b0f89f217f06f53495c011828494e379517edb22220d1e08076081d2fd8", + "31fa2d2738687e84aeb791d1da281c35e009d609352b79e938b035c718a47530", + "32188bec2e0424b77ea0f3633c8be658cc35a739de5842dced47bb4cf29b2e85", + "3226aef36f5280e1a5f7aa6921eed13bc6f959316ac2e0b64006c092639f7580", + "3282223e993dd22d7998813c953ad2266e639d3d9f861168eb41a0e6accb55c8", + "32bd77fac7231b667834724f6fc99fb379885d7c0b217fbd5f741a8e40904b96", + "32d78508833e9de23b051b65fac7099f8a8582b13576e65ddd950cdba06bb3db", + "32eacb2a02b9c5c57ac5384a2c6670c7c1f2164a9895398fcc42d78f3ef0cacf", + "331915366ec39333ff7c232f1838bf30be30e90aebc2e357e9d36d7024e1131b", + "33452c95909e9c0cdfa89c3cc04a7b27102859779a85dd966559b993eb94a7b1", + "3351793a8c5db83f2f1877a1cc8bb57f7d76f3e614c55dbd58e8489491e24af5", + "337ea7d073d3cd9be0463afe0eb37c755c15240f54d10ed971cdfd4f09618d8b", + "337f9f4695b3e8a6d6ef006ebf5b1cde828e91ea4237ecc5992d6291292ee7cf", + "33ac16e402f3f2824708a8187c19404f952acb49f1ee8b4bbe5312ad585fb131", + "33ad56c3d64bd864c3a927b12fb95ac99c6dee0e6abc244fbf4b61149b16be74", + "33b0a6867562a918e8702903b476fa7100a1b356b6613d2e9c25035e31d0791c", + "33bd5cc6473cc48632de8dd7f0ae4c6d99d772cd07e29e5bf3b37f4f9e142d66", + "33c5f3455ea8abbe73312f89915385361981af36a329a1a357c3d7274c2a1d2f", + "33c7e4b1a1be597198141e9f4506cb9c9e0955b01e6fa2103b428cd8d187f5b4", + "33e3575cb0a63713d10b724814597d6160c2549a3ac1947125c10dfc02d66286", + "33f483a510152f19c6c4220525d2227535ac5ceb9fca7483f95ec74122dc0b73", + "33fc9890af9071f76d49957ca4596b2255dac4564c3238d02f369030ee0acde1", + "34169e0b266acc6516cf0abecb99fbf15d96010e9937ccd43726762a9716ac8b", + "3421114015b80b306d6ff05cda48ef6742e935d3e9f1fbd4a79c6eb378665f10", + "3432acb18a5419f76a291dc59126ca2a71d0ded8c0c5e6da08d58ac040d5a264", + "347682ce1cb3ed264f55a6f66f7908b2a5b4aced184463812ad49a7a785bdde7", + "34b271c48e0c0b2b2b29ac13d525b4a4fba844419fb6b88cd0c1b78cfca8c3a4", + "34ccab9180bf8852ae598aae302da522f7e2366d54fde419f9e990b4abcf1722", + "34ccb47ee7a64b14318d84d641a7046684f71ec4d01aadc54c3203a1085ac817", + "34f3ead3029e94a2a25c940365600dc03afde42ba6da925938431506431d3585", + "3543fb202207aa6628c47078d7f21f7590defca0b9eb7af4ba12ed5aa445c4e5", + "355cff0ae44e80d74b235b926e1e414860320ad4a101ec70fa99f9cc2481b626", + "35905154d89fd0df546a823190c2754e6d51f79cd30024c9c1acb9f2c7eda1a4", + "35a7d85bb77a51ffe29fa15e956a970b54810c1f618adbd8547a4efba9df46e5", + "35da11f528bc82d5f9ebf9f5bb9bc25c766cff532216af0646b1d8f15afd2baf", + "3619b67cc0937e9f89964936c4e2ef1897f48ad30b83e4224812c1222882511d", + "36253466aa8a53fc51bb8fc037d21924520d59aa36259be54b53f8d8c67015e5", + "36714da17a776533a53fdb0fac7fc0b051a1bd0bf85320ee3bbb09ff68da5b9a", + "36bfdfc702c50ddc4995779357499347962e08280e108d1f67fc6377ee56e225", + "36c654f26c8c4b5e4db51a5c1f2213b8206a5e4c88d84d5dcec4a7268b42265f", + "36d2173dcb129d98a1fbb9a25ecff8613706c046e4ccf8b49c07d0f8043f6d9a", + "374d2651979eeadfad2699f5a557d7db7174073b84eec55c9704fc171d768e12", + "376cff84d5eaabcd84de92e8037c98ff298ac1d372866331c49efed4d8ad0e15", + "37b247ed57b009b125d27738b5bec0372247fb76cb1cf225c16dff94019aa8fb", + "37c6a25a9fefc754d6d25c419a6485068c7a1dc1af5d07280145cbf7e849fa8b", + "380d3341ffe5b7eb9332109706e251b68211796ad2714d403a50e8809939f4b0", + "382e7932bbe71318dee082bd946788f610fa948c114d3a3bdd763553fb10aa59", + "3837c28ddc9543e2beff4e98d311d4a7ad2e675019dcfa0f38bc54dbdd00e1fe", + "384b8311b0fd3f43ae9e3c803c42ce0b5ac4f23f5dc1c1785def1cd01f34e99f", + "38680b605ac4b6e46bda8a39889e50dd8df12cc82c303e31928a16cf3e310cc5", + "387200a627634514bb8321770d460b668c9516375226ff1b129c54e00e743267", + "389b17e5ec1ac64f3b51977b3ebf9279409572fbbd1cc5779e5986f10ea0eaf6", + "389c9edea933534cebf8dccb4b7e03b96ebac870ab1c83bac3193688e5596152", + "38adf5a5620c39b5b3cd62b452de47b44487adbfcd855ff75b37eaa331277cb0", + "38be75538254cfe4d5e535b5bfeb08d320e36d2fc09e9d3fe0dbc369a03c6f2e", + "38c5e9c98332a91abd7c6654227b1eabf1717bc4c48eae5ba54d4250a1b8faae", + "38d86b76bfad976d136d0a0a6ebf1220b57bb58fc6d0c6bb1161474740643564", + "38f011dfb9503dbb7d7c5547e05c955e684a5de2a8a2431a40cde7c23e1b30a1", + "38f2f6a1c0398f9a45a2c8c4efe37de7ac56d3031102eede259aba10d6ae3fc1", + "392d57bd5fbc504434787a303f8e1c56064fd33fbb145766cf7e7f1a0088ba1b", + "393003d60574390abc414e5bad9f145b5a4382b3a34606d19e0f2612fce42834", + "396066036fb59e21a23d3dcd772be7f1834dc79369b98db5aab76fac639557ce", + "39907f7d1ac19d644f06417cd795e3ebcccc940383f1bd078e1b5c839f79057b", + "39943f932a5282c29a1719b4bd14cb195599345a3df6889c657702b127783e38", + "399da7528776a819912236825acf69f6cdf7b3ab904d49f16e970f0aed4053db", + "39aaf3de02157ee50731d29a7af009637f923a488b32a28213893cde9d796982", + "39cb6cbd40abd3a3b830ff12a87b423447419a54aea2df12ae424db1928a3aea", + "39d727d8064649b6d18281804dcfcc9d26bd27ff747b7a74252c7bfa62c6c01f", + "39dc7dc27988f1ec49f46b721feaabc2b63126a44c585f447c54abf33be9616d", + "39e5e558b78627797fa5fc55aab54c3353435867baecad912dc8c54062da8458", + "3a5343a13cc9fc4fe5e37203776a45663a5e988c50955ba5ee1ef78bc4f77623", + "3a573bba76d4f43d6e64238fa21e38aac63f8a2c511db774ae118d40c509ca29", + "3a8e9ea5bc01a4d394a2b7a91bf89b579395bd802683f24914a6c822969a7417", + "3a9ad0d2857e8b0dfac3bd237dfceb08c1db01b79a0169a868b8d588a589d35b", + "3ae9e28b8f68bb9676cac4f93d99c9704ef82ed42794bb2a9aca59f52833ede3", + "3ae9e7b2eb9044e1be1281a28177b8dc4ea1b4eca35f88a11157bc8f68e257f5", + "3aead832e9e872bc3a1d9324fd90910c8a904255c92159b2cb3202241d1c50b5", + "3b12a56b380ae79ca6c9a061b7e7902544a9cf51eea1b627206e44334fb3fc10", + "3b16dd05c50b6534be4523f8f34e0778455ad90ace4091604f18356a14b8edac", + "3b1bca7064e1eb773d2a2a55e2151392dbe3a9a44ae12a14017d258edb8842ed", + "3b29e3d5850f9099ae05d5171456f0a735ea8f07c137492557a93f425ef7e7c9", + "3b3b58ffa394f68cd450f57e00369649704aac9e4e954fe2d0c89f7ae75b634b", + "3bb376999dc724ca9695e469e5db3da657c83841343a498132d5849bf8e1db46", + "3bd236b7cd8f3bf3461f6e6eedfeb97ef0b0fcadfbc4a42164eea6546bfe0c72", + "3bea6795b923ffc0354dd3e3c427b242af16597e2c327e0bfd164026f1cc8c8a", + "3bec1a06ba82fe284d099ae5f195379ed8b870dc62a4ffbed1a67437805e3eaf", + "3c0d4da7c7013e037a799b975326ceb560a905a9381dce02c6540d46404e6dcc", + "3c3768c794292f1cb9a8e23c221abd3cfb5483be5893be5302dda43c86dd70f7", + "3c529806ad8b1c37819898031c2e8ac6a27d47b259acb7cf1691f51689c86651", + "3c611a1bddfd25ae3b7170d90da70818ca5e5327cefe4454b6628da9a08a6db4", + "3c7cc070b7fe2fada929c8cf26fb697e3c58062a6fe3ca348e1a1eda76b37625", + "3cb6873e95612d0fb931a822a8b55afed47585c707ae3f6136a682b244ea0f9a", + "3ccefeaa917898a3cd386dfe4d65199796e1cfc8bb4a73985af3f1e4ce3a26a1", + "3cedc03408a8b3da093700161a478a062762373e3a952e36a2e11992c6e089e0", + "3d2ff9c749ff902f5bfba3fd3f7a269b5aa37487001a7318e50f09fc043bb09f", + "3d3102629f40f0f6fc575a5a87ae7d73b895a8960e8746b056fba7f368a2595b", + "3d32a74525e557244497297607dd36cbbf4777e0778888152e1603d600b26564", + "3d60580449a07508fa4162d6c188e21c4dcea4014ae77b7b4908ff786d9456ab", + "3d74f4db788aa8bfd1a108c3cf1bed0958332e64df508f002310fdd13fb57e1b", + "3d74fce3a56854f176b3c1660203d4424bd3557c09ca2c224a3403946da2cc94", + "3dc736a1700bbadd6eb95f52a70b1d397f49aca10d5c94a3fe6f9ce946fedad3", + "3de589be6e9c38b0b10243bd7c2bc3055d643f29c3944339db80ab405b3a1c32", + "3de99f1d97b00c3baa52658450c06e846b07ea78cbe8a3338b7be49b03fbc357", + "3dfd24ce8083825ac0413bd174d48661f4e7a666faa6d6f54592d49fc244ce81", + "3e18849ab3807d887c0f4fa62a2bbf6e76a41dea1ac7950ab5a1ba4cf96af931", + "3e77980932066d7cbe8263a65005bcc889b865516454120c1f48f94c2fef9dbc", + "3e960f77007ccc87387c15f7aafe210ae38438f8f6c8ff64551033770046c8e1", + "3eda1adf6b8f49d33323ae8e6bc96a1cca229571f5927a804a6b878fef00e3c1", + "3f0c251e7b67fb6858967a27754f29ad110c89c0b158c4f2e5a96c05fd81b888", + "3f420e76b844336a91049960db17157980dfa4d5916d0b814d78cddbf57c4f86", + "3f5d876571875f965c438a8a58c96b7f32234911546f2c1590b83a5198acaa3d", + "3f6e99c36c36dd12dacfbe23c7ac87082c419842dc12bec4f74829f764a8e6f9", + "3f8f5d84da330a6c7016ad72d234c174aaf723b45cbe5d3a9145e675c2575568", + "3faf0136d7affd01be02bf5a9698317ee8a6ff6f6a0ba75779670d18c66a38b4", + "3fd34303b59e12e06bd772fc1b9792580da8a50e4d69dbf78702fe7bec9b8c0b", + "3fe4f6a0da2df15f272d64d4f9ea68425ad01fa3f0f6ee452ab9cffc98c34b18", + "400a9724d93c3ba1b67e2e8cec7225516e819676e4733d50ed33edafcab0a707", + "4012b33f2ecf8d1baf925426438baf8632a4fdb1f52dd767a572496a457e5ee3", + "4018bc333ed65ada31d42f42e5e51b29c6e927157e58cd60f037c1f26c49599b", + "404ef49e9e6d25710b2c12929318e60fd4601328688785d65844608044de48bf", + "40c9410e7b8ade1cb3a6fa71487e22f7892b5f85da1fbd14f433fabbc48cb583", + "40ebe8d29f30b9acd3993e9d72f36d2511b3a81c8d50a25b6304d3ebc4504eb9", + "40f4b6329e19c0ad22fb492fc79edcd242a2183f9fa3aff583dfcff16beaeceb", + "4119243479f2a04dfa7843c8a9357b64a7bbc4bb9378d364ba66938fa29993b3", + "41582150bb4e6f0252ef9eb9b8d9ccdc6d9111291411798bdca84198a1d29e79", + "41602d2b2ae7b9d18ce632b67e369cba36d272d2b0f412cd30a481cfa0cd49d1", + "417be49aa3741eb0120073ae71337f11595f05d93525afe5dcdcec42bcf24df5", + "41beec4ea2210c90956299942b14f90f8c6c450035b05f56a7b5a8fdd54db85a", + "41dfdd7f8a8debeeb472a7afc5a6a55d1139a39985f07eab8effaa20a412af3d", + "42090d888f6cf0a9bb0956166fbf65d0d16b4ee84bfb7975afa821ad8a10a869", + "421469c5402db04f1276760314af84edb5f2c5ca93e0d77a31f31cebacf0f96e", + "422872dac53957684f5fb702d6e11b39f4108487bb848c0d26b7b4376cef69a5", + "423003117279ba621fa3481afa37e09c1966b3fa7e49608f95ba53db066c4bdd", + "427ee0af0d810082509973505dda1b7b1befe244be465f40b3e642047132adf6", + "42a0a9a7c4bb7b6e66ec7f68c0905d2a4554f50cd078ea3465b2c3e4b6a7e085", + "430794193b81c68d824dc36a195b9570fab05a4a0b5b0be3c5ae5698cc82b6d4", + "434ba4341c13fc1fc6d71d9eee95ea2a39c06771630b215effea797ced42e9df", + "43536f1ba3748225251c8d65c5a5ba97747caf695d420f85e4b62457c8a9db9d", + "4356382bf08cc0c2d865bf6a0e018cbb2b90415c5dd37f5207c5a874e7e1b0c8", + "435933e580253d372d941f3a6b5be5352dbcf61dcd2355007b7f8369c5dc3728", + "43977f363939808102fc0d611a5e27cc924308b2a5a88939a46be95452a0f070", + "43e04de95dfed0fbb2c29a1c7485d3d6c66cc66ea7ad695e830915f1358c3a4b", + "43f0895baa6de45c86e086854700cc2ab2841f2afe5911496244f45ab3818109", + "440e2bcaa12caf38dd81cbd60439f72067e1767478c6b77e3613f76edb11e8c8", + "441a8a6e8fdc88b18e5a5174eab8a9f9b5110d15a3c5329ba01b9ed9da185aa7", + "441b9b6bb84b083a33a31a436fb9dca500ec5d7ca0051a1b7813eb2684be624f", + "4441befd5fd6304f1a65bf2406aa25ad86a0f30ca7bd4f5cd4deae4d384f1ca2", + "448596ed1a8fb48786ed62faa09ff8a9cc1ddb31179cfc0ba5eb60ee00400112", + "4494dfcb21b68387a1e0da5394813e325547a389598b224ce8b10b7bdb7ec945", + "44c16b61c03f9214c769001b2444ea84acbc91960c125fec67b7596862bb4bf3", + "44c9620d8eeeff6ccccd98d825af7354ed4eb4400131e03b5ea04409113b39e6", + "44da5db5b8e8740a38435d426e4eb9f855beeafa03098faaebc33ccea9d0b744", + "45244adc20436d2f5a427b80f4f1c15a73d8579ea5d2dbfb4bfc72e5f997d115", + "4557c4cd811e0166e218978bdb775fb6ea299c9603e3c20de6afb68e24706d9e", + "456b221337cbc952c55988375ec557286728cd771ef9408f1c764a4073a1e8f2", + "457bffb709cb5ccf6977d714e33b256783469ec035f37c071395e36e4cf4debc", + "45893369e05b3cf063d77f451ff8a6417a1f97bd6b4838c8dec22bef43bac523", + "45989e843b597dec4790098482c0063a6acd65d1c01bba9e8131bc813df32ad6", + "45dbc8cf1ef6e1c9ffc1c8b79f3e7513ac92f1e46154a0dbb7edf3d03ee898cb", + "46382d94897e05a802f9482b1e10fa0ade6053254dcabfc957022abbe14e500f", + "464151651c3adc93f0644b3f9b94b0e98dbb04136e92735dbbfe83b18658074d", + "4696efbc5ce832da96a9ef1c6f515d0e944264dd070cba698e9e89323ca114ae", + "46cd422c309d4f4394c38d7bcae4ce458613793042ad31626cca4a303cde1a51", + "46e704548e226c14dea88e9ff1acaf4de243139614902fdf3f5997d9c13fb34f", + "4783f6f77bb4629c69d6822d64cb2da730348eaa8a136a24194d24d53b83f559", + "47c6f0de97d395643f179a9f3247d2768707b5964c336eee0765202a603e393c", + "47ed9d7edc63708dae6e1f25927c5231c86b6ba6717983516b994ed544bb2e8a", + "47fefb31541d1c4cce72568f220febded0b5c4ec858cc6c1147050d8878b8083", + "4801fda1d5dd623aa273a8369de75adfae8657152a36650b6e8002da433117c6", + "48126690b86a0aecf87d8e7251173c0020ff593b458133ae8b560d6d9f5370da", + "4828a99d5491c1b8f564b661ac866eef5afea099c7f7ad2d67924e2c735a004e", + "484af435548059ee71cf025a6b8895e8576d14575677ef293037221f0624cc09", + "4856458e1a9f38fa310eba2ca517513c74fdbd8d62cf29196f5656460f42e268", + "4871fb0f89facb7ca44af55e031344953f8ef45c7b7585984f6fdf9e5946fe74", + "48a465de4fa18a9eb99967acba19b560edf8a061be1926210d5e96c837f18b11", + "48c30858be8459db0ca75bf71f5121e2f0b6ad18a328108363184efb2fecb713", + "48fa46445e65a643e29937c69e0538384ba7f1feef2088a73a6c39990ba26549", + "49253c445ce10685f3fef55e41d69dab439b9c9c07ad1d7fa96bfcfbb343e1e8", + "49371386ec950ac3831816f28cb358301a13abbdb3c56f5fb42003916cd59796", + "4938be3383313071b1269b37d6459e6a87242f58492b02dbc44f335d1a83c5ca", + "496d3f9c41547b2fa460c2054fb7c381747424a75117b3e471d5f5ced0067c3c", + "4979a34d6cf19c97162debc3a657c79cb50dabd39c747623d71f2c520c93f3f2", + "497cdb2183fd3f04cc5a0c73dee40295e9bf0a15a9d91234b7bf617280cba693", + "49817e31b53b5d8068fdda4e2e63369d63d548bed45065c03e04df508046367c", + "49b173f5c0aaed30183ede6f3fe27ac6e96db679cb07d35802d9576bcd672d88", + "49d89c167a538636e2cd7a049b633cc8c7f28d0e86860f2c30b3dbd3de4c8c92", + "49db0c2118ffe8b7f171b29e2e464253a1b10c551ae05ab473386a050c8cbcb6", + "49e4287b6ee26db52e0b2ba0229cb0cf295899071855dc3f665150df4a689ced", + "4a207c0a8a8e584fa8a3df1ad6d6a7572faec70672511c3f9fed0b5d608eed50", + "4a42d21f25dc546223c6113b33a861a36837ebe67506cbe9b0c1b75c8722d517", + "4a5e398567c199f6d294a82583b892fb3b0d3b5c40fded337ee09b634132d9ee", + "4a64db3dc71ce8b6eca7ff5091a0277e2b80ad862567edf4210832e7010a5521", + "4a77ffbfbd81da55bd8c5706cdb05b1dd3120466e1ac2face12b1206e0805a6f", + "4a7f6bac80765327bb0a049c8fc3cfce86f18cae7c8a1c6a57cd7dabec934d52", + "4ab9fc7611e72acf1d3a55fd6ebeb87ffd5b1efedee167da17118bb5705faa3e", + "4b09b1b44158cf0cc3de2907ced21de6de81b27ee094308de5e889de05fd66a7", + "4b0cc785d418234ca91e5b634d702589cd328168efca29e8cca374b3f0806b0a", + "4b0de9893a759494abf37b80d8c8b14487ba7c1f4c2933e3684cf0522d3b2e96", + "4b17ff76e6f9846ad966ec619bbfecec4c15f6155b12d82b97d286994927448f", + "4b5befafa28e84c8bb80aec49d60255781ed80956037db55bce16eaf8dcd7587", + "4b6543e98732849e1e2ba3b993f7da9340803e9e91a1f5f4c030ca972f720c1c", + "4b65ae471b05d20e97ee83c2c32fdc74e406e3019d0e9361b025dd1c2bff3584", + "4b82bb99d206dbe881b684309528b9a233f7c01aa2de42ada61be3497547eadd", + "4bb7a372a1e1de11644233db31b82c7d11e50c0628e7c8f58b2e21f988d4333d", + "4bc0b35b5c19fe1e718251ebaa8fcc726b3d0bcaf4dcf4552ddaf4bd822f97ac", + "4bd9eb8a050cdd74cb9e1348106cc1978b8607f0d34304f7bb856d86f99deae4", + "4befd84c3ba8fea7a5c64dd302955250b0d3ec2e3db7dbb7a1c2d2123865f273", + "4bf3d35df938a7a2900e0f5cfd4a174643b3752b74b69f762a579433444cb9d5", + "4c1549462902d97cee0fafa319ff25c0acc9a9d8b0f0c2fc046123b53e16f107", + "4c32c99d4808fc50a5c7d89b8a1fa32f8dad7f0ff8e06e748a1e4be8199b375e", + "4c33528daaa8c4791b76b49410b4081aa6bdaf0ed2db5bc7dfcd25f31ebc6a06", + "4c8d5cdb0ed963ff600d8701fc929d22a60812685d3b69868d4f42c9fea1bf6b", + "4cd852ab8d9d6c1307b5d037a8c2cbca1b9f0f651b9a4a8d75869ec89910a4e2", + "4cdaf3234c626e8930f9660042f269725faab25c1d6eac7714cf28ada00149ca", + "4d2c2a2036be8d1337f8f878f270dff29bd50ee2cabe82f4ebae082de71208a6", + "4d607073270ea17f497483ffd6baa03ae8fd6e770bf0e0094fc5f3ad9b1b38eb", + "4d747008088d0d38ae9c13e763ea7e35a3fab5859e0169b563aa077c1f5652a0", + "4da2ccad66b0320dfe85a3a91f9552177cac3735266fe014d2037a5a67257796", + "4da980d9ac346c38717f4a7e2361c05f0a5cfab7ede3d665368b67a7bddda4d5", + "4db5086479c285af9c641e7f93dfc4d3a7ecded77e9e29e377b7a654640e87c6", + "4dc96a99b46cd33924f46cb16d8a924dee4bcda345895a6c522eeead47208432", + "4dd7e0eec3c93e06a0151da75caa5a8dd09225d92f887f216671c589cc2f27c4", + "4e06b20920cb157f51231aa2aaa4b05d984392e7e0feac54112621758cabc3cf", + "4e129f2f9c65a1628d79caceade78b6ea056984fcc2e274f68ab01aebfe2140a", + "4e199e45884bb64c5eea327a917cf04e6de8e285df60592eb2b4afaa33729c43", + "4e26a5a289e257ad3df1d62ef154be1e724e71f19bbdc4d0ef54b0b5eb18aa8d", + "4e379fc475c786d5b1be61d19965483d11829f54164b0010d4283a33a8f60845", + "4e3ed35ca52c2e01a7cdd729e7068c1c03a5dc841390d426b04070b750adf764", + "4e4872b564cc0e5f969c3249956cf8aabcc79b0438edd7b02c65391e915fb6d1", + "4e7ae1e39495741171f3aaf74402143b8cf31d863924de41c80a69bb6186b89a", + "4eae55f012e712d2e10100b46eb90c597c8a0ce62bd167ee2f286e161434e0de", + "4eb8937b123b44f09c2574a572e384c7511a3ea950eac45d4aace87c2f44c391", + "4ed094cb2bb666fd3da8853a36964758d91dfe66d1bdd347e0f7c8b8c718ea0d", + "4ed6453dd0eadf7321ebfe6d030fefe0bb7daf1e52cf195ce43db0ff2fa5f4ab", + "4ed9df471fa4c9d9b7d6c29eaf127e1743ef80322b706f10008711fbdfad772d", + "4f186bd841dbf7ddaaca22aa1552a1e191844e3b80866158fcc37236159b7531", + "4f3d06d0b1335a13cb225acca00e85b351b3a5637c9f84e70bf434d52be4e93b", + "4f42fbaea85d8740f2a250daa88a9ad7dae53a07ab210746cc1526de742265a9", + "4f4bb1a8cf80e2c0847873006a5ddecc6871ee627588f1afb032a56779eb2cbf", + "4f865a399d2d50af188ffeb535c333d2017387ebf230f0b13e0e741dd35a350c", + "4fe42b5a3e9d4c0f6a2be6eacdd1a0cf45affa7cbb687c50b143da1179fe9c18", + "4feb47d1ff7a9cb4022ec5fc0914c6fbce0dd9f1a1a7af70a36c7c83949ddda3", + "500325e34d1afe2d2b79d83e7ebcc7e4b1f140c8113bc29fe712cf9298392906", + "504d7971b807bdc6ff779b6c274db38dafda22ac1a2b7d50becee1b3b6a2dad5", + "508a89c82aa71579d982948d197a6967e9bc64f5ec0b224467c4cfed96b88785", + "50cc531f76a921df9b6e2eef827c5921580930c58434a2778b01083e55967446", + "510d7c189bbf686a248aa8172d9553649b5d82cc389ec42c7f1c95bc3af9cb7a", + "511b16f4f474db69694757b57b5805f47decbecabef68ad4050812a972718198", + "5121cfd3d97b25de8e92fa91c182282af8d11d6a9035fd522bd2c9cfa1c3c075", + "5147c91d7c07fa96291abe56b71abe33fe5c8260d37c54b7f4c2008e802fd36f", + "5169e603e9ff88c8b5c21b97f23a0a4f96ddcb40ba25754c332c7cb0edb1640c", + "51ecd39f98794af8a8842cd2d4b41bc9cf9117825ff9915efd474464b2e5faa7", + "51f34f7fceaa4c434accbd8bd8596ac00f4a92642a437f46750ef45e718b69db", + "51fc880d84d3d8acb797768c4ebebbae41bdfc303777b4e52ec7ad95c08ed39c", + "522748f349598f8a5b5af35825819ab6ee2456f91aeccf1083dd7f6353480edd", + "5266828a56701e05b3c0ebfe65103f9d484b036c59b4e1eb0bda970bd5cf63bd", + "527e94eb60c278adb854c2849e5e1d508b6ef13f651ecd7aac0916528851f8be", + "52aa00e24b3fe442d1dd5d535a6b8e2287ee8a5e2bd85b334c377bd26b7bfca6", + "52b2e38e8e13617822a9291ee6c5365b5626ebe01d4272f39f75a7c9335c848e", + "53129650eca54ff4f0a67bb3cd93ed89c64d5dc87d945e9e8ac09ce72f8650ac", + "53214dd5fd650a6d93dc2d3ec89c65cc7d4503cdf665e7ae2b2bfb4895fbb112", + "5326f5cb0bfc01d08b7c17fea34cbdddb4d1a4fa5d907435c15a747b0750af20", + "532f79785cd9091700d6a0ec6a83d5d023e71e56ce704f91e3f5d92ec12a9af6", + "5335fd622f89353e5f12da72d7f570a680ea20fde15a4d030cc3e682917f3e08", + "535c9f3b9205d8d835cdf757b11cd0fe848c1d8bac1c660daa0e710abfac44c3", + "5368d271fa15e996184438f729026a741ceafa7b702e2f5319a02ed4a7537096", + "536ca7234857569d044bce93cd8058dab7eab3b19f4c4a56f2ac347a811fb435", + "53d325afbbaf0a085c8f4dab65d0ad4f6f3e48ad455df4b92efae664e3ed30cb", + "53f7214b828aa5afaf391ef2c4802d1f8d0c782a1d9bb71ff86c023134cda878", + "543c4fd0c392645155f627609c7d430a4e4d06e576863c514b20e1a564381e21", + "54a84ebc0507a9a9e843dc592c639f3b43c7d8e94c6019d71029c2d8e15dd930", + "54cc4fd24af0cacded407bd4f3bc683e25a9e969d89a010d36a0dbd586a2fc3a", + "54dc85bf612eafb02c47ebf4d67017c8de63bb7ced7344de32de836ea5551797", + "550f94d77469ebf1c5b125a72fed8df79c24a28f669fd339d74aa22ddac9b5a6", + "5513af533ca18fd5629a153368b994ac91b0449602a26297479018ab1f926071", + "552e2500e0c22bfc971e5fae6456053307211e2511a4b3f4edf67248e9e60c7b", + "55a8869fa9dc2191f7aad76c572cd5fbb72b826f2aa1566f0ede2fa89ec2a987", + "55fec48f1e3e713fd61d8afafc86d0bf0bb63b88212d7a3524d42201e4363d47", + "560da590c9a2d82030a253d044f3622ecb732576a98175b89f80efcf56d029ef", + "56330f94ec1211784277ffe3d0e6eb9d8570375c1502885ac9d0353bbe9b92ca", + "564b347329fe24e1e6550303be041361ae32520e753adcb0fa3b70d24f42953a", + "565a04ee80c7f76dd537e36950a9760676142d67f41fd998fc8295a59495246a", + "566a331cdaa86a9e637b843b124b48b2e492baf6e9a0856458f6192486912efc", + "56a30765776309f19a9e9cdd1f6d9456a23a1363636db79a0cf650bb7e31c547", + "56a94d11f9dd24ddc273a377cee421a7041081158351923c2bcdf1c180d3eae2", + "56afdf35be6e9e568b089419fdc113e430ed702e03f627833150a9b0f8b359fc", + "56b07e2d192eb70cc5a1dd8717ea1d104734dc3bf0247a85ec610a35ca8f5461", + "56bc76794589d321520fbcf73773be5152d5026d6117d231d5b49aac792836bb", + "56d9a85095e43f325d13a495567d69055af77d9eb0564a401e32cb3bc89336a5", + "56fd2f352e422edf3479dfdd0d0730885bcdc6e8140a93fb29efa33ce2e1e847", + "5750d0299820d644772be4cdb8b00a3dd45729a401c65134cb2a76cab7e4fb2e", + "578c2107d5f26305eed742fdc8f309810093277f54a5d831cb9d62431ff3761b", + "57aeaa2185be958180717afd2d67c85c071eac34214811e3478263c4d376c152", + "580347991ea4f30e4f531a31f6c561c15bde59a529de296194e4a35c2460692c", + "5810e162ef494be2cf77abb454b744a4f104dfc64a8bf30d81d8e065b3a0d7ce", + "58146c983a0dadd6f68056f5d0b5b285c1296cea6662566ad16257119665a177", + "581e0f0f3b40618fd8ef5e2f13555c891e6ad26e779700fbc9e7ac7a74370e85", + "585214c1720d13b39d0c41df6a0222e8eba17741364312ff3dc9aa1dab87743b", + "585e3e07fc2ef6b1dcbc173166d50ab12c9d23ab5b1ac9b6d2755fce6f4873b9", + "58773571cee830e082df51f368c9a63a7f20d290046b62633fdd995ede99418b", + "587db8c7d804f20dea3cd62a9c01057cb3b49601e4589440d2017bd02d638cc6", + "588a1e302469a7df1104d8bb094ca75222ee8da70b0d7f5466871c8247366fcf", + "5890e575bfe1cbd1718521af938e9e6042bf386075a3f2d18fa875cafb6bb992", + "5896dc528d2b7eb93675b4b1cc555efee4ff2eeccd4c3dfc8d36bceb18cf0e6e", + "58bafb2f5bb8a3b1eb354ecb07e3453d00501c6b5baed70aaab8391c3f400a2a", + "58dd3c4e88e304f5b06967264379c278b8fc6f530fe74a4d3aeeb00507badb80", + "58e1f86fa5ce3520e7d8f58889ee0d0c51e0157b76f12db19c7380abae59ccb0", + "58fd614ae697249681124a8c8fb27476cd7ac07bcb9bb5bec862323be835e426", + "59097dc61bdc1cb165e5fe9ef02fa705888f8737f2832e5e5355c1308efb9e82", + "591dee517c139be11a013a596aab5e3d4319d2d056f49a8cd9eee65859af7c6f", + "594c5ef422269e28c70f7684ae1395e7912c61532d21daa7e6c1372555afe0ae", + "596fbf2cced5eaa06817fe6cf08e2573bc73576833d15c973378433a3530b2c6", + "5973d3dab148e37f2b2ff8453d4c0db021866575b4278e4c53e4df8a700ab04a", + "59827f3a93b7f8823af0a16a43d17644aef2f7086b6138074a2df2a6ce5e6a48", + "59c53eacce4fab10ac3a923492063f77745b6b08badcc44256d82f74886e9e3d", + "59e170ca4b004d729ec102c5c05d2ddc53d12c725e025d112037eeb9dd788f7b", + "5a5d038adcb164227b02c45d95d545b73dc5aaf5c29f34d2dbae3866d12182ff", + "5a5faac5b9dd300ae710f7742a8f72e16d8ec858b153f397e45af70731cb6d43", + "5a65be1f3c040bd2c6e2fb583752e5dad7987e11e451d59e690f90ef14dee23c", + "5a665b818961b4ca6b32bf841165108602b3e27012b836b0f85d7561852d6d56", + "5a9e2a164309d31ea17ff845570e206871eb9f4a062edca50fa000b56818d599", + "5abe82d5f6eb9f97fc34764a7867c2f48f6ba3bef63e0c0db0283cb233d970db", + "5b077baa508db040e7bf3405264cf235c7d95ae59f33c54ce4fdd41a2297badc", + "5b0803ecdd5ae32f9940911e057431a19da47f172667396a50231caee56138dc", + "5b09088a32a4d31b4fa61cd7be14176699d042b759c867afe3664b6d4a3c8d54", + "5b29442d8070ea3f34d021c2229abbb1203ce47102ccc375e500ee28380a4999", + "5b48e419b56f1e0ac888f78d01256d7d9792181d244f7cf160bc6b33b86fa0ec", + "5b5d08a56b25f363a8b8eb54f1b66af6fd8d5e1d37ff19b56f08dde5868ba810", + "5b739a71f1b3b09bbf3d54543b662b0ea4eb22fe35809aa0ac14f2fdc343b6cd", + "5b8782b98afbfb414e51f003c063aca6dcc358ce27695ea52a0cb111b7ee3863", + "5b8ff653c10153620887a41370f6f1f7a862d06f9ff05002391c76ae0c7a7287", + "5b98921008beb016b9637d68f80eb8ad495391b3f2dbaa77ad64a705c315cbbf", + "5bc1788d1e6b4a285d30916d2f3846a50e57f7ca7a018eedebd0cfe875df9aa0", + "5c03f2b3a7c6f501261291db173abefc3d9d73a4094af49d78fcf6b76e505c5d", + "5c18ab98cb1bc02ac71d430fb9236c4b59f142d9d3ed4edac095b38ad1e6ce6f", + "5c24a8958cb0c33ff7617e4fd9770cb9e5b1f0d9932017bf0b7e9c7322516d53", + "5c25781bf806586340bf9b17c1f1dfc990e6dd21cadddd271448866f44d1c684", + "5c3b624fba16f6088eddcb0c9b9422eaa1aab3175b5e89732601d788440d1c16", + "5c4ed7d029b580c166984451d8c1fd42460bbba0a224c77b4e69f70cf1428a69", + "5c59eb61bb8b5953a21997aba7dd19476888f96424d4aa3e9f2eee253bfe9b25", + "5c7c610d7cc5465f9cbe4714c35ec928553493d0a16c9a63a0e57f8e4878fc6c", + "5c8b92dda4de3de4816c98e1d58444dc30f0e79493330a36d857be85bc4d3b79", + "5cb6260be3c98ae3ea566ba38130c44a50bf48304db36059d4c05f345458f0fb", + "5cd356062f8e1ada45e2ab1fed44d13c8b551331f1d485e4b6860466f78a8372", + "5cd993f6b096c7825ff9dac935488ce03f8dad2b060d67bf2505f22491292f5c", + "5cda3ced22b8cbf4998a18a9577a5ca62a34e37b38d118367d9d11d2d7d9b3b5", + "5d11328833f680ff28da140865138c3ba6383e972a7d95425a7f199a499017ac", + "5d13e695115cf2fd4c780da5bd73aa6774b7ceb334ebcd1bcede0e7716870565", + "5d1dc41467ed9f448b1dc703c862c1f6d51ed29037aef1e74964b8341f96034a", + "5d349e1d596ad3e2b74a6bbf355fdc4259476cd6977ed29600cf27ae50b32f2b", + "5d4597dfd3afff704c7d40fd84648673786f0d4a6776311ccac43be8832138f2", + "5d5c5ed5ef38e292d47d4e9f0849a6a550f03eaa18bf6f50d757cffe8d5a306e", + "5d8923c6ca6ca759ca9b471a2e08fc501c0ca65a8f38fc4e08fdbbf9b1c37702", + "5d90f1c2ea5e0973c16286af4fee863e69bd2c5dabf11e68efa4b30d15a38ba9", + "5daee84cb06225feda64408ceeac2f96556185059251680e18b092e894f6bf16", + "5dafd93880dfb3436f97a5eea1ecaa129a2b5b0a2765fd61b7ad752f4f45a8ef", + "5db84722f01e750446f95efb0f7b1e3e5be4e972df48183ef6449dbc33831e54", + "5dc3312ffcdd9b0392d350a016568b35465c467f4a8af3f8df8a01bd8d0b1a99", + "5dc49e102663cc053f4489ca3edcf207447454b65efbdeaa94f23a019e50b17d", + "5dc9d6774b9e06240c24fea6ac9ad4ab0a1c0fdae3b9236eb445bf7e05eb303c", + "5e58ca69ef681e85b55a5ee44016c5a277003ffb6fb80d76e1924d7911355b36", + "5eb24ff067b2d15897c338e09f972b4278a6a86b9167986594fcbfa1c4bc9c5a", + "5ebf7cc08cdfafc0c31b22d4f9c6765865ff9e63d72971016abc8c4dcec766e8", + "5ec3e6a888e606119f06059312103a6c70d1ddd2765dc399093457245bd07189", + "5eded2b78b90beb97074676bf47a1f3243b84ecd855a25aabc030a5a81102b55", + "5ee5a0f04e5de073e8b2e1ea283e8811305fb610276f297c8280b061019e3738", + "5eec32ec10e48bfc7f746c5560512e381d9c44eede3873a4d40f2854efd57476", + "5f9c8bf3d60bc1a4b796feff5e5340f352d4b70eeee37d3acd503fade03e1acf", + "5fde1897d8bbba084ff2a87e8fc4c103418060c56244f796f8b102d3ef6b70ae", + "5ff67526f77dbeabeb3eb9af58aae32ed2bd0af1c859deb6830c92b283cb720a", + "60072d766b1383035d005e1516a319b9c435c4e3ce59b69aeb5342f07dc380bd", + "604d392d36d6214c60fba0281725ef2125e573177d58d54d6a6f8183b73c3f66", + "60537dcabb3d7bdc1e312753525dd233fa8a7745c77ee26c77241a7886d0209c", + "60589d3cafbed181eecd4cc288e853fca6bcff092197b3b42995411ac6e3a2d2", + "6059c8c920478b870ae974d4422cba0ca49af2bdbda2fbbf10838cdcfa5c0387", + "60709160950e949aaf4b06a81bb046990a4a2699414d23ff7390998bf98867d7", + "60f873f907ee5d4f835f6bf079ea80950c32b0b885402692574ffa1bf340a0bd", + "612acbf37c5e1db127331fb6a7a9faa48b65756a36cccb51315bdd2663362227", + "6137250e31cccb6df9aef545f0af9e6f425d29f57df8c0bd59adde97f2a35066", + "6162e12b54d9c1dd7d764333543b0c1ffcf5660c585575595c04636beadcbd55", + "6171a6e20e268325d4a4a3fcf435febf5bd2572239d5e36dbc1ac4b091e86dc6", + "61ae93a705ef2553ecaf50ee273b55ca0e859f6be1b5501cc15ec1f78750f7f5", + "61bffd76ee2fc12fe529aed3fea7b656429f720be71e72d304b05ad787ffa1aa", + "61ecf85f0dcb3ce79c049cbf4246d31c3e9ee4fd97924b4d9786ae87ca6b6a6e", + "61f9efcef6f5be426991f087dd75332705677487880af8042569287517516522", + "620ade8ce0fc6e5781a57ab85ae405c9f62ea0360ec2a02d441ab4b6fb9f39d0", + "621d7c8e6248697e85db92cb715da91464290c722f306015957949842a1c820b", + "624d16569d89bc9da4b8f535ce8ed165b63ec7b7a290e116bdaaff37d0a09f2d", + "62a21d830ac26c364928f82f927d53233bfc87386ff2e97163fb981bdac82a0d", + "62db198b6f4c3f1a462a174b3d5d5a6078bc2dc15790360bdd7116e0f1e7f31c", + "62f9affb54a325d03ea5e8311b6fa62e04263251010f8f1e0545824b5250fc9e", + "62fbc7ac4cabfc232dfe0d8a78c55fc656073e2f6642396082245c9c98dd4998", + "63157ebe4e96307820a0d937d2a37bae3f9c14cca6cb651150156ec5c6d36448", + "6335cb3ff64c3cf8541dab770520a7cbcb6e3c8bec899c10393eefa9814f6ac6", + "6363f95dace9764ebb6dd5e4cd446dfe73be55be5f9e0b063ab5406af571075b", + "637340653f23ca873db36e16f9e98d6a1a1d7bd96baa07cf2afcad5123c182ee", + "637ccf26a58ec061ad2ee59d1f87c95f71f0bf8fa029b7ee9797b2a909c36ccf", + "63e61f2694503b1cd61ab7dc4c33759d884c3137d7e4f2a61709c562a01790cc", + "64642476b64edf3a64d69b2bf3231a5175d46c8a7db906ccb95271a787a799e1", + "646c682297f626e5d120f89acc519b581e8778b3b1dc6ffe16476a9835f3bcf0", + "6489ad49812acdb7da4187f73f716b78e1cd0b589e05fac85e5c75b1ec5cb1f1", + "6490ba8c17967e7a7203bce09753747dbdc0480c3f827fd4db746591b610a1d6", + "64d4e42816228f1b393c4571a5ab6e78acdc6e1a207df3bb942305c9de8d1c61", + "650b7cd2d1d95428a6e9ccf0dd9a40a72f1c636182446dce691c3169a007a87c", + "651182a33f678ada4b45eb1079ef7e8e105ae14b5bad8cb3952539de77cd7d05", + "6530adeb56d3e3fff647f3ee8d34065bed5ae533ea00772bce081e753c861291", + "655231a11c2e677666b6d7ba200414a8446e912ca60e5924b9ce1c6f8998f5d7", + "656f782ba203277a60d4a4c5a65a83feb7b139b96c5d5b0ecd132f21fb365f8a", + "65714992f01962d6e80f7afbd4c8e6332b419e436e0cd5d985958d03a754d787", + "6578e518dddb95d215dd3a7dbd1577616364d9359162715ab59552d3ec77d308", + "658b002d9661970ccaf80be9e81500d4a55990b813b813fa0846edece61b92b3", + "65a71290538408a4d8e1f694b3148e0f09b74528fe161cfe3f2778d148f23b26", + "65b2ebfcd8434bc57234573a03281c5ea42211945505d4bc7926620e7c037883", + "65f741be093e64aa33aea625d6cb2a0da9d0f66794d345d4d36a64c194f88a89", + "66106dad07085d8716cd3a57c38719985cfee9d7da7b9e75d42d653fde524a29", + "66278a5f3ea6f31ed36c22b8fffa8d927eda8d7f17f75bdcdcb3e476a900b092", + "666560495c0db35d8963004e3610db8147cba9b15b48e2b5298340a7a82411eb", + "669c686bd276c775b5e02b1268da9fb8d07c792351d0016576732ee0c9bbe7be", + "670869367ca4532f46d9214993c0f31081562026087e6d64941a9c4a7901875b", + "672bcec3d5308dc7eb9a921e5ef64fecce84cb150510afea304e61326daca37b", + "674b60cbcefefa4501172470e2073dbc4946967875c5451bbb39242b91c6ce02", + "675b54560835499f03023d08441c0ebd1e0d08f93c67df4d03476181c60717e2", + "675ee443873b67c1a0d4acf4fa7d50e28b2fb70531bf3af261d528478e59c0a2", + "677a7bf5d7402745cef77d25c358406a21a994b948b84b9627004b3eae0ff40a", + "67ae8e470a9c09f61434cfe21c78698058e3d595b0f68b7edb31c359c990f1c4", + "67b7367d7b14561a50011d56a056e3b8e4a46f32d6e44f48f335474600119352", + "68492785665af90395b524536599ba8101495f429eeed201c267d1a9486b8765", + "6873449ea7587bd681061ea00fb7a11653cc4d9611d6544a074bd95e1bd04434", + "68738cbc5f45b159196c3cb22db18e4c8daf229c4980efcb501427372dff8f66", + "6874e93c2da9e538c7b3817ab71c77b8523fc4ac9d4abc4ef62b46fca1afb8e5", + "688a3beeed114db0cc09136ff3e1f826a4c0009eeb0a5dba639d70c5b8479d10", + "68a2fe7ed3dbacfc51faf72da293adfba3d7aa87c52353002c82da7f71ad400b", + "68af249b1ba166e2336cd9e42a28de6d46fe383904832c4d6537e7352321d073", + "68b27de105169ec72e0d2b178ba1eb0c3fc0a98b4a7995b2b3e517c32a96c4bd", + "68db72e7e93af0434222b65801ab46dce3761afe35c0a144fc7954013456918d", + "68dfd709de674c13150cc384e52feef725a9bf0783546e621a96fbaa2bff3dfd", + "68e32d09a1873503fcd8c4473df014183b75fd59524c232428413f0e7c26069c", + "69247df2ed55b582d2560bfb3db2e758123d594cd88f2dba1e0fc4bfc33b1d29", + "69249f79f5aaaeac200aabe1636937412dfe678a99cd79bd6de302a2c889fb94", + "696e2f5c8d58f45c1879a88969d911893c391ec6cc625afbc119c5d31c1137ad", + "697e0f9df442d5d8ef90a2bdb81ddf81ecadbb8870c89f7021d46763fd8aef76", + "69a3819da4c4f0c147398fb5b7da7d804192caf89f06956ee2348b2ae023d499", + "69acfb9fbe3cd486ef3b93739328d90eb35b2431b15e512b081e8eae896d1e08", + "69ba2a070b08ab1d9e2b171e5ff96da782a6b2abe6d78847e82441fd8f6e48b3", + "69dad4799c84f69e502f17319717729db7b879694541dd198f51c79060f2b2f9", + "6a01220adc0cb957e87d4781fd8dd89b108d1e507b27296ad28773a0840567f9", + "6a0e6e4014dba325b39185dbf1a7a08e2cd10ea7c02bae337c795d68fa6ffafe", + "6a2a3bf1cda8da3cfd63ae6e9f6fbb55fd2ec6385749e7ca09f27edcede57584", + "6a2e45123f24f5b2a79c78a7eb96ab498b77a0247d3cb1d3a1086ba6c026e1ec", + "6a79de837ad6b4d5b78063a389d5a30efc754182c9a450f685bbab55fff0ebf5", + "6a7cea629eb5f5f3bc04f856a20ad4252f2ff11364e698f2bccc18891c9c0ff3", + "6aa8fa03e2650e4784898ef4e27a212426010d0cca9f9cea722b5a9d050197dc", + "6ac918e60536affa2c356c46bc2689ba51862f52e0c4da168f637655fd085de3", + "6acda15ef961735cc7d69ca607a4e11cb962c535ad6adab5bc2953ff51d6e9ef", + "6b166244dfacb745cd227432fd643b7259f9cddc517b05245f62ea70a95cf934", + "6b228ebec266d57dcaa49b58b1db767784f6486d45ee95dc131450a0479c9afe", + "6b27ba831dac670539efbe9131002e9213375a9a26af267c308e5f855c2486bb", + "6b4cf277783c75898a2559ae6e129f1a5af61eea9943b9d6a7d1e2248194141d", + "6b5c7b0afa8e4b320afd0b89e313c9b5fa62fa33bdeb7cdefc1673f9c4acfd4a", + "6b79946dea25c3685d5237e236c8931782449933fce6917c0c7204d6d7133fe6", + "6b937b0554365fa001d5016d0af872c629c510ab04a6e9e8580bccaae07047ce", + "6b9d661d4eb6199238cc5cf77affbd3616aec0b9a794418f256a14003b87c11a", + "6bcf7405aaf5b98665c658334cd647d60106a8b76717466a9c504d28cbc31c61", + "6bd312d9d400813835b84759667398a5c0b8fc2e05b5461da95fd5babc3ee785", + "6bdc16be5569ecceab477b89fe2586151da5c490333f5e0a1f596c4bf83f6b39", + "6c0b4e79c200c629e640557990dd39fd1b5fbf70f4ae153c137d6f35a563bca2", + "6c2f06a98f5f92eda14193ff996f38782ebae82f94c856ccfb148c4c5bf5f7a0", + "6c7fa58045d8afeadadb92380fa79952b72c66c3ca4a199ffd05b405009fe5e1", + "6c870298d4b3b9a594029a58de126c7d3912bd4c8aa8aa0956f96e0c94214237", + "6c93ceb690805df915a0d11307c3e898862ac2a8e2f03545d9f9f31333d97b9d", + "6c9d4b7e0d99d0168eda469e87ddcf684c3ff33d2a120784b3b6669089ff107e", + "6ccc1357aa0f377fa2486a55ec540541ef6f9143b4215c14f2533188603b0fd0", + "6cd5081e4647c16bfaa05928de0b79d48228f7163b2c097a78b43258081f0489", + "6cdd98c3d0ee8d4a939eee49d26bbc9744fc9e22e88563d636ad5b1d4572b211", + "6ce8a8829a4467d50859e5d48868ef249c31b4281ecface940b240804498a9cc", + "6cec9c740ba668e1bc5979a2f406be9d4326eaa440b49583caff5fff16a4a974", + "6d024da7f213afb6dddd051098fce2e2fc219009d7621833e4e0bfb3d459e93a", + "6d3a64c3c7d6bdcd11e121de5b26f8bb5b8ec0e16dd8e0a9614abb6279689583", + "6d3d79a21ed625cf8c2bd6fa5233f3b5eb112cea3b896a466208ff48db23837d", + "6d8429ee4d3409cf09dd4c1e1ee8d17ec622f3f65f2d04d2d5b46d3a1e19ce10", + "6d8b1d53d160411812b2e63ed22f7fab8a2865cc200ec6dd7a103a121405c5a9", + "6db6f17d30a1e7d929b1574447e0a58ae84fb52e785dc8ffeef72988e7b9918c", + "6de68bf5dbfc8d0a38db7653c7b28a6172424c2f8bb2e31a5318bd0636a34832", + "6e46fa4e2fa1ecc23219527e1ee220eb785973a0f8b404956297d5d6594a3847", + "6ec6d1488d2dade7f32eb4be6868b7ce41161dc4b88344dfb788047806eb3fcc", + "6f3203130b9b3bd29b81708bfc5fcaf3d48926c9fdf3e98eb330c02a99d4d702", + "6f6226cd7173c25f9283a26873c340a53ed94d4a734a6c38366fff4c82cbe894", + "6f8a46c31c1aceac05f0039bf6e9a36b209475ef18a584f1cb8d5e3bda48094d", + "6fb212de4b5bef6fc983bddb206fcb4978547681a9542de68b5abd7f51546fd7", + "6fd4ed214af38642cc6bb474abd126841f82ca4516af7d676fc525f3107eb4de", + "7005b0152b7e62e70a4ec3548246bc1b1d5729feb4958c2fd3688c137f69893a", + "70068c35f76f137bcbffaf69bfcf8103099b381ee31719b508515abb965fd586", + "701bec823b7e553bae03ac497ebec8a40592fea30cca8d91cf9efad75e6ad7ef", + "7064f9086f3605fd49bff512a1172dfbd0ceaf58a88410d163a8cd1a0ad34f94", + "7072448eee55c0872253c8a40a5bfdc2c36b8f7b1be17bebe825468540882db9", + "707966f3f0d9b4a67d3379ecf693b83b2a1e1c217358494af3504b95c137fe17", + "709835aa30c7d0a66f14b43475d8db201bf2bd39510d9ebdfad8848b84fc42cf", + "70ab03aa506b299df87e64cd28986c66d4a8dd1afa44066323e9cc1d9f89a6bb", + "70bcf0249de578d517dcd2d9ce720351b3aebc2b3408f92ffc31c3f264797cb1", + "70e8eb1a392cd7468a49fe73f3aaa99481ab7019d354ff555dbfa1901298e9d7", + "70fe15efe7ac3eafb95ce5f3f89daeaf76a65535fb98d50b60a029e4484de29f", + "71024b56d0f6a22d5fe9e7acd8ee1e7ec9a651721f305a2c7f3e4d5849781f5c", + "710f89d19eea86719bec11c771e48def42e5fae229de5f1f094639beb3c85efd", + "713f861df41f12f45b212350544849e022cd8665397f31300942f1ef32f1bc18", + "716c7dcc84d0813272c889fa45fdc626ef28b96df90f083f0f95e51b5909b7c3", + "7170b2876d55cff7bbbadde1cf60918fa82c96b47a419d0dddfe68656bb15f7e", + "71ccfa1efe119f4b3400c25f08c0cb253d6e84a3034595291688d0390362e7ef", + "71d64e8f87b796850cfde842ddfcc5397530526668b4758e95e3ff352ffb808a", + "7208d392e8466c55bd4db2b793267d4f537812981de57808548a24d575c00802", + "7241dc8b47fc985afcdf062800aaa0e4b3288f68ae23e6420d4174936ce1ffc5", + "725960adbfacfff83b82b7df54c856df7d95600c14505b43941eb4a624f2d833", + "7261079205d5dfa4e29d586c9b6d03ca1ee3e2225ef85043a4a4eaf8583df9cc", + "728722ded8b3e23f89f3fe1884bdbf69cb04c6f5b843bb307b4a62e2a2964762", + "729dfd823bb3d1363bbbe0f6beea353b4b9cdf5e47c3b6af17ffc74b854a2d88", + "72aabf4eaf51267a223ceecc011f0e24ba882cb2833bfb09f99d7000e4ce2935", + "72d78fd5e3c79370db8423179341cd9bba72ff6bd98a54b2a915a698196bef03", + "72e1b778da03bbf91d47c4c0fdb930d7ad38f76cf0490408e8b4df6eedac1ea6", + "730722ba5721418ddd857592947efd2f0e6f8166eaa20bf9cbab54a8f4b0a3cc", + "73546fed85b1e74894df9b8f32aa7bd56c478286997a8c92b08c67af0e5335f8", + "7355830e37a3f3593f8f485125738b6e7caaf9813042a1aa65a78148f98393e6", + "739d4dc3543c4be6ec96158cd1f81024d14489153b712afb0a7baa9a136af44c", + "73e3407bdb5f83b252fcf7c2ccd03c3a15690c91e77ab9a390e9d6e133770587", + "73f1a153831e1cab13cc9cb150325007b4b7ed7d147a5aef39665059d0d87327", + "73fb5ecd247e8ee18252483813c02d62106af8bc146cd4e3bd3a321f20cc866a", + "7461c878ab37114c8fa861bede453b9f180844dd0024a1031f8d0d5d4350be43", + "7485d46799828ad55580d2370456706f56e6b531b241d1c09d6e60fe61680738", + "7493553d2dbed8c03679691050369d2cbd955cfeb2e09469b115893f92f81f1e", + "74b8a267ac18fcae42bfc59b17bbfc54cb50efbc4983b4fc0de89fdbec45f26b", + "750b82983cca089581424966a9da92809d4739ff8a84fc81448b6c2144f0c151", + "750f7f9b8560ab8aca11e03d8825159f4a02c97263c4b164cc6b6cf8c627a367", + "7512074a62a9ea08ed848461d4b771a087bf6d0806d9d8e9de9f109de0b62713", + "754eeedfe7b3dcb1fb680ffd21ca550bea5bc1eed3c387b5889d69c739bf82dc", + "7550a0b2274ac6eadda85f132246d97cfdd1fbc17a8c60247c93e8a7d6d42352", + "7552868610e7a0492a47d7bb17f915b243f75b08fafc0ec2122db8ca25d02ba1", + "7574a2bed3690cf7b377627edac03b504ff654934a0e4df77254cab0678b3da3", + "758fbeca0e118842f28a52b4c62c1fbb9828f2e0a7c78c41db50bdc603bb82db", + "75a4c50f6b7b5f9416c27fabfc24bfce47505c2507c9e5c40689f1cf8edbe796", + "75b0312a7fe659ec2619fc0a00188c1c25648195d6e7d9a6f7064d3accd1b897", + "75b3017f16c079d88f1448c1f4b9665aeff8c770ee0174a59a6dac4af0945054", + "75c9c4a1f7af09bd21062dfb98f95b17143fb282d7ecc42f082167e36209e5b0", + "75dae78bf5c4fcc772bd1ae3d1e61f73ca0fd329342829d93701ecdb05cbe776", + "75f25b711155c22abd881f62cbdbe0bfe8236697ac644ba420b88a50799d6682", + "75f96d7235913b25c9433b1a0ec68104bc222192d7a8813010730f2a0659b3c6", + "7622d99087890010662d5c0ded17c8d5a71587f4e39fa1d389c45a24b04cbed9", + "762512c3c71425a88180aeb09010834265222f4af4988ca83e9d4b453058e003", + "763be34aaf034de4d21129b44404ebdc6f95c5e5aa07140a0215b3a831c0c975", + "7658c48e7502d6d1d0b8fbffe97f4df627e7974f75cdbb1b9ab6a31495a94ea2", + "765a03507d35544d425b3d04d0b5f8af819630c18ec83638bd7ca85fab92f351", + "765ea782be299b543c8481425b1da923aaa737acf2b736c47584089744724891", + "766672d016c0c6eb36997037b67867ab2c694726af1d88c375228194dfb90a87", + "767a47a19834085dd8e10c9d3d66ac5964793f8a9513a35b2dde4d1429df7a48", + "76a179bb27b3e317f8d4e66765cf407ff74fe9839e80bbd0d6de766e8bc90823", + "76d19a21140ea79bc1b758ce772ade48566e5a967ba98fc6319541f9e4cab3c5", + "7703f063aec2da8a8985f6e719c2a68d00f1ac73cb75800e295741fa4d40ea37", + "770e2608d147738abd66ba9367f4d31123a567da4e4237c08a58d1b357f6ba26", + "774be2ef21c84a0f0b67221f61609824f9e43ae746ceaa6ffdd72ddd8da88c49", + "774ec128960025ac079c5cc7860ba8866b328c0a910e4ee97d9cbc3948293a74", + "7771b23e1a872781d3c859508c8f81f5f0145aadcfd51d4937479e8e767e7326", + "777e1e2ad048ec24134ae9e9a2cc09bdb9e65db4a33aaf26a5571378fb576c86", + "779ed248c556b5bf3b3c773212e4390243e14167ee6adacbdc126e00057141df", + "77d7f1daec219ca893d2724e410ba5179573dffd22ac21b1ab2a67fc65159508", + "77f2ffd9c903d1cb59f3bb79ddb89b412ca1ee3529d555074ed4cd2e575d330f", + "77f65063c77913e76134b4c37ff835f373526c410e5b367c54fe4d8d613e473a", + "781ac47130a2dc741d2bec0037bbbf1faff9e21ed424934baacace67e827e599", + "7834f835213ed6839d9b2abd93cf8ac01ebe5bdca61c9855dcdb515765e61e2e", + "7841c519c3dac274e9c8c618fa6201314f47dd892aa879e0f41619ece8f58376", + "7847d17d7e5ba94d471bb710e54d9e653488c9753dd1d12661e44c04cb4793e7", + "78c796b942fe136b1056c697af7179df8ab8dd78e2c2bb385d9774710d9816e6", + "78c9fec65bf1ce348f0f3a33687b29f95d7056f34c580a0e4292702501694c01", + "78d4919ce6c9044423da92ac281f463a4de59ff03f9a54382fef3465b64f2aa1", + "78e0d7e8b4413c0759ce0b6c2bbb6a132b7ff8aed71e0ec46ca0d3669080c781", + "78f5bfe6f58ca23b2d85c54acea293e7e19ba60b63768934d1b65d1b44b31364", + "7910445c446ded88f3ae81db2004fe916506fadf83017f1d6c0481764fbc726d", + "791e0feef999d39967656eb542ccd5a5172cea4650d58f23bcb8fa8622551d0d", + "79493b75a4fdce4eaaf946864585719108489ee5f6dbb8440a70987d7ab3a14c", + "794f1d793b6807ddb2da2933851557a4ff3326c21d6a717897a475e7c8305e11", + "79665c355e938b7c9a5085d02e1f4454bbd1fa42d99ff76d56a0d37eede5505a", + "796cc42123f9a58eaa22cb3b3388e40160491dca862fb0a2aa17db03b45174be", + "79851d82ad4f83e06d475cd976b34d79a0884bf77624a6443337c585f5e7d88d", + "798ab8eda49c99f52524fab0e6c29e2fcea9c25bb7eb29e55d24f8bd928f5885", + "7997158c039d6cb8a0206f33a7fac43f12b6c05c50ab9523c5d319a07c9f47a4", + "79a8920832ba7837cd6d4b63760c7db8963bc9c4d5cc84b02d1656de6861cd11", + "79e246581d6965e0b6397f31780e76d0e681e72a94a031bd976890872e8c682f", + "79f7f98c4c91e9507352044f016d9373175a6410538d1d74765f5013de13efe9", + "79fe72616f57da92fbc0c265358909235cd86d5107d0e7d613adf8a67a868b91", + "7a0e515bd7dd1fc037d2793b5bb09d987967b38c7b0cdbaf484bdeebb6fffdc1", + "7a0e65a743023d8e6f5162be55768a3b850f64e76af441e3a5b99bf0528ec159", + "7a111a1887f6e3391d63b0141a7175207f02679e816cea6845b3486565b6cb17", + "7a1a59f3aa4d6387e5a5a2d61a4f4409428633616e29fa29d50ea0fc53bcd339", + "7a29256a151c5dfa124e5c339bf9e5bb74c2125c128b04d6fe46abf2a758f712", + "7a5b2cda69206c48b9e8d535cd1e96ea9cb9185fab4ce335a2956bcc05b0b115", + "7a70990914d64abb8b318d3bb2a06651e1832f21937c00651a6df6f061cd4842", + "7afef4f1d28f68d33ac3ffb659441e0102096da753d5c20969c536ea0dfed7e5", + "7b2cf8dd9d56b7ca7a7bfd1fa949e72969257010fc1d480bd54e043796eb182d", + "7b421ee486724cf4f7e562d916b247b7648a152d31428856c35c2c2db4813bd2", + "7b689b58e64f9ac6bee1eb14628884451cee03f41fdf2178916537c2b56133a0", + "7b6c5f005c475b264b2a7bc64a14240082a0ca96eaa41b4f9d59a0ed65e7d381", + "7b92e26c917b401a6baaa8cecdf723c0cd3d59666c778dca685bbc12ab9c6f5e", + "7bb5aab6218927cd29574273c5cdd63d39849a57c0ec9b59340302dac1ec5c5f", + "7bc0ebc0825f60d79a197e4d27689643d65798e598057d0ae33f72cd39a8d1ea", + "7be40af82f79492703783517086251c2be0511bcf1dd2b1659e4d627055acad2", + "7bf300a78229f582b67dce564df3158f02c03a7d17064a78d4b041ce41ffc3a4", + "7c13771c61816ecda59ab2026f2d2a85c882cfba9b967d5d41c31e338602eca1", + "7c249a595ca1f06717edfa3b35b2b487e2c2a47c29f75e4d52f4ac7625e24b41", + "7c5f4b2df36cdc780578afdd9ddf6d84a6cdf997977d0351ac0ff19bb403551c", + "7c671c670ce130f0d602bd6d0573ba85c6e754639d122ac4993bcea07d5e1b47", + "7c7f4330d058eff280e31afa750d2679d4e686f89f57a402ae5851cb33eaef95", + "7c809ee513fae51056cf753009b516a7d1c46af0c8b9d53761cde8467416e843", + "7c89620ab5745c5c096fed9a789fb4ef9a7038f3a03ac2a6aa033a960cea0759", + "7c90ac3e8b713181be6385e25e948a2bfe21ed28b0c882153787eb8be239c2d0", + "7c9aa9f7348291d8af6529b509632b76f898e46070c8b8cc5a74209f1f8e384c", + "7cf13a99136bc97bc57c8cbd7faf6c461f688084df1ec3a9d65f79c11c600491", + "7d4aeaf8a6c226ade1ca11ee3ae0217ebbe29c768e83b525129fd74895c0830a", + "7d653fa63d81af93ef7779035399a4e53411ec8a7f651256d7f721c1da327da5", + "7d6d2253f232f8dd1a23a77557f48814e176bb670dced172ff3ff7270115e2b4", + "7d83af9e9367d2ff00fe5848cfdb4ba392479eb3f1fe223f6828c6a735ce24e1", + "7d83e4f7e37ec9b9962eaf3c010021be13d664653d07999ca6c6a81e4fa3c427", + "7d9c7b07ccc1daa04385683f46ed47f8ff248fceaf24e67db501c065e8cb21b8", + "7da2911323411aaa29a9a089c7b549d57f54086f1831e8d11e614fb17e786326", + "7db20772098831748a7981a5ced53fd95f5e03e737349f7ac3900188cdd22f5a", + "7dd2da493ec5174b6763f9a0162334bf09572691c2d8908df6fa21d18cc67fa4", + "7de9b973552d85e3f6badbb2ee3721a640a18b97840ad0259039d6e6df85ff05", + "7dedf5136a89643bd0641830635e4f5a6ae5d68d3ecd6e4abc658a39da276f96", + "7df266244513d7042584a4a8e1b07a928fdcfd2eb2568be855f438b14502c66f", + "7e256ccc54272dfadec975e2192dcc8b899784a82c8eaafafc2320af6270edc1", + "7e4083c02abf3d1745e121c2404d91b27f2b0b972ee71c98dfc2fabc838eab8c", + "7e53515a2846c4ac3e3005a0d705ff1bc47fd08a4186c5bf3a5d4d342b7b76f3", + "7e648858b5dc20e69179df0316b44b36ba017448ff6bec6310da2a37e48093fd", + "7e6a0e320a76b2d7c4571302adb31af0cfc155f957166d39d44d4ad842d38c82", + "7e7078f9ba22a164fc61c762fb5db6d955ca1ca86a3b50b8961e9e5a3627e79e", + "7e80a3b28e467688135d12d509131fde13a4f17f81614a7b1f9d19379793e765", + "7ec7e941978523b6bb23d2fd3d55d6fe33879c6aebbd84fbb185117091c12b72", + "7ee3852676020e282bda9cef1c1045e25441647116ab309a214ec1f5f6f1610f", + "7eeaba0caf143763754abd5a641189db220b4f37abce91f1c66ea992dce96336", + "7eebdb6297c64378e8304231ece5291bd19ec690a37b2b58c90880cf89ad3bd8", + "7f175640c764b7df9f0cbc95bae421d5f9297fcf62160d65949b5b100557f9eb", + "7f792c5911a60663e0e9fa68a8f6be943a925cee621084cf3956ca175b27749e", + "7f823f60346810266ccd7ba91ea8984ca742aa3baf849349c8e0475b749dd68e", + "7f87c627f7f7412fee152b6c47e6eb8b6a13bd38fb9397d50a10f0be461e214f", + "7f8834cbc4e365c22d0e3918efe4d32714772b2c52fc2f836ba60f52662caeb8", + "7f98e2cfd1b02cbf7445f2f3c3fbe301e820dd82abfe4177d51c3285a507f3b2", + "7f9d4060748a069f6786af1e4860c6880d39e038135439b0b2458ad5d3c595a9", + "7f9df7d75ebce46939549aeb0324983263bb1d809264f4903d434452c7026514", + "7fb3d3b49f404b6314e29baa782768af91c6415ebd255ea3eded9dc3f584abad", + "7fbcaaf24c8235d541a5609f45fd7144db69a5defadf4bf6b518f9ef5f163e5e", + "7fcba24680c7ad95062aeb55090e10028546a5fd4f4a697d538593deeadabef7", + "7fd23e3b2cb3db7f81e7730037a1d3e53fc9774948a8b85613600590ac7d0f92", + "7fe0b89997665c24edc4173f17d379fb0241804761719324b77a3ad19cb134dd", + "7fec9ff3d09a0bcbe3db33b8066c3bacd1d09326c89e6e453e8c42ea49fda27e", + "7ff731d055995f2c7355f7a59174ff09d3ad70234c6ee73b5b29c2a0622d13df", + "8014b101a0e5de562df78d8629d92a6b367ecfce7c3f7eb968ca1178dcc94883", + "803fca0d27e485372ff299d0138f27c97487af7708745803e2eb557e40c71172", + "806f014e6df16a1654bf06eaf08ed3b60c299fc7b5d1c37d433b3a0ecaa81655", + "8096a2f2dcbbff46a37e430b9e9b6894d5f6b5c01c93a8187900dd7909ea0126", + "80b0d0b784604d32fdce1b467680d1630259424aaa829573fa50c6444cc0b1de", + "80f52225f3eeef206f805d640ed82231bf7c01b9f7cfee7ed524b1b299ce1197", + "80f9218c5e9118528da5dfd6780556e8443856b35e876c1685d2d25887b92367", + "810aa4c9218c0639ea1d9d7fd3811415d7444dc97f831a0dcaf75ef802242f74", + "812c2ac233a568a4da77e7b7487e4472579ac9bbd8ba095177caf7f1e2406b9a", + "814f2af21544b1b03d045f29be5b5d6defb02be970b8a4d85036944c3a408843", + "819ca88d90bf6c86b097fe09a8704530168f1b8931116e9e55affbaafd0ace33", + "81acd0d4f293b61cb77a0cfa090543beac2ab5792ca1e88447b16374fd1dc4cd", + "81d76fcd53d510136dca6c06bdd90619de32a48b1caa2da941bdcb661bf8af8b", + "81fe898b8769901447fb50c71d7d0627025ad9f7d533d59cc9f7d17eaaeefa60", + "82489daa4c05ab621452941f901dd0a684e640632f76836541fa3f45bdc27782", + "825ff15f0782c3ed5bdbb6412e3bb77aacf35c880840c8273142230c0df2c2c6", + "827fcbbece238ab1a8bfc63dc4ae4a456a40d215b9bb3f7c9383d49c1460d217", + "82826c41b67c800350648444e150c3a009db7301fb4c693f84f848977dd3e9fc", + "82d78f23b734d512517876ace50d7e7e19d9dcad4db80774ed3609ed8d3e49e6", + "83171042fc2b4dfc78224fb2b00fa8f1d00a038680000ea14462f14c604eba46", + "833133f91df4b418393445068bc2e9e93a065c463c8c8a25d3521e84f6414a93", + "834597c90083fb02ca00b90a38092f814fdb011897ed0cb4ee5c25bedb69d940", + "83aa7ee537812e09eb16fef58bf15fbc5efb929c298c5d5e1be5682bff8efd68", + "83b61aaa99d449b594ef00dfaa7d958c36d90dab01db14b2e04ee0db9c365445", + "83cbe1bf2b5a688a257ca8d424d4384d53a97da158822b9f086511e145839c23", + "83f7029e77724c8160a53346097015aeacaaaabb5e4e8e3930f21c82663783be", + "842402ca8d019c79cc2980fc26018ecdc2d715015f164225eb0112cf8b619587", + "84363142eb13b6d6a8c1ca2c282aeac21af2d69006b75733eea7b1ade7666d27", + "8463f76ac8b1f2ca1f2d0968a0dd799a8bb3d66f09eecc92341b45b3838840a5", + "8472c08437e9d065a8dfe76e302935c88138b65679974121599273edf2cd8d41", + "847cc4a4ff20a7cdb636264448eb18745c302f2ab0a10f1f61e1345cbe0bd5c2", + "847ef669e72c32f996d6a8245081aed7da92401654351337adeceac90899c4ab", + "848273eda2ed1190523b8ae401df6071e574af543b7d4237f4819ea8e795fb91", + "84bf01b039a60611ae8cb61ae6361a304f17ba7063425fa40f9836d7e76fd51c", + "84d4126224dd508c633f787902a15d111b6faca854ad91c470b008b84c466704", + "84d6d9ce5417edfffb97ce2f1947dd3b28381572a4bf02b041311b8a2ae6d3de", + "85026b06797489a856e2a5c1101cc0dcf0eaaa1b44ecc856f7df064abaed8e4e", + "8504c3f425c74ff723cb3a7f5c7e8df33f66e1eaa213cd2e71dca4f35491e68a", + "852359a7965660c3aa1296b56e4e98ecf4eb217a50e89ef12718539d31e0dfad", + "85374924744d63a016f40be7fbfbfcc00ef1baa374f4e9c31f35a039e4916d9b", + "856321fc3f1b3564e2631eb636aa5ece8bff0a96516788fb7ae44a68de39d799", + "856a22bd189c9505240da1131d3d09c44441396b58ee0031f0e69abecfbe5784", + "8571d2f43d6f338aa2691f71db7d7e10ead6c44764d7acfb62593f92afe32be0", + "85c19a7874cff0defb3daa6ed2411c6a300d40630127db54c7b1d72ab218ec8f", + "85d06bc43fa85e3a54b821385cf84b32ac25b95ae86a0d6448ec3a097436bae1", + "8602cd81454b46b26250823f90f178accf605c3cde02e5f940e4dd2ac634f131", + "863cb5c4d023cbdad947ac7cab27045fb066171a906913f4b8ff04ab9458217f", + "864c42307df55fcbe1f520b716e132e19f1ee4c0e4581b8fb51d1c8e0a2be428", + "86645cf08cbfdbba0c35ec11fccf41629fcc698746c542a776d96f1809aeafef", + "869892d5fe6eb6d2dbd806e2aa54b95f3e4087c76fed7f496ad8da9151917325", + "86c2413964f42e8db6d102682b3527c1d54fd1f36fcd167c75d1783ec9b2c682", + "86d5aa48fabe29b7086c8c48b33733fb5086507c03ce152f3dcff551fe403cfb", + "87251660f1c93538bd393e443bdea252c12178ad7b1fe3717c4cb456da1d5633", + "8726a1f04292bdbc4d137c58d3322127e3c73f977afe4f13da68584a409fab08", + "874aff72a30aa094c623b419ad6367bc9469095f544350d65af2c75f8fd78c60", + "8772efce2dcbd728dc3e6be593fcef89602e115e612aa50ede25ec3edab73f66", + "87a7443e2f33778a9f446b2e6bc81415c501381ad2f80caecd6cc5dee29a9e07", + "87aa334d871a911f774f07e7097a23960868f396ec4ef24f2d2a4d40a9d09150", + "87c60d890ad8c5c2a38be11978989d57c185a25969e84bcd8ba976a7f25070b4", + "87d1b82dd094c4acb9ad58a0ffbea7a5563a9936cd78f6e7fee778ebdcfcbe70", + "87dab3f34020532d6b44f52e4b7a035fd33bc5e4bab4c286faa547834a63b77d", + "87e20d968cd2fe30891243f403b3252a266e8db5fc90396caca386e302d0b8b0", + "87f4c982994ca15c1fc3d4d333ccf487b6548242feaab2b0dfdafda419bd0067", + "880779342113d7f59f2ac7f35283e3fa21c7f5635b19f6b2cfffd6bf6c9af087", + "881aa0facead55798fc83556e0e59977b06d1ec3cf3c254d86462c2dfce80736", + "884553201eb494d42a1993226344515c35c0d161c21d4a665a4239cfd45e005f", + "886ecdf27bbbdf53c2139328d18c174afcec3341a5aa4bd5ddc67e42a87c00c0", + "88a8b7550d896043b38333c24505d28737df4e73ec5fbc816795af28d9295048", + "88a9bdd66fc2344f8aff0b0d71c502b15dcc7bbe157324d6e40dd3058cc364e6", + "88b630562d5d85aa3ff2a6669b4bbfe5192067fc50d1cbcd58374f5d9d48c6f2", + "88ba13465bce1bb05049f99681cd787431e3a53903f0d3f51cfae12f95db0902", + "88c5f45e6084c9eb35b4377db489699d8f8fa81ced9a5c74341ae8f5217a0bdc", + "8918af679922da6f209ec9c23cf601bcf108176e52ff7fe2fcc20eb375fac7af", + "893c0f99b45c6e1ac223a4b470273b1a89431fb4ab140984aef93375e78e5e77", + "8981937b36eb34f0d585e60606ce56e8e67173cc3f04a22f46618d9f6e682e25", + "89875aece432ca420311ff4a23c57fd5b4c505e3c3b559165275845ae5e0e76b", + "89aae336d8ab2ca0ffe06a15efc2abde66c39f2d3f40fce5e85f111606b5e2a8", + "89b8caf01caefe1f5e873dee97904df01c28959a164af479fed1407c92add483", + "89c455c362afed63936ab400f5655351e685606805fec2d47dc36c40e15b971e", + "89c4b6d6bf0f071af66ee795ff42407c4fe150cb8e301997b721860501ab9a7a", + "89c84ad21dffffbd89fd8391e4a68536cd65177f9024c0fbb33ed0aa1988e9a9", + "89dc9547f563493b4c062ca9c1b56e277c0b973a53b4e11dec83468a4af1aab5", + "89e82b72fe11cbdc6c4685419428e65f1b96655d4eafa2343e891fd1f6429ea2", + "8a1f12fc3045a1ae803dd949931b6c477310b8636ed7c23b39af03b3dda64076", + "8a6baf027cbf0e268695248bd5dad15399913f37908c4a678622ca58b25ca568", + "8ae1576dcbfb6e37484071b60abefbee4a2d4183f8e8b21abcda98d615be1d51", + "8af1a4cecf6072ba4dfe2e726448b9d79c9350f80a171fc963e58b7fffcb0fd5", + "8af5596c49e5933241edd73e1c0349b84b28d9a9ef0bdb0b3f08dadcb3592d24", + "8afe556c423b4d558efaf132ca16b5fcd8ec07eab04614e72fcb67a700eedfe8", + "8b142c5aeb5e5c2ba168580db094b7b6da729837abe6199de738f104555e63df", + "8b181304e3a183cff73e26e12f68004c48cfbc7f66edb371622561c9ffb6d175", + "8b7f3e0da8bbb4bd381dd5faf22f3506c22b01f117a180f56f2060a15f7f3128", + "8ba5cc41b621f68b1da7a950361afac2ebd406bb12f8f21e1571be2680cea395", + "8bb3960b9aa49d3c7daf93c0b27a2e82677fe70954a5b1b8b40ff51e01f7b651", + "8bb4e371cb956240c12c6824fc1c42f113160973b79c22572e5140a5a29d4542", + "8bb8b135de8b7901b1b456ea52976b19d15090a7de2bc5afc5cf0e408102dbd6", + "8bbdea659991c41fc41e5d01605860cd53b22535b761954b847e34eb452c1bb0", + "8bcf7f0413754b850c9b24fa87068f48e73c559327425e1bfc82cae6fc4d6b89", + "8bd8787b5b3659b6315c160145c30b3aa5f799d10b61624b530ff90396a5b7ef", + "8c26ad6b9984963d835deae72d8b24856d38fef889bb05c10c85b3fd7a25cd6a", + "8c379436af1581c0c376678bb99b148d4756e39e9446cd804858c12009b2d373", + "8c4099078c2c0f6c69528c976e9308f5104108e2c6cfe95c3d2399672f5279b3", + "8c45266d075cdcfe5d59cd03fb88bc25a7a6ff511ef0b0cf9ae47fe5363b7d53", + "8c4cd3870bc9dc20ba127862a0ab3aac857ed1a4d0e2340dc7937eda0d115c21", + "8c4feea60b5188293e370c26dd06528cf6bdfb30518b3869007d3f80bdc3ef13", + "8c7148ff82aafe024eba8a2ebb64d6fd836fd75cbaac428abba1d92627fb07e0", + "8c7bdd8a6c39e9ac5222fb2b16861c1b24367708dc596f1bcc76c2c7e9c2b578", + "8d432dfd008cd8ced880729bfc45f3d548bc54922e5e6d4bacf680bcdc0a480f", + "8d5d02f66be9555b666419f16215b79e6929aa14a678c08d09d2480bc2a82b6d", + "8d717570e2a44975ba3d53e6575d4a295717dcf2564d14d62a2089a372a8fe34", + "8d72bf24d3835df975bd47aace23e91fe60bdb5cb0ba997ca78e52fbad343414", + "8d78008b2455a61d192635e692cf968000425c1b87e8ff46fd02b3816dcfae23", + "8d878504470b79be572a45051abba37a9ece730557e77120508eeb24db203851", + "8d96dec83d0411426e3245574e8c35f5b73c8df79c1390b684cae3a7950d76ae", + "8dc3568da85a0ea8b8b2d7c0bdf1cf458129cff09d04cbcb860575bd5d82ed95", + "8dc80b3bac6de50377d73b41145b48bb3ca6d23e99fc278cc28c1dba42bc1c5c", + "8ddd2ff6afd3e415f3a1316579c35833de765f667b9501e7e42da54f9eb4c7e5", + "8ecf4a9b224c936daa0053ab30a1858c67e74b17f2c82725389198ba26733706", + "8efc955a5a8e460ec1b481cef6a4a6f51552e6e1e49355f9feb912a0650757a4", + "8f030cf10304c8a821ff762ece320e3489870be3b40569b6663cbfafbe45d90d", + "8f03736ef3ed40b3dd7126f1c5b63059ec0a5b2ae71f67fb7d97de2fc16d2cf5", + "8f145b6dc0b3004631db47f710f7ab74a258adabbec957e85f67f15a99e5ef7e", + "8f4cb1bd07998f4bcc079c05285b81793fed07c094cb52e506b5fe2d654575d5", + "8f529f8b797d5cddfb9dbd137631316ec57cc23c6b3cd5249562e7fc8d8d80bd", + "8f91ee16ae905b7524533a6fcf40137005c080e4608becccd2c865d1e0933420", + "8f9d44973e1d625174356a6be2636e7f56339aa695ddc592dc65cd79f2355954", + "8fbead8e6f6dad8d2bd48456546b24c66bb9735ca70566daafb9118cdc7c83b5", + "8fdd16b6d541b7b81911d14d7f6430046b8069fda31d5c512e78f371f7df215f", + "8ff40cf299ee3337cd7fba77cb92fad05cbf3430ba4891c9545793a70b3a0384", + "9041d1ecb1c173f04808b57a9819a53b3156c1fe87df224597680c170297208a", + "9097a466fefe8548f577da17fd853422773485880e2d7c6085e8950694a316b1", + "90d310d3cc0848220d1011df2ecb0ef6045971926c923b6fa9ac2e8fbf8b2b50", + "90d7b9d3d86e904e19237a6f574643863bbc9146092f09a1b63fb18bcde427ff", + "90dbc96061612277e1e925b9beac596373152e74125ae7a9213bd1a1f8585e08", + "90ee3c873af25ebd77ed889cf4ef6410bb4b9eb00b32572f4651e93bd3b4743f", + "90fbcad54d4abeb3e68dbd7c4b923eabb630fc065a70246b8796627b3e3a1e96", + "911b8543dac7382e9e25b4dee5ce3774927cf78f57cb05eabdf33b3cdfa41e65", + "914f8bfa81a2c9fe0b168c645bbe45399099ace3ca439353229216f9a1b99587", + "91518c76c3bd6245cf1b051b958d8270e9d81f0373bfc56467dc896de7fa856d", + "91aeab64cf94aa7690f78bb5bb2edf91d64e5d8a037094b19a9c3e5feb183a60", + "91cabeda5e16e572b89e53f6ba3988fd8416fa7b03653f06969c126752dbed64", + "91d65e83e4fa2e40a29660e8697d3dedb1894b25601c3bb80df559389a9ca07c", + "927b077b9994f92cc17262f57066e2731c19fdf59f1ad23e9a187ad8a8e6a21e", + "92b4ffc72368d645dc6a5d181ec6cccaa6b2d5733d289a6fbd0d49da064ad778", + "92bacb8e4cb433020247552fb9c6cabd6455018488f8992ac6c202820aebe864", + "92c9c9caec17b174b853604267805f1caaaced45888eeb894cb5f07c3da60bf9", + "92d6eb7919388ebad69f0cdc179024407f2b0f10ade265a6bb82ceb3dd533319", + "92e08317b5264a5c75af6bd32f8f9d19d35dc2ae77a8fcebaee74b6349e2053d", + "9317c21bf5f13c7615b404074d4bdcd514250758aa7dec348b461e3ba7de2ea1", + "93717b7c08182de46766bedabbdd152710832ce74aa1e15d30d2d66748a74660", + "93d4e4a8d37483366afce2294a63b8e94031c2eb2595f06809f3fa154b8973bf", + "93da984a0282e86731dfcd961a8ce43b862a14c42d5b5d1e8874ffdd200d30d5", + "93f71412244e33e67e82e17efea5d58a9ecead2928ca36ac1e8cef2457f09296", + "93f7370ce73039faa1adee0d0cab9177772da8cb361d27ac1254cfdf74dfdd57", + "9433d8a34075bb592cf9c36bb1be21da08bdec9f9ef0718a0b61943b321b2098", + "945c821f819b42bed83f5450d180a86c23e6ce00695f7a18a105e2980b2ac2a2", + "948738b6923f5c6f6af94eaf47ada5f24436125e7b2d7532f2cb9b3f8a1ac64d", + "948b1fb3a88a247eacfa7863b50f8933ec9f89a59ce7cc3a17589ca825f5ad76", + "94a3567ffc300b6b0f7aa8e3de115ea0f5ada4d64e2184d2fc236c39056d500f", + "94ca3253fac86271dc0564b9de6cbeff335102bf817c744eb6d6434596879179", + "950c886241c044f7515eb00ccc25b1e9603899a3dfb4abf94727a1476bf37b05", + "9540e5430f9fac88e79b1e21bc8252affe0dc5893218218b2ff1839999223005", + "95473d6a2079786547cc70ab7f130c85e58839a865eebbcc11b0b75730e53b25", + "955f30c3eaf38525cd9d5503c4b1bea7b23c299cd6c8a6eaa6a6caa3748b24cb", + "95b77329d4972e58cce72f2ea588d614e3cf39cb424cd628e90e58fa2868a4f0", + "9620c1973d15d900e0cdfc235abab14254b5d29f240b8382ae9425fd47d18dcb", + "96290f319d1f2eb52a9f427d1df9bc5e5c149094606c64f51a172bb8111e1d96", + "963991dc96009d33a16333cd8dfd1995a7d274957508b25bb9f3c00683af0a85", + "9648696e71b0f84e436a6dd018c574b57d85ef948f6d5cc577a4964b9da64f68", + "9657dcc5c71ffa5ec11af1b35558bacbc681beefaf10373f74dfae9c13687f6f", + "96848b6b4616088854f302f4c39f7879232f208212d4816b6390ac11fb0eac1b", + "96af2b09a6a25be1f7f654be30ff50d473192cce28c69a5e89a1016ef5371093", + "96deb6e25f941a15a80a373e2ecb92809fa08772ea70db16466a16debe9842fd", + "96eb4c60b1ed6d1cdbb6d0a6e9df4e6d3072c037f5bf8fbb08d6f49fc0e96791", + "970c5fae48051e2e51273d468c630a6bb8abd4abf92cdc1e040ab2e5fe641145", + "97104c8814d8d54be64a2effe279e2de76ece7d9946d17f2bc721fc9ead045fd", + "97af82b0ce5128d252a339103965d1c99eca7a42ebbc19da35325e2cd716023e", + "97e800caad83fd0ea5bfc639fd4764cfe8a7a0fbb1268176e05015bdcff4ba19", + "9807f31242666c19189d7cd3d7007e6c617b49dabb548b216832e72d253341c9", + "980a7b92dfc3cb82bfde2b7660855cdff3a4960d863818224580dccbcb82a403", + "9811203c2c5bfc2d0a16b56c0aca2650534579d776a9cf0d2a949816af975180", + "981c07276056bc24a9bc32b733dedf96f088faf23bae6f053a5381b357f4a639", + "983888a1b90bcbf0f9188f2c89e0871aeef8a39bebfeac066c11418959c43089", + "98701e77e0b6c6887b96faf4f415863fe540bf5ea720db6c1df7da2bd60475f5", + "987defec81908223dae92da9b57746a6942845b340ba3f79cf8538dd83eda1fd", + "98889046817a4b80a4f39a5cb29ee9aacda0898af6ea7c02b82cbf7c1a75cf85", + "98b46bca82a90ea11dbd8c72a45f3697ba61059d391cb02b2cfbd98331ed4db1", + "98c28626cf398fb8e30aaf97dcc9b33e1eda33d6581088f76fdc2c5cbec9c375", + "98d35a6606cf800ca8622e8df0beb098445ed83ac11f1a14eebf21dffcf368d5", + "98ffdad2141e58ba3f1d928a7521c82d40aea01042acccab30f527e31d6d0c9f", + "996f5a33720e295a48e74c5316d54bbee57d280916362ca8f77e5d519db411fd", + "99984f7515c7d2a04b908a483c70a7cb0f2ca25605858592104a41e709665b30", + "99b4c019bfc6e68eb07326258763d2a30e257474a0da57b9791e6531c62e9e25", + "99bb9b16e809868a3c0e8195dfca2107f4bb888cf7a569b8e62a1dc8813d7323", + "99d4f33215908f8993f4002c40e20672c3831f527217db39469b57ac7bff8861", + "99e21fb94cc1441231170fed489593e95f264396012ce6d9f1887a4c693559b2", + "99e9ecd54eee0b8c28f53c9887b0f7f6e76a07fc2bc3defce7b09032c47fd533", + "9a528a5fe792a2afe2443a7f889a837a87c2f143271855c941c898e5bcb0b671", + "9a5f1226f926f45bcea2f48d1bbd21fb46b8607b63fdd464c5d429e41685a80e", + "9aed6498f33b8d9b4030e31861866f063c68284ff77e0d52c533eb19676a51e1", + "9aef46bf4b917c607a06ea8f84a33fc09b03e46d4161aafbccb836359b1cdf20", + "9b03128ac34991380aaf809fa4f6008fca9964dca37bb9a141af5be04b2dfd19", + "9b1c963738e61cb6be8b2320ec1f5d55b632d42de6852b77a1bdd26fa590f2ec", + "9b72c6281f55aa675ab785fbecd32f58bfed0a6b427b612b0872230734a2f6f4", + "9b93401d713707902cc42358969ae2e8a3f67a7abf682e6c9c662b6a4cd02c8e", + "9baaeff6a1950b08fe74b5224b5c12994587023cca5d9ccd79f2d5b9a9ed5961", + "9bd0c6c55c6ff96e5f018636211870a2aa45bd2f53041e5d6103e05afdbcadd7", + "9bda8160ad9e098a3db0011e26abcf4ae8137edffc8c7484befbf9c861f034ef", + "9c60ce7d1e20540bf17f23636fc186c7084acb8d696a47803d3a5667bfa20589", + "9c8ab72ebb2efef086b21855d5a4ae3f5292616e5a9c876f8bec8e28da0fddcf", + "9c91b3f38a7c6d1d242212504e68ba0a67a684a0d485f15dcd357bd19b411040", + "9ca581e36ee1174533857b9d926754b77009f9080246afaa63d9748d2d066095", + "9cab822d1fe9746b3beac1f5de7e24436882ed4a39ae207f3c9a51cf5b5097f0", + "9cf21be9a626e2ec27da045de4d7f4cce775ff1474fb7b4b1d361407f15bbc18", + "9cf6e80299c6ecc47c8d79c3c5f98265011381bcbf756a12c629ace25d71e6dd", + "9d20f9cd2e12a69400d1753f325f197a34cde2feaa0a6a41cbc8d114cb24d407", + "9d4d421526e0ce02fe621aefed63966502e7a6f1aefd76f27d7c5e7b57760626", + "9d5312a67b24be3aa7949b355c9627bb9982a017b95e263fb65e40ffabbbef35", + "9d5fc103de859786dedacc04713afa5f5ec908af5a8d4b6f654e4f35332f53fd", + "9d86dcec27b7472f83a0109dc136502848b7506e05e3d2b05a983f635bc617e7", + "9d91de5aac37435138b7a0fd208877e7f22c0f1ec931edd49824d81790758ccb", + "9e090e14941f35c1426fb43e188ac85fbf1d1a199e057b80de662be34937b5bf", + "9e0b3c8611c3983453fb5d666148abeb23e5f988cd09a0e5a34f61fb059d6898", + "9e0e34f721896560c995500d5251f2fac8c4562403cfb13ec4a2abf496d8e1a9", + "9e14ec38f8398efaf34b5b15bba9e47ab2019fad693dc51b513bbe274d9d160c", + "9e22c3ed476b2c040f1d9357094d17f92f041d539117569d75326ca8959e42dd", + "9e34c2de58f482b4d4e2a962ca938c8513d8555258088d2815b729ac114a5d10", + "9e397f028c0b7df416cfdbdab53e0b8a607d2348dd807343849ccd811e6dd429", + "9e513c1e436f575aaf5bd5c7df54304d8390ae88da87efdd3c9df0dfe3b395c9", + "9e81082b39b55ed18042c2c5371a652ed32337f6383d9e714cfeeadf99dd5851", + "9ea47633f98cf0baacd4966b055be5d3e71087b274b18821abfeefa6d304550a", + "9f06acce477a36f418f3cc826c27ccb555ee6a836ddcb8452cd76a766d8d4565", + "9f23be74bb52beb0da75166d46ee912a3ee0bf66954efee53a92ff558b0603ab", + "9f73734372aefdeadfafabfcff94725115e0cf4f9b15c18b6d671c93bc45d4b6", + "9f965d1380ea525701e2cc137a76dbf657d135908edb119b5ade89d517c642c1", + "9fbbc8610042a490eccef51f5d910410f875d56ea7ec41b455ce0d7f6fb76e1e", + "9fdb058c747884260689eae9bda96e5499101e7c85685c7441d62133692bf1af", + "9fea119b522607162bd0849202a5af02fbbcfcc9630713404eb24acf7fe44770", + "a038947bb5d061ac5c76c52746b1f3a3177cac57b6990c86694207fde7c6d52e", + "a0b1daea0fa36875362462eb688f7f3fdc0d602feb9df26f1ef223d31cc2da74", + "a0cc9e51bc6b5aac329313a6901cf0320d67cdc2ade26f4db2d279ec352b42f1", + "a0ea2d1b59920fdbf1565c05bf05ea3eb49b8a893dee51b93bf7d443d947b85e", + "a1240a45d1455232215af2dab413d40b707bb262e4183f3068350332abb33b22", + "a16a139ef9ca152e81fa85305fe2b5cb805153c6bdd2ea89b48a0f247ebbe12a", + "a178707a23e01e9e0d753982ef537e8ebd27f10881be88d117ebffd125c84112", + "a1828553aaf5d9ec4efdfa0a925fb03449f6db4b13c376691b34a6f4775b439e", + "a18640480013082dd7e7a22d73e66f35cc59f1fd1b93f5ed3de1cec43413bd5f", + "a1cb72da3133be47ced5d3fcab95755ef24d418f76f4fb0b4d47e6e8ff89a470", + "a1d812357c1be25f45e129a8b8635d6a2e8d8b2eeb0248984bbd64459918b28c", + "a21d104b5dd167736bce9902303d4ba1b1f02237c9b4a5f6eff0a7439042895e", + "a23825f174104e351285c36f589dc37770f5bc835d7d0e920e09573b704e62cc", + "a24336324a1e3800247283a001ffbf0c66ba08c1bfd2194aaa88e91da4c2a16a", + "a2456c53189fb0ad5fa63355cc5c3041e80729e753cad05922be0d65b48d2fe1", + "a25f880fefbc75dece8b8cfcab410d6383a6e87ba0d63b18cacf501ef8d0f4fb", + "a271158686e7f14f011abf7ccbfa93f0b0a444e3b356c8f623aa73fe8013d82c", + "a2a7b5c40a898db6db90eca16c9f41edde6feaadb445b78161abbd523d4ee84f", + "a2b0b4b8b334464890d9a5d9d6f39b05260fab4f43ba90faab4ee969594848a2", + "a2c6b407b6b42eaf49eb9077bff37c976cec72ddf64de83b95d34b75d5e09718", + "a2ec6a6102628055650d617239939fcbe2324cf862641123946607ee1eea542b", + "a335b1d36fab56c1b1ccfd69f872f9d3b702b312e2f8d64b3b63990e2a02b32b", + "a36b7eae29896149c7696386376724ce04884f2cf9ae685a42133a31bded0fd5", + "a38fe3c04414577e28a405c41a2ff11ccd80b808dfeef1bc20e36264da1368c7", + "a42c6de83dca6e8a8483d1725cd6a3f78811ae75be16ab8afa4d9a95965f30c6", + "a440cf910bbfe95dc9a81c122775b244009e5fbea638563e5760e4e482038dcc", + "a4776777ba6678085120b1ae5cbcf2d7739848f9ebfe6cb6d7b7bee59653964c", + "a49914e2c5326aff76aa60d7f5ee3c3640d622c9e8b4c68b37c3ccf680fbd7c7", + "a4bb7e0ab08ef465f0b663f20a013ec927c25d17ee3d4111eed4044b1d1c2980", + "a4c7c3bcb82cc25073907469db3e203766cb08f4cb9e2396165243e0708db374", + "a4c80a95b8f850c2dfbf410aa2f795d233320f9714d5237e5120063e6b5ee2c0", + "a4cffc4bf7e71400877b3811d34b1038e95f6053ee7be10b8990b485c7112140", + "a4ee41ba2c8e777a9555f962d285865998c7df97062acfdb9ce6caa79aa76bb1", + "a4ef3eea01ae764762e35970478dc48efcb323c6465751ac59d5101dea270504", + "a546c967ddb57c8307ccb939b6a2be51ec7639d4678662bb377661aaa3c80b38", + "a56d066f7b27d828d979ac0852165457388eb9fb7ccf509ce55be322a79336eb", + "a5def6fa74e09ff25a2be14c68cb7685ae5b85c27241afd2204da0810134086e", + "a5df13d7ca9fd83e65a0a5795994ed24336471126925fb292b760d5e8bc7ea98", + "a67c76e319bee136016fec9557c156c4c8715121226474fb21005392a92ad059", + "a682b5600e66458787495ec446edb276970f001498e62f2f7416d7445f85c8cf", + "a690a4c2865eaa72e8d5f2985a25fb8a77c688b977fe7f670d5ec3f1e41a9285", + "a69ca4c0ce5b896dcce10da086405f8d109c21fdc48bc2f174d6da28a516739e", + "a6ce6ff75d73dd15b9755db511202f12f64f6cc578ae887d28c1b5429c60df8d", + "a709d08dbcf24d3d86edf9333fb4b2f3767c49860619eb5844abebd7e8995130", + "a70dead3d5c69f30999bd47228b9a51d8b6a03009aa88510f1862af51e3d29f5", + "a71fff06653ad8cf199d4e6f37db51a88391f423f5ff498188361c1e7b6d34ab", + "a75e488dc3e774199bc944e1bb796138ca794094f25e6cd6fcc2697ff39752e8", + "a7acb1e2a2622b4f46a3c77fecb918f935fbee00f4698ddf64ca13299e0d779c", + "a847b4f5daa6a2730017bed869d6fbb7592a82e92d83e337a9f379a7bf071aea", + "a84ba5d88853d9f39f58293080d7675c8a20280d0950ed4b34f403b07fc94294", + "a864d70f399f5f45dec5d6850ee6cac2e405af6d81e08a0167230b795711345c", + "a8b1d5189fb760726fa336137e4278c0dbdd9c6b263f24be466d4d2e04af99a0", + "a8c190caef1cd42699635f6ab8c273c11a9c540afd96e14745854499dc90747e", + "a8cc6da58014750451c1aba1cc1b8ae1eb1f439ee203d1462972e4cd1238194d", + "a8ebc50ebd06ac561c84209d42e3395d57b7a927aed1790db3217dbb6d261ab6", + "a90aeee371c90fa42d40c1b41c630ec063f748040bc5006fd9c2b88372430d46", + "a90ee73b0f090b36a3b906fc3ba254d3d921f539e4fdf8178fa2764c3c515ff7", + "a951575db1c5632e10bbf2f3012328ad5e9db87c8c3d6642165a5e59c3790092", + "a953dae70fc62946d83ceeb303e8ae3838b0b0930ee3daec62514a2859a0f5d7", + "a953eb424669375abde80755e7260407805606f07e0a2475802425430ccfa20d", + "a981bdeb4c49cc8f338c986c9c80e5b0fb4179c88f512d472dbc4d32b6ddd837", + "a9baa63d7dab29028f641775aeabe1073f5bd65eafa0395caa9dc2b473acb87b", + "a9c76a3cd3ac63866e2fa0680a392bcc7fe67d652a27ab62c2f3fe50d07e90c3", + "a9d5e8a3bb7e612f9263323961dea5f8e066e61fea3ec027af77816c7545414c", + "aa055331e79144b7c663468f03e6fe569cd36baf77fa333e08bd6a7e3d4f242f", + "aa1d01d96bd3a988acec7c407d567aee6f731c70a4f449796468e2883a1a20c2", + "aa2c77c6f377cdfb433d54423d96748858541f03da3d8943586a1913416fc9f5", + "aa5b0cf7fa87a46134f260b548a74341fcecab639ad38e324f09dfa171c272e2", + "aa5c9f8b4c37a2dc2f29ee3e60e2b49487c0561c5a698c0fac5ef89eafc02d61", + "aa64551893a506054432e9e661b5efb425769dfbd9965568d21ad1eb10dff7d3", + "aa6ce1d0818f17de327f41b9a0c29e1cb143e49f5a039965bc42659ea7492197", + "aa97ca3097e3987f4519063fc8728c5b3b01365ccaf191553fcd10197670106d", + "aae9480c83f3cc084dca70e76f618f82b87bf2fb2e617143edf01665ef506093", + "ab62a849fcc9de65ce3e9f885d959a2b3454622150816ab94e10c0d8d5f0af14", + "aba3af84c0c932cb2ac57217c3c01c65cab49ec15f8773ffe5bbba1c7fdb4051", + "abdd082537018e878916cd25555617da6b91fa387c23f45b53fd80da215f15c7", + "abe660e96896609144d8f5994cae6cd441200a09e16a5a9080469f3023b91e86", + "abfc23b6077fd0a9b69141b3798a24ae1bf70e8a6e985057ad09c3c1c9e17246", + "ac2b9a3dcc45274eb9f2cd32498118f539bacfdf5705c3a9a60d68b8be68cebf", + "ac4a30459d90b1ab88ed2c6d96d1e6afa2aa1a59e708ff7350de36b260e4bd64", + "ac91bb18dc539462084798d093ebabc3c38ecdc88db10582b97bd1ec4063b1d0", + "aca5ebdeb25a6839fdf7d99c3c40094535dcc69fd098c51ea7e86f870a768388", + "ad1fea799f9b928ae94a7a9f820465a3d54a164276dd68f1fbc771e2458ea95d", + "ad4aa588253fd0b75effba32d3f3f62d2a0b20499b4b2ff20183c9e9dce8921c", + "ad4d7e703b7b024c40df24238902ab1f66c74d3982da912c354dbdb120b00a18", + "ad545207d62ac1a22cdee8252b85b32924c5e16627dd921a96a9c327c21ee4c9", + "ad9016fbde8ab97a81bc34a102503eb570e2d1cf3aa49de6d37cb6eb272261df", + "adb558ba9f031272bd864ed4c02cb66079249969b4a84714d8e904450bad301f", + "adf7df02e5ca3584edd4f5cdbcb27252efea4db7d58d67f833789d016a911c43", + "adf817abe34fd96eb5bb476d4ecd70d5838d3d1535b1a1fa7f89e48deba3f691", + "adfc6ca3cf146e6b70ec610f433048f8f2f61e4b0a9a61b00bcbd548aeac7c68", + "adfe8570209520592870637d0c6d89dfb53ddbf0d5162c38e7ffa79c154e6823", + "ae282a9feba3cb183bfc8ae5f3029dea7f85af44579e140d29111a5ba56a8a8f", + "ae4a88c7e31198956db2ffbb52fd170f8391acb7af4b75f0f7e17c52ed28be38", + "ae5753a98c376c9e7d29d50ddc0e0a6c9eaa15a6fa983e15fd5bf6a70dc1cbb2", + "ae61bafda0632145d2c840cbf5b9ee880f5d515b1dfdf84255024d3aa9ebb866", + "aeb84e1cc03533df9a567948e3096a4651622ddb8c65d273ac6ed9cacb76b27e", + "aec8370a06a5d1b6bc64be94fcffea3bbcb31bd7c3f87072f6eabe7c2f74353f", + "aeda0d224f68d7d2271fb38d4cdea0bab20c6d678bac9e7415be86e1428a8f2f", + "af0eab4a7ff3a9f565c5d441ee48c6728a8d2572bb47ba29bf054ad05c830fa2", + "af3d43fd362bb702008257f845c0fbd7fb0b247038bf11c8577d02b448ab9644", + "af40469b4be5f0f237a06204c6410d570c9b1713dd294125dca3fd71e5e67947", + "af93c5d79d26229544af7194357c4a6874a024c4f513e73ffb9b6fff55d8f800", + "afb9f2bf7e8bd33d310cdc0db5a756e3a4783e435816d67a201b8ed3f797b1ef", + "afc38038f2d082d7566ca44715c132c0d531f317a1a3d54a88178e187124b18b", + "b02963656a19f9f7dbd1c3065b907b214dac7cec8ae7b0ca8bc882a31112e43d", + "b02f7b28daa1a2f7aab7632e16b8cac778bc70c3e8a1b92d2d9bbf3721e3a4ca", + "b03c695f732f3d794a58bd6ccb30774a7e9f77e6335038a1b81997cfd39be8ea", + "b0641b7d679fce235e06223f2fc03382a110fda99b2624b408fdf9edd18c9a02", + "b067b6ae3257a455bfa37d6317ea11da0285343065480c68b88fdbf1f564c456", + "b088b8fb0d23433e44bfb8495d0bcdf14a90a8688a40a33d77ffa1434e4bd021", + "b08f6cb49d8f2c0cf7fc4529717ef050a3d5ac48257e43bcbb21ffdf9ef56721", + "b09d4968b098e1be873a624d4735318198e28acc81516d23ff2a718d5e05da0e", + "b0c9a587407aff5c3b96b28cb92227db0849d7b1fb4dd54ed01e9e194748fda6", + "b0d272d65e86b83427688ceb352a31af5bcabcac115cbeda72ce0cd8df37c3c1", + "b0f9d0a383327c41c70b0d99cc1e17d9e4b0f1a1766c4b1c410a847baec0eaab", + "b1188a1e16f91b3e25c4437f118bede028ee084b2d52ef9cc226ac04a33076f2", + "b148ce2a75b2aafba33d67b9414f1a8b5a355f84f16d4c0b6438da02ff71d9dc", + "b14c6e982b47a9f7cbc83e2178db724a8dc1a43b96d54f4d28ccc5f2a03de985", + "b164746f375cbce5e956785bb3571c970f97f0e3ea06b58f63e6f031ba2b1d58", + "b16a445cba1d8945f315d29452fc1bb6b36006f64c9dd3a71818a89102c48dd7", + "b16c7b13be12df7a0a17561e577053f14140827fbe7f4e220b1d3db6df56a2f0", + "b17984a1dea67fbaa09b778d2318f797f0b0096ae071c7380d13b2913befd546", + "b186db261318ce9d0eec255ee4aa33caad5d23c3fbfcb7b0cc74a01c7ccfec43", + "b1940f4e9449beaa5e5553b1396d1bf7ae7cc18512cb8927bd44996302e44010", + "b19d1adb2abaf3a1939ab6fdd40020c0fe19361f943dfba64dee935859ee905f", + "b1a5bac13634f77e22d4f4d18e98ea341b72743117f0720cb05cfeb06188a979", + "b1d03318fe70818bd4a120e56e58c65af108ca8089ca38b33aab4d2f49666b22", + "b1e5ef44a5efd3a383ae8ccbbae9e741e221ea1fbcc5d05bd23ed1f2fc3e5fa3", + "b1e84bde7f97e4c7525cdf0971c26f570dfa7821392c3414da9bf6d49989be9a", + "b1ff72519a4f541cf01e4ae32345cede5aaf9d36c67d965e8981c26a139c0460", + "b260bbe4a2dff2585030152513fbd0eecfa57c138379e0087ea6f906c03201a8", + "b286fa0eb10cb2daf7436686cd9582632439c280b04b6b7539e1cf79a9f11f93", + "b2ce9e4d70b36c0cd47252751ed47b8c7765839750ab046b270f863a82f19fa5", + "b2efd303d551237f7c1248e6619bbf925479058a16a716f102d67b94e494131b", + "b31cae085781532bef6eef09b4bfa31dacf2bc02dc20fa2d8d922a7e9f985f43", + "b322dac84aec62b55e85728718e2a0d9ddab14bfebcbde23f2eb6b949dd4759e", + "b341b490f5f8c125d57f9d9633f8af89f022d09083e79b096a593a6b23336d13", + "b34596b6121b6f5302625bfc13325f97a351ee3824e7b08171554f043e7213e0", + "b37d030e04976ad9717464b22b3854a50114d068ed51509060bc73376a484116", + "b37d200ac7c0c826ddce7614f9224f2365e8ab7732483874f4ca181394c2a5e5", + "b390d84397dacfca64a5a0e664c32603b4064baa103beea8c699d807793ee9f7", + "b3a19406b80a96b58ffb2186de2d9ecbcb3ea04f0079be0b038fc3676d2f5763", + "b3c4733af59558e62a4d1f9f5ac676e225e448d7cf70c381d9a3927d5f49c450", + "b4090022ae5f781ec0846b887e8576989db3ba34f51aa66d8b619eb4ff73f665", + "b464834e6ccec4397ee4c4f488c26779ddf6a067b7258b25fcd31bf1d61c77f4", + "b46d29e256ffda81cd2812cbeb60119d4c3521ed84384a74970b74876f81bc7b", + "b46df5a380426793e33ecd6a5cf0ef9cac8fab77a13a0d9a1dbf80ad98ff4f2e", + "b4d3a904bae484b4816a592f36be4cfc175b6af30db3d5fdbc11d3692a42ceef", + "b4e84424287c53a682e4e2c85b57e730c130cbdcaeee4c5f08e481b0c9767469", + "b5731d35376fc114009c9ecc6feb8061e4326f8127de14c332ae7fd0f1c8e8f9", + "b5740ca18f0bc9a3f6fa9ca025e89a4f4ae9c571ff67d75890cb3d6414463e69", + "b57d18041db98a5c8a9f58f983b9b0d8771bbfad555450164542942faa0da287", + "b590eea2c394badae79c7964c2752de3e07cd42725af5153ad6e0f4f0f81560d", + "b5af2eac889304f3f2106cc0cd4e4269657876909cf0b1a6a5e1422caa69c026", + "b5bac383570e16b15b2e55caec37e1761c6ec382a8b744174d29687b4508a94c", + "b5c30c652903550a6533882aa07c3c99df4e03ad64056ba2df1245cd8d2556f2", + "b60f4d4880ebd0615405671fadd4969804cd9ce9c249ff4af97d235871d45324", + "b6266da8eded1619177f4f8b08f28b6dc348b362741465078fe9e3f05d516da9", + "b647017f986b484639ada0c1521f4cbf3d09bb6d852f17be1892fb2e22bad569", + "b657cd7a60e964d2f7b07dc6eaf7e1a4e91f6fb08c90180d3dfa1792260fffc1", + "b67925a01180149949620a0dddcc956c5af9cc1579fb8026c717ac2c2d6f81d8", + "b68f5c89a885438b835515c63cfc0cf094b9b480cbf4067fabc35fc73e54ae70", + "b69f06e4f6f273d7a8a6150e56c64e6ea920641fb286e654a6d548a0ec5e3ce7", + "b6afb900709a13953b9833e507e5b0c53761bf68ba514642a45fcf3fe7dacea2", + "b6b8789681c6078777268a6ff1098d67a2176a210b052275e8bdfd378f0b7be7", + "b6bb805f019769e7c22c3099958cc7079d8261a3ceb6760f2307b824170a06b0", + "b70a4c5b5ad5ef61ce3dfe870e9a3f04dce28b863c72efd2302aaf9d0284dabd", + "b712f04ad27bfc82a95a0a9539c5cf93b2ac2b0f938ef593d5301645a3dd8232", + "b7358990749f89a239d0c35e4454c1f2256f2696f43c0b9a6dfca0a01e8c234d", + "b73736da3712e2ac62d7296642cc9b095a26ff278454b9c800c8e88ac19b0c68", + "b75225cc5620623a5f0aeb1ae56a99714cea6aac6ad220343d4ac6fe3681e22b", + "b75f3b57a8761038dc34e8c06242932cd7a3a42a4684ad04c3d0aac1c8e01f35", + "b79f8493c9f22103dc29fb01c9606f726684a35e81826d67290a251a494b2d6d", + "b7d02801cf1eaa121d3e3e958f782edd778ca42dd1acb0d63030f50b6ecae0f4", + "b8089da48eb95d0fa7aa7178c148f878c61fc5c425e163276050d66769b39570", + "b848f9c1cacdfec2ca9ec77091fa926373901a81e89893ffd44b8e34e813c6ac", + "b8a81aac98d4107d71a94ed1e7dc03918b99566b6fef297c43daa713c1e0aec9", + "b8ac6150c2c004604580621d18eb3af49871dc2626c1d951334225bc65d1c438", + "b8b5a6b49726a5e7522829273581631f66139462cee12dbd41ca7b93d8012a3f", + "b8b736ff78f5dd8bdae19eefa05ac7204f4f07187d163341e9316f2caa7921aa", + "b8c3bf17a916c8272eec662a19fcd3ba15885deb7013e3165443147be8624afa", + "b8d7cedc5d15d681707f2f38e63c1b38836456b98577792d229443c26cffebe8", + "b8df782b5cc955e789adf0cc2dba357393d352da099418073e9a9c255f0f9797", + "b8fd7e25bc04fd03c01b2c376dcd64328a543ebcd9d53564ac8113be74b1aac4", + "b928ed0ade3ad7ae4d974356691db8dd8a7fa18ed135a9aef8d746184aeda3a9", + "b93fba82189b4ca8c32a2557f46a130f50b57171e1951004e5378a2d8fad05fd", + "b97305ccba8b3ff42187b7db6c2859328d1799cd55cd581fd3cda2ab303cfd93", + "b9a967c30fe2122901176bf823703e243ef604bc34f75753fe0d8161c46578ef", + "b9acce70b7e12e8b34844749eea5419deb55261a739d1519432c2cfb66dc567d", + "b9c540741090fe4388183904f3f19738a7b727103ccb50bdcaa87abe0a812dec", + "b9dc3d032fec920e4b54a5f3e44c3d2ecf6c8c09745da579b3fe1a22eefa9bbe", + "b9ef19b21ea3253daab567caafe43363eafd77cf8df382c906231547b7d1eade", + "b9f2cddfcd21ccd881f6178c8388173f4997370ec9a2a5b3a3f10309af4cea8f", + "ba3e5e077781c9e12718fd602c426becd35555b333346d5c8b8d0e4b8d59fb75", + "ba3ea31cba469ba9522e17bd900bea020ef196b15bb4f6082ba322d71c7cd2aa", + "bab819a3345cc38eb9d2f4b3d594f7040b75114388442241d3dcc83ce9289ff7", + "bacfed8aa647656b317f21fae6db862e24fc59481a66338f040977139dc3d059", + "bad11986a7cf2e25876f9ef3449c835f2a40ce7ea2f8bbe7da9bdd9beda4a3d8", + "badae713e39d2bb3121a95ff0de7e6fd013bdb9dc86119b49f7edf04222f175a", + "bae48bf75fa4d9c777766969cea7efa83e9175b9735a27008fe457060dac51ab", + "bb17ee93889768038584160ae37403acb9401651630ab3a405abd65214f8e1dd", + "bb1a561eeef5a67d08fa6850904216b1d7593e365271505e36cee5c56e6e886b", + "bb53e6ce9e0343c8e4a1a689015ffcbb21f3db895b2ad5ae51472dfae8f98759", + "bb811974b2088e8fdc2606f0c8c076e55e48972c5213cf3234945c63bb9aa8d3", + "bb82c9ba73073bc7bf00d3315367a6e01d136bcd9a9f0677b9ee642fd2a59227", + "bbb54ad3f3e4d91ef75bc242b35a9a28625963d91f7c123d99d53c2ddd58ba60", + "bbe0f13c903510cd9b65c6bcbe40c141847bd3289e8d27a17c494d98104f1ad6", + "bbea53187321d8577bf4de25cbaedc339cd30cab7900e2d0f9dfceff92e30cd9", + "bc491ccaea000e7928611c69f1b3e7ab5f24f03ada9c29fcbb69f7fc05aec117", + "bc65a7f288310c5ea632964514744d5054e03a6862fb478ff41f6ebfb5e4d749", + "bc6adec90a6dd989815d2ebd2d3d122e5472c349eaab762ddfac58eaf23a9f57", + "bc8cb19dfa4512030c9cfd5171d50b53c2524193b48d715df8bb2f349d24a0be", + "bcb180173212c62ba2392972845d7d66e70d6efbf4d99f19703185b405d8d6b3", + "bcd8b31f715818910c72709e4e53ec79e715058b3b8b20b88d421485b142705a", + "bd2121f0f8916cb944dc0b9cce499d8c8201bb9ed855484806832f0e3774cb14", + "bd55dfb70701260beac95f7ae6c29b0cc675b33a56815d4368ba5a5a9b2fb960", + "bd68f00870fa2e09f15aaa7d6f7f5e3c0e0c8e9df9fd13caeb5ba5ec721129c3", + "bd6bf91e43218326db970a15eb8c67d056a452ea307dfa1826cb6633acf7da0c", + "bd6e313ca24faa2d2310f0608762f6c5fac8ed5041e62e6e8b5c7c80edfddeef", + "bd932b7f64314837ebadb99027ac33363e1ee0729674016f754c0ccad8e8f101", + "bdc0a195aca3787839f81cfb8e1b323702fa9650364a46c5fcd573a2c49f60c7", + "be0b46e7fc8a373d20a3fcfbafb196c04fad15e6cb4292c552dc243ae1323f09", + "be4265c2119ae94dd3dd6269a67b63bac95797b3cba30b31349625b1607919a6", + "be543b6873587292c6e6a41172e329c02e95e4a64e504f19ea40c4cb61acaeaf", + "be6c45e7f69bc3a50da61647200e2ac1ae02bbb1a8ced24f59103c3044bf24a3", + "be7d2336c310249ab90834b1f81317af638e6dadbaf11073b38092b368151dbe", + "bea496e9c1a81e100601aac156c4c56a930441a8a23838954bbb3323a21047fd", + "beacab588382cbada4b9001299ebb9fc2af37622555922c641cccb25aec00506", + "bed8a7542d9679d11feae206a51d62c627da9221a91b6835876f1600176d7dc3", + "bf0cd750da6ba9752ef29d82afc70701c2f466f4f7762efb7c587c973e66d8ea", + "bfb2e72e247a7f021efea6c2ec93634f2e549a87d06a6d518c04b2b8ced37d7b", + "c00ac129f5040ff62e1ed55dc195170aed57ceecf978448290f4c98b6f2b7cda", + "c00e8564f343e9d6d7bbf1516dbba91b91abf3c89dd7b0637417f310e9e51245", + "c01a0385e94fc2fcccf46f244ca591e67737a37030caf872033f60323dd42c4a", + "c02899f51d77c003d5e83858d1c9b0f4f01b04de3f7642167221b4f13a0b2982", + "c094c64f759b68c09c223f21b1cfd965b766db99af89b76efe2170546a2ecf6a", + "c0c3bc99b53fd5e09e26d8814ddf1ba8920932745eef252fc09e240d1c0e0d3e", + "c0dc7ebe001f9d7d104c791c9de10c31255c5a850e14fc31dedffa78e6541e83", + "c0fc70c7452a8d99076484534e581dba98bd1e867317810feb0a103d2e5c37b1", + "c10514926b6b320fbab45c471f278074787252ffec0aee89aa55d78ee98abc39", + "c116eb782fbedff0cc22f419af954749c075c7af985c9eff7d0c165d76144ba8", + "c16d20b8b99da3786748ec64c23f1d4514e948d12aa8f5bc5187a38454282226", + "c16e7d672eaf324069aa23d5d6a1b5f138a706dcdb9d093c6ef42bd08d2b890f", + "c1791b84f1d8438c1ca830aa9d7176dd9115d3ceb6fd23aa7b204fcb05165ce3", + "c1a4323c6bd7607433d6ddf4c76e4502e8b7a6b3b72fa385c906cdef3699cc93", + "c1ad7d9b7be6927a9c31647b1f69ebb2da21c369c6a817099e6976b981c10deb", + "c1c21f7564d5ef802a6cb2a5fc333b10067d5814640df797701eb8fe4b114e7b", + "c1c2b04639ef27293b03468107a7b2017f793a7c5ce17232db08ad396c23fcdd", + "c1c84ed5e9f2228c43bd94eb64102cd7d02ed700cf4f712be746eec31807506c", + "c1d0a5bb9e5792eb0903e793a7e924bafd953d7e53982e0f31e650162658cfbd", + "c1e5ec301d21dd885bc19330ff1cc10a1e028bc22cc866015a3e550ae1762066", + "c1ed275f0d09590a03749fbf16e0767add13f879a89ca1c55950f460feeaf205", + "c215aada505a3b2c249a1aa1e3e6b5ae1817942bc390c1e2366096e8e438f996", + "c221e076bc9d99365c242a0b180ee5895b305510d85fe5b1e1e4fca94ecef40a", + "c2456af35e9845a2a973de93fb3bbf8031c8f2e22dba10d9496fa138065731ca", + "c2461e62faef7d19e61d6a84bddd561ee9746eb30d0b28779b0f24b0ca84dba0", + "c25b6a2ca778cb99b613fe51208ffc8ef01a48f9516645701c248300d6ceceab", + "c273d409d18c86f08b74523ff515897dfce4c5bf272ef773512fa8328eb3e808", + "c27aaa5652f29d382f2e42082cb18850ddbe91d2c142d8cf62d16fa36e6dc829", + "c299e5c291e69611dad567130400548bbca22948ac9a2be5d86cf0be1ed59c8c", + "c2bb18b10dc4435b7225f86e6c9ddeb04669f2b2a062cb6d28fcb9fe61e7d29f", + "c2c199ea4cefaead9e23e15c44d8f94855398419bc748403163652b18e0f14cc", + "c2da835ea1010e60a621f955df541bb0d11ad051b342041497657f9d3a0f44bd", + "c2f750075e3b223f783bba59ca6cb7537feff873a77554742e170652df62a787", + "c2fa1db7cc0f17f1b740a1d3da1b0c2ce7e1cfb7abc7f92eb342cbdc0336983a", + "c308697a46022bb28dd37829dd583edbc414f1646d6ba88d8e788d74fe75e3f4", + "c33f6e1d953c2e2459176f207c5837fb768cc7eddbeadf4619aa2777546de62e", + "c35b5d06952680ddbc500436b29ba9576bc8f11bcca1f5977d6714edc5510ceb", + "c35b835d08530179fdaab703ae0f98b31eb83ff1b1a9707c341029c0516d955d", + "c37bc89dca3af800f44aadf1d78e4f7e3a3e89af77ea633d31651e1a95eda347", + "c3af48670ea6f3200c2ff85ace910f488e272e926055891d86b862e0b353bfe3", + "c3c41e5552308fa4970fbda82afa59b66f489c27a7db4d5d94a60d0db8c8039c", + "c3f8ff3e207767558db61456c7f127af9d79c2d0eb50a8270bce5ea45db2c543", + "c40d78ed2ee68ff20485aef5e34579f5f4054a6fcf5d9b305e7291b701bad3e3", + "c44cb714ef7ca78ac4a00b50760b3abec03706a7545d5031a3a5ab948f7e3830", + "c4736ad4146e4335178db8a9741c652d5d05833fd419a4d44301e034289f86d8", + "c52d67b0644f775244057009d1d807cf959238e8b094a325b89130b66fb13549", + "c5935c567c338bdd710e01da38b97eb94bf8a373c524ffa90f06e141ca21e0bb", + "c5bd03645c8290866a1d39f3b34ff2bb3e804bf5a0d19d1c38b6f3327ead98aa", + "c5c447a38bbbb89458230f11f0b3732d0fc3dfa2e7dc93aa8177bd348c3d8b13", + "c5d45b34ecba22e99f79cd23ae74c5a3af84174d5379bcda1564d7fa6fec5495", + "c5d4d80494eec80c6ed832e4cd5132266d42a0076ac5053fafe48ddc173abaf0", + "c5f95ddaaf9299738b4281a94832b4558835d454e6f79f15b9b41df9e4f5d760", + "c5fe2f8f0295e4674f673532c94ee14bd9cb9a81e0bc9cde32814bb3576865fb", + "c611664dfe03fc5ee5c7fbe647d0d25e57afaa76bfcf90ec9e82f3331b41dcb5", + "c61963f555a95cfc758ce53612e8f0e4cd2841cbb614aff60605dc6ed944617b", + "c620c3c9379df967de6fd8f4c84db0679e9d2dad81cbfb3e2d77d5ce454aa3a2", + "c638f92723f379d1dc48135cdd1a27c1edfddbc74c7ffaab10cd4879227f346b", + "c63f67dffc9e3aa8d60e13154e76d0db63e3fa987ef4c910f9912ca63a0f6f04", + "c64430ee7118581e5bde54d8e9856c15054d73a04f4899f64ab8b59a320e9365", + "c646b12f66f51279cd7c736b218ad1f413585472da68bb60bb92369f7cc66a03", + "c660922239aca421a5cba165941bf8cf071cd9b118cdeec290fe7f5095a38663", + "c67c22645c881371accb498fdc5160ca3f3a69cebea7d152970ae608de1f480a", + "c6c3f5e34e82d2e364d88b418291b2cc75a632eab80e23a502411656fe5e2af8", + "c6d7bbf143e0a344523a1d435ff82552e0ea7c64c6eccff80f9bbe7d56107045", + "c6e87c32c76b5a2e213fde365de66d9f0fc396743bd3d759388be531c3299d42", + "c6ea608314b0fb56168c2fcf3fe90f112a36997db1f7f52b767255a85dd2de9d", + "c6f1e298f5c323a4991b06bf766f36b810701cb0522045b27f73aa8a2d8081d5", + "c71638e6645944d5989bff886d6604db71549d3e91a8138277d2f9d85c9bfd26", + "c729c560656c7822b69406e8dbfddec6ac00bab0a37680eeca66c468bf163a39", + "c753105e888b26ab8eb96284b0c6ae789b38ec094f3a428b913dab070eb0451e", + "c76ecb80f592ab61cf9933db4a4ead1e171011c713a0d4b7de404783b40f05bc", + "c774bd3b040c5bed6d13003b638a72d4e9a2ba11dcabb1ab4a80a641d5d7d37e", + "c7835c562b64fa359cc458c940feb654af46e8b10e2e84fea0e25f077b9f1e41", + "c7848b1fee9b082918121acfb53913b4a8ab7103e792b429e76c9657150e070a", + "c78bf4ee459dd9e2a4acd4a83fb6178ae4f3ec991c0571dbaf28b6cd57407261", + "c79a23775dd5de96daa4f02e7aa240eb03a1da8f63b61e81bdcb891e29416905", + "c7dec169ce85e8407d5039e463a0fe4a5686cbd6f9167dda51caac44cabbf1a1", + "c7f25797d0ee04f41ae508ba50790f03ae751892f9e24bef0a3e447fa25822d5", + "c8bb562f8f805383903a5574b9f84445414378205aab1d3d22b5559a7aeef6cf", + "c8d9cd0644ce09573c71564657ee8e0a78a66162fc7d207a8567710c8f7d7fda", + "c8e463592713e1c7727569c6e667d8918152fc4cc28c15fa75ce9b41239475bb", + "c909c842b7d32ea12f44d92ac041cbdc2c64248e9522ae465168aaff9e899909", + "c91c361212e050943a81e7bafb3d4f2b3a17c63f4373f240593f76233b48e5db", + "c92bfca22b81581e51629b216c9389ede9575af63c0cb28097310c7035952bd4", + "c94ee22263a5ae8a3bab787be129a555d8c2b662564b238c6d11cf1d88fae542", + "c9785645bf1b8f42290c3c10e0036dea320a18d1bbd8df0d8b57f974565bc139", + "c9d61d4310b1e84c3208b667dba9ae044e0acfc1fe96b96d8e2a11fd1f4d9322", + "c9ebe28a3b00f9cecc4ec7d71ff046f07fb16d4377fe33f68711a4145243b523", + "c9f778d144174b667b4657826699683e72d081916ab3d4443ea5cf1de63b5ebe", + "c9fb44774055fb5761f738df4c8bbda5c59121dc85ed27811ee16a7b17b8a234", + "ca02950b9ed42a6cd522865e22c5d5da36b6f4cd1206ce8f36fec5c2a9fb7100", + "ca0ecdc8c99fe496f1a0728042f34bb8477aac8d6f6eb988220200fa60ba3d8a", + "ca894012f7454a07dc52cb1a1f0f3ff3df6ac6c9066f13c331ed5c1d73eb0f3b", + "ca8b4797bb2a647ab2051a4a299d4e1ffbed14988e084b66330bac78a06aca3c", + "caea21412875078c8e0d019066c50138b67a46835a5cc2c25945dae6a099059a", + "caeb0797f58b1448e6f64d2636587d63efbb6e27607c26a6fa3b660296288487", + "cafff4262ddd03cd199ebce7acff792b8e0f906c2f90c0f7b556f8faaff4228b", + "cb1e6c8197a549f105abbef457a379bd4d71aa5c35d63d780733cf1f77e3e5f1", + "cb308fddcab3858dfba8e2d1705b9ca9aea23cdaf9dcbfddf4b77d6db5db9040", + "cb3d92ef090fd329dceea69ac6d48aa42aa88149701769b1a4625ba80cbceefa", + "cba2b38f8e1d3963539ceaa0303e8669a015599a0e01062fbe4e82e8f359c667", + "cba925939b83f0d297ea329100f6dd9ec95d783625be4ba508391646ad22388e", + "cbc4cbe38d4e3f010efbfc9d4a6d021783b5f8d66a9edebb9e6afc046ef0ee4f", + "cbd9adb58482cd85fc5c54d0a13ebb0e0be278c3edcfe7a42b8b246cc8b46e27", + "cbdfe9dad04b6df20ab06dc513414ca71c92b6b01d7e74b79a13ab76ae590474", + "cbe87cae84854bc4177b85a47b953f39523e61561c2488db4b4d64374cebf4f1", + "cbf3c36ef8b5885ced1792a28616aaabddeaaf368413906e9f7ff6ac103c9ad0", + "cc1299ddca29d3f47b825ac93c64b361242fa25803f328fca13441684a4c7a63", + "cc1b30d08252f2224567abf1482c833f629a31afede28893f45868a3122d025c", + "cc8b9c23a4766d262667973e449c8e1fb6f5ea9182359fe848983de8e69420f5", + "ccae31e6c91561c34ae01567e9f442e6ff4ae66e8b2ff665c8fb8199b0222253", + "ccc80694b03de71ba7eb7837ce36fd2c8ff6cc6a7e121f1a0010e21394ec7d28", + "ccc9952cb786577ce9a48ec50a19331a7e3b5aeaed521227fe6b36ca1650e8a4", + "cccab9452697a5822f7228d9ab3198a4e3f2b73810aa78705005edf91c8b68ff", + "ccccbbabccb218f9dd35e3245a04c9f58c86e3d11edef57aa70d17ed0a47ca5a", + "cce04483aafe15033058fc57a6adae63474c8886b7fbd33c2a214c02b64b66c5", + "ccf474053b489d3012904f3d2c634e04ed67f79fb7f258d12174ceffe4d581ee", + "cd03e2df6e42a8883dfbe44e1c0efc58bf63aa6f6d32d47fedd8bba1e23a973e", + "cd1847ee00a06fcac86360000d06315bdcb62abf7dac02e82799277175c37a03", + "cd1c9e3e32e853d95f9f87241c0907ac60f999a01c96947e80829a4cb472a2f8", + "cd1de18adf0a6ef2f23a850f3b25be3407533a2b7cce889d7ebc5ef5c137a3df", + "cd456c7ce5577a4d33433324f28e893d9257b69bbcb020525de57e17f7ca2379", + "cd6590458cb77b5c7aa0bc28bb00582e6f5b6693f59af5d38df77b4db73b7aa3", + "cd90dd5b59b902f716500ba7c07bdcc5f9d0265757f2d1367a47af2c143adb32", + "cdf4874cc5a4273cab51aa533b5fb08d7eb14cab1179572e46c68715013abf23", + "cdfc4db314c0d71b8350c0022310e352a396e01c94d446739e8fc33f6603a4c7", + "ce80b2a6b2332251556de83f456f5be22ddf5c98131073cfb4db9469f6c06eb0", + "ce8347d62ffbfe3ef6cc58ae9ad45b4a527df7b40b899e0d7e09e5cdfb0d254f", + "ceca7e2e8199bad3a54ecbfb618ba5292f78d1f3731bd38a7f8f9999ff8bb941", + "cee38d8376a08f3ca2660337ae409abce9f7f89102199b7b55be928111c3e34a", + "cef24ac7588db91521430146ed719c060006d2d6ed9315fb63fb24975dc094e4", + "cf66b4aae4a945bbca0bab0f912f0a584ac11475c3caa7c65afc461052f2d415", + "cf6b951dda214e04ac328b5e572449c55180815430fcb0c181cf61da6e998975", + "cf844beaf2e6c685aaeeffb989285275203a16da7b6c92a4478a82de33bb6982", + "cfb6ab02ca975d55b0a6c1878cc4a3f22ee15f7f9ce6642e1490eaac8fb8da77", + "cfc5f785a585166a29ef45f8f3887658907612b4704cad2606fdd86580d9306e", + "cfdd01086c7ef1d3ec816295c00994d3133a0c9fc0f097c579a38f67ec965436", + "cfddea7938adca95988dc306017212e307a7d9d29300a49e14781cfe582b3cb0", + "d01fe0cc347b04c146e981c9f3c86510cb9b4ac16c75439afcafa961284857eb", + "d070e8c8a5b0b356d8fbc0d5e50dfbc8cba88939108ff3bb17c8cee06ef92dba", + "d073ee715c3c64a67b1dc52f02c0645f2e16dc4b8934d870046d611511240593", + "d0a2f11cdc874d7797789d80bb38d398145d118b2f161f08d6f171b571673b78", + "d0b0069247d86164502ffe2c678701ba41e1ab19cfdaa75461bc56b6b257a399", + "d0b354e0d29108ea1816cefb8c4aef012ef0c74a5dcea0fb7683de54eaad24a6", + "d0db6137e2be2f61916b26e69df32c0a1525d91867ec2a6baccd1b05dbf733b7", + "d0f579e298d6c00b884b463527e40a31c703d780772924526fc4601c3ef12a72", + "d12dd9f2c736de3dbf0ab05b0c4715204d65d682112f01d6f0a881719086827e", + "d164afffcdfbad0c8b8829a9dfeb78cc90d6a4245e4744a2f4f9462af84ab382", + "d192fa159a7984c4b8d688747b547f3c310c85249445eda2a9bac352ade8b47a", + "d1a0d5cce1543295d9ac16a559390ca2ef9625fbec0a148a88662b3bdbc07566", + "d1a395ca68b59f7acdb1ae4b11b826a5ce25f79c9184abf7dbdac2d87d2a53de", + "d1ab005534a41df9b1eb7c7af7275121d5cac159e60361088178022ec008693b", + "d1b5feafb908fc6394d1e094bef4e084d8003e475c7314265c8e0fa1bfc63484", + "d1c84b48656978636a6b54ab2d8c5a289df23fe8ba0cd8d8f4e1dfc7873112c2", + "d1f34f69500b5ea583cfcdb117ef5cda1352b29fff00113740607e752e8d522f", + "d219e8bb14cc94caa1833f01f066fa4bd963a3624286e9fca56ed00866356b39", + "d23cd9664d80088a3a9a96950b1d0859a1e198edbc456bbeded1a83db91be9b6", + "d240fafaf43c653236bbf6eebbf58cb3b479c5f0bb18c1552eb55949f8abc091", + "d24203d0614be5b8da34b8412d279a17f0441bd4381ace8178a1c044142b48b0", + "d25a40be63ec05dd9772fbfbc282715edcdea3bac2a6050a637367a6be37f0d6", + "d26c279d4047b77c1a4a9a95383b7487e2e22bc12aee632ae36466fb6c02a0c0", + "d28ac0816f3c9f0bf18e3d08d90ebd4a752239f4f23d06bff4f47cc0c8423ce4", + "d2cc4fbd2e43a98674c6c4e68b970c0ddfc139b3d0db1f70db40e8210e8402ad", + "d2d2ec5406caaa6d416c4acbf9dba6eaa4c259b8a97dd1203fc90bd92cff0bf7", + "d2eddfd7392636028a12000fe3b0d06120fbc56d0b5bb92f34a7a52ec0feaa82", + "d33d3d89031950904ea761667310076f6b95a4cc5ca930b7b31b91131f7121a1", + "d3638aaebe3f77b329913705ada57e8ecbfe424f447bb858b96a4aea55f7cd29", + "d37f403d2e1a6ad27ffa735476b4bc9cc58d1031e7a6ee8c42ae69932c9c56d6", + "d3da0f35aa5303b3ababfc1859247dd0328001748311a88cd77376fadaea8a70", + "d41d6abf63e47839422bfe192daeb310402091b9e7e024b25ec534131766d17e", + "d41f562e5632185f7696701cf7294f8deb89416449143d857d247de6eabd1101", + "d436ffd00330337a12dc5021ca62dd16010db019b10d98dbc3d88d878eab6fb4", + "d43d1f6f23de682b557ae10b04fb03b9578a38684ebb2c75ad53680a39bc6b42", + "d4887e482e42cb9ce2d5a0350d934526145985b1e312e711792f24c38a951ed9", + "d4af0e0f77184ac84939d282b93b1018614e0fb91a40d6a0aaea1109f37d8301", + "d4c681632a45ce471ed9a1798c14e9f6bc335eb80162e5f1371611462d639d14", + "d4f514dc89a34da7c37a834886b1bd0144061a816f77fe6cef2bfd066955d183", + "d52bed0de5961ce7653d028389a47d304bbd5ec2fb492688d88e01c20c9e9751", + "d53949e80f745b4d64e9553b036572189079413f781fcd75d9c956d7a09aaa53", + "d55253c49dd42e9a2dec3363f86c4c8c1184894b8f07fe730a51c6660b1edbb9", + "d55b36cf5c50bc67d8bb6b130ab1cf86a08e9317ea19ea7ea9a3df4645d6646d", + "d5737cab82e5a56e1ace27cd0dfecf5e81a85a5b80e246956891a90b2ae92e5f", + "d581b656c72970c3bde09f7167783111a25b8fbc77dbad63a9a5042e6e1c02d3", + "d586cc1e5ffecd95fdc88554b909d8221f17d16a4726358d2bc327a2d254b8df", + "d5c3519f7ae4e585e1c3cd23634b0d912b7ef200686b7ab800b31be4b3e272fb", + "d5ccdc32319e9be322f989a297b46a9a0b100979c009b6ba6deb1bec2f3365f7", + "d5ee28369b14bfb5029dccae76c41cc3188b947e4f7c045d5945febeec161b14", + "d60003a162be418f023b13145c3b1429932f426df639d42f7eda9c12b25f09bb", + "d61fe033fb6964bad0a578ee4c4536c6634c299ac42a84d963a3743a618558ac", + "d62433ee8b0120631f7f4af20bdee7f25e9625184b009a790d3cbd85e25cd0d3", + "d624e46ef452e34d883a49229c0136009bea51e568c553540b303a7d2c94cf82", + "d639559a862fb3e47517bc78973650a49c35c11e75d16771260d838e65ce4854", + "d6a34c88ccf7607bfea26640f4fedb3197571406ef9225dbebb65252b12dc1c4", + "d6ad863a975be3ebe83c4ad6f1ce0c26beff8b91b3fe36887fd13ea576a6e568", + "d6ef592700ea7d33fbd82279ac402a40bf172cbe8b4903576bba1299252fb198", + "d6f318978802318d0e5ac779d37df16f5a0d831be6d654aa57f1e88c33c4417e", + "d71f2212c6c3b2416020d45b3e01d88c807edcf43f6b83e95b4cea0de13befef", + "d728f827d33c65af123c6671f18997b4e8f26289ed3f940faa60b4eabac80ea0", + "d74727b5d722908bbbb3ecbc7cc5b5d356a3c09635fdb78e3fb23a8d9a090622", + "d76911df66f2f4125ce433021ad75a37b8f11dc1e98310a3a5eb00bd2585c467", + "d7691a888b5923650a28fe3758cff2345bb1ae8d80ee8f44bc43de702259da8c", + "d783b3620e8f7e8de20c14c7c66a742cde0bde86556180ee97689ab3626c19de", + "d79918e84e2867f0b93095aa13d9df173a1306205e2c62a26c95d5e75af9a4f8", + "d8051dc72cec1c8f55d0163c4f6606d459ff34a268c1c639323160148604f86a", + "d8117061146d01c9377d0a4142990c2a3eb36399b490ad643275ebfac9e53933", + "d86a6cb10f44ae323616539741800437205a848404368791f18793c8697c27c0", + "d88c67aaeecd2123733c147b75b18a7e23d67ac8abaf7cebc51f004b001cbf94", + "d9064a1bb6a2b88b73173f4e5258feae5b2d2022e91639f3ddeee2c6adc6597f", + "d90c3aae92540da66419e78719f2313129851b552e2f3d21effabdbd0b311e5a", + "d93cae2b91c10e8b3f169f19ce95c444cc9c8b2b46c47e4556d2830ace7381c1", + "d93e2156fbfa90b936ae9334eb4dfda6fbd1b597cab22cb2c90be37a77932af2", + "d9406b1fc98b36f27aaae17128b74ec926f8ad01aa3dac817410f4cff97acba3", + "d956c4f0e28406e4c7b6bef2d310b7df8fa0cbd440eedb9c06f3ece7c5e4ee42", + "d957c2a65445dae0b2e0f0bc24514a284d28a55cf39ee1cee5a8d3e688b925bf", + "d9931641acc262693d83fef4e7c13905f928c7442a51ad8fdf6e1d6f74646982", + "d9a77f4a690ed3e06d58633a85543fd0c80cc41136e7f258c47f1552fb075343", + "d9f3a7eb5261064dfb2740a6bb3e8dfe8cabce064c91099800bf73db39f00fe9", + "da09f8de280cc5727564880afd5046632dc2d4650e1576b3569c862785995645", + "da11ecebd84b29778efb2c763b45ba7e11e55c26ea665e136dc6148fc81385a7", + "da151d715afef7c4222446b54520107722f3c1294f727eec51cee471cd982e01", + "dab4e81d25783c9bd0f74aee16ee181acf0af223526b364ec8945928d7c673e9", + "daf8e49ac3283bbbed06fe7c1cb3f514bbcccf5e32dd189c4cab86c74f013636", + "db1e47ccc5090a6d2281a7bb9d33077e0659d7bb7c697b21a9e649b6a3e3745e", + "db290975bec0205404d595bae0d1f9f9ee1fd7bc26f9175121ad300bcb31bd59", + "db2e86a479922463332e6278dbf55e367557fcab64dda06d3cc7de5c22906df6", + "db4b658fe727a99f255ea3957d2d3529947b6caac5ec48fe25c1708db1fb984a", + "db5d60dbab7a234266c0956d863a5f5c5a100ae4067a4d46e833133108272f8c", + "db91903e6de419db6c737b68d92f7f85721221ebe4c27813f1cd08cef5ecce5b", + "dbb5bbcf389e31bf48d994286a8395a089e7a6de641dc06567efc06a8125c08b", + "dbc1a9a5ff51b0d3d41c72676706adef865ab3e45d26ff2a05a993487a3a8029", + "dbcf50070e709dee8be0cef48068aa573818c8de9d5b7c440bd01c1719eef6a6", + "dbd36f5d6918f0b3f987d0d18bde8833f41024e3019ed93025db2b78afaf7f49", + "dbed23923526ee3ad9c1ba501c4c7150bfad61ab420e5e711e2b4d4e76192d25", + "dbf0ff60e08b430fcdaa06cedddef05cdb2fd205061012a0ae4113d9e22c7c18", + "dc1e5c4c9a22b0020c807905cfd648879658bf757e481c49d677f7ab315ccb3a", + "dc234840669526b8354dedff55c633a89fae838cdce0ae3c777b20bd0ff8a5b0", + "dc48000c38c5cdb704442f68e2b1faee6e061401cb9df35454212f46001bc316", + "dc6c45d3fb6024fbec0d0c553ceb9182658161d39902d52c837631db86229ae4", + "dc700ef65fb407d53f0af7dc467f57549ed1c4b6ccf8f743f380ada78301915c", + "dc78b2e1c869e9e4cef3a00177f73ecaf699a511a4bfb691b0be761977d35758", + "dca104eb57c18b8fbeb17dd08fd35b458a0cd4c870e1202653e4452fc46ccd00", + "dcc4579b164b16e503e32fc81f19470ff3982c05d4f61a41a063d1c25adf8e71", + "dcd3bae5bf55c08d3de86830ddb7571d60fcd64bf47134a6cd8fa7d15d10fc4e", + "dd29a8e572daee4a427ebdfcfd9398d6ff0384d4d2fce00c2eeb5055d110b081", + "dd8cb0abc314ebe78432a219bbfb7ada743bb081249088447dbcc4e4cc70694d", + "dd938aa5437ea65e7b2e1a8ab074abc3fc74de10e486f9256553eee63262b54b", + "dd95113e1f59e4c6968eb8bffa51d996a80a07bceeafedf95a1c8c58b302715d", + "ddd7e5361fe63c2df295a8cc68e8dd7678892867ff1cd46dd837175f30fe4175", + "ddd82ffd7398afcd1b42a4859b65af105c55dfbe21564d87addedf163028a75f", + "dde71388ecf1192c2b1cf1b0bb1341e5800ccf19e54417b3d6a3a791e60e97f9", + "ddeaa5c8e0724599daca62a8952f9f3dbdcabb48576783d4a5921d67bf6290a4", + "de1f40bae42ab53dc5a1a9154c96b94109cb63db2939d268d8c27be0f50c72aa", + "de2f03f07e8f5c21b5d86803f1c512267e8cdfc5580d1f7767383bc7dfc14501", + "de484ca0333fb6a1426bc6e8423a3ecf0e8cc6176a92c3d55155189b9e5bd488", + "de6da3d412c132de0603ee71fbf57148663c9652ca08988d366a2bd38032211e", + "de79493ba0e7a9bee56efd8a6291d858b5a59c282990fd67bcd24b8bb3e8529a", + "de7d61196c0267bbc0f2e2350157db8607f5ca2005af62e78b6c492a53e627d1", + "de8b3255316fb1ba2eab755aa341fb1645b172862215080cdbe07d595f6e1a67", + "dec04ecea85213d947af9b3d88ed75d20fef74ebdb0a5b319477e7f40a2b019c", + "dec68eb79dc49e5ebef9247cdb8ca00020536d911cdb3d504a44903e0f13813a", + "dee4c4ad7cce6ba20f17b691f3829bab814297ad16f25a705c017156787705c0", + "df72209a247f2662c87a21897664b840207d054f5a609083c4e4bd58c913dede", + "df97baa5c6e3317a4b7199a23efdf9739b610abcf987760302ca7749e6bee48e", + "dfa0964ccd4294a4786d98030d7f786a51de585ab9fcc66e9c5af18d7d61fa22", + "dfad806f4668af0fdeb83f2124e6049759bfc6a1ba71467f70a3e241d50db23a", + "dff55f5278fbef8d13bf51671d73b47fe02ae344308f737e751d224de5ae0c9d", + "e00b6adec3f29f19699b8ba00228d2c78f2d72bb996d0c4ca64e232b18bc36ef", + "e0348e9295bee7097b7872c7f50ca11c038e81dc92e7f76edf179e8ce64728bd", + "e09ce2a126107b366b8f8967eb0e8acd23e46f2100aae5d70dd3320ba973d21f", + "e0c8e621f083e33474b8b839fa19f52a709ad11cef6da76b1000469c3a4d84e4", + "e0cc2ed2a8b47079fd14310f898e4a16dbc27fddb31b8855b89f3d9ecc5cb42f", + "e0ee1ac0a2e7f6e414418734e3f8accbc6f9f09832a454b5ead8ab7eff5bd3a4", + "e11e96d715757aae37c340799218673e837289d9b80fae6534ec3f91dcb01699", + "e13160d8f2421179342e179055d54da9de069a1ccd13c16dc04928481d8d024a", + "e13c156e3c6b33114561f9a03066a2d1f474760c197ab3e22dda33996abc6225", + "e1490b6ff8322a5aa02caa25ee79e0e8b6dbd41368f32b055a83b38b3f20b072", + "e155b99821af5d977974bf58e1807c82d232156f53041f13339bcac76da04221", + "e15d71bda0132aa953c779dc21d8eaec2160b70dfe978c4a64c59cfaec344854", + "e1833ccd30ff1b270164c4ca86a26f362a7b4d378abe951db23af2437292a7a5", + "e1a0656684ec308cae1e1c9973c7db7ac78b8891d318efa3854fd0ccd6b55223", + "e1e126ccf4e5b5d3f928fb86c7c249c071384cfb6d6c49f59748fe062b89270f", + "e21ce0dc2ee47f468c8e7b8a13c61223d7379531ba486d121feec867fd8e6e7f", + "e22bc8f83c92edc88ee5e4dae3d64c162633b5e1e73d21dbcc85adfaf90b809b", + "e22d4e35d578d3bc10e8c0bb073103c053b3ae33c0f42e5f40ee6f843acff811", + "e2306bf5516952d77c9a0b4a41ab36ceabd3bd43413b14a2dceddb4c0342d351", + "e27557f997a9172b8decf36332ec7af376eb8cfbcce1577279b2a79db50b74ac", + "e2b6036fc7df03dd49b347b097e61a41f1608ff15d2268190f460d5d4d17eaa7", + "e2cca3b592eb4f08c768a27df03111493aa29246e2f896a08bbd1316dcc9c8b5", + "e2d431394164bd0c41c89f9b8adf6088435011a64d396a874e55abdd7269e4bd", + "e2e4de26dd09f5f661a27182517d173c2b5784b62b363fb94489ff795bbce5af", + "e2e71862d3b63a9440de0b65100f8f3a5f868f4685a2a8f514805033a88c9ef6", + "e3161696c0ff6c0c240ab5c8a39baf2cee8329d591aa4219abb2e6a8f339c0af", + "e31ec9fbd684cebfe22f4fd79d494c1940868bbb1b706892044aea02326b6703", + "e331d60266d5fb71a2cf5b4e342c841d9e08c449c00bbf5f21b41061d5e7916f", + "e3d29931e874f9fb5e99c63c1da162bd8be6723ba3b15e2a3d115353a095984b", + "e3ed4db7194f77d1944709b3080b212e4d1148881725a441b6a89a66747d152f", + "e401536f4e5869d0aafdcee910b572aed4bc9214aeb90a0b3ac026b6f2a36b15", + "e42ef36206320b49147f2d4545be3c521ffca988fdb68b93de1e150d74f45d80", + "e437847d2cd237d56bb0f4e94d08b96d54a346efbf5e00bd4403001bca402b3e", + "e49a197326a94c87b190f6dd10eeea7819b2bc890e12060ed234afdc5c549f21", + "e4b7acbe1a91e932a5ff62c651e8bbce6f578675db5c9f073bc0f4f595a4ba7e", + "e4c8ffb54e9731092c01d80eb64f15c244287cd8ce2755042e43812f9ae308dc", + "e4eb7f4d1ddbcc4ffb54b2bb8cf2f0a3d25bb5bc69d96ce8d49ca9aaa1346e6e", + "e4fbe2e7051ce629dbf11bec69934aa615a86ce36f050d38530722b082d96f11", + "e4fc8dec6fb6f8ce4464735036575318c30469a8841f3421203662d728c7863d", + "e5560868dd89e19db66b810054778ecfdfb79dbee842c17cd59d5f9259c8e745", + "e57a10a2ec65c306f28602a78f921856b4da3dde464448a376161c9207ad5642", + "e59cfb000b5e37220df6031d4b75f9caa5064d8e250d4c71555ccb917f9f4f53", + "e59db623511c79fa44e3331011355c450de3267fdee43f73a6e1c3be6d535c58", + "e5c3d31414b03e9ebffec47d7e6539abacca63883994e7797d0449e759904e8d", + "e62dc0358bd61478c2d074d0fd346a78581416f6da7f5badf1b14638501df1b3", + "e65253c864af09523087a95d6e93bb100f4d23a89d0cf9e7e149ca7d75796ebb", + "e663a99cbff1eedab7d4eefdf715b20b94288ea6aeea064d3fa5652427a79223", + "e6f99b65c2153acbba6bfceaedb81e6022862e7ca651aab3f2a6dfadd0b50d1e", + "e705fd902403d9cc9857a3cb79418b2309a2a73617d28b1ebfd9afb8affd6903", + "e720b0907b7c559e2ad11daf498bce7e81b90e04f6b7daeb902fcb445315791d", + "e7304b63fa61c5bf44279793451cce774c87dea397e7ebcb2db6415df9b98818", + "e74b7a55a37a76b850f7df1fdfbd4007b58480460e036b16cd2aba06a8170201", + "e74d07886141e7b7f17aacef7846cb3851eca347bbeb9bf9e12115209ea7a89d", + "e77a1c97ce579a6463489cdeb9d291a1deef4981ed63886f079a0cad1b71b24c", + "e7cc860c5ee1d94b6ddb825db86998db190ac87293ce08f59ce1f18720460690", + "e7efe271fb083d4aa855fc07a557b0ded89c5af326a13937164f32d4cd5ef697", + "e7feeb46423d16a543c678b1d9e622de494d4ec17a1d51340eac3b6ad0f8af20", + "e80068e7a72ce50a0bd03b687439a393a446208f30c8e9a042f4987eb1995c8c", + "e81a7edda92f777c9606500d272f71dcaad22047e5043f9760d290a7b9f5c210", + "e8314d44fdc7f77c3837fb3f8a00072e6957542d09afab98e04c0078a54f0de2", + "e86b8058966437966248ca46b07426dfb0a3f053336043edb21475056a20fde7", + "e88c64b97523bd0fcd353a58a965d2d49cb5d8d4ff848a2984bfe33ce0ae52aa", + "e893a900a6014aae7750c958e2ec9b4cff8b442c2d7330cad9ca387b4541587a", + "e8a7b252b4bd1f8edb0ab2702106d8186b92de10c4669bd7c2edc7a6e162c5ef", + "e8e85111e0a93a445476d3004291721ec6f144962efea2a6c2ba30b1aba3569f", + "e93bfc5bc342a1682e229484069d4cde0420f1092212a91dd88b20618aef5f3e", + "e9748c12d086fff748c634103440f42822b4d30d330de751fb7ebd16fbb65fa5", + "e97a1a46b82cf13f9847dfbc47322620100f5113275f800ed2440577f989165f", + "e9a667208b6e17c44ae2a228803f68bc443fd9890863921ba73faa43e0531ed3", + "e9c70d4e8fb45ba63db4e9e130bfe555bebb46a30dac84bc65efcccd5bcf74fc", + "e9d4b30aace05a47643d8a3306b7607b465c9af32f343e6c27e15afbe2ed9ac8", + "ea7322d351765ee91211e726f2a2e71b148614ff858476f2d314faff22537095", + "eaa227c5a58d2552cdc53b0ea798ef846e669c58bf6e1236a9f3ac8df3d5bad7", + "eaa547e3196dbfc5bf312fd6ea2d6860d9ae18fce3872e91d94a259e4146bc55", + "ead919499710acc08c0afeceef3fbf1e194220008ff82f204e997b21f0c4d1ae", + "eaff850268079777284976516c8b4e741a24febc416e9616a9b0ce64d0a39bc2", + "eb078772e8c9314f42cf175c06ec13c2a59fa5156144670cd43b0cac5ff3a7d1", + "eb17893ad911455c681dd6323aa29fba6306391cc375a157a2febb36171a11f9", + "eb5f7c73c14782faf0db5f533399f067787dc7c0042fd8b4dc6117e4b7dd4488", + "eba58ab888533f8846fdc9933d6d36e7325f463e132129ed5e040d70375a2393", + "ebba08751abc0f9c4f5074fae111f889f1cbc096694fdcca5011e5d753d5c630", + "ebed887eb371af718ef561d2d7488b49d834caadd3af8c5a0544b68efb7b7526", + "ebfb11f55ace4943dcd0b4c8c78d8e3316e1d4d22c73aae77880a891d2273e12", + "ec0c3c07db268251f8cdc4756dd27ebdd9a37526cbb44d0cf5a9c1f2ed52ed7c", + "ec4d703860817d225317b3ca6fe99f767d67bf59c14201c1109b89fbc2703fbe", + "ec4de8153a1f4115e7e6eb09ca88b9a38349ac2993c137e62bbf7d638e1e5415", + "ec54410110150cd6c1ebd1d382d132d0627034a939e7b6ef93c2170db5a5ec62", + "ec59c3922f577bac3bb7132377d65a763a014b6cac1269973bef2f8541a63075", + "ec9081187fd745bfa1c3c7f9d05a46149def1200bbc2772183179385df2525fb", + "ec9da9a275700cd08b7f6cc22ad718bf73621761924b7366ddbcbf6a783c85a5", + "ecbb1070a795551da662c74f7809c8ca608ac26e70e1ddca1c73c026143465d7", + "ecbb17cf37f94c9c7b3f89231851aafcbe0925e7e4d7eb96d5ff54427e014aa5", + "ed0b2d5e9d81368ac6029b17c0b2126f43dd2eec9201c76bef086c3a4f0cfb17", + "ed1e78181721b3f5aa283d9ebf81462ef18abe21419874ced31934518045a5f0", + "ed545e76d727586c418f0744d339402358a3aa5540da6a397c659d86a0bc38b5", + "ed616c04d4a39d94cc2066556580c2263bd90b6fe017b8fe86b8ca13f019ae4e", + "ed7f80b722d4c5d6cdee6e8c5c621f956149df086e3723cd4064ba0d5be6838f", + "ed82fe48d398b7d2cf5fd18e30e272c93df532b8c2c1467ebf9ca356c33f909a", + "edc1ef0e343b63c4b1fc0ac740deb67667e878d75a060dd098ed0e06050f649c", + "edc5ed01c736565a07399504b2f911ad7a8b02670da0dc1eeb2ca5a780cf8d09", + "ee3750871ccfbf83bb92d4c931e88d78a6e592eb39409134c1a18c8461722172", + "ee9448da8478284de63644ea593656cd7c9fd513177eb3e58283ba84a8c1079b", + "eec253d9e8deb02ffa7e8a78467d48c67b5e593e36f46b2cb74fd3804823d5bd", + "eed982158217d01a1d5b5c628a6aee7f4aa9758aa44e35ae6b48ae4b0b36bc23", + "eee65bd6251f9ce7acacc87488126d67cca954487a386830d5e4f3304bfd5b70", + "ef0211b2a54bc7071af61659e4a0487f611c1ffb44d722cf8b6a7f47a266559d", + "ef0aeed7749cc3cdf075c187cf83633e54450383a329c6e2c0dd9d9c893d7271", + "ef4c5f83618c0f55d9ec021f80e2c8feed4a3044076ab4825cdec43b091da1d1", + "ef588ad2795460519841155c165a062f282045f733d1ed6d5cb1d548a306fbf1", + "ef8a439b7f1d5e93bc57ef1607c0aeb59f791fe38a8a8280daad941542de8cf8", + "efad264184bd7aa09ef2eb1b312f42ac1954241db19463bb7ce5f07e3b9c7419", + "efd01beab68ff2d4fcb1e4f0286207fcbfb74427295d34575d8ceb391c9c5806", + "efdf1bc52ff67b6eca8933310fe35f6e2d914fe396c9ea7b65cc8eada41c3fa6", + "f021fbaabe1c089409a58f516da03fd2e19c31b074e05be889fca1151e50fe49", + "f037cd9b21c0dbb33b365f4bd160389d80fa5373a748b3dbf9be255f0fe64338", + "f03eaf76ad7edd7350374cba202dc95389556f56bf4eb3c02a32c61551fb8921", + "f04338b6f05255d1a4efc150f3a1a2d5ba1533e874fba4247596b3442faba88f", + "f057e1afc93790016ca44dc2cffdbf67c7d98134488954fe16fca88479c0f422", + "f0611536592f851e763da6c07e4f7b6fe767250d886b1bc1f5ac527edf740d29", + "f06b9f2044daa583f152be142881eb31a0b7caf424e1e2596359a7d2eb00524b", + "f071dd23de9824926063c391a86f4129f95dbaf9034032c70f807bfee403f15a", + "f0750574f2b36357f51ca81b9248c6b2d7154de18878f944a4a438a7d26e26ce", + "f082fca71ee6941d5c60d24a7c7358b281461c8c0a2e2c55e4373f220c7a25b3", + "f0a1e26d253c2e1370acaf11fcd05ada4585b63dc572ea53813e7f0870337be9", + "f0c004c62fcb2c23aece72277c417f7d33656d9d9e0e05187494ef6f29056018", + "f0cc7ffae79b48422c342ed7ee7ab2f9c117c7d9b92102ea48b05a11e47402e7", + "f0cd108d4d1b11a4bed5645b1b295f9ff0699a6c07434ee79fb7764684c53617", + "f0d7c76e820802f368a015e8589209c0b1ff010af501316b0e4a82e7590cf821", + "f124e0aab0ff513e104eb3ba2ff2c974505d10ffd7bfb3087781fca6d1f2c852", + "f12c64024ab41f66b085ec4528a3d78c82c6dd83c61812ecf475a1e0f12f9004", + "f13265389b401334fde5646bc13b7e7c4f8a05c630878615034f77541b1396f3", + "f14f547e4ae201c702c05d04b1024a226baab1f7f20e74559c94c8b5800e0359", + "f167fa0d6df35de3a5ba0b255a2ab45436efeb3b143a7cdaf0e8a6a68ec77f6b", + "f19f0f821f7cddeb39e9fce9ac5bc78d6a24e64275b28b2290779ea1bf99fd19", + "f1b00f410ccaef0d7ae613b337569f38d35d6520b51cfbe13970ba7263367342", + "f1c038bbd103b1de60b0ffa944d940168af7f5cb28ad2f619f58a53c5b63eacc", + "f23baa5741bccea2fb140328e4e00ab0f5cb1e1e424701bcd5a4c4be33f6e098", + "f23e67c6b6e6c08a3e110eaa2331c1068330ed9eea335a54606d1d2e156ed26f", + "f25aed755465da7d5c3b4e3c95a25600cc4bc635e9640e7ef993d1c38040b344", + "f2669e86800b3022fd2a15123b693b94b45af55681b255b85a85dbada47af1da", + "f2a404f19cfe1b22fe2af0d23235a5f7146d784a3b1a0692d6bb8bdb3044b552", + "f2b1b88aa102690a9c394f9214a04cfddc9c153861459fb689f46bcc63f8f4e2", + "f2b5f26abaeded0cdbbbd330a97f60a2cb3af54e0e4014f0cb1c0251a2989be3", + "f2f5a1978daeeaf25c8ffa085e09762e27630f6001497d9ad2be7923124031a3", + "f3aae264c5324b53d7c9dfa3c93a217cc9acb8b59786172b2436713fb7bf04db", + "f3acee3ba45918475e3cb3f57aa38f918024778585bb85d7ad778e5e8ff12046", + "f3b1ea2e2365db574a8289dfbed5fb65424c09de311a0297f2229dcc65e52243", + "f46ad557fe453b456d94d7591a9482f084e3c502e334a445ca0cfab1d465c943", + "f46bed6638a9631ec01f7450a8edffdfd1cd597efa1e3c18a7b971ec9b56d5c4", + "f4826ae6b3ed964c98c870304a5985692a6401ad5d2bd799fbf648f3e455a44b", + "f48b5f5117b250997fe6c64af116867753b003ba870eb9e70cc511a7d6cc8830", + "f4b1501caa6b19f21b015ab1cfb44af7a77d9cf11436e9b8e74ed73802fd4274", + "f4d2e4d7dd527388994c2677651db31a41e812bde44eef9c6683a97ba479129c", + "f4ee92a108e08b0836423cb6772b501eb452a4ae5400192dc88bce99518f91c7", + "f4f047a3a53e130dee3f20a27f71637e3b13c6437223c8a641b6b4a9f6df27ef", + "f4f81e3dda331425e5f951652c77ee7deda8d3fb5cbc5a2bc1e9ea53db2b9320", + "f51a262120089fda895a9f80697ca8b686e8c057f19ee736daf479b7826271d5", + "f55b877df863c6b4a5a3e233892e11c00368a8a49ffd2dfd7a1feada89702cd1", + "f573ff54d3b07aaf198fef31cf632c82a409ed59eba85845ff96b0367d964b1e", + "f5750ff4545c2651debea6d7c4464bb4d92721eae8fbecea7c011af94b084fb5", + "f58cceafea23ddd305ba490e7f8cdd8ae1df5b02f66e07bd698b2d1353ec578d", + "f5939607dc859b81f8b9eeda2fbb4bf58e9ac3f3f259f2141284fa44fe350cc7", + "f5deaac65043d57651d7e4530e62761b8014a54771fb2fed110eb0b30e86f3d3", + "f6041f907c89e38602fc4f723a7ba880ba6ba6293494427f433675fc139c5740", + "f63d5f0e4107f691719d2c7e1b0aa0fa40a7fd868acce8040031abdc046fe06d", + "f63f72c986c0997616af03a1c7a6a2a01ce412229e7cd8f2592de4b60789a350", + "f65732cbd65b8f3f8b4b3257ae9be69f496d828b288a643c54070577e23694c8", + "f65b8d423e8c5a5869491f91860120e2ad932f87aa646d51ccc5c97e478e8b74", + "f67204bc7998220c42535161b3da945d9522fb4f66d804b7bda97391d18ed41e", + "f67a153cc00c83debd8241b7b91eb4ebc6efda81127a3143614b4ec202362dbe", + "f67c79723180993631bcd816141ea11c780d74ce1f167dfd20d64eee34020ee3", + "f68e1fb5c0a10c18c1b436659a676060f2ff347549074643cc376516187d5e18", + "f6b57a0cddb03aed35a73ccd10e928ba35eec1f593614a9621e383a15a4bbc59", + "f6fe4d7f7d890ab3e96cd6deecace45ccf2b7703dfe4d7cbd19fc38bf96a4fd4", + "f72103979549dc42fd342428606c57e0056e1ad7aa6367c9ad53b6eae3ba84b1", + "f73acce0d0caaee1fbb643dfb565a921a94617a3349d93a8469b9760fb7d2dd2", + "f77aeecf0782ee72b26dbd0bc2afdd9474c7a16608b383e35e8e7247ca1719ca", + "f7869b4843638e7f3dfae398dda8e0a7560cd9c5aee80270a5267ae8866cf734", + "f7897905419d70b23db05fc0a1001d0b1a3b0af0f832117479edb4e0143bf2fd", + "f78b1994b24b4fdae392f7b46573346d468ed74676c110875db6860536411972", + "f792dd2b474132905f755c2f31fdc6d04573311b6838f293d104ecbba7ef9bf1", + "f7a7d978a948f36568666db2f266897d1d6db97c881afa967813a1d90c861ed0", + "f7ac2ef367d2595d4316d92f2f5a1e0b3e3ef0c52b13bc6903ab626e0768daef", + "f7c9b0a8be9f9fccfb26c2cd67badd6c4baae6fa4720cb9b9ce32718d0d42ce0", + "f7d15ee450b14cfad3a63d8a40a182d6cf2d729e5427d1bd43b1d514fdb2ac18", + "f844dac31b49b7e2be2afc4507c6835c6275992bed428d5c2811406eb0636ffd", + "f8856132a7ee942f9935d819d8bd1ac2e8db73d3ad1e3504231ec44eac8b4b2a", + "f88d7b87fbb90b5eed6107d4ca238347cbf7bd9960d11e6dbd6202336ba47a48", + "f8bff3207a9c4d7acba4b30d1fe9159776e6dc1cfb0a684cc73660264474adcd", + "f8df2bc65afbba35fc3a1f266114766feec630ea2a193325a0610fdde899e85b", + "f8e8e4500b2f9425a8c19279d62346ba5015130418ab023656960a73308e39c8", + "f8f3be4f9c461671164909f3a6a4360a0a18664a5783a29b2f869345d28689ba", + "f9044f70f67651a5074910319f059bd29ba17d5aaa643cd022af04092ec5b8fb", + "f9373e58869e091a50e8fca190323e20ad36dad195a8bf0aafb84d405ffb41e6", + "f9426611ad2ed31b5fab523266149e5a7076e35454e8b4b21b69761164b56c46", + "f952fb8637baad559eb2a8d8541a65dbdd0453b111a4f459b15c12e741274545", + "f963862beab529f79ec7450689199006366b9f9eb602dcf88829b30549b0dff9", + "f9b818c580b690669d02210dfef753cf6d8cdeee736dd43edf3e3873be4f3593", + "f9bf5accfa2f7af0cd3c16bc75bdbecfad15455e969c43a2386ea5e0dc164b76", + "f9fd406f361fbc59236ff4e5db9fb03bf34b0d52a314e50b9a06463060c39b9a", + "fa03e828df8c0572b8a53b3ede30b1093736785e63a68fbebcab58688c8d25fd", + "fa03f44e759f0f2b45d4ae60b6bdf1f883c7b28c8e584f4c9dbdfd0da8acf021", + "fa2d7063043b250978556d2efd5c69280ac146530785e68eded6c8c4a2272f2a", + "fa3375c4d494fbde579c72af92d9b836e215537068065aa024adda70b3e2fb87", + "fa44d75d52a573658e4fb168bff6591313105d2b215ec101d87c8003e5ae1ee3", + "fa601c703bdf0f05901fda4215366afbd6b8da68a13a1acb1ae5a7d0ca551dad", + "fa86d37d13aa868ffe88887c502f54675183fb72a6bf0f0eda73353cc93a6f1c", + "fad28332415da4ed66e0a02710033fae4e7974bd895649061f6fe1447aa48eb7", + "faeecb415d0620338cffc4971adac667836da581e69d97a32e8cff554889bb23", + "fb0c91e2f07bb12e03c91114d97b28f384dacc22f55c9c2534fbb5341594ad79", + "fb1c4b9dce86b632b78ffea1501e361fcff7a4d914c99eeb5646a1483c17a75a", + "fb1ca8b33a54a6968735c70926614093f25f1485371b0b5c94a50866f1561c02", + "fb4006f12e6539562628afbdffc1114c6109fcef4021e01a31272d54f7973643", + "fb61451de49728bbd653e2fb66435c9d7551b5716e4d795fb8ce5de400c31628", + "fb63b0298dea67ca91f29157c90022fdd2f6b692c615cd54213b93b435c04a68", + "fb87ea8200382d0ee7b1a4ddf3ca0d7068d3f3967fbc1cdab44b22301b06a094", + "fbaa467be2afcb06c9ee22987e7ea16fa3e79c192296b6c4129f07c8f60714be", + "fbb08add215e5c4520b18356ee5e2cdcfb7397916a93d3df4d9e0dd0232af675", + "fbb5155bf20f9145c10c42fe56c7cf1d1c6b7a8d5f7c748184a431c101113f0b", + "fbcbd4a4a21904d22d4d54896b1429e206a079dce30c8eca0933af05232a14a4", + "fbfd191d086e9d8c6ca4a41d248079bd285124897f997b424966b625f3a67a1a", + "fc31a936db1402433f3384dfdab6f361b46d3556e4c7d95f52bb844018d491b1", + "fc4ddb1b80c7669514ad1ff5d62406a0fe7e8688782d5189a850dcff081340d0", + "fc6c4e67ebdb78dcb98619014a4419b074e164bd5e992991575dfd2a66291e4b", + "fc72d6536dfdeb226151f1905bd74a43e8f04d2e7c4b12c36f6c3433c50d0ce7", + "fc7dd06302e49c42e86e4e8a4e9e093e918964e38a9e35ce206109c4b92b5391", + "fc8a7c5bbd30a4fe7b08b44f2057a67dfcf5588a1ffc249d405f6bf9756da974", + "fcc5e54f7c00f05f6f78b3cf6c1daeafcd911a02240265d44cbc2b853d6a5ab3", + "fcc6d55972e8e70cf5ea47dab7fc1b7efa3f4dc786c5a33ba46e4ae892947a72", + "fce6f3c6682f3b46233f31f7424fe98ca8855762fc84632c4019464a3c06caa5", + "fd057104993516baf2087ccaf9a5f36fb95373e33a94a82810c7a7ad048306f9", + "fd18b02e97f3f2538c2c21b3d618c0f0d47678fdd8494ad14a0fcc5ad1bd95f5", + "fd2c9a9f34990dbe30ab307f0950e8a255c1d575db5e24ca54618ecd92be3263", + "fd2debd3f338a433e73fa8321ffd1581b173f088dea2590a4850810f55616bf5", + "fd44d05b0be2cf03d5487ef0eacdbb3b76e5c00fd547db7623d7d26cda7fcd3f", + "fd4bc5136625be7774541ac903c7c4f174833235a42a9853adfd8371097f6fc1", + "fd4ee2313d81fb52f0930d955c668ed97fe3b85bf1a1c92c6d28c7253c84fab7", + "fd68aad1a332f4bca84e4a4c75d469604e8f51a2bb6e02f734349ba870efac86", + "fda15d0579879a85bf0495c5862956f75fb23e793ba45977c389721b079c6c77", + "fda896bad99a7e6f63b12edb5586f732b73ba6187afddea5b9f2c1b9deec86cb", + "fdc4083896df70525d6cb8838de7de6e510385561045aae9b6a11d9eb7174957", + "fddf26cb81896c40e3033f1d43c911b706fb58b681757d1c607e5cd8ba997df5", + "fde50489ec700e54054b2c5a63ee1b596e1dd9d25e1178b9d97bda5cadfb99c0", + "fe0d45c84237f46dbcd90a2d90b7d7103a5b1ae79d9641dfaa7e7b5d7c155382", + "fe2db61e327fde7d349bff423eedb5664b2d8d268a1050d50d700e44a4a4ec84", + "fe45796d92ed2ffa9edadee0a8ed0baf06a01ea06f7e2b3b280a3660399707a0", + "fe4834ee67e3dc86f371db6861ab5514c8451744449c22c80ddfe0855b2441a4", + "fe5b72d2a8a29732c5307f43f7212a492ff5720c9eec8e98b31c01b57293e447", + "fe7e015b2a149f7e3addde43798f0860be1f92ef8e0beb0032e9282e858f1a2d", + "fe7e12db4ca7b495ab3de2b8e812ea23219baf38291e7a5836944eee383c1362", + "fe860d68dbfdccebe089b378ebc4e3beaea70d1a8b24d2334c47799a5edff5a7", + "fea5a8461835b1382a1110a77224df1ce8a067bed704f1a6076904647f068b1d", + "fef252c6261533f1b9784314052f8e1e48ac6bf10dfc302c24b718ea18b32274", + "ff22d9c91fd4d74876b2b41c3a653963d348c8f2040039d5bd0af665a5b59af5", + "ff3168092a52292e5e43e7db6331bc539b4b37f1df12d19b1c616ff3c2dc0ea0", + "ff3542e42ba107e85406f63a9d670e4d53f34d98f13b7e53baa3de8f44b05879", + "ff3f03ddfca7aae413d0716ae39a988d7964e752108c348a4a4423616d85f81a", + "ff55ea16c630a7b728e6616bb5427410845da8046007564c6258d7d02b986859", + "ff6427309ced42e9287b4540496c6ede902122ee4ca6763c4267c75792f11ef5", + "ff65174fde7ad22bd4838e4c6a501c904ea8406aaa9a233f1f15b266abb8e1a1", + "ff7e5550a6afa9804c92139375dbd8bdb6930b456f9266869cd42c7384cbc6d2", + "ff88a6da61a4a0fc99ea252508a180d48600f13212246fe266b5a02b64ac14d3", + "ffc2588c1509c663c1406c1494a663b87978bc0c3620feedae2857f5c22ca4d7" + ], + "include_outcomes": true, + "label_counts": { + "negative": 24912, + "neutral": 6636, + "positive": 1156 + }, + "leakage_contract": { + "deployment_must_not_read_outcomes": false, + "nontrain_outcomes_are_evaluator_only": true, + "train_split_only_for_retrieval": true + }, + "max_flat_action_dim": 112, + "num_groups_exported": 2044, + "num_groups_scanned": 2873, + "num_groups_skipped_by_split": 829, + "num_rows": 32704, + "object_embedding_content_hash": "fe97ac428d6b0ed4da7060eb6f697c1392b4a8a09988a10c0f633b1e457bd568", + "object_embedding_manifest": { + "dataset": "embeddings", + "dim": 64, + "extractor": "rgb_object_layout_v1", + "num_embeddings": 2044, + "path": "object_embeddings_rgb_layout.npz", + "reads_outcomes": false + }, + "observation_embedding_manifest": { + "dataset": "embeddings", + "dim": 32, + "extractor": "rgb_jpeg_stats_v1", + "num_embeddings": 2044, + "path": "observation_embeddings_rgb_stats.npz", + "reads_outcomes": false + }, + "retrieval_index_allowed": true, + "schema_version": 1, + "shard_content_hashes": { + "charts_00000.npz": "43e04927549fe3aadafe63067bb82c16c1167c54d0c8ebd47d4114a814f338f1" + }, + "shards": [ + { + "num_rows": 32704, + "path": "charts_00000.npz", + "sha256": "6f3211ef91cac305045b540e71fa40156ee697abff9db3c8f994149a2fc1b663" + } + ], + "skip_counts": {}, + "split": "train", + "split_fractions": { + "test": 0.15, + "train": 0.7, + "val": 0.15 + }, + "split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "split_policy": "stable-hash", + "split_seed": 0, + "state_hashes": [ + "000b25ed5631161d0436064f18bf2927ebfd97feec3b027cb7a3fc7f802faf32", + "005c6ef90461f4461053d2f248598e4dc200fdb34d95079ba9b6ee9cdc796282", + "006bca42e3c79ccd3a2cfe015bf7c713f69f779e9f9b6e05e6d60e05adc1ef2d", + "00889249a42766e52451f1930d859b4e9c6a18eb2d2a23e4a466065e1ec6cd8f", + "0091b5c64f92215ad5e12cb639483bcb463e589fb1442461b3b7f2e1f99bd18f", + "00a20929cfedf805498d877fc9a08bf4e18093022ea54d5e205ecd22b0aff85f", + "00bf989afb844173147aa217cda23c2d540be3549b8d54cee51524018f0886d4", + "00c9e85ebf618703709c8e915feb24d5f9fc6ef4b1ba77a7fb4bbd07f931b588", + "00f7d5c91383370fa6a6cf7e0da5227d76d4148a8d74c28136bb2e2c32e9d5b7", + "010ae081b71f7c0e58868fcb7737f45102882974c383ff4d8c1b02ce1d895661", + "0117fdd7ec26170b0745e65b08ea88008b09508b62e77d62a69473f535c25715", + "012a634e5771527516fd013df470d5b27c8a795d224f217fa9655c378cbdc6ae", + "0147c0991e5aec13d38c9a12625b9824d5e7a497734845173f31878b6daf3369", + "01642e4ace471a0ee60cd4a110c921359d11750c72c4a4a17274dc285f8de946", + "016859c711e2d334a59ec1acb53b154655e7875d1c095191ac681a0d6f2d459b", + "01a740cfda480c25d931fd14305d6b661829ab5ebeb80ab3efb4bc1b7d400958", + "01cabb034d28efce276c7c37f7c4d1b3fa2c9d00b140b109d619358e1a99ac3d", + "01cc39c8b7c4511f037106757c86d8a9147e5306a4d30e1bae3b41bde0455895", + "01d46a34d74e86012c60ccbcd0cf56dbba216277d41a455b8f04dd1367dbf373", + "01d93a2edcd9393b7d5f2543c37638ee18a7f731976a7f58e144de69747df1fd", + "01f9e4c4ee47d37a28ff9581e3c433583b0b46f8edea284260559029e7c04bf4", + "020d8719f66fdc0b0c47e07ef57b3ac2a06173c2c743a699b972fe3996758e9a", + "02251ce01393ea7b7a290137bfe09e465ff090cee97d821901fb100038b3fb16", + "0251eba2dbfe3f1be9cbbb27321cef00827b19394ac802bde83bcce72d7be6df", + "02752960d18784921c0f4b6022bb7a034bce292b03909ebbd0353c12d48ab314", + "02860b143f2e95695756174fceb8ddb331e7510aa0454f124250edb7863fd5c5", + "02af5467f98ae64c370ed93e1b882b5259adf7e1a41a00e36d0b0a64bb6ef756", + "02df6abd276ead475e81d9a264ff6c4187b0505a8c8d36e0300ee9fde4d4c731", + "031cb9954afaa630fd3a550eb9641912cb237d06db40d1acd81a7f6cc7ebdc31", + "032090600396eef932cda799b7e4ca43ec002031b155583d41ddf5f884b0e066", + "03422e256c0166427e92805dd77424b2025b7be0dfa3d2cacdd17e844c91a80a", + "035290871ce5028fce982cb3000b928e26a7daddacfa891677740136e5f415c6", + "0393682c39f9ad90feafe5ccadfe0e5cd11db16f15286b404bc4ff620d0a9164", + "04016ddff5f00fa3ef76c2246256ff48d5c7bf896cb3563f358e0ca9b5cad5d4", + "04042fe5cd41177bde33eb3dfc7c022d68426c8bce2b386b6c3d744b3b6c8ba9", + "0413d5ba033ddb1d8be7f3fd905b3a3fea63495b9e2bc6636af73a0cb799b926", + "04463ccaaca5d28ca99f63f4f53e070efa4ad2299568bd63a9105702336b5ab6", + "04553237c84ff09a6543e606ab9df00f3fdd3359b381e9cfcecb9f0cf8f66c58", + "04736afd75d9b70defe4c540c6b91495c1d483916acab8b9cb88954faaf5d7fb", + "04891991eaac4f5fe2628582b59594f435c2f81d561f2c42ec1dd98ea57408b9", + "04980e08e9f08128a95235e4570f5c3b26e9900930f8e1901b1cc805055d835b", + "04e020a7f9b5a6b03c4d0cf02f1a5226b30f77918560b372a4a86342886926b3", + "04f06ba196bb7367e0483cf316dadeaec8bee82115844252a7139a70a7c202a3", + "050edccb9a8c424bb0da9dbe686f19e78196f374d0a0a6d8f47b92cb2f8766fe", + "05165e6acce39716961beb3bb0ee00c1158ebf8fa3658dc70048d10cd99093cf", + "0518bcd40edaebdb67c915e851d4938473f26ce6dca50a21e69f83fc32288c89", + "054a1bb649b430a86f7802a9457cc6a83d15cd5b8ab6bf8adf410843ce7aa13b", + "05550c96fe8ba235ebbc76dd6570b63df19358382aafb3937726a87d268713b7", + "0583a22e745ea93e01a9d2f9b53642289d9a05b222a73c0fecf8e710fa1b59e7", + "05b5cf5f903d65c126181a2bf230af182068934b33d8184849b6b8981f68ebdb", + "05c6d0e8769e7b9de479e7e02edf2eeda58b5ebddebf2e1f15c962e257d49e40", + "0602af5175c035dfbe398a4014adbeedfe194d75ed41a0b7eb8a79b48ff6bccf", + "060d8bf103ac5fde82e3703ca8e2578b4f8c7bb7c5cb738f5d1209b919c61dba", + "06257bd3fa90f0afcecc239680999406ab79980eb303d45a2d1fafea670e4a38", + "069f55a960c8724588c1ffd62afe2fd0bd83efd2e34cf6a5168032643e6352f2", + "06eb82c273f65f490d85d60a39406f6cf2318d14ee9596ecb1cf7fcc3ac9f8da", + "0743154e958abc52c160fce6065de25d968ab60c7c7a536d400d2295bfc27255", + "079b15a218441cffd25d9f3aa501b9143a3292bb7005f9eae05704e55f3b38ec", + "07de76536f57dde25b6235024f5a41e215d6a13d9f823e2ef50e4e84c87fb739", + "07f15fafa5887d8328b676cfb87c6f8f7dcae9cb187b8b6969e2f3a4ef37e8d0", + "083bb24051300e75dbd52e0202170794bc727994f278922c2d158af777a3f2cd", + "086bbd948427f350f1d964b288a729ae80da38291e409fb7e0f0c688d99562d8", + "08713adb8a3215938b380fbc0c63f235e063847962fe02dca3f6d20113fea1cb", + "088e20d6cc294aded49228f0ac9a7191e7cc9fb848bc19a6ec9d03fb7258748b", + "0890c32a99d90761a2f31733eedbd91b1980913ef735752d889c0710fe572764", + "08b122b9a1c8a40b5965898017846a28d6377690db7128b7b31417f45d1e407b", + "08b58c89333a6a06c7714c86704b3f06bd2ad23a18360ca5e8ac7a2bde0649ce", + "08c6791c2cfa2927363fe34b6af41498531aa6816ad3edd85c3f9835a1ac1014", + "08db846838425971e9024677f1cc8e070c502bcbeabaf05f6f3631dbed7a3ed6", + "090f2f8b58a9dc0ae63008a8fac34a18b2aa93522542bb472a1202fa81e69465", + "091953502d8a89e9db47809dba001d5756c622e3aba8318722f8da0054bf92f5", + "092e168f994875a90490b287f3aeb7ad7ec27428bfd07d0e1ec1645dd2e01af8", + "093675a243ffec08a1c179a6757e916d5cd8dd9cc46921469e62bbf3bc12fba1", + "0947a97e05cd96908d88dd2db1de6be0175c37c2a635e5506ab0ef60723e9244", + "09a45c5c82ae74f3fa1dbfdd04e9c9cbb68ed2af9f96dbf6cbe731b22c5f00c6", + "09f192983656d6b34d005d8824dd443c5aa2a86212e1ef363d3ca398a61127bc", + "0a2105ebcf3307835abd4a7f5a7ea78b1b8061a118837a41879beec8ac9c4333", + "0a39ffbc1c4821c186d48f34e505d1dd889f98280e19b2967c7296fdbd4686d9", + "0a7a2bb9c6b1a50d1a9522163191dde71a243bd834f093cefb2ff1481dc637bc", + "0a7da33713c8db7bd3bc44ae27bb34e21ee7301152b2bb9d76626ec934e57f5a", + "0a994de647582a2ed0d6ce05c5ee895783f3d0a242425bd51bce5a205f5e0c58", + "0aacc576c7717acf5a36d239d777b8220079ff790301e35589802e502b65cc6e", + "0ab3580b189b821099bf7214360927a818a8e415f10d4ee7006a7d6ed518446f", + "0aec7bce1ce582391a1644d3411acad983fd582a91af97f199631e487d483d9a", + "0afa74ff678a87361267b42d66b59fab3466e8dbf8b95a5f891c25ee4a0a6ee5", + "0b034ef0b1596741e3285665c5bab78be118eeca05fac3b0685c0e405e60b01a", + "0b856ee1f7054bfe985a8a4929618c2f8c241b97d548c3ba2edd0ce3c8636aba", + "0b8bf874669ae24a086caf5364a73cc92e86e1baa7da60ddd21cc9dc58cffe54", + "0bedc58061fd753c26bcf63e803d240be5e7f5a044b0628b5d5e456b35345fad", + "0bee130933c311df93b7e72ecb227b6894a91763ffc616339264184411326529", + "0c0ab38d6a39a42e4cbcecdc53231d83e1b1e645f5b24b1a6fdc8ed2213d108b", + "0c3ffc0a5ed70915980abb6a49e16622554ee49a5730d559dddfab9115b6ab85", + "0c7a98ae4b33c672b70b431ab840bf764156ad0a3143c8df83ae1af6ffea6551", + "0c95367a82ef1977f91b7bdece3b7d452ac5947ac67445d20680d4324e6d70e3", + "0cb834d2ef4c99da95a871846baf987a939a014b4cca2f156974445a21c89196", + "0cc76f5c6a58bb1c614293fa3e9294f64fd03eb893cbd05576a9e93ee282ae61", + "0cdbd10f984a65865896ad1dc53188dacf141bf7ef001a0762752abc4207d819", + "0cf838444d8848a8b327d97bc22b1e3ecf4c14bffb554ebe9dbcccf35078d516", + "0d2cf77cb168280d8d9d3b9ca4d212baf17223c4da3f1f0dbedaf7d95bbfb42b", + "0d31cf5b5687ceab9361200aff34004ff93663803b4644b27d1d959f457d6090", + "0d77208f3083852c88c631f5a4102ad7684253ee7a8f2ecb5105725de98a9f22", + "0d84d509f5df6bc0a68efbeece7432b092fa13b52609b417755a8db67632970d", + "0d95a334465d12b48b4882fcb17e7bcf3c9c1e4dc69e753f044d91f7525654e0", + "0dc074e26e6b013f64ab01a46d120a9f9ddcffb756692e9dd7105039c88af908", + "0de8d03b14fedb9561b9015c19f8ecaf855b222a2423b0fc9a502374ce4a6c2a", + "0e0cd8063d131f6694b924dbd28dc10dfaf98b2f7df5ff6bb011ab411cc528ef", + "0e33d93a2b330cc4879f36904e581a044e9934fe4c2f28734201782301bb9a40", + "0e55cd1f05f691f5f7316ced762325ce7b77786edb3e9e65486d53c6e34d123a", + "0ea0a077924dd4cecbe0a3cf6e56b97055a69138eabc98f73d668cdc55a8d342", + "0f5b30575f5d8d9fbb772dde91794d51ae97b2f4a84b8986885d9ce9982bd455", + "0f60d64a0247253787e778e9e05d247a67c0c1e6ad497d8d596c13a8515015f1", + "0f76c338852c84ba09db812d81c87051f18e3a4175f866f1748dcbf8051d58b3", + "0f87489e07b3fd86800dc412d9dba47d153dca7d2a2bf1a2bd499bdf2c6ad911", + "0f9b2fb89c06a7b3f998a4e878ac7359a60ddb43ce1d9d46c5298eb2ce78ae10", + "0fd61f330351d6e64c0b9d1e6e17e73760f6e839bce6dfd46aa8958b0a00e0d7", + "101dfa78c8e779ffd065d635677a806d1220e5d870d64ac8bd95376840c9e7c0", + "101f400f3a1a701669887d3522d66c35e1dd66e64a55542c144f74dd4c7228a1", + "101f8a873e2ff5f5f2422c858ee11adcadb2eb241c8e329c5ca8d6b156feca65", + "102bd69c2a0b6d60b3e093cef71d13b91187ab631524b3fde873a75a795f82a1", + "1030bb1d5219a2c2d7b4560f0b8fed5a0b7645b178bf2a25224f3e8a1620a5e4", + "1047bd5df24c19ad178a6ba3b0e1108087b995bcf277520efa1ba27511aca823", + "1057678ed417e6ca1ca396a9143da3da3c7a44d7824c00178707a929a980993b", + "105fb9809fe0bc910f4fa8fb60b5a9bca2a517ed9781d2c99525cef6bd7a7c33", + "1076382ed2d4fc79f71805538631478e0cbb956e3884829ea3582c5ee8511b31", + "109679dd2218dfbb038bcb415aa96ae23b53c35a1a9cf156656633fd33b12051", + "109be5cf5ff9399f14ee012b0e509b867c597216a88f62b615f0fa4f79cb313d", + "10e050af1016ad6490f688299853f4c6a26b49fc52e36d8e84bb16b1d6242f63", + "1106d1fdacb747420db88c70e23d0522e849e8ec815670c0ff7c826e4a75d53f", + "111c5243a9942e1524b878a9931a8bab82b9f22dcb1a804a118b7d5745e1c557", + "1132f5177070b804b78222ed58670c88c119a10914c7cd820a71b18061fccc4c", + "113618a22be4bee42ee2bb850f8ccc18363063f6b3cdc8a30a8ca58b9b39fd3e", + "1142609193b022cf38eec3ec0870d6c51b2c09eaaf6138a8d55deb11a6c5f22d", + "11450674cb59ca3534c525d225161770cb93022c728f4835a7f52574e959525d", + "114707d425d986244f3e56284f7bd171563cf48974a42fd7a9280980c67a3f95", + "11773ee6fd7210cf7c79880cb39fa7cf7ab2829df19080e82cad70c285ae4268", + "11b4798cc26f0599eecdc28b4d71317e23c4acf33bb37093c50a39a188631945", + "11b56f2db4d76ba9adf943bc17191c2cb6e5a85d85a1208593cb7c1be386ed44", + "1209427eab050c56945e46d87c5c97c8037c55ddae6e4f0aa97a12c04b38226d", + "1233ac6385a62a731f93e267599b5c5169646ecc236c05196bb6256a4601e223", + "12a42cd1df46b7dcdbe7e2b9a63a24bf047a07cb453c4064a2f92be8e58e7a95", + "12b0cf2d082125418884cb235be8f0cf114690e28e8e68ff8de681b5a0a801a0", + "13149b1fd7056e439febeb35a6867dcbb149e9081d37fa40e83b1f2918ea5e6a", + "131b8be76b5625f24c2b90ef450e09681b8105c48bb24612d260f7abe159b85d", + "133e83462cc2534113fba38b5377d44f087fd0aa62fb17119a2d775c4d36ae13", + "135c96d100c2bbb210c05b0fd0ef4847d85bce2f7db80ff53a6db6a34388ecb8", + "13600885db220c1356adecf1f18bea3b6548495d5a2dd92685a3405e6dc32601", + "139065da5be130bbc91483d942c99736367a5b8e88fed0352cea387a7a8dd805", + "13ea6dc51f604aa5829da023023854a2495048e1f0760c9b3693d1b6d6cbc295", + "140b360113e9fb4ba876e619c0f710422a72e52423a3a5c80a7ed758f1681024", + "1443c885727a11619a79c4d8beb93b4f70e2eb0d682dfd1cf245d1e66fa67beb", + "145fb05e3a2ae5ce368a34f6fcd96b7ef31dbf82c770652fdb4a93e1309f5cad", + "146051089e993bfdffdd37eaaa68c9bede30409ab45c27b17c19c9bdfdfcc447", + "1480b5f9e56ed421dfa2682a888b5340df9e3417b2bf8c26eb628970dec5a891", + "14a571926fb18de1ad06566225635e5562408c6b1e2f78c46b959033de658e70", + "14b5b48bc2a11d15c666669958c19a316df7da7ae93585e4fc29782003f22093", + "14eb14bc234531012f5a646c2481f6f876e53d1549adc80ef13030c6ca20ba62", + "150e06f72127d24afa4e0520537a06b3c6948119d290c73c8bce75462e52b706", + "151401539a88f6fd7ef8230cac5454f2ba76c8e3413bbb677dafdf138979eadf", + "152b07d8b0784d39ddbc1d8033cb660d667cda442b82103cf2cee4bc344b62e1", + "1589857b2270441a120ba8e1402c97cb04438de24c09d023d2255f6916f012e7", + "15c85017940438224999372bf58525a0aa05fae056b9b62153189e1b19774ea7", + "15d9ef11aa04e54d29e0e84e71fe641dc54224f835c497ea74c18a6c2f710e53", + "1627d9af029612789104cebb56c6f6ca76795364d6ea2d2bb6eb4911ae636d1d", + "1652f6d3ba2e0a8818b2449312190f6e266553fe636d1f9ab834dd7984a01386", + "167ef625245b0fba78daf7d0bbbc5e6fe0aefaa9dbe1e57dac61fa631b475138", + "16946e5f2cb7e92c9b2e2be8f3e6e03e7c4a7b884986bd8db1b427600a1e6591", + "16a262c3f0d6c6f5df9156d5b413d2ac9c076056df7e3b059c0bacfd32a02c74", + "16ed36710a71acf4e83a0c735591a8d8c90197e92a38efcc8f5ecf64a1868ec0", + "16ef1975b3bdb478bc236271efdc9cffab09408902a5f97ce16f5d71136edb63", + "1705a03558836eb72c3179c8c3f78886201b56975c7b1ee7ef61a9cadeb4a57b", + "1711dc5e3e5570e0e338a8747c3c62bcafd93fb89e20f4596ccdf0da8c1a31ce", + "17218e8f2c3254089caec4db721323a772301788f1ae0e1d540fb82101af6777", + "1729956f439d4baafa124ad35906905bce52a38cee105d0384caf2bf4b1b8c16", + "1748ee905ebf6e5544ca8a20a2dc34e69272edfde8c325294b5a6e7feaed3445", + "174d8e6fd793f2b7032fbf0d980e5f4d8bd766c85f9873782481450c8baf2078", + "178a0f21298df6295ec0adc89eed35c2c7b0010375ade0b3ea639cb06e4c7c34", + "179c1cca46f3cb63e3f75c021f41dee206bc6a554eed8b17a49522eeb93db65d", + "179db1683a5d69775c9698822868220d9908f07495335ad2473ab81fd62943b6", + "17b9ff51808a80a062bbbf618fd89a0d21a707ddfeadc23243cb82a6aa2f9241", + "17bdbd2ced1b33326fd60ffe6f5af3a474d9323baf21761a637897088221a48f", + "17caa6df22d23dbeeba5faf91e1d00809f676ec09c65139d493616baa2d814c7", + "17cb801862c13d2966d77f9841a7a3e020ec4eaf3b76f14adb6aa1816047b308", + "17e6e2f1435e3d01d6654206b99090f419461dbfb37993d5a7fc128169279cfa", + "180c43a6c0de8a43420887064895b3aa42bc535816446275fee6a6a6af523c30", + "184ce76e1968d8a3369ccd694d266187a1189d85515eef1759b0c1c6f784d838", + "1859ef068a39acc92b320dadbb60f49dcec9bb2799dbc8c6f25f54151240f0ff", + "18ab98247de6b45481ea7b0934e111b3fb070f46173f5f586bb9d45f4cd3c700", + "18c879052385c74614d1a44ec0ca146ff4413a852dc5ed816189f45047c61ce5", + "18df2522938daae00fc6bf415f08a5503d5d4b600b060b5e514628b10a8363dc", + "18e88569169f27453c3a17a88edbe98bddf953e9022234f2e5a0bcba4849d111", + "18fff60f2e7239cd15e46ca71f35847df2b47d7e499474cdd1d3512894750bbd", + "191f38fba735695d000d80a4521af73f630fea434a5676d87d1b27c544bd10cc", + "193ddaf83e3bfa7913045010dc99c78a7900f686f81bb1a64c72c8402131b494", + "194d8b0abc3141bb77465d1a491f4c23166105fef6a0ec0b46dbab0fa0b27a97", + "197409c1266c07fe3c0307fa9dbc526ccee630d158390c766a97a8b3b521d16a", + "19abf2d41c0502ea57f2d87647da2fd9096befc28cb60052e5c1f9a03815c5fc", + "19b6783b167f943186601698d9096375a33996a58185b4fd8f6a1ba54222b021", + "19de583fa8e8a4aa2ba3b2362b6f4a4d5139dd4c45718c1b6a13f34cf5614017", + "19e44efb9c3709c75976aa81dae9336fabd9f3a16efd14724881cb3730964579", + "19f460b02fce9592cf785408a8e17e51bf35598a9f123262c56d9cbf144a1aa0", + "19fa18cc19f1a47e94f37519208bb6dcefc4b91981884f101182e1025c76433f", + "1a296224e9477d2b02c8f3f9abdb975a926f5b16f09a262c00a1db29c7e29127", + "1a5a2327dbf13f4d8062a3a676c7c3be2acc4469f771f49f385b6f608303f885", + "1a5ca891c3e8863c73d87d409285913f3ff31d7d816ff7d769b18424406d0b72", + "1a6fe7be0ae3c5a1eddbdea9cdb59ffec5cbe937bad1daea304c4a4c33ffdfc8", + "1a770631048156b7cb85f7f1c3604ba29f06fb0194113c0cdebb9741cb21cf0d", + "1a9e514e8ebae8faf4aa9b2561e82c4da462ac342967fd23eeb62d69c788c37f", + "1a9fad98a2bc4349a33356f65471b768c20daab285e0ec8926cc2cbd64519950", + "1adf55444a134beb42228bb9fdcea16e6ed4ea5514f059ad5f241bec3c07e58a", + "1b44657071336edcebbd0778f6a189a4c5d407483596e41b82cf2caa223f7184", + "1b4d8ab0d77e3a7b204f36bd251a684a81f307e3e5c353299c03e46527bcbe57", + "1b692f8a75a468d9d73d47e69bed36a362534ac7439e4b7ffb46a28ca6ae4300", + "1b938aacb3f82d2eba748f4d3ce039643df540aa52b8a226b1c47559dfd95300", + "1ba01ade66bbd22f55c0b6c4fa5d5ad80ebf7c31f10ff208dea06bfa5acb145f", + "1baf7fdda1317cc8e1aa739628f9f87d48b7e0fdf26e88fcad20f570602d58d6", + "1bd4b7607ffc276d7dea3a901f5c49dd25aa8450579c89f9983848e7d9b68b8b", + "1c260a74d7a7e4a86538fd40f497ddaa09cf0bdb87d3cc4a2f491d1b76cad2e5", + "1c2f381834ffddbaa77f1c0985c687d61ef5d1d87723ba0026aac8c53f6c61a0", + "1c4b6cbb1a2a0dd5e7445c1927f918866bc9ea76cea0d3c46c79421831ed4b09", + "1cb958ce0de35812bd01a62f72785aeaf1d2a0b942e3ea7be0b3893287d0bcfd", + "1ce18ff05a935eb586210f789946ff86af6512f5626fd0f6f3d90aa823c59b26", + "1cf5eb085a340488bf274e08c6abbafb32b94ff4456da6286a373409848f1373", + "1d53c386b644f1386934ba661826894ad14674b148d6199883619500cbfeac72", + "1d693019454f375a0ee1f16e9a0cc473cbf941f780b4eab01c210c6ab7d1ddc3", + "1d78802c9564ceb0481995708138e6af5daf67039ec5fffaa7b6e62c2a09b7af", + "1d8b9fdf05159ad613bdf0714f03e8b12dc33b3f78389bb315d13c336f8497e6", + "1d9d0ad3ae1bd0b3c2584d50c9f535b95f6f981c0c306a903ab3a14622fa4e83", + "1da17175ee9ae5d0b54fbb2a6afe84ab8f750228fec9acbe63af0fdc34e86a27", + "1dddb86c43db3223ceeb167535187e1dc4f77f558311137ab9598e173f42fcf4", + "1e37d61f66212b48b8f89e47c6a666b81600b191dbf1b93cd77301e19b59bc1d", + "1e60e9b00d53547071ebd296021a97f135965fc4d14108f39b2422776efadafa", + "1e898b25e792d44792dae03c892d9509da8138095c2ee7ff05f747df251ef63d", + "1e9370a197078bb8a937eac632798a8fc73f985b22a08e3e4f4418a7d252a53f", + "1e9a4d1e09c1d1eac8f0ddfc3732c0a6625aed1cfb4792cc9f0c35b226bea6d3", + "1ecff6afd06d6caa8afee5203626181994051d7a71897efddc9be4c42af91989", + "1ed9801a7d77ece07e35f4a3dcb5ae794b3fcb8d8b30472ba56281fb661950d7", + "1f1974c9c42bcb251ec0a6fd11a37ba070cba50c94b94d4d8173f17c7b3a8669", + "1f2197f0f714f26bbceeadb235929047fa69008d79e878abd900f1599e4eaaaa", + "1f5676917e8606c554ef1e53c14771da295818cbcc218eb002cdd7f1ed92cac8", + "1f93422b066c8f014f661708d7b347338edd9f76fe8a7e7b2318c15128416d7d", + "2006d578e24eebea02053b839228dc47b6caeacf097a56d9d747b1819df472ae", + "20403264c1f7c368497fb1b777303827fa9983e06820d7281639af0bf3f4e96e", + "20646ba67096da4fb1ab927a11101408c675c2eb98f47805aaa769a8b3db67f9", + "20bb27f8873c4d93c946b6891f3e513ec84ca2b6f09a73049952615b0248e599", + "210a7a5467b03c8bc0c2cf65ac6d5db5d76a98b285bc0a1e9c9c117b1768021d", + "212234d9d3b54e23a153e6222de1af78eabd8b93df9f6436078ee4b2a3223f68", + "2126cfceae5b8e513d078433eef8fe23109167b12bbff8d1c360ccee15bd0e61", + "2149243c6ba95fedbe578ddf8ac61ff2325b3ebc36705d3e278142f3806a062f", + "2151099bc42497224ddfdf04bd012b865f8565cf17b37ec2ccd7a4a73403b1fc", + "2163bf387c2bf5b804e987bf19844e67db7a90704c25bcad773aef04d4d859a5", + "21c4f19658aba33d1fb5c99d6e0d6c0c139f5218e44d78a77ffa5284896141b9", + "2207b41b5d1d5d2916d616e2af7d8aefecafb390937d39fa6e2a540a3bd10e4d", + "220831fdb2b9d540590561a6fa4d2b97ce6d6482f2ce58f567f9089ea8387831", + "22280f10fa1c1437d2664ada33ffba110c92a720bd7d5d657803207dd18535c7", + "22a36a44c5a57339ad24d7023c86fa5588a6300e8637b51351dd7ac95da29064", + "22f355bcedcb413a3a1dc60fe601ecca2584fcb94b8f2b56491c6361d23791f2", + "22fe153e2c7dc2a356f3551ddab83540615177d8066b248bf72168113be476cc", + "2314f1f7d20b72aa7d1d275a77ff5e154b3ca6e5beecce0cff9cba24fb95f873", + "23523c218d0de27396382b1ed9debbe4d895135feff96a92af88cafe23eee549", + "235c8c8b46ccec4d668a3c9c9acf7e1691674e6d76bd31cdc5b8919d9c62d4bb", + "23750121de6757ae56f097ccf73f62973ab859693341fc53986837e6cdef1a50", + "2376eb1a205defae31db9d21736c3a9981730d3e8de1c237e084b331f56f0226", + "23c8cc053a3e3937b369c9a82b97dbab7c1b3418079d83bb808c8ee6a8cb977f", + "24037ecc9e35cbc55aaec3c4beac71aa1b73ed68c84ba0882c32d53a156835d3", + "24063a84bea566065e3eb027a021d7321f6073ab346602abf8bbd590a787e463", + "24418099cd07ceb506972b9e4e1787eea959ca7655ea273a189bdd687bb39f54", + "2442a7acfa983a1ea38aa0f9d72e26e644620ea256c96537a9ea9146d9453022", + "245709bde24a4091af4962982e892c9e60ecbeebb4467db6ce16bdaadcfb3d9d", + "2459c1f49eef7a04b4e30c35ab426b182f335c3502bff32a32b15b42f73ad032", + "24894d48044a516cc510afc78da561f4baffc729562753188a040e7378af3d26", + "248a9d49901fe119fb30582e47717afe7ae45690000a45fcc8ee35518753d89f", + "249ecfe12202bdf2985659df13fd59ccb8802ae2dc6188499f1386d7362c0cab", + "24a0451be2da0b8e65249852523eedd2ec51ed37745039663151334f2023c326", + "24c84ffd078a3121a4145a43c56e4f1a2bf0af2f3cfdb189f98c838731066f04", + "25063678270b59e0dca5a6da6eba3292f23b73d217c9cac51605b15560e77120", + "2518b9c1010b8d9b2d7d1cd22c1483a017a24ce6480e7d11b44e414ef44b0765", + "25348dd365bc036dac3ca8c5d136c534990cb2faa13ed65b9beaab47203d8084", + "254464dd9570b6db1fd50422d4c43c65e00e79e73b7f67805753d87663fa78c1", + "254ab0473645a7d33ea1c7177f62059d16b3aed7d778d712fa260af6b41fe5ed", + "2562e4795a7465ef02f84e915ca1b58067581b7131b29252f9dfdea193c17fbc", + "25b468e54479d43367920591e3ed6aeebd9abf6f10e1eea7317a9385008d1170", + "25c0469c9130572e4564ba51afc94384fe426a6bb26817c98e57d5bb2226ffda", + "261a4e8be9e066d28fe884861cda7b2f9157f907f5fbc1e367352c42f3878753", + "26212fe5d8453ae051b0ac57fe878c9227f645900a62cf21bcbfdf6a0fc34c7b", + "2656e30609ebc0b05b195b9c0e09ddca9d3dcf58b9c2407eb1baa979794c909e", + "266db8f2f7eb2cb3582b4069583e6e96c68f8b4c2111cac4214cd4572c1bf692", + "2679142ca668cd2ed57827268096b51b059693540395cb5346e1c31aaa903641", + "26b4a45547ce31c9abe7b9e18b8dde2b06083198e899247c49f3da64f310c18e", + "26b9354f82f2e5c5d5264dd5e3d91b597a90830ed119946f4e086ecbb69e4c1c", + "26d40782ae484f44ff1d3a6c9a760c457b48f9cc56194998c017ca391b7d597a", + "26db33a4e5b448543a2a7852a03a797297f34982049f1bd0dc7ebf3807f9f981", + "26e92dd8552c48b677811eca3aa2246840723432f17775fafe537fdd1fb61302", + "26f397e39bd00131ad71a7aad609c68e156ba48c68b264166234d946ab4419d2", + "27066882d04770d7e9e64e700a19a6f9370b9a34ad3802ca1f9bdf19c37770bf", + "270b00f071e7989acc5fce6eb464501e6b57d9c336002964e9eb65617383d72f", + "273a74693860dfe7fcfb511a287decae91dd4c8a72b71af522017505694a6794", + "27407481c77c04104695dc23de6a69977aef5457e2ad988c1644523701f13d7e", + "2740a0e42da39c2fa8c568cbe32098b1c3d225756665665556f3f7f6a43a0495", + "27858d3f5cb16b93823d8914d2f8c3372e7c686c9e54b1ffd81c5b934ad1e5d8", + "27a7460340351ea1d4644e1db88e17f3eb72666eb67d741f28010308ab3a7a41", + "27b0bb9643a61edfdc28887735cbc0983b19ca2d152c614359ca281fa5d5d57a", + "27b242c5b4cf58073dc81620e1d34d802768844a3a6b1f6175330bcd5dfd74fd", + "27bcda081c48a50079e57660ce90ed1035693e751717c2460abc3724fce99a8a", + "27e7476fd5be3d2d690ae3d918652b34fb880dcbdf5868fdc723b92538497555", + "280189bb83c0b59ea0a19cc763241246d9c875cc395a2a44b8981b58913503ab", + "28124203f106909f1df4f1eb9f24106733b41600a23bc9ee10f3500647c78219", + "2828d558dee61df8b392e18e255063d3cc328b0bbf2bee4245c4378cf3452423", + "2855125525b095570c5c67a3e29341c7acc8d003430f0eaa7c9ffd3b62a4b82d", + "285ebf730d8e2e3c9de075c89cd76833a4604d7bf38e55e1f232514825053166", + "286613917a09de3d887c18c08de2c8318e746292f5960f14629b6837fcea83e4", + "287ab60d93d0337301bcbfe0c86095fc2efa96ec49f298a840203dfa6a6811cb", + "287eeaf0dc4a3c7d7ea6d5a3dc340ad31e89a936ea03e1418e7d334500e9bac8", + "2885cd50b962d924a12f2dce415a974cef12dd736a2d3d6808497099ad4242fb", + "288a9d61d75d3a9ac82165408ab2f83d773bc6976d0a596343e0115f37baef8f", + "288cdd186e53aa4e0eeda60339143adc8cf1b20a7568a5275edaeb2ac5d44d30", + "2892fb34d6fdf1048f8348e29c1e2c3eaff561a648a9016ecac7390198e0a8df", + "28a2333a43a7143735131c539910d2b9a76961757224329d64832322fe34de18", + "28b23e9ea4c919838a761bb97144a6472f73bd6f7713209efcee7f30f2735306", + "28b9b8824154fa65a6140ea02b8b19d23f258f3981b58eeb1157aa0157d12a55", + "28efb908bbddbda776159898adba8ac6c589dc6b3d37829b590c807ed2eac206", + "292c695b9d20625d27207381f8a3e70898aee3771622f38b71e090a86afc0a58", + "294a4a2c1a832c0388fe80f5beecfaf4ca644cd360c07309573cc1577bcd972e", + "294e7b801a7eb160b3a4e5a424080598ba1f1b3f73400f5138a60c3ef0bccffc", + "29744e52041dfdb69de822316a881dc2e91ac164a8483c53c3f8cea36de54686", + "29b0d55d68bd9dfa81634efce57079b22ef6fc8199b081b7c1381731de5c48a6", + "29c4bc894cc82a5ab9c0b81405b35d21f9f9cff75b7f5f33413a1fe01a7a7943", + "2a2c87339df141c33a723fbfdd435a731a0d6852dd65ca14e95e6abf6d92e75f", + "2a4492b8f60864d826a76eed372bf588abf7407050f4fcdbb4b3ccfb73559183", + "2a5f58badc336d55b3dc5815488b62b0667ec6fb54d6bae91243f46c9062b24e", + "2a943b3c5c254092702cda0c505dc040cbabdc7e04bd7658b35481f0c5f699bc", + "2aaaf70ea612431d001d3466f0b41627abfe991e7ecf768ebe8734785812e243", + "2aceba1f4c30f8131dc9ffac25a01aebf23b15c34b19f5fa8926aa4e63a7cd8f", + "2ade1798df72258a18c346a6051b1ee1779e06d9dad7c27137403468fc630bac", + "2aea7980b8e26c8d434644d2b1a4c65ccffc39ba85b00eb8de41675c42756667", + "2b0881118dff8870ad09e53fcd20f0e4663e45c8ca5f38df1c4bf17361a2a13e", + "2b08e0a8c9d17fd5c672d90351625b6dd1ab26e2a77f68ae04ef065c3a3b8758", + "2b360994ca527b6f16c2aecba4370306e7afbd6c7d20995e38f96bba71da6be0", + "2b38841187c474862deb3f9e0eb5065ada3046739c7ca2b8f62f0c85a62fc8da", + "2b42c4ee1c3d0d4d83dab44e08dd6ede637b0ddd7c16b8190b5b08988715b45e", + "2b4f44a83e139d6234b2609433bf8436c4460bcfe236addcfbe63595c8ec994f", + "2b621cec625ced5ee26b5a268b34a664e21be930eaacf81e052434a037cdeac3", + "2b7cc30d5340a2b66dcf09fdd5c6e5ca85bd388bfbd3345630990eed07578f80", + "2b9cf96df99a58f4ebce65d21e097d819c00d44ad2fd27a98d3308aa88e6b3d1", + "2ba4807fa727358531b1957300c9abdfb1ba797fabb7a7dec22a4b2dbb8c8171", + "2bc9be0fd0d56c399fd79f919ba90ce52faaab5bc69a62519ebd214e17489472", + "2bcc96d40c2a72958ad61b4a1aa2123c8d356bee6aa1ab4a661135a5bc90569b", + "2c1654b1ef1308a13bfe2ff712e531a32297032794072730a00ece77f458fe61", + "2c27838dad793f315a486df8361278039fbb7199008c026422040c445f2b9f86", + "2c387f23eb35c3d4a52f59dc80cd87a9fbd1ddf6314944a60e7fba59d0765ed3", + "2c43cc1bbee25412c64d941f66df407bfbf6b4fa6dec9128336bb1f301c6d7a2", + "2c4432fe90d8817433832a4800d0bf40c932a4692404b1fa73eab43bfade88a7", + "2c70cf437ff6f42edeb8f76651a33fbf7230d6f0b672cdd08bca19916c1e30e5", + "2c7e589a02095c96e26ff938206dd582151dabadacfc5a8721ef8cfa86527b94", + "2c956b88706537a241524c1abc8ab57a320c8711f8503cb8aaa4bb17f3200cba", + "2c9add30016eb8b582370afb233e756a2c0e71441505cd96142f27e36e88c1d9", + "2ca3401452d5d5866ec65739ab2665c8b5f0be1c614dd7059de089fec3a711e6", + "2cb99fea0cad751b8a4a1b941ed87172d52d55fb43fb83638e31b9efbf4233fb", + "2cc0152d812d1e4a0d511ff3c3a9916e776aa614a366f1f73d3106e6897bc52e", + "2d1bb608b747735c0b5a69c4e51fd79ed6b5cb826d2a61cfa642cd13f8989b41", + "2d2dadaec7a8758c8dc8309497a471fa2e019e60f9d6cd3aa19b4ec0a51d2b99", + "2d4cd5e46d3aacbf9b82a6edbe2411a74c4a93212246c1bb2372372e4bf3274d", + "2d683c2c3b49680b6de9b50e2276b7bdcc137261f37cabba58a7748d299a5494", + "2d7a92c2c3fa714388b8ac5806366ceb8121ad8f5d2f8b9984b2fd1ec6b06e37", + "2d89f2626ffa73c6be327d17d2a6db0779eae4beca522ce7f24665f70ccad02d", + "2dbf667179319df46d094d753e44d5dfaad646777e08b29b4c51d58c6447496a", + "2dd0949bc9ed9322b494caa15ea82863a6a116f20e0e6651c1d5e6b6eb43f52c", + "2de26fa321cc0f2e464c49e500b33d39797c3b6be17a60668063ba26dfa76e42", + "2ded610c1de810d2c462f51208ee2d8f471d9a3995a6bc6932c5fac629560bdf", + "2e119b1f0a1c7fb0f1b2c3f5ec1db7e89f7657c009a48b309538b396e36b2ad4", + "2e28d2c5a315902609c458005876a85506dafbdf01751fdcaab44a68d0a3bfa2", + "2e6dd85f441bd65e0fc4220329cafb0408430e5d8de2ab07a5990095ff43e78e", + "2ea77ca16c79e921ab5851baac1f0fb33529d7c76a7d2526fe3e403761fde1bf", + "2ecfa522280f5eac8bdf12c4ddf4c4a33c5b60f36f306a8e2561bd7c05d3f148", + "2eecb84249abc69da89a65bc6cafef171db054f0399e30b7cd2778512941fae8", + "2f53841fd202a63ce5944e746c15fa26e17302e631b041f1b64022545e851d77", + "2ff450770fb2fcaa0867b408f1a5e66c2b65770311e4c4ca0eaf4e68c6e04a3f", + "30511fbb595277baec4c896231d4e9295ae744a425e66949fc1c9f745e951a7f", + "30653eee6ae7bbc39c131176c37c2b9fd5ec2c50008eb84f5ae21627c1fc12c5", + "306a71235602fe4889afe2a1efc2fec8907396abc8edb9b814c710abda504325", + "30aae25405ca1260a5434470f4cf866a9039e557139dc0260f75f35dcba741bf", + "30d6674f194cb2b76d9e31310c49eed068a97c0f27598c46d930a2f6fa59ea10", + "30e628d6fa4953917a5c581772f8ddee9fbc06c63a4d2956bed54b6ea7323c73", + "310e11e670166746bec5b4b32994cc60083db42de75f51d351e07fcdc7d3de46", + "31121904f1bcb9dca43ffd7e0ba17e8b4553fefb8385245b890a54e572f8ed32", + "312ce46a276e976e2336873ecf88d8e6ce8da47e9d1c280fe7934b1cbce52834", + "3133263be7c1529f939a42c98f9f5cf54339ab8cc8ad88ecf24b653c2ab2e526", + "31619efc33f3642822abcc8df062b55ed10485df2a6b91649e86018ab2f2c9f8", + "316d5b71d81c1d3b43e88a63aa20bfca9a2ae50618a4dbd9c3338f5eacbbbe25", + "31c8c284b98bdd6e6b6e76e48688b98041b67fde8ed080bb821b723063f24866", + "321231bad3b78e54ae2e36b015490b90756bb489d54c3f4a474665fa9db95b23", + "32136646f2c328cdea70a654cc9aabf770a36f339acbeae5f8948e40c0e2d3da", + "3229ed80b09d389cd3ab365025abc3e5a0541efccf5213b41200a8dc12d24ffb", + "3253500b87ad1014ba69e19395cd95459110bc8b1dc37e4a906778e19d71d04d", + "3257111be2381ffac0f0e668d5a3ba794eb8c4c2dd03e57bf13bc6d4471b88b7", + "326c201a620cc66c4a4d90735a7009ad601df58ca76002c49a0cb39831f37d71", + "32a616312a1e0a3a8b2b427320a861a432451b405d7ba02a7158d76b055f39a9", + "32b93c82ffe40795f963cc8a2f0b18d9ceae277f114b027f79e4d36fe9b085ab", + "32f8abe41933ba53748abf536e0e0ccf4cc4e79bd65f4c33f1df78f52b45d970", + "32ff9812853c478ec4ba6d031cd30a38f47f295d2103f6bdd1e28450958fc14a", + "3338ae2f7ba490655d53c884258f45d848437fd173c216a38837ca2cc09f8e5a", + "333988cd2d484d4f92a17e109724085c52ad8c70ce1759c53553210e50d0f0c9", + "3355c5148bb66e9b9638e671d428c271469a8ca2de794275c94af852e8209424", + "337b9dab25d59bbfeac2ddf9ff3b194f0ed6c876b99b071cb42fb2574a58a0d3", + "33b73f56f58baa737f8591faf7618590c2ecd86bcba29259e69585da9dbc521a", + "33bcda29c685106c1c0f44fdb5946a975f15b59275e8ae3d7c0ad7f8d404987c", + "342b09937c0ecbdf5b01afe4d178961d86950c31daa9d90fbb5f9bfebb5d8a3b", + "34373b6e9907806a7985134899c6979611d5c9e7e176d41b97e5880ecbed2fa4", + "344a203506196c9215b5e9f57aacda02e90cadd3a9430befb19349d13afd9178", + "3472ab26eca549d514b26dde208c03dcfe5a7a764be1c5514b9bf3aecc6a66f1", + "3480bb22f27a0d18b2df530a5dce8893367e0f3855fe65da9a1be6638f97e27c", + "34937198be44c29e1ed3159e7ec17dacb6df05fc6cc9c0b1663beef0e95462f1", + "349c29d22cba205fe77921efacec0ac6ef8b71ea131f273e91dcc5d042771f75", + "34b6365aa65685adfb11f115ccaea4ad1bebf026fd9e7b0318313a7d75a27c18", + "34c35b63588dfdcb6be0ce3eb3398ccb276afe2f2ea8a6ac3a30921c858d9b53", + "34d179a8f3ec05d380cf2667f5bc24b504fe9a125ec62f5759bf76225d5e50c3", + "34e69d9035c99f06a5aa13492a356a90693976f53d37de5868016b5519877481", + "34f74afd2a016440c205ac27a027ca4ca31ae30eb360a5241462ff6cfde32dd0", + "3540be5c5eb3cd046b586a603ddd6b954bfae55c076a97a22f77e4dd40ac9493", + "354ff8174aa039d4a22a628025b8a9a6ec415f7a6a6cf5874a722c1f36b7da40", + "360027a341c08574c0477f28fdfc05e786916e797b7559af6348a68273739d25", + "362be82bc53aeba93a56f0c4cd5f2a371fcac8f45d5e7240b2866865e8a03804", + "3690398863a71cbf8c4fca9137a29a0e6e15b71470a2f0fa1cdf06a5d0986bd0", + "371b1838c0b76c4959eb92dbb7a645db0e915ffa25c3906b2959d5cc627c3fde", + "37337b4f60f4e996080e8d99ab02621bc21eab68b26607ed1e8ff6a7797b0ea2", + "373f72cd6fdabdb9446a1a73ffcec5022c36c185aa4036ad5429b02f7f6ed666", + "37522879f3d15105cda2d3ab6b48fed2900198e74e844df6fe086b385d0a9122", + "3790c32a444c5c6f3edb16980563d016559ab40aa9a48700e292b413223ea56e", + "379eec9aab24cee73bdca8e950ba019a1f57af3f88bf83f14540496a223e8420", + "37ecce1d750a93d7ff5e6ce79171495cc95a596a892a565921fba2898fdd868b", + "3827b9b917f9e9c1e0c92f935c31ae04e70a8ecc430e1a17df84688c499264cc", + "382ed00337d1740c1abcba4e6b069e5c19fae28ca9f551087ad3d8223b10e821", + "384398950404d022e3d937bfbeea616ed984f4800284a2548af95bd875e718e0", + "3859548b787c8da0ffb479bccf43817cec3aa1e6ae9c065952140c89add2b017", + "385bde3762f16b48d40ffca0f7e13cb61a3685cac72681c1ea6d46b8565fcbe7", + "386620005ea0a9aeca9e2115e73f6a315f651d1e6a33c570438104467964ff23", + "387f75f652efc68830d25b8fa3e05b3e67b26413579c7ca31092a97da03ebe90", + "389fe21426476539bc015e368b4b1db1d44dd895995e1aa3a9f082fdac2b995b", + "38ae7ecb7169e273a8ff2ace53eec67dd77941dfab4ca3610f84cf489ac06c1e", + "38f13a2e254e38adef39a763b20eb8dd3ff8fdc529bae0429891336ee448c286", + "39000ae05180499f9212cda76d3251fc437ba01b55d5cc9387822f0673bfd14b", + "390f26525ac002e5a9812985150e0b5a53bd4bc13d39d33dff76c58c5943ed1c", + "3922d60da591994cba312d2f2a8fc8d8117c09b20b198274129a070f113cd5c6", + "392d57bad566f771464837d462a4f940f6bb8b307ab19c5e6cd0b93b94c90d10", + "393395504d82649ea066d408f61d423ade8334b7d7bc95f2949b1a91a3c21898", + "393400bd5410c8a61b80eb939fd77272f5baad15e273c7401838185b5fb8d548", + "3937272e73fb81158ba96b14584d01423a8d789a1c439c3ac19085358e4d605f", + "3940d2f105eaadd7273423da9474f33a55787dc9d4e3c4c334ec2fe28f4a568c", + "39516f23294ca9fbc6114cb20c160c6b7f0c205d63889193cb85f834e1bb329f", + "39570eea5b1d6207bbfc9b1792bf673b1c6aaf0e8934cf232a771fd4eb0b42d5", + "395a62303d9fbcbf1a19506d29fd1a95337a3e998115b8f45602dd04f19f27d1", + "39764a681f8e76c170113a5ffb08d4678cefde18fbe999acd5134bc1333c4a86", + "39b3d97686cdbb4c091fdd041c7ce589bab27b9a794f798052cb0cd13fe8873e", + "39bc2f74fa976e69d6dd2a8c599e695dd3ce6bd2a28fb764ace7acc236da158f", + "3a0a35b88e4e046bb4809979d6c0638f0e673a23342bd0f845602f9d68d83446", + "3a8799a5aeb66e6b62a8f9d1971a49d558032ef93454ef38af18d6ce59d58c64", + "3a9d4a48fd9eff24b30053f0563e2cd0a8fb066a1f20f131840a902aa8d29440", + "3abcf8119ef5d704352ac5c3393f8905d420e5cb6e269df9355c046d33513644", + "3ad8e387b6b41cbd255f061e746d929211b17dfa1dc4c2bb3f988a731a6a1467", + "3adc8c039c613b98ff8c20e0670f7a758ed2e9e39feb135d9f5365a08ad47615", + "3aef058e6d5fda406e7e9d808a96e70f18ba8e13162edb2908661f7cf00ce7f9", + "3b072f9113509c9d2cbbf1c8844cdef49e2a497e972c6f6671e89870fdcf7012", + "3b2fd03b31505badf3cc0c0ae855f48bd2783c9893c35e9ccc88f55110271fae", + "3b76d52cda8f218ad86f297985c5938c24a4b6b632854f3011494b7cb744268a", + "3b9e0b44fe5d97d8001018fd4570ff07c06d80ba961d940a98c4472ed3ae83f8", + "3ba52aff6dc34cf2624a72e08e61cd95f3de7295b25b0cd669b267c9bab081b9", + "3ba52cf7bc2fb85c021648f84774c9837dc9612bd7a619c517c1c66e4ef44875", + "3bb56255d36bc07beb145539c98a47d0d3aa1ba617524b5948e3b2c22ad1645c", + "3bb92d37f525b8e3c1af2c617cc6ce9c7780bb4041eadded976f54ceed6cd655", + "3bc4dbf30cb70d551f8355bcca87cecc0d215b5744e0b50c9b0c783ced0a21da", + "3bc883d49816b88701ebaaebcba750fba11e2680392e23ee31f486c0c5862b68", + "3be2384dee2306d6a9284b55449b778ce79b220658a9b1bd21dd9118ab14ee21", + "3bf059c7df21bfa5f8b729db597f2ccfa845b0b3b1e732fc1e10f4425bcd3782", + "3c780b30a7f1aa34eef3a076aeb0d0c9e0b6233d652829a80d6d09464475a2ac", + "3c7aeffc56ec06c0d657ea706dbf66038fb53b3ab45ce57abd52c188a56adc31", + "3c9c25ac5614ff2f35703f3c8b16d214c19cfaa98be57f16c62419a05248f1ec", + "3cae26f451b61e3273296c01d01d60a12c16719d2ba21ec35e56f42cb37196b9", + "3cb0441e03264e1f033bcbc9a5bbb0d4410cde112281ac6549a918354325e2fb", + "3cfd4dace103cd378e4de22dadcac1e856aa3297ba31185bf3324ad0de33c9ea", + "3d178d2959ab3643134b553b2bf9240b0faadd3169023160903ed01a928021fd", + "3d2baf4da5720c05a85d5458cab3caf75b88dbc812e96696877dfea74760ecee", + "3d5b80db97a27e956b860cef66224a4e13774242ec0cbd808cf605e22dff6ed4", + "3d6e35262a8b1c59a282867c635be36e6d3a19a8d926f243f8811ec445b36782", + "3dbd487c90e62aeb025e3e17bf1dada719a29d9d02eb890b08f8d335ce9bfa82", + "3df597c5e716d2d5c9df79f49d6d278fa9cb40e3c3dc9b62d0d570a8f6e844ea", + "3e8457255e410c29054eb03ecbe8b264f1b3c5f74cb676fae8febbe706f1ac01", + "3e9459ca0a452caa4ffe6c08f088f780b56014a8fceb4475633d52b79ec71102", + "3e9cc0fb1dd20c269828064dca21f3594ddb7a2b2de3befbe6a5aa7d122210f1", + "3ea36a3f2725b618cef04969dd397b1631dda55ae703a926f78e6ce73bacd4ce", + "3ec85c9d41a20e1fbd27a3d3a341d28dee5281763720a6e6198b3bee1f336d37", + "3ee3d21edd0432f918f2b2f2da8d028916a311c6d1db676955e5b985ead589b4", + "3ee6a26718544fff1c7ecf9b145d04616e2405948378f9ec342251f56d9c233e", + "3f006610eb2c7a2e531da7e7e997d21202e3510dc0ac56a508ea567c3b7f863f", + "3f39376f90046200bff93b447e0e6a0dceac5d3c783811289e0b98cac2c429ee", + "3f836c4a17f0b9d0af547785829a9330ea4702b83d86098ff7dd612c3fc4f642", + "401436e97b2f59480c5c391ab5bcaac864876386e2bf617497cf181fe45690fa", + "4021fc9ac7e29c383c2a74fcecbb0bbffc07306dd5ad62445cdc087135e45ad2", + "405ce23ce6f135491c13e994d6051c77b74bd5133bf7b92fdd05083a78a306d2", + "40a61127f13657828dfbb6be83c8d0ddc2ab8b6e6ea8b0d23ecceaaed43f9250", + "40bc5caf332cfa6dff82a65aba8c88b49d8d9258240b7dce0a0a208ea44e2397", + "40fd462b084290e729d6e367d8e05017492546885d3dadadacc0e1c5d045131a", + "412893375eef842497f0b60c9a438f557df0632085d4bc926c66b080c40673e4", + "41b4b99877e41a0d93cb88c9bc81db9ef7d14c83c6b9357b01090b103131406b", + "41cf7462c3ae41845da13e06bc518236acd8fd18f38cd7e1b279c38678392caf", + "420435e1d25b95bf6ab1c9a55c923e409d5da847168091b3760226e243ba2c14", + "420eeea9117f59559bd541412bcb29cf8554b2ebb1d4b8638371e8d73b15ff2b", + "42218ce98ad3d8d50bc9cb306062f0baa439662373548602415acc7e59d6e9fd", + "42558ca06ed02105c1166604d705f4f767a1e36c7a78618437fc36f0fc7ba0ac", + "4255e9bfe477e94d2520a86c29f52e6041a4b05683c78288689f0d3254db0d02", + "42649b4f1d4cfb99397aaa33778874f4012567a6f2dd991393b094ae0596857d", + "4265b41d4e437989877b223b245e1063734921b5400fd04ba45ac53c5ffe90c6", + "4267a36f64ce7838ad9e10bfdba933172a098d263bb63c86ad57267027942326", + "426b2a61d885d5483224791a3c349ddbaee80a35e9c40f80bfa17fa1f7ddf578", + "42777dcca97c89d7e5dab61e5d9d50f5296e32579e2cb8063baf1f68b8921d90", + "428f274d9a39516b33a8ce19bbf358eea69640c134c81028375e2d49aaa891a9", + "42922521409bdb49b86404d39136e93ea75eb0580955c13fe8cd2cba85756686", + "42d0312b0a32fbd94ca1b9eaa4b8e3129fa96947832f97a6a44d15675e697585", + "42f5ca82146746b778bfe5ec97acedd150aea7e3294c9898623fb86a53b70300", + "430d0e82480ebb89b687f5648879a0c26589aa9c9f887d3327b279c330b2b591", + "43295a9e46b5e2ec8e671c5a130fdd7f942b7efb434d66f7b7835620e2a859a5", + "4349b4d1462a48bca23750e495a84a3b950f705dfcb3d11c0cffe385eaa19ee3", + "436a928e4fe1f3e129a24d447dd62821a8b2a5c9055189f0dea60828b7f361a3", + "438e8de1b84bb27d2c051c461ba76ebc5de760814e0988577684197b0005b3f5", + "43bb7b981ac7954e20a941c170ae59219c91c77a2966d776942db5e4028f24ee", + "43cc161ca50b1b7d86a7e025187296c87c6d289f8e9e7bb86f8b6d1edaf7f026", + "441586d4cbf2db66b066c6a372442092654c3d87158fd9e46e6091fb5feb634c", + "4424a4bc86c0f46c8093e3bed4442f576579feacfc43f2366f36e4fe6edec869", + "44650da1115b47ef24e431eaaebc63ce3482b9888382be4309e9e08ae417f4e6", + "447de9ca3a4c845fd9fff0d19c05e240558368ea14d8a481281c42c1b3f12984", + "4498af1a55b9b415d066ded464919d2a55edb6969b8cc00092adbc52edba84e4", + "44cb0c73cd673eb2469fbeb0f283e7eeea30e78fd242c7fee547887a1acc4f92", + "44cc9e16828fa11b5d0a4541d3a29383c8f97c4eaad6f8b8cd7ad9b804f77951", + "44d383d0c482930c758e9abf75ef1b4145b7b91a7f3ba4a8e41563c30cc60009", + "44d43be322fe6f463b76c7e48e789ff01e07d3712606fd6155204d9e96fa7f53", + "44e9bcea8279515cc4de38c7c1b47cf7db5b1d5fe7b3373f99f31a334341f2b3", + "45092edc1b76ae550294388a7008a4673b601988e5d4dd47e0d5e42fb3e1ef01", + "45607e5b779450299b4a4e3d2bec1d1bfaa5de408b81221fbf109d782b8dcc06", + "4591e8241665cbe9eb8e1e475f97037849633d40bc1e3dbec237d87d292913fa", + "45c3b0d13cc47a16729ce2bdc9c4463a2b73f299ec743fb2c3cd93dfdab96286", + "45d6fe5461af10acda5c405e9d1a098a709cdbd59c89c6da0b41ab7b9565c360", + "465d1b63766e37d26df675c644396e902d587b2ed23cb7654dcb05fa05543133", + "46844362e00bd617fccf5dec58b106fc8b40c69054f5ea96fd3f08a6f7106d10", + "468917e21826e30fbf98f2b87d396d0472fe5b98c216eb12819fc588e2288f32", + "46affc777d8595127e09bdbabb975693f690868625ca609955ac8bf61669c912", + "46c916e7635bbf037b90447dde28310890f8b755c054927867966849863981c8", + "46d8a3840800d6114f9e23ff4c6920318183c011fae7580a86042feb881ebfb9", + "46dba5eba3ae9cab894e7240ac8ea3ee6475750d6c22bc1ab75749ad11c90516", + "477dea60e11636b609f77bf206708ea7dbf025d7729d90f692a3e732b08b21fc", + "478481dae1cc617790b53d41d5783e8000b51a433ce3bd0979e2fd5512f0324e", + "47966500228dc154f1e5184b4eb5c1d7079d2cf0ab6d6873751954f85ac2fdf7", + "47d6af26ce42dec6de81734d68bf4d2e12153415a0048b9645858e67a46e57d0", + "47de3552f64938c412f636cf0c628e63813bc8464b3d15d6b39b25bc509f718b", + "47e7e6462074a1a7dbe81ecfe3c03772620597901fc0063e09b85f360053f888", + "47edf18219d7e7f42ce9b3fe479cfcadf04f52ae02a7cd02586b965f5a9163d3", + "47fadbe243f65705a8d6f991f805a499136f721c1165f8e6f93f186f0a0007c7", + "48130f01e7241c2a0cc995b602940e1a52ee6d6e4bbd1f64580f9ed0817685e6", + "482d446437c7d574e8059cb27f6f258acf28bb2570996ad698ace431b7bb501f", + "4844fa3bfab190bfb8f62b622ca7923aa2473fcb339f49964d2043046948edf5", + "48873a484e5925c760df63afa2c3c1a66198254db1ca5717cdd081b5c79c0516", + "488c31db771d6fb4a209ccc928baea074f090e7fb56ff863b2e569d9ed1bc372", + "48a37c04e31111c927f2a066db22f491afe23df1eb94d9d9288c999f471aa982", + "48e31de5c3115f218c5eae6d25a2d4cd097cc97480aa61a93b876eeae03a1871", + "491364a39d23fd232b902a1bad2e3b9aa7884909a16969eaecaca24c4d125257", + "4938fbb14d7f0a430b584a9a5d28a53667e267d6669b4c6419dd6ec9bb6b77f4", + "4983a313916042ed6a0c6c2bd9d48cb8a2904585e99db84a6a7d9bbbb105f99f", + "499aa49d7834b33ab4bd44e7f51e7fca36bb850bfc76b3b5b8542db1df09e628", + "4a2726cdf7b53daa50ce0d8f82988880cb1ab67aac8e3eb096e386b06ee377eb", + "4a2f05baddba7f86c20f9bd95ec3a75ba9e604c10ddcb3c0b46f0926feba4bc5", + "4a459a84cefab2ae16c9dd41d5a5e38abbdddf9e88763fbea8a358e8f0a45450", + "4a4a6d458d69c78b42eecc5d11a1565e210edac4cc7754485a4f7dc2ec353a10", + "4a6ef7f11a9b59fff7083b81620a26bbf8bd849ccea64eac497b37a58ae69826", + "4a7ffb8d73d336b41de75ae007d05cb60d78fa29f979886dc1f4758cf66a14ff", + "4a871532448ee3c6d421c8a462db8e257039a1dd1547315b004671a4a3b79eec", + "4a8761e3fbb633cf41fee33db760dcf10f1c1c281fe188bdce901883371631ef", + "4ac00b172d90a8f879b320d58f5ce6f177b4f0f1f2fdc4896d8513121e1bc444", + "4ac2003f5ff908998db16efd8639d53cb3d5d26d28fa7221ed0f8163f6c953d9", + "4b01d0ffe4419a55d185fc6a50f1b80d629d87d5d44caa68a4f002cad0331eb7", + "4b10518d510ef77e3c7337bf0761660350d2350b22cea604bf6da5cfc569ecf7", + "4b20bff40caf59eaf8e5b8468dbb031f2239f249f8eeb09eea3a2c5aab036f8f", + "4b3210a8d9fa69888af9d8c56c0def3bb79e5e05f0a8be419e0bf58797864eda", + "4b5b4b5c0af753edf62720b09c8461fbf6917106b875aa95e5429c8b66e08c76", + "4b6a1d6c205a38fe3b7c533c282dd7b29ac4b5ff6864409396020a569999b374", + "4b7f80cc29961955ea40c7d3567b35a02164cd1533d958d883be82c4ba37b31d", + "4b8a1def4d36441651de8f179889ad3bd052bb0becb9605941116db4018c95f8", + "4bbf36b7c2c14caba6070bbae9458af2c813b599c8f6f3c6e2cf9705b21a99fd", + "4bc646e0147b8c2f1d49768a352f413b830a2332df659f02e944e942d47a4786", + "4bd9d910a8041060f92d96bf5eb568ce0140fe65e0015b82973300cf0e5b0688", + "4be8d922a5de73ec325c9248fc1416187e5c829799430a4dd333e4a7553150ea", + "4c1afc535fe4498f8e6ac334816ed0b9bbbabf835f590d477284915804ec6b02", + "4c54b6a006e39b19e32a114c684e4687e5bdb997e3c1d32299df1a6d821e51e4", + "4c6311c9c35a3420cdf7020ede4be8fd5f7b42235f8f7691677857ecd3987a48", + "4cfd7bdd44bea3204d1c2a9061be31965670616e9683866ae44294f5376a6f40", + "4d04a412a956ca16c52fc241daa05d7607215f93580007a531b0d254fd9f52d7", + "4d052ee0eb223728de2c5566ab2bc2532d03ad930b5f2cc92f0a40c1e75dc478", + "4d6bfda867e611caabef7d76612f0b5aaa6418d8225f4e6d42fa953a950d17e4", + "4de79945a00baf1aa259d5875bfcc7727bf66a46dc8fa9bc2231d72b821d48bd", + "4de923ffbb46a3f8e6ac63c98a68521fd21c1ab0f77649c95a30bacabead4e5f", + "4dee6a3056ebdf8fadc76c651ada138068fced8ecb685dead613ff9f1ce21912", + "4e1e2a65b95720debed07bccc2ab2fb4530f46961a49a58b3d06121dba5ac69d", + "4e39cb23a75019301edf20fc52aafa4cd6af401443571086d35b2bd0578b742f", + "4e5a8e5c4554d8fa8a888d419e03dedf35272f60c09c06494bdd6025f1aaf47a", + "4e5e55b9e0c37032de19375ef4f580467a7d16c24c117a936a64b1acedfbe1f9", + "4e96aab0b69cd5652b2226c538170f17d2c99dd6fbe6558086d131f1ab426fba", + "4ec8791a63530bde19bd54031d055d324e66610201287ff0a15c2b1a2fda1534", + "4ed68cecaf66306974ee0a1c02ac6d5522fd388a6ebf63f1260ed6bd778e6ccb", + "4ee615c9b08f46dde44466f10ad5597cb8da99157dcf86430e3409d29ee2e6e7", + "4f2bfd951868f8f07f40b14079b16caf8ad4ff9cb426f1f444e36016d29c6d3d", + "4f44da2e6a0ee3611a2f59dcbd74e92f9003581abbe0ffdf0eaa4441c485f5cc", + "4f5acdad32a85f1f0913249599e5c961a1bceeca6258089074bd6c79525915ff", + "4f7ba8c30d185c5a7dfa3e424b3c9e8f2a2bffb63293f27438fe741526458cb2", + "4fbfc62a8da5a39510020c9ce45669045efe132b191c2e611b5db80f6ff1911e", + "503aa9adefdf6b5f8bc9fabe338b915ccb08a00434799b4b2a9887bd2475433a", + "507197a8f0ca6fbc5ea59f39fb0f1f93f463b87ccef674d37f216214cf048ec6", + "508d052b57de3a3b4e3c114fd09f3c0e1390e61331e018ee6636a39d34c9ea42", + "508ee9e55ec34d55b104c6fc261d9eb746e4e70512ad86cc0b4309420b47ab66", + "508f1063fa711383380384576cf4b882743780c40f5b068d19ca20f21a78498f", + "50dd754d35675a097c9ac8e548f50f62d860771bebba8ca0c32e8db8da81bc38", + "5106b9ea61a0c1ad92726963b125faff828e1c21cd1872565f50b1cd3aec3886", + "51164d3f4bf9e63d7d12f2557f54de4f38cfd15f84d3c9ac6aba4e7b4f7cf00e", + "511f1c32d909dba225053747092b92f90309b832ca3b77266befcaaa1127124b", + "5125d348c3b07f6c3178ec0d4f202d1651e134a8c5c54b12c2a51f703328f066", + "513fcc34df2dbdeeff9e169cbb3da37e3f749e7fc1015c015a9cd546b0044572", + "5145f5c1bbdd99afaa2fd7f44f5e9e1f65aa81e02190f562903a48943cbf81d6", + "5177307c8c4ff23aecd65de6866fb2638d3d7ec38857cfa69a6b1735d3ae4ceb", + "519a433335a476427b4e44e79c849f394218055b0b3d55bb5d827d57404235f0", + "51ebce2f08cf13324964a1a686b9d497861cc807cf4f352c649a44836c3e94a5", + "520c9cec4ebc993350ca357fcfb6a8c3f6b94ad865e20f46ddcb517c414bf9dd", + "5220fecc9698d8c201b67957faed62daa1a60bacef7cf5654cddf7e2b7ccc75c", + "52259a41e170c1c126230665c1e0e2b9fcd62ebc41309bf079f2d8372c1e6d95", + "522e80a35e589f1d0ad9f4f7b6b6cd0b718e84a27b5001690f1e62f9b932360b", + "5238327c3ec5fa63a5a4cb5dbadda9d28c0486b41c53f91661a2130bd715baf8", + "5244e6ef0b5ed9f3207115544961ddc1ebc1911fd6964bacb01d8bf76c00d659", + "524c7192963314b4237bce27dd2894d568e7fc3372c9b7d76e772e53bed3f6b2", + "5255f663fa401436723b5d5f7894ed1097b9ff37dfaf3d2b746e8f5ba8467982", + "52764263ad81241602654110f7beef81da96da4ec7b8c60799edbd5857587e37", + "52a60872f2deacda62eae94debfa7f6940e2d58712570cab942848334ec35422", + "52f3bca068bbdcd19eaff06f35b06f7e535d88eec057180ef79c1d408b3b6630", + "52fa559775b9d6bc1a7a600084774c5960ed4a3bb7fc494ebf021e5cd1cbfdfb", + "52fad5a8f3e0c1a5efe527864537eb8c3ee9ba1d18e21bd0553925d1a4dac3cb", + "52ff1f7716b9f49065fbbdcc8ef2ce9ddb14b91daa85fdb2729237d0e59f6cc5", + "5305de227271bd209a44b2db1b715e7d261f4c402556d75a55e0f1b527f60567", + "530b04bc8910bae2df31a59dc8ac9b7ef4f4dc854b82f552a23044fb168e24c0", + "5330727dddfd3cfbd27b98ab33714e4c1acd7aefc2386acb5fd788a7657ce576", + "533ba29ec6475d2a3d97751dcf7cd534b0ac20aa01c93b7bb6fbf637e84f4acc", + "535739cdc04feef32fb71d37d31105fda85e8465082bf605b5441435aada8542", + "5387d01cc44757a81ae1ccf1111f3a77fe8c210cdf6ed1a991192a80f4cdb0d0", + "53b018be0e50f8ad61778e2330ed74ec2ffc8bec9b5d06a8cba967046f7fe86b", + "53f26ef79ca484c2bff9c31ed2c0b2d6ac2fc5a437d9b29fb1c5ab9dca002e8e", + "542faf6ec88e75b833dc99ce6d5fd4d30eb3373afc85f9f535fcabc38d0ccf1c", + "5468da1bfe41c940866fee9927d6945071614195f680f2b64b83c55bba3192f9", + "5494e181e6577586ec41dd3e5abc372a33e765cb03daf56d3b20db748dc79b80", + "549cc016d0d8b032362d4d1649408bfffa31446c2590b2e12804ebbdc47fb2b6", + "54bf2c3e0ed21b66938eb7623c57deac2f09ee563dea39c3e14d86a9217d2654", + "54d8b11fa7465c7082792100e111f8e33492e254f3892a41af6f78fc123d2454", + "54e5ea5fe2853c502e96c870d0111a15365a5f6bce1f3193a7b9c348645481c0", + "54e9dd90dc1debc55834f9788aaded500adec9d7619cc66817eb3b8eda688b94", + "55043e55212e6205cba548efc9313fdbb51cfa8cca57640fbff9dc8aab528853", + "556445fd46eda083b9309061f7e2fa74f036037f632b9b970eeb6722d23303ff", + "5573a9b44b03589da0cabf94c4a8e8937abc3d157356d864293a721fffd6b64c", + "55bf14f4a1a1eb840ef9a1784a65ba2d04e37319e45a2acf8349860cbef6a232", + "55d1286538ca7f5a4fd5356054a3b8f1dc9526876d81e36a0bdf741c802f1650", + "55e77f7587c4d4a56c639ef0c76bf425495c87ba7fe9cc3ca0c1cf436418d585", + "55fa2fdacb594e28831ed04b9b0fa4a047abd8e550f7d5eb724679036e528d53", + "5613fb98363341a6104fac31488fe8fcf907baf29f54278bbd9df7342c0f8685", + "566626d5c211e1d702e139dc08be33384e6d823d5cec75a6cb7b3310c6998a0b", + "56c1062c0c0ed6a0ba04664b78a6355cd2fcbcdac3aa93bc06819def28aa37f9", + "56e70514de14b66605e2b2e23053ed771e60f1e0de950cc1fd0c510a9a5301c2", + "5706b028c6b3503ed4ec4b5f490a5ccd746f48f3a8520d31d1d83cca2b8594c4", + "5709be9646cf910153f3f75a496597712cb7411e2f9447c7fef7fd492b32286e", + "571925f0b66c738b52105644b24558c21771db1f4432842dcb87a1ab468a74a9", + "571d8a20fccfe7b2de32adddf80c3e3f437341d84ae5fea77e5c55d8d9201dfc", + "5725a7914b447c22de18d20518bfd925addee19a17d889f3625836ebcadc972e", + "5740788c353fc6c81713fd4349e0db8b199ae0e1e8e4d5adc84dac9087e53e47", + "5743c10d43e919f0b522f535e9682357ae759de4f2044f5913aa11483dff0881", + "5743f3b3ae3820ce68338256058963f97e55e5242640d1dc30f974297d9391a3", + "57441d4c49333dd837ecafa0038bebcee2d990af8ecb55d4ff3b34ee9e0fe376", + "5792777bcf215d5f29e1271849674a435ed2a491f59bc498a24f761716bc833c", + "57bf1e0123dac8c3ff51ddd5c0e66620bcc27ae225e8c35927529b2fde62383f", + "581c0ff964ecde7b34cfd519ffec1d7846dacb2d96dcaab573231b5e824f341f", + "582bf2770b2a7a438b46966f406f82fa7f23dd8ad0f4c516837964af31808117", + "583ff7d92c90bc9811ad96d829f25bd68db1d0f5baefc02207c3c55179ffbd1e", + "58587d94c44401d22c7570b425d78588aaa439b5ba8a02532cc32fafd64b790b", + "5861d48f3a22385c21966caf1cbff9e534b2bc6d9b24b04a1c2bc92a175101cc", + "586df6b16d0371f939f01dd0fca066135a548ec526170a270879a775c4d6e8a7", + "58953bd77119c5a75dc8db52eae11aaf841ce9f6acf9c724aac634e0916c98d9", + "58af007dbfbb09cda0c25a22146bde68c6101750a61cc2bb16773de4fdd0288e", + "58c399854348d4943780f0593aa64371481e164045bc1e02af6cd5435b68a781", + "58dbdaab4d64ff0407fed71b3944f8a45d07694f2a39bb72cad56d94606c61e6", + "58e1c39129cfeeb249160e0272d7a56800521b08204d103bdce6b8867569c0ae", + "591de6a6a561511570150c61e0dc026f7babbe07e022131910a92f7523704d93", + "593124d7a97e611c01bcd92a663ff46fe9cf2d26cb22366fe35be7232ea1908c", + "5951167c8577ebc5e40b7323822b0b24a5da3268fbed0121456adcd5546090b7", + "59958e12c7427964002f36c93678bd6093ddc70364b2a735f592ac7b53d303e4", + "59a558b41fee06ccd7a1c840b603c99b1de92ec6ea6deed3152c01066a533212", + "59c8aee345f6da52cfba936abbdd140b1f318a80b616fa7778d12bab59819e3a", + "59f012671896e2a38807e6186a07c1111855296e31327c5fdd2a1dba5b785f3f", + "59f0eebad0170449246bb9826bc605a71fb122133ad7d4e7d99c69603f8629e2", + "5a07dac7bd8ad5b87afa1bc38b0276e6480da48f5e6d17b7825c8098bf88c1ec", + "5a0ba0d9ac4d2cd1f66309b325bff32bbe54816c4dcfc42064244df39d65a2a1", + "5a28a5206f5de7aa53c740e1b5b5b6cbbed409faebf4eb9f803d7c63a5198e79", + "5a3bfba80d88b4a3978f734acd65bebd13219893a477685dd8a71d513d02f424", + "5a7cf8dda296956ca9d978266b4cba91b8c4923c9b21ea98e740a35cb1901b50", + "5a82c9e9b43a93b1d64933552ec8836d741ca1406b29f8ff0c121ab3447c795a", + "5aa9412ceba584bc77c6bbabf4fd18ed16253403928e49524ed59535670e5e1c", + "5aaf3e59356111f7d87a5be93f33806c428219dfa4ab46dc5790f8503e31c581", + "5abfec56f23ffc8b44eee5e710ce21cef1efa1b0987dec1ecf800891da7b1001", + "5ac17fd232e43b44cbc0abeb3705d2538a647767056a2973c6dc82f24ed377fc", + "5af847ef35663184ba6a26712cb6a7344a96f2fcf273fa3a292b4bb46701bd02", + "5b2ff51ee5fc993256b0d9a247a5fef9874fab06a2fb561da7741d48c98c4700", + "5b4f43cff4bb797638bfd67b131632468a2c82a8f46e8028c4b2f70e5c374e5f", + "5b5cee99d8eef512becf6b4c6f2adc28a7674b2a5b2b034f6299ef87e135cc1a", + "5b81733f6c2d4875de6d68334be907addfee1ee916eb178a17b4a3532e57ce79", + "5bb74799899c9c13b3de7b14145ee9a36e517b3255b98fc68ed551e3f168eaca", + "5bf76a6863797b85f12c1e9657dbb98cbfeda76a85e5649366bb51c2e25ae0ea", + "5bfd333e5ee50ebc3ef7343214119e6485fcfe62177243c9ed44ca94ab862a42", + "5c0660c0a889c03272b8cb05ddefc3bb0f3208eceb119af380c167b29c1152a9", + "5c12ed807c668a70cf1e43c34b6976fa59d4a75b94cf5f46e6a66553dc551001", + "5c3800b2100255c46b68a1c6b3949bf252472c389be1176689bbe9072892db81", + "5c7875a957f4a01842286acce36973d6f1b43f0b7be8cf309ce3d3fc1b8a576a", + "5c81f028f79f4c7834047b6f0e6a9099cddec1698b588f49fb095cf79ce66096", + "5c89cbf0a0308716fa318e4b90d30aa624a3420ecbcdbab00885ded6dec88f6e", + "5c95d9702aa1067cb6ed8b2ca35dced65dd135880926fe90f6723d32342ddbfc", + "5cb3005a1e6be72d09b1fae86de8b44c7aef9f7b3fbcaf7be1314471d533b087", + "5cc8076479ea7c2283aa6ca570696a1ff9830912ffa176e470a8360a0b94b1e8", + "5cdfefce4177c0db9df5fe05c4d9838a299082541dc99c4a2b2ca62eb09973d0", + "5d197d6f8601540b20d9c8650ac6c7db604ea308f7ebaa6455f4f4f22bb9095a", + "5d5711143100eee894c191c7308a6310af56b08efd5d96764e4633c39d472f9c", + "5d77ac5abf8d323e7a23e3e0fb820cbd184050de53e1f6592102f3ce1888782f", + "5d8d05965144cc8ed052b1ebea298aeaa8acc790f297bf6b360d8799ffefe1a5", + "5da330a2d7539cb8079584a52f6d704f1dbed9de147c04d8001ff5e11b0536ea", + "5daa16db0f4094f7ea1436e6b4bc0ff8eb07efc22c3b63675509a52a235c51a9", + "5db712157f9dc54dead72fde0ed3dea5df4cdc2f9572b76c471c5c22b9232bef", + "5dd622ce281dfa0b24d79e8399b00c68bc086ea44ce4326920fb148b0e6f4d78", + "5df83d897e2924cb350b4a3305db4d44e1d066c990aed69b29e43e0f6698f533", + "5e1afb6bc141daf303dbab37f66ee10ebbe8b31ab3c5656946238e2eafd6340d", + "5e24f0ff25337bcf32fac07d4030e897af5dabefcfbeb5f1cf2e5403ba82b9c2", + "5e3432bde574450d89842f92daf06262e8cc64d281cc107ce3f103dd32cc0eb9", + "5e4b5c42a003ceaea591f2895a3e0f3102897dd8b1ecfe91a8bd5b5f562e96f2", + "5e589de637f00be39babfdfb626967ed63a1cd173634ed66cd240dfd436de90f", + "5e685214a2659cfab9e0ef45c26ecf4f5698b852eac4238043d62e60fa539519", + "5e6eb52c3e99332abab43be7928ca3633f011753547397f5a314b3e166035a48", + "5ea7348a56ef47dcb20a2c685063b693a67edab31a66863d7db871acf8dc477a", + "5eb9a73a975f07b91f8b3e72ba5c8456bc036afa6e319978295073db5afc4a6f", + "5eb9c1eaefa61748f03a84a8985aa3574ff723f4ded3b96cc6fd312de9f478a7", + "5ef8af7db8c8467bc40fa36bd555fafe62a1ab4e9946b732bd14b8b7785fda5d", + "5f250cb3bcadecde0bd95597f5d4fba1b26740d4d34d012595438f01455e9e59", + "5f2f285781ede7bd7228d4bc4cd74e2e679af4ab6f79188c80ea37525786638c", + "5f3490442150aef643b464dc288c4a064717afa7e6fe82a0db06d12bb4376fc5", + "5f3ade9c6ab3e134e04cd4a11338b71299eb710e0b2aebeb923e7dbe832020e3", + "5fe9694c951479112476e396fb24b6701c41954c967143688ecf85829914c8c9", + "5ffe2d07115394912d1f6396ab7085e45e9aa33eb85d986ecc24d1b720cdaa1e", + "600952a30695601677ef7a4b490fbcca600890c72bb1fcba30e6608f52a3b048", + "601938fde46de7f2ea79924da1579b5c5776505e76b41d86957a714552446738", + "6050db010826953bf2c017835d673e3201d38873160548b2efd17911823f3310", + "607db073c05e934b6c553b86c9b277d3ff247c7d7a6b8cfb70158246d56eb0be", + "607e3ebbfd8fa181ff70f397c7b22d0ad49597cb5cc85ead3b09821969d1e139", + "609f5fcd87f8b7224c173b7705321238e7406b3a54298777d7fec85ae6eb2ada", + "60e15063301376c03a8acf67435bef4242d6ae4b1836722c25a338b8c1bf4024", + "618d56cecf230518ff456c9888a33aaa90bf6b63edbca25f220851a7d2047900", + "6192d9e645ce38581a2ceafa43b9f5a92a38ea33439bd627797e16f54d0d2792", + "61a51010912747df01da5cd08f06e107a55b3e04a0ca7f99169432a3c2ff7872", + "61cd4801f08d19367b08eefc34e54ca6118b7c9e45e4e40bfd123134f8dc6425", + "61ce3e015ff0053f90457530820a7a8d46fe13e094e36b34c455a9c5ba1259b3", + "61f7a10997a6de054614fb67050e4ba86dafac0e7e82cac440206469de7de82b", + "62188821e56e67a9095da01261d799a8f4303257116295a6d51560bf7f96d272", + "621d8c4f8a40c4cc92b77b97df8bcc633317af6589211de5485863c573897f32", + "625750077e5ef43a7436d8a4e067f2bc09fc9c037e0a37b9c6116697c9af3c53", + "62702c6a819aeb5926918fc55115d3ad2a7667b0465dde4335e9f15cc5e91e33", + "62767847449284fce1f687e9c8015a80ae44b4e5b3d741177f2d5323e438f32b", + "62b1c59113149af010c492e62e5ccb43ec4adc764b9536cd0852fe9bcf2c897f", + "62dc61fd40b554612b1ba15c8cfc78a3a034eab498edbb67758f37664179fc37", + "62df329d815d6957eb334e427fe52d6ce5d18863f4e2e812c982004f24bd1d3b", + "631fc475f1c091e56e359a9aa4471130ab9a5895582c35da8b378c747967ba26", + "633a4fd35f046f06b4a4dc0e5a90b1e477763a2124535e3e425c78e25527bd4a", + "633bf032214ab02575b9908ade4b36342e8801f3eb6dda731f807b6aa89547c8", + "6341a2b834df92c235865fc9924d178335a4744ac558f4a2f96bf9b25441bd0d", + "63606c5d2195afd67a85725058a3dabb0855cc4b56ed7d66048433b523200118", + "6376c53f85f5ef69d5b59e136d2ad58d7b045fe50d0696662cc988f8bf145673", + "637b7b47f8f66722655cd95433ff31ae247dc7285c3348a7abf5dda931b84f1d", + "638daf7eb92d77f62d09ad78bb2c3f915e9809db388c9262deffd339d42f8b10", + "63a27ddfb2656a6745bb08f49faec53d3e559269161b2ee647134804ec18305a", + "63a986aaa6d340794b5a30bfeef97f8fb7b8fb91e641a447f129bf1dcd635f55", + "63e0e5c2a713106bccfd18bd22f26b4403229956df862889834518e440e6416d", + "63ef9a580cb94305c24269d0542a6ec21cd0f1e2c91e57f55ffbac99bc7163a6", + "6423ea5b8363fe772ca1e62ed51591ef0e2c7cad699b917253275cf8a71321db", + "64365e6dd5353b5442d4700932937991b7775997dd01d8fe6453abc7cd9a3e0b", + "643db93d793e3e730baa3209418e81e3f268dbb1e7a481ddcf59300a95b2d8d7", + "647a28aad4b02ff76bf2259ce3e1c17ddc7b444c3ac52705e681c64cb175291d", + "647f41c5b6dfa99ea202bde2c72d94b72b96e1e16cb297d7e00557a37c227154", + "64a7a3087e7a42aa66ea37486727c3874a65ac8611e49acf44d804965ea352b4", + "64d351d40a34f7884df2f840b7fa21727f8fed6df9e05e5b1fa6bdddbec49ebf", + "64d5c7ace5d90a493707fde2ba69c96975472d619423de17d258a14a73d6c04c", + "64d807ab260eb9d668450a8f1935359de7f4df518fbd924d220a81c6329301f6", + "64f76ea78bd84b0f135d3ede3c9285c77bb30e844a15dcc5eaf6301ba843cff7", + "6513fa0432f4384c96452465cc6c3482864a8e8fd9e7805b5a6a162e049ae062", + "653a93f89a8da3751e456c49059224b46698b894b64350b7e3c63f44658e0d8f", + "65647f03bd50562243692b63cc1fdfc4a754a814a21bf410cf4e0c48540d95c0", + "659c2ec601cdf969de0359d6b82058e778b563e99581d841af4c429ddaa1edb4", + "65a83ee09df22d92c5b76d1ed8e2e1f8d63cc02c4abe190763340d3d8313eeab", + "65ade03bf0d5ee974f9c8bb9d4c4a00a66b48e1bbfc7386d34c7a4170d16c49e", + "65cdab08c1e4a155c6f8e5617b93d8648b3b6a5912d4b790d02857d3507be9d6", + "65e5649aea03c20cd090502514db112dc347491b82ab8eca8b6f394a5c1b51fc", + "65ef0031ba9a2d6b0ec1fe5f29e47b3450c348c8d790e5e4d4900697c304f440", + "65f80b4528d187f9bcb15ee1794c0f43689c5d0b08bc2a75eaf2a31c52bd341c", + "65f9260cf8b1be768366e28a641a648f4769425ce0b37774259be274d7a64038", + "6605f42fbdae3b25ab1530fdb2e5d131f1b1ad59ed024da45c2da15fc8fce44e", + "6620b6219ad7c9bb82b4fc14b69fdbe3e0bb3f0a843350dea575f2a68352f8bb", + "662575859843a090be178ef11c3ff949b26975089fe6a0dc0efe041da3363c95", + "66477eb8101c0eae9ca8d6e00fc5af48b1ebe634380b7a368bc9887036d9edc9", + "6656cf871958da5c32aeceafe2572dd475124aaaf701cdc309f577d418044b5e", + "667f496aaae9c55dc80a3241ed1680bdd57dfda585610cc577356107c47a67e9", + "6681e3c1384378849a7a9295a3c4a4753b4b65445095201402636da3729f2322", + "66f368c851503c59694f9ee32a9f32927b8fe9fa1e80dbbb701f87bcc048e57b", + "66fcec4db0d060ad0ae50ac16bcf92257fb5cebf0a8cacaf0097b4a3cf590463", + "67252f580d6b891d16bfaf4180aed1b01ebb7ab5d6ffe0ff6dbcc3f315c1cad1", + "675c6a52ee8c1097c94aabe8f89da07c84bfb1729b4fc83d082035b4098cbf39", + "67aa1cb881d30869a3568440aebfd1ff0c8e52743ffdf4deb1f648dfa9ec27da", + "67aedb4ea431c92855d75899e571ffc8a0566485bce9ba162205149c951b08fb", + "680b5d1f7823434a26a54fa967b9243361a3bc8505f6d3014aad56e8255fee8e", + "683d4185c296a570ac27cc4108e8c1b7a14bd682b0b04879cddc4786448ff178", + "68516ac9f1a2c06694609dbe17e755798c4972d137779cbbed90bda9ab0dbe94", + "6873b65c87de9d43a3be6c7f6c3dad94b85dd07d3063f127fb8ff9736ed74855", + "687dd21c9c339af790cc61e3520fcb729bdd986e40acb3cbac5b21882b0a7d7a", + "689c596a1a402a711028033b1e00b31e7a30d93605654124841b30283edbfde6", + "68d2c928a05ee93db92e1161dd420921dfc98e3bd055cea5383ed8c450cb9a10", + "68f0dda42d4f45fe70c134d627e1ffa69479dd40159fd5d8164b14649e82b2c4", + "68f2c4b0e3f60bf7f398c9b6aa28bacf6d2f5e7234b3f0857e3d06920c214c04", + "691a1ea063bea59e2ea787fd37e06c333998f93705c02f8c88badb80f0e08abf", + "69296a5c433d9a8b385e82a2c65bee4a2de9a80b4f77441559a9ea83f3f6f1a8", + "6974bf3ae3d564d5bcc545a797bc0b2e11f69be43b840119bcf58374566c858b", + "6974f12d69e3af683d1ab7a7d20a865486783d23b4e1c282b154cb6faaebac71", + "698c4b79a847e21ce2dfceb683173bb2517d401de64c50844cd4ff796bbd98cc", + "698d53ba96b04fa3bbc0ff889ce2ec3398df89fe1d09022d4c1870a6360c9074", + "699c232eb369996e6e1955d8755aa4df666ea7adbd9171521c1106deaf28a2b9", + "69d39be858ab5a0d1731a1a0de016d5ef8d400c011d243b6a5182f1adf354ee8", + "69e7227a17f559f45e0e53f484042ed1b12edea43a4f1c9115d0134f7dda7465", + "6a3a2abc2148efeb99cf92d43659fe4437ad3f381fc5cec5cf18bc297c6fec96", + "6a690eb87882ab82e8202172c9cd7f6e3c9fb672069eb0ab8dac0e7dc7d05de0", + "6a75bcba06bd24c12169079e32622867190ffaf07100424ed00e1858c69822db", + "6a91e9529558fbf167a7cf0893a07bd8e16faed710a86e06af068fa15bb67873", + "6aa592fba84736f361ddcf66707fe15babd6f50b5f1fd49c21c272ecfd7f205d", + "6b4719b2a5a8835e0740c73c2bbf6b94213866794b174ecfece0f737f1cd001d", + "6b51b99bbc348bee5c4e4604da974fc566d70867a4d0e011d55a252e06cb0db2", + "6b7822407423f5266f91eba2227e33c75eb6cb85dc67efc76b07fc06ddfad88d", + "6b9663fe2428b5109a5721f7e8866e9e69d8a62b1f270019ecc36a347603e821", + "6b9c5aefef9b58f1e8e192fff34cf257b08e466aeaf65a2853c9a0d88a19502f", + "6bcf9d9355d514c522b60268466703f3338beddfec4380bd24d7c022b8ec87c7", + "6bd01a73d20d03563f91657146f0a6e11c082e8da4d74ab6f76bd146ad9610e2", + "6be5b5b85ec4edb66ee377e4682a06b3882e7b7c6ea4b8b7f7d3fc74337e2dbe", + "6bfccfa455cc73d12acfec2372639c0a80e7e809e43f3e9aa536677a4497fb60", + "6c06419d83f639d799195f586480be6d1f65ad715cb88b9f2700086f6b70e594", + "6c2f8a4d7b49e4bfd26d82b1779e7fd4b6ce7f78de2e47b7005a63132718bc98", + "6c66f8b2ea659391d439527556839fdf14b43d7104cfaceaa2c50e54b7bbf966", + "6c7909f2a90be8e7963c586b202e174f445d8d3d7dec3b6e864e136502484008", + "6c7b914983c75cd033d8e9266cc5df962bf7329113b58dbd8b29830e93ad4e7c", + "6c8284565dfd154013854f88ef41a73491447a92b69ee0313f8266d2007737f9", + "6c86c884635b1713342b8a2104ab5ffcf17c5f5e388a2f1396f36280c8135b66", + "6caeaa904146cf8358cb8ccf09c6a44c64e1942acd24fba6432ebfe674209788", + "6ce3537f885d05ca6480438c7ce3e87d1f7d2521eb0f31f5c08e47cf301be28a", + "6d36f8a775d3eaa6a2c0b0ad390e01f35bb6b8ea75409807ec9b9a3c93be592c", + "6d4fe2a0608ef4ef22a00655ca12540eb6636334c5202261a66dbb6d10a58ea4", + "6d896096e52d1cccf599dc7fa94fcde94fff526220ca8c4ffc626c42495df1a2", + "6d96d09c7e5a6a9112a35e96e7b57c65de7883a1f51f9a5db07146ce857e07cd", + "6db2b190ba0fd5fc752e7285ae272f7a391803e52660e928273987805c9ea2ba", + "6dd7c52cad4d13afdcda3ca78630bf63ac86ee745524929fa91c1cc9ccb544cd", + "6df1495b93ab4d7cb31c85976c88767a2906f21144a7a8b856ed57c5d584700d", + "6dfeb5a052c7866d1798360e88d5714fab6693772b66e0417d45c74b5763c33d", + "6e21c0b9482031bc9f387ddd33a53ab464c4a00c1eae99f69ace97054c81abae", + "6e428659f64733b176f25f790389d3f4347ce2862ab67d3ae1fd4ca9a42cd330", + "6e586147e1f0d42aeeee54bdb5236d097e305ab7aae76514ee6f4acd0bceed63", + "6e6347ede261653f4adc6b01200384e013ba4103de13e31f7cb82cb17eb3e109", + "6e63daf41296594baabea559f145029b9fc5364946a47ad7a4899ae6f6c8f9f7", + "6e90c6bd75b3a980e10388fdefdce25fc783e51729cf6f00c8f652b0f4355365", + "6ea1ebde2882b97883e62dfd80d3918f6c8de7406f0de4ce9619c25b2438755a", + "6eaf4819e64e8f094c8f1964c77bb2aeb952e2691b520f2ec2fa7e11f52fdcb3", + "6ef70efbf5db1a44d48e3738de90d485aea621d09b3efc0a541198a5d5b20bc0", + "6ef9b67e2d1d4d36228c7b5218e96042ae7ad65312c2022c52b4b14c67686487", + "6f25c4816728e39049eb9e3a648e2ad713e25e33507c05f83c84126d5507052f", + "6f3ab3548e1e543697f0eb90ee19d20ee9fb9bbd8b940606a4b0fd8ee6595c0f", + "6f3b4625f41a1f8b908a4f6c900231e22e702083803b05f1de408a1b58cc987d", + "6f3e158f32511ccc6690cd472aa9f02321cf7d353e167241a7c6ff92134c5994", + "6f4218c3d3934cb40b30b14df5a72de35d8dbd1146159083cdabe234471186bc", + "6f79af20b759c197b9bd3b1b63a504fabf62e15f570d70c5a8afc28aee5559aa", + "6f89168841ee0bd7f09a457fddb74638c7bde20dbcc26695b6a702ce2126e883", + "6fa265f656f2ee4d7a2b8c55b477ded670ecf48496fb9c4afd98bc0d32d39ebb", + "6fa315dac5b32cf7abf9acfe27c648859b2ccaa3b6e161adeff396ade3723978", + "6fbb766aa08fb955aa7577e9a7f2ef8d9fafa9dd0fc8d3521ebf5f460301cfda", + "6fc5414ef35c32d41c91fbd549f5bdbc0d8b0db216280c83544ab0b8738b6f00", + "6fcd3d40f63cbd59389972fb750ee27e1adcdef6a5274943948f40f56116fb14", + "6fe35d8f5fa325e35f4cde71cda545655b1c653f1772bfd8ce8fcdc280162ada", + "6ff0db043be5147c89ea98552ecdb3d43eb872a907cc4797c01bb226d980f5b2", + "701eca67391efee74e0293d67cfd39e615baa695474d81970834e9e62c7d3cf9", + "70333759946dbc2f084534f94f76d6604dc5dc3ebde87e79bf4bbdc12a7dfe9a", + "703768057979fb6f20c571ccc72439a6595a20f77c4b01d4b2d1bed855e1cd42", + "703b9ba161ba6e11b97f64d87fcdb54c9f53272699dbefe94990d83d2271590f", + "703f7f554bbbfe686479fb152fef9e4e0f8310a827e86b8515adf24b07a620ed", + "707e3480e3adf85aac5025238159eb94804c76fd671e28e23176d75048a511b3", + "709f5003d5a255e258b1cb7a6b18c421a2e1826b448c72ed8847eeac5497ef3e", + "70b3a8211a06b1d0caa06b96b49926e57109a461918cc02186fe43840ed9cc3e", + "71088e9aae4a8185d5d27950295b872a98ea093428f97864c6ae60a04f332a98", + "714c7d5079aa57ea41eaf89e238ecf1f20918147887e41bcb8d6b2f6dd05d8f7", + "71673963d8dad3c8041c1c46fef6c23da0c49ea1df404712d1d2cfec197b0574", + "717caea1d37df06e06b5cb5b9cbee7081e65491abdddc40c5f073f499eb97c89", + "71826c795a682894dabef3ff7ce3208029ce95212b8b8149010dd01522a124a3", + "71853c6e61e8a444c03052320348c810c0073bfb0d31ebddcfb9c4e9c007c02c", + "718c33f21779928837d659c7614a844276b6e76a5fcc9b62a092a25fa2c29184", + "7192b4a737df2076f40c10e6893c2ffa904b42694f1938960401a11cf8738d68", + "719712ae6554824342c77f11e57208d5fa78d40528df8a18e61d48b54430ed47", + "71be758ab30f48eba0225f88b5e4677fd61309234c44078db44801c879ba3074", + "71d9b63e6b32b3a79ceab2553a5d4b8d0d060c02034ce041a0d86081287d851e", + "721f9f634bf6ebd5c7ebed59495abec3cd11c9b72631477995ecba11e5dfd8c2", + "7233fc7cf525a3643d7abe89bfc1f9b2cd0a11f9936a54d7797967f646c07116", + "72462d383c86b76753eb6817608dd89ec609c406cbae7ed4e717b595909de462", + "725ff03054935ab9923ad90ecaca69238e39046d3fbd14000e0b698401754d8f", + "72695a3c8e402351d00ad77e266b6ba8a282aca36289e88e36c61709877919fc", + "72e78cfdeeffe0bd2597a166d4da21981b87977df5f8c29d250cb10e51cb6581", + "72f4d67ba20569c6b7596ec75c0e033971e148698de7f4bebcdae8fa8554db17", + "72fa79b3fa115e9812888bf78b82495c5f4cc6a234ce5adbc3eb386984c3184c", + "72fc5100b1654ec387007afe56d8bf5eca6072396db1f7d944936eea41695f21", + "73024882c66bcc50e1eb8fa5534a49ae5dfef16fa8869944b961cfab6364c0af", + "730b69a152e1b93d05dc8d4dd430cf48831c4edf53558e148161613762d5e6c4", + "73298093213dcc7424d426cb063933c34faad99e3ecc54f6a1f9da12e0757b16", + "73466bffa17e0f61131673c325dde5e475e2534dced253fa6c2c041060fe8d16", + "7359f0b367c18c3514982a300af59a3715a3cfbbf68ffe9619532be27eddc830", + "7364b82fd2ea76068008093d85b78ebb04a31672269cf9b25c4fd1d1a4c6190f", + "739ed04dab3d45687f8a69040dd7d3d9193876342abc769ea1f83d4cf177934a", + "73a4760cfd30333ca79502c85b198c3776db9749fa4c4b38edca375db671140f", + "73ab62d6f93a31d65bf4b47d0752b58e6bca8ee092e30a0b75ed176165770b60", + "73e19802a19244d6ba391c14e7e910267e01e9d9a1823acff209361dba4f418e", + "742111f5094433a5f24f30d74366d717136d0c7f310c03cff6e4c7e4d2f16972", + "743c7e0bcaeefc11b5879252821ec7ba7f7e617480ba2989bfda8db33d43b8a9", + "74684ee19ac68b9d5652c36f29d803ac7fb131c41bba960ba12cbfb38f420374", + "749d83f2bb09989fe35c1b6eabec5bc4c5730e2dc87ca1f2cf173ea5fea2034a", + "74a3878c4403073afe2c4a2779a4460b958fb4687381f2f40792a88a808cff1f", + "74b8ff4e4480e118d6a2f4918ed91f2bc0585219cdbbf6062ce8fb1996f647d1", + "74ba9ba206af46de2e61639e2df0374a8248861b38702c513176fff43a393c41", + "74d5ad78c8c335d3eb3e4b4be59efe582982aa2f607c15de98d25794595b3139", + "75049ffb0430bb1c3b413f523d00860d9ebe79608c5b96638715b44457d8580f", + "7521f48d4093fd7a9f6f645199fcf1745851467131ac37581781ef7d2b225df6", + "7540f3ca514bcf1ec786b4b7d11ba9f68eee79fdf891ed8c4f711c40e5ba8b25", + "7579cec0a34fdeefe775b1c85d11e936c39b6429e30189253c52d8a9230aff9f", + "75926f87d7473582ce80f378332a95c7c20b6a27ae07575eaafbd3203b23322a", + "75b7d995c7b34a8667be75ed3691324e218b44f94f772c4b2c3765401f01d106", + "75e79e1f3af52e503798047cccc630e2d91cf5525851c0884290c03dfde1aa80", + "760dbbc813ccd9ae23ce4af764de33059e888bf038d7a65e5d6816892f9cc103", + "764089293fede6ed51e409246c5cb35248fd9f2538c7ef10c37b36cda222fcad", + "767475c5471db706e5873690e0ce6b40160e0cfffae596f2e0609f971f41f849", + "769b6e89a2fbebc067461c2c54e8f79f242c729bd7206183ba670dc65b275f82", + "76b0bc810534bba4a859942e88a604edb3d394b8421d516298302ff23557409d", + "76e965b06bd3d10aa463f44812e8ea04eb8e67c95f382595a70be22c42dbdd4b", + "76fd68580002a202777d8bee3d257cc28cffeee5de1799bec497fac4427e0c04", + "77250960a3a38b0e46d4366dd3387196ab80ef8e5c4e224587156f973a55e094", + "773fc1d055664538e2caff1408b4786eceb0348027158c0ce749ee031220ac93", + "776acd210e8fe45c59dcc77d5c4c9249019230746124fd78e363d5842cce43d4", + "7794ba7ff588975da27457e5de81fd2021a7d5f8657c33672e36b38cd7bd13a0", + "779804956eb5a224028889be5dffaaea3d890ee8403e346b8a68051f8c122260", + "77cffae22072a98ce40bd10afed90a8a4ef52833566e3b14c4fe5c3603f0555d", + "78196725b339be6d51593df871fa995b35e14ed2dd70cac976afa3cb7330c05e", + "78435438eadc53e4b65b80e6f52e7d3b8f989b81bad7b49fb4b06e6fed2ce3db", + "7858336ed2c00e5618549fbbd572a335c911c09a4976bef5f8ac1f08a5933920", + "785bf60336e1b28f1892dcbfba28bfa6898b93802fe99093ce45e8fc308c7916", + "78622374cece0a3dd71545fd5579fa5ba742fb0e275d452adedbe3bb821a5edb", + "78698ce70fc4b006e2d8f6374ba1cfcf74c0a9c4b29b075c957d92f942cd73d0", + "786a54e1510ecf3e0c15b22d8b1cf54d50d94297cb6bae026b522534374c78af", + "78a73a8ad0f98bc4a4d8638ad881b815a3e32bf2a2949f180602e7875134aa1c", + "78a8a9c1a3b7908c79c7f78d726116cc70a98e29b738ac171e11b2fb25e1151c", + "78afb450b01a4b9fceeeb74d3b40965be61cba7e43cd68c7dbd7028e89268bb3", + "78ee6e15729bc5127e4e4fa362dc21a2482f39a6fd3cecbb9de2fff3ad884382", + "78f740cf7bf686664e07a6dfd745854f5b16eb9d211acefbbccf1f903b3c7fcc", + "79393d7ca4a928730d3ba7c7b33d3ac19a72629becdbfd5266baa2058dc936ec", + "794bb3b63d8687eb22490f6255abac5d7e1a4eb7f1852766700090be9c4ed871", + "7970e3c540590f3d367a29d9b5fb6c0fd23c809dfc652e359680d9e03bd1c86b", + "797f3cafd57e7c848b365be2a79eabc1cadf42857f5a2abc1eb2850b082a1f42", + "798f00477a084973c5c1f4a93cde9839dce6ca25634a4bb83e78e5e3ce9e703a", + "79a445fe8117b072375d8cb67d2ce686e79cca3d029ce8a703255513edd715c7", + "79b2565cd36a5aa7332f4cd279edc77d9daf30771d5d7b961967603af3d770df", + "79b9c50866c5fa2e695540ce7a267c0889d0f5f3f30abfb820d44a407009c1fe", + "79d537183284063b3c119f5bf642b045592f50620b0feb4e444beec840a003e9", + "79f36210ee715b0f9867bf90584758664f37847083647139d14935ead9805adf", + "7a1e81189da79ee94e93d47aebd5bcd6fd85b716169e9561f213678523a4a17c", + "7a51f2d1ce1a72560afb60675ff504b47fa01bb657a0dccf975a371525239d89", + "7a88199756be7bd36308fdb0db2319a89bad7be1a4336c7eb15f92eadede9be0", + "7a9442e42d1113539ae1654ed1ecbe54c67bf72e55e467c57688c0f4e4d31e43", + "7aa0646914320a193ed7cb20bbbe65d900bc659a2a581f5a44055c60197ed130", + "7aa4be1cc1fb76f5ee2d307f24110c84d565971c030a848da542be8edf937c07", + "7aa850a2ca05f925ad97095285f06535102b93bf507e85dbbd53f1ca3d000952", + "7acaf7a8d5f3879c46553fec4f100fb2a1dc9f8ed0c0d8a883489427ef2d28ea", + "7aefadb4ddffb69dd946a207c21f412da4554397a8b06865e239c602d2f41a55", + "7af5155fdd59640ac809c59916c3620598a785fae44a3e702aae488c119c5198", + "7b16ac75723b6f9894eb0599c963725531588337e41ad546eb8395327d5e0697", + "7b21c9201cb94c0510eadce9d4cd9a069c097f584a50c41e7968ce3fabf34d6e", + "7b85738369d19b76ee12a33138b6fd71f12b61c541322bd99b6e8c76f28d8f84", + "7b9501a5c4baca2b8e964d4fd1e9d6ac724d7a0a1c0160cfbc089920f900fcf1", + "7bb52dacdb2687faf5c22439851c2c834dd54c485d7543d0aed875ee4a1a394c", + "7bbfca7a917b961e83f03bf91996b05f5668bc93facc9cb94ed1fef301a115b6", + "7bc8cd8c3a7c5107c3c2bcbea68a070cf1ac231a74057404dee584fdb70c0a03", + "7bdb0e3427bab33b42fe14d28182c3273838199c7b439da9e1bf2fbbe30ce74a", + "7c01f13b6ffeaa36f3d6754644c6b5f5f62ba7d3adff55ea78ee05281a7151b1", + "7c13410bc3b2cafdecde4586c74704a26f911d86a07f147af9af91f24fac60fd", + "7c15efa53621d248accaee695bbc7adce989c445911c33070ed5377af5a738df", + "7c276fdf802f10307b08c02b3597e7ddca1ff05d81e58426241b978b1295bd4e", + "7ca8ff06f6624f3dd7194560b088908c42e11b69a2b015b252e84e391519fcdf", + "7ccc8fa8248956bd43d77506a4c23697e30fa61f2f5602063e885c278657066a", + "7cd938fabf4f322c80e458c82e4e29ba87807a0d77af5714dbf740c14d4096d2", + "7ceb188274648cdcac071879cdf812555a194a99b60c088d389b7db82149584e", + "7d1cb0401330cbba5c232aab8f71fb185f52d939d5a474c481db009167520432", + "7d26d34a80b40971e91ed54ebe1c94d874fa06f7ca0cc1710112faf2d02410eb", + "7d28016ba3f18a075f25cd72c23c8aa3248df56228fb1a52a723d9a24e726368", + "7d4978222a424efebe70733bbf7ebf56ab35a773b50ccb82a690b07d65576549", + "7d4da588cc4a42eb2dbf5920937d1af624734e7b94c9e9fd35e000de9b19958f", + "7dce8a0adb7f8202cfa2bb850fd97ff64521cadc0ca4a4d564ef4b6abe260b69", + "7e14c223156ea6e79def806060fd0c9888732cc9959313d42a56e6a384e1828d", + "7e23d0dd4cd6bd622866dec150a1d38b77187d2895e0dda5229305902ab23be7", + "7e7c44b1e6c90932b9d8b3496d700cea042dc79e088873bd16584bfb0065d21c", + "7e9b8e888284ef44482249468bcf1d3c05e248d8f83365adfa032d7155b7c39c", + "7eaf4730cb6b13e2ac7db67cf61b6a29f5715cbd294685f743fe5b1545a2e173", + "7ec7ac968403f21ebc8bac18aeae134a206b12c7524c4057e2b47a42fecef24c", + "7efe2a5615bc2ac81c2a8c56a439dafbb7e04bc721a6857129b0dbb7156a49a8", + "7f02c41794b097eede7d73b9d380ee74f7df101cb0fcb2c5ed18268fa95dcf87", + "7f4eeac34b3358cd586794fe9fb121217a86464deb4b1493bfd413c57ae77af2", + "7f6c61ff00f23fd6ae6b96d61a303f6e44363ff4d1679b13b9cef244e814cc84", + "7fd7e3e665311b4248ea1af06a835d00dae5a427922abe1f898fe66a958e5382", + "7fdaf4bdab2643b67df88ccbdd1cf0d5878361070c6374a12939e7cf93bc733b", + "80313a3c133d5918239dbaf1248d1b2abb392f39b294c81ea689af6ee0707fd2", + "8031bebb0d9126c9be5daa6beeca271b8f7879957fe35e0e47c17060eb9b7811", + "8074859bcf62c319d896a8f29d98216f8991ab9bdf49f879ab36d33dd155d9d2", + "80ad51c06c23fceb63beb5462345dcb9a2d1b67054015609ca1189c2308e8f8a", + "80ae9c3044117ffd27c3f375f0e29c9e75d2a3edba632de71adfb25ce66b0771", + "80e4cae673b060511ceade38c0a68359b9e878300c9d5d0259b6552deaa5a1d6", + "811b18091fbf1a6e5d1452fbbdbc2e9ecab73191974e0214e38bf70cbd0010fc", + "8124f366da84fb287245aca463572ef89c274a6360cc0e9bdee7d477ef041215", + "81518eb16b5d44c22df9334eb65fac0dcc85031b0c946172485819306c58aa8e", + "818dbba72cafd429131e635b3ef6f3dcd5f3d40c359d6f615235f11eb769bd27", + "820c4004f3ca9ed9556f3e68fdcdd8ab6e394dba0f50e532d682344629184ee0", + "827972c6d14855d08db1c34677e97fd79a18c0bd9fb2b9e9e45f6a0a05a2952c", + "82d226725b39f5e86eb5bbfdca67c52fa239ee867160c45a87c90194f9c3e509", + "82f4be5f4a60f1413f24d360f20b179fded451bb7f109f1f6b7a15a7b5393f80", + "83134fa9b0a26a50c2fcdf580f8ab7160a8404bdfb887cf727d7588c080b4ce7", + "835422aa07b758d79e0f7f35f6e5db7bf413b872aa8a71f2abaa82a96bc8c799", + "836ad77eb86cc6612922f0fbd4a848187aa5c64219811459d4bbebfe01ecc345", + "837b550ef7d135d2c0dcd5984d065228dd5442d3351d9cca94321162d8b14af1", + "838f6d15062b30326ad7199db9d071f7507bccf1fceb33f3e8a204d285382faa", + "83d185f09165dc5d972254bea55079367c28a0fb300ebbf3fe92bd74fe2d93c4", + "844a4ccb1843663785075b9e9f858bfd930d305b1a5c84753cf9c0a2a0295da7", + "845d12f4cfa04b863ab3301a97d26363b521787bc09ed660cb283520b35e9b9e", + "848898dbe0c01cc84e63e5f2095e35255b43bc886f0e27eaa81e8b8260a297ed", + "84a94da8a7ac516f98b806874e3ffaf05d4dbc506126b3dbfe613fdead407985", + "84bd9538257b7814607343164e7ed9b5e4342b8b4f7cd3545f72fc1ddd06896a", + "84cf3d3a21806b2e0a25f2137a8df5a2838f1b0b747657386e8e4018a9f7009b", + "84e808828a69df041c29bac0e30230b711a94e906ac9820f06655213f786b6e7", + "84f7e204b791c99dd098aadcde86e776a5ceabcfd3061aa4c654a7635de22f9c", + "856769850310c7f661ab3a987a49aac3f19b580db8f96ab79daafa718e247f79", + "8570530e03ba5bd517b643180a182c36fe9c233ee4f5cdcfda9adf6544d97e5b", + "8574f58478a831ff1c012fef160d321b0e4a0c4e0aa1bfd4f02ce1a1fe3c9b43", + "85a6f82c5d501f4f85312ee5c78761f3711c14fe917f5141fea49f97adb7aa27", + "85beba3967198e7e0def9d27d36611e722d7660db0c44da4dc75932d501c438f", + "8631f5eb9a74ac31db1a05ed767754d8ec31f089a80f38116a71014bda213882", + "863bb5e180baa89cd2770b88f31edf3988b20dbbac613a409311b9465e593c5c", + "864a2260f794a96c454c1e311409f8b7eb71278f8cef9f6f1b610de80c6f347b", + "865f0cf27ae5e60a2195c621c0ac35b3bcc9324ec41b8986fbc714ba480f9ea5", + "8699aad8ce4823f923740806ea40269c4ef238e140c811865958e98d665cb08f", + "86d85370fe7dd74a8023657798682dda13199fab06e13bd306d4a19dd8193a0d", + "871cb4c9c8f611b49d7743fd17eb92d14640bbdee058490611bca68dda174f3d", + "872c8acb3d12e47c416d8c271212920cc4654d7339111e4d89065ae2f6b15285", + "873f4d11489712ec090a4451aa926f41fd3800f396ab4613c7055f9710a41565", + "874f46d9052509cd07edba8888b99776024653a9dba3aa2ac48a63f232f7eff2", + "87afdca89f997e5f096f51341578d2ab8b8c5f1f091e2011d0df87b28570cbd2", + "87d36a47ee715c077744344ee6e94618940f64b34d8f232fa15248e96ec679e6", + "87dc2cc17041f5f8857c27415fac432301a74c3f7cc6e3ab1e6b10a0f124ad34", + "87fe37b344acf82bdc4a7d88fea87ed6570374d3511bdab793b90e0d2e1f2a0d", + "883ace1beaaea3781f22637789a1a7fa17bfb1ce7542e941d81bb468a7d4c6e5", + "8846f36ac79147eb10014d75f6fd6585d01f3490d72496a0bdd74fc5b1e7f181", + "884db55185577f00049d89caa0ffd4f57d3fec4fde57b2baec680c6e4ff4836e", + "885febe6854c1e5acb517005f8a6da8f35ed7099c45193832ca4fa5e3ce91848", + "8882fa13167d65377a7ba90a7a4dcedcfac1e80b54c84a5b946296074674ba31", + "88bc8e9c23670d5bd20a398979d863b137b267aa1fd7d8c84678d3f72a163be4", + "88cc71bd445254f00b73120853a0b61cfd04c7a6f912d657198109ef8deb7020", + "88da3f334bdf2598c22e1c4cfc7e4f27813fab04b71a1c94fbd394963104f5ac", + "88ef35c4215e6661b6b13b887413f6cfb5fe007fbe0eb7c31ab8f8667d49f939", + "88f24ed049e9cfdc5e6e52664281db32ee6d605f8880ff9795f339c8b8cb6ca7", + "88fc7fb8d44b1eb7909f8ee97234e419796e3b2262957bf06855040d53b30e20", + "8906b268b99b4ac3c7eade7be1ff2265be2c7fd00c92ab2c1ef2a3a4fd4d2cdc", + "8909455ec2ed4207fb2252286af435945ef7f4441b14dcdb2233b2c7e473c62c", + "892da162016430553dc0ebf569bcde69a9d968cfb7fd52167ed6e243a99af167", + "8981887e94552109b3cd72aff3dfa3c9d76eaaaabebf0254c890e8a58ddcab8c", + "89a3a2ac4e3d523b93cf14975ce15e5fd39483055aea8eb9754967463ef4bb22", + "89d52a436b5a0a0f5caf3ec0dae813d38b2ec6f90f9b6ad984155b2ef628451a", + "89fffa58748f397976e7995342db915a9ef79590640577796c7d99ee674e0eaf", + "8a12f9756e56fecb2d2654523cc2b63ea0219362fcd3ab082bb95812968ec31b", + "8a258b19bfa17e45608be2a3076e02f86b2bebced13facefc1f28a5c503e3958", + "8a38325262b8f4fab2f9586fcb48caa450546bbf8881b2512fc7ae56f0463327", + "8aa00ccd9ec106af3f8d6269121ff1dbd381673b740a26106da23ba717c5267c", + "8b09e0958549cdad4bbcb5072d88bb28b2d16887b3580879796753f2ca73dbfa", + "8b1b7db85dfb029764987b5d104bb3bd23bce31c4c15285536218025bcde4bc3", + "8b2e208352c7bf5089566c2285fb25c19a7e4c68d62a1a45ba3eb0642f5a44a5", + "8b3e1ea8341e7a38998e66b52af92276c2ab7d089fe710bbebe40c37fdf9c210", + "8b3e415c88f3f697e9264f31cec88cb364776b3b2174b6ec9d2c9ae8ab41bb2a", + "8b455bed9daf8f050877d9ad9a1d0008e1c5b5b47af5f2608ff6d63d7f2778f4", + "8b4dcf783f047800dfce66ac900496e13d11389aa43f11d0ba0aa17a53d35900", + "8b4e9df3726ccc0a874a7657bea1c03e50a1e63855a8fa4aec0ca050d16c5cf8", + "8b91b7882239b55dccd3f2a3b699b870aa6e3cf9820fb32f7f64fe0975716466", + "8bc15315ec593223c023b73d8e47bc4fa17ec93d3eb420598028076918e9e59a", + "8bd6765985b355a8426754aae723dea54f2e1ac544ff0a1a3442941e6febaac8", + "8be8a2bdaa02f79e7cdb4c7c3906826fa6bf90a20bcf9285cad4b3b105aeb2ba", + "8c02e41c60007b69416eb7689404b46186087ccfe0f5989f20784c2c196b7b1b", + "8c8d4af696736585ce185cc8af41d0b6109a2ad22d4bfa782ea7ef9a3834e55f", + "8c8e11c3bd2c532170df6ca8a165d44f1e707806299fe6ffcab13a6f2e122521", + "8cb3a08d1ccf19c8725c6a98ed7b3386e5799fab11ace8108c2883579f682714", + "8d177f26ef02fd353303673b91e46d8390241c1c279dbdb22c991b3c4e9f51e7", + "8d28413eeca9c997dac5d6ec5ab6ab12517e44b8c4a51ca3ba7c859491df3952", + "8d9ef2d42ea02122ee5863b202c05351599d62885d2b98f96bb737e4ee6ebc33", + "8dbb7ce9aa0f47e558051a724cd235cdeda295cef4ae6af454076079bdff514a", + "8dcfd3fb6ca84c1f95efc8f897513707d70ebce78c2518aa8f65e814e71f3545", + "8df113b565bd3cc6cec8965e630711bb168b1cc181f53bce4c6b7ce61ba56aac", + "8e22151cba0ac0a1e46d5ddce156fd63cca4eae19b77238cca06c8c8ce821eb8", + "8e2eb253feab70a0380720d136c671a52ebd609d5c1272ccdc6e3ad461435156", + "8e4f60fbc9c964d0ebbd10f8b9265be69a3a3edc2119ea12426ef43119d4ad5c", + "8e5ceea749d717624820f70c22b33842bca091f4ae50be442a7ec0f09c83303a", + "8e6d9e83a4e8ecd5052dfac7c65bafb828ac22f1788885c36a58ec7aca9d27d9", + "8e9c2d035ed8eff3f545b75e289b7e582c7968100ea73e6fa33b0be1d5bb50b8", + "8ead263dc6ed57e40bc9a47ef5e9f5c3e646b986f2ea12f7e6892ead5de2ab8c", + "8f170427867d4318616b12da898037de0f181a5431d3acded4af696cf1326370", + "8f4222b557e6df356279166322bbd1705e8a56c35891f90c1530cf4bb2eb54a0", + "8f44f3b78ef1575f9b970421c721350dc71b5db2af4dec9abaab2ca7578652e0", + "8ff84b817ee44a102d307bb8083ced2d3cdedf4cf0079e6c459c7b677de3a329", + "901a8a97dbb978e7d127091a6fba78c0400b75a12bbf3d417b28fb0af80b6782", + "905bcd129bbf0751e06533ac956befd3daaaa9ee2e6645cc6fa3ecf71fbb9a89", + "90695b391187c27e2685fb7e84f2652a507fb2a02449a190d3267bf7cb3f5b45", + "906b270dd3e6ce8f729128dd2cb3031beb593dc7429e4dd36b13f401cdf792b5", + "908adfeb33ccf16b1cbda5b092c58b268e734268ff6c60cfc466cc856adf2baa", + "90a9b33fee83fcf0446eb2bd3672676120fa0145c84b4a808785d5cf67fb307d", + "90b17242f7b1bf044dcb0c8b348a6b20c4f43458b8a4263562653701f136c4ef", + "90cde577fd11b1b1735407d95de549f9c0a53404b7f2d12fcd870cb927e9f059", + "90d2284908e605264442186b3da69f670bf446ef3616ef6e730a093e090e2b3a", + "90ef7fa7bf2f7675c592189ef0b9aee190f6b142d6df567bc67852a3fe46d9af", + "91436958d8b110e882c5c4a525733c39f6c910e196f4deb9f42603dd7c4603ad", + "91629fdb7a0c28df18abce3897b3b96b2554f0a3b724004d415f8cd7a3a47e92", + "917dd26021510d66f517c0d401ecb5d7db01a25a1195fc4ecb9a7a363814268b", + "919a549f4845967ac2f847626e2b00056eecf902c91d9a4be8378a41640fbf34", + "91bd8da2cffc4096f89a36c40aa3d00916cdcc73f8a681efa0222941be36543e", + "91ffe0cf3888e203f9d71c38b09f1fa3d34ba073342f26cc49bdd72e92ab0206", + "920c0664db9a83dcfaf66ae2e13aef9ade367d3221902ff79efbec849060b80a", + "922aa008e201c6d77a55d168be1f336b56911fc60a133a2ed900f3cf13d73431", + "92563ce948515f1da8aceedf199f6ed7dd85ecc6de1f26c98b99655deafe03cc", + "92697ea9bfbcfec599c6d210f730325aaf05166191250d1f162d162258961af0", + "926aa9c880645612620c26ddcd7a444ffabb7b7edf31b3bc2d22b50819698712", + "9291a90d8529ecbfa4905aa368ef4026cb7a430e3e6714b9b8f334dbe5cb96d2", + "929700cddcbccce723799f822366c39243dbc4b470d2164354f1168a9b7c97a3", + "92bf7e3f8a5589cf3497c2a9dc4b945e169b87e44a1c01240873eb41bb77ee8d", + "92c5af8113a4129e4e6b5e6a9e16a45471fdde0ec55ab2cf1f64aa1fba547d5d", + "92c8bc3fc53f871322fa08f65bd33d640fdc555892939d08208ae59cbc888a18", + "92e75868a799ff640fc178d95d09d447e721af90c051b2ce8d970f9037c04670", + "931e6ab771d72e7d47b63fc46fccaea8b91fcc80aa97a456bee36974cac43006", + "932f61eac4752e0bd55dae596f5fb82c6a5655beed2cc0f209ff157dbe89e180", + "9363a4d2b67faf2bb77955660010ced2a615c568d3761c57a3bf2ff6404ec4a6", + "94a44eb8c02107750fa3c557560e6bec0aef76e28851d236b2d6cf85565a07f5", + "94d38a120b004f5a6b66229add7e74e72dbd074772ba15d1742d4a647d2713c8", + "951e91b14a12ad616ca8991f1132fa4f763d1858822d0157ebffc429b49cae4c", + "952cdfdd6d836ab796722cb133ce2ee16b14348de138f70a79f1c6af33ad9c38", + "95312bd603102ebca1809ff72dbbf8e073586e5c0a4c68cacc6199d1e1224e6d", + "95406e941d31e39578bd8abbdb76f1cb2b3eb7bcc360261d849b45b641718be8", + "9568476e7af8a3f709e95241c456daeb7c9b80accea85bb5f7f71ad6b4387f33", + "956ff6130d82abbacde17e05a5bf2923b8a4a993b65c424d4f9c95a30427714c", + "9593d227ea9654f8734f7c51145c6d3eee4179603c050ccd5642cb50d2f790d1", + "95a05b8ab460184690fc31a8a13698b286f0e92bdcb33fb5cf6af66a29b3ab47", + "95a5f3efbe93d44838d1f96f3621c477d24e786415f4c81c9879dd403e4b4278", + "95c254bc14997a2e0c244f9213f7a2230d543948e2c41e38937cb69235ce7eb8", + "95c4df01b8d01f9506b8814f2631c677d1604853eb37cc30abd19bd33be912c5", + "95ca30ba70b76786b6fbc3b5aad9ca496be3168819291372dae2eb2b6e70e234", + "965257f692454bf6a792cebadc934b730ecfa2a179d3fe46df2232fb32d57d91", + "96d82f098fbb8db5b98ec2e8f8ec63a3641b4c660c41aaaab61cb09ac4ae15ba", + "96f19408441d972335469bf0b4364ad2cd39f3e435ba1b0e02972576769b7970", + "96fa526e45620e358f25a3d8752067f54c93554cca27cf49eb56c7913c4e8f7b", + "97320818d86e303e534a01a6dcf3774bfb866340af2a33897cdeb7ddd3f8de06", + "9747edba14159f5fb25629223ff12e9bc0d216302826344d592c5105e428919c", + "975bf5d76668602e6ba980cee7f109c9434a3a37c1b8b35d93fb5e8e7a608a18", + "977685b7cffb73f1ef26fa6a4f2b7fb5bd358a4f8f839052a2c89dbbb8a09670", + "97abf8d14bff7ab218a3cfe00a321ac0e109c828bf806129aa3b2c33ee6fae09", + "98002f7653792d2b2c41892baeed0c7871b7856cb26b78caf2195b83d11d135d", + "98302df238b9b63340092368d30b1e5290ffdca36d5a5b3d24a79f63f7dcce21", + "9834e883bee15cbce032514b9ab9335f702664589143d62a1fc71a032af3d901", + "98722a51369c5ca8848ee4144a01909eaaafb0722e5b8c04ce0c87d9aa096dab", + "9875cd5eccda046323b2e00db35a097674bff9ec0541d5b3eed73f5f8e39caf1", + "987eb90a6d108de4e939e23419ffe5a45a8f0caba5219dc6d14f049b0393ce76", + "987f394e337fb104b903dda46a5301aaff287730007f46ef25afadae5b7dbc78", + "98b3c272f364d556fa622ce5c522e9fc0f060dc4aef276bb44dd1d46d5e9db10", + "98d225867299292f12703bffec008805ea807aee642be225324659d309d6b226", + "98d302541fa8f33ca7ca098bd107aea4ed97518af5859ebb10c8a6ea039bae2f", + "98e07240600a5cea93c946519efac38d57c0b3858ae6a47638bea60c21879471", + "98e654baa488ccf9b9007a817167c6973be3bce500d7f03d1fdd73298cae5e1f", + "99428849c4d6e31bebde27e9099ab532683054ce8c153a9d10a7b65e56f932bf", + "9945f0c167dffd618b6f84eaf7a8709781a93caa6e0ba0a6c2393c54e5b7f3b7", + "9958c8980b2b1b12f3715a6f85c7aca1486f43cc2ce26172440b894f9fcbac94", + "9982a84e51afd9d4e847ca4dec6ad6d8fe88c8849074f780cc82a921f363473c", + "999257b0754b17852fb928aaf87ed897706aaa3588e087cfe4f1eea6960450aa", + "99b3b41376a8c953826ee94c16fdb7a8043cacc75e44148bc4d48cad1c6758b2", + "99c3c79104bcaaffc5c38c1c85b4b96bc953dbaba19994863b50bab0a91c4dc2", + "99e82a068e68ff5b9d9b6de848bb2669cee8923cdf9256a0794faac08625f004", + "99f302b695c73ae896aa6a57bb5c781cd38940e32e98672cd1c6387ca00a73ea", + "9a42dbb9f9aa960515c14ac7992fa86c323fd97df6deb622daf0e340b30e4f93", + "9aabcb855d31dc16f8bed9fc53fd312eaa100f5b252bbc59eacfa0743e7f12c7", + "9ad7db7364cd7d7d8697de9aea722bfec1955477728fef1bd63114bdb4e42224", + "9adcd005b974106a7f456d51d6b603fbd89c076c1d74fb6965712e9c35e073b6", + "9ae8a425fd5b7d3f65a6939b75afa5f607f45380d5fd4a54146672e940a05b1e", + "9b4eb0e364d3349b3d3f10bbe399fd8ce1368111f97d521c7636a349572fc063", + "9b572af176a0ce08ca4766f2fcfae2a2a6f44ce8138c941be9aef566beb694ad", + "9b804167e135aac643a0d5144bc61a7c188d47eb0a6e17f0ccfb5cb08183022e", + "9b8670857d068299982e3e12da1e040cbc4089da59f6652f7b8ae34584c276c0", + "9b9cb54e75f99488e11db80a28db4063de7fb107041291028130bb6637a82dac", + "9b9ddee0c23e3340dad578eaf73967b63092832f46e26829f7d05b493ff48473", + "9c0f001259d0ecae9b91379fc580c00e436b58d9715f51e6e39ddf6459606997", + "9c22c606bc416d61f094cc2dc17efd03777200b2037a27f5d485c97884b231e3", + "9c304d4792c3378127ba41e6a5ce0730ceb9098676fe4bbd4dcc5c55be621260", + "9c4b420bda3ad85c9610c1ddb0930fdae31c0509c31a65d65f48ac32d72eb2a4", + "9c4ed10cefd936aa1c795de9324409d690e05d54899d1a9e1bfc432bc45e9b31", + "9c5428b74d0734c1798659a5c2a60cdd104a631bc5c2cd482108ede09800926f", + "9c58dd0f52e8ade322cd6875f3d98cff28ff26657432258b9248a9b60400f9cb", + "9c6eb67983bac3503d1fa4f3e747f02c7a25d3de09fbd646cfff2d010810f726", + "9c97222a640a39803482f4e08a69e01a0450a3be06fe4fc9641cf0131b0e86c6", + "9ca67f9e97473ad9f858f9fd62d6c2f9c7709049980088db569a4728f7623aca", + "9ce73e107b7cba29d5ad4ac9b3a920127a0d4b2f7c81ad090d2152bd4f1bcd48", + "9d14045957382f9a922548f9dbe983f021f3d102ac0b4d2c2658dc4700d3b952", + "9d1592120d27fe3d51e75420ae1269de3b705afc3dbb5b89cce8de4ccb52b16e", + "9d3c066c345a544216a3e13515026d70b30b2bf65442999ada7ee95942fda828", + "9d80dd48429037b3827ba214b30643adf392eec87a0c3ba9d2167e12557a58a6", + "9d80ee70666e9432e5b6b7c04bf60f050ef6ef8ddc40d02486ab6eadc0cc1191", + "9dbd67c8b94fe422989baf694e2aa381769117e3664a310407b683265fa032ad", + "9df49ef86b0558c59f03245b2744e5ec1ed8f24add5adda97f61534de4fe4c5f", + "9e2772494f3e3a4d63bea115187ea31220252ed837a201930c9015ad51d4c0b7", + "9ec4ac3f75b6b18803e497d20e16b30f6b5a98d905e8afb47e895ed64aead6cc", + "9edd451a1be159f01f6432f0ad82ef62570ee660cdced89909882eb0d78c5a90", + "9eee7d2578410d51fa92385eff3d2357afc8f8e200163f99102ed21acfe7839d", + "9ef26a47cf3da4e812aabe3aa2cd7e1e6b73276f12f82643281967d5dc7bbd45", + "9efbab6a3cec41e6dc9c76eed55e3b427f7b4926803c4373ff88ebac5b4a1d87", + "9efd4b51ad36d81e214f27d70fb6e36ecc4c3566a1b659f78d6259742fcd9bb4", + "9f1009087c7b37ae089117668f728aaa5f3ecb22baa8c604cbb56eda5096b1f0", + "9f438393cc1d673b507b5163f7f99fa21fc9af792a0578aa7c117d8ef2d9614d", + "9f4608d4eab9ae10ac70d874f81e00f0c06ca749edd729478bc739c0a8fc1976", + "9f537e23b1e63ebdaeba8c83d6a910e290c782791d0652b8bed9fadc802ae38b", + "9f720ffc8c64a4886030d06ee66ddba984162e4e90567fdcd1e371052c71ce7d", + "9f83ec16c6dfdc98808e6539b3af01ce77f36771064307578895db61f041d296", + "9fa6a3c0a80ac376f44c6818436d1ba91c105d80f58bd984524b3801c57d4033", + "9ff10d17184c153e9069f5b824aec53bd5f435325952615bf79a6e0a430dd425", + "a03e7717c89a3aaa577413130e453605801d85ca172be688e483096d54d40777", + "a05f659e06fb5ca28fc989c75b2092aa41efc477340b3143c980c624439bd579", + "a070b405d69bb48b537162f88d038381a8b8253fdf90a8f080ec52c83da09f8c", + "a078011b24a347ae2c5be1d725a3d4c6ffe933a58d37fe7c5e5085315a9793ed", + "a079e506e625b7b14bf23dfc1608663d4d16ea2a8e7aa8b9e50662ba862a93f1", + "a0a32a724d874bd58f98b5f7fb50b1d157a6ebc89f6efede46e9142b9fd5d16f", + "a0b6c02e901895db03c5ea136c311331a2982addeabca9117e552b623ea8646a", + "a0bf65f9ebed8f1cd09c82107f1c208997cf8b2a17c9e2b6b588e0d3f1b3f609", + "a0c09e25ef17e1692554bdd8e22ba9a13b8919a359ae77ca15e82a818f9cc228", + "a0df1828ea87abce9638cfc85257ec619ed709cbe2192fc222708c13c256e38e", + "a0ea59161b4c689a3ecaf735f546c10d007594c544fc010f836cb7d01ea5d197", + "a122c64266b92c9dee0fa5ad5d81772d0180f88416ca5ce41e7fddbfa8ca4cd6", + "a123e568191dbccc159dbf0e2610a7dd4d4cd1d901d5fd255d84b36e93171dd3", + "a1279e59cdc47708df9e5014384f1f7824738edf0986e296526bc7763f6dface", + "a12b0d2bd7e72eb43a9219cbb0b6c5c593d94ddbcd01820f06a7a12d7a30c575", + "a14884b94402c0efadd3fc5e9dff09bfed8621c78ac78a3e50340e6e06b2a36f", + "a1854a9e9f442c7bd612070ce1950c4ed7094fee3b31beeac51ea5f545fd0c57", + "a18bd1bd8c4c4b16779ad1695fb61ebaf9d343672af48bee35b47fe1414968a3", + "a1a5267ab45977532a38fbff6777b9544779982f6f4e83c549de66b63a4ba068", + "a1ac8f2d5688e6cd4c46638539fd8e8fc274b8077ef84fd47e3cb3e148864cfe", + "a1b96a861d0b7be41db93fdf76b9ca4b90e24e451e2895b9d90cbdec14cfca30", + "a1bad290aaa74df21c1058d796b601398ebb6a0223dc2ab35831eb8d222fb5ae", + "a1dbe74c9cf79fc19bee00cae7b280cbeca083839382f8b8ffd335846c4f1ef1", + "a1e8da8c76d26d280439cb1705749ba35d2ed1cad5321c1556c45d65e5e0e26f", + "a23a151a9824adf8b8e54ea8f026762ceb74574168c081a84ee58f86948adada", + "a251b51e2297b9639a578d470f6813079c90edb65aec82753a9a1ac005dec569", + "a267aff604611134b035bdd7e8d744e6a463af69a6b4f9083a31b23d584e7946", + "a2ab534a12a3231a13d0028ae76a9672c44afad2b9b1e01ee05c49d375ae442c", + "a2b9e6fc6089a192910ffa79508000f4f459f5533cdbf6dcf7a23ceb3a745c6c", + "a2bf76422d88c525965efcecea741bd1f94a6c86c8608a6843ae2601f56aaa5d", + "a2f189b2f7404dd679f481e41b8d61fecf5da0d92409edef587cd6463f037895", + "a341833440fd48aea26699fdc1624a52eb3db99cd103932100d6a55bb6b0ee78", + "a366b08e727752b0261a79be508a76bf912873a09755df37d9d550f16436e359", + "a37b642ebec2bc4f64baece4a7c71adcd4d4cfbf1a2220e14058b3edc38fa6dd", + "a3803b601e55ef8bf2797ab8276df011c7eeef666383810ff7ade10167c04019", + "a3b25f99314074318a875968b4626d99a5b42dcf7b8c402be4c2698bead84d08", + "a3cc1097f0e358e7aaa41eccc5cc0001745cbbac0ca383a6d0185cc111c9d6ed", + "a3ccd3ee0afb191d916a5727318209623918e7cd970493710189c20786956af1", + "a3d40ca71837fabb3666dcd6bb36d375d4b6ff612a79e53c51ea31b9d7ec3dee", + "a3dc406694c180921d703ab6353d32c97a8e7222eea4e4d4c163f6779b381129", + "a40af89757b4e1f7e4c4fadcded4a6cc724a18d86d2c57296c12458f1d762eea", + "a4354136ce6b985a5d3d980778b5fddaa5ba5f445b7ce6270a9c5f85963a1a7e", + "a43cab807744dcf2bbd2d088e8cd43af87096233291eae48711afa8040b8249d", + "a49d7dc6f8218a62df895bbdb45910dc182f25ac2b7344427198f0218d0438e8", + "a4b8b9783603e6964a3fe485be7d515cdd6f9cd92baab45ae4563ce8a445995a", + "a4fca0e1b8fa35e1450019470bbacdef5dc45aac4feed5e002547d9d6f59476b", + "a5351a3819981a736641c3c50a4d6ca3faabe5ae8e32edc28bc1cef141c40d6b", + "a5617e451323f4ddb657cc1ac0c9b2b08ba01d0c9ee6b5bb90adef7d4c333568", + "a5776d9e11b439905583492e705b91a1b42b34abc508d79ce0e0722f6ca6acef", + "a583abb75ff1fa4bb7d52db7b9109127fe822d281aa9003149444f944021bafe", + "a5b422afc53594c54b4c4b16be01e923e9c79f8b593736f49ac225c951cd8c14", + "a5dfd9da32550e164a6c63eb08c1f6a2547987d2b33969ff7b0215f8ee622dc6", + "a5e995f954f1646970f15c779b6fe6be94c0ee52144ff5a2e9d72eff33ffeb2a", + "a621ad9f22c1fec373c9baffcff0c941a8d7f792f0a18586dc5125a627878116", + "a6683640657870fe56c4af8333bb373efe582acd241443c5de17ba3453619bb6", + "a6791df4f079d4614a3a1f216f1d86eaa5f3bdc98bd273f56b69b0592b41dd26", + "a6eddb8ff9858cd674acb7cc225eca94121cf50c2a9de3ef2ac05ee0adf5b0f0", + "a6f6c8f1dd7d6a28fe80567947e1c77305fdfb7190e3f94f79b2cee4b8449d3e", + "a702a62a0ce17078c88d1416a10b81c9aec85b78d2a706cbf77272e2efc1a4f7", + "a708819bb0384b19c9ebcba75702ae96f641ee3cf79fd712077e727912c0457a", + "a724449ef878facb7ee0b008e220ea864963f0e7e39cdd31a43864ef9921025c", + "a74224ae36d9293a32a0505281ad4f1733bbe621873bb0c984f8954d316daab3", + "a743bd88fce22643ef830c23b68506dd375abac6465ad60b4cb8351040e62454", + "a750263eddf945283109f003cd8046cdc153df71899d09b30556ca8fff014fa1", + "a772cf9ca1bf588c36712a36ddad337b29af5ab0401c3264dae1a0bfd0a8bf0f", + "a7d7fdd7df2d66a787622eb9a27212a1dd3f1f8a07c7e77c3be9fc959d771e69", + "a7eb99092abaf7f8b9d0670b5906ddf805a1773f1b4f62f6dc9fb5538132a693", + "a7f0ca54f2700303978ec97d9f365ad1f917ce94f606d2595f0a539985bd6e1b", + "a841679522059f62ce5a950cad182a3ef1ff961be3332d2f38192bf05624b9ab", + "a86b3699691acf7e426f5c0ce4272cfac4d634a3a106902dbf1938e2435dbc2d", + "a89081c801b7308c86904b0f441fea806e7acceed9c420f90078295498919d44", + "a890bfd6614b9ff75e4acf074b26d9f7875caf6d281bbceda0e42ecf78a7455d", + "a89c9af7c6d1d0818e2fdaaf3dd920dd20d4afb730e80b8d2640bdf1fba75ea4", + "a8ce83c637cca3cd88eda23cec113036b92b12ea61ee6bec7f5a00ccb2f946ec", + "a8f018689fafbdf00bab10aff5d77035210177e8a86a94d7ac6f631271851cb3", + "a90d7bdac4e8db9bd85268ea2b0601d32b28138dbb1af43ea9ed8475656fc803", + "a914075c60372981b697d3f25a5d0303feb5707b71cbdb81aa59e20afa38a69c", + "a933b42767b675f23c55a3bd0888051a90a6db5c12e528a1109f45c86d265b9c", + "a93d49bda71d2aadc8ab5ebc10c9136812a71ddd0e0e3a257930a06d9ad120a1", + "a950359abacff930799b98035d2dee1a0c84943487c21065b25c417a480728e3", + "a956b733a72c588a886dd80f07b9a50a7c50c71eb090eedb9813bca88fcd414c", + "a9d932a62fa73f6d6a96d146fe28c041b9452dccd5a6d0131f996ebeab7ab8f0", + "a9e1f90410ff4001727d240e543b1da201dca7540cdbd5720fbabc52d55555c5", + "a9f0c0de38ec436a4af0aeb96a04249d84df15ccc080c0225546d5006fccf5ac", + "a9f7ffb91f44bd27ad011c0721a6697ddf77dcd7e05a2473c2b2ae4b45d6ed67", + "aa49fbe535345917bb469c8f07a4bb19b263320a1b878247504ace3b668429ff", + "aa634009f9ffa363413b6b8490516ec76b3e37545a792226271e5dae9333653b", + "aa66f54b57d2ce29fd90750a393ddad20927e731f383c2bde590741963435ceb", + "aa685e32c88aac051f95e9663d74d92536871bb1570ca2f880e3649d72dcb4df", + "aa6cbf5104804f5cb7e86fd753e4e5369ca2299c248fa5386b2c6441ecf79107", + "aa89b61df7e81d444be2c2c9b7d86a55f27d70841a4009d08bf2f6f8809b7426", + "aa8f9285a8e65fce85ef3357eaa84a104d40ef1658045a2b95c913c664d90598", + "aabd8f2fd6a25e53262d8c8cf76418c49fbe2de5ec7f69b813a9013e64b2a608", + "aac35f875f1806cd3f3547362515bcda9d6151c2231d41c0f8bb62ffaaa40e8f", + "aaee3bf077a76386aa71f808b32df60a5b57cf0ad84413c35999abfdaed06797", + "ab30982785dbf8b2be3b1781cc58d1e85fcd0e061889e46d23b7abe964202dde", + "ab8664776033525115237381c15dfc31c621d66c93c0d28c58952644d0bfca09", + "abb567bcb73b46d0668540edfb8b70a816a17fc2d26e8cce5978631a3c0a4b71", + "ac2020054cd25ce5be47042209efe44a780470610210e47a07bae9b1d75c6d51", + "ac59fde9d43a11dd89fedeb2d49a2ad8e722b2be68b9fd5c5ac9520fb0cadcb0", + "ac6dabee5b5707147eb79e3bb6b790df357dc26f77e37e5b6af43845234e82a1", + "ac7a9078774e79a56683c947338c4740b4d0218ce1296b660c91fd8fae06a0d4", + "ac908e26fed79816dea69ab9577341edd806b565fcd971f8a86fff8e3db4b932", + "aca04b419e208d72cb2769900f22462f9df8b0502bcefec22556bdedef44a110", + "acc4ea38d065d2f3336ed088cc991b07e581d37456d4370fcd40d81f6f36b7e6", + "acc5c7ac1c8bc43afd9fc0d870d8a65432604c53294a77ee31eb6f60637d130c", + "acec02d51cbb9f79b85881f375b122f7fd1bd7483964f86729ad4aa8cbd922ef", + "acf9d606aa42cdce69f9bf465a11ddefbfb50d2b8800efae9644276aea08b2a4", + "ad0bf4a8f205d5339e85a4212cef2b20b7ced6280c2e9f5f45ccb004b6af322a", + "ad511b083a403a2b6d901119d09bc310afbe220f66b4cbc997ce6e1b43923b62", + "ad7bc98c757fe2eef34f27ca38ebf39cc7560a6475ea5fb65b53a6ac62361296", + "adc0bc79f7cdebfda01aefe2fa44fb0406fca002c0afe4f286022b704074625b", + "ae16ab0f91f2f878129865aca27304a11882aaf09f7a1f6c9c886216601fdec6", + "ae21a5986a959789da9807444a242b62794e9551de2b3e33d472a60a891e9181", + "ae39a4ac22a55fcf1a8a78ebdc58fb8d1fe5f7a6b4a51f0ea36e65cea0c9c868", + "ae39ab3cbe928abdccc9f6eec045dd500e343938bad19e3ef945b06d5fa16e60", + "ae454bc89861d4a5c6244640aa99db82c11f4196f6386e5a66739bc9a458e534", + "ae5bfe44bf7768585ce234c04d1d0494bddf10ccd26964ba59f0dd47f0c2b268", + "ae8e2940d3dc04d7a6a8a6aa07aea6b1c7e943d11ade25e6dd5e489dbe0ee8d3", + "ae8fdce92af832a981dea810c831d8c267879922314f42c4e9f6275776c154d7", + "ae936a3462d7e22809bd94ae5931ae70430f00031ae97dd2bd0e7ff78343e728", + "ae9ec3f3a37a86ce3e8f4605da3f65632e534875fb5bd4ade1db25c1beb1e1fa", + "aeff9c08f60704690cebdcb3f6ec1b9095c9e7d75504a384fbc0bbc11c9d5341", + "af1db02874a387fbc5f94838d3a0b80d3a940e21e95730f6b7d4cd6bcb6dc022", + "af40ef6e683b838b9d629021d698846993a024b916555f7af5ceebcc0dcb73d0", + "af5d9abbeb59f0510b4d1f5d5b4bebfafabafbc03e030c467c68fc1026a60db1", + "af6ac547c28b21459f56a8506afb797e43f8186c9c08519b19983ba48a8e48e3", + "af94252639de288cd696f79c779ec18bdd6f087b485d465a9a2d53be647e4bb3", + "af9874ea784ef6a5422de0f475430314d5c5c80f2d627c9c4d5b101f2dbd5c95", + "afb8867def2977d96ab00821322d5f342c123bee85e33c9530001026a22b1fc2", + "afcee626e7e0ddf98780a26e9ce0627de9b45db75e8bf1d50f86844c0c8e9018", + "afd476b9cf674144af007dc146cf82e7dbc8fe486b80888bfc9d82f70588a23b", + "afe06523c00a6e5b945484aebfad43ef059ad8d648cb1bcc680ca26c034a6f80", + "b02acd3d0ec221ad7b8893f16b79fcd8bbd831f74409baaa2184b623a4c63829", + "b03763967523aa5f431d80ce741e8418d8764ba8316afaa6bd804d51f9247553", + "b078f6c1ab0b1f8866eb77d939d2d70f9ec748e9a45cc09fee00a1b9f8cc7898", + "b07c85be90171841365bd23640932d9a41511e389f7ffb9adfcc5ab66d89bf57", + "b0ac14cdfcd1f096709f5e82ca65c67c2e0a27f2ba6d688a954c356501cb7d20", + "b0b04d5848dc77d7e5908eb8c0a6df6887c743bfe1078b13b14571ca3269ccfb", + "b0b999be0229df105d8b877277e75f63a392daf57752c18b3fc14a6c1ff674be", + "b0c43e875e30694fc93cfee195abe7069714ae7655347bc779694bc60694b637", + "b0d3870ec2a569f0f320dd68d4c728fa7c82c1423ad214286e3e5f9de19086cb", + "b0e33fe309f08f0aa5750e1184da66b702ddef2b9ecbbcf2e3fa3d35b465a72e", + "b10d1de22a03af66a12caeede346d3b183929a809d5beda7f13a42e69eb190ec", + "b11f3bcb872439562b3233ed7bc52f2786a0b449a41d6701b74666208f092e5a", + "b12ad0ccf78825d7347c3a7bf4aa5d5ca460dec3ee1584db25f09518fe20baec", + "b156c635011a93a648f076dd68f26a0c9aee6e5e674ee4a207653d65f451338c", + "b1628145ee10ce0474906869897ccda3c70dcc5334d33223d950aeb2ff9f0f57", + "b18d03055f8f39c8a459af611514a947e6b08e51d17cec2d74f2159cb22c50ec", + "b1f012120595870ca5d56c46ee5882430dd91ac6be210da64182088b15236bfa", + "b2366b6bca134f7a8b7347dac3bb0b100db6d3cf1de9ea0eeb417e1a324b9538", + "b24726902e3146d4ab97bf67178ed77840fe5d9ff6942d105a48c79f6deaa0c2", + "b288350c5709578836c834d4955a2fb3faacf583df46a9edfc32c44c116a8ba9", + "b29ee483be358080d8f6db6047e42600fd7481adc4b86dbee6facc4199b39ea6", + "b3088bbd2f0cb560ca5edd8269eefde6e52e98807dcd2daaaa04ff531f49a983", + "b30afabdba5dc3dd57cfc03f8c8142972748ebe769127fb481171ec84d5e6782", + "b30f9b407fb8ab04e4108164a282def889f0c6a659a6925051dd82835f739858", + "b35a6351bae968e768fa415340252ca2358af96c3181ec6a2b414656302165f1", + "b3a4be33968e42a4b8851312c7d9c996e078c7ddb25aa484a34e559c65d17fb0", + "b3dd9456f5fd8e73e0a9c7b0ef5bbade876a54e0b1f91868efbefaed8e6b62c0", + "b3fcd7c5df285b09027b8ad2356c12e05736373385da2ba556796de7e43b3094", + "b417c0e7dac8894fe8b58e4a9b2ae13525ef8065d68181dba35457570e7e19b8", + "b44002ae51ac10d1d0851b77a8c5e5109e7369fae9d73b459e7b3aa3f0f6cbdf", + "b45ee5251159a5e01dc599e6e1a30a304859484cbd066d9ecb6d8cc0e714ec27", + "b4782cfa6af28c51d1a53555e241f4f68589f6e89c84b7246261344db3bfae0f", + "b47f6e0b37f9793776fa983dba2dd994a42c32346b4b1f4c9278ffa6ce3529fa", + "b48314748c45c8dff55a2316ed448bd72f20d231778e80368376bbe483a2b1b8", + "b4a51cfb2f14fac383167432544f2b82e7e120014acf948aaadd33ae1ed331ca", + "b4ad08fb9090c4ab98399dbb2e98a5cf0b334bd88580b38250cf13c9d386b4fb", + "b4b38ab0899c091d3ddfc7a88603261cc3446a70c5bc09d3ebf651f3f0d6832c", + "b4ccdc6ff79881b8e5ce94394ba1e8000be7611793c98624c09b7737325617e2", + "b4e174aaa52c030cf6b034355335fdc549f44681908979ff6500ea6dde0cb67a", + "b513c611ee93c3033d681ac34dc5334fcae74e64dcdd1cdd16ca84269a4cdb6d", + "b5326045d5af7216b123b3f45417dccbf6e9cd74ce36508cb1801793f1ec789c", + "b58a15be3759b35178f2a230d09976799f0bfd2bd4b47f730ffc5133c8ab0d37", + "b59a22535f3efb4fbb1a9e3cb7a414a382ee70835a3a34bfe3f7304879a0076c", + "b5b5a06c87e9b6185f039dc51b297bcb413f09f577fd54dabb2522256e0d95cd", + "b5bbbafcf9f6b175e3dff8d0aaca59059cf2299247cd7d825304a26a13e25b13", + "b5c1a87db5d6e07258cff72927b1820ab5e83b10c0029456d43476447dfa473f", + "b5c242e5c5c2dd850f6448fcf141f42aee95261f7cc5ec287991dbc845ee0cec", + "b5c7c8e294414ef68009c0ae31352d550b47a4ef468a29e034a81acff91f6c40", + "b5d318dd57a9c80b3e8c3fb510c8109ad38bee6a3fe0e4a495619be1758227da", + "b5de3dfc01a42bd025e966c95808dde6a80c364b73cbf8f0d42f1f5a4e5eb023", + "b5ebd74e821ac2afbf6df49f7d9d4071750dfa86cfb91dba56c403af731dccd9", + "b5f2d01087a437e4ad8e7ed4a37db82ed01e39b8ce4fc4e5213cecd480371678", + "b66b4005cbfc9b8c8548e0348990af5da72fdb3bfe9fe1c0e01852ae0ee02eb8", + "b684f330e18b73f11336a3ff76bab18dbdbbdcbbb4e9401707118fb8d319ccba", + "b6b886286302a6c5faba0e67d898b1b416706ef980e779930e8673699c7811ce", + "b6d1a49654eb4109c9a1426b33c4b3eee473f744c7dbb6f0fd0bc859d4d07ff7", + "b6d8eddbc4f2611621906fd72f117fb090909ae263c6583654380efda492570d", + "b75ec2550dcf9a890403ef27db99f24849cc1f172d08d68e57e5aa2639b6db3e", + "b7fe798ebcf5157ee4dd11701414ff41e15d63f74ca55895e53db3a3e6b7d68c", + "b82d43dc022095090b121186bf6029625117111d18c836eb0b4d463a2a00d70b", + "b84de002e997f9f9e0df1349dcd3238bd91f30dd163180a17dd7cddaa2df57ed", + "b922630a7c7fd034f00cf9d2f940f32862470390df5e6b97858a32e3b3411290", + "b98ea5c34a50ce0e0dd1cdfd6075f7b1832f0d5d1252b7459feaf0a9ddc56dea", + "b998048a5255c631eb43e8cc0c8b892e29b7b9de408e4d99fb3447862c734e7a", + "ba0e2abe57d4e8976f7b6dee8c399cc76d672974a63968bb22e74fa3f3c07c42", + "ba17117fe128ec4754d6f6d360227770b44ae0d3c0ccd6cd676c15e61a959b6e", + "ba51369f3f4aa59d6bf7f75cb9359705bc8da0df0c6d68c8608f4286d76d333d", + "ba73c3a2d34ebf5fb4faf99ec8b0d20104f805520fcc4242316692e239584e31", + "bb0abbf276e17a783bcb113938c42f3c3c47a263a3bdf13e3efe2881ed0cebd9", + "bb14ff83d06e928142c58f803627b6b6067f7d7010725f231d3bc3991537325a", + "bb679d31a017ae04a18eaebff3f6fb8f429b92f73b9da58c65f8f662af751fda", + "bbb6568d2f337cb7194aa2b2aec0fb6a188c46d06a714fd27f0eb79f524c8315", + "bbd3c5a145a404a237f0d5d6900cc4b3c1767160954322a286491c874a50da5d", + "bbeecbe54c37f58d527baf903067f9dad4708779c45cd500d58c01f62b7f3b32", + "bbfcf175ab1c5bc358e67c71a7e7b68a7ff599eec1846eadc93113b45f15f036", + "bc2bd8f50cf3c34d2cb52e4acb44cbccfcf12768a96e14c39f8a117dc8d1f120", + "bc547c6f717ee21173258f4c941a3a3c4acd7c8578ebf8d35ef8b98dff04ddfd", + "bc5f61fb37b1c01c737ab760249b7f12da334e3adaa471ecf20116794b71c1e4", + "bc64c4c70b7cd5ff2acec0c8eb0b7de2b95c0f95df1107612c6c86aacd750b75", + "bc6eaf5d6e55fd6755fd47c42ba065cfd837ee952d1baba7caebd5b18b5a04b4", + "bcb3fc371ffa17c1c073cc2accd53923efeb08a01d24ed7d806532ed460eeb2f", + "bcb499ca031752491bf542f8ec41e8be53b4ade5d1955f0e25156181d3bdc13d", + "bcd9ad96c094d2b37d619be8f3fafa21e1d7c541e355a5dd8aba1427e2c5da0d", + "bcdd90d4e12d43067f3bed7dc8774068e4f42335771892bdb17e8a69afcee82e", + "bce86da49ca3b3e13bce9c8a69617ebb5e761edbaed7ac9cb914e3ea681d14d6", + "bcf87d42810cf1d06f5b7e1c63f715ac6407cb2fa729488030612a8c5be72cf7", + "bd23a935c7370573992718d3062157b935bdba9e90d5ce38a2f039792ac7706e", + "bd36a8c360935cbe0fbdf5b58237bb8eec29a2620100a7ea3363f1b977e044c9", + "bd4742ece96e7508c7fe3019146b9e886d39ffa7c03cbc486edd29e807e97ac3", + "bda5ef41f329a3daca0dcc206bf1fa8e743c267226364a383065e9b453d7a5a8", + "bdb0721795731acec21268778b65231fab27a672cf0c89d144ce4de017dd3da9", + "be02f86b8ffa1a1090f50d83e4592e652fab03b76eac2de1d3bd1461630612b3", + "be267196ec200de80b7d4ffa4fbee39772cf74936647dd847851289092f42f62", + "be6cc9db2102eeccb24879afe2971e6fe5aaeda2ef1042fbeeeba79a1a1c5635", + "be77bcf22d386e03355561d906a4d01f536230205251150ad805e06b9e4bcced", + "be9678da0d814cb4b23b38dcc097182a377fd1c709439cae4d9d8549ede3e1ef", + "bebe2a929aaac911b350d2d921386029b225bf1e3d3d08da214a9a5dc3bb62a0", + "bed10168e4ddf079981c1b85076f28a3ce061584175feec439e0f737b950865b", + "befdb6fb0fb17e561e51bee3afca24285c8055bdcd5ed793ac12cc49939185e6", + "bf120a9a0f7df85c1ba2da3658c8978748e3c3b1bebaa82fa8d8439ecb4da32e", + "bf12504a200f2bc6c6502eb732bbf3eea946be65c0837729ef477e1ede70ce20", + "bf21233f56d276bfa502ad634f5e8c7f7ea04421db5d894da274b2652d65eec4", + "bf3eb231d9cb0dca6d3f37766e4f5ce99a62ac72526d638dd7f28a8923834afa", + "bf7e9e164281f29ef0eb79a1e142065d86ce32e9c927c9d94c079ffa98174eb3", + "bf90af2b98795bd827adf4c4b2d0b7a583b37f29240a8f5ba4edcf20c5a9f3c1", + "bf97fb94799f55df9377e1a428aea35a9aafafaa53a5b8d65d902542d8c6fd23", + "bfd67afb7e0d59083ae71a16f8540a54890e5fb890350363b0fe33e344e8879e", + "bff20b1fcfb36e27b079aae822ec04b4b25a2a5809a1be3387eb49542ebf8735", + "bff67f7214a8e1eb41e7bd601e76092330f41d8ac24b5824972c1a4d6c050be3", + "c024a64ca09b458d19a05c552070c1342763c6d68337bca48f0097432fc3595f", + "c034c03b47b63c202ec02eed6420878b1c4bb0e2ac260d91df82c54f1ca190de", + "c04ed340bae75e79cdd78942e9335de2837e00b273efa911e62f7e53eab5ef25", + "c05e4185c5bc92b4a260dfc81e9faee47539d118f13474933e5ccfcf0be3f740", + "c06524578ce761107a87557d97f85990be11337eaf5e3ca90fdc4b260baa75ff", + "c06549de3238c0c6e7b2f07f003b2f0ffb4c9c27edff4bfe101a5795e0bc6f5f", + "c072202f9b3b28a346cde65ce651963430c942590739690c3084bca8119015f3", + "c07233637a353018d3f7bea24ae21554a01430c39ee202d62a458c13b199c5af", + "c072353b9e9217c2abaed2c476d9a771b759c59578cceb93d264740aec6a4384", + "c07921530aa4faf4287314001c48ba740df2388ca8ec94dfbfb74a3eb50d58bc", + "c08c7e43682e8e4c87d8be637f9d82d4b82429e4aa372e6a2e30d373e93b4d32", + "c0a1f61328f349ecf9f32d154d912097d236ca5200c509253c56dbb2ae7fa413", + "c0b5635ded2ca1c72e4ae38072bf0bb69a5cbbd13eb59acd42599e3e50aa4066", + "c0c4e1754896994da712d47d8507b7f875e99a4d5f65910d989c73aa800d109a", + "c0d55523dbf8dc16c804f877dc889ddebcd2a5359ee7148c21d78935c2fac7f9", + "c0e43d2d0c4f96d5a4d1d3ab1f1469c31d8efd1c05132a75461cc6e8c826f0b9", + "c0ed6130b0bdd0359e918cb5d3b2f6423a42fb6efc4af3d556d1cf4e984c0cec", + "c0efbdbb716f87ad84e409e078f1919460f3de01f2b33e53ce0ffae28b8941c7", + "c0f652e25c47ebc5fc2e18598b76e83c656643bdb75602a92289fb01fa0066af", + "c102e2f2303f130e73f2b2dd75b6603fdaaba996a1ca4ba043e9c136f7c15420", + "c1253ef0ffbce0d7f76f114b7184a12dc245ef7cdf7eb3ea2d51332101a4ea08", + "c16f94d277ce05d085f72444894a40c6ec0479c0a49398e2da51073effdf2fbb", + "c16f9bcd2c4d8f79169ad9955e892aad835f6ced928d1579087faf42955bdc19", + "c180609592cc3870f3dc650c79360da83f5102115b4df9aadd7023e0b7691554", + "c18677c93d36734e09db302b680fad865fdad625dd42ac98a56afe0bb23c7b79", + "c1b11ab212a92c06204de8eb8e33db59c06fe2cc9350b692f91238af02b77270", + "c20c660029c5bb5c25ab60ffa6cbd63eb50e1ea43dac36926bd32c87e0d191c9", + "c21d65e6dabab9487a553c295e2149e194dd0b17a36b209dda8c6acbc49b691c", + "c21fcd5859d67db5a7fb511fe21e68f37cf6d0ae1a7eb3695e3e46e74d1caf67", + "c2288e2a9b6d9e23c664e5a238815c204fc92379ca4b1de8e369d491015f3fa6", + "c24a3fcab5943b4336cb25e2c4496b62055967aa48d8876577b9c7bd67cf7c80", + "c2a1d0fefbd6321505cb03d69d49cdef50ab1cb792fec9cf4a23a2dd0ff2cb62", + "c2cfee3f872b61bf99bf2bd71381f5c7cd279e249bf804b7b8e2861ccafc7776", + "c31106c0aafcb04e0f97c35994ecb7e89348b7948861d5261adfa6579c5f4f1f", + "c33032b249679e0f5113a3448b152fc582acce16b6c145398ea5da4b13027e68", + "c34b9d242a85f0dd41f34a88dafc8faf1b3ac03488731cd6131f4083819dfbb8", + "c354687dcc12700beaa4c12307ed810a27f3ce079cda07e476c5f80acdf9d8be", + "c3612869eed6d45631154807b57745bca7ce167ebf45787d9bd5bcf8b234fc74", + "c383a9a9768dc0b25b1259b736264f763ef1b1c651cdf24a475bd267c3b951d6", + "c399984d6b0c9d1999e2805b521bfe0a27488982249c821889da7727d760f1ac", + "c3b28e516f3b74a15b151adcf82fd9aed69d90a91aea048fb03ebf3ccccd7de3", + "c3d2ab670f1ccd44a266fff4bde9c07cd7c7911142e4a8abdd2e23a07d822b3f", + "c3da57acf36595c800970f7880c8852e7c04d7daad8a0f27105cefb8068eb5e2", + "c41af0d05d256ace803806241254b43a9ce52c0fa512ac9a1a5027c6a26a6bb2", + "c450137ddf1ac2940dcec86a56e99ff42ee74a420d86eb28cdebbf2a25960012", + "c4536a2b42d40e58686bed1dc5c422da58fe3cc6dc8e7e9e6f9ab1e9120019b2", + "c48a0a0c5647f069fb67e33e9475256d4f7edffebed6d1472825ee72753aba88", + "c4aaaf90719f53cd17a3187436bb383c505c1ecdbe6907c43f5513461b3acae4", + "c4cab42c78c297b8fcc09523dd25ae6a8a8086e5b4263ac26e18f1a333ea1787", + "c4d45104076f6520102d542848213e0889b2d563c9777c13fd4389c7929f0a70", + "c4da74fd5cb8e52860b27008effe3da41e6360d9934a66a52020437cb0c0a153", + "c4f814cc6ed768d0e56a13d19679631906b07418c3e033ea24f143d827a40193", + "c529c725b71da442936bb8ab052712caa8812c1f3176170f460642223274a589", + "c542024eaaaa752bfc798493ee4945c876146979711fb9cc49c9b71d84a508ce", + "c552a359e5e1a4bd1ff4d6a5313e086d4fc9382444d7bbdc6372d25052301ce1", + "c55839974c8b040c7bbf1adf0077afb3d0e699044c74be927217f2c445f7091e", + "c55c2a1121a5d3fe7144f9a737652ea0e0ccac440b34ca5ae6f739322c14971a", + "c5616f4c2c8e787dc1a217841c54667eccaff4d25a6f203811b31b8171a283bf", + "c59ee61269705faaf361a3fa88eea141e108dfe28a427c6e488b13ae88de7165", + "c5a5a6d1f10a0a1d8d7eb22f171e311c031144da9c8e1aaff5225089cec00163", + "c5a6a1c441fb287fc9d2416bddfa134666d87296bc9079fdb40ef3dd88f834af", + "c5b2951e583e3f5d7bab20aa1f620dc8e62071cb0d9fcf36011cee5c79f24f0a", + "c5c0af94a126c1b7d8be21886530273bf5db83f4ecbe624034dd36af5038a16e", + "c5de598be6e1b4db13e787c719b2852ab2988caa4d1909389f5c60a3b4839232", + "c5e4cdc0aa0d84e16a372597c6261b520cad72f635271c0942fcfac0a3aa6849", + "c65f6259756a3f7330763fc5b04a03c6a7bfa4d7adba7b24cb52bf25f8ccf582", + "c667a9f939ab7cbc24f16808a496913e96c956f1a30dcaf190ddebbd2e2c37c3", + "c685c7deda7bee11ae26a52bd9c2e0bc69725db3da8c11ca31de05126b59444c", + "c6a6f1bcce0906960d0db892e8a31402332a77c0a34f30af2d1a1fe4577f1ca5", + "c6e52e8fc9cc1326987043c2e5cd275d9739742e759dc500b48dc1e1afb9990e", + "c6ff80d4c29f48b524d61b8d7637ee883a639940f373acaca91985a8fd2760fb", + "c723c0f98c601d0f562a1069910741b66a215bf266cf88a2bf46d39e54c8337c", + "c73658bc27f27a413c5b379e75960bb121ee0ffb6e20209bb8ba3822d0d7b73c", + "c76e6da2382e79e7b091a92fe16a4b1a4329533e5ccf782dfcac91a092585722", + "c79078deea0cabe9b6cda04bbd16a271bf55231129a32d3923b5d61f0e164493", + "c7b43b191278a9fbe9cca46d7250b30521923e3c969a1c377a266770d2994618", + "c7ce0c2ae2170931ae88ed3aa666e043a9571ac1c725ad769b055a7249bc1fae", + "c7ecfe011cdb48ac32b4a4ffc380ffcd844620329f25b3eb275dec9548fdb13f", + "c850c40819cf6be6039abc709e34a90a41d0299aacd5432bde96a32b187c7939", + "c8d5f6a03cf5578ff3c27f20418a5fd7e2f75936fdf843e7c6dde8119b2202b1", + "c8d631c5dd8e4329af5d97d73eb7d0bd2a0c7f18ce291731c4ae9b5540a300ba", + "c92d8d90bb4f444c193024326c23bbf6e700afa1564e9212e1c39529ed08c2b6", + "c932e5515c72dc46b1695007001ce1032a77d3a8b1b313c2c892ef4b2c9d7d4b", + "c98f48a896e4f421facf2c022912144fffe504e3db694226092561f802f22830", + "c995710e45ca54270ba28255ecf88460be706305607354f0ea067c3843aa43ae", + "c9ab4df781c5b5c2ce6e3521532cdf914185234fe0f0a4d8efe2f882a53ed368", + "c9b3e6759a8e4bd06b705333df6b1e39d0f285fe2f389f5aff84c1d543ba0512", + "c9bc72a44e5f32adc8d7f3408a983a192e4fe97bfa408126b3074c0b759499fd", + "c9cba066b29ed7b13272cd5ed7ba87332d37b1ca861a14523b68537a4bcfcf91", + "ca0dfd82bd822da0e32efd6402e176a98e724020726654e2317af944b1ae8471", + "ca79360ba5ff1285e65649a6d3a83a99010576e08d618eceb90ad103aaf03490", + "ca88c8f7dc5e0b1dc4b8f5000cab018d8a2c7b1c1f160311bafcb5e388a9452f", + "caa3bfb83ed58e30949a863b79a2af63f9e16ee16b7d7c311d14362e907fbe9f", + "cacc91e38cde889f3a3ed7ed8b914d6ffbb1231c0a4ed13e2bcff10fc3464b43", + "cafd41841a6971c11257fb7e6189c7785498e20635721574650fc381c539681e", + "cafd9a430506f2cbc372666b6543a890bc6934870dd4b5ee9040a47032390b94", + "cb0517ee94da12b8eee80c94ff7346510d18d7e285f951a87e78762cc47d9e58", + "cb072729d8819401aae11fca56357df42a34fb91fd5b902cd006d7fc36af2f24", + "cb10006e3b9263921ecadde5362e86bdf3a8146c22db36e458f8f4bcc6e526f4", + "cb1312d8be0cbdb89d8cb7c9e4971edd45995484af46b1efd4fe7f1825e4031a", + "cb15b5de90f1f391f397471faf423fd6e70259c74cc4209a792986e4e9d57bbf", + "cb1da3e4cfac0866b4f42bb6104d1ca463379e9a1b15b2ad7a1669d70f457ca7", + "cb21511ac20b769340b179d9e5b6f1683b3a13fb44093f68ca225be514b20b83", + "cb8904eaf404eca0238f1f2c1532feb5746bd94ed9d7577383ad11a8f4e62d77", + "cbddf4858f884f84126efe1c43b65fb1269d916d0f94ee73cc4b3b5dc2949e7e", + "cbf5706c38795e3efdadf678690e074e3f2512ff9868d8665655811d1053a5dd", + "cc170a33d8290a975c2c965f0893eb728ac611d55b3b06b87be576b85bb57877", + "cc25f4b88d039fa902ba63f48ac72b27a1c82c7a1552040ceeab9dc788052d3d", + "cc30c9c182b14d50bca7900e9af56708d83adfe54179fbcae40b2db471db46e2", + "cc4f1dea9120114b4b3bfabbc5d96c080ce5ec16d50501bd4543a2817e6b834b", + "cc68e4b42102346f64fcfc28a5407d758d87cad2b058b6b500587e60b1d54188", + "cc88dd027865c2036edfe3895b017b69977f1cbf6a90cb1ef8596f5e0d886f78", + "ccacaec81b3af38e6b70ce193c830d1081aa02a06cf99dbd5c13e035afc0a526", + "ccd0ebc51296dfcf4e63cf0560dda61e542e6aa4183958b1595f30a01057778e", + "ccf399adea2bbc343bd217129eb44c3eb6516916108c3b4e8d126179356fb07f", + "cd371cb03771295a33e5ad744fd245c02d34de6b06609f3923a6c95554b66775", + "cd380281327a8079b9eec2f202ec9f51f4008f9ce6d096f523dc9901015bd901", + "cd3a0a64425073cb18a40113b21de741a01166932eac1b1b8bd358dc85832af5", + "cd456c636492018ca447218c02d8fa52ef443b088cc6a57a3aaa83aebd71c370", + "cd4b0b156adb451ce45ef6f93d44353b81d1ae76ee0d8eaacab68832c26ba075", + "cda8fef741223993341e4ed5d553000635a1bbb0c66a2b0a5723542077dcdfc9", + "cdac002519a0b137114e4761cfa75f82e189ce7ed319655f15c37ef04e4896d5", + "cdc0e13996861748790ed8d99cf83a43a8bf8c1d2063dc89ab890325a5eefc15", + "cdcfcbeb98cd99aa567ab0c7bff236b3572cc9a2500507b189f6f28de7425860", + "cdd0ae1676d3f8a072fc5218723f6e2e515a8258b82f51bacdc2a3f78c9b2348", + "cde33a13b4c099644749d2facbf2d4341a9e9ee10354095f774310cdcdefabb1", + "cdf8d5aa0a6cc5b51a2944f3c570de8794e9e945317c3e1a5aecd87cee420299", + "ce005b7688f239258d6e6714763b3a37aabfbc9ec585ac5fb624e228c2a102b3", + "ce0823ac2f688885cf9e1a6a3f45844332ea6a428d685c1cb041feeec764ca32", + "ce7855544d63ef0544126ec17880120aeb4cd2b834c5bec78f33e409a1b117c3", + "ce796c3e11929e06bda4cf091d8108ddbbdb1f16b95cbb86dbbe3950b36dc8b8", + "cef2f0750274e9d6fb19a80350a41255bf66e58e100a4bfb1fff6833308eb661", + "cf0491fdfbf94a1fdabf6bc700e5bae8a8aa4ede01b3174cd79f8c8fdefa8fb6", + "cf1206186d32ff4c5fcde159ed0322a4a16ee0be90a79192f80830cab670ca56", + "cf1421b73484081f4683ac57aba50831a901313c0e4010c12485e5b4d2d54445", + "cf1b6ca2a327b7a0553e51527285f213016fc8fcdb2841895a5350141066bbd7", + "cf596932c32af9f99e3a9d9c4a434c18d49b0698a62ecc760ceda02606a8e77d", + "cf6e1977d9fc2dc361ce318e15d9e0200f80299349b480efbc05bbbd8fc53c85", + "cf82c6faacff4d071925e08e101145a1d140a21a95c246e6a0c221837cbdf464", + "cfb37033c2cc4ff84000d6587e152ab3bc37bf44d06a3b8d8371e3ec36b0b7d1", + "cfb3713401986e4be228d6139d98c1b4b5059d8eaea055cc4817c965d0827347", + "cfc5981766f3757e4f015d9cb9c905266e1e5a068d91cc23c9fb54d0f4626177", + "cfccd1f89b646ed289d68e41563e44cefc6d903526af3b51e003dd0e6ea2eb4b", + "cfe2005d6603a710aad7afa82c13c9b27c382f62cd602d371e2505dde16fa1aa", + "cffb93ee92c901b673855d2fc800de2344a16785b8e750abffa64b12b4643e81", + "cffd8cb1754770684de7bdb190219a8c81d077ab861f6f1a2838d5b8caa24af1", + "d016b007e6a831837ffe35c8ab482f5c1220e3a96fdb81c493576dc94498bb0c", + "d03d5a0fd96b5ac4ed689dd49336670aacb7b3d0f727347766b71119d421b34d", + "d04f3c5d101b4e1c5c63a1f09bbe3af3cb7a1e3acbef5b6ad46ba8af4059b743", + "d07053f7c2fe9a0bc31c4bf4223236226952198b195504b958c6f67bc3e24a0b", + "d07e1e4b8e401e7a91201526e70e8f8b9c894db55cd961c81fe855d3053bc5b8", + "d09c34f46a32f80b3780e18e54292a344cb74eed88945a3da07a69d7d3dca009", + "d0b47688297a825cc4a6d72cfb45e24d17a5da9d8f3d43cba2af8f13dbf3621c", + "d0dd6364b3886aa40068a8537aa6719ecb9d3a7e185ca5338b3bd4eae43aac31", + "d0f14e441af05c93061d2b8b45ec72b5191e1c998991ab103ee48902537dafef", + "d1667d0fdb73ec24000d778bb95c383e345dea0315c43ae462e510731b674b18", + "d1695227ba09c29e9e60c5d5305e2b1812134ef5bb04656b7d54846e970c51b4", + "d1831fdbd3721ac4d291560f54ff189aaf13adb759d344b8360495a5436f49f6", + "d1ca52ea943499cb88d83ed1a1d4696727e30fcf9e1c1a888f1f8661de1e5b64", + "d1e292907c03ea6dc5f98dea5f5e8d8d514fbf4cdafcd28ab4a8755c637cfe34", + "d21907d45a7fe36bb446dea4c790889f472b6de896800ed0d68c0d73ec23edff", + "d22d1baeb2fd07a953539f8ffe95fa404fe31cba8058f2234ec48007e0ac9cb9", + "d22f899fae63fc221efb42c1b8c03c110d1d9b365c3f17267261e5c445454f4e", + "d2309680eac9074c3d554b75c8477b7466b29c0ca6140b4bb61dcca09c877a6b", + "d254fb11bcd49612eb4d2189ed13085213ca8f113c9dab52eb93979adb7dcffd", + "d2771a1ea6f906469c07a1e8fef04e6e643d707d7125a38478f997778c1cc90d", + "d28acf5fbcf2510e976669e6ab05fb86db09a1dfd1eaed44fb96c5363dfe8733", + "d2addf95aa41f5e13a6ad590b2713f0ea7670624ceb3df45f5e6f0847fe73863", + "d2d9d70590b8374c968d8d982113565843967802241d93306f007f9b79c1a99b", + "d302274ec5e8a3e1aeaec35db97550937668e42bdbf98e53948dd4b834171dea", + "d3138688b8d2099d4601de9ffed504cedb79976e47aad6522e2d4ff6f13d5994", + "d3169979e682be12dd5305e08dbc513429285768617d1b7a98efa20286501d58", + "d31b09e18c9a5c15705119822e09ed574aa647bc38e21d57355968a65fbc8a05", + "d322f0d4bf8d5e22bc3835516eac087d05a568838effc1b703114422b44d5cb4", + "d34180ef4ca91e938c1245aa42202fd82b50fb53a8a2c42e4fc8eb478c24b02f", + "d3524da45fa4191acb576840d36223f45e1bd7e25e406b6150005897291bf4e0", + "d365b706c8193737aa79d0ce71d6d7489ea46780b17df89ad02a66c13d9c1eb8", + "d371916d7c47430779a55070128d0311441547566cd9f18afdfea10e69191ba9", + "d37493a4cf9bae160adeda969501f311b2c616a56327b492b027721b48801afc", + "d382ea48424cfc817aaff993d824152802559459d71465f25a13e90436043cf8", + "d388e45a663cbf86578fe0cb8d19c0166b4b11e0d1a3b48103390561e7c0acad", + "d3d1dedb04ba701168bcf97bfff1d5589fafcda8ea1153f361a0bc94e3689996", + "d3e4dfff94666f5375e8da4d0a1c4167909eecfdef85ee5f7f85900e08797095", + "d3fd3eb99a4b1e0cd42b805bee67fa353a11bd39c51935e07233d751e01800fd", + "d4177cb874b7a7307336b8774dbec5db48ba5bb281861f49f45ba104c609c56d", + "d42724b6e4f8209ea7139fb53c761b47e923fc360ab6c2c93727d2880c4d2ec4", + "d4445cb8041006a06f30940f452fb25ab5718c99054d263aec03e37ffa554a0b", + "d45f77a1c72f28f05ad7a0d26f74ab8e263d381cfc8e330920e7ff92de4e7e78", + "d461cf05c001b85297342bec480204f28d5573fb3c2c7a0f7588bb8e9aad4dee", + "d487e8e2502e45a634f6d83a3d784e66e203476dbdac51742edae02202a73ccf", + "d4911ee2e307625b1af25cd1ce9e9e42e8f1013a53e9c0e6c5ed1aa6c762879f", + "d498da45a43990513ae88b937d74bfa2a2eaa5d6c20d4fbd8aebd2a4a1a77feb", + "d4cb4c626dc946ed93e1e15283bc68a6b313b2449ec60051b1d25589277b8bc7", + "d4ec74e5fac26adc6ac7afb3b1fd3ed6ab7a2288d882660a95d0d949844e3eac", + "d51c08ca2193e5449fe9d63168b50130e9700ef88c377c6b79f8eff85ce9bc18", + "d530bb300a81c9bcf846ef44ba7d1bf309be17f5647b366e0b321990d6566e8d", + "d532cd7d6f2171be22137ed4b733dd0c79efcd8f560663a4e6530051b69dca1f", + "d54c515b8daaac7db5022b06c73f56eb455768823e71a8c7105c70702c9f0def", + "d55a55aef96c54b8234b8c52f62e44e93728af52d16af7818b952416007eee1d", + "d56f0479216784c4f31c98511368936d9ec46f53ef36c30779033bf833716bc7", + "d5866744aa5485039257eec813b3c798ae708920b48c50c55f3ac91a7d8b313d", + "d5963fa825935ade10ebeb3a5ab89f7c29b5a89859b6e4d9a05b99f3fe76a6e6", + "d5c42ad099d946dd538d27751c569e77104260c88208ebb17303e037b33c8de2", + "d5c64604cb597fa9233354ad4c6184f50e075e3f608fc4e066e39ce41485133f", + "d5cfbba01fec87614efa807630f78b2d40aa00c3f54ef03d5b3d8313eb964564", + "d5e76dc3ce2665b4abb00a96203625c578cfece64c99dff710b28b6ffa184fe4", + "d5fceb5d50d5e30dee026f099317ac53d4a1641564148f8994f97a7fd3794a1e", + "d604760d8544d6548d357c8fb7f7b56b51452ba1d9d21e3374f8135977800802", + "d61e5c37f01be9e876f3202371ec9eda45c423b52980e16b0f15058ec2bf9e8b", + "d6629213250691aec35c30304183005f2177b5b7930ae4a0264ac795944d6779", + "d66418fe345340cba51a03ec40d1dbfa622275dc1fb4fb4839071b8bad8531ac", + "d6937eadac2b6c3d0375e46e5492df69273ab4945242e12f5628df05457158a0", + "d6db4c02d1721ac2784f2c309b97d099d69a2c73b80f2b53af0e478f6ead5828", + "d6deced34e316b96c5ad0ee3041f14f090a8115b76ff4765e1ef0b1d1c5936c1", + "d70eee734906d8157ca2a4adc428a0279e887ea9df38850873a991d40b363b08", + "d74097f6558874f34dc121b870a042ae2798ef27e12dc8a8ce49fa834817d81e", + "d75e0996adcd6c44d978b2b56b517d28d52b277a00ea557b949d71eb6731c1f9", + "d7812ecddd19c0888a7ddc1b19504b9d13ce3b9f08594e8af96f539716bb12c8", + "d786281abd5086e355ca25841d15069947ee83ff1febee9f91a3a6b985e3529d", + "d7aa736b01c8f04fdf206f169aeaa16b5f11775467f659105030ea3a635f66e3", + "d7c5ef08eb5c2c55507d0554d06da42f076ba5dd8c4cd7ebe95eb9cdd72ecff7", + "d7c808f88a1ee2dd7e0508779b7296f465c97a790a1525c38f27d2956730c8f5", + "d7ceec1687b45dda147c887a9a1747e8b29eed1ab2736335e4f941c40bc36f50", + "d7f0bfacc9d7918ef2c50b51f2760b9399f1a087a00a26d12551f63f6c4f3bcf", + "d8041796f4cbba48d6f3e468f182eea71c719e1b4515082b170adb67fa5640c2", + "d827cdcb1f4a28085009d79166e0766be4ceadbec5654943377567edd6d1a9ab", + "d88522c4f93e5c1e6670d82ea79ca300ffbc0f894ffc9430224000ef782f8fa1", + "d8a1213d2b8a725dad7dd1f6c94e46c5402c708b811d1964ff09ed5640b8e29f", + "d8b9c5e74d8d3e00148b98b8ba7b62cf980c8d0d59a576bc372d6c95ffa25b5a", + "d922ae2fa9fce5b9a8bca376cb078bab270b7d38baf9d2f30e3f9f5740432dc0", + "d98c7ec8189d9bdbd3096d624214e8490e275e1e14130031e146652aeb1de3af", + "d9a8b49b226594f5688d633f92b6dadbdfe670ae73c2ed8b8c6f8f11a74a50b8", + "d9cf4ab91413fcca5d5ebc76de93248aa95deb050b70ba1ef5329cfd9813c918", + "da06ca9f1bd4c1116e446737bc45bc24139610e4d86cc12c49912575f0ad3702", + "da0b0dc816f0a9044f295db801b007b657e1bc9119c0d2973d864f992479dbf0", + "da1c6e9277c0a19b8158e84dd9580a3164df4c2766a9866dbf11776c1196b07b", + "da2aa0974c90a4b92ac0847661095b9ac66d42c72563d0764baa12494a398183", + "da372dcf7884fd6dd5878f012626f38cd2c0a0c00ede4fe1e9b5d1c627f025d7", + "da40f051387657cb3841435ef1eb8de999d0b481a498c56ce69e405499948e52", + "da459a3bd11b2eb7dc03c679865c777fa390bd595961a61937a5994056aaddcf", + "da46003d68d66486d0dfad911aaa7dc805d34d411b78d471ada75cfc3ac403aa", + "da78a8e6bed8c5080da3ee344b34a450c3dc3c8b397ae24a8f6f1073e4000786", + "da97b31bae61c6b47bcbd6df3e428cb672e7ce25e615dba3d3e786113d88b8b5", + "daa90216b4b743a3cf2c9ad5b266a3c73c54207705060a7a9012f0e30317e131", + "daf3c47d33f1186ec51d15b5c6543abf982cc1557a3a6e1744e911a07e164b0d", + "db0e91755ff469e050b088764aff3c0ebbc90339907aa7bb9a5bcfce8c41b185", + "db1ac06f5945913376b6e0738efbb5dece43cad713af671cb01ead48ff4320e1", + "db2b2e354461be6b9fd1d9f65f83f608c82f37dc34b4123fc46a7170b0aa320e", + "db4d2acf1804f056caa6338a0c1121ebe7e872eb623820d2faf28f33033e589b", + "db8af8c419ca5c9e68fefafcc963f225091d9a50843113db6693d9e4bb4e0d2f", + "db9e83e985cacab6d1fdbca4aeae0719315ed7812438c1614b407f86dee3e6c1", + "dba3f95e581a67af2cc56bcd034a08321d9429a6a4a6558e0dc45e8d6e49c4c5", + "dbf89cef1ba7324ab6b407e149ac280eecac0c8b5820bc3d7448391ca59aaf70", + "dc06ce6ab156ba6a1b0200c9b2b3da835da67acd9a2a0d949d2d99d4b4bacf22", + "dc5f6880b7ab6efc2987eb81ea52bd648018d5f2dfaa1793f37640af324f35fe", + "dc9f64fe826159e80dc4a3b3e431c2ebfb89813de20dd3728fef9075dddb7180", + "dcbf7df88c53f2241c6d065d9706914e797856c427a9a7748f1a63714e035b57", + "dcc17cdd0fe848571eb98a651b77b0fca689d676576e396cef0e1e6213309cc6", + "dcd9c20f60c91d021109231b65061a5110cef9adfabc1381bf85aa06f3e4cb7b", + "dce77d78d4180ba19dd613af265e892ce7f35409a0e0d06331d101ab388d5d67", + "dcf195c7d41770f7c6c6a8f9869603f5d971e42f6322417ea4f804c457e858c5", + "dd428ea6b43db62554dc7f571fc9c9f4d735af775c0220921bb872acf95b9a21", + "dd5fb7c5f41c05f16a82beafa5bbf35b793c75a6cbf7c55f9bb6e1d78d5f13c4", + "dd61cece503090b07d75c50d1586fe3b5e7d7244216f79adb88ae8f631c0b5c0", + "dd814e024eef95fbbd404fe77becdd1b85b1cd84a1e6669e45198a45a7581e73", + "ddd1bb658f9157c87bd8b8e37c73e56b5fd70d5776d39e89af71a6b5fa75a5f9", + "dde0cb6bb333a5d275d2a42260faffe1f3c35a21cd71f9e6ce2dc65c5c364dd3", + "de2b5031bb88c517a74c9c7ce89d5663604cad24204589e5f04990665617bb19", + "de3c9542ce9158d9b846bca4c0e919fdaaeedcf781e0f3a532fbbdb0ef4c5225", + "de3d5a79d11855a00ba7a1fcf7b77a1f53fc6e6e64832cd9eafa469d0488c4c0", + "de4e63972c76033624e557a1ebe022edb003eae1549e6c00e897c55e547c34de", + "de5e4e21a02c52d22cbf382a6a968e4a68f0be6d26a80e997135747fbbac0e3b", + "dea0a83b2388d405870ab8d20e6701d12f7770a0ef891d35e8a6c60a24f549db", + "deb7bdd29c4ebab19db23e28bdd715a7e08efc63f1bd6c99937993271d26bec2", + "deea6967477e2d07a943fa1612a56e40712723c4cac464676ba95d03745c5712", + "df22cfdbe476a57a3a5b02c2ce8942fbbdfa2bef3af34e3ef92dd087d7848fb9", + "df32dadc5ea31fbf05bfc17765453c4b752ce748ea5c11f3eaf461cb4c4cf86e", + "df36ee603a248f177b0f7faf15812cc77475edc5e0eb4b3c2b90630d54211fe8", + "df3fdb2f1e2b9712dba895c5dba6d01ecfc53b1dfdd4a63a1db80df373135f3b", + "df762849fc9890946bcc0b5797d4d924e430849dd3c2859dff2c4ba7f009172d", + "df79c31a3880ff4606e683dea00a9f2d9c667074a3e15c7cbf315e51a8d47919", + "df9e0382e541f41c49bbf53abb33be62a5d85dd0901b3364dda0956337788d3a", + "e01fb81986221bb00578cc81023e8ae9d432c72ff0177fdde4057d96bcceb463", + "e04134fc94dbef26800f264b5643d7502849a1f4b1fc6af5ddc23a1a8f7af260", + "e04cd88d4542f2cfd01ba7a165a772991dd8df58dc9876b6a6633cbf5204dd32", + "e053ce301796e47ad5b2eed409cbe269442cbb8169c7c3135f9905161c621b07", + "e0756dca1abef5b526f8ae94b26d938328865bb3d80245bf87f8b4fe291612cf", + "e07f02c55b571705efbd7fd1acc36170409c3da0abbc1324f2f87179e4100de5", + "e0b00d3bedb2f4d54817338493a959e7725e0dadf958f41949ba385709a3b716", + "e0fb21f8466f8b028c0a8ec4ce48a6991ce3f73b80c1a300fabf49ea1f11b5e2", + "e0fd0bab853b2375c1458b581b9c4eb10a892740899b2730f77d5cf452315f50", + "e1116deb7a312c0773f7504fa68643e7642d7e07edc6f76ecf0e7f2317e4eaae", + "e116e702614e7e8dffd26029d472791b380e98b24796b9b0729c1431cb11073b", + "e13b045d58d8eb0173956697314814b0818b090fe63a02b4d383f050bf947302", + "e149e8d8cc8840f38238c52e2e4e095923ba0c1ba192fbd4115df6c612e69238", + "e15cf52cd356a8e4b4c394beab2276b76fddd40d600f4979bf671b8a8ae8d908", + "e165291d79e8169ad9ece304325eab1ff728bf614fd944a40d06ca1213341f43", + "e17bd7410c31d2059a4638b5d8798f1863143f7a099d1b60e03308a8c0065fda", + "e1e001d88d85d034c5aee96eb55f261c5318176bfb9acfdbeaa8c16c5302c6b9", + "e1e228481bbac4845368d96e9979b2e28a4d9c96046d94d4eb75ee8add763555", + "e1eda798f2f368b8857b491ca6731a72e832ea80fd29033e752239e12597c627", + "e1edaacf2d378c94eb4fcfdf03a1f080e3a7f2f5498718719b203f7e3f5b1abb", + "e1ef250815b1426c96147c4072ab677ef792e1109910427d9cced32834e50a4a", + "e20c328c0bad99a61bd6801575f9d5f9e8e1f31f4d6567cb9ab4ed4cff110771", + "e21d30dd0a17f4e3a48c59368fcc31959cdd1081319e280121522a0cffeb886f", + "e231c038fed93023890dd0e30ba99e3d0bd4f43d22f14c82756c138840a31774", + "e2422f3fc47eed98843f471f11ee45a4d5017d6ba68f5977ab3108eb2cda62a0", + "e249962c602b987e4520b848a2433337a2615e537a00dca6df1ba86146614d09", + "e299be1a347f7ee34b86aa0acd5d1251813ceda172c868b32c74c927067b830c", + "e2a8a71bf005884953ba2c79222dce333654a560e088e9cf38dac85680b1f735", + "e2b5b8cd854bdecc517cbd0a199160bebb1e56907278546f752c1d5a0e93a57c", + "e2b8f68c5ba84769f8c704bf4157ccd3e99b01b121dd3a4e8cf3a6792b8d6d39", + "e2c05ca53adfa0f9b3e5c432b2d6aa14c7825cf879d6befc9b384a02a1128624", + "e2cce80dae5af088e4d5a2f938be265a32d40a5aa29e216adc94c6af85769342", + "e2d6766f2fc6adec1f494b3c97ada10bc6b3dd89fffaa6b9c7774c0140e702e8", + "e305c6624bc5303875d34fbb385598ae3418b4f5036fe9763d6c5ba5b3261fa2", + "e328ec3ce32b204b4b8d6bf0f63bb2937b8070ab8158d792b746a4cf5bef0112", + "e3290eebc099a02b3bda9c30ab81e7a91f11b4635d3d68ba488795fceb7292ee", + "e354482685966bcfaa5ccd17d666dec791abb29017bed1cff35c1f3b48427196", + "e371f2ecf7e74a3b1a21a0d2dbd76ff061ede68511f0e504ca9f3b0562f666ff", + "e38810c68a7a67fdc05d198102f932ec2302ed507f9268ba85bbf84940be2bb5", + "e395caeba922f8b90effac45dc8e59be8078bce39141d8faf98ac3b4a2be832d", + "e3ad990bf0964b8d707c91fa546956dae31678ed77f2bdb827b9cc7659ce9718", + "e3bc95a739f1260d432bccbf2bb6e8e588e957d1128bca21b8175ad944922069", + "e3ca4bb6e4d1ee23cc72e89ad926308195c8cb17ab4fa4e240ccba8b99bde1e0", + "e3e53335a806f53b0df5e5632c0a5ce39d062736641aa05d81be66bece91cdee", + "e406fde4c19b80fa9ab6dd1a9dc4482b579dda8e387196c8218a5e2d36cf767f", + "e45054295a867fd9f601fb78e919ee099ae8bd87be5bd91f9055d9b5c95ba1c7", + "e4833559ec5fce3dee483ba72c851d1ead07e7a693163e5ec4d14afd7d3e4ada", + "e49d798fd16d767825c4ad445843bc037f30c731303cf07626052611103e69dc", + "e4afd66f886e9c1db0d2fc427c08f765d3f5ea8a57a76c1f3f675dcd1693b797", + "e4b88f8bc22ac0755f21e7e57a5daa1242c722dd27e5d88218a71fcd377044ea", + "e4bc88414306bf55b62cb428f40f2eb464a9b27b3d65dbe712f85ead7604bfcb", + "e4cbf1b81022ebb6a3f4ca228bd6e1d533e051087cb3be4dae37a6b335b64b27", + "e4e28808ae8dfd46216a2fb9252d8f9afafa858d11cf775076b653c004cb32db", + "e4e2bb9879bfece542a719a3d80a69b09404c4de19ea69c7c27e2d2f73c6ce92", + "e4f68bc2988442906fb3146a0c313b75bf7d98c78b03fbba3449eba222b4e2ff", + "e4fbaaedb205db953e8d24af46d3937040d565da17dd3c1a6358598ec4065910", + "e5154a7d17607655f0bdfe3160a78ccf8e0ba51ea39acae0376350886f8e1845", + "e55299d0bc052cb970f3e49f110d9d50c45c8c5aac7b7ded84020228bf1ede03", + "e556d85a21c5f358d7d57d3c4126a0476dd687aed40bfc633440a5f9114c0a62", + "e5e8e61a6acf4409b3b546bc34c4c5c3df6f8230d7e4ccfc33a2679da4ba8940", + "e608c7c92c378f3a308d3f279f0eaa97c8fb7e7d8dd97ea445b1c690362a3a38", + "e60b254147e2585fbbf72d47ea2954b9a673d8717c7d3e910bc513246e5089c0", + "e61fac6a1d5695f0b9010ab423fa50e035f5a29cc33ad776963371df7620b0b9", + "e6a517e9dfa6af4da7ac8a6d5ec19af46219981a80f523afa4e93b8a807c69e6", + "e6c72ed82e109f3797ee8861b619ab6088818c184d9ad5e96af3cb2381b7e95e", + "e6d6496eb3909c504ba87b96fb20e29c2a2cb1d6d15222570dacbdb4d6e0b239", + "e6daf10fc8aacdfb63cebf93b94bec2053e8fea782c7e9e02034387db2cab032", + "e6f45d4600f715028e9eefe9ca9ebeb9e34923defa01abcb5ebe3a997d8f077a", + "e758b129c07dbf4795ff3c35ddfb08b94af172a15cd90c21805ddd3269795d7c", + "e75ca95dce785b8977f2b24392174c21ce5b772035b87b24fd8f7df2894e8b04", + "e7669041b406fe2fb9605a2dea02b57ed708712921edaf3049e30c15c0e49870", + "e769fd9bd29296b03b3bfa821ce720a82c7fad4f6c3545defa203fb70f3f5890", + "e778fc043354b52215651eee4314565d3b8a52b3060f76014c43e768a0723ad5", + "e77fa12e38c90a278fb7af1465517449accfa94a16aed341a3c9f7c8a19d2acf", + "e7b9e659a56a75a9cc66abaf2878026f9ec9ea7b6753b744fadc05e8b4fe2e49", + "e807a1dae7d3b43b44646da39c2f93eac68f75c5d408bb4c7df269b2c31bfa6b", + "e809ef944112bcb401a996b7c48e9c60d7c951635b878b53a9a4e8e7198d5771", + "e8210d2b3a53f28d90cd4c0d5d563c4974afb154e58a950de60a6ce1e94b98a0", + "e83f1bc3ef5cdb9898f71848e2455310e173d9e625f6f62622ee9ca1f067d6cb", + "e8a3e432bb9a9a7a777dda70ead4e914e3852169e92cc1213435d2800a4b0d15", + "e8ab701f9c746b2050930d13cd14c1a4da09470e84a4af00182b240e0190f2c2", + "e8ddd4a423f53d9e73e89994910905d4177eba6ba48a22b6e94fb46c2eedbdc4", + "e8e065640856dc967ec7df4f09627d3df869baf4768b8b9f52d5455aa3824b98", + "e8e555f7ad29c9b6cf48afe7e613a9e7326b0ed49d8c4fa98574dce237cc8bd6", + "e8f7b249bfc5fbf13cf71c70f52c4cfe7ba312f9b2eda18ed1b85f95947dcf0b", + "e931fc764175ccbc37d92970ca353c3ef516278cde0fd8302140552eab5eee68", + "e935cfce5873192c80a1489d0905c0ec0a2c93bf66fddb4525e4ba1a261bad2e", + "e94950b99f9e3a0dccef7b2aad372368c1d0aa60ed9d46f44440f91152bfa9d8", + "e9c3fced5b9ee5c1b0564c4b0e97bdf8036c915928d6d71306c6a8153a3dbf52", + "e9f20a5843a9b56c9f60c76fe1a4a20502d3ac52f6a79e6bb0fddfe57c734f4b", + "e9f7e71b5ac971ee7edd8954535c7bb37a9b4423bf2b4c334a4d837a719d6940", + "ea5df06840f024216c7060de9e0e27e831a22648e6cc40f3cb3c39f9ca632790", + "ea677fca7f169e12f3c208c112812333ff2c7a322418d181c9a173155006659c", + "ea73e0364115fda77575b697cebee340ad79f5d70bb0c3e1a84fcf9366041a13", + "ea8c900045b92ce2c1fcfbde002b9cd3fba951e0930ffa5506af460aa540aad3", + "ea9fb8c996a5e0d3355b766f1236bcd7a1fa9071da553f229d72821e270d5241", + "eaa4e3b998e89bd22997baed140e1377accf6b6bae9a9d1e86054d583885d929", + "eada10e544c804df305e002ae188e023f385ec4ec1aad6532c21a721981d28c3", + "eadf398b6f49fcd70b7893a706e3ca3cec3931e2f79f1bf0478725d835df8094", + "eb2184aec299ddbafd802d22f3c13fd5f78cc0b39aa17d20217480ea1b539150", + "eb21f51a46305dbcfafcae086fbd51bbd1bba56b0c8cea89d67ea33f61e8e540", + "eb49069e0d0803d055f3d2498d02c39396653693a7bad1d32dc458e3118a2218", + "eb715472371cabefba3465b6b77fbff6aa0fc2b4f00c8fe7e55da7cdb1e57c16", + "eb829ab015a5470d658bfc7cf8d8a4f9ed7b0649fc04477878c2c5bb086ac465", + "eb919932d5bcb710486119df62f73e823b2662eb21355d6a3229976093c579e3", + "ebf1af91b52cd8ca4efe45784949e3cc2e8f246d7f4360202ae6f3749a725c3b", + "ebfc8c6d358667cd32e57e33ffbf9aa00db477c85e0e78f25cd2889ffe8a0fb2", + "ec128b250d230602ce125413a544982ece6e917ebe41620a35b3c2da7e64751f", + "ec248ceff6e47b10f0d67f19204dde463c9a1408cc04ba680abb10619a4edd29", + "ec34aae96b40eb7e90848804ed9ceccdb00855bf550a25ea89d1e9387475d256", + "ec3ff85db7bc96ce057455dac92b0d1e8b1a53813902595a4ecfb7eb248527e4", + "eca600fa3e3f7716c80a84047650bb56523b5eef746421cfd879f7de4e393677", + "ecd38846f9a886c1c8ced9a9bccb141c92951cf2c1a0762d75b46a4e27ac1979", + "ecfeccdc4944cd60927341295292880891b0047b9767a3067c38a6a88fc892e4", + "ed3411e5dbd97935027bab83e0de5f1844b0203fdb471007197c8fd12abcbb75", + "eda0be2d9847f3bed6e1f0897845672968add857c9059f415357dbd6a513948c", + "eda11580ce1dcbae0093fc69bea154e1b5e2c429c40a29a07cb6fa005413095f", + "edcf7090faa657a30800ba1effb6f5336ce6c6da7d67f34fdd7bdef9ae1799cb", + "edd3144bce78365312149662b177843b7be7ebc94852e60db48883321cd1c28d", + "edd8157eaf2fabdfa0228ccf1bbdf8da438b5c85aad7e5e2b16415e03225fc9c", + "ede58af4da98df112c7e8d3e3bf12d1dc95cd6a2c2133c5eeb854eeb92549e48", + "ede62eb7c52421c015c5c55624a6f4eeb50adf742ced0cb0e8103793174b4336", + "edea1d001bd33c4521b4976f8c90c447ceb5f39377994d58cbe80b20760855a1", + "edeefba63f3b8bdd4193ebd0f4bde202512ec7186db9680f83ea3503d2bf2221", + "edf05e34b8627e131cc9b0011ee2814775b3ef1f15e7df2a994799b77349e766", + "edf49d0ccc646cd8e65b71c90acbe0577f1e70a44488387d10d5af0ff0ec4ac1", + "edfe570ddea7c44dcf50e5d209f1d0858ac2f6d1f5f4314a47b9291c1e7761eb", + "ee27d30b6be2213a9955755bf53c689e22a1d36d6d937c6e5ce30f033baf1ff9", + "ee2eba2147ebe6e0be37840a004b67948c629fdba8fdfa4717838f8f14fe6655", + "ee58e5f3069e7a722ea63d9f4adc2a70363f44578c99b653e9242f51966d3557", + "ee67b0345ba3a7220dc499ea6cd43247969026767eca25819111f4437406e5f5", + "ee81040064f573d55bc892a32c0a6a9633ecb54f5e92dbf3e5250168a9ff7f8e", + "ee9ebd625b437d3e95188418939dc28c7bf4ba90a218a9ad07adda258499360d", + "eeabdf9f8b0db1bceb0bae0890b5fdfb9326eedbc40dcb85c7f26c6e14f3d37c", + "eed3e97400c4186ba00d90ef1ef61a2add556118dc8721a9d0cf4fd1677821ab", + "eee22c4fc66a21e9250f8e48c48b4b20312e15ad4dc92eda8fd1a8e84ec3b28d", + "eef0e5551b49791fa467c9827c9b6e3bf48fb58d15608a863f3b54531d6e7c2d", + "ef014ca51bfe5dce1a5cadcb691a09e845e1f189e197edcb1b057577e7db2b09", + "ef21ffe1a20987ae754772d981827616c2430fb2ee65fc446dc12b890ac44691", + "ef5bc23227ee57f961995d4db74011fa4e4f67326381286a17660c10f2bdaf2a", + "ef5e8925c07e682028717a42aaf87e36955876bcf1b2cf90c11c9f31047dc205", + "ef650facfe06596e2790017a7de7284ae386555ce70df6ac6792510b0c7782d0", + "ef651f38f75bb4454622f634118195c8fa2f380a02c85e542d0d5e8d40a312d3", + "ef98d4518adc923a900d7f16d0eee156539b83cf1eed9b777c661648296c8ac5", + "efa37ee575b79f5cfe55a72d9df54837d8ad4596af2cd6c9be85fcc147f607ee", + "efc630fb564e71a9a422d6fa7b95119128cb8e883fc59f15dbecfe36e9ecaa44", + "efdc2d00f2661a9477844c248a90bf38774d9ac165b55caa12814e13ffda9f6b", + "efe84d821bfa612112cbb36d005691aaf4429ed7d43863abd6028e2628aed9e8", + "efed2f5dff554dffd1f0016ade25144b7bfc1b05491f22dfd977473d08d90243", + "f002dc29bf45877bccf6944d2b2f85ca3bb5e62003414b4dd8d0da1a8f0239f0", + "f039cb6e4f3b43eb5fbd395eb0922df5deff0be36ebe5339171e5fda1a175bc0", + "f064a324d8450b5bf59a974ff9ca07478be793974aadacac4ba283d522ce1031", + "f0653049808872abb6486f7a0237281fea4cdbf633c2b9b7377b97c036c5b636", + "f0878d7d54e80a2f4cb870d77044819820cec0639b0c3ed55d68060e02aa2534", + "f0c51e190f9e1d34a145c340fcd2f3702770119d1504da2e90ee8cf61ebc98ce", + "f0dbdfdffab87118e1549236292638f7846f0346115debc135c9f7ab76481624", + "f0f6100f7f043d80e2d8ce1d779db6b497b549634415cf66ddb07b5cfc32a9ba", + "f0f910c3e7bdc180f5d4bdcf9969badcd6f9ce1ad0021aaac14c0e6268b93593", + "f113925bba3c99674527e3a1674af50ccc53272dfa59bb7e3ac95713e0263cc9", + "f11a0117829cfa3cba8005d6f7853eb1b972bd0c946a92175a4e94725e03721e", + "f11fa2ba5323f3df29ad79ad699aa5f4a6de4b5a44843d15393588dec3aecd42", + "f12815e47d204a27bd5b1f5f7d006bbb61f7bc8ebc81a5a238e3ca323bf50fe9", + "f140b3f6e23c614a37a424499d2195d9390aebede5e7d7249528f8c97ac4f8d4", + "f141613c3556bc991572da8b83f7e173bc25c96339e883c0419ef6909a1b4c9a", + "f176f0da7b059ac9e4eaf82d45200a273674b1a3c7762d988c8b45f72a9f5c9f", + "f18f388ab015171d6aab7c42b2d0b7460f2051e823cc7209f11a2a158eb9e69f", + "f1e8698545df73640275657660f6d64d39e679ad11a8e51db2beff9efd7d575d", + "f201a6a126befab7f6ecc164e97bd97e7fb96d41c449d5afed4e70bc38dca2ef", + "f219e244f6d65c5d2d342f118c7f3b86f8d17e9fefb358dc2bac5517a93f4403", + "f232edac8808e595c7660ed3940acc5757889c36fc23cbaf94c0256c0ca0e4de", + "f235b5c48056676e8b72a6ab386dffb67c99cd2e777a366a7bf2271009947a0f", + "f275219cb79a11c529c59fff67cbe509a3b4b494baa631231dc2ebdc434cd4e7", + "f297043452426d4badda87fe216cfaf3218081193df564a8e509b1e88436183a", + "f2eb09aa74a3916a82b93cbb45e64b6987b4b0db2bb41508039da083373192a4", + "f3202ff50ba57b89a223afda873dc154246a315d02cabd86d92e79aac87b43ee", + "f36947f97e0c076049cf7bbfc4b1f9e29a44ccd257ba819c8a4390ccd8c104f4", + "f36fa28a0d82ff6459afb0cc4c214a3f00a2f2c05ce33a7bac83708b92e8f790", + "f370cfc1ec34d5682a22d438f262a6466af86ebb6075f28a95852a8a9e1368fe", + "f3bf79df83a9498bef664e804ba1ac9abb0540d1aae420d363aca076e898e2c5", + "f3c7e653d4b7b48da3d6827590d2571ec55a0e2fc93089bbf0729b5118932cf5", + "f3e69d248ea724fffe205c18afedb24684d9e9732f7123bb52cb080c0813276a", + "f424781bd9b11fb67fcdee09cb5e383ad7eb8e73c9549f7cb2da216a5e597c38", + "f451ebfabdb362efd297cbce185efb7a1fbcbcb0ee813e81eb06cdc8ab9241e9", + "f4607d690baf013d7a802204f90eff95d514e819f345a9ca28a1ce98f372d49b", + "f46093cd26aaa07510e0b7bb6ea87df4989f67d720ede606e1ab1efe80d63d76", + "f4af0630de6af54e875ca3b5d161441a92c31a3d2755c4679296ec56ed1a4208", + "f4d75bd491dfcf2f2794dd3ade4da77c7f9d4ae201ba391c79c3885bd2a3a032", + "f4d97df423579ef674f017107340bb087391536562fd7949514ec8a0d6f3457e", + "f510d227beb77e05cc85eeb4cbaf7a037ab4696ddd28337cb3d066a514e20344", + "f5811d175868d36ea16bbf4edfaa39742b67911bc911f6407c853d4a60c2004b", + "f5a72bbabf09277fb14b7d4c26a8ad14fb18e35b770b84c63adff036b1130211", + "f5a9086e74abd4849d1138c889e6363e1ad753a23eea3b336c9b5098dcbbbbf0", + "f607d317a2120ccef46ceca3f4b01107e96dfafbf7e59b9eaac61518a9384894", + "f648007ae7f6484e0bbbb8540f09dd3799d97892127e3719cf24202000f58eed", + "f657f2650b495c498d64853a352dbddc540ef5fbf88c00612eb41fbff9b443c0", + "f687ac299deaec19007e4a2e760d551d30b34298db875526723ccc4fb29c4543", + "f6919a9c489ddc3392482baf57b77b9d6dcf57701f91d52f39622b411357b59c", + "f700762310adf887ff5ccb9d8345a9b8c70fb579f323066c2a784120612038a3", + "f729f479bac37a44de358818074337e1df23ac866b68c37ce94bad896108f756", + "f74f1b83529ecd86b42df89fb3db019bd37dbdfc0cebc44c1a8295617836aacb", + "f7601fe05dbc775a3669d8cb7056b902646da54214fb50515b5e46c2409e2c6e", + "f7691fdfdb3293b4544fc7a352f0f0bb36f9dc5f9f7be92351ec0713b440fba3", + "f7702d266606741d021f0a71da9cf4742074f901e150b84f94d792303db58e6f", + "f785b6dd3ff60c5fe59dd1c191c094844203febfd64e047f37c5d77173f3f66d", + "f7ee42dc5ffe55dbbe926bfb15e73fc7856a879741e98cc2dde8d0d93d5c3952", + "f7f4125d2f2282786b18a12631ecfdc43b84d6831f40e667c2e29307177a24df", + "f80da66a27af19b2789125c6b3f6d7c1761be79cf443343933c1e9dcf40fc996", + "f80fca2b74648131a8a080a1e8ab8a6f7d611bcbfb2c96206e11f4c6ccedcb35", + "f82c04cf08a5d7ffbc2052d4f95bed1564bfb469e14b3d1abebd310baa7a318f", + "f847a0c8222272968b74e8818e2021323d0ceb7e2e2a59f40e6ce7259b249359", + "f85696718a674949094404ab65331bcf680c5db6db8bbce2d0ca66cb795c2937", + "f87e78a9e384348c22fd07fef02a503472337a3cbed6fe3b0592793fda93d178", + "f881585742358979d9c7078d9838a7cbab4cf29046bd22840a529cf9a7687a10", + "f896d7cc6906c2c999a594df63f5db5abca03fd217c6d22b4c083c8f1a847dd8", + "f8bd539030c47ef4ff3a14d114494cdb38bab3c1e3bdc3265b293fa311d6370d", + "f8d90aaeeabd0645344faedbe4a1f9b051f2bb897fc5fb001e02a6de84f45a3e", + "f9323ea2196bfa6deea220e7416a5fffa89909695221079f04b0259bea505bca", + "f95f0181a9ef304fcf345420a6136c5dd128dedf7d29ea48d056f24cdb990061", + "f987345695896ce10e780add318656a40f3036d2dfa95daf70776c2d89909d70", + "f9c47c28d51431a91e70491c1ca8cab97e592279a94f9607368228549ad57936", + "fa07c7f20c27c4bf9145d52efbe65dbec7fe7e8e0a74543437b056b2fa40b98e", + "fa1b71dfc12a4f385623fe52b9f2454dd162217414070904c282e74116447700", + "fa31e969c67e0e4953325d3a91af4672b1f3d68ee721c1f819f040838966c625", + "fa39b77c7e175c3622da909e0c48dc141bc2421906f477bcdf99ba3ca389d679", + "fa47b56854afae0bb9ae5678b80137f64bbfa32e3a177583c97dde0caae50788", + "fa8721a6a99ad2ee7c2e31f26f5a28a03aa2e6cedb9f099c81b1947da8c079de", + "fab9244a98d17007ca287be76d375653d42376c8a40c9837d697b5a9d54442e5", + "fae85d58a94ee8ac87350af04247eb429dc13eba424d783babdb3b40962959fb", + "faf20ae2f6ea21bc12fcba0fed95fc2d2c66fa525eccb574b82d2e775d33e72c", + "fb0104a9ad8c9302afa9844b799df77114a1f599ae73fb70ca182c18657746d6", + "fb06e5d2ecd8c36cadbee98f013d650a6c07bb2040955789da0d930f95d97e87", + "fb6c478eee898b673291ae1ccaec9d0c1e3118308994e6e9ed615548ea682490", + "fb7a06547c76adb0113a91ae5e50d2de4241269eb736261a6943d7cf576ec5ef", + "fb8086cbe71654a8393479e76bc7865b3813caf6dfeb169c9b65234afdf316ea", + "fb8db1967a1bbbdcde29dbce41454d1cb692aa293ac147ebbc0ce88f94357194", + "fbae6636795b7b141ee220b0c648196fbf00d711eb5b0730b7981fcc79d98e0c", + "fbddd6b3a7a38eb467e4dcd74d53870097e4244fe2869fecfca9e44780db9ab1", + "fbe4c736b1b9641c14842d2f9c2d6a4a10e6838033b53bf5bbabfab58c8932d1", + "fbe89b5c637a113631eb2ebfc096d50b3df29a2732f57a74aa8b237b547a04f6", + "fc288e5b6716568bfc9909a0e3645678a20e33954d8cf3222dbbee1033550e88", + "fc32e8d507aa88822efa6300d76556838e30a182d9b8084a59c05ad83a83a15f", + "fc437094c0ca48af5d90e3736f836efc88679b90be80bfbcf782d4459d7b089d", + "fc501b523438dab044b23b812eda04e81412320fabdf2c7dbdc8830909c2d660", + "fc66d4b1c09bd0e559c579f228d3c38a078e45e9d1305d4cfff09a47329a0bbb", + "fc86885e0b1a0de5f04adcfaf51758add43ef7ff78fe43cc43a50cf664176586", + "fccd668b17136cdbb99e09013df6c6dc0a5a91f1eb1bf22f989a21cefa20ae82", + "fcd1b5d1a225fba957d02e346ad9878be14a6204b601bc7f2d3e9a6b06216294", + "fce089f68c0ae00d5da1e754906c73f1c3dda9140a37617dd6eb83c4b2c2ba48", + "fce84ffad30fafc2e31b69dcd740f4d6d675a37c7639473101c780d5061b1a03", + "fd17ec469ece96894cbad6f7c7e391c4a4c110283bcce36d923414a824f835ac", + "fd386d3ba151f051475e843a9e93e392fc7cd8f3193b21c2a78870ef190a8629", + "fd56a8f2722ba03958b443a558c4e13465f2984cd852cdcf87dab011f33b24a8", + "fdaa56189cd67b46adeae1059d510dbfb12edbcdc518bfa2e400140fe3b09bcf", + "fe19349e611215277bbee9b125024dec258002ca7dfcd3a7d6e2b5935dcfed50", + "fe196574c75b0c689806a6ec1f4ea29cb90df6513c167c4c8f168a3cc4af9676", + "fe1c600d5df592d4c32d16ff5e494006f5c98b172fc2b4f6e1697a713ff561ee", + "fe2b7d4e8a2279775c1366b1447b7c2ca7b9c78f80250778eeee02afe18fb26a", + "fe35b9a5ab367c6ce9b369d900c2e296de0d65106a374f90da0fb7339e6a1f8e", + "fe605b386e9ad2f29da2d05a1ecbeeddca2f4e96e997dc7f936f2da629381fc3", + "fe79db37640f9939ca2efc575b9ea6f0ada86bcfc2fa03c6d33decbd4adea4c6", + "fe8a0a9d5dd6d92788187afa795eaac740e028d09f80a76682e4f0c33a5f136b", + "fe94f8c3fc0116c93a617302afdda03a1964b143983d60e257509a690fedca37", + "febc1c3d860db8a9fbe1fee5d8c1f502d19797294cac207b19f790661312b229", + "fee93db0be8b667fcad98db04b454a25912e56cf54f3fb8637fdbc0eabdcf2a6", + "feeb03e7148982cbe914b41c73f9e6f399ccdfccdac69269fb3a92d214b253c2", + "fef7239ff6ce720d7a669ff227efa115fdeec7720a1dd0c35e5b1fdc58e9bd6a", + "ff07c472654ffc93b1e5cc2d73137b9c00da43f91f81c163d8f1ce6e3549ce46", + "ff1b6d0ed5dc56d68521316910c4d5f1e9e4c149199a4fc4ea61f27bfc7292d0", + "ff25cc50d0ec9a6667cb3562ea5a4db8311bafb7b6da50189ef2b88e8f792eed", + "ff3206f9c2acccdd3717c968f3a263cab72d9bd0ae23a9d3457d0129e01f6af9", + "ff6826aba61a52110343bc8400371d982003dfba256edac36ebd3559a2b924b0", + "ff79f3a026549cc2971adfd50a9b5308ea8e872e0cd3b1f88f9923d18d9ef983", + "ff7b6aaafe05fb6bfae3c4b5d4d2085bc210bf47117d02702d8b3aeabb31f0aa", + "ff9132bfcc7b97ca2285f63b6b98c9e5ca470341af195fa9ab9b33e5597cfd77", + "ffb1eb364e359c1ea440fd1aaae3ace5524bcb3e61da03b813c249d7f77de6f7", + "ffd5b2e7c3e69af9156ecbf8478ef0d2911bb1ceb72ff6c5dc2207e71ce09a82", + "ffde31f75b901ba39b1a965ac5540655b4a238fbff179c2e51a2ead30aada872", + "fff33bb40f0d5856f295bd86e969edec5147c0aac5672ff4f959b4dfad835dfd" + ], + "task_counts": { + "LiftPegUpright-v1": 5664, + "PickCube-v1": 11264, + "PullCube-v1": 4240, + "PushCube-v1": 5872, + "StackCube-v1": 5664 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_split_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..26988e766c6751325c7f72d33b199bd133350cde --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/source_split_hash.txt @@ -0,0 +1 @@ +867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/split_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8533e80f235606cef4ba0119a3ac3806d63a0983 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/split_hash.txt @@ -0,0 +1 @@ +9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/table.tex b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..e8f238faca54e2beddb03b32a4ed0f1db12dd669 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 69 & 0.2754 & 0.1884 & 0.4058 \\ +base\_utility & 69 & 0.7181 & 0.5449 & 0.9239 \\ +branch\_car\_at\_8 & 69 & 0.3487 & 0.1943 & 0.4943 \\ +hidden\_chart\_oracle\_success\_at\_8 & 69 & 0.6667 & 0.5507 & 0.7826 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 69 & 1.4000 & 1.1926 & 1.6081 \\ +outcome\_ptr\_at\_8 & 69 & 0.4638 & 0.3478 & 0.5797 \\ +pairwise\_causal\_calibration\_ece & 69 & 0.2266 & 0.1956 & 0.2556 \\ +proposal\_oracle\_success\_at\_8 & 69 & 0.3478 & 0.2174 & 0.4928 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 69 & 0.0725 & -0.0725 & 0.1739 \\ +proposal\_oracle\_utility\_at\_8 & 69 & 0.8180 & 0.5915 & 1.0681 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 69 & 0.0999 & -0.1436 & 0.2862 \\ +selected\_success\_at\_8 & 69 & 0.1739 & 0.1014 & 0.2899 \\ +selected\_success\_gain\_over\_base\_at\_8 & 69 & -0.1014 & -0.2174 & -0.0290 \\ +selected\_utility\_at\_8 & 69 & 0.4693 & 0.3037 & 0.6778 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 69 & -0.2487 & -0.4637 & -0.0852 \\ +selector\_regret\_at\_8 & 69 & 0.3487 & 0.1943 & 0.4943 \\ +success\_selector\_gap\_at\_8 & 69 & 0.1739 & 0.0870 & 0.2754 \\ +success\_support\_gap\_at\_8 & 69 & 0.3333 & 0.2174 & 0.4348 \\ +success\_total\_car\_to\_hidden\_at\_8 & 69 & 0.4928 & 0.3623 & 0.6232 \\ +support\_gap\_at\_8 & 69 & 0.6088 & 0.4128 & 0.7915 \\ +total\_car\_to\_hidden\_at\_8 & 69 & 0.9307 & 0.7182 & 1.1327 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/target_index.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/target_index.json new file mode 100644 index 0000000000000000000000000000000000000000..bf98fd3b10c4105d20658db53fde359eba827cfc --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/target_index.json @@ -0,0 +1,929 @@ +{ + "audience": "evaluator_only", + "base_candidate_types": [ + "policy", + "policy_residual", + "anchor", + "base", + "expert" + ], + "candidate_type_counts": { + "expert": 419, + "near_miss": 838, + "no_op": 419, + "random_negative": 4190, + "wrong_direction": 419, + "wrong_gripper": 419 + }, + "content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "dataset": "/scratch/knguy52/dovla/experiments/six_task_h16_collection", + "deployment_candidate_excludes_expert": true, + "deployment_clean": false, + "embedding_content_hash": "e4374b56912fdb6814eb0320518733d1602a90e0dd2f9268f65c8a148751f963", + "epsilon": 0.05, + "format": "cil_charts_npz", + "group_hashes": [ + "008d30588ad015db473bad0a71c13039cb1d7725fd6345f85f61b4f135a5914f", + "010f7c7c4e75115a818b44fcca59b1b036ccd9e3bef5c336a53008f8a7b0c90c", + "01349bd14e96a28b27485fa7b4152088fee51e14509318f34e89cccdcd369c86", + "026a5501ef47551172cabd8769a56ef391320107d1c27757256c6804a0bc52d2", + "029379cca8b2cc44b70649722a5e1a91cb6e609ca2ca13f242e01ba12bfca76d", + "029627d900d662439e83a0985be181083aee1388afd0887f6e0fbe802d8d33f0", + "047b5299a16717c2aea07ceeb278229f0b4993fd8fac84c7c7added14a47ee77", + "04d0793c098cc4e382d401ded0ea3d56c9677d2ad834b5901da53ccda9e7d606", + "05989b6549b3ff8e8e6c76d5ac972447f33600ec31b946a59777fd915ee136f6", + "059f2165f0f92f81ec8c873a978423ce310464a3cc27b1b08eb88fa7d5a5b3ed", + "05ec227fb1c3e112625c754c648e95e326449767ace3e1602902db3ad71e378e", + "05f2d0050690209c8abcbbf38499e0e195f5fe9078d5964a2fa4d351dc340a4f", + "072314f777216049af237bb018df2e562ad21accca2928fc4a1728c4c0a70dca", + "0746e4881ae94e6843c1d7e85f889ac5196b2a58ca2f1cf5e2c731735ccd70f8", + "086e7180f0fc674f6b011339a3ab06feb8a4d914744385734c4af6938d63ff60", + "08bce62400dec82cb7f3de6cfdd7e831fc3733bb7d920e8a13144702403468d2", + "0a68ad44f481250311d0f54d46938b1fd557893af5d9efca367ded1c61e3c2e8", + "0b50e4bf134f6df63b79deb0d5c1e8428d36b52ac3b97260c5d2d3f3286c025d", + "0b5309f59e3b116aaafffd79932bf558afe148cfa503ee100797c472adbb66dc", + "0b677f82a30861323128c400c43186f364d115c3646f77e0b5b6fbd9aba93af8", + "0b7b725b4ba7064a7598c6b283e2bea2f81f95b08e5390df39bdca14cf174397", + "0be3d37ad2ea7b9c4768dc6be2b505a8569caafe7e3c850bfc5f466cb00ac715", + "0cd039b28bc0a289515b0552105237c97f010ea8f52f1d79d15ac23b2c5f51b5", + "0d02743f28fb0fc84f8aa9b502d34a15c82714c2afb9c8ea8a0e2731a134cba8", + "0d5ad7cb6f339dd2b38a88d486f273f01a85efad9cf16205236872fce6299e6e", + "0f0804f0ceaa4f105cabcbe15bcf37c28c8b2deb76adbc77d10f106b31d1a19d", + "0f64d8c709a8f757bc9a972ac48f24f5f9f618bc9dc10c00bdced3ea5995cffe", + "0ff07d6f8629a231fac814ae3f8a9d18d9f2d5a5d3144b16a20ae7d1aa202cf4", + "115cc5a336dcfd42aaf2e70e91399dcaea0c67511c2d2776f0583d533f3ab1a2", + "1214e59fe6c7a0a2c552d75d76556b94ed956f9483f43c23c9c918ef74bed5c4", + "13bdb78c493f25aaa60ae00be517b8d81a019edd3fed9db1d110730c682cf077", + "152c1154221df1c61379262f4431935a6643a8b056939e6067a24277dbd95144", + "1736a3e41589f068c5583aa47361c5a65a38c756d7c83dd2b291e533f8ccf29b", + "185899d72412376f441583214f89c8b4f5919141ecc7d26040d421ce38aa827a", + "1859b39dcf1792ba35aae7f6525f1c5bebabaa6cf96764b6d91efd065b7c5b72", + "19164061e9741ae8144c089d22914c6687147fb1eb9769ac6138ef3ff3aa3bd6", + "1985928f34b5eb98e58fc7b2e0ba76f219326d56aae7e44d5e05b7c0e37a6a47", + "1a6b5ec8a4bba5d6cc47c74786b709c3f7fcc745cecf70a73dab8b5e1196081a", + "1af3466db9afc8d2081db54c173835b8ca6499309fe4f3f3ab21c9e73e73c360", + "1bd01f63cd4031a48713788ec00bd2f31261029118e648e055cf33c285d3ed15", + "1c59c34bb7ec1e57a6c4cb6b99b155a268a5905621800c9d03e32626d2aaedc3", + "1c84f9a3ba87babf7017b4310f0995d45d5e2a6ec3856e30afa5792152a078df", + "1e0dd390e3e7b388f54330562460b0c813e82ad862d26c66b42c6c3d2e46e2bb", + "1f7225e5a9e00d9ab87b51faeb0874623ac983f56dcdf4a855d0da29f861cbda", + "1fa325165da53abe38062a07aba9d719b26a7e890f8663f320ba2eae96072d14", + "21707fe0167d5227f7f416747410e40ae3f78c6c2b825361ff3f3091d56f42f2", + "21b0c38799dbd91817654ead09968002f251beb305f2a0dd488e70665cb691e0", + "222b477e56cdc6dd529442698720ed23b2fb6089d0a934ed3cb97104669ad1be", + "227321345173cebc9198ba533e27dc7f68bfd22fe075783a60ec7c32a37ed68d", + "240fa7babba75db5977aa68dce1f0d77015a455839915d71ebc76b252acf7399", + "24bf7cb1b06503ef7d2d0d33b11d92f096a2d9f043ec3454dd24190c2f447410", + "26ac77dfd2a9ee1f8346cc862a4c76e04a552d4bf09a95dba0fd72e86c22d544", + "26faf0b54827c513fd7411133f3d655c69be1762be7262b7c95a59a53c8e5376", + "28ffdf18cab5b40f176891bfad5f86b67c78fe0a164a773f0ea4c9e4eb3e2e89", + "2974927c1ec6cdb00ca053ab7064ccfc1528d35084141b0b55baf42bd1679e94", + "29c243497d2fe43220d59c5689f78e8fc25be720880c7c96fe3bb83cee571730", + "2c519a92b9c61870a25bd465e387ddd56cf785dac01d3d799f1336071a688cf6", + "2cc11b7818e043aaa695b6cb39e23ca678666972630ac06f4f9f233d1f7f0808", + "2d3b45195ad9548f6d927e45edd80c0472fe6b99444d7bae2078279d94f38dcf", + "2dd4376fd28fc3d5be4efdb5e4d0788a73c5e2e017433df7afcdf4e295073e71", + "2eda514c8c3130822ecfb887b782233433591d277b4d5544f35869951575a3d5", + "2f62b63771280281f0bd7b76aeaf94b18feb3d9e77d5f4c3e6c941bd1b24f682", + "2f6c7e63a96faafb2769f3bad09af0dfb334b7572bed6de64bf9e9fe48a46bb2", + "2fc4bcded9e3b5488db04aa8d76d91801cd086ca4f9b1297717a52bc367e9b16", + "2fd10e31d0173bf7895979bebccc703d39b3bac009853985801006d7d6e2005d", + "300fe4259c0457ea26c5c5ddcc0ddd771457c628008b83e328aad35d009a7988", + "31a5cc9dd9197d83dbbd3d3ca4142fbf28b2fc8201bb741270773cb1894cfc7b", + "320f6a196e59b2b4edd608ebf7ee79f17c8ad5466630843e775035c4846b90c7", + "3228ee331db1a462dda0f3f48e0e61c214e7605e1108cce578ea5f2fd595f1d8", + "333b07c20959eea559a91f453fed7eb811df3829c14be6ce7a0f2cf0d2b43920", + "33555b978758146c6ef5db44724a5c7d0c6bd220b6f59563564431729312103e", + "33e88827861239c6dd5fc396fc3565daba3dba151859cb16e84bb12f27cdecf7", + "3404ffd881c45722234ea44705c34a255fd4c7bcb8bf6bac83c63d67405866bd", + "3419d427c11b72a992314179016bb0a9148e4e87233c09d9a2670db8fcca245f", + "34fe8be6a6881aede4d578671114f9ecebff8022ff512590dcb98d7eb56136a7", + "362adb9e2f87f785eea8ff421cc49108735a1156b118db43f7dfb3640de23566", + "363bb1c2cc2d0815667febd486acfcfbd569db438fa75e5e535e1533c735964f", + "36b9c0d8e8dbf42d0ecf7b80842f7ece7e817229b8d9d81b9abaf789f2c84bad", + "38d12b23358dd923d0dea27b84fe3be71eb3da4d43c60a7437fd0d79429cc151", + "391a987cfdc35a7003584b41e687fcf13b3ea16a0bdea3579e38b9b68283dbff", + "3975173592d19589ee2483b3a9c729a1b09e357dcb9b99289e91e364a55d34a4", + "39e42a7c1b6df133a041cae19e34e0f2b67091d7aaa9fdc857e8141d1a02e0b2", + "39ea289587d7c4a9fd58e23fb6d50bf46589a1a9d30d37f385ad74547d61262a", + "39f53df98841de6d36ebe7d912d78c71b7a4880cc7bc08db0e52dab1678006fe", + "3c6ca95fd406cdcabd3371a318249e933f024d2d96c627aa2735a8cf7fae7804", + "3c90d5d7da13b39be123adb5d07405cd5957e4bfb7c514020dcd16c358ab80c3", + "3cad974195456ee5a2db5145a90ca0f2558e212a4c893a0616803f4b58326e77", + "3d546da4e682f1e5e2e378874bb802a821fcf1a2cd8f3ad02fb4c2c2539ed504", + "3d7dd030d76915bc67fd843d4a0f7f16e93fe56d1ed2dfceb77e3c1e846aa2a4", + "3ea9916cefe474f99143e9d69e0363de2625d9ce34d093043f1731a1a36ed598", + "4085a47a2f6b662206c0c7fe411d7b713e52f429bdefb47b0aee4f7f7eb40065", + "40fd8700ab3b156b4b0f82ea40b48dd00341dac9dea27e31a89bb53587a81fc2", + "4177ad99c83f31a03223c2c57ad0633ee8d930604b5da6143288bb9d95a073c5", + "4199a7885060607ed8db3b1218328efd16d4d2ccf80bc1a81d0bc590ebed3232", + "426e498c237a053ceccae427cb0541c4b237d508b4464f772fa89f78625b4120", + "427b4bd0585c94622bc045b12e05ac66ac352697c428ca2962489b2e099e099d", + "434cf031465e32cc99d0be0df1be0e209bf206de47292393be87ae541ffaecc0", + "438a37d6b4bd0e2fd1adfd430acdf90e336993bd959f79d5bf71fb37a825150d", + "43ceee1b0eca37c6537d26a22ae452abb5881e096dcc2413a056be46ffceddf4", + "43d3ff2bed6137b1430b5b5a3b10e54c39ecddcccbbd9a41e9a16ec6604be4cc", + "44632cc411cf104f311f9118f831b8b096458e112ba5644c00e7c4c230cfcca6", + "44bb557346fda02484ec041ca21fb14dfb3de5c9721d5d129ef0e9ba51181358", + "450ebccf76e662aaf43e9bde1076c30ef05eb7a7ca03cc5e40d242d4088010ab", + "459c03ca4b12a9927ec189194a6dc2732521761d69a41490909503ed15d0be51", + "45d4b64e2aecad1a79fdcc592484933cf1a922e6ef3db9e9aa626ffe9e4607fc", + "46c00e829542ab0f4441761c7a1c22ac053712d38c1f1cd57bc6bed307e62feb", + "47416d5350e161048d0d62abae130ee6e13e620c1453601bb68f12566d3a8aa8", + "4884708d82754bea1caf4baf09354f1013b3932daeecfa07713ee2ebc17ad8b5", + "48e3602d53ab14882fb5a3fc964efb696a97b153329b4be01765b62164b31cc5", + "48fdfb473480e244ab50a0aac797379a1d447f455569fcb32d54fb3eea07a388", + "49c4c3c498d85c4b459939a083610984c215e633e361f0ccd2d6e11fa3fbce39", + "4aa7582e97431f30c9db15f9ed5bb7568ca31cbc5861c2c64abc20827ae986fc", + "4b63bed5359a285c67e9ff7d84280f7b0a40c641104fa0a1a50af5ce0ca419d7", + "4bf3a488c5402697c040893291a578005cb34907d802b68844fc5774842462ab", + "4c6d7902a1e22b2bf6f7f4167a860be5ed39be0aed7c909f23dc649a728b3cea", + "4d0c7ab2ff619a787e1e947cfcf1c100528c47c16c90aebe19e3281d5747a55e", + "4dba02fb52c247ab7473ed24861f38cfb281456a57a480e4d64457180ebd2f70", + "4f81b09cefc1d9a5f91ae4cc1d5b2788c7c0e0b384e17512e82d4bf023c24999", + "506d3395d1567f02c0c44436838ea9889d35976ecbcb7ea30f9a50adca34ae3a", + "50ab237de0f3ad4f4d676158d6f474657d272f8ae0dba7ca4950f917c0233b85", + "50b8547d0dfb7ea95945ad82c54b630cc3ab54e0ab01033a9f19166fa41f58d9", + "50dd63169fcb6480776acb6cf689077b9fe565bd6acffec4e86bbe8b12411424", + "5196ffdffe946a62f4500745d1912f3fdb667cf7cd0cafa5ae7a035b896db048", + "51d45b6d1badda9da12208d1b9a1ff775403485fda5a8652a0e8d7d499f4939b", + "51f1b9dd8bd0de616f527f8848f95818854511884d434a2b394115e1bcc7a070", + "52479f35626dbd059b00c6c86e23379ab399d11ce8042b4399b3bc9c099fcc2b", + "52fee2dc2bd87e3c1a6c0620c5c627b2cc30490936440fc6122e0432b983f5c4", + "53d530ffa92afed3ce280ed6285c4888ba3bb59ff1cbd07fac523a5318f00ec3", + "5480d25e04a50a517a951793e91afcdbc3ea583236d08af15ca500be17218b43", + "54b739816771589b437a20ea18ad6c027b5bc534aa88a3be253cce7a37acd3f1", + "56137914bc27ccc2862b7a1a1452b01a33e6fa0d263c6afa6ab0607d40a8f099", + "5741844eeabbf8c6c21ca90af591fd3c50da313baa957a4053557992b8cc4866", + "5747a957d9972c2d47c89fd759afc4a8531a338971ce57954a785f61761f57c7", + "57acbd0e674472d2028036494f2e7e3ef2d052f4edac55fed900c1dc89f1347d", + "5897e01933db895c1bcadd473c11310268cbf5caf38c818f81d4bc96e93a4f10", + "5910b96219ff6c27133fcd192fdbc1c14b2930f1eba4593f03a31db8b7e3291f", + "5987281f5dfa9bd2089a2ca5da16d8ab94bbc2b908f1d04bf6548d9d79d6f0b3", + "59d30ed358c5b684b4eb3b6a32c5c62986761e80d85b519e1a48f6b639ccbbce", + "59d9238f47488e2bdd936a0d1c63ea4e763e511fb9b5c903862e239c35475ed4", + "5a678af69532034429641cbeabdcdcd84eade06b83dac385dd6d8d7ea36195b9", + "5bf1b42a54729b348c61228315ecb19a7b2ceede4d0e7b8dc9afa8891bc81d58", + "5d0f3f38df807dd181beb9335c96aec831ddabc4d9a634aa7c54334ab16c1d75", + "5d4e9ddcebff2ec81105e8ae79f57de3c984200f9994e88c466a0aa07c8e1320", + "5d515949f2580c7204d7e0ea416bb67a32d528e4ff3041f37bd94c08b56c5357", + "5e418b7e9d172a9813cabe7b9bf34c168db366015d182770fae70f0ac41422c0", + "5e78b744add77bebb64bcf18bf09a83d5597e058afec7410d5da6eb169100c30", + "5eac061f4ceb6a0fc89ae4a1cee37ca22bf27174233561d82042b577f736242b", + "5ffa8307514fd765cae152d50388c2481438d15691116a76f78c1fcef9d2ba57", + "600b24bbb9011d209e230efc0651ecbfdf9192844de83e795d5993ee034c8f06", + "602956f2c532b51b46cdd0c64db43929f489e5804e760c69806aed0e26b3514f", + "60c343e6f949a3098f6f33740971af76511890cd47b68c88f7ce1894d6d46b0e", + "61497e92cfa672eff759375d67829981565e5be07bb87bda0caf2525f38171c4", + "6214f701d647152e08612cd72bd1e15b64329676d053324533b8c928d6e6c9d4", + "62f0b0da66a12c0b2bb3b7f1324906246d8aa8c22dcf738728d169df29b85cbb", + "6349e990f1235293d2dbad51467abd02bf7dab9fe63a77e1f5e661bb5e1a1672", + "63b494c4eaafcc2da75d4d1235d198b3c0b0c06f6bf762a6a5fc7baeb1a9e385", + "64010a7e425884a51a7a5195beffa3367cb63dc05d7d9af85e6089f959e7e589", + "64bd5fbc7e70465034494d5dd5147ece3426224ab802b7051041606c9af60ea3", + "6518e953250447419c08c59d2ee155fd52ff507b28eeb3a83433bd89432a37ea", + "651c0e51e5e9e00565b3824a7f42082fd052a2df3b8df4e1caee0cd8e50e90ce", + "65afb3b6725976a6ee42025db81e84c935fdc10361f657ebb33d9cfdc442d502", + "662a3ab296db484aed6b385a16f29bce8467cf9e45bd85b7d4603ef7aa75a911", + "665000f91e6250f222e1d20d8821ac20395c23127d25b29b2a18d824efe5d95c", + "665b59a56dfe89b21ce7f3e1078d4070fc5d6c6aa9c32fc101782b87c94da10e", + "66db6cd423a630f043ba70653d9cb2d3912d39d74ba20aba42f4ed49be24515c", + "6707e8af129b5c01f0227ea9337cc805a44e08594231bff59fced200904b68ca", + "6762618ec7b24c3b7302187e22d53c4c8fc36c5479d927e7b22f82df0096ab81", + "68aa926879c32f5120a16375df788268f6d466b722c653eae0cb489b64c2715e", + "68c7948e1397aae7ada0e711538dbde0b894aeb6c65a2a28b789072222c7d04b", + "6978f93cc00cdf51bfb5d79aef73aa1e6d9cefa78fa0452fbd6c7d536692e095", + "6a4016ddc183de9be99d891de4af7b52506be085bdbd7b623afe6ae67d380743", + "6aa36251dd6981548c9da087d57cdd769ef61ed00e6b91d9ef9ad66fd8042151", + "6b317d5548424aa5a1e695f983622fffea91e6839ad1998a13000c2510316cd6", + "6bed9461397e0217ff6532c954a833dc28fece0cb46e647554a189e66524cb97", + "6c583b7ca612a308e259dfa946fa8cf45fc8b8ba31262221f833ff3f44783095", + "6ef52217b53a0484ef0bc7e133620ce5236c4268111f9f9e6562732ddb3c1bf4", + "6f019701fa2e5327cdf14bbfefca64df0f5aeecb6928fb52e5a01acae8633fe1", + "6f25c0d6d8191dec23c8d22c518212f1d2507ad4e3d4e4ac34869a5e3d988cf5", + "6fb7eb6b8b430c8ef441afd0ace15a4c42dfd23b3ac134c399abfa06390058ea", + "7258d7dd2d99620e2f9563f387392f056353d9cd0feaa9065c2a8be194a95f91", + "730b0ee7d1a69be505cfa2d8f82ef12619218566820fcbf2c2049d781cedf6ad", + "7345a52d3dfba26a5528debf07f380ed05b6e5a1dd313e7e108226d74edb2724", + "75038a8ab21048d173a77df2a2d9410cf256f6a587ef333f82d545a2b0da5495", + "772ecbeaf77547ccf7b918354bc234fffe749626c7bc4b6644544711300e3795", + "77a5e2b24fd4ffc2325efccbe53d2d0af89831ac3081665825a5d19b0f99bf58", + "77b31ca7d2196bab4fb31060a8c79365e306514ebcdaadc089a12ad03b7ab734", + "77da10edb61dd80d381746bfb7929ee08eaf96f64b65f535695e5f076fce2861", + "792e6b2c4a8865ef31fdb6e497bf4c8299a0cefa2fe47777c8b20dc4be86fae9", + "7a19124e06706c046acb04bc05c52c019ab35a5c0bc2d9d48e752d4ae0942009", + "7b750fab861ef3d12d2b387ed46455aa8b65ad72ade5ca7d253bfc8d8408c160", + "7bd6f159ff7920fed389c26281943a38f556af30c29d4aa402d5d1ad808a39d4", + "7d913b1ad623536ac5812120629dd8cb0455511983d5c0d1d26277b9aa0b5105", + "7e010a7eadabd6135dcbc0a81d71a0c4a40d9ec4d8e4a41b9bc887d64fbf8c8a", + "7e030ac9c9322183567b1955bd4924366422f8ba5246e22a232e5932702f2069", + "7f64fa79eb60a5f4b50f1b33da9b9b3d99737cef21da808c825c33092615cbc8", + "7f9ff6f99c5de72a9908c3fadfde6298fadfdb9eb36c32219e4a7fa10892f642", + "7fdaabf2a4be809033161efb0bbc2828cbefe75c439216450b11d705a706a1e6", + "80bda934d0265c977e22f84b9a47bc985de3740895abeee0e54cad76fd791093", + "8196baac48edec4fd98a232dd60880cfc89bfb76b772d9ce7c857a4051429a44", + "81afc453735d484ff81d6fcb46b6738fc9de062504b255a5d366d15833b2b93c", + "82ba96b0db417c596fbe2c6f05edd0aeb2e82e0903016792d0bbcfe6551bd17e", + "840446c401008c1f4f1b1f82237092cbc1a020e55cc02ce94792c3fd4c4f4ee0", + "8488122d93a4ca2429fe5228db45ba7185a2290931adc5c1c99e8cbab534ac39", + "860f89295df9fcc82bbdc1b569404fbc284cd0db6eb4aedac0d72ceae19e27aa", + "865a93b7d07abbf7233f13edf40ac7ac9c8013b44bb5b1d8f56f6628c9db00e6", + "86d9accc836046ab5fe18c08db4a4d0d5d65eef66c487b81d39befa68a892c69", + "8744f7b069ab27db815e0d8ddafb255992a61c37084b201454809d668506fd34", + "8755482d237160251dbe35fe0020ba85c14e541289f1dec4f6494305fbd8df89", + "87a181cd2f9e280ca7cdc7f0fde3e07112a363891302d8682860c081c35921b4", + "87f6acac31dff38cd80e58f8371fff6142712d897de20092d418c4a430a80633", + "88a012cdc526b418b4aa88816d251083f4853e870482f8e924e8b6a8af8ba731", + "893c75484cb41d8965549dea2b8a2bc089cb463a1e6f8469546dc92c8705de39", + "89d95882eaacf6f7cb28c46b23ba22c3ca11986ad0365890f461a800c7d087d4", + "8a41d9b49b6fb8253d7df35219e7867110656805953c6d665f128bc74765ef89", + "8a6dc00f6e7e5cd26234fa4eeb4d15a8547fa593afc36997459216137d1300b7", + "8aab254e85254d5cf1f1103bed51c9cdf8d6e287dda1b267b7d2adf0d513eb2e", + "8b7b0c0c36a755ab9fdc1c810ccf6abe54e24c1fd381c8da44fe1fefd0d05d76", + "8d11dd1ec76d295ab531cef7f05a62c5e138f2084880886a78c6855f907a19f4", + "8db7ec8b19bdce5f9886dc235e95546452a592b45a0e0377fae405b80564eef6", + "8dd973f2ade114309f025349bd4852d7fb3fed114871ab48e19583c48260626b", + "8e2579fa847f97c9029eda36fde5da04309c4e42188364edd4ade1cafdbaa213", + "8e94f3617db242fa9ca8ec1d20a20134c46fb1301f1c585863055e5e1ccfc87f", + "8ef7802d4d36504e28d179e70878185878721e7aa7b791aeb9f4510ae926e73f", + "90287d800986804804b7ef7bcdeaa94888d09b6329918ddc0136dbd31c7ac8c0", + "913df90c98f124b2ac5f5565e5e86e1043f79f34f1c6410e7843886c4bfd138a", + "9143d67f035c7c08bf56e3bb65f4d597492147e3bd06e575d05b9617fd5a2e6b", + "914a0ddd825254d368ed88a4e4c0e1492b62b685b54d43a74eb809c21c0b057d", + "9155e1827763664471baf371a8b288f53632986039fe13fe8bc519af27b7448e", + "9180decef7665bdc3dfc930e38fdaa742261568d8a323e5ebcbb5dda83ab7435", + "92c3f4c9a5c73d021484d9c22b11dc3f0a5938d92e8b4a3ac29a540daf62cdae", + "93308310bdac1dc182ac074cf0f452d5857412a3dc381d6adab000413862c294", + "93fe1b775c9d5a005b4c24ed4cdcbeace7c443fd96acde340cf37531e0d958c8", + "94adae68e8f80732228ddd487310402d7af461be7ebb5f483fcc14fad5c84368", + "954022aeea5a16063fab63a3400d1e5ae2e4c1cf3af56c94d437dcb613d6eb9e", + "95848d52f9d2b349485284334c4efe48bc0c9d0e4f3993dc3fa0b93917c0c794", + "970ae009f20045d56e65f7c177ebc783dc82bf777b5dec048d9ffb62c2e7073d", + "975adc9af6b12272dc8484adb4210f1137470d6c77b2982cf4809f2cff9b712c", + "980d048a07e2ed128f0cfc5768babeb12f7b8c1102dd264d40304becd711ada4", + "982679e3e46a917a3cff3891b491fba40130ce6ec338858450dd68b6dd6ee16a", + "98d2821446cb8cf7f5342382fc409bf4e1531234751095148028095c4d68c0b9", + "997b672b7ce7473d2751343cde3d2aaeaf840cb344d368ebbadd3d2205a3f079", + "9a0729ed13e288c93f267bb3557f4698a73ed5aa31a55a8e84b558b94825f262", + "9ab25a8cc539aeaf87f23200ed1c3f9667d44963b7f6da1e903f16736bc509e3", + "9ada1587db68d41c577e46d5046862a766502fb2dafa18a139e5f797b035c295", + "9b6021119181becda810b14ab492087af39bbb14c7b7a73d51f6e9f274aa83eb", + "9bc128b3db4a990e7db2c97b5c7761eb5452183da0e8b815fe35c85de239251e", + "9c72c21529d08c5069a521af5f213b99dc9741ab154c07c3bd0b4937801e6cfb", + "9d09df5c56e375bc9454202fa81f76636c4357c73e2f5000368c4300c30c9d8a", + "9e25bb1895a025f06794aba14ab0cdbbc2d4c900b747d42d48957a823e0bab5d", + "9e71e5498441b4b84fda9d4a381c841cdf9dda4efa3e0911ec8866e7fb9a9d6b", + "9ed736060546283eaf7a9e6d7f2be31ab3cfcce0d01a7602dc9bf3311dc84567", + "9f30c7f9e60ba8535bf2d90500801f28f4177b66ea3a2366eb50ef7ff7f9b9db", + "9f3b2d807346a6cb4c758c408fef30a978ae8b2b82ea419f13d447daad058f5e", + "9f9a2aec986ef7f8a6fdf55160dcae5c969717a28c50a5bd8a3b46bc0ad415fd", + "a00ac37fd929bfad607c2c6799f93375b96a8f3011cf2f49ff9f47201ca01987", + "a12758b2b9df7c5e40f379dbc2c5374e2fa0508e5f21c7b5cc38dda34dcc7102", + "a20b63cd6f96f12ec818d2ee4b0e6f2017b73aab05981d5a46d2c466e7d82432", + "a2727388face2b4a7efc7b6c58838a198a9e1f420e0b8852b925386e77a83f70", + "a2cd9da5ec1623c7aa739bda183c8101f171d8713dcd39db1ac5871ff56b4017", + "a5119d9b97da13e0e5999ef6c91227ce8a3b17de9bb77acf4abec2e00259b822", + "a544ddf0f57c78a44f54dc946cb6885693ed7372a89f15bd914569f1d9849ee3", + "a54c68ec203d07c2a967d02825037471086ba238263be088d59d9c7f00612ed6", + "a5ba2e61cefc84d8908c6b2bb830d91790177360ad02b3a744c640ff5b45d6df", + "a72c67526742bf14cf400ad3515ed8b87c2fc7bb4f95070a0faea29c8ea984d9", + "a775848264bf7b4a3777a556d11f38e2994d92714c0c7734b5a4d46162c84226", + "a83bf42daf636e7a77f77955757dcca4224440cff075bf7eb0370795f9d8663c", + "a93f33032986ce24e99eb38f10fbd9a62d75462d5adeca5b4e236cef0836bd90", + "ab78cf2c57454d46dbd7fea49f3293f01e7c6dc9ae10e2eb2b4fb20a3648ecdd", + "ab8b5ff6eae18b911f6e8dc11c2a34f65b1c9b6d116c91a00b8a8792096688ed", + "ab9e518e9c2bd1aa4fa1e395553620371f0c79804c89f5daeb6ce18d5b66af23", + "ac5ecc054864495a34f0b9a16a5473b786d7497474773dcb84947ed204009aad", + "acca44ec9bfa0a02833526adbaf86c7041a0ac281b28cb7ee91f18fd09b078f9", + "ad8f425b29b9b12876e4fc5fd18c3f867046ae41c81ec55cf0b6d208e85dd742", + "ae3fc0f559f37778cfdf27f640bd35a16aa6b76c4ffecf8d42baf37e4a5a7905", + "aefc7849c632d4044a926f20f1d94b82af1b7ed6dc0167d2cc249b218a343a93", + "af28d652f179b9718c6a9a9e74f9e8e7f12c437a5ad54a75d80a82c23dc8e644", + "af82611bf1b54c6df4bc9285d7264be4ed86af1711891cbdf484def25a7d98ef", + "b0ea0f0cf25530508c4a9f67de55afe8038355b7e0b860af7ba9f04aa31a3e06", + "b1bc36c06168dda8d850b543cbd2d870694b4869230cd985b20dc8b6ca23c1d1", + "b20791ea51ec80c181aa6a54ebc7d1383974764c55ee7dd3cb3cca64948266b9", + "b34c178c6a7c97b598af258fa1a791d9c0c61e8432517e3d91953c0cde131d7c", + "b40f7a9d22cda22a063bc22dd3b4db5c3598a98609594494193ed48dc8192cb0", + "b461e54387e0a47ece2b22c3dcc65391d86517ead678ec33a5d901fbbadc0926", + "b4ccc21e5ea8003a945bf634d6b800fc44eee0aad713f12e57e1984c525a65f9", + "b714c7af15a1b9b8b710eb5c7611595076321e58f55c377e69c9121093522bcc", + "b8004aa52a284d2556c496f43d30dc6fb1cdd091695e1e50c1a90f0d65f92b09", + "b81b7e5043ba8065513ae1e47dd72ba75abd98d33e82a7ca7e78e051cbe43cf2", + "b8876098ea9a407eb70ae6ec08951a4f55579c2e0c08716a7c6cc43156beba4d", + "b9285dbe600a0d883bb1a8106ab84908e1daec78d16497c41f6d64424cb98313", + "b97e8fd40a8dbf94cb8e2011dac90a9480e3f112d431fc70ab75ac357644f8e2", + "b9b5303b9443950e596764e3ab43ba388dc88bd9e9fa4db7feef060ce896c62f", + "b9c2abe9f6ecb2b7b77d64e7191d9ed09a36554e9421caf1b2e65c5463365230", + "b9fefd5cdff2e6c28088cc0a53e346a2c097998df8b6fc297222153e1daf65df", + "bb2bbefd2a016c0ad8e9e3e8858cd69d918f0eca2138b4f2289c7afd8ac1f4d6", + "bb30b5791e4307912aa9adbb690455c076e533c6285272da8d12e78f51db283c", + "bbf53aba9adeea04e3f958a95feecb13ac2d29b45a69478899314a996c78e544", + "bc6192bc3b2826b4e1b0b4d086536b6565c66d2c8bd66378de084b53625943dc", + "bce154fde165db7793c519eb7cd7b398ff52e9cd42578142e5c8ee8fb0656547", + "bd1dc7793ae70f7d999d0e13a56b492cda6f3ab0de6301699294ad15b001ac7b", + "beb32919cb900c53a3953442a5d071436449a4b9ec5b2bce6056f3d0a1f87714", + "bf6f5ac70503dd191546b89ffc869bc3e28efae51bf05ec330535f6e5286e301", + "bfc5863c1a54513bc8256de6b55beb115b2fa384a58f23530fcf523dcbb52e1d", + "c0b1165b54a3e58bae71e154a65337aed1918c1d065439d9a3f9d65f7d1324a3", + "c10afbedc26936296433deb12073cbd34317fe0ab7899c916c8cfe4001642ac8", + "c14e6d6cb9f0b03afcc8cc196c3a29e3e9e2306becbe004eb2313e306cd0a74c", + "c17f9ccd9fd2ea3d7d9230a42289690c9decb88e0a880a42037a79e94b100e02", + "c309e10ae79aa8821732639fddbc5af8d7480dcab2d0316dd331ad859a03eb98", + "c390b96eed7402670db4562842a2aaa9581848dee1538a66faef88134835c045", + "c39690ec883cace98ebcccef51fe48fc75b323082a2c0ad951b40531383a9717", + "c403e6f15abb1ca9d51a883ecc88cfcee37d64058e19559d87cc8e9659369ddc", + "c628c9f4b7dd0a2931ccd0a68cdb70335c49d125adc341c75ced526e0b5d7439", + "c66503618c5c0194e190365bfdd6f18fc85f6e8b14f248c0724575689225af85", + "c669ba0197f300ec35c9cda7a239e8a69e73488459a625ab9476b09624287715", + "c9ab26239629a397da5a94679f176211063e4fabfd3c41a2481405a5f70b2e71", + "cae0fe58a9afff0750883faf68e9d4ed511d4adf9f799828e6c7c331d0b2733d", + "cb0f5ddc19e8db59d7affe16f76ef2585cc9a4c992ec4aa144bd25fbaae42eb0", + "cb42cf2a13ecee4f5c3585d182f1a5e0172acde5cdb5dca20df348c61d5707e5", + "cb7117229c57ca9d821d8ec24d4a922d8348a5fe3325939bf84dfb18f11eb5a9", + "cbde24769f686efa9b3c354173679e351c5ea0d76fadb3e69593a7ead04a1ee7", + "cc70ac956671e95234876cc83614c2f88db9110ecce9381fd0ee64803b299429", + "cd36b417bafdabd7721c88c17d2c29c33e9a11cc0594f52a54950906d4a44fbb", + "cd47f108a726fc44a0eaaf3d4bde604dbdf8c41861f2c9a64ef11c7a36c8d6a5", + "cd4b3470071fd954fce44c84264d4bc9e61790b37d2b5e9374efef01316e6dfc", + "cda1f2569cdb637d0550db64ff59e80ba7330792ac95484c37cb0e3d113eccf8", + "cdf79c0142aaa8c5353ce2d676a01a9865903051dc987c8b73cdd73c76ec9c23", + "ce43ed1b892a3a7b12e0bcadb397a918718cd979e9e0795395c986f56bc2a750", + "ce7c329ebd2e1b1d9bbc17a9b9b74bcd914df56e03745413b6dce38eb4772bcf", + "cf2039c1d46672c9f9a5515c3d60575e6be00fcb93c2cbc1477459ff50955ef5", + "cfc4f4161ce90cb06c135ded875919f4675c107e786fb6adaa9070e9fe56d34b", + "d0ac396c83185c55d031ae1fbfa33bbb017d14a316cabb55503b6fe772fdee36", + "d11329eb550769a78aa2d9f8be77708dd9ed7527d706dad9fa2ed230fa38fe5a", + "d140b4148ccabf6f483642b7ce61825956b6e0a84ca2bd32a08a98f84a200a0a", + "d152d1675da0f33cf86b1664536034d4c9f4ddfc901e51c018b87280284756df", + "d2d52cebb51643d7f8ad3b2e52987b7c7f68e66e172418fb42636767f927ffb8", + "d42152e189ac72bdfc7e818137ba41cb5265a008aa3cdcacfdc560f5af5341e0", + "d49d739f974de58a57bc4f74207ae2bc8d8e66361adeaf55e961a27aa1eaae65", + "d563f93f87d17243f00fe477d8e78157ae7edf218e5c88e941cde003bea36a6f", + "d5a907cbc19ff057e4dafd6cbd075d983f7e71085148ddff5920871bc2167575", + "d5e7f6b0de3d88f2eef5cb8276e5f45caa1fb6e902025627b04b490b5d98ade1", + "d65210aa0d78bb32b64e066dcb64b6f8f71f71779a5d499354ddfeab22f1332d", + "d6f1ed8d805236ccf9f98fd65500bb4aa9ad8c41f8a071ddc29ca8e752929ae7", + "d7ad91e423e48f64697d40ac5d829b3386ed420ea0fd9841676704f0a062dbb0", + "d87661fbf955f5f80d07effe80080355aaeddb42c6cb0409ebf62ecb5393b674", + "d90b629f237efb49c3ab6d0aff93c838b4db299e6d7feee7636508d12b5c5fdf", + "d911da1cc1aa1a02712d5b1a762cf0b5d5ab389492c1a4b0559299f05493d3ec", + "d99afab46c17b26a07d8f71fedfadaca28767a8f350a2d9589736ce7da6cc3bc", + "d9df49c2c7600b22a4c93bb4b1f9dc9b1157f0117b215b7d1c00e26770ed5582", + "dab08cc34ed2fcdd470d03536ca642ff4624b381eaf9d536195bbe17ebe9c17b", + "db064204080f8555787bb6221183aa561add66a3383b4ba439f45c6ee5a2d96b", + "dccbc6279f3222dbde7759e12a7dda491c55472378491e24482e4ba527959a06", + "dd5269687a6c1f75d0234677f460ecef52a58ff78ab2f9ca6822a1316b36c5c4", + "dd87ae7fff40771e69e0a6180d1305fc6d1d0ce432e9aced610fb3f49194e7c6", + "def0de3ee663dc1c9d1b8df1f0127da4720d8dc16adffce81c060c330dda769b", + "df1f96b7a4242177ecb40fbebf6ad7091106e10dcb7b8be68cdf62ae9532d7ad", + "dfaa421efe0204df2a51dd0f34cc72c3310aab26426b107e3973b954cebb730c", + "dffea2882ee92b26abce07a5376e46308bc5e7a44749f72858eadc0aaf862fa3", + "e0077b61aca56e8aa2a018e68311f0e6aefb5d030952537cb7eef012f41369a1", + "e031d45f43b36e3034b2966c9f75218e710ec72dcc2a5e970c1d4244e4913e31", + "e0de0c7672d9da979ef118f4983eb9ad6ef0402fc8e6f5935c75326e26231f68", + "e1b16c59c51132e39f175f624b134609dc6280d818f8c47d9b6eea77e7b193bf", + "e2679dd5d22d82f7095519aa46615e702dcdf209163df9cdb734d295f70ed8cf", + "e2fa296a21818b123e3d7f367826c5e997b1371c858beb73610b110ca8ce4839", + "e50781f70c95585a91804012aa888669166226a18d9fd8d104cf901f565bb067", + "e579dd010621e89c430a10b2996eb3b216029e121a77237418f7280ad0f5d088", + "e61eb554944b8e39f620309e492c5ac7361b46cd9706619aa516e1c87ab2f11e", + "e6ad5e3ada47a44524ddd841faee42a564c8b8fff2eb255362771dd9f3cd5a55", + "e76923fb355d375ab50ff13eeb1ffa782eaae18ccabfa627716742e9c1390653", + "e7afca6e54dddeefdb0e7f1b02cc0cf2abf00561ecc8086fb882ed8ced4928f9", + "e84d29a0132a44780d83de149ff7581d54b4bc7acba7241efd3758dc613e3cc4", + "e906c0d38c20ca1f640313df9905bb9a39b48d7fe15c57a2ab75f1565a191fe5", + "e9173b067eb363df353546f8567b60b5122ced586ee8c343ccbb3bda6ab73592", + "e9795c286533eb785b99b4ec8b8d78f7a1eb8bb725e0d7916cafbeee1ca8003a", + "ea4b1dc02232d972aefbc800b4156284095112baf01168056ddc24b8a247f512", + "ea7664f177fe6c9fea15443fbf74a78eaed5618ae0f37bf189a0b726d17a5d4f", + "eb3537c28c26f4f61dee000695de989b41525c029c7d92eaff101a6c58fbbfdb", + "ebba25c528191213226e54e837bb727c0dd9fb48ecfcbfce91412f7fddee51a5", + "ebd7b1abe0d22ca80aa8f56e1d62c8a38ff410749cc212d3da44eb580195e0f9", + "ed4e3c8d856aaa25e7adb8ef0ab63276a38f2de8d211ddff05b794283c206470", + "ed82a193afed8645087d6ca4289eb20157008afd83272002a5669d25cfc51400", + "ee15e6d1acea63bf964c93eee42be5dd3029058f8770dd9caf4d5337a2c01bd9", + "ee5d303b876c4dac62384687b039f9658fff97c84fd87b08ff59b99a50a2c6cf", + "ee623702315a339ba49bed2fe17471ae117b38e2e55d2469e7cdd4c8ab88565a", + "eeea07e1f63be94a7ee57285c6c517fd3ffefdb1f48e02d1318b7d0d204db5f2", + "ef727e08aa8670e19394b9f923ac237625629c01368ef4a89e9cb565aad5d2ff", + "efc515168087302956545ad66a1b2923d107e4142ad40dc14c43ebfbc769e247", + "f08cf650a03124cb4ff3d44fd4a70abe0342eb048dca6644323b335770a9a083", + "f1176ced474ef2f0ade0eea28d45b69b1c172cb97124465766a696984397688a", + "f2143c8de49b21841348de9bd75cac7ffb13b71d118249abb1917698450f638b", + "f28ef034d38ffdec076a212d91fa453834eeab284db013d48370f45b23ac1e1e", + "f2f034448bd3ed4bf77478619b5c901fcb70f6a8f3ecab9dbbaa143a6953d9d0", + "f4d897d2739e313ae9a7f4fab2d18e96b751d08263efa31b92c65c39b569a099", + "f5636406724c3560b13585d91d6a1b58716ad37a776edc88af285d7b05f70d76", + "f56756762723fb4248c7c9cbc4288058fd1bf9a5b92dddc39c7454bdac885124", + "f582d63ab10b23f5177f0615badaa3359f0a0f7a56fb4165437c0ccb3bcceccb", + "f661f3a1841fceb3f780a7b40257af20ed3e704c463eeecf783b2c749d7a916a", + "f6685d129fc3f57ba35b71ce97af43735c428fee02c6c50fca3b8dfcce32da38", + "f72fe1265f2add1e7058bae000d66be9f4ec18762d6413e0bc58548f043a59ab", + "f731dbd87c85ffe281c1d8740aa242fa9a7b1d7210e0b1ca69592d11f6f11f8e", + "f7d9e96925c19e11d9a4329b0ff009a383d758857a44287115687ddc7d098d93", + "f7f300fb09d9d92cf906384f012ebe09835fb0d03b902e2a230c001b302b9274", + "f8fac33ac4e76d8146004ac03838c9e4977f31db2e3226693233e411ae4c5cdc", + "f919f6e81d3e9733aa6e62846129ac10370ca19d3e72ff8594876e9e77334efa", + "fa49239daee14b0d44eed2285a78ae6d7ea97346a5f71a9439f394afac870c63", + "fb3c26f9e5d25c1414b4eab1076ffe73023df0950ccb8798d2b2359e7060ff69", + "fb7d518eeb067ac77240f19ac6cece874b2af904d69198f83fab2f6f7de203f9", + "fb9f9805796702942ff8ce23f4b5f2e2abcecba678a82eaa95eb2f35d3d37cc6", + "fc3e326d60df78352c6da66ac491963b7c457797ebcd1e96c2436594fe59a6c7", + "fc6bb05f32006e045e36a3300044ffd7205edd7a2ace65ddb7fc77df71abd6f6", + "fc96267aab2baa5d3ae73bb0adc76accb567533bcbf6a69ddf358b7ece8e51de", + "fd03ea8635c19ccd8b6496bccce844e72697d6a638a7fe75234441e839b7615c", + "fdbb4ed5157a5396e2fb421e74d4d4cf4bf3131cca117026ea1f51d77a857994", + "fe4209d418bd8745b7e03e0450b65aa434d749e4f0b0f3793a1849ebbe1f93cb", + "fe54a9ab1415ed784e1c9d3aaa035570a6cf495a610171edc863f5743524e7f6", + "fe578fe58231dea95dce9a9201b55208564b4a59479d910d2144111d16e809eb", + "fe62a651f7a3b22cfc8af16729766ce8b0dba209d70e68d12dea3c7ff7c5d3c9", + "fe895f4166f609e50dcbadd349f7534f86356f5dc12f47d6511059ddd8fcb4d1", + "feaa87eec12904d90fed4e3ae7cd9189e9adf521af0c660ccfc811e39398c52a", + "fee98f221151bb980cdb862306da346bf3b0d9cec0b9cbdd9c2ba13ef635201a", + "ff1366cc917bc6dcdc630665d80888e067a67153e0291eb502ce1c061598ceb9" + ], + "include_outcomes": true, + "label_counts": { + "negative": 5134, + "neutral": 1325, + "positive": 245 + }, + "leakage_contract": { + "deployment_must_not_read_outcomes": true, + "nontrain_outcomes_are_evaluator_only": true, + "train_split_only_for_retrieval": true + }, + "max_flat_action_dim": 112, + "num_groups_exported": 419, + "num_groups_scanned": 2873, + "num_groups_skipped_by_split": 2454, + "num_rows": 6704, + "object_embedding_content_hash": "411ee7d4c96458879f14fc9460c0e10d5309d83aacd5552c6bcbe089e81838bc", + "object_embedding_manifest": { + "dataset": "embeddings", + "dim": 64, + "extractor": "rgb_object_layout_v1", + "num_embeddings": 419, + "path": "object_embeddings_rgb_layout.npz", + "reads_outcomes": false + }, + "observation_embedding_manifest": { + "dataset": "embeddings", + "dim": 32, + "extractor": "rgb_jpeg_stats_v1", + "num_embeddings": 419, + "path": "observation_embeddings_rgb_stats.npz", + "reads_outcomes": false + }, + "retrieval_index_allowed": false, + "schema_version": 1, + "shard_content_hashes": { + "charts_00000.npz": "6f6ea9dd2f6809cbed119598842a1dd6baaf5db099587a2748a00264aafae4e8" + }, + "shards": [ + { + "num_rows": 6704, + "path": "charts_00000.npz", + "sha256": "538fac2468d1bde33b9e76f5233e331040a8c76ff5fd224727bd8140686f4071" + } + ], + "skip_counts": {}, + "split": "val", + "split_fractions": { + "test": 0.15, + "train": 0.7, + "val": 0.15 + }, + "split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381", + "split_policy": "stable-hash", + "split_seed": 0, + "state_hashes": [ + "00febbd1a7ad126d88b65aac55a35a9c4c63bc11e0f6e45344ddd4969242b950", + "0161713e96d5c5e1a45422cbbd0367210454a00f7d6e9840771090f4849fcb5b", + "0271bb52312dbcc4b57060d87dd5b3a514bb13bb1e20b235ad8204d61ad50a24", + "0544a922f6c9c662f2c9238466411bd1942870f7f182a508fd124141ef777c01", + "054fa57f5f9a67e3839a4c9674f9146dda13d27aa43c99ab5f87dedb1c0c2712", + "069933a0983331f68db2455b6a2b4fa216a97e119e1fb5b42a65adc252aca505", + "06a95069882817d6afbcaae8ec6bc12ed361c0a9446c56639880a8b9223c0c74", + "06d2eae8a8bb4c4503348959d65191f1f652edf6267e750861de79a6e0f1b1be", + "08353e08c94f126c924237211ac6e55a78461a8ff88eb8a1dbddbf844b6f2ec0", + "0905ba70e353c374ec760960c11b7a7914e3a0f401bf94a7772225dc39a94cc7", + "0a48350b21b9cbc27742e61775d9a3cd611daa5905f96a2cc3c59625d4b081ed", + "0b7be6179dacba118f3fdbe0e890ffc2d4ae1d15a87a215f41574bda0b24907a", + "0c4be6df74c82811154373ce6212cb0d7766cfe37020a987e1c1a553fc82aba4", + "0c5278be095a7a37efb891610364a48e4a4d84d044f63fab14e15e9351d2a9d8", + "0cab5c0c4fd5dd2183a8bc5891151fb5e5697bb605a5f7aa3c85f87f0503a804", + "0cf52a33f7d5f568029fdd8a8a3b634cdc7443ec30e9c127a73f25543ccd2c30", + "0d7cc8eda8b418bc8993e1fdf3fd64353a35fafd2501660fe5d87c6f7ef3d692", + "0d873ad4e7f5c34b9002a03754159bc9f2641e80bb46292920d135468141f582", + "0d8db9d71f59cad390736c77c29a6ee9f9f24de0c5250e1b8115164f4bc33f66", + "0dc29c4c945813a6553cd08d2dddb3982abec790f14fca7f70dfab255302960e", + "0dcfcf107cd9dfedc537d2da4969654c2c4fc3d04b1176f8e1426633edc8fae3", + "0dff2cb77a11d1888d2a48a76c8da04fe4e0daa54b5194bdb5ea100e2eb48ab2", + "0e23f36f5e66b1e4a3a7c074a6757653fecb8ebf1beebcd1e78033f5c2656a35", + "0e3910690cb8e214f63534f41f2acc8afec60992e30a2ed4d2fb61fcd93604a2", + "0edee0a0681248fffae8e5677c3658aa6a8b59d9eb3e33d90ea3027923b85216", + "0f0d99b44408bc284f59809b1266764117c2426549ba6b7e22b897e4c24cdfd1", + "0f19062a067ca2c041986c3de45c4e8d4e0f32838adcf5b52c29dac097394b47", + "1023ebe09fd0537cc30774636aa5c98411ae6ed1c2dc259a2624bbd063eaf23e", + "1113eef231093ad70c840dc75559d4ca74bd3d8dc4aaefa1c0b04354fe6776af", + "11f96b224bfcd397ee4f98e388350e3b65418b00a9ab4d2746128b42b12b43d5", + "158c57e861f5dbf4fafa3370948c03172a7354fd34511a04af24d744cca2d11e", + "15a9a5593a8b8ab430be4f23e1279949867e92ff352f3e2033c69495d6f636f1", + "16a1c922cc898f22123974a540e044a4ab649945807f078770480f7297e8e90c", + "16ddcd38b720cafaa913dab508ceefddc7620486bb42e9e3b0a9b81e7155543f", + "16e7bf765913827062088fe3aab96fcdad44b4615afb788a2a07e434d272b389", + "171fce66778284ce4d146eac1ca6327cab37d27bbc4be27a2a9fff473038245b", + "17547f1d97d7d63690d4f5ae56430531c2bcb8c2289abdea53e0151d4bd9b79d", + "18d69dcc611026a63a4602aa482606f68eb3ce416d10ba04f9fadd6b3622a991", + "1921b5d9fe877551d561618aa96a78ddc9480216ba450d537d5e4b27206c4999", + "19cb279a5068d0754add0249c6c2577e07f80aa33c56746853c796f5418fc7d8", + "1a43f6367e6d71af41f099f5216e60ec305d43486fe887af0c2f19849f406b75", + "1a5d3ae36c55dc60075de4acb795d4258bd60953aaca4786039280f763703323", + "1ad053ad6219459720e464e61ba1165cbbe5210a308e3f80962cc0e4376e38fd", + "1c665a2c6b863f8415c790f358f9138f56958247ed05638712d6357c208708da", + "1da1e71bbc20c3fee4c7494196a98750f0081b55357d61ee7cdc53945bb7965e", + "1de189fddeff2424f521f5026da163632e4417fac1f9336255f0e62614ee5bb2", + "1e6a33aa31538f769c69f26afd8c3fb7c5f0290023735688823d00e5e34f22db", + "1ed8594ffd73dc1116a58995bf378e36dbf065e9af2b003e643eb9933e7422dc", + "1f7544e2474c3c34d37aa4fd972618a6f29b1e84a054ebe0dffc58c8a2e7a71b", + "1faf9206f8f302ab744b3d89effef3a2a055b0e600c874ec05045043e1dfbfa7", + "20191c4fcda51e9e83b7e86251c8a03607d12daf307afbc00d51d85e4e7def84", + "2053f765d67e1e99fa2f2d67b7c8dbdc0edda26a58e6602fd99f9153ec1e826c", + "207798bd670f5ca13a6fe29e77f180432de0fe8e09512311636a9c49b6d27523", + "20cc15006255a0f0abb3e7f4e59313cbaee4bb4c70b8fda8bf24a8429d7bfeff", + "213e926b1edf4eb61706e283cd09447dbb4e7e529420a94918b6dcc6593f0b29", + "2144d1c886bb5f05a02f9f782ab4ef7451d251b76cf8774d23862aaedf517c32", + "2171371427e17c1995dad1d3526082f7b27faa7ee8f8d115d3e1f5a29fe1bba6", + "21736847d7537ba2fbee1d21f9e7c6f55dd5c9df2239977d889770ddf12484b2", + "225fc26bfe12be03306bfd0e7e321879c3e0515185c5941d4162e432fe6ec2a3", + "22af0dbc179d12978f8a50de641f892441c0fa6ff71eca94e9a7d4fb6d805446", + "232da4dad7749216ed41234b7490922221df235896962a2155fcb14e975705d9", + "23466412690e3c55214d93f7681b36635a8e1359f0ede7506e7c8801e12a3fb7", + "23b377bbdaf00ea187f5864295d60d63f998e9923abd69ee7308b5224d0e37f7", + "23c0c827d7f0d4742515d5227c112062f3eea5b8a374fae7b9f04a34b8a1ddf8", + "25438255a311c51c782bd1ca4591dbe9229bc3be0160d5ef35b016acb0850b61", + "256f6415a7e248845e28ac5e7d542da9867ae0f5a8765a3eae7809291bec1e3a", + "279cef5e2ebb26ddcc87c670075598b35cbc4366b39e9fd22c16533fd9f6f79a", + "28a15d04c873a079c1d06c470d2d77b8e96e025c30b859f176dba529fdcc514e", + "28e0ea4b46d79646462b01eaa2c675b5872098bba0bf3d66a75d1c1a750261ce", + "290401966fd54d78073a1984e84efd046b150ad5b91413e6f5bbcfd35a27c19a", + "294c656a8b36af0bf9249985a8fdac65c4530e7c81d78a0b226f721ad6f9d2fe", + "2a018927dc86ad3fbecca897ae6ec63485d334434829d3670fe94776f8444bea", + "2aba3b30872a625774dc16ad32767e78fcc9e0a1e8e800f976967682c2fabf89", + "2b68052385906b5436a650c00b9c3ffe0253943c3a86a40187fb964d4fb2870c", + "2b7e6e58aaa56a83b488a80cf01f40bfd6fbd5941855bb29c2ce21221ef78374", + "2bc8e606bc7f0a2875ad9a588f3e7a0aa32df858fe5542ff8f7c102cc9c9f14a", + "2d0fab9e61b78cd62d1ca7d76326481635d04d61812c74ab9cbcc2b0e0a3ab1a", + "2d666e1b77c3039e2574d2c71967860a070f2166462f049a3245013d8bb9ec71", + "2d71b5cd66d133bb09dab6f0f35dd703104df4b77255a9ee1a8315d1ca6f59d7", + "2e247e7348b1c46bfa9b0068a40df1f45a4ab4d9c7ac366e14d0a9f99e7d63ab", + "2e7618499962df0d0c566b843a23f7f4fb31301bc9cbab802d427206f496cce8", + "2e89f06988ec2b92a1c8b18b2b3697ee0404bde77298adca7ce981dc1b1a7417", + "2edbff51b1020474d5e181f95bfd43c6fbfed555f6b5d5687917c0817d4ba0e5", + "2f08ab47ee8e608031042183ba73f76856f60fb0262a8adabbdd29700e91da9d", + "30928509ef611054e0fec9b07ce84529aa3afb560aab8a0afad741c493c3efc8", + "312adebbcbcb2b79ef60eb52a89d93e0dbced13464e3bf66ecfbec402be9c7af", + "32bb5ed54dbb72237062117185d38803a2821325ae9186773df5dbd2ee73155e", + "3301879024b9c79869a02253edec7b100d7677fcb2023acd6e9ae7c9cf46308f", + "333fc8cfc5f76fd98905846b73548b0023cac4f5f10cf49d3ea5efd8e0c15fff", + "3376c21147b72dc0474a3899c9eb74d175f9b78d228913dd7c3a27bc18f6af3b", + "34a2ee666ee8be188582c8f7c239b6ff96df48562fe53f34fbc9006c61b87b23", + "35f48e0a0b32a3966c94cbbb83fac82d9d6c89b6ae9194c446e085b265ef05b3", + "36178548038fbcdee704bc77af6a77590b37e6a3915480091ac1d77d9fe92a59", + "3622d0ee6c6d00a746bda15fb39d7044c31bd30e627a4a7ba5737477cc2c788f", + "3655a761324c8dc8844e8ba8b9a4772e2453ea55f7c1520925ef154dfd43df4a", + "36c71852489c6888128d3ff5c2e62d599759c400acb473bcc0a06b2f9acf066b", + "36f646f44b39a2775ef7f08cdd6256eb68b045ff9d2ef373e8826165c2fbe98d", + "36fa82aa1f5bee9ced8fb9c2bde5246529f5735b6aef2abfa3398721a222afcd", + "3718724484b8054be4ba240b10122a71068067812d05e7b9963932051445168b", + "371f27874971dd4d10050be4bbbc1e38e9afe1d8bdf5de5bbc6b3b480334e7ce", + "37487957af8670032f1ebd2698dffcfc99b5e2931094bbce10a1f5a41d17e5b6", + "376260250744115f58dd61526c1195fa32865622d2bf0daab219ef3491f00a8a", + "377d211e4bed8b5fe1de5384be5ba8ac62161b150784f0e7d2b6dc3568e4d508", + "37fdb61efd2db8d7ee146b537977800e6c42f93a35f1b452a8a76ae50aa2e1a5", + "3802165b910c46febd084599f3ad3bb68e7f94946aa316b3be4077dc64d6892c", + "380464d744f9af313aefefee8de18cddcbd32aa3e09aaf4ae13b60c0cdf2c535", + "385b3e8838fe0639b71bdb89d173cde82ce32e8218e8a6900f7d544855d3402c", + "3864afe56d299c4e93a7b4b270e10fb22a03024afefbbc4676456fb2bf09ee2a", + "386a215a2dedbc0166ce9e574929a13416acf7edf161c0c281a7d3752c1664f9", + "399ab40ccb7266269c21475263d0e7264b2b983c333db7aa09ca0c3ff532101e", + "39e3ae6e36f8d2befbcd1d844052de9ec49b87d041c1152f23ce51b5fe4a7365", + "39fc414f3d1216defb30c0764ca78e922de5fae956c8f6fcc0c6cd3ff3c03c5d", + "3a3c4ce053b1d397bbba192d6cfcf15b8c08f6e43f2286971a9321c4a1b9f783", + "3add4fdf06b87cfe5ea9fbb1f1a6fdb2b2f53181cb4f4819abaa511222c728a4", + "3b1c9478132c0351de3a4efd39668453b42aa1eb060e7f6a1def9379b74c4012", + "3b76ae93b0a9d50a284a8556fe33d6bd5255ea98d1c84822d166a4e5a2380a90", + "3c8e3b6a6980d85319a45619c74f75689a10d57455f60ad363b0db859fe33cb4", + "3e44df029d4120641875fb004a2a937c49fe4779a7ce5b1c15df1c7c1e3393a0", + "3e7dbcf1d6545bf86b069dd68aa26c7b2d4deaf965dca2f37a2c7b327ec007a8", + "3f27b12be9a2fd631200a9fbd83b29deecde1e760ec54035723fe0793de1e445", + "4136b9be1dde7532b54d1ce1f96598e153c15c3a76ec32b95813cc38b070b954", + "41498d0adc8e023aca47897192364c2ce5745b089f33013b8b9deb69e24b942a", + "4167b16627d6e167a2dbd1be8484490ec0c6da9de79b106634ad88b9b65e93b9", + "4171cf812123e2b6f87fb20b97a1992b79181a242809805cb68c7ec70913dd6e", + "41919df4cca85f0b531324cfffd6584be4830bc8bd9a988fdf7881d30a8dde56", + "4236092dcd909a41dc844bb5007d916aaf2142ba0c7332fbf96947b5b8ee61b1", + "426d33af3a135919ef5d2b06dabf99d1a1c15ed3b1bb2b9e4c2978a5a107b0cf", + "4372ac740d1727b2385f90171db7c748a03a6fa3849940c98f4b9754a7b29d43", + "442548c3a3f961d677d15b34974fe1729a5c922853972634cd4ef9a66a130b3e", + "446f95aee72b7659374dd227a9ffe3150e198e66df6434c3b4d8c9e563c35ffd", + "449d06ce2a53a15c8070a0555ccfe750c1f25fed1317c69aa040aa181a24353b", + "454c2f13e99ab69415a54b5e6b0d4d5f934892e1b5083e568be81894e1ca2a9b", + "464f92220100eb1bbd1b22e9d03626667f349d7ea6f74c23f1bcef7fc88d5ba0", + "47a9c4f96c6872687900bf48c444e916d883ae0147879a5136565d9f781aaf1e", + "4843e199e6ede40f1655b18465b9caba2d2d1afb569e977520703101f30088d7", + "49417479651902941b3e22068b8f19676e4f484938e9068df2d418739e62cc18", + "4a6cab077475699ab9b8df7a8e6207d267d0d2358da13cf6493010703c398c6c", + "4aa88d7007545b593c65410ddba0b240f986229a5b3eaf101d52da62d1d145f1", + "4ab7c6c04df7d5502cd7a523e550cff7e0e5cd589b392491a7331b95ee9da323", + "4b81a042ffa2df32f8f83b5604e4a00759eda66d89a404a2558eafba6d4fee6c", + "4bf18fcd7c9afb43f6ae51b4dc792f1c53af66240e23b0fccb23b7c3d8aebc1c", + "4c03b46384e7d52ff10e415873ecfa8c5c15e6a35dcad5767afed802c3d1200c", + "4c9e5cab65487fb267c99ba75713cfb3a2a030f49e6cfcfb4f49ba84f398714b", + "4d0131e19ef58b57be98408e41c9a58aa60c41ab4dd830f587cf209bd3972720", + "4d25e2d4328ed58c8f62e38711768a25b57fdc929210950785171937151217ba", + "4e5b04961304eba5fdde7a3060e1a8a98a4a72f1f260a71090f499a79769aa8b", + "4fbaa3eadac6794630ca6e6754e7f252387e419761876d548179007e6e70bdb5", + "5079ae092765e2fb5b517d123bd809195e2b2d26b55db1509aeaeefaac437113", + "507eb82aa5d9d9c694ec304b6dfc7e339efd292c0f36d8ad69de93022801fd97", + "50f8c0bafafcc8b5c0908d4a4ce21c87bf5b8dc7524cf3513aec538e4efde930", + "52bc6084f0ae006eb0c52083a34981493ce7918c8d02d4b5467b92a16cad52a9", + "52cf2ba1f84ccfbf1812ba17714ffb951d5a5503465df2fb7e14fd81930c82b9", + "53e0f8565233b4daba347cd65931b0d03d3be14b00b81f53eb19e8732f7fad75", + "54276b11b5dc801246bf871ed20acea86c35bc3e48d65668292c54894cda707d", + "5573e534e62463125b164ba4d53d7ff101f2040bc3e43ea9f8e66e8a78a66432", + "55c17cc62f79dcc544b49163b3e763a294a0b661482408b35ec48b2afc4eba4e", + "5797212ef801c231eeb5e6157caeb769c9f1c954c945c6e41f6b1de27fe6ea5a", + "58157eae91b262404e5660758e41c875ce3d2823b2e4e5fc30e8a3e1384561ae", + "5842b9664210226005965140e501df8cfd1193dc1def74032e3fb54e6b9217ef", + "590d891c164938e5e47d3dd069e1c81dced1f8a6e52a3020aa35ddce011f6dfb", + "599decd9be71c7b255562b1d9623c90819211305e579d8ad4a3af74c5ce8c4d6", + "59f04a0707f4139930d13192eb42ef9cf0a5193fa98f46e64ec927d3bd34f662", + "5a53a1e8f981c816e78f88933c5d2cf7aba4d9a9a28e953315d817abf3e937cf", + "5a8eec221fecdba8dfeba990ba2ef824a40da43c4167f70dac07ffd3335e8fbf", + "5b231a214ee5db29e6857bfe09c6a36d9dbd7dd2b85aa25528d47902099bd8c1", + "5c670f5657e3227b5ec3bc5199ed81e653deb046b4f6fb539c40281da5887418", + "5d8536a1f3f9d4054766e67598a6212a995f62d36f57fd8241d4432b800d429b", + "5d95de9ba01f727e8fad9c3476b8010bc5753d93b9f775ec94e307aeaab0488d", + "5dd35ce96ea763c051ec4c81442ea8b847f0d357a99fd385ca5b09e9cce36a22", + "5e1aee289d761d9dc1711bc6b3c97c0eeddc16b630c815839a156e42e7f39195", + "5e4eff8ba21f97afac6f8d2f6467b6e285e21dba4b21c6bc0f58d2f086edc2ab", + "60a111176ae35f70f9585afa440fe7f3d18977376c267480fa3094c3393adf62", + "60c84890cf7803f6ae595c3566b6aa4176405f2a8ba83c0f6cd6e000247ca271", + "60f630ba7d00adbbbe73b85429e665bd3c041464321cb346d27774365cab4682", + "61aa27300c1a31db1051074f03440d13ac491b0a5577f0b3de363e5a3b189bd9", + "6206882c83f707aa1f3259cd8cbc1f890f45bc4a85124cc602dc3318eeb6d3d8", + "635c92d494d7796b0c2f484147287f53a89a765e464d054cafb688b9d96655cc", + "6361a57a00da67d6102113b3f3ba0bbb7634c08cbd2096db4289f234380f6778", + "657d6b2557b1397c99b24a9e5777a3e28bb62b4a942ccf9da164c08e6a655e0d", + "65b892cbc5957bc2cc57375a544114a6475b8d9c274b5204d6e0d9c1b9cae15e", + "65de868c437f8dcddad0e37f6533940fde34b39148b9f63104f41d8145610798", + "65fc6b04f85a036c86076adfc04b741aedfe77a79a90db37b4023174ed4604da", + "6728ff8f1d45daa81b955edc59625766ed7b9e5b26f65edeb3dcd646675cdda1", + "69a10964b21da0fecd5c463c6816f544cb1efb299cfcd7a50a4b3a8ae59a9e0d", + "69b1edf50126e222eedc15ef65e9aef983fb9d340ccb13cb2740f509cb7019a9", + "69b58c84600892b4fa2cef6512cc93f04e14415058a2e2275bfdbb6ffece1495", + "6b71c1274c1625f9d19ef6d1f7044365d9dbff6bb252a6445648ff98c073baed", + "6ba83bb33eb0852b8a304b335ab26c5716a00bafeb0b1619146263c25ee0966c", + "6d4eb328e7293fa796088ce8a44db34d6e177e36d4bbb103f369859a08842c5b", + "6d9ecd81dd1ed4c5e6fc7bcec4e40d9097f9f818422e122794aaddfe260780eb", + "6dab1cd77a4ef3a9aee45a8537f58ab30d455d55700428a5ffab4022c2ee9d7d", + "6e29a3f512c9a33928ddfaeb68dae4c9ff946678744704d45ee053726b09e5e2", + "6e2d61579abd8f5acab73369a191a425e369300e3de717d6b5e8b1c79cfb2fae", + "6e6db234d66d1aab39325523447331e08d1255a7d81c67c10348b3abb03f24b3", + "6eea19a0e64ae3f4ec5dc0e5178341635ff858ec9c2f935976c054dafcdc05e4", + "6f76d0be63f1094f195d03f468ab82e36ae3f5f846d96d70812668b5d8e2f7db", + "6fcf3251ef3b9a95d9d2917e3604c3e2c1c05d2f4004fbe0686920abf3574575", + "71415a4b8c2ece02ddf3e9f1abe357c30c11029627c5510ec937582f36eb2939", + "71de47b052ee81350c5ad7224bc7dcbb5be7c4fd375f0821a479c31d82c9bdbc", + "727ca5e770a26fb5fa53155067cd05cc54bec3392d6a1856765f966ca660f9a4", + "72a6b851451517c6bdda780093ffb0e1f713db1580fb466d1dd390c2603f1d23", + "73e64ca24a747c2550b8a82dd413d5fe30e26f03fa95771d660c4b21561b7877", + "7498d1733ee3bfc7e7cdb11d6ad5a65369420da5ed5db9c4d752ae710f7e6e00", + "74e57ea458f6a044861dd23b053a31372df73e0b47e139b1c30d126edf775ee3", + "761134143d39da9361ac778266206266d961c8a76d60b063ff66903a0c51bb87", + "77e531ba2e0fdfb9d13b55ac7c023bec7acdbc08a87322fbec534cdc5020410a", + "77fe4e7237e2a146bc81019079776768d43ca9d55e19baf25f6db7ead3cb3fa7", + "7a56f9770759b68564c8bdbb38baa964b4acce93a3687488b18bf4a82561be32", + "7c419f28cac41a890964171a7ede35e4df22f5166d04b34ba2f1f5c530e895cb", + "7cb35abb83b346ffac1f0a58212082119a81f64f5cceedbd7b8af7810f2ebc07", + "7cfc2744e8e2c90dec0a973312a07f72b33ccff72595d6531f66c31ed099c6e1", + "7eb1f194992237cecaa642db7c5140b22a58ccf39c3aea4a21afb463b58b728f", + "7f60c45a4a714bf1066d53bf7d2b72abac5194fa5979d3aa68b93b64c9facecc", + "8243a06a68ff61d79c805f2c45bcd25fea70bea8fad91df198b432ab470ce9f8", + "8350ee6948c386f427a68e9cce71b828dda126548c1704c5aca0d040de3e1ec5", + "839775ebcdb6e011391fc3c3a66abd24907fcccd0d2c939b5cb390a65fb76f07", + "83d122ef72fc3ac252eefea7659e06a26f887d24374c04a5d750299f76aaed7c", + "843281e77ecc5a5210dbcd0b0b98edf7b95537fb6cdacbfb74fea2f8b11d8a51", + "85044cb38abf6ea983bce7170966ab3062fcdf01b28e25048481770a3418b82d", + "852cb696bf9bcb0a2c8c25bb5d111528a92006f3345eb8a47b3eef0efa343d72", + "85654a7b2d601c501d06ccec3bdcba969240c6862c59a34fa4ee9e3216a6f978", + "85797e1a9396533183d3551f5f62e960350b2cc8692881077242461722204027", + "86f1519103b9098184c2719a2553fcb4f24abf9635e7db4f4e09808b8af84f2c", + "86fde29349d8159afd645bd5b7c5556bf74a7563f5d27f5252cce6523681f6c8", + "87100034cff240e23384afe44efb0b09e90603f382896bd58c186faf0fe4aeb5", + "8751179f0094283c5cf1ddede063340f3a84784459a23264adc86421baf930a9", + "87d27f35acbb6f4734a61bf3a13edbe579fbb5d4ccf246ab2378e24b7558253b", + "88b7ccfb5fa2d3d4bec816f96e0a2dc5a15ce4ea602df297725a58dcb0806732", + "896df52c27a5c01769e1e6e9fbfbb795f43e7de78ac34f34a40bb37c147de197", + "896e19fbf4b3372093f28200f82ff64bedf1d38b1f7fe349d998496342162082", + "89e41d06c9b85a5e85183168ae099df4f324852eed8ec1314e347767563fea96", + "8a69885fdc86627e9cf7ec2880a8c8348b0cd02df31962c0c4d2c389a1a066df", + "8a8bdd653602f2f158019a5f439e3a899cf0a85921aa14cca082d6e603560909", + "8ae4bf13a85c96359e899a85b465784a6bdc84f8f36315f706c34d955beabd29", + "8b6ae722b7977d64014fa0af8f179df23ff208fd2bfae650d6c2277dde6979a3", + "8bc2f96c2f5cfec2923d02f631b6adb72273f0596f19d1de7b39d20acf0b6b12", + "8bc5b83371b1c90418b00374bfd06f8db036885a988bd6833ab0c06fb1ea98df", + "8bca1c6617cbbfd5ce3cfa106987cf26130f8539c26044e1a22a2f51fb1aca1f", + "8c05750c1ca939baf32d25d1f2d92a61800b49a76426ec28c1559d59606a61eb", + "8cb8ad276815c315802c4d97c5820cee5b1077c684875ebebd549432c2c4a812", + "8d091dc0975a50221f3b19a0569128ca17b1a52d01fe047e380b0aaf0e63b52d", + "8d314a3c1bf7b6c300dfecf79320d58bf6d2c23fb174f92d5c64d5e8542a96d7", + "8fd961b8dda83744a40dc807e6d7d2a3ea9d5d1bd799c1e1d5b2fefa666d6ec4", + "8fe2c0bd9c12919205e8c8b29ea61bd96d89616e9b7498b2ee8013446d5dd10f", + "902b91e3a5080e9f94da80889ceaa51c5f31d964846008622b6ddbeb07e28392", + "90e57c604e72df20a9694ebcd97e3fd9f00c5e5d8ec239e84a7b8a0c431e66f8", + "92928d4b346d02eac80d60c98e94f9f1b113f7d01134383db932754c4c5bd3d1", + "9393b510d59ccd16774e17789b3a01f90a64554dffdb9eac197d050c0bf5d09a", + "954890c63eef6f575f68e4f37adcaf21f4a2e343bb291d0b5585d0ea6d46a6b3", + "954c283eebad88edcd070cdb36ab55e457424f3fcb9210f59386cd3e4ee4e74f", + "956fbfd3ce94d4619ffdd9194deb85dc6d82e89fa199cc978e0cfca1d3e8ad6b", + "9588b1651ce73d10cae6b77ac6b991868c806a63979dc3b4be971f686e9723b8", + "95af0aec0162909a0addf5f0002455abb2bd61577e28562a5ed00c2260cb8fbe", + "964083a52374c2723dcb95bfed5636587a298497bf58e1183b60af055c46f397", + "96909c33a461e46a0d938d888e4371d1295f04535166bc16c9dfbb35c9d552da", + "9bd4c58f5a4d057257cae54e018f4eb88f9816f3cf7f88949345453a70512ff9", + "9c674f43e5ec1a37fb7eeb8264a5b4d785f17f716f1897c41616ed37333ba70a", + "9c6faadf417db194a1a2e68cfc6a4792339ca759edfe44c747fb4c516d7a66c2", + "9ca9acceac4275ff088b9bcd89d78019a7f71f284365c0681882194a8af11390", + "9ceaf7a3ba12f73c742a6f86a81c708c03240515b9cc5b89cae86ec4601a744e", + "9d8ede4ea688ed3e8334867c8f34e34e5c070ac993ae9bcf04489e7a25aec07c", + "9dee4ee61296b82b98e8972d47e802ee8ac8a5008b5e03cd5b130d0edfa15c77", + "9ef46448570d1e27b98284d64a08cec2d25a8118fad8017d44a99f4f6c061c47", + "9f1bd954340e3d26d52465ccbcd4b72baa30a3ce520e2a6e4358b5617b176963", + "9f585ccfb9a92e299ff0f76a9a777b4c5eeec4189f4fa9ab50c1cf3d74c21e05", + "a17f4334476d179113e2db7a6dbff71ecae86b10dfb2ad7fe2d8cc9135b4f2a5", + "a3170ffc83f7e0cc197350a8d74d85500d474673998ef1c62f1ddc3f364eb53e", + "a31eca43deeec05441c35933fc20b48f45b191042da018d7f6d97232c1ed9ce3", + "a322d6d1f0b71546ad4e9f746bc3fbf1742df0ecc431dca06584e44f29328e43", + "a35f0920ead6068375bb19cceff473cc971e9b578a0bc963fe7e617e2fdea9d3", + "a43f30a7fdf9c7a1561c6eb9aaca66890cdec58295265ab8d305ff076e36e0a8", + "a467e1194c7dc803eec26bc9535be71b58444bb5748d179e2a4e5e0662d3e0de", + "a49644cf53180202b00609f3cddc005a01e2086bb63369c711ff0950c8f8c214", + "a4b7c79b9897333ce6ceb761dc6854a40225ee9211a54101ba64dcbc92e0a6fe", + "a52945e1d0442f75f9a004ecb76d0e2b17e6137b124e73953d23c99aefa7749b", + "a52d52c4694251b5798e3478578d4d39fe1e964a70548f6388cd3c5e5cdb0fa7", + "a53fbb7223f7fdf1f7b641c51f0f8005ec0f32cfce74d9276001d8d8ea1151dd", + "a59577e4bbf52734f1931725d54881da6d66231bc9fe4c238d840b8c5444fa4b", + "a59ba23421ebc70cc23935fec64df1f948ef7e38c9ae6ae5d51491aea195046b", + "a939547a17e2245da625a539382110eaadcb4aa5b5e0821c15d6ae3a017f55e4", + "aa1da523736c9af8a5c7af126aef2900c7e06c0ba501113ecaed22f8982f7cda", + "aac283530a6c5a5e222a951421b18af4933a1f893ddc009d2e500340dd5d5abc", + "ab4a571a4eae6c30751bdee0aed400dc75363f94dc0f4b7782cb4556e83c23f3", + "aca0c3bd9fdcf931da31abcc3e4ec560120c83ab86deee1e627b5fe8e06b21b6", + "aef1e30af94b53299662a893e7d3a08099e1a9e2431e39096bd55d3eefc3953c", + "af28f08a766c0dfa0650194c27fbfb2b9b057e676e959dcde8c0dc04ba0372a6", + "af680a60c99d6ed6f75252add7208bbac1f27c4263be29008866b314963f9933", + "af8587819a0c45ae5a25878cc8bedc571a5a14d719ff7ad25bd3b088af294b3b", + "afb57d77ba6a109f6435059dac6617930c19567b3a9a5c690932e04ef7b7d45a", + "aff6e17d88bd7ef94395eaa52ed7f8bfa1a84a322d28c1535cdbaeff3fe137c8", + "b015523bce4598e2b2d077d075a5eed6fae4235b2851862ec3edf2a2a454b89b", + "b018d81f318adafd8aeaf9997b18e1e23da168b5fc28e640cf50d570dcc9e315", + "b10f856b83f430458887304c2f3609fc5bf70132d9546fbbf763183d9c8ec5b6", + "b12d970acf0fb01d206f2547177e9c4008abb0d81d20a8df2faf566176cb042f", + "b1daedac8c8e33c5708b2cb66bd406e57633cf05b6bbd95c11f9e752b35da4b5", + "b246439be3988b28f0fee9316443e3a92a37bf97ce2be5c748f04ddf800418ff", + "b3ae5672ebbcf36d12467c9fefcd760018e4b5733a34c41bce09e760bfaa9715", + "b50ce258f9c2c3a1b1284b7a573a2fa4a11ebb787929d39011a47002463ea7df", + "b50fca44807c252ef01dd95698262033154f8707905f779c0a20abc6439ca0d9", + "b5a7e7146a98035523ef6490183cdde177444103df52e1217bf8e72ada4d6328", + "b6324c2d34799565940dcccdd23e1360e974a46b44c209a6a7a9fdabf790d044", + "b66654f95bae5b30f1a326c90dc930b954368834385eaa21725571afe94b354d", + "b677f0aee329cf1b061cc0e2601a9c33218c5c10c0cace5820236313a0825d3e", + "b679de5b8df5d6b74ce1c8331418e05575117dfb8903a5381b6e92d9eb0b6a4a", + "b68d92aa8fccb4f75dc0ad09e9b6d567f7d40d421ae5321b36bb0a48236e598e", + "b72fa8f582c7fcc689962e8ffd5f8fe3f837aba4cd11ce7687532498c11fede9", + "b7761c8e81ab553d5b43ef3e570e6639d8d4fd3d3b217198f7bcc86b0078db4d", + "b7f2c74cc6970aa943740ba5c7fa947e4dc991233fd35e51412ced790d20298d", + "b8c14ae75cbc0d0dcb2d256815e6e4fab110ea29a3582201c0df5cebe47544d5", + "b8c87db8cf709cb8f4bbd8e684279f796def8980d809fd174a79690113de7207", + "bac19e11a6db7315a718dcf8236ca104336de292ca5553afe82f9418239b4271", + "bae7c6cd0a3c6e5f44834d64879b260dece1a4232f3fcb1f01756b03391c8831", + "bb5d7e079e9ab21431f60dd6c4495a3984001039620203c0962965e5f42de079", + "bbd5c9fa1463d0d789c66b421b274e424b1289de879bc2095a39924521dcdf06", + "bc107f5bfc824a5872fb29eb153fe68133664f84a0bda0c1f040629cc5722e55", + "bc28efd0ad227d98655ebf16a86d39e0eeca17d2f7e4a1bb14562b9772777332", + "bd570017cc62932d88f56095bb4fc99aa0d3f718317c089808b4649309f57322", + "bed8189cf521818dded6793b749c12a91df30fa033b1d0d628b74b078aa3d17f", + "bef655041d427966b37d503ec3ee6a608a397e34e50b3faf19c7c8010d32e9e9", + "c0b19497f20c96ff5a998cdfd8fe901b2636548e67f7521e17d8c1c3a344b119", + "c1fe0dcfc510b321f89d86e9e28d959be4b9f7191bbdfe75d94aa00ba907fecf", + "c29fa01c7d7c2300dc9f248122d101f2ae7096dbdff573efbb51b271db18a237", + "c2c4d1a16f5c13da6ead578d5ea39e79319c2fbe4b950d9a9eef447f0d885985", + "c3207049b4fe55b2130b223f9d327b21f11dc51c5ca20eca2a8d75c13f6000e5", + "c4c4cd450d60c5748c8fc36d10c75f182cfe952711617dacb76a0b0e7391a35b", + "c53776402fe96a74a9aa02b2ae732f279a3ed258aebd35e7f7347da28491856d", + "c5608d269923a2a3ee69d360c52f71f1eea591a603304c908ddb465f737c0782", + "c56cf2d189daaefc764ff1bbcf667d4625be73dbcdfe9c42e0f503d59b5ccd23", + "c60a678a37f48d744f7ba785c4adc7de65317982199801ebd603090315c85ca6", + "c6244a7b5686ad0e553cd0fb349481c14e4390ec7a65d414da7dead452a535d2", + "c6dba1af7f4d12fe2720b67f0ba6ce4984cf9d47c6d288f2467681d02f5af6d4", + "c7418675ff86e0833d60ea80a8c8a0f9b99d239421ca68164c6824a99a276f54", + "c77af3500dc3ee06cb5dd5ef1e7cffd70a3c7bca08b9c00196884cc3dae8db90", + "c9feb7939ea3febccdec91c170527060563ee0dbf0dac5b75092ef5c5bcc9e6f", + "ca67eca42b424c6374979d63df4eeae00381446c7c7eea0934a3a21b9352cd9c", + "ca83a160910b077d3ca4b6f0185b4a99549d8287d187a4c204ffcb6867514887", + "cbddadeb108dc36a2cf5e686e2cb3a143946afe47ceb15dcc185af4b272e3777", + "cc47d275990836af665da5f4dcb0c118d3d4897d376ebeac53ba6f207dbb8a64", + "cd015dbf90e521f2443eb714d30785c1d21240e44dc10e1f8464ff146866f6d1", + "cd7d0bcd5fbfa262877644a5d88d8e2b6016e6cb9a880761c1cc1823c4e6c621", + "cdd8100bc160c8c866913e8a033cce5f402efb1fc1538247d4cbcda3e1216659", + "cdef8f05ddad0db32cc966def25db6898a05c42bd15391b77c619eaf5ab2a904", + "cf4dd171afff5c96a512d37c25ea9e292f1cca25a2d267b2aef735d51605de17", + "cf99daf73630fb1d3448b7bbf0b4ba0558c886dd8ae1fd77a38a0b610fa1ea3a", + "d0a9e8772c316ac795de39d315b5ce38b72472b4df16d6ea6ae473c94c853a0b", + "d0c4b2351028946098f41c3fe28981d2aee8e3f5caf4ba4fcb12462d0c2fa12c", + "d0d11e3470945260027f6fefa6a4d0a3cbd29d2ecb9b9420a14ff6fa952c957d", + "d0f6f9c2fb93b869433b409b0093f0e15e226e6d750632a90a0e6e970a3464b6", + "d1ac7a2471d394057a588ec47c74a0b78e7cc2f495bb1585b184eb72dfd4ccce", + "d2615c76b309b1437f0d1f0294b8e88fc5fe3e9259a41a9ae4cdf137ef73609f", + "d3723031d231e59cd40ba8632e01ec7604f5197a48a45898cd0e314541331c87", + "d440f8f0db5b04f2ada14de102a3b3a7e3e774ab487a6bde0275a4344d62dd16", + "d584d77a56c70ad17c27a192efd074d4f5f74452f2c62ebae6b627d80ae063f6", + "d5b1bf93014f200928b481640c3ae57597cf73a0f1668a77e07ddd8683a81508", + "d652ef33b9ffeeff77ae17aeeea979ab8c1d403431bdf245867abf4566b54689", + "d66bff5b50f3ca994a03faa77246bf9e3a8193a4903fec25d5f56f7fee31611a", + "d66e373e83c0d5bf391ef83eb47b2ea4971409df16bf2d2d3f90e4819c641cc6", + "d723c5e330147c034be5d384e3c539f323c50b01326b901b3f78ec8a29d13b41", + "d842c9b772dbd601bfad1c3d830df5aacbb685507247666a80d8ef7ec0ab4db2", + "d92d42c9d643429671fe1f7384014ca15ae89ce49be0c5d524d2172938585986", + "d9efd2aa3c0a57f3bb93916653cfee963e4b1141917d703e61509d8d6dab7c78", + "da5e94dc767915969e1c8188d29035197a22ba0b9d8be1cebc04e5a8203646a0", + "da96925490c07aa86ee90d9b7c31bd59fdc56d27c891cb6728059322add4260b", + "db258468bee6ca4d5ca59334c4e45ca958aa6df89ceadebead6c04ec77feca35", + "dd2f29960e09d45097d7c7324130b96c27ee8483c4dca1b0045dd58a158b4838", + "dd794a7a3fcf1fe4a062e9664df3eecc6f45f0f15bb1672fe552f117acc1f646", + "dd97f6d3742257341118baa40d95ac216a80d20d39d47ec9c17e37234151d783", + "ddcb6f80de7e1ee656dabcf2566d3a78fb97806bbb6077a66d0b1e9d8a3e251c", + "de0beb18c2c3a2c895ca5f7a01665041577ef8373f8037591a1b1bd9b906e13a", + "dedc28fe961870e02089cb419fe3be918a27dd6568af8724f7e6879f749cd029", + "df67a5c802fc30ab72aeababbb0e93dd8e5b42cd9646ebc54ff9f47240c6f361", + "dfaa69ca87d3288942dcaf896c9ba8b8a6b0ba61006631efac7e521e8b543c9f", + "e128aa827a20a9ca1a057cf1991d49872999d78244f9f9d126399e8aa70b4841", + "e170828ddffedb42c27cf62dc347fca5ebcde643e3b4585691189c321cfda9c4", + "e1bb15a4b9aa87ea3327bde96322b226d9e407fee96824e7281a4142d9c32592", + "e1c5715b0a722ae7aa244fc75ea30dcfa239f6af640e6ba5d4937d22c85cb982", + "e24cb331e947114a105313fec3abee7bba02e65792962b11f789ff65354bca55", + "e34faf75bfbb6ec5d6fea460d34a300f33d61d5544249ce0adb97e3c9a8bf469", + "e417c01c804a4b846fb4ab0bc5f5e635822dd469814b6f454344a371a0c54846", + "e4a3df05b8fa77155ebe3f0e59b48ed9727f70e8624623cb69596dd0ccb2acff", + "e55e578cf3630b4265a38473546efd00faeaccae3b69cafbd46c38b43810159b", + "e58a39b6c5e754b1bfd881b5e6b3072eb678e7909a173dbe589b1f1046cc3c61", + "e614a0919bca602cb5416a1b69b025e7679766f28a630f8fe36a044c9b6c19a2", + "e730a01ae3f75be9423d32d2fc782f0250689840a9f35f552f9234956b91b651", + "e7cbfefa2b400f6c2a4088c06e774c341f1fec283d7e6de39d6aa38f8fb16eff", + "e83f6805a42db291bb19df80219ebf6257f4ca9965f7c96acf9c84c19845466f", + "e8b62f0abf8484fd663882a6423adc003b27b151b2e081544cdb328480b0efec", + "e968c0a922d5fbb0063602dfac63455d45924717d817299133a56a73fa9c85d9", + "eb68932633b2c16e1761b398a52354406182ae85cbc65ae7e27dbcfdb667d6af", + "eb961efc66c3dda1fceeffc0f5dcb270b6f354484fe5ff983e7252533d2db5b6", + "ee27e25f902e654569ef7a8ded0e028eea1f7b48cce472d02537f9c58b90f31b", + "ee9c47b6572c11e77808aa5278255d5baaa53327776a2eb1f3c02ac21706667e", + "eec2faad73968e9d8df0cc9171b24b238365533e490c3c1da68f2a01b3a58dac", + "f01042ebfb2d85430a7264450902c9d7f8f208343cbfa398de5efe062158cb10", + "f0262b9ee515bf10133044931f0078e678627bee08a7021dae6532b1d10652eb", + "f07a5ec90031146f8a1b92c877a9ef88d90b67725b687daa47850afcb8a4f5bf", + "f137e08187e31a78bf93686f27df66e24ba62fb370115cf099ab3452b3e84934", + "f18c1737e19dbb670db3262115d9d640533cf10ba73f8196ddb5a7d582744feb", + "f194576d56014f50ba3f94c92baf120c3dbcebe176b661154484c2a92b7b40c5", + "f3088bbb09667126643e781a17542b06cf97833d29b414ff29cdfc0c249b0025", + "f36b21cc9f9651ab59717c67501c154929fdb69d2124fcf87beb5cb4eddc385b", + "f37badc006085509e7f393cc62d362f41a6300fe2a54c46c6e0763b4afbfe7f7", + "f3bc1c4b3902b647d6660ae83b6a272dbf80b85cbcb7b57d74afbe94d6d754b1", + "f54e7d6b2f7b5aaa89a466187ae089784a605b946d50d221e21185b14ad2e5a0", + "f5579f35655c4c99b49931e63782e64494470ac21c1693fecacad3d3d54291bb", + "f70bac72a9d4055c60922c63e2f2cb3b77aa90250cc071a96f67d3ab658a4a12", + "f7899c890521825ae7f31b8c107f2ac9cf8e50097d4c06b92e226a772ba7b925", + "f82052e29e6ca52c7c633334e5c939674ca671cf5a592db59f882e88c5bbfdac", + "f937b82c14d253a7eecc1fc8effc545517af158c38d181727dcb8205e42a27bf", + "f9576ef7044564daed5e3bd01e196df3a69109244971a106c44d8108f286f08d", + "fa65af636e0484758f08f422fe84c5ccc340fd7f33a96fe5bd63e2a9c110fe6c", + "fc969058a53f8827440a2d1750091bc41aaded65e487bc033c81f2140b997321", + "fd18f20b9ea23bcb6fc5091c74383ba65a8b58c19f12da01c74b82ce5f821843", + "fd5336c29326b4bcacb4c55cedbe14a1d5e61ef269102110f643f665d1f8f03e", + "fdb70a2a5773acfdd00811abbadb690df8422aaaed7bd93893f576e2f5704bac", + "fe2303f90cb10eb75f13d551f88d2103bfb4f0c83682f807942f4a768b6852d1", + "fe6e280d14d7dd0fe23a487c684330c060896eee257e80121bc188a4f6110c09", + "ff5cbd0b1fdca636b746e9d39e0c2e984969c249bd093bcdd886290622f87775", + "ff6bbc3315c35a873ae49a7aea0911da2039eb0bdfd754beec5b119d5ca46b29" + ], + "task_counts": { + "LiftPegUpright-v1": 1056, + "PickCube-v1": 2384, + "PullCube-v1": 928, + "PushCube-v1": 1008, + "StackCube-v1": 1328 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed0/train.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/train.log new file mode 100644 index 0000000000000000000000000000000000000000..05319279e0bb3fe48a513ec9a1a0937eb02ea910 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed0/train.log @@ -0,0 +1 @@ +CTT checkpoint trained separately; rollout used checkpoint=runs/ctt_residual_base_context_obj_seed0/model.pt diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/command.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/command.txt new file mode 100644 index 0000000000000000000000000000000000000000..6245a9c501e359bbb4a875c12f3590dd3ff0ab30 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/command.txt @@ -0,0 +1 @@ +python scripts/eval_ctt_generated_rollout.py --checkpoint runs/ctt_residual_base_context_obj_seed1/model.pt --source-index data/cil_charts_rgb_refs/train/index.json --target-index data/cil_charts_rgb_refs/val/index.json --out-dir runs/ctt_residual_base_context_obj_rollout_val69_seed1 --k 8 --pool-size 0 --neighbors 8 --max-target-charts 69 --group-batch-size 1 --device cpu --sim-backend physx_cpu --render-backend cpu --restore-tolerance 1e-5 --delta-scale 1.0 --bootstrap-samples 200 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/config.yaml b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/config.yaml new file mode 100644 index 0000000000000000000000000000000000000000..01fcaebae913e71c3f5e7467969cf31439663788 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/config.yaml @@ -0,0 +1,18 @@ +bootstrap_samples: 200 +checkpoint: runs/ctt_residual_base_context_obj_seed1/model.pt +delta_scale: 1.0 +device: cpu +exclude_self_source: False +group_batch_size: 1 +include_targets_without_positives: False +k: 8 +max_target_charts: 69 +neighbors: 8 +out_dir: runs/ctt_residual_base_context_obj_rollout_val69_seed1 +pool_size: 0 +render_backend: cpu +restore_tolerance: 1e-05 +sim_backend: physx_cpu +skip_metrics: False +source_index: data/cil_charts_rgb_refs/train/index.json +target_index: data/cil_charts_rgb_refs/val/index.json diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/data_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..49b12b50537d95327f9c62ddd41910531a6c14a4 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/data_hash.txt @@ -0,0 +1 @@ +1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/eval.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/eval.log new file mode 100644 index 0000000000000000000000000000000000000000..3d65976412deaee2f45e25b919d53e6822b5a280 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/eval.log @@ -0,0 +1,766 @@ +start +importing gymnasium/mani_skill +imported gymnasium/mani_skill +loaded checkpoint=runs/ctt_residual_base_context_obj_seed1/model.pt +loaded charts source=301 target=69 +generated proposals rows=69 total=552 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=PickCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PickCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +rollout complete rows=69 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/git_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/git_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8b137891791fe96927ad78e64b0aad7bded08bdc --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/git_hash.txt @@ -0,0 +1 @@ + diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_candidates.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_candidates.json new file mode 100644 index 0000000000000000000000000000000000000000..6d6369c332f5651cee5cfed23a9d9b5a22044f7e --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_candidates.json @@ -0,0 +1,44442 @@ +{ + "candidates_evaluated": true, + "checkpoint": "runs/ctt_residual_base_context_obj_seed1/model.pt", + "decoder": { + "delta_scale": 1.0, + "lossless": false, + "name": "linear_keyframe_decode", + "source_code": "spline_tangent_code stores start/mid/end residual keyframes" + }, + "exclude_self_source": false, + "k": 8, + "neighbors": 8, + "pool_size": 8, + "report_type": "ctt_generated_measured_rollout", + "rows": [ + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7270061373710632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7270061373710632 + }, + "base_progress": 0.7270061373710632, + "base_success": false, + "base_utility": 0.7270061373710632, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7210943102836609, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7210943102836609 + }, + { + "contact_quality": null, + "progress": 0.2108287811279297, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2108287811279297 + }, + { + "contact_quality": null, + "progress": 0.5322304368019104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5322304368019104 + }, + { + "contact_quality": null, + "progress": 0.4597426652908325, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4597426652908325 + }, + { + "contact_quality": null, + "progress": 0.5059094429016113, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5059094429016113 + }, + { + "contact_quality": null, + "progress": 0.5588076114654541, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5588076114654541 + }, + { + "contact_quality": null, + "progress": 0.5301380157470703, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5301380157470703 + } + ], + "candidate_progress": [ + 1.0, + 0.7210943102836609, + 0.2108287811279297, + 0.5322304368019104, + 0.4597426652908325, + 0.5059094429016113, + 0.5588076114654541, + 0.5301380157470703 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-df961fe24a43e56d1716", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-484ea653bbca2de3a4a2", + "generated_tangents": [ + [ + -0.1740616410970688, + -0.16297969222068787, + 0.3772149682044983, + 0.7074242830276489, + 1.0587588548660278, + -0.018649250268936157, + -0.3348178267478943, + 0.004648327827453613, + -0.04392655938863754, + 0.20681168138980865, + -0.08656638115644455, + 1.245284914970398, + -0.06480005383491516, + 0.1428927332162857, + 0.2228441834449768, + -0.015324490144848824, + 0.0933811217546463, + 0.022266343235969543, + 1.2185115814208984, + 0.05619916319847107, + -0.020767703652381897 + ], + [ + -0.12723924219608307, + -0.04269175976514816, + 0.3771720826625824, + 0.6217846870422363, + 1.2183494567871094, + 0.1744009107351303, + -0.3855992555618286, + -0.025565162301063538, + -0.04204275459051132, + 0.1668231189250946, + -0.10222139209508896, + 1.104973316192627, + 0.021443843841552734, + 0.3351121246814728, + -0.0014446936547756195, + -0.05466555431485176, + 0.05066956579685211, + 0.1515975147485733, + 1.4147889614105225, + 0.030329227447509766, + 0.015080541372299194 + ], + [ + -0.21145252883434296, + -0.16480331122875214, + 0.3572320342063904, + 0.44956710934638977, + 1.2530062198638916, + 0.2183370441198349, + -0.15831303596496582, + -0.19049617648124695, + -0.03198334574699402, + 0.027901150286197662, + -0.02730824053287506, + 1.1245977878570557, + -0.09650805592536926, + 0.4930015802383423, + -0.05702989175915718, + -0.020286599174141884, + 0.1561012864112854, + 0.10981842875480652, + 1.1236357688903809, + 0.09788975119590759, + 0.21223600208759308 + ], + [ + -0.21923606097698212, + -0.08897554874420166, + 0.020573318004608154, + 0.4900379478931427, + 1.4295775890350342, + 0.36279985308647156, + -0.6743503212928772, + -0.1906130313873291, + -0.01927298493683338, + 0.1211085319519043, + -0.17763853073120117, + 0.7397709488868713, + -0.09032189846038818, + 0.2276231050491333, + -0.05368965491652489, + 0.10843819379806519, + 0.17454448342323303, + -0.09307925403118134, + 1.185486078262329, + -0.16541698575019836, + -0.05778945982456207 + ], + [ + -0.2743591070175171, + -0.21796829998493195, + 0.18193334341049194, + 0.5038225650787354, + 1.3598613739013672, + 0.12141576409339905, + -0.6088373064994812, + -0.1826959252357483, + -0.08674545586109161, + 0.08357211947441101, + -0.07634951919317245, + 1.0663490295410156, + -0.07500910758972168, + 0.36990493535995483, + 0.07827837765216827, + -0.07535171508789062, + -0.033485040068626404, + 0.03464619815349579, + 1.285482406616211, + -0.1284266859292984, + 0.15608970820903778 + ], + [ + -0.2402840405702591, + -0.16710659861564636, + 0.3115612268447876, + 0.5065975189208984, + 1.0599058866500854, + 0.15393267571926117, + -0.33368104696273804, + -0.19817450642585754, + 0.0020447690039873123, + 0.08052072674036026, + -0.04305257648229599, + 1.2484194040298462, + 0.007736295461654663, + 0.2714783549308777, + -0.04003826901316643, + -0.09442171454429626, + 0.11606928706169128, + 0.0512305349111557, + 1.2150667905807495, + 0.06886446475982666, + 0.08871621638536453 + ], + [ + -0.22926358878612518, + -0.12117899954319, + 0.3107870817184448, + 0.5271912813186646, + 1.063497543334961, + 0.049579598009586334, + -0.4694610834121704, + -0.2212059199810028, + -0.007535628974437714, + 0.10281668603420258, + -0.14292535185813904, + 1.1758768558502197, + -0.0920785665512085, + 0.1954551786184311, + -0.09687457978725433, + -0.03090819902718067, + 0.2185717076063156, + 0.07259805500507355, + 1.1084318161010742, + 0.046909987926483154, + 0.0721399113535881 + ], + [ + -0.26756858825683594, + -0.1288309246301651, + 0.2943589389324188, + 0.6360929012298584, + 1.0180401802062988, + -0.037748053669929504, + -0.28414690494537354, + -0.09688475728034973, + 0.03188071772456169, + 0.1313062161207199, + -0.17894881963729858, + 1.2895761728286743, + 0.04807606339454651, + 0.30058038234710693, + -0.22448518872261047, + -0.05610359460115433, + 0.18521246314048767, + 0.12355383485555649, + 1.268186092376709, + -0.10017992556095123, + 0.07076788693666458 + ] + ], + "generated_utilities": [ + 2.0, + 0.7210943102836609, + 0.2108287811279297, + 0.5322304368019104, + 0.4597426652908325, + 0.5059094429016113, + 0.5588076114654541, + 0.5301380157470703 + ], + "group_id": "ms-484ea653bbca2de3a4a2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7242962718009949, + 0.7209752798080444, + 2.0, + 0.19684189558029175, + 0.21780464053153992, + 0.19326990842819214, + 0.22509369254112244, + 0.20856516063213348, + 0.21041584014892578, + 0.19898515939712524, + 0.20721960067749023, + 0.20830264687538147, + 0.19554394483566284, + 0.22549311816692352, + 0.19481703639030457, + 0.2172507494688034 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.4380870461463928, + 0.4186934530735016, + 0.6294705867767334, + -0.08003801107406616, + 0.5882754325866699, + 0.006411761045455933, + 0.0, + 0.27439412474632263, + -0.43702489137649536, + -0.26345914602279663, + -0.3102925717830658, + 1.1419475078582764, + -0.05133828520774841, + 0.0, + 0.11964668333530426, + -0.04283810034394264, + 0.021698474884033203, + -0.3213534653186798, + 1.444929838180542, + 0.058529436588287354, + 0.0 + ], + [ + 0.18277202546596527, + -0.31313952803611755, + 1.5374140739440918, + 0.9199619889259338, + 1.58827543258667, + -0.42588773369789124, + -1.307411789894104, + -0.1648276150226593, + 0.17999683320522308, + 0.7365408539772034, + -0.2825929522514343, + 2.1419475078582764, + -0.44205278158187866, + 0.3557547330856323, + 0.11787930130958557, + 0.04347073659300804, + 0.5306029915809631, + -0.6201959848403931, + 2.444929838180542, + -0.7078790068626404, + 0.7410526275634766 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.614823579788208, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7115094661712646, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4821052551269531 + ], + [ + 0.2415536344051361, + -0.2574644386768341, + 1.3600168228149414, + 0.8914169669151306, + 1.311052918434143, + -0.13483268022537231, + -1.4109176397323608, + -0.007070749998092651, + 0.30199557542800903, + 0.867316484451294, + -0.0111103355884552, + 2.228789806365967, + -0.26305240392684937, + 0.4046628475189209, + 0.25534987449645996, + 0.29946526885032654, + 0.777583122253418, + -0.535997748374939, + 2.2955121994018555, + -0.6670190095901489, + 0.8460665941238403 + ], + [ + -0.212661013007164, + -0.6653347015380859, + 1.8750245571136475, + 0.7373600006103516, + 1.2695744037628174, + -0.7584145069122314, + -1.695502758026123, + 0.05721297860145569, + -0.04310476779937744, + 0.4696756601333618, + -0.2954420745372772, + 1.9897286891937256, + -0.6554491519927979, + 0.5805841088294983, + 0.17026324570178986, + 0.3433569669723511, + 0.3734014630317688, + -0.8102566003799438, + 2.792973279953003, + -0.8463170528411865, + 1.0759955644607544 + ], + [ + -0.039009034633636475, + -0.7773740291595459, + 1.064732313156128, + 0.712108850479126, + 1.270837426185608, + -0.5491334199905396, + -1.7919384241104126, + -0.5782418251037598, + 0.4058297872543335, + 0.5231752395629883, + -0.521438479423523, + 2.2066850662231445, + -0.5258626341819763, + -0.09100466966629028, + 0.24944378435611725, + -0.2271452248096466, + 0.9009795784950256, + -0.8140718340873718, + 2.6494152545928955, + -0.5153427124023438, + 0.671539306640625 + ], + [ + -0.3804699182510376, + -0.5808351039886475, + 1.8389031887054443, + 0.7141245603561401, + 1.0997881889343262, + -0.1678517758846283, + -1.884540319442749, + -0.3411717414855957, + 0.4830858111381531, + 0.8356180787086487, + -0.8238832950592041, + 1.743321418762207, + -0.3751307427883148, + 0.5275082588195801, + -0.08622422814369202, + 0.23666565120220184, + 0.802552342414856, + -0.8389871120452881, + 1.9817423820495605, + -0.4874984323978424, + 0.4390762150287628 + ], + [ + 0.4497724771499634, + -0.7880887389183044, + 1.4436085224151611, + 1.083756446838379, + 2.2298290729522705, + -0.03008332848548889, + -1.0250177383422852, + -0.6266056299209595, + 0.3751387298107147, + 1.1566712856292725, + -0.6914000511169434, + 1.953734278678894, + 0.08129626512527466, + 0.9812837839126587, + -0.4981238543987274, + 0.6697847247123718, + -0.05150604248046875, + -0.06941354274749756, + 1.9621039628982544, + -1.345557451248169, + 0.43397778272628784 + ], + [ + 0.03950639069080353, + -1.037135124206543, + 1.1041697263717651, + 1.109439730644226, + 1.144017219543457, + -0.07355684041976929, + -0.8942177295684814, + -0.16419151425361633, + -0.12125559151172638, + 0.36935141682624817, + -0.7298048734664917, + 1.4954023361206055, + -0.018926143646240234, + 0.4787553548812866, + -0.3127076327800751, + 0.6351315379142761, + 0.5993886590003967, + -0.5143972635269165, + 2.429173469543457, + -0.6674217581748962, + 0.5048977732658386 + ], + [ + 0.9763806462287903, + -1.1549197435379028, + 2.0378684997558594, + 1.2418556213378906, + 2.427785634994507, + -1.1326193809509277, + -1.5491721630096436, + -0.8937685489654541, + 0.2583051919937134, + -0.10687285661697388, + 0.2628750801086426, + 1.6517728567123413, + -0.21639958024024963, + 0.7581590414047241, + -0.29044294357299805, + -0.08927425742149353, + 1.3009071350097656, + -0.9239004850387573, + 2.5070056915283203, + -1.3246500492095947, + 0.24751555919647217 + ], + [ + 0.9781984090805054, + -0.13759100437164307, + 1.8698725700378418, + 1.4942535161972046, + 2.5587658882141113, + -1.1335766315460205, + -0.6910277605056763, + -0.1434302181005478, + 0.4021325707435608, + -0.12275522947311401, + 0.6571680307388306, + 1.9964357614517212, + -0.1442476212978363, + 1.3201656341552734, + 0.2586786448955536, + -0.38222962617874146, + 0.9950553774833679, + -0.8371140956878662, + 1.8977141380310059, + -0.795723021030426, + 1.505234956741333 + ], + [ + -0.6250126361846924, + 0.4231366813182831, + 1.701646089553833, + 1.6233667135238647, + 1.0861127376556396, + -0.3387698233127594, + -0.9825164079666138, + -1.1511213779449463, + 0.11523068696260452, + 1.5823121070861816, + 0.5840664505958557, + 2.145885705947876, + -1.2145874500274658, + 1.1558246612548828, + -0.5131542682647705, + -0.4715161919593811, + 1.0824741125106812, + -0.7667124271392822, + 1.8457963466644287, + -1.228658676147461, + 1.2112116813659668 + ], + [ + 1.1640653610229492, + -1.1697030067443848, + 1.6035369634628296, + 0.2161661982536316, + 1.4819366931915283, + 0.2569611370563507, + -1.3529595136642456, + 0.41208329796791077, + 0.03121529519557953, + 1.731370210647583, + -0.8131548166275024, + 2.996675729751587, + -1.1173620223999023, + 0.7474988102912903, + 0.21459203958511353, + -0.2133086621761322, + -0.22307389974594116, + -0.6306187510490417, + 2.0490896701812744, + -0.49808698892593384, + 0.8432145714759827 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.014628380537033081, + -0.18207009136676788, + 0.5374140739440918, + -0.04374772310256958, + 0.5882754325866699, + 0.25251278281211853, + 0.0, + -0.23516394197940826, + 0.12562739849090576, + 0.3381730020046234, + 0.033050984144210815, + 1.1419475078582764, + -0.3290551006793976, + 0.0, + 0.04625481367111206, + -0.0026901662349700928, + 0.0754578709602356, + -0.1481514573097229, + 1.444929838180542, + 0.049897611141204834, + 0.0 + ] + ], + "predicted_scores": [ + 4.409553050994873, + 3.6382431983947754, + 3.63432240486145, + 3.5137364864349365, + 3.2669029235839844, + 3.1935641765594482, + 3.182760238647461, + 2.740438222885132 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1231780997", + "selected_index": 0, + "state_hash": "771fb752a186163f05a49a9b404284bde55d89838769a764fca1d6d1a9419652", + "stored_base_utility": 0.7242962718009949, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.528373658657074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.528373658657074 + }, + "base_progress": 0.528373658657074, + "base_success": false, + "base_utility": 0.528373658657074, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.21175099909305573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21175099909305573 + }, + { + "contact_quality": null, + "progress": 0.20219288766384125, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20219288766384125 + }, + { + "contact_quality": null, + "progress": 0.19861702620983124, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19861702620983124 + }, + { + "contact_quality": null, + "progress": 0.20537595450878143, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20537595450878143 + }, + { + "contact_quality": null, + "progress": 0.19838964939117432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19838964939117432 + }, + { + "contact_quality": null, + "progress": 0.2148684412240982, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2148684412240982 + }, + { + "contact_quality": null, + "progress": 0.2115357667207718, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2115357667207718 + }, + { + "contact_quality": null, + "progress": 0.2025710493326187, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2025710493326187 + } + ], + "candidate_progress": [ + 0.21175099909305573, + 0.20219288766384125, + 0.19861702620983124, + 0.20537595450878143, + 0.19838964939117432, + 0.2148684412240982, + 0.2115357667207718, + 0.2025710493326187 + ], + "candidate_source_chart_ids": [ + "ms-df961fe24a43e56d1716", + "ms-13d8bae55d3c3710138f", + "ms-502dba1003c09ec60ba0", + "ms-0660d5f65df5fc4b6132", + "ms-8f3d4e1d07bb5dcc7c57", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb", + "ms-e1b6bcb41532a800d3cb" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank6", + "ctt_transport_rank0", + "ctt_transport_rank5", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-4b4cd12d70c0615e0930", + "generated_tangents": [ + [ + -0.12567324936389923, + -0.054757699370384216, + 1.0230728387832642, + 0.8730899095535278, + 0.21579238772392273, + 0.4350303113460541, + 0.12024950981140137, + 0.12823566794395447, + 0.01874651573598385, + -0.24843716621398926, + -0.25908493995666504, + 1.3926934003829956, + -0.17717412114143372, + 0.11273403465747833, + 0.09128392487764359, + -0.11082588136196136, + 0.130533367395401, + -0.07427425682544708, + 1.185415506362915, + -0.3137575387954712, + -0.017678260803222656 + ], + [ + -0.12893421947956085, + -0.14310193061828613, + 1.139750599861145, + 0.8206130266189575, + 0.20182399451732635, + 0.28020334243774414, + 0.2846207618713379, + -0.021337375044822693, + 0.12925907969474792, + -0.18140903115272522, + -0.21781983971595764, + 1.3943719863891602, + -0.07744288444519043, + 0.08842577040195465, + 0.07450462877750397, + -0.08268576860427856, + 0.08181801438331604, + -0.08562545478343964, + 1.1875466108322144, + -0.07574416697025299, + -0.08042079210281372 + ], + [ + -0.4525997042655945, + -0.13613227009773254, + 0.6503928303718567, + 0.931410551071167, + 0.35046908259391785, + 0.9532989263534546, + -0.06574034690856934, + 0.012613564729690552, + 0.042612239718437195, + -0.17629987001419067, + -0.01874808967113495, + 1.4223198890686035, + -0.21591919660568237, + 0.19703838229179382, + -0.11318287253379822, + -0.05161397159099579, + 0.24200108647346497, + -0.19728456437587738, + 1.2009154558181763, + -0.18358485400676727, + -0.12039703130722046 + ], + [ + -0.010334953665733337, + -0.08441674709320068, + 1.1696398258209229, + 0.9284533262252808, + 0.26577818393707275, + 0.2814810872077942, + 0.18289008736610413, + 0.05493806302547455, + 0.11425623297691345, + -0.25960254669189453, + -0.3063563406467438, + 1.5965182781219482, + -0.11598813533782959, + 0.14736324548721313, + 0.09953732788562775, + -0.06987813115119934, + 0.08677676320075989, + -0.07662118971347809, + 1.0562846660614014, + -0.10489632189273834, + -0.24773052334785461 + ], + [ + -0.23086850345134735, + -0.15534354746341705, + 1.0204118490219116, + 1.0651427507400513, + 0.17302680015563965, + 0.35944873094558716, + -0.08795249462127686, + -0.09896808862686157, + 0.11917176097631454, + -0.17487865686416626, + -0.12784412503242493, + 1.5219006538391113, + 0.06432041525840759, + 0.1888321042060852, + -0.0535670630633831, + -0.12329638004302979, + 0.20388533174991608, + -0.09481458365917206, + 1.2969107627868652, + -0.14185336232185364, + 0.1616964340209961 + ], + [ + -0.08113265037536621, + -0.05291309952735901, + 0.9791842103004456, + 0.922683835029602, + 0.3093123435974121, + 0.33274784684181213, + -0.05834740400314331, + 0.048475250601768494, + 0.10372401773929596, + -0.2137639820575714, + -0.28827419877052307, + 1.6551964282989502, + -0.11236998438835144, + 0.48802006244659424, + 0.04900050908327103, + -0.1199604868888855, + 0.20941756665706635, + 0.03269365429878235, + 1.160142183303833, + -0.09298726916313171, + -0.13338792324066162 + ], + [ + -0.18320231139659882, + -0.19712179899215698, + 1.0537443161010742, + 0.8514035940170288, + 0.14848870038986206, + 0.31337594985961914, + 0.08552402257919312, + -0.005987539887428284, + 0.17298458516597748, + -0.2623000144958496, + -0.22141033411026, + 1.4968657493591309, + -0.3683769106864929, + 0.4715902805328369, + 0.1347920298576355, + -0.15936803817749023, + -0.08030088245868683, + -0.10295622050762177, + 1.1988164186477661, + -0.12891122698783875, + 0.13912439346313477 + ], + [ + 0.03518760949373245, + -0.17133961617946625, + 1.2623594999313354, + 0.9834651947021484, + 0.05818794667720795, + 0.12489740550518036, + 0.09438782930374146, + 0.08768085390329361, + 0.23926232755184174, + -0.20809391140937805, + -0.37154942750930786, + 1.4849345684051514, + 0.056320399045944214, + 0.32554954290390015, + 0.0560346283018589, + -0.08457501232624054, + 0.021852806210517883, + -0.11967034637928009, + 1.2465505599975586, + 0.10155147314071655, + -0.07965892553329468 + ] + ], + "generated_utilities": [ + 0.21175099909305573, + 0.20219288766384125, + 0.19861702620983124, + 0.20537595450878143, + 0.19838964939117432, + 0.2148684412240982, + 0.2115357667207718, + 0.2025710493326187 + ], + "group_id": "ms-4b4cd12d70c0615e0930", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7225138545036316, + 2.0, + 0.6951258182525635, + 0.18915215134620667, + 0.20867900550365448, + 0.20041285455226898, + 0.20335882902145386, + 0.20815956592559814, + 0.2024221420288086, + 0.1940193623304367, + 0.20689067244529724, + 0.19995209574699402, + 0.21901433169841766, + 0.19443893432617188, + 0.20502614974975586, + 0.1959807276725769 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.09018868207931519, + 0.3819495439529419, + 0.9586174488067627, + 1.0247726440429688, + -0.179018035531044, + 0.125870943069458, + 0.0, + -0.1166897565126419, + 0.07711493968963623, + -0.24461698532104492, + -0.005436837673187256, + 1.737370491027832, + -0.4425608515739441, + 0.0, + 0.24725346267223358, + -0.03395397961139679, + 0.643236517906189, + 0.05422210693359375, + 1.2738127708435059, + 0.18164724111557007, + 0.0 + ], + [ + 0.6095315217971802, + 0.35729360580444336, + 1.9586174488067627, + 2.0247726440429688, + -0.2820541560649872, + 1.044631004333496, + -2.239265203475952, + -0.14250506460666656, + -0.4363628029823303, + -1.244616985321045, + -0.32413747906684875, + 2.737370491027832, + 0.24581307172775269, + -0.053983256220817566, + 0.2877395749092102, + 0.18538890779018402, + 0.45247530937194824, + -0.9457778930664062, + 2.273812770843506, + -0.8183527588844299, + 0.5866778492927551 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -4.478530406951904, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.10796651244163513, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1733556985855103 + ], + [ + 0.8415398597717285, + 0.17323988676071167, + 1.9669679403305054, + 2.188286304473877, + -0.386857271194458, + 1.3415166139602661, + -2.06968355178833, + -0.38555508852005005, + -0.6971946954727173, + -1.0110756158828735, + -0.19943451881408691, + 3.0229673385620117, + 0.03575621545314789, + -0.26961493492126465, + 0.5202459692955017, + 0.0921676978468895, + 0.7439858913421631, + -0.724668562412262, + 2.116668939590454, + -1.0585715770721436, + 0.7851569652557373 + ], + [ + 0.6909375190734863, + -0.018118590116500854, + 1.677350401878357, + 1.9505902528762817, + 0.11725521087646484, + 1.2839276790618896, + -2.0543243885040283, + -0.38843441009521484, + -0.6865814328193665, + -1.5103397369384766, + -0.2952713072299957, + 2.384809970855713, + -0.10558199882507324, + 0.29018354415893555, + 0.33942633867263794, + -0.19196103513240814, + 0.35257238149642944, + -0.8657878637313843, + 2.5445473194122314, + -0.706183910369873, + 0.7214455008506775 + ], + [ + 0.8977934122085571, + 0.7311913371086121, + 2.203964948654175, + 1.686698079109192, + -0.338996022939682, + 0.631161630153656, + -2.3877103328704834, + 0.07195566594600677, + -0.6892462968826294, + -0.9634823799133301, + -0.5621621012687683, + 2.479537010192871, + 0.675940752029419, + -0.047649919986724854, + 0.14082925021648407, + 0.19517391920089722, + 0.15626585483551025, + -1.3974462747573853, + 2.759931802749634, + -1.166107416152954, + 0.18018442392349243 + ], + [ + 0.7796773910522461, + 0.06440207362174988, + 1.5159449577331543, + 1.6007665395736694, + -0.12459242343902588, + 1.5781673192977905, + -2.5707991123199463, + 0.034798189997673035, + -0.44772639870643616, + -1.3837177753448486, + -0.5199699401855469, + 3.1566030979156494, + -0.22805768251419067, + -0.5881848931312561, + 0.49021050333976746, + -0.08376060426235199, + 0.5395568609237671, + -1.3038995265960693, + 2.37483811378479, + -0.615712583065033, + 0.7884383201599121 + ], + [ + 0.4830939471721649, + -0.12824741005897522, + 2.133068561553955, + 2.42085337638855, + -0.1641906201839447, + 1.6598048210144043, + -2.5325045585632324, + 0.06623834371566772, + 0.14292877912521362, + -1.85994553565979, + 0.01158285140991211, + 3.2622194290161133, + -0.017848491668701172, + -0.08653604984283447, + 0.2199593186378479, + -0.0463244765996933, + -0.04547742009162903, + -0.5160569548606873, + 2.1955723762512207, + -0.364714115858078, + 1.0323584079742432 + ], + [ + 0.7637265920639038, + -0.4020084738731384, + 2.4714059829711914, + 1.33060622215271, + 0.1464400589466095, + 0.7656213045120239, + -1.9972842931747437, + 0.42333751916885376, + -1.118815302848816, + -1.5639967918395996, + -0.32944348454475403, + 2.0334420204162598, + 0.37165334820747375, + -0.2861292064189911, + 0.6376298666000366, + 0.8521975874900818, + 0.6845490336418152, + -1.515796422958374, + 1.5925071239471436, + -0.4112323224544525, + 1.1011840105056763 + ], + [ + 1.456933617591858, + 1.0030901432037354, + 1.1997551918029785, + 1.7652757167816162, + 0.03292572498321533, + 1.302201271057129, + -2.748408317565918, + 0.7148768305778503, + -0.5089412331581116, + -0.5701506733894348, + -0.40410247445106506, + 3.586395502090454, + -0.37424683570861816, + -0.6850852966308594, + 0.9945629835128784, + 0.5821471810340881, + 0.8666700124740601, + -1.1607441902160645, + 2.1460094451904297, + -0.7912135720252991, + 0.6301633715629578 + ], + [ + -0.09475094079971313, + -0.28330767154693604, + 1.8353716135025024, + 2.3373594284057617, + 0.7110240459442139, + 1.9665697813034058, + -1.8502212762832642, + -0.40577346086502075, + -0.03506562113761902, + -0.7167142033576965, + -0.39153769612312317, + 2.3472883701324463, + 0.08497899770736694, + -0.9910531640052795, + -0.32254326343536377, + -0.11388416588306427, + -0.4873260259628296, + -0.8018304109573364, + 2.295731782913208, + -1.325164794921875, + 1.2513983249664307 + ], + [ + -0.32657402753829956, + 0.8198520541191101, + 1.211392879486084, + 1.974273443222046, + -0.11595214903354645, + 1.954041838645935, + -1.5547080039978027, + 0.06433123350143433, + -0.079216867685318, + -0.5887003540992737, + -0.6015424728393555, + 2.756274938583374, + 0.641853928565979, + -0.8642687201499939, + 0.07965350151062012, + 0.11391039192676544, + 1.10663902759552, + -1.506749153137207, + 3.222418785095215, + -1.0799858570098877, + -0.12158209085464478 + ], + [ + 0.9412136077880859, + -0.5255860090255737, + 1.9096710681915283, + 2.113077402114868, + -1.0082788467407227, + 0.1505374312400818, + -3.0413341522216797, + -0.24721917510032654, + -0.20225940644741058, + -1.6247544288635254, + -0.33320704102516174, + 1.8443769216537476, + 0.7569764256477356, + -0.5270920395851135, + 1.1813421249389648, + 1.0764858722686768, + 0.01533237099647522, + -1.4125463962554932, + 1.82404363155365, + 0.054037630558013916, + 0.45250433683395386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.056152522563934326, + 0.005679905414581299, + 0.9586174488067627, + 1.0247726440429688, + -0.054595887660980225, + 0.044631004333496094, + 0.0, + -0.09190673381090164, + 0.09592300653457642, + -0.24461698532104492, + -0.007907181978225708, + 1.737370491027832, + -0.029382914304733276, + 0.0, + 0.06305502355098724, + 0.03764393925666809, + 0.0785532295703888, + 0.005711078643798828, + 1.2738127708435059, + -0.04058480262756348, + 0.0 + ] + ], + "predicted_scores": [ + 0.612500786781311, + 0.4029788672924042, + 0.06764078140258789, + -0.08695489168167114, + -0.1073904037475586, + -0.16207697987556458, + -0.37850794196128845, + -0.9772553443908691 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1025740212", + "selected_index": 0, + "state_hash": "5b394fca1712c0667d126198527a4f0ba81893d536a5fcbf7b90f5b13fa703dc", + "stored_base_utility": 0.7225138545036316, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7181311249732971, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7181311249732971 + }, + { + "contact_quality": null, + "progress": 0.655661404132843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.655661404132843 + }, + { + "contact_quality": null, + "progress": 0.6736156344413757, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6736156344413757 + }, + { + "contact_quality": null, + "progress": 0.7093296051025391, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7093296051025391 + }, + { + "contact_quality": null, + "progress": 0.7143670916557312, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7143670916557312 + }, + { + "contact_quality": null, + "progress": 0.5601552128791809, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5601552128791809 + }, + { + "contact_quality": null, + "progress": 0.7090306282043457, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7090306282043457 + }, + { + "contact_quality": null, + "progress": 0.5006552338600159, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5006552338600159 + } + ], + "candidate_progress": [ + 0.7181311249732971, + 0.655661404132843, + 0.6736156344413757, + 0.7093296051025391, + 0.7143670916557312, + 0.5601552128791809, + 0.7090306282043457, + 0.5006552338600159 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6e268d1e2915740816a1", + "generated_tangents": [ + [ + -0.0008221715688705444, + -0.15447860956192017, + 0.06838101148605347, + 0.3342396318912506, + 1.789905071258545, + 0.028334319591522217, + -0.055450379848480225, + 0.0549965500831604, + -0.048723749816417694, + 0.21258844435214996, + -0.10498260706663132, + 0.9088835716247559, + -0.0790761411190033, + 0.09338332712650299, + 0.08490000665187836, + 0.0025108912959694862, + 0.03347918391227722, + -0.059780195355415344, + 0.752911388874054, + -0.1534477174282074, + -0.13718003034591675 + ], + [ + -0.16093336045742035, + -0.11970759928226471, + 0.17358586192131042, + 0.05782929062843323, + 1.6275742053985596, + -0.09557829797267914, + 0.18253520131111145, + -0.15749138593673706, + 0.043381333351135254, + 0.12774048745632172, + 0.12142147123813629, + 0.9491201043128967, + -0.02470564842224121, + -0.012363627552986145, + -0.28332972526550293, + -0.02575576677918434, + 0.014457911252975464, + -0.098793163895607, + 0.9115554094314575, + -0.36201727390289307, + -0.13929247856140137 + ], + [ + -0.24218355119228363, + -0.0708344355225563, + 0.12442180514335632, + -0.18104180693626404, + 1.7634239196777344, + -0.052961185574531555, + -0.30474936962127686, + 0.026587843894958496, + -0.08119779080152512, + 0.07087457180023193, + 0.11938828229904175, + 0.9018123745918274, + -0.07314527034759521, + 0.40944308042526245, + -0.1938321590423584, + -0.021801553666591644, + 0.07719245553016663, + 0.01955544948577881, + 0.7524850368499756, + -0.05436146259307861, + -0.09793844819068909 + ], + [ + -0.05114002525806427, + -0.10038959980010986, + 0.006561249494552612, + 0.12133711576461792, + 1.7032814025878906, + 6.608664989471436e-05, + -0.0943724513053894, + -0.13517552614212036, + -0.028819995000958443, + 0.1371464729309082, + -0.12953326106071472, + 0.9723052382469177, + -0.0935530960559845, + 0.12432572990655899, + -0.21406587958335876, + -0.031927481293678284, + 0.16789288818836212, + 0.07786032557487488, + 0.7261624336242676, + -0.14685063064098358, + -0.10996684432029724 + ], + [ + -0.04979391396045685, + -0.11791586130857468, + -0.004456192255020142, + 0.05855506658554077, + 1.7235608100891113, + 0.11105094105005264, + -0.024040699005126953, + -0.12752288579940796, + -0.014224421232938766, + 0.07338318228721619, + 0.010501477867364883, + 1.0183629989624023, + 0.0008610785007476807, + 0.2317407727241516, + -0.18003568053245544, + -0.062449511140584946, + 0.06018295884132385, + 0.04666796326637268, + 0.8320387005805969, + -0.10812510550022125, + -0.11861711740493774 + ], + [ + -0.11588315665721893, + -0.1507032811641693, + 0.13556641340255737, + 0.21135267615318298, + 1.7329940795898438, + 0.10381265729665756, + 0.4070972204208374, + -0.1715426743030548, + -0.005405336618423462, + 0.11465444415807724, + 0.0011274665594100952, + 0.9490469098091125, + -0.02240428328514099, + 0.30045509338378906, + -0.1598256230354309, + -0.02153543382883072, + 0.1829117089509964, + 0.0506272166967392, + 0.8583177924156189, + -0.06775277853012085, + 0.08185550570487976 + ], + [ + -0.12218575179576874, + -0.05966152995824814, + -0.03340598940849304, + 0.17769727110862732, + 1.6792542934417725, + -0.022392183542251587, + 0.007063388824462891, + -0.023685961961746216, + 0.008037716150283813, + 0.13172943890094757, + -0.10718705505132675, + 1.0250279903411865, + -0.05941393971443176, + 0.2312353253364563, + -0.3399696350097656, + -0.00429912656545639, + 0.15994682908058167, + 0.10264082252979279, + 0.8822011947631836, + -0.2537164092063904, + -0.16122901439666748 + ], + [ + -0.3161572813987732, + -0.08427201211452484, + 0.16332033276557922, + -0.06050175428390503, + 1.7552568912506104, + -0.23629158735275269, + 0.22806505858898163, + -0.0823492705821991, + 0.0047934032045304775, + -0.02825797349214554, + -0.10792965441942215, + 1.0314810276031494, + 0.23326674103736877, + 0.4444473385810852, + -0.30170097947120667, + -0.020457323640584946, + 0.05768272280693054, + -0.07658644020557404, + 1.0223991870880127, + -0.09513147175312042, + 0.1868864893913269 + ] + ], + "generated_utilities": [ + 0.7181311249732971, + 0.655661404132843, + 0.6736156344413757, + 0.7093296051025391, + 0.7143670916557312, + 0.5601552128791809, + 0.7090306282043457, + 0.5006552338600159 + ], + "group_id": "ms-6e268d1e2915740816a1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7257392406463623, + 0.7165254950523376, + 2.0, + 0.72465980052948, + 0.7087006568908691, + 0.2306610643863678, + 0.7143489122390747, + 0.7045385837554932, + 0.5905889272689819, + 0.1919487714767456, + 0.1914421170949936, + 0.2230607271194458, + 0.20312398672103882, + 0.19757536053657532, + 0.19724789261817932, + 0.19653666019439697 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.44810131192207336, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0264720916748047, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.9882198572158813 + ], + [ + -0.4707597494125366, + 0.09937268495559692, + 0.6395220160484314, + -0.40935325622558594, + 2.4514670372009277, + -0.02397046983242035, + -0.21874001622200012, + 0.16799744963645935, + 0.0360843725502491, + -0.09719255566596985, + -0.8619975447654724, + 2.2499821186065674, + -0.4933375120162964, + 0.35259634256362915, + -0.34735018014907837, + -0.517992377281189, + 0.6015943288803101, + -0.7677568793296814, + 1.5267367362976074, + -0.16888000071048737, + 0.3771851658821106 + ], + [ + -0.8530820608139038, + -0.6707327961921692, + 0.7970488667488098, + 0.34902632236480713, + 2.1393990516662598, + 0.5651285648345947, + 0.113044872879982, + 0.2987540364265442, + 0.4139185845851898, + 0.41691792011260986, + -0.4906405806541443, + 1.5687955617904663, + -0.8899085521697998, + -0.01883232593536377, + 0.24473188817501068, + 0.554391622543335, + 0.23424841463565826, + -0.10042615234851837, + 2.1214542388916016, + -0.15780393779277802, + 0.5807479023933411 + ], + [ + -0.6562768220901489, + -0.059059903025627136, + 0.24766850471496582, + 0.06101328134536743, + 2.5052053928375244, + 0.5204365849494934, + 0.1595638245344162, + 0.47176462411880493, + 0.3036114573478699, + 0.13197286427021027, + -0.2551935017108917, + 2.185537099838257, + -1.0984127521514893, + 0.3213496804237366, + 0.49875718355178833, + 0.1223822832107544, + 0.2575722932815552, + -0.7017092704772949, + 1.7495591640472412, + -0.19501343369483948, + 1.1214004755020142 + ], + [ + -0.45427173376083374, + -0.041353583335876465, + 0.8143320083618164, + -0.39680930972099304, + 2.4542014598846436, + 0.382559597492218, + -0.9993512630462646, + 0.5202391147613525, + 0.5282149314880371, + 0.4165073335170746, + 0.43153735995292664, + 2.5625123977661133, + -0.721943199634552, + 0.6765530109405518, + -0.6697012186050415, + -0.0637422502040863, + 0.6919896602630615, + -1.033813714981079, + 2.317282199859619, + -0.13503669202327728, + -0.07009828090667725 + ], + [ + -0.548885703086853, + -0.609938383102417, + 0.7181543707847595, + 0.11203774809837341, + 2.478057861328125, + -0.6170704960823059, + -1.0383654832839966, + 0.3524467349052429, + 0.5748242139816284, + 0.42987847328186035, + 0.013119548559188843, + 1.3954555988311768, + -0.6403790712356567, + 0.025414258241653442, + -0.7780460119247437, + -0.23124903440475464, + 0.48347827792167664, + 0.05266502499580383, + 1.692553997039795, + 0.362108051776886, + 0.6330842971801758 + ], + [ + -1.3070502281188965, + -0.947432816028595, + 0.4198402762413025, + -1.1768442392349243, + 1.7843327522277832, + 0.7587301731109619, + 0.7447960376739502, + 0.9411643147468567, + 0.3013244867324829, + -0.3862171769142151, + -1.0263384580612183, + 2.97957181930542, + 0.1849566102027893, + 0.08799406886100769, + 0.1984272301197052, + 0.5277765393257141, + 0.2931878864765167, + -1.1385071277618408, + 1.884846568107605, + 0.23083388805389404, + 0.6359583139419556 + ], + [ + -0.5375385880470276, + 0.16159841418266296, + 1.260543942451477, + -0.6744770407676697, + 2.1646244525909424, + 0.6190755367279053, + -0.746461808681488, + -0.38674360513687134, + 0.8897944092750549, + -0.594306468963623, + -0.05647379159927368, + 1.7390328645706177, + -0.037059009075164795, + 0.470071941614151, + 0.010003047063946724, + 0.08421775698661804, + -0.4613104462623596, + -0.22854508459568024, + 1.772307276725769, + -0.29165884852409363, + 0.6179300546646118 + ], + [ + -0.6489957571029663, + 0.4526708126068115, + 0.5115790367126465, + 0.31459903717041016, + 2.9655239582061768, + 0.07586031407117844, + -1.1667817831039429, + -0.7146965861320496, + 1.0295621156692505, + 0.1932714432477951, + -0.2703063488006592, + 2.9046523571014404, + -1.5882594585418701, + 0.3030351996421814, + 0.6543201208114624, + -0.448961466550827, + 0.4115087389945984, + -0.4087216854095459, + 2.6432881355285645, + -0.6797424554824829, + 0.2669999599456787 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.22510427236557007, + -0.26737168431282043, + 0.10375632345676422, + 0.12666502594947815, + 1.379244327545166, + -0.01088210940361023, + 0.0, + 0.23721180856227875, + -0.10981421172618866, + -0.021816372871398926, + -0.071512371301651, + 1.0360438823699951, + 0.09019589424133301, + 0.0, + 0.03181244805455208, + -0.06466590613126755, + 0.2068846970796585, + 0.14229199290275574, + 0.7610219717025757, + -0.03211131691932678, + 0.0 + ] + ], + "predicted_scores": [ + 4.490880966186523, + 4.102152347564697, + 3.7473788261413574, + 3.7030088901519775, + 3.611773729324341, + 3.593114137649536, + 3.482645034790039, + 3.4754390716552734 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1475440973", + "selected_index": 0, + "state_hash": "38ff5a264c3c39ff1ec8756d84e622963a11ff1ac61568ba57fbea0c114c3140", + "stored_base_utility": 0.7257392406463623, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7239725589752197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7239725589752197 + }, + "base_progress": 0.7239725589752197, + "base_success": false, + "base_utility": 0.7239725589752197, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7220139503479004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7220139503479004 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.650579035282135, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.650579035282135 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7230307459831238, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7230307459831238 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.7220139503479004, + 1.0, + 1.0, + 0.650579035282135, + 1.0, + 0.7230307459831238, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-8864796e31bbe93d7c5d", + "generated_tangents": [ + [ + 0.08788160979747772, + -0.2383939027786255, + -0.032137900590896606, + -0.24412868916988373, + 1.4860432147979736, + -0.21342450380325317, + 0.09996834397315979, + 0.1382390707731247, + -0.15099269151687622, + -0.020627014338970184, + -0.142374187707901, + 1.287447214126587, + -0.051568299531936646, + 0.22556555271148682, + -0.1834896206855774, + -0.0788751095533371, + 0.1916874647140503, + 0.022342637181282043, + 1.0566089153289795, + -0.5432512760162354, + -0.18783268332481384 + ], + [ + -0.05326157808303833, + -0.22410079836845398, + 0.09589865803718567, + -0.13422852754592896, + 1.596210241317749, + -0.4352465569972992, + 0.4212307929992676, + 0.16992929577827454, + -0.09806247055530548, + -0.0010492801666259766, + 0.17228838801383972, + 1.011678695678711, + -0.0015822947025299072, + 0.22310665249824524, + -0.19998541474342346, + -0.005391527898609638, + 0.14887210726737976, + -0.1188158243894577, + 1.1391692161560059, + -0.7205156087875366, + -0.13284984230995178 + ], + [ + -0.04978486895561218, + -0.19524957239627838, + 0.026632994413375854, + -0.16501560807228088, + 1.507580280303955, + -0.34590357542037964, + 0.40128493309020996, + 0.12344897538423538, + -0.10562635958194733, + -0.0436982586979866, + 0.033674925565719604, + 1.1360576152801514, + -0.11289465427398682, + 0.13492539525032043, + -0.2354370653629303, + -0.05626790225505829, + 0.17636601626873016, + -0.12393470108509064, + 1.126055121421814, + -0.7666268348693848, + -0.3608725368976593 + ], + [ + -0.01939263939857483, + -0.24679048359394073, + 0.03107011318206787, + -0.4890434145927429, + 1.533837080001831, + -0.4929920732975006, + 0.25275158882141113, + 0.2871931791305542, + -0.21316978335380554, + -0.0804167315363884, + 0.1186186671257019, + 1.0964572429656982, + -0.24419304728507996, + 0.5774073004722595, + -0.31122124195098877, + -0.08002664148807526, + 0.3209894001483917, + -0.00044849514961242676, + 1.0110256671905518, + -0.17076098918914795, + -0.4503469169139862 + ], + [ + 0.06518993526697159, + -0.2147245556116104, + -0.0637262761592865, + -0.17908763885498047, + 1.4877252578735352, + -0.3291391134262085, + -0.018538206815719604, + 0.11814204603433609, + -0.14750580489635468, + 0.061548177152872086, + -0.23010414838790894, + 1.2080109119415283, + -0.09372454881668091, + 0.09325084090232849, + -0.22445476055145264, + -0.09234264492988586, + 0.2706632912158966, + 0.044122859835624695, + 0.9624550342559814, + -0.6028077602386475, + -0.22579365968704224 + ], + [ + -0.00460207462310791, + -0.2785096764564514, + 0.15270927548408508, + -0.18722903728485107, + 1.50177001953125, + -0.19188493490219116, + 0.4806599020957947, + -0.026199862360954285, + -0.05291987955570221, + 0.03865250200033188, + -0.1597832441329956, + 1.3016701936721802, + 0.031150877475738525, + 0.22909942269325256, + -0.23353204131126404, + -0.06382535398006439, + 0.25345098972320557, + -0.08922843635082245, + 1.1146588325500488, + -0.5792299509048462, + 0.06494437158107758 + ], + [ + -0.21892715990543365, + -0.1725882887840271, + 0.11764785647392273, + -0.29704079031944275, + 1.6328375339508057, + -0.6228445768356323, + 0.6463073492050171, + 0.14286065101623535, + -0.18001601099967957, + -0.2367975413799286, + -0.14022648334503174, + 1.2718926668167114, + 0.10375061631202698, + 0.6114175319671631, + -0.3505202531814575, + -0.05591515079140663, + 0.232391819357872, + -0.10737200081348419, + 1.3376154899597168, + -0.4605124592781067, + -0.07286329567432404 + ], + [ + -0.022343099117279053, + -0.16998058557510376, + -0.030634313821792603, + -0.08502086997032166, + 1.4842581748962402, + -0.37542346119880676, + 0.16227364540100098, + 0.20214256644248962, + -0.11115231364965439, + -0.009610556066036224, + -0.2007465362548828, + 1.3154736757278442, + -0.12881764769554138, + 0.2673048675060272, + -0.31555965542793274, + -0.03339279443025589, + 0.26468798518180847, + 0.05638900399208069, + 1.1049151420593262, + -0.7345172762870789, + -0.23666653037071228 + ] + ], + "generated_utilities": [ + 0.7220139503479004, + 2.0, + 2.0, + 0.650579035282135, + 2.0, + 0.7230307459831238, + 2.0, + 2.0 + ], + "group_id": "ms-8864796e31bbe93d7c5d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.728268027305603, + 0.7257949113845825, + 2.0, + 0.7287554740905762, + 0.72548508644104, + 0.7011849880218506, + 0.7089968919754028, + 0.689879298210144, + 0.722068727016449, + 0.7046384811401367, + 0.1933039277791977, + 0.629345178604126, + 0.18965840339660645, + 0.18740439414978027, + 0.2032846063375473, + 0.20379847288131714 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.3859826922416687, + 0.1119421124458313, + 0.6443181037902832, + 0.11836245656013489, + 1.863237738609314, + -1.166780948638916, + 0.8025261163711548, + -0.5342166423797607, + -0.2057022899389267, + 0.8987585306167603, + -0.525823712348938, + 1.6212486028671265, + 0.25674968957901, + 0.4037417471408844, + -0.4561460018157959, + -0.18429945409297943, + -0.1778680682182312, + -0.22491087019443512, + 2.095893144607544, + -0.3484030067920685, + 0.47628670930862427 + ], + [ + -0.446367084980011, + -0.19229409098625183, + -0.03271818161010742, + 0.39123907685279846, + 1.1786221265792847, + -0.16883885860443115, + -0.3337137699127197, + 0.7789723873138428, + -0.12953178584575653, + -0.028340458869934082, + -0.871502161026001, + 0.7702060341835022, + -0.645424485206604, + 1.0858956575393677, + -0.05413459241390228, + -0.14592209458351135, + -0.03685358166694641, + -0.686826229095459, + 2.084512710571289, + 0.11898508667945862, + 0.20845119655132294 + ], + [ + 0.09377387166023254, + -0.03508002683520317, + -0.29283326864242554, + -0.9559528827667236, + 2.0994086265563965, + 0.02743762731552124, + 0.6819101572036743, + 0.607413113117218, + 0.4378519356250763, + 0.6497513055801392, + -0.5086554288864136, + 0.9556822180747986, + -0.8518581986427307, + 0.31636691093444824, + -0.4196639060974121, + 0.6866374015808105, + 0.5175505876541138, + -0.572692334651947, + 1.651871681213379, + -0.7227057814598083, + -0.33861052989959717 + ], + [ + -0.42610836029052734, + 0.10144555568695068, + 0.7105271220207214, + -1.291037678718567, + 2.2307684421539307, + 0.17725014686584473, + -0.08436906337738037, + 0.9681323766708374, + 0.5482062697410583, + 1.368379831314087, + -1.021532416343689, + 2.5197110176086426, + -0.3508763909339905, + 0.7584415078163147, + 1.0978648662567139, + 0.2626119554042816, + 0.6115584373474121, + -0.7777479290962219, + 1.7278802394866943, + -0.15240566432476044, + 0.4239119291305542 + ], + [ + 1.0207308530807495, + 0.8557561039924622, + 0.4489574134349823, + -1.0117379426956177, + 0.8262714147567749, + -0.630092442035675, + 0.09812727570533752, + 0.21663640439510345, + 0.23771141469478607, + -0.47456812858581543, + -1.3189021348953247, + 2.269665002822876, + -0.7066116333007812, + 1.1826255321502686, + 1.0058563947677612, + 0.1694127470254898, + -0.5649446845054626, + -0.32043421268463135, + 0.46698564291000366, + -0.9052557945251465, + -0.12436902523040771 + ], + [ + -0.5202209949493408, + 0.30583277344703674, + 1.2163622379302979, + 0.4305659234523773, + 1.7276779413223267, + -0.1535550355911255, + -0.5289345383644104, + -0.632349967956543, + 0.9393473267555237, + 0.38391292095184326, + -1.1998018026351929, + 1.9739618301391602, + -1.3405330181121826, + 0.048939138650894165, + 0.6369897127151489, + 0.629721999168396, + 1.388192892074585, + -1.065996766090393, + 1.9119879007339478, + -1.0432264804840088, + 0.9627590179443359 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.13860583305358887, + -0.06957411766052246, + 0.1504676342010498, + 0.1671028435230255, + 0.5675687789916992, + 0.16587841510772705, + 0.0, + -0.21137550473213196, + -0.21542736887931824, + 0.2565074861049652, + -0.00923699140548706, + 0.5251069068908691, + -0.06964048743247986, + 0.0, + -0.19547803699970245, + 0.10278619825839996, + -0.19206219911575317, + -0.1487000733613968, + 0.44575393199920654, + 0.0725468099117279, + 0.0 + ] + ], + "predicted_scores": [ + 3.7122392654418945, + 3.7117695808410645, + 3.6351206302642822, + 3.6117537021636963, + 3.4393064975738525, + 3.282926082611084, + 3.219355344772339, + 2.964003086090088 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "551504635", + "selected_index": 0, + "state_hash": "ab6587e0b2a3b5f21f23c56daf8894249774912a3b20794bd49dc1b78ec736a7", + "stored_base_utility": 0.728268027305603, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.42231252789497375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.42231252789497375 + }, + "base_progress": 0.42231252789497375, + "base_success": false, + "base_utility": 0.42231252789497375, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.6624154448509216, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6624154448509216 + }, + { + "contact_quality": null, + "progress": 0.22238026559352875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22238026559352875 + }, + { + "contact_quality": null, + "progress": 0.7256770730018616, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7256770730018616 + }, + { + "contact_quality": null, + "progress": 0.20876221358776093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20876221358776093 + }, + { + "contact_quality": null, + "progress": 0.39452430605888367, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.39452430605888367 + }, + { + "contact_quality": null, + "progress": 0.2241959571838379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2241959571838379 + }, + { + "contact_quality": null, + "progress": 0.21765725314617157, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21765725314617157 + }, + { + "contact_quality": null, + "progress": 0.2077002078294754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2077002078294754 + } + ], + "candidate_progress": [ + 0.6624154448509216, + 0.22238026559352875, + 0.7256770730018616, + 0.20876221358776093, + 0.39452430605888367, + 0.2241959571838379, + 0.21765725314617157, + 0.2077002078294754 + ], + "candidate_source_chart_ids": [ + "ms-8bc7aa419d17368d81b5", + "ms-13d8bae55d3c3710138f", + "ms-df961fe24a43e56d1716", + "ms-0660d5f65df5fc4b6132", + "ms-8bc7aa419d17368d81b5", + "ms-e1b6bcb41532a800d3cb", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank0", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b04b489f4fbad93b65ca", + "generated_tangents": [ + [ + -0.473727285861969, + -0.20458243787288666, + 0.3475378751754761, + 0.8252710103988647, + 0.24466358125209808, + 0.17423851788043976, + -0.03964203596115112, + -0.0001462697982788086, + 0.006239580921828747, + -0.043422676622867584, + -0.26038965582847595, + 1.1473183631896973, + -0.1878947913646698, + 0.04016990214586258, + 0.15174512565135956, + -0.06292980164289474, + -0.0837620347738266, + 0.11048615723848343, + 1.1570909023284912, + -0.30989381670951843, + 0.030177026987075806 + ], + [ + -0.1426912099123001, + -0.19952040910720825, + 0.7745941877365112, + 0.8301906585693359, + 0.20359830558300018, + -0.17863845825195312, + 0.18299150466918945, + -0.12927794456481934, + 0.16794560849666595, + -0.04269062727689743, + -0.29995664954185486, + 1.075852394104004, + -0.1310909390449524, + 0.10846757143735886, + 0.11009477078914642, + 0.006240988150238991, + -0.03404878079891205, + -0.02286781370639801, + 1.2750709056854248, + -0.24338920414447784, + -0.14805647730827332 + ], + [ + -0.17457570135593414, + -0.11718995124101639, + 0.7044237852096558, + 0.8478370904922485, + 0.11586553603410721, + 0.13718518614768982, + 0.03240591287612915, + -0.013515517115592957, + 0.06891965121030807, + -0.1174798533320427, + -0.29726698994636536, + 1.138895034790039, + -0.26907211542129517, + 0.06772702932357788, + 0.19067975878715515, + -0.008191946893930435, + 0.05094030499458313, + -0.040464892983436584, + 1.3260605335235596, + -0.41114822030067444, + -0.24100342392921448 + ], + [ + -0.09082283079624176, + -0.15988582372665405, + 0.8348315954208374, + 0.9153009653091431, + 0.21004009246826172, + -0.11935034394264221, + 0.026137322187423706, + -0.10805058479309082, + 0.16905617713928223, + -0.1175743117928505, + -0.39345020055770874, + 1.2242575883865356, + -0.10758864879608154, + 0.12500585615634918, + 0.13719435036182404, + 0.03037506714463234, + 0.014696374535560608, + -0.006907284259796143, + 1.1555595397949219, + -0.337271511554718, + -0.4327709376811981 + ], + [ + -0.4098537564277649, + -0.21750971674919128, + 0.232845276594162, + 0.7502044439315796, + 0.006616652011871338, + -0.00454305112361908, + -0.010837703943252563, + -0.08243179321289062, + 0.06806407123804092, + -0.16616925597190857, + -0.07266721874475479, + 1.1514806747436523, + -0.29811978340148926, + 0.0646565705537796, + 0.10928624868392944, + -0.1019655168056488, + 0.12408410757780075, + 0.06523144245147705, + 1.180974006652832, + -0.2171332836151123, + 0.07417988777160645 + ], + [ + -0.24468393623828888, + -0.16898521780967712, + 0.7371393442153931, + 0.8363014459609985, + 0.13242918252944946, + 0.010619372129440308, + -0.013288170099258423, + -0.12980088591575623, + 0.21491104364395142, + -0.04816807061433792, + -0.3035173714160919, + 1.2466926574707031, + -0.3947046399116516, + 0.41484034061431885, + 0.16444221138954163, + -0.039556294679641724, + -0.06334884464740753, + 0.02021704614162445, + 1.27815580368042, + -0.2796225845813751, + -0.030325546860694885 + ], + [ + -0.09441088140010834, + -0.08173932135105133, + 0.6346042156219482, + 0.961563229560852, + 0.24901160597801208, + -0.10678330063819885, + -0.11417979001998901, + -0.07817527651786804, + 0.16771452128887177, + -0.09400541335344315, + -0.3943471610546112, + 1.3348896503448486, + -0.2699192762374878, + 0.4237706661224365, + 0.029781069606542587, + -0.04438348859548569, + 0.06384485960006714, + 0.05673319101333618, + 1.1843684911727905, + -0.4529908299446106, + -0.2420814037322998 + ], + [ + 0.008482202887535095, + -0.18334625661373138, + 0.8706309795379639, + 0.8725862503051758, + 0.1143718883395195, + -0.0940537303686142, + -0.001456618309020996, + -0.021944910287857056, + 0.2779668867588043, + -0.08584963530302048, + -0.44209742546081543, + 1.2187726497650146, + 0.06693956255912781, + 0.26411381363868713, + 0.07009257376194, + 0.015542935580015182, + -0.03625498712062836, + 0.016178786754608154, + 1.3165087699890137, + -0.25032639503479004, + -0.20187541842460632 + ] + ], + "generated_utilities": [ + 0.6624154448509216, + 0.22238026559352875, + 0.7256770730018616, + 0.20876221358776093, + 0.39452430605888367, + 0.2241959571838379, + 0.21765725314617157, + 0.2077002078294754 + ], + "group_id": "ms-b04b489f4fbad93b65ca", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7198216915130615, + 0.7247374057769775, + 2.0, + 0.22730140388011932, + 0.21130704879760742, + 0.2076549530029297, + 0.2006116509437561, + 0.21285679936408997, + 0.19614991545677185, + 0.20440611243247986, + 0.2015489637851715, + 0.19511334598064423, + 0.20091164112091064, + 0.19681888818740845, + 0.19675713777542114, + 0.20853900909423828 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.30706143379211426, + 0.2797066569328308, + 0.4565920829772949, + 0.7342244386672974, + -0.28428468108177185, + 0.012420237064361572, + 0.0, + 0.19500797986984253, + -0.1289721131324768, + -0.3524337112903595, + 0.07460063695907593, + 1.7684156894683838, + 0.11335399746894836, + 0.0, + -0.5014548301696777, + 0.4537479281425476, + 0.1963198333978653, + -0.025568604469299316, + 1.6286182403564453, + -0.0963512659072876, + 0.0 + ], + [ + 1.3070614337921143, + -0.7202933430671692, + 1.456592082977295, + 1.7342244386672974, + -0.15130555629730225, + 0.9519147872924805, + -1.4541058540344238, + -0.6432010531425476, + -0.26959508657455444, + -0.3406839668750763, + -0.4419664740562439, + 2.768415689468384, + -0.3271861970424652, + -0.13685251772403717, + 0.23842881619930267, + 0.20406432449817657, + 0.3414191007614136, + -1.0255686044692993, + 2.6286182403564453, + -0.9051820039749146, + 0.6850019693374634 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9082117080688477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.27370503544807434, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.3700039386749268 + ], + [ + 1.0634682178497314, + -0.8590207099914551, + 1.6810569763183594, + 1.8525642156600952, + 0.06716938316822052, + 1.0774534940719604, + -1.177553415298462, + -0.7412493228912354, + -0.3193308115005493, + -0.4253617525100708, + -0.4890587329864502, + 2.624048948287964, + -0.109145388007164, + -0.10380380600690842, + 0.20137116312980652, + 0.20110605657100677, + 0.06929725408554077, + -0.9908366203308105, + 2.5783159732818604, + -0.7248227000236511, + 0.919882595539093 + ], + [ + 1.2083125114440918, + -0.9112091064453125, + 1.1255995035171509, + 1.9678500890731812, + -0.3424606919288635, + 1.350451946258545, + -1.5799542665481567, + -1.014689326286316, + -0.3427739441394806, + -0.24094250798225403, + -0.4992668032646179, + 2.403038740158081, + -0.29080134630203247, + -0.1569681167602539, + 0.4877905249595642, + -0.16464389860630035, + 0.41957730054855347, + -0.6997033357620239, + 2.80957293510437, + -1.0509651899337769, + 0.8937826156616211 + ], + [ + 1.2133954763412476, + -1.0171141624450684, + 1.1572445631027222, + 1.8050156831741333, + -0.29570114612579346, + 0.7672240734100342, + -0.976697564125061, + -0.8219587802886963, + -0.746895432472229, + -0.346658855676651, + -0.7942779064178467, + 3.0403196811676025, + -0.6172577738761902, + 0.3615778088569641, + 0.546725332736969, + -0.06415437161922455, + 0.43834370374679565, + -1.332592248916626, + 3.029940605163574, + -0.5720348358154297, + 1.1660562753677368 + ], + [ + 0.7937494516372681, + -0.45201122760772705, + 2.0367026329040527, + 2.1590728759765625, + 0.38935917615890503, + 0.5424834489822388, + -1.391097903251648, + -1.0657228231430054, + -0.44411367177963257, + -0.3797546923160553, + -0.03612968325614929, + 2.867069721221924, + -0.26322364807128906, + -0.32477179169654846, + 0.007963955402374268, + 0.053216055035591125, + 0.36466139554977417, + -0.7477234601974487, + 2.170649766921997, + -0.46266868710517883, + 0.7669195532798767 + ], + [ + 1.0920023918151855, + -1.0678586959838867, + 1.4391454458236694, + 2.155820608139038, + -0.7347199320793152, + 0.6208673715591431, + -1.6380563974380493, + -0.6589961051940918, + -0.8267081379890442, + 0.011062979698181152, + -0.1169244647026062, + 2.743579626083374, + -0.41342735290527344, + -0.5980290770530701, + 0.49618369340896606, + -0.09863747656345367, + 0.2179805338382721, + -0.6006474494934082, + 2.2965025901794434, + -0.97348952293396, + 1.2447597980499268 + ], + [ + 1.8286218643188477, + -1.486896276473999, + 1.8610713481903076, + 1.792364478111267, + -0.06209900975227356, + 0.7138283252716064, + -1.0023046731948853, + -0.8588749170303345, + -0.30936315655708313, + 0.33444294333457947, + 0.2771298885345459, + 3.0452492237091064, + -0.6618144512176514, + -0.747363805770874, + 0.19540274143218994, + -0.05051566660404205, + 0.5057645440101624, + -0.3770519495010376, + 3.028423309326172, + -0.19366812705993652, + 1.1766406297683716 + ], + [ + 1.057653784751892, + -0.15160071849822998, + 1.1646175384521484, + 2.517653226852417, + 0.037323400378227234, + 1.4844036102294922, + -1.2659661769866943, + -0.9694168567657471, + -0.792635440826416, + 0.15484732389450073, + -0.6592323780059814, + 2.426063060760498, + -0.8063763976097107, + 0.37280529737472534, + -0.1984816938638687, + 0.3488052487373352, + 0.36773377656936646, + -0.9600323438644409, + 1.801730990409851, + -1.1618295907974243, + 0.526383101940155 + ], + [ + 0.8403785228729248, + -1.7131881713867188, + 1.7484346628189087, + 1.3915244340896606, + 0.038706690073013306, + 1.7859117984771729, + -0.6996253132820129, + -0.12596702575683594, + -0.07181461155414581, + -0.8556877374649048, + -0.6567700505256653, + 2.935109853744507, + 0.3808797299861908, + 0.1733608990907669, + 0.9097857475280762, + 0.7001722455024719, + 0.14087507128715515, + -0.32659411430358887, + 2.278188943862915, + -1.87721586227417, + 1.438266634941101 + ], + [ + 1.259635329246521, + -0.5244767665863037, + 2.201894998550415, + 2.2337045669555664, + -0.333431601524353, + 0.7420244216918945, + -2.0896377563476562, + 0.00931471586227417, + 0.6261725425720215, + -0.49785688519477844, + -0.9404669404029846, + 3.756554365158081, + 0.5252982378005981, + -0.3158212900161743, + 0.4053819179534912, + 0.8204222917556763, + 0.5041900277137756, + -0.43809741735458374, + 3.519717216491699, + -1.0316743850708008, + 0.3069072663784027 + ], + [ + 0.9530733823776245, + -0.011007547378540039, + 2.396453380584717, + 2.6382436752319336, + 0.29263433814048767, + 1.6715352535247803, + -0.819398045539856, + -0.5916767120361328, + 0.006437480449676514, + -0.3942071795463562, + -0.5187541842460632, + 3.677276849746704, + -0.4376804232597351, + 0.3331298828125, + -0.3693196773529053, + -0.3432707190513611, + 1.104022741317749, + -1.0780245065689087, + 3.0859882831573486, + -0.26956111192703247, + 0.8377041816711426 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.30706143379211426, + 0.024911105632781982, + 0.4565920829772949, + 0.7342244386672974, + 0.01698923110961914, + -0.04808521270751953, + 0.0, + 0.10375595092773438, + 0.23247671127319336, + -0.12021404504776001, + -0.19280612468719482, + 1.7684156894683838, + -0.23127278685569763, + 0.0, + -0.1010780781507492, + 0.00952194631099701, + -0.16307592391967773, + -0.14496707916259766, + 1.6286182403564453, + -0.28656232357025146, + 0.0 + ] + ], + "predicted_scores": [ + 1.8552892208099365, + 1.3998653888702393, + 1.3820189237594604, + 0.7555887699127197, + 0.6081521511077881, + 0.341136634349823, + -0.13221046328544617, + -0.282153457403183 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113268940", + "selected_index": 0, + "state_hash": "76e9a962970055fa7565c007db1b35e003cd352908be4cf573066d8b58e2d443", + "stored_base_utility": 0.7198216915130615, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7171411514282227, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7171411514282227 + }, + { + "contact_quality": null, + "progress": 0.7212136387825012, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7212136387825012 + }, + { + "contact_quality": null, + "progress": 0.6921262741088867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6921262741088867 + }, + { + "contact_quality": null, + "progress": 0.7184799313545227, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7184799313545227 + }, + { + "contact_quality": null, + "progress": 0.7070601582527161, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7070601582527161 + }, + { + "contact_quality": null, + "progress": 0.7001742720603943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7001742720603943 + }, + { + "contact_quality": null, + "progress": 0.7255622744560242, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7255622744560242 + }, + { + "contact_quality": null, + "progress": 0.6983166337013245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6983166337013245 + } + ], + "candidate_progress": [ + 0.7171411514282227, + 0.7212136387825012, + 0.6921262741088867, + 0.7184799313545227, + 0.7070601582527161, + 0.7001742720603943, + 0.7255622744560242, + 0.6983166337013245 + ], + "candidate_source_chart_ids": [ + "ms-062addb9da32a0c2ea97", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69", + "ms-b483cac145e16ccdf42a", + "ms-efbde8652e75274fbd69", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69", + "ms-b483cac145e16ccdf42a" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-c663f7e73e7e8376a05c", + "generated_tangents": [ + [ + 0.018496453762054443, + -0.08482980728149414, + -0.21568617224693298, + 0.09971177577972412, + 1.6243064403533936, + 0.10376907140016556, + 0.020362496376037598, + -0.014710485935211182, + -0.0076336730271577835, + -0.028992556035518646, + -0.09916447848081589, + 1.0256421566009521, + -0.04061979055404663, + 0.05338793247938156, + -0.09438146650791168, + -0.0546598955988884, + 0.10169292986392975, + 0.04009826481342316, + 1.0276858806610107, + -0.12354743480682373, + -0.05365872383117676 + ], + [ + -0.05339689552783966, + -0.10680296272039413, + -0.07668808102607727, + 0.1379472017288208, + 1.5675547122955322, + 0.1497737318277359, + -0.010280996561050415, + -0.055953145027160645, + -0.01853945478796959, + 0.0033985525369644165, + -0.09700677543878555, + 0.9669918417930603, + -0.15829509496688843, + 0.37960100173950195, + -0.019056398421525955, + -0.04062753543257713, + 0.07642152905464172, + 0.07597062736749649, + 0.8810792565345764, + 0.0319751501083374, + -0.0648588091135025 + ], + [ + -0.03835053741931915, + -0.10996833443641663, + -0.09504029154777527, + 0.09926941990852356, + 1.6195778846740723, + 0.17754143476486206, + 0.08033245801925659, + -0.030462920665740967, + 0.0072348159737885, + 0.016059696674346924, + -0.10309233516454697, + 1.0032544136047363, + -0.18356046080589294, + 0.33337730169296265, + -0.037213798612356186, + -0.03972916305065155, + 0.10433965921401978, + 0.08191090077161789, + 0.888705313205719, + 0.03629118204116821, + 0.0033277571201324463 + ], + [ + 0.09127193689346313, + -0.055760692805051804, + -0.1374853551387787, + 0.07743537425994873, + 1.5472488403320312, + -0.00636233389377594, + 0.04363471269607544, + -0.00877659022808075, + 0.029708439484238625, + 0.05226445198059082, + -0.05625463277101517, + 0.8850756883621216, + 0.035779207944869995, + 0.015579409897327423, + -0.09338870644569397, + -0.0615127794444561, + 0.1847681999206543, + 0.05029042065143585, + 1.1262969970703125, + -0.029379457235336304, + 0.02394700050354004 + ], + [ + 0.04843737930059433, + -0.0003103986382484436, + -0.2789051830768585, + 0.16971105337142944, + 1.7634544372558594, + 0.504429280757904, + -0.27973997592926025, + -0.01923424005508423, + -0.021087726578116417, + 0.08221027255058289, + -0.2855764627456665, + 0.6937445402145386, + -0.07779139280319214, + 0.1338072568178177, + -0.13744857907295227, + 0.08373387157917023, + 0.19297969341278076, + -0.04320131242275238, + 0.9147657752037048, + -0.14992932975292206, + -0.1147456169128418 + ], + [ + 0.030949018895626068, + -0.05405410751700401, + -0.2067166268825531, + 0.09814414381980896, + 1.584350347518921, + 0.12914511561393738, + 0.032954901456832886, + -0.10106584429740906, + -0.015615474432706833, + 0.0002103671431541443, + -0.1916314959526062, + 0.9277292490005493, + -0.03392323851585388, + 0.12778586149215698, + -0.11368101835250854, + -0.07895809412002563, + 0.08963510394096375, + 0.051576659083366394, + 0.8910223245620728, + -0.002006441354751587, + -0.08677718043327332 + ], + [ + -0.01074892282485962, + -0.1378961205482483, + -0.2911452353000641, + 0.17210707068443298, + 1.687772274017334, + 0.11322876811027527, + -0.07297337055206299, + -0.05377556383609772, + -0.026861606165766716, + 0.023169003427028656, + -0.09588069468736649, + 0.8563389182090759, + -0.10155078768730164, + 0.2857053577899933, + 0.054930489510297775, + -0.07985828816890717, + -0.07473190128803253, + 0.060383960604667664, + 1.02923583984375, + -0.10958270728588104, + 0.020274892449378967 + ], + [ + -0.059429556131362915, + -0.07215353101491928, + -0.23873195052146912, + 0.1298384666442871, + 1.5628283023834229, + 0.12706680595874786, + 0.03786039352416992, + -0.09929674863815308, + 0.02840629406273365, + -0.03223598748445511, + -0.10132608562707901, + 0.8569584488868713, + -0.037724435329437256, + 0.07865964621305466, + -0.10110992193222046, + -0.07663080096244812, + 0.02457106113433838, + -0.005532324314117432, + 1.0890499353408813, + -0.1768168807029724, + -0.013077616691589355 + ] + ], + "generated_utilities": [ + 0.7171411514282227, + 0.7212136387825012, + 0.6921262741088867, + 0.7184799313545227, + 0.7070601582527161, + 0.7001742720603943, + 0.7255622744560242, + 0.6983166337013245 + ], + "group_id": "ms-c663f7e73e7e8376a05c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7233090400695801, + 0.7198824882507324, + 2.0, + 0.7120869159698486, + 0.27145418524742126, + 0.23090600967407227, + 0.2526220679283142, + 0.2656436562538147, + 0.264312207698822, + 0.27572086453437805, + 0.296668142080307, + 0.2128777951002121, + 0.22965002059936523, + 0.2196153998374939, + 0.22027263045310974, + 0.19939398765563965 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.038699887692928314, + -0.3685438632965088, + -1.3628361225128174, + 0.0675731897354126, + 2.4906270503997803, + 0.8971554040908813, + 0.06863358616828918, + 0.14689946174621582, + 0.0500560887157917, + 0.5852231383323669, + -0.5345680713653564, + 1.7433128356933594, + -0.5520187020301819, + 0.36285334825515747, + 0.06837204098701477, + 0.04091557487845421, + 0.26646411418914795, + -0.3870337903499603, + 1.7901486158370972, + -0.46956348419189453, + 0.5412080883979797 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.13726717233657837, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7257066965103149, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0824161767959595 + ], + [ + -0.09956050664186478, + -0.6439464092254639, + -1.5331499576568604, + -0.2275703251361847, + 2.2570624351501465, + 1.145668387413025, + -0.19933581352233887, + 0.15889883041381836, + -0.07524599134922028, + 0.6580387353897095, + -0.5754004716873169, + 1.6105921268463135, + -0.25637149810791016, + 0.37256744503974915, + 0.17931225895881653, + 0.286465048789978, + 0.07884949445724487, + -0.35266172885894775, + 1.548264980316162, + -0.2917717695236206, + 0.3307930827140808 + ], + [ + 0.13072630763053894, + -0.5927069783210754, + -1.4328954219818115, + 0.1252535879611969, + 2.5439467430114746, + 1.119347095489502, + -0.12134809792041779, + -0.09485669434070587, + -0.040146272629499435, + 0.4683093726634979, + -0.6791773438453674, + 2.1314187049865723, + -0.4368498623371124, + 0.5876930952072144, + 0.27027207612991333, + -0.1808907389640808, + 0.037005022168159485, + -0.6130290627479553, + 1.60811185836792, + -0.8427046537399292, + 0.5648807883262634 + ], + [ + -0.019807856529951096, + -0.40178078413009644, + -1.5710430145263672, + 0.3568149507045746, + 2.8693275451660156, + 1.2080440521240234, + 0.45903119444847107, + -0.07479061186313629, + 0.2587376832962036, + 0.37667450308799744, + -0.5607666373252869, + 1.9825769662857056, + -0.7874732613563538, + 0.48894837498664856, + -0.04161303490400314, + 0.2237108200788498, + 0.12043289840221405, + -0.8541218638420105, + 1.3131444454193115, + -0.8318716287612915, + 0.7653946876525879 + ], + [ + -0.039324238896369934, + 0.18316024541854858, + -1.2052042484283447, + -0.020045489072799683, + 2.441659927368164, + 1.3116326332092285, + 0.04364071041345596, + 0.617156982421875, + -0.5366188287734985, + 0.5837841629981995, + -0.6157102584838867, + 1.946446180343628, + -0.44197604060173035, + -0.12030342221260071, + 0.2635759711265564, + -0.34203875064849854, + 0.3788418769836426, + -0.01673591136932373, + 1.7612254619598389, + -0.8283271193504333, + 0.44542378187179565 + ], + [ + 0.020479684695601463, + -0.3728349208831787, + -1.2566280364990234, + -0.4903675317764282, + 2.7149407863616943, + 1.1410136222839355, + 0.18438707292079926, + 0.15751159191131592, + 0.5994289517402649, + 0.7441410422325134, + -0.5757432579994202, + 2.025078058242798, + -1.1044142246246338, + -0.24153441190719604, + 0.7623711824417114, + -0.07102327048778534, + 0.7491155862808228, + -0.853058397769928, + 1.1346031427383423, + -0.4604499638080597, + 0.3380756974220276 + ], + [ + -0.0438276082277298, + 0.21775567531585693, + -0.9180121421813965, + 0.2164689004421234, + 1.7669851779937744, + 0.7701845765113831, + -0.4171431064605713, + -0.44690096378326416, + -0.6473215818405151, + 0.5592721700668335, + 0.1537061333656311, + 1.2201651334762573, + -0.02862715721130371, + 0.759445309638977, + -0.08159622550010681, + -0.038715001195669174, + 0.8136870265007019, + -1.129607081413269, + 1.5099153518676758, + -1.2117259502410889, + 1.2496943473815918 + ], + [ + -0.6884663105010986, + -0.7670764923095703, + -1.3375780582427979, + 0.42161574959754944, + 2.4925031661987305, + 0.18111640214920044, + -0.4373469054698944, + 0.6228322386741638, + -0.3739543557167053, + 1.3811917304992676, + 0.05884122848510742, + 1.0582032203674316, + -0.38174134492874146, + 0.7282494902610779, + 0.9550073146820068, + -0.6478474736213684, + -0.467768132686615, + -0.22179017961025238, + 2.1456754207611084, + -0.39938730001449585, + 1.08339524269104 + ], + [ + 0.7042056322097778, + -0.4828507900238037, + -1.914047122001648, + -0.7084186673164368, + 1.87782621383667, + 0.42005008459091187, + 0.511604905128479, + -0.2043081521987915, + 0.9001545906066895, + 1.083567500114441, + -0.9796027541160583, + 1.3277345895767212, + -0.5564201474189758, + 0.6932777166366577, + -0.5934457778930664, + -0.296670138835907, + 0.14519014954566956, + -0.8612968325614929, + 0.8968887329101562, + -0.5584146976470947, + -0.434501588344574 + ], + [ + 0.19210004806518555, + 0.5024477243423462, + -0.4993329644203186, + -0.826042115688324, + 1.6430644989013672, + 0.9877098798751831, + 0.49057987332344055, + -0.13611945509910583, + 0.7228546738624573, + 1.4937405586242676, + 0.18012672662734985, + 2.188062906265259, + -0.6084107160568237, + 1.0874407291412354, + 0.5744171142578125, + 0.9375161528587341, + -0.43115919828414917, + 0.19787153601646423, + 2.652085781097412, + -0.529538631439209, + 0.03626328706741333 + ], + [ + -0.014053341001272202, + -1.351341724395752, + -0.7293765544891357, + 0.9410406947135925, + 3.139275074005127, + 1.3463419675827026, + -0.728021502494812, + -0.6353646516799927, + -0.19685329496860504, + 1.5315749645233154, + -1.2550475597381592, + 1.7411813735961914, + 0.14418578147888184, + 0.7038239240646362, + 0.1004495620727539, + 0.7590892910957336, + 0.9287852048873901, + -0.23726919293403625, + 0.8381044864654541, + -1.2906644344329834, + 0.9584261178970337 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03644278272986412, + -0.12137487530708313, + -0.3628361225128174, + -0.027813076972961426, + 1.4906270503997803, + -0.10284459590911865, + 0.0, + 0.15174832940101624, + 0.015595663338899612, + -0.10680150985717773, + -0.14723241329193115, + 0.7433128356933594, + 0.25961190462112427, + 0.0, + -0.23801246285438538, + -0.10784421861171722, + 0.09500086307525635, + -0.06586915254592896, + 0.7901486158370972, + -0.10978007316589355, + 0.0 + ] + ], + "predicted_scores": [ + 3.528031587600708, + 3.484841823577881, + 3.465104341506958, + 3.426609754562378, + 3.389573335647583, + 3.3381567001342773, + 3.1684746742248535, + 3.0973165035247803 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1528740417", + "selected_index": 0, + "state_hash": "c02fdfdd2339e4215d60b2221be925e066a7763f9e3bba3ff33a15f20e286f5a", + "stored_base_utility": 0.7233090400695801, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15342965722084045, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15342965722084045 + }, + "base_progress": 0.15342965722084045, + "base_success": false, + "base_utility": 0.15342965722084045, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0375710129737854, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0375710129737854 + }, + { + "contact_quality": null, + "progress": 0.4434976577758789, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4434976577758789 + }, + { + "contact_quality": null, + "progress": 0.025715351104736328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025715351104736328 + }, + { + "contact_quality": null, + "progress": 0.025355923920869827, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025355923920869827 + }, + { + "contact_quality": null, + "progress": 0.035843171179294586, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035843171179294586 + }, + { + "contact_quality": null, + "progress": 0.46439141035079956, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.46439141035079956 + }, + { + "contact_quality": null, + "progress": 0.022827064618468285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022827064618468285 + }, + { + "contact_quality": null, + "progress": 0.021351909264922142, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021351909264922142 + } + ], + "candidate_progress": [ + 0.0375710129737854, + 0.4434976577758789, + 0.025715351104736328, + 0.025355923920869827, + 0.035843171179294586, + 0.46439141035079956, + 0.022827064618468285, + 0.021351909264922142 + ], + "candidate_source_chart_ids": [ + "ms-a509a572b3871e591d26", + "ms-b7e7fe66a6c86898eec2", + "ms-307f9067d5b66f55b3a6", + "ms-4015c296239564108da2", + "ms-0541a144afe57b5a7ae6", + "ms-b7e7fe66a6c86898eec2", + "ms-5df54a8baaceb8c73c8a", + "ms-23a355317920e192147f" + ], + "candidate_source_task_ids": [ + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank6", + "ctt_transport_rank3", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "generated_tangents": [ + [ + -0.04052545130252838, + -0.2974761128425598, + 0.9359121322631836, + 0.10208287835121155, + 0.8169476389884949, + 0.31597915291786194, + 1.6885921955108643, + 0.35994845628738403, + -0.1680283099412918, + 0.12475381046533585, + 0.18452417850494385, + 0.3003499507904053, + 0.5231201648712158, + 3.625150680541992, + -0.0822700560092926, + -0.13376817107200623, + 0.465038001537323, + 0.08038262277841568, + 0.4713194966316223, + 0.05367746949195862, + 3.2971887588500977 + ], + [ + -0.1903712898492813, + -0.37053966522216797, + 0.6688979268074036, + -0.09197451919317245, + 0.6773920059204102, + -0.33712926506996155, + 1.4484882354736328, + 0.5138263702392578, + -0.09785021096467972, + 0.03438278287649155, + 0.12166035175323486, + 0.31852588057518005, + 0.380852609872818, + 2.716904878616333, + -0.4005071222782135, + 0.046017542481422424, + 0.6103469729423523, + 0.3996133804321289, + 0.5668272972106934, + -0.316438764333725, + 2.8927831649780273 + ], + [ + -0.21704740822315216, + -0.330172061920166, + 1.100143313407898, + -0.07240776717662811, + 0.5212385654449463, + -0.10958275198936462, + 1.6004505157470703, + 0.07333774119615555, + -0.12773768603801727, + 0.06542205810546875, + 0.07344947755336761, + 0.17908638715744019, + 0.11419609189033508, + 3.368091344833374, + -0.08816054463386536, + -0.17370833456516266, + 0.5846867561340332, + 0.07153494656085968, + 0.4638330936431885, + 0.6930655837059021, + 3.6226603984832764 + ], + [ + -0.12587983906269073, + -0.38620221614837646, + 1.1821459531784058, + -0.057351693511009216, + 0.3969283103942871, + -0.13062044978141785, + 1.5191221237182617, + 0.05112460255622864, + -0.12875741720199585, + 0.08850830793380737, + 0.12764394283294678, + 0.24970731139183044, + 0.2700720727443695, + 3.2572789192199707, + -0.09465007483959198, + -0.2016996443271637, + 0.47044676542282104, + 0.07808393239974976, + 0.44408103823661804, + 0.5866963863372803, + 3.515298366546631 + ], + [ + -0.29369062185287476, + -0.3233182430267334, + 1.038780927658081, + 0.031081125140190125, + 0.8599079847335815, + 0.02698427438735962, + 1.6414964199066162, + 0.28055518865585327, + -0.10770316421985626, + 0.03712392598390579, + 0.08561195433139801, + 0.17864517867565155, + 0.18073830008506775, + 3.289355754852295, + -0.14902560412883759, + -0.24006064236164093, + 0.5561586618423462, + 0.10910765826702118, + 0.6538932919502258, + 0.8370549082756042, + 3.7762296199798584 + ], + [ + -0.2538868188858032, + -0.40941011905670166, + 0.7809090614318848, + -0.07721935212612152, + 0.5145487785339355, + -0.44785603880882263, + 1.4584362506866455, + 0.524945080280304, + -0.13737700879573822, + -0.18502366542816162, + 0.16651324927806854, + 0.3745180368423462, + 0.3228949010372162, + 2.64192271232605, + -0.46043604612350464, + 0.10471723973751068, + 0.6481974124908447, + 0.3921470642089844, + 0.5390013456344604, + 0.0029841959476470947, + 2.9022462368011475 + ], + [ + -0.14602552354335785, + -0.2900434732437134, + 1.1657943725585938, + -0.02164776623249054, + 0.6434773206710815, + 0.05961035192012787, + 1.5957074165344238, + 0.2755030691623688, + -0.09505859017372131, + 0.09135580062866211, + 0.18444988131523132, + 0.20256099104881287, + 0.17112037539482117, + 3.125737428665161, + -0.21520298719406128, + -0.1718806028366089, + 0.6482832431793213, + 0.10868270695209503, + 0.675834596157074, + 0.9381958842277527, + 3.60792875289917 + ], + [ + -0.13458235561847687, + -0.33202123641967773, + 1.1956603527069092, + -0.06914085149765015, + 0.22907499969005585, + -0.234523206949234, + 2.0782017707824707, + 0.20605845749378204, + -0.14640741050243378, + 0.1175045520067215, + 0.12905830144882202, + 0.2542654871940613, + 0.1759064495563507, + 3.1949563026428223, + -0.2317231297492981, + -0.22757183015346527, + 0.5861898064613342, + 0.24816322326660156, + 0.29490092396736145, + 0.5639578700065613, + 3.59490966796875 + ] + ], + "generated_utilities": [ + 0.0375710129737854, + 0.4434976577758789, + 0.025715351104736328, + 0.025355923920869827, + 0.035843171179294586, + 0.46439141035079956, + 0.022827064618468285, + 0.021351909264922142 + ], + "group_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.6040759086608887, + 2.0, + 0.14582239091396332, + 0.5723002552986145, + 0.10770338028669357, + 0.10128920525312424, + 0.08499570190906525, + 0.14736615121364594, + 0.07181884348392487, + 0.11773071438074112, + 0.06549974530935287, + 0.06568789482116699, + 0.10681070387363434, + 0.05432777479290962, + 0.0625491738319397, + 0.06991603225469589 + ], + "instruction": "Pick up the cube and move it to the goal position.", + "negative_tangents": [ + [ + -0.072060227394104, + 0.2043294906616211, + 0.8692872524261475, + 0.020746350288391113, + -0.004742175340652466, + 0.05779099464416504, + 0.0, + -0.0498790442943573, + -0.20218299329280853, + 0.41556206345558167, + -0.2306707501411438, + 0.32090064883232117, + -0.2089032530784607, + 0.0, + 0.05318737402558327, + -0.04637153446674347, + 0.04412045329809189, + 0.08670947700738907, + -0.10918038338422775, + -0.059010982513427734, + 0.0 + ], + [ + -0.5282122492790222, + 1.204329490661621, + 1.8692872524261475, + -0.8456907868385315, + 0.42858073115348816, + -0.3134727478027344, + -2.5308754444122314, + 0.1655428111553192, + -0.12024828046560287, + 0.0010203085839748383, + 0.2592625319957733, + 0.2679504156112671, + 0.25922709703445435, + 2.851924419403076, + 0.0039860643446445465, + -0.014717570506036282, + 0.01173020526766777, + 0.10578173398971558, + 0.05138798803091049, + 0.3216937780380249, + 2.9243662357330322 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.061750888824463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.703848838806152, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.8487324714660645 + ], + [ + -0.6558206081390381, + 1.3553937673568726, + 2.057647466659546, + -1.0405915975570679, + 0.2629950940608978, + -0.28233209252357483, + -2.589219093322754, + -0.06961745023727417, + 0.02406928688287735, + 0.2341437190771103, + 0.49766814708709717, + 0.4930538535118103, + 0.370309442281723, + 3.111422061920166, + 0.06151486933231354, + -0.2028706818819046, + 0.12007936835289001, + 0.2164750099182129, + 0.16723006963729858, + 0.26430171728134155, + 3.0643396377563477 + ], + [ + -0.18174728751182556, + 0.8890945315361023, + 1.5388343334197998, + -0.4532086253166199, + 0.3864310085773468, + -0.45410382747650146, + -2.623199462890625, + 0.5360933542251587, + -0.04706605523824692, + 0.11160072684288025, + 0.5460608005523682, + 0.39992716908454895, + 0.17028246819972992, + 2.8760411739349365, + 0.29450932145118713, + -0.3028962016105652, + 0.06744809448719025, + 0.1687500774860382, + 0.42097336053848267, + 0.3965923488140106, + 3.28550386428833 + ], + [ + -0.4313088357448578, + 0.9752410054206848, + 2.0068275928497314, + -0.5783756375312805, + 0.5761726498603821, + -0.5289677381515503, + -2.321643590927124, + 0.0023015737533569336, + -0.07372284680604935, + 0.05664224550127983, + 0.03747282922267914, + 0.4900178611278534, + 0.31588616967201233, + 3.018609046936035, + 0.12529893219470978, + -0.2872275412082672, + -0.2500917911529541, + 0.0006694421172142029, + -0.39133700728416443, + 0.6382719874382019, + 2.810041904449463 + ], + [ + -0.5560051202774048, + 1.4042110443115234, + 1.662226915359497, + -1.4059122800827026, + 0.8242433071136475, + -0.10998553037643433, + -2.7556281089782715, + -0.26875895261764526, + -0.4201813042163849, + -0.371376633644104, + 0.7718684673309326, + 0.6452833414077759, + 0.027421817183494568, + 2.3879575729370117, + -0.47588983178138733, + -0.045049820095300674, + 0.0838315486907959, + 0.10257662832736969, + 0.4181908369064331, + 0.2650004029273987, + 3.006909132003784 + ], + [ + -0.4896228015422821, + 1.0570447444915771, + 1.408416509628296, + -0.7269805669784546, + 1.023213267326355, + -0.22378163039684296, + -2.1348509788513184, + -0.4423682987689972, + 0.26834720373153687, + -0.5835619568824768, + 0.3675343692302704, + 0.3307996988296509, + -0.01537257432937622, + 3.4422543048858643, + 0.3622148334980011, + -0.26806333661079407, + 0.4238347113132477, + 0.2583734393119812, + 0.10431534051895142, + 0.2077663242816925, + 2.446356773376465 + ], + [ + -1.1077957153320312, + 0.4289678931236267, + 1.9882663488388062, + -0.3021383285522461, + -0.006195276975631714, + -0.22295325994491577, + -1.8697566986083984, + 0.9590625762939453, + -0.0024877265095710754, + -0.3088582456111908, + 0.6517632007598877, + 0.4271944761276245, + 0.34532302618026733, + 2.704721450805664, + 0.7557594776153564, + -0.8073300719261169, + -0.7479258179664612, + -0.19068273901939392, + 0.14240311086177826, + 0.2233867347240448, + 2.1774275302886963 + ], + [ + -0.9354162812232971, + 1.5971527099609375, + 1.0168101787567139, + -0.7164653539657593, + 0.519247829914093, + -0.7477964162826538, + -3.338376045227051, + 0.7212944030761719, + -0.6477807760238647, + 0.38193029165267944, + -0.1568622589111328, + 0.47457069158554077, + -0.30742186307907104, + 2.6848907470703125, + -0.4251534044742584, + 0.6623334884643555, + 0.03348249942064285, + -0.2996619939804077, + -0.8319172263145447, + 0.7883892059326172, + 3.5870752334594727 + ], + [ + -0.44698429107666016, + 0.3201255202293396, + 1.5405346155166626, + -0.9956957101821899, + 0.8944320678710938, + -0.4196317791938782, + -2.010188102722168, + 0.7199084758758545, + -0.49276289343833923, + 0.4512734115123749, + 0.48276829719543457, + 0.22657281160354614, + 0.8205416798591614, + 3.6653342247009277, + 0.6818231344223022, + 0.4660317897796631, + -0.14916014671325684, + 0.5328372716903687, + -0.5979922413825989, + 0.3124805986881256, + 3.110726833343506 + ], + [ + -0.5630171298980713, + 1.934722900390625, + 1.4954721927642822, + -1.684873342514038, + 0.133146733045578, + -0.4359630346298218, + -2.681238889694214, + 0.8904907703399658, + -0.48221057653427124, + -0.7635119557380676, + -0.23804441094398499, + 0.5691602230072021, + -0.23042798042297363, + 3.760842800140381, + 0.3684825003147125, + -0.20475278794765472, + 0.04838913306593895, + 0.17439094185829163, + 0.07042267918586731, + -0.1653938889503479, + 3.844882011413574 + ], + [ + -0.3705383241176605, + 1.5082495212554932, + 0.9002141356468201, + -1.3291171789169312, + 0.15676641464233398, + -1.2286558151245117, + -1.7006758451461792, + 0.34438908100128174, + 0.32962289452552795, + -0.6556114554405212, + 0.3502740263938904, + 0.7518025636672974, + -0.49080890417099, + 2.10355806350708, + 0.36426910758018494, + -0.4822288453578949, + 0.7496572732925415, + 0.43558254837989807, + 0.34650692343711853, + 0.9977220296859741, + 1.9853770732879639 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.005563676357269287, + 0.2043294906616211, + 0.8692872524261475, + 0.08163481950759888, + 0.05995750427246094, + 0.012857973575592041, + 0.0, + 0.01583634316921234, + 0.04724548012018204, + 0.018815450370311737, + 0.02742627263069153, + -0.03381502628326416, + 0.007930099964141846, + 0.0, + 0.01774722896516323, + 0.08064591884613037, + -0.06814983487129211, + -0.002621002495288849, + 0.05759798362851143, + 0.05164051055908203, + 0.0 + ] + ], + "predicted_scores": [ + 0.6592466831207275, + -0.18288078904151917, + -0.2230769693851471, + -0.33697351813316345, + -0.6092725992202759, + -0.8914775848388672, + -0.931504487991333, + -1.0792096853256226 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1273247072", + "selected_index": 0, + "state_hash": "237fafab6bfdb561d66c862026f803fbd19d4007f6313bdb3d79b6dd74f69327", + "stored_base_utility": 0.6040759086608887, + "task_id": "PickCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.050487834960222244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.050487834960222244 + }, + "base_progress": 0.050487834960222244, + "base_success": false, + "base_utility": 0.050487834960222244, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.019472381100058556, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019472381100058556 + }, + { + "contact_quality": null, + "progress": 0.05719713494181633, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05719713494181633 + }, + { + "contact_quality": null, + "progress": 0.06821080297231674, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06821080297231674 + }, + { + "contact_quality": null, + "progress": 0.028275012969970703, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.028275012969970703 + }, + { + "contact_quality": null, + "progress": 0.01960599422454834, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01960599422454834 + }, + { + "contact_quality": null, + "progress": 0.11701449006795883, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11701449006795883 + }, + { + "contact_quality": null, + "progress": 0.05281931161880493, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05281931161880493 + }, + { + "contact_quality": null, + "progress": 0.06312280893325806, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06312280893325806 + } + ], + "candidate_progress": [ + 0.019472381100058556, + 0.05719713494181633, + 0.06821080297231674, + 0.028275012969970703, + 0.01960599422454834, + 0.11701449006795883, + 0.05281931161880493, + 0.06312280893325806 + ], + "candidate_source_chart_ids": [ + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0345d8ac530f88cefd61", + "generated_tangents": [ + [ + 0.444324791431427, + 0.10367916524410248, + 0.012270689010620117, + 0.8968409299850464, + 0.2659638524055481, + 1.0732413530349731, + -0.21330034732818604, + -0.09056296944618225, + 0.6343738436698914, + 0.874171257019043, + -1.1261749267578125, + -0.17651215195655823, + -0.6824302673339844, + 0.21632888913154602, + -0.8118916153907776, + 0.3421381711959839, + 0.47831493616104126, + 0.186706081032753, + -1.1168158054351807, + 0.15592551231384277, + -0.3649512827396393 + ], + [ + 0.582933783531189, + 0.07501409947872162, + 0.3667686879634857, + 0.6543165445327759, + 0.3305560350418091, + 1.1277859210968018, + -0.5874252915382385, + -0.17955923080444336, + 0.48543867468833923, + 0.6742415428161621, + -1.0333203077316284, + -0.10667729377746582, + -0.5693666934967041, + -0.16236604750156403, + -0.6417573690414429, + 0.35614097118377686, + 0.472110778093338, + 0.25263339281082153, + -1.2747080326080322, + -0.06723153591156006, + -0.5342777967453003 + ], + [ + 0.4774540066719055, + 0.18879170715808868, + 0.052444279193878174, + 0.7714061737060547, + 0.2214621752500534, + 0.9115142822265625, + -0.4279770851135254, + -0.04090888798236847, + 0.6480913758277893, + 0.8157578110694885, + -1.042747974395752, + -0.2258063554763794, + -0.696947455406189, + 0.05961869657039642, + -0.8991007804870605, + 0.30625736713409424, + 0.5233581066131592, + 0.20415931940078735, + -1.189032793045044, + 0.15570494532585144, + -0.2326734960079193 + ], + [ + 0.44649672508239746, + 0.09742087125778198, + 0.13266673684120178, + 0.7603572607040405, + 0.282711386680603, + 0.9564565420150757, + -0.46242356300354004, + -0.03456386923789978, + 0.5279194712638855, + 0.7715417146682739, + -1.0219554901123047, + -0.18551328778266907, + -0.4615861773490906, + -0.11304500699043274, + -0.8485602736473083, + 0.314506471157074, + 0.5243592858314514, + 0.30175650119781494, + -1.1856787204742432, + 0.014193445444107056, + -0.30259281396865845 + ], + [ + 0.38412991166114807, + 0.102728471159935, + 0.06790709495544434, + 0.6844552755355835, + 0.18033036589622498, + 1.2175039052963257, + -0.2505999803543091, + -0.10151886940002441, + 0.5712025761604309, + 0.8772667050361633, + -0.9988381862640381, + -0.10765290260314941, + -0.45664942264556885, + 0.0903095155954361, + -0.9287569522857666, + 0.3178274631500244, + 0.523028552532196, + 0.10677517205476761, + -0.9136781096458435, + 0.3343772292137146, + -0.033195048570632935 + ], + [ + 0.4742353558540344, + 0.1852404773235321, + 0.27466344833374023, + 0.3616723120212555, + 0.31565362215042114, + 0.30515414476394653, + -0.34123581647872925, + -0.07583582401275635, + 0.5641093254089355, + 0.5736047029495239, + -1.1779217720031738, + -0.012221962213516235, + 0.10746058821678162, + -0.6907521486282349, + -0.7577171921730042, + 0.38214731216430664, + 0.21964217722415924, + 0.2696443796157837, + -1.424448013305664, + -0.14528992772102356, + -0.24489116668701172 + ], + [ + 0.3809819221496582, + 0.2657419443130493, + 0.24924719333648682, + 0.44967177510261536, + 0.6870844960212708, + 0.7106812000274658, + -0.5713879466056824, + -0.0011958479881286621, + 0.48955443501472473, + 0.8217517137527466, + -1.05900239944458, + -0.6630968451499939, + -0.3793579936027527, + -0.5009958148002625, + -1.023776650428772, + 0.21992097795009613, + 0.5339664220809937, + 0.10243518650531769, + -1.1657459735870361, + 0.12472578883171082, + -0.18491411209106445 + ], + [ + 0.5386635065078735, + 0.11535128951072693, + 0.250006765127182, + 0.8108882904052734, + 0.6256585717201233, + 0.8495814800262451, + -0.23052239418029785, + -0.1735210120677948, + 0.5670768022537231, + 0.7462118864059448, + -1.0848647356033325, + -0.3679772615432739, + -0.16919347643852234, + -0.35942530632019043, + -0.8746117949485779, + 0.2851692736148834, + 0.38496536016464233, + 0.25257858633995056, + -1.0041775703430176, + -0.1322382092475891, + -0.18745839595794678 + ] + ], + "generated_utilities": [ + 0.019472381100058556, + 0.05719713494181633, + 0.06821080297231674, + 0.028275012969970703, + 0.01960599422454834, + 0.11701449006795883, + 0.05281931161880493, + 0.06312280893325806 + ], + "group_id": "ms-0345d8ac530f88cefd61", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.007898053154349327, + 0.0058197579346597195, + 0.052657388150691986, + 0.016109248623251915, + 0.1698339879512787, + 0.008177022449672222, + 0.12333428859710693, + 0.1150994747877121, + 0.15141737461090088, + 0.08515926450490952, + 0.1523766964673996, + 0.04073335975408554, + 0.027175527065992355, + 0.1798098385334015, + 0.08658158779144287, + 0.0618208646774292 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2244938164949417, + -0.708204448223114, + 1.4058912992477417, + 0.19480958580970764, + 1.1751772165298462, + -1.2265020608901978, + -0.8636159896850586, + -0.07021944224834442, + 1.2708500623703003, + 1.0246398448944092, + -1.7615737915039062, + -0.44495686888694763, + -1.2927910089492798, + 0.27074551582336426, + -1.6349679231643677, + 0.9748592376708984, + 1.0210484266281128, + 0.36906859278678894, + -2.528050661087036, + 0.7326674461364746, + 1.553327202796936 + ], + [ + 0.35595428943634033, + -0.8224175572395325, + 1.1192705631256104, + 0.4673205614089966, + 1.1115962266921997, + -1.4676826000213623, + -1.057816505432129, + -0.008461203426122665, + 0.9729399085044861, + 0.8019976615905762, + -1.4944581985473633, + -0.21817980706691742, + -1.3722625970840454, + 0.3557305932044983, + -1.8202308416366577, + 1.2552388906478882, + 0.7855625748634338, + 0.5709130167961121, + -2.4140331745147705, + 0.9063631296157837, + 1.282220721244812 + ], + [ + 0.19927366077899933, + -0.7607958912849426, + 1.4928451776504517, + 0.38265493512153625, + 1.2970150709152222, + -1.2964622974395752, + -0.5547049045562744, + -0.2585906684398651, + 1.3020356893539429, + 1.0737547874450684, + -1.5593467950820923, + -0.1617565155029297, + -1.4868489503860474, + 0.15243440866470337, + -1.4100472927093506, + 0.7991768717765808, + 1.0339381694793701, + 0.6228882074356079, + -2.8132376670837402, + 0.7685560584068298, + 1.8755284547805786 + ], + [ + 0.6393168568611145, + -0.2793915271759033, + 1.4690577983856201, + 0.4745514690876007, + 0.9989874362945557, + -0.7359552979469299, + -1.2171094417572021, + -0.39386802911758423, + 1.1646839380264282, + 0.6661151051521301, + -1.9162347316741943, + -0.43809595704078674, + -1.3973355293273926, + 0.7664861679077148, + -1.553026795387268, + 1.1995127201080322, + 1.285676121711731, + 0.822955846786499, + -2.574894666671753, + 0.3542059361934662, + 1.5986168384552002 + ], + [ + 0.26509445905685425, + -0.5443360805511475, + 1.2502005100250244, + 0.6112543940544128, + 0.924185037612915, + -1.0554168224334717, + -0.7927505373954773, + -0.07152043282985687, + 0.9228929281234741, + 1.545881986618042, + -1.6973015069961548, + -0.496639221906662, + -0.8727940320968628, + 0.5596445202827454, + -2.1939897537231445, + 1.239556074142456, + 1.4699029922485352, + -0.182618647813797, + -2.7263355255126953, + 0.976334810256958, + 1.0402796268463135 + ], + [ + 0.1353418529033661, + -1.1921336650848389, + 1.6913124322891235, + -0.43279919028282166, + 1.1331026554107666, + -1.8503069877624512, + -0.46699997782707214, + 0.5785945653915405, + 1.5516390800476074, + 0.38185298442840576, + -2.192314386367798, + -0.6304612159729004, + -1.6237419843673706, + -0.06383657455444336, + -1.2674099206924438, + 0.7967972755432129, + 1.2453874349594116, + 0.01532512903213501, + -2.6341590881347656, + 0.8366512656211853, + 1.9056174755096436 + ], + [ + 1.0916199684143066, + 0.23665857315063477, + 1.7737994194030762, + -0.8046588897705078, + 1.944507360458374, + -1.3606563806533813, + -0.022103488445281982, + 0.5769228935241699, + 1.0605480670928955, + 1.3100652694702148, + -1.9243974685668945, + 0.43040379881858826, + -1.9046732187271118, + -0.49132758378982544, + -1.9095335006713867, + 1.8850300312042236, + 1.366606593132019, + 0.37651723623275757, + -2.86037540435791, + 0.7753443717956543, + 2.072996139526367 + ], + [ + 0.7277938723564148, + 0.23822063207626343, + 0.4208102226257324, + -0.18717050552368164, + 1.1951282024383545, + -1.1622966527938843, + -0.6617333292961121, + -0.6339871883392334, + 0.8901646137237549, + 0.9977425932884216, + -1.8568902015686035, + 0.3291373550891876, + -0.9377886056900024, + 1.0705609321594238, + -0.90934818983078, + 1.7013394832611084, + 1.5952999591827393, + -0.05327540636062622, + -2.546459197998047, + 0.15792781114578247, + 1.2277774810791016 + ], + [ + 0.9283657073974609, + -1.421413779258728, + 1.6497737169265747, + -0.1984737515449524, + 0.9705085158348083, + -0.8917888402938843, + 0.026184558868408203, + 0.09181642532348633, + 0.6335604786872864, + 1.6531281471252441, + -1.4918432235717773, + -0.01735416054725647, + -0.5425058603286743, + 0.7866935133934021, + -0.7011347413063049, + 0.9797741770744324, + 1.6596866846084595, + -0.13051137328147888, + -2.692556858062744, + -0.05617690086364746, + 0.9771407842636108 + ] + ], + "predicted_scores": [ + 7.805641174316406, + 7.689274787902832, + 7.685960292816162, + 7.47231388092041, + 7.412343978881836, + 7.375511169433594, + 6.744673728942871, + 6.324068069458008 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "1ca7c651b2dc4638488078318fe8aa293aa6f6da78672d440752daa9f2d4e8a3", + "stored_base_utility": 0.007898053154349327, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07743936777114868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07743936777114868 + }, + "base_progress": 0.07743936777114868, + "base_success": false, + "base_utility": 0.07743936777114868, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008482594974339008, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008482594974339008 + }, + { + "contact_quality": null, + "progress": 0.01349447201937437, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01349447201937437 + }, + { + "contact_quality": null, + "progress": 0.01016932725906372, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01016932725906372 + }, + { + "contact_quality": null, + "progress": 0.009373247623443604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009373247623443604 + }, + { + "contact_quality": null, + "progress": 0.010430972091853619, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010430972091853619 + }, + { + "contact_quality": null, + "progress": 0.009613573551177979, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009613573551177979 + }, + { + "contact_quality": null, + "progress": 0.013550917617976665, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013550917617976665 + }, + { + "contact_quality": null, + "progress": 0.012914061546325684, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012914061546325684 + } + ], + "candidate_progress": [ + 0.008482594974339008, + 0.01349447201937437, + 0.01016932725906372, + 0.009373247623443604, + 0.010430972091853619, + 0.009613573551177979, + 0.013550917617976665, + 0.012914061546325684 + ], + "candidate_source_chart_ids": [ + "ms-61bde0225d6f4175b7cb", + "ms-d6da14bffe59c0265db7", + "ms-61bde0225d6f4175b7cb", + "ms-02311c7783bc67e188b4", + "ms-61bde0225d6f4175b7cb", + "ms-02311c7783bc67e188b4", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-0819eb89e627595e7e1b", + "generated_tangents": [ + [ + -0.48170340061187744, + -0.21298520267009735, + 1.315117359161377, + 0.6920210123062134, + 0.8527080416679382, + 2.968205213546753, + 2.1501686573028564, + -0.38050714135169983, + -0.048633918166160583, + 1.1450988054275513, + 0.2232743203639984, + 0.8262364268302917, + 2.342627763748169, + 2.3650028705596924, + -0.36532288789749146, + -0.06950965523719788, + 1.331042766571045, + 0.2704000473022461, + 0.6595650911331177, + 0.8094047904014587, + 1.3616341352462769 + ], + [ + -0.17983202636241913, + 0.14181523025035858, + 1.1113064289093018, + 0.33009806275367737, + 1.7241795063018799, + 2.376654624938965, + 1.1097676753997803, + 0.05465759336948395, + 0.09483547508716583, + 1.7570924758911133, + -0.1716618537902832, + 0.960301399230957, + 1.9596649408340454, + 2.3049609661102295, + -0.7374430894851685, + -0.03532176837325096, + 0.5762314796447754, + 0.7728564143180847, + 0.9117649793624878, + 1.0454025268554688, + 0.3472132682800293 + ], + [ + -0.31008750200271606, + -0.12151334434747696, + 1.2201170921325684, + 0.19237235188484192, + 0.738303542137146, + 1.855515956878662, + 1.6113858222961426, + -0.37851646542549133, + 0.030104465782642365, + 1.3002598285675049, + 0.01885172352194786, + 0.9363909959793091, + 2.283543109893799, + 1.0001118183135986, + -0.49234700202941895, + -0.06432265043258667, + 1.016754388809204, + 0.5973330736160278, + 0.5052860975265503, + 1.2347865104675293, + 0.29192376136779785 + ], + [ + -0.2867894172668457, + -0.13257068395614624, + 1.4791605472564697, + 0.12920957803726196, + 0.975943386554718, + 2.172316312789917, + 1.9733011722564697, + -0.2797960937023163, + -0.054635435342788696, + 1.6207730770111084, + 0.18025602400302887, + 0.7074676156044006, + 1.452094554901123, + 2.140620708465576, + -0.23117363452911377, + 0.001552867703139782, + 0.9780837297439575, + 0.4659976363182068, + 0.7417269945144653, + 1.0483758449554443, + 0.8611186742782593 + ], + [ + -0.40189963579177856, + -0.1369675099849701, + 1.2212437391281128, + 0.1413283348083496, + 0.8650162220001221, + 1.8716392517089844, + 1.6848230361938477, + -0.4276892840862274, + 0.04519989341497421, + 1.122190237045288, + -0.018321841955184937, + 0.7497153878211975, + 2.335747480392456, + 1.097080945968628, + -0.4920797348022461, + -0.019546911120414734, + 1.1053532361984253, + 0.5397530198097229, + 0.4506194591522217, + 1.1697564125061035, + 0.3101489543914795 + ], + [ + -0.3615391254425049, + -0.14622633159160614, + 1.4674757719039917, + 0.10478755831718445, + 0.9942489266395569, + 2.164763927459717, + 1.7419607639312744, + -0.27240440249443054, + -0.04424624145030975, + 1.5607510805130005, + 0.17398256063461304, + 0.7515172958374023, + 1.4062234163284302, + 2.099848747253418, + -0.1732201874256134, + -0.03870294988155365, + 0.9826485514640808, + 0.5652058720588684, + 0.8408015370368958, + 0.9496859908103943, + 0.6621719598770142 + ], + [ + -0.046975865960121155, + 0.024709030985832214, + 1.1837114095687866, + 0.10256123542785645, + 1.2242372035980225, + 2.677042245864868, + 1.2298593521118164, + -0.22442549467086792, + -0.009645089507102966, + 1.488405466079712, + 0.21199652552604675, + 0.42241954803466797, + 1.7646032571792603, + 1.6054023504257202, + -0.39166170358657837, + -0.08306765556335449, + 0.7367458939552307, + 0.873846709728241, + 0.7609628438949585, + 0.3406764268875122, + -0.1312771737575531 + ], + [ + -0.08826951682567596, + 0.086859330534935, + 1.1238945722579956, + -0.005096703767776489, + 1.1703311204910278, + 2.7813732624053955, + 1.3670010566711426, + -0.2854817807674408, + 0.013575018383562565, + 1.4255802631378174, + 0.27501311898231506, + 0.44776588678359985, + 1.9851607084274292, + 1.3412727117538452, + -0.36598777770996094, + -0.09238594770431519, + 0.7466386556625366, + 0.863097071647644, + 0.8219146132469177, + 0.35138940811157227, + -0.11985167860984802 + ] + ], + "generated_utilities": [ + 0.008482594974339008, + 0.01349447201937437, + 0.01016932725906372, + 0.009373247623443604, + 0.010430972091853619, + 0.009613573551177979, + 0.013550917617976665, + 0.012914061546325684 + ], + "group_id": "ms-0819eb89e627595e7e1b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07638250291347504, + 0.11176031827926636, + 0.139996737241745, + 0.06282445043325424, + 0.08466696739196777, + 0.0763862133026123, + 0.05953681468963623, + 0.06854937970638275, + 0.06834080070257187, + 0.09309522807598114, + 0.14572584629058838, + 0.07182137668132782, + 0.020047685131430626, + 0.05106111615896225, + 0.03756682202219963, + 0.061605654656887054 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.04522436857223511, + -0.3834587037563324, + 2.32004976272583, + 1.6386330127716064, + 1.508955717086792, + 1.681371808052063, + 2.9060468673706055, + -0.5984516143798828, + 0.36428770422935486, + 0.7442611455917358, + 1.163742184638977, + 0.4921322464942932, + 1.9586588144302368, + 3.335318088531494, + -0.02929070219397545, + 0.5442408919334412, + 2.4529502391815186, + 0.17471881210803986, + 1.804263710975647, + 1.5498225688934326, + 1.9392809867858887 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.19627264142036438, + 0.015629824250936508, + 0.723259687423706, + 0.0313839316368103, + 0.02149224281311035, + 0.7142634391784668, + 0.0, + -0.29767167568206787, + -0.10304851830005646, + 0.6168417930603027, + 0.0271989107131958, + 0.369077205657959, + 1.5924487113952637, + 0.0, + -0.11617732048034668, + -0.1419914811849594, + 0.9400725364685059, + -0.04139697551727295, + 0.7166774272918701, + 1.092864990234375, + 0.0 + ], + [ + -0.18197131156921387, + 0.30914804339408875, + 1.4557228088378906, + 1.4375734329223633, + 0.9600651264190674, + 1.7448713779449463, + 3.031123399734497, + -0.6073551177978516, + -0.4749099016189575, + 1.3016932010650635, + 0.32392996549606323, + 0.6961254477500916, + 2.1961381435394287, + 3.730100154876709, + -0.6154190897941589, + -0.3908892273902893, + 2.0994153022766113, + 0.18937554955482483, + 1.5752670764923096, + 2.2457275390625, + 1.5629570484161377 + ] + ], + "predicted_scores": [ + 8.962811470031738, + 8.02236557006836, + 7.837157249450684, + 7.748305797576904, + 7.6356892585754395, + 7.552002429962158, + 7.480719566345215, + 7.393957614898682 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "582daf047d78a1b01018d54f5798143c4be6129fd7b6cbc025fc85818e519f3c", + "stored_base_utility": 0.07638250291347504, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19400137662887573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19400137662887573 + }, + "base_progress": 0.19400137662887573, + "base_success": false, + "base_utility": 0.19400137662887573, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03977566957473755, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03977566957473755 + }, + { + "contact_quality": null, + "progress": 0.035518765449523926, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035518765449523926 + }, + { + "contact_quality": null, + "progress": 0.037184059619903564, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037184059619903564 + }, + { + "contact_quality": null, + "progress": 0.04727194830775261, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04727194830775261 + }, + { + "contact_quality": null, + "progress": 0.1163921132683754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1163921132683754 + }, + { + "contact_quality": null, + "progress": 0.039057254791259766, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.039057254791259766 + }, + { + "contact_quality": null, + "progress": 0.03655803203582764, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03655803203582764 + }, + { + "contact_quality": null, + "progress": 0.059245508164167404, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.059245508164167404 + } + ], + "candidate_progress": [ + 0.03977566957473755, + 0.035518765449523926, + 0.037184059619903564, + 0.04727194830775261, + 0.1163921132683754, + 0.039057254791259766, + 0.03655803203582764, + 0.059245508164167404 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-25ab20bb3da8ba34be95", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "generated_tangents": [ + [ + 1.4961535930633545, + 0.05711425095796585, + -0.8380736112594604, + -0.07539501786231995, + 0.31329917907714844, + -2.6488101482391357, + -1.0738818645477295, + -0.968364953994751, + -0.2544800043106079, + 0.4324849545955658, + 0.6627989411354065, + -0.28087976574897766, + -0.10343340039253235, + -0.02613060176372528, + -0.7529232501983643, + -0.30540382862091064, + 1.263474702835083, + 1.0633282661437988, + -0.3979564309120178, + -0.3800346851348877, + 0.41512221097946167 + ], + [ + 1.3072959184646606, + 0.1105508804321289, + -1.073121428489685, + 0.022990912199020386, + 0.5359436273574829, + -2.6330926418304443, + -0.9600021243095398, + -0.8445543050765991, + -0.1756526082754135, + 0.4960522949695587, + 0.9204374551773071, + -0.1704983115196228, + 0.166866272687912, + 0.08802354335784912, + -0.9753002524375916, + -0.4139297604560852, + 1.4275983572006226, + 1.2034046649932861, + -0.23188930749893188, + 0.3031361699104309, + 0.40181833505630493 + ], + [ + 1.308129906654358, + 0.13780780136585236, + -0.9447993040084839, + 0.24220460653305054, + 0.3176955580711365, + -2.2120866775512695, + -0.9479312300682068, + -1.114196538925171, + -0.15016527473926544, + 0.4687451422214508, + 0.6838050484657288, + -0.22989246249198914, + 0.43679264187812805, + -0.037165701389312744, + -0.783089280128479, + -0.2977108359336853, + 1.2360730171203613, + 1.1537435054779053, + -0.3863477110862732, + -0.001897364854812622, + 0.1392533779144287 + ], + [ + 1.3749992847442627, + 0.2382805496454239, + -1.0212786197662354, + 0.15463030338287354, + 0.38636261224746704, + -2.4216132164001465, + -0.9702731966972351, + -0.7767324447631836, + -0.18187862634658813, + 0.48575687408447266, + 0.7979539036750793, + -0.17321276664733887, + 0.08161547780036926, + -0.05776302516460419, + -0.9654657244682312, + -0.3173145651817322, + 1.4733184576034546, + 1.1904693841934204, + -0.20466378331184387, + 0.3063088655471802, + 0.34561169147491455 + ], + [ + 1.6630033254623413, + 0.061981506645679474, + -1.2176945209503174, + 0.16619658470153809, + 0.615795373916626, + -2.1796154975891113, + -1.4509468078613281, + -0.5507602691650391, + -0.08558899164199829, + 0.42375171184539795, + 0.5340151786804199, + -0.13156241178512573, + 0.11130085587501526, + -0.16660358011722565, + -1.1031697988510132, + -0.2901856601238251, + 1.0634026527404785, + 1.2348135709762573, + 0.20170143246650696, + 0.18915006518363953, + -0.20062124729156494 + ], + [ + 1.2891536951065063, + 0.1130342036485672, + -1.0005327463150024, + 0.2583622932434082, + 0.5921269655227661, + -2.4853010177612305, + -0.9217981696128845, + -0.747860312461853, + -0.18771840631961823, + 0.4645868241786957, + 0.9687589406967163, + -0.21854183077812195, + -0.012618690729141235, + 0.22672508656978607, + -0.97975093126297, + -0.2944067418575287, + 1.4229469299316406, + 1.1545466184616089, + -0.2660664916038513, + 0.30267637968063354, + 0.28235435485839844 + ], + [ + 1.1726384162902832, + 0.014311477541923523, + -0.8084231615066528, + 0.16792312264442444, + 0.046148285269737244, + -2.0844624042510986, + -0.8476629853248596, + -1.0396668910980225, + 0.054600343108177185, + 0.7791575193405151, + 0.3479359447956085, + -0.40113091468811035, + -0.083014577627182, + 0.16177353262901306, + -1.2966257333755493, + -0.18281859159469604, + 1.1668022871017456, + 1.0944393873214722, + -0.06395062804222107, + 0.33366262912750244, + 0.6274118423461914 + ], + [ + 1.6430341005325317, + 0.23615755140781403, + -0.6891779899597168, + 0.22973570227622986, + 0.5483315587043762, + -2.0632615089416504, + -1.6444897651672363, + -0.6461735963821411, + -0.22430431842803955, + 0.5634874105453491, + 0.8024416565895081, + 0.06881408393383026, + 0.5734598636627197, + -0.7092761993408203, + -0.931363046169281, + -0.1820114254951477, + 1.4591095447540283, + 1.3638323545455933, + 0.09794610738754272, + -0.029864966869354248, + 0.16142325103282928 + ] + ], + "generated_utilities": [ + 0.03977566957473755, + 0.035518765449523926, + 0.037184059619903564, + 0.04727194830775261, + 0.1163921132683754, + 0.039057254791259766, + 0.03655803203582764, + 0.059245508164167404 + ], + "group_id": "ms-0dee962a5901fb8b5bbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23161475360393524, + 0.23611454665660858, + 0.1380448043346405, + 2.0, + 0.25444933772087097, + 0.25831863284111023, + 0.22753827273845673, + 0.2483263909816742, + 0.21622240543365479, + 0.17534737288951874, + 0.23816657066345215, + 0.09714888036251068, + 0.05844185873866081, + 0.12679120898246765, + 0.0472835898399353, + 0.03277532383799553 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7612967491149902, + 0.192471444606781, + -0.2699308395385742, + 0.017076104879379272, + -0.14400714635849, + -1.9032442569732666, + 0.0, + -0.28746497631073, + -0.2857179641723633, + -0.09024286270141602, + -0.03538453206419945, + 0.06432074308395386, + -0.2535901665687561, + 0.0, + 0.06180626153945923, + 0.16256888210773468, + -0.031329914927482605, + 1.563896656036377, + 0.04795600473880768, + 0.02485179901123047, + 0.0 + ], + [ + 1.2132649421691895, + 0.536715567111969, + -1.4273607730865479, + -1.0538015365600586, + -0.5170869827270508, + -3.4203505516052246, + -2.3889083862304688, + -0.7717514634132385, + -1.1465628147125244, + 0.8393621444702148, + -0.4419952630996704, + -0.23069629073143005, + -0.0637764036655426, + -1.5193647146224976, + -1.0273863077163696, + 0.39812424778938293, + -0.02874515950679779, + 2.7962026596069336, + 0.2370568960905075, + 1.5884850025177002, + 0.9987253546714783 + ], + [ + 2.1451404094696045, + 0.09629477560520172, + -1.8500688076019287, + 0.012462526559829712, + 0.13087880611419678, + -2.4062256813049316, + -3.303264856338501, + -1.3855981826782227, + -0.0292893648147583, + -0.17000514268875122, + 0.0005041323602199554, + -0.7487195134162903, + 0.06868582963943481, + -2.107095718383789, + -0.5890907645225525, + 0.1777246743440628, + -0.5576009750366211, + 2.767899990081787, + -0.3420543074607849, + 1.1357430219650269, + 1.8840694427490234 + ], + [ + 2.032343626022339, + 0.4649246335029602, + -0.5597869753837585, + 0.37935349345207214, + 0.14823919534683228, + -3.389478921890259, + -1.972822904586792, + -0.0014220476150512695, + -0.6147069334983826, + 0.9483420252799988, + 0.641323983669281, + -0.7033213973045349, + 0.4629128575325012, + -1.6774024963378906, + -1.1420938968658447, + 0.5891244411468506, + -0.712251603603363, + 2.1924612522125244, + 0.18694229423999786, + 0.2795591354370117, + 2.4723148345947266 + ], + [ + 2.299381732940674, + -0.6742787957191467, + -0.886760950088501, + -0.1637115478515625, + 0.32663029432296753, + -1.9267126321792603, + -1.9424545764923096, + -0.3586595058441162, + -1.1000827550888062, + 0.45532336831092834, + -0.4314403235912323, + -0.12869995832443237, + -0.032513588666915894, + -2.046614170074463, + -1.2942235469818115, + 0.3962012231349945, + 0.797533392906189, + 2.202794313430786, + 0.4938191771507263, + 0.15863806009292603, + 1.745660662651062 + ], + [ + 2.107815742492676, + 0.963295042514801, + -1.2979395389556885, + -1.0134333372116089, + -1.319197177886963, + -3.176518440246582, + -2.4782180786132812, + -1.3465216159820557, + -1.0473096370697021, + -0.22631770372390747, + 0.7881518006324768, + -1.4033093452453613, + 0.3855947256088257, + -1.3816941976547241, + -1.1257284879684448, + -0.7653472423553467, + 0.6845007538795471, + 2.9796371459960938, + 0.1967305988073349, + 1.4200057983398438, + 1.2843091487884521 + ], + [ + 1.7303649187088013, + 1.0728741884231567, + -1.1676844358444214, + -0.20355769991874695, + -0.8773125410079956, + -3.2701919078826904, + -2.1479928493499756, + -0.044036924839019775, + -1.252698302268982, + -0.01393243670463562, + 0.6447213292121887, + -0.7088868618011475, + 0.11592492461204529, + -2.1431162357330322, + -0.8284967541694641, + 0.4642922580242157, + -0.5207862854003906, + 1.8997660875320435, + -0.6910213828086853, + 0.2906065583229065, + 1.4081878662109375 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7612967491149902, + -0.5331565737724304, + -0.2699308395385742, + -0.20421931147575378, + -0.22613269090652466, + -1.9032442569732666, + 0.0, + 0.09110367298126221, + 0.009229481220245361, + 0.004811525344848633, + 0.656868577003479, + -0.28485387563705444, + -0.31901538372039795, + 0.0, + 0.08918565511703491, + 0.3355104923248291, + 0.026827186346054077, + 1.563896656036377, + 0.13480885326862335, + 0.02485179901123047, + 0.0 + ] + ], + "predicted_scores": [ + 8.347381591796875, + 8.0756196975708, + 7.89379358291626, + 7.591610431671143, + 7.5797038078308105, + 7.3529510498046875, + 7.292175769805908, + 6.160061359405518 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "73f2826502af62b561f13c32614947588918c3deb631165739e66d787008fa98", + "stored_base_utility": 0.23161475360393524, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.04583283141255379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04583283141255379 + }, + "base_progress": 0.04583283141255379, + "base_success": false, + "base_utility": 0.04583283141255379, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.10429390519857407, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10429390519857407 + }, + { + "contact_quality": null, + "progress": 0.08981126546859741, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08981126546859741 + }, + { + "contact_quality": null, + "progress": 0.09254121780395508, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09254121780395508 + }, + { + "contact_quality": null, + "progress": 0.09323513507843018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09323513507843018 + }, + { + "contact_quality": null, + "progress": 0.1434544175863266, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1434544175863266 + }, + { + "contact_quality": null, + "progress": 0.17300955951213837, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17300955951213837 + }, + { + "contact_quality": null, + "progress": 0.07903516292572021, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07903516292572021 + }, + { + "contact_quality": null, + "progress": 0.09951392561197281, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09951392561197281 + } + ], + "candidate_progress": [ + 0.10429390519857407, + 0.08981126546859741, + 0.09254121780395508, + 0.09323513507843018, + 0.1434544175863266, + 0.17300955951213837, + 0.07903516292572021, + 0.09951392561197281 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-7a044cb6c7bb03f3ea10", + "ms-38dff128761905bb1507" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-126eb8b36769f9ba4671", + "generated_tangents": [ + [ + 1.369253158569336, + -0.06315869092941284, + -1.080492377281189, + 0.09179672598838806, + -0.2259550392627716, + -2.9094576835632324, + -2.2019600868225098, + 0.937443733215332, + -0.023995475843548775, + 0.0738721489906311, + -0.1052713468670845, + 0.8359808921813965, + -1.4244933128356934, + -0.34113651514053345, + -1.115409016609192, + -0.15458451211452484, + 0.9955348372459412, + 0.5989454984664917, + 1.2079029083251953, + 1.6112141609191895, + 1.669655203819275 + ], + [ + 1.2912083864212036, + -0.19512198865413666, + -0.994770884513855, + 0.07655856013298035, + -0.049782752990722656, + -2.7013258934020996, + -1.6116347312927246, + 0.7569692134857178, + -0.018285728991031647, + 0.1288003921508789, + -0.2150503396987915, + 0.7750152349472046, + -1.0736428499221802, + -0.5305038690567017, + -0.9757967591285706, + -0.22260208427906036, + 1.1594197750091553, + 0.4849216938018799, + 0.7854194045066833, + 0.9466932415962219, + 1.5160409212112427 + ], + [ + 1.578649640083313, + 0.02785012125968933, + -1.048768162727356, + 0.10640284419059753, + -0.25175657868385315, + -3.3711423873901367, + -2.2970876693725586, + 0.8743171691894531, + 0.06778844445943832, + 0.1173376590013504, + -0.26985153555870056, + 0.9261730909347534, + -1.1985284090042114, + -0.24936388432979584, + -1.0118955373764038, + -0.11626236140727997, + 0.8804306983947754, + 0.7422818541526794, + 1.240435004234314, + 1.416860580444336, + 1.6711128950119019 + ], + [ + 1.4255388975143433, + -0.0031580328941345215, + -1.0690832138061523, + 0.10249030590057373, + -0.2593812644481659, + -3.1884021759033203, + -2.243687868118286, + 0.930073618888855, + 0.011781872250139713, + 0.10563268512487411, + -0.11607181280851364, + 0.9018707275390625, + -1.4074382781982422, + -0.26366329193115234, + -1.119073510169983, + -0.16597124934196472, + 1.0663888454437256, + 0.6679109334945679, + 1.2316689491271973, + 1.572657585144043, + 1.5834970474243164 + ], + [ + 1.5968835353851318, + -0.11812939494848251, + -0.951828122138977, + 0.28660598397254944, + 0.2038881480693817, + -2.8580515384674072, + -2.4814679622650146, + 0.6503607034683228, + -0.05494828522205353, + -0.08175159245729446, + -0.18631088733673096, + 0.776267409324646, + -0.5315053462982178, + 0.2381022572517395, + -0.4410700798034668, + 0.003059820272028446, + 0.4818568229675293, + 0.5917049646377563, + 1.395589828491211, + 0.8140818476676941, + 1.0847562551498413 + ], + [ + 1.5958155393600464, + -0.14150354266166687, + -1.0692274570465088, + 0.45605435967445374, + 0.0012110322713851929, + -2.7148196697235107, + -2.5512993335723877, + 0.570069432258606, + -0.04990261793136597, + -0.1281956136226654, + -0.10317870229482651, + 0.8229105472564697, + -0.5673810243606567, + 0.19114671647548676, + -0.4908102750778198, + -0.04287712275981903, + 0.4610198438167572, + 0.6574446558952332, + 1.4220271110534668, + 0.718690812587738, + 0.9527000188827515 + ], + [ + 1.1580145359039307, + -0.18998830020427704, + -0.8047693967819214, + 0.0764504075050354, + -0.06095534563064575, + -2.9506022930145264, + -1.5477747917175293, + 0.726123571395874, + -0.0107507836073637, + 0.12318387627601624, + -0.031517282128334045, + 0.9004573225975037, + -1.2188684940338135, + -0.2621389627456665, + -0.974763810634613, + -0.2896084785461426, + 1.1361896991729736, + 0.5284179449081421, + 0.9712277054786682, + 1.2001430988311768, + 1.5238471031188965 + ], + [ + 1.5085656642913818, + -0.12596197426319122, + -0.7328596115112305, + 0.3691893517971039, + 0.1983453780412674, + -3.1568164825439453, + -1.879227876663208, + 0.5138417482376099, + 0.11119545996189117, + -0.16166532039642334, + 0.4548158347606659, + 0.729375422000885, + -0.1761901080608368, + -0.025958046317100525, + -0.8443230390548706, + 0.015648137778043747, + 1.0883651971817017, + 0.6242193579673767, + 0.7178710699081421, + 1.7087934017181396, + 1.506760597229004 + ] + ], + "generated_utilities": [ + 0.10429390519857407, + 0.08981126546859741, + 0.09254121780395508, + 0.09323513507843018, + 0.1434544175863266, + 0.17300955951213837, + 0.07903516292572021, + 0.09951392561197281 + ], + "group_id": "ms-126eb8b36769f9ba4671", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05304628610610962, + 0.14190539717674255, + 0.1985912322998047, + 0.1210482120513916, + 0.18570420145988464, + 0.05259696766734123, + 0.1827520728111267, + 0.15291261672973633, + 0.07772710174322128, + 0.15551692247390747, + 0.08802877366542816, + 0.07544025033712387, + 0.08058679103851318, + 0.10843702405691147, + 0.023751219734549522, + 0.00727109145373106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7799564599990845, + -0.0863567665219307, + -0.04370701313018799, + -0.04603701829910278, + -0.003975361585617065, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.02617853879928589, + 0.009054642170667648, + -0.050068795680999756, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + -0.007249712944030762, + 0.0760425329208374, + 1.0404269695281982, + -0.04351449012756348, + -0.010379016399383545, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + 0.005863763391971588, + -0.04370701313018799, + 0.010433673858642578, + -0.05825299024581909, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.22243881225585938, + -0.007246788591146469, + 0.19961372017860413, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.11177664995193481, + -0.1665668487548828, + 1.0404269695281982, + -0.09226274490356445, + -0.18212640285491943, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + -0.7482962012290955, + -0.04370701313018799, + -0.2370874285697937, + 0.3763168752193451, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + -0.441777765750885, + -0.44057023525238037, + -0.6443575024604797, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.008161306381225586, + 0.22021520137786865, + 1.0404269695281982, + -0.09226274490356445, + -0.35497862100601196, + 1.0522968769073486, + 0.0 + ], + [ + 1.7799564599990845, + 0.04087676852941513, + -1.043707013130188, + -0.8482787609100342, + 0.3640388548374176, + -3.3430333137512207, + -2.8715221881866455, + 1.8326367139816284, + -0.4197058081626892, + -0.040397901087999344, + -0.37471577525138855, + 1.7451434135437012, + -1.6306462287902832, + -0.7785218954086304, + -0.5121169090270996, + -0.7797847986221313, + 2.0404269695281982, + 0.9077372550964355, + 0.645021378993988, + 2.0522968769073486, + 1.2749258279800415 + ], + [ + 1.6519862413406372, + 0.1747584044933319, + -0.9129671454429626, + -0.6869257092475891, + 0.12357090413570404, + -3.2319328784942627, + -2.704270839691162, + 1.7347972393035889, + -0.40612348914146423, + 0.23568139970302582, + -0.3912501931190491, + 1.8630532026290894, + -1.8548930883407593, + -0.9794526100158691, + -0.25177696347236633, + -0.6884964108467102, + 2.237109422683716, + 1.1246912479400635, + 0.7490227222442627, + 2.320941686630249, + 1.2039268016815186 + ], + [ + 1.7142105102539062, + 0.3941279351711273, + -1.021533489227295, + -1.1548089981079102, + 0.5052449107170105, + -3.714951992034912, + -3.04028058052063, + 1.529634952545166, + -0.22215746343135834, + 0.19690367579460144, + -0.12262976169586182, + 1.4847477674484253, + -1.490212082862854, + -0.7427945137023926, + -0.7506002187728882, + -0.9221256375312805, + 1.9211885929107666, + 0.5114957094192505, + 0.30887648463249207, + 2.4437777996063232, + 1.3625030517578125 + ], + [ + 1.3140592575073242, + -0.1129835918545723, + -0.9130957722663879, + -1.1089756488800049, + 0.7560886144638062, + -3.3889732360839844, + -3.1182656288146973, + 1.7436503171920776, + -0.9962512254714966, + 0.43109408020973206, + -0.8453629016876221, + 1.2057833671569824, + -2.194112777709961, + -0.22479140758514404, + -0.7258806824684143, + -0.9702881574630737, + 1.8160195350646973, + 0.649415135383606, + 0.6846981048583984, + 2.1997463703155518, + 1.7655411958694458 + ], + [ + 2.6430320739746094, + -0.7696453332901001, + -1.765415906906128, + -0.8281408548355103, + -0.1440155804157257, + -3.3335886001586914, + -3.398141860961914, + 2.4138498306274414, + 0.47537559270858765, + -0.9514593482017517, + 0.2674855887889862, + 2.6307411193847656, + -0.7876039743423462, + -0.35013434290885925, + -0.7511636018753052, + -0.04397457838058472, + 2.123702049255371, + 1.1604151725769043, + 0.7323322892189026, + 1.88338303565979, + 1.5078264474868774 + ] + ], + "predicted_scores": [ + 6.472078800201416, + 6.394108295440674, + 6.16207218170166, + 6.00885009765625, + 5.819354057312012, + 5.714887619018555, + 5.364053249359131, + 5.0096964836120605 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "2390fc23ce8b50bdcdcbb1ab5f0583d2dad2b94fb871db73b1a8401265f59987", + "stored_base_utility": 0.05304628610610962, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-cad6645f8175e38ad155", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-babaeb20129c46551281", + "ms-1f19fd22562f87bdb776", + "ms-c1620854ba52f09e24d5", + "ms-067cac2e4dbeddb12239", + "ms-067cac2e4dbeddb12239" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-16661b2fd2c85f57a66c", + "generated_tangents": [ + [ + -0.18380220234394073, + -0.24929334223270416, + 1.262943148612976, + 0.03600206971168518, + 0.6148139834403992, + 2.245206356048584, + 2.704965114593506, + 1.3962924480438232, + 0.010732992552220821, + -0.047270484268665314, + -0.05522475391626358, + 0.6706928014755249, + 2.471439838409424, + 1.5786356925964355, + 0.8490850925445557, + -0.3888009786605835, + -1.2239018678665161, + -0.33014625310897827, + -0.20082581043243408, + -1.5346240997314453, + -2.185572624206543 + ], + [ + -0.315105676651001, + -0.5645406246185303, + 1.2408192157745361, + -0.16152966022491455, + 0.7254892587661743, + 1.8949651718139648, + 2.4952545166015625, + 1.769245982170105, + 0.08672025799751282, + 0.04487885534763336, + -0.36564403772354126, + 0.8081681728363037, + 1.9177643060684204, + 2.0841763019561768, + 0.9675657153129578, + 0.02687590941786766, + -1.5230522155761719, + -0.45724719762802124, + -0.15677958726882935, + -1.3005516529083252, + -2.746328353881836 + ], + [ + -0.37123948335647583, + -0.5263715982437134, + 1.122902274131775, + -0.22985431551933289, + 0.8552618026733398, + 1.766284465789795, + 2.3940672874450684, + 1.7402807474136353, + 0.06402721256017685, + -0.005862034857273102, + -0.3764722943305969, + 0.6854610443115234, + 1.8561197519302368, + 2.278413772583008, + 1.0196105241775513, + 0.04680681228637695, + -1.5317866802215576, + -0.403542160987854, + -0.19488957524299622, + -1.4346392154693604, + -2.7704172134399414 + ], + [ + -0.626388430595398, + -0.346606969833374, + 1.2113229036331177, + -0.6287609338760376, + 1.2463550567626953, + 2.577420711517334, + 2.93330979347229, + 1.5317102670669556, + -0.28259995579719543, + 0.018710888922214508, + -0.1620306372642517, + 0.8240635395050049, + 1.637794852256775, + 1.1724345684051514, + 1.263065218925476, + -0.29181909561157227, + -1.5987719297409058, + -0.3505403995513916, + -0.4393131136894226, + -1.3932018280029297, + -2.506209135055542 + ], + [ + -0.4232896566390991, + -0.6276735663414001, + 1.3259775638580322, + -0.5200741291046143, + 1.3620741367340088, + 2.0042545795440674, + 3.035946846008301, + 1.632419228553772, + -0.5040328502655029, + -0.16054072976112366, + -0.14927446842193604, + 0.37346094846725464, + 1.4454445838928223, + 1.481978416442871, + 1.261528730392456, + -0.20199286937713623, + -1.7897307872772217, + -0.1670408695936203, + -0.10157498717308044, + -1.161111831665039, + -2.15732741355896 + ], + [ + -0.40514546632766724, + -0.10850674659013748, + 0.8810243606567383, + -0.7218068838119507, + 1.0397483110427856, + 2.2089436054229736, + 4.277473449707031, + 0.968049168586731, + -0.20366573333740234, + 0.2481432557106018, + -0.09063944965600967, + 0.19565847516059875, + 0.7704454660415649, + 2.1513984203338623, + 0.9093822836875916, + -0.2663203477859497, + -0.7597174048423767, + -0.0018222182989120483, + -0.27158141136169434, + -0.4617255926132202, + -1.760399341583252 + ], + [ + -0.4684767723083496, + -0.2348737120628357, + 0.8931154608726501, + -0.22056005895137787, + 1.024080514907837, + 2.415644645690918, + 3.497183084487915, + 1.203537940979004, + -0.14209936559200287, + 0.6001217365264893, + 0.0164608433842659, + 0.21254536509513855, + 1.8652161359786987, + 2.435795545578003, + 1.3397101163864136, + -0.16474942862987518, + -1.0032472610473633, + 0.19162122905254364, + -0.17032292485237122, + -1.4556047916412354, + -2.5731472969055176 + ], + [ + -0.3986818790435791, + -0.32030463218688965, + 0.6388684511184692, + -0.2922161817550659, + 1.1398611068725586, + 2.398139238357544, + 3.4355506896972656, + 1.4486161470413208, + -0.1690330058336258, + 0.20826128125190735, + -0.14162737131118774, + 0.22350215911865234, + 1.9856444597244263, + 2.3961915969848633, + 1.4219110012054443, + -0.10985557734966278, + -1.0580624341964722, + 0.24057930707931519, + -0.3399960994720459, + -1.33854079246521, + -2.2894551753997803 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-16661b2fd2c85f57a66c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23635749518871307, + 0.1864171028137207, + 0.18280422687530518, + 0.16576561331748962, + 0.2967458665370941, + 0.2833873927593231, + 0.2587948143482208, + 0.2695174813270569, + 0.2239263951778412, + 0.15686756372451782, + 0.2505500912666321, + 0.12250886857509613, + 0.16453635692596436, + 2.0, + 0.13450095057487488, + 0.11914549767971039 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.12099093198776245, + 0.06886856257915497, + 0.2952747344970703, + -0.1764048933982849, + -0.12411242723464966, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + -0.046127498149871826, + 0.12581980228424072, + 0.042072445154190063, + -0.06942430138587952, + 0.7517287731170654, + 0.0, + 0.690804123878479, + -0.06023237854242325, + 0.049633681774139404, + 0.01943105459213257, + 0.013327360153198242, + -0.8525059223175049, + 0.0 + ], + [ + 0.3599843978881836, + -0.34296315908432007, + 0.2952747344970703, + -0.3797188997268677, + -0.41771262884140015, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + 0.06634362041950226, + 0.138022780418396, + -0.2342309057712555, + -0.36017337441444397, + 0.7517287731170654, + 0.0, + 0.690804123878479, + 0.10550708323717117, + -0.35431206226348877, + 0.16382712125778198, + 0.06674730777740479, + -0.8525059223175049, + 0.0 + ], + [ + 0.567882239818573, + -0.4729353189468384, + 1.6256722211837769, + -1.5372763872146606, + 1.0487556457519531, + 1.5327277183532715, + 2.2847697734832764, + 2.6703484058380127, + 0.4013517200946808, + -1.3629915714263916, + 0.6385155916213989, + 0.2664203345775604, + 1.2691267728805542, + 1.2942520380020142, + 2.2084813117980957, + 0.6982719302177429, + -0.8422143459320068, + 1.070263385772705, + -0.9558742046356201, + -1.3996182680130005, + -1.4242117404937744 + ], + [ + 0.6830785274505615, + -0.5584439635276794, + 0.841848611831665, + -0.5293771624565125, + -0.14354348182678223, + 2.097932815551758, + 2.9733188152313232, + 2.603226661682129, + -0.1639159619808197, + -0.918163537979126, + 1.002030849456787, + 0.40132570266723633, + 1.1751198768615723, + 1.455662727355957, + 2.3202755451202393, + -0.18584474921226501, + -1.3483099937438965, + 0.0660635232925415, + 0.4330971837043762, + -1.1989169120788574, + -2.0896859169006348 + ], + [ + 0.44418102502822876, + 0.07469291985034943, + 1.6960139274597168, + -1.329486608505249, + 1.3067576885223389, + 2.2694292068481445, + 2.0299551486968994, + 1.6317954063415527, + 0.7472026348114014, + -0.12518703937530518, + 0.3035789430141449, + -0.5889836549758911, + 2.345867395401001, + 1.9470158815383911, + 0.8398547172546387, + -0.6090301871299744, + -1.5060018301010132, + 0.5695335865020752, + -0.6025367975234985, + -1.8703374862670898, + -2.728482246398926 + ], + [ + 0.31904691457748413, + 0.04995492100715637, + 1.3823881149291992, + -0.5106721520423889, + -0.2236303687095642, + 0.9921484589576721, + 1.5933295488357544, + 2.741320848464966, + -0.21808990836143494, + -0.7775652408599854, + 1.224307894706726, + -0.08168366551399231, + 2.692474842071533, + 2.269430160522461, + 2.3270838260650635, + -0.2901480495929718, + -1.2098140716552734, + -0.07570821046829224, + -1.1340655088424683, + -1.9999847412109375, + -1.6802332401275635 + ], + [ + -0.6257733106613159, + 0.3840647339820862, + 0.9012271165847778, + -1.248098611831665, + -0.1650761365890503, + 2.7645747661590576, + 2.7588071823120117, + 3.0172548294067383, + 0.4120163321495056, + -1.6110656261444092, + -0.10423526167869568, + 0.5334746241569519, + 1.7809263467788696, + 0.5690644383430481, + 1.5881801843643188, + 0.8860509991645813, + -1.1502506732940674, + 0.7821243405342102, + -0.549727201461792, + -1.326378345489502, + -1.5100535154342651 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2773973345756531, + -0.13796104490756989, + 1.2952747344970703, + -1.0323810577392578, + 0.5605664849281311, + 1.8606208562850952, + 2.439758777618408, + 2.353257894515991, + 0.24642756581306458, + -0.861977219581604, + 0.4799116551876068, + 0.264199823141098, + 1.7517287731170654, + 1.4282559156417847, + 1.690804123878479, + 0.1215878501534462, + -0.9322434067726135, + 0.8536378145217896, + -0.35758620500564575, + -1.8525059223175049, + -1.910990595817566 + ], + [ + -0.4916617274284363, + -0.5934253931045532, + 2.074575424194336, + -0.9512085318565369, + 0.3705800175666809, + 1.691048264503479, + 2.2907681465148926, + 1.532329797744751, + 0.8991785049438477, + -0.7733803987503052, + 1.3577321767807007, + 0.9844847917556763, + 1.7379451990127563, + 2.050845146179199, + 0.7804086804389954, + -0.8400509357452393, + -1.6339564323425293, + 0.9922879934310913, + 0.6006971597671509, + -1.0612030029296875, + -1.3134464025497437 + ] + ], + "predicted_scores": [ + 8.758014678955078, + 8.409639358520508, + 8.26655387878418, + 7.458409309387207, + 7.388648509979248, + 7.007550239562988, + 6.89915657043457, + 6.521210670471191 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "74b116fd9ceddd5b2261edd96a14934a4b4a78a40191a654d6743661b819f407", + "stored_base_utility": 0.23635749518871307, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.00048063197755254805, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00048063197755254805 + }, + { + "contact_quality": null, + "progress": 0.0005253751878626645, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0005253751878626645 + }, + { + "contact_quality": null, + "progress": 0.0004003842768725008, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0004003842768725008 + }, + { + "contact_quality": null, + "progress": 0.0003489255905151367, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0003489255905151367 + }, + { + "contact_quality": null, + "progress": 0.00048579773283563554, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00048579773283563554 + }, + { + "contact_quality": null, + "progress": 0.001480837701819837, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001480837701819837 + }, + { + "contact_quality": null, + "progress": 0.0006714065675623715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006714065675623715 + }, + { + "contact_quality": null, + "progress": 0.00032782554626464844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00032782554626464844 + } + ], + "candidate_progress": [ + 0.00048063197755254805, + 0.0005253751878626645, + 0.0004003842768725008, + 0.0003489255905151367, + 0.00048579773283563554, + 0.001480837701819837, + 0.0006714065675623715, + 0.00032782554626464844 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-8e12f7ad446482627dd3", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-20e1efbb745fd02abc95", + "generated_tangents": [ + [ + -0.12346334755420685, + -0.19985295832157135, + 2.1784703731536865, + -0.31124866008758545, + 1.529472827911377, + 2.947808265686035, + 2.6920390129089355, + -0.6265994310379028, + 0.13566288352012634, + 1.5676236152648926, + 0.112566739320755, + 1.0675935745239258, + 2.570864677429199, + 2.6500132083892822, + -1.4526606798171997, + -0.22069929540157318, + 2.1557509899139404, + 0.96132892370224, + 0.9685867428779602, + 2.9575047492980957, + 2.7185823917388916 + ], + [ + -0.43995237350463867, + -0.2050541639328003, + 1.7482885122299194, + -0.251895934343338, + 1.8548591136932373, + 3.3218228816986084, + 2.2479407787323, + -0.3862256109714508, + 0.015753012150526047, + 1.78085196018219, + 0.24729987978935242, + 1.187187910079956, + 2.186312437057495, + 3.038965940475464, + -1.5351717472076416, + -0.28550881147384644, + 1.970921516418457, + 0.9219973087310791, + 0.912335216999054, + 2.5899159908294678, + 2.3246777057647705 + ], + [ + -0.250826358795166, + -0.18446028232574463, + 1.9133923053741455, + -0.46357783675193787, + 1.6548621654510498, + 3.0725228786468506, + 2.4449257850646973, + -0.5827616453170776, + -0.009590886533260345, + 1.732719898223877, + 0.17942655086517334, + 1.117271065711975, + 2.3779704570770264, + 2.668149948120117, + -1.646291732788086, + -0.323905885219574, + 2.1908156871795654, + 1.0647403001785278, + 0.8036691546440125, + 2.8001701831817627, + 2.425461769104004 + ], + [ + -0.28640180826187134, + -0.2839859127998352, + 2.0572218894958496, + -0.42631280422210693, + 1.5923094749450684, + 2.8822576999664307, + 2.911482572555542, + -0.6053820848464966, + -0.04821009188890457, + 1.8190683126449585, + 0.24333569407463074, + 1.1664683818817139, + 2.697253704071045, + 2.843883991241455, + -1.71109139919281, + -0.26010245084762573, + 1.9914753437042236, + 1.162591814994812, + 0.9151231646537781, + 2.488668441772461, + 2.022063732147217 + ], + [ + -0.13244841992855072, + -0.17563793063163757, + 1.8345932960510254, + -0.5187920331954956, + 1.943190336227417, + 3.448361396789551, + 2.255242109298706, + -0.4262901842594147, + -0.10808656364679337, + 1.8234599828720093, + 0.2276066541671753, + 1.2756623029708862, + 1.9931296110153198, + 2.609802007675171, + -1.434948205947876, + -0.32289913296699524, + 2.1416492462158203, + 0.8319113254547119, + 0.8000109791755676, + 2.658656120300293, + 2.234731912612915 + ], + [ + -0.38166344165802, + -0.15863287448883057, + 1.6745790243148804, + -0.5549147725105286, + 1.5172958374023438, + 2.7655439376831055, + 2.8303303718566895, + -0.28214988112449646, + 0.051939524710178375, + 1.2991156578063965, + 0.2312173843383789, + 0.6783193349838257, + 1.7863367795944214, + 3.8908531665802, + -1.1424500942230225, + -0.4551764726638794, + 1.4787436723709106, + 0.5913362503051758, + 1.0201154947280884, + 2.6787302494049072, + 2.5280520915985107 + ], + [ + -0.6116804480552673, + -0.2950592041015625, + 1.8596454858779907, + -0.35463833808898926, + 1.1682342290878296, + 2.9465863704681396, + 2.540924072265625, + -0.2936297357082367, + -0.08103088289499283, + 1.50174880027771, + 0.025975782424211502, + 0.493008017539978, + 2.202176570892334, + 1.8654035329818726, + -1.3604581356048584, + -0.29839661717414856, + 1.7678908109664917, + 0.6027181148529053, + 0.622581958770752, + 2.6514248847961426, + 1.7808696031570435 + ], + [ + 0.1034335121512413, + -0.25065457820892334, + 1.947960615158081, + -0.7060551643371582, + 1.1494879722595215, + 2.454498767852783, + 2.7128725051879883, + -0.9030486345291138, + -0.08790617436170578, + 1.8212023973464966, + 0.2397160530090332, + 1.1197707653045654, + 2.8676016330718994, + 2.4723026752471924, + -1.5656120777130127, + -0.5339487791061401, + 2.1438119411468506, + 0.7354217767715454, + 0.49033674597740173, + 2.791179656982422, + 1.8338626623153687 + ] + ], + "generated_utilities": [ + 0.00048063197755254805, + 0.0005253751878626645, + 0.0004003842768725008, + 0.0003489255905151367, + 0.00048579773283563554, + 0.001480837701819837, + 0.0006714065675623715, + 0.00032782554626464844 + ], + "group_id": "ms-20e1efbb745fd02abc95", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13665315508842468, + 0.1696699559688568, + 0.049180448055267334, + 0.05942712351679802, + 0.22736775875091553, + 0.0799722671508789, + 0.17261533439159393, + 0.21202586591243744, + 0.18980810046195984, + 0.24345096945762634, + 0.08349347114562988, + 0.11624568700790405, + 0.17245106399059296, + 0.07060472667217255, + 0.041495781391859055, + 0.028585990890860558 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.08645056188106537, + -0.060760222375392914, + 0.6493666172027588, + -0.016124725341796875, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + -0.2971414625644684, + 0.023367643356323242, + 0.6914786100387573, + -0.1027369350194931, + 0.1733318567276001, + 1.2948026657104492, + 0.0, + -0.30476057529449463, + -0.10394972562789917, + 0.9822354316711426, + 0.055961012840270996, + 0.004969358444213867, + 2.122485637664795, + 0.0 + ], + [ + 0.046016186475753784, + -0.24098780751228333, + 0.6493666172027588, + 0.05737954378128052, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + 0.22233793139457703, + 0.3896067142486572, + 0.6914786100387573, + -0.061455681920051575, + 0.17170286178588867, + 1.2948026657104492, + 0.0, + -0.4717581272125244, + -0.5722612738609314, + 0.9822354316711426, + 0.3690842390060425, + 0.33333829045295715, + 2.122485637664795, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.590132236480713, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.670657634735107, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.27303409576416 + ], + [ + -0.4654892086982727, + -0.6659128069877625, + 2.053840160369873, + -0.17420679330825806, + 1.1238669157028198, + 2.773953914642334, + 2.9706993103027344, + -0.9652835726737976, + -0.6603021621704102, + 1.957632303237915, + 0.30636119842529297, + 1.0932379961013794, + 2.4988603591918945, + 2.1735024452209473, + -0.7229697108268738, + -0.6565687656402588, + 2.1097376346588135, + 0.79427170753479, + 1.1421128511428833, + 3.200641393661499, + 3.3294520378112793 + ], + [ + 0.8783296346664429, + -0.6790766716003418, + 1.3789844512939453, + -1.578566551208496, + 1.2084856033325195, + 1.906846046447754, + 2.7278928756713867, + -0.8570022583007812, + -0.3114888072013855, + 2.660332202911377, + 0.4615887999534607, + -0.16199523210525513, + 1.9830265045166016, + 2.4224376678466797, + -0.9260885715484619, + 0.5555675029754639, + 2.5474588871002197, + 0.0070996880531311035, + 1.627825379371643, + 3.9487287998199463, + 3.03330135345459 + ], + [ + -0.1274794638156891, + -0.26015421748161316, + 2.5414626598358154, + -1.0220305919647217, + 1.4708960056304932, + 2.0982167720794678, + 3.1457836627960205, + -1.0992621183395386, + -0.9075721502304077, + 0.8330968022346497, + 0.15631377696990967, + 1.3095502853393555, + 1.8582203388214111, + 2.101317882537842, + -0.9184049367904663, + 0.4676415026187897, + 2.784534454345703, + 0.4529607892036438, + 0.6403366327285767, + 3.1566896438598633, + 2.5268735885620117 + ], + [ + 0.9289348125457764, + 0.1545984447002411, + 1.6792887449264526, + -0.6400681138038635, + 0.5956984758377075, + 2.5183756351470947, + 1.6261346340179443, + -0.019961506128311157, + -0.2638699412345886, + 1.310987114906311, + 0.4996453523635864, + -0.06606829166412354, + 1.9830868244171143, + 2.392476797103882, + -0.3871336579322815, + -0.7797489166259766, + 1.927225947380066, + 0.4438522458076477, + 1.0257132053375244, + 3.211881160736084, + 3.374328851699829 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05862678587436676, + -0.042836256325244904, + 1.6493666172027588, + -0.7465556859970093, + 1.1865841150283813, + 2.873413562774658, + 2.2950661182403564, + -0.48127153515815735, + -0.11381363868713379, + 1.6914786100387573, + 0.06965510547161102, + 0.8288453221321106, + 2.294802665710449, + 2.3353288173675537, + -1.3047605752944946, + -0.39009055495262146, + 1.9822354316711426, + 0.9993376731872559, + 0.81731116771698, + 3.122485637664795, + 2.63651704788208 + ], + [ + -0.045653510838747025, + 0.00397084653377533, + 1.3591042757034302, + -0.6414353847503662, + 0.8377937078475952, + 2.8277816772460938, + 2.203674554824829, + -0.1951642632484436, + 0.12272481620311737, + 1.7411555051803589, + -0.004818864166736603, + 0.8769108057022095, + 2.243906021118164, + 2.4328396320343018, + -1.6330946683883667, + -0.4807296395301819, + 2.2988247871398926, + 0.8021057844161987, + 0.6970767378807068, + 2.8728103637695312, + 2.288278102874756 + ], + [ + -0.4273069500923157, + 0.16743427515029907, + 1.538269281387329, + -0.7002415657043457, + 0.7622614502906799, + 2.805328369140625, + 2.267277479171753, + -0.6541028022766113, + -0.2980780303478241, + 1.456940770149231, + 0.5105806589126587, + 1.040317177772522, + 2.4321720600128174, + 2.696331024169922, + -1.1555191278457642, + -0.025772064924240112, + 1.6384800672531128, + 1.4557092189788818, + 0.6594157814979553, + 3.1446406841278076, + 2.249678134918213 + ], + [ + -0.6431475877761841, + -0.24224388599395752, + 1.3122833967208862, + -0.16711556911468506, + 0.9232780337333679, + 2.850522994995117, + 2.645284414291382, + -0.379825621843338, + -0.6750738620758057, + 1.1236441135406494, + 0.4938291311264038, + 1.1075668334960938, + 2.203859806060791, + 2.799032688140869, + -1.359815239906311, + -0.5104091763496399, + 2.0970702171325684, + 1.057171106338501, + 1.1167995929718018, + 2.8190994262695312, + 2.2184524536132812 + ] + ], + "predicted_scores": [ + 7.416408061981201, + 7.266858100891113, + 7.034653663635254, + 6.855306625366211, + 6.838082313537598, + 6.716786861419678, + 6.6193695068359375, + 6.177856922149658 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1375492786", + "selected_index": 0, + "state_hash": "56acb365a02ebac0ac4893531e168df3c430b486416156feeea40eeb178bda3e", + "stored_base_utility": 0.13665315508842468, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2dc5138251afd26a040d", + "generated_tangents": [ + [ + 1.7014609575271606, + -0.18039368093013763, + -1.2970715761184692, + -0.6534698605537415, + 0.3687649965286255, + 0.38120630383491516, + -0.239515483379364, + 0.751300573348999, + -0.02512221969664097, + -0.17351478338241577, + -0.7087518572807312, + -0.45235753059387207, + -3.3236494064331055, + -2.2830886840820312, + 0.26985061168670654, + 0.1224251389503479, + 0.3557276725769043, + -1.178897738456726, + 1.1012275218963623, + -2.4061923027038574, + -1.28949773311615 + ], + [ + 1.600577712059021, + -0.31889867782592773, + -1.0949269533157349, + -0.7782318592071533, + -0.03460490703582764, + 0.11920750141143799, + -0.9855067133903503, + 0.6091313362121582, + 0.1639746129512787, + -0.45358240604400635, + -0.3156372904777527, + -0.21796345710754395, + -3.02553129196167, + -2.235438823699951, + -0.003896709531545639, + 0.19991938769817352, + 0.7951566576957703, + -0.8402144312858582, + 1.218689203262329, + -2.647265672683716, + -1.5202679634094238 + ], + [ + 1.7732620239257812, + -0.08437895774841309, + -1.4931981563568115, + -0.1378997415304184, + -0.0962563157081604, + 0.34288352727890015, + -0.7458742260932922, + 0.5131850242614746, + 0.03603658825159073, + -0.2769846022129059, + -0.28220295906066895, + -0.19899293780326843, + -3.198810338973999, + -2.617182493209839, + 0.13802781701087952, + 0.05464818701148033, + 0.3968123197555542, + -1.293387532234192, + 1.251478910446167, + -2.528707265853882, + -1.2376025915145874 + ], + [ + 1.7733588218688965, + -0.07844860851764679, + -1.2190992832183838, + -0.2284640371799469, + -0.32739758491516113, + 0.6094720363616943, + -0.29514384269714355, + 0.4748133718967438, + 0.062315359711647034, + -0.05596692115068436, + -0.4144563674926758, + -0.36419641971588135, + -3.6123297214508057, + -2.5737216472625732, + 0.10197815299034119, + 0.17784062027931213, + 0.5930182933807373, + -1.0416651964187622, + 0.9636204838752747, + -2.3183629512786865, + -1.5335577726364136 + ], + [ + 1.7985514402389526, + -0.037546902894973755, + -1.2421534061431885, + -0.5172708034515381, + 0.053566426038742065, + 0.3809960186481476, + -0.05482292175292969, + 0.6663963794708252, + -0.03202047199010849, + -0.12443280965089798, + -0.5220405459403992, + -0.44478660821914673, + -3.471993923187256, + -2.3812077045440674, + 0.18697009980678558, + 0.08937080204486847, + 0.308956503868103, + -1.1457538604736328, + 1.0770564079284668, + -2.7350029945373535, + -1.3568449020385742 + ], + [ + 1.7711821794509888, + -0.11741582304239273, + -1.2946592569351196, + -0.36421170830726624, + -0.1146346926689148, + 0.456108033657074, + -0.23670202493667603, + 0.5579202175140381, + -0.0011934908106923103, + -0.12236564606428146, + -0.39571714401245117, + -0.28584516048431396, + -3.3783440589904785, + -2.509171724319458, + 0.22266556322574615, + 0.08078525960445404, + 0.35258013010025024, + -1.213260293006897, + 1.0058355331420898, + -2.471781015396118, + -1.4205639362335205 + ], + [ + 1.9132753610610962, + -0.04391380399465561, + -1.0892757177352905, + -0.30472588539123535, + 0.13552846014499664, + 0.15591594576835632, + -0.6045941710472107, + 0.36314308643341064, + 0.1025685966014862, + -0.09091076999902725, + -0.15325510501861572, + -0.19675669074058533, + -2.6142191886901855, + -1.9814730882644653, + 0.12692448496818542, + 0.010162748396396637, + 0.4250797629356384, + -0.9183775782585144, + 1.0481324195861816, + -2.7949914932250977, + -1.3203645944595337 + ], + [ + 1.7334052324295044, + -0.057985819876194, + -0.8837970495223999, + -0.24913650751113892, + -0.10631182789802551, + -0.24190834164619446, + 0.056062906980514526, + 0.6961460113525391, + 0.07683011889457703, + -0.2773747444152832, + -0.3860301375389099, + -0.47417378425598145, + -3.164853811264038, + -2.0257632732391357, + 0.2156425416469574, + 0.07730689644813538, + 0.34279489517211914, + -1.0879709720611572, + 1.0649265050888062, + -2.7084548473358154, + -1.4397176504135132 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-2dc5138251afd26a040d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1373138427734375, + 0.1398601531982422, + 0.11154793202877045, + 0.07755142450332642, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7500379085540771, + -0.32870030403137207, + -0.3281364440917969, + 0.020404458045959473, + 0.03884005546569824, + 0.26651138067245483, + 0.0, + 0.2614295184612274, + 0.02006034180521965, + 0.6256309151649475, + 0.10119366645812988, + 0.0365104079246521, + -1.9807136058807373, + 0.0, + -0.4040032625198364, + -0.3390212059020996, + 0.444367915391922, + -1.3744113445281982, + 0.3090496063232422, + -1.6372854709625244, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7500379085540771, + 0.1347455233335495, + -1.3281364440917969, + -0.3984496593475342, + -0.6640693545341492, + -0.7334886193275452, + -1.63351309299469, + 0.4696621894836426, + -0.03721961751580238, + 0.0896599143743515, + -0.8988063335418701, + -0.047963276505470276, + -2.9807136058807373, + -2.542311906814575, + 0.5875393152236938, + 0.41857796907424927, + 0.32660895586013794, + -2.3744113445281982, + 0.9575246572494507, + -2.6372854709625244, + -1.9586280584335327 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.26702618598938, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.08462381362915, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.9172561168670654 + ], + [ + 1.7928142547607422, + -0.046942055225372314, + -1.3733142614364624, + -0.4212188422679901, + -0.5566414594650269, + -0.9966021180152893, + -1.7926092147827148, + 0.37284746766090393, + 0.18181614577770233, + -0.056899696588516235, + -0.8793746829032898, + -0.06206277757883072, + -3.2539288997650146, + -2.3651227951049805, + 0.543649435043335, + 0.4969414472579956, + 0.3953157365322113, + -2.6153955459594727, + 1.239845633506775, + -2.9289329051971436, + -2.1138057708740234 + ], + [ + 1.4313925504684448, + 0.3287010192871094, + -1.4217545986175537, + -0.11546039581298828, + -0.38684383034706116, + -0.3866080045700073, + -1.4945378303527832, + 0.4180656969547272, + 0.08673545718193054, + -0.30675017833709717, + -0.8370212316513062, + -0.3124745488166809, + -3.2861690521240234, + -2.5144338607788086, + 0.7133253216743469, + 0.5272455215454102, + 0.6288780570030212, + -2.7024707794189453, + 1.0730262994766235, + -2.603350877761841, + -1.8104162216186523 + ], + [ + 2.0841901302337646, + 0.18348857760429382, + -1.4159170389175415, + -0.5868262052536011, + -0.3290694057941437, + -0.8189557790756226, + -1.811798334121704, + 0.8927561044692993, + -0.4510294198989868, + -0.1373635083436966, + -0.46978816390037537, + -0.2017078548669815, + -3.1228857040405273, + -2.619961977005005, + 0.7154320478439331, + 0.4623824954032898, + 0.8127838373184204, + -2.5683043003082275, + 0.8921399116516113, + -2.799823522567749, + -1.6220401525497437 + ], + [ + 1.5681514739990234, + -0.11813957989215851, + -1.066882848739624, + -0.4835556447505951, + -0.4833664894104004, + -0.1345365047454834, + -1.872452735900879, + 1.059093952178955, + -0.3537854254245758, + 0.5130321979522705, + -0.7769420146942139, + -0.4462882876396179, + -3.249842882156372, + -2.2144932746887207, + 0.6907999515533447, + 0.14545467495918274, + 0.8816275596618652, + -2.4357430934906006, + 1.424092411994934, + -2.656961441040039, + -2.3529839515686035 + ], + [ + 2.0935373306274414, + 0.20671290159225464, + -1.0638595819473267, + -0.37751641869544983, + -0.5913090705871582, + -0.3702518939971924, + -1.3611384630203247, + 0.8432458639144897, + 0.4395832121372223, + -0.19578908383846283, + -1.2028591632843018, + -0.07535962760448456, + -3.2909016609191895, + -2.9216225147247314, + 1.1581575870513916, + 0.9284459352493286, + -0.3347962498664856, + -1.9277539253234863, + 0.8971017003059387, + -3.245394229888916, + -2.1499102115631104 + ], + [ + 2.5108895301818848, + -0.2977973222732544, + -1.458537220954895, + -0.5705423355102539, + -1.4453105926513672, + -0.4854125380516052, + -2.383378028869629, + 0.39735814929008484, + -0.3523370325565338, + 0.4586251974105835, + -1.2388068437576294, + 0.1082226037979126, + -3.5858566761016846, + -1.8809643983840942, + -0.18688076734542847, + 0.0871247947216034, + 0.0062812864780426025, + -3.012958526611328, + 1.0266033411026, + -3.1649136543273926, + -2.6460556983947754 + ], + [ + 1.2948591709136963, + 0.21422013640403748, + -1.3904463052749634, + -0.13111025094985962, + -0.1661812961101532, + -0.4983506500720978, + -2.0006139278411865, + -0.4048727750778198, + 0.10130663216114044, + 0.1955089420080185, + -0.27839744091033936, + 0.8026529550552368, + -3.2792744636535645, + -1.6839708089828491, + 0.6468846201896667, + -0.2772523760795593, + 0.45975640416145325, + -2.612403631210327, + 1.6571701765060425, + -1.8971503973007202, + -1.2593979835510254 + ], + [ + 0.9154124855995178, + -0.2073056846857071, + -2.1218466758728027, + -0.4230979084968567, + -1.2478010654449463, + -0.5439814925193787, + -1.185287594795227, + -0.06685006618499756, + -0.7776275873184204, + -0.015365853905677795, + -1.3256038427352905, + 0.46163398027420044, + -2.8853814601898193, + -2.387786865234375, + 1.2300693988800049, + 0.9541484117507935, + 0.29864931106567383, + -2.931433916091919, + 1.3643410205841064, + -2.001633882522583, + -2.6548502445220947 + ], + [ + 2.035454750061035, + 0.03935282677412033, + -1.9080866575241089, + -0.17657285928726196, + -1.4843900203704834, + -0.4207392632961273, + -1.554489016532898, + 0.1909814178943634, + -0.5061874389648438, + -0.420978844165802, + -0.7634562849998474, + -0.7818551063537598, + -3.374903678894043, + -2.5324442386627197, + 1.2213574647903442, + 0.9676649570465088, + 0.5052346587181091, + -2.0974624156951904, + 0.9588823914527893, + -3.6367201805114746, + -1.953636646270752 + ], + [ + 1.4856661558151245, + 0.3259274363517761, + -2.1635096073150635, + -0.8793277740478516, + 0.2686290740966797, + -0.340938925743103, + -0.8516338467597961, + -0.3830271363258362, + 0.5376158356666565, + -0.2721732258796692, + -1.2276334762573242, + -0.38624608516693115, + -3.346294403076172, + -3.093522071838379, + 0.6735774278640747, + 0.5564011931419373, + -0.5450864434242249, + -1.5815953016281128, + 1.4068235158920288, + -2.7531042098999023, + -2.515698194503784 + ] + ], + "predicted_scores": [ + 6.710046291351318, + 6.684288024902344, + 6.168863773345947, + 6.0246429443359375, + 5.804898262023926, + 5.693931579589844, + 5.307399272918701, + 5.256102085113525 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565621826", + "selected_index": 0, + "state_hash": "f6edd849b0b51788d7be25e402fb514cb777ccf220797faaf769000814df02cb", + "stored_base_utility": 0.1373138427734375, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08252689987421036, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08252689987421036 + }, + "base_progress": 0.08252689987421036, + "base_success": false, + "base_utility": 0.08252689987421036, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06141128018498421, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06141128018498421 + }, + { + "contact_quality": null, + "progress": 0.06181597709655762, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06181597709655762 + }, + { + "contact_quality": null, + "progress": 0.14589862525463104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14589862525463104 + }, + { + "contact_quality": null, + "progress": 0.05644259974360466, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05644259974360466 + }, + { + "contact_quality": null, + "progress": 0.061918895691633224, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061918895691633224 + }, + { + "contact_quality": null, + "progress": 0.058040279895067215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058040279895067215 + }, + { + "contact_quality": null, + "progress": 0.04945683479309082, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04945683479309082 + }, + { + "contact_quality": null, + "progress": 0.04462442919611931, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04462442919611931 + } + ], + "candidate_progress": [ + 0.06141128018498421, + 0.06181597709655762, + 0.14589862525463104, + 0.05644259974360466, + 0.061918895691633224, + 0.058040279895067215, + 0.04945683479309082, + 0.04462442919611931 + ], + "candidate_source_chart_ids": [ + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-206099bab6bfe7c84fce", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-387e6771a55be98f1a59", + "generated_tangents": [ + [ + 0.06006329506635666, + -0.08010709285736084, + 0.11080029606819153, + 0.32980427145957947, + 1.7240791320800781, + -0.362089604139328, + -0.2584540843963623, + -0.19833067059516907, + 0.30151551961898804, + 0.07162415981292725, + 0.2010030746459961, + 0.20961271226406097, + -0.03765740990638733, + -0.08270052075386047, + -0.09435108304023743, + -0.049414895474910736, + 0.108904629945755, + -0.16463418304920197, + -0.03854423761367798, + 0.15900826454162598, + -0.04328756034374237 + ], + [ + 0.009599551558494568, + -0.16914266347885132, + -0.1386336386203766, + 0.30693379044532776, + 1.6530368328094482, + -0.07611559331417084, + -0.1791226863861084, + 0.02553841471672058, + 0.19469773769378662, + 0.027957111597061157, + 0.45823749899864197, + 0.32478460669517517, + -0.1743784248828888, + 0.21550936996936798, + -0.10836927592754364, + 0.054596196860075, + 0.16475197672843933, + 0.03313766419887543, + 0.32187891006469727, + 0.3524985909461975, + 0.12549921870231628 + ], + [ + 0.6465438008308411, + -0.3374776244163513, + 0.12796393036842346, + 0.2827477753162384, + 1.7577989101409912, + -0.5565595626831055, + -0.16577357053756714, + 0.01457950472831726, + 0.24365079402923584, + 0.04987485706806183, + 0.8056530356407166, + 0.12356974929571152, + 0.26063719391822815, + 0.6298707127571106, + -0.145112082362175, + 0.06279179453849792, + 0.17149415612220764, + -0.29395294189453125, + 0.5806790590286255, + 0.426554799079895, + -0.4817831814289093 + ], + [ + -0.053341493010520935, + -0.1865396499633789, + 0.16841968894004822, + 0.11713874340057373, + 1.4590277671813965, + 0.3799755573272705, + -0.681486189365387, + 0.0640789270401001, + 0.1813700795173645, + 0.0997859388589859, + 0.4823245406150818, + 0.2732934057712555, + -0.20591291785240173, + -0.20029090344905853, + -0.24545499682426453, + 0.04443333297967911, + 0.1731298565864563, + 0.21911995112895966, + 0.1807205229997635, + 0.3559965491294861, + 0.2999149560928345 + ], + [ + -0.027210667729377747, + -0.03206159919500351, + -0.052471309900283813, + 0.01967671513557434, + 0.7426930665969849, + -0.8253703117370605, + -0.018247127532958984, + 0.12169646471738815, + 0.19982509315013885, + 0.18069101870059967, + 0.22702598571777344, + -0.37949854135513306, + -0.7075455188751221, + 0.003782108426094055, + -0.10478651523590088, + -0.0896352082490921, + 0.09950701892375946, + 0.07654224336147308, + 0.5347628593444824, + -0.2050948143005371, + 0.24008597433567047 + ], + [ + -0.10568918287754059, + -0.07966116070747375, + 0.003670215606689453, + -0.08083397150039673, + 0.8740999698638916, + -0.6896226406097412, + 0.08204978704452515, + 0.07617051899433136, + 0.1713121086359024, + 0.2213243693113327, + 0.31011971831321716, + -0.4065733551979065, + -0.656135618686676, + 0.05357886850833893, + -0.1855739802122116, + -0.11018005013465881, + -0.04411895573139191, + 0.08190812170505524, + 0.586898922920227, + -0.3115938901901245, + 0.23166456818580627 + ], + [ + -0.11853678524494171, + -0.060251377522945404, + 0.036512553691864014, + -0.0002815127372741699, + 0.7336574792861938, + -0.8709181547164917, + 0.08950820565223694, + 0.04132549464702606, + 0.16058886051177979, + 0.2268449366092682, + 0.2850486934185028, + -0.40542882680892944, + -0.6018311977386475, + 0.1696666181087494, + -0.09339651465415955, + -0.09056708216667175, + 0.09688669443130493, + 0.05742751061916351, + 0.5989552736282349, + -0.15574336051940918, + 0.22451287508010864 + ], + [ + -0.22560490667819977, + -0.031416166573762894, + 0.11794865131378174, + -0.3758751153945923, + 1.021733283996582, + -1.0908676385879517, + 0.201805979013443, + -0.062230855226516724, + 0.10675548762083054, + 0.26547670364379883, + 0.044261425733566284, + -0.1574232280254364, + -0.8401191234588623, + -1.262336254119873, + 0.0014948472380638123, + -0.12965939939022064, + -0.27413487434387207, + -0.11683200299739838, + 0.5190190076828003, + -0.29211071133613586, + -0.27497655153274536 + ] + ], + "generated_utilities": [ + 0.06141128018498421, + 0.06181597709655762, + 0.14589862525463104, + 0.05644259974360466, + 0.061918895691633224, + 0.058040279895067215, + 0.04945683479309082, + 0.04462442919611931 + ], + "group_id": "ms-387e6771a55be98f1a59", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.03822460025548935, + 0.18199902772903442, + 0.20744429528713226, + 0.05679210275411606, + 0.07646497339010239, + 0.038853906095027924, + 0.0824488028883934, + 0.05751838535070419, + 0.02475176379084587, + 0.040490686893463135, + 0.027167975902557373, + 0.06515108048915863, + 0.04604649543762207, + 0.014139334671199322, + 0.0258652176707983, + 0.034187138080596924 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0643576979637146, + 0.02293895184993744, + 0.03272289037704468, + 0.02799379825592041, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + 0.05611860752105713, + -0.010578691959381104, + 0.03396539017558098, + 0.754948616027832, + -0.12030506134033203, + 0.05740344524383545, + 0.0, + 0.01674991101026535, + 0.0017660856246948242, + 0.1110122799873352, + -0.29033517837524414, + -0.015073239803314209, + -0.062492311000823975, + 0.0 + ], + [ + 0.03256732225418091, + 0.029803231358528137, + -0.11524808406829834, + -0.03975573182106018, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + -0.11635321378707886, + -0.12916040420532227, + -0.10934318602085114, + 0.754948616027832, + -0.10127544403076172, + 0.10553282499313354, + 0.0, + 0.2124219685792923, + -0.13985675573349, + 0.20122885704040527, + -0.29033517837524414, + -0.09104716777801514, + -0.33053046464920044, + 0.0 + ] + ], + "predicted_scores": [ + 7.301035404205322, + 7.189646244049072, + 6.6587982177734375, + 6.655889987945557, + 6.1351423263549805, + 6.031546115875244, + 5.582923889160156, + 4.6993608474731445 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "6b753a027f9378ff308946331f6bb658d452d8e77220ab0cd6fc813be0ac75cc", + "stored_base_utility": 0.03822460025548935, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03917040303349495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03917040303349495 + }, + "base_progress": 0.03917040303349495, + "base_success": false, + "base_utility": 0.03917040303349495, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11718571186065674, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11718571186065674 + }, + { + "contact_quality": null, + "progress": 0.08475744724273682, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08475744724273682 + }, + { + "contact_quality": null, + "progress": 0.06714028120040894, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06714028120040894 + }, + { + "contact_quality": null, + "progress": 0.035049181431531906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035049181431531906 + }, + { + "contact_quality": null, + "progress": 0.03447306156158447, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03447306156158447 + }, + { + "contact_quality": null, + "progress": 0.02684718370437622, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02684718370437622 + }, + { + "contact_quality": null, + "progress": 0.02741086483001709, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02741086483001709 + }, + { + "contact_quality": null, + "progress": 0.0036935408134013414, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0036935408134013414 + } + ], + "candidate_progress": [ + 0.11718571186065674, + 0.08475744724273682, + 0.06714028120040894, + 0.035049181431531906, + 0.03447306156158447, + 0.02684718370437622, + 0.02741086483001709, + 0.0036935408134013414 + ], + "candidate_source_chart_ids": [ + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-be219977a9db8dbb1e58", + "ms-98eb81debb5186ebbe38", + "ms-7dedf60ad41ea2d92ee5" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-38f7298184a2c5005e94", + "generated_tangents": [ + [ + 0.6369849443435669, + 0.11492690443992615, + 0.0840025246143341, + -0.7244514226913452, + 0.8756675124168396, + -1.535658836364746, + 0.612249493598938, + 0.3143526613712311, + 0.04067815840244293, + 0.2646087110042572, + -0.8137031197547913, + 0.3005238175392151, + -1.509477138519287, + -0.9800235033035278, + 0.00546921044588089, + 0.1681281328201294, + 0.005790531635284424, + 0.14234024286270142, + -0.4064610004425049, + -1.5683271884918213, + -0.9911508560180664 + ], + [ + 0.5584995150566101, + -0.05000460147857666, + -0.1664465367794037, + -0.5341358184814453, + 1.0766503810882568, + -1.296900987625122, + 0.026495516300201416, + 0.017091497778892517, + -0.03880733251571655, + 0.20170888304710388, + -0.6933715343475342, + 0.6240185499191284, + -1.3246278762817383, + -0.7883918881416321, + -0.15517935156822205, + 0.212482288479805, + 0.05112868547439575, + 0.20218025147914886, + -0.336897611618042, + -1.534984827041626, + -0.8085364103317261 + ], + [ + 0.41416382789611816, + -0.06269974261522293, + -0.047496646642684937, + -0.6089473962783813, + 1.2396233081817627, + -1.3737013339996338, + -0.11145180463790894, + 0.05461525917053223, + -0.07937335968017578, + 0.1598551869392395, + -0.6420341730117798, + 0.6470789909362793, + -1.4348926544189453, + -0.8785654306411743, + -0.29571256041526794, + 0.17546021938323975, + -0.1828174740076065, + 0.14676928520202637, + -0.29687660932540894, + -1.3849177360534668, + -0.8743813037872314 + ], + [ + 0.4911530613899231, + -0.042739931493997574, + -0.3767957389354706, + -0.6375311613082886, + 1.306060791015625, + -1.2735764980316162, + -0.08663177490234375, + -0.23911377787590027, + 0.09168751537799835, + 0.38278016448020935, + -0.6272723078727722, + 0.5073744058609009, + -1.6354384422302246, + -0.742485761642456, + -0.10793615877628326, + 0.2449941486120224, + -0.21595244109630585, + 0.2828562259674072, + -0.22117012739181519, + -1.4895732402801514, + -0.644891619682312 + ], + [ + 0.2277255356311798, + -0.16269108653068542, + 0.18027782440185547, + -0.10532813519239426, + 0.7146583199501038, + -1.3409571647644043, + 0.010533004999160767, + 0.09202136844396591, + 0.0896114632487297, + 0.23388461768627167, + -0.4258403480052948, + 1.0549230575561523, + -0.8153125047683716, + -0.25280433893203735, + -0.3791922926902771, + 0.1472392976284027, + 0.007149025797843933, + 0.072731614112854, + -0.04462990164756775, + -1.410667896270752, + -0.78948974609375 + ], + [ + -0.20279093086719513, + -0.20314247906208038, + -0.0337354838848114, + 0.0771024227142334, + 0.8623231649398804, + -1.8566323518753052, + 0.12431898713111877, + 0.006875365972518921, + 0.18823981285095215, + 0.36663636565208435, + -0.35092443227767944, + 0.3065592050552368, + -1.2975765466690063, + -0.06951725482940674, + -0.21218442916870117, + 0.20042358338832855, + 0.06910833716392517, + 0.05749613046646118, + 0.10933664441108704, + -1.560270071029663, + -1.0320936441421509 + ], + [ + 0.13047151267528534, + -0.22059795260429382, + 0.020146965980529785, + 0.19127678871154785, + 0.9241453409194946, + -1.7760614156723022, + 0.15420421957969666, + -0.04240788519382477, + 0.18307676911354065, + 0.22342398762702942, + -0.33989664912223816, + 0.9492338299751282, + -1.38172447681427, + 0.3790102005004883, + -0.2612534761428833, + 0.15426896512508392, + 0.12225785851478577, + 0.1312655657529831, + 0.02084413170814514, + -1.6546058654785156, + -0.9936606884002686 + ], + [ + -0.4820966124534607, + 0.016768746078014374, + -0.1510896384716034, + -0.04225653409957886, + 2.0557963848114014, + -2.255887269973755, + 0.05668759346008301, + -0.1425926387310028, + 0.1668226569890976, + 0.40121230483055115, + -0.6604657173156738, + 0.6876393556594849, + -1.0268025398254395, + 0.43825095891952515, + -0.3937377631664276, + 0.44952335953712463, + 0.25725775957107544, + 0.053308889269828796, + 0.01985764503479004, + -1.5205094814300537, + -0.5356847047805786 + ] + ], + "generated_utilities": [ + 0.11718571186065674, + 0.08475744724273682, + 0.06714028120040894, + 0.035049181431531906, + 0.03447306156158447, + 0.02684718370437622, + 0.02741086483001709, + 0.0036935408134013414 + ], + "group_id": "ms-38f7298184a2c5005e94", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1194220781326294, + 0.1767096221446991, + 0.0505807027220726, + 0.026383202522993088, + 0.09035396575927734, + 0.11899666488170624, + 0.059947751462459564, + 0.08427222818136215, + 0.041255753487348557, + 0.05034176632761955, + 0.02779809758067131, + 0.011037330143153667, + 0.020422041416168213, + 0.049474477767944336, + 0.03517866134643555, + 0.02796018123626709 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.08923250436782837, + 0.05675226449966431, + -0.1250607967376709, + -0.07566308975219727, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.001595139503479004, + 0.026048749685287476, + 0.1681501865386963, + -0.11568224430084229, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.18507611751556396, + -0.15127414464950562, + 0.07637521624565125, + -0.15284772217273712, + 0.0320390909910202, + -1.8774652481079102, + 0.0 + ], + [ + -0.2749255299568176, + 0.2578675150871277, + -0.46759605407714844, + -0.5041444897651672, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.13614141941070557, + 0.11120480298995972, + 0.15478569269180298, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.01827770471572876, + -0.38639169931411743, + -0.18168723583221436, + 0.3532869219779968, + -0.5437159538269043, + -1.8774652481079102, + 0.0 + ], + [ + 0.4466920495033264, + 0.07532927393913269, + 0.8145812749862671, + -0.963495135307312, + 1.3846542835235596, + -3.0133423805236816, + -1.0476078987121582, + 0.6759306788444519, + 0.5485053658485413, + 0.634345293045044, + -0.7682492733001709, + 2.2281322479248047, + -2.7100830078125, + -0.8660445213317871, + 0.7648183703422546, + 0.607989490032196, + -0.6848516464233398, + -0.06598936021327972, + 0.2030458152294159, + -2.9672060012817383, + -1.4293488264083862 + ], + [ + 1.0058056116104126, + 0.3426892161369324, + 0.6375484466552734, + -0.5828341841697693, + 1.7963712215423584, + -3.3760194778442383, + -0.9086123704910278, + 0.20509907603263855, + 0.10472571849822998, + 1.1158980131149292, + -1.0710231065750122, + 2.203950881958008, + -2.4466300010681152, + -0.3184358477592468, + 0.43084120750427246, + 0.32133060693740845, + -0.6005205512046814, + 0.07880236953496933, + 0.52874755859375, + -2.7805917263031006, + -1.6972103118896484 + ], + [ + 0.48786461353302, + -0.007949680089950562, + 0.41629794239997864, + -0.2277727723121643, + 1.0953291654586792, + -2.7825636863708496, + -1.7178279161453247, + 0.8412191271781921, + 0.6389099359512329, + 0.7100303769111633, + -1.1701807975769043, + 2.8546230792999268, + -2.570579767227173, + -0.2383948564529419, + 0.4282584488391876, + 0.27555936574935913, + -0.4073549509048462, + 0.49247825145721436, + 0.6483681797981262, + -3.3716540336608887, + -1.891394853591919 + ], + [ + 0.17139381170272827, + 0.831203818321228, + 0.1966383457183838, + -1.5011540651321411, + 1.811257004737854, + -2.6416001319885254, + -1.8791863918304443, + 0.03218984603881836, + 1.0182924270629883, + 1.5643268823623657, + -1.2954871654510498, + 2.249680757522583, + -1.9383416175842285, + -1.4303685426712036, + 0.48937559127807617, + 0.2848057746887207, + -0.6113857626914978, + 0.6453920006752014, + -0.13643476366996765, + -2.514732599258423, + -1.8046306371688843 + ], + [ + 0.1380026936531067, + 0.5165722370147705, + 0.9639596939086914, + -0.3682268261909485, + 2.2811193466186523, + -3.5902557373046875, + -1.047336220741272, + 1.3064022064208984, + 0.7264143228530884, + 1.3356540203094482, + -0.41242116689682007, + 1.8184869289398193, + -3.016441822052002, + -0.8501555919647217, + 1.0147737264633179, + 1.0417438745498657, + -0.24422766268253326, + 0.3441629707813263, + 1.075059175491333, + -3.178764820098877, + -1.7388081550598145 + ], + [ + 1.0805304050445557, + -0.5906903743743896, + -0.32329434156417847, + -1.7024246454238892, + 1.3012806177139282, + -3.3655173778533936, + -2.0100932121276855, + 0.28595322370529175, + -0.42309290170669556, + 0.9014679193496704, + -0.11985284090042114, + 3.033172607421875, + -1.890408992767334, + -0.049424707889556885, + 0.8836197853088379, + 0.07151538133621216, + -0.6898007392883301, + 0.6614744663238525, + 0.7956362962722778, + -3.344180107116699, + -1.4918344020843506 + ], + [ + 1.4192681312561035, + -0.5825785994529724, + 0.11329725384712219, + -0.08465814590454102, + 1.0504729747772217, + -2.438842535018921, + -0.9858216047286987, + -0.35852980613708496, + 1.4301284551620483, + 0.37676453590393066, + -1.3725404739379883, + 3.114079475402832, + -2.0786290168762207, + -0.5835205912590027, + 0.1985606551170349, + 0.260464072227478, + 0.41686397790908813, + -0.104665607213974, + 0.6382055282592773, + -2.7474825382232666, + -1.663126826286316 + ], + [ + 1.0832164287567139, + 0.11555392295122147, + 1.2108526229858398, + -1.5446858406066895, + 0.9112894535064697, + -3.3027408123016357, + -1.2419135570526123, + -0.057880282402038574, + -0.07349961996078491, + 0.026286423206329346, + -0.897603452205658, + 3.2857306003570557, + -2.232046365737915, + -1.2509676218032837, + 1.1900405883789062, + 0.8446143269538879, + -1.339659333229065, + -0.07882180064916611, + 0.1166829764842987, + -2.662961721420288, + -1.0079293251037598 + ], + [ + 0.43863287568092346, + -0.2024492472410202, + 0.0567094087600708, + -0.3029903173446655, + 1.612892746925354, + -3.26739764213562, + -1.7898839712142944, + -0.1649491786956787, + 0.2927369475364685, + 1.020188808441162, + -0.10171866416931152, + 2.3230679035186768, + -2.615206003189087, + -0.08625847101211548, + 0.9057100415229797, + 1.4201092720031738, + -0.3173455595970154, + 0.09508261829614639, + 0.9565507173538208, + -2.2954390048980713, + -1.2229193449020386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.03360384702682495, + -0.028338372707366943, + -0.12341004610061646, + -0.04655510187149048, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + 0.032517433166503906, + -0.02922782301902771, + -0.033792078495025635, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + -0.06465893983840942, + -0.026740312576293945, + 0.046701133251190186, + -0.06953387707471848, + 0.02321198582649231, + -1.8774652481079102, + 0.0 + ] + ], + "predicted_scores": [ + 7.032159805297852, + 6.676544189453125, + 6.1592559814453125, + 6.096284866333008, + 3.760338306427002, + 3.450634002685547, + 3.272002696990967, + 0.3568611443042755 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "dee046e5a3c9b5e78be27f1df1f4f235344ec818a1a7439afe5f4cb30139cf34", + "stored_base_utility": 0.1194220781326294, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.009124159812927246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009124159812927246 + }, + "base_progress": 0.009124159812927246, + "base_success": false, + "base_utility": 0.009124159812927246, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008222500793635845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008222500793635845 + }, + { + "contact_quality": null, + "progress": 0.0046975016593933105, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0046975016593933105 + }, + { + "contact_quality": null, + "progress": 0.0073589482344686985, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0073589482344686985 + }, + { + "contact_quality": null, + "progress": 0.0043724775314331055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0043724775314331055 + }, + { + "contact_quality": null, + "progress": 0.021352291107177734, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021352291107177734 + }, + { + "contact_quality": null, + "progress": 0.00515224551782012, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00515224551782012 + }, + { + "contact_quality": null, + "progress": 0.00400706147775054, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00400706147775054 + }, + { + "contact_quality": null, + "progress": 0.007424016948789358, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007424016948789358 + } + ], + "candidate_progress": [ + 0.008222500793635845, + 0.0046975016593933105, + 0.0073589482344686985, + 0.0043724775314331055, + 0.021352291107177734, + 0.00515224551782012, + 0.00400706147775054, + 0.007424016948789358 + ], + "candidate_source_chart_ids": [ + "ms-6758deb06644a14aa722", + "ms-6758deb06644a14aa722", + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-397e502908d16b7804b0", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-482c5c8d208b5907a2c2", + "generated_tangents": [ + [ + -0.017186760902404785, + 0.09956637024879456, + 0.20753830671310425, + -0.20426759123802185, + 1.5893971920013428, + -0.5114763975143433, + 0.11638316512107849, + 0.10756231844425201, + 0.1158156618475914, + 0.5714477300643921, + -0.904312252998352, + 1.3716343641281128, + 0.6861156225204468, + 0.3100910186767578, + -0.3616923689842224, + 0.09996774792671204, + 0.5461421608924866, + 0.715141773223877, + 0.2816997766494751, + 0.738487184047699, + -0.15864297747612 + ], + [ + 0.23175430297851562, + -0.17619842290878296, + 0.929126501083374, + -0.018297672271728516, + 1.4659910202026367, + -0.45366737246513367, + 0.301045686006546, + -0.08241283893585205, + 0.21580976247787476, + 0.5826863646507263, + -0.8205847144126892, + 1.3259111642837524, + 0.6864414215087891, + 0.6886419057846069, + -0.6587576866149902, + 0.08108201622962952, + 0.47818803787231445, + 0.7640861868858337, + 0.0036778151988983154, + 0.6387270092964172, + -0.2671339213848114 + ], + [ + -0.04621201753616333, + -0.005012210458517075, + 0.2569892406463623, + -0.2733756899833679, + 1.7086939811706543, + -0.38714930415153503, + 0.15246009826660156, + 0.09361942857503891, + 0.12655338644981384, + 0.5312187671661377, + -0.8678405284881592, + 1.317182183265686, + 0.7126821279525757, + 0.5091283917427063, + -0.472143292427063, + 0.17529332637786865, + 0.5172862410545349, + 0.7350562214851379, + 0.10154630243778229, + 0.7145820260047913, + 0.010804355144500732 + ], + [ + -0.3709810972213745, + -0.18172338604927063, + 0.4522005617618561, + -0.09949532151222229, + 1.7164051532745361, + 0.190280020236969, + -0.047384679317474365, + 0.06561936438083649, + 0.2878030240535736, + 0.7176128029823303, + -0.9203264713287354, + 1.4038488864898682, + 1.487691044807434, + 0.42478305101394653, + -0.5498186945915222, + 0.22853001952171326, + 0.904656171798706, + 0.65569669008255, + -0.36996662616729736, + 0.20872294902801514, + -0.3177125155925751 + ], + [ + 0.31387752294540405, + 0.20017869770526886, + 0.2086392045021057, + -0.32494959235191345, + 0.8306301236152649, + 0.37293708324432373, + 0.10171589255332947, + 0.06858208775520325, + 0.294421523809433, + 0.4255017936229706, + -1.0117416381835938, + 1.2504631280899048, + 0.6738172769546509, + 0.4711087942123413, + -0.5773358941078186, + 0.0011362014338374138, + 0.13909029960632324, + 0.8214422464370728, + 0.10278940200805664, + 0.5087759494781494, + -0.17116862535476685 + ], + [ + -0.32143181562423706, + -0.05319594591856003, + 0.39582347869873047, + -0.03692084550857544, + 1.5359551906585693, + 0.23442868888378143, + 0.028827577829360962, + 0.07867465913295746, + 0.2888440787792206, + 0.7149316072463989, + -0.7780476212501526, + 1.5521657466888428, + 1.118637204170227, + 0.5799349546432495, + -0.6084085702896118, + 0.24349932372570038, + 0.7352807521820068, + 0.8135769367218018, + -0.30793333053588867, + 0.23084181547164917, + -0.18994954228401184 + ], + [ + -0.2828400135040283, + -0.06221558153629303, + 0.38384413719177246, + -0.02375875413417816, + 1.6100142002105713, + 0.2748015224933624, + 0.18585950136184692, + 0.019401252269744873, + 0.2511083781719208, + 0.6879197955131531, + -0.8648056983947754, + 1.5771504640579224, + 1.1772615909576416, + 0.5997141599655151, + -0.6419427394866943, + 0.23534314334392548, + 0.7537817358970642, + 0.9064120054244995, + -0.3188903331756592, + 0.45716267824172974, + -0.2197788953781128 + ], + [ + -0.5289236307144165, + 0.050729669630527496, + 0.5570802092552185, + -0.1773999184370041, + 1.289454698562622, + 0.27887582778930664, + -0.08914440870285034, + 0.11143083870410919, + 0.5173144936561584, + 0.4815208911895752, + -1.0125471353530884, + 0.9513201117515564, + 1.7803000211715698, + 0.7554020285606384, + -0.7401578426361084, + 0.2986412048339844, + 0.5340263247489929, + 1.0491249561309814, + 0.06565281748771667, + -1.1585078239440918, + -0.2989015281200409 + ] + ], + "generated_utilities": [ + 0.008222500793635845, + 0.0046975016593933105, + 0.0073589482344686985, + 0.0043724775314331055, + 0.021352291107177734, + 0.00515224551782012, + 0.00400706147775054, + 0.007424016948789358 + ], + "group_id": "ms-482c5c8d208b5907a2c2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0359329991042614, + 0.1834145486354828, + 0.04271410033106804, + 0.03202331066131592, + 0.12089761346578598, + 0.03575414419174194, + 0.07773216813802719, + 0.1062050461769104, + 0.10849495977163315, + 0.08369352668523788, + 0.14167599380016327, + 0.008830646984279156, + 0.014355997554957867, + 0.02518286369740963, + 0.07873731851577759, + 0.015250584110617638 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.007817894220352173, + 0.01831631362438202, + 0.14601469039916992, + 0.10826835036277771, + 1.6591682434082031, + -0.005238652229309082, + 0.0, + -0.02753499150276184, + -0.009624123573303223, + 0.5762790441513062, + -0.3171614408493042, + 1.4374253749847412, + 0.7921857833862305, + 0.0, + -0.037753358483314514, + -0.025671005249023438, + 0.027724146842956543, + 0.4680846929550171, + -0.08415359258651733, + 0.033902376890182495, + 0.0 + ], + [ + 0.4543180465698242, + 0.20962828397750854, + 1.14601469039917, + -0.33832865953445435, + 2.659168243408203, + -0.7141264081001282, + 0.3707960247993469, + 0.21857939660549164, + 0.6796972155570984, + 1.5762790441513062, + -1.3171614408493042, + 2.437425374984741, + 1.7921857833862305, + 1.7551628351211548, + -0.19566796720027924, + 0.6842412948608398, + 0.11985036730766296, + 1.468084692955017, + 0.29799214005470276, + 0.3715361952781677, + 0.2624160647392273 + ], + [ + 0.2115812450647354, + -0.09577503800392151, + 0.7509078979492188, + -0.6239187121391296, + 2.4163239002227783, + -0.5409671664237976, + 0.16154196858406067, + 0.5615253448486328, + 1.0765573978424072, + 1.4198161363601685, + -1.430187463760376, + 2.5212466716766357, + 1.9029841423034668, + 1.5550287961959839, + -0.5099657773971558, + 1.0103299617767334, + -0.21028786897659302, + 1.4441221952438354, + 0.5605546236038208, + 0.6799722909927368, + 0.5554409027099609 + ], + [ + 0.46792101860046387, + 0.08014003932476044, + 1.5437835454940796, + 0.08789932727813721, + 2.3046298027038574, + -1.1965231895446777, + 0.8527317047119141, + 0.22510167956352234, + 1.0473172664642334, + 1.3720369338989258, + -1.2376489639282227, + 2.066061019897461, + 2.033992052078247, + 2.166865825653076, + -0.46199846267700195, + 1.1675353050231934, + -0.22657811641693115, + 1.7576167583465576, + 0.766140878200531, + 0.1856600046157837, + 0.008163660764694214 + ], + [ + 0.9431964755058289, + 0.6587716341018677, + 0.8381389379501343, + -0.8407833576202393, + 2.3666741847991943, + -1.05813729763031, + 0.1429125964641571, + 0.10640819370746613, + 1.0277029275894165, + 1.193630576133728, + -0.7695048451423645, + 2.9484994411468506, + 1.1866973638534546, + 1.5341145992279053, + -0.1225738674402237, + 0.9121334552764893, + 0.7116963863372803, + 0.8198207020759583, + 0.6562460660934448, + 0.1118304431438446, + -0.2320157289505005 + ] + ], + "predicted_scores": [ + 3.8373167514801025, + 3.3415451049804688, + 3.233798027038574, + 3.2228729724884033, + 3.134660005569458, + 2.501694917678833, + 2.3772544860839844, + -0.7967339754104614 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "903f78d5baa541d73612cac5e063bb553fba566c0d4adcb9603aa055d710fb15", + "stored_base_utility": 0.0359329991042614, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14663122594356537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14663122594356537 + }, + "base_progress": 0.14663122594356537, + "base_success": false, + "base_utility": 0.14663122594356537, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09237110614776611, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09237110614776611 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.07816246896982193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07816246896982193 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.054367441684007645, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054367441684007645 + }, + { + "contact_quality": null, + "progress": 0.08874932676553726, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08874932676553726 + }, + { + "contact_quality": null, + "progress": 0.0857056975364685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0857056975364685 + }, + { + "contact_quality": null, + "progress": 0.08813277631998062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08813277631998062 + } + ], + "candidate_progress": [ + 0.09237110614776611, + 1.0, + 0.07816246896982193, + 1.0, + 0.054367441684007645, + 0.08874932676553726, + 0.0857056975364685, + 0.08813277631998062 + ], + "candidate_source_chart_ids": [ + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + true, + false, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-4da03300d92420e80ed0", + "generated_tangents": [ + [ + 1.7368111610412598, + -0.21339505910873413, + -0.46265074610710144, + 0.06460139155387878, + -0.7958738207817078, + 1.979134202003479, + 1.6510772705078125, + 1.8144487142562866, + 0.31472790241241455, + -1.1421655416488647, + -0.3903200626373291, + -1.0712122917175293, + -2.1565475463867188, + -2.780268669128418, + 1.7021327018737793, + 0.03109576925635338, + -0.903856098651886, + -0.03948545455932617, + -0.08760222792625427, + -1.6022825241088867, + -1.7862193584442139 + ], + [ + 1.7032325267791748, + -0.17047926783561707, + -0.16288384795188904, + -0.07070240378379822, + -0.43538016080856323, + 2.7562496662139893, + 1.8079917430877686, + 1.817337155342102, + 0.33617568016052246, + -0.7495534420013428, + -0.22707799077033997, + -0.6103841066360474, + -2.676668167114258, + -1.630541443824768, + 1.8960447311401367, + 0.28720083832740784, + -0.6609141230583191, + 0.08464326709508896, + 0.18430621922016144, + -1.458815574645996, + -1.265209674835205 + ], + [ + 1.70179283618927, + -0.03528985381126404, + -0.4249545633792877, + 0.05410349369049072, + -0.7084158658981323, + 2.627464771270752, + 1.7215704917907715, + 1.9313647747039795, + 0.2992633283138275, + -0.851211667060852, + -0.294365257024765, + -1.0870718955993652, + -2.2758162021636963, + -2.242332696914673, + 1.7648634910583496, + 0.07617968320846558, + -0.6996333599090576, + 0.046212613582611084, + 0.009328782558441162, + -1.4902095794677734, + -1.5844112634658813 + ], + [ + 1.5555816888809204, + -0.049911241978406906, + -0.3536982834339142, + -0.07961110770702362, + -0.8340397477149963, + 2.6909291744232178, + 1.3236160278320312, + 1.910525918006897, + 0.2639979422092438, + -0.9818429946899414, + -0.2043403685092926, + -0.979907751083374, + -2.3934974670410156, + -1.9812959432601929, + 1.8485350608825684, + 0.1398877203464508, + -0.735511064529419, + 0.19586089253425598, + 0.05111268162727356, + -1.325878620147705, + -1.5703483819961548 + ], + [ + 1.69390869140625, + -0.047605711966753006, + -0.3597279489040375, + 0.019434362649917603, + -0.6562896966934204, + 2.535365343093872, + 1.896613359451294, + 1.8285937309265137, + 0.3076530992984772, + -0.7159000039100647, + -0.42773759365081787, + -0.8712296485900879, + -2.540210485458374, + -2.202735662460327, + 1.726596474647522, + 0.10543574392795563, + -0.5718258619308472, + 0.1633746176958084, + 0.06928750872612, + -1.283116340637207, + -1.3769266605377197 + ], + [ + 1.8496158123016357, + -0.044849198311567307, + -0.609849214553833, + 0.21328625082969666, + -0.6746882796287537, + 2.7659895420074463, + 1.4111237525939941, + 1.929183840751648, + 0.256539911031723, + -0.8107463121414185, + -0.2945350706577301, + -0.7436599135398865, + -2.236917495727539, + -1.543531060218811, + 1.7369017601013184, + 0.1844581514596939, + -0.7065438628196716, + 0.06091751158237457, + -0.16380739212036133, + -1.0527808666229248, + -1.074257731437683 + ], + [ + 1.8270145654678345, + -0.08395204693078995, + -0.3144706189632416, + 0.05772817134857178, + -0.39466285705566406, + 2.3055927753448486, + 1.9214544296264648, + 1.8403788805007935, + 0.3090434968471527, + -0.8092614412307739, + -0.42020419239997864, + -0.915605902671814, + -2.6384880542755127, + -2.0004937648773193, + 1.7308602333068848, + 0.08055980503559113, + -0.5612534284591675, + 0.15524426102638245, + -0.05449339747428894, + -1.3552777767181396, + -1.4205656051635742 + ], + [ + 2.286088228225708, + -0.033173397183418274, + 0.03115794062614441, + 0.14390608668327332, + -0.08776077628135681, + 1.8534377813339233, + 2.061161518096924, + 1.9325462579727173, + 0.1618713140487671, + -1.0712156295776367, + -0.5439500212669373, + -0.7075928449630737, + -2.4705185890197754, + -1.7517262697219849, + 1.651858925819397, + -0.13163228332996368, + -0.5243056416511536, + 0.37641942501068115, + -0.2565738558769226, + -1.6566424369812012, + -1.2345231771469116 + ] + ], + "generated_utilities": [ + 0.09237110614776611, + 2.0, + 0.07816246896982193, + 2.0, + 0.054367441684007645, + 0.08874932676553726, + 0.0857056975364685, + 0.08813277631998062 + ], + "group_id": "ms-4da03300d92420e80ed0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18184101581573486, + 0.1803790181875229, + 0.09926676750183105, + 0.20791122317314148, + 0.26338279247283936, + 0.2025860995054245, + 0.2575649917125702, + 0.23894403874874115, + 0.2174234390258789, + 2.0, + 0.1918075978755951, + 0.10365168750286102, + 2.0, + 2.0, + 2.0, + 0.15933696925640106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.1168947219848633, + 0.13667210936546326, + -0.051466986536979675, + 0.03360617160797119, + -0.19173425436019897, + 1.7646396160125732, + 0.0, + 0.9886866807937622, + 0.19114632904529572, + -0.3927978277206421, + -0.21108940243721008, + -0.031822919845581055, + -1.6683034896850586, + 0.0, + 0.9908627271652222, + -0.05301591753959656, + 0.08875280618667603, + 0.04224884510040283, + -0.03718748688697815, + -0.06952071189880371, + 0.0 + ], + [ + 2.1713812351226807, + -0.41443270444869995, + -0.23275701701641083, + 0.6524178981781006, + -0.25828492641448975, + 3.331055164337158, + 2.47529673576355, + 1.4984867572784424, + -0.396874338388443, + -0.725897490978241, + 0.1870076060295105, + -0.4014764428138733, + -2.239036798477173, + -2.218061923980713, + 1.6540865898132324, + 0.013689309358596802, + -0.22782719135284424, + 1.2475574016571045, + 0.7231295704841614, + -0.281164288520813, + -1.4593302011489868 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 2.1168947219848633, + 0.045049574226140976, + -0.25911271572113037, + 0.6112266778945923, + -0.5530642867088318, + 2.7646396160125732, + 1.9622430801391602, + 1.9886866807937622, + 0.28067120909690857, + -1.392797827720642, + -0.5793421268463135, + -1.031822919845581, + -2.6683034896850586, + -2.812993049621582, + 1.9908627271652222, + -0.28380459547042847, + -0.7305920720100403, + 0.7978808879852295, + 0.1574108749628067, + -1.0695207118988037, + -0.9408642053604126 + ], + [ + 2.228377103805542, + -0.19656506180763245, + -0.23844876885414124, + 0.6676552295684814, + -0.7639804482460022, + 2.937652111053467, + 1.882341980934143, + 1.8207013607025146, + 0.27702635526657104, + -1.549763798713684, + -0.5955427289009094, + -1.185685157775879, + -2.8421716690063477, + -3.054234266281128, + 1.691805362701416, + -0.37051287293434143, + -0.43653741478919983, + 0.514695405960083, + -0.0132378488779068, + -0.899186372756958, + -0.8349848985671997 + ], + [ + 2.425011157989502, + -0.03216857090592384, + -0.010275468230247498, + 0.4265761971473694, + -0.7737585306167603, + 3.1123812198638916, + 1.5686779022216797, + 2.112421989440918, + 0.3475198745727539, + -1.3452436923980713, + -0.6067132949829102, + -1.1150503158569336, + -2.461210250854492, + -2.8643558025360107, + 2.185579299926758, + -0.6223857402801514, + -0.8183724880218506, + 0.8074819445610046, + 0.536148726940155, + -0.7826522588729858, + -0.6682058572769165 + ], + [ + 1.6831411123275757, + -0.19312052428722382, + 0.010244876146316528, + 1.1032377481460571, + -0.4994945824146271, + 2.9000508785247803, + 2.187715768814087, + 2.200220823287964, + 0.4744641184806824, + -1.6715843677520752, + -0.493664413690567, + -1.459894061088562, + -2.90458345413208, + -2.3254852294921875, + 1.4726771116256714, + -0.21338894963264465, + -1.293912649154663, + 1.3146823644638062, + -0.09375743567943573, + -0.5040683150291443, + -0.9596792459487915 + ], + [ + 1.6738543510437012, + 0.6350066661834717, + 0.19669359922409058, + -0.1396545171737671, + -0.9625160694122314, + 2.207883596420288, + 1.7678128480911255, + 1.319273829460144, + -0.267609566450119, + -0.5747214555740356, + 0.19376236200332642, + -1.4490230083465576, + -2.6397883892059326, + -3.2567262649536133, + 1.2646806240081787, + -0.14523190259933472, + -0.050948500633239746, + 0.6943628787994385, + -0.1503342241048813, + -0.7313232421875, + -1.7930638790130615 + ], + [ + 2.139432668685913, + 0.34125545620918274, + 0.6370852589607239, + 0.8591548204421997, + 0.14890235662460327, + 2.6835381984710693, + 2.324920892715454, + 2.0091233253479004, + 0.13480046391487122, + -0.47407835721969604, + -1.102479100227356, + -1.0710523128509521, + -2.3226847648620605, + -3.6534740924835205, + 1.8767409324645996, + -0.01776304841041565, + -1.167153000831604, + 0.693484902381897, + 0.563054084777832, + -0.17544519901275635, + -0.8315125703811646 + ], + [ + 1.1254388093948364, + -0.03027276322245598, + -0.8753920793533325, + 0.3578681945800781, + -0.4256642460823059, + 1.7686536312103271, + 2.253190517425537, + 2.8274970054626465, + 0.3938461244106293, + -1.4084759950637817, + -0.7252857685089111, + -0.07288885116577148, + -1.7749395370483398, + -2.1030378341674805, + 1.92497980594635, + 0.20398634672164917, + -0.03521168231964111, + 1.4702520370483398, + -0.36331063508987427, + -1.5291211605072021, + -0.8369957804679871 + ] + ], + "predicted_scores": [ + 5.142221450805664, + 5.011483669281006, + 4.925361156463623, + 4.686418056488037, + 4.659388542175293, + 4.544834136962891, + 4.415966510772705, + 3.512410879135132 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1747178030", + "selected_index": 0, + "state_hash": "87c74ddf97c53ddeeaae98467b4a13b1e0e499bd94eddb617759d39164cab542", + "stored_base_utility": 0.18184101581573486, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.009015560150146484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009015560150146484 + }, + { + "contact_quality": null, + "progress": 0.01863894797861576, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01863894797861576 + }, + { + "contact_quality": null, + "progress": 0.010752241127192974, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010752241127192974 + }, + { + "contact_quality": null, + "progress": 0.007264256477355957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007264256477355957 + }, + { + "contact_quality": null, + "progress": 0.007598539348691702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007598539348691702 + }, + { + "contact_quality": null, + "progress": 0.006039460655301809, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006039460655301809 + }, + { + "contact_quality": null, + "progress": 0.0047194757498800755, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0047194757498800755 + }, + { + "contact_quality": null, + "progress": 0.004894236568361521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004894236568361521 + } + ], + "candidate_progress": [ + 0.009015560150146484, + 0.01863894797861576, + 0.010752241127192974, + 0.007264256477355957, + 0.007598539348691702, + 0.006039460655301809, + 0.0047194757498800755, + 0.004894236568361521 + ], + "candidate_source_chart_ids": [ + "ms-e84163d9ef2e7014daca", + "ms-6a3add21affa4796f9be", + "ms-f85a653af630821f2572", + "ms-dc26b448cce4164a23e6", + "ms-e84163d9ef2e7014daca", + "ms-7eaea428dfdef7255a66", + "ms-0e592951fa32139b6dc9", + "ms-6a3add21affa4796f9be" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-57be9ae181c077e7821f", + "generated_tangents": [ + [ + -0.673485279083252, + -0.11057630181312561, + 1.162347435951233, + -0.4004848003387451, + 0.401690810918808, + 2.4316258430480957, + 1.357682704925537, + 0.14826972782611847, + -0.14617441594600677, + 0.9371858835220337, + -0.2728007435798645, + 0.8436618447303772, + 1.2050950527191162, + 1.2057795524597168, + 0.9029747247695923, + -0.2893470525741577, + -0.2767745852470398, + 0.2502163052558899, + 0.37158024311065674, + 1.3908026218414307, + -0.16359004378318787 + ], + [ + -0.60336834192276, + -0.17724356055259705, + 1.0772401094436646, + -0.3030449151992798, + 0.5300936698913574, + 2.089449882507324, + 0.9355648756027222, + -0.14148828387260437, + -0.02768166922032833, + 0.5793574452400208, + -0.39049065113067627, + 0.28496310114860535, + 0.9720772504806519, + 1.229799509048462, + 0.7502161860466003, + -0.16310334205627441, + -0.2300533801317215, + 0.15497493743896484, + 0.37238848209381104, + 1.4026505947113037, + -0.38596704602241516 + ], + [ + -0.5618137717247009, + -0.02821844071149826, + 1.1908972263336182, + -0.3832213878631592, + 0.8535086512565613, + 2.410517454147339, + 1.0895622968673706, + 0.1309642791748047, + -0.04606577008962631, + 0.7881467342376709, + -0.10882357507944107, + 0.5313479900360107, + 1.2998323440551758, + 1.5107380151748657, + 0.7132909297943115, + -0.2532687187194824, + -0.0633615106344223, + -0.10792712867259979, + 0.5407611131668091, + 1.3837144374847412, + 0.0009638965129852295 + ], + [ + -0.583907961845398, + -0.08239202201366425, + 1.225914716720581, + -0.514095664024353, + 0.5955853462219238, + 2.2436342239379883, + 1.195970058441162, + 0.15324531495571136, + -0.11245833337306976, + 0.8402239084243774, + -0.32664012908935547, + 0.7294513583183289, + 1.5859030485153198, + 0.7674247622489929, + 0.5557411313056946, + -0.4299098253250122, + 0.018211573362350464, + 0.234456866979599, + 0.24528399109840393, + 1.4658477306365967, + -0.04363994300365448 + ], + [ + -0.7100796103477478, + -0.10140429437160492, + 1.123480200767517, + -0.5277652740478516, + 0.6968957185745239, + 2.523554563522339, + 1.3010798692703247, + 0.10423418879508972, + -0.09796861559152603, + 0.9269856214523315, + -0.2893550992012024, + 0.8262448310852051, + 1.342462182044983, + 0.9518347978591919, + 0.7416843771934509, + -0.3507519066333771, + -0.06938178837299347, + 0.23946513235569, + 0.33395281434059143, + 1.284440040588379, + -0.1417977213859558 + ], + [ + -0.5256441831588745, + -0.08357777446508408, + 1.140179991722107, + -0.29000991582870483, + 0.25538700819015503, + 2.684236526489258, + 1.121407151222229, + -0.00708770751953125, + 0.023272830992937088, + 1.146262764930725, + -0.20857489109039307, + 0.5451996326446533, + 1.8633259534835815, + 0.9819211959838867, + 0.564016580581665, + -0.35496509075164795, + 0.1230098307132721, + 0.48420006036758423, + 0.3613831698894501, + 1.1079778671264648, + 0.261741042137146 + ], + [ + -0.7110994458198547, + -0.22062639892101288, + 1.2708433866500854, + -0.4487105906009674, + 1.060157299041748, + 2.3540022373199463, + 1.404695987701416, + 0.01504121720790863, + -0.03042696975171566, + 1.1687805652618408, + -0.1746625006198883, + 0.856299877166748, + 1.4687488079071045, + 2.0115089416503906, + 0.41887542605400085, + -0.12903989851474762, + -0.04874922335147858, + 0.10019809007644653, + 0.2882075607776642, + 1.2084572315216064, + 0.4652540683746338 + ], + [ + -0.7296510934829712, + -0.21878530085086823, + 1.2190682888031006, + -0.37322354316711426, + 0.9713817834854126, + 1.8391281366348267, + 1.2307020425796509, + 0.026589691638946533, + -0.03206340968608856, + 1.2204389572143555, + -0.10162902623414993, + 0.7409164905548096, + 1.6027806997299194, + 1.6609978675842285, + 0.4686562716960907, + -0.07023249566555023, + -0.049426332116127014, + 0.08257109671831131, + 0.5998162031173706, + 1.7177071571350098, + 0.5814725160598755 + ] + ], + "generated_utilities": [ + 0.009015560150146484, + 0.01863894797861576, + 0.010752241127192974, + 0.007264256477355957, + 0.007598539348691702, + 0.006039460655301809, + 0.0047194757498800755, + 0.004894236568361521 + ], + "group_id": "ms-57be9ae181c077e7821f", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.28031715750694275, + 0.19804009795188904, + 0.15024888515472412, + 0.23104076087474823, + 0.21580509841442108, + 0.013450047001242638, + 0.20318475365638733, + 0.22562944889068604, + 0.14178292453289032, + 0.15763869881629944, + 0.27735981345176697, + 0.1909358948469162, + 0.06666654348373413, + 2.0, + 0.060366373509168625, + 0.05085722729563713 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.06739383935928345, + 0.06319573521614075, + 0.8779754638671875, + -0.03682136535644531, + -0.07093041390180588, + 1.3382306098937988, + 0.0, + 0.07168813049793243, + 0.01971740648150444, + 0.4866671562194824, + 0.07402461767196655, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + 0.03574584424495697, + 0.021435141563415527, + 0.0021790266036987305, + -0.06565424799919128, + 1.1393508911132812, + 0.0 + ], + [ + -0.17397063970565796, + -0.08753994852304459, + 0.8779754638671875, + -0.09028738737106323, + -0.17196941375732422, + 1.3382306098937988, + 0.0, + 0.022144019603729248, + 0.1154828891158104, + 0.4866671562194824, + -0.09212043881416321, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + -0.2021142691373825, + -0.022606968879699707, + 0.0021790266036987305, + 0.03272634744644165, + 1.1393508911132812, + 0.0 + ], + [ + -1.0144902467727661, + -0.023572498932480812, + 1.8779754638671875, + -1.0368213653564453, + -0.03799626976251602, + 2.338230609893799, + 2.239948272705078, + 0.16168542206287384, + 0.07221447676420212, + 1.4866671562194824, + -0.3557073175907135, + 1.3622210025787354, + 1.896626591682434, + 2.0382778644561768, + 1.4886415004730225, + 0.1997920423746109, + -0.4386618137359619, + 1.0021790266036987, + -0.3174663782119751, + 2.1393508911132812, + 1.9118322134017944 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.479896545410156, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0765557289123535, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.823664426803589 + ], + [ + -1.0723698139190674, + -0.05779706686735153, + 2.0670831203460693, + -0.830497682094574, + -0.17302533984184265, + 2.2277159690856934, + 2.0435242652893066, + 0.08191673457622528, + 0.08711377531290054, + 1.416495680809021, + -0.5633984804153442, + 1.3216067552566528, + 1.8660295009613037, + 2.2198705673217773, + 1.6887608766555786, + 0.06658947467803955, + -0.4010601043701172, + 1.1803714036941528, + -0.15144284069538116, + 2.308220386505127, + 1.8460347652435303 + ], + [ + -0.8901733756065369, + 0.3378032147884369, + 2.2360360622406006, + -1.207388162612915, + -0.34048303961753845, + 2.460798978805542, + 2.100795030593872, + -0.1847335249185562, + -0.06362440437078476, + 1.2987924814224243, + -0.11972835659980774, + 1.170393943786621, + 2.0620977878570557, + 2.4285032749176025, + 1.3666276931762695, + -0.01825574040412903, + -0.4620283842086792, + 1.1138218641281128, + -0.5787107944488525, + 1.78761887550354, + 1.740113615989685 + ], + [ + -1.1796774864196777, + -0.12369386106729507, + 1.546891689300537, + -1.0500564575195312, + -0.026454759761691093, + 2.177122116088867, + 2.0003931522369385, + 0.13697585463523865, + 0.37715619802474976, + 1.9734747409820557, + -0.16338464617729187, + 1.3775254487991333, + 2.3054146766662598, + 1.5546050071716309, + 1.2645983695983887, + 0.3812609910964966, + 0.02191823720932007, + 1.0944219827651978, + 0.09844094514846802, + 1.8145865201950073, + 2.0624771118164062 + ], + [ + -0.5514534711837769, + -0.4441075921058655, + 1.8971614837646484, + -1.6112515926361084, + 0.1306135058403015, + 2.736365556716919, + 2.270737886428833, + -0.35948747396469116, + 0.03968408331274986, + 1.061530351638794, + -0.29366230964660645, + 1.5066968202590942, + 2.34443998336792, + 2.3445637226104736, + 1.2286460399627686, + 0.13919776678085327, + -0.21014492213726044, + 0.9723342061042786, + -0.5822539329528809, + 1.8932968378067017, + 2.159864664077759 + ], + [ + -0.9470903873443604, + -0.7433359622955322, + 1.798203945159912, + -0.5850675106048584, + -0.7611241340637207, + 2.2761964797973633, + 2.0114798545837402, + 0.7724847793579102, + -0.23453289270401, + 0.7384847402572632, + -1.022956132888794, + 1.0059813261032104, + 1.652310848236084, + 2.6376113891601562, + 2.148331642150879, + 0.9844839572906494, + 0.18531352281570435, + 1.6280735731124878, + -0.06812705099582672, + 2.2684829235076904, + 1.1641287803649902 + ], + [ + -1.729402780532837, + 0.15382106602191925, + 1.0736894607543945, + -0.5141006708145142, + -0.6476962566375732, + 2.0120980739593506, + 2.8703343868255615, + 0.5578619837760925, + -0.8154823184013367, + 1.4325131177902222, + -1.0468122959136963, + 0.9670607447624207, + 1.973939061164856, + 2.7364890575408936, + 1.8303502798080444, + 0.8089333772659302, + -0.41138362884521484, + 1.8146085739135742, + -0.11676108837127686, + 2.3541221618652344, + 2.3875224590301514 + ], + [ + -1.8409323692321777, + -0.2118261456489563, + 1.5354409217834473, + -0.3873547911643982, + 0.7043789029121399, + 2.2068612575531006, + 2.5864486694335938, + 0.8632195591926575, + -0.10360705107450485, + 1.6340630054473877, + -0.017105937004089355, + 1.90720796585083, + 2.608898639678955, + 2.3387808799743652, + 2.3744726181030273, + 0.13692857325077057, + -0.7716406583786011, + 1.3477450609207153, + -0.9811362028121948, + 2.15514874458313, + 2.298004150390625 + ], + [ + -1.2484530210494995, + 0.5977066159248352, + 2.8256869316101074, + -0.3418053388595581, + -1.031638503074646, + 1.7023751735687256, + 1.4487309455871582, + -0.3013373017311096, + -0.3682725131511688, + 2.4404053688049316, + -0.8263542652130127, + 2.1105899810791016, + 2.5769248008728027, + 1.6751525402069092, + 0.8367876410484314, + -0.1554209142923355, + 0.17985862493515015, + 1.0017495155334473, + -0.3176954984664917, + 2.6737403869628906, + 1.7722500562667847 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -1.0015538930892944, + 0.43126925826072693, + 1.2786648273468018, + -0.7182899713516235, + 0.06109866499900818, + 2.323270797729492, + 1.3882524967193604, + -0.2783380150794983, + 0.35085979104042053, + 0.5870303511619568, + -1.0134913921356201, + 0.4675567150115967, + 0.9185858964920044, + 1.9441449642181396, + 1.3693488836288452, + 0.05759717524051666, + 0.19058758020401, + 1.9360744953155518, + -1.0018632411956787, + 2.043489456176758, + 1.1226990222930908 + ] + ], + "predicted_scores": [ + 6.7034149169921875, + 6.5890398025512695, + 6.502756118774414, + 6.359658241271973, + 6.356502056121826, + 5.557772636413574, + 4.942564964294434, + 4.4306511878967285 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "9c4de49d50d74c8c571620759fff1d43be23093a5f66e946f7ae6d75cb26ade5", + "stored_base_utility": 0.28031715750694275, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1544450968503952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1544450968503952 + }, + "base_progress": 0.1544450968503952, + "base_success": false, + "base_utility": 0.1544450968503952, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07292797416448593, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07292797416448593 + }, + { + "contact_quality": null, + "progress": 0.06293817609548569, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06293817609548569 + }, + { + "contact_quality": null, + "progress": 0.07210147380828857, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07210147380828857 + }, + { + "contact_quality": null, + "progress": 0.057973701506853104, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057973701506853104 + }, + { + "contact_quality": null, + "progress": 0.05558790639042854, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05558790639042854 + }, + { + "contact_quality": null, + "progress": 0.07254187017679214, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07254187017679214 + }, + { + "contact_quality": null, + "progress": 0.07161635160446167, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07161635160446167 + }, + { + "contact_quality": null, + "progress": 0.061803679913282394, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.061803679913282394 + } + ], + "candidate_progress": [ + 0.07292797416448593, + 0.06293817609548569, + 0.07210147380828857, + 0.057973701506853104, + 0.05558790639042854, + 0.07254187017679214, + 0.07161635160446167, + 0.061803679913282394 + ], + "candidate_source_chart_ids": [ + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-ff129e39095c2861b522" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6388b5d2356c556c3e55", + "generated_tangents": [ + [ + -0.3968668580055237, + -0.38062024116516113, + 1.7399386167526245, + -0.2414710968732834, + 1.1255995035171509, + 3.039266347885132, + 3.546168088912964, + 2.1424098014831543, + -0.005538465455174446, + -0.07015720754861832, + 1.9714783430099487, + -0.36543887853622437, + 1.792925477027893, + 2.405517339706421, + -0.2426699697971344, + 0.47847115993499756, + 0.714500367641449, + 0.3246252238750458, + 1.9411358833312988, + 1.951286792755127, + 2.6931686401367188 + ], + [ + -0.4288380742073059, + -0.1759163737297058, + 1.9252293109893799, + -0.2241262048482895, + 0.9497015476226807, + 2.679028034210205, + 3.121192216873169, + 2.2287421226501465, + -0.10378557443618774, + -0.2547786235809326, + 1.5193686485290527, + 0.1322452425956726, + 1.9332810640335083, + 2.3389811515808105, + 0.043793801218271255, + 0.525780200958252, + 0.974789559841156, + 0.10781662911176682, + 1.7040464878082275, + 2.568972587585449, + 2.3547987937927246 + ], + [ + -0.6143622398376465, + -0.20890432596206665, + 1.8667266368865967, + -0.4120851755142212, + 1.2120611667633057, + 2.5719242095947266, + 3.503678798675537, + 2.175184488296509, + -0.025717923417687416, + -0.1382434070110321, + 1.5058507919311523, + 0.19175328314304352, + 2.1061060428619385, + 2.4802355766296387, + 0.09369316697120667, + 0.5243374109268188, + 0.6584591269493103, + 0.10176264494657516, + 1.612694263458252, + 2.428436279296875, + 2.6008424758911133 + ], + [ + -0.5036369562149048, + -0.18751846253871918, + 2.042149305343628, + -0.4116286039352417, + 1.0883207321166992, + 2.580353021621704, + 3.3304309844970703, + 2.2497329711914062, + -0.08341985195875168, + -0.08701028674840927, + 1.4502122402191162, + 0.2606508731842041, + 2.2266836166381836, + 2.2686614990234375, + 0.1270129680633545, + 0.5080239772796631, + 0.7286629676818848, + 0.08464661985635757, + 1.6795477867126465, + 2.7060940265655518, + 2.65683913230896 + ], + [ + -0.4743857979774475, + -0.2309812754392624, + 2.2059569358825684, + -0.2929331064224243, + 1.1745760440826416, + 2.7660915851593018, + 3.2718822956085205, + 2.2960586547851562, + -0.1819061040878296, + -0.16178375482559204, + 1.6534020900726318, + 0.2426091581583023, + 2.1609795093536377, + 2.7593913078308105, + 0.008061695843935013, + 0.8117608428001404, + 0.8180800676345825, + 0.15561014413833618, + 1.6041498184204102, + 2.6488308906555176, + 3.138012170791626 + ], + [ + -0.23429040610790253, + -0.30268633365631104, + 1.6476975679397583, + -0.2855158746242523, + 1.16886305809021, + 2.704505205154419, + 3.3356595039367676, + 2.3955564498901367, + -0.029977740719914436, + -0.06803838163614273, + 1.5985625982284546, + -0.014424756169319153, + 1.702675700187683, + 2.734076499938965, + 0.018823731690645218, + 0.8042119145393372, + 0.4351709485054016, + 0.27187758684158325, + 1.7853443622589111, + 1.9134089946746826, + 2.8341336250305176 + ], + [ + -0.6073477864265442, + -0.18093299865722656, + 2.125244379043579, + -0.2429467737674713, + 1.350313425064087, + 3.0189356803894043, + 3.2697079181671143, + 2.1868600845336914, + -0.0840703621506691, + -0.37311649322509766, + 1.5795865058898926, + 0.45110803842544556, + 2.3109750747680664, + 2.7224833965301514, + 0.04407507926225662, + 0.7730731964111328, + 0.8389228582382202, + 0.0641772449016571, + 1.5328953266143799, + 2.33486270904541, + 3.0427112579345703 + ], + [ + -0.5567232966423035, + -0.2194473296403885, + 2.038078546524048, + -0.39234840869903564, + 1.2892754077911377, + 2.6152706146240234, + 3.3098623752593994, + 2.311594247817993, + -0.19186042249202728, + -0.34591808915138245, + 1.69652259349823, + 0.16723546385765076, + 2.188263416290283, + 3.0482404232025146, + 0.06031256169080734, + 0.7795422673225403, + 0.8894026875495911, + 0.07335814833641052, + 1.4641940593719482, + 2.4214982986450195, + 3.0559797286987305 + ] + ], + "generated_utilities": [ + 0.07292797416448593, + 0.06293817609548569, + 0.07210147380828857, + 0.057973701506853104, + 0.05558790639042854, + 0.07254187017679214, + 0.07161635160446167, + 0.061803679913282394 + ], + "group_id": "ms-6388b5d2356c556c3e55", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08186312764883041, + 0.11667267978191376, + 0.09751813113689423, + 2.0, + 0.27431026101112366, + 0.11760994046926498, + 0.20386962592601776, + 0.19832774996757507, + 0.22735612094402313, + 0.1356993168592453, + 0.1713169515132904, + 0.1792232096195221, + 0.06848804652690887, + 0.1094626635313034, + 0.11555596441030502, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0018978863954544067, + -0.4448504149913788, + 0.7108399868011475, + -0.33157044649124146, + -0.062200844287872314, + 1.4758033752441406, + 0.0, + 1.4153923988342285, + 0.14116239547729492, + 0.00777280330657959, + 0.5541017055511475, + 0.07029426097869873, + 0.26837193965911865, + 0.0, + -0.015542566776275635, + 0.3573532998561859, + 0.6168417930603027, + 0.3612897992134094, + 0.369077205657959, + 1.5924487113952637, + 0.0 + ], + [ + -0.09549225866794586, + -0.20593886077404022, + 1.7108399868011475, + -0.1554051786661148, + 0.9377991557121277, + 2.4758033752441406, + 2.4156031608581543, + 2.4153923988342285, + -0.2459496259689331, + -0.9922271966934204, + 1.5541017055511475, + 0.4461512267589569, + 1.2501436471939087, + 1.7566912174224854, + -0.8014360666275024, + 0.04551727697253227, + 1.6168417930603027, + 0.5393297672271729, + 1.369077205657959, + 2.5924487113952637, + 3.1126015186309814 + ], + [ + -0.11610143631696701, + -0.3961976170539856, + 1.9081053733825684, + -0.14575888216495514, + 1.1381605863571167, + 2.7434439659118652, + 2.5136890411376953, + 2.6401467323303223, + -0.45453745126724243, + -0.9589858651161194, + 1.4083189964294434, + 0.5800129175186157, + 0.9940754175186157, + 1.7450767755508423, + -1.0145976543426514, + 0.3312625586986542, + 1.4729199409484863, + 0.2520747482776642, + 1.3623151779174805, + 2.5670394897460938, + 3.3589630126953125 + ], + [ + -0.2318952977657318, + 0.024853751063346863, + 1.829875111579895, + 0.09877045452594757, + 1.1515556573867798, + 2.2239503860473633, + 2.153250217437744, + 2.8057901859283447, + -0.1800437569618225, + -1.2225371599197388, + 1.421208143234253, + 0.6627710461616516, + 1.3895055055618286, + 1.856442928314209, + -0.5928603410720825, + 0.4364123046398163, + 1.7852745056152344, + 0.3345714807510376, + 1.1167365312576294, + 2.7676005363464355, + 3.041402578353882 + ], + [ + -0.2227625697851181, + -0.012646213173866272, + 1.7988159656524658, + 0.21457992494106293, + 0.8877373933792114, + 2.9257700443267822, + 2.2101094722747803, + 2.2874886989593506, + -0.09066365659236908, + -0.49909964203834534, + 2.042389392852783, + 0.6287553906440735, + 1.074569582939148, + 1.7805378437042236, + -1.1840007305145264, + 0.3728531002998352, + 1.165173053741455, + 0.3054528832435608, + 1.4924153089523315, + 2.5540966987609863, + 2.664964199066162 + ], + [ + -0.6864494681358337, + 0.119636669754982, + 1.2460956573486328, + -0.6367122530937195, + 1.4080986976623535, + 2.5262017250061035, + 2.508194923400879, + 2.1323020458221436, + -0.43302130699157715, + -1.2678899765014648, + 1.0870052576065063, + 0.46296897530555725, + 0.765267550945282, + 2.3093044757843018, + -0.46595194935798645, + -0.43870145082473755, + 2.028986930847168, + 0.26284897327423096, + 1.5542359352111816, + 2.530392646789551, + 2.7435970306396484 + ], + [ + 0.18175412714481354, + -0.478390634059906, + 1.286731481552124, + -0.8066962361335754, + 1.4787592887878418, + 2.1033878326416016, + 1.750453233718872, + 2.047861099243164, + 0.3163147568702698, + -0.30588728189468384, + 1.0766479969024658, + 0.5822455286979675, + 1.673675775527954, + 2.039386510848999, + -0.62891685962677, + 0.615638792514801, + 2.018416404724121, + 0.2530737817287445, + 1.34544837474823, + 3.2287912368774414, + 2.9459850788116455 + ], + [ + 0.5059412121772766, + 0.17952178418636322, + 1.3319568634033203, + -0.8850483894348145, + 0.6336387395858765, + 1.775376796722412, + 2.3280270099639893, + 1.7305657863616943, + -0.02617873251438141, + -0.7963138818740845, + 1.9142115116119385, + 0.6635866165161133, + 1.3493314981460571, + 1.9258644580841064, + -0.3795176148414612, + 0.7861210107803345, + 1.0032029151916504, + 0.8491684794425964, + 0.5918128490447998, + 2.399027109146118, + 2.3932981491088867 + ], + [ + -0.34448128938674927, + -0.2996852695941925, + 0.9233196377754211, + -0.22890916466712952, + 1.6932814121246338, + 3.465762138366699, + 2.2005038261413574, + 2.6472361087799072, + -0.8981531858444214, + -0.2658765912055969, + 1.8755900859832764, + 0.2962568402290344, + 1.8239326477050781, + 1.0176585912704468, + -0.7912051677703857, + 0.6727061867713928, + 2.199978828430176, + 1.015731692314148, + 0.43087130784988403, + 3.5876548290252686, + 3.893789052963257 + ] + ], + "predicted_scores": [ + 8.308979988098145, + 7.565439701080322, + 7.398580551147461, + 7.091735363006592, + 6.928080081939697, + 6.910342693328857, + 6.837721347808838, + 6.50447416305542 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "38d539e46140304e3f157cbc207aa0d9489e5470633c216a0379906af445a1ec", + "stored_base_utility": 0.08186312764883041, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06173701956868172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06173701956868172 + }, + "base_progress": 0.06173701956868172, + "base_success": false, + "base_utility": 0.06173701956868172, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.057138841599226, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057138841599226 + }, + { + "contact_quality": null, + "progress": 0.06841854006052017, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06841854006052017 + }, + { + "contact_quality": null, + "progress": 0.015207032673060894, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015207032673060894 + }, + { + "contact_quality": null, + "progress": 0.016131063923239708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016131063923239708 + }, + { + "contact_quality": null, + "progress": 0.08538073301315308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08538073301315308 + }, + { + "contact_quality": null, + "progress": 0.0494011826813221, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0494011826813221 + }, + { + "contact_quality": null, + "progress": 0.02298945188522339, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02298945188522339 + }, + { + "contact_quality": null, + "progress": 0.018225232139229774, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018225232139229774 + } + ], + "candidate_progress": [ + 0.057138841599226, + 0.06841854006052017, + 0.015207032673060894, + 0.016131063923239708, + 0.08538073301315308, + 0.0494011826813221, + 0.02298945188522339, + 0.018225232139229774 + ], + "candidate_source_chart_ids": [ + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-7e88428497a6b13936c4", + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-a2518826fa0a816cc46f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-638d2ba1917eff4d09e0", + "generated_tangents": [ + [ + 0.3360844850540161, + -0.5631576180458069, + 0.42826926708221436, + -0.05306026339530945, + 1.9138963222503662, + 0.6474291086196899, + 0.6063404083251953, + 0.3220643401145935, + 0.11976703256368637, + 0.2562875747680664, + 0.060126449912786484, + 0.5298478603363037, + -0.12947794795036316, + 0.5538510084152222, + 0.46762537956237793, + -0.05959991738200188, + -0.09273485839366913, + -0.21236087381839752, + 1.016845703125, + -1.1638799905776978, + -0.523855447769165 + ], + [ + 0.23037973046302795, + -0.527307391166687, + 0.47113341093063354, + -0.14062219858169556, + 1.9487664699554443, + 0.7238055467605591, + 0.21989122033119202, + 0.2621961236000061, + 0.03199471905827522, + 0.12914234399795532, + 0.31201422214508057, + 0.898644745349884, + 0.1890799105167389, + 0.3102106750011444, + 0.4614345133304596, + -0.21547609567642212, + -0.1286732405424118, + -0.11163227260112762, + 1.0411545038223267, + -1.0815834999084473, + -0.31273379921913147 + ], + [ + -0.05227935314178467, + -0.5841974020004272, + 0.8214445114135742, + -0.005569174885749817, + 1.8276751041412354, + 0.35953664779663086, + -0.018803954124450684, + -0.10537424683570862, + -0.006136782467365265, + 0.08871615678071976, + 0.25584447383880615, + 0.8758326768875122, + 0.09765169024467468, + 0.3560537099838257, + 0.45599764585494995, + -0.355105996131897, + -0.0563969761133194, + 0.12093324214220047, + 0.5417991876602173, + -1.1594804525375366, + -0.038095176219940186 + ], + [ + -0.05506591498851776, + -0.5924087762832642, + 0.6706127524375916, + 0.5257781744003296, + 1.266045331954956, + 0.20478864014148712, + 1.0375064611434937, + 0.10195954889059067, + 0.12375635653734207, + 0.2946399748325348, + 0.10719247907400131, + 0.6814927458763123, + 0.3802690804004669, + 0.8426738977432251, + 0.2701503336429596, + -0.2513330280780792, + -0.04093451797962189, + -0.1308555155992508, + 0.9174430966377258, + -0.7879283428192139, + -0.527397871017456 + ], + [ + 0.3089306950569153, + -0.6023566126823425, + 0.49269524216651917, + -0.13358056545257568, + 1.939436674118042, + 1.0642074346542358, + 0.5651950836181641, + 0.31345751881599426, + 0.03243149816989899, + 0.09227952361106873, + 0.007948838174343109, + 0.4876777231693268, + -0.012213468551635742, + 0.5890823006629944, + 0.5992427468299866, + -0.08782917261123657, + -0.2024044245481491, + -0.30906999111175537, + 1.0151417255401611, + -1.3159689903259277, + -0.5578432083129883 + ], + [ + 0.08377283066511154, + -0.6841627359390259, + 0.7012265920639038, + 0.1769711971282959, + 1.3453123569488525, + 0.32215750217437744, + 0.06348186731338501, + 0.1281505525112152, + 0.08698300272226334, + 0.2377529740333557, + 0.15218651294708252, + 0.6287205219268799, + 0.23558709025382996, + 0.38711631298065186, + 0.34391283988952637, + -0.2935248017311096, + -0.36701059341430664, + 0.19629979133605957, + 0.7564046382904053, + -1.1271963119506836, + -0.17813050746917725 + ], + [ + -0.03945416212081909, + -0.5818068981170654, + 0.6312779784202576, + 0.20385882258415222, + 1.534226417541504, + 0.47082000970840454, + -0.05443471670150757, + -0.030519947409629822, + 0.026358027011156082, + -0.0040660277009010315, + 0.1990625560283661, + 0.8969831466674805, + 0.30952897667884827, + 0.7024948596954346, + 0.29324811697006226, + -0.27860313653945923, + -0.516141414642334, + 0.17933474481105804, + 0.6370826363563538, + -1.00701904296875, + -0.35897037386894226 + ], + [ + 0.04158759117126465, + -0.60334712266922, + 0.442973792552948, + 0.1053362488746643, + 1.5350306034088135, + 1.008374571800232, + 0.4564563035964966, + 0.19211281836032867, + 0.029262414202094078, + 0.22067029774188995, + -0.0864112600684166, + 0.329892098903656, + -0.2380591779947281, + 0.4292786717414856, + 0.28826022148132324, + -0.3610919117927551, + -0.18386386334896088, + 0.20696431398391724, + -0.11754748225212097, + -1.646545171737671, + -0.08137795329093933 + ] + ], + "generated_utilities": [ + 0.057138841599226, + 0.06841854006052017, + 0.015207032673060894, + 0.016131063923239708, + 0.08538073301315308, + 0.0494011826813221, + 0.02298945188522339, + 0.018225232139229774 + ], + "group_id": "ms-638d2ba1917eff4d09e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.028925320133566856, + 0.11176212877035141, + 0.057868462055921555, + 0.09960909932851791, + 0.07349967956542969, + 0.029011588543653488, + 0.06443297863006592, + 0.05166193097829819, + 0.025834282860159874, + 0.056667208671569824, + 0.04477008432149887, + 0.019685089588165283, + 0.018628716468811035, + 0.08587563037872314, + 0.02232462167739868, + 0.010265171527862549 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04120570421218872, + 0.038339823484420776, + 0.5979268550872803, + -0.05306033045053482, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.3407015800476074, + -0.21580028533935547, + -0.036036908626556396, + -0.004389047622680664, + 0.5364620685577393, + 0.026709645986557007, + 0.0, + 1.0126078128814697, + -0.10034149885177612, + -0.16565871238708496, + 0.07125610113143921, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + -0.5276790261268616, + -0.19392354786396027, + 0.5979268550872803, + 0.05016906559467316, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.6852816939353943, + -0.21580028533935547, + 0.04531368613243103, + -0.09276384115219116, + 0.5364620685577393, + 0.021915271878242493, + 0.0, + 1.0126078128814697, + 0.29255303740501404, + -0.16565871238708496, + 0.14600259065628052, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + 0.5910290479660034, + -0.038985610008239746, + 1.5555552244186401, + -1.028314232826233, + 2.3103959560394287, + 0.8163888454437256, + 2.064336061477661, + 1.168483018875122, + -0.21718227863311768, + -0.19770365953445435, + 0.11310696601867676, + 0.6850457191467285, + -0.06804794818162918, + 0.33076921105384827, + 1.7529149055480957, + -0.562712550163269, + -1.6352436542510986, + 0.1224215030670166, + 3.354771852493286, + -1.1317338943481445, + -1.7348618507385254 + ] + ], + "predicted_scores": [ + 6.9197821617126465, + 6.764666557312012, + 6.156739234924316, + 5.908979415893555, + 5.825287818908691, + 5.521308422088623, + 5.370540618896484, + 4.671911716461182 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "5d6f4227fab3b4bb36756c9a15c32b6fe6c9a0ae4b07b3f20a2bd65c2594df13", + "stored_base_utility": 0.028925320133566856, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.007633765693753958, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007633765693753958 + }, + { + "contact_quality": null, + "progress": 0.004399994853883982, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004399994853883982 + }, + { + "contact_quality": null, + "progress": 0.005148271564394236, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005148271564394236 + }, + { + "contact_quality": null, + "progress": 0.0037710864562541246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0037710864562541246 + }, + { + "contact_quality": null, + "progress": 0.0034768779296427965, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0034768779296427965 + }, + { + "contact_quality": null, + "progress": 0.007886429317295551, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007886429317295551 + }, + { + "contact_quality": null, + "progress": 0.0032363932114094496, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0032363932114094496 + }, + { + "contact_quality": null, + "progress": 0.0019465485820546746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0019465485820546746 + } + ], + "candidate_progress": [ + 0.007633765693753958, + 0.004399994853883982, + 0.005148271564394236, + 0.0037710864562541246, + 0.0034768779296427965, + 0.007886429317295551, + 0.0032363932114094496, + 0.0019465485820546746 + ], + "candidate_source_chart_ids": [ + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-a6a6de64c39abad45f86", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-652ad552ba98d2736620", + "generated_tangents": [ + [ + 0.44328463077545166, + 0.08944632112979889, + 1.1233186721801758, + -0.6342301964759827, + 2.6212942600250244, + 2.7777037620544434, + 1.1180120706558228, + 0.3905259370803833, + -0.13039368391036987, + 0.8636714220046997, + 0.347335547208786, + 0.7396171689033508, + 0.21301797032356262, + 0.320237398147583, + -0.6468695402145386, + -0.2346024364233017, + 1.3507697582244873, + 0.013264164328575134, + 1.0822482109069824, + 2.213974952697754, + 2.575228452682495 + ], + [ + 0.22460892796516418, + 0.10911282896995544, + 1.2078988552093506, + -0.5381326675415039, + 2.284000873565674, + 2.28365421295166, + 1.9541079998016357, + 0.29657286405563354, + -0.12368787825107574, + 1.0407379865646362, + 0.520634114742279, + 1.1993212699890137, + 0.11919161677360535, + 0.37520527839660645, + -0.8313935995101929, + -0.02070493996143341, + 1.2995599508285522, + -0.13885383307933807, + 1.587677001953125, + 1.851771593093872, + 1.5285829305648804 + ], + [ + 0.11763723194599152, + 0.16928982734680176, + 1.2548750638961792, + -0.6948763132095337, + 2.777177333831787, + 2.4812123775482178, + 1.6804273128509521, + 0.44601404666900635, + 0.009550084359943867, + 0.7694616913795471, + 0.38177919387817383, + 1.2609425783157349, + 0.40404900908470154, + 0.8937294483184814, + -0.7064406275749207, + -0.07946361601352692, + 1.6611992120742798, + -0.11573787033557892, + 1.6066079139709473, + 2.0395593643188477, + 1.9090783596038818 + ], + [ + 0.014741957187652588, + 0.11712643504142761, + 1.5054949522018433, + -0.6112051010131836, + 2.2916512489318848, + 2.1692728996276855, + 0.9762204885482788, + 0.23812434077262878, + 0.027299486100673676, + 0.525099515914917, + 0.2947130799293518, + 1.2027831077575684, + 0.7017966508865356, + -0.37153053283691406, + -0.7292154431343079, + -0.28864961862564087, + 1.8713526725769043, + 0.05484373867511749, + 1.2416728734970093, + 2.16310453414917, + 1.9149621725082397 + ], + [ + 0.15705810487270355, + 0.21274326741695404, + 1.4380555152893066, + -0.7557823657989502, + 2.262181043624878, + 1.7250597476959229, + 1.7626023292541504, + 0.10116852819919586, + 0.0020106295123696327, + 0.8254652619361877, + 0.36281219124794006, + 1.2800666093826294, + 0.33038368821144104, + 0.35335612297058105, + -0.5088874697685242, + -0.11496816575527191, + 1.6460429430007935, + -0.10266251862049103, + 1.1497715711593628, + 2.248767137527466, + 1.8903944492340088 + ], + [ + 0.007033973932266235, + 0.054758474230766296, + 1.300036907196045, + -0.8114298582077026, + 2.267390489578247, + 1.9414392709732056, + 1.1136937141418457, + 0.29846006631851196, + 0.003335862420499325, + 0.5853330492973328, + 0.24868065118789673, + 1.1058058738708496, + -0.03307062387466431, + 0.25670677423477173, + -0.29727616906166077, + -0.06248842179775238, + 1.3406250476837158, + 0.08400772511959076, + 1.0439934730529785, + 2.1453020572662354, + 1.3445807695388794 + ], + [ + 0.17126376926898956, + 0.1989465355873108, + 1.42512047290802, + -0.7550135850906372, + 2.20028018951416, + 2.0143473148345947, + 1.4277539253234863, + 0.1929885447025299, + -0.022833550348877907, + 0.9543581008911133, + 0.2628089487552643, + 1.1975512504577637, + 0.07181712985038757, + 0.05113094300031662, + -0.5681172013282776, + -0.16490671038627625, + 1.6664643287658691, + -0.013347640633583069, + 1.3300714492797852, + 2.0404887199401855, + 1.8152005672454834 + ], + [ + -0.00817921757698059, + 0.002253517508506775, + 1.5720202922821045, + -0.6353524923324585, + 2.2705748081207275, + 1.2103488445281982, + 1.6786835193634033, + 0.13466280698776245, + -0.10966622084379196, + 1.0458556413650513, + 0.22026807069778442, + 1.342624306678772, + -0.26158997416496277, + 0.6853857040405273, + -0.7264761328697205, + -0.12796680629253387, + 1.7027888298034668, + 0.17405883967876434, + 1.5154500007629395, + 1.6549060344696045, + 1.3723491430282593 + ] + ], + "generated_utilities": [ + 0.007633765693753958, + 0.004399994853883982, + 0.005148271564394236, + 0.0037710864562541246, + 0.0034768779296427965, + 0.007886429317295551, + 0.0032363932114094496, + 0.0019465485820546746 + ], + "group_id": "ms-652ad552ba98d2736620", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08378833532333374, + 0.10458396375179291, + 0.19086095690727234, + 0.04039820283651352, + 0.15980230271816254, + 0.0428888201713562, + 0.2367449700832367, + 0.18185830116271973, + 0.17593276500701904, + 0.16103637218475342, + 0.15250858664512634, + 0.05226989835500717, + 0.1701633185148239, + 0.11590366065502167, + 0.05217729136347771, + 0.03575938940048218 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.09009888768196106, + 0.08798432350158691, + 0.718157172203064, + -0.6351244449615479, + 1.405301570892334, + 0.6596384048461914, + 0.0, + 0.060225605964660645, + -0.0894029438495636, + -0.10323715209960938, + 0.052752673625946045, + 0.5354657173156738, + -0.013395614922046661, + 0.0, + -0.3450838029384613, + -0.11172826588153839, + 0.7912200689315796, + 0.33702391386032104, + 0.5586241483688354, + 1.8093180656433105, + 0.0 + ], + [ + 0.557296633720398, + 1.087984323501587, + 1.718157172203064, + -1.6351244449615479, + 2.405301570892334, + 1.6596384048461914, + 1.7430553436279297, + 0.3360595703125, + -0.2521040737628937, + 0.8967628479003906, + 0.26587578654289246, + 1.5354657173156738, + -0.12294083833694458, + -0.022678710520267487, + -0.416314959526062, + -0.26016199588775635, + 1.7912200689315796, + -0.16825227439403534, + 1.5586241483688354, + 2.8093180656433105, + 2.2589924335479736 + ], + [ + 0.2635665237903595, + 1.0005124807357788, + 1.973160743713379, + -1.7548720836639404, + 2.176497459411621, + 1.5416926145553589, + 1.9556171894073486, + 0.2215275913476944, + -0.5432531833648682, + 0.8434366583824158, + 0.3559061288833618, + 1.6657240390777588, + -0.3025258779525757, + 0.04401832073926926, + -0.17062894999980927, + -0.1509227454662323, + 1.7901219129562378, + 0.0010261386632919312, + 1.726505994796753, + 2.9778714179992676, + 2.0530292987823486 + ], + [ + 0.23987123370170593, + 0.8904718160629272, + 1.467047929763794, + -1.990342378616333, + 2.7685346603393555, + 2.0486786365509033, + 2.013002395629883, + 0.006436288356781006, + -0.25436922907829285, + 1.1101897954940796, + 0.4448293447494507, + 1.3105798959732056, + 0.07367905974388123, + 0.23779401183128357, + -0.34580594301223755, + -0.35948312282562256, + 1.4435756206512451, + -0.35472720861434937, + 1.801603078842163, + 3.0683224201202393, + 2.2330009937286377 + ], + [ + 0.6738064289093018, + 1.3973212242126465, + 2.0960373878479004, + -1.735988736152649, + 2.5216798782348633, + 1.1975955963134766, + 2.234865665435791, + 0.4602587819099426, + -0.7169049382209778, + 0.9412583708763123, + 0.7081141471862793, + 1.1730585098266602, + -0.016052812337875366, + -0.10474976152181625, + -0.03458032011985779, + 0.044317394495010376, + 1.6339548826217651, + -0.5292693376541138, + 1.5412750244140625, + 3.018833875656128, + 2.6051149368286133 + ], + [ + 0.6305910348892212, + 1.540043592453003, + 1.626009464263916, + -1.8823078870773315, + 2.2431485652923584, + 2.1866872310638428, + 2.3003249168395996, + 0.25838330388069153, + -0.2936764061450958, + 1.2413102388381958, + -0.12014040350914001, + 1.743309497833252, + 0.3618801534175873, + -0.37178292870521545, + 0.11553955078125, + 0.04856392741203308, + 1.2029554843902588, + 0.04960852861404419, + 2.0190422534942627, + 2.5883281230926514, + 2.7636377811431885 + ], + [ + 0.5004369616508484, + 1.4795310497283936, + 2.325191020965576, + -1.7464677095413208, + 1.9271103143692017, + 1.2483245134353638, + 1.9852193593978882, + 0.24291110038757324, + 0.4463852345943451, + 0.5267151594161987, + 0.32179588079452515, + 1.2356117963790894, + 0.03201058506965637, + 0.2881847023963928, + -1.0345205068588257, + -0.4171831011772156, + 1.5080864429473877, + 0.11479203402996063, + 1.9481834173202515, + 2.840816020965576, + 2.427428960800171 + ], + [ + -0.3133848309516907, + 0.43174320459365845, + 2.305823802947998, + -1.657350778579712, + 2.010749578475952, + 1.9222235679626465, + 1.3648147583007812, + 0.1938892900943756, + -0.6084526777267456, + 0.7855725288391113, + -0.07715931534767151, + 0.9140809178352356, + -0.9242425560951233, + -0.4681842029094696, + -0.07827043533325195, + -0.23733587563037872, + 0.9328996539115906, + -0.6088451147079468, + 0.8040639162063599, + 2.620466947555542, + 2.5148611068725586 + ] + ], + "predicted_scores": [ + 3.608150005340576, + 3.326437473297119, + 2.825561761856079, + 2.500056743621826, + 2.399468421936035, + 2.315864086151123, + 2.0968191623687744, + 1.6071484088897705 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "8c1ab306d1e82f81085196c60b3c846d2e2eb20ebc538597af359a6252623e7e", + "stored_base_utility": 0.08378833532333374, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03784912824630737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03784912824630737 + }, + "base_progress": 0.03784912824630737, + "base_success": false, + "base_utility": 0.03784912824630737, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.04070079326629639, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04070079326629639 + }, + { + "contact_quality": null, + "progress": 0.035504478961229324, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035504478961229324 + }, + { + "contact_quality": null, + "progress": 0.016018351539969444, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016018351539969444 + }, + { + "contact_quality": null, + "progress": 0.057549476623535156, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057549476623535156 + }, + { + "contact_quality": null, + "progress": 0.029227495193481445, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029227495193481445 + }, + { + "contact_quality": null, + "progress": 0.029378632083535194, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029378632083535194 + }, + { + "contact_quality": null, + "progress": 0.03666381165385246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03666381165385246 + }, + { + "contact_quality": null, + "progress": 0.04491941258311272, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04491941258311272 + } + ], + "candidate_progress": [ + 0.04070079326629639, + 0.035504478961229324, + 0.016018351539969444, + 0.057549476623535156, + 0.029227495193481445, + 0.029378632083535194, + 0.03666381165385246, + 0.04491941258311272 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-48178f52a96de3d1ead0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-674fad0e7c042d23629e", + "generated_tangents": [ + [ + 0.9003629088401794, + 0.12425516545772552, + -0.30955418944358826, + -0.14490333199501038, + 0.5139075517654419, + -1.4620239734649658, + -0.17373090982437134, + 0.3943895101547241, + 0.12960994243621826, + -0.4282251000404358, + -0.20047026872634888, + -0.7172543406486511, + -1.6864415407180786, + -1.2767021656036377, + -0.46521663665771484, + -0.10093821585178375, + 0.3085433840751648, + -0.628913164138794, + -0.023214906454086304, + 0.03263598680496216, + 0.2237262725830078 + ], + [ + 0.9199603796005249, + 0.08735062181949615, + -0.3462953269481659, + -0.08642083406448364, + 0.5941511392593384, + -1.2761945724487305, + -0.2588474750518799, + 0.418645441532135, + 0.06355312466621399, + -0.35064005851745605, + -0.17876100540161133, + -0.647356390953064, + -1.6682637929916382, + -1.2554212808609009, + -0.4986473321914673, + -0.071780264377594, + 0.30545884370803833, + -0.6120912432670593, + -0.05120024085044861, + -0.14255347847938538, + -0.027466237545013428 + ], + [ + 1.0517892837524414, + 0.03220818191766739, + -0.12867340445518494, + 0.05941849946975708, + 0.31146305799484253, + -1.538813829421997, + -0.5194803476333618, + 0.7237954139709473, + 0.08202748745679855, + -0.2483474314212799, + 0.05738503858447075, + -0.7206576466560364, + -1.5642082691192627, + -0.7618886232376099, + -0.4593932628631592, + -0.1864432692527771, + 0.15552642941474915, + -0.3930931091308594, + -0.008394837379455566, + -0.05159834027290344, + 0.005968749523162842 + ], + [ + 1.182978630065918, + 0.15921711921691895, + -0.2721805274486542, + -0.20114116370677948, + 0.4701843857765198, + -1.541680932044983, + -0.27385467290878296, + 0.41247761249542236, + 0.12139921635389328, + -0.45550674200057983, + -0.18808534741401672, + -0.9331966638565063, + -1.6188327074050903, + -1.374698519706726, + -0.4295809268951416, + -0.13094736635684967, + 0.24121223390102386, + -0.6172925233840942, + 0.25906991958618164, + -0.11272430419921875, + 0.3728640079498291 + ], + [ + 1.0054574012756348, + -0.14592623710632324, + -0.30663397908210754, + 0.03191231191158295, + 0.6533571481704712, + -1.5023176670074463, + -0.08405900001525879, + 0.3093627095222473, + 0.1181986853480339, + -0.21082532405853271, + -0.27991580963134766, + -0.4078807830810547, + -1.3492491245269775, + -1.0489988327026367, + -0.4215231239795685, + -0.05872068554162979, + 0.28000956773757935, + -0.563254177570343, + -0.07488834857940674, + -0.4309867024421692, + -0.27577027678489685 + ], + [ + 1.010310173034668, + -0.12428206205368042, + -0.3011971414089203, + 0.1363234519958496, + 0.6821796298027039, + -1.4914907217025757, + -0.21904844045639038, + 0.3578451871871948, + 0.14482904970645905, + -0.28952568769454956, + -0.14728403091430664, + -0.42694759368896484, + -1.5940645933151245, + -0.9595520496368408, + -0.377424418926239, + -0.04055410623550415, + 0.26222702860832214, + -0.6909810900688171, + 0.010868072509765625, + -0.3265906870365143, + -0.3279358446598053 + ], + [ + 0.842851459980011, + -0.10780809819698334, + -0.22249743342399597, + 0.1693369448184967, + 0.5579236149787903, + -1.3317362070083618, + -0.11668068170547485, + 0.2850192189216614, + 0.10159546136856079, + -0.27994829416275024, + -0.16368147730827332, + -0.4000256061553955, + -0.9346339106559753, + -0.8347570300102234, + -0.3967498540878296, + -0.12279963493347168, + 0.300127774477005, + -0.4666268825531006, + -0.2992909550666809, + -0.48686856031417847, + -0.16061455011367798 + ], + [ + 1.1743879318237305, + -0.09086474776268005, + -0.5411816835403442, + 0.09675449132919312, + 0.8503625392913818, + -1.6534641981124878, + 0.3966398239135742, + 0.40719252824783325, + 0.12983600795269012, + -0.48168712854385376, + -0.04600173979997635, + -0.3031688332557678, + -1.4280933141708374, + -0.6938177347183228, + -0.17034503817558289, + -0.04279131442308426, + 0.4690910577774048, + -0.7339193224906921, + 0.25339069962501526, + -0.31196919083595276, + -0.19958674907684326 + ] + ], + "generated_utilities": [ + 0.04070079326629639, + 0.035504478961229324, + 0.016018351539969444, + 0.057549476623535156, + 0.029227495193481445, + 0.029378632083535194, + 0.03666381165385246, + 0.04491941258311272 + ], + "group_id": "ms-674fad0e7c042d23629e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07829316705465317, + 0.080319844186306, + 0.11335399001836777, + 0.15264075994491577, + 0.14950451254844666, + 0.07767044007778168, + 0.12988285720348358, + 0.10878288745880127, + 0.15573063492774963, + 0.10951457917690277, + 0.06582474708557129, + 0.02477661892771721, + 0.07745532691478729, + 0.03348777815699577, + 0.10050909221172333, + 0.033770203590393066 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.5533217191696167, + 0.5758762359619141, + -0.6480545997619629, + -0.10892429947853088, + 0.7943102717399597, + -2.7399702072143555, + -1.8395006656646729, + 1.447904109954834, + 1.4495785236358643, + -0.055675506591796875, + -1.3537942171096802, + 0.38944748044013977, + -1.6196211576461792, + -1.2710431814193726, + -0.9744676947593689, + 0.057668983936309814, + 1.8350472450256348, + -1.2618876695632935, + -1.2385435104370117, + -1.7976703643798828, + -1.9491409063339233 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9072847366333008, + 0.06923278421163559, + -0.14729690551757812, + 0.15558937191963196, + 0.40412187576293945, + -2.342223644256592, + 0.0, + 0.2589508295059204, + 0.35182610154151917, + 0.10176032781600952, + -0.3489869236946106, + 0.13150231540203094, + -1.3123579025268555, + 0.0, + 0.10376164317131042, + 0.0909266471862793, + 0.1919306516647339, + 0.28928786516189575, + -0.4625590741634369, + -0.30475670099258423, + 0.0 + ], + [ + 1.9072847366333008, + 0.05373973771929741, + -1.1472969055175781, + -0.47107377648353577, + 0.3914180397987366, + -3.342223644256592, + -2.4980990886688232, + 1.2589508295059204, + 0.8455543518066406, + -0.7174267172813416, + -1.0694091320037842, + 0.33346641063690186, + -2.3123579025268555, + -1.1108808517456055, + -0.33852219581604004, + -0.7232105731964111, + 1.145590901374817, + -0.7107121348381042, + -0.5049349069595337, + -1.112295150756836, + -1.244770884513855 + ], + [ + 1.808228611946106, + -0.18174676597118378, + -1.2057973146438599, + -0.6115378737449646, + 0.11874309182167053, + -3.3499772548675537, + -2.3861043453216553, + 1.2540870904922485, + 1.1210845708847046, + -0.953934371471405, + -1.3015953302383423, + 0.6089286804199219, + -2.1655566692352295, + -0.8375923037528992, + -0.26793593168258667, + -0.998324990272522, + 1.2481704950332642, + -0.8320856690406799, + -0.376935750246048, + -1.0882012844085693, + -0.9857641458511353 + ], + [ + 2.291844606399536, + 0.05574151128530502, + -0.9760321974754333, + -0.5710926055908203, + 0.8478215336799622, + -3.22356915473938, + -2.591245651245117, + 1.2184754610061646, + 0.855300784111023, + -1.0267595052719116, + -1.3318730592727661, + 0.4197496771812439, + -2.0368778705596924, + -1.3685163259506226, + -0.582176685333252, + -0.8082455992698669, + 1.3435251712799072, + -0.44022125005722046, + -0.06775334477424622, + -0.7692400217056274, + -0.8863754272460938 + ] + ], + "predicted_scores": [ + 9.506608009338379, + 9.447779655456543, + 9.234846115112305, + 9.209221839904785, + 8.154670715332031, + 8.111454963684082, + 7.987610340118408, + 6.852998733520508 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "6802b2d1fb8a5afa54b7321e2bb938176de48d74f28e9b2a9c741c23547f0ef1", + "stored_base_utility": 0.07829316705465317, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.12791956961154938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12791956961154938 + }, + "base_progress": 0.12791956961154938, + "base_success": false, + "base_utility": 0.12791956961154938, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07348597049713135, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07348597049713135 + }, + { + "contact_quality": null, + "progress": 0.06691998243331909, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06691998243331909 + }, + { + "contact_quality": null, + "progress": 0.0687488317489624, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0687488317489624 + }, + { + "contact_quality": null, + "progress": 0.07327929884195328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07327929884195328 + }, + { + "contact_quality": null, + "progress": 0.03650514408946037, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03650514408946037 + }, + { + "contact_quality": null, + "progress": 0.05396809056401253, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05396809056401253 + }, + { + "contact_quality": null, + "progress": 0.07279130071401596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07279130071401596 + }, + { + "contact_quality": null, + "progress": 0.13260090351104736, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13260090351104736 + } + ], + "candidate_progress": [ + 0.07348597049713135, + 0.06691998243331909, + 0.0687488317489624, + 0.07327929884195328, + 0.03650514408946037, + 0.05396809056401253, + 0.07279130071401596, + 0.13260090351104736 + ], + "candidate_source_chart_ids": [ + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-1ba37e30b90a3b28e62f", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-728b865677134d3788e3", + "generated_tangents": [ + [ + -0.39034903049468994, + -0.37187659740448, + 0.3935436010360718, + -0.15706869959831238, + 0.9356510043144226, + -0.19463494420051575, + 0.4388798177242279, + 0.7160687446594238, + 0.19778768718242645, + 0.15543249249458313, + -1.5249929428100586, + 0.8594302535057068, + -0.7428938150405884, + -0.5552669167518616, + -1.482895016670227, + 0.9974061250686646, + 1.7214030027389526, + 0.01351088285446167, + -0.2140149176120758, + -0.6090185642242432, + 0.4767541289329529 + ], + [ + -0.4475897550582886, + -0.3542618751525879, + 0.5799123048782349, + 0.12671992182731628, + 0.9840754866600037, + -0.1267329454421997, + 0.6425981521606445, + 0.8072195053100586, + 0.3213188648223877, + 0.23512426018714905, + -1.4188973903656006, + 0.6924213171005249, + -0.7114902138710022, + -0.19320417940616608, + -1.6145719289779663, + 0.9080706238746643, + 1.6354997158050537, + -0.19772334396839142, + -0.10790109634399414, + -0.6084801554679871, + 0.5870579481124878 + ], + [ + -0.4405338168144226, + -0.3784378170967102, + 0.5646163821220398, + 0.09684327244758606, + 0.8023883700370789, + -0.29672256112098694, + 0.5263015627861023, + 0.7598861455917358, + 0.30964475870132446, + 0.22663670778274536, + -1.4080426692962646, + 0.7018111944198608, + -0.7842860817909241, + -0.44861429929733276, + -1.646507740020752, + 0.8910717964172363, + 1.6700502634048462, + -0.20267723500728607, + -0.17231038212776184, + -0.5862141847610474, + 0.39029455184936523 + ], + [ + -0.34456372261047363, + -0.41968315839767456, + 0.7650783061981201, + 0.20784354209899902, + 0.6980795860290527, + -0.3027670979499817, + 0.8314871788024902, + 0.3897552490234375, + 0.30897587537765503, + 0.24327963590621948, + -1.3717589378356934, + 0.8708892464637756, + -0.5835776329040527, + -0.5545505881309509, + -1.8845280408859253, + 0.8902562260627747, + 1.5212655067443848, + -0.349348247051239, + -0.12915486097335815, + -0.6258441209793091, + 0.06586413085460663 + ], + [ + -0.6155652403831482, + -0.3373787999153137, + 1.1049389839172363, + 0.24397826194763184, + 1.1622910499572754, + 0.4856007695198059, + 0.5087839961051941, + 0.5048795342445374, + 0.21637371182441711, + 0.2952132821083069, + -0.8826477527618408, + 0.8144375681877136, + -1.2008814811706543, + -0.34453094005584717, + -1.3298373222351074, + 0.8847464323043823, + 1.4294888973236084, + -0.2635078430175781, + 0.4943919777870178, + -1.6388778686523438, + 0.4366621971130371 + ], + [ + -0.1289636343717575, + -0.4173429608345032, + 1.0081381797790527, + -0.022463232278823853, + 0.6412431597709656, + -0.6562385559082031, + 0.38539278507232666, + 0.7804857492446899, + 0.3084360659122467, + 0.006881922483444214, + -1.3052064180374146, + 0.37727031111717224, + -1.4309247732162476, + -0.7981573343276978, + -1.413253664970398, + 0.8792598843574524, + 1.9208170175552368, + -0.3883278965950012, + -0.5910097360610962, + -1.0852289199829102, + 0.38884368538856506 + ], + [ + -0.9370386600494385, + -0.3018230199813843, + 0.8576704263687134, + 0.3421913683414459, + 0.9998913407325745, + -0.38233909010887146, + 0.663680374622345, + 0.6609971523284912, + 0.42053672671318054, + 0.07217412441968918, + -1.364300012588501, + 1.2068727016448975, + -1.0331658124923706, + -0.07561425864696503, + -1.806665301322937, + 0.7029359340667725, + 1.1926920413970947, + -0.3325579762458801, + 0.057507649064064026, + -0.39939096570014954, + 0.16372086107730865 + ], + [ + -0.3966712951660156, + -0.34475117921829224, + 0.453570157289505, + 0.05661824345588684, + 0.8598929047584534, + -0.015782281756401062, + 0.1667470633983612, + 0.6924062967300415, + 0.2899540662765503, + -0.09931730479001999, + -1.3228330612182617, + 0.12798099219799042, + -1.021296739578247, + -0.6155245304107666, + -1.633471131324768, + 0.8296821117401123, + 1.7047876119613647, + -0.4488670229911804, + -0.6817136406898499, + -0.7421263456344604, + -0.2532697319984436 + ] + ], + "generated_utilities": [ + 0.07348597049713135, + 0.06691998243331909, + 0.0687488317489624, + 0.07327929884195328, + 0.03650514408946037, + 0.05396809056401253, + 0.07279130071401596, + 0.13260090351104736 + ], + "group_id": "ms-728b865677134d3788e3", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05257302522659302, + 0.05559448525309563, + 0.11835493892431259, + 0.04238405078649521, + 0.16897141933441162, + 0.02001136541366577, + 0.1609041690826416, + 0.14248299598693848, + 0.14091376960277557, + 0.17822234332561493, + 0.22658847272396088, + 0.1736571490764618, + 0.18659374117851257, + 0.12753915786743164, + 0.07164359092712402, + 0.0063618822023272514 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04065108299255371, + 0.05943702906370163, + 0.0039327144622802734, + 0.10760968923568726, + -0.3465921878814697, + -0.6843758821487427, + 0.0, + 0.3034271001815796, + -0.38217779994010925, + 0.019724853336811066, + -1.1125431060791016, + -0.016437113285064697, + 0.09826856851577759, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + 0.055904269218444824, + -0.2721228003501892, + 0.07830148935317993, + 0.0 + ], + [ + -0.9075424671173096, + 0.06243014335632324, + 1.0039327144622803, + -0.6845037937164307, + 0.050797268748283386, + -1.6843758821487427, + -1.3340954780578613, + 1.3034271001815796, + -0.29618290066719055, + -0.035879287868738174, + -2.1125431060791016, + 0.9835628867149353, + -0.9017314314842224, + -1.01132333278656, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233 + ], + [ + -1.117501139640808, + 0.23503510653972626, + 0.9669153094291687, + -0.8317459225654602, + 0.06331060081720352, + -1.5474717617034912, + -1.3840656280517578, + 1.240980863571167, + -0.053158730268478394, + 0.10333722829818726, + -2.2084054946899414, + 1.1802411079406738, + -0.8987206816673279, + -1.136216402053833, + -1.9180536270141602, + 1.4563629627227783, + 2.625094413757324, + -0.1296394169330597, + 0.3340006470680237, + -0.7212131023406982, + 0.9384214878082275 + ], + [ + -1.2640053033828735, + 0.2415706366300583, + 0.9424604177474976, + -0.8472410440444946, + 0.35185110569000244, + -1.8022713661193848, + -1.2391949892044067, + 0.9333417415618896, + 0.03309231996536255, + 0.22933605313301086, + -1.7521884441375732, + 1.0360857248306274, + -0.6365935802459717, + -0.7614871859550476, + -2.103987216949463, + 1.5381183624267578, + 2.11069917678833, + -0.5904175639152527, + -0.07874824106693268, + -0.3884509205818176, + 1.0602728128433228 + ], + [ + -0.7705078721046448, + 0.18350359797477722, + 0.579454779624939, + -0.5434049963951111, + 0.2274666726589203, + -1.7208446264266968, + -1.5971848964691162, + 1.6145182847976685, + -0.6915329694747925, + 0.20122811198234558, + -1.8584421873092651, + 0.9026989340782166, + -0.5741626024246216, + -0.5299967527389526, + -1.544856309890747, + 1.7236231565475464, + 2.1288580894470215, + -0.7592856287956238, + 0.3358893394470215, + -0.2929669916629791, + 1.67848801612854 + ], + [ + -0.3686429262161255, + 0.3043383061885834, + 0.7294905781745911, + -0.8736650347709656, + 0.3122289776802063, + -1.5051836967468262, + -1.8593363761901855, + 0.8389579653739929, + -0.8785254955291748, + -0.5520821213722229, + -1.8517990112304688, + 1.3417646884918213, + -0.32390326261520386, + -0.42145586013793945, + -1.6530441045761108, + 1.7202519178390503, + 2.180105209350586, + -0.4262692928314209, + -0.1161729246377945, + -0.08550512790679932, + 1.1130309104919434 + ], + [ + -0.9971213936805725, + -0.4085577130317688, + 0.9404422044754028, + -0.9977414011955261, + 0.4963066577911377, + -1.4445728063583374, + -1.8557989597320557, + 1.3813822269439697, + -0.9350934028625488, + 0.6622016429901123, + -2.0982701778411865, + 0.5712680220603943, + -1.3476169109344482, + -0.8056140542030334, + -2.2595674991607666, + 1.7601569890975952, + 2.5158329010009766, + -0.8607662916183472, + -0.010314732789993286, + -0.9802933931350708, + 0.587103545665741 + ], + [ + -0.538765549659729, + -0.2632336914539337, + 0.574018120765686, + -1.3315669298171997, + 0.23096206784248352, + -1.4029772281646729, + -1.2746275663375854, + 1.674896478652954, + -0.6276504993438721, + -0.06763898581266403, + -1.44939124584198, + 1.1256837844848633, + -0.6540690064430237, + -0.7850072979927063, + -2.472240686416626, + 1.201231837272644, + 2.056978702545166, + 0.4725829064846039, + 0.7320148944854736, + -0.47291314601898193, + 1.2244172096252441 + ], + [ + -0.5474913120269775, + 0.3537777066230774, + 0.1717698574066162, + -0.9464929103851318, + -0.24061335623264313, + -2.5145106315612793, + -1.9605224132537842, + 2.171875476837158, + 0.5462924242019653, + 0.2324821650981903, + -2.4616305828094482, + 1.3455290794372559, + -0.703314483165741, + -0.6289631128311157, + -1.848944902420044, + 1.755210518836975, + 2.649005174636841, + -0.931267499923706, + -0.3109248876571655, + 0.21283608675003052, + 0.6994417309761047 + ], + [ + -1.4273285865783691, + 0.48591291904449463, + 1.4724129438400269, + -1.1116547584533691, + -0.04224221408367157, + -1.1567587852478027, + -1.0686161518096924, + 1.7429871559143066, + -1.1814582347869873, + 0.07730802893638611, + -2.1004433631896973, + 0.45253968238830566, + -0.7293854355812073, + -0.4477439522743225, + -1.986203908920288, + 1.1158549785614014, + 2.0316240787506104, + 0.5763667821884155, + -0.48857587575912476, + -1.07709801197052, + 0.5611822605133057 + ] + ], + "predicted_scores": [ + 2.4478979110717773, + 2.437049627304077, + 2.4245622158050537, + 2.25917649269104, + 1.7486274242401123, + 1.7416589260101318, + 1.6707737445831299, + 1.5316259860992432 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296", + "selected_index": 0, + "state_hash": "62e00728e448102460275bc2c35150dbf429ab0d5670a6706fa41256273bac3c", + "stored_base_utility": 0.05257302522659302, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20529556274414062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20529556274414062 + }, + "base_progress": 0.20529556274414062, + "base_success": false, + "base_utility": 0.20529556274414062, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03273266553878784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03273266553878784 + }, + { + "contact_quality": null, + "progress": 0.03373098373413086, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03373098373413086 + }, + { + "contact_quality": null, + "progress": 0.04705389216542244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04705389216542244 + }, + { + "contact_quality": null, + "progress": 0.05021713301539421, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05021713301539421 + }, + { + "contact_quality": null, + "progress": 0.04120592400431633, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04120592400431633 + }, + { + "contact_quality": null, + "progress": 0.09449902921915054, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09449902921915054 + }, + { + "contact_quality": null, + "progress": 0.051247138530015945, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.051247138530015945 + }, + { + "contact_quality": null, + "progress": 0.04602358862757683, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04602358862757683 + } + ], + "candidate_progress": [ + 0.03273266553878784, + 0.03373098373413086, + 0.04705389216542244, + 0.05021713301539421, + 0.04120592400431633, + 0.09449902921915054, + 0.051247138530015945, + 0.04602358862757683 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-6ddecc7f074b9e267409", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "generated_tangents": [ + [ + 0.9338359832763672, + -0.027668345719575882, + -0.15829375386238098, + 0.25817638635635376, + 0.500524640083313, + -2.3095037937164307, + -1.6588237285614014, + -0.8597820997238159, + -0.04362007975578308, + 0.9941971898078918, + 0.5525499582290649, + 0.25136032700538635, + 1.3738535642623901, + -0.22684724628925323, + -0.536362886428833, + -0.22348515689373016, + 1.0505791902542114, + 1.003051519393921, + 0.3198562264442444, + -0.801285445690155, + -0.5484545230865479 + ], + [ + 0.7688387632369995, + -0.11683923006057739, + -0.6016196012496948, + 0.21707341074943542, + 0.31842467188835144, + -2.13503098487854, + -1.047135353088379, + -1.1803016662597656, + -0.016026286408305168, + 0.7579044103622437, + 0.4563840329647064, + -0.14257311820983887, + 1.113267421722412, + 0.3017989993095398, + -0.44573020935058594, + -0.40615108609199524, + 0.9448165893554688, + 0.7917150259017944, + -0.0023890435695648193, + -1.0027339458465576, + -0.49909844994544983 + ], + [ + 0.9584357738494873, + -0.08497299998998642, + -0.4230339825153351, + 0.1546584665775299, + 0.354657381772995, + -2.7537901401519775, + -1.4100141525268555, + -0.8773391246795654, + -0.050078339874744415, + 0.8806989789009094, + 0.7529194951057434, + 0.0011425763368606567, + 0.4982249438762665, + 0.3286147713661194, + -0.3247344195842743, + -0.3099651634693146, + 0.9654372930526733, + 0.6347294449806213, + 0.03375589847564697, + -1.0854305028915405, + -0.07017646729946136 + ], + [ + 0.8489947319030762, + -0.11245530843734741, + -0.5541179180145264, + 0.26276764273643494, + 0.3777993321418762, + -2.4220778942108154, + -1.3048346042633057, + -0.8763859272003174, + -0.015903599560260773, + 0.8433221578598022, + 0.6016063690185547, + -0.022166937589645386, + 0.8738051652908325, + 0.5042930245399475, + -0.4785439372062683, + -0.39346086978912354, + 0.9362626671791077, + 0.687313437461853, + 0.2061292827129364, + -0.8717151284217834, + -0.2886638343334198 + ], + [ + 0.7763317823410034, + -0.18735788762569427, + -0.49614909291267395, + 0.1925407350063324, + 0.45760008692741394, + -2.593526840209961, + -1.2330846786499023, + -0.9266390800476074, + -0.04245000332593918, + 0.7759215831756592, + 0.6631491780281067, + -0.09900155663490295, + 0.869513750076294, + 0.6292188167572021, + -0.44907987117767334, + -0.465933233499527, + 0.9126246571540833, + 0.6969174742698669, + 0.19138744473457336, + -0.9333896636962891, + -0.2863849997520447 + ], + [ + 0.9015936851501465, + -0.24430496990680695, + -0.47113338112831116, + 0.26044490933418274, + 0.6222957372665405, + -1.9738842248916626, + -1.7597627639770508, + -0.5862915515899658, + 0.05656706541776657, + 0.7552425265312195, + 0.3856826424598694, + 0.1264311671257019, + 1.013861060142517, + 0.677474856376648, + -0.594429612159729, + -0.3007287085056305, + 0.5179783701896667, + 0.7459592223167419, + 0.47634759545326233, + -0.7555156946182251, + -0.5965601205825806 + ], + [ + 0.8412076830863953, + -0.14837463200092316, + -0.5811554193496704, + 0.550194263458252, + 0.07614865899085999, + -2.2806615829467773, + -1.4382452964782715, + -0.7184593677520752, + 0.2265947163105011, + 1.004583477973938, + 0.593015193939209, + -0.2310028076171875, + 0.7951648235321045, + 0.6527782678604126, + -0.9342862963676453, + -0.1985575556755066, + 0.900602400302887, + 0.8346605896949768, + 0.1322655975818634, + -0.5390253067016602, + -0.08271494507789612 + ], + [ + 0.6555678844451904, + -0.3034543991088867, + -0.32602766156196594, + 0.2733928859233856, + 0.41943785548210144, + -2.308108329772949, + -1.034306287765503, + -0.8407673835754395, + -0.06308501958847046, + 0.7547457218170166, + 0.611559271812439, + -0.22087693214416504, + 0.7729818820953369, + 0.7875890731811523, + -0.4229137897491455, + -0.3362942934036255, + 0.7964543700218201, + 0.6681042313575745, + 0.1465258151292801, + -0.9407565593719482, + -0.21448808908462524 + ] + ], + "generated_utilities": [ + 0.03273266553878784, + 0.03373098373413086, + 0.04705389216542244, + 0.05021713301539421, + 0.04120592400431633, + 0.09449902921915054, + 0.051247138530015945, + 0.04602358862757683 + ], + "group_id": "ms-7ea98b8f4d2220347ee0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18600693345069885, + 0.07475636899471283, + 0.27452313899993896, + 0.20920448005199432, + 0.18411576747894287, + 0.27989473938941956, + 0.19416475296020508, + 0.10948207229375839, + 0.123555026948452, + 0.1468026340007782, + 0.12693151831626892, + 0.11466284841299057, + 0.07585643231868744, + 0.12908481061458588, + 0.22501108050346375, + 0.06431269645690918 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.06974434852600098, + -0.02351950854063034, + 0.04567508399486542, + -0.07042121887207031, + -0.038941264152526855, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.08617860078811646, + 0.9112892150878906, + 0.7374391555786133, + 0.06539928913116455, + 0.03787863254547119, + 0.0, + -0.021528124809265137, + -0.04566385596990585, + -0.03331831097602844, + 1.6466763019561768, + 0.04444807767868042, + -0.27494049072265625, + 0.0 + ], + [ + 1.1160619258880615, + -0.10018755495548248, + 0.19327251613140106, + -1.3990073204040527, + 0.672177255153656, + -3.303851842880249, + -2.832740545272827, + -1.902563214302063, + 0.39503830671310425, + 1.542332410812378, + 1.6850805282592773, + 0.6842766404151917, + 1.1400834321975708, + 1.670598030090332, + 0.631821870803833, + -0.26601675152778625, + -0.2725638151168823, + 3.003178119659424, + 0.23009130358695984, + -1.0388810634613037, + -0.19119825959205627 + ], + [ + 1.031497836112976, + -0.2943698763847351, + -0.11846622079610825, + -1.3361037969589233, + 0.6225074529647827, + -3.545761823654175, + -2.5078811645507812, + -1.4024183750152588, + 0.6736462116241455, + 2.093475580215454, + 1.9168713092803955, + 0.6377919316291809, + 0.5105260014533997, + 1.8172619342803955, + 1.1300766468048096, + 0.1616329848766327, + 0.03408849239349365, + 2.2435190677642822, + 0.735376238822937, + -1.5760226249694824, + -0.7514586448669434 + ], + [ + 1.7542359828948975, + -0.7351009249687195, + -0.854544997215271, + -1.223644495010376, + 0.07327532768249512, + -3.150663137435913, + -2.5486767292022705, + -2.110039472579956, + 1.0206904411315918, + 2.1492340564727783, + 2.187993288040161, + 1.2547765970230103, + 0.8607165217399597, + 1.1150517463684082, + 0.9828771948814392, + 0.5030426383018494, + -0.4691219627857208, + 2.2485060691833496, + -0.14985057711601257, + -1.9480454921722412, + -0.05157876014709473 + ], + [ + 0.6040669679641724, + 0.5665496587753296, + 0.24913550913333893, + -1.6600215435028076, + -0.10813313722610474, + -3.8031911849975586, + -3.226392984390259, + -1.5098689794540405, + 1.0684192180633545, + 2.1557064056396484, + 1.545849084854126, + 1.011389970779419, + 1.113816499710083, + 0.9937302470207214, + 1.372757911682129, + -0.4055754244327545, + -0.23403912782669067, + 3.0650274753570557, + 0.1330145299434662, + -0.7907523512840271, + -0.6428619623184204 + ], + [ + 0.30788952112197876, + 0.6807127594947815, + -0.035938769578933716, + -0.8788743615150452, + -0.30191969871520996, + -4.111589431762695, + -2.432647705078125, + -1.7479791641235352, + -0.321926474571228, + 1.9823087453842163, + 1.6617395877838135, + 1.3319180011749268, + 1.5937504768371582, + 2.0502309799194336, + 0.63725745677948, + 0.7232187390327454, + -0.23885302245616913, + 1.8472312688827515, + 0.3300834596157074, + -0.43588966131210327, + -1.2869892120361328 + ], + [ + 0.3636443018913269, + -0.4284004867076874, + 0.2675324082374573, + -0.4861376881599426, + 0.2239738255739212, + -2.813422918319702, + -2.4891929626464844, + -1.165071725845337, + 0.6442201733589172, + 1.1830427646636963, + 1.177597999572754, + 0.33577069640159607, + 0.6362329721450806, + 1.5409071445465088, + 0.29718804359436035, + -0.5603961944580078, + -0.8089720010757446, + 2.6271419525146484, + -0.5456377267837524, + -0.2824941873550415, + -0.7960987091064453 + ], + [ + 1.1001086235046387, + -0.9762114882469177, + -0.00277864933013916, + -1.704659104347229, + 1.2373628616333008, + -3.385840654373169, + -1.6910454034805298, + -1.470743179321289, + 0.9354362487792969, + 1.9378679990768433, + 1.515415906906128, + -0.3002503514289856, + 0.21796274185180664, + 1.542279839515686, + 1.7353785037994385, + 0.23501873016357422, + -0.16601301729679108, + 2.1989498138427734, + 0.8671687841415405, + -1.6252647638320923, + -1.2141560316085815 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06974434852600098, + -0.15391214191913605, + -0.22159501910209656, + -0.07042121887207031, + -0.09706336259841919, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.1400253176689148, + 0.9112892150878906, + 0.7374391555786133, + 0.13415807485580444, + -0.10778343677520752, + 0.0, + -0.08470147848129272, + 0.1231045201420784, + 0.08352774381637573, + 1.6466763019561768, + -0.005736887454986572, + -0.27494049072265625, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.125826835632324, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.152101516723633, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.8942345976829529 + ] + ], + "predicted_scores": [ + 6.813526153564453, + 5.844024658203125, + 5.787663459777832, + 5.626963138580322, + 5.517650127410889, + 5.478634834289551, + 5.153843402862549, + 5.113950729370117 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "a2b9f3b3ea6458c6734081cea861cf54026796a0a594bf9be3657d28f12e9b3c", + "stored_base_utility": 0.18600693345069885, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18213583528995514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18213583528995514 + }, + "base_progress": 0.18213583528995514, + "base_success": false, + "base_utility": 0.18213583528995514, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0009908875217661262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009908875217661262 + }, + { + "contact_quality": null, + "progress": 0.0009231368894688785, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009231368894688785 + }, + { + "contact_quality": null, + "progress": 0.0015073418617248535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0015073418617248535 + }, + { + "contact_quality": null, + "progress": 0.0008463859558105469, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008463859558105469 + }, + { + "contact_quality": null, + "progress": 0.0013398727169260383, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013398727169260383 + }, + { + "contact_quality": null, + "progress": 0.003711601020768285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003711601020768285 + }, + { + "contact_quality": null, + "progress": 0.0017419258365407586, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0017419258365407586 + }, + { + "contact_quality": null, + "progress": 0.0007060368661768734, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007060368661768734 + } + ], + "candidate_progress": [ + 0.0009908875217661262, + 0.0009231368894688785, + 0.0015073418617248535, + 0.0008463859558105469, + 0.0013398727169260383, + 0.003711601020768285, + 0.0017419258365407586, + 0.0007060368661768734 + ], + "candidate_source_chart_ids": [ + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3", + "ms-02311c7783bc67e188b4", + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-8e12f7ad446482627dd3" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7f57045c114ddad153b5", + "generated_tangents": [ + [ + -0.2520824670791626, + -0.12511616945266724, + 1.573218584060669, + -0.14429140090942383, + 1.4808824062347412, + 2.862978458404541, + 1.6637825965881348, + -0.03156566619873047, + 0.019331930205225945, + 1.0417245626449585, + 0.6316964626312256, + 0.9478929042816162, + 1.1161257028579712, + 1.8866373300552368, + -0.7375503182411194, + 0.019532859325408936, + 1.4928796291351318, + -0.3573065996170044, + 1.1581472158432007, + 1.3615963459014893, + 1.4436187744140625 + ], + [ + -0.2731807231903076, + -0.24097032845020294, + 1.4526458978652954, + -0.4595770239830017, + 1.4015209674835205, + 2.5857861042022705, + 1.8386640548706055, + 0.08633071929216385, + -0.1831485480070114, + 1.0925453901290894, + 0.4738825261592865, + 0.8631149530410767, + 1.3541682958602905, + 1.2599204778671265, + -0.7336021661758423, + -0.03448072075843811, + 1.5850486755371094, + -0.1531260758638382, + 1.0067245960235596, + 1.349545955657959, + 1.4664242267608643 + ], + [ + -0.10694728791713715, + -0.3016720414161682, + 1.4750951528549194, + -0.4589652121067047, + 1.6018683910369873, + 2.2743873596191406, + 2.048210859298706, + 0.18516378104686737, + -0.16088849306106567, + 0.9327735900878906, + 0.24240219593048096, + 0.8371343612670898, + 1.4592478275299072, + 2.012094736099243, + -0.5596979260444641, + 0.09212283790111542, + 1.5698059797286987, + -0.07501949369907379, + 1.0781712532043457, + 1.850982904434204, + 1.4062294960021973 + ], + [ + -0.1476268321275711, + -0.22388184070587158, + 1.3541926145553589, + -0.2221907526254654, + 1.7256944179534912, + 2.4829025268554688, + 1.8799159526824951, + 0.05385105311870575, + -0.2196202278137207, + 1.1014631986618042, + 0.4526349902153015, + 0.8024947047233582, + 1.130302906036377, + 1.5131126642227173, + -0.8158919215202332, + 0.031249038875102997, + 1.8092244863510132, + 0.014496967196464539, + 1.0285019874572754, + 1.4472846984863281, + 1.641628623008728 + ], + [ + 0.09493135660886765, + -0.28286826610565186, + 1.3929522037506104, + -0.5253861546516418, + 1.688690185546875, + 2.262230396270752, + 2.0344462394714355, + 0.09672358632087708, + -0.13036280870437622, + 1.1764590740203857, + 0.35523638129234314, + 0.9216479063034058, + 1.0209664106369019, + 1.6644783020019531, + -0.5358803272247314, + 0.0413481704890728, + 1.5671550035476685, + -0.1557922512292862, + 1.1109559535980225, + 1.4154980182647705, + 1.7753100395202637 + ], + [ + -0.30561399459838867, + -0.2675154209136963, + 1.2992031574249268, + -0.46828600764274597, + 1.1049857139587402, + 2.2260189056396484, + 1.5147428512573242, + 0.4077915549278259, + -0.11710146069526672, + 0.6827633380889893, + 0.5014147162437439, + 0.5580408573150635, + 1.3217647075653076, + 0.8362540602684021, + -0.33639612793922424, + -0.002032805234193802, + 1.166864037513733, + -0.2901707887649536, + 0.7729716897010803, + 1.439924955368042, + 1.058825969696045 + ], + [ + -0.0660819411277771, + -0.2754341959953308, + 1.2881159782409668, + -0.43712514638900757, + 1.6505377292633057, + 2.328268051147461, + 2.0120089054107666, + 0.24228665232658386, + -0.08295539766550064, + 1.0956268310546875, + 0.14585113525390625, + 0.9994098544120789, + 1.4498875141143799, + 1.777565836906433, + -0.49545717239379883, + 0.1572200208902359, + 1.5885310173034668, + -0.29035139083862305, + 1.238010287284851, + 1.7241621017456055, + 1.612553358078003 + ], + [ + -0.1133287101984024, + -0.18578703701496124, + 1.5263078212738037, + -0.3058190941810608, + 1.4516069889068604, + 2.3268725872039795, + 1.9249842166900635, + -0.07982030510902405, + -0.13312064111232758, + 1.1845663785934448, + 0.624777615070343, + 0.9806438088417053, + 1.4735302925109863, + 1.2689952850341797, + -0.848861575126648, + -0.012123910710215569, + 1.6709089279174805, + -0.17237071692943573, + 1.0478421449661255, + 1.3791472911834717, + 1.4720901250839233 + ] + ], + "generated_utilities": [ + 0.0009908875217661262, + 0.0009231368894688785, + 0.0015073418617248535, + 0.0008463859558105469, + 0.0013398727169260383, + 0.003711601020768285, + 0.0017419258365407586, + 0.0007060368661768734 + ], + "group_id": "ms-7f57045c114ddad153b5", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.11277659982442856, + 0.1479199379682541, + 0.07541074603796005, + 0.042439721524715424, + 0.15517297387123108, + 0.03584214299917221, + 0.1660110056400299, + 0.12549421191215515, + 0.227622389793396, + 0.09924932569265366, + 0.18198737502098083, + 0.11130931228399277, + 0.10854844748973846, + 0.01617596670985222, + 0.08437560498714447, + 0.04182587191462517 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.013166621327400208, + -0.12399804592132568, + 0.48011910915374756, + 0.4925293028354645, + 0.6256281137466431, + 2.213102102279663, + 0.0, + -0.4044480323791504, + 0.1722363382577896, + 0.3389366865158081, + -0.08761294186115265, + 0.21791136264801025, + 0.9396321773529053, + 0.0, + 0.06717276573181152, + -0.5456919074058533, + 0.722014307975769, + -0.051606684923172, + 0.7784175276756287, + 0.5338386297225952, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.9143741130828857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.455221176147461, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.263336420059204 + ], + [ + -0.47406601905822754, + -0.37954479455947876, + 1.123671293258667, + -0.8903511762619019, + 1.5515481233596802, + 3.7488584518432617, + 0.9699987173080444, + -0.23581115901470184, + -0.2149372398853302, + 0.8883978128433228, + 0.5803861021995544, + 1.7047908306121826, + 2.149672746658325, + 2.2741973400115967, + 0.2513730227947235, + 0.6982267498970032, + 2.1354174613952637, + -1.151021122932434, + 1.9550246000289917, + 0.5981679558753967, + 1.338004469871521 + ], + [ + -0.1836935132741928, + -0.42300617694854736, + 1.7664438486099243, + -0.5576877593994141, + 1.0626983642578125, + 2.5276265144348145, + 2.3329849243164062, + -0.4834023118019104, + -0.25722599029541016, + 0.5209662914276123, + 0.24240870773792267, + 0.8533134460449219, + 1.5531291961669922, + 2.304234504699707, + 0.7609332799911499, + 1.0344312191009521, + 2.270921230316162, + -0.926997184753418, + 0.9014018177986145, + 2.259265422821045, + 2.172292470932007 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3910996615886688, + -0.9467402100563049, + 1.4744372367858887, + -0.2748112082481384, + 1.753147840499878, + 3.3321292400360107, + 1.7170560359954834, + 0.09920699894428253, + -0.11250317096710205, + 1.5213767290115356, + 0.0891318991780281, + 1.0193004608154297, + 1.7396382093429565, + 2.202378988265991, + 0.1176290512084961, + 0.5433868765830994, + 1.6113250255584717, + -0.2701912522315979, + 1.7789868116378784, + 1.246854305267334, + 1.6763029098510742 + ], + [ + 0.3669140636920929, + -0.8853410482406616, + 1.1054010391235352, + -0.008406088687479496, + 1.683157205581665, + 2.8090970516204834, + 2.1524839401245117, + -0.1140800267457962, + -0.16068093478679657, + 1.3338439464569092, + -0.34659242630004883, + 1.4494925737380981, + 2.0485284328460693, + 2.244007110595703, + 0.4862309694290161, + 0.6211890578269958, + 1.5708913803100586, + -0.22552455961704254, + 1.7964838743209839, + 1.386763095855713, + 1.3543281555175781 + ], + [ + 0.7718826532363892, + -0.5710152983665466, + 2.151604175567627, + -0.5648711323738098, + 2.1273765563964844, + 3.657623291015625, + 1.9845168590545654, + -0.1599704772233963, + -0.6097310781478882, + 1.135812520980835, + -0.09587362408638, + 1.555975079536438, + 2.3309824466705322, + 1.9832960367202759, + 0.46482378244400024, + 0.6467846035957336, + 1.7182108163833618, + -0.0750286877155304, + 1.9485927820205688, + 0.959426760673523, + 0.9474185109138489 + ] + ], + "predicted_scores": [ + 5.310606956481934, + 5.179131507873535, + 5.123929977416992, + 4.9957756996154785, + 4.984136581420898, + 4.951163291931152, + 4.9353485107421875, + 4.836304664611816 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524", + "selected_index": 0, + "state_hash": "4885f9acf2601b3d2c201bc8e00ef617845ad2a6d08f96d585249345f457be89", + "stored_base_utility": 0.11277659982442856, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0056487321853637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056487321853637695 + }, + "base_progress": 0.0056487321853637695, + "base_success": false, + "base_utility": 0.0056487321853637695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.020783662796020508, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020783662796020508 + }, + { + "contact_quality": null, + "progress": 0.006764829158782959, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006764829158782959 + }, + { + "contact_quality": null, + "progress": 0.007911324501037598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007911324501037598 + }, + { + "contact_quality": null, + "progress": 0.008485655300319195, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008485655300319195 + }, + { + "contact_quality": null, + "progress": 0.012919445522129536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012919445522129536 + }, + { + "contact_quality": null, + "progress": 0.010243098251521587, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010243098251521587 + }, + { + "contact_quality": null, + "progress": 0.005925814155489206, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005925814155489206 + }, + { + "contact_quality": null, + "progress": 0.011702279560267925, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011702279560267925 + } + ], + "candidate_progress": [ + 0.020783662796020508, + 0.006764829158782959, + 0.007911324501037598, + 0.008485655300319195, + 0.012919445522129536, + 0.010243098251521587, + 0.005925814155489206, + 0.011702279560267925 + ], + "candidate_source_chart_ids": [ + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "generated_tangents": [ + [ + -0.9797090291976929, + -0.5788966417312622, + 0.5761638879776001, + -0.4251173734664917, + 0.8595467209815979, + 2.4331881999969482, + 2.6097702980041504, + 0.20743997395038605, + -0.1089591458439827, + 0.5487211346626282, + -0.6175524592399597, + 0.2773182690143585, + 1.6332672834396362, + 0.7114748954772949, + 0.8582735061645508, + -0.3945067524909973, + -1.4017373323440552, + 0.9111741185188293, + 0.6223698854446411, + -0.056299954652786255, + -0.10510754585266113 + ], + [ + -0.6735737323760986, + -0.5392934679985046, + 0.7878432273864746, + -0.6181210279464722, + 0.8703170418739319, + 2.469273805618286, + 2.7541191577911377, + 0.2942640483379364, + 0.007552893366664648, + 0.7389305830001831, + -0.5672551989555359, + 0.3732055425643921, + 1.6452428102493286, + 0.7301114201545715, + 0.8220322728157043, + -0.3352580666542053, + -1.3437676429748535, + 0.9306057691574097, + 0.5657291412353516, + -0.26815271377563477, + -0.3904983699321747 + ], + [ + -0.5472486615180969, + -0.5470925569534302, + 1.0276633501052856, + -0.4575492739677429, + 0.9992425441741943, + 2.3985705375671387, + 3.057952404022217, + 0.2488958090543747, + -0.049797892570495605, + 0.5111393332481384, + -0.6462520956993103, + 0.3110223114490509, + 1.320737361907959, + 1.1416326761245728, + 0.7001436948776245, + -0.3477555811405182, + -1.1263864040374756, + 0.8989945650100708, + 0.8363519906997681, + -0.2654411494731903, + -1.2923177480697632 + ], + [ + -0.6852919459342957, + -0.6334052085876465, + 0.8921171426773071, + -0.6946732997894287, + 0.9066221714019775, + 2.20363712310791, + 3.036403179168701, + 0.3311169147491455, + -0.05185820162296295, + 0.6375935077667236, + -0.44766172766685486, + 0.4774964153766632, + 1.9750186204910278, + 0.5735405087471008, + 0.7067320942878723, + -0.3437744081020355, + -1.3345177173614502, + 0.8593479990959167, + 0.49963071942329407, + 0.052784264087677, + -0.4160422384738922 + ], + [ + -0.7206347584724426, + -0.3388293981552124, + 1.3264834880828857, + -1.1894015073776245, + 1.3139410018920898, + 2.424651861190796, + 2.700961112976074, + 0.4642212390899658, + -0.3942778408527374, + 0.03071877360343933, + -0.7663148641586304, + 0.41479724645614624, + 1.3833167552947998, + 0.9138265252113342, + 0.7195596694946289, + -0.3014485538005829, + -1.6018716096878052, + 1.0623793601989746, + 0.7466589212417603, + -0.04536920785903931, + -1.7899843454360962 + ], + [ + -0.6311175227165222, + -0.35981082916259766, + 1.4574148654937744, + -1.164613127708435, + 1.1249423027038574, + 2.357187509536743, + 2.628722906112671, + 0.5517063736915588, + -0.33276084065437317, + 0.3055139183998108, + -0.6316373944282532, + 0.2753061056137085, + 1.4839552640914917, + 0.6826318502426147, + 0.8178143501281738, + -0.25516846776008606, + -1.5381749868392944, + 0.9670345187187195, + 0.7789851427078247, + -0.31182414293289185, + -1.4498058557510376 + ], + [ + -0.5135475397109985, + -0.21973976492881775, + 1.2926387786865234, + -0.8528269529342651, + 1.0104479789733887, + 2.3066048622131348, + 2.498617172241211, + 0.5369632840156555, + -0.23872369527816772, + 0.3226558268070221, + -0.34781765937805176, + 0.4087653160095215, + 1.2539739608764648, + 1.2790660858154297, + 0.9861194491386414, + -0.2710059583187103, + -1.2331801652908325, + 0.7592136263847351, + 0.9374886751174927, + -0.45170706510543823, + -0.9161456823348999 + ], + [ + -0.8477624654769897, + -0.25506559014320374, + 1.6048439741134644, + -0.9381506443023682, + 1.2483799457550049, + 2.365689754486084, + 2.4276623725891113, + 0.45283880829811096, + -0.4671296179294586, + 0.06111642345786095, + -0.6230047941207886, + 0.23739725351333618, + 1.2195711135864258, + 1.1104482412338257, + 1.0702121257781982, + -0.44958576560020447, + -1.368348479270935, + 0.8212172389030457, + 0.6473431587219238, + -0.1580667793750763, + -1.0589468479156494 + ] + ], + "generated_utilities": [ + 0.020783662796020508, + 0.006764829158782959, + 0.007911324501037598, + 0.008485655300319195, + 0.012919445522129536, + 0.010243098251521587, + 0.005925814155489206, + 0.011702279560267925 + ], + "group_id": "ms-81f4e8858b2ba90aea2e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.01361372135579586, + 0.007307033054530621, + 0.021326780319213867, + 0.04093001410365105, + 0.15059307217597961, + 0.013598322868347168, + 0.09240063279867172, + 0.1631626933813095, + 0.05027441307902336, + 0.05712141841650009, + 0.1538398563861847, + 0.06023097038269043, + 0.020811915397644043, + 0.019836824387311935, + 0.1318017691373825, + 0.051341354846954346 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06500570476055145, + 0.046472612768411636, + 1.2439939975738525, + -1.446747064590454, + 1.7077040672302246, + 2.291205883026123, + 2.1016674041748047, + -0.6354803442955017, + -0.4465000331401825, + 0.9666666984558105, + -1.1514010429382324, + 0.45639365911483765, + 1.613925814628601, + 1.0568119287490845, + 1.3792463541030884, + -0.7852597832679749, + -1.576931118965149, + 1.1258286237716675, + 0.9614027738571167, + -0.9620904922485352, + -1.7528356313705444 + ], + [ + -0.23400194942951202, + -0.21453839540481567, + 1.458961844444275, + -1.701543927192688, + 1.4158955812454224, + 2.2648162841796875, + 2.2270467281341553, + -0.5239131450653076, + -0.44378146529197693, + 1.0904332399368286, + -1.223331093788147, + 0.45784294605255127, + 1.6057316064834595, + 0.9251953363418579, + 1.308751106262207, + -0.6351475715637207, + -1.5950077772140503, + 1.0428481101989746, + 0.6781296730041504, + -0.9007717370986938, + -1.880659818649292 + ], + [ + 0.2324782907962799, + 0.05129373073577881, + 1.1239540576934814, + -1.1853233575820923, + 1.3290393352508545, + 1.933064341545105, + 2.192976951599121, + -0.23718085885047913, + -0.279358446598053, + 1.3028839826583862, + -1.4676742553710938, + 0.16475489735603333, + 1.67984938621521, + 0.6600161790847778, + 1.5654886960983276, + -1.1545368432998657, + -1.5593464374542236, + 1.1976646184921265, + 0.5715436935424805, + -0.9047394394874573, + -1.9614591598510742 + ], + [ + 0.7452290058135986, + 0.614987313747406, + 0.7383754849433899, + -1.5856380462646484, + 1.264894723892212, + 1.7337512969970703, + 2.502504825592041, + 0.040616750717163086, + -1.1382783651351929, + 0.7095704674720764, + -1.014717698097229, + 0.5126742720603943, + 1.3401073217391968, + 1.4275505542755127, + 1.4739655256271362, + -0.7308655381202698, + -0.9416123032569885, + 0.661049485206604, + 0.9453439712524414, + -1.5968079566955566, + -1.715583324432373 + ], + [ + 0.15219944715499878, + 0.5165310502052307, + 1.0654370784759521, + -0.5756344199180603, + 1.2834129333496094, + 1.6370232105255127, + 2.414426326751709, + -1.1256691217422485, + -1.1171015501022339, + 0.5337101817131042, + -0.9850769639015198, + -0.35428309440612793, + 2.4350547790527344, + 1.8563504219055176, + 2.1929399967193604, + -0.1821107268333435, + -1.4600040912628174, + 1.5430631637573242, + 1.4937412738800049, + -0.8152608275413513, + -2.020042896270752 + ] + ], + "predicted_scores": [ + 7.898220062255859, + 7.632265567779541, + 7.578484535217285, + 7.495541572570801, + 7.006176471710205, + 6.1332879066467285, + 5.817028522491455, + 5.645639896392822 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "562e8bc3804d1d310e0eaf612dbc110b8efc94514458dacf551ef33f4f0b8ee7", + "stored_base_utility": 0.01361372135579586, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.00698717450723052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00698717450723052 + }, + "base_progress": 0.00698717450723052, + "base_success": false, + "base_utility": 0.00698717450723052, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06249414011836052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06249414011836052 + }, + { + "contact_quality": null, + "progress": 0.05666528269648552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05666528269648552 + }, + { + "contact_quality": null, + "progress": 0.06223900988698006, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06223900988698006 + }, + { + "contact_quality": null, + "progress": 0.0536121129989624, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0536121129989624 + }, + { + "contact_quality": null, + "progress": 0.07487839460372925, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07487839460372925 + }, + { + "contact_quality": null, + "progress": 0.058169882744550705, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.058169882744550705 + }, + { + "contact_quality": null, + "progress": 0.08651334047317505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08651334047317505 + }, + { + "contact_quality": null, + "progress": 0.052170634269714355, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.052170634269714355 + } + ], + "candidate_progress": [ + 0.06249414011836052, + 0.05666528269648552, + 0.06223900988698006, + 0.0536121129989624, + 0.07487839460372925, + 0.058169882744550705, + 0.08651334047317505, + 0.052170634269714355 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-48178f52a96de3d1ead0", + "ms-45fbd749d5ed3fe37c13", + "ms-9229ce141dd41a11fac1", + "ms-9229ce141dd41a11fac1" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8a72b73ac452bc190aeb", + "generated_tangents": [ + [ + 0.7237734794616699, + -0.3463215231895447, + -0.2813967168331146, + -0.7398946285247803, + 0.4126674234867096, + -2.4756922721862793, + -1.0576188564300537, + 0.20339497923851013, + -0.15946611762046814, + 0.041883088648319244, + 0.3290709853172302, + 0.38147589564323425, + 0.6676110029220581, + -0.22153611481189728, + -0.5712802410125732, + -0.49059584736824036, + 0.556411623954773, + 0.8972428441047668, + 0.23936396837234497, + -0.05750294029712677, + 0.5669397115707397 + ], + [ + 0.5318090915679932, + -0.2204359918832779, + -0.23124584555625916, + -0.6424493789672852, + 0.3152037262916565, + -2.814680337905884, + -0.6020680069923401, + 0.3010162115097046, + -0.2979036867618561, + -0.022432439029216766, + 0.6397415399551392, + -0.01696258783340454, + 0.44693610072135925, + 0.908657968044281, + -0.3844669461250305, + -0.6219145655632019, + 0.6164681315422058, + 0.609011709690094, + 0.45481422543525696, + -0.10028764605522156, + -0.10990327596664429 + ], + [ + 0.4268692135810852, + -0.1830245703458786, + -0.28259745240211487, + -0.6784018278121948, + 0.41127753257751465, + -2.7967660427093506, + -0.5581141114234924, + 0.29672208428382874, + -0.3161696493625641, + -0.02425564080476761, + 0.6270208954811096, + -0.03540460765361786, + 0.4001066982746124, + 0.8597201704978943, + -0.3618530333042145, + -0.6466072201728821, + 0.4462053179740906, + 0.5887701511383057, + 0.4716850519180298, + -0.07090075314044952, + -0.05993114411830902 + ], + [ + 0.42195379734039307, + -0.17336474359035492, + -0.1927911937236786, + -0.6151036024093628, + 0.32705777883529663, + -2.8485512733459473, + -0.4864002466201782, + 0.3123778998851776, + -0.3475083112716675, + -0.043378494679927826, + 0.5556626915931702, + -0.04166272282600403, + 0.49940237402915955, + 0.845285952091217, + -0.3649314343929291, + -0.6653127670288086, + 0.5573483109474182, + 0.6050751209259033, + 0.47031983733177185, + -0.06376247107982635, + -0.06030736863613129 + ], + [ + 0.8675592541694641, + -0.17233529686927795, + -0.1716870367527008, + -0.622383177280426, + 0.4599394202232361, + -2.6053483486175537, + -0.8403750061988831, + 0.08160973340272903, + -0.17525219917297363, + -0.15138384699821472, + 0.9165785312652588, + 0.2648965120315552, + 0.340472549200058, + 0.5224915146827698, + -0.08234900236129761, + -0.4862605929374695, + 0.8968313932418823, + 0.4751356244087219, + 0.7731359004974365, + 0.3415035009384155, + -0.01751391589641571 + ], + [ + 0.22726790606975555, + -0.18214154243469238, + 0.15191107988357544, + -1.019789218902588, + 0.2853069603443146, + -3.1014511585235596, + -0.4352756142616272, + 0.30375105142593384, + -0.2323322594165802, + -0.2173788845539093, + 0.534632682800293, + 0.16938892006874084, + -0.14465510845184326, + -0.04447142779827118, + -0.24358361959457397, + -0.513020396232605, + 0.3160269856452942, + 0.4808957576751709, + 0.6584339141845703, + -0.32977449893951416, + -0.24890568852424622 + ], + [ + 0.6190431118011475, + -0.3089068531990051, + -0.32904091477394104, + -0.6154882907867432, + 0.42806077003479004, + -2.5397346019744873, + -1.0147783756256104, + 0.12143614888191223, + -0.2780408561229706, + -0.153457909822464, + 0.8323661088943481, + 0.2215633988380432, + 0.37340179085731506, + 0.4195871949195862, + -0.7584972381591797, + -0.7267059087753296, + 0.5783241987228394, + 0.7702854871749878, + 0.2830463647842407, + -0.3344297707080841, + 0.25418585538864136 + ], + [ + 0.5480592250823975, + -0.31458598375320435, + -0.47533008456230164, + -0.6371848583221436, + 0.6769095063209534, + -2.691488742828369, + -1.0633811950683594, + 0.1997028887271881, + -0.29389750957489014, + -0.10647069662809372, + 0.8385688066482544, + 0.32284706830978394, + 0.4793175756931305, + 0.4523414969444275, + -0.7055636048316956, + -0.7248621582984924, + 0.5520171523094177, + 0.8804596662521362, + 0.24362146854400635, + -0.1687656044960022, + 0.17645736038684845 + ] + ], + "generated_utilities": [ + 0.06249414011836052, + 0.05666528269648552, + 0.06223900988698006, + 0.0536121129989624, + 0.07487839460372925, + 0.058169882744550705, + 0.08651334047317505, + 0.052170634269714355 + ], + "group_id": "ms-8a72b73ac452bc190aeb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.02298194169998169, + 0.01763375662267208, + 0.009361605159938335, + 0.02526181936264038, + 0.09214292466640472, + 0.022978028282523155, + 0.07704488933086395, + 0.056830525398254395, + 0.045355141162872314, + 0.06095697730779648, + 0.10329639911651611, + 0.05954315513372421, + 0.03003152273595333, + 0.008136491291224957, + 0.05599620193243027, + 0.011116823181509972 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.0913797616958618, + 0.3305599093437195, + -0.0778055489063263, + -2.223818778991699, + 0.9291321635246277, + -2.905466318130493, + -2.4575929641723633, + 1.1732350587844849, + 0.22933529317378998, + -0.8654288053512573, + 2.1122353076934814, + 1.5987088680267334, + 1.144788146018982, + 0.40604642033576965, + 0.10343833267688751, + -0.7513935565948486, + -1.054006814956665, + 1.396103024482727, + -0.16642586886882782, + -0.04196041822433472, + -1.2630513906478882 + ], + [ + 1.3690472841262817, + 0.2738865315914154, + -0.27599766850471497, + -2.431976556777954, + 0.7025187015533447, + -3.155372142791748, + -2.6898910999298096, + 1.1788215637207031, + 0.19074484705924988, + -0.7378551959991455, + 2.3089826107025146, + 1.3478807210922241, + 1.3458092212677002, + 0.3417958617210388, + 0.24155153334140778, + -1.004831314086914, + -1.3499912023544312, + 1.311987042427063, + -0.25646817684173584, + -0.3180008828639984, + -1.3290650844573975 + ], + [ + 0.9032705426216125, + -0.3011789321899414, + -0.21911907196044922, + -2.884286403656006, + 0.319735586643219, + -2.613290548324585, + -2.175532817840576, + 1.5185720920562744, + -0.1741684526205063, + -0.4879068434238434, + 2.721827507019043, + 2.1753995418548584, + 0.8214534521102905, + 0.7874165177345276, + 0.013686276972293854, + -0.7485443949699402, + -0.764183521270752, + 0.9452257752418518, + 0.07210072875022888, + 0.2788098156452179, + -1.8349425792694092 + ] + ], + "predicted_scores": [ + 4.453597068786621, + 4.2043070793151855, + 4.053958892822266, + 4.031412601470947, + 3.84814715385437, + 3.5510828495025635, + 3.0950734615325928, + 3.057717800140381 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "722240dc5e14c5a30badc5fbec9f15ecf388cf6402fb77013c8f9dbdcd100846", + "stored_base_utility": 0.02298194169998169, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08285123109817505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08285123109817505 + }, + "base_progress": 0.08285123109817505, + "base_success": false, + "base_utility": 0.08285123109817505, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.08203518390655518, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08203518390655518 + }, + { + "contact_quality": null, + "progress": 0.09714335203170776, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09714335203170776 + }, + { + "contact_quality": null, + "progress": 0.0648718848824501, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0648718848824501 + }, + { + "contact_quality": null, + "progress": 0.06826772540807724, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06826772540807724 + }, + { + "contact_quality": null, + "progress": 0.03956645727157593, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03956645727157593 + }, + { + "contact_quality": null, + "progress": 0.042625367641448975, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.042625367641448975 + }, + { + "contact_quality": null, + "progress": 0.04983709380030632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04983709380030632 + }, + { + "contact_quality": null, + "progress": 0.05635470151901245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05635470151901245 + } + ], + "candidate_progress": [ + 0.08203518390655518, + 0.09714335203170776, + 0.0648718848824501, + 0.06826772540807724, + 0.03956645727157593, + 0.042625367641448975, + 0.04983709380030632, + 0.05635470151901245 + ], + "candidate_source_chart_ids": [ + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-a96bca1ef0b74f686018", + "generated_tangents": [ + [ + 0.5346310138702393, + 0.10992571711540222, + 0.6128008961677551, + -0.2714889645576477, + 0.3584376573562622, + 0.25554028153419495, + 0.1413051187992096, + 0.11463594436645508, + 0.011281419545412064, + -0.11737047880887985, + -0.9030520915985107, + -0.049557045102119446, + -0.29371047019958496, + -0.6933405995368958, + -0.3945266306400299, + 0.11014603078365326, + 0.2626229524612427, + -0.27674490213394165, + -0.7403897643089294, + -0.3901497721672058, + -0.7334052324295044 + ], + [ + 0.14372357726097107, + 0.04398868978023529, + 0.5261341333389282, + -0.2815788984298706, + 0.6882809400558472, + 0.42881765961647034, + -0.21752256155014038, + 0.15735551714897156, + 0.04994235932826996, + -0.037333764135837555, + -0.8268375396728516, + -0.032546281814575195, + -0.6295778751373291, + -0.45782583951950073, + -0.402691513299942, + 0.07683300971984863, + 0.39632636308670044, + -0.23021282255649567, + -0.7886610627174377, + -0.5787997245788574, + -0.6151546239852905 + ], + [ + 0.16368357837200165, + 0.08141957223415375, + 0.7052232623100281, + -0.5786411762237549, + 0.8425356149673462, + 0.6810296773910522, + 0.21797403693199158, + 0.3181007206439972, + 0.1156037375330925, + 0.12721092998981476, + -0.9957047700881958, + -0.3546638488769531, + -0.9460336565971375, + -1.3903577327728271, + -0.4220317304134369, + 0.15521006286144257, + 0.5781416296958923, + -0.23526884615421295, + -1.3471155166625977, + -1.1188604831695557, + -0.5476807355880737 + ], + [ + 0.22467413544654846, + 0.024278812110424042, + 0.8184769749641418, + -0.648155927658081, + 0.848567008972168, + 0.735680341720581, + 0.06917771697044373, + 0.2127572000026703, + 0.13531038165092468, + 0.08184245228767395, + -1.00473952293396, + -0.3373028635978699, + -0.866546630859375, + -1.5128990411758423, + -0.36625927686691284, + 0.14370355010032654, + 0.5097633004188538, + -0.30661845207214355, + -1.2713730335235596, + -1.129964828491211, + -0.5226327180862427 + ], + [ + 0.1036408394575119, + -0.011204499751329422, + 0.9784748554229736, + -0.3923414349555969, + 1.0034300088882446, + 1.2108299732208252, + 0.0016484856605529785, + 0.03118811547756195, + 0.0231051966547966, + 0.26200950145721436, + -0.9888348579406738, + -0.37069010734558105, + -0.6912773847579956, + -0.6350822448730469, + -0.3739113211631775, + 0.08453267812728882, + 0.1155906394124031, + -0.18932513892650604, + -1.0536084175109863, + -0.9525995850563049, + -0.4120785892009735 + ], + [ + 0.19068212807178497, + -0.002145826816558838, + 1.1100620031356812, + -0.543897807598114, + 0.7654709815979004, + 0.928135871887207, + 0.01588571071624756, + 0.2787608206272125, + 0.03331562876701355, + 0.13259316980838776, + -0.9413942098617554, + -0.4548082947731018, + -0.7601237893104553, + -1.2860647439956665, + -0.5193300247192383, + 0.05155609920620918, + 0.5452077388763428, + -0.17883668839931488, + -1.3284802436828613, + -0.9959831833839417, + -0.5929776430130005 + ], + [ + 0.04121547192335129, + 0.31499946117401123, + 0.8823180198669434, + -0.37555986642837524, + 1.2201275825500488, + 0.557954728603363, + 0.321776807308197, + 0.3943336009979248, + 0.13487793505191803, + 0.1472679078578949, + -1.0082677602767944, + -0.1733517348766327, + -0.7004083395004272, + -1.5310494899749756, + -0.31477823853492737, + -0.015043312683701515, + 0.7494775652885437, + -0.4257533550262451, + -1.37750244140625, + -1.2458014488220215, + -0.699881911277771 + ], + [ + 0.3205135464668274, + -0.0065469890832901, + 0.8198845386505127, + -0.42302414774894714, + 0.42646801471710205, + 0.5846861600875854, + 0.21706122159957886, + 0.26053255796432495, + -0.09887508302927017, + -0.2193397581577301, + -0.8402920961380005, + -0.2699149549007416, + -0.8194273710250854, + -0.502998948097229, + -0.6821227669715881, + -0.03476265072822571, + 0.7032755017280579, + 0.36467689275741577, + -1.1770315170288086, + -1.1577614545822144, + -1.343900442123413 + ] + ], + "generated_utilities": [ + 0.08203518390655518, + 0.09714335203170776, + 0.0648718848824501, + 0.06826772540807724, + 0.03956645727157593, + 0.042625367641448975, + 0.04983709380030632, + 0.05635470151901245 + ], + "group_id": "ms-a96bca1ef0b74f686018", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07923875749111176, + 0.1507580727338791, + 0.09708476066589355, + 0.1572066992521286, + 0.09626718610525131, + 0.0781460627913475, + 0.07252912223339081, + 0.04244875907897949, + 0.026137471199035645, + 0.03892441838979721, + 0.04886416718363762, + 0.10530619323253632, + 0.0477486252784729, + 0.0898323506116867, + 0.01968318223953247, + 0.0289560966193676 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.48996758460998535, + 0.9053895473480225, + 1.8985445499420166, + -1.3594893217086792, + 1.6097791194915771, + 1.132408618927002, + 1.8319320678710938, + 0.46463528275489807, + 0.13208985328674316, + 0.8792686462402344, + -1.868321418762207, + 1.1490275859832764, + -2.0193891525268555, + -2.0637927055358887, + 0.11448347568511963, + 1.2957178354263306, + 1.293219804763794, + -1.7789109945297241, + -0.4312591254711151, + -1.7441130876541138, + -0.06536102294921875 + ], + [ + 0.03654795140028, + 1.7676925659179688, + 1.048339605331421, + -1.366391658782959, + 2.5093846321105957, + 2.160305976867676, + 2.8675124645233154, + -0.309950590133667, + -0.5841768980026245, + 1.3030188083648682, + -1.2228018045425415, + 1.1373907327651978, + -2.831526041030884, + -2.2111129760742188, + -0.14044174551963806, + 1.660491704940796, + 1.5215414762496948, + -1.0137104988098145, + -0.40579986572265625, + -1.9571893215179443, + 1.241256833076477 + ], + [ + -0.2826732099056244, + 2.2520718574523926, + 2.418309450149536, + -1.7572941780090332, + 1.5832278728485107, + 0.6361246705055237, + 2.2009639739990234, + 0.9109495878219604, + 0.22027403116226196, + 0.7075455188751221, + -1.2762477397918701, + 0.8630691766738892, + -1.2986351251602173, + -2.567821979522705, + -0.4690782427787781, + 2.1363346576690674, + 0.1853562593460083, + -1.541505217552185, + -0.033681899309158325, + -1.0161254405975342, + 0.06877446174621582 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03534998744726181, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.3504420518875122, + 0.0, + -0.013733148574829102, + 0.021970629692077637, + 0.014389097690582275, + -0.7432765960693359, + 0.006671249866485596, + -0.9755387306213379, + 0.0, + -0.027152668684720993, + 0.2708500623703003, + 0.052558183670043945, + -0.7615737915039062, + 0.020878344774246216, + -0.2927910089492798, + 0.0 + ], + [ + -0.1400495320558548, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.5714048147201538, + 0.0, + -0.3271818161010742, + -0.04413123428821564, + 0.16598933935165405, + -0.7432765960693359, + 0.2282017469406128, + -0.9755387306213379, + 0.0, + -0.4382812976837158, + 0.2708500623703003, + 0.2094714641571045, + -0.7615737915039062, + 0.23577484488487244, + -0.2927910089492798, + 0.0 + ] + ], + "predicted_scores": [ + 4.269467353820801, + 3.8671813011169434, + 3.827446937561035, + 3.6798555850982666, + 3.2768988609313965, + 2.685136318206787, + 2.673711061477661, + 1.6891785860061646 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "61714505b9ac6cbb1c6deff698bb3e96fd54c7cdb8f0d4307bfdd236634525c0", + "stored_base_utility": 0.07923875749111176, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1380254477262497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1380254477262497 + }, + { + "contact_quality": null, + "progress": 0.06922783702611923, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06922783702611923 + }, + { + "contact_quality": null, + "progress": 0.102073073387146, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.102073073387146 + }, + { + "contact_quality": null, + "progress": 0.12990880012512207, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12990880012512207 + }, + { + "contact_quality": null, + "progress": 0.11760101467370987, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11760101467370987 + }, + { + "contact_quality": null, + "progress": 0.11701671034097672, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11701671034097672 + }, + { + "contact_quality": null, + "progress": 0.14365969598293304, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14365969598293304 + }, + { + "contact_quality": null, + "progress": 0.10479718446731567, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10479718446731567 + } + ], + "candidate_progress": [ + 0.1380254477262497, + 0.06922783702611923, + 0.102073073387146, + 0.12990880012512207, + 0.11760101467370987, + 0.11701671034097672, + 0.14365969598293304, + 0.10479718446731567 + ], + "candidate_source_chart_ids": [ + "ms-37384bede4d1c03f3b80", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-ac4fb33f3ab4db853995", + "generated_tangents": [ + [ + 1.8757209777832031, + -0.2798987627029419, + -0.491509348154068, + 0.037609606981277466, + -0.8015756011009216, + 2.0208706855773926, + 1.6965241432189941, + 1.9253427982330322, + 0.4048178791999817, + -1.3228524923324585, + -0.4634227454662323, + -1.3139301538467407, + -2.516993284225464, + -3.13143253326416, + 1.5752646923065186, + 0.16432787477970123, + -0.923065721988678, + 0.17932958900928497, + -0.18268033862113953, + -1.8566339015960693, + -1.7394579648971558 + ], + [ + 1.877700686454773, + -0.27677154541015625, + -0.2079942524433136, + -0.09529922902584076, + -0.356905460357666, + 2.6387462615966797, + 1.9372978210449219, + 2.0541677474975586, + 0.4495166540145874, + -0.9983639717102051, + -0.34587231278419495, + -0.8706223964691162, + -2.940394639968872, + -1.9517396688461304, + 1.815815806388855, + 0.4263441562652588, + -0.6645351648330688, + 0.3294494152069092, + 0.1476365178823471, + -1.818439245223999, + -1.324389934539795 + ], + [ + 1.7460854053497314, + -0.14227360486984253, + -0.44563987851142883, + -0.10550718754529953, + -0.7599748969078064, + 2.783186912536621, + 1.4664199352264404, + 2.082426071166992, + 0.38119378685951233, + -1.1542381048202515, + -0.32885488867759705, + -1.1708210706710815, + -2.741654396057129, + -2.316239833831787, + 1.7502933740615845, + 0.27585113048553467, + -0.762572169303894, + 0.3741384446620941, + 0.0003533363342285156, + -1.8107967376708984, + -1.5808926820755005 + ], + [ + 1.9009313583374023, + -0.11730101704597473, + -0.517284631729126, + 0.05539482831954956, + -0.6659588813781738, + 2.6490137577056885, + 1.847862720489502, + 2.114100933074951, + 0.42314252257347107, + -1.0808172225952148, + -0.45225653052330017, + -1.3092527389526367, + -2.716824769973755, + -2.6231470108032227, + 1.593016266822815, + 0.22196482121944427, + -0.7252891659736633, + 0.31342220306396484, + -0.057871073484420776, + -1.9106976985931396, + -1.616092562675476 + ], + [ + 1.8682342767715454, + -0.1502419114112854, + -0.4085184633731842, + -0.021281033754348755, + -0.5494136214256287, + 2.4912312030792236, + 2.0649428367614746, + 2.020284652709961, + 0.4087589979171753, + -0.9137766361236572, + -0.5366708636283875, + -1.094678282737732, + -2.9595882892608643, + -2.596893310546875, + 1.6457347869873047, + 0.20525941252708435, + -0.6062728762626648, + 0.3353874087333679, + 0.02441948652267456, + -1.6362388134002686, + -1.350596308708191 + ], + [ + 1.9847166538238525, + -0.15845981240272522, + -0.395762175321579, + 0.08717381954193115, + -0.3476361632347107, + 2.264495849609375, + 2.0956692695617676, + 2.025761365890503, + 0.4366755485534668, + -1.0068161487579346, + -0.5347964763641357, + -1.0649518966674805, + -2.9739842414855957, + -2.2935690879821777, + 1.5841917991638184, + 0.1982099413871765, + -0.6307572722434998, + 0.3693506717681885, + -0.1537345051765442, + -1.7523667812347412, + -1.5008445978164673 + ], + [ + 1.9415417909622192, + -0.13299265503883362, + -0.6431517601013184, + 0.1701856553554535, + -0.5841852426528931, + 2.8527026176452637, + 1.545745849609375, + 2.0428261756896973, + 0.34041354060173035, + -1.0155531167984009, + -0.48445674777030945, + -0.9317442178726196, + -2.4904329776763916, + -1.8709954023361206, + 1.6128383874893188, + 0.28925085067749023, + -0.7339987754821777, + 0.13303901255130768, + -0.27472424507141113, + -1.367954969406128, + -1.1774617433547974 + ], + [ + 2.4350316524505615, + -0.1626695990562439, + -0.07594266533851624, + 0.18432152271270752, + -0.015824690461158752, + 1.9474067687988281, + 2.2089436054229736, + 2.021188974380493, + 0.19317109882831573, + -1.33009934425354, + -0.6722767353057861, + -0.8716684579849243, + -2.8951737880706787, + -1.9773768186569214, + 1.5057123899459839, + -0.029739627614617348, + -0.5092992782592773, + 0.4653567671775818, + -0.3313637971878052, + -1.94559645652771, + -1.1394827365875244 + ] + ], + "generated_utilities": [ + 0.1380254477262497, + 0.06922783702611923, + 0.102073073387146, + 0.12990880012512207, + 0.11760101467370987, + 0.11701671034097672, + 0.14365969598293304, + 0.10479718446731567 + ], + "group_id": "ms-ac4fb33f3ab4db853995", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.2651830017566681, + 0.22676818072795868, + 0.22454051673412323, + 2.0, + 0.31214088201522827, + 0.21535459160804749, + 0.18962346017360687, + 2.0, + 0.1830430030822754, + 0.15030252933502197, + 0.12050267308950424, + 0.17022761702537537, + 0.11389490216970444, + 0.07058519124984741, + 0.07076358795166016, + 0.18658748269081116 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 2.0962133407592773, + 0.057441189885139465, + -0.2733909487724304, + 0.4170292317867279, + -0.7772165536880493, + 2.858414888381958, + 2.1098427772521973, + 1.8366810083389282, + 0.017507493495941162, + -1.6127345561981201, + -0.39152991771698, + -1.017486333847046, + -2.815659999847412, + -2.787520408630371, + 1.6748408079147339, + 0.2986273169517517, + -0.6001855134963989, + 0.5475705862045288, + -0.28012242913246155, + -1.5382447242736816, + -1.6652274131774902 + ], + [ + 2.1338062286376953, + -0.07656055688858032, + -0.7531881332397461, + 0.48830646276474, + -0.36628106236457825, + 2.381558418273926, + 1.7315943241119385, + 1.838928461074829, + -0.050897032022476196, + -1.171897292137146, + -0.8350631594657898, + -0.8261303901672363, + -2.2635834217071533, + -3.2190167903900146, + 1.3416705131530762, + 0.44279617071151733, + -0.7299901247024536, + 1.2929750680923462, + -0.0032969415187835693, + -1.5000369548797607, + -1.6060972213745117 + ], + [ + 1.5521352291107178, + 0.03659695386886597, + -0.44382959604263306, + 0.21355122327804565, + -1.0471938848495483, + 3.194620370864868, + 1.9892739057540894, + 2.5084128379821777, + 0.531404972076416, + -0.9168282747268677, + -0.49052953720092773, + -1.1588059663772583, + -3.156189441680908, + -3.278008460998535, + 0.921509325504303, + 0.49181532859802246, + -1.0228296518325806, + 0.5866082310676575, + -0.4882396459579468, + -0.7946261763572693, + -2.119061231613159 + ], + [ + 2.7138938903808594, + -0.3406146764755249, + -0.32174432277679443, + 0.9962238669395447, + -0.558576226234436, + 2.049264907836914, + 1.8703808784484863, + 2.113438367843628, + 0.9615883827209473, + -1.9615708589553833, + -0.0062177181243896484, + -0.5808464288711548, + -2.1758205890655518, + -2.219890594482422, + 1.022050142288208, + 0.7201904654502869, + -1.3177021741867065, + 0.4089192748069763, + -0.0315951406955719, + -0.7215378880500793, + -2.069262981414795 + ], + [ + 2.5701913833618164, + 0.3137266933917999, + -0.5592924356460571, + 0.0026398897171020508, + -0.19283491373062134, + 2.2903807163238525, + 2.637770652770996, + 2.5494186878204346, + 0.9258015155792236, + -0.7146614193916321, + -0.686233639717102, + -0.7985042929649353, + -3.431988000869751, + -3.2178471088409424, + 1.319100260734558, + -0.15109702944755554, + -0.2694774866104126, + 1.0912322998046875, + -0.6380462050437927, + -2.0780839920043945, + -2.1209774017333984 + ], + [ + 2.6195321083068848, + -0.2646539509296417, + 0.15159690380096436, + 0.8749752044677734, + -1.1848887205123901, + 2.8789353370666504, + 1.6037814617156982, + 1.3343384265899658, + -0.2254078984260559, + -2.041776657104492, + -0.9811336994171143, + -1.3441768884658813, + -2.2502636909484863, + -1.9091612100601196, + 0.9959704279899597, + 0.42473238706588745, + -0.5330828428268433, + 1.6945891380310059, + 0.4069111943244934, + -0.5568764805793762, + -1.472819447517395 + ], + [ + 2.9485361576080322, + -0.6907311677932739, + -0.057036399841308594, + 1.2457408905029297, + 0.09488356113433838, + 2.6509125232696533, + 2.0378005504608154, + 2.278454303741455, + 1.1734973192214966, + -1.853393316268921, + 0.09614759683609009, + -0.35674989223480225, + -2.1597909927368164, + -1.9672894477844238, + 1.9760816097259521, + -0.47744297981262207, + 0.021782994270324707, + 1.6328153610229492, + -0.8569514751434326, + -1.0814869403839111, + -1.7780723571777344 + ], + [ + 2.1897926330566406, + -0.3883178234100342, + -0.6492674946784973, + 1.3223958015441895, + 0.30954307317733765, + 2.031817674636841, + 1.878535270690918, + 2.5030624866485596, + 0.6644855737686157, + -1.7382179498672485, + -1.1862120628356934, + -0.7967990636825562, + -2.8425841331481934, + -2.3635308742523193, + 2.0061349868774414, + -0.6015456318855286, + -1.429253339767456, + 1.2773160934448242, + 0.04122643172740936, + -2.0497689247131348, + -1.1791472434997559 + ], + [ + 1.586853265762329, + 0.06395213305950165, + -0.8944701552391052, + 0.5118889808654785, + -1.1310638189315796, + 2.865907669067383, + 1.3295278549194336, + 1.0862370729446411, + 0.6303088665008545, + -0.3754369616508484, + -0.04129624366760254, + -0.7044278979301453, + -2.923245429992676, + -1.7837438583374023, + 1.5932514667510986, + 1.2467985153198242, + 0.13401156663894653, + 1.6706414222717285, + -0.816297709941864, + -1.2893165349960327, + -1.0985345840454102 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.1383795738220215, + -0.1483764797449112, + -0.27178800106048584, + 0.26738643646240234, + -0.12845781445503235, + 1.6495440006256104, + 0.0, + 0.9713280200958252, + -0.050046056509017944, + -0.36543774604797363, + -0.022124826908111572, + 0.08494043350219727, + -1.671510934829712, + 0.0, + 0.5271386504173279, + 0.3898613452911377, + 0.21280205249786377, + -0.2029280662536621, + -0.01802508533000946, + -0.35994601249694824, + 0.0 + ], + [ + 1.7570452690124512, + 0.07137972116470337, + 0.02318662405014038, + 0.2907020151615143, + -0.2308352291584015, + 2.89892578125, + 2.537220001220703, + 2.238605499267578, + 0.16207315027713776, + -1.6996564865112305, + -0.8050198554992676, + -1.2014062404632568, + -2.5764517784118652, + -2.393839120864868, + 1.1276613473892212, + 0.31928181648254395, + -1.036102533340454, + 1.1266506910324097, + -0.1944657564163208, + -1.5998979806900024, + -2.151465892791748 + ] + ], + "predicted_scores": [ + 4.508659362792969, + 4.334898948669434, + 4.305202007293701, + 4.188601493835449, + 4.163142204284668, + 3.8224165439605713, + 3.804182529449463, + 3.057375431060791 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "0b62c42c3beda0eb959692d55792514223681b522edd0a5e73247682cf0bfe95", + "stored_base_utility": 0.2651830017566681, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-fbd73493bcf73542f735", + "ms-fbd73493bcf73542f735" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "generated_tangents": [ + [ + 1.5776278972625732, + -0.1062684953212738, + -1.2771344184875488, + -0.49981874227523804, + 0.2987932562828064, + -0.07345153391361237, + -0.3306434750556946, + 0.6023690104484558, + -0.031822457909584045, + -0.06422949582338333, + -0.8920311331748962, + -0.25830867886543274, + -3.039278984069824, + -2.0247509479522705, + 0.04372836649417877, + 0.1492304801940918, + 0.49795687198638916, + -0.8894179463386536, + 0.9733495116233826, + -2.2327277660369873, + -1.1248815059661865 + ], + [ + 1.6623307466506958, + -0.02801577001810074, + -1.2038532495498657, + -0.416922390460968, + -0.03998517990112305, + -0.010470360517501831, + -0.17300570011138916, + 0.550474226474762, + -0.02714109607040882, + -0.04892220348119736, + -0.7027192711830139, + -0.3052824139595032, + -3.2295403480529785, + -2.228353500366211, + -0.1067059189081192, + 0.07792852818965912, + 0.47764676809310913, + -0.8404160141944885, + 0.9186916947364807, + -2.3990609645843506, + -1.2066744565963745 + ], + [ + 1.656821370124817, + -0.0799698457121849, + -1.4930826425552368, + -0.01621711254119873, + -0.22033074498176575, + 0.0406893715262413, + -1.0481154918670654, + 0.41119152307510376, + 0.033923156559467316, + -0.2097930908203125, + -0.40410226583480835, + -0.000824272632598877, + -2.937533378601074, + -2.3680219650268555, + -0.1172158420085907, + 0.029197171330451965, + 0.5653212070465088, + -1.001155138015747, + 1.157625436782837, + -2.386183023452759, + -1.1381199359893799 + ], + [ + 1.6274237632751465, + -0.06096906214952469, + -1.1917328834533691, + -0.09866362065076828, + -0.43721234798431396, + 0.2296428680419922, + -0.4563702344894409, + 0.37857958674430847, + 0.05558808147907257, + 0.02650347352027893, + -0.5547478795051575, + -0.15131524205207825, + -3.2839534282684326, + -2.348494291305542, + -0.15734140574932098, + 0.17000021040439606, + 0.7693645358085632, + -0.7704904675483704, + 0.860443651676178, + -2.102745294570923, + -1.4043538570404053 + ], + [ + 1.6805323362350464, + -0.10195273160934448, + -1.293046236038208, + -0.20656415820121765, + -0.23258602619171143, + 0.06793885678052902, + -0.46606773138046265, + 0.4363333582878113, + -0.003076324239373207, + -0.040726251900196075, + -0.5399624705314636, + -0.11603021621704102, + -3.1387341022491455, + -2.31392240524292, + -0.04789737984538078, + 0.05541032925248146, + 0.5314755439758301, + -0.8912479281425476, + 0.912960946559906, + -2.218932628631592, + -1.2900742292404175 + ], + [ + 1.548425555229187, + -0.29463839530944824, + -1.0629558563232422, + -0.6135776042938232, + -0.1539422869682312, + -0.18624693155288696, + -1.1895885467529297, + 0.4904552102088928, + 0.17714644968509674, + -0.40043675899505615, + -0.40535709261894226, + -0.14600098133087158, + -2.7475781440734863, + -2.0295424461364746, + -0.2928716540336609, + 0.1679936647415161, + 0.9342039823532104, + -0.6481351256370544, + 1.1030620336532593, + -2.437485456466675, + -1.411930799484253 + ], + [ + 1.6877155303955078, + -0.033127088099718094, + -0.8584600687026978, + -0.12578636407852173, + -0.3914439082145691, + -0.6135951280593872, + -0.2536333203315735, + 0.5843752026557922, + 0.07948585599660873, + -0.23855450749397278, + -0.49482861161231995, + -0.3175610303878784, + -2.8263332843780518, + -1.8088093996047974, + -0.07201462984085083, + 0.061782415956258774, + 0.5911457538604736, + -0.8975171446800232, + 0.8092808127403259, + -2.417825937271118, + -1.2020542621612549 + ], + [ + 1.8183302879333496, + -0.07024002820253372, + -1.0422594547271729, + -0.10619249194860458, + -0.06513074040412903, + -0.27954554557800293, + -0.9251890778541565, + 0.3007574677467346, + 0.09522706270217896, + 0.02375529706478119, + -0.23301813006401062, + -0.11444693803787231, + -2.2615537643432617, + -1.7828730344772339, + -0.1611703336238861, + 0.018185578286647797, + 0.6350998282432556, + -0.7584947943687439, + 0.9100536108016968, + -2.5712146759033203, + -1.1600972414016724 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.149173304438591, + 0.06221481412649155, + 0.1385849118232727, + 0.06656398624181747, + 2.0, + 0.10397814214229584, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7398372888565063, + -0.02836926281452179, + -0.4292973279953003, + 0.028730571269989014, + 0.009982943534851074, + -0.3464317321777344, + 0.0, + 0.009800434112548828, + -0.06816069781780243, + 0.030653484165668488, + -0.00554502010345459, + -0.06216755136847496, + -1.9802618026733398, + 0.0, + 0.01717478036880493, + -0.05601567029953003, + 0.010390102863311768, + -0.8072932958602905, + -0.05775028467178345, + -1.9342114925384521, + 0.0 + ], + [ + 0.7398372888565063, + -0.1961192637681961, + -0.4292973279953003, + -0.0017886161804199219, + 0.06461763381958008, + -0.3464317321777344, + 0.0, + 0.42972221970558167, + -0.5797654986381531, + 0.19436834752559662, + -0.23844504356384277, + 0.1733923852443695, + -1.9802618026733398, + 0.0, + -0.27269166707992554, + 0.04145494103431702, + 0.12319430708885193, + -0.8072932958602905, + 0.4949256181716919, + -1.9342114925384521, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7398372888565063, + 0.2242223471403122, + -1.4292973279953003, + -0.537040650844574, + -0.9353823661804199, + -1.3464317321777344, + -2.159137487411499, + 0.489885151386261, + -0.006211067084223032, + 0.09238272905349731, + -1.0055450201034546, + -0.04778099060058594, + -2.98026180267334, + -2.576570987701416, + 0.2992324233055115, + 0.34874510765075684, + 0.5165960192680359, + -1.8072932958602905, + 0.834364652633667, + -2.934211492538452, + -2.0790040493011475 + ], + [ + 1.9740948677062988, + 0.4187958538532257, + -1.2225801944732666, + -0.6112513542175293, + -1.0962133407592773, + -1.4875872135162354, + -2.1883857250213623, + 0.6603437662124634, + -0.20142723619937897, + 0.1697872281074524, + -0.8377580046653748, + -0.16444604098796844, + -2.87237286567688, + -2.4153482913970947, + 0.03898555040359497, + 0.5175161957740784, + 0.7254846096038818, + -1.9598647356033325, + 1.1053451299667358, + -3.172945976257324, + -1.9334450960159302 + ], + [ + 1.5938913822174072, + -0.018363401293754578, + -1.1851862668991089, + -0.20344951748847961, + -0.9226735234260559, + -1.3553601503372192, + -1.932102084159851, + 0.6446930170059204, + 0.30503973364830017, + 0.3522501289844513, + -0.9527953267097473, + -0.15980565547943115, + -2.9574379920959473, + -2.9056527614593506, + 0.6069583892822266, + 0.3722725808620453, + 0.7255810499191284, + -1.8503519296646118, + 0.5613387823104858, + -3.095244884490967, + -2.023747444152832 + ], + [ + 1.6626678705215454, + -0.02834661304950714, + -1.0125243663787842, + -0.5629516839981079, + -1.230878233909607, + -1.414373755455017, + -2.42856764793396, + 0.8520264625549316, + -0.1296539008617401, + -0.22410845756530762, + -1.0539721250534058, + -0.060277245938777924, + -2.740527391433716, + -2.1194698810577393, + 0.6826534867286682, + 0.7535779476165771, + 0.35573869943618774, + -1.9124289751052856, + 0.7101565003395081, + -3.326658010482788, + -1.8195855617523193 + ], + [ + 2.2881288528442383, + 0.6951426863670349, + -1.0978585481643677, + -0.008003771305084229, + -1.0754073858261108, + -1.7734596729278564, + -2.139570474624634, + 0.9205271601676941, + 0.29560548067092896, + 0.49158430099487305, + -1.2606573104858398, + 0.43956148624420166, + -2.831833600997925, + -2.960129976272583, + 0.6832801103591919, + -0.03894898295402527, + 1.036036491394043, + -2.127277135848999, + 0.6759494543075562, + -3.1690073013305664, + -1.9895020723342896 + ], + [ + 1.9020261764526367, + 0.6575945019721985, + -1.3994193077087402, + -0.5040701031684875, + -0.6857678890228271, + -0.7056208252906799, + -1.5344107151031494, + 0.7615087032318115, + 0.6105715036392212, + 0.19082021713256836, + -0.6303516626358032, + -0.4614318609237671, + -2.3375303745269775, + -2.188478469848633, + 0.6886799335479736, + -0.03237125277519226, + 0.483882874250412, + -1.3982734680175781, + 0.23996800184249878, + -3.3969180583953857, + -1.5116117000579834 + ], + [ + 1.666227102279663, + 0.6915197968482971, + -1.6392340660095215, + -0.9650112390518188, + -0.2591104507446289, + -1.5417896509170532, + -1.695828914642334, + 1.130251169204712, + -0.555443525314331, + -0.4872768521308899, + -1.0994479656219482, + 0.5934816002845764, + -3.5218958854675293, + -1.8239338397979736, + 0.7150859832763672, + 0.8003170490264893, + 1.0113730430603027, + -2.1905770301818848, + 0.8595032095909119, + -3.039506673812866, + -2.3675880432128906 + ], + [ + 1.59856116771698, + -0.036071136593818665, + -1.700652837753296, + 0.27602267265319824, + -1.608276128768921, + -0.875938355922699, + -1.6728798151016235, + 0.8851535320281982, + 0.7461313605308533, + -0.7497223019599915, + -1.0928030014038086, + -0.4735495150089264, + -2.223254680633545, + -2.5322465896606445, + 0.044906944036483765, + 0.6160990595817566, + 1.3697642087936401, + -1.6367510557174683, + 0.086475670337677, + -3.323115825653076, + -2.058189630508423 + ], + [ + 2.285855531692505, + 0.6314746141433716, + -1.2119637727737427, + 0.039707839488983154, + -0.9751565456390381, + -1.7796330451965332, + -1.5261995792388916, + 0.5450726747512817, + -0.8933275938034058, + 0.7903657555580139, + -1.5466060638427734, + 0.5994364023208618, + -3.370340585708618, + -3.1641077995300293, + 0.3225497603416443, + -0.39524197578430176, + 1.152822732925415, + -1.508494257926941, + 1.1221402883529663, + -3.361729860305786, + -1.9251339435577393 + ], + [ + 2.2599079608917236, + -0.005553290247917175, + -2.4053664207458496, + 0.39961689710617065, + -0.40980857610702515, + -0.9905421733856201, + -1.7244036197662354, + -0.06417369842529297, + 0.6376181840896606, + 0.5168956518173218, + -1.6740490198135376, + -0.1833474189043045, + -3.9368975162506104, + -1.7146861553192139, + 0.9366075992584229, + 0.701168417930603, + -0.23131346702575684, + -2.740941286087036, + 0.908729076385498, + -3.6231396198272705, + -1.3161351680755615 + ], + [ + 1.819408655166626, + -0.5504165291786194, + -1.302573800086975, + -0.6927691698074341, + -0.7548186779022217, + -2.1824703216552734, + -2.4921669960021973, + 0.6337187886238098, + 0.1691494882106781, + 0.7547447681427002, + -0.4371253252029419, + -0.6562469601631165, + -2.313655376434326, + -2.4974589347839355, + 0.05105690658092499, + 0.3473445773124695, + -0.4244192838668823, + -1.8233305215835571, + -0.1157217025756836, + -2.0408761501312256, + -2.2010600566864014 + ] + ], + "predicted_scores": [ + 7.347308158874512, + 7.290104866027832, + 7.171411991119385, + 6.953240871429443, + 6.723456382751465, + 6.7069830894470215, + 5.547750949859619, + 5.501445293426514 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2054899126", + "selected_index": 0, + "state_hash": "b26c5f64bc7e877098323281f2b58e7850275b30405bc411ed7ffc7cf69f4e26", + "stored_base_utility": 0.149173304438591, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.037537217140197754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037537217140197754 + }, + "base_progress": 0.037537217140197754, + "base_success": false, + "base_utility": 0.037537217140197754, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01185232400894165, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01185232400894165 + }, + { + "contact_quality": null, + "progress": 0.015052755363285542, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015052755363285542 + }, + { + "contact_quality": null, + "progress": 0.01309619378298521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01309619378298521 + }, + { + "contact_quality": null, + "progress": 0.01102795172482729, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01102795172482729 + }, + { + "contact_quality": null, + "progress": 0.018680056557059288, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018680056557059288 + }, + { + "contact_quality": null, + "progress": 0.014400660991668701, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014400660991668701 + }, + { + "contact_quality": null, + "progress": 0.0062745013274252415, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0062745013274252415 + }, + { + "contact_quality": null, + "progress": 0.025354644283652306, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025354644283652306 + } + ], + "candidate_progress": [ + 0.01185232400894165, + 0.015052755363285542, + 0.01309619378298521, + 0.01102795172482729, + 0.018680056557059288, + 0.014400660991668701, + 0.0062745013274252415, + 0.025354644283652306 + ], + "candidate_source_chart_ids": [ + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-6f26118aa94bacd82250" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c832439976a37f52d094", + "generated_tangents": [ + [ + 0.01794205605983734, + -0.39928489923477173, + 0.3047671914100647, + 0.640389084815979, + 0.7059146165847778, + 0.5030750036239624, + 0.16233357787132263, + 1.2212294340133667, + -0.27749332785606384, + -0.5608906745910645, + -1.718010663986206, + 0.508609414100647, + -1.7132834196090698, + -1.3535130023956299, + -0.03722262755036354, + 0.2721763551235199, + 0.49909234046936035, + -0.5537380576133728, + 1.2883248329162598, + 2.141218900680542, + 1.7727383375167847 + ], + [ + 0.02859903872013092, + -0.45314890146255493, + 0.33511263132095337, + 0.5635606050491333, + 0.6740097999572754, + 0.4368559718132019, + -0.17657595872879028, + 1.1521787643432617, + -0.20417176187038422, + -0.40049248933792114, + -1.694342017173767, + 0.7616839408874512, + -1.620996117591858, + -1.0670340061187744, + 0.032699525356292725, + 0.24202735722064972, + 0.5946812033653259, + -0.3700817823410034, + 1.4160525798797607, + 2.2401106357574463, + 1.7921828031539917 + ], + [ + 0.05263262987136841, + -0.3874813914299011, + 0.23047152161598206, + 0.7403726577758789, + 0.7350418567657471, + 0.2908567190170288, + -0.19339340925216675, + 1.2352060079574585, + -0.17305763065814972, + -0.21827661991119385, + -1.5216920375823975, + 0.8042436838150024, + -1.7348999977111816, + -0.8976197242736816, + 0.06708423048257828, + 0.27360913157463074, + 0.537377119064331, + -0.3399698734283447, + 1.4630978107452393, + 2.1092727184295654, + 1.8591285943984985 + ], + [ + -0.4147792458534241, + -0.2259741574525833, + 0.46644696593284607, + 1.522256851196289, + 0.5814188718795776, + 0.4100750982761383, + 0.19133654236793518, + 1.4422869682312012, + -0.31723716855049133, + -0.39828479290008545, + -2.055227041244507, + 1.1731226444244385, + -1.3083711862564087, + -1.6941227912902832, + 0.27829301357269287, + 0.3670850992202759, + 0.4991532564163208, + -0.01967155933380127, + 1.4013078212738037, + 2.4548299312591553, + 2.3025670051574707 + ], + [ + -0.0542437881231308, + -0.43662893772125244, + 0.34782475233078003, + 0.6523323059082031, + 0.6649909019470215, + 0.12254741787910461, + -0.16712021827697754, + 1.0428181886672974, + -0.14330950379371643, + -0.2890681028366089, + -1.5606493949890137, + 0.8559022545814514, + -1.7993046045303345, + -0.7523421049118042, + 0.08717244863510132, + 0.24994580447673798, + 0.5766638517379761, + -0.32058560848236084, + 1.4456861019134521, + 2.2457237243652344, + 1.8380358219146729 + ], + [ + -0.38087594509124756, + -0.26499029994010925, + 0.06930813193321228, + 1.2164201736450195, + 0.5663763284683228, + 0.603212833404541, + 0.4198690354824066, + 1.1472017765045166, + -0.37199628353118896, + -0.4269323945045471, + -1.9943755865097046, + 1.146256685256958, + -1.4226115942001343, + -1.6951652765274048, + 0.04609443247318268, + 0.2929285168647766, + 0.8115304708480835, + -0.18005909025669098, + 1.7425360679626465, + 2.6117398738861084, + 1.821550726890564 + ], + [ + -0.35506951808929443, + -0.3280073404312134, + 0.29627925157546997, + 1.6572011709213257, + 0.8197150230407715, + 0.3277698755264282, + 0.41143620014190674, + 1.386468529701233, + -0.2305210828781128, + -0.5448424220085144, + -1.7443983554840088, + 1.1193647384643555, + -1.1633388996124268, + -1.593414306640625, + 0.4279046952724457, + 0.31593361496925354, + 0.4422547221183777, + 0.10983540117740631, + 1.2414162158966064, + 2.3535211086273193, + 2.1370372772216797 + ], + [ + -0.40747785568237305, + -0.13480070233345032, + 0.7814823985099792, + 1.509238362312317, + 0.5356488227844238, + 0.6098292469978333, + 0.35830628871917725, + 0.8087968826293945, + -0.07927663624286652, + -0.06282895058393478, + -2.015357732772827, + 1.3790664672851562, + -1.7812633514404297, + -1.6387113332748413, + 0.15258343517780304, + 0.23151299357414246, + 0.7743162512779236, + -0.21275068819522858, + 1.7635765075683594, + 2.566110610961914, + 1.2568166255950928 + ] + ], + "generated_utilities": [ + 0.01185232400894165, + 0.015052755363285542, + 0.01309619378298521, + 0.01102795172482729, + 0.018680056557059288, + 0.014400660991668701, + 0.0062745013274252415, + 0.025354644283652306 + ], + "group_id": "ms-c832439976a37f52d094", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0808674544095993, + 0.13129448890686035, + 0.11199682950973511, + 0.2594742774963379, + 0.17684690654277802, + 0.11511725187301636, + 0.17100229859352112, + 0.12906593084335327, + 0.12404869496822357, + 0.10653237998485565, + 0.15492436289787292, + 0.05800905078649521, + 0.04767048358917236, + 0.09533219039440155, + 0.03300365060567856, + 0.07704520225524902 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025296777486801147, + 0.11629299819469452, + -0.024434387683868408, + 1.0812468528747559, + 0.04022188484668732, + -0.01547396183013916, + 0.0, + 0.05167186260223389, + 0.07629208266735077, + -0.00870610773563385, + -0.5777420997619629, + 0.07687681913375854, + -0.6336902379989624, + 0.0, + -0.043449461460113525, + -0.029798656702041626, + 0.18944299221038818, + 0.03022998571395874, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + 0.05685463547706604, + -0.3914306163787842, + 0.4578048586845398, + 1.0812468528747559, + 0.10990457236766815, + -0.18159747123718262, + 0.0, + 0.05167186260223389, + -0.5002377033233643, + 0.04299694299697876, + -0.5777420997619629, + 0.02747189998626709, + -0.6336902379989624, + 0.0, + -0.08638033270835876, + 0.17561247944831848, + 0.37304359674453735, + 0.2538231611251831, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + -0.3902576267719269, + 0.03852551057934761, + 0.16114424169063568, + 2.081246852874756, + 0.22516529262065887, + 0.8184025287628174, + 0.3388257324695587, + 1.0516718626022339, + -0.061541248112916946, + 0.22279328107833862, + -1.577742099761963, + 1.027471899986267, + -1.6336902379989624, + -1.6284003257751465, + -0.24836044013500214, + -0.3427097499370575, + 1.1894429922103882, + -0.15580247342586517, + 1.7312084436416626, + 2.8623785972595215, + 1.0248357057571411 + ], + [ + -0.5903857946395874, + -0.10628272593021393, + -0.07594148814678192, + 2.194340944290161, + 0.11632551997900009, + 0.942036509513855, + 0.06351110339164734, + 0.7966359853744507, + 0.1702205240726471, + 0.1549067199230194, + -1.439386248588562, + 1.3076335191726685, + -1.3458164930343628, + -1.90281081199646, + -0.2967699468135834, + -0.27783676981925964, + 1.0556988716125488, + 0.06510581076145172, + 1.6170521974563599, + 3.027111530303955, + 1.154527187347412 + ], + [ + -0.7559880614280701, + 0.694898247718811, + -0.3551136255264282, + 1.8615511655807495, + 0.9249891042709351, + 0.969862699508667, + 0.4452719986438751, + 1.5801937580108643, + 0.13303470611572266, + -0.13816028833389282, + -1.2399333715438843, + 1.5769362449645996, + -1.5071980953216553, + -2.0578060150146484, + -0.8449342846870422, + 0.3327537477016449, + 1.6781622171401978, + 0.4016150236129761, + 1.8172147274017334, + 3.1609320640563965, + 1.4916398525238037 + ] + ], + "predicted_scores": [ + 3.0802533626556396, + 2.8079328536987305, + 2.694960117340088, + 2.5704212188720703, + 2.3174843788146973, + 2.289219379425049, + 1.906691074371338, + 1.363757848739624 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "0f2810f22383259329f6e94fdc8384176c664f2ed476924e59db7cda4f4a4863", + "stored_base_utility": 0.0808674544095993, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09138330072164536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09138330072164536 + }, + "base_progress": 0.09138330072164536, + "base_success": false, + "base_utility": 0.09138330072164536, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.011022846214473248, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011022846214473248 + }, + { + "contact_quality": null, + "progress": 0.005815426353365183, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005815426353365183 + }, + { + "contact_quality": null, + "progress": 0.006854832172393799, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006854832172393799 + }, + { + "contact_quality": null, + "progress": 0.004729151725769043, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004729151725769043 + }, + { + "contact_quality": null, + "progress": 0.004355867858976126, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004355867858976126 + }, + { + "contact_quality": null, + "progress": 0.0065706572495400906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0065706572495400906 + }, + { + "contact_quality": null, + "progress": 0.0038128693122416735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0038128693122416735 + }, + { + "contact_quality": null, + "progress": 0.00446764612570405, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00446764612570405 + } + ], + "candidate_progress": [ + 0.011022846214473248, + 0.005815426353365183, + 0.006854832172393799, + 0.004729151725769043, + 0.004355867858976126, + 0.0065706572495400906, + 0.0038128693122416735, + 0.00446764612570405 + ], + "candidate_source_chart_ids": [ + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c96b415f083f60857334", + "generated_tangents": [ + [ + -0.8631720542907715, + 0.20969156920909882, + 0.476644366979599, + -0.04481770098209381, + -0.12367299199104309, + 2.0115714073181152, + 2.2624404430389404, + 0.13628721237182617, + 0.3867899477481842, + 1.0296491384506226, + 0.0583997406065464, + -0.09586712718009949, + 1.128807544708252, + 0.6050926446914673, + -0.4575827121734619, + -0.020532377064228058, + 0.11341233551502228, + 1.2037279605865479, + -1.155188798904419, + -0.1706058233976364, + -0.18583622574806213 + ], + [ + -0.39789462089538574, + 0.15221968293190002, + 1.0981792211532593, + -0.16742363572120667, + 0.3057832717895508, + 2.5257771015167236, + 1.3183159828186035, + -0.17978835105895996, + -0.012723429128527641, + 1.254101037979126, + 0.31563514471054077, + -0.06348013877868652, + 1.7093688249588013, + 1.2359894514083862, + -0.3803277015686035, + 0.06949549913406372, + 0.30669891834259033, + 1.182504415512085, + -0.2994729280471802, + 0.037694960832595825, + -0.916029691696167 + ], + [ + -0.3888220191001892, + 0.12620966136455536, + 1.0046055316925049, + -0.19541215896606445, + 0.3842652142047882, + 2.5759148597717285, + 1.3602116107940674, + -0.18259736895561218, + 0.047947175800800323, + 1.186171054840088, + 0.3989625871181488, + 0.0273311585187912, + 1.7371598482131958, + 0.9497014880180359, + -0.33922454714775085, + 0.028795022517442703, + 0.2447018325328827, + 1.1618549823760986, + -0.17465215921401978, + 0.10633102059364319, + -0.841964602470398 + ], + [ + -1.1787970066070557, + 0.01931753009557724, + 0.9009875655174255, + -0.049869269132614136, + -0.04317525029182434, + 2.61747145652771, + 2.009066343307495, + -0.18116849660873413, + 0.3759390413761139, + 1.1102315187454224, + 0.13905367255210876, + 0.17136947810649872, + 1.3530523777008057, + 1.0468418598175049, + -0.4049355685710907, + 0.07246808707714081, + 0.2904800772666931, + 1.3014835119247437, + -0.9384945034980774, + -0.20737358927726746, + -0.6260594129562378 + ], + [ + -1.0669664144515991, + -0.05625876039266586, + 0.8500634431838989, + 0.025441870093345642, + -0.23586323857307434, + 2.4919440746307373, + 2.0146644115448, + -0.2825126349925995, + 0.34640994668006897, + 1.0405712127685547, + 0.27461907267570496, + 0.2203475683927536, + 1.263142466545105, + 0.81635981798172, + -0.28405964374542236, + 0.007491793483495712, + 0.40737515687942505, + 1.2977722883224487, + -0.9127889275550842, + 0.06070518493652344, + -0.7309354543685913 + ], + [ + -1.635523796081543, + -0.009781613945960999, + 0.7616024017333984, + 0.02520376443862915, + 0.05396144092082977, + 2.5448250770568848, + 2.040884494781494, + -0.298483282327652, + 0.3895706236362457, + 1.0330209732055664, + 0.4657478332519531, + -0.09435263276100159, + 1.0137053728103638, + 1.0496138334274292, + -0.1867525279521942, + 0.08877302706241608, + -0.019419148564338684, + 1.3915016651153564, + -0.559948742389679, + -0.1923726201057434, + -0.7642843723297119 + ], + [ + -1.2364280223846436, + -0.05301504582166672, + 0.8328337669372559, + 0.09453615546226501, + -0.14185777306556702, + 2.3135950565338135, + 1.8596365451812744, + -0.4325282871723175, + 0.3867894411087036, + 1.0150127410888672, + 0.16291148960590363, + 0.18627028167247772, + 1.208469271659851, + 0.7966751456260681, + -0.4114797115325928, + -0.056879088282585144, + 0.3379962742328644, + 1.3211537599563599, + -0.9629276394844055, + 0.21863311529159546, + -0.7556114196777344 + ], + [ + -1.4423089027404785, + -0.10011017322540283, + 1.0570999383926392, + -0.004705637693405151, + -0.1491108536720276, + 2.23101806640625, + 1.9499225616455078, + -0.38821807503700256, + 0.3906157314777374, + 0.9672142267227173, + -0.07669848948717117, + 0.36660289764404297, + 1.3273799419403076, + 0.3710804283618927, + -0.30547404289245605, + 0.005290810018777847, + 0.1930892914533615, + 1.2460944652557373, + -1.0674211978912354, + -0.03995099663734436, + -0.8713643550872803 + ] + ], + "generated_utilities": [ + 0.011022846214473248, + 0.005815426353365183, + 0.006854832172393799, + 0.004729151725769043, + 0.004355867858976126, + 0.0065706572495400906, + 0.0038128693122416735, + 0.00446764612570405 + ], + "group_id": "ms-c96b415f083f60857334", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15946325659751892, + 0.16313008964061737, + 0.2226802557706833, + 0.18744397163391113, + 0.21545825898647308, + 0.15985511243343353, + 0.16218391060829163, + 0.26426148414611816, + 0.184181809425354, + 0.24352896213531494, + 0.14179718494415283, + 0.1026589497923851, + 0.1274908483028412, + 0.05275428295135498, + 0.08544860780239105, + 0.07143914699554443 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.16191565990448, + 0.4561867117881775, + 1.1241415739059448, + 0.08947351574897766, + 0.32140541076660156, + 3.6895532608032227, + 3.6284518241882324, + -1.8666775226593018, + -0.22328677773475647, + 1.48662531375885, + 1.4737720489501953, + 1.35750150680542, + 2.507124423980713, + 1.9315069913864136, + -0.8371121883392334, + 0.5339290499687195, + 1.0061794519424438, + 2.149812698364258, + 0.08911214768886566, + 0.17410686612129211, + 0.3549031913280487 + ], + [ + -0.14804881811141968, + -0.5333454608917236, + 0.7481307983398438, + 0.22411420941352844, + 1.7204210758209229, + 3.908766746520996, + 3.3698034286499023, + -1.6747719049453735, + 0.2769082486629486, + 2.042428970336914, + 0.44748249650001526, + 1.9492043256759644, + 2.918121576309204, + 3.2413060665130615, + -0.6994609832763672, + -0.3854917883872986, + -0.6702244281768799, + 2.408869504928589, + -0.5672509670257568, + 0.5941196084022522, + 0.46011871099472046 + ], + [ + -0.46204906702041626, + -0.4236203730106354, + 1.8582539558410645, + 0.8416532278060913, + 1.2386386394500732, + 3.7320151329040527, + 2.242549180984497, + -1.8652616739273071, + -1.0259326696395874, + 1.3383660316467285, + 1.7068204879760742, + 0.7941814661026001, + 3.3389508724212646, + 2.7797679901123047, + -0.9363516569137573, + 0.17812880873680115, + 0.6111975908279419, + 2.066016435623169, + -0.28941091895103455, + 1.501695990562439, + -0.35273995995521545 + ], + [ + -0.7471081614494324, + -0.029650092124938965, + 1.019482135772705, + -0.30564722418785095, + 1.380143642425537, + 3.247985601425171, + 3.0548973083496094, + -0.5582134127616882, + -0.920264482498169, + 2.3859832286834717, + 1.1080355644226074, + 1.5716153383255005, + 3.089156150817871, + 2.043484687805176, + 0.1913408637046814, + -0.7557333707809448, + -0.6297664642333984, + 1.7829738855361938, + -0.6967166662216187, + -0.10757219791412354, + 0.229112908244133 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027880847454071045, + -0.2692106068134308, + 0.12925803661346436, + -0.1745016872882843, + -0.1231425404548645, + 1.9967715740203857, + 0.0, + -0.1433020830154419, + 0.2667914927005768, + 0.8867976665496826, + -0.057856738567352295, + 0.5789351463317871, + 1.7831475734710693, + 0.0, + -0.35565105080604553, + 0.293251097202301, + 0.012226536870002747, + 1.121243953704834, + -0.15060651302337646, + 0.03409510850906372, + 0.0 + ], + [ + -0.7738120555877686, + 0.08056297898292542, + 1.1292580366134644, + 0.4734484851360321, + 0.8768574595451355, + 2.9967715740203857, + 3.0509116649627686, + -1.143302083015442, + -0.1310054361820221, + 1.8867976665496826, + 0.9421432614326477, + 1.578935146331787, + 2.7831475734710693, + 2.6037895679473877, + -0.5597631931304932, + 0.11406822502613068, + 0.2593899667263031, + 2.121243953704834, + -0.1791832000017166, + 0.5773167610168457, + 0.41381677985191345 + ], + [ + -0.6024693250656128, + 0.29007619619369507, + 1.3939473628997803, + 0.3474765419960022, + 1.1848478317260742, + 3.127882719039917, + 3.3329083919525146, + -1.2855982780456543, + 0.20972535014152527, + 1.8329110145568848, + 1.267270803451538, + 1.709198236465454, + 2.927372455596924, + 2.9659509658813477, + -0.7555317282676697, + 0.015305742621421814, + 0.6009905338287354, + 2.188507556915283, + 0.07961152493953705, + 0.4547080397605896, + 0.20378917455673218 + ], + [ + -1.2702982425689697, + 0.06162632256746292, + 0.9447290897369385, + 0.837199330329895, + 0.5848468542098999, + 3.4801836013793945, + 2.8138537406921387, + -1.5083554983139038, + -0.45768383145332336, + 2.08561110496521, + 1.4951050281524658, + 1.8484808206558228, + 2.5617129802703857, + 2.942486047744751, + -0.562404215335846, + 0.17441512644290924, + 0.7880480289459229, + 1.8807158470153809, + -0.043379828333854675, + 0.39090824127197266, + 0.43096962571144104 + ] + ], + "predicted_scores": [ + 8.127728462219238, + 7.716030597686768, + 7.650069236755371, + 7.320975303649902, + 7.23729133605957, + 6.941411018371582, + 6.868640422821045, + 6.600250244140625 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "bf24d506cf320effed2cd7faf94d8618b0626bf8d5c3abc1f837513597c18231", + "stored_base_utility": 0.15946325659751892, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0626688003540039, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0626688003540039 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 0.0626688003540039, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ebacae0135af2832dfbb", + "ms-003a2225c0892dcda683", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683", + "ms-3d090ac2459c4b891dd1", + "ms-ebacae0135af2832dfbb", + "ms-003a2225c0892dcda683", + "ms-6d46dae312649c977b64" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + false, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "generated_tangents": [ + [ + 0.06849304586648941, + 0.020963147282600403, + 0.3397200107574463, + -0.21450698375701904, + 1.7600600719451904, + 0.3978930115699768, + 0.08446690440177917, + 0.04985088109970093, + 0.009671914391219616, + 0.1878652274608612, + 0.06474080681800842, + 0.43082863092422485, + 0.01677122712135315, + -0.09526801109313965, + -0.9732409119606018, + -0.11426205933094025, + 1.0602449178695679, + 0.355584055185318, + -0.368932843208313, + 1.6154060363769531, + 0.22390800714492798 + ], + [ + 0.15004707872867584, + -0.08960972726345062, + 0.33218637108802795, + -0.31179484724998474, + 1.9093825817108154, + 0.37549805641174316, + 0.18935957551002502, + -0.00799529254436493, + 0.011870434507727623, + 0.1430848091840744, + 0.12518903613090515, + 0.36604756116867065, + 0.08603277802467346, + -0.06902790069580078, + -1.046745777130127, + -0.18465444445610046, + 1.1558502912521362, + 0.2851211130619049, + -0.33572494983673096, + 1.7115893363952637, + 0.13107410073280334 + ], + [ + 0.19285762310028076, + -0.04629793018102646, + 0.389565646648407, + -0.19866038858890533, + 1.7930231094360352, + 0.46391668915748596, + 0.06795725226402283, + 0.051617443561553955, + 0.06245651841163635, + 0.12304775416851044, + 0.04547211155295372, + 0.43040698766708374, + 0.028770118951797485, + 0.04088864475488663, + -0.9964492917060852, + -0.06462139636278152, + 1.080371618270874, + 0.2642951011657715, + -0.34562748670578003, + 1.6415646076202393, + 0.2722638249397278 + ], + [ + 0.11304552853107452, + -0.03762324899435043, + 0.4021502137184143, + -0.2493474781513214, + 1.8291187286376953, + 0.3095042109489441, + 0.09451401233673096, + 0.05497932434082031, + 0.006364226806908846, + 0.13321825861930847, + 0.12507623434066772, + 0.307923287153244, + 0.11472919583320618, + -0.15573225915431976, + -1.0519554615020752, + -0.13943922519683838, + 1.1995705366134644, + 0.30124446749687195, + -0.24065834283828735, + 1.7303709983825684, + 0.11594261974096298 + ], + [ + 0.038671910762786865, + 0.021258823573589325, + 0.3150809109210968, + -0.3030160069465637, + 1.8387234210968018, + 0.4528977870941162, + -0.08211082220077515, + 0.12862318754196167, + -0.009231509640812874, + 0.114659883081913, + 0.08934962749481201, + 0.4509127736091614, + 0.013727247714996338, + -0.1062411218881607, + -0.982166588306427, + -0.0884760320186615, + 1.0436427593231201, + 0.4198005199432373, + -0.24441659450531006, + 1.7569730281829834, + 0.18441389501094818 + ], + [ + 0.2280730903148651, + -0.06717413663864136, + 0.2869489789009094, + -0.13354362547397614, + 1.5840575695037842, + 0.29217830300331116, + -0.1405428647994995, + 0.1144861951470375, + 0.09890282154083252, + 0.18530166149139404, + 0.10732823610305786, + 0.5616626739501953, + -0.0902877151966095, + 0.09286195039749146, + -0.9488819241523743, + 0.04549604281783104, + 1.0309386253356934, + 0.4310721457004547, + -0.028140664100646973, + 1.790179967880249, + 0.22344520688056946 + ], + [ + 0.024200841784477234, + -0.027662798762321472, + 0.20946550369262695, + -0.2739817500114441, + 1.816767692565918, + 0.4621601700782776, + -0.015732914209365845, + -0.08524897694587708, + 0.07004743069410324, + 0.17961004376411438, + 0.04790934547781944, + 0.45547717809677124, + 0.062212854623794556, + 0.14574454724788666, + -0.9427071213722229, + -0.13454841077327728, + 1.087843894958496, + 0.16174688935279846, + -0.2898705005645752, + 1.7132596969604492, + 0.19047726690769196 + ], + [ + 0.09942897409200668, + -0.024153687059879303, + 0.44161221385002136, + -0.17972099781036377, + 1.815826416015625, + 0.5074065923690796, + 0.1741277277469635, + -0.06894567608833313, + 0.06373905390501022, + 0.11198487877845764, + 0.06291908025741577, + 0.397016704082489, + 0.1716410219669342, + 0.0040760338306427, + -1.018132209777832, + -0.0712416023015976, + 1.095371127128601, + 0.2742389440536499, + -0.36693036556243896, + 1.5890514850616455, + 0.356515109539032 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 0.0626688003540039, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20931994915008545, + 2.0, + 2.0, + 0.2008315920829773, + 0.09476469457149506, + 0.05857282876968384, + 0.10009106248617172, + 0.07648500055074692, + 0.05889680236577988, + 0.014406224712729454, + 0.09262995421886444, + 0.15997587144374847, + 0.04682888835668564, + 0.09751574695110321, + 0.06459556519985199, + 0.20335909724235535 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7537296414375305, + 0.4158197343349457, + 1.4326180219650269, + -1.4107877016067505, + 2.9579341411590576, + 0.9669660925865173, + 1.4339183568954468, + 0.5987398624420166, + -0.2018759399652481, + 0.7105501890182495, + -0.03802938759326935, + 1.3799947500228882, + -0.6568997502326965, + -0.37473973631858826, + -1.857657790184021, + 0.038770925253629684, + 1.931185007095337, + 0.6317195296287537, + 0.1380508691072464, + 2.954265832901001, + 2.9641003608703613 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8678367137908936, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.7494794726371765, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.928200721740723 + ], + [ + 0.6226351857185364, + 0.34334811568260193, + 1.492361068725586, + -1.5917733907699585, + 3.1334922313690186, + 1.0313001871109009, + 1.2075668573379517, + 0.3046378493309021, + -0.11550238728523254, + 0.4499495327472687, + 0.16646838188171387, + 1.5877203941345215, + -0.6488803625106812, + -0.5760968327522278, + -1.8349132537841797, + 0.14188142120838165, + 2.0770444869995117, + 0.5806879997253418, + 0.23913151025772095, + 2.8127381801605225, + 3.0649962425231934 + ], + [ + 0.5975998044013977, + 0.722756028175354, + 1.7548110485076904, + -1.166775107383728, + 2.6060047149658203, + 0.6675758361816406, + 1.125187635421753, + 0.3373170793056488, + -0.1842741072177887, + 0.8186168670654297, + -0.31481224298477173, + 1.2082113027572632, + -0.9804694652557373, + -0.14131735265254974, + -1.700500249862671, + 0.07075637578964233, + 1.5488730669021606, + 0.549576461315155, + 0.034750498831272125, + 3.227431297302246, + 2.9464566707611084 + ], + [ + 0.844982922077179, + 0.7132244110107422, + 1.4430484771728516, + -1.3331272602081299, + 3.1703803539276123, + 1.1851513385772705, + 1.355688452720642, + 0.6115275025367737, + 0.2616361379623413, + 1.2030918598175049, + 0.05625858157873154, + 1.6050300598144531, + -0.7155078053474426, + -0.021488159894943237, + -2.073806047439575, + 0.5387547016143799, + 1.6223118305206299, + 0.7140727043151855, + -0.06027664244174957, + 2.7503156661987305, + 2.6344075202941895 + ], + [ + 0.5821706056594849, + 0.78206866979599, + 1.8119354248046875, + -0.8252442479133606, + 3.038909435272217, + 1.3900940418243408, + 1.7462186813354492, + 0.005083680152893066, + -0.27262064814567566, + 0.867202877998352, + -0.5555376410484314, + 1.172202706336975, + -1.2016760110855103, + -0.06355220079421997, + -2.2925045490264893, + 0.6078628301620483, + 1.7895314693450928, + 0.8720078468322754, + 0.472573459148407, + 3.377012014389038, + 2.6039178371429443 + ], + [ + 0.3789674639701843, + 0.3270051181316376, + 1.056584358215332, + -2.0788497924804688, + 2.948819875717163, + 1.6022355556488037, + 0.8599810004234314, + 0.9570736885070801, + -0.6380544900894165, + 0.3103100061416626, + -0.42896610498428345, + 1.7150276899337769, + -0.8975871205329895, + -0.278381884098053, + -1.305256724357605, + 0.3637259304523468, + 1.8004443645477295, + 0.16446983814239502, + 0.19097819924354553, + 3.43778657913208, + 3.538135528564453 + ], + [ + 0.6903814673423767, + 1.1327197551727295, + 1.8523926734924316, + -2.0428731441497803, + 3.1870431900024414, + 0.5352929830551147, + 0.6004904508590698, + 1.2120530605316162, + 0.16280107200145721, + 1.119707465171814, + 0.7612010836601257, + 0.7890661954879761, + -1.3447637557983398, + -0.1703537553548813, + -1.1010353565216064, + 0.32831913232803345, + 1.5534476041793823, + -0.2119135856628418, + -0.562728762626648, + 2.4702467918395996, + 2.5528204441070557 + ], + [ + 0.9177048206329346, + -0.04751536250114441, + 1.4819984436035156, + -2.1542677879333496, + 3.522390842437744, + 1.8177759647369385, + 2.2213237285614014, + 0.8223017454147339, + -0.1973809152841568, + -0.053272128105163574, + -0.5862784385681152, + 0.6421516537666321, + -0.8713911771774292, + -0.9495017528533936, + -1.6454768180847168, + -0.38891172409057617, + 1.648432970046997, + 1.4951133728027344, + 0.5893608331680298, + 2.449934482574463, + 3.419659376144409 + ], + [ + 1.2956775426864624, + -0.3119873106479645, + 1.5930066108703613, + -1.7682520151138306, + 2.4385876655578613, + 1.4679458141326904, + 1.4536705017089844, + 0.31068816781044006, + 0.660710871219635, + 0.8104690909385681, + 0.483506441116333, + 2.1112046241760254, + -0.22927385568618774, + 0.17858651280403137, + -2.485126495361328, + -0.9187790155410767, + 1.2810213565826416, + 1.3727718591690063, + 0.44044917821884155, + 3.4018898010253906, + 3.39054799079895 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0012856721878051758, + 0.00795695185661316, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + 0.11055737733840942, + 0.0, + 0.02143383026123047, + 0.0025591105222702026, + -0.030314385890960693, + -0.0125969760119915, + 0.3799947500228882, + 0.0076233744621276855, + 0.0, + -0.857657790184021, + 0.013764806091785431, + 0.9311850070953369, + 0.07110422849655151, + 0.03423507511615753, + 1.954265832901001, + 0.0 + ], + [ + -0.24627035856246948, + -0.16095152497291565, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + -0.023127973079681396, + 0.0, + -0.24312204122543335, + 0.1389055997133255, + 0.11457735300064087, + 0.06618259102106094, + 0.3799947500228882, + 0.07616078853607178, + 0.0, + -0.857657790184021, + 0.09948430210351944, + 0.9311850070953369, + 0.006249547004699707, + -0.33439844846725464, + 1.954265832901001, + 0.0 + ] + ], + "predicted_scores": [ + 4.393341541290283, + 4.325108528137207, + 4.305380344390869, + 4.273581027984619, + 4.09778356552124, + 3.936275005340576, + 3.919905424118042, + 3.758706569671631 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "534717141", + "selected_index": 0, + "state_hash": "5318995ba859471b8678d6594274c7c0565a1ccfd83ea302378cc2884dc5cc32", + "stored_base_utility": 0.20931994915008545, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14809735119342804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14809735119342804 + }, + "base_progress": 0.14809735119342804, + "base_success": false, + "base_utility": 0.14809735119342804, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0006723999977111816, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006723999977111816 + }, + { + "contact_quality": null, + "progress": 0.000547130883205682, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000547130883205682 + }, + { + "contact_quality": null, + "progress": 0.000514427840244025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000514427840244025 + }, + { + "contact_quality": null, + "progress": 0.0005189180374145508, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0005189180374145508 + }, + { + "contact_quality": null, + "progress": 0.001536289812065661, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001536289812065661 + }, + { + "contact_quality": null, + "progress": 0.0004230936465319246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0004230936465319246 + }, + { + "contact_quality": null, + "progress": 0.00036319097853265703, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00036319097853265703 + }, + { + "contact_quality": null, + "progress": 0.0003688931465148926, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0003688931465148926 + } + ], + "candidate_progress": [ + 0.0006723999977111816, + 0.000547130883205682, + 0.000514427840244025, + 0.0005189180374145508, + 0.001536289812065661, + 0.0004230936465319246, + 0.00036319097853265703, + 0.0003688931465148926 + ], + "candidate_source_chart_ids": [ + "ms-f2b2b2a1aa2863c3676f", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-da142025733b53fcf7eb", + "generated_tangents": [ + [ + -0.3432873487472534, + -0.4166366457939148, + 1.7469239234924316, + 0.30275484919548035, + 1.5726878643035889, + 2.5822014808654785, + 2.0489354133605957, + -0.5852302312850952, + 0.1062394306063652, + 1.4572888612747192, + -0.3047958016395569, + 1.0039513111114502, + 2.0345330238342285, + 2.8583405017852783, + -1.4076637029647827, + -0.2795407772064209, + 1.7358763217926025, + 1.0890778303146362, + 0.5476433038711548, + 2.262739658355713, + 1.6884771585464478 + ], + [ + -0.44322335720062256, + -0.26510465145111084, + 1.6426684856414795, + 0.22649741172790527, + 1.5892977714538574, + 3.0924112796783447, + 1.9100933074951172, + -0.5058478116989136, + 0.1344156563282013, + 1.6353282928466797, + 0.02294306457042694, + 0.9665766954421997, + 1.906577706336975, + 2.7980704307556152, + -1.6457966566085815, + -0.30786964297294617, + 1.8597486019134521, + 1.1984976530075073, + 0.6639366149902344, + 1.9491913318634033, + 1.9728260040283203 + ], + [ + -0.14347891509532928, + -0.29903554916381836, + 2.006197690963745, + 0.12976646423339844, + 1.1190180778503418, + 2.7899868488311768, + 2.288774013519287, + -0.7226301431655884, + 0.24866513907909393, + 1.41183602809906, + -0.07595058530569077, + 0.925747811794281, + 2.2851498126983643, + 2.5220324993133545, + -1.641467809677124, + -0.29011446237564087, + 2.005598306655884, + 1.2708896398544312, + 0.7066189050674438, + 2.3896970748901367, + 2.3964414596557617 + ], + [ + -0.05751855671405792, + -0.253709077835083, + 1.705299973487854, + -0.1536070853471756, + 1.6768748760223389, + 3.253661632537842, + 1.874450922012329, + -0.5398855209350586, + -0.022087017074227333, + 1.677821159362793, + -0.002833440899848938, + 1.1556262969970703, + 1.8816803693771362, + 2.575993776321411, + -1.5646936893463135, + -0.34078413248062134, + 2.111616373062134, + 1.0539075136184692, + 0.5210786461830139, + 2.0769670009613037, + 2.0071918964385986 + ], + [ + -0.387844443321228, + -0.3073009252548218, + 1.5910156965255737, + -0.09482955187559128, + 1.246790885925293, + 2.478865623474121, + 2.3712637424468994, + -0.3380632698535919, + 0.1314142942428589, + 1.1512407064437866, + 0.058152489364147186, + 0.5780254006385803, + 1.717081904411316, + 3.6755905151367188, + -1.2382253408432007, + -0.4301864802837372, + 1.4169405698776245, + 0.7797537446022034, + 0.803180456161499, + 2.1079227924346924, + 2.019592761993408 + ], + [ + -0.20618392527103424, + -0.2934543788433075, + 1.82016122341156, + -0.037838131189346313, + 1.3345353603363037, + 2.859126567840576, + 2.06943678855896, + -0.713410496711731, + 0.09354620426893234, + 1.6099059581756592, + -0.027280479669570923, + 0.8799170255661011, + 2.104429244995117, + 2.486814260482788, + -1.7883286476135254, + -0.37900424003601074, + 2.04034423828125, + 1.3327462673187256, + 0.5964900255203247, + 2.144643783569336, + 2.1147611141204834 + ], + [ + -0.21850784122943878, + -0.36162781715393066, + 1.9714752435684204, + -0.03462591767311096, + 1.3450486660003662, + 2.7020928859710693, + 2.5649144649505615, + -0.732650876045227, + 0.048771344125270844, + 1.6956018209457397, + 0.029450014233589172, + 0.9816755652427673, + 2.4808883666992188, + 2.607961654663086, + -1.8436342477798462, + -0.269349604845047, + 1.900127649307251, + 1.4064507484436035, + 0.6604788899421692, + 1.8866121768951416, + 1.769606590270996 + ], + [ + 0.10653827339410782, + -0.3439488410949707, + 1.8145986795425415, + -0.35351812839508057, + 0.9562007784843445, + 2.389096736907959, + 2.441350221633911, + -1.0330595970153809, + -0.026264620944857597, + 1.7622023820877075, + 0.0689915269613266, + 0.9659565687179565, + 2.6354854106903076, + 2.3394391536712646, + -1.6571424007415771, + -0.5705158710479736, + 2.0164456367492676, + 0.9216766357421875, + 0.38031086325645447, + 2.2025082111358643, + 1.607088327407837 + ] + ], + "generated_utilities": [ + 0.0006723999977111816, + 0.000547130883205682, + 0.000514427840244025, + 0.0005189180374145508, + 0.001536289812065661, + 0.0004230936465319246, + 0.00036319097853265703, + 0.0003688931465148926 + ], + "group_id": "ms-da142025733b53fcf7eb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.14843207597732544, + 0.1717694252729416, + 0.09356185048818588, + 0.23231086134910583, + 0.19341367483139038, + 0.04822012037038803, + 0.23024770617485046, + 0.16561973094940186, + 0.2643269896507263, + 0.12403462827205658, + 0.12689702212810516, + 0.16619956493377686, + 0.13481859862804413, + 0.07537531852722168, + 0.11867459863424301, + 0.04740651696920395 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.04848463833332062, + -0.18164527416229248, + 0.6558789014816284, + 0.038211286067962646, + 0.08492374420166016, + 2.265157699584961, + 0.0, + -0.09011825919151306, + 0.0018594712018966675, + 0.6884076595306396, + 0.027596401050686836, + 0.06117302179336548, + 1.278116226196289, + 0.0, + -0.3951960802078247, + -0.08061015605926514, + 1.07906174659729, + 0.3229694366455078, + -0.21076995134353638, + 1.8564467430114746, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0903544425964355, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.570276260375977, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.736253261566162 + ], + [ + 0.08095964789390564, + -0.7926032543182373, + 1.571103811264038, + 0.07894467562437057, + 0.2701656222343445, + 3.806621789932251, + 2.203669786453247, + -0.042464032769203186, + 0.7168105244636536, + 1.8671371936798096, + -0.30838707089424133, + 1.7242767810821533, + 1.8847527503967285, + 1.305934190750122, + -0.42665380239486694, + -0.2973874807357788, + 1.1881210803985596, + 1.6921244859695435, + 1.070060133934021, + 2.067784309387207, + 1.916501760482788 + ], + [ + 0.7620636224746704, + -2.1605899333953857, + 2.1443262100219727, + 0.17886832356452942, + 1.6259945631027222, + 3.787412166595459, + 1.3663169145584106, + 0.49799659848213196, + -0.6301277279853821, + 2.275850534439087, + -0.6735645532608032, + 1.7504758834838867, + 2.7512195110321045, + 1.7462939023971558, + -0.6835377216339111, + 0.36623528599739075, + 1.405979871749878, + 0.6472548842430115, + 1.4791045188903809, + 2.7838869094848633, + 3.0856707096099854 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.29792410135269165, + -0.30587875843048096, + 0.6558789014816284, + -0.4171115756034851, + -0.1733894944190979, + 2.265157699584961, + 0.0, + 0.14607194066047668, + 0.05651267617940903, + 0.6884076595306396, + -0.0433836504817009, + 0.07522618770599365, + 1.278116226196289, + 0.0, + -0.41462820768356323, + 0.26706579327583313, + 1.07906174659729, + 0.44956159591674805, + -0.37072819471359253, + 1.8564467430114746, + 0.0 + ], + [ + 0.18432100117206573, + -0.9473621249198914, + 1.5389783382415771, + -0.041053857654333115, + 0.6282865405082703, + 3.276425838470459, + 2.1839330196380615, + -0.3612370491027832, + 0.23081421852111816, + 1.8094429969787598, + 0.10581490397453308, + 0.8865737915039062, + 2.5146288871765137, + 2.2134878635406494, + -1.6751070022583008, + -0.08268362283706665, + 2.303063154220581, + 1.3061904907226562, + 0.4770132899284363, + 3.109205722808838, + 2.509927272796631 + ], + [ + -0.5288646817207336, + -1.5067847967147827, + 1.376499056816101, + 0.13231416046619415, + 1.048660159111023, + 3.5048489570617676, + 1.9678237438201904, + -0.14848726987838745, + 0.05475667119026184, + 1.5529061555862427, + 0.1597936749458313, + 0.7318928241729736, + 2.17749285697937, + 2.0782582759857178, + -1.5473355054855347, + -0.6715520620346069, + 1.7043644189834595, + 1.1190917491912842, + 0.4214729070663452, + 3.3400330543518066, + 2.3147284984588623 + ] + ], + "predicted_scores": [ + 6.292049884796143, + 6.263817310333252, + 6.2627787590026855, + 5.9403767585754395, + 5.853697299957275, + 5.806769371032715, + 5.642268180847168, + 4.94965934753418 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "0026cc5b3762b520baadbda7bf138a2d24fffa5b2306f245ddbc1d1556cccc22", + "stored_base_utility": 0.14843207597732544, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1362423449754715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1362423449754715 + }, + "base_progress": 0.1362423449754715, + "base_success": false, + "base_utility": 0.1362423449754715, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0132790207862854, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0132790207862854 + }, + { + "contact_quality": null, + "progress": 0.018407067283988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018407067283988 + }, + { + "contact_quality": null, + "progress": 0.012948970310389996, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012948970310389996 + }, + { + "contact_quality": null, + "progress": 0.010082741267979145, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010082741267979145 + }, + { + "contact_quality": null, + "progress": 0.010856767185032368, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010856767185032368 + }, + { + "contact_quality": null, + "progress": 0.009846587665379047, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009846587665379047 + }, + { + "contact_quality": null, + "progress": 0.006558021064847708, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006558021064847708 + }, + { + "contact_quality": null, + "progress": 0.012989739887416363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012989739887416363 + } + ], + "candidate_progress": [ + 0.0132790207862854, + 0.018407067283988, + 0.012948970310389996, + 0.010082741267979145, + 0.010856767185032368, + 0.009846587665379047, + 0.006558021064847708, + 0.012989739887416363 + ], + "candidate_source_chart_ids": [ + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-76f27d9a935b25e15c38", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-76f27d9a935b25e15c38", + "ms-cf4b63d109a88a4c42bf", + "ms-a2518826fa0a816cc46f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-e2bad6e99e8a281207cc", + "generated_tangents": [ + [ + -0.7656202912330627, + -0.5836631059646606, + 1.9936965703964233, + -0.25754499435424805, + 1.807642936706543, + 2.8049769401550293, + 2.9517674446105957, + 0.8411537408828735, + -0.3716524541378021, + 0.4764257073402405, + 0.8208474516868591, + 0.7875684499740601, + 0.23823460936546326, + 0.508261501789093, + 1.0950506925582886, + 0.2447732537984848, + -0.6680842041969299, + -1.4543466567993164, + 1.4240951538085938, + -1.0104715824127197, + -1.3162401914596558 + ], + [ + -0.5956717729568481, + -0.3503824472427368, + 2.43707537651062, + -0.17598645389080048, + 1.5012032985687256, + 2.822638750076294, + 2.460059642791748, + 0.8111618757247925, + -0.4014160633087158, + 0.16674283146858215, + 0.9034915566444397, + 1.0867396593093872, + 0.3830781877040863, + 0.3912196755409241, + 0.9169946908950806, + 0.25626760721206665, + -0.6009308099746704, + -1.4893838167190552, + 1.5943150520324707, + -0.9501956105232239, + -0.7393081188201904 + ], + [ + -0.8630020618438721, + -0.4366946816444397, + 1.6524393558502197, + -1.1390119791030884, + 2.5286006927490234, + 2.949692487716675, + 2.2771573066711426, + 1.2820297479629517, + -0.40615230798721313, + 0.5594807267189026, + 0.6452037692070007, + 0.6294150352478027, + -0.3940696716308594, + 0.023169033229351044, + 0.9016968011856079, + -0.031307801604270935, + -0.6735029220581055, + -0.6876105666160583, + 1.6249415874481201, + -1.031238079071045, + -1.4563292264938354 + ], + [ + -0.832272469997406, + -0.5986323356628418, + 2.341871976852417, + -0.19841353595256805, + 1.8155076503753662, + 2.685831308364868, + 2.803917407989502, + 0.8019734621047974, + -0.3884523808956146, + 0.48511818051338196, + 0.8000732660293579, + 0.9763474464416504, + 0.22082993388175964, + 0.5804189443588257, + 1.028027892112732, + 0.17335057258605957, + -0.5768880844116211, + -1.429967999458313, + 1.5494601726531982, + -1.0149294137954712, + -1.4201065301895142 + ], + [ + -0.6742351055145264, + -0.6398972868919373, + 2.234661102294922, + -0.05855080485343933, + 1.8243978023529053, + 2.7808637619018555, + 2.83819317817688, + 0.7586811780929565, + -0.38342413306236267, + 0.353824257850647, + 0.7527822256088257, + 0.9232964515686035, + 0.30970773100852966, + 0.364149272441864, + 1.1255438327789307, + 0.18194139003753662, + -0.4114264249801636, + -1.3955891132354736, + 1.3439805507659912, + -0.8595713973045349, + -1.4836050271987915 + ], + [ + -0.8021159768104553, + -0.44574612379074097, + 1.7953410148620605, + -1.218800663948059, + 2.4733567237854004, + 3.018044948577881, + 2.455137014389038, + 1.213944435119629, + -0.4098770022392273, + 0.5149925351142883, + 0.5362481474876404, + 0.6469517946243286, + -0.38892173767089844, + -0.03415808081626892, + 0.9222234487533569, + -0.038790859282016754, + -0.5011613368988037, + -0.7243276834487915, + 1.4363141059875488, + -1.137031078338623, + -1.4899466037750244 + ], + [ + -0.8298236131668091, + -0.7571960687637329, + 2.4112040996551514, + -0.7422088384628296, + 2.5757224559783936, + 2.3544819355010986, + 2.362041711807251, + 1.3503568172454834, + -0.32672879099845886, + 0.36957573890686035, + 0.42405596375465393, + 0.7359752655029297, + -0.19414299726486206, + 1.5960707664489746, + 1.154114007949829, + 0.02908744290471077, + -0.5586186051368713, + -1.3680403232574463, + 1.2902143001556396, + -0.6922940015792847, + -1.7041513919830322 + ], + [ + -0.8196470141410828, + -0.4303642511367798, + 1.8968323469161987, + -0.3930642902851105, + 2.3315858840942383, + 2.9916892051696777, + 2.4671459197998047, + 0.9331362247467041, + -0.4813539385795593, + 0.3563936650753021, + 0.9445043206214905, + 0.8429049849510193, + -0.7790067195892334, + -0.4683154821395874, + 0.867400586605072, + -0.05478433519601822, + -0.5078667402267456, + -1.0063856840133667, + 0.9468156099319458, + -2.225613832473755, + -1.4846243858337402 + ] + ], + "generated_utilities": [ + 0.0132790207862854, + 0.018407067283988, + 0.012948970310389996, + 0.010082741267979145, + 0.010856767185032368, + 0.009846587665379047, + 0.006558021064847708, + 0.012989739887416363 + ], + "group_id": "ms-e2bad6e99e8a281207cc", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13993772864341736, + 0.16482847929000854, + 0.20825156569480896, + 0.16188287734985352, + 0.13454246520996094, + 0.14753298461437225, + 0.14504572749137878, + 0.06676968187093735, + 0.09772863239049911, + 0.0585557222366333, + 0.12863430380821228, + 0.10292482376098633, + 0.07993738353252411, + 0.03641533851623535, + 0.04985213279724121, + 0.1516878604888916 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.3249012231826782, + -0.9610954523086548, + 2.054325819015503, + 1.018629789352417, + 2.393625497817993, + 2.2159433364868164, + 2.098443031311035, + -0.49140021204948425, + 0.044472046196460724, + 0.8045110702514648, + 1.4511574506759644, + -0.3077920377254486, + -0.052587300539016724, + 0.036380887031555176, + 0.7875679731369019, + -0.3928021192550659, + -0.09456783533096313, + -1.4901847839355469, + 1.1711406707763672, + -2.1349995136260986, + -2.076193332672119 + ], + [ + -1.4271517992019653, + -1.114187479019165, + 1.618905782699585, + 1.0621737241744995, + 1.6005314588546753, + 3.017716407775879, + 2.3444156646728516, + -1.0368298292160034, + 0.6155713796615601, + 1.0049463510513306, + 1.3708372116088867, + -0.27396243810653687, + -0.19098177552223206, + -0.5735776424407959, + 0.863328218460083, + -0.4340485632419586, + 0.09502258896827698, + -1.8856427669525146, + 0.9514620304107666, + -2.3912711143493652, + -2.6571128368377686 + ], + [ + -1.0088777542114258, + -1.803802251815796, + 2.6037826538085938, + 0.763943612575531, + 1.7241847515106201, + 2.433204174041748, + 2.273603916168213, + 0.06359493732452393, + -0.17274422943592072, + -0.23299545049667358, + 1.7126038074493408, + -0.3797045052051544, + 0.7567765712738037, + -0.9981052875518799, + 1.0371832847595215, + -0.5120593309402466, + -0.059397101402282715, + -1.9757622480392456, + 0.7378945350646973, + -2.6195693016052246, + -2.178041458129883 + ], + [ + -0.09093856811523438, + -0.14735746383666992, + 1.350311517715454, + 1.477137565612793, + 2.134960174560547, + 1.7331886291503906, + 2.24615740776062, + -1.7148966789245605, + -0.3949584364891052, + 0.4433448612689972, + 0.9034961462020874, + -0.23445925116539001, + -0.6371403932571411, + -0.038229942321777344, + 2.010648250579834, + 0.14472149312496185, + -0.26303571462631226, + -2.3003270626068115, + 0.565727949142456, + -2.586710214614868, + -2.098479747772217 + ], + [ + -1.8506121635437012, + -1.4700038433074951, + 2.4313056468963623, + 1.6264371871948242, + 1.1548577547073364, + 2.9678268432617188, + 2.990251302719116, + -0.25485146045684814, + -0.36889538168907166, + -0.14847010374069214, + 0.6971698999404907, + -0.6752342581748962, + 0.44545215368270874, + 0.663374125957489, + 0.6998850107192993, + -0.6500043869018555, + -0.04876291751861572, + -2.446995258331299, + 0.7760334014892578, + -1.38918137550354, + -1.5372083187103271 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.053252458572387695, + -0.006605625152587891, + 1.0103552341461182, + 0.25736117362976074, + 1.010573148727417, + 1.4690883159637451, + 0.0, + -0.0840916633605957, + -0.24044206738471985, + -0.06801849603652954, + 0.061444878578186035, + 0.11697901785373688, + 0.03979811072349548, + 0.0, + 0.054485201835632324, + -0.11758939921855927, + 0.31161248683929443, + -0.602575421333313, + -0.021049559116363525, + -1.3257782459259033, + 0.0 + ] + ], + "predicted_scores": [ + 5.009342670440674, + 4.948630332946777, + 4.570794105529785, + 4.540657043457031, + 4.3454999923706055, + 4.340945720672607, + 4.03071403503418, + 3.644404888153076 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "f054ce13f25dcda8808ebde2bee98ca84bfbdc3fc328a0eb831e21faada3fa4c", + "stored_base_utility": 0.13993772864341736, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09695577621459961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09695577621459961 + }, + "base_progress": 0.09695577621459961, + "base_success": false, + "base_utility": 0.09695577621459961, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.023269018158316612, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023269018158316612 + }, + { + "contact_quality": null, + "progress": 0.019655028358101845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019655028358101845 + }, + { + "contact_quality": null, + "progress": 0.0087813138961792, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0087813138961792 + }, + { + "contact_quality": null, + "progress": 0.00755995512008667, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00755995512008667 + }, + { + "contact_quality": null, + "progress": 0.0067829289473593235, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0067829289473593235 + }, + { + "contact_quality": null, + "progress": 0.007289707660675049, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007289707660675049 + }, + { + "contact_quality": null, + "progress": 0.007393876556307077, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007393876556307077 + }, + { + "contact_quality": null, + "progress": 0.006907304283231497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006907304283231497 + } + ], + "candidate_progress": [ + 0.023269018158316612, + 0.019655028358101845, + 0.0087813138961792, + 0.00755995512008667, + 0.0067829289473593235, + 0.007289707660675049, + 0.007393876556307077, + 0.006907304283231497 + ], + "candidate_source_chart_ids": [ + "ms-206099bab6bfe7c84fce", + "ms-6e27d292a9cbea108fc9", + "ms-6e27d292a9cbea108fc9", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e30fd80adc089e9f697a", + "generated_tangents": [ + [ + 0.6366334557533264, + -0.11214189976453781, + -0.4352327287197113, + 0.7022861242294312, + 0.18992498517036438, + -0.21438473463058472, + -1.4528279304504395, + -0.11280220746994019, + 0.31944581866264343, + -0.01835080236196518, + 1.1632746458053589, + -0.19994431734085083, + -0.3558635711669922, + -0.00420457124710083, + -1.0711610317230225, + 0.19876058399677277, + 1.2058967351913452, + 0.17612312734127045, + -0.0037953853607177734, + 0.8510705828666687, + 0.757475733757019 + ], + [ + 0.6585342288017273, + -0.07911577820777893, + -0.5741208791732788, + 0.9375303983688354, + 0.1638108491897583, + -0.8045713901519775, + -1.2619781494140625, + -0.13599571585655212, + 0.40331730246543884, + 0.05331644415855408, + 1.224998116493225, + -0.17791005969047546, + -0.42211830615997314, + -0.6020296216011047, + -1.0580703020095825, + 0.20305536687374115, + 1.3169106245040894, + 0.062639981508255, + 0.12919948995113373, + 0.728853166103363, + 0.4308769702911377 + ], + [ + 0.12071378529071808, + 0.059990979731082916, + -0.6425563097000122, + 1.2545583248138428, + 0.4109281003475189, + -0.17255926132202148, + -1.140244483947754, + -0.349547415971756, + 0.40955424308776855, + 0.14000645279884338, + 1.4145702123641968, + -0.18921741843223572, + 0.016987502574920654, + -0.194206103682518, + -1.1323565244674683, + 0.17514415085315704, + 1.281287670135498, + 0.19112029671669006, + 0.1002194881439209, + 0.47168827056884766, + 0.5684411525726318 + ], + [ + 1.006769061088562, + -0.12850505113601685, + -0.028462618589401245, + 1.168389081954956, + -0.12616848945617676, + -0.8076877593994141, + -1.4858224391937256, + -0.3880215585231781, + 0.842804491519928, + 0.21694205701351166, + 1.554677128791809, + 0.2051052302122116, + 0.5004428625106812, + -0.5582298636436462, + -1.3873852491378784, + 0.1259801685810089, + 1.7126113176345825, + 0.6904522776603699, + 0.5261470675468445, + 0.9457487463951111, + 1.440118432044983 + ], + [ + 1.0136741399765015, + -0.18631194531917572, + -0.0763501226902008, + 1.2448190450668335, + 0.15769533812999725, + -0.3354029357433319, + -1.5129380226135254, + -0.4687686264514923, + 0.8125385642051697, + 0.3846355378627777, + 1.539689064025879, + 0.115292489528656, + 0.4957970678806305, + -0.7441828846931458, + -1.591048240661621, + 0.0773932933807373, + 1.7895101308822632, + 0.7715845108032227, + 0.3651684820652008, + 0.7716142535209656, + 1.3820078372955322 + ], + [ + 1.105075716972351, + -0.18184205889701843, + -0.16723933815956116, + 0.927869439125061, + 0.06029707193374634, + -1.5241950750350952, + -1.7320661544799805, + -0.46492311358451843, + 0.895631730556488, + 0.45828625559806824, + 1.648598313331604, + 0.16798025369644165, + 0.4141702950000763, + -0.5209448933601379, + -1.7534955739974976, + -0.09653037786483765, + 1.6231496334075928, + 0.5987874865531921, + 0.5790414810180664, + 0.9431485533714294, + 1.6359766721725464 + ], + [ + 0.9705630540847778, + -0.22713980078697205, + 0.02578490972518921, + 1.078351378440857, + -0.0194435715675354, + -0.8012266159057617, + -1.3823676109313965, + -0.4644630253314972, + 0.8608686327934265, + 0.3069694936275482, + 1.529152274131775, + 0.18881826102733612, + 0.41185715794563293, + -0.6967524886131287, + -1.5064692497253418, + 0.028784722089767456, + 1.7600440979003906, + 0.6636291742324829, + 0.4427874684333801, + 0.9378055930137634, + 1.3367422819137573 + ], + [ + 0.8924802541732788, + -0.19990727305412292, + -0.023123115301132202, + 1.1342636346817017, + -0.09567445516586304, + -0.7180756330490112, + -1.235316514968872, + -0.5094244480133057, + 0.9015052318572998, + 0.27604198455810547, + 1.4410655498504639, + 0.30212128162384033, + 0.6821057796478271, + -0.8145237565040588, + -1.5531606674194336, + 0.01960892602801323, + 1.6762187480926514, + 0.7282889485359192, + 0.4589124917984009, + 0.8844152092933655, + 1.1597541570663452 + ] + ], + "generated_utilities": [ + 0.023269018158316612, + 0.019655028358101845, + 0.0087813138961792, + 0.00755995512008667, + 0.0067829289473593235, + 0.007289707660675049, + 0.007393876556307077, + 0.006907304283231497 + ], + "group_id": "ms-e30fd80adc089e9f697a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.12557005882263184, + 0.11941087245941162, + 0.10894306749105453, + 0.060310762375593185, + 0.18920521438121796, + 0.11678676307201385, + 0.14083251357078552, + 0.12988364696502686, + 0.18350017070770264, + 0.1286114901304245, + 0.06744732707738876, + 0.04428015276789665, + 0.08399941772222519, + 0.0742691159248352, + 0.05598282814025879, + 0.09703926742076874 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.044197678565979004, + -0.12298870831727982, + -0.37176433205604553, + 0.5503368377685547, + 0.5411782264709473, + 0.6186751127243042, + 0.0, + 0.28674131631851196, + 0.08053566515445709, + 0.3295105993747711, + 0.8157306909561157, + 0.1715114712715149, + -0.43090736865997314, + 0.0, + -0.5639103651046753, + -0.48903688788414, + 0.7485904693603516, + -0.4474331736564636, + -0.18787053227424622, + 1.2062294483184814, + 0.0 + ], + [ + 1.3153167963027954, + 0.7455203533172607, + -1.0113484859466553, + 2.097999334335327, + 0.7455009818077087, + 0.3843202590942383, + -0.16713692247867584, + -0.8400754928588867, + -0.3845864534378052, + 1.065396785736084, + 1.712794542312622, + 0.803345263004303, + 0.6529266238212585, + 0.6022698879241943, + -1.1222561597824097, + 0.06260442733764648, + 1.101211428642273, + -0.133493572473526, + -0.8626402616500854, + 1.8406728506088257, + 0.8011860847473145 + ], + [ + 0.23532289266586304, + -0.516789972782135, + -0.3119599223136902, + 1.501338005065918, + 1.4930639266967773, + -0.419913113117218, + -0.6910054683685303, + 0.30246037244796753, + 0.041916340589523315, + 1.2685117721557617, + 2.5912232398986816, + 1.2864888906478882, + 0.3821069598197937, + 0.4677737355232239, + -1.84663724899292, + -0.8953573703765869, + 2.3256053924560547, + 0.18948839604854584, + -0.5580936670303345, + 1.528401494026184, + 1.4914395809173584 + ], + [ + 0.6903802752494812, + 0.3962534964084625, + -0.8207939863204956, + 0.5973623991012573, + 1.310564398765564, + 0.6411572694778442, + -0.2032768577337265, + 0.25504088401794434, + 0.6581551432609558, + 0.3938753306865692, + 1.0127826929092407, + 1.6796927452087402, + 0.4867250621318817, + 0.05769670009613037, + -2.026149034500122, + -0.17722094058990479, + 0.9414948225021362, + 0.6044259667396545, + 0.03290170431137085, + 2.9423327445983887, + -0.060306549072265625 + ], + [ + 0.47530579566955566, + -0.6085019111633301, + 0.12691906094551086, + 0.5673632025718689, + 0.08076304197311401, + 0.8255819082260132, + -0.9781221151351929, + -0.796129584312439, + -0.6745100021362305, + 0.7784937024116516, + 1.7613102197647095, + 1.4703174829483032, + -0.36175650358200073, + 0.16939619183540344, + -0.9399533271789551, + -1.2696547508239746, + 1.3314080238342285, + -0.3690394163131714, + -1.0323668718338013, + 2.305772304534912, + 0.27392566204071045 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.6770331859588623, + 0.09077263623476028, + -0.3141186833381653, + 1.5503368377685547, + 1.0133107900619507, + 0.12400172650814056, + -0.2399108111858368, + -0.4914711117744446, + -0.11917559802532196, + 0.7716305255889893, + 1.8157306909561157, + 0.7167991995811462, + 0.5690926313400269, + 0.6154549717903137, + -1.5639103651046753, + -0.5166093111038208, + 1.7485904693603516, + -0.15158627927303314, + -0.46638885140419006, + 2.2062294483184814, + 0.7902159690856934 + ], + [ + 0.9456368684768677, + -0.3964601159095764, + -0.5845282077789307, + 1.2133667469024658, + 0.5843623876571655, + -0.02685314416885376, + -0.006928771734237671, + -0.2159697711467743, + -0.3017333149909973, + 0.40500110387802124, + 1.9734654426574707, + 0.46303731203079224, + 0.33380845189094543, + 0.7758987545967102, + -2.060479164123535, + -0.955610990524292, + 1.942940592765808, + -0.23636659979820251, + -0.10507094860076904, + 1.8367226123809814, + 1.2234702110290527 + ] + ], + "predicted_scores": [ + 5.982050895690918, + 5.516965866088867, + 4.392928600311279, + 4.120024681091309, + 4.05231237411499, + 3.795178174972534, + 3.6413815021514893, + 3.2022411823272705 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "ca8873454d7cf9723b1e743355bf255813544ae3212d80abc954fd347d7adf27", + "stored_base_utility": 0.12557005882263184, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07411724328994751, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07411724328994751 + }, + { + "contact_quality": null, + "progress": 0.09047380834817886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09047380834817886 + }, + { + "contact_quality": null, + "progress": 0.10569536685943604, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10569536685943604 + }, + { + "contact_quality": null, + "progress": 0.14950889348983765, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14950889348983765 + }, + { + "contact_quality": null, + "progress": 0.09969130903482437, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09969130903482437 + }, + { + "contact_quality": null, + "progress": 0.08115202188491821, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08115202188491821 + }, + { + "contact_quality": null, + "progress": 0.13061003386974335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13061003386974335 + }, + { + "contact_quality": null, + "progress": 0.08743151277303696, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08743151277303696 + } + ], + "candidate_progress": [ + 0.07411724328994751, + 0.09047380834817886, + 0.10569536685943604, + 0.14950889348983765, + 0.09969130903482437, + 0.08115202188491821, + 0.13061003386974335, + 0.08743151277303696 + ], + "candidate_source_chart_ids": [ + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-19cb10d10516bd0216a6", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "generated_tangents": [ + [ + 1.6360913515090942, + -0.18474355340003967, + -0.03315272927284241, + -0.35874706506729126, + -0.23833909630775452, + 2.607821464538574, + 1.9053452014923096, + 2.068326711654663, + 0.44662436842918396, + -0.9570363759994507, + -0.5049992799758911, + -0.6835821270942688, + -2.8723480701446533, + -1.9566744565963745, + 2.0397720336914062, + 0.3789702355861664, + -1.0628761053085327, + 0.10153849422931671, + 0.13642095029354095, + -1.9212422370910645, + -1.6213682889938354 + ], + [ + 1.6279715299606323, + -0.04317447543144226, + -0.33085086941719055, + -0.22415505349636078, + -0.4503195285797119, + 2.551999568939209, + 1.8113257884979248, + 2.193518877029419, + 0.39876192808151245, + -1.102653980255127, + -0.5429314970970154, + -1.158248782157898, + -2.6006979942321777, + -2.550006628036499, + 1.8820801973342896, + 0.12466619908809662, + -1.0567195415496826, + 0.018112868070602417, + -0.06478580832481384, + -2.0482516288757324, + -1.9957387447357178 + ], + [ + 1.6535975933074951, + -0.0813259556889534, + -0.30771657824516296, + -0.26517102122306824, + -0.40425264835357666, + 2.509902238845825, + 2.004323720932007, + 2.105619192123413, + 0.40929552912712097, + -0.9626333713531494, + -0.636191189289093, + -0.9176632165908813, + -2.89888858795166, + -2.5282859802246094, + 1.9083274602890015, + 0.1351604163646698, + -0.9505917429924011, + 0.09362629055976868, + -0.04731979966163635, + -1.804837942123413, + -1.690934181213379 + ], + [ + 1.7347869873046875, + -0.1579766422510147, + -0.4947598874568939, + -0.055146366357803345, + -0.4505763649940491, + 2.7247345447540283, + 1.3791768550872803, + 2.0401041507720947, + 0.31636542081832886, + -1.0958385467529297, + -0.6046792268753052, + -0.8150632381439209, + -2.4669017791748047, + -1.9254041910171509, + 1.8212796449661255, + 0.2587243318557739, + -0.9763078093528748, + -0.07706935703754425, + -0.25398099422454834, + -1.5144741535186768, + -1.5001959800720215 + ], + [ + 1.7521071434020996, + -0.11762430518865585, + -0.216354638338089, + -0.23457713425159454, + -0.18162685632705688, + 2.2410361766815186, + 2.021794557571411, + 2.1394143104553223, + 0.4362010955810547, + -1.0363415479660034, + -0.697080671787262, + -0.9890108108520508, + -2.9245851039886475, + -2.2857704162597656, + 1.9089828729629517, + 0.16052402555942535, + -0.9754478335380554, + 0.13712504506111145, + -0.11094871163368225, + -1.9151535034179688, + -1.7413411140441895 + ], + [ + 1.4861899614334106, + -0.15332084894180298, + -0.3576917350292206, + -0.24738585948944092, + -0.22742795944213867, + 2.1283457279205322, + 1.9224722385406494, + 2.095451831817627, + 0.44908690452575684, + -1.2427468299865723, + -0.548572301864624, + -1.1474500894546509, + -2.6523094177246094, + -2.038431406021118, + 1.9554672241210938, + 0.2681941092014313, + -0.8971112370491028, + 0.2167607545852661, + -0.17372861504554749, + -1.7628464698791504, + -1.8130526542663574 + ], + [ + 1.969941258430481, + -0.055220190435647964, + -0.21834620833396912, + -0.3075636625289917, + -0.4150499701499939, + 2.355985164642334, + 2.297492265701294, + 2.101154327392578, + 0.2317568063735962, + -1.137794017791748, + -0.5858346819877625, + -1.0449132919311523, + -3.045583963394165, + -2.1876230239868164, + 1.9643102884292603, + 0.13033004105091095, + -0.7871422171592712, + -0.08854912221431732, + 0.10471479594707489, + -1.6661694049835205, + -1.8294498920440674 + ], + [ + 2.2868597507476807, + -0.15640106797218323, + 0.23856395483016968, + -0.0673302561044693, + 0.002996191382408142, + 1.9117286205291748, + 1.9574682712554932, + 2.075385808944702, + 0.137465700507164, + -1.3756444454193115, + -0.7577681541442871, + -0.8775328397750854, + -2.7816855907440186, + -2.069620370864868, + 1.7326915264129639, + -0.07838009297847748, + -0.7329941391944885, + 0.2459639608860016, + -0.30372434854507446, + -1.9334983825683594, + -1.538361668586731 + ] + ], + "generated_utilities": [ + 0.07411724328994751, + 0.09047380834817886, + 0.10569536685943604, + 0.14950889348983765, + 0.09969130903482437, + 0.08115202188491821, + 0.13061003386974335, + 0.08743151277303696 + ], + "group_id": "ms-e9ed5d488afdfd6449d1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.22621813416481018, + 0.26591962575912476, + 0.22478708624839783, + 0.074407659471035, + 0.3134628236293793, + 0.19173070788383484, + 0.2438776195049286, + 0.26689577102661133, + 0.24477621912956238, + 0.24051737785339355, + 0.2738616168498993, + 0.04947046563029289, + 2.0, + 0.04723116010427475, + 0.08354459702968597, + 0.06540697813034058 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.6524851322174072, + -0.023243099451065063, + -0.05794792249798775, + 0.1783304661512375, + -0.3767814338207245, + 1.4893875122070312, + 0.0, + 1.024657964706421, + 0.3163401186466217, + -0.49971193075180054, + 0.05114656686782837, + -0.13273626565933228, + -1.9796202182769775, + 0.0, + 0.818771481513977, + -0.13705189526081085, + -0.21289467811584473, + 0.3914462625980377, + -0.6477798819541931, + -1.0717217922210693, + 0.0 + ], + [ + 2.363477945327759, + -1.042197585105896, + -0.8369066119194031, + 0.17341986298561096, + 0.6040215492248535, + 1.9536304473876953, + 2.2788424491882324, + 1.5015904903411865, + -0.07916292548179626, + -0.6816228628158569, + -1.3893373012542725, + -0.2375274896621704, + -2.5662975311279297, + -2.7615673542022705, + 1.151218295097351, + 0.29418760538101196, + -0.4583754539489746, + -0.4555256962776184, + -0.6620294451713562, + -2.534846067428589, + -1.916935682296753 + ], + [ + 2.0104916095733643, + -0.539351224899292, + 0.13553066551685333, + 0.9507300853729248, + 0.6636125445365906, + 1.5649433135986328, + 2.0495975017547607, + 2.832026958465576, + 1.1830073595046997, + -1.6508865356445312, + -0.26773035526275635, + -0.3988863527774811, + -2.8235108852386475, + -2.2897520065307617, + 1.4448039531707764, + -0.5226297378540039, + -2.1615242958068848, + 0.8214395046234131, + -0.755327045917511, + -2.571093797683716, + -1.7457375526428223 + ], + [ + 2.166736602783203, + -0.3969564735889435, + 0.5171664357185364, + 0.12789201736450195, + -0.9176173806190491, + 1.8307013511657715, + 1.834255576133728, + 1.3594005107879639, + 0.13751044869422913, + -0.9923363327980042, + -1.136932611465454, + -1.2923126220703125, + -2.445897102355957, + -2.4779889583587646, + 2.6843042373657227, + 0.10015779733657837, + -2.0766196250915527, + 0.1269409954547882, + 0.16698843240737915, + -1.2709240913391113, + -2.7113101482391357 + ], + [ + 1.0476312637329102, + -1.2484432458877563, + -0.8652145862579346, + 0.8706620335578918, + 0.6091254353523254, + 1.9634168148040771, + 2.105522394180298, + 1.2906999588012695, + -0.3275601267814636, + -1.0110361576080322, + -1.6514918804168701, + -1.376278281211853, + -2.345550060272217, + -2.4931728839874268, + 1.00270676612854, + 0.04506785422563553, + -0.9420130252838135, + -0.6248484253883362, + -1.469360589981079, + -3.059006929397583, + -3.495709180831909 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.6524851322174072, + -0.3197403848171234, + -0.07008056342601776, + 0.38692954182624817, + -0.013657404109835625, + 2.4893875122070312, + 2.3340368270874023, + 2.024657964706421, + 0.35216206312179565, + -1.4125897884368896, + -0.8262880444526672, + -0.49936625361442566, + -2.9796202182769775, + -2.9234721660614014, + 1.818771481513977, + 0.09626653790473938, + -1.2128946781158447, + 0.10123217105865479, + -0.7020833492279053, + -2.0717217922210693, + -2.5944693088531494 + ], + [ + 0.9812780022621155, + 0.2612687647342682, + 0.8081930875778198, + 0.3689844310283661, + -0.16453352570533752, + 2.8411240577697754, + 2.2401933670043945, + 1.7904633283615112, + -0.04837304353713989, + -0.6139949560165405, + -0.8549875020980835, + 0.3574656546115875, + -2.8409454822540283, + -2.1011672019958496, + 1.382948875427246, + 0.6656897068023682, + -1.5090553760528564, + 0.4825518727302551, + -0.3345044255256653, + -2.26375150680542, + -2.561906576156616 + ] + ], + "predicted_scores": [ + 4.722866058349609, + 4.667067050933838, + 4.577579498291016, + 4.560670375823975, + 4.475306510925293, + 4.332956314086914, + 3.9019358158111572, + 3.673485279083252 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "5608eea5b7e99b5fed6b316f742ecd34044df87286b2338705d8472043fc4375", + "stored_base_utility": 0.22621813416481018, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09514755010604858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09514755010604858 + }, + "base_progress": 0.09514755010604858, + "base_success": false, + "base_utility": 0.09514755010604858, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.042074382305145264, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.042074382305145264 + }, + { + "contact_quality": null, + "progress": 0.0623781681060791, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0623781681060791 + }, + { + "contact_quality": null, + "progress": 0.0336853452026844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0336853452026844 + }, + { + "contact_quality": null, + "progress": 0.00692981481552124, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00692981481552124 + }, + { + "contact_quality": null, + "progress": 0.03991667553782463, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03991667553782463 + }, + { + "contact_quality": null, + "progress": 0.03802645206451416, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03802645206451416 + }, + { + "contact_quality": null, + "progress": 0.07131137698888779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07131137698888779 + }, + { + "contact_quality": null, + "progress": 0.0379769392311573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0379769392311573 + } + ], + "candidate_progress": [ + 0.042074382305145264, + 0.0623781681060791, + 0.0336853452026844, + 0.00692981481552124, + 0.03991667553782463, + 0.03802645206451416, + 0.07131137698888779, + 0.0379769392311573 + ], + "candidate_source_chart_ids": [ + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ec013396e4aff6170ee1", + "generated_tangents": [ + [ + 0.4611658453941345, + -0.2873786985874176, + -0.07860538363456726, + 0.752954363822937, + 0.3158806264400482, + -1.1932144165039062, + -0.1740119457244873, + -0.6403985023498535, + 0.8440989255905151, + 1.2669352293014526, + -0.9001718759536743, + -0.8188923597335815, + -0.23858115077018738, + 1.0344558954238892, + -1.2292507886886597, + 0.6932545900344849, + 0.7712217569351196, + 0.6993335485458374, + 0.37747806310653687, + -2.2667934894561768, + -0.9057581424713135 + ], + [ + 0.39145201444625854, + -0.343828022480011, + 0.15941423177719116, + 0.7066751718521118, + 0.14841711521148682, + -0.40486249327659607, + -0.396170973777771, + -1.064365267753601, + 0.946660578250885, + 1.1499437093734741, + 0.005964670330286026, + -1.1780930757522583, + -0.300137460231781, + 0.7000108957290649, + -1.237982988357544, + 0.7131509780883789, + 0.7316395044326782, + 0.7953580021858215, + -0.03141322731971741, + -1.3973627090454102, + -0.8751441240310669 + ], + [ + 0.4933302402496338, + -0.44363027811050415, + -0.5209032297134399, + 0.6641716957092285, + 0.1200324147939682, + -0.9006333351135254, + -0.2204020619392395, + -0.599134087562561, + 0.9412590265274048, + 1.5974762439727783, + -0.8537541031837463, + -0.7195894122123718, + 0.21629825234413147, + 0.8109884262084961, + -0.9227805733680725, + 0.6385137438774109, + 0.8771730065345764, + 0.5583455562591553, + 0.38907456398010254, + -2.736330986022949, + -0.6876829862594604 + ], + [ + 0.580592930316925, + -0.4862269163131714, + -0.2889409363269806, + 0.5998220443725586, + -0.006053805351257324, + -1.1369870901107788, + 0.5363792777061462, + -0.7946500778198242, + 1.0766104459762573, + 1.6346819400787354, + -1.053938388824463, + -0.575809895992279, + -0.29470497369766235, + 0.9115384817123413, + -0.5781656503677368, + 0.5710615515708923, + 0.7287531495094299, + 0.40843814611434937, + 0.09897521138191223, + -2.882838249206543, + -0.6922982931137085 + ], + [ + 0.4615640640258789, + -0.5395740866661072, + -0.4543243944644928, + 0.7728146314620972, + -0.055400550365448, + -0.8873399496078491, + -0.1899268627166748, + -0.7802066802978516, + 0.9693182706832886, + 1.6709871292114258, + -0.8543050289154053, + -0.7539804577827454, + 0.06042942404747009, + 0.8117468357086182, + -1.0044642686843872, + 0.688340961933136, + 0.9000950455665588, + 0.5628482103347778, + 0.24863645434379578, + -2.7251012325286865, + -0.5936688184738159 + ], + [ + 0.469229519367218, + -0.49117475748062134, + -0.5210379362106323, + 0.7249901294708252, + 0.04420997202396393, + -0.7754863500595093, + -0.16021692752838135, + -0.704777717590332, + 0.9539754390716553, + 1.630562663078308, + -0.8701502084732056, + -0.7330318093299866, + 0.02480393648147583, + 0.8277038931846619, + -0.9628077149391174, + 0.6552332043647766, + 0.9154318571090698, + 0.545397937297821, + 0.26340970396995544, + -2.6805334091186523, + -0.6075135469436646 + ], + [ + 0.19687849283218384, + -0.6519871950149536, + 0.2553129196166992, + 0.574405312538147, + 0.1867075264453888, + -0.6016026735305786, + -0.4330863356590271, + -0.9723740816116333, + 1.0654808282852173, + 0.950093150138855, + -0.17083856463432312, + -1.5525258779525757, + -0.48422351479530334, + 0.5935631990432739, + -1.1726895570755005, + 0.6031124591827393, + 0.8103089332580566, + 0.9438478350639343, + -0.31401270627975464, + -1.7797350883483887, + -0.5831897258758545 + ], + [ + 0.41737931966781616, + -0.540668785572052, + -0.6665561199188232, + 0.5068308115005493, + 0.22947601974010468, + -0.8732736110687256, + -0.032761573791503906, + -0.6664251089096069, + 0.9593477845191956, + 1.7728954553604126, + -1.0882853269577026, + -0.7038149833679199, + 0.3145146667957306, + 0.8262359499931335, + -1.0099703073501587, + 0.7345770001411438, + 0.9556244015693665, + 0.45601099729537964, + 0.2565111815929413, + -3.041599750518799, + -0.5785009860992432 + ] + ], + "generated_utilities": [ + 0.042074382305145264, + 0.0623781681060791, + 0.0336853452026844, + 0.00692981481552124, + 0.03991667553782463, + 0.03802645206451416, + 0.07131137698888779, + 0.0379769392311573 + ], + "group_id": "ms-ec013396e4aff6170ee1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.010581731796264648, + 0.049839019775390625, + 0.06402494758367538, + 0.012959301471710205, + 0.11614581197500229, + 0.012509982101619244, + 0.09607907384634018, + 0.1605721116065979, + 0.1708059310913086, + 0.055588722229003906, + 0.2052522599697113, + 0.0690404623746872, + 0.11059975624084473, + 0.03047112748026848, + 0.24502670764923096, + 0.22897732257843018 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.16510963439941406, + -0.011389851570129395, + -0.16074393689632416, + 0.8547754287719727, + -0.09704409539699554, + 0.09471678733825684, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + -0.29076793789863586, + 0.0122823566198349, + 0.059903502464294434, + 0.0, + 0.13894814252853394, + 0.2961620092391968, + 0.025543689727783203, + 0.01881265640258789, + 0.052319109439849854, + -1.1815040111541748, + 0.0 + ], + [ + -0.6906574964523315, + 0.5966935753822327, + 0.1873793751001358, + 1.8547754287719727, + 0.16492335498332977, + 0.4949548840522766, + 0.624785304069519, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233, + -0.44174641370773315, + 0.866999626159668, + 1.0255436897277832, + 1.018812656402588, + 0.6033775210380554, + -2.181504011154175, + -0.4328075349330902 + ], + [ + -0.68684321641922, + 0.35380983352661133, + 0.41641929745674133, + 2.0589241981506348, + 0.20610027015209198, + 0.2780776619911194, + 0.6000017523765564, + -1.8711607456207275, + 1.71079421043396, + 2.484286069869995, + -0.12494567036628723, + 0.3767848014831543, + -0.59480881690979, + 1.3215627670288086, + -0.18650269508361816, + 0.5971667766571045, + 1.00047767162323, + 1.2905888557434082, + 0.7682754993438721, + -2.280778408050537, + -0.26339399814605713 + ], + [ + -1.0669870376586914, + 0.5963312387466431, + 0.24800772964954376, + 1.5324974060058594, + 0.049914538860321045, + 0.33535104990005493, + 0.9774697422981262, + -2.318913698196411, + 1.14731764793396, + 2.1673643589019775, + -0.3369846045970917, + -0.13161931931972504, + -0.7527784705162048, + 1.2647147178649902, + -0.5229569673538208, + 0.6999921798706055, + 0.654599666595459, + 0.9745514988899231, + 0.5995839238166809, + -2.3844704627990723, + -0.7404599189758301 + ], + [ + -0.660759449005127, + 0.560627818107605, + -0.19398994743824005, + 1.7401045560836792, + 0.006708189845085144, + 0.2270365059375763, + 0.9201371073722839, + -1.6917380094528198, + 1.8382139205932617, + 2.184737205505371, + -0.07137502729892731, + 0.13004565238952637, + -0.8376798629760742, + 1.3597201108932495, + -0.8258764147758484, + 0.521450400352478, + 1.4799309968948364, + 0.7742871046066284, + 1.007819414138794, + -2.398472785949707, + -0.7120757102966309 + ], + [ + -0.3511212170124054, + 0.6072589755058289, + 0.6547060012817383, + 2.306784152984619, + 0.03337647020816803, + 1.1676279306411743, + 0.48420602083206177, + -1.7057392597198486, + 1.7822117805480957, + 2.741852283477783, + -0.970782995223999, + 0.3188708424568176, + -0.1370975375175476, + 1.740339994430542, + 0.18519049882888794, + 0.2771400809288025, + 0.8003557920455933, + 0.8813821077346802, + 0.9829010963439941, + -1.9104408025741577, + -0.7000454664230347 + ], + [ + -1.1815872192382812, + 1.0524637699127197, + 0.5610368251800537, + 2.407832622528076, + -0.2576238512992859, + 0.1467430293560028, + 0.27881303429603577, + -1.6810011863708496, + 1.5921515226364136, + 2.929525375366211, + 0.19559642672538757, + 0.6552202105522156, + -1.3889155387878418, + 1.1560360193252563, + -0.4104887843132019, + 0.8887493014335632, + 1.283969759941101, + 0.7811828851699829, + -0.10630232095718384, + -1.4329519271850586, + -0.5646982192993164 + ], + [ + -1.4926873445510864, + 0.11535564064979553, + -0.6240940690040588, + 1.631784200668335, + 0.7930782437324524, + -0.35493338108062744, + 0.20104485750198364, + -1.5626122951507568, + 1.3110854625701904, + 3.0686309337615967, + -0.8419785499572754, + -0.0010939836502075195, + -0.49860918521881104, + 1.5949748754501343, + -0.938766598701477, + 1.762406587600708, + 0.625543475151062, + 0.20789378881454468, + 1.1513795852661133, + -2.3980860710144043, + -0.36970049142837524 + ], + [ + -1.0271471738815308, + 0.6133835911750793, + -0.3870598077774048, + 1.9299802780151367, + 0.9606215357780457, + -0.19876420497894287, + 0.29684266448020935, + -1.6164112091064453, + 1.3925114870071411, + 1.9474635124206543, + -1.2669438123703003, + -0.6821421980857849, + -0.7869906425476074, + 1.3602250814437866, + -0.14778050780296326, + 0.9938437938690186, + 0.6618988513946533, + 0.1858283281326294, + 0.8312947750091553, + -1.7043554782867432, + -0.23048581182956696 + ], + [ + -0.409736305475235, + 1.5572996139526367, + 1.0024399757385254, + 1.6385810375213623, + 0.83731609582901, + -0.1629825234413147, + -0.05952054262161255, + -1.0968945026397705, + 0.728961706161499, + 3.1370975971221924, + -0.44591856002807617, + 1.149456262588501, + -1.3439809083938599, + 1.738822340965271, + 0.4063607454299927, + 0.4372418224811554, + 1.276847004890442, + 1.4612212181091309, + 0.6917726993560791, + -1.914740800857544, + 0.5415292978286743 + ] + ], + "predicted_scores": [ + 3.913698673248291, + 3.4669954776763916, + 3.412660837173462, + 3.3919060230255127, + 3.289900302886963, + 3.232205867767334, + 3.2286953926086426, + 3.156780242919922 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296", + "selected_index": 0, + "state_hash": "f5a19cf07e6a6cc71c95ce01e976d34d1eeae68c8a93ecc985fd0a78cb0c1a5e", + "stored_base_utility": 0.010581731796264648, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.010638435371220112, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010638435371220112 + }, + { + "contact_quality": null, + "progress": 0.008713643066585064, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008713643066585064 + }, + { + "contact_quality": null, + "progress": 0.00959573220461607, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00959573220461607 + }, + { + "contact_quality": null, + "progress": 0.008243958465754986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008243958465754986 + }, + { + "contact_quality": null, + "progress": 0.011892616748809814, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011892616748809814 + }, + { + "contact_quality": null, + "progress": 0.015581388957798481, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015581388957798481 + }, + { + "contact_quality": null, + "progress": 0.01546515990048647, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01546515990048647 + }, + { + "contact_quality": null, + "progress": 0.014228303916752338, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014228303916752338 + } + ], + "candidate_progress": [ + 0.010638435371220112, + 0.008713643066585064, + 0.00959573220461607, + 0.008243958465754986, + 0.011892616748809814, + 0.015581388957798481, + 0.01546515990048647, + 0.014228303916752338 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-003a2225c0892dcda683", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-0285f5815c210ac00bec", + "ms-0285f5815c210ac00bec", + "ms-be219977a9db8dbb1e58" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ee30ebaf98622759d888", + "generated_tangents": [ + [ + 0.8207482695579529, + -0.7308071255683899, + 0.4467986226081848, + -0.5334179401397705, + 2.624948263168335, + 0.5768913626670837, + 1.3773689270019531, + 0.401658833026886, + -0.25230899453163147, + 0.5512906908988953, + 0.06494399905204773, + 0.5098161101341248, + -1.6437782049179077, + -0.833240807056427, + -0.7026548385620117, + 0.3165251314640045, + 1.241769790649414, + -0.34677284955978394, + 1.7037112712860107, + -2.0828754901885986, + -1.2344740629196167 + ], + [ + 0.7710335850715637, + -0.7674400806427002, + 0.5049636960029602, + -0.6269207000732422, + 2.7183949947357178, + 0.6444479823112488, + 1.2842581272125244, + 0.3312671482563019, + -0.23908039927482605, + 0.5727101564407349, + 0.09744159877300262, + 0.4604488015174866, + -1.661820411682129, + -0.7683685421943665, + -0.7152273654937744, + 0.27507102489471436, + 1.1888011693954468, + -0.3454775810241699, + 1.620046615600586, + -2.0872716903686523, + -1.3310085535049438 + ], + [ + 0.6475136280059814, + -0.8064519762992859, + 0.15487346053123474, + -0.6583085656166077, + 3.0244264602661133, + 0.6566344499588013, + 1.0700241327285767, + 0.1811072677373886, + 0.10540540516376495, + 0.697022020816803, + -0.19983208179473877, + -0.18796342611312866, + -1.7084141969680786, + -0.6339097023010254, + -0.7315196394920349, + 0.42225131392478943, + 1.3073558807373047, + -0.5681096911430359, + 1.7822608947753906, + -1.9828650951385498, + -0.6053508520126343 + ], + [ + 0.29915285110473633, + -0.8396425843238831, + 0.20700645446777344, + -0.414063036441803, + 2.8157193660736084, + 0.6158472895622253, + 0.6684472560882568, + 0.008738711476325989, + -0.05560358613729477, + 0.3467683792114258, + -0.06346379220485687, + 0.3750460743904114, + -1.9538394212722778, + -0.40554577112197876, + -0.431382954120636, + 0.3815584182739258, + 1.0538214445114136, + -0.5597825646400452, + 1.3493223190307617, + -2.371721029281616, + -0.935927152633667 + ], + [ + 0.41692185401916504, + -0.8120754957199097, + 0.8173114657402039, + -0.4108160734176636, + 2.753605365753174, + -0.0567987859249115, + 1.0259952545166016, + 0.23923759162425995, + 0.10675296187400818, + 0.27663564682006836, + -0.009385555982589722, + -0.25673994421958923, + -1.5019313097000122, + -1.0777069330215454, + -0.17191405594348907, + 0.5699157118797302, + 1.1479648351669312, + -0.564826250076294, + 1.2419342994689941, + -2.078266143798828, + -1.4185867309570312 + ], + [ + 0.9519003033638, + -0.8409497737884521, + 0.5436072945594788, + -0.33745941519737244, + 2.6066904067993164, + 1.4436250925064087, + 1.1239246129989624, + 0.553543210029602, + -0.11368918418884277, + 0.634536623954773, + 0.004998352378606796, + 0.24750663340091705, + -1.602338194847107, + -0.6645449995994568, + -0.3664837181568146, + 0.25969475507736206, + 0.9689200520515442, + -0.46969062089920044, + 1.658595085144043, + -2.1279382705688477, + -1.1879879236221313 + ], + [ + 0.9801964163780212, + -0.8598592281341553, + 0.5351187586784363, + -0.3519566059112549, + 2.586333751678467, + 1.453399896621704, + 1.0645297765731812, + 0.5686471462249756, + -0.10753000527620316, + 0.6423960328102112, + -0.048477739095687866, + 0.23440031707286835, + -1.6481211185455322, + -0.7002952098846436, + -0.3652022182941437, + 0.24543322622776031, + 1.0314792394638062, + -0.382108211517334, + 1.6628272533416748, + -2.21238374710083, + -1.1133114099502563 + ], + [ + 0.6216107606887817, + -0.5996929407119751, + 0.2552225589752197, + -0.4646618962287903, + 1.9257380962371826, + 0.7422885894775391, + 1.1481587886810303, + 0.2559194564819336, + 0.05205418914556503, + 0.47481974959373474, + 0.2907453179359436, + -0.0073999762535095215, + -1.6014293432235718, + -0.3677322268486023, + -0.4897036552429199, + 0.12193399667739868, + 1.0392929315567017, + -0.4940047264099121, + 1.0735681056976318, + -2.2955286502838135, + -1.1116067171096802 + ] + ], + "generated_utilities": [ + 0.010638435371220112, + 0.008713643066585064, + 0.00959573220461607, + 0.008243958465754986, + 0.011892616748809814, + 0.015581388957798481, + 0.01546515990048647, + 0.014228303916752338 + ], + "group_id": "ms-ee30ebaf98622759d888", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08405601978302002, + 0.09846439212560654, + 0.04595305770635605, + 2.0, + 0.08787640184164047, + 0.032481253147125244, + 0.08755072206258774, + 0.058936577290296555, + 0.10809408128261566, + 0.06723630428314209, + 0.0625009536743164, + 0.1415095031261444, + 0.08326160907745361, + 0.10944461822509766, + 0.0673408955335617, + 0.033725105226039886 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.4581542015075684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9406367540359497, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.470216989517212 + ], + [ + 1.6340367794036865, + -0.30366095900535583, + 0.2830277681350708, + -1.1627482175827026, + 3.58767032623291, + 0.6565636396408081, + 1.4321975708007812, + 0.8704822659492493, + 0.14327874779701233, + 1.0190284252166748, + 0.3078446090221405, + 0.7927982807159424, + -1.9305998086929321, + -1.7108646631240845, + 0.10729888081550598, + 0.9427294731140137, + 0.41770225763320923, + -1.7673753499984741, + 1.3209179639816284, + -2.8053033351898193, + -1.5330681800842285 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.00628507137298584, + 0.0595628023147583, + 0.39481037855148315, + -0.006641983985900879, + 1.8433213233947754, + 0.14830267429351807, + 0.0, + -0.24425631761550903, + 0.42130333185195923, + 0.41399094462394714, + 0.478841632604599, + -0.007461488246917725, + -0.5437495708465576, + 0.0, + 0.255215585231781, + -0.07331699132919312, + -0.1736500859260559, + -0.285358190536499, + 0.567956805229187, + -1.7103407382965088, + 0.0 + ], + [ + 1.0371580123901367, + -0.33467406034469604, + 0.9327473640441895, + -0.5396467447280884, + 2.1487879753112793, + 0.6294846534729004, + 0.6252679228782654, + 0.5140941143035889, + -0.20645928382873535, + -0.04528486728668213, + 0.14780870079994202, + 0.430325984954834, + -0.9293508529663086, + -1.5147639513015747, + 1.1104007959365845, + -0.05476301908493042, + 0.1755150556564331, + -1.3470268249511719, + 1.675830364227295, + -3.2944674491882324, + -1.9277915954589844 + ] + ], + "predicted_scores": [ + 5.316679000854492, + 5.273787975311279, + 5.226407527923584, + 4.974737644195557, + 4.743207931518555, + 4.480126857757568, + 4.326788902282715, + 4.008129119873047 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "636680137", + "selected_index": 0, + "state_hash": "2de607a9c69005130e51ff2b551c0eeedc85c58fef796682e6527bb5f719ff7b", + "stored_base_utility": 0.08405601978302002, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.026694953441619873, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026694953441619873 + }, + "base_progress": 0.026694953441619873, + "base_success": false, + "base_utility": 0.026694953441619873, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07018190622329712, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07018190622329712 + }, + { + "contact_quality": null, + "progress": 0.0666942223906517, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0666942223906517 + }, + { + "contact_quality": null, + "progress": 0.055874746292829514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.055874746292829514 + }, + { + "contact_quality": null, + "progress": 0.04966646432876587, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04966646432876587 + }, + { + "contact_quality": null, + "progress": 0.05415312573313713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05415312573313713 + }, + { + "contact_quality": null, + "progress": 0.07510259002447128, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07510259002447128 + }, + { + "contact_quality": null, + "progress": 0.060902971774339676, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060902971774339676 + }, + { + "contact_quality": null, + "progress": 0.060523372143507004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060523372143507004 + } + ], + "candidate_progress": [ + 0.07018190622329712, + 0.0666942223906517, + 0.055874746292829514, + 0.04966646432876587, + 0.05415312573313713, + 0.07510259002447128, + 0.060902971774339676, + 0.060523372143507004 + ], + "candidate_source_chart_ids": [ + "ms-1a4979e969aa331fb126", + "ms-53a86042af3c0887e556", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-38dff128761905bb1507" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-f3f1d07cfd59b823da47", + "generated_tangents": [ + [ + 1.470210075378418, + -0.01580379530787468, + -0.7283998727798462, + 0.055736392736434937, + 0.24472132325172424, + -1.1284493207931519, + -0.8600251078605652, + 0.3851696252822876, + 0.3760157823562622, + 0.12302403151988983, + -0.6882843971252441, + 0.061757877469062805, + -0.3473249077796936, + -0.3466219902038574, + 0.0132053904235363, + -0.13766108453273773, + 0.5573180913925171, + -0.3471088409423828, + 0.34647753834724426, + -0.36234331130981445, + 0.2744932174682617 + ], + [ + 1.5544204711914062, + -0.205126091837883, + -0.8731956481933594, + 0.2661944329738617, + -0.2335115373134613, + -0.869574785232544, + -0.7200550436973572, + 0.6391658782958984, + 0.3534799814224243, + -0.03703191131353378, + -0.453675240278244, + 0.24169741570949554, + -0.5494473576545715, + -0.46163302659988403, + 0.10946279764175415, + -0.02392527088522911, + 0.4711993932723999, + -0.33371812105178833, + 0.49010616540908813, + -0.19618189334869385, + 0.27094411849975586 + ], + [ + 1.329495906829834, + 0.027531512081623077, + -0.8849279880523682, + 0.3290958106517792, + 0.33424481749534607, + -1.3913294076919556, + -0.16062289476394653, + 0.22481419146060944, + 0.28562772274017334, + -0.21471309661865234, + -0.7556926012039185, + 0.1228054016828537, + -0.2151045799255371, + -0.048544853925704956, + 0.04217083379626274, + 0.12947796285152435, + 0.14930139482021332, + -0.47361016273498535, + 0.8292598724365234, + 0.23861980438232422, + 0.3284529149532318 + ], + [ + 1.4044618606567383, + 0.005583565682172775, + -0.7097935676574707, + 0.2668057680130005, + 0.25619059801101685, + -1.5797830820083618, + -0.1833033561706543, + 0.32690665125846863, + 0.2806861102581024, + -0.08240661770105362, + -0.6182770133018494, + -0.02741682529449463, + -0.10848620533943176, + -0.20988096296787262, + 0.03131095692515373, + 0.09103158116340637, + 0.20413587987422943, + -0.45211684703826904, + 0.4914407730102539, + 0.17220258712768555, + 0.2225353717803955 + ], + [ + 1.4604007005691528, + 0.05853419005870819, + -0.6606404781341553, + 0.36987581849098206, + 0.22956697642803192, + -1.6750952005386353, + -0.29594236612319946, + 0.22983431816101074, + 0.25976860523223877, + -0.0760643407702446, + -0.6543765068054199, + 0.08448074758052826, + 0.0864664614200592, + -0.3004239797592163, + 0.0643731877207756, + 0.09490689635276794, + 0.21696867048740387, + -0.41205132007598877, + 0.4456923007965088, + 0.07428351044654846, + 0.196548730134964 + ], + [ + 1.6114256381988525, + -0.22972920536994934, + -0.8154833316802979, + -0.016338661313056946, + 0.3510611951351166, + -0.9570256471633911, + -1.0122497081756592, + 0.44647374749183655, + 0.22919197380542755, + -0.22349590063095093, + -1.0027257204055786, + -0.20639553666114807, + -0.21644142270088196, + -0.27758657932281494, + 0.40001967549324036, + 0.10671809315681458, + -0.2702145576477051, + -0.3088212013244629, + 1.0284866094589233, + -1.0518794059753418, + -0.12535235285758972 + ], + [ + 1.6862655878067017, + -0.19620074331760406, + -0.7826117277145386, + -0.13826322555541992, + 0.46432220935821533, + -1.1855347156524658, + -1.1719279289245605, + 0.4701594412326813, + 0.24714942276477814, + -0.21692991256713867, + -1.0339958667755127, + -0.25352704524993896, + -0.2177039086818695, + -0.25296932458877563, + 0.45234382152557373, + 0.11219623684883118, + -0.23155592381954193, + -0.34111160039901733, + 1.069090485572815, + -1.0809414386749268, + -0.07589679956436157 + ], + [ + 1.6199008226394653, + 0.004985533654689789, + -0.6341373920440674, + 0.2836281359195709, + 0.20008470118045807, + -1.8935352563858032, + -0.021761029958724976, + 0.08875052630901337, + 0.3210280239582062, + 0.01811107248067856, + -0.6016245484352112, + -0.14748689532279968, + 0.020007193088531494, + -0.16946063935756683, + 0.07198408246040344, + -0.05364494025707245, + 0.07550117373466492, + -0.3124585747718811, + 0.7698216438293457, + -0.2978877127170563, + 0.28643712401390076 + ] + ], + "generated_utilities": [ + 0.07018190622329712, + 0.0666942223906517, + 0.055874746292829514, + 0.04966646432876587, + 0.05415312573313713, + 0.07510259002447128, + 0.060902971774339676, + 0.060523372143507004 + ], + "group_id": "ms-f3f1d07cfd59b823da47", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07767709344625473, + 0.07153411954641342, + 0.08480638265609741, + 0.06087514013051987, + 0.15252648293972015, + 0.0776997059583664, + 0.1476559340953827, + 0.17124073207378387, + 0.12327007949352264, + 0.12037092447280884, + 0.08562581241130829, + 0.09917601197957993, + 0.07137986272573471, + 0.039922356605529785, + 0.04496842622756958, + 0.05561637878417969 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.9664052724838257, + 0.20301294326782227, + -1.1196715831756592, + -0.17260856926441193, + -0.5341777801513672, + -2.678128242492676, + -2.431448459625244, + 1.5603530406951904, + 0.03585217520594597, + -0.3520800471305847, + -2.473386287689209, + 1.0162299871444702, + -0.7868502736091614, + -0.9102090001106262, + 0.9442894458770752, + -0.41473180055618286, + 0.7349075675010681, + -0.8863289952278137, + 0.8796874284744263, + 0.028114937245845795, + 0.24920660257339478 + ], + [ + 1.900488257408142, + -0.00838443636894226, + -1.2311820983886719, + -0.361179918050766, + -0.3987593352794647, + -2.551008462905884, + -2.701385021209717, + 1.7117550373077393, + -0.20423506200313568, + -0.3429180979728699, + -2.6584055423736572, + 0.8018259406089783, + -0.856230616569519, + -0.901038408279419, + 1.2131764888763428, + -0.6892215609550476, + 0.6726197600364685, + -0.7798839211463928, + 0.6868253946304321, + 0.07731680572032928, + 0.30107203125953674 + ], + [ + 1.6257587671279907, + 0.275219202041626, + -0.7838636636734009, + -0.5118123292922974, + -0.3429456353187561, + -2.342111349105835, + -2.7158243656158447, + 1.7038744688034058, + -0.1860528439283371, + -0.11617843806743622, + -2.1894750595092773, + 1.2306283712387085, + -0.8203369379043579, + -0.5157363414764404, + 0.7951346635818481, + -0.464976966381073, + 0.8823708295822144, + -0.7671404480934143, + 0.9700374007225037, + 0.21184596419334412, + -0.11244213581085205 + ] + ], + "predicted_scores": [ + 5.896638870239258, + 5.579070568084717, + 5.217473030090332, + 4.840173721313477, + 4.642172336578369, + 3.875715970993042, + 3.8069164752960205, + 3.643287420272827 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "3f477a29208b02558b99bd0f3305d0f1845c277d32e0b5afdafa71012673383b", + "stored_base_utility": 0.07767709344625473, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07080724835395813, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07080724835395813 + }, + "base_progress": 0.07080724835395813, + "base_success": false, + "base_utility": 0.07080724835395813, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06590981781482697, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06590981781482697 + }, + { + "contact_quality": null, + "progress": 0.05606792867183685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05606792867183685 + }, + { + "contact_quality": null, + "progress": 0.06440688669681549, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06440688669681549 + }, + { + "contact_quality": null, + "progress": 0.056011125445365906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.056011125445365906 + }, + { + "contact_quality": null, + "progress": 0.07260851562023163, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07260851562023163 + }, + { + "contact_quality": null, + "progress": 0.07159297168254852, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07159297168254852 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.07599961757659912, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07599961757659912 + } + ], + "candidate_progress": [ + 0.06590981781482697, + 0.05606792867183685, + 0.06440688669681549, + 0.056011125445365906, + 0.07260851562023163, + 0.07159297168254852, + 1.0, + 0.07599961757659912 + ], + "candidate_source_chart_ids": [ + "ms-dab80e1bb7987577e33d", + "ms-bfe867e8d518b9f8333a", + "ms-bfe867e8d518b9f8333a", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-081952a63087fc8f83cd", + "generated_tangents": [ + [ + 0.03375854343175888, + -0.04900426045060158, + 1.412408471107483, + -0.725378155708313, + -1.2397005558013916, + 0.14692215621471405, + -0.09779292345046997, + -0.13894176483154297, + 0.02070929855108261, + 0.1501135230064392, + -0.03075685352087021, + -1.318574070930481, + 0.20384398102760315, + 0.12198208272457123, + 0.06595951318740845, + -0.13841527700424194, + 0.14516130089759827, + 0.28113842010498047, + -0.8382633328437805, + -0.33905622363090515, + -0.09004026651382446 + ], + [ + -0.02888302505016327, + 0.00361696258187294, + 1.3578001260757446, + -0.7423804998397827, + -1.3380978107452393, + 0.24790945649147034, + -0.1035570502281189, + -0.03865240514278412, + 0.05394285172224045, + 0.16857779026031494, + -0.013176776468753815, + -1.4046103954315186, + 0.1299164593219757, + 0.24455326795578003, + -0.00902726873755455, + -0.11642973124980927, + 0.15607956051826477, + 0.36178815364837646, + -0.761135995388031, + -0.15585646033287048, + -0.08467072248458862 + ], + [ + -0.019810110330581665, + 0.007485195994377136, + 1.3393781185150146, + -0.7851817607879639, + -1.3124110698699951, + 0.2607835531234741, + -0.13392162322998047, + -0.0397568941116333, + 0.04677657037973404, + 0.16579826176166534, + 0.04543188214302063, + -1.401707410812378, + 0.14570406079292297, + 0.223608136177063, + -0.044904474169015884, + -0.11624743044376373, + 0.20332372188568115, + 0.3622475266456604, + -0.7356930375099182, + -0.14989958703517914, + -0.05062338709831238 + ], + [ + 0.04672608524560928, + -0.06441795825958252, + 1.452209234237671, + -0.6598774194717407, + -0.9569069743156433, + 0.3073361814022064, + -0.018876492977142334, + 0.0018425732851028442, + 0.054364390671253204, + 0.12689965963363647, + -0.10602091997861862, + -1.0276288986206055, + 0.2296665608882904, + 0.0032745152711868286, + -0.02115141972899437, + -0.20479948818683624, + 0.2083372324705124, + 0.532421886920929, + -0.6153956651687622, + -0.10267405211925507, + -0.1843487024307251 + ], + [ + 0.18768692016601562, + -0.08090052008628845, + 1.4347021579742432, + -0.6663864254951477, + -1.2294483184814453, + 0.08292755484580994, + 0.0018686950206756592, + -0.01563642919063568, + 0.09956304728984833, + 0.31591761112213135, + -0.014304853975772858, + -1.1225000619888306, + 0.4266897737979889, + -0.1096440851688385, + -0.00591939315199852, + -0.19483500719070435, + 0.13410940766334534, + 0.34820324182510376, + -0.6537925601005554, + -0.20497937500476837, + -0.020299524068832397 + ], + [ + 0.1646152287721634, + -0.18114203214645386, + 1.7602574825286865, + -0.620148777961731, + -1.4567086696624756, + 0.4231996536254883, + -0.2829062342643738, + 0.1680794060230255, + 0.1296631395816803, + 0.1684209704399109, + 0.05274005979299545, + -0.9986904859542847, + 0.13165435194969177, + 0.2405969798564911, + -0.10494042932987213, + -0.26405394077301025, + 0.0836157500743866, + 0.45522111654281616, + -0.9746655821800232, + -0.14122386276721954, + -0.11801105737686157 + ], + [ + -0.2704783082008362, + -0.1814546436071396, + 1.643018364906311, + -0.703923225402832, + -1.0324947834014893, + 0.9079009294509888, + -0.27390748262405396, + 0.007359892129898071, + 0.0805899053812027, + 0.18501780927181244, + -0.022130578756332397, + -1.0192875862121582, + 0.43525174260139465, + 0.07570907473564148, + -0.09988804161548615, + -0.2945282757282257, + -0.024339601397514343, + 0.4812130331993103, + -0.7962149977684021, + -0.3062295615673065, + 0.00640125572681427 + ], + [ + 0.08928649872541428, + -0.15247970819473267, + 1.6031819581985474, + -0.7279216051101685, + -1.0372061729431152, + 0.47576549649238586, + -0.16235554218292236, + 0.18651781976222992, + 0.05945601314306259, + 0.1023881584405899, + 0.076483353972435, + -1.1296696662902832, + 0.20422998070716858, + -0.07566863298416138, + -0.051602523773908615, + -0.2550467550754547, + -0.15156693756580353, + 0.5413152575492859, + -0.7147372961044312, + -0.09824743866920471, + 0.015150845050811768 + ] + ], + "generated_utilities": [ + 0.06590981781482697, + 0.05606792867183685, + 0.06440688669681549, + 0.056011125445365906, + 0.07260851562023163, + 0.07159297168254852, + 2.0, + 0.07599961757659912 + ], + "group_id": "ms-081952a63087fc8f83cd", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0006054937839508057, + 2.0, + 2.0, + 2.0, + 0.09041815996170044, + 0.03755016624927521, + 0.10023769736289978, + 0.08501018583774567, + 0.10316208004951477, + 0.13556814193725586, + 0.0687420666217804, + 0.16420090198516846, + 0.07440806925296783, + 0.03177754580974579, + 0.05512912571430206, + 0.05822136998176575 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05530714988708496, + -0.0015276968479156494, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.028481781482696533, + 0.0, + 0.07634823024272919, + 0.10118435323238373, + 0.01472395658493042, + -0.021184463053941727, + -1.3624117374420166, + 0.06702810525894165, + 0.0, + 0.064424529671669, + -0.13110561668872833, + -0.04677841067314148, + -0.0666094422340393, + -0.8611308336257935, + -0.02982822060585022, + 0.0 + ], + [ + -0.13999104499816895, + -0.16139763593673706, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.17549771070480347, + 0.0, + -0.15019088983535767, + 0.2207401990890503, + -0.06357991695404053, + 0.10177183151245117, + -1.3624117374420166, + -0.05549207329750061, + 0.0, + -0.1271420419216156, + -0.057942673563957214, + -0.279840350151062, + -0.11312717199325562, + -0.8611308336257935, + 0.1586582064628601, + 0.0 + ], + [ + 0.2317524552345276, + -0.2474955916404724, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.2997032403945923, + 0.0, + -0.3180278241634369, + 0.1991673707962036, + 0.26833486557006836, + 0.005899060517549515, + -1.3624117374420166, + 0.032782405614852905, + 0.0, + 0.17653466761112213, + 0.5550464987754822, + 0.13015127182006836, + -0.11312717199325562, + -0.8611308336257935, + 0.020897403359413147, + 0.0 + ], + [ + -0.7016198635101318, + 0.312343955039978, + 2.582733154296875, + -1.5283455848693848, + -2.530975103378296, + 0.7002967596054077, + 2.0224196910858154, + -0.031223494559526443, + -0.07945594191551208, + 0.7471928000450134, + -0.05505431815981865, + -2.3624117374420166, + 0.4337144196033478, + 1.385794997215271, + -0.18244768679141998, + -0.22281835973262787, + -0.5003347992897034, + 0.8868728280067444, + -1.8611308336257935, + 0.19736243784427643, + 0.2646070420742035 + ], + [ + -0.41503679752349854, + 0.3199126422405243, + 2.3341803550720215, + -1.5432449579238892, + -2.6036009788513184, + 0.8186125755310059, + 1.889835238456726, + -0.20609386265277863, + 0.0827421247959137, + 0.8381537795066833, + -0.0944519191980362, + -2.4805798530578613, + 0.3916063606739044, + 1.2392640113830566, + -0.2266693413257599, + -0.33516526222229004, + -0.3536234498023987, + 0.9749420881271362, + -2.1439476013183594, + 0.3261662721633911, + 0.13869160413742065 + ], + [ + -0.6778035759925842, + 0.3674989640712738, + 2.3216195106506348, + -1.6129508018493652, + -2.3233790397644043, + 0.37060582637786865, + 2.3460428714752197, + 0.2269098311662674, + -0.269074022769928, + 0.9810001254081726, + 0.023842941969633102, + -2.478595733642578, + 0.46739575266838074, + 1.1671457290649414, + -0.41542381048202515, + 0.030661001801490784, + -0.48762595653533936, + 0.7803086638450623, + -1.7778995037078857, + 0.10219369828701019, + 0.08611755073070526 + ], + [ + -1.0292376279830933, + 0.05653715133666992, + 3.017642021179199, + -1.2965388298034668, + -2.1434991359710693, + 0.8112732172012329, + 2.3553943634033203, + 0.14745590090751648, + -0.42473679780960083, + 0.8983602523803711, + -0.5389888286590576, + -1.9595260620117188, + 0.8665074110031128, + 1.7057592868804932, + -0.36306142807006836, + -0.6694610714912415, + -0.3699086308479309, + 0.7089512944221497, + -2.2496888637542725, + 0.1759258657693863, + 0.13106243312358856 + ], + [ + -0.2738665044307709, + 0.13772070407867432, + 2.3146424293518066, + -1.9326081275939941, + -2.760838508605957, + 0.10491085052490234, + 1.9823331832885742, + 0.3494338393211365, + 0.3550114631652832, + 0.7471076846122742, + -0.06264704465866089, + -2.2210469245910645, + 0.5386381149291992, + 1.431454062461853, + 0.1620958298444748, + -0.00858105719089508, + -0.8883554935455322, + 1.1764272451400757, + -1.8280467987060547, + 0.30690017342567444, + -0.32479730248451233 + ], + [ + -0.8548018932342529, + -0.2078946828842163, + 2.740286111831665, + -1.6552648544311523, + -2.470123291015625, + 1.0607044696807861, + 2.4267592430114746, + -0.16535070538520813, + -0.2640931308269501, + 0.2447068691253662, + 0.5616287589073181, + -2.808340311050415, + 1.0820176601409912, + 1.2134929895401, + 0.2653927206993103, + -0.454708456993103, + 0.03163468837738037, + 0.8172736763954163, + -1.8861355781555176, + 0.45078086853027344, + 0.29641321301460266 + ], + [ + -0.7025277018547058, + 0.5556451678276062, + 2.3374006748199463, + -1.9256110191345215, + -1.7495018243789673, + 0.9721168279647827, + 1.8326293230056763, + 0.5355191826820374, + -0.5788277983665466, + 0.19061005115509033, + -0.6322742700576782, + -3.031383514404297, + -0.1651831567287445, + 0.8584551215171814, + 0.2280312329530716, + 0.27221786975860596, + -0.3728860020637512, + 1.102063775062561, + -2.5995376110076904, + 0.7578867673873901, + -0.3784477412700653 + ], + [ + -7.212162017822266e-06, + 0.552659273147583, + 2.0397679805755615, + -1.8501746654510498, + -2.5457160472869873, + 1.0514975786209106, + 2.5270917415618896, + -0.7367458343505859, + 0.7676748037338257, + 0.7428221702575684, + -0.538831353187561, + -2.7423243522644043, + 0.07969537377357483, + 1.0069079399108887, + -0.8100140690803528, + -0.9174093008041382, + -0.9411231875419617, + 1.0430617332458496, + -1.4683198928833008, + 1.0334573984146118, + 0.4721405506134033 + ], + [ + -0.41159626841545105, + 0.41878798604011536, + 1.6363914012908936, + -2.4960880279541016, + -3.049954891204834, + -0.09567022323608398, + 2.3269245624542236, + 0.11159682273864746, + -0.7885257005691528, + 0.44582536816596985, + -0.47802892327308655, + -1.5574463605880737, + 1.0153859853744507, + 2.258706569671631, + -0.36591836810112, + 0.49115514755249023, + -0.9523675441741943, + 0.5892280340194702, + -2.1512646675109863, + -0.7744256854057312, + 0.592117428779602 + ], + [ + 0.1343594193458557, + 0.45838433504104614, + 2.6735801696777344, + -2.005676746368408, + -3.340463161468506, + 1.0152086019515991, + 1.429718255996704, + 0.7929191589355469, + -0.2598758339881897, + 1.6483545303344727, + -0.7411167025566101, + -2.8978958129882812, + 1.3759692907333374, + 1.4385837316513062, + 0.2961248755455017, + -0.042098239064216614, + -0.6380948424339294, + 1.0523303747177124, + -2.1783251762390137, + -0.045405834913253784, + 0.7960561513900757 + ] + ], + "predicted_scores": [ + 8.38859748840332, + 7.865166664123535, + 7.739889621734619, + 7.458866596221924, + 7.421019554138184, + 7.341989994049072, + 7.153594970703125, + 6.231618881225586 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2141802816", + "selected_index": 0, + "state_hash": "d1cf06bba4675ad6a6942dc4f6fbade890fbf21b5c3068f8bbe30cfc60c54d9b", + "stored_base_utility": 0.0006054937839508057, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-2ba02dcefa15a9371706", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-121bc2e6676c1ac10911", + "generated_tangents": [ + [ + -1.1428537368774414, + -0.33681827783584595, + 0.0769180953502655, + -0.5968054533004761, + -0.571502685546875, + 0.25492945313453674, + 1.8293170928955078, + -0.05916261672973633, + 0.2201186865568161, + -0.46115565299987793, + 0.1269274204969406, + -0.9132931232452393, + -0.09839442372322083, + 0.7744705677032471, + -0.06471119821071625, + -0.14481939375400543, + -0.20126263797283173, + 0.7576097846031189, + 0.2796335816383362, + 0.01933467388153076, + 0.22309339046478271 + ], + [ + -0.43985962867736816, + -0.023176170885562897, + 0.3756997585296631, + -0.24748322367668152, + -0.6525080800056458, + -0.2818254232406616, + 2.2367281913757324, + 0.08405214548110962, + 0.15601615607738495, + -0.06820777803659439, + -0.1090114489197731, + -0.1929427981376648, + -0.14624667167663574, + 0.92344731092453, + -0.08865220844745636, + 0.05167548730969429, + -0.40641969442367554, + 0.49160391092300415, + 0.0954277366399765, + 0.14885711669921875, + 0.6119989156723022 + ], + [ + -0.6306641101837158, + -0.1735406219959259, + 0.028355330228805542, + -0.8203088045120239, + -0.5015040040016174, + -0.03623749315738678, + 1.7209930419921875, + -0.052967727184295654, + 0.2429560273885727, + -0.29548078775405884, + -0.2207215130329132, + -0.5763352513313293, + -0.6688967943191528, + 0.4102092385292053, + 0.20549078285694122, + -0.10159313678741455, + -0.16518981754779816, + 0.543495774269104, + -0.06937819719314575, + 0.16048121452331543, + 0.6708518266677856 + ], + [ + -0.33497750759124756, + -0.034328460693359375, + -0.29067251086235046, + -0.6124038696289062, + -1.2642409801483154, + 0.3559309244155884, + 1.6858468055725098, + -0.061229854822158813, + 0.16996949911117554, + -0.09348472207784653, + 0.07988788187503815, + -0.7375262379646301, + -0.446968674659729, + 0.48836642503738403, + -0.1606544852256775, + -0.1050918698310852, + -0.5172917246818542, + 0.659605860710144, + -0.5452120900154114, + 0.11138984560966492, + 0.6398909091949463 + ], + [ + -0.8535206317901611, + -0.03217511251568794, + -0.45911410450935364, + -0.9529755115509033, + -0.764872670173645, + 0.03175976872444153, + 1.339918613433838, + -0.03322230279445648, + 0.3067573606967926, + -0.3839905261993408, + -0.28448960185050964, + -0.6497014164924622, + -0.5434535145759583, + 0.2834795415401459, + -0.3409568667411804, + -0.09876580536365509, + -0.08029772341251373, + 0.8834956288337708, + -0.282196581363678, + -0.38060763478279114, + 0.9672904014587402 + ], + [ + -0.9470295906066895, + -0.159866064786911, + -0.022715657949447632, + -0.5708368420600891, + -0.43117791414260864, + 0.011789724230766296, + 1.926875352859497, + -0.05345553159713745, + 0.2844993472099304, + -0.3308572471141815, + -0.25868457555770874, + -0.4164997339248657, + -0.8099724054336548, + 0.42035311460494995, + -0.0224350206553936, + -0.06410782039165497, + -0.04351837933063507, + 0.6609387397766113, + -0.12267306447029114, + 0.2131757140159607, + 1.0377119779586792 + ], + [ + -0.7540832161903381, + -0.18264012038707733, + -0.12035420536994934, + -0.6487157940864563, + -0.6398594975471497, + 0.12603461742401123, + 1.8564980030059814, + -0.11238265037536621, + 0.20606793463230133, + -0.31462809443473816, + -0.10243957489728928, + -0.5333985090255737, + -0.7816828489303589, + 0.5273482799530029, + -0.06624205410480499, + -0.14748543500900269, + -0.18167315423488617, + 0.6917931437492371, + -0.18754485249519348, + 0.11445602774620056, + 1.0951846837997437 + ], + [ + -0.6739012598991394, + -0.43815380334854126, + 0.03737080097198486, + -0.7623136043548584, + -0.26687294244766235, + -0.7074288129806519, + 1.153752088546753, + -0.20895999670028687, + 0.41342222690582275, + -0.5485003590583801, + -0.32419297099113464, + -0.5121111869812012, + 0.003171265125274658, + -0.05138956010341644, + -0.19933244585990906, + -0.012433379888534546, + 0.0928570032119751, + 0.8527677655220032, + -0.1134442389011383, + -0.04123127460479736, + 0.26661014556884766 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-121bc2e6676c1ac10911", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15086030960083008, + 0.08354458212852478, + 2.0, + 0.1453719437122345, + 0.19646552205085754, + 0.10343040525913239, + 0.1443832516670227, + 0.13976123929023743, + 0.17431864142417908, + 2.0, + 0.12949708104133606, + 2.0, + 2.0, + 0.11397694051265717, + 0.1271887570619583, + 0.025320395827293396 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.39665424823760986, + 0.08244594186544418, + -0.026901081204414368, + -0.959688663482666, + -1.2648048400878906, + 0.05568990111351013, + 0.0, + -0.048488348722457886, + 0.03576555848121643, + -0.012020409107208252, + 0.08576416969299316, + -0.25812816619873047, + 0.03589746356010437, + 0.0, + 0.0016258209943771362, + -0.04025750607252121, + 0.010670006275177002, + 0.5142596960067749, + 0.06220144033432007, + 0.07906517386436462, + 0.0 + ], + [ + -2.15720796585083, + 0.13025060296058655, + 0.41441529989242554, + -1.4498568773269653, + -2.6386165618896484, + 0.5572659969329834, + 0.7093658447265625, + -1.0835764408111572, + -0.896172285079956, + 0.07744038105010986, + 1.1224509477615356, + -1.6321502923965454, + 0.877888560295105, + 0.4978562295436859, + 0.5774499177932739, + 0.8093571662902832, + -0.8521628379821777, + 2.054164171218872, + -0.6021197438240051, + 1.0350128412246704, + 0.823830246925354 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.39665424823760986, + -0.2284793257713318, + -0.027931194752454758, + -0.959688663482666, + -1.2648048400878906, + -0.17197783291339874, + 0.0, + -0.06679429113864899, + 0.1646350920200348, + -0.11601293087005615, + 0.08576416969299316, + -0.25812816619873047, + 0.1479925513267517, + 0.0, + -0.04833713173866272, + 0.22201494872570038, + -0.16177666187286377, + 0.5142596960067749, + -0.10178101062774658, + 0.21631890535354614, + 0.0 + ], + [ + -0.8824495077133179, + 0.0034109503030776978, + -0.5904110074043274, + -1.400491714477539, + -2.4447474479675293, + -0.00789790228009224, + 1.6208159923553467, + 0.26819390058517456, + -0.23575741052627563, + -0.7184416651725769, + 0.9072303771972656, + -1.2077033519744873, + -0.21496516466140747, + 0.8258882761001587, + -0.4575563669204712, + -0.3655202388763428, + -0.5908474922180176, + 2.0210399627685547, + -1.2132787704467773, + 0.9719676971435547, + 0.24625355005264282 + ], + [ + -0.6953714489936829, + 0.4026735723018646, + 0.6648998856544495, + -1.356613278388977, + -2.953742742538452, + 0.037713564932346344, + 1.53464937210083, + 0.027994811534881592, + 0.03435474634170532, + -1.1939163208007812, + 0.9877840280532837, + -1.180513620376587, + 0.4064505994319916, + 0.5743880867958069, + 0.1558205783367157, + 0.3796072006225586, + -0.6676555275917053, + 1.4665511846542358, + 0.04170459508895874, + 0.17645099759101868, + 0.2388516366481781 + ], + [ + -1.4095232486724854, + -0.06358731538057327, + -0.7774888873100281, + -2.806776285171509, + -2.44041109085083, + -0.8258867859840393, + 2.383592128753662, + -0.093393474817276, + -0.24701429903507233, + -0.5286431312561035, + 0.7741218209266663, + -2.111828565597534, + -0.31807881593704224, + 1.3003538846969604, + -0.9306488037109375, + -0.8708829283714294, + -0.3694953918457031, + 2.169797420501709, + -0.7153271436691284, + 0.029519766569137573, + -0.5629553198814392 + ] + ], + "predicted_scores": [ + 3.663315534591675, + 2.5850982666015625, + 1.273209810256958, + 0.41306161880493164, + 0.03598073124885559, + -0.15662100911140442, + -0.1887272298336029, + -0.43141189217567444 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264", + "selected_index": 0, + "state_hash": "d1280a9e9362f8da3290de84fdd0f0b78475d470cd0230e590ccaf8f5428548a", + "stored_base_utility": 0.15086030960083008, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-22f153f0d4fa656018e6", + "generated_tangents": [ + [ + -0.24887557327747345, + -0.027390284463763237, + 0.5010677576065063, + -0.9348056316375732, + -1.5222833156585693, + -0.04675397276878357, + 0.5733861327171326, + -0.15539279580116272, + 0.08523792028427124, + 0.10814549028873444, + -0.18766281008720398, + -0.9836422204971313, + 0.05895397067070007, + 0.2705014944076538, + -0.14306843280792236, + -0.103380486369133, + -0.055262550711631775, + -0.014810875058174133, + -0.4295213222503662, + 0.024319976568222046, + -0.1443595290184021 + ], + [ + -0.6374868750572205, + -0.17066138982772827, + 0.5716707110404968, + -0.8789207935333252, + -1.7522239685058594, + -0.16476508975028992, + 0.44131967425346375, + -0.15306147933006287, + 0.058665692806243896, + -0.11219052225351334, + -0.07430637627840042, + -1.1802771091461182, + 0.40951308608055115, + 0.21393123269081116, + -0.2618885040283203, + -0.22483773529529572, + 0.10827963054180145, + 0.059042468667030334, + -0.25649404525756836, + -0.10458812117576599, + 0.4529889225959778 + ], + [ + -0.6623293161392212, + -0.07495532929897308, + 0.21865785121917725, + -1.1720353364944458, + -1.9080228805541992, + 0.6288790106773376, + -0.21692156791687012, + -0.04327043890953064, + 0.01988872140645981, + -0.02546333521604538, + 0.10489708185195923, + -1.2112760543823242, + 0.010973095893859863, + 0.2623225748538971, + -0.2310495674610138, + -0.34748461842536926, + 0.128828763961792, + -0.1510370522737503, + -0.2398608922958374, + 0.15933069586753845, + 0.51133131980896 + ], + [ + -0.47105252742767334, + -0.20062479376792908, + 0.5346060395240784, + -1.0590448379516602, + -1.8472039699554443, + 0.0377831757068634, + -0.05174052715301514, + 0.034425780177116394, + 0.027260316535830498, + -0.053590960800647736, + 0.14692896604537964, + -1.1483964920043945, + 0.03161340951919556, + 0.3398374915122986, + -0.2776331603527069, + -0.16293086111545563, + -0.075893834233284, + -0.055674538016319275, + -0.4315546154975891, + 0.21798521280288696, + 0.35815271735191345 + ], + [ + -0.6245862245559692, + -0.20884281396865845, + 0.6665024161338806, + -0.9951554536819458, + -1.9768967628479004, + 0.18941600620746613, + -0.0761062502861023, + -0.03308330476284027, + 0.02400665357708931, + -0.013298399746418, + 0.13496777415275574, + -1.254115343093872, + 0.07984712719917297, + 0.32589688897132874, + -0.2589893341064453, + -0.18994814157485962, + -0.045686766505241394, + -0.06592737138271332, + -0.43024200201034546, + 0.15589860081672668, + 0.16848532855510712 + ], + [ + -0.6612503528594971, + -0.20185846090316772, + 0.6815662980079651, + -1.04066801071167, + -1.7090041637420654, + 0.4628135859966278, + -0.09681999683380127, + -0.15471616387367249, + 0.04341607540845871, + 0.06519214808940887, + 0.14938311278820038, + -1.0014803409576416, + 0.19993892312049866, + 0.5284703969955444, + -0.23039305210113525, + -0.24038387835025787, + -0.13825036585330963, + -0.046910956501960754, + -0.34253668785095215, + -0.30616962909698486, + 0.05990007519721985 + ], + [ + -0.7707524299621582, + -0.05616701394319534, + 0.7455941438674927, + -0.9382203817367554, + -1.903231143951416, + -0.1696189045906067, + 0.017838507890701294, + -0.20468854904174805, + 0.11596565693616867, + 0.15395723283290863, + 0.08011648803949356, + -1.3235644102096558, + -0.08012253046035767, + 0.30108708143234253, + -0.3165774345397949, + -0.191693976521492, + -0.04941864311695099, + -0.0692630261182785, + -0.5177285075187683, + 0.3827189803123474, + 0.12820693850517273 + ], + [ + -0.6125665307044983, + -0.24611927568912506, + 0.7538248300552368, + -1.0436112880706787, + -2.0779056549072266, + -0.08610925078392029, + -0.04694008827209473, + -0.026910915970802307, + 0.049526311457157135, + 0.07596208900213242, + 0.06273334473371506, + -1.0408450365066528, + -0.15906131267547607, + 0.47979414463043213, + -0.09796953201293945, + -0.26035764813423157, + -0.37497830390930176, + -0.10670261085033417, + -0.4887915253639221, + -0.07884900271892548, + 0.03693243861198425 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-22f153f0d4fa656018e6", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 2.60770320892334e-06, + 3.129243850708008e-07, + 6.109476089477539e-07, + 2.0, + 0.20781123638153076, + 2.0, + 0.16136802732944489, + 0.15920132398605347, + 0.16996727883815765, + 0.14405608177185059, + 0.15874098241329193, + 0.08526891469955444, + 0.0828208178281784, + 2.0, + 0.034265756607055664, + 2.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.3937166929244995, + 0.17073306441307068, + -0.5225132703781128, + -0.8479964733123779, + -1.713785171508789, + -0.19215916097164154, + 0.0, + 0.8004336953163147, + 0.4925512969493866, + 0.3161412179470062, + 0.10414126515388489, + -0.9857386350631714, + -0.4383167624473572, + 0.0, + 0.05412057042121887, + 0.021832358092069626, + 0.5262433290481567, + -0.24631118774414062, + -0.20008409023284912, + -0.04257836565375328, + 0.0 + ], + [ + -1.3937166929244995, + 0.2384568303823471, + 0.3933030962944031, + -1.847996473312378, + -2.713785171508789, + 0.20405007898807526, + 1.2454378604888916, + 0.0006654821336269379, + -0.07783810049295425, + -0.002305123955011368, + 0.18527865409851074, + -1.9857386350631714, + -0.009703370742499828, + 0.7078995108604431, + -0.08121174573898315, + -0.031056255102157593, + -0.40927961468696594, + 0.5954965949058533, + -0.8039777874946594, + 0.05187072232365608, + 0.2283477634191513 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.490875720977783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4157990217208862, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4566955268383026 + ], + [ + -1.5219146013259888, + 0.23501667380332947, + 0.5663670301437378, + -1.892876386642456, + -2.744781494140625, + -0.04844887554645538, + 1.3053655624389648, + -0.11823110282421112, + -0.02477371320128441, + -0.17061075568199158, + 0.22949494421482086, + -1.854048252105713, + -0.025548618286848068, + 0.7958164215087891, + 0.12320645153522491, + -0.23218455910682678, + -0.6995364427566528, + 0.8081986308097839, + -0.558222234249115, + -0.18823161721229553, + 0.04578563570976257 + ], + [ + -1.608424186706543, + 0.022089049220085144, + 0.15639334917068481, + -1.982436180114746, + -2.6509876251220703, + -0.01820056140422821, + 0.95000159740448, + -0.08617040514945984, + -0.46914470195770264, + 0.3453706204891205, + -0.020491018891334534, + -1.738605260848999, + -0.0859340950846672, + 0.443394273519516, + 0.09907735884189606, + -0.38721394538879395, + -0.3415156602859497, + 0.524202287197113, + -0.7343614101409912, + -0.22556106746196747, + 0.4535057842731476 + ], + [ + -1.7957028150558472, + 0.593864381313324, + 0.06846150755882263, + -2.0563552379608154, + -2.2595136165618896, + -0.12188448011875153, + 0.9899688959121704, + -0.14783401787281036, + -0.22327497601509094, + -0.32116562128067017, + -0.07331573963165283, + -1.910165786743164, + 0.264303058385849, + 1.0567166805267334, + 0.016188345849514008, + 0.159999281167984, + -0.22885318100452423, + 0.705758273601532, + -0.8778964877128601, + 0.4381847381591797, + 0.19897118210792542 + ], + [ + -1.499897837638855, + -0.2043466418981552, + 0.42094048857688904, + -1.430327296257019, + -3.1506710052490234, + -0.08664844930171967, + 1.4003938436508179, + 0.24064289033412933, + -0.35471564531326294, + -0.37839531898498535, + 0.47059693932533264, + -1.7703200578689575, + 0.101543128490448, + 0.8470064401626587, + 0.16171279549598694, + -0.0019259154796600342, + -0.6455138325691223, + 0.5694639682769775, + -0.9997703433036804, + 0.5366826057434082, + -0.11106891930103302 + ], + [ + -1.8847153186798096, + 0.21543154120445251, + 0.9309400320053101, + -2.1095237731933594, + -2.136512517929077, + 0.6948637366294861, + 1.233637809753418, + 0.577111542224884, + -0.5727867484092712, + -0.6055393815040588, + 0.8738593459129333, + -2.6654083728790283, + -0.171565979719162, + 0.8934992551803589, + 0.17394188046455383, + 0.6209473609924316, + -0.1149752140045166, + 0.6160256266593933, + -0.5519447326660156, + 0.7195507287979126, + 0.5175096392631531 + ], + [ + -2.138639450073242, + 0.36738795042037964, + -0.33481496572494507, + -1.9255597591400146, + -2.2420146465301514, + 0.31966179609298706, + 1.3172208070755005, + -0.48551350831985474, + 0.27687638998031616, + -0.41806119680404663, + 0.05362778902053833, + -2.3665032386779785, + -0.47635504603385925, + 0.081889808177948, + 0.542697548866272, + -0.7215265035629272, + -0.906980037689209, + 0.6977691054344177, + -1.4041197299957275, + -0.11211369931697845, + 0.5149455070495605 + ], + [ + -1.4869698286056519, + -0.30130720138549805, + 1.28191077709198, + -0.9839465618133545, + -2.2501890659332275, + -0.20459605753421783, + 1.3607591390609741, + 0.6349532008171082, + -0.8627415895462036, + -0.623334527015686, + 0.21189503371715546, + -2.2093186378479004, + -0.4655338525772095, + 0.7016672492027283, + 0.6916180849075317, + -0.0807163268327713, + 0.1465948522090912, + 0.05083376169204712, + -0.8381134271621704, + 0.015291888266801834, + 0.05565965175628662 + ], + [ + -2.365525245666504, + 0.45377761125564575, + 0.2984481453895569, + -1.3690096139907837, + -2.843494176864624, + 0.32462257146835327, + 0.5082458257675171, + 0.7033615112304688, + 0.31817564368247986, + -0.4890395700931549, + 0.09036337584257126, + -2.904804229736328, + 0.560937762260437, + 1.649811029434204, + -0.9795587062835693, + -0.46419358253479004, + -0.4739357829093933, + 0.34800490736961365, + -1.0712809562683105, + 0.9167320132255554, + -0.48252081871032715 + ], + [ + -1.2987136840820312, + 0.7370830774307251, + -0.49668365716934204, + -1.201794147491455, + -3.552318572998047, + -0.5406968593597412, + 2.1693286895751953, + 0.5925421714782715, + 0.7903083562850952, + 0.991620659828186, + 0.7936639785766602, + -2.3832802772521973, + -0.621701180934906, + 0.43542394042015076, + -0.14415577054023743, + 0.15921397507190704, + -0.395956814289093, + -0.2778439521789551, + -1.069305419921875, + -0.5347616076469421, + -0.26581722497940063 + ] + ], + "predicted_scores": [ + 3.5058493614196777, + 3.4791247844696045, + 3.096482038497925, + 2.2233731746673584, + 2.211974620819092, + 1.8748934268951416, + 1.7593693733215332, + 1.1505695581436157 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007", + "selected_index": 0, + "state_hash": "3139b3ac90f6fb958f14d875d082c95c8569f039086e0789e6c9e079d602b945", + "stored_base_utility": 2.60770320892334e-06, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0320269912481308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0320269912481308 + }, + "base_progress": 0.0320269912481308, + "base_success": false, + "base_utility": 0.0320269912481308, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03511819243431091, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03511819243431091 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03361482918262482, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03361482918262482 + }, + { + "contact_quality": null, + "progress": 0.024507999420166016, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024507999420166016 + }, + { + "contact_quality": null, + "progress": 0.02013157308101654, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02013157308101654 + }, + { + "contact_quality": null, + "progress": 0.03976954519748688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03976954519748688 + }, + { + "contact_quality": null, + "progress": 0.03125926852226257, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03125926852226257 + }, + { + "contact_quality": null, + "progress": 0.017352595925331116, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017352595925331116 + } + ], + "candidate_progress": [ + 0.03511819243431091, + 1.0, + 0.03361482918262482, + 0.024507999420166016, + 0.02013157308101654, + 0.03976954519748688, + 0.03125926852226257, + 0.017352595925331116 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1", + "ms-2423891634e7b469eed8", + "ms-ad5107846b45eae5a0ab", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "generated_tangents": [ + [ + -0.004802003502845764, + 0.2759769558906555, + 0.6961096525192261, + -1.6054247617721558, + 0.24079948663711548, + 0.44251441955566406, + 0.6913051605224609, + -0.21571427583694458, + 0.21488012373447418, + 0.13592529296875, + -0.28689852356910706, + -0.7922946214675903, + 0.25793585181236267, + 0.3460880517959595, + 0.2650951147079468, + -0.18585707247257233, + -0.016776978969573975, + 0.05982615053653717, + -0.6826059818267822, + 0.22354018688201904, + 0.2553693652153015 + ], + [ + -0.04533401131629944, + 0.2646772861480713, + 0.5887936949729919, + -1.6549843549728394, + 0.3267868161201477, + 0.3581558167934418, + 0.31044822931289673, + -0.11962872743606567, + 0.20861420035362244, + 0.23288512229919434, + -0.17209821939468384, + -1.001847743988037, + 0.16749736666679382, + 0.5874131321907043, + 0.004180684685707092, + -0.21890732645988464, + -0.09830166399478912, + 0.20511867105960846, + -0.6198068857192993, + 0.19094139337539673, + 0.10831240564584732 + ], + [ + 0.04220472276210785, + 0.22109778225421906, + 0.8459436893463135, + -1.8022898435592651, + 0.2696196436882019, + 0.42059656977653503, + 0.5853767991065979, + -0.1801830530166626, + 0.19578561186790466, + 0.03888130933046341, + -0.07634315639734268, + -0.7004210948944092, + -0.04044097661972046, + 0.5887544751167297, + -0.06970055401325226, + -0.24952712655067444, + 0.0805043876171112, + 0.26777398586273193, + -0.7303264737129211, + 0.4835692048072815, + 0.367800235748291 + ], + [ + 0.047049954533576965, + 0.13327544927597046, + 0.8369796872138977, + -1.7121621370315552, + 0.4671766757965088, + 0.3856254518032074, + 0.8522987961769104, + -0.08665236830711365, + 0.2445538192987442, + 0.20203416049480438, + -0.2101040482521057, + -0.45600658655166626, + 0.41030821204185486, + 0.2601064443588257, + 0.0742850974202156, + -0.1906077265739441, + 0.11767930537462234, + 0.060356974601745605, + -0.5563361048698425, + 0.3837745785713196, + 0.4662744104862213 + ], + [ + 0.0006069839000701904, + 0.14383821189403534, + 0.877548336982727, + -1.7358375787734985, + 0.12807147204875946, + 0.3142473101615906, + 0.5067259669303894, + -0.10054326057434082, + 0.22263167798519135, + 0.06229785829782486, + -0.14894098043441772, + -0.7591889500617981, + 0.23409780859947205, + 0.30805134773254395, + 0.06809236854314804, + -0.3235054016113281, + -0.12104277312755585, + 0.29004836082458496, + -0.8910035490989685, + 0.4976123571395874, + 0.2881072759628296 + ], + [ + -0.02980591356754303, + 0.07565620541572571, + 0.8593282699584961, + -1.7921754121780396, + 0.2695343494415283, + 0.5140743255615234, + 0.5875495076179504, + -0.1574336290359497, + 0.2650357484817505, + 0.12073545157909393, + -0.23395144939422607, + -0.7751498818397522, + 0.42846986651420593, + 0.2149398922920227, + 0.01605881378054619, + -0.31510379910469055, + -0.07233808934688568, + 0.21103742718696594, + -0.6775739192962646, + 0.5546218156814575, + 0.290467232465744 + ], + [ + 0.02690904587507248, + 0.058692567050457, + 0.8474045991897583, + -1.8158884048461914, + 0.3521440923213959, + 0.5875942707061768, + 0.5325207710266113, + -0.020325347781181335, + 0.25149399042129517, + 0.0481838583946228, + -0.17914846539497375, + -0.7939544916152954, + 0.19633254408836365, + 0.35983195900917053, + -0.10683563351631165, + -0.3130522668361664, + -0.09634657204151154, + 0.24662891030311584, + -0.6263746619224548, + 0.6420769095420837, + 0.35792800784111023 + ], + [ + 0.10404586791992188, + 0.08910278975963593, + 1.0987014770507812, + -1.8449687957763672, + 0.5307130813598633, + 0.5096020698547363, + 0.7059164643287659, + -0.104676753282547, + 0.21691840887069702, + 0.05176727473735809, + -0.2859335243701935, + -0.5509306192398071, + -0.05278941988945007, + 0.3020511865615845, + -0.043806713074445724, + -0.29579946398735046, + -0.16649185121059418, + 0.051625266671180725, + -0.6595144867897034, + 0.19266369938850403, + 0.3640836477279663 + ] + ], + "generated_utilities": [ + 0.03511819243431091, + 2.0, + 0.03361482918262482, + 0.024507999420166016, + 0.02013157308101654, + 0.03976954519748688, + 0.03125926852226257, + 0.017352595925331116 + ], + "group_id": "ms-27c9f70e97b8850cc3ab", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.09440004825592041, + 0.14095592498779297, + 2.0, + 0.09219714999198914, + 0.05765572190284729, + 0.02530825138092041, + 0.07676126062870026, + 0.06643831729888916, + 0.047928065061569214, + 0.07107676565647125, + 0.05561619997024536, + 0.02124820649623871, + 0.08948668837547302, + 0.10203869640827179, + 0.06119762361049652, + 0.04547174274921417 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.984761953353882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.7230467796325684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.554086923599243 + ], + [ + 1.0343247652053833, + 1.287368655204773, + 1.867376446723938, + -2.4465248584747314, + 1.036996841430664, + -1.2664310932159424, + 1.4399157762527466, + -0.8265262246131897, + -0.28319495916366577, + 0.16331201791763306, + -1.5917816162109375, + -1.5816115140914917, + -0.1664504110813141, + 0.7195659279823303, + 0.29144588112831116, + -0.18517816066741943, + -0.08826595544815063, + -0.8266173601150513, + -0.7933371067047119, + -0.938305139541626, + 0.5359267592430115 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3968086242675781, + 0.5359851121902466, + 0.7176156044006348, + -1.52370023727417, + -0.1481943428516388, + 0.10887271165847778, + 0.0, + -0.12331700325012207, + 0.14385993778705597, + -0.041522324085235596, + 0.16089874505996704, + -0.6189663410186768, + -0.01699581742286682, + 0.0, + -0.07216246426105499, + -0.15073662996292114, + -0.134138286113739, + -0.06049776077270508, + -0.26776671409606934, + -0.019470810890197754, + 0.0 + ] + ], + "predicted_scores": [ + 5.22308874130249, + 4.920225143432617, + 4.658702850341797, + 4.609931945800781, + 4.31458854675293, + 4.129289150238037, + 3.913191795349121, + 3.680939197540283 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1881223807", + "selected_index": 0, + "state_hash": "0f58a11c36c331c9abab00795189dccbc7e0aebf985ca9df67bf9da756f154e3", + "stored_base_utility": 0.09440004825592041, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.10841864347457886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10841864347457886 + }, + "base_progress": 0.10841864347457886, + "base_success": false, + "base_utility": 0.10841864347457886, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0735837072134018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0735837072134018 + }, + { + "contact_quality": null, + "progress": 0.03691600263118744, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03691600263118744 + }, + { + "contact_quality": null, + "progress": 0.04955050349235535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04955050349235535 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0934627503156662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0934627503156662 + }, + { + "contact_quality": null, + "progress": 0.03192968666553497, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03192968666553497 + }, + { + "contact_quality": null, + "progress": 0.03504982590675354, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03504982590675354 + } + ], + "candidate_progress": [ + 0.0735837072134018, + 0.03691600263118744, + 0.04955050349235535, + 1.0, + 1.0, + 0.0934627503156662, + 0.03192968666553497, + 0.03504982590675354 + ], + "candidate_source_chart_ids": [ + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + true, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-2867deaca7e98582620b", + "generated_tangents": [ + [ + -0.20429296791553497, + -0.14876165986061096, + 1.3290013074874878, + -1.352597713470459, + -1.7126870155334473, + 0.14912866055965424, + 0.07702407240867615, + -0.3980390727519989, + 0.06364188343286514, + -0.1360224485397339, + 0.11478905379772186, + -1.6165939569473267, + 0.2900989353656769, + 0.009237505495548248, + -0.11541570723056793, + 0.05542559549212456, + 0.420320600271225, + 0.6107426285743713, + -0.5372239351272583, + -0.5682095289230347, + -0.25948551297187805 + ], + [ + -0.10409478843212128, + -0.12906688451766968, + 1.1847866773605347, + -1.091176986694336, + -1.3266043663024902, + 0.37149542570114136, + 0.48397350311279297, + -0.33552107214927673, + 0.19832918047904968, + 0.18402515351772308, + 0.03066684491932392, + -1.435766577720642, + -0.03562536835670471, + 0.27757394313812256, + -0.29364967346191406, + -0.005042250733822584, + 0.4285711348056793, + 0.28734320402145386, + -0.42920011281967163, + -0.5686603784561157, + -0.49396613240242004 + ], + [ + -0.13803307712078094, + -0.1262870877981186, + 1.149946928024292, + -1.1331455707550049, + -1.315546989440918, + 0.3669312596321106, + 0.599021315574646, + -0.28965067863464355, + 0.1906404346227646, + 0.17868971824645996, + 0.013034090399742126, + -1.4281377792358398, + -0.04523172974586487, + 0.11331256479024887, + -0.2622690796852112, + 0.03134184330701828, + 0.3662893772125244, + 0.2786247134208679, + -0.48805147409439087, + -0.6206440925598145, + -0.62861168384552 + ], + [ + -0.06871075928211212, + -0.11510556936264038, + 1.2270950078964233, + -1.3125711679458618, + -1.0540797710418701, + 0.5840364694595337, + 0.32354772090911865, + -0.39099714159965515, + 0.053422100841999054, + 0.1083730161190033, + 0.026813482865691185, + -1.2090234756469727, + 0.19731321930885315, + 0.13857580721378326, + -0.3366085886955261, + 0.05450739338994026, + 0.7534447908401489, + 0.5692466497421265, + -0.5715690851211548, + -0.510240912437439, + -0.04246090352535248 + ], + [ + -0.11054052412509918, + -0.052226945757865906, + 1.1467020511627197, + -1.2065396308898926, + -1.3488514423370361, + 0.520066499710083, + 0.20574578642845154, + -0.23715922236442566, + 0.10530725866556168, + 0.25740188360214233, + 0.005739815533161163, + -1.4566844701766968, + -0.03515699505805969, + 0.46808958053588867, + -0.2493971586227417, + -0.010118776001036167, + 0.5703317523002625, + 0.5680418610572815, + -0.627251386642456, + -0.46288031339645386, + -0.06490163505077362 + ], + [ + -0.04272349178791046, + -0.10831273347139359, + 1.4016040563583374, + -1.3271615505218506, + -1.4304304122924805, + 0.4315873682498932, + 0.05909159779548645, + -0.1402525007724762, + 0.08273803442716599, + -0.12461645156145096, + 0.17579922080039978, + -1.3578174114227295, + 0.18397226929664612, + -0.09607651829719543, + -0.05287511274218559, + 0.07294613122940063, + 0.438185453414917, + 0.6284604072570801, + -0.4433620572090149, + -0.5687687397003174, + -0.2140549123287201 + ], + [ + -0.18141140043735504, + -0.1062038391828537, + 1.3859483003616333, + -1.3377981185913086, + -1.3497600555419922, + 0.41398143768310547, + -0.22816812992095947, + -0.15845686197280884, + 0.10322701930999756, + 0.03054284304380417, + 0.2757967412471771, + -1.4107712507247925, + 0.39524540305137634, + 0.19922864437103271, + -0.0873475968837738, + 0.06450699269771576, + 0.5114575624465942, + 0.6249605417251587, + -0.5176184773445129, + -0.6053492426872253, + -0.32059231400489807 + ], + [ + -0.1258784979581833, + -0.1408306062221527, + 1.1380115747451782, + -1.2596936225891113, + -1.4162650108337402, + 0.22117295861244202, + -0.3289192318916321, + -0.31275537610054016, + 0.07880052924156189, + 0.31489723920822144, + -0.15400105714797974, + -1.4290878772735596, + 0.11055895686149597, + 0.4349272847175598, + -0.3734752833843231, + 0.10176998376846313, + 0.545526385307312, + 0.583720862865448, + -0.6139087080955505, + -0.31675440073013306, + -0.017446845769882202 + ] + ], + "generated_utilities": [ + 0.0735837072134018, + 0.03691600263118744, + 0.04955050349235535, + 2.0, + 2.0, + 0.0934627503156662, + 0.03192968666553497, + 0.03504982590675354 + ], + "group_id": "ms-2867deaca7e98582620b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0924256443977356, + 0.05646410584449768, + 2.0, + 2.0, + 0.183049738407135, + 2.0, + 0.16824382543563843, + 0.1424575299024582, + 0.08230158686637878, + 0.15871968865394592, + 0.16382521390914917, + 0.05336849391460419, + 0.07674244046211243, + 0.10727545619010925, + 0.06594061851501465, + 0.045542001724243164 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.44511327147483826, + 0.05827039107680321, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.2619132101535797, + 0.0, + -0.37528133392333984, + -0.07968637347221375, + -0.07514983415603638, + 0.19317352771759033, + 0.03868463635444641, + -0.06793206930160522, + 0.0, + 0.13187909126281738, + -0.20905672013759613, + 0.030073702335357666, + 0.21597933769226074, + -0.0761370062828064, + -0.11662125587463379, + 0.0 + ], + [ + -0.15486939251422882, + 0.24427275359630585, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.44790399074554443, + 0.0, + -0.37528133392333984, + -0.29508399963378906, + 0.7311322093009949, + 0.19317352771759033, + 0.09925723075866699, + 0.1767415851354599, + 0.0, + -0.5768744349479675, + 0.1300974041223526, + 0.21151533722877502, + 0.21597933769226074, + 0.04431194067001343, + -0.040631115436553955, + 0.0 + ], + [ + -0.29166290163993835, + 0.09681949019432068, + 2.2608249187469482, + -1.3577959537506104, + -2.195330858230591, + -0.13884210586547852, + 2.5595173835754395, + -1.3752813339233398, + 0.390796959400177, + 0.673005223274231, + 1.1931735277175903, + -0.41860872507095337, + -0.24474714696407318, + 0.3015621602535248, + -0.2400326132774353, + 0.14207373559474945, + -0.26512715220451355, + 1.2159793376922607, + -0.3237651586532593, + -0.7466862201690674, + 0.30001604557037354 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.119034767150879, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6031243205070496, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6000320911407471 + ], + [ + -0.3580354154109955, + 0.03320832550525665, + 2.5546932220458984, + -1.138798713684082, + -2.260084629058838, + -0.04964215308427811, + 2.8354909420013428, + -1.6732244491577148, + 0.21277914941310883, + 0.5097208023071289, + 1.387328028678894, + -0.691602349281311, + -0.28483790159225464, + 0.35576167702674866, + -0.0605899840593338, + 0.3764685094356537, + -0.28031954169273376, + 1.063202977180481, + -0.03841540217399597, + -0.5410391092300415, + 0.5580100417137146 + ], + [ + -0.5578174591064453, + 0.3727092146873474, + 2.1929497718811035, + -1.577579140663147, + -1.8962993621826172, + -0.2802738845348358, + 2.1670117378234863, + -1.0752147436141968, + 0.16372516751289368, + 0.6087679862976074, + 0.8854888081550598, + -0.18080148100852966, + -0.18821784853935242, + 0.010591715574264526, + -0.12560874223709106, + 0.15850795805454254, + -0.1898641288280487, + 1.1823316812515259, + 0.06655246019363403, + -0.4014361798763275, + 0.4376043379306793 + ], + [ + -0.43790310621261597, + -0.12778332829475403, + 2.6857643127441406, + -1.6293541193008423, + -1.9850949048995972, + -0.7167636752128601, + 2.465301513671875, + -1.4102312326431274, + 0.2071499079465866, + 0.5742072463035583, + 0.9525313377380371, + -0.34116798639297485, + -0.04760183393955231, + -0.05569759011268616, + -0.4777999520301819, + 0.17186680436134338, + 0.0841393768787384, + 0.7708102464675903, + -0.2784478962421417, + -1.2458651065826416, + 0.46140795946121216 + ], + [ + 0.06170770525932312, + 0.45555436611175537, + 1.6506444215774536, + -0.9394457340240479, + -1.9117670059204102, + -0.6910014152526855, + 2.40145206451416, + -1.7531975507736206, + 0.10217079520225525, + 1.2514894008636475, + 1.3236911296844482, + -0.6504120826721191, + -0.6943472027778625, + 0.7029284238815308, + -0.7387814521789551, + -0.33777421712875366, + -0.8187699317932129, + 1.8448829650878906, + 0.00849190354347229, + -1.394152045249939, + 0.918946385383606 + ] + ], + "predicted_scores": [ + 5.722990989685059, + 5.510813236236572, + 5.459251403808594, + 5.288208961486816, + 5.2233147621154785, + 5.001323223114014, + 4.794158935546875, + 3.848588466644287 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1189392917", + "selected_index": 0, + "state_hash": "e8cf9b95f09f32d8e7942b114792725bd89c392a97ff7e6468ebf09df1bb0d21", + "stored_base_utility": 0.0924256443977356, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.13927479088306427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13927479088306427 + }, + "base_progress": 0.13927479088306427, + "base_success": false, + "base_utility": 0.13927479088306427, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06193861365318298, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06193861365318298 + }, + { + "contact_quality": null, + "progress": 0.060320109128952026, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060320109128952026 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06495748460292816, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06495748460292816 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.06193861365318298, + 0.060320109128952026, + 1.0, + 0.06495748460292816, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + false, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-365d8c6de6de16f66744", + "generated_tangents": [ + [ + -0.0793064683675766, + -0.04591337963938713, + 1.4583491086959839, + -0.593203067779541, + -1.700812816619873, + -0.0134030282497406, + 0.04298475384712219, + -0.10123550891876221, + 0.006442335434257984, + 0.4131436049938202, + -0.08257017284631729, + -1.2423640489578247, + 0.16897401213645935, + 0.18572698533535004, + -0.08300364017486572, + -0.030812395736575127, + 0.3930715322494507, + 0.3161470293998718, + -0.4375561475753784, + -0.6313263177871704, + -0.21668821573257446 + ], + [ + 0.0020146816968917847, + -0.023522598668932915, + 1.520439863204956, + -0.4879358112812042, + -1.5140752792358398, + -0.13699603080749512, + -0.0178375244140625, + -0.12501052021980286, + 0.01563800685107708, + 0.3766712546348572, + 0.0915779173374176, + -1.244942307472229, + 0.11321845650672913, + 0.25958627462387085, + -0.06212689355015755, + -0.009170486591756344, + 0.30351322889328003, + 0.3310752511024475, + -0.4037807583808899, + -0.2220911830663681, + -0.1283075511455536 + ], + [ + -0.20166529715061188, + -0.11073032766580582, + 1.2266387939453125, + -0.7656883001327515, + -1.1065425872802734, + 0.10294624418020248, + -0.036587536334991455, + -0.3598079979419708, + 0.07628034055233002, + 0.19569899141788483, + 0.10280755907297134, + -1.2576781511306763, + 0.2309914529323578, + 0.2545529007911682, + -0.08116030693054199, + -0.1079796850681305, + 0.23373401165008545, + 0.3920135200023651, + -0.257668137550354, + -0.26219671964645386, + 0.03141540288925171 + ], + [ + 0.1136198490858078, + -0.04602309688925743, + 1.4528672695159912, + -0.4521344006061554, + -1.4777216911315918, + -0.03439553081989288, + 0.07159629464149475, + -0.03103436529636383, + 0.07056211680173874, + 0.2610095739364624, + -0.04178794473409653, + -1.2863779067993164, + -0.16831350326538086, + 0.10637831687927246, + -0.06459861993789673, + -0.016607005149126053, + 0.32327282428741455, + 0.37286895513534546, + -0.4181337356567383, + -0.3303793668746948, + -0.00886024534702301 + ], + [ + 0.07044044137001038, + 0.013210754841566086, + 1.4144206047058105, + -0.5527143478393555, + -1.404444932937622, + 0.08353669941425323, + -0.02880951762199402, + -0.011849001049995422, + 0.02369382232427597, + 0.24913308024406433, + -0.10446097701787949, + -1.3512086868286133, + 0.0598885715007782, + 0.15145491063594818, + -0.10982738435268402, + -0.08647167682647705, + 0.40291428565979004, + 0.3543710708618164, + -0.4759449362754822, + -0.4454573392868042, + 0.06395728886127472 + ], + [ + -0.06900635361671448, + 0.12228089570999146, + 1.208706021308899, + -0.48433423042297363, + -0.9781535267829895, + 0.22853729128837585, + -0.11004656553268433, + -0.05800589919090271, + 0.0071481578052043915, + 0.12424708902835846, + -0.11923506110906601, + -1.197339653968811, + -0.20530453324317932, + 0.15419232845306396, + -0.29404327273368835, + -0.114983931183815, + 0.430389940738678, + 0.3056219220161438, + -0.43837523460388184, + -0.37814196944236755, + -0.016082659363746643 + ], + [ + -0.029048293828964233, + -0.01479404792189598, + 1.4270713329315186, + -0.5820334553718567, + -1.4748575687408447, + 0.20198768377304077, + 0.02460220456123352, + -0.16334596276283264, + 0.041051752865314484, + 0.2036067247390747, + -0.1663990616798401, + -1.2124314308166504, + -0.010271072387695312, + 0.19432416558265686, + -0.0633213222026825, + -0.04832817614078522, + 0.40308159589767456, + 0.462817907333374, + -0.4925278425216675, + -0.2808868885040283, + 0.11546452343463898 + ], + [ + 0.04306834936141968, + -0.014999547973275185, + 1.445180058479309, + -0.603166937828064, + -1.408811092376709, + 0.10178462415933609, + 0.0807824432849884, + -0.12137889862060547, + 0.06345739215612411, + 0.24071167409420013, + -0.17246851325035095, + -1.1719390153884888, + -0.12095233798027039, + 0.2868489623069763, + -0.1355857104063034, + -0.0820237398147583, + 0.31586211919784546, + 0.4032604992389679, + -0.5581992268562317, + -0.2176150232553482, + 0.003830462694168091 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.06193861365318298, + 0.060320109128952026, + 2.0, + 0.06495748460292816, + 2.0, + 2.0 + ], + "group_id": "ms-365d8c6de6de16f66744", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20231953263282776, + 0.12319359183311462, + 2.0, + 2.0, + 0.1590207815170288, + 0.019343823194503784, + 0.1322992593050003, + 0.09479346871376038, + 0.1768283247947693, + 0.1054161787033081, + 0.16278749704360962, + 0.08510464429855347, + 0.09564858675003052, + 0.07681193947792053, + 0.1334480196237564, + 0.08148902654647827 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.09058855473995209, + -0.022760659456253052, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.07837581634521484, + 0.0, + -0.05228453874588013, + -0.0679071694612503, + 0.026890814304351807, + 0.03908693790435791, + -1.4526653289794922, + 0.04786187410354614, + 0.0, + 0.019858457148075104, + -0.05534793436527252, + 0.17386651039123535, + -0.024165034294128418, + -0.05880159139633179, + 0.044484615325927734, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.063174724578857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6561365127563477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0078073740005493 + ], + [ + 0.05168074369430542, + 0.27186232805252075, + 2.0911388397216797, + -1.6812132596969604, + -2.3280625343322754, + 0.38023167848587036, + 2.5090978145599365, + -0.2604735791683197, + 0.04725922644138336, + 0.8121852874755859, + -0.6659450531005859, + -2.2462568283081055, + 0.49988994002342224, + 1.9786770343780518, + 0.02324119210243225, + -0.45050501823425293, + 1.1065142154693604, + 0.5724118947982788, + -0.6320602893829346, + -0.7826599478721619, + 0.47463753819465637 + ], + [ + 0.46382713317871094, + -0.06331050395965576, + 2.657292366027832, + -1.1560972929000854, + -2.765941619873047, + 0.7805119752883911, + 2.3117406368255615, + -0.3644295036792755, + -0.25628525018692017, + 0.7623568773269653, + -0.4176616668701172, + -2.646364688873291, + 0.40904325246810913, + 1.7654002904891968, + -0.23315654695034027, + 0.026929572224617004, + 0.8236149549484253, + 0.8322737812995911, + -0.7612741589546204, + -0.4725078046321869, + 0.5895571708679199 + ], + [ + -0.26226741075515747, + 0.3922410011291504, + 2.450082302093506, + -0.9020822048187256, + -3.028742551803589, + 1.0746917724609375, + 3.104015350341797, + 0.3752234876155853, + -0.009036362171173096, + 0.3652568459510803, + -0.7608495950698853, + -2.337242603302002, + -0.03944078087806702, + 1.972575306892395, + -0.32381975650787354, + -0.45855021476745605, + 1.6927499771118164, + 0.8358478546142578, + -0.8610806465148926, + -0.6741821765899658, + 0.4013170003890991 + ], + [ + 0.5790615677833557, + 0.8308188915252686, + 3.141986846923828, + -1.218785285949707, + -2.0300495624542236, + 0.4300342798233032, + 3.0737316608428955, + -0.09167198091745377, + 0.4237035810947418, + 0.7421274185180664, + -0.637946367263794, + -2.1245827674865723, + 0.4184945523738861, + 1.2377147674560547, + -0.4068492650985718, + -0.03615438938140869, + 1.5385099649429321, + 0.51850825548172, + -0.5737811923027039, + -1.472043752670288, + 0.45800793170928955 + ], + [ + 0.38714414834976196, + 0.25513744354248047, + 2.9400248527526855, + -1.8116681575775146, + -1.7500431537628174, + 1.403240442276001, + 1.9709672927856445, + 0.6755956411361694, + 0.49445489048957825, + 0.19490671157836914, + -1.1990764141082764, + -2.803361177444458, + 0.499578058719635, + 1.164682149887085, + 0.9588776230812073, + 0.5154257416725159, + 1.0328537225723267, + 1.1309876441955566, + -0.9365650415420532, + -0.9663279056549072, + -0.032733798027038574 + ], + [ + 0.7417005896568298, + -0.12406697869300842, + 3.0325844287872314, + -0.5656485557556152, + -2.5084800720214844, + 0.27374324202537537, + 2.2886011600494385, + 0.5997821092605591, + 0.9623891115188599, + 0.38451504707336426, + -1.4501185417175293, + -1.6908833980560303, + 0.6743149757385254, + 1.8593690395355225, + -0.018024832010269165, + -0.20444278419017792, + 1.364742636680603, + 0.34230074286460876, + -1.4427728652954102, + -1.5577378273010254, + 0.5589802265167236 + ], + [ + 0.3448556661605835, + -0.5947309732437134, + 2.3888885974884033, + -1.0376405715942383, + -2.218167781829834, + 1.568857192993164, + 3.0995821952819824, + -0.4682580530643463, + 0.3072889745235443, + 0.14532917737960815, + -0.567415177822113, + -3.085655689239502, + -0.6656016111373901, + 1.6447856426239014, + 0.8135664463043213, + 0.3758860230445862, + 1.399398922920227, + 0.11528515815734863, + -0.531513512134552, + -0.6104787588119507, + 1.4474445581436157 + ], + [ + -0.18966330587863922, + -0.7208725214004517, + 2.586766481399536, + -0.6973344087600708, + -3.2187445163726807, + 1.4117884635925293, + 3.0797414779663086, + -0.29149553179740906, + 0.09620378166437149, + 0.874613881111145, + -0.8004729151725769, + -2.0231733322143555, + -0.5948910713195801, + 1.5080900192260742, + -0.750367283821106, + 0.8155628442764282, + 1.6562600135803223, + 0.8091130256652832, + -1.1569771766662598, + -1.4121384620666504, + 0.3738836348056793 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06428352743387222, + -0.015524566173553467, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + -0.1607142686843872, + 0.0, + 0.27928173542022705, + 0.05664636939764023, + -0.03914147615432739, + 0.1700872778892517, + -1.4526653289794922, + 0.05555404722690582, + 0.0, + 0.18240894377231598, + -0.05239863693714142, + 0.17386651039123535, + -0.21921080350875854, + -0.3728579580783844, + 0.11250120401382446, + 0.0 + ], + [ + 0.008915487676858902, + 0.540733277797699, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.22902190685272217, + 0.0, + -0.3544532060623169, + -0.2348114550113678, + 0.04456782341003418, + 0.2907915711402893, + -1.4526653289794922, + -0.1788860559463501, + 0.0, + 0.07762694358825684, + -0.07040916383266449, + 0.17386651039123535, + 0.44059625267982483, + -0.19159233570098877, + 0.09149748086929321, + 0.0 + ] + ], + "predicted_scores": [ + 8.251832962036133, + 8.237282752990723, + 7.584033012390137, + 7.5751519203186035, + 7.472279071807861, + 7.430319786071777, + 7.395534515380859, + 7.284758567810059 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1997090716", + "selected_index": 0, + "state_hash": "98870eab9f394b355af9b8a6d689b3bcd574af94db5d1a9139d0aed0b633b959", + "stored_base_utility": 0.20231953263282776, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03280790150165558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03280790150165558 + }, + "base_progress": 0.03280790150165558, + "base_success": false, + "base_utility": 0.03280790150165558, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.02276340126991272, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02276340126991272 + }, + { + "contact_quality": null, + "progress": 0.022152185440063477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022152185440063477 + }, + { + "contact_quality": null, + "progress": 0.0349019318819046, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0349019318819046 + }, + { + "contact_quality": null, + "progress": 0.022732555866241455, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022732555866241455 + }, + { + "contact_quality": null, + "progress": 0.021209537982940674, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021209537982940674 + }, + { + "contact_quality": null, + "progress": 0.023332014679908752, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023332014679908752 + }, + { + "contact_quality": null, + "progress": 0.020442917943000793, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020442917943000793 + }, + { + "contact_quality": null, + "progress": 0.022646263241767883, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022646263241767883 + } + ], + "candidate_progress": [ + 0.02276340126991272, + 0.022152185440063477, + 0.0349019318819046, + 0.022732555866241455, + 0.021209537982940674, + 0.023332014679908752, + 0.020442917943000793, + 0.022646263241767883 + ], + "candidate_source_chart_ids": [ + "ms-3f6cd47af5e9e6781b73", + "ms-fe38b164397dd669f391", + "ms-2423891634e7b469eed8", + "ms-2cbe83da02847008bf24", + "ms-2cbe83da02847008bf24", + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-3c326d05fec66e1d6bea", + "generated_tangents": [ + [ + 0.40442317724227905, + -0.7725453972816467, + 1.1501598358154297, + 0.17600640654563904, + -0.30155789852142334, + -0.39580318331718445, + -0.01875162124633789, + 0.26155561208724976, + 0.4048941731452942, + -0.02681455761194229, + -0.3096141219139099, + -0.6872912049293518, + -0.36405548453330994, + 0.06340265274047852, + 0.1773771047592163, + -0.10477298498153687, + 0.015425518155097961, + 0.20689745247364044, + 0.04955346882343292, + -0.3902435600757599, + 0.24647946655750275 + ], + [ + 0.07905979454517365, + -0.909687876701355, + 0.9376540184020996, + 0.43800172209739685, + -0.023324698209762573, + -0.23537349700927734, + -0.45633143186569214, + 0.2921690344810486, + 0.6057440638542175, + -0.07589032500982285, + -0.563623309135437, + -0.505800187587738, + -0.07873103022575378, + 0.17948520183563232, + -0.20254218578338623, + -0.10596145689487457, + 0.1744166612625122, + 0.10523907840251923, + 0.24562320113182068, + -0.25142624974250793, + 0.3499114215373993 + ], + [ + 0.12745730578899384, + -0.8414797782897949, + 1.0682847499847412, + 0.2258983850479126, + -0.2101956009864807, + -0.13761919736862183, + -0.23796921968460083, + 0.1290353685617447, + 0.5689228773117065, + -0.03213111311197281, + -0.42360377311706543, + -0.6594093441963196, + 0.03247526288032532, + 0.12020916491746902, + -0.26870110630989075, + -0.13109737634658813, + 0.28618288040161133, + 0.21626964211463928, + 0.12039704620838165, + -0.22269728779792786, + 0.14697611331939697 + ], + [ + 0.29885730147361755, + -0.911459743976593, + 1.0089999437332153, + 0.2503260672092438, + -0.20167016983032227, + -0.3472200036048889, + -0.6033332943916321, + 0.09250596910715103, + 0.5052357912063599, + 0.009694404900074005, + -0.46168193221092224, + -0.6105486154556274, + -0.10397589206695557, + 0.11901242285966873, + -0.3067343533039093, + -0.0861128717660904, + 0.0857761800289154, + 0.08606605976819992, + 0.10337814688682556, + -0.34056398272514343, + 0.31238046288490295 + ], + [ + 0.3344447910785675, + -0.9550344944000244, + 1.1186020374298096, + 0.22837793827056885, + -0.14751920104026794, + -0.3237929940223694, + -0.5763834118843079, + 0.20042864978313446, + 0.5201164484024048, + 0.004939809441566467, + -0.4772302806377411, + -0.6474586725234985, + -0.18470191955566406, + 0.19634193181991577, + -0.2683328688144684, + -0.03984227776527405, + 0.1518973708152771, + 0.03422252833843231, + 0.15046000480651855, + -0.20712175965309143, + 0.3553316593170166 + ], + [ + 0.20436400175094604, + -0.9022265076637268, + 1.1143196821212769, + 0.43258896470069885, + 0.17536607384681702, + -0.18271684646606445, + -0.692637026309967, + 0.212854266166687, + 0.5240234732627869, + 0.03801193833351135, + -0.5399991869926453, + -0.6230814456939697, + -0.15395104885101318, + 0.21651341021060944, + -0.31853556632995605, + -0.025904826819896698, + 0.07033790647983551, + 0.1579502373933792, + 0.06720687448978424, + -0.1857805848121643, + 0.3302036225795746 + ], + [ + 0.16572201251983643, + -0.9039570093154907, + 0.9914646148681641, + 0.3511832654476166, + 0.0878506749868393, + -0.038403719663619995, + -0.7526358962059021, + 0.19371949136257172, + 0.523228645324707, + -0.07204826921224594, + -0.5434116125106812, + -0.7394533157348633, + -0.10109508037567139, + -0.06650365889072418, + -0.10940064489841461, + 0.08765175938606262, + 0.30040937662124634, + 0.18968750536441803, + -0.09119495749473572, + -0.35894227027893066, + 0.4195023477077484 + ], + [ + 0.25426024198532104, + -0.9746049046516418, + 0.9145423173904419, + 0.2769142985343933, + -0.01973554491996765, + -0.06788142025470734, + -0.673834502696991, + 0.0617547333240509, + 0.5495027303695679, + -0.057230062782764435, + -0.5406963229179382, + -0.6831774115562439, + -0.18949377536773682, + 0.19629457592964172, + -0.2762845456600189, + -0.09926141798496246, + 0.1114891842007637, + 0.07077242434024811, + -0.08504712581634521, + -0.26889699697494507, + 0.45010533928871155 + ] + ], + "generated_utilities": [ + 0.02276340126991272, + 0.022152185440063477, + 0.0349019318819046, + 0.022732555866241455, + 0.021209537982940674, + 0.023332014679908752, + 0.020442917943000793, + 0.022646263241767883 + ], + "group_id": "ms-3c326d05fec66e1d6bea", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18246883153915405, + 2.0, + 2.0, + 0.11388173699378967, + 0.035728439688682556, + 0.05159728229045868, + 0.043176501989364624, + 0.04580867290496826, + 0.02321702241897583, + 0.040450215339660645, + 0.04055213928222656, + 0.024657413363456726, + 0.017660364508628845, + 0.044850513339042664, + 0.031234994530677795, + 0.06195336580276489 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.018136948347091675, + 0.10074883699417114, + 0.11574286222457886, + 0.0, + 0.1410461664199829, + 0.38714247941970825, + 0.2381301373243332, + 0.06927800178527832, + -0.6138286590576172, + -0.33484986424446106, + 0.0, + 0.012610852718353271, + 0.5407634973526001, + 0.07784794270992279, + 0.3537199795246124, + -0.1602279543876648, + 0.002093076705932617, + 0.0 + ], + [ + 1.9327397346496582, + -1.7429183721542358, + 1.8877019882202148, + -0.33001917600631714, + 0.49746543169021606, + 0.47390231490135193, + 1.7802670001983643, + -0.8589538335800171, + 0.37976425886154175, + -0.18104670941829681, + -0.8859321475028992, + -1.6138286590576172, + 0.34242650866508484, + 1.2023069858551025, + 0.555707573890686, + -0.35764527320861816, + -0.10027600824832916, + 0.4702257812023163, + -0.6294202208518982, + 0.1926015019416809, + 0.5916687846183777 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.5605340003967285, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.404613971710205, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1833375692367554 + ], + [ + 1.7389304637908936, + -1.4791877269744873, + 2.158487319946289, + -0.3266654312610626, + 0.5907376408576965, + 0.43260636925697327, + 1.9136874675750732, + -0.5760036110877991, + 0.3071907162666321, + -0.4073694348335266, + -0.8863769769668579, + -1.8163083791732788, + 0.5969879627227783, + 1.196010947227478, + 0.7674136161804199, + -0.2368052750825882, + -0.12123071402311325, + 0.36782756447792053, + -0.8187789916992188, + 0.15793970227241516, + 0.43327897787094116 + ], + [ + 1.9560860395431519, + -2.008063793182373, + 1.6910814046859741, + -0.2576735019683838, + 0.647503137588501, + 0.5322431921958923, + 1.9268598556518555, + -1.2501428127288818, + 0.775363564491272, + -0.0965004414319992, + -0.5331029891967773, + -1.639766812324524, + 0.44450435042381287, + 1.5980249643325806, + 0.9402694702148438, + -0.1454993337392807, + -0.2541998028755188, + 0.6709543466567993, + -0.44338259100914, + -0.043697476387023926, + 0.7131086587905884 + ], + [ + 1.4609386920928955, + -1.6896755695343018, + 2.174467086791992, + -0.34207087755203247, + 0.962078332901001, + 0.28904104232788086, + 2.0044631958007812, + -1.184704303741455, + 0.8424319624900818, + -0.6237084865570068, + -0.9171007871627808, + -1.5616872310638428, + 0.3556978702545166, + 1.5233795642852783, + 0.7210954427719116, + 0.028062552213668823, + 0.21759141981601715, + 0.2823620140552521, + -0.588395357131958, + -0.12581703066825867, + 0.7275858521461487 + ], + [ + 2.087728500366211, + -2.0203187465667725, + 2.4627456665039062, + 0.09322953224182129, + 0.4771752953529358, + 0.2439241260290146, + 2.3431313037872314, + -0.7176190614700317, + 0.8298519849777222, + -0.47738587856292725, + -1.4635096788406372, + -1.2598087787628174, + 0.008061617612838745, + 0.9835944175720215, + 0.10656380653381348, + -0.1548040360212326, + 0.20115382969379425, + 0.4581896960735321, + -1.197885274887085, + -0.28882738947868347, + 0.17229679226875305 + ], + [ + 2.535038471221924, + -2.2647459506988525, + 1.280318260192871, + -0.8243153095245361, + 0.36149507761001587, + 0.01678559184074402, + 1.259200096130371, + -0.5383518934249878, + 1.0661393404006958, + -0.1134992390871048, + -1.0009663105010986, + -1.8629896640777588, + 0.5329263806343079, + 1.1952301263809204, + 0.5380017757415771, + -0.21264730393886566, + 0.33232176303863525, + 0.9111320972442627, + -1.0945085287094116, + 0.7546109557151794, + 0.8692140579223633 + ], + [ + 2.3966352939605713, + -2.3338406085968018, + 2.0661728382110596, + -0.1174439787864685, + -0.28071242570877075, + 0.8010929822921753, + 2.485050678253174, + -0.5360689163208008, + 0.9712552428245544, + 0.5487139821052551, + -0.46108368039131165, + -2.0631186962127686, + 0.6046813726425171, + 0.9991111159324646, + 1.299267292022705, + 0.3742367625236511, + -0.7156072854995728, + 0.7805436849594116, + -1.3091955184936523, + -0.5538330674171448, + -0.20684683322906494 + ], + [ + 2.614837884902954, + -2.24919056892395, + 2.2911505699157715, + -0.2939787209033966, + 1.1876089572906494, + 0.15644106268882751, + 1.7348854541778564, + -0.7635170221328735, + 0.5141214728355408, + -0.17918013036251068, + -0.8421157598495483, + -1.5084272623062134, + -0.18822845816612244, + 1.8389434814453125, + 0.24652135372161865, + 0.38509100675582886, + 0.5761538147926331, + -0.15904095768928528, + 0.0013433098793029785, + 0.6120359897613525, + 1.3977106809616089 + ], + [ + 1.5386989116668701, + -2.0177698135375977, + 1.8581953048706055, + -1.2861346006393433, + 1.0945217609405518, + 0.8363389372825623, + 2.708517074584961, + -0.9201098084449768, + 0.8546208739280701, + -0.058566831052303314, + -1.5932974815368652, + -2.015472412109375, + 0.2565937638282776, + 1.988462209701538, + -0.04372602701187134, + -0.8754178881645203, + 0.27551382780075073, + -0.29693683981895447, + -0.2824576497077942, + 0.40926772356033325, + 0.7658611536026001 + ], + [ + 2.8735361099243164, + -2.675261974334717, + 1.459324836730957, + 0.380817174911499, + 0.6231281757354736, + 0.03267902135848999, + 2.296022415161133, + -1.587423324584961, + 0.7833051681518555, + 0.6618785262107849, + -1.7110093832015991, + -1.436572551727295, + -0.14784568548202515, + 1.525026559829712, + 1.4909214973449707, + -1.0601696968078613, + -0.5454426407814026, + -0.09859517216682434, + -1.4344520568847656, + -0.058256953954696655, + -0.3444262146949768 + ], + [ + 2.0854334831237793, + -1.3952012062072754, + 2.651411771774292, + -0.7663257122039795, + -0.05179548263549805, + 0.8797686100006104, + 2.0904030799865723, + -0.8943268656730652, + 1.0378066301345825, + 0.08841629326343536, + -0.36119532585144043, + -1.236596703529358, + 1.0752983093261719, + 0.349973201751709, + -0.19527214765548706, + -0.8867040872573853, + 0.18460334837436676, + 1.454195261001587, + -0.6220338940620422, + 0.9919598698616028, + 1.229588508605957 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.009577363729476929, + -0.06759434938430786, + 0.05406871438026428, + 0.0, + 0.003663182258605957, + 0.0492824912071228, + -0.038241639733314514, + -0.02580159902572632, + -0.6138286590576172, + -0.056222498416900635, + 0.0, + -0.026217222213745117, + -0.051223963499069214, + -0.031640589237213135, + -0.010870158672332764, + 0.08728909492492676, + -0.020528942346572876, + 0.0 + ], + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.06735062599182129, + -0.023807168006896973, + 0.1439460813999176, + 0.0, + -0.19209599494934082, + -0.14847975969314575, + -0.22449488937854767, + 0.09710657596588135, + -0.6138286590576172, + -0.1854294240474701, + 0.0, + -0.17072844505310059, + 0.09961000084877014, + -0.017846792936325073, + 0.22415585815906525, + -0.18760734796524048, + -0.060083091259002686, + 0.0 + ] + ], + "predicted_scores": [ + 5.217313289642334, + 4.2235822677612305, + 4.106942653656006, + 3.900104522705078, + 3.644498348236084, + 3.5433480739593506, + 3.4885756969451904, + 3.16633939743042 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524", + "selected_index": 0, + "state_hash": "6333fcf80f76890398820f27d1a99efe459cdb6ff795201d73e3bcedc966664f", + "stored_base_utility": 0.18246883153915405, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18095605075359344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18095605075359344 + }, + "base_progress": 0.18095605075359344, + "base_success": false, + "base_utility": 0.18095605075359344, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03734047710895538, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03734047710895538 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.077694833278656, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.077694833278656 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.03734047710895538, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.077694833278656, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "generated_tangents": [ + [ + -0.7903228998184204, + -0.18409757316112518, + -0.14013144373893738, + -2.5101399421691895, + -0.9746879935264587, + -0.32710710167884827, + 0.23014850914478302, + -0.35718467831611633, + -0.18316799402236938, + -0.11985679715871811, + 0.04373310133814812, + -1.4271233081817627, + 0.26793184876441956, + 0.49707841873168945, + -0.33715230226516724, + -0.6178942322731018, + 0.7486673593521118, + 0.5280073881149292, + -0.09113270044326782, + -0.6513424515724182, + 0.3264918029308319 + ], + [ + -0.6438853144645691, + -0.21396349370479584, + 0.1571500301361084, + -2.6774978637695312, + -0.9535077214241028, + -1.0579779148101807, + -0.03666716814041138, + -0.33191463351249695, + -0.08599188923835754, + -0.1596742570400238, + 0.0914076641201973, + -0.9398535490036011, + 0.6325937509536743, + 0.6105518341064453, + -0.3370731770992279, + -0.5366656184196472, + 0.6772626042366028, + 0.4953550100326538, + -0.21307837963104248, + -0.4033760130405426, + -0.02836097776889801 + ], + [ + -0.8709462881088257, + -0.32414430379867554, + 0.20400968194007874, + -2.6513242721557617, + -0.7906050086021423, + -1.0159655809402466, + 0.037367284297943115, + -0.05022421479225159, + -0.07747355103492737, + -0.1263035535812378, + 0.0053431689739227295, + -1.0356826782226562, + 0.7024953365325928, + 0.5128731727600098, + -0.34005260467529297, + -0.4671139121055603, + 0.5004689693450928, + 0.5561464428901672, + 0.11805525422096252, + -0.11512330174446106, + -0.008380204439163208 + ], + [ + -0.8215875029563904, + -0.37770628929138184, + -0.01451060175895691, + -2.6621549129486084, + -1.2418372631072998, + -0.8316209316253662, + 0.04187417030334473, + -0.17782306671142578, + -0.043381065130233765, + -0.1426493227481842, + 0.02887875773012638, + -0.9600236415863037, + 0.3839133679866791, + 0.539447009563446, + -0.28831005096435547, + -0.5900147557258606, + 0.35402023792266846, + 0.5745357871055603, + -0.0773160457611084, + -0.39950305223464966, + 0.1341574490070343 + ], + [ + -0.5483207702636719, + -0.2609912157058716, + 0.2517930567264557, + -2.571366310119629, + -1.1163084506988525, + -0.960135817527771, + 0.07333788275718689, + -0.3056282103061676, + -0.052837468683719635, + 0.17141781747341156, + -0.024515755474567413, + -0.7675437927246094, + 0.7875030040740967, + 0.323081374168396, + -0.3825334310531616, + -0.43633607029914856, + 0.32293570041656494, + 0.5625973343849182, + -0.003449380397796631, + -0.2669273614883423, + 0.16390341520309448 + ], + [ + -0.5376284122467041, + -0.23660890758037567, + 0.18803945183753967, + -2.50242018699646, + -1.0672125816345215, + -0.782788872718811, + 0.03912481665611267, + -0.23303145170211792, + -0.03477063030004501, + 0.1316252201795578, + -0.022717773914337158, + -0.8641567230224609, + 0.5775023698806763, + 0.19517473876476288, + -0.3026233911514282, + -0.570724368095398, + 0.3069618344306946, + 0.5117232203483582, + 0.04628857970237732, + -0.13032598793506622, + 0.0027305632829666138 + ], + [ + -0.7411304116249084, + -0.45104676485061646, + -0.06603893637657166, + -2.5610299110412598, + -1.1784672737121582, + -0.6976543664932251, + -0.02025526762008667, + -0.11350211501121521, + -0.05391823500394821, + -0.04498853534460068, + 0.17176511883735657, + -0.9517258405685425, + 0.384677916765213, + 0.39348429441452026, + -0.21921876072883606, + -0.548501193523407, + 0.2978198528289795, + 0.41421887278556824, + -0.1791597306728363, + -0.34533971548080444, + 0.08193863928318024 + ], + [ + -0.960375189781189, + -0.22278209030628204, + 0.1317155659198761, + -2.5167276859283447, + -1.2622530460357666, + -0.9951598644256592, + 0.19187024235725403, + -0.15784722566604614, + 0.027605026960372925, + -0.22090288996696472, + -0.1198325976729393, + -0.900932788848877, + 0.5397241115570068, + 0.024562105536460876, + -0.33324357867240906, + -0.5202938914299011, + 0.4540517032146454, + 0.48335450887680054, + 0.2966420650482178, + -0.1541118025779724, + 0.2457282841205597 + ] + ], + "generated_utilities": [ + 0.03734047710895538, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.077694833278656, + 2.0 + ], + "group_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.17512565851211548, + 2.0, + 2.0, + 2.0, + 0.10078734159469604, + 2.0, + 2.0, + 0.06518048048019409, + 2.0, + 2.0, + 0.05268773436546326, + 0.056524619460105896, + 0.07100173830986023, + 0.16304254531860352, + 2.0, + 0.07095202803611755 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -1.7685134410858154, + 0.4561518430709839, + 0.8554723262786865, + -2.9925076961517334, + -1.652122139930725, + -0.9696183204650879, + 1.440989375114441, + -0.1663060486316681, + 0.005464707035571337, + -0.08078178763389587, + 0.5969918370246887, + -1.7046172618865967, + 0.3472420275211334, + 0.7052499651908875, + -0.026682782918214798, + 0.013620201498270035, + 0.002864290028810501, + 1.5629743337631226, + 0.06437517702579498, + -0.20232005417346954, + 0.06808048486709595 + ], + [ + -1.571894645690918, + 0.49223509430885315, + 1.2358425855636597, + -2.7074813842773438, + -1.6053731441497803, + -0.9737302660942078, + 1.2407286167144775, + -0.41521748900413513, + 0.27269765734672546, + -0.08917579054832458, + 0.4081580936908722, + -1.3627166748046875, + 0.07880738377571106, + 0.7477485537528992, + -0.2180848866701126, + -0.13109977543354034, + -0.3875364661216736, + 1.8194886445999146, + -0.06430798768997192, + 0.07869948446750641, + -0.12621839344501495 + ], + [ + -1.7744590044021606, + 0.367733359336853, + 1.3591725826263428, + -3.3725903034210205, + -1.8434977531433105, + -0.8745664954185486, + 1.6967558860778809, + -0.44676536321640015, + -0.6177129149436951, + 0.03244173526763916, + 0.4171296954154968, + -1.4895254373550415, + 0.9296896457672119, + 0.15846145153045654, + -0.006281482055783272, + 0.6454329490661621, + 0.6225314140319824, + 1.6578954458236694, + 0.3061527609825134, + -0.41300463676452637, + -0.2527916431427002 + ], + [ + -1.7233139276504517, + 0.020191192626953125, + 1.0188184976577759, + -2.873849868774414, + -2.027033567428589, + -1.1650824546813965, + 2.0792155265808105, + -0.8799198865890503, + 0.6924232840538025, + -0.7650182247161865, + -0.04520058631896973, + -2.468881130218506, + -0.13257521390914917, + 0.6857031583786011, + -0.10177457332611084, + -0.7609032988548279, + -0.08249430358409882, + 1.7503252029418945, + 0.1621951013803482, + -0.22467197477817535, + 0.43942898511886597 + ], + [ + -2.3507251739501953, + 0.16114166378974915, + 1.5912489891052246, + -2.1838369369506836, + -2.061230182647705, + -1.7918524742126465, + 0.8407515287399292, + -0.3446531295776367, + -0.7727420926094055, + -0.2617931067943573, + 0.6622774004936218, + -1.05295991897583, + 0.13840271532535553, + 1.2008695602416992, + -0.509428858757019, + 0.7052024006843567, + -0.6614989042282104, + 2.0029079914093018, + -0.82997065782547, + -0.9390356540679932, + 0.7515151500701904 + ], + [ + -1.7504347562789917, + -0.13171464204788208, + 0.7620843648910522, + -3.4511537551879883, + -2.5023341178894043, + -0.8595067858695984, + 1.6361833810806274, + -0.4263937175273895, + 0.005345506593585014, + -0.8147202730178833, + 0.4142894744873047, + -0.8735606670379639, + -0.4809738099575043, + 0.9412479996681213, + -0.6222366690635681, + -0.8843234777450562, + -0.8842994570732117, + 2.4228873252868652, + 0.6301752328872681, + 0.727215051651001, + 0.5050253868103027 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.7685134410858154, + -0.04630154371261597, + -0.002802610397338867, + -1.9925076961517334, + -0.6521221399307251, + -0.013848721981048584, + 0.0, + 0.014648675918579102, + -0.0837627500295639, + -0.04761720821261406, + -0.024705946445465088, + -0.7046172618865967, + 0.0914837121963501, + 0.0, + 0.04683725908398628, + 0.048148274421691895, + 0.03868643194437027, + 0.5629743337631226, + -0.043637245893478394, + -0.08927876502275467, + 0.0 + ], + [ + -0.7685134410858154, + 0.1268729865550995, + -0.0878140926361084, + -1.9925076961517334, + -0.6521221399307251, + -0.08497345447540283, + 0.0, + -0.23278282582759857, + 0.1505891978740692, + -0.0029508620500564575, + -0.03223156929016113, + -0.7046172618865967, + -0.18528565764427185, + 0.0, + 0.009110454469919205, + 0.006383728235960007, + -0.012130538001656532, + 0.5629743337631226, + 0.23608635365962982, + -0.06009632349014282, + 0.0 + ], + [ + -0.7685134410858154, + 0.6076252460479736, + 0.2745842933654785, + -1.9925076961517334, + -0.6521221399307251, + 0.03038167953491211, + 0.0, + 0.33581337332725525, + 0.35011863708496094, + -0.06728862971067429, + -0.4030081629753113, + -0.7046172618865967, + -0.44776007533073425, + 0.0, + -0.01595483534038067, + -0.040477462112903595, + 0.28290560841560364, + 0.5629743337631226, + -0.42285358905792236, + -0.11823103576898575, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.881978750228882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.410499930381775, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.1361609697341919 + ], + [ + -1.5711963176727295, + 0.5461994409561157, + 0.8384940028190613, + -2.960416078567505, + -1.9055238962173462, + -0.7765143513679504, + 1.4308912754058838, + 0.10734349489212036, + 0.011199884116649628, + -0.028424743562936783, + 0.5893871784210205, + -1.7020739316940308, + 0.48640453815460205, + 0.9950923323631287, + 0.20675025880336761, + 0.11047534644603729, + 0.09110014140605927, + 1.3974875211715698, + -0.21079741418361664, + -0.3005509674549103, + 0.09192514419555664 + ], + [ + -2.12906813621521, + 0.3435881733894348, + 0.7164700627326965, + -2.750021457672119, + -1.3681434392929077, + -0.8937081694602966, + 1.6475064754486084, + -0.1447279453277588, + -0.44103574752807617, + -0.5038806200027466, + 0.9006609320640564, + -1.8256494998931885, + 0.433420330286026, + 0.3506351411342621, + 0.4619242548942566, + -0.3905109167098999, + -0.01803053542971611, + 1.9505069255828857, + -0.339522123336792, + -0.20715753734111786, + -0.410531610250473 + ], + [ + -1.194382667541504, + 0.9652969837188721, + 0.41722428798675537, + -2.6203644275665283, + -1.139541506767273, + -1.5412108898162842, + 1.667648434638977, + -0.5256404876708984, + 0.17716272175312042, + 0.30056339502334595, + 0.09983009099960327, + -1.8219844102859497, + -0.23339274525642395, + 0.5763673782348633, + -0.4285883605480194, + 0.1476086974143982, + -0.3515554666519165, + 1.2458031177520752, + 0.3135184645652771, + -0.33711761236190796, + -0.09264840185642242 + ], + [ + -2.0614774227142334, + -0.4080510139465332, + 0.5476135015487671, + -3.0662269592285156, + -1.497420310974121, + -0.11606889963150024, + 1.03195321559906, + -0.3406473994255066, + 0.12844708561897278, + -0.4317864775657654, + 1.035879135131836, + -0.7729326486587524, + 1.0862895250320435, + 0.16315871477127075, + -0.14380013942718506, + -0.8498450517654419, + 0.9780882000923157, + 1.0358695983886719, + -0.5399289727210999, + -0.10386796295642853, + 0.8144621849060059 + ] + ], + "predicted_scores": [ + 7.710109710693359, + 6.939748764038086, + 6.586607933044434, + 6.2206501960754395, + 6.1800737380981445, + 6.078464984893799, + 5.686131000518799, + 5.517265319824219 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "794181626", + "selected_index": 0, + "state_hash": "4043b3c43ca4f115e1ea4fbcd98c7d08f797df2efbfe6b67bcffa5592a8aa36d", + "stored_base_utility": 0.17512565851211548, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08194400370121002, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08194400370121002 + }, + { + "contact_quality": null, + "progress": 0.09846144914627075, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09846144914627075 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.009868413209915161, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009868413209915161 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.2353062629699707e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.2353062629699707e-05 + } + ], + "candidate_progress": [ + 1.0, + 0.08194400370121002, + 0.09846144914627075, + 1.0, + 0.009868413209915161, + 1.0, + 1.0, + 1.2353062629699707e-05 + ], + "candidate_source_chart_ids": [ + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + false, + true, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-58950fed04d203b97ab9", + "generated_tangents": [ + [ + -0.1266256421804428, + -0.002259131520986557, + 1.6213842630386353, + -0.7543456554412842, + -1.6134655475616455, + 0.1789816915988922, + 0.22414876520633698, + -0.13272294402122498, + 0.005732003599405289, + 0.31355682015419006, + 0.10824506729841232, + -1.4673373699188232, + 0.34103503823280334, + 0.4597412943840027, + 0.1806682050228119, + -0.0032630981877446175, + -0.08528648316860199, + -0.011055469512939453, + -0.704582929611206, + 0.005777209997177124, + 0.024259299039840698 + ], + [ + -0.26199430227279663, + 0.005657803267240524, + 1.5621333122253418, + -0.6827386617660522, + -1.7485694885253906, + 0.5028383731842041, + 0.3064003586769104, + -0.09494590759277344, + -0.04572809487581253, + 0.3929176330566406, + -0.07712054997682571, + -1.3926876783370972, + 0.45926740765571594, + 0.45114612579345703, + 0.1326114982366562, + 0.008343080058693886, + 0.029313594102859497, + -0.019701898097991943, + -0.5967046618461609, + -0.2046085149049759, + -0.039422422647476196 + ], + [ + -0.030679523944854736, + 0.10015365481376648, + 1.5865848064422607, + -0.7421698570251465, + -1.6257531642913818, + 0.35852259397506714, + 0.30258017778396606, + 0.0038167834281921387, + 0.05335818976163864, + 0.26822400093078613, + -0.003624599426984787, + -1.505874752998352, + 0.12448570132255554, + 0.1746404618024826, + 0.17631147801876068, + 0.019554976373910904, + -0.028907790780067444, + 0.029448434710502625, + -0.6635255813598633, + 0.011510729789733887, + 0.060159601271152496 + ], + [ + -0.37533628940582275, + -0.009217534214258194, + 1.356131911277771, + -0.9588888883590698, + -1.2638320922851562, + 0.2709219753742218, + 0.1071615219116211, + -0.28787726163864136, + -0.024955475702881813, + 0.18631234765052795, + -0.004405900835990906, + -1.5419903993606567, + 0.4139575660228729, + 0.3009853959083557, + 0.09969498217105865, + -0.07751719653606415, + -0.08473540842533112, + 0.14005310833454132, + -0.4224899411201477, + -0.0025836527347564697, + -0.0714116245508194 + ], + [ + -0.24527902901172638, + 0.20554082095623016, + 1.314713478088379, + -0.7239284515380859, + -1.1570618152618408, + 0.689300537109375, + 0.32333698868751526, + 0.02252228558063507, + -0.08762882649898529, + 0.07853666692972183, + -0.09377797693014145, + -1.4810137748718262, + 0.02623167634010315, + 0.2645219564437866, + -0.027446914464235306, + -0.059286776930093765, + -0.014142453670501709, + -0.022253021597862244, + -0.6485402584075928, + 0.06886330246925354, + 0.055783092975616455 + ], + [ + -0.18931667506694794, + 0.11410576105117798, + 1.4706931114196777, + -0.8522800207138062, + -1.6707148551940918, + 0.5215132832527161, + 0.09922018647193909, + -0.11198189854621887, + 0.00743284123018384, + 0.1963292956352234, + -0.09166524559259415, + -1.4035396575927734, + 0.2550733983516693, + 0.3028809428215027, + 0.10759639739990234, + -0.017149310559034348, + 0.11416398733854294, + 0.16294322907924652, + -0.5847540497779846, + -0.00019565224647521973, + 0.02579958736896515 + ], + [ + -0.1178145557641983, + 0.11055108904838562, + 1.5651174783706665, + -0.8255586624145508, + -1.5410065650939941, + 0.4679739773273468, + 0.1807832419872284, + 0.025051862001419067, + -0.01573147065937519, + 0.2161959707736969, + -0.04220806807279587, + -1.5332847833633423, + 0.3336579501628876, + 0.25811365246772766, + 0.053960882127285004, + -0.017516309395432472, + 0.06422293186187744, + 0.04104362428188324, + -0.6408693194389343, + -0.12048624455928802, + 0.13836044073104858 + ], + [ + -0.11564154922962189, + 0.105094775557518, + 1.4515327215194702, + -0.8464374542236328, + -1.6001012325286865, + 0.3799157440662384, + 0.13040688633918762, + -0.06551563739776611, + 0.01518826000392437, + 0.24243757128715515, + -0.11430519074201584, + -1.3762128353118896, + 0.15279600024223328, + 0.3622298836708069, + 0.05797265097498894, + -0.039949700236320496, + 0.04767826199531555, + 0.1508132517337799, + -0.6215991377830505, + 0.07235613465309143, + -0.13865765929222107 + ] + ], + "generated_utilities": [ + 2.0, + 0.08194400370121002, + 0.09846144914627075, + 2.0, + 0.009868413209915161, + 2.0, + 2.0, + 1.2353062629699707e-05 + ], + "group_id": "ms-58950fed04d203b97ab9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20183078944683075, + 0.18959736824035645, + 2.0, + 0.0385412722826004, + 0.15877258777618408, + 0.03365510702133179, + 0.17151890695095062, + 0.13845136761665344, + 0.11539244651794434, + 0.1388089954853058, + 0.15365298092365265, + 0.0967668741941452, + 0.0908290445804596, + 0.06094485521316528, + 0.0377749502658844, + 0.028643101453781128 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.028361663222312927, + -0.489355206489563, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.16713106632232666, + 0.0, + 0.5545767545700073, + -0.18274790048599243, + 0.2772042155265808, + -0.45601847767829895, + -1.3811633586883545, + -0.1571006327867508, + 0.0, + -0.004826217889785767, + 0.2916078567504883, + 0.34807145595550537, + -0.37882840633392334, + 0.08759653568267822, + -0.07658697664737701, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.797813892364502, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.654926061630249, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4572639465332031 + ], + [ + 0.17793144285678864, + 0.008656054735183716, + 2.343496322631836, + -1.9455183744430542, + -2.4697093963623047, + 1.0036901235580444, + 2.5124316215515137, + -0.29308056831359863, + -0.02035735547542572, + -0.059014320373535156, + -0.5863867402076721, + -2.254472255706787, + 0.3354390263557434, + 1.381989598274231, + -0.27729421854019165, + 0.5023907423019409, + -0.8654648661613464, + 0.39588332176208496, + -0.5682056546211243, + 0.20400357246398926, + 0.48529666662216187 + ], + [ + -0.5227808952331543, + 0.12448178231716156, + 2.4855635166168213, + -1.6602094173431396, + -2.3857760429382324, + 0.323077529668808, + 1.9053025245666504, + -0.698577344417572, + 0.619890034198761, + 0.3500756621360779, + -0.1476593315601349, + -2.8024404048919678, + -0.21084021031856537, + 1.0928113460540771, + -0.3938775062561035, + 0.4228009581565857, + -0.39060449600219727, + -0.31665533781051636, + -0.9205541610717773, + 0.03774743899703026, + 0.3333621621131897 + ], + [ + 0.10275222361087799, + 0.08925776183605194, + 2.0934102535247803, + -1.379586935043335, + -3.3245811462402344, + 0.6494860649108887, + 1.990661382675171, + -0.021198302507400513, + 0.4400750398635864, + 0.39176517724990845, + -0.8639137148857117, + -1.981496810913086, + 0.01414397917687893, + 1.2019504308700562, + -0.7839368581771851, + 0.108054518699646, + -0.17177742719650269, + -0.36239588260650635, + -0.5078206062316895, + 0.4236416518688202, + 0.25084057450294495 + ], + [ + -0.8110343217849731, + -0.11216431856155396, + 2.8897132873535156, + -1.319732427597046, + -3.1141648292541504, + 0.416613906621933, + 2.3244988918304443, + -0.6579729914665222, + -0.39938226342201233, + -0.08646327257156372, + -0.3429824113845825, + -2.4165658950805664, + -0.6488300561904907, + 1.4793217182159424, + -0.8628695011138916, + 0.6845848560333252, + -0.7794754505157471, + -0.014068186283111572, + -0.27444708347320557, + 0.8030790686607361, + 0.08975183963775635 + ], + [ + -0.23147638142108917, + -0.26693814992904663, + 2.8227345943450928, + -2.3573813438415527, + -3.3905396461486816, + 0.38961973786354065, + 1.9831907749176025, + -1.2505497932434082, + 0.00624750554561615, + 0.9689428210258484, + 0.11981219053268433, + -1.5865073204040527, + 0.4901374876499176, + 0.6158997416496277, + 0.3767610788345337, + -0.025944411754608154, + -1.4681568145751953, + 0.48217374086380005, + -1.5973502397537231, + -0.7199738621711731, + 0.5119494795799255 + ], + [ + -0.19429966807365417, + 0.7059361934661865, + 3.2694599628448486, + -2.656050682067871, + -2.5502617359161377, + -0.23801833391189575, + 1.628230333328247, + 0.36624830961227417, + -0.08827590942382812, + -0.2098810374736786, + -1.1745270490646362, + -2.2196781635284424, + 0.44475334882736206, + 0.48411858081817627, + -0.25864526629447937, + 0.2009485512971878, + -0.9008907079696655, + -0.3571908473968506, + -0.16894888877868652, + -0.6256314516067505, + -0.4078131914138794 + ], + [ + 0.22205571830272675, + 0.6026778221130371, + 1.8579424619674683, + -2.404672384262085, + -3.728808641433716, + -0.19806396961212158, + 2.8292319774627686, + -0.2435564398765564, + 0.7903643846511841, + -0.376501202583313, + -1.0294175148010254, + -1.7672805786132812, + 0.5992995500564575, + 2.044847249984741, + 0.0011147260665893555, + 0.18739356100559235, + -0.952850341796875, + 0.8702673316001892, + 0.038269996643066406, + -0.4372258484363556, + -0.37835830450057983 + ], + [ + -0.3783646821975708, + 0.08899453282356262, + 2.721517324447632, + -2.3160932064056396, + -2.0679585933685303, + 0.549463152885437, + 3.141833543777466, + 0.09782099723815918, + 0.5830706357955933, + 0.7134941816329956, + 0.2986952066421509, + -2.9980335235595703, + -0.6391466856002808, + 1.075132131576538, + -0.7696593403816223, + 0.4343666732311249, + -0.911625862121582, + -0.31454378366470337, + -1.5935450792312622, + -0.8993679881095886, + -0.3811922073364258 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.1209426149725914, + 0.10675475001335144, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.0875239372253418, + 0.0, + -0.0486987829208374, + 0.06580346822738647, + 0.24258655309677124, + -0.20229026675224304, + -1.3811633586883545, + 0.4185009002685547, + 0.0, + -0.1431063860654831, + 0.15154658257961273, + 0.17190229892730713, + 0.10405468940734863, + 0.036865830421447754, + -0.1988307237625122, + 0.0 + ] + ], + "predicted_scores": [ + 7.98698091506958, + 7.975624084472656, + 7.712305545806885, + 7.680033206939697, + 7.596920013427734, + 7.543159008026123, + 7.519227504730225, + 7.204469203948975 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113371261", + "selected_index": 0, + "state_hash": "1b9018834a77531260bf8dfa74a9832916fe215bda78b5367b061bb4e0ce2333", + "stored_base_utility": 0.20183078944683075, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.01641605794429779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01641605794429779 + }, + "base_progress": 0.01641605794429779, + "base_success": false, + "base_utility": 0.01641605794429779, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1069791316986084, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1069791316986084 + }, + { + "contact_quality": null, + "progress": 0.05026264488697052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05026264488697052 + }, + { + "contact_quality": null, + "progress": 0.09185349941253662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09185349941253662 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08999918401241302, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08999918401241302 + }, + { + "contact_quality": null, + "progress": 0.1404910683631897, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1404910683631897 + } + ], + "candidate_progress": [ + 1.0, + 0.1069791316986084, + 0.05026264488697052, + 0.09185349941253662, + 1.0, + 1.0, + 0.08999918401241302, + 0.1404910683631897 + ], + "candidate_source_chart_ids": [ + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-9f61e09c05b4c8f74366", + "ms-9f61e09c05b4c8f74366" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + false, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-651bc0395d78d2819158", + "generated_tangents": [ + [ + -0.504991352558136, + -0.21939893066883087, + 1.3444803953170776, + -0.743743896484375, + -1.3850276470184326, + -0.031286150217056274, + 1.1078267097473145, + -0.10409319400787354, + 0.06178978830575943, + -0.17713001370429993, + -0.0072335004806518555, + -1.1815955638885498, + 0.18793776631355286, + 0.005217045545578003, + -0.26193767786026, + -0.16495974361896515, + 0.25650161504745483, + 1.0371205806732178, + 0.04905816912651062, + -0.4537602663040161, + 0.5095398426055908 + ], + [ + -0.35190099477767944, + -0.2557993531227112, + 1.6199679374694824, + -0.9254664182662964, + -1.6567935943603516, + -0.3687204420566559, + 1.2310676574707031, + -0.07828259468078613, + -0.05586294084787369, + -0.5629380345344543, + 0.2418472170829773, + -1.8919719457626343, + 0.3157327473163605, + 0.3827357888221741, + -0.23709455132484436, + -0.34630805253982544, + 0.32932648062705994, + 1.38849675655365, + -0.07390868663787842, + -0.10740266740322113, + 0.666833758354187 + ], + [ + -0.34218829870224, + -0.27783042192459106, + 1.4953527450561523, + -1.1027888059616089, + -1.684415340423584, + -0.512134313583374, + 1.1624705791473389, + -0.330881804227829, + -0.15998603403568268, + -0.36967188119888306, + 0.10534857213497162, + -1.759790301322937, + 0.3465334475040436, + 0.548625111579895, + -0.18405956029891968, + -0.38847607374191284, + 0.3852667808532715, + 1.4972847700119019, + -0.4169442057609558, + -0.24077847599983215, + 0.5337702035903931 + ], + [ + -0.34306859970092773, + -0.2906871736049652, + 1.4421676397323608, + -1.031981348991394, + -1.768282413482666, + -0.7896502017974854, + 1.0640686750411987, + -0.1864977478981018, + -0.10034267604351044, + -0.377536416053772, + 0.04089305177330971, + -1.8937641382217407, + 0.4347366392612457, + 0.3884339928627014, + -0.15927137434482574, + -0.3172740340232849, + 0.4115046262741089, + 1.4296550750732422, + -0.29879748821258545, + -0.2692466974258423, + 0.6491756439208984 + ], + [ + -0.4007565379142761, + -0.19153250753879547, + 1.4736835956573486, + -0.9515355825424194, + -1.1882541179656982, + -0.3834197223186493, + 0.5090863108634949, + -0.011358320713043213, + -0.0645751953125, + -0.3433118462562561, + 0.08590733259916306, + -1.5041146278381348, + 0.4623602330684662, + 0.184320330619812, + -0.04183097556233406, + -0.29643651843070984, + 0.5616769194602966, + 1.236170768737793, + -0.08876863121986389, + -0.4893401861190796, + 0.5256282091140747 + ], + [ + -0.5128392577171326, + -0.2719690799713135, + 1.5210754871368408, + -1.0445642471313477, + -1.494227409362793, + -0.4725043475627899, + 0.9738064408302307, + -0.22282347083091736, + -0.1320306956768036, + -0.44933080673217773, + 0.061698026955127716, + -1.736612319946289, + 0.35390862822532654, + 0.40314024686813354, + -0.10215525329113007, + -0.39843082427978516, + 0.468929260969162, + 1.4518229961395264, + -0.31548869609832764, + -0.2277843952178955, + 0.580947995185852 + ], + [ + -0.3465387225151062, + -0.32379454374313354, + 1.3886327743530273, + -0.7451854944229126, + -1.4972631931304932, + 0.1540457159280777, + 0.280448317527771, + 0.03724488615989685, + 0.0381455272436142, + -0.16173213720321655, + 0.10003554075956345, + -1.3670969009399414, + 0.22218278050422668, + 0.24737313389778137, + -0.3057235777378082, + -0.20689114928245544, + 0.4633018970489502, + 1.149208903312683, + -0.19519853591918945, + -0.471104234457016, + 0.6061217784881592 + ], + [ + -0.3432949185371399, + -0.2844322621822357, + 1.3986402750015259, + -0.8788204193115234, + -1.512843370437622, + -0.16554951667785645, + 0.3919046223163605, + 0.0830933079123497, + 0.02433175966143608, + -0.36533844470977783, + -0.01762431114912033, + -1.3434616327285767, + 0.05337068438529968, + 0.29428839683532715, + -0.1534549444913864, + -0.22810466587543488, + 0.5390974879264832, + 1.1905592679977417, + -0.23922199010849, + -0.4439091384410858, + 0.5905386209487915 + ] + ], + "generated_utilities": [ + 2.0, + 0.1069791316986084, + 0.05026264488697052, + 0.09185349941253662, + 2.0, + 2.0, + 0.08999918401241302, + 0.1404910683631897 + ], + "group_id": "ms-651bc0395d78d2819158", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 7.152557373046875e-07, + 2.0, + 0.0, + 0.0, + 0.18490922451019287, + 0.0, + 0.18801423907279968, + 2.0, + 0.18292954564094543, + 0.06743006408214569, + 2.0, + 0.08718416094779968, + 0.0016145259141921997, + 2.0, + 0.0, + 0.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.13822472095489502, + 0.052123308181762695, + 1.2542681694030762, + -0.5262794494628906, + -1.515254020690918, + -0.022995710372924805, + 0.0, + 0.07829460501670837, + 0.056280869990587234, + -0.006563127040863037, + 0.00594717264175415, + -1.0464823246002197, + 0.028334707021713257, + 0.0, + 0.04214516654610634, + -0.0032968670129776, + 0.025241270661354065, + 1.0898823738098145, + -0.09987176954746246, + 0.009208589792251587, + 0.0 + ], + [ + -1.138224720954895, + -0.12241673469543457, + 2.254268169403076, + -1.5262794494628906, + -2.515254020690918, + -0.4036164879798889, + 2.5164785385131836, + -0.48240265250205994, + 0.09632497280836105, + -1.0002868175506592, + 0.6668505668640137, + -2.0464823246002197, + 0.5158075094223022, + 0.48990970849990845, + 0.02257649227976799, + -0.23748205602169037, + -0.2213379293680191, + 2.0898823738098145, + 0.1738651841878891, + -0.37470543384552, + -0.06145218014717102 + ], + [ + -1.0695267915725708, + -0.1775755137205124, + 2.1683619022369385, + -1.4577077627182007, + -2.7917678356170654, + -0.632304310798645, + 2.3911960124969482, + -0.4637458324432373, + 0.35316434502601624, + -0.8419650793075562, + 0.8253071308135986, + -1.9648017883300781, + 0.2602742314338684, + 0.4741981625556946, + 0.17542533576488495, + -0.15374207496643066, + -0.4857789874076843, + 2.119079828262329, + 0.44966256618499756, + -0.3760189712047577, + 0.18533895909786224 + ], + [ + -0.7955029010772705, + 0.23637625575065613, + 1.9978463649749756, + -1.8726613521575928, + -2.2562904357910156, + -0.3048732876777649, + 2.261690378189087, + -0.6620026230812073, + 0.06043636053800583, + -1.091504693031311, + 1.038455605506897, + -1.9805728197097778, + 0.7370907068252563, + 0.5352527499198914, + 0.26298987865448, + -0.2739749252796173, + 0.14720074832439423, + 2.0173659324645996, + -0.16336800158023834, + -0.5858949422836304, + -0.1647021770477295 + ], + [ + -1.6335437297821045, + -0.08274295926094055, + 1.833235502243042, + -1.758880853652954, + -2.0250279903411865, + -0.8149738311767578, + 2.4238271713256836, + -0.33182430267333984, + -0.16191768646240234, + -1.4929053783416748, + 0.8467401266098022, + -2.4791038036346436, + 0.2757382392883301, + 0.16471439599990845, + 0.18122689425945282, + -0.2995648682117462, + -0.34926944971084595, + 2.063732147216797, + 0.3726923167705536, + -0.259366512298584, + 0.36020517349243164 + ], + [ + -0.9912663698196411, + -0.19695568084716797, + 2.342681646347046, + -2.1077044010162354, + -3.031508684158325, + -0.6591198444366455, + 2.5516788959503174, + -0.8631293773651123, + 0.2481168806552887, + -1.107275366783142, + 0.5796921253204346, + -2.479954719543457, + 1.0449233055114746, + 0.40612131357192993, + -0.12985309958457947, + -0.18829946219921112, + -0.6196849346160889, + 1.6295442581176758, + 0.4197757840156555, + 0.02911210060119629, + 0.5378711223602295 + ], + [ + -0.6689683198928833, + -0.5626275539398193, + 1.738625407218933, + -1.919468879699707, + -2.6463799476623535, + -0.32590651512145996, + 2.287154197692871, + -0.9975759983062744, + 0.7224722504615784, + -1.427150845527649, + 0.572014331817627, + -2.2473490238189697, + 0.8538937568664551, + 0.0030456185340881348, + -0.1059482991695404, + -0.003734409809112549, + -0.2650519907474518, + 2.554786205291748, + -0.24461163580417633, + -0.6881535649299622, + 0.5301691293716431 + ], + [ + -1.5373890399932861, + 0.42636638879776, + 2.9180266857147217, + -2.126905679702759, + -1.825490117073059, + -0.6280819773674011, + 2.593716621398926, + -0.6465503573417664, + 0.5798410177230835, + -0.5376753807067871, + 0.5435326099395752, + -2.610151767730713, + 1.0700397491455078, + 0.7953996658325195, + 0.5072687864303589, + -0.02665230631828308, + -0.6953800320625305, + 2.0521392822265625, + 0.751386821269989, + -0.7565212249755859, + -0.5263289213180542 + ], + [ + -0.7768699526786804, + -0.18900753557682037, + 1.7192409038543701, + -2.2835047245025635, + -1.713951826095581, + 0.004747450351715088, + 3.2709672451019287, + -1.1866822242736816, + 0.38759592175483704, + -0.7857802510261536, + 1.539168119430542, + -1.3250088691711426, + 0.8453556299209595, + 0.5691420435905457, + -0.701140820980072, + -0.6115058660507202, + -0.8829987645149231, + 1.5742011070251465, + -0.3939879536628723, + 0.45414644479751587, + -0.23744438588619232 + ] + ], + "predicted_scores": [ + 7.084019184112549, + 7.08164119720459, + 7.0393757820129395, + 6.938310623168945, + 6.919251918792725, + 6.6739912033081055, + 6.482789039611816, + 6.373105525970459 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1483645336", + "selected_index": 0, + "state_hash": "857c28f61d13d7ee5a59956a87579f6c17f6c601c1680e574691e095e373d439", + "stored_base_utility": 7.152557373046875e-07, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15461373329162598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15461373329162598 + }, + "base_progress": 0.15461373329162598, + "base_success": false, + "base_utility": 0.15461373329162598, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.04685409367084503, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04685409367084503 + }, + { + "contact_quality": null, + "progress": 0.025345027446746826, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025345027446746826 + }, + { + "contact_quality": null, + "progress": 0.04131391644477844, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04131391644477844 + }, + { + "contact_quality": null, + "progress": 0.04660575091838837, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04660575091838837 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.04232649505138397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04232649505138397 + }, + { + "contact_quality": null, + "progress": 0.01882496476173401, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01882496476173401 + } + ], + "candidate_progress": [ + 0.04685409367084503, + 0.025345027446746826, + 0.04131391644477844, + 0.04660575091838837, + 1.0, + 1.0, + 0.04232649505138397, + 0.01882496476173401 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-1c4df81cce643ca4fb3e" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-9fad82fd288e4296e354", + "generated_tangents": [ + [ + 0.20766592025756836, + -0.09938950836658478, + 1.423433780670166, + -0.17177465558052063, + -1.3757376670837402, + -0.15777432918548584, + -0.24454718828201294, + -0.14580971002578735, + 0.26739832758903503, + 0.43735209107398987, + 0.041180569678545, + -1.3525785207748413, + -0.08127036690711975, + -0.23888324201107025, + -0.17473816871643066, + 0.14641612768173218, + 0.3547530770301819, + 0.006695836782455444, + -0.4463213086128235, + -0.29608821868896484, + -0.10081437230110168 + ], + [ + 0.34539926052093506, + -0.1247783973813057, + 1.5992226600646973, + -0.4015488028526306, + -1.4918618202209473, + -0.004178911447525024, + -0.8372355103492737, + 0.056121423840522766, + 0.23947948217391968, + 0.6515242457389832, + -0.2394605576992035, + -1.434688925743103, + -0.3629516065120697, + 0.14668413996696472, + -0.3211246132850647, + 0.26727861166000366, + 0.40521925687789917, + -0.09060023725032806, + -0.1244821846485138, + -0.5918140411376953, + 0.25310218334198 + ], + [ + 0.19805660843849182, + -0.1373877376317978, + 1.395082950592041, + -0.11858675628900528, + -1.5370681285858154, + -0.11043974757194519, + -0.12877482175827026, + -0.15525361895561218, + 0.3476954400539398, + 0.6211431622505188, + -0.1982797384262085, + -1.2383601665496826, + -0.15672820806503296, + 0.04531017690896988, + -0.16811484098434448, + 0.14060468971729279, + 0.2006876915693283, + -0.06664876639842987, + -0.22397667169570923, + -0.5180864930152893, + 0.097849041223526 + ], + [ + 0.17813198268413544, + -0.16313879191875458, + 1.4762564897537231, + -0.0008421540260314941, + -1.3843040466308594, + -0.09591269493103027, + -0.381478488445282, + -0.16069743037223816, + 0.2906929552555084, + 0.4442158341407776, + 0.0563124343752861, + -1.268372654914856, + -0.1029319167137146, + -0.24049584567546844, + -0.3363857865333557, + 0.12519130110740662, + 0.4553292989730835, + -0.04446130990982056, + -0.4481552839279175, + -0.33072778582572937, + -0.06801219284534454 + ], + [ + 0.09426534920930862, + 0.038272030651569366, + 1.1666783094406128, + -0.16440847516059875, + -1.0368421077728271, + 0.07006518542766571, + -0.32122868299484253, + -0.22001880407333374, + 0.3071807622909546, + 0.3413297235965729, + -0.10435640066862106, + -1.1479319334030151, + -0.3591143488883972, + -0.2282538264989853, + -0.413745641708374, + 0.06576137244701385, + 0.4322214126586914, + -0.18121947348117828, + -0.3420878052711487, + -0.4371933341026306, + -0.12038460373878479 + ], + [ + 0.2011847198009491, + -0.018461860716342926, + 1.3517429828643799, + -0.25713682174682617, + -1.3855352401733398, + 0.030164361000061035, + -0.38910025358200073, + -0.11146190762519836, + 0.2730139493942261, + 0.4387199878692627, + -0.025306664407253265, + -1.3061028718948364, + -0.04865357279777527, + -0.15673215687274933, + -0.2664736807346344, + 0.08021986484527588, + 0.40260010957717896, + -0.03546345233917236, + -0.4927268624305725, + -0.45318055152893066, + -0.07600349187850952 + ], + [ + 0.20030882954597473, + -0.16111081838607788, + 1.4788992404937744, + 0.09693899750709534, + -1.4145588874816895, + 0.15468072891235352, + -0.5813713669776917, + -0.047689273953437805, + 0.2658916711807251, + 0.45151978731155396, + 0.08848687261343002, + -1.2820138931274414, + -0.13745823502540588, + -0.2831106185913086, + -0.390105277299881, + 0.1235877275466919, + 0.5074005126953125, + 0.023937925696372986, + -0.3375319242477417, + -0.36059510707855225, + -0.15831956267356873 + ], + [ + 0.3738653361797333, + -0.1765938252210617, + 1.487797498703003, + -0.28906771540641785, + -1.2941184043884277, + 0.12946821749210358, + -0.5777290463447571, + 0.020496457815170288, + 0.2621426284313202, + 0.4477204978466034, + 0.17428931593894958, + -1.3336917161941528, + -0.3482901453971863, + -0.01711834967136383, + -0.3248338997364044, + 0.15900002419948578, + 0.4918249845504761, + 0.05082593858242035, + -0.3648627996444702, + -0.19320496916770935, + 0.061057500541210175 + ] + ], + "generated_utilities": [ + 0.04685409367084503, + 0.025345027446746826, + 0.04131391644477844, + 0.04660575091838837, + 2.0, + 2.0, + 0.04232649505138397, + 0.01882496476173401 + ], + "group_id": "ms-9fad82fd288e4296e354", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.19493231177330017, + 0.06623455882072449, + 2.0, + 0.15396031737327576, + 0.16184639930725098, + 0.025735855102539062, + 0.134642094373703, + 0.15606965124607086, + 0.12982505559921265, + 0.14038166403770447, + 0.09718823432922363, + 0.015769004821777344, + 0.09577034413814545, + 0.057760089635849, + 0.047366172075271606, + 0.035309627652168274 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.0622546412050724, + -0.09277644753456116, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + -0.005287081003189087, + 0.0, + -0.005681931972503662, + 0.03343720734119415, + 1.0536105632781982, + -0.00917971134185791, + -1.333775520324707, + 0.03716959059238434, + 0.0, + 0.019381746649742126, + 0.026990801095962524, + 0.21505475044250488, + -0.029777705669403076, + -0.14875531196594238, + -0.048092663288116455, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.959449768066406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.103323459625244, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.591951370239258 + ], + [ + -0.13909977674484253, + -0.26507115364074707, + 2.099381446838379, + -1.432984709739685, + -2.215683698654175, + 0.18431386351585388, + 2.42065691947937, + 0.3679453730583191, + -0.24726781249046326, + 2.055530071258545, + -0.4911385774612427, + -2.0511183738708496, + 0.04109602048993111, + 2.784976005554199, + 0.005537793040275574, + 0.05149966478347778, + 1.124292016029358, + -0.6457813382148743, + -1.226535677909851, + -0.28760284185409546, + 1.0270601511001587 + ], + [ + 0.30531278252601624, + -0.4336698055267334, + 2.6217315196990967, + -1.5083816051483154, + -2.5322670936584473, + 0.06784650683403015, + 2.3866119384765625, + -0.09851980209350586, + 0.2774229049682617, + 2.157179355621338, + -0.9796892404556274, + -2.8201849460601807, + 0.08135517686605453, + 2.818830728530884, + -0.6848310828208923, + 0.37913888692855835, + 0.7420558929443359, + -0.22611916065216064, + -1.5158166885375977, + 0.0614812970161438, + 1.3468891382217407 + ], + [ + 0.27444392442703247, + -0.22534464299678802, + 2.616544246673584, + -1.2606401443481445, + -2.0848917961120605, + -0.23496872186660767, + 2.1798665523529053, + 0.7968885898590088, + 0.26037585735321045, + 1.5473439693450928, + -0.233917236328125, + -1.867544412612915, + 0.48288947343826294, + 1.9728566408157349, + -0.4071806073188782, + 0.5528238415718079, + 1.6528295278549194, + -1.0936471223831177, + -1.062675952911377, + -0.5311015248298645, + 0.9778987765312195 + ], + [ + 0.5435068607330322, + 0.24469684064388275, + 2.6045663356781006, + -1.284223198890686, + -2.6095967292785645, + 0.9732104539871216, + 2.741159200668335, + 0.6031707525253296, + -0.5612202882766724, + 2.4774889945983887, + 0.1548280119895935, + -2.5516507625579834, + -0.0651187151670456, + 2.3864798545837402, + 0.130750373005867, + -0.1318313330411911, + 1.6559979915618896, + -1.0673086643218994, + -0.5477704405784607, + 0.18809780478477478, + 0.7076419591903687 + ], + [ + 0.31294435262680054, + 0.0263526514172554, + 3.0396268367767334, + -1.3165022134780884, + -1.8981198072433472, + 0.5049136281013489, + 2.31935715675354, + -0.4664327800273895, + -0.56757652759552, + 2.7719295024871826, + 0.21425002813339233, + -2.3925602436065674, + -0.26789456605911255, + 2.008218288421631, + -0.1879691481590271, + -0.34721577167510986, + 1.6638673543930054, + -0.30173182487487793, + -1.8719977140426636, + -0.8951039910316467, + 2.094865083694458 + ], + [ + 0.533157467842102, + 0.308005690574646, + 1.7770951986312866, + -0.6794775724411011, + -3.2953453063964844, + -0.15337547659873962, + 2.5854313373565674, + 0.07320006191730499, + -0.24033141136169434, + 2.5622174739837646, + -0.9168426990509033, + -1.5317132472991943, + -0.5942863821983337, + 2.2703769207000732, + -1.0709311962127686, + 0.6969255208969116, + 1.5304101705551147, + -0.7959219217300415, + -0.9165043830871582, + -0.8046965003013611, + 0.638719916343689 + ], + [ + 0.7393907904624939, + -0.13299384713172913, + 2.4281864166259766, + -1.861674189567566, + -3.3330533504486084, + -0.18516618013381958, + 1.6492295265197754, + -0.6932566165924072, + 0.06020718812942505, + 1.1755363941192627, + -0.016285300254821777, + -3.0671546459198, + -0.6052039861679077, + 2.6829426288604736, + 0.7021514177322388, + -0.477348268032074, + 0.466697633266449, + -0.9403301477432251, + -0.8690205812454224, + -1.1888904571533203, + 0.5360406041145325 + ], + [ + -0.0713023692369461, + -0.9776724576950073, + 1.2895376682281494, + -1.6489125490188599, + -1.4475497007369995, + -0.6822370886802673, + 3.14093017578125, + -0.5356632471084595, + -0.7523763179779053, + 2.766268253326416, + -0.8215572834014893, + -2.753042697906494, + 0.10929550230503082, + 2.570046901702881, + -0.34730663895606995, + 0.8763631582260132, + 1.3356308937072754, + -0.14330342411994934, + -1.1863114833831787, + -0.30006271600723267, + 0.7264611721038818 + ], + [ + 0.35943418741226196, + 0.22802312672138214, + 1.7158434391021729, + -1.4605827331542969, + -3.3450541496276855, + 0.23581582307815552, + 1.759531855583191, + 0.7012510299682617, + 0.0034519433975219727, + 1.4857903718948364, + -1.2866599559783936, + -2.1787426471710205, + 0.8807055354118347, + 1.9066205024719238, + -0.9918697476387024, + -0.32221919298171997, + 1.1986194849014282, + -1.187239646911621, + -0.5752912163734436, + -0.015687286853790283, + 1.775393009185791 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16282694041728973, + -0.1309112012386322, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + 0.15446226298809052, + 0.0, + -0.15669959783554077, + -0.08878482133150101, + 1.0536105632781982, + -0.09701520204544067, + -1.333775520324707, + 0.10988402366638184, + 0.0, + -0.06573517620563507, + -0.2108776718378067, + 0.21505475044250488, + -0.16805344820022583, + -0.14875531196594238, + 0.12529203295707703, + 0.0 + ] + ], + "predicted_scores": [ + 7.332526206970215, + 7.2481184005737305, + 7.235976696014404, + 7.188525676727295, + 7.012310028076172, + 6.965161323547363, + 6.905152320861816, + 6.820371150970459 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1176943004", + "selected_index": 0, + "state_hash": "0d923438bf2b227e3521cb937d011f5bddf9b553a882853098b702ec31090614", + "stored_base_utility": 0.19493231177330017, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06781657040119171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06781657040119171 + }, + "base_progress": 0.06781657040119171, + "base_success": false, + "base_utility": 0.06781657040119171, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.00024640560150146484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00024640560150146484 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05664622783660889, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05664622783660889 + }, + { + "contact_quality": null, + "progress": 0.00476934015750885, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00476934015750885 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.00024640560150146484, + 1.0, + 1.0, + 0.05664622783660889, + 0.00476934015750885, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + true, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "generated_tangents": [ + [ + -0.2442924529314041, + 0.1922399252653122, + 1.2617125511169434, + -1.0334233045578003, + -1.6436376571655273, + 0.2199554741382599, + -0.06904351711273193, + -0.16711318492889404, + 0.05172731727361679, + 0.07282762229442596, + 0.28897908329963684, + -1.388846516609192, + 0.1773185431957245, + 0.13332222402095795, + -0.046502482146024704, + 0.21015708148479462, + 0.28959494829177856, + -0.45105117559432983, + -0.6802242994308472, + -0.24409805238246918, + -0.05618833005428314 + ], + [ + -0.30237674713134766, + 0.15285466611385345, + 0.983101487159729, + -0.8516850471496582, + -1.0975096225738525, + 0.6081944704055786, + 0.6999843716621399, + -0.12977352738380432, + 0.20687785744667053, + 0.16245761513710022, + 0.10769888758659363, + -1.2781031131744385, + 0.3068641722202301, + -0.5050348043441772, + -0.06470249593257904, + 0.29483672976493835, + 0.2722031772136688, + -0.5721697807312012, + -0.0724286139011383, + -0.6215910911560059, + 0.49704527854919434 + ], + [ + -0.18169476091861725, + 0.19710227847099304, + 1.2476732730865479, + -1.0144259929656982, + -1.620957374572754, + 0.2072249799966812, + -0.06506145000457764, + -0.08163696527481079, + 0.08300820738077164, + 0.04539757966995239, + 0.2998795807361603, + -1.4227242469787598, + -0.027084678411483765, + 0.1919259876012802, + -0.07722999155521393, + 0.16494181752204895, + 0.22130902111530304, + -0.33830904960632324, + -0.7020812630653381, + -0.33060234785079956, + -0.13652271032333374 + ], + [ + -0.3333394527435303, + 0.1445809006690979, + 1.3304860591888428, + -0.9550267457962036, + -1.5173897743225098, + -0.11261460185050964, + 0.14770564436912537, + -0.23509421944618225, + 0.1560247838497162, + -0.012800522148609161, + 0.2957661747932434, + -1.4266231060028076, + 0.15834972262382507, + 0.08385638892650604, + 0.012017462402582169, + 0.2219429463148117, + 0.3547346293926239, + -0.4124802350997925, + -0.6573758125305176, + -0.023219317197799683, + -0.1346246898174286 + ], + [ + -0.1128702312707901, + 0.1393568515777588, + 1.321090817451477, + -0.9430903196334839, + -1.5495328903198242, + -0.13003066182136536, + -0.1664838194847107, + -0.17644405364990234, + 0.12526345252990723, + 0.11281096935272217, + 0.27010899782180786, + -1.229363203048706, + -0.0460125207901001, + 0.18365833163261414, + 0.009704198688268661, + 0.2022450566291809, + 0.3173149824142456, + -0.4220997095108032, + -0.8078009486198425, + -0.23880226910114288, + -0.21057868003845215 + ], + [ + -0.1460224837064743, + 0.12933729588985443, + 1.2848962545394897, + -0.9112169742584229, + -1.6112854480743408, + 0.058030061423778534, + -0.17862147092819214, + -0.1863740086555481, + 0.15170565247535706, + 0.07708920538425446, + 0.16633179783821106, + -1.0974202156066895, + -0.05451437830924988, + 0.17298021912574768, + 0.008827831596136093, + 0.20871904492378235, + 0.2916363477706909, + -0.4815630316734314, + -0.7385351061820984, + -0.08161972463130951, + -0.08039110898971558 + ], + [ + -0.08636783063411713, + 0.1789580136537552, + 1.2403063774108887, + -0.9093590974807739, + -1.5502665042877197, + -0.08800885081291199, + -0.1913967728614807, + -0.10208499431610107, + 0.11641716212034225, + 0.15578410029411316, + 0.25646504759788513, + -1.179133415222168, + 0.002452850341796875, + 0.2192654311656952, + 0.02191675826907158, + 0.242428258061409, + 0.3431023955345154, + -0.4159812927246094, + -0.8238541483879089, + -0.04872231185436249, + -0.16364553570747375 + ], + [ + -0.3311907649040222, + 0.1384386271238327, + 1.4478973150253296, + -0.9109632968902588, + -1.7710649967193604, + 0.5832236409187317, + -0.21054184436798096, + -0.179454505443573, + 0.21354767680168152, + 0.2505671977996826, + 0.5328578352928162, + -1.3277925252914429, + -0.054305195808410645, + -0.03966668248176575, + -0.14212284982204437, + 0.14360544085502625, + 0.5255115628242493, + -0.058621689677238464, + -0.5408797860145569, + -0.33775684237480164, + 0.0875023603439331 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.00024640560150146484, + 2.0, + 2.0, + 0.05664622783660889, + 0.00476934015750885, + 2.0 + ], + "group_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0793772041797638, + 2.0, + 2.0, + 2.0, + 0.1144598126411438, + 0.04584880173206329, + 0.08543911576271057, + 0.1307348757982254, + 0.10944780707359314, + 0.07284757494926453, + 0.05719476938247681, + 0.09351089596748352, + 0.13337698578834534, + 0.039893895387649536, + 0.09353269636631012, + 0.08224260807037354 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.016482055187225342, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.0903974175453186, + 0.0, + 0.012217208743095398, + 0.010632793419063091, + 0.03624187409877777, + 0.04524834081530571, + -1.3283450603485107, + -0.00880381464958191, + 0.0, + -0.014520794153213501, + 0.5632922649383545, + 0.003262042999267578, + -0.11586570739746094, + 0.0014255642890930176, + 0.015446990728378296, + 0.0 + ], + [ + 0.04007059335708618, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.12278485298156738, + 0.0, + -0.0962398424744606, + -0.08766138553619385, + 0.29553186893463135, + 0.10519622266292572, + -1.3283450603485107, + 0.00032788515090942383, + 0.0, + -0.13836456835269928, + 0.5632922649383545, + -0.13551044464111328, + -0.11586570739746094, + 0.07183873653411865, + -0.03684847429394722, + 0.0 + ], + [ + 0.08037775754928589, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.33122140169143677, + 0.0, + 0.20786377787590027, + -0.42098182439804077, + -0.17684945464134216, + 0.1062583178281784, + -1.3283450603485107, + 0.28051087260246277, + 0.0, + 0.7470210194587708, + 0.5632922649383545, + 0.03247350454330444, + -0.11586570739746094, + 0.4809897243976593, + -0.1744043231010437, + 0.0 + ], + [ + -0.4189682602882385, + 1.1229554414749146, + 2.8264856338500977, + -2.2389113903045654, + -2.4226768016815186, + 1.2001769542694092, + 2.1568119525909424, + -0.016906127333641052, + -0.24083517491817474, + -0.23574936389923096, + -0.13933053612709045, + -2.458132028579712, + 0.40251442790031433, + 0.870932936668396, + -0.28115370869636536, + 1.4463695287704468, + 0.9248121380805969, + -0.9938455820083618, + -0.680131196975708, + -0.051702484488487244, + -0.4663417339324951 + ], + [ + -0.6373537182807922, + 1.3288449048995972, + 3.1669085025787354, + -1.4171695709228516, + -2.242492198944092, + 1.6900161504745483, + 2.778874158859253, + 0.47899991273880005, + 0.3402186334133148, + 0.4507899582386017, + 0.8237879276275635, + -1.7865383625030518, + 1.1732149124145508, + 1.5826115608215332, + 0.4416889548301697, + 2.222722053527832, + 0.6443713307380676, + -1.8598005771636963, + -0.9690301418304443, + 0.3504742383956909, + -1.1371408700942993 + ] + ], + "predicted_scores": [ + 9.808351516723633, + 9.605087280273438, + 9.494207382202148, + 9.47028923034668, + 9.351375579833984, + 9.00699234008789, + 8.98760986328125, + 7.942086219787598 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "998761162", + "selected_index": 0, + "state_hash": "78cc63bdeba9f4d6d6da2efd23c0f7f6631321321e917a70299a7c8025e71d47", + "stored_base_utility": 0.0793772041797638, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05380085110664368, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05380085110664368 + }, + { + "contact_quality": null, + "progress": 0.045698270201683044, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.045698270201683044 + }, + { + "contact_quality": null, + "progress": 0.07195276021957397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07195276021957397 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.07060344517230988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07060344517230988 + }, + { + "contact_quality": null, + "progress": 0.07577565312385559, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07577565312385559 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.05380085110664368, + 0.045698270201683044, + 0.07195276021957397, + 1.0, + 0.07060344517230988, + 0.07577565312385559, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + true, + false, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-fc8084d5cd7da937fc02", + "generated_tangents": [ + [ + -0.25766533613204956, + -0.12600606679916382, + 1.309084415435791, + -0.5940244197845459, + -1.1092166900634766, + 0.5150706171989441, + -0.052182137966156006, + -0.018023774027824402, + 0.027493583038449287, + 0.243815615773201, + -0.08884450048208237, + -1.4417210817337036, + 0.0016799569129943848, + 0.07788446545600891, + -0.0633881539106369, + -0.05702465400099754, + 0.3619932532310486, + 0.22092172503471375, + -0.33220547437667847, + -0.20806413888931274, + 0.058834560215473175 + ], + [ + -0.24930612742900848, + -0.10163244605064392, + 1.2350733280181885, + -0.5576799511909485, + -0.9712830185890198, + 0.23886027932167053, + -0.014269083738327026, + -0.05562072992324829, + 0.0043869102373719215, + 0.2794634997844696, + -0.03582601994276047, + -1.3612873554229736, + 0.17686626315116882, + -0.24185006320476532, + 0.008196085691452026, + -0.03828829526901245, + 0.23057204484939575, + 0.1730470061302185, + -0.35135817527770996, + -0.18045026063919067, + -0.09966912865638733 + ], + [ + -0.1390257030725479, + -0.07180877774953842, + 1.373719573020935, + -0.7151130437850952, + -1.1378800868988037, + -0.11026304960250854, + 0.05199959874153137, + -0.137319415807724, + 0.06817563623189926, + 0.06592569500207901, + -0.063407763838768, + -1.4731884002685547, + -0.0001252293586730957, + 0.04567163437604904, + 0.0496208593249321, + -0.04812358692288399, + 0.3729042708873749, + 0.20860879123210907, + -0.3157980442047119, + 0.04793208837509155, + -0.033076465129852295 + ], + [ + -0.08036945760250092, + 0.00599384680390358, + 1.1883916854858398, + -0.5798259973526001, + -0.9847074151039124, + 0.38269490003585815, + 0.5433768630027771, + -0.11317121982574463, + 0.09527748823165894, + 0.021453171968460083, + -0.06421821564435959, + -1.3068289756774902, + 0.279157429933548, + -0.27088087797164917, + 0.0208757221698761, + 0.04182553291320801, + 0.11511608958244324, + -0.009401455521583557, + -0.058999449014663696, + -0.36277535557746887, + -0.017688199877738953 + ], + [ + -0.06852550804615021, + 0.004076119512319565, + 1.2988475561141968, + -0.7716856002807617, + -1.3796987533569336, + 0.12243293225765228, + -0.1760903000831604, + -0.0433368980884552, + 0.04176665097475052, + 0.15399578213691711, + -0.12742996215820312, + -1.4812989234924316, + -0.01767835021018982, + 0.14924274384975433, + 0.007744971662759781, + -0.032446056604385376, + 0.2803436517715454, + 0.27891480922698975, + -0.3602532744407654, + -0.09992602467536926, + 0.1479855626821518 + ], + [ + -0.12720204889774323, + -0.027446024119853973, + 1.2578574419021606, + -0.685033917427063, + -1.2497880458831787, + 0.11687909066677094, + -0.1411479115486145, + -0.08844837546348572, + 0.012235293164849281, + 0.13517729938030243, + -0.0583876296877861, + -1.3469491004943848, + 0.08236351609230042, + 0.25899428129196167, + 0.0673624724149704, + 0.0028378991410136223, + 0.22267533838748932, + 0.28155064582824707, + -0.24954187870025635, + -0.1778385043144226, + 0.08610621094703674 + ], + [ + -0.0639880895614624, + -0.002814624458551407, + 1.2676571607589722, + -0.7168252468109131, + -1.4181642532348633, + 0.12310458719730377, + -0.21598517894744873, + -0.040600910782814026, + 0.012664916925132275, + 0.16312536597251892, + -0.09044265002012253, + -1.3518496751785278, + 0.005789220333099365, + 0.18833549320697784, + 0.044131435453891754, + -0.012436660937964916, + 0.1918671429157257, + 0.3091844916343689, + -0.3346096873283386, + -0.016804546117782593, + 0.03484569489955902 + ], + [ + -0.010094091296195984, + -0.0019205734133720398, + 1.295836091041565, + -0.780786395072937, + -1.3846473693847656, + 0.20473794639110565, + -0.160369873046875, + -0.0183841735124588, + 0.03672788664698601, + 0.11521317064762115, + -0.08856336027383804, + -1.5128268003463745, + -0.15641868114471436, + 0.14340858161449432, + -0.0047133006155490875, + -0.0030567143112421036, + 0.22386585175991058, + 0.3224785625934601, + -0.4089009165763855, + -0.1121964156627655, + 0.03727264702320099 + ] + ], + "generated_utilities": [ + 0.05380085110664368, + 0.045698270201683044, + 0.07195276021957397, + 2.0, + 0.07060344517230988, + 0.07577565312385559, + 2.0, + 2.0 + ], + "group_id": "ms-fc8084d5cd7da937fc02", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.120394766330719, + 0.1662210375070572, + 2.0, + 0.0270967036485672, + 0.16474007070064545, + 0.014024153351783752, + 0.13986149430274963, + 0.17214345932006836, + 0.19527268409729004, + 0.08535367250442505, + 0.1354738473892212, + 0.11415943503379822, + 0.05034071207046509, + 0.10602898895740509, + 0.01804502308368683, + 0.09680917859077454 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.4742112457752228, + -0.6877930760383606, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.0012008249759674072, + 0.0, + 0.04633669555187225, + -0.307714581489563, + -0.22357922792434692, + -0.35245198011398315, + -1.486854076385498, + 0.15573161840438843, + 0.0, + 0.5157145261764526, + 0.243257537484169, + 0.2085912823677063, + 0.48896148800849915, + -0.06008961796760559, + 0.4603208899497986, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.883587837219238, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6201469898223877, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.5597648620605469 + ], + [ + -0.8389496207237244, + 0.5873794555664062, + 2.327263116836548, + -2.175445795059204, + -1.844232439994812, + 0.15723112225532532, + 2.920079231262207, + -0.5335484743118286, + -0.6761462688446045, + 0.48151493072509766, + 0.2968217730522156, + -2.99812650680542, + -0.27805766463279724, + 2.0262699127197266, + 0.10151885449886322, + -0.7244771122932434, + 1.0176090002059937, + 1.278473973274231, + -0.2260074019432068, + -1.0207180976867676, + 0.7682583928108215 + ], + [ + 0.2503487765789032, + 0.20628705620765686, + 2.289511203765869, + -0.8669976592063904, + -1.9807379245758057, + 1.1820356845855713, + 2.8384368419647217, + 0.43699324131011963, + -0.688888669013977, + 1.0812339782714844, + -0.18395745754241943, + -1.6906647682189941, + -0.040969282388687134, + 1.254612684249878, + 0.24077603220939636, + -1.0562812089920044, + 1.2436273097991943, + 0.2388666272163391, + -0.5016235709190369, + 0.4570234417915344, + 1.355008840560913 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.10285450518131256, + -0.2634444832801819, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.12851905822753906, + 0.0, + 0.06062778830528259, + 0.006933145225048065, + -0.08638572692871094, + 0.32057398557662964, + -1.486854076385498, + -0.1531560719013214, + 0.0, + 0.14251592755317688, + 0.07826228439807892, + -0.24055713415145874, + -0.11116695404052734, + 0.200389564037323, + -0.07051187753677368, + 0.0 + ], + [ + -0.07637833058834076, + 0.6696901321411133, + 2.130587100982666, + -1.9006763696670532, + -2.6654670238494873, + 0.06729818880558014, + 2.65816593170166, + -0.0612010583281517, + -0.1226595938205719, + 0.9200000166893005, + -0.4352770149707794, + -2.266615390777588, + 0.48697614669799805, + 1.5162358283996582, + -0.014945849776268005, + -0.4459138512611389, + 1.1448073387145996, + 0.8031747937202454, + -0.6817312240600586, + -0.20262596011161804, + 1.1690938472747803 + ], + [ + 0.519248902797699, + 0.27624276280403137, + 1.911887764930725, + -1.8784401416778564, + -2.9212372303009033, + 0.7762739658355713, + 2.547428846359253, + 0.08362685143947601, + -0.1671719253063202, + 0.8596282601356506, + -0.17510351538658142, + -2.220954656600952, + 0.4918195605278015, + 1.357650876045227, + 0.3462192416191101, + 0.2476332038640976, + 0.6965686082839966, + 1.0915483236312866, + -0.6818809509277344, + -0.7465203404426575, + 0.724744439125061 + ] + ], + "predicted_scores": [ + 7.541661739349365, + 7.425927639007568, + 7.374334335327148, + 7.179238319396973, + 6.926530838012695, + 6.9085612297058105, + 6.650173664093018, + 6.590436935424805 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1991414747", + "selected_index": 0, + "state_hash": "127cf975da326f174c815484811a6acc4855bf07460a2aa5b0d728cb2a1f41c7", + "stored_base_utility": 0.120394766330719, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20275163650512695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20275163650512695 + }, + "base_progress": 0.20275163650512695, + "base_success": false, + "base_utility": 0.20275163650512695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2330203801393509, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2330203801393509 + }, + { + "contact_quality": null, + "progress": 0.15594589710235596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15594589710235596 + }, + { + "contact_quality": null, + "progress": 0.23662996292114258, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23662996292114258 + }, + { + "contact_quality": null, + "progress": 0.24087366461753845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24087366461753845 + }, + { + "contact_quality": null, + "progress": 0.22966185212135315, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22966185212135315 + }, + { + "contact_quality": null, + "progress": 0.22573722898960114, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22573722898960114 + }, + { + "contact_quality": null, + "progress": 0.2381056249141693, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2381056249141693 + }, + { + "contact_quality": null, + "progress": 0.16225522756576538, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16225522756576538 + } + ], + "candidate_progress": [ + 0.2330203801393509, + 0.15594589710235596, + 0.23662996292114258, + 0.24087366461753845, + 0.22966185212135315, + 0.22573722898960114, + 0.2381056249141693, + 0.16225522756576538 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3afd8959363e917591d4", + "generated_tangents": [ + [ + 0.032345257699489594, + 0.041534118354320526, + -0.24840888381004333, + -1.115391731262207, + 0.43810999393463135, + -0.3190232217311859, + 1.095834493637085, + 0.0745425820350647, + 0.1691260039806366, + -0.14132744073867798, + 0.06961572170257568, + -0.23097893595695496, + -2.2369840145111084, + -0.4671887159347534, + -0.019553054124116898, + -0.044790834188461304, + -0.010421991348266602, + 0.2782835066318512, + -0.2650759816169739, + -1.255564570426941, + -0.2518557012081146 + ], + [ + 0.052113763988018036, + 0.0841289609670639, + 0.16394153237342834, + -1.261644959449768, + 0.39749136567115784, + -0.4661403000354767, + 0.7681432962417603, + 0.11206910014152527, + 0.16414090991020203, + -0.08916645497083664, + 0.011096607893705368, + -0.444663941860199, + -2.5357465744018555, + -0.42860186100006104, + -0.05279140546917915, + 0.059909138828516006, + 0.09880596399307251, + 0.38401365280151367, + -0.3973008990287781, + -1.5253477096557617, + -0.2788736820220947 + ], + [ + -0.14080624282360077, + 0.12565378844738007, + -0.20969948172569275, + -1.1504992246627808, + 0.746421754360199, + -0.6103098392486572, + 1.072847843170166, + 0.1000339463353157, + 0.14912262558937073, + -0.18910285830497742, + 0.030119994655251503, + -0.42367202043533325, + -2.2265522480010986, + -0.28670209646224976, + 0.07863615453243256, + -0.1160237044095993, + 0.049855709075927734, + -0.07195411622524261, + -0.2410619854927063, + -1.5276737213134766, + -0.1106393039226532 + ], + [ + -0.13328681886196136, + 0.1056923121213913, + -0.14332684874534607, + -1.0966047048568726, + 0.6499765515327454, + -0.5430305004119873, + 0.929847240447998, + 0.06825277209281921, + 0.1386578381061554, + -0.1759842038154602, + 0.0729876309633255, + -0.282988965511322, + -2.3079235553741455, + -0.14611150324344635, + -0.03398582711815834, + -0.10941538214683533, + 0.055496156215667725, + 0.08245377987623215, + -0.1715477705001831, + -1.4508602619171143, + -0.12715402245521545 + ], + [ + -0.06993065774440765, + 0.1536913365125656, + -0.16998520493507385, + -0.8637540340423584, + 0.7124572992324829, + -0.5082904100418091, + 1.2236427068710327, + -0.03231893479824066, + 0.157712921500206, + -0.18595099449157715, + 0.22639310359954834, + 0.059861332178115845, + -2.0127789974212646, + -0.3475462794303894, + -0.1140376627445221, + -0.09047801792621613, + -0.10816304385662079, + 0.08630485087633133, + 0.33980488777160645, + -1.3979809284210205, + -0.17245376110076904 + ], + [ + -0.06523267924785614, + 0.1315969079732895, + -0.28416815400123596, + -1.009912133216858, + 0.7519988417625427, + -0.5380936861038208, + 0.3884163498878479, + -0.014463379979133606, + 0.12786871194839478, + -0.10217460244894028, + -0.056670717895030975, + -0.13899099826812744, + -2.1328392028808594, + 0.08389313519001007, + 0.08188736438751221, + -0.07830917835235596, + -0.04422105848789215, + 0.14191380143165588, + -0.10145944356918335, + -1.582294225692749, + 0.05496809631586075 + ], + [ + -0.15150649845600128, + 0.17804746329784393, + -0.07315006852149963, + -1.1448009014129639, + 0.7629889249801636, + -0.5661367177963257, + 0.521133303642273, + 0.08745068311691284, + 0.1308426409959793, + -0.19754517078399658, + 0.1423840969800949, + -0.17677173018455505, + -2.2040855884552, + 0.31156861782073975, + 0.03675226494669914, + -0.13841646909713745, + 0.06883640587329865, + 0.3033646047115326, + -0.09109467267990112, + -1.582838535308838, + 0.06170547008514404 + ], + [ + 0.18227604031562805, + 0.12305095791816711, + 0.29672008752822876, + -1.265738606452942, + 0.3126491904258728, + -1.3182971477508545, + 1.3083362579345703, + 0.07891415059566498, + 0.13188202679157257, + -0.1381712555885315, + 0.18966078758239746, + -0.22495943307876587, + -1.6865249872207642, + -0.4436098337173462, + -0.06243528053164482, + -0.2181418091058731, + 0.18943697214126587, + 0.13939020037651062, + -0.350314199924469, + -1.6874771118164062, + 0.05776550620794296 + ] + ], + "generated_utilities": [ + 0.2330203801393509, + 0.15594589710235596, + 0.23662996292114258, + 0.24087366461753845, + 0.22966185212135315, + 0.22573722898960114, + 0.2381056249141693, + 0.16225522756576538 + ], + "group_id": "ms-3afd8959363e917591d4", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8582140207290649, + 2.0, + 0.18031787872314453, + 0.07887399196624756, + 0.5568947792053223, + 0.16715207695960999, + 0.5635193586349487, + 0.18913890421390533, + 0.5682768225669861, + 0.10102970898151398, + 0.09163376688957214, + 0.12909311056137085, + 0.12299326062202454, + 0.09030237793922424, + 0.13531166315078735, + 0.06874948740005493 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.15699657797813416, + 0.6247961521148682, + 0.107974112033844, + -0.3284463882446289, + 0.22832846641540527, + -0.12728703022003174, + 0.0, + -0.2739208936691284, + -0.051639966666698456, + -0.2660524249076843, + -0.2539072632789612, + 0.09800386428833008, + -1.84090256690979, + 0.0, + -0.017676634714007378, + -0.018830953165888786, + 0.1583891659975052, + -0.07512116432189941, + -0.01770937442779541, + -1.5237011909484863, + 0.0 + ], + [ + 0.4675478935241699, + 0.6247961521148682, + 0.3345896601676941, + -0.5404787063598633, + -0.06834304332733154, + -0.12728703022003174, + 0.0, + -0.14999517798423767, + -0.2730908691883087, + 0.08837036788463593, + 0.45600926876068115, + 0.410434365272522, + -1.84090256690979, + 0.0, + 0.4969322681427002, + 0.22034725546836853, + 0.2921968400478363, + 0.5872196555137634, + 0.19443652033805847, + -1.5237011909484863, + 0.0 + ], + [ + 0.41189882159233093, + 1.6247961521148682, + 0.12157746404409409, + -1.328446388244629, + 0.9316569566726685, + -1.1272870302200317, + 1.276085376739502, + -0.029595039784908295, + -0.014941371977329254, + 0.004283940885215998, + 0.6188332438468933, + -0.49464237689971924, + -2.84090256690979, + -1.8046140670776367, + 0.0007966384291648865, + -0.006994657218456268, + 0.014263113960623741, + 0.6869955658912659, + -0.2918644845485687, + -2.5237011909484863, + -1.7481608390808105 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.552170753479004, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.6092281341552734, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.496321678161621 + ], + [ + 0.44720953702926636, + 1.4417266845703125, + 0.4182493984699249, + -1.547792911529541, + 0.990667462348938, + -0.9114025831222534, + 1.535681962966919, + 0.21791213750839233, + -0.30513259768486023, + 0.013669610023498535, + 0.7976462841033936, + -0.6395556926727295, + -2.5809314250946045, + -1.571488618850708, + 0.038753002882003784, + 0.04824339970946312, + -0.11262937635183334, + 0.9187417030334473, + -0.1308273822069168, + -2.59043288230896, + -1.9625922441482544 + ], + [ + 0.5162960290908813, + 1.2868248224258423, + 0.2058304250240326, + -1.4812198877334595, + 1.085485816001892, + -1.05157470703125, + 1.1480523347854614, + 0.2796477973461151, + -0.1532609462738037, + 0.12988577783107758, + 0.8581418395042419, + -0.6760798692703247, + -2.769819736480713, + -2.192568778991699, + 0.10523252189159393, + 0.05066283047199249, + -0.08061028271913528, + 0.7438116669654846, + -0.07841359078884125, + -2.3079848289489746, + -1.4200246334075928 + ], + [ + 0.4112767279148102, + 1.8745536804199219, + 0.5301340222358704, + -0.9917724132537842, + 0.8905197978019714, + -0.8391705751419067, + 1.3120794296264648, + 0.3723032474517822, + -0.19477176666259766, + -0.35689839720726013, + 0.2843710780143738, + -0.45066511631011963, + -2.812030792236328, + -1.588818073272705, + -0.09205310046672821, + -0.5067525506019592, + -0.41988417506217957, + 1.0423402786254883, + 0.1759319305419922, + -2.945967197418213, + -2.05134916305542 + ], + [ + 0.46365559101104736, + 1.105017900466919, + 0.466808021068573, + -1.5382812023162842, + 0.8287478089332581, + -1.4617033004760742, + 0.8807560205459595, + 0.2107943296432495, + 0.2954367995262146, + 0.45928943157196045, + 0.6575880646705627, + -0.5414218902587891, + -2.711836099624634, + -1.6827114820480347, + 0.3175691068172455, + -0.38371264934539795, + 0.34041139483451843, + 0.34132614731788635, + -0.8783347606658936, + -2.133782148361206, + -1.3974260091781616 + ], + [ + 0.13586324453353882, + 1.6158640384674072, + 0.24489957094192505, + -1.4942185878753662, + 1.1602565050125122, + -1.648895502090454, + 1.1836220026016235, + 0.17831489443778992, + -0.5400142669677734, + -0.42907604575157166, + 0.5951722264289856, + -0.7885013818740845, + -3.2272911071777344, + -2.208972692489624, + -0.4857475459575653, + 0.5623884201049805, + 0.02566027268767357, + 0.590702474117279, + -0.6581307649612427, + -2.7863473892211914, + -2.031818151473999 + ], + [ + 0.21801596879959106, + 1.0595136880874634, + 0.9040011763572693, + -1.1076470613479614, + 1.2744677066802979, + -0.7150566577911377, + 1.0042598247528076, + 0.5841938257217407, + -0.2832033336162567, + -0.45343834161758423, + -0.17037296295166016, + -0.5152367353439331, + -2.1963798999786377, + -1.5652427673339844, + 0.03223085403442383, + 0.07496664673089981, + -0.1515023559331894, + 1.233866572380066, + 0.4301607310771942, + -2.489328384399414, + -1.8620245456695557 + ], + [ + -0.15804454684257507, + 2.2168471813201904, + 0.9374236464500427, + -1.8158314228057861, + 1.114590048789978, + -0.9279311895370483, + 1.890993356704712, + -0.8653054237365723, + -0.5551189184188843, + 0.4295695126056671, + 1.1614964008331299, + -0.5840407609939575, + -2.900725841522217, + -2.579883575439453, + -0.1900041103363037, + 0.42368441820144653, + 0.38376376032829285, + 0.9654008150100708, + 0.10608989000320435, + -3.332181453704834, + -1.8879828453063965 + ], + [ + 1.0901068449020386, + 2.576328754425049, + 0.5122992396354675, + -1.9627916812896729, + 0.026246964931488037, + -0.3622438311576843, + 0.9286589622497559, + 0.644849956035614, + 0.9211093187332153, + 0.3528344929218292, + 0.3661169707775116, + 0.20941221714019775, + -2.582620143890381, + -1.6867890357971191, + 0.09869468957185745, + -0.8795469403266907, + -0.2239246666431427, + 0.5206944942474365, + -0.20716872811317444, + -2.8072235584259033, + -2.3385443687438965 + ], + [ + -0.0005125105381011963, + 1.8920408487319946, + 1.0018010139465332, + -2.2556979656219482, + 0.7338991165161133, + -0.4559611678123474, + 1.027335524559021, + -1.0165901184082031, + -0.14792552590370178, + 0.11212217807769775, + 0.8827763795852661, + 0.1683982014656067, + -3.2136244773864746, + -2.4300284385681152, + -0.9183067083358765, + 0.10752613097429276, + -0.2955099046230316, + 0.31867098808288574, + 0.06515705585479736, + -3.416110038757324, + -1.5137704610824585 + ], + [ + -0.39508435130119324, + 2.55599308013916, + 0.128604456782341, + -1.8821169137954712, + 0.6769157648086548, + -0.7477259635925293, + 1.6593644618988037, + -0.005859369412064552, + 0.22320973873138428, + -0.2906110882759094, + 0.22343486547470093, + -1.488371729850769, + -2.81829833984375, + -0.939697265625, + 0.9867495894432068, + 0.2955887019634247, + -0.11908379942178726, + 0.473803848028183, + -0.11059392988681793, + -3.471217155456543, + -2.3480191230773926 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.018576741218566895, + 0.6247961521148682, + -0.017397306859493256, + -0.3284463882446289, + 0.044771432876586914, + -0.12728703022003174, + 0.0, + -0.005512619391083717, + 0.03525955602526665, + -0.005881692748516798, + 0.04493492841720581, + 0.056808531284332275, + -1.84090256690979, + 0.0, + 0.11075251549482346, + -0.05448189005255699, + -0.041690655052661896, + 0.010008454322814941, + 0.056893616914749146, + -1.5237011909484863, + 0.0 + ] + ], + "predicted_scores": [ + 2.6485273838043213, + 2.5635921955108643, + 1.288849949836731, + 1.2780762910842896, + 1.010661244392395, + 0.6195508241653442, + 0.10887739062309265, + -0.662818193435669 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "75070641", + "selected_index": 0, + "state_hash": "bd1bbc057450972b9fa6e08a54ebd8c6d82d09997dad16cecf9584632b705b7b", + "stored_base_utility": 0.8582140207290649, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8343751430511475, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8343751430511475 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8134292960166931, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8134292960166931 + }, + { + "contact_quality": null, + "progress": 0.8216007947921753, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8216007947921753 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1820054054260254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1820054054260254 + }, + { + "contact_quality": null, + "progress": 0.8494137525558472, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8494137525558472 + } + ], + "candidate_progress": [ + 1.0, + 0.8343751430511475, + 1.0, + 0.8134292960166931, + 0.8216007947921753, + 1.0, + 0.1820054054260254, + 0.8494137525558472 + ], + "candidate_source_chart_ids": [ + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-0949097d19e597fc7f63", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + true, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-3ff1aedb72f65a182b58", + "generated_tangents": [ + [ + -0.062439724802970886, + -0.10267174243927002, + -0.08697965741157532, + -0.19292812049388885, + 0.04958716034889221, + -1.9831255674362183, + 0.0265466570854187, + 0.09313465654850006, + 0.17003506422042847, + -0.06069370359182358, + -0.008215725421905518, + -0.1732316017150879, + -1.980164885520935, + -0.14224843680858612, + -0.05720065161585808, + -0.04062411189079285, + 0.02359837293624878, + 0.06030084192752838, + -0.06397882103919983, + -1.6451139450073242, + -0.21163910627365112 + ], + [ + 0.03654243052005768, + -0.08168097585439682, + -0.04141226410865784, + -0.14411795139312744, + 0.12869243323802948, + -2.002037525177002, + 0.007157176733016968, + 0.06532534956932068, + 0.17071294784545898, + -0.03254286199808121, + -0.017103321850299835, + -0.14140141010284424, + -1.9333590269088745, + -0.06879577040672302, + -0.021911438554525375, + -0.07094591110944748, + 0.03496462106704712, + 0.07464239746332169, + -0.0960855782032013, + -1.587310552597046, + -0.1497659981250763 + ], + [ + -0.08239680528640747, + -0.16734471917152405, + 0.009770750999450684, + -0.19258294999599457, + 0.15593227744102478, + -1.92607581615448, + 0.13474443554878235, + 0.07099077105522156, + 0.22003060579299927, + 0.013289451599121094, + 0.038875825703144073, + -0.1812708079814911, + -1.9331516027450562, + -0.14263375103473663, + -0.11658428609371185, + -0.09669330716133118, + -0.08281369507312775, + 0.034088134765625, + -0.03991749882698059, + -1.654839038848877, + -0.25345224142074585 + ], + [ + -0.11683295667171478, + -0.1470959484577179, + -0.015281647443771362, + -0.09597066044807434, + 0.03893783688545227, + -1.9797533750534058, + -0.15978527069091797, + 0.010778605937957764, + 0.14734525978565216, + -0.09725389629602432, + -0.2448519766330719, + -0.14496874809265137, + -1.9619742631912231, + -0.06363658607006073, + -0.09696879982948303, + -0.06293077766895294, + 0.02727481722831726, + -0.05131717026233673, + -0.3007820248603821, + -1.738222360610962, + -0.42688825726509094 + ], + [ + -0.12231095135211945, + -0.04192669689655304, + 0.03312569856643677, + -0.2702970802783966, + 0.24234843254089355, + -1.9157201051712036, + 0.19673866033554077, + 0.009997710585594177, + 0.15686525404453278, + -0.06183413416147232, + -0.09602666646242142, + -0.1328202784061432, + -1.9324674606323242, + -0.2542661428451538, + -0.05770087614655495, + -0.06654544174671173, + -0.0003515481948852539, + 0.004999503493309021, + -0.03302639722824097, + -1.501014232635498, + -0.4634285271167755 + ], + [ + -0.2972021698951721, + -0.10989681631326675, + 0.01714882254600525, + -0.004400297999382019, + -0.022506117820739746, + -1.9709454774856567, + 0.08642077445983887, + 0.223464697599411, + 0.15293675661087036, + -0.1213032677769661, + 0.06315916776657104, + -0.3256334662437439, + -2.0076959133148193, + -0.11435651779174805, + -0.04275403544306755, + 0.03250744193792343, + 0.10640215128660202, + -0.08467166125774384, + -0.0460490882396698, + -1.8005802631378174, + -0.41277238726615906 + ], + [ + -0.32271140813827515, + -0.19143876433372498, + -0.14872053265571594, + -0.08058717846870422, + -0.2637145221233368, + -1.8081475496292114, + -0.21086162328720093, + 0.23122139275074005, + 0.10887749493122101, + -0.06519048660993576, + -0.1338232159614563, + -0.4990749955177307, + -2.027585029602051, + -0.10016399621963501, + 0.12119326740503311, + -0.04883003234863281, + -0.0022823214530944824, + -0.14061720669269562, + -0.3593866229057312, + -1.7351727485656738, + -0.5648698806762695 + ], + [ + -0.1143321543931961, + -0.21972617506980896, + -0.23139998316764832, + -0.20535056293010712, + 0.11902617663145065, + -1.966908574104309, + -0.08657747507095337, + -0.27134639024734497, + 0.17673714458942413, + -0.24420922994613647, + -0.014107845723628998, + -0.5075947642326355, + -2.299177646636963, + -0.0649813711643219, + 0.07382363826036453, + -0.05025056004524231, + 0.11171112954616547, + -0.0808776468038559, + -0.43903303146362305, + -1.7131781578063965, + -0.6973515748977661 + ] + ], + "generated_utilities": [ + 2.0, + 0.8343751430511475, + 2.0, + 0.8134292960166931, + 0.8216007947921753, + 2.0, + 0.1820054054260254, + 0.8494137525558472 + ], + "group_id": "ms-3ff1aedb72f65a182b58", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.851725161075592, + 2.0, + 0.8126396536827087, + 2.0, + 0.7855525612831116, + 0.7821746468544006, + 0.6115735173225403, + 0.5732660889625549, + 0.7798006534576416, + 0.09109212458133698, + 0.8456447124481201, + 0.16636842489242554, + 0.0955686867237091, + 0.0822453498840332, + 0.0018809139728546143, + 0.08579078316688538 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.09319563210010529, + 0.18433167040348053, + 0.03857926279306412, + 0.20899039506912231, + -0.19732166826725006, + -2.7716264724731445, + -0.9089346528053284, + -0.012062903493642807, + -0.001710914308205247, + 0.02447224222123623, + 0.6588084697723389, + -0.43759873509407043, + -2.7224881649017334, + -1.560755968093872, + -0.002114389091730118, + -0.004199527204036713, + 0.015975257381796837, + 0.7779304385185242, + -0.31706690788269043, + -2.4826252460479736, + -1.5341289043426514 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.8178693056106567, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.121511936187744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0682578086853027 + ], + [ + -0.10748189687728882, + 0.29221633076667786, + 0.12325333058834076, + 0.21938443183898926, + 0.017790749669075012, + -2.485055446624756, + -1.1110891103744507, + 0.062413301318883896, + -0.2567306160926819, + -0.15453597903251648, + 0.549060583114624, + -0.18322163820266724, + -2.5428390502929688, + -1.8537542819976807, + -0.07324761152267456, + 0.28765758872032166, + 0.16924789547920227, + 0.6205734610557556, + -0.4268453121185303, + -2.6385459899902344, + -1.5760513544082642 + ], + [ + 0.44683152437210083, + 0.3713763356208801, + 0.1084144338965416, + 0.2590586543083191, + 0.1997859627008438, + -2.787759780883789, + -1.1921976804733276, + 0.33878061175346375, + 0.019987009465694427, + 0.021556159481406212, + 0.36814823746681213, + -0.2135106772184372, + -2.416769027709961, + -1.3776189088821411, + -0.3500843346118927, + -0.26303720474243164, + 0.10322260856628418, + 0.9827795624732971, + -0.3747403025627136, + -2.6419029235839844, + -1.4116339683532715 + ], + [ + -0.2883707880973816, + 0.02873578667640686, + -0.11175497621297836, + -0.106661856174469, + 0.03059367835521698, + -2.3899149894714355, + -1.0789995193481445, + 0.2610729932785034, + 0.4800889790058136, + -0.45478329062461853, + 1.0867195129394531, + -0.7417076826095581, + -3.09529185295105, + -1.730720043182373, + -0.33508142828941345, + 0.10841497033834457, + -0.38902348279953003, + 0.44348275661468506, + 0.0854053795337677, + -2.3334603309631348, + -1.7851121425628662 + ], + [ + 0.16367590427398682, + 0.05515189468860626, + -0.3197028636932373, + 0.5501840114593506, + -0.3621746301651001, + -3.0582237243652344, + -1.135798454284668, + -0.15054848790168762, + -0.5578444004058838, + 0.3691062331199646, + 0.9388304352760315, + -0.19103051722049713, + -2.7536559104919434, + -2.093773603439331, + -0.037153881043195724, + 0.19014233350753784, + -0.36205345392227173, + 0.24469107389450073, + -0.5916419625282288, + -2.7417092323303223, + -1.637229323387146 + ], + [ + -0.3646707534790039, + 0.6696967482566833, + -0.44303250312805176, + -0.4559982419013977, + -0.6542845368385315, + -2.9630653858184814, + -1.4721050262451172, + 0.5010843873023987, + 0.5162685513496399, + 0.42702627182006836, + 0.6571387648582458, + -0.08207055926322937, + -3.278073310852051, + -1.7742136716842651, + -0.6448181867599487, + -0.45361432433128357, + 0.3905653953552246, + 0.7770582437515259, + -0.19482973217964172, + -2.094736337661743, + -1.083003282546997 + ], + [ + 0.9472348093986511, + 0.6904511451721191, + -0.2936761677265167, + -0.34556156396865845, + -0.7593099474906921, + -1.9670193195343018, + -0.7599589824676514, + 0.04467743635177612, + 0.6557444334030151, + 0.2765021026134491, + 0.042136549949645996, + -0.37541040778160095, + -3.161308765411377, + -0.8803613185882568, + 0.38967010378837585, + 0.4423010051250458, + 0.019825218245387077, + 0.9605399370193481, + -0.874176561832428, + -3.0791730880737305, + -1.3198856115341187 + ], + [ + 0.8605663180351257, + 0.04459291696548462, + -0.8886744976043701, + -0.38333380222320557, + -1.0364235639572144, + -3.0104973316192627, + -0.889693021774292, + -0.08674328029155731, + 0.20159615576267242, + 0.1069621816277504, + 1.4727201461791992, + 0.09883639216423035, + -3.129849433898926, + -0.8773520588874817, + -0.2628254294395447, + -0.3415844738483429, + -0.35313117504119873, + 0.3155616521835327, + -0.556646466255188, + -2.0992307662963867, + -0.7895610928535461 + ], + [ + 0.39962780475616455, + 0.3406272530555725, + 0.05363208055496216, + -0.06018832325935364, + -0.5703235268592834, + -2.5974044799804688, + -0.6783060431480408, + 0.7815009355545044, + 0.35641810297966003, + -0.00838596560060978, + 0.6228271126747131, + 0.25608590245246887, + -2.5914058685302734, + -0.8502807021141052, + 0.726007878780365, + -0.6774719953536987, + 0.6261520981788635, + 1.149846076965332, + 0.6782730221748352, + -3.391972541809082, + -0.7993651032447815 + ], + [ + -0.29702335596084595, + 0.6166410446166992, + 0.6319320201873779, + -0.6542804837226868, + -0.8892693519592285, + -1.8599720001220703, + -1.3999924659729004, + -0.3017314672470093, + 0.9357973337173462, + -0.5596981048583984, + 1.3143360614776611, + -0.30643409490585327, + -2.028008460998535, + -1.3395459651947021, + 0.6212120056152344, + 0.6739580631256104, + -0.5840877294540405, + -0.06664055585861206, + -0.23446696996688843, + -2.0225906372070312, + -2.438906669616699 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0649428591132164, + 0.09363124519586563, + -0.014398600906133652, + -0.012044146656990051, + 0.03117428719997406, + -1.7716264724731445, + 0.0, + 0.05836518481373787, + 0.012665675021708012, + -0.013142907992005348, + -0.06252282857894897, + -0.06281104683876038, + -1.7224881649017334, + 0.0, + -0.06448972970247269, + -0.03696601092815399, + 0.030633779242634773, + -0.0031400322914123535, + 0.0327644944190979, + -1.4826252460479736, + 0.0 + ], + [ + -0.04005520045757294, + 0.2738151252269745, + -0.05346817523241043, + 0.20275527238845825, + 0.10101960599422455, + -1.7716264724731445, + 0.0, + -0.36066195368766785, + -0.21821273863315582, + 0.22022001445293427, + -0.0596311092376709, + -0.17951729893684387, + -1.7224881649017334, + 0.0, + 0.3205486238002777, + 0.25526633858680725, + 0.17392843961715698, + 0.5170043706893921, + 0.43821561336517334, + -1.4826252460479736, + 0.0 + ] + ], + "predicted_scores": [ + 3.6719894409179688, + 3.591271162033081, + 3.2816109657287598, + 2.9038634300231934, + 2.802067756652832, + 2.5342326164245605, + 1.3963682651519775, + 0.9829980134963989 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055567618", + "selected_index": 0, + "state_hash": "027cc529c99c00db47a01ecfc12ab432557fbd911e8ff3c8a56adf2d3404549f", + "stored_base_utility": 0.851725161075592, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19648407399654388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19648407399654388 + }, + "base_progress": 0.19648407399654388, + "base_success": false, + "base_utility": 0.19648407399654388, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.16174951195716858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16174951195716858 + }, + { + "contact_quality": null, + "progress": 0.5578349828720093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5578349828720093 + }, + { + "contact_quality": null, + "progress": 0.5522037148475647, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5522037148475647 + }, + { + "contact_quality": null, + "progress": 0.5545214414596558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5545214414596558 + }, + { + "contact_quality": null, + "progress": 0.5589519739151001, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5589519739151001 + }, + { + "contact_quality": null, + "progress": 0.23065254092216492, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23065254092216492 + }, + { + "contact_quality": null, + "progress": 0.812659502029419, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.812659502029419 + }, + { + "contact_quality": null, + "progress": 0.12922577559947968, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12922577559947968 + } + ], + "candidate_progress": [ + 0.16174951195716858, + 0.5578349828720093, + 0.5522037148475647, + 0.5545214414596558, + 0.5589519739151001, + 0.23065254092216492, + 0.812659502029419, + 0.12922577559947968 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4048f8f6e047422e3c79", + "generated_tangents": [ + [ + 0.11197315156459808, + 0.006205298006534576, + 0.14926928281784058, + -0.7924947738647461, + 0.2881510853767395, + -0.07056090235710144, + 0.8740236759185791, + 0.11901829391717911, + 0.17200277745723724, + -0.11694551259279251, + -0.08643137663602829, + -0.4421881437301636, + -2.138794422149658, + -0.139741912484169, + 0.003947321325540543, + 0.0577222965657711, + 0.09705755114555359, + 0.5465931296348572, + -0.4082576036453247, + -1.3069524765014648, + -0.2806330621242523 + ], + [ + 0.07726870477199554, + -0.07570934295654297, + -0.20511028170585632, + -0.6203000545501709, + 0.45177608728408813, + 0.07481899857521057, + 0.9583245515823364, + 0.1163330152630806, + 0.17529936134815216, + -0.16057321429252625, + -0.03535432368516922, + -0.08222764730453491, + -1.8852554559707642, + -0.3469465970993042, + 0.053696177899837494, + -0.024394530802965164, + -0.046366795897483826, + 0.426835834980011, + -0.2268209457397461, + -1.2934815883636475, + -0.207806795835495 + ], + [ + -0.08031533658504486, + 0.01641693338751793, + -0.2207373082637787, + -0.5965136885643005, + 0.6481350660324097, + -0.04586876928806305, + 0.9781064987182617, + 0.04301615059375763, + 0.15092270076274872, + -0.1742050051689148, + -0.006424359977245331, + -0.3280750513076782, + -1.7509621381759644, + -0.025145605206489563, + 0.09848727285861969, + -0.07737380266189575, + 0.00167161226272583, + 0.11688224971294403, + -0.29587846994400024, + -1.4826886653900146, + -0.18090125918388367 + ], + [ + -0.05299849808216095, + 0.05311792343854904, + -0.26579973101615906, + -0.5007749795913696, + 0.6133293509483337, + -0.04339100420475006, + 0.5321841239929199, + -0.03950536251068115, + 0.08820459991693497, + -0.03519446402788162, + -0.07128775864839554, + -0.1457923948764801, + -1.879436731338501, + 0.19897763431072235, + 0.05538950860500336, + -0.04888976365327835, + -0.007486552000045776, + 0.2481449842453003, + -0.04760858416557312, + -1.4677531719207764, + -0.024621397256851196 + ], + [ + -0.09349143505096436, + 0.004638437181711197, + -0.14963886141777039, + -0.5424951314926147, + 0.5620851516723633, + -0.10637667775154114, + 0.8044445514678955, + 0.0581660270690918, + 0.15918996930122375, + -0.16691237688064575, + 0.020559489727020264, + -0.15404486656188965, + -1.784014344215393, + 0.04818182438611984, + 0.03167671337723732, + -0.10784728825092316, + -0.032836392521858215, + 0.2474530190229416, + -0.20835059881210327, + -1.3472936153411865, + -0.16309085488319397 + ], + [ + -0.06473344564437866, + 0.028126433491706848, + -0.1466488540172577, + -0.3547529876232147, + 0.5873199701309204, + -0.0310448557138443, + 1.0894575119018555, + -0.025085359811782837, + 0.17570744454860687, + -0.19183149933815002, + 0.13417312502861023, + 0.1717030555009842, + -1.63825261592865, + -0.21345572173595428, + -0.019779633730649948, + -0.10790978372097015, + -0.1684931069612503, + 0.27209532260894775, + 0.2623617649078369, + -1.3589808940887451, + -0.17549178004264832 + ], + [ + -0.07936891913414001, + 0.07177864760160446, + 0.0379214882850647, + -0.49842533469200134, + 0.6703141927719116, + -0.004151239991188049, + 0.5484176874160767, + 0.05874086916446686, + 0.0968005433678627, + -0.14093080163002014, + 0.030639145523309708, + -0.1879361867904663, + -1.7977654933929443, + 0.4428902268409729, + 0.013650596141815186, + -0.1640920341014862, + 0.07403738796710968, + 0.35591626167297363, + -0.08696019649505615, + -1.4611952304840088, + -0.044204503297805786 + ], + [ + 0.25739431381225586, + 0.022951796650886536, + 0.2976742386817932, + -0.8237043619155884, + 0.2583261728286743, + -0.7832725048065186, + 1.4842185974121094, + 0.10086319595575333, + 0.11956450343132019, + -0.06298255175352097, + 0.12854579091072083, + -0.29761233925819397, + -1.3621857166290283, + -0.11717723309993744, + 0.00656508281826973, + -0.2573537826538086, + 0.13421666622161865, + 0.31478530168533325, + -0.2815800905227661, + -1.6235404014587402, + 0.0009451210498809814 + ] + ], + "generated_utilities": [ + 0.16174951195716858, + 0.5578349828720093, + 0.5522037148475647, + 0.5545214414596558, + 0.5589519739151001, + 0.23065254092216492, + 0.812659502029419, + 0.12922577559947968 + ], + "group_id": "ms-4048f8f6e047422e3c79", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558006286621094, + 2.0, + 0.1986379623413086, + 0.03769078850746155, + 0.5361635684967041, + 0.18274834752082825, + 0.17300468683242798, + 0.12688571214675903, + 0.13836368918418884, + 0.07454231381416321, + 0.0899321436882019, + 0.10402637720108032, + 0.1711854636669159, + 0.17255032062530518, + 0.039613887667655945, + 0.10826331377029419 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1422826200723648, + 0.21033799648284912, + 0.1978604793548584, + 0.059162259101867676, + -0.2529524564743042, + -0.15161612629890442, + 0.0, + -0.2264404594898224, + -0.2209387719631195, + -0.025619791820645332, + 0.06467145681381226, + 0.07422274351119995, + -1.7048513889312744, + 0.0, + 0.15056702494621277, + -0.06247536092996597, + -0.1033044084906578, + 0.1334916353225708, + 0.2784581184387207, + -1.4811434745788574, + 0.0 + ], + [ + -0.44999396800994873, + 0.271686851978302, + -0.4267011880874634, + 0.2730966806411743, + -0.01798880100250244, + -0.04680868983268738, + 0.0, + -0.09472770243883133, + -0.06993720680475235, + -0.06752754002809525, + -0.13884133100509644, + 0.24737784266471863, + -1.7048513889312744, + 0.0, + -0.09056920558214188, + 0.4132460653781891, + 0.04097161442041397, + -0.25072288513183594, + 0.49742037057876587, + -1.4811434745788574, + 0.0 + ], + [ + 0.17392832040786743, + 1.2103379964828491, + -0.3708428144454956, + -0.36093437671661377, + 0.744896650314331, + 0.23929360508918762, + 1.4388959407806396, + -0.04149632900953293, + 0.015302501618862152, + 0.023577755317091942, + 0.5954978466033936, + -0.3274606168270111, + -2.7048513889312744, + -1.646976351737976, + 0.0006552413105964661, + -0.008917607367038727, + 0.022643378004431725, + 0.7492771148681641, + -0.26030659675598145, + -2.4811434745788574, + -1.684959888458252 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8777918815612793, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.293952703475952, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.369919776916504 + ], + [ + 0.1792769730091095, + 0.9774190187454224, + -0.5587559342384338, + -0.5885956883430481, + 0.9904338717460632, + 0.41647347807884216, + 1.4140102863311768, + -0.2462461292743683, + -0.1271664798259735, + -0.2736872732639313, + 0.8877567052841187, + -0.6213628053665161, + -2.937077283859253, + -1.3732452392578125, + -0.0015218837652355433, + -0.27707284688949585, + -0.16698139905929565, + 0.4978373646736145, + -0.479316771030426, + -2.7234396934509277, + -1.6759265661239624 + ], + [ + 0.1658722460269928, + 1.3070863485336304, + -0.4893108606338501, + -0.04209986329078674, + 0.7449368238449097, + 0.3420809805393219, + 1.4472918510437012, + 0.08132945746183395, + 0.13181470334529877, + 0.22494959831237793, + 0.22593030333518982, + -0.49233078956604004, + -2.7544617652893066, + -1.7597702741622925, + -0.365922212600708, + -0.17204320430755615, + -0.1821347028017044, + 0.6334611177444458, + -0.33006516098976135, + -2.2188992500305176, + -1.5452594757080078 + ], + [ + 0.18608316779136658, + 1.23563551902771, + -0.5727517604827881, + 0.10049733519554138, + 0.9073957204818726, + 0.5650919675827026, + 1.8946568965911865, + -0.17719107866287231, + 0.34025683999061584, + 0.44525983929634094, + 0.6176816821098328, + -0.2832425534725189, + -3.1802117824554443, + -1.8703548908233643, + 0.07433810085058212, + 0.1340292990207672, + 0.497944176197052, + 1.0071699619293213, + -0.44033437967300415, + -2.7992849349975586, + -1.8935935497283936 + ], + [ + 0.40270811319351196, + 1.5102664232254028, + -0.8070739507675171, + -0.16390679776668549, + 0.3215266466140747, + -0.16522419452667236, + 1.0900052785873413, + 0.07780961692333221, + 0.5627153515815735, + 0.5386761426925659, + 0.2949375510215759, + 0.059314221143722534, + -3.2417359352111816, + -1.899662971496582, + 0.4160631597042084, + 0.44336366653442383, + 0.47819244861602783, + 1.162381649017334, + -0.7831735014915466, + -2.625349521636963, + -1.2278814315795898 + ], + [ + 0.8529313802719116, + 1.5106499195098877, + 0.19478559494018555, + -0.6338159441947937, + 0.7529481053352356, + -0.24222904443740845, + 0.9723314046859741, + -0.14078867435455322, + -0.5563283562660217, + 0.5440018177032471, + 0.004017293453216553, + 0.019366413354873657, + -3.1521430015563965, + -2.0841636657714844, + -0.2760852575302124, + -0.5531485676765442, + 0.22684869170188904, + 1.100138545036316, + 0.046634942293167114, + -2.429711103439331, + -2.280839443206787 + ], + [ + -0.4760962128639221, + 1.8579951524734497, + 0.2322864532470703, + 0.19953304529190063, + 0.48002323508262634, + 0.7484984397888184, + 2.073361396789551, + -0.04567806422710419, + 0.04004429653286934, + -0.5635936260223389, + 0.5135918259620667, + 0.3908633887767792, + -1.9335949420928955, + -2.442507743835449, + -0.6620182394981384, + 0.4175874888896942, + 0.7711085677146912, + 1.2790143489837646, + -0.5464571714401245, + -3.2030673027038574, + -2.3768434524536133 + ], + [ + 1.0231709480285645, + 1.1804075241088867, + 0.11513572931289673, + -1.23623526096344, + 1.4848686456680298, + 1.1120561361312866, + 0.9504602551460266, + 0.8234015703201294, + -0.04407603666186333, + -0.37310001254081726, + 0.24512693285942078, + 0.4242170751094818, + -2.4985997676849365, + -2.1206297874450684, + 0.5509956479072571, + 0.8686066269874573, + -0.5043656229972839, + 1.3957195281982422, + -0.07662208378314972, + -2.371795654296875, + -2.2404346466064453 + ], + [ + 0.49735015630722046, + 0.3600291609764099, + 0.3103446364402771, + 0.6051294803619385, + -0.15457379817962646, + -0.5570405721664429, + 1.2383109331130981, + 0.9059417247772217, + 0.7879754900932312, + -0.31492647528648376, + 1.5871094465255737, + -0.8906484842300415, + -1.8698136806488037, + -2.401073694229126, + 0.3560212254524231, + -0.6580788493156433, + 0.9037240147590637, + 1.614617109298706, + 0.3788090944290161, + -1.6909503936767578, + -0.8765208125114441 + ], + [ + 0.10883396118879318, + 1.8993752002716064, + -0.7673929333686829, + -0.0936923623085022, + -0.15508884191513062, + -0.1517968773841858, + 0.465425968170166, + 0.43258219957351685, + 0.46641960740089417, + -0.3528187572956085, + 1.09894597530365, + -0.6569450497627258, + -2.025486946105957, + -1.2820806503295898, + -0.21141839027404785, + -0.9515489339828491, + 0.7230286002159119, + -0.21062928438186646, + -0.32822558283805847, + -3.289642333984375, + -1.7125959396362305 + ], + [ + 0.37358880043029785, + 2.052347183227539, + -1.212618112564087, + -0.6321570873260498, + 0.7031511068344116, + -0.3477393090724945, + 1.2687753438949585, + 0.8023800253868103, + -0.031013749539852142, + 0.35525181889533997, + 0.5262036323547363, + -0.9288194179534912, + -1.7686026096343994, + -2.128396987915039, + -0.7001423239707947, + -0.12872236967086792, + -0.012955078855156898, + 1.0237239599227905, + 0.03636348247528076, + -2.0397119522094727, + -0.9761244058609009 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.060203760862350464, + 0.21033799648284912, + 0.0356404185295105, + -0.03044089674949646, + 0.024639129638671875, + -0.032671064138412476, + 0.0, + 0.016544528305530548, + 0.04633430764079094, + 0.005854343995451927, + -0.014876246452331543, + 0.0028293728828430176, + -1.7048513889312744, + 0.0, + -0.05953434109687805, + -0.04181046783924103, + -0.0018185675144195557, + -0.09216523170471191, + 0.019338399171829224, + -1.4811434745788574, + 0.0 + ] + ], + "predicted_scores": [ + 3.702712059020996, + 2.98641037940979, + 2.442595958709717, + 1.7719107866287231, + 1.769231915473938, + 1.3229467868804932, + 0.3650352358818054, + -0.19932720065116882 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055157354", + "selected_index": 0, + "state_hash": "ae21d2b71722381a530d3a9259d02ea6728eb8a4a84fafc40ccd653d62ad3968", + "stored_base_utility": 0.8558006286621094, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19844311475753784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844311475753784 + }, + "base_progress": 0.19844311475753784, + "base_success": false, + "base_utility": 0.19844311475753784, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.20458371937274933, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20458371937274933 + }, + { + "contact_quality": null, + "progress": 0.24029652774333954, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24029652774333954 + }, + { + "contact_quality": null, + "progress": 0.22018474340438843, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22018474340438843 + }, + { + "contact_quality": null, + "progress": 0.19370315968990326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19370315968990326 + }, + { + "contact_quality": null, + "progress": 0.2239307016134262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2239307016134262 + }, + { + "contact_quality": null, + "progress": 0.20504295825958252, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20504295825958252 + }, + { + "contact_quality": null, + "progress": 0.22744446992874146, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22744446992874146 + }, + { + "contact_quality": null, + "progress": 0.2148163914680481, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2148163914680481 + } + ], + "candidate_progress": [ + 0.20458371937274933, + 0.24029652774333954, + 0.22018474340438843, + 0.19370315968990326, + 0.2239307016134262, + 0.20504295825958252, + 0.22744446992874146, + 0.2148163914680481 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-5b26c33feed6992386b1", + "generated_tangents": [ + [ + -0.07354520261287689, + -0.1256038099527359, + -0.2062130868434906, + 0.5030757188796997, + 0.02786637842655182, + -0.8338708877563477, + 0.06525328755378723, + -0.013071492314338684, + 0.11664233356714249, + -0.05924617499113083, + 0.03110441565513611, + 0.14111444354057312, + -1.826629400253296, + -0.10948537290096283, + -0.10933530330657959, + -0.020234961062669754, + 0.06310255825519562, + 0.09728805720806122, + 0.033154457807540894, + -1.6914780139923096, + -0.2264210283756256 + ], + [ + -0.2372485250234604, + -0.20508919656276703, + -0.29013410210609436, + 0.6007887125015259, + 0.13013434410095215, + -0.689579963684082, + 0.348114550113678, + -0.03366383910179138, + 0.04903995245695114, + -0.07776796072721481, + -0.16477221250534058, + 0.09789817035198212, + -2.2579095363616943, + -0.1864905208349228, + -0.10923396050930023, + -0.12563833594322205, + 0.1276419758796692, + -0.016522720456123352, + 0.18242934346199036, + -2.0922954082489014, + -0.45239725708961487 + ], + [ + -0.20454083383083344, + -0.15289542078971863, + -0.12778088450431824, + 0.40291568636894226, + 0.024638861417770386, + -0.5686707496643066, + -0.06602460145950317, + -0.0829559862613678, + 0.04839726537466049, + -0.028551049530506134, + -0.0814475491642952, + 0.05080345273017883, + -2.0805954933166504, + -0.04724736511707306, + 0.0011115968227386475, + -0.041859667748212814, + 0.05227665603160858, + 0.14725975692272186, + -0.08195555210113525, + -1.9680421352386475, + -0.20420211553573608 + ], + [ + -0.09986400604248047, + -0.11947405338287354, + -0.08526220917701721, + 0.4571826159954071, + 0.07406111061573029, + -0.5261439085006714, + 0.0399547815322876, + -0.02421511709690094, + 0.09648419171571732, + -0.037118442356586456, + 0.055190496146678925, + -0.016269028186798096, + -2.046588659286499, + -0.06308583915233612, + -0.0685170441865921, + -0.03924035653471947, + 0.1361769139766693, + 0.08701697736978531, + -0.024386346340179443, + -1.9275929927825928, + -0.1816137433052063 + ], + [ + -0.25593799352645874, + -0.17109103500843048, + -0.4388103187084198, + 0.3898616135120392, + 0.155392125248909, + -0.4137926399707794, + 0.2990950644016266, + -0.1853753626346588, + 0.10374453663825989, + -0.06233047693967819, + -0.25568926334381104, + -0.10958299040794373, + -2.4645986557006836, + -0.13059793412685394, + 0.07097207754850388, + -0.06706021726131439, + 0.056096628308296204, + 0.03948955237865448, + -0.05719456076622009, + -2.224764585494995, + -0.3590715825557709 + ], + [ + -0.254397988319397, + -0.09702186286449432, + -0.1946195662021637, + 0.5826245546340942, + -0.06090956926345825, + -0.5958397388458252, + -0.012904822826385498, + -0.008259877562522888, + 0.0874367207288742, + 0.018531017005443573, + -0.05752967298030853, + 0.22654378414154053, + -1.7283936738967896, + -0.08846975862979889, + -0.03372752293944359, + -0.016586128622293472, + 0.0731029212474823, + 0.06927242875099182, + 0.03214584290981293, + -1.7384560108184814, + -0.271386057138443 + ], + [ + -0.5649675130844116, + -0.19363243877887726, + -0.306924432516098, + 0.6226949691772461, + 0.36562594771385193, + -0.5397567749023438, + 0.45113444328308105, + -0.16688323020935059, + 0.09275840222835541, + -0.08188348263502121, + -0.2544727623462677, + 0.08943998068571091, + -2.107145071029663, + -0.08943602442741394, + -0.046965304762125015, + -0.17391133308410645, + 0.005414009094238281, + -0.10378427803516388, + 0.016249656677246094, + -2.1992218494415283, + -0.3649754822254181 + ], + [ + -0.3835487365722656, + -0.17246809601783752, + -0.22917434573173523, + 0.48758628964424133, + 0.16649049520492554, + -0.5598640441894531, + 0.3399524688720703, + -0.1241578757762909, + 0.11243285983800888, + -0.0111432746052742, + -0.12439332157373428, + -0.014379605650901794, + -2.411860942840576, + 0.00608748197555542, + -0.0877176821231842, + -0.18000683188438416, + 0.10511918365955353, + -0.008005529642105103, + -0.0656428337097168, + -2.1491360664367676, + -0.4590318500995636 + ] + ], + "generated_utilities": [ + 0.20458371937274933, + 0.24029652774333954, + 0.22018474340438843, + 0.19370315968990326, + 0.2239307016134262, + 0.20504295825958252, + 0.22744446992874146, + 0.2148163914680481 + ], + "group_id": "ms-5b26c33feed6992386b1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8326650857925415, + 2.0, + 0.15131878852844238, + 0.15007886290550232, + 0.5293327569961548, + 0.06490398943424225, + 0.5425760746002197, + 0.20163774490356445, + 0.15259261429309845, + 0.23924465477466583, + 0.12766429781913757, + 0.08033241331577301, + 0.07788355648517609, + 0.13886235654354095, + 0.09716498851776123, + 0.11682072281837463 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.06662726402282715, + -0.14569896459579468, + 0.04974943399429321, + 0.539892315864563, + -0.19018621742725372, + -0.5884796380996704, + 0.0, + 0.12748029828071594, + 0.11038697510957718, + -0.03454907238483429, + 0.08700031787157059, + -0.13105249404907227, + -1.9082190990447998, + 0.0, + -0.0033895689994096756, + -0.15730009973049164, + 0.15846982598304749, + 0.11786291003227234, + -0.02387581765651703, + -1.5966379642486572, + 0.0 + ], + [ + -0.06662726402282715, + 0.059319257736206055, + -0.19934123754501343, + 0.539892315864563, + 0.07902508974075317, + -0.5884796380996704, + 0.0, + -0.3428957760334015, + 0.08558038622140884, + 0.28825634717941284, + -0.32335156202316284, + -0.3631777763366699, + -1.9082190990447998, + 0.0, + -0.3765835762023926, + -0.5554376840591431, + 0.3468284010887146, + 0.875933825969696, + 0.11397302150726318, + -1.5966379642486572, + 0.0 + ], + [ + -1.0666272640228271, + -0.6187554597854614, + -0.8818012475967407, + 1.539892315864563, + -0.428971529006958, + -1.5884796380996704, + 2.0647666454315186, + -0.03600159287452698, + 0.032059766352176666, + 0.030456876382231712, + 0.14377687871456146, + 0.20562076568603516, + -2.9082190990448, + -1.332404375076294, + -0.014788046479225159, + -0.0006958844605833292, + 0.023885300382971764, + 0.5519534349441528, + 0.1840958297252655, + -2.5966379642486572, + -1.4879226684570312 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.129533290863037, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.664808750152588, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9758453369140625 + ], + [ + -1.2971861362457275, + -0.7044790983200073, + -1.134492039680481, + 1.5821952819824219, + -0.7159937620162964, + -1.7615629434585571, + 2.031708002090454, + -0.25998222827911377, + 0.10180196911096573, + -0.24491730332374573, + 0.2074550986289978, + 0.2793378233909607, + -2.752415657043457, + -1.3360638618469238, + 0.03524862602353096, + -0.1854405552148819, + -0.1696440726518631, + 0.7172544598579407, + 0.0028989464044570923, + -2.680950164794922, + -1.2071388959884644 + ], + [ + -1.465846300125122, + -0.34887754917144775, + -1.046374797821045, + 1.703241229057312, + -0.19744430482387543, + -1.2604204416275024, + 1.9822474718093872, + 0.18854136765003204, + -0.146264910697937, + 0.059789031744003296, + 0.39534950256347656, + 0.29636046290397644, + -2.6612584590911865, + -1.5028973817825317, + 0.059081435203552246, + -0.11711112409830093, + -0.07238245755434036, + 0.25233787298202515, + 0.12625277042388916, + -2.2362401485443115, + -1.7053974866867065 + ], + [ + -1.1892943382263184, + -0.20831987261772156, + -0.8013461828231812, + 1.7197901010513306, + -0.39134201407432556, + -1.5160430669784546, + 2.4594008922576904, + -0.1669330894947052, + -0.337530255317688, + 0.15848031640052795, + 0.05738166719675064, + 0.41757503151893616, + -3.1393940448760986, + -0.9257140755653381, + -0.5068305134773254, + 0.1864723265171051, + 0.45369866490364075, + 0.6036283373832703, + -0.018199875950813293, + -2.8028440475463867, + -1.3847744464874268 + ], + [ + -1.1111165285110474, + -1.191490888595581, + -1.361661434173584, + 1.197826623916626, + -0.29219919443130493, + -1.7355475425720215, + 2.0579750537872314, + -0.08019472658634186, + 0.5140348672866821, + -0.5474883317947388, + -0.03311571478843689, + -0.060686588287353516, + -2.3427765369415283, + -1.8552747964859009, + -0.36303937435150146, + 0.01966947503387928, + -0.5651369094848633, + 0.22617560625076294, + -0.16322728991508484, + -2.421419143676758, + -0.921785831451416 + ], + [ + -1.012363076210022, + -0.6483663320541382, + -0.7951939702033997, + 1.2563079595565796, + -0.5323562622070312, + -1.5913673639297485, + 1.406038761138916, + -0.39036238193511963, + -0.12943506240844727, + -0.4373972415924072, + -0.20462383329868317, + 0.12806488573551178, + -3.570098400115967, + -1.8897531032562256, + 0.6369156837463379, + 0.2483828067779541, + 0.6809477210044861, + 0.8138958215713501, + 0.6604747176170349, + -2.717491865158081, + -1.9953765869140625 + ], + [ + -0.378162145614624, + -0.23438718914985657, + -1.2589956521987915, + 1.844052791595459, + -0.8936926126480103, + -1.230797290802002, + 2.5580525398254395, + -0.21314577758312225, + -0.05935201793909073, + 0.46811148524284363, + 0.38594305515289307, + 1.0049593448638916, + -2.491497278213501, + -1.3096134662628174, + 0.15605805814266205, + 0.7682130932807922, + -0.4010901153087616, + 0.8638691902160645, + 0.5755372047424316, + -3.324826240539551, + -1.6860322952270508 + ], + [ + -0.6920164227485657, + -0.5060986280441284, + -0.5146759748458862, + 2.3206489086151123, + 0.26810598373413086, + -1.7696658372879028, + 1.4922571182250977, + -0.6155052185058594, + -0.5581206679344177, + 0.4068271219730377, + 0.5812678337097168, + -0.2056950330734253, + -2.227752923965454, + -0.7079744935035706, + 0.3153572678565979, + 0.019053615629673004, + -0.10691303759813309, + -0.18750637769699097, + 0.5490041971206665, + -2.7611911296844482, + -1.0861625671386719 + ], + [ + -0.9623568654060364, + -0.821997880935669, + -0.0017323493957519531, + 1.8667385578155518, + 0.39793866872787476, + -1.3673040866851807, + 2.8636348247528076, + -0.739641547203064, + -0.4186078906059265, + -0.21357670426368713, + 0.6820193529129028, + 0.09538085013628006, + -3.242215633392334, + -1.023385763168335, + -0.2367737591266632, + 0.9248462915420532, + 0.04536557197570801, + 0.2294020652770996, + 0.15174128115177155, + -2.509904146194458, + -0.5901334881782532 + ], + [ + -0.565070390701294, + 0.37745314836502075, + -1.7464253902435303, + 1.4869437217712402, + -1.187619924545288, + -2.3802757263183594, + 2.5987753868103027, + 0.5777139663696289, + -0.04772312939167023, + -0.49332448840141296, + -0.515097975730896, + 0.751485288143158, + -2.7030012607574463, + -1.5739474296569824, + 0.8266244530677795, + -0.9454089403152466, + 0.7678747177124023, + 1.0557427406311035, + 0.0584852397441864, + -2.8590199947357178, + -0.8677505850791931 + ], + [ + -0.2763708233833313, + 0.28772038221359253, + -0.7008944153785706, + 1.3402774333953857, + -0.78607177734375, + -2.1210508346557617, + 1.4908761978149414, + -0.4155166447162628, + 0.40930652618408203, + -0.1769891083240509, + -0.20092366635799408, + 0.5321645736694336, + -3.2144649028778076, + -1.2440845966339111, + -0.247700035572052, + 0.7174559235572815, + 0.11071103811264038, + -0.023232340812683105, + 0.6976174116134644, + -2.9463140964508057, + -0.5927516222000122 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.06662726402282715, + -0.05619156360626221, + -0.02257448434829712, + 0.539892315864563, + 0.053197890520095825, + -0.5884796380996704, + 0.0, + -0.04219311475753784, + 0.010949447751045227, + 0.027476036921143532, + -0.02041013538837433, + -0.01791653037071228, + -1.9082190990447998, + 0.0, + 0.01256529800593853, + 0.014024228788912296, + 0.03545428439974785, + 0.019800782203674316, + -0.022738635540008545, + -1.5966379642486572, + 0.0 + ] + ], + "predicted_scores": [ + 5.118069648742676, + 5.116032600402832, + 4.928212642669678, + 4.704899311065674, + 4.558428764343262, + 4.518015384674072, + 3.5779058933258057, + 3.563319206237793 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1522272686", + "selected_index": 0, + "state_hash": "10034ca87ce4b5c491c0aa23d2a30389ba8265fa695d65198bf719c54565cf2a", + "stored_base_utility": 0.8326650857925415, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8388344049453735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8388344049453735 + }, + "base_progress": 0.8388344049453735, + "base_success": false, + "base_utility": 0.8388344049453735, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8183318376541138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8183318376541138 + }, + { + "contact_quality": null, + "progress": 0.22806134819984436, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22806134819984436 + }, + { + "contact_quality": null, + "progress": 0.832514762878418, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.832514762878418 + }, + { + "contact_quality": null, + "progress": 0.19844911992549896, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844911992549896 + }, + { + "contact_quality": null, + "progress": 0.22729773819446564, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22729773819446564 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.20421144366264343, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20421144366264343 + }, + { + "contact_quality": null, + "progress": 0.21390068531036377, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21390068531036377 + } + ], + "candidate_progress": [ + 0.8183318376541138, + 0.22806134819984436, + 0.832514762878418, + 0.19844911992549896, + 0.22729773819446564, + 1.0, + 0.20421144366264343, + 0.21390068531036377 + ], + "candidate_source_chart_ids": [ + "ms-5dfbb16e173e1a3e2bd6", + "ms-0b226017db7f07c5307e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-5a456ddf97d294548cf1", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-63ac245579b1188935ad", + "generated_tangents": [ + [ + 0.07592315971851349, + -0.13614404201507568, + 0.24823014438152313, + -0.01298859715461731, + -0.09450054168701172, + -1.8092811107635498, + 0.15673872828483582, + 0.037664398550987244, + -0.0279239509254694, + -0.15241870284080505, + -0.010709114372730255, + 0.02850356698036194, + -1.1655819416046143, + 0.08682677149772644, + -0.022702503949403763, + -0.06397686898708344, + 0.0005460083484649658, + 0.23710030317306519, + 0.17263144254684448, + -1.1171483993530273, + -0.3485688865184784 + ], + [ + 0.07267247140407562, + -0.0802840143442154, + -0.08107021450996399, + 0.07116827368736267, + 0.08196194469928741, + -1.653926968574524, + 0.07945936918258667, + 0.01596371829509735, + -0.00013408716768026352, + -0.047363825142383575, + 0.06136802211403847, + -0.02504594624042511, + -1.290826678276062, + -0.026758089661598206, + -0.11897893249988556, + 0.00425499863922596, + 0.12613841891288757, + 0.06169748306274414, + -0.061475396156311035, + -1.1233642101287842, + -0.1260521113872528 + ], + [ + 0.060978569090366364, + -0.02506651170551777, + -0.024420887231826782, + 0.19880282878875732, + 0.08790360391139984, + -1.5615947246551514, + 0.10512334108352661, + -0.00031866133213043213, + 0.03373006731271744, + -0.10571563988924026, + 0.06250378489494324, + 0.11451734602451324, + -1.1461238861083984, + -0.03493809700012207, + -0.13956454396247864, + -0.050033316016197205, + 0.09889145195484161, + 0.06260648369789124, + -0.029381930828094482, + -1.1260510683059692, + -0.1741926670074463 + ], + [ + 0.09063458442687988, + -0.07695324718952179, + -0.05717006325721741, + 0.036723554134368896, + 0.13756972551345825, + -1.5772385597229004, + 0.06240609288215637, + 0.03608326613903046, + -0.026225006207823753, + -0.027511008083820343, + 0.05826742202043533, + -0.012191027402877808, + -1.243361234664917, + -0.067467600107193, + -0.0990309864282608, + -0.020708836615085602, + 0.17371058464050293, + 0.04112726449966431, + -0.059579998254776, + -1.1515450477600098, + -0.16000548005104065 + ], + [ + 0.06826478242874146, + -0.09602241218090057, + -0.08379694819450378, + 0.06933766603469849, + 0.19838720560073853, + -1.4886646270751953, + 0.05686655640602112, + 0.013776406645774841, + 0.016440080478787422, + -0.03296954184770584, + -0.009033150970935822, + 0.07672037929296494, + -1.180536150932312, + 0.029635459184646606, + -0.10914972424507141, + -0.047345105558633804, + 0.13566727936267853, + 0.020698681473731995, + -0.06398943066596985, + -1.1226403713226318, + -0.05157192051410675 + ], + [ + 0.0007637739181518555, + -0.12595009803771973, + 0.04650917649269104, + -0.0979643166065216, + -0.017885595560073853, + -1.7403672933578491, + -0.022388488054275513, + -0.009271219372749329, + 0.018568366765975952, + -0.052760593593120575, + 0.06903066486120224, + -0.07851454615592957, + -1.2405943870544434, + 0.019612088799476624, + -0.04794960841536522, + 0.026756152510643005, + 0.20309510827064514, + 0.09076309204101562, + -0.23707938194274902, + -1.091101884841919, + -0.15543323755264282 + ], + [ + 0.09207674860954285, + -0.08228801935911179, + -0.04116174578666687, + -0.0035973042249679565, + 0.15192490816116333, + -1.4146941900253296, + -0.02971857786178589, + 0.1076693907380104, + -0.055346980690956116, + -0.0685649886727333, + 0.10692813992500305, + 0.1420077085494995, + -1.13576078414917, + -0.02274516224861145, + -0.09511969983577728, + 0.022226441651582718, + 0.02580389380455017, + 0.16851267218589783, + -0.054017484188079834, + -1.016629695892334, + -0.20070743560791016 + ], + [ + 0.04372203350067139, + -0.06514435261487961, + -0.08617410063743591, + 0.029892414808273315, + 0.06151294708251953, + -1.5051560401916504, + -0.042172372341156006, + 0.059974148869514465, + -0.004093525931239128, + -0.0804525837302208, + 0.046910591423511505, + 0.027034729719161987, + -1.095505952835083, + -0.10103793442249298, + -0.09881904721260071, + 0.0494321808218956, + 0.11947246640920639, + 0.06747278571128845, + -0.074625164270401, + -1.0337462425231934, + -0.2033267319202423 + ] + ], + "generated_utilities": [ + 0.8183318376541138, + 0.22806134819984436, + 0.832514762878418, + 0.19844911992549896, + 0.22729773819446564, + 2.0, + 0.20421144366264343, + 0.21390068531036377 + ], + "group_id": "ms-63ac245579b1188935ad", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.855198860168457, + 0.8138954639434814, + 0.20312510430812836, + 2.0, + 0.7862945795059204, + 0.7843606472015381, + 0.6004383563995361, + 0.15506848692893982, + 0.15924105048179626, + 0.132452130317688, + 0.05381816625595093, + 0.09561765193939209, + 0.07140937447547913, + 0.10806754231452942, + 0.023168057203292847, + 0.07957816123962402 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.007633227854967117, + 0.02389371022582054, + 0.2181050330400467, + 0.026324495673179626, + 0.1209479570388794, + -1.3937149047851562, + 0.0, + -0.2493116855621338, + -0.17256586253643036, + -0.06812826544046402, + 0.019445210695266724, + -0.2657848000526428, + -1.0657541751861572, + 0.0, + -0.04306507110595703, + -0.07170329242944717, + 0.035314902663230896, + 0.06211268901824951, + 0.18158195912837982, + -0.7849693298339844, + 0.0 + ], + [ + -0.011555496603250504, + 0.0016381663735955954, + 0.019193550571799278, + 0.26171422004699707, + 0.08122420310974121, + -2.3937149047851562, + -1.53977632522583, + -0.00580335408449173, + -0.0020052569452673197, + 0.01787502132356167, + 0.4615197777748108, + 0.1938573122024536, + -2.0657541751861572, + -1.4660639762878418, + -0.00267898291349411, + -0.0038005562964826822, + 0.0179074015468359, + 0.5644202828407288, + 0.2690354883670807, + -1.7849693298339844, + -1.3308851718902588 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.07955265045166, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9321279525756836, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.6617703437805176 + ], + [ + -0.16036809980869293, + 0.11436319351196289, + -0.14857570827007294, + 0.09414434432983398, + 0.02274160459637642, + -2.426297903060913, + -1.7686878442764282, + -0.26862773299217224, + 0.16241224110126495, + 0.24208402633666992, + 0.4166663587093353, + 0.07841069251298904, + -2.2318098545074463, + -1.2228643894195557, + -0.08888694643974304, + 0.20051127672195435, + -0.14729085564613342, + 0.5799955129623413, + 0.33039355278015137, + -2.00492262840271, + -1.4948352575302124 + ], + [ + -0.3820815682411194, + -0.3679526746273041, + 0.18845602869987488, + 0.4168931245803833, + -0.007936321198940277, + -2.700572967529297, + -1.795093059539795, + 0.1203194335103035, + 0.19610334932804108, + 0.2749679982662201, + 0.8331882357597351, + -0.06589585542678833, + -2.333136558532715, + -1.1008005142211914, + -0.02170141413807869, + 0.24186654388904572, + -0.3734443485736847, + 0.5144574046134949, + 0.3134346902370453, + -1.5088386535644531, + -1.1978046894073486 + ], + [ + 0.13625949621200562, + 0.47836095094680786, + -0.05790846049785614, + -0.09614765644073486, + -0.14281350374221802, + -2.522264003753662, + -1.0473889112472534, + 0.4792231023311615, + -0.4869813621044159, + 0.17644789814949036, + 0.8425257205963135, + -0.28899794816970825, + -2.5508456230163574, + -0.9762126207351685, + -0.16381770372390747, + 0.1586131900548935, + -0.3460317850112915, + 0.6056831479072571, + 0.7327044010162354, + -1.494289755821228, + -1.4671605825424194 + ], + [ + -0.40018847584724426, + -0.028111951425671577, + 0.31324630975723267, + -0.22690656781196594, + 0.2902344763278961, + -2.9841673374176025, + -1.4654184579849243, + -0.1155615746974945, + -0.4125809073448181, + 0.6098153591156006, + 0.8544195890426636, + 0.22702345252037048, + -1.7853796482086182, + -1.3474400043487549, + -0.454882949590683, + -0.3122820556163788, + 0.22399993240833282, + 0.05299121141433716, + 0.07802119851112366, + -1.3896000385284424, + -1.894580602645874 + ], + [ + -0.6587541699409485, + -0.6905457973480225, + 0.10385943204164505, + -0.2785833477973938, + 0.2321375161409378, + -2.565701723098755, + -1.1605603694915771, + -0.6775721907615662, + -0.30887091159820557, + 0.1186944767832756, + 1.0924456119537354, + 0.5009081363677979, + -1.8667830228805542, + -1.2491121292114258, + -0.11290804296731949, + -0.03346216678619385, + -0.28778207302093506, + 1.2255009412765503, + 0.6856998801231384, + -1.1723419427871704, + -1.1220875978469849 + ], + [ + -0.5019480586051941, + 0.07311037182807922, + 0.5101989507675171, + 0.7159117460250854, + 0.005071915686130524, + -2.6556949615478516, + -2.321233034133911, + 0.6035679578781128, + -0.13785676658153534, + -0.4309522807598114, + 0.9664275050163269, + -0.020235419273376465, + -1.6894025802612305, + -0.7511875033378601, + 0.5713916420936584, + 0.24962951242923737, + 0.40067923069000244, + 0.12209588289260864, + 0.03222101926803589, + -2.27744460105896, + -1.9113109111785889 + ], + [ + 0.7552750110626221, + -0.8085685968399048, + 0.20896124839782715, + -0.6043995022773743, + 0.733860969543457, + -1.610467791557312, + -1.0368051528930664, + -0.02292630262672901, + 0.6320546269416809, + 0.14528857171535492, + 0.8372507095336914, + 0.41523146629333496, + -2.339751958847046, + -0.6185142993927002, + -0.5750091075897217, + 0.12324154376983643, + 0.2804809510707855, + 1.0503896474838257, + -0.15392154455184937, + -0.9555137753486633, + -1.517809510231018 + ], + [ + 0.8853163123130798, + 0.09475164860486984, + -0.13681088387966156, + 0.08429434895515442, + 0.13555681705474854, + -1.5859344005584717, + -2.1957859992980957, + 0.9337366819381714, + -1.0016963481903076, + 0.11365362256765366, + -0.31175100803375244, + 0.7412776350975037, + -1.0797133445739746, + -0.8527018427848816, + 0.6110302209854126, + 0.006565961055457592, + 0.38363590836524963, + 0.5695288181304932, + -0.2547180950641632, + -2.6268043518066406, + -1.748514175415039 + ], + [ + 0.513801097869873, + 0.7078304886817932, + 0.7309613823890686, + -0.32746708393096924, + -0.36263394355773926, + -3.3305888175964355, + -1.2108653783798218, + 0.5104080438613892, + -0.26447102427482605, + 0.6361017823219299, + 0.5986918210983276, + -0.48598653078079224, + -2.4424209594726562, + -1.4694169759750366, + -0.5649402737617493, + -0.22020848095417023, + 0.6405193209648132, + -0.0698668360710144, + 0.7155715227127075, + -2.352860450744629, + -0.7502654194831848 + ], + [ + -0.9911022782325745, + 0.9677192568778992, + 0.5195661187171936, + 0.6733791828155518, + 0.7589308023452759, + -3.3654959201812744, + -0.6360958814620972, + 0.5926312208175659, + -0.06773044914007187, + 0.3785405457019806, + 0.5934506058692932, + 0.6434512138366699, + -2.891094923019409, + -0.8924227356910706, + 0.8036434650421143, + 0.6109268665313721, + -0.6607317924499512, + 1.0173940658569336, + 0.2546094059944153, + -1.4653143882751465, + -2.0709643363952637 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16686837375164032, + 0.1354454904794693, + -0.11150697618722916, + 0.7247087359428406, + 0.2886054813861847, + -1.3937149047851562, + 0.0, + 0.6050093770027161, + 0.20291854441165924, + 0.5827622413635254, + -0.2687305808067322, + -0.25171223282814026, + -1.0657541751861572, + 0.0, + 0.29065752029418945, + -0.023413484916090965, + -0.24587245285511017, + 0.4600236713886261, + -0.4278547465801239, + -0.7849693298339844, + 0.0 + ] + ], + "predicted_scores": [ + 3.951364278793335, + 3.529320478439331, + 3.5285229682922363, + 3.487807273864746, + 3.3132495880126953, + 3.2474350929260254, + 3.2326605319976807, + 3.150263547897339 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "211245842", + "selected_index": 0, + "state_hash": "fb52e84b6addb8bdddf02a80d5c8567e0fa27195729e8440271b710441061b7f", + "stored_base_utility": 0.855198860168457, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8184402585029602, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8184402585029602 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8130596876144409, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8130596876144409 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8125015497207642, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8125015497207642 + }, + { + "contact_quality": null, + "progress": 0.8125571608543396, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8125571608543396 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8167412877082825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8167412877082825 + } + ], + "candidate_progress": [ + 0.8184402585029602, + 1.0, + 0.8130596876144409, + 1.0, + 0.8125015497207642, + 0.8125571608543396, + 1.0, + 0.8167412877082825 + ], + "candidate_source_chart_ids": [ + "ms-875a4668671c02054a3c", + "ms-20f3cfcf183c69853b1e", + "ms-875a4668671c02054a3c", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + true, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "generated_tangents": [ + [ + -0.01070578396320343, + -0.033470772206783295, + -0.15495410561561584, + -0.14183250069618225, + 0.07164767384529114, + -1.9417678117752075, + -0.14187884330749512, + 0.08544448018074036, + 0.14689677953720093, + -0.052584730088710785, + 0.03513490408658981, + -0.05415037274360657, + -1.8725054264068604, + -0.1501309722661972, + -0.03508613631129265, + -0.05812845751643181, + 0.0007632821798324585, + 0.0408368855714798, + 0.020252972841262817, + -1.7189569473266602, + -0.2775786817073822 + ], + [ + -0.012276023626327515, + -0.028644850477576256, + -0.09336325526237488, + -0.14155516028404236, + 0.09984803199768066, + -2.1003611087799072, + -0.07957345247268677, + 0.02338235080242157, + 0.18073223531246185, + -0.0357944592833519, + -0.012997716665267944, + -0.08227553963661194, + -1.8983495235443115, + -0.14400701224803925, + -0.05123366788029671, + -0.02659599855542183, + 0.0619058758020401, + 0.08809824287891388, + -0.07122787833213806, + -1.5974328517913818, + -0.20015138387680054 + ], + [ + -0.11515431106090546, + -0.17193876206874847, + 0.10262271761894226, + -0.28938037157058716, + 0.2151399850845337, + -2.1313931941986084, + -0.0048177242279052734, + 0.16256849467754364, + 0.14841464161872864, + -0.09813383966684341, + -0.12788528203964233, + -8.824467658996582e-05, + -1.9873358011245728, + -0.5370582938194275, + -0.07476823031902313, + 0.007620302960276604, + -0.1157131940126419, + 0.0862252488732338, + 0.14170050621032715, + -2.0510191917419434, + -0.8264507055282593 + ], + [ + -0.08699685335159302, + -0.06080082803964615, + 0.035507649183273315, + -0.30851852893829346, + 0.21045854687690735, + -1.9924968481063843, + -0.0053428709506988525, + 0.0917435809969902, + 0.15357144176959991, + -0.022615395486354828, + -0.041588179767131805, + -0.14054468274116516, + -1.841922640800476, + -0.34916096925735474, + -0.0639040470123291, + 0.004495611414313316, + 0.10388081520795822, + 0.044780537486076355, + 0.0870068371295929, + -1.650801420211792, + -0.46704408526420593 + ], + [ + 0.06315898150205612, + -0.0010309480130672455, + -0.05888590216636658, + -0.12963604927062988, + 0.1410176157951355, + -2.07487154006958, + -0.0625162124633789, + 0.038195788860321045, + 0.1573835015296936, + -0.21881094574928284, + 0.008190367370843887, + -0.23996657133102417, + -1.8577485084533691, + -0.29064762592315674, + -0.3194462060928345, + -0.15451104938983917, + 0.1985035091638565, + -0.03591302037239075, + -0.2407069206237793, + -1.6201457977294922, + 0.11997450143098831 + ], + [ + -0.051500990986824036, + -0.1544715166091919, + -0.09336391091346741, + -0.3871646225452423, + 0.26705265045166016, + -1.8083488941192627, + 0.11108162999153137, + 0.19076180458068848, + 0.12387299537658691, + -0.03509994596242905, + 0.12095065414905548, + -0.3259950280189514, + -1.9535678625106812, + -0.30912458896636963, + 0.030186571180820465, + -0.03327091783285141, + 0.07141990959644318, + -0.007241472601890564, + 0.20521318912506104, + -1.9050300121307373, + -0.5703409910202026 + ], + [ + -0.14716701209545135, + -0.05128244683146477, + -0.017126411199569702, + -0.20498310029506683, + 0.2128160148859024, + -1.8568142652511597, + -0.07626485824584961, + 0.13453879952430725, + 0.16995331645011902, + 0.013863474130630493, + -0.11347896605730057, + -0.14959144592285156, + -1.8593852519989014, + -0.23283414542675018, + -0.0664626955986023, + -0.004108800552785397, + 0.10919519513845444, + -0.053353384137153625, + -0.07882529497146606, + -1.6868853569030762, + -0.5271542072296143 + ], + [ + -0.06839938461780548, + -0.005638685077428818, + -0.0015881359577178955, + -0.326743483543396, + 0.29128238558769226, + -1.9975172281265259, + 0.08928051590919495, + 0.019716590642929077, + 0.15808995068073273, + -0.07500874251127243, + -0.057167477905750275, + -0.04409746825695038, + -1.923764705657959, + -0.2519254684448242, + -0.06714454293251038, + -0.05821181833744049, + 0.044128432869911194, + 0.06208115816116333, + -0.020185887813568115, + -1.5056889057159424, + -0.45270946621894836 + ] + ], + "generated_utilities": [ + 0.8184402585029602, + 2.0, + 0.8130596876144409, + 2.0, + 0.8125015497207642, + 0.8125571608543396, + 2.0, + 0.8167412877082825 + ], + "group_id": "ms-6c8d06bace98a28a8e4e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8583254218101501, + 2.0, + 0.8125, + 0.12893107533454895, + 0.8582976460456848, + 0.15346482396125793, + 0.18961156904697418, + 0.13901273906230927, + 0.10113435983657837, + 0.11161506175994873, + 0.09923169016838074, + 0.16662254929542542, + 0.04211682081222534, + 0.024710848927497864, + 0.05722464621067047, + 0.8584086298942566 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.10696407407522202, + 0.3957799971103668, + 0.15132969617843628, + -0.32772913575172424, + 0.005261177197098732, + -2.2650365829467773, + 0.0, + 0.3958877623081207, + -0.5120954513549805, + -0.4317946135997772, + -0.3970779776573181, + 0.5317854285240173, + -1.8659565448760986, + 0.0, + 0.1198807880282402, + -0.11760073900222778, + -0.6048114895820618, + 0.15180528163909912, + -0.532695472240448, + -1.6115882396697998, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.120756149291992, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.070481538772583, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0495758056640625 + ], + [ + 0.30105674266815186, + 0.4776385426521301, + -0.22267001867294312, + -0.13300693035125732, + -0.11095686256885529, + -3.138023853302002, + -0.793931245803833, + 0.16969743371009827, + -0.16339385509490967, + 0.02508675307035446, + 0.654399573802948, + -0.22830897569656372, + -3.1443004608154297, + -1.4291013479232788, + 0.005021120421588421, + 0.005560946650803089, + -0.1901366114616394, + 1.0211858749389648, + -0.09353887289762497, + -2.3457469940185547, + -1.406853199005127 + ], + [ + 0.4160586893558502, + 0.13316196203231812, + -0.21056225895881653, + -0.25006812810897827, + 0.060951750725507736, + -2.8735690116882324, + -0.9062713980674744, + -0.28707224130630493, + -0.2396206259727478, + -0.19859735667705536, + 0.29039445519447327, + -0.24077445268630981, + -2.644789934158325, + -1.8330551385879517, + -0.23438936471939087, + 0.04620177671313286, + 0.1472887098789215, + 0.9629149436950684, + -0.020509399473667145, + -2.7985036373138428, + -1.2463163137435913 + ], + [ + 0.2486386001110077, + -0.022333532571792603, + 0.14628465473651886, + -0.8381847143173218, + 0.21282373368740082, + -3.585387945175171, + -1.2156991958618164, + -0.08670558780431747, + -0.3178575336933136, + 0.4322021007537842, + 0.6421236395835876, + -0.33922863006591797, + -3.004714012145996, + -1.19637131690979, + -0.015640778467059135, + 0.10077434033155441, + -0.449941486120224, + 1.0431067943572998, + 0.2899783253669739, + -2.500624179840088, + -1.6148788928985596 + ], + [ + 0.46770766377449036, + 0.03902164101600647, + 0.2556130588054657, + -0.45893940329551697, + 0.08939866721630096, + -3.163339853286743, + -0.7599828243255615, + 0.3327450156211853, + 0.3252364993095398, + 0.3067358434200287, + 1.1394517421722412, + 0.472858726978302, + -3.1504058837890625, + -1.9780522584915161, + -0.26712900400161743, + -0.262444406747818, + 0.20269782841205597, + 1.336982250213623, + -0.06957009434700012, + -2.0651021003723145, + -1.8484716415405273 + ], + [ + 0.48348987102508545, + 0.19676931202411652, + -0.20689666271209717, + -0.21521174907684326, + -0.5048230886459351, + -3.491732358932495, + -0.7390058636665344, + 0.18321916460990906, + -0.1509065330028534, + 0.27166980504989624, + 1.004181981086731, + -0.5531666874885559, + -2.531594753265381, + -2.0243897438049316, + -0.44593361020088196, + 0.49786314368247986, + -0.006632372736930847, + 0.24423635005950928, + -0.6331305503845215, + -2.7707719802856445, + -2.1364808082580566 + ], + [ + -0.5684993863105774, + 0.0623701810836792, + -0.10449839383363724, + -0.7858190536499023, + -0.057080868631601334, + -3.981018304824829, + -0.6721940636634827, + 0.10191962122917175, + -0.7216448783874512, + 0.3870577812194824, + 0.5508396625518799, + -0.6811783313751221, + -3.6131227016448975, + -1.7484180927276611, + 0.34220433235168457, + -0.16193144023418427, + 0.16876991093158722, + 0.6509957909584045, + 0.2548854351043701, + -2.0779964923858643, + -1.5782216787338257 + ], + [ + -0.1224658265709877, + 0.22953394055366516, + -0.41326335072517395, + 0.19802969694137573, + 0.6866055130958557, + -2.6415462493896484, + -1.049781322479248, + -0.6788480281829834, + -0.8187242746353149, + 0.6141759157180786, + 1.3649951219558716, + -0.36378854513168335, + -2.2142889499664307, + -1.3759111166000366, + 0.583359956741333, + 0.6860965490341187, + 0.8285356760025024, + 1.4575884342193604, + -0.4101440906524658, + -2.218319892883301, + -1.5390111207962036 + ], + [ + 0.3368571698665619, + 0.4528505206108093, + -0.2985517382621765, + 0.37766069173812866, + 0.2064778208732605, + -2.8070714473724365, + -1.5598305463790894, + -0.9032005071640015, + -0.48420336842536926, + 0.639835000038147, + -0.3801422119140625, + -0.1376074254512787, + -3.369077205657959, + -1.9231767654418945, + -0.16211053729057312, + 0.53387051820755, + 0.6583098769187927, + 0.7611914873123169, + -0.46404945850372314, + -2.7554476261138916, + -1.4307246208190918 + ], + [ + 0.6002402901649475, + -0.14213827252388, + 0.30183425545692444, + -0.20118463039398193, + -0.16981711983680725, + -3.4212875366210938, + -0.2252577543258667, + 0.34123149514198303, + 0.003176456317305565, + 0.7433648109436035, + 1.1403348445892334, + 0.5709706544876099, + -1.8894051313400269, + -0.7200812101364136, + -0.7235198020935059, + 0.2147149294614792, + -0.28744715452194214, + 1.0323235988616943, + 0.5815067291259766, + -2.582993984222412, + -1.2160475254058838 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.019069351255893707, + 0.012574762105941772, + 0.07413499802350998, + -0.04097828269004822, + -0.03991875424981117, + -2.2650365829467773, + 0.0, + -0.06082560494542122, + 0.05821063742041588, + 0.07896332442760468, + -0.037061095237731934, + -0.08879614621400833, + -1.8659565448760986, + 0.0, + -0.022465867921710014, + -0.026843247935175896, + 0.04789978265762329, + -0.019972383975982666, + -0.04040021449327469, + -1.6115882396697998, + 0.0 + ] + ], + "predicted_scores": [ + 2.5245156288146973, + 2.4051806926727295, + 2.306995391845703, + 1.821117877960205, + 1.7071008682250977, + 1.702606201171875, + 1.620788812637329, + 1.4197536706924438 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1665739511", + "selected_index": 0, + "state_hash": "0f77663511063f17b0962024501b863c62b881513459d8bfa3ac00bbaed67de4", + "stored_base_utility": 0.8583254218101501, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.19993644952774048, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19993644952774048 + }, + { + "contact_quality": null, + "progress": 0.23993994295597076, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23993994295597076 + }, + { + "contact_quality": null, + "progress": 0.22335931658744812, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22335931658744812 + }, + { + "contact_quality": null, + "progress": 0.23456653952598572, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23456653952598572 + }, + { + "contact_quality": null, + "progress": 0.23101213574409485, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23101213574409485 + }, + { + "contact_quality": null, + "progress": 0.2455206662416458, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2455206662416458 + }, + { + "contact_quality": null, + "progress": 0.8128684163093567, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8128684163093567 + }, + { + "contact_quality": null, + "progress": 0.14169609546661377, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14169609546661377 + } + ], + "candidate_progress": [ + 0.19993644952774048, + 0.23993994295597076, + 0.22335931658744812, + 0.23456653952598572, + 0.23101213574409485, + 0.2455206662416458, + 0.8128684163093567, + 0.14169609546661377 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-72777024349c2aacd5e0", + "generated_tangents": [ + [ + 0.055865637958049774, + -0.055816806852817535, + -0.20907989144325256, + -0.769936203956604, + 0.4880841374397278, + -0.25204890966415405, + 0.6985225677490234, + 0.12398729473352432, + 0.16473756730556488, + -0.21713915467262268, + 0.05861528217792511, + -0.06095767021179199, + -1.8979934453964233, + -0.37624984979629517, + -0.006911199539899826, + 0.010136563330888748, + 0.02695821225643158, + 0.22647030651569366, + -0.2013503909111023, + -1.2692904472351074, + -0.1853887140750885 + ], + [ + -0.020478785037994385, + 0.044863805174827576, + -0.18460753560066223, + -0.6313502788543701, + 0.7127556204795837, + -0.28846850991249084, + 0.3333684802055359, + 0.016965195536613464, + 0.09148606657981873, + -0.08868777006864548, + 0.012398604303598404, + -0.08372506499290466, + -1.7294925451278687, + 0.13085003197193146, + -0.06228635832667351, + -0.050725340843200684, + 0.06951582431793213, + 0.03120531141757965, + -0.1471652388572693, + -1.3888170719146729, + 0.022654831409454346 + ], + [ + 0.05123429745435715, + 0.010771278291940689, + -0.3759067952632904, + -0.5952674150466919, + 0.7524579763412476, + -0.19433516263961792, + 0.3558697998523712, + 0.07158229500055313, + 0.08915022015571594, + -0.0989876464009285, + 0.0979427918791771, + -0.15328934788703918, + -1.7377667427062988, + 0.19969624280929565, + -0.13484250009059906, + -0.04985309764742851, + -0.009539350867271423, + 0.005046576261520386, + -0.09007668495178223, + -1.4380772113800049, + 0.043121084570884705 + ], + [ + -0.12803484499454498, + 0.0055353716015815735, + -0.20564386248588562, + -0.741146445274353, + 0.7627297639846802, + -0.4963206946849823, + 0.6548652052879333, + 0.05438297986984253, + 0.1399819552898407, + -0.24557217955589294, + 0.0346328541636467, + -0.2714666724205017, + -1.7525880336761475, + -0.13117019832134247, + 0.07491924613714218, + -0.04336576163768768, + 0.06672023236751556, + -0.11204515397548676, + -0.2286921739578247, + -1.4248881340026855, + -0.15738847851753235 + ], + [ + -0.07982590794563293, + 0.044089801609516144, + -0.13705822825431824, + -0.5091069340705872, + 0.707309365272522, + -0.37458983063697815, + 0.8120944499969482, + -0.00697346031665802, + 0.16757282614707947, + -0.2451404333114624, + 0.22887039184570312, + 0.20996689796447754, + -1.660861611366272, + -0.24643395841121674, + -0.09035596251487732, + -0.04664681851863861, + -0.10761897265911102, + 0.05824191868305206, + 0.2820228338241577, + -1.3844363689422607, + -0.12130829691886902 + ], + [ + -0.13600154221057892, + 0.015058614313602448, + -0.13416323065757751, + -0.7091516256332397, + 0.6443045139312744, + -0.4889087975025177, + 0.5162445902824402, + 0.04842694103717804, + 0.13782557845115662, + -0.2266004979610443, + 0.08909663558006287, + -0.11199542880058289, + -1.8013991117477417, + -0.019768044352531433, + -0.024277430027723312, + -0.04220620170235634, + 0.033375561237335205, + 0.03389929234981537, + -0.1650262475013733, + -1.3381359577178955, + -0.12047871947288513 + ], + [ + -0.15009067952632904, + 0.05454045534133911, + 0.029337078332901, + -0.6240274906158447, + 0.6465781927108765, + -0.4041072428226471, + 0.3130103349685669, + 0.07021210342645645, + 0.09378987550735474, + -0.21047285199165344, + 0.11681286990642548, + -0.14802634716033936, + -1.8183677196502686, + 0.3068639636039734, + -0.010608140379190445, + -0.11435514688491821, + 0.07564397156238556, + 0.13713252544403076, + -0.13190573453903198, + -1.4660191535949707, + -0.04035669565200806 + ], + [ + 0.19345687329769135, + 0.036093391478061676, + 0.3382440209388733, + -0.9522699117660522, + 0.27695274353027344, + -1.1500868797302246, + 1.2254087924957275, + 0.09756684303283691, + 0.10196477919816971, + -0.11644334346055984, + 0.16047650575637817, + -0.19358840584754944, + -1.3692970275878906, + -0.3326490521430969, + -0.07179197669029236, + -0.18614377081394196, + 0.1879936307668686, + 0.07348190248012543, + -0.2861347198486328, + -1.5783159732818604, + -0.08773162961006165 + ] + ], + "generated_utilities": [ + 0.19993644952774048, + 0.23993994295597076, + 0.22335931658744812, + 0.23456653952598572, + 0.23101213574409485, + 0.2455206662416458, + 0.8128684163093567, + 0.14169609546661377 + ], + "group_id": "ms-72777024349c2aacd5e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8314920663833618, + 0.1595238745212555, + 2.0, + 0.16971935331821442, + 0.5892879366874695, + 0.1617850512266159, + 0.1877371072769165, + 0.5782577395439148, + 0.12074722349643707, + 0.11237785220146179, + 0.08792310953140259, + 0.1209893673658371, + 0.13739405572414398, + 0.0773405134677887, + 0.09497407078742981, + 0.047447383403778076 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.002835560590028763, + 0.3854118585586548, + -0.06168194115161896, + -0.16570186614990234, + -0.05051702260971069, + -0.43052077293395996, + 0.0, + 0.024141468107700348, + -0.011737043969333172, + -0.05502588301897049, + 0.013536334037780762, + -0.000911470502614975, + -1.6284780502319336, + 0.0, + 0.030459173023700714, + -0.05656937509775162, + 0.03393230587244034, + 0.06894856691360474, + 0.00450655072927475, + -1.2762084007263184, + 0.0 + ], + [ + 0.2191334217786789, + 0.3854118585586548, + 0.5048428177833557, + -0.16570186614990234, + -0.20026534795761108, + -0.43052077293395996, + 0.0, + -0.026082046329975128, + 0.03163425251841545, + -0.04950854927301407, + 0.1969367265701294, + -0.05762401595711708, + -1.6284780502319336, + 0.0, + 0.002695085946470499, + 0.34764212369918823, + 0.6051180362701416, + 0.9295560717582703, + 0.38285934925079346, + -1.2762084007263184, + 0.0 + ], + [ + -0.04624907672405243, + 1.3854118585586548, + 0.05864633619785309, + -1.1657018661499023, + 0.7997346520423889, + -1.43052077293396, + 1.1980153322219849, + -0.023269198834896088, + -0.026228107511997223, + -0.009256521239876747, + 0.6858966946601868, + 0.03280947729945183, + -2.6284780502319336, + -1.7920862436294556, + -0.0004090145230293274, + -0.00536443293094635, + 0.007614931557327509, + 0.7401981949806213, + 0.11622340977191925, + -2.2762084007263184, + -1.6813035011291504 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.3960306644439697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.584172487258911, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.362607002258301 + ], + [ + -0.13329119980335236, + 1.1171538829803467, + -0.19378040730953217, + -0.9341158866882324, + 0.8714147806167603, + -1.659934163093567, + 1.1658833026885986, + 0.23766514658927917, + 0.21810689568519592, + -0.05730185657739639, + 0.4946695566177368, + 0.17369553446769714, + -2.559515953063965, + -1.5605767965316772, + 0.08475297689437866, + 0.22485606372356415, + -0.2591644823551178, + 0.9637633562088013, + -0.08890970051288605, + -2.1015453338623047, + -1.5709789991378784 + ], + [ + -0.3507236838340759, + 1.1737966537475586, + 0.10341192036867142, + -0.7761039137840271, + 0.47493064403533936, + -1.2297213077545166, + 1.4181653261184692, + -0.1981543004512787, + -0.18248644471168518, + 0.17768341302871704, + 0.843591034412384, + -0.07443197071552277, + -2.6062240600585938, + -1.750958800315857, + -0.15503963828086853, + 0.18218432366847992, + 0.002181168645620346, + 0.8641887903213501, + -0.026101917028427124, + -1.9398812055587769, + -1.8561092615127563 + ], + [ + -0.22363319993019104, + 1.533349871635437, + 0.4454386830329895, + -1.5271000862121582, + 0.8945828080177307, + -1.105146884918213, + 0.7171626687049866, + -0.20207461714744568, + 0.31458696722984314, + 0.20214562118053436, + 1.0593571662902832, + -0.4655032455921173, + -2.8809642791748047, + -1.580764651298523, + -0.22098299860954285, + -0.08731096982955933, + -0.15165574848651886, + 0.89030921459198, + 0.611029326915741, + -2.2792654037475586, + -1.798964262008667 + ], + [ + -0.3702698349952698, + 0.9771865606307983, + 0.27564477920532227, + -1.7643829584121704, + 0.7298657894134521, + -1.875503420829773, + 1.4598875045776367, + 0.47600361704826355, + 0.13439032435417175, + -0.23905222117900848, + 1.2192013263702393, + -0.44705769419670105, + -2.603621482849121, + -1.3898506164550781, + 0.16782721877098083, + -0.2559890151023865, + 0.2965841293334961, + 0.8750308752059937, + 0.1521819680929184, + -2.3417282104492188, + -2.2663588523864746 + ], + [ + -0.31705379486083984, + 0.866703450679779, + -0.09125347435474396, + -1.5959886312484741, + 0.8041162490844727, + -1.7037310600280762, + 1.6304850578308105, + 0.11580469459295273, + 0.2654762864112854, + 0.12200001627206802, + 0.6904033422470093, + 0.19859254360198975, + -2.7615463733673096, + -2.059410333633423, + 0.5591834783554077, + -0.4801292419433594, + -0.3343847393989563, + 0.8787070512771606, + -0.14748375117778778, + -2.0334713459014893, + -1.1611082553863525 + ], + [ + -0.11324095726013184, + 0.8777624368667603, + 0.8215147256851196, + -1.5343353748321533, + 0.881126344203949, + -1.0607860088348389, + 0.5398488640785217, + 0.6020828485488892, + -0.5065795183181763, + 0.02798180840909481, + 0.33880186080932617, + 0.320717990398407, + -3.3552045822143555, + -1.1435439586639404, + 0.19279444217681885, + -0.5245973467826843, + 0.30647164583206177, + 1.3726601600646973, + 0.3123217523097992, + -2.9675674438476562, + -1.7007204294204712 + ], + [ + -0.6432619094848633, + 0.79607754945755, + -0.49864423274993896, + -1.0710605382919312, + 1.0709084272384644, + -1.612175703048706, + 1.2555015087127686, + 0.2873077392578125, + -0.36858057975769043, + 0.05837755650281906, + 1.1025807857513428, + 0.7949575185775757, + -3.2468795776367188, + -2.556741237640381, + -0.6579093337059021, + -0.8874800801277161, + 0.7922345399856567, + 0.4745541512966156, + 0.07610362768173218, + -2.041800022125244, + -2.2069954872131348 + ], + [ + -0.9264093637466431, + 1.9828436374664307, + 0.7907096147537231, + -1.260971188545227, + 1.7548933029174805, + -2.2685132026672363, + 1.1821955442428589, + -0.49968039989471436, + 0.10982713848352432, + 0.2384604662656784, + 0.2075445055961609, + -0.7872775793075562, + -2.076058864593506, + -1.1888585090637207, + 0.7447717785835266, + 0.2355569452047348, + -0.3163020610809326, + 0.3282558023929596, + -0.4075327515602112, + -3.2451019287109375, + -2.4086263179779053 + ], + [ + -0.7190575003623962, + 1.3702023029327393, + 0.8307373523712158, + -0.7533494234085083, + 0.574251651763916, + -2.3376822471618652, + 0.5512753129005432, + -0.04391670227050781, + 0.2242431342601776, + -0.46594029664993286, + 0.5667901039123535, + -0.12832416594028473, + -2.9870448112487793, + -1.2032369375228882, + 0.17313259840011597, + 0.5932936668395996, + 0.7428228259086609, + 0.6570835113525391, + -0.1891915649175644, + -1.3703107833862305, + -1.7630409002304077 + ], + [ + 0.799586832523346, + 1.0164330005645752, + -0.49519193172454834, + -0.6983597278594971, + 1.7632484436035156, + -0.9710203409194946, + 1.5416830778121948, + 0.9653489589691162, + 0.06498706340789795, + 0.2132291942834854, + -0.1650814414024353, + 0.06786654889583588, + -1.8782389163970947, + -2.3576126098632812, + 0.9410911798477173, + -0.576595664024353, + -0.8031080365180969, + 0.7120062112808228, + 0.6479252576828003, + -2.3928208351135254, + -1.4952012300491333 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.11725310236215591, + 0.3854118585586548, + 0.06966311484575272, + -0.16570186614990234, + 0.06442362070083618, + -0.43052077293395996, + 0.0, + 0.11858775466680527, + 0.0573427751660347, + -0.14938217401504517, + -0.01657569408416748, + 0.060478124767541885, + -1.6284780502319336, + 0.0, + 0.016584808006882668, + 0.18205130100250244, + -0.13064442574977875, + 0.15621143579483032, + 0.056775350123643875, + -1.2762084007263184, + 0.0 + ] + ], + "predicted_scores": [ + 2.7105655670166016, + 1.4601608514785767, + 1.3950504064559937, + 1.3776217699050903, + 1.21635103225708, + 0.9927445650100708, + 0.2479965090751648, + -0.32676640152931213 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "256770874", + "selected_index": 0, + "state_hash": "3a7aa4ba27a5b34cdeb0f31ce9818588b1dd4ede96bf546f07302616c2e9b3a6", + "stored_base_utility": 0.8314920663833618, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20372186601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20372186601161957 + }, + "base_progress": 0.20372186601161957, + "base_success": false, + "base_utility": 0.20372186601161957, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.23941433429718018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23941433429718018 + }, + { + "contact_quality": null, + "progress": 0.2004055678844452, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2004055678844452 + }, + { + "contact_quality": null, + "progress": 0.21979865431785583, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21979865431785583 + }, + { + "contact_quality": null, + "progress": 0.5798611640930176, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5798611640930176 + }, + { + "contact_quality": null, + "progress": 0.20053516328334808, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20053516328334808 + }, + { + "contact_quality": null, + "progress": 0.5538582801818848, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5538582801818848 + }, + { + "contact_quality": null, + "progress": 0.5764124393463135, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5764124393463135 + }, + { + "contact_quality": null, + "progress": 0.2330804020166397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2330804020166397 + } + ], + "candidate_progress": [ + 0.23941433429718018, + 0.2004055678844452, + 0.21979865431785583, + 0.5798611640930176, + 0.20053516328334808, + 0.5538582801818848, + 0.5764124393463135, + 0.2330804020166397 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-ecb37ebbab21a506d8ca", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "generated_tangents": [ + [ + -0.038756996393203735, + -0.13550937175750732, + -0.2540381848812103, + 0.33375242352485657, + 0.4462161660194397, + -0.5883610248565674, + 0.053501129150390625, + 0.07820597290992737, + 0.13572408258914948, + -0.07118258625268936, + -0.10641970485448837, + 0.1457289159297943, + -1.7172857522964478, + -0.010731622576713562, + -0.06439921259880066, + 0.042354196310043335, + 0.03466056287288666, + -0.004692539572715759, + -0.011387646198272705, + -1.3083875179290771, + -0.1264006495475769 + ], + [ + -0.1580553501844406, + -0.13328713178634644, + -0.19537189602851868, + 0.41709479689598083, + 0.3933730721473694, + -0.3096438944339752, + 0.07337459921836853, + 0.09225863218307495, + 0.08545009046792984, + -0.016713984310626984, + -0.21659067273139954, + 0.23247233033180237, + -1.602565884590149, + 0.052193187177181244, + 0.00322757288813591, + 0.06677122414112091, + 0.056176647543907166, + -0.03963209688663483, + -0.003780156373977661, + -1.3334197998046875, + -0.15710443258285522 + ], + [ + -0.14634673297405243, + -0.1670856475830078, + -0.1505264937877655, + 0.22932136058807373, + 0.46648484468460083, + -0.29467034339904785, + -0.030024707317352295, + 0.024578526616096497, + 0.10371974855661392, + -0.07803692668676376, + -0.17516982555389404, + 0.09712421149015427, + -1.818817377090454, + -0.0137137770652771, + 0.06714317202568054, + 0.05008891597390175, + 0.014884591102600098, + -0.0014598816633224487, + -0.03815457224845886, + -1.4960157871246338, + -0.1800711750984192 + ], + [ + -0.13525371253490448, + -0.21858741343021393, + -0.27348700165748596, + 0.4115125834941864, + 0.47482720017433167, + -0.3542366921901703, + 0.4323655068874359, + 0.07173498719930649, + 0.10754535347223282, + -0.1506441831588745, + -0.38532698154449463, + -0.014619454741477966, + -1.9378687143325806, + 0.0027394890785217285, + -0.14780347049236298, + 0.008556980639696121, + -0.02571706473827362, + -0.15478719770908356, + 0.15509457886219025, + -1.6268761157989502, + -0.2344937026500702 + ], + [ + -0.07550613582134247, + -0.14450731873512268, + -0.1280311644077301, + 0.2581559419631958, + 0.4942222833633423, + -0.2489149570465088, + -0.0064125657081604, + 0.0913766473531723, + 0.10777343064546585, + -0.060408152639865875, + -0.05955008417367935, + -0.009624660015106201, + -1.820928931236267, + -0.0243748277425766, + 0.004844978451728821, + 0.022344592958688736, + 0.12104836106300354, + -0.03878173232078552, + -0.03758198022842407, + -1.5328521728515625, + -0.18792152404785156 + ], + [ + 0.021220877766609192, + -0.24586030840873718, + -0.46019771695137024, + 0.3124466836452484, + 0.41348084807395935, + -0.2183077037334442, + 0.5402253270149231, + 0.1507055014371872, + 0.054333195090293884, + -0.17292925715446472, + -0.3890281021595001, + -0.02466900646686554, + -2.089529037475586, + -0.1611459106206894, + -0.1178891658782959, + 0.03982258588075638, + -0.05764146149158478, + -0.16406626999378204, + 0.03445519506931305, + -1.498215913772583, + -0.35786673426628113 + ], + [ + -0.17654405534267426, + -0.2302348017692566, + -0.4488113820552826, + 0.20745614171028137, + 0.5087523460388184, + -0.03024144470691681, + 0.3668936491012573, + -0.029347702860832214, + 0.14453846216201782, + -0.1354864239692688, + -0.3882730007171631, + -0.07349419593811035, + -2.1526310443878174, + -0.08613543212413788, + 0.04476073384284973, + 0.06055107340216637, + -0.021915599703788757, + -0.04606534540653229, + -0.059882551431655884, + -1.699329137802124, + -0.21382218599319458 + ], + [ + -0.308411180973053, + -0.23533658683300018, + -0.2005605399608612, + 0.31021276116371155, + 0.46984899044036865, + -0.2295137345790863, + 0.40720218420028687, + -0.0017099827527999878, + 0.1529761403799057, + -0.09738033264875412, + -0.293779194355011, + -0.05512559413909912, + -2.0737287998199463, + 0.09915192425251007, + -0.1006896048784256, + -0.04626504331827164, + 0.010933130979537964, + -0.12180183827877045, + -0.08540168404579163, + -1.6375348567962646, + -0.426442950963974 + ] + ], + "generated_utilities": [ + 0.23941433429718018, + 0.2004055678844452, + 0.21979865431785583, + 0.5798611640930176, + 0.20053516328334808, + 0.5538582801818848, + 0.5764124393463135, + 0.2330804020166397 + ], + "group_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8363267779350281, + 2.0, + 2.0, + 0.16206762194633484, + 0.5390529036521912, + 0.08151453733444214, + 0.5261932611465454, + 0.5265869498252869, + 0.5459305047988892, + 0.18360589444637299, + 0.16689825057983398, + 0.15426510572433472, + 0.0028923004865646362, + 0.15516109764575958, + 0.0506085604429245, + 0.19578850269317627 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.0005862116813659668, + 0.18783891201019287, + -0.02734696865081787, + 0.09201061725616455, + 0.010171383619308472, + -0.4856022596359253, + 0.0, + 0.10349468141794205, + 0.4567214846611023, + -0.1893925815820694, + 0.12298360466957092, + 0.3369159996509552, + -1.6010875701904297, + 0.0, + 0.304210901260376, + -0.3049007058143616, + 0.3435661494731903, + 0.3158445656299591, + 0.2931552231311798, + -1.2709691524505615, + 0.0 + ], + [ + -0.726529061794281, + -0.8121610879898071, + -1.0273469686508179, + 1.0920106172561646, + 0.3926190733909607, + -1.4856022596359253, + 1.8979665040969849, + -0.029249273240566254, + 0.008005961775779724, + 0.013417234644293785, + -0.10860864818096161, + 0.43491366505622864, + -2.6010875701904297, + -1.493168592453003, + -0.01047949492931366, + -0.0018908085767179728, + 0.019280990585684776, + 0.20726272463798523, + 0.45750224590301514, + -2.2709691524505615, + -1.4645930528640747 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7959330081939697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.986337184906006, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9291861057281494 + ], + [ + -0.9019005298614502, + -0.5901641845703125, + -0.920559287071228, + 0.991828978061676, + 0.2407551407814026, + -1.2655587196350098, + 1.7652356624603271, + 0.03909382224082947, + 0.04142814502120018, + 0.12125641852617264, + -0.2822783887386322, + 0.4936577081680298, + -2.5754144191741943, + -1.5686157941818237, + -0.22672900557518005, + -0.08431098610162735, + 0.09805622696876526, + 0.2479260265827179, + 0.4450404644012451, + -2.1972453594207764, + -1.4420135021209717 + ], + [ + -0.8822115063667297, + -0.9738405346870422, + -0.7449414730072021, + 0.8784142732620239, + 0.4583987295627594, + -1.3989503383636475, + 1.8331178426742554, + -0.006558071821928024, + 0.30383312702178955, + 0.2117801308631897, + -0.2538969814777374, + 0.13478082418441772, + -2.584805727005005, + -1.325882911682129, + 0.23593956232070923, + -0.2618964612483978, + -0.30780693888664246, + 0.24083535373210907, + 0.430472195148468, + -1.946203589439392, + -1.5077860355377197 + ], + [ + -0.39491432905197144, + -0.8895903825759888, + -0.9160035848617554, + 0.7220491170883179, + 0.36837494373321533, + -1.4429160356521606, + 1.4078493118286133, + 0.03969770669937134, + 0.007204015273600817, + 0.056278251111507416, + -0.4780465364456177, + 0.9121701717376709, + -2.512721538543701, + -1.6225675344467163, + -0.42074650526046753, + 0.15076839923858643, + -0.24544812738895416, + 0.08152849972248077, + 0.1987975835800171, + -2.712414264678955, + -1.9573136568069458 + ], + [ + -0.6879937648773193, + -0.8859440088272095, + -1.1152405738830566, + 0.8750674724578857, + -0.05955067276954651, + -1.5194538831710815, + 1.876329779624939, + -0.2068542242050171, + -0.5498799085617065, + 0.46419233083724976, + 0.42569708824157715, + -0.02127784490585327, + -2.5905418395996094, + -1.411050796508789, + -0.21327723562717438, + 0.20716798305511475, + 0.20993508398532867, + 0.33476364612579346, + -0.09872657060623169, + -2.5958731174468994, + -1.6374839544296265 + ], + [ + -0.5723236799240112, + -0.28514552116394043, + -0.6157487630844116, + 0.6151646375656128, + 0.32639649510383606, + -1.7217803001403809, + 2.5557403564453125, + 0.532052218914032, + 0.5977330207824707, + -0.5707970261573792, + -0.21707558631896973, + 0.8829224109649658, + -3.1287553310394287, + -1.0988328456878662, + 0.38229066133499146, + 0.0032835237216204405, + -0.3569248616695404, + 0.0031215548515319824, + 1.1060712337493896, + -2.3536932468414307, + -1.383341670036316 + ], + [ + -0.7901747822761536, + -0.8977189660072327, + -1.4548863172531128, + 0.5700533986091614, + -0.13018959760665894, + -0.8057670593261719, + 2.4019269943237305, + 0.5368256568908691, + 0.5227565765380859, + 0.28199371695518494, + -0.1637011021375656, + -0.2898564040660858, + -2.9142980575561523, + -1.8492696285247803, + 0.7177971601486206, + 0.2360217124223709, + -0.13862104713916779, + 0.9262527227401733, + 0.39495596289634705, + -2.470649480819702, + -1.3260751962661743 + ], + [ + -0.6053685545921326, + -0.3819047510623932, + -0.258634090423584, + 0.19338607788085938, + 0.19874510169029236, + -1.3407407999038696, + 2.5238022804260254, + -0.6922282576560974, + 0.8137098550796509, + 0.39421388506889343, + 0.09789760410785675, + 0.12572914361953735, + -2.518153667449951, + -1.6892600059509277, + 0.503250002861023, + 0.88390052318573, + -0.7520112991333008, + -0.3356115520000458, + 0.15291047096252441, + -2.5788943767547607, + -0.9403089284896851 + ], + [ + -0.464814156293869, + -0.6034223437309265, + -1.6505372524261475, + 0.6883972883224487, + -0.3758969306945801, + -1.785051941871643, + 2.605452060699463, + 0.550828218460083, + 0.4742209315299988, + -0.19333000481128693, + -1.002321481704712, + 0.48782211542129517, + -1.8015108108520508, + -0.8254865407943726, + -0.7333722710609436, + -0.4948355555534363, + 0.6016485095024109, + 0.5632485151290894, + -0.24367755651474, + -2.201673984527588, + -0.5806979537010193 + ], + [ + -1.4628195762634277, + -0.4433446228504181, + -1.9926375150680542, + 0.33977705240249634, + 0.03379333019256592, + -1.6042804718017578, + 2.221484422683716, + 0.7110810279846191, + -0.41500210762023926, + 0.08095695078372955, + -0.23683495819568634, + -0.43207624554634094, + -3.080759286880493, + -0.9071659445762634, + 0.47082334756851196, + -0.9117310047149658, + -0.06435621529817581, + -0.6813669204711914, + -0.2741369605064392, + -2.3763599395751953, + -1.000203013420105 + ], + [ + -1.382751226425171, + -1.7026821374893188, + -0.11846715211868286, + 1.5398613214492798, + -0.5885447859764099, + -1.8825289011001587, + 1.5456308126449585, + -0.320659339427948, + -0.964028000831604, + -0.1812019646167755, + 0.14226128160953522, + 0.4527861177921295, + -2.714468002319336, + -1.9451607465744019, + 0.195057213306427, + 0.16905252635478973, + -0.9247990250587463, + 0.9112424850463867, + 0.79449862241745, + -2.714893341064453, + -1.6473028659820557 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.02016383409500122, + 0.02509528398513794, + -0.02734696865081787, + 0.09201061725616455, + -0.04331648349761963, + -0.4856022596359253, + 0.0, + -0.022040890529751778, + -0.003832761198282242, + -0.01408458687365055, + 0.06165061891078949, + 0.02554956078529358, + -1.6010875701904297, + 0.0, + 0.009550465270876884, + 0.03566932678222656, + 0.013732164166867733, + 0.017749279737472534, + 0.0699148178100586, + -1.2709691524505615, + 0.0 + ], + [ + 0.023195505142211914, + 0.029890060424804688, + -0.02734696865081787, + 0.09201061725616455, + 0.06108534336090088, + -0.4856022596359253, + 0.0, + -0.034551952034235, + -0.15152660012245178, + -0.21811580657958984, + -0.063543401658535, + 0.23667243123054504, + -1.6010875701904297, + 0.0, + -0.1614403873682022, + -0.17222382128238678, + -0.09706380218267441, + 0.3660350441932678, + 0.13013160228729248, + -1.2709691524505615, + 0.0 + ] + ], + "predicted_scores": [ + 5.028995513916016, + 4.867879390716553, + 4.800228118896484, + 4.730734348297119, + 4.398209571838379, + 4.280922889709473, + 3.719674587249756, + 3.216965436935425 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "957101489", + "selected_index": 0, + "state_hash": "e8473553a2b17f5adf09e08e7271b5502d53a51225c17abac3d9f78906cf8c46", + "stored_base_utility": 0.8363267779350281, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8431589603424072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8431589603424072 + }, + "base_progress": 0.8431589603424072, + "base_success": false, + "base_utility": 0.8431589603424072, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.2240862250328064, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2240862250328064 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.23456276953220367, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23456276953220367 + }, + { + "contact_quality": null, + "progress": 0.22421211004257202, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22421211004257202 + }, + { + "contact_quality": null, + "progress": 0.23834753036499023, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23834753036499023 + }, + { + "contact_quality": null, + "progress": 0.23887677490711212, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23887677490711212 + }, + { + "contact_quality": null, + "progress": 0.2396199107170105, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2396199107170105 + }, + { + "contact_quality": null, + "progress": 0.21505582332611084, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21505582332611084 + } + ], + "candidate_progress": [ + 0.2240862250328064, + 1.0, + 0.23456276953220367, + 0.22421211004257202, + 0.23834753036499023, + 0.23887677490711212, + 0.2396199107170105, + 0.21505582332611084 + ], + "candidate_source_chart_ids": [ + "ms-5bddb39f792f20e5e6d4", + "ms-9228f8295b9891cf011c", + "ms-5bddb39f792f20e5e6d4", + "ms-9d12ad4ebdf37419994c", + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-ce9505086ea115f86649", + "ms-9d12ad4ebdf37419994c" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9a14552b7db695dee92c", + "generated_tangents": [ + [ + -0.24398012459278107, + -0.31837475299835205, + -0.39673200249671936, + 0.8768613338470459, + 0.41880494356155396, + -0.886460542678833, + 0.2791355848312378, + 0.07774002850055695, + -0.05111357569694519, + -0.166623055934906, + -0.20629540085792542, + -0.10491454601287842, + -2.0003256797790527, + -0.25472474098205566, + -0.12438620626926422, + -0.10710087418556213, + -0.08980457484722137, + 0.03481768071651459, + -0.139867901802063, + -1.9910016059875488, + -0.5777000188827515 + ], + [ + -0.011119157075881958, + -0.24807676672935486, + -0.0015007555484771729, + 0.6639519929885864, + 0.3065972924232483, + -1.6503907442092896, + 0.07185950875282288, + -0.008339866995811462, + 0.026413122192025185, + -0.061842434108257294, + -0.04869193583726883, + 0.01619863510131836, + -1.5339487791061401, + -0.15631748735904694, + -0.08074803650379181, + -0.008532043546438217, + 0.04662767052650452, + 0.07369226217269897, + -0.2059488296508789, + -1.5359117984771729, + -0.08076417446136475 + ], + [ + -0.061953917145729065, + -0.32794779539108276, + -0.5045595169067383, + 0.786717414855957, + 0.33011600375175476, + -0.819882869720459, + 0.48401767015457153, + 0.1185966208577156, + -0.08473946899175644, + -0.11563808470964432, + -0.2017354667186737, + -0.15378880500793457, + -2.1040890216827393, + -0.30424827337265015, + -0.11217138171195984, + -0.05279594287276268, + -0.10907794535160065, + 0.04559807479381561, + -0.23217111825942993, + -1.800889492034912, + -0.6683998107910156 + ], + [ + -0.036740511655807495, + -0.2286078780889511, + -0.1605977714061737, + 0.5439430475234985, + 0.2564154267311096, + -1.5536032915115356, + 0.047141581773757935, + 0.03982469439506531, + 0.010757035575807095, + -0.0548592135310173, + -0.14562514424324036, + 0.12549127638339996, + -1.5212243795394897, + -0.11536450684070587, + -0.12896764278411865, + -0.032481104135513306, + 0.01559726893901825, + 0.10565237700939178, + -0.29879236221313477, + -1.5924403667449951, + 0.10036706179380417 + ], + [ + -0.037515848875045776, + -0.178071066737175, + -0.2516103684902191, + 0.6320818662643433, + 0.348877489566803, + -1.0222082138061523, + -0.011195868253707886, + 0.03477014601230621, + 0.04416806995868683, + -0.07414370030164719, + -0.0005400404334068298, + -0.012319117784500122, + -1.7142136096954346, + -0.11643184721469879, + -0.1803487241268158, + -0.03259944170713425, + -0.005131468176841736, + 0.12992960214614868, + -0.1465526521205902, + -1.624044418334961, + -0.26597124338150024 + ], + [ + -0.1794966608285904, + -0.1724695861339569, + -0.2557346522808075, + 0.7137835025787354, + 0.21937555074691772, + -0.8046422004699707, + -0.026013940572738647, + 0.05393823981285095, + 0.02173120155930519, + -0.011962853372097015, + -0.07620421797037125, + 0.0318894237279892, + -1.587060570716858, + -0.11730077862739563, + -0.10359573364257812, + -0.03643292933702469, + -0.018543973565101624, + 0.11555609107017517, + -0.16775581240653992, + -1.6180734634399414, + -0.24911972880363464 + ], + [ + -0.1586340218782425, + -0.16423314809799194, + -0.2150205671787262, + 0.77131187915802, + 0.4593883752822876, + -1.0750209093093872, + 0.08453720808029175, + 0.035815492272377014, + 0.035750649869441986, + -0.10813844949007034, + -0.026688888669013977, + -0.0509973019361496, + -2.2673110961914062, + -0.061440303921699524, + -0.0829656720161438, + -0.07193587720394135, + -0.03457425534725189, + 0.07937327027320862, + -0.21057280898094177, + -1.794555902481079, + -0.21171388030052185 + ], + [ + 0.01266525685787201, + -0.2515493631362915, + -0.032141655683517456, + 0.6023002862930298, + 0.32514604926109314, + -1.6528481245040894, + 0.06609141826629639, + -0.0161876380443573, + 0.010724145919084549, + -0.06089041382074356, + 0.013896647840738297, + 0.0018012821674346924, + -1.564214825630188, + -0.09429937601089478, + -0.04476979002356529, + -0.05024196580052376, + 0.016884565353393555, + 0.0802685022354126, + -0.22384709119796753, + -1.529163122177124, + -0.02373480796813965 + ] + ], + "generated_utilities": [ + 0.2240862250328064, + 2.0, + 0.23456276953220367, + 0.22421211004257202, + 0.23834753036499023, + 0.23887677490711212, + 0.2396199107170105, + 0.21505582332611084 + ], + "group_id": "ms-9a14552b7db695dee92c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.838127613067627, + 2.0, + 0.15671375393867493, + 0.17482513189315796, + 0.5684618353843689, + 0.19147612154483795, + 0.5643423795700073, + 0.1653006672859192, + 0.11670731008052826, + 0.14578887820243835, + 0.16058449447155, + 0.10923929512500763, + 0.13408902287483215, + 0.06845971941947937, + 0.02954423427581787, + 0.08425161242485046 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.22581616044044495, + -0.32600927352905273, + -0.054001130163669586, + 1.2553811073303223, + -0.1383839249610901, + -1.9753923416137695, + 0.0, + 0.09536236524581909, + 0.017635054886341095, + -0.06946105509996414, + 0.06630897521972656, + -0.10150641947984695, + -1.9112331867218018, + 0.0, + -0.23092225193977356, + 0.04748937487602234, + 0.3013715445995331, + -0.22072738409042358, + 0.0733548104763031, + -1.6514825820922852, + 0.0 + ], + [ + 0.22913068532943726, + -0.32600927352905273, + 0.24692901968955994, + 1.2553811073303223, + 0.3704858124256134, + -1.9753923416137695, + 0.0, + 0.036500319838523865, + 0.12820330262184143, + -0.09674224257469177, + 0.2668613791465759, + -0.14372897148132324, + -1.9112331867218018, + 0.0, + -0.27894535660743713, + -0.3322452902793884, + 0.16300754249095917, + -0.24123603105545044, + -0.2359180599451065, + -1.6514825820922852, + 0.0 + ], + [ + 0.0198141410946846, + -1.3260092735290527, + 0.011403528973460197, + 2.2553811073303223, + 0.5258303284645081, + -2.9753923416137695, + 1.8956809043884277, + -0.028990529477596283, + 0.007858656346797943, + 0.03128144145011902, + 0.5457771420478821, + -0.14793801307678223, + -2.9112331867218018, + -1.5743238925933838, + -0.011861175298690796, + -0.002163209253922105, + 0.015663353726267815, + 0.7587639689445496, + -0.07801014184951782, + -2.651482582092285, + -1.5696079730987549 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7913618087768555, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1486477851867676, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1392159461975098 + ], + [ + 0.13539990782737732, + -1.2862521409988403, + 0.28441768884658813, + 2.3658432960510254, + 0.4678441286087036, + -2.8791494369506836, + 1.8408029079437256, + -0.11893181502819061, + -0.1259741485118866, + -0.20943574607372284, + 0.4765416979789734, + -0.40862777829170227, + -2.8372273445129395, + -1.4697245359420776, + -0.08606886118650436, + -0.06554404646158218, + -0.27424973249435425, + 0.907739520072937, + -0.10130079835653305, + -2.6951611042022705, + -1.807652235031128 + ], + [ + -0.15379676222801208, + -1.7091000080108643, + 0.31153741478919983, + 2.2179949283599854, + 0.859725832939148, + -2.6038026809692383, + 1.98358952999115, + 0.22934654355049133, + -0.16202330589294434, + 0.02916553243994713, + 0.7702362537384033, + -0.0907563641667366, + -2.6495683193206787, + -1.394291877746582, + 0.2534489035606384, + -0.07077484577894211, + 0.10660970956087112, + 0.7866172194480896, + -0.41323983669281006, + -2.4847500324249268, + -1.5881152153015137 + ], + [ + 0.14051103591918945, + -0.8684865236282349, + 0.45214012265205383, + 2.499347686767578, + 0.310441792011261, + -2.875823497772217, + 1.4898443222045898, + 0.3396439850330353, + -0.033532168716192245, + 0.20189781486988068, + 0.3506202697753906, + 0.04831346869468689, + -2.4730942249298096, + -1.4369138479232788, + -0.4840375483036041, + -0.17814871668815613, + -0.1863224357366562, + 0.47034892439842224, + -0.5315045714378357, + -3.019744396209717, + -1.2571722269058228 + ], + [ + 0.5503659844398499, + -1.517655372619629, + -0.20272362232208252, + 2.5588791370391846, + 0.7160627841949463, + -2.551017999649048, + 2.423107385635376, + 0.18953359127044678, + -0.4268995523452759, + 0.014377228915691376, + 0.969605028629303, + -0.2440275251865387, + -2.385143756866455, + -1.4786417484283447, + 0.44389280676841736, + -0.11543146520853043, + -0.5315021872520447, + 1.2627880573272705, + 0.24797001481056213, + -3.17976713180542, + -1.8759949207305908 + ], + [ + -0.07551594823598862, + -1.6101253032684326, + 0.3926098048686981, + 1.726672649383545, + 1.0158472061157227, + -3.367068290710449, + 1.4499058723449707, + -0.5616514086723328, + 0.002434568479657173, + -0.5033466815948486, + -0.12249338626861572, + -0.07335222512483597, + -3.5959439277648926, + -1.0662813186645508, + -0.034436360001564026, + 0.512275218963623, + -0.04822078347206116, + 0.6117934584617615, + -0.745093822479248, + -2.2700355052948, + -2.090803623199463 + ], + [ + 0.6751457452774048, + -0.7857800126075745, + 0.11759950965642929, + 2.7748122215270996, + 1.3096001148223877, + -2.3352341651916504, + 1.4677345752716064, + 0.7219796180725098, + 0.21788907051086426, + -0.3245394825935364, + 0.08372032642364502, + -0.6393120288848877, + -2.8002729415893555, + -2.157836437225342, + 0.11339983344078064, + -0.2834126651287079, + -0.3824191987514496, + 1.130112648010254, + -0.8028184175491333, + -2.3846678733825684, + -1.8259286880493164 + ], + [ + -0.01608770713210106, + -2.1245148181915283, + -0.175740584731102, + 2.754368782043457, + 1.0032098293304443, + -3.7700157165527344, + 1.3539870977401733, + -0.7906507849693298, + -0.8311541080474854, + 0.8180835247039795, + 0.10450628399848938, + 0.01135176420211792, + -3.396463394165039, + -0.6868918538093567, + 0.22346650063991547, + 0.693323016166687, + -0.47104692459106445, + 0.8188157081604004, + -0.6246469020843506, + -2.4006571769714355, + -2.431541919708252 + ], + [ + -0.7931793928146362, + -1.7645766735076904, + 0.23125068843364716, + 2.816321849822998, + 1.1080517768859863, + -3.8820090293884277, + 1.8420056104660034, + 0.4984684884548187, + -0.9149360656738281, + -0.028868291527032852, + 0.14621934294700623, + 0.546608567237854, + -2.7883894443511963, + -1.3650472164154053, + -0.055590901523828506, + 0.41940492391586304, + -0.7235082983970642, + 1.1734092235565186, + -0.6126058101654053, + -3.217271566390991, + -2.512387275695801 + ], + [ + -0.5335642695426941, + -2.139784336090088, + 0.9540140628814697, + 1.7781559228897095, + 1.2117886543273926, + -3.71681809425354, + 2.461766242980957, + 0.4546531140804291, + 0.09786976873874664, + -0.8281612396240234, + -0.19971781969070435, + -1.1042157411575317, + -3.047807216644287, + -1.0697336196899414, + -0.22256404161453247, + -0.3800235986709595, + 0.3603542447090149, + 0.7543611526489258, + -0.23184219002723694, + -3.5156431198120117, + -2.1170644760131836 + ], + [ + 0.27338743209838867, + -0.9654052257537842, + -0.2700650095939636, + 2.6957814693450928, + 0.0643424391746521, + -3.094449281692505, + 0.8981494307518005, + 0.19603312015533447, + 0.5969407558441162, + -0.45200687646865845, + 1.466981291770935, + -0.45456382632255554, + -2.997983455657959, + -1.1903986930847168, + 0.5722758769989014, + -0.9932370781898499, + 0.28074830770492554, + 0.6244409084320068, + 0.6411240696907043, + -3.2179243564605713, + -1.905179738998413 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.025789255276322365, + -0.32600927352905273, + -0.02888326905667782, + 1.2553811073303223, + 0.013092517852783203, + -1.9753923416137695, + 0.0, + 0.04008308798074722, + -0.05201057717204094, + -0.008613351732492447, + -0.06225985288619995, + -0.0397564172744751, + -1.9112331867218018, + 0.0, + -0.02836347185075283, + -0.0055325529538095, + 0.053034864366054535, + 0.009740114212036133, + -0.006228044629096985, + -1.6514825820922852, + 0.0 + ] + ], + "predicted_scores": [ + 5.9339399337768555, + 5.8429741859436035, + 5.83306884765625, + 5.777254581451416, + 5.714264392852783, + 5.678617000579834, + 5.648623943328857, + 5.636460304260254 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264", + "selected_index": 0, + "state_hash": "49d38eabb7404e0028c75dd98e1239274b5dc2f04b9db8653001e44369ca8f3a", + "stored_base_utility": 0.838127613067627, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8326728343963623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8326728343963623 + }, + "base_progress": 0.8326728343963623, + "base_success": false, + "base_utility": 0.8326728343963623, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8636302947998047, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8636302947998047 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.8636302947998047, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-c80e71e55d51d8595499", + "ms-5dfbb16e173e1a3e2bd6", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-19a3669e096e98232ec8", + "ms-c80e71e55d51d8595499", + "ms-19a3669e096e98232ec8" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-9f5d57fe2153fd267774", + "generated_tangents": [ + [ + 0.08413292467594147, + -0.08230706304311752, + -0.1254267394542694, + 0.2186259925365448, + -0.004199966788291931, + -1.5646209716796875, + 0.17910218238830566, + -0.059523820877075195, + -0.042556025087833405, + -0.08612646907567978, + 0.30891841650009155, + 0.00041578710079193115, + -1.151727557182312, + -0.004442453384399414, + -0.20556923747062683, + -0.05447538197040558, + 0.06253249943256378, + 0.21837827563285828, + -0.07886528968811035, + -0.8268441557884216, + -0.10379719734191895 + ], + [ + 0.005435451865196228, + -0.15224504470825195, + 0.2607381045818329, + 0.0070578306913375854, + -0.23332613706588745, + -1.6677610874176025, + 0.15069997310638428, + 0.02010825276374817, + -0.03953012824058533, + -0.14700442552566528, + 0.18694975972175598, + 0.08739671856164932, + -1.0308794975280762, + 0.17629662156105042, + 0.01297353208065033, + -0.11081932485103607, + -0.05898500978946686, + 0.36896753311157227, + 0.12984052300453186, + -0.7175573110580444, + -0.30426302552223206 + ], + [ + 0.06530517339706421, + -0.10118457674980164, + -0.09261497855186462, + 0.12752801179885864, + 0.029137805104255676, + -1.4337760210037231, + 0.16093605756759644, + 0.010899156332015991, + -0.05387471616268158, + -0.03502560406923294, + 0.2865021824836731, + -0.025219827890396118, + -1.016312599182129, + 0.0006787031888961792, + -0.11459082365036011, + -0.06721201539039612, + 0.22419260442256927, + 0.14956511557102203, + -0.07550835609436035, + -0.8552228212356567, + -0.09761044383049011 + ], + [ + -0.010899603366851807, + -0.09557810425758362, + -0.07494232058525085, + 0.11577621102333069, + -0.04265552759170532, + -1.4771164655685425, + 0.13611218333244324, + -0.030279889702796936, + -0.07786482572555542, + -0.1113763228058815, + 0.2983352243900299, + -0.02195592224597931, + -1.1468371152877808, + -0.09548032283782959, + -0.14314137399196625, + -0.05801401287317276, + 0.13833624124526978, + 0.19742122292518616, + -0.14581972360610962, + -0.8729411363601685, + -0.09893390536308289 + ], + [ + 0.04760611802339554, + -0.04970837011933327, + -0.025667399168014526, + 0.2508976459503174, + -0.04721599817276001, + -1.3973768949508667, + 0.19110479950904846, + -0.019008412957191467, + -0.013932688161730766, + -0.08488116413354874, + 0.2985996603965759, + 0.07291415333747864, + -0.9242931604385376, + 0.02658630907535553, + -0.12224647402763367, + -0.06941595673561096, + 0.1371287703514099, + 0.1649552285671234, + -0.05273208022117615, + -0.8101409077644348, + -0.06959079205989838 + ], + [ + 0.011269211769104004, + -0.09086939692497253, + -0.10519930720329285, + 0.167340487241745, + -0.06838038563728333, + -1.5224459171295166, + 0.16651907563209534, + -0.0974019467830658, + -0.057201795279979706, + -0.12136388570070267, + 0.19467008113861084, + -0.015901654958724976, + -1.0124092102050781, + -0.044310301542282104, + -0.10065591335296631, + -0.06727073341608047, + 0.19224049150943756, + 0.1459444910287857, + -0.2303667664527893, + -0.8499047160148621, + -0.06792102754116058 + ], + [ + 0.16562232375144958, + -0.10833266377449036, + 0.09600090980529785, + -0.1374075561761856, + 0.30824172496795654, + -1.4698164463043213, + -0.30176299810409546, + -0.06370925903320312, + 0.010365153662860394, + -0.0443614199757576, + 0.32736533880233765, + -0.3612250089645386, + -1.0171478986740112, + -0.5918875932693481, + -0.16154873371124268, + -0.059748485684394836, + 0.29379719495773315, + 0.39021843671798706, + -0.05616903305053711, + -1.054394006729126, + 0.27992331981658936 + ], + [ + 0.04429534822702408, + -0.108091801404953, + 0.08059525489807129, + 0.2930544316768646, + -0.06860846281051636, + -1.504648208618164, + 0.1273336410522461, + -0.021768197417259216, + 0.005642636679112911, + -0.08264502137899399, + 0.25489139556884766, + 0.10974524915218353, + -0.8973052501678467, + -0.04159153997898102, + -0.042293477803468704, + -0.12202993035316467, + 0.050917476415634155, + 0.17859452962875366, + -0.14225858449935913, + -0.8196880221366882, + -0.11021232604980469 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.8636302947998047, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-9f5d57fe2153fd267774", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8544998168945312, + 2.0, + 0.14427810907363892, + 0.1566883623600006, + 0.7847082018852234, + 0.6161996126174927, + 0.5900160074234009, + 0.12891459465026855, + 0.5811972618103027, + 0.12705880403518677, + 0.12002001702785492, + 0.07508957386016846, + 0.1335657238960266, + 2.0, + 0.5254277586936951, + 0.1602712869644165 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.07957560569047928, + 0.1778574287891388, + -0.06101015955209732, + 0.08770006895065308, + -0.13665570318698883, + -1.1242008209228516, + 0.0, + 0.20265339314937592, + 0.03117954358458519, + -0.08283126354217529, + 0.051615357398986816, + 0.006725620478391647, + -0.7304134368896484, + 0.0, + -0.03756266087293625, + 0.051428791135549545, + 0.13959813117980957, + 0.040047526359558105, + -0.04084143042564392, + -0.31500017642974854, + 0.0 + ], + [ + 0.31541678309440613, + -0.16606420278549194, + 0.09380461275577545, + 0.031209349632263184, + -0.10083025693893433, + -1.1242008209228516, + 0.0, + 0.09663133323192596, + -0.095388263463974, + 0.4012776017189026, + 0.051615357398986816, + 0.167547807097435, + -0.7304134368896484, + 0.0, + -0.6302167177200317, + 0.05524101108312607, + -0.14198890328407288, + 0.040047526359558105, + 0.5201720595359802, + -0.31500017642974854, + 0.0 + ], + [ + 0.004154659807682037, + -0.0034282731357961893, + 0.005892834160476923, + 1.0312093496322632, + -0.15828494727611542, + -2.1242008209228516, + -1.4059314727783203, + 0.002088826149702072, + 0.0016306412871927023, + 0.005124530289322138, + 1.0516153573989868, + -0.05936579033732414, + -1.7304134368896484, + -1.2081255912780762, + 0.010161396116018295, + 0.0017865521367639303, + -0.006222852971404791, + 1.040047526359558, + 0.07346899807453156, + -1.3150001764297485, + -0.9752891659736633 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.8118629455566406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.4162511825561523, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9505783319473267 + ], + [ + 0.05903958901762962, + -0.05713164061307907, + 0.27611011266708374, + 0.9773679971694946, + -0.2122824341058731, + -2.198127031326294, + -1.1234419345855713, + 0.22609303891658783, + 0.14432841539382935, + 0.13455358147621155, + 0.8475942611694336, + 0.19007176160812378, + -1.4536203145980835, + -1.208316445350647, + -0.24895474314689636, + 0.08610261976718903, + 0.16216596961021423, + 1.2356702089309692, + 0.07108174264431, + -1.4601069688796997, + -1.0005295276641846 + ], + [ + -0.2862337827682495, + -0.34814661741256714, + -0.3317800462245941, + 1.383265495300293, + -0.3485583961009979, + -1.7799506187438965, + -1.6264256238937378, + -0.24356596171855927, + -0.18337340652942657, + 0.3351142108440399, + 1.36115562915802, + -0.4404119849205017, + -1.4858057498931885, + -0.9229021668434143, + 0.3078569769859314, + -0.18743832409381866, + -0.3414495885372162, + 1.0045359134674072, + -0.1438894122838974, + -1.5133963823318481, + -0.7331042885780334 + ], + [ + 0.14600041508674622, + -0.10555070638656616, + 0.06090353801846504, + 1.1715545654296875, + 0.14829085767269135, + -1.8377182483673096, + -1.2019046545028687, + -0.33082208037376404, + 0.27619102597236633, + 0.3936208188533783, + 1.4077775478363037, + -0.2165551781654358, + -2.094696283340454, + -0.7843748331069946, + -0.13239514827728271, + 0.208020880818367, + 0.17500364780426025, + 0.5608562231063843, + 0.5641188025474548, + -1.6192113161087036, + -1.1969176530838013 + ], + [ + -0.03659747168421745, + -0.2400680035352707, + 0.49718841910362244, + 1.0422266721725464, + 0.13088186085224152, + -2.068427324295044, + -1.1215412616729736, + 0.06697356700897217, + 0.19986751675605774, + -0.5827334523200989, + 0.9792195558547974, + -0.5810520648956299, + -1.7176907062530518, + -0.6760571002960205, + -0.08971267938613892, + 0.36716267466545105, + 0.4829793870449066, + 0.7045803070068359, + -0.3341261148452759, + -1.3750282526016235, + -1.086517095565796 + ], + [ + 0.09393255412578583, + 0.6277099251747131, + 0.653377115726471, + 0.5602235198020935, + -0.8483365178108215, + -1.7264833450317383, + -1.698084831237793, + -0.3303340971469879, + 0.43070027232170105, + -0.13159427046775818, + 1.0274523496627808, + 0.4760425388813019, + -2.1360630989074707, + -0.6506720185279846, + 0.3130929470062256, + 0.6801184415817261, + 0.6575731039047241, + 0.9053056240081787, + 0.18805859982967377, + -1.3225764036178589, + -1.0653810501098633 + ], + [ + 0.536294162273407, + -0.026106586679816246, + 0.11593607813119888, + 0.2775042653083801, + 0.1801595836877823, + -1.590940237045288, + -0.6846873164176941, + 0.5080465078353882, + -0.7678324580192566, + 0.24115782976150513, + 0.2675415277481079, + -0.3879689872264862, + -2.1869571208953857, + -0.7176487445831299, + -0.15474224090576172, + -0.5326727032661438, + -0.12406099587678909, + 0.411626398563385, + 0.3519209027290344, + -1.6924495697021484, + -1.4648362398147583 + ], + [ + 0.2688097357749939, + -0.7928301095962524, + -0.39608004689216614, + 1.3961963653564453, + -0.026196151971817017, + -2.263073682785034, + -2.044499635696411, + 0.055598195642232895, + -0.485431969165802, + -0.058985911309719086, + 1.781836986541748, + 0.7104123830795288, + -2.0781984329223633, + -0.6968748569488525, + -0.1437186300754547, + 0.8155919313430786, + 0.5188283920288086, + 0.23954623937606812, + -0.6596511602401733, + -1.3867920637130737, + -0.7448391318321228 + ], + [ + 0.11259422451257706, + 0.43877506256103516, + -0.585435152053833, + 0.12099790573120117, + 0.5070493817329407, + -2.101219654083252, + -1.3759371042251587, + -0.7516231536865234, + 0.7018498778343201, + 0.7899036407470703, + 0.9772143959999084, + -0.7032701373100281, + -1.4699928760528564, + -1.4471495151519775, + 0.4750489890575409, + 0.8803708553314209, + -0.25032687187194824, + 1.3429648876190186, + 0.7985087037086487, + -0.5786607265472412, + -1.563289761543274 + ], + [ + 0.4965847432613373, + -0.8452511429786682, + 0.21300016343593597, + 0.28797537088394165, + -1.0059970617294312, + -3.040114402770996, + -1.1050269603729248, + 0.7826927304267883, + 0.8645586371421814, + -0.9447845816612244, + 1.0121192932128906, + 0.5123239755630493, + -1.5470938682556152, + -0.6170141100883484, + 0.2538997530937195, + 0.912939727306366, + -0.8613666892051697, + 0.9517492055892944, + 0.7185115218162537, + -2.1534924507141113, + -0.6818757057189941 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025899745523929596, + -0.03433045744895935, + -0.044045113027095795, + 0.031209349632263184, + -0.014532476663589478, + -1.1242008209228516, + 0.0, + 0.10768167674541473, + -0.020356399938464165, + 0.09216367453336716, + 0.051615357398986816, + -0.011370409280061722, + -0.7304134368896484, + 0.0, + 0.04558054730296135, + 0.038496289402246475, + 0.02504396252334118, + 0.040047526359558105, + 0.01217309758067131, + -0.31500017642974854, + 0.0 + ], + [ + -0.5198070406913757, + -0.8798871040344238, + 0.04596509411931038, + 1.55931556224823, + 0.31027060747146606, + -1.5283797979354858, + -1.6053413152694702, + -0.2754230797290802, + -0.012456716038286686, + 0.7998818159103394, + 1.5184144973754883, + 0.05059843137860298, + -1.85283362865448, + -0.9533926248550415, + 0.9978112578392029, + 0.8525571227073669, + -0.9611027836799622, + 0.32179969549179077, + -0.448661744594574, + -1.9452965259552002, + -0.7132223844528198 + ] + ], + "predicted_scores": [ + 3.505180835723877, + 3.2975637912750244, + 3.262356996536255, + 3.181797504425049, + 3.159857749938965, + 3.157637357711792, + 3.1126620769500732, + 2.846342086791992 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1607773860", + "selected_index": 0, + "state_hash": "fdc7379ab4491c9f65860cadd3b2f3c9f2335f26aae91fb1c06d3a4edf858d6a", + "stored_base_utility": 0.8544998168945312, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8272150158882141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8272150158882141 + }, + "base_progress": 0.8272150158882141, + "base_success": false, + "base_utility": 0.8272150158882141, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8165659308433533, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8165659308433533 + }, + { + "contact_quality": null, + "progress": 0.21969212591648102, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21969212591648102 + }, + { + "contact_quality": null, + "progress": 0.20318710803985596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20318710803985596 + }, + { + "contact_quality": null, + "progress": 0.21245209872722626, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21245209872722626 + }, + { + "contact_quality": null, + "progress": 0.20310749113559723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20310749113559723 + }, + { + "contact_quality": null, + "progress": 0.24723665416240692, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24723665416240692 + }, + { + "contact_quality": null, + "progress": 0.8245300054550171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8245300054550171 + }, + { + "contact_quality": null, + "progress": 0.20159128308296204, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20159128308296204 + } + ], + "candidate_progress": [ + 0.8165659308433533, + 0.21969212591648102, + 0.20318710803985596, + 0.21245209872722626, + 0.20310749113559723, + 0.24723665416240692, + 0.8245300054550171, + 0.20159128308296204 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-20f3cfcf183c69853b1e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b0fee26e1db967f90b81", + "generated_tangents": [ + [ + -0.016336917877197266, + -0.013360157608985901, + -0.07621291279792786, + -0.16371239721775055, + 0.0668400228023529, + -1.894248366355896, + 0.008704543113708496, + -0.06707262992858887, + 0.0719880759716034, + -0.15066036581993103, + -0.08019492775201797, + -0.0692489743232727, + -1.5693426132202148, + -0.240192249417305, + -0.0804758071899414, + -0.01940889097750187, + 0.033196404576301575, + 0.04620952904224396, + -0.1469961702823639, + -1.5181262493133545, + -0.29363399744033813 + ], + [ + 0.027893297374248505, + -0.008285533636808395, + -0.08378562331199646, + -0.17221945524215698, + 0.21182218194007874, + -1.9948073625564575, + 0.03760448098182678, + 0.035435259342193604, + 0.0717722699046135, + -0.08506845682859421, + 0.04238452389836311, + -0.03463172912597656, + -1.7876969575881958, + -0.13388626277446747, + -0.0628129243850708, + -0.01397827547043562, + 0.034210845828056335, + 0.04611770808696747, + -0.11658340692520142, + -1.548569917678833, + -0.21235740184783936 + ], + [ + 0.041436389088630676, + -0.07316534221172333, + -0.045554667711257935, + -0.22406762838363647, + 0.07410885393619537, + -1.6889369487762451, + -0.057614028453826904, + 0.10750889033079147, + 0.03681034594774246, + -0.06785174459218979, + 0.005821295082569122, + -0.10103631019592285, + -1.4945741891860962, + -0.16003330051898956, + -0.04352296516299248, + -0.0019179899245500565, + 0.0614660382270813, + 0.03905145823955536, + -0.05980569124221802, + -1.4910571575164795, + -0.31295046210289 + ], + [ + 0.1022481769323349, + -0.06372478604316711, + 0.09434723854064941, + -0.08511318266391754, + 0.16531969606876373, + -1.787405014038086, + 0.029011934995651245, + -0.0098990797996521, + 0.11715134978294373, + -0.12257704883813858, + 0.04646285995841026, + 0.013985782861709595, + -1.4870665073394775, + -0.20644120872020721, + -0.029565926641225815, + -0.041694074869155884, + -0.07414861023426056, + 0.018175959587097168, + -0.06981590390205383, + -1.47420334815979, + -0.30480608344078064 + ], + [ + 0.016742020845413208, + 0.015241581946611404, + -0.0390644371509552, + -0.12326564639806747, + 0.11400949209928513, + -1.7120423316955566, + -0.029703915119171143, + 0.02878272533416748, + 0.1282055675983429, + -0.13336879014968872, + 0.06595300137996674, + 0.07015843689441681, + -1.4451978206634521, + -0.16225583851337433, + -0.0900433212518692, + -0.04399387910962105, + -0.04886709153652191, + 0.07016231119632721, + -0.03312358260154724, + -1.4111404418945312, + -0.27658745646476746 + ], + [ + 0.0614650622010231, + 0.017464391887187958, + 0.005555987358093262, + -0.21283464133739471, + 0.2246522158384323, + -2.001433849334717, + 0.09043154120445251, + 0.10920168459415436, + 0.03578871488571167, + -0.2796453833580017, + 0.08667211979627609, + -0.17847344279289246, + -1.6167254447937012, + -0.32877546548843384, + -0.3370673954486847, + -0.12383782863616943, + 0.16609779000282288, + -0.03975054621696472, + -0.3205159902572632, + -1.37526273727417, + -0.07098342478275299 + ], + [ + -0.020552828907966614, + -0.0018603764474391937, + 0.048248082399368286, + -0.4000368118286133, + 0.3300015926361084, + -2.0215983390808105, + 0.23077704012393951, + 0.048726677894592285, + 0.02718084678053856, + -0.18375235795974731, + 0.020289212465286255, + -0.02788648009300232, + -1.767735481262207, + -0.16084523499011993, + -0.07335028052330017, + -0.03634614497423172, + 0.015561208128929138, + 0.05741630494594574, + -0.14920943975448608, + -1.3750662803649902, + -0.5028982162475586 + ], + [ + -0.05880250036716461, + -0.042254041880369186, + 0.07410252094268799, + -0.39490199089050293, + 0.25762513279914856, + -1.9657152891159058, + 0.1290152668952942, + 0.10992196947336197, + 0.03848196193575859, + -0.12064739316701889, + 0.03580477833747864, + -0.09004402160644531, + -1.7064241170883179, + -0.24991311132907867, + -0.09742112457752228, + 0.009243233129382133, + 0.09109720587730408, + 0.027836203575134277, + -0.0063446760177612305, + -1.503725290298462, + -0.5057412385940552 + ] + ], + "generated_utilities": [ + 0.8165659308433533, + 0.21969212591648102, + 0.20318710803985596, + 0.21245209872722626, + 0.20310749113559723, + 0.24723665416240692, + 0.8245300054550171, + 0.20159128308296204 + ], + "group_id": "ms-b0fee26e1db967f90b81", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8397466540336609, + 0.8594310283660889, + 0.14638642966747284, + 0.09418028593063354, + 0.7838175296783447, + 0.5982922315597534, + 0.5913184881210327, + 2.0, + 0.20996521413326263, + 0.17419946193695068, + 0.11757439374923706, + 0.1645697057247162, + 0.05191062390804291, + 0.08658662438392639, + 0.04989185929298401, + 0.05867189168930054 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1510201096534729, + -0.21724483370780945, + 0.14240525662899017, + 0.003108598291873932, + -0.11609727144241333, + -1.6516358852386475, + 0.0, + -0.09976008534431458, + 0.061503566801548004, + 0.3070257902145386, + -0.08051937818527222, + 0.05735237896442413, + -1.6075372695922852, + 0.0, + -0.19649454951286316, + -0.15815469622612, + 0.2621642053127289, + -0.17578476667404175, + -0.034231506288051605, + -1.3808586597442627, + 0.0 + ], + [ + 0.014505341649055481, + 0.0403006337583065, + 0.5529788732528687, + 0.0714028924703598, + -0.41683781147003174, + -1.6516358852386475, + 0.0, + 0.41446545720100403, + -0.16278114914894104, + -0.4971221685409546, + -0.3640303611755371, + 0.02483367919921875, + -1.6075372695922852, + 0.0, + 0.5711643695831299, + -0.5244585275650024, + -0.5069849491119385, + 0.31709733605384827, + -0.4254770278930664, + -1.3808586597442627, + 0.0 + ], + [ + -0.044587142765522, + 0.03136333078145981, + 0.014485275372862816, + 0.04301099479198456, + 0.47328460216522217, + -2.6516358852386475, + -0.32303211092948914, + -0.01855158433318138, + -0.003608003491535783, + 0.029763570055365562, + 0.6359696388244629, + -0.1304762363433838, + -2.607537269592285, + -1.7957698106765747, + -0.009414590895175934, + -0.000939749414101243, + 0.018994441255927086, + 0.7486392855644226, + -0.06401278078556061, + -2.3808586597442627, + -1.722450852394104 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.6460642218589783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.5915396213531494, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.444901704788208 + ], + [ + 0.18585285544395447, + 0.17557260394096375, + 0.08478302508592606, + -0.17644454538822174, + 0.7477058172225952, + -2.614673137664795, + -0.3018328547477722, + 0.2204062044620514, + -0.0497702918946743, + 0.1961878538131714, + 0.5304322242736816, + -0.15872131288051605, + -2.604066848754883, + -2.053436517715454, + 0.18444111943244934, + -0.0024187257513403893, + 0.0054171085357666016, + 1.0256288051605225, + 0.02724038064479828, + -2.426236391067505, + -1.6594898700714111 + ], + [ + -0.5335133075714111, + 0.12298225611448288, + -0.12197553366422653, + 0.0424814410507679, + 0.002416938543319702, + -2.8428826332092285, + -0.17827168107032776, + -0.3377644121646881, + 0.04258471727371216, + -0.15910103917121887, + 0.9169985055923462, + 0.003966793417930603, + -3.069180727005005, + -1.407257318496704, + -0.47542956471443176, + -0.2210284322500229, + -0.10021048784255981, + 0.7806655168533325, + 0.3228837251663208, + -2.058253049850464, + -1.556677222251892 + ], + [ + 0.20687156915664673, + -0.5511468052864075, + 0.3945963978767395, + -0.16104432940483093, + 0.4936041831970215, + -2.1271684169769287, + 0.02855244278907776, + 0.544975996017456, + -0.3336343765258789, + 0.029475202783942223, + 0.8120307326316833, + -0.3246060609817505, + -2.885145425796509, + -2.2030622959136963, + -0.28730231523513794, + 0.19937433302402496, + 0.19840951263904572, + 1.1192030906677246, + -0.3995746374130249, + -2.5594136714935303, + -1.1545796394348145 + ], + [ + -0.5497553944587708, + -0.39125651121139526, + 0.2725587487220764, + -0.27965104579925537, + 1.0584466457366943, + -2.1044363975524902, + -0.6898342370986938, + 0.36552220582962036, + -0.2289373278617859, + 0.24627351760864258, + 0.7968363165855408, + 0.14776766300201416, + -2.988917112350464, + -1.6111457347869873, + -0.5116392374038696, + 0.030796781182289124, + -0.08262427151203156, + 1.0816104412078857, + 0.336497962474823, + -1.684936285018921, + -2.290850877761841 + ], + [ + 0.18322327733039856, + 0.6938241720199585, + 0.5252001881599426, + 0.06770005077123642, + 1.120344877243042, + -1.8558356761932373, + 0.18930694460868835, + -0.7033196687698364, + 0.7861698865890503, + -0.0965423658490181, + 0.015094935894012451, + -0.5303559303283691, + -2.6036770343780518, + -2.589250087738037, + -0.468231737613678, + -0.5901691317558289, + -0.7109078764915466, + 0.986366331577301, + -0.7634903192520142, + -2.984590530395508, + -1.1651530265808105 + ], + [ + 0.7039692401885986, + -0.5346179008483887, + 0.1466517597436905, + -0.6284777522087097, + -0.07994896173477173, + -2.4640438556671143, + -1.102164626121521, + -0.26685386896133423, + -0.3165925443172455, + 0.8653677701950073, + 0.232782244682312, + -0.5054740905761719, + -3.400317668914795, + -1.4077497720718384, + -0.09987740218639374, + -0.27442091703414917, + 0.5428174138069153, + 1.5128605365753174, + 0.747156023979187, + -2.0367631912231445, + -2.464085578918457 + ], + [ + 0.4474923610687256, + -0.7824500203132629, + 0.5928975343704224, + 0.57558673620224, + -0.17630118131637573, + -2.176387310028076, + -0.3299017548561096, + -0.6172446608543396, + 0.5065999627113342, + 0.46063295006752014, + 0.5951727628707886, + -0.02366364747285843, + -3.254279136657715, + -0.8880517482757568, + 0.3714323043823242, + 0.12449074536561966, + 0.6677311658859253, + -0.20713549852371216, + 0.5062628984451294, + -3.2156763076782227, + -1.9652986526489258 + ], + [ + -0.437069296836853, + 0.615723192691803, + -0.7743021249771118, + 0.20228458940982819, + 0.4320293366909027, + -2.0403428077697754, + -0.32251936197280884, + -0.7280393838882446, + 0.10878247022628784, + 0.9395551681518555, + 0.6021312475204468, + -0.3748174011707306, + -2.0175063610076904, + -1.9166245460510254, + -0.10793138295412064, + 0.6512030959129333, + 0.3747653365135193, + 1.5698660612106323, + -0.7995363473892212, + -1.8378328084945679, + -1.0564453601837158 + ], + [ + -0.7214477062225342, + -0.5686705708503723, + 0.8819299936294556, + 0.7319017648696899, + 1.0695421695709229, + -2.0338172912597656, + 0.2973436415195465, + -0.19576850533485413, + 0.4506514370441437, + 0.9037411212921143, + 0.0996239185333252, + 0.5076434016227722, + -2.611851930618286, + -2.7078182697296143, + 0.022706281393766403, + -0.4812851846218109, + -0.40778082609176636, + 1.6509652137756348, + -0.5721896886825562, + -1.8604621887207031, + -1.2943172454833984 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.017527949064970016, + -0.13547566533088684, + -0.31396082043647766, + -0.06045208126306534, + 0.7718918323516846, + -2.996134042739868, + -0.0199548602104187, + 0.26194432377815247, + -0.303554505109787, + -0.09068091958761215, + 0.3007984459400177, + -0.39815279841423035, + -2.626121997833252, + -1.8776893615722656, + 0.25959014892578125, + 0.20619672536849976, + -0.25346171855926514, + 0.40209779143333435, + 0.1943899542093277, + -2.464694023132324, + -1.5338099002838135 + ] + ], + "predicted_scores": [ + 3.2439491748809814, + 3.1261205673217773, + 3.0300378799438477, + 2.903808116912842, + 2.8831100463867188, + 2.7466049194335938, + 2.478484869003296, + 2.3214616775512695 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1358421325", + "selected_index": 0, + "state_hash": "b8e04d64f31138fff8f83f1bb017f9e8515565a2ada2ba5787ad5f06b796589c", + "stored_base_utility": 0.8397466540336609, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8307622671127319, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8307622671127319 + }, + "base_progress": 0.8307622671127319, + "base_success": false, + "base_utility": 0.8307622671127319, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.20941472053527832, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20941472053527832 + }, + { + "contact_quality": null, + "progress": 0.24078069627285004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.24078069627285004 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.16757088899612427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16757088899612427 + }, + { + "contact_quality": null, + "progress": 0.20940162241458893, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20940162241458893 + }, + { + "contact_quality": null, + "progress": 0.22608521580696106, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22608521580696106 + }, + { + "contact_quality": null, + "progress": 0.8135863542556763, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8135863542556763 + }, + { + "contact_quality": null, + "progress": 0.8580727577209473, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8580727577209473 + } + ], + "candidate_progress": [ + 0.20941472053527832, + 0.24078069627285004, + 1.0, + 0.16757088899612427, + 0.20940162241458893, + 0.22608521580696106, + 0.8135863542556763, + 0.8580727577209473 + ], + "candidate_source_chart_ids": [ + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-852c671b0adeccfa4bf1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + true, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "generated_tangents": [ + [ + -0.007829621434211731, + 0.01891593635082245, + -0.19329121708869934, + -0.005221456289291382, + 0.8205686807632446, + 0.10241826623678207, + 0.26574987173080444, + 0.008135095238685608, + 0.07516167312860489, + -0.02350630611181259, + -0.11826420575380325, + -0.04830430448055267, + -1.471059799194336, + 0.09093007445335388, + -0.04718199744820595, + -0.014181646518409252, + 0.0020204484462738037, + -0.017738118767738342, + -0.05534696578979492, + -1.0768465995788574, + -0.037861645221710205 + ], + [ + 0.02366016060113907, + -0.06201127916574478, + -0.32058408856391907, + -0.002542644739151001, + 0.7281009554862976, + 0.3995829224586487, + 0.8869548439979553, + 0.11554411798715591, + 0.02445787936449051, + -0.1949370801448822, + -0.267350435256958, + -0.24438369274139404, + -1.6549205780029297, + 0.12907633185386658, + -0.23692730069160461, + -0.05998361483216286, + -0.07307992875576019, + -0.08992688357830048, + -0.20582860708236694, + -1.2334067821502686, + -0.3705994188785553 + ], + [ + -0.04514935612678528, + -0.013184353709220886, + -0.04945489764213562, + -0.04480566084384918, + 0.7700527906417847, + 0.00011171400547027588, + 0.2241833209991455, + 0.02297687530517578, + 0.04185006767511368, + 0.049113623797893524, + -0.18785682320594788, + -0.1471807062625885, + -1.612541675567627, + -0.0796525776386261, + -0.03811833634972572, + 0.0030864905565977097, + -0.042008593678474426, + 0.014863431453704834, + -0.12185788154602051, + -1.2912936210632324, + -0.05844557285308838 + ], + [ + -0.020709052681922913, + 0.0016984492540359497, + -0.10843166708946228, + -0.05650597810745239, + 0.7973015308380127, + -0.06671230494976044, + 0.39062896370887756, + 0.10322524607181549, + 0.11411242932081223, + -0.2520296573638916, + -0.2025318145751953, + 0.10886607319116592, + -1.5927718877792358, + -0.04515893757343292, + 0.008747205138206482, + -0.016902105882763863, + -0.0842159241437912, + 0.06362536549568176, + -0.18833810091018677, + -1.1561708450317383, + -0.04434993863105774 + ], + [ + -0.06029124557971954, + 0.02984365075826645, + 0.02232035994529724, + 0.08511334657669067, + 0.758011519908905, + 0.08001004159450531, + 0.0702548623085022, + 0.08262918889522552, + 0.08510931581258774, + -0.137472003698349, + -0.09991353005170822, + -0.08376145362854004, + -1.514747142791748, + 0.18701723217964172, + 0.014249257743358612, + -0.06069842353463173, + -0.030516967177391052, + 0.06509275734424591, + -0.05639326572418213, + -1.169434905052185, + 0.05315692722797394 + ], + [ + 0.06198125332593918, + -0.022218966856598854, + -0.30206623673439026, + 0.06313300132751465, + 0.874203085899353, + 0.14009854197502136, + 0.2918381690979004, + 0.06727392971515656, + 0.04293106496334076, + -0.01505974680185318, + -0.06775247305631638, + -0.15592628717422485, + -1.4857100248336792, + 0.13839712738990784, + -0.14423401653766632, + -0.041462600231170654, + -0.1070481389760971, + -0.05557180941104889, + -0.01071956753730774, + -1.1340205669403076, + -0.06076802313327789 + ], + [ + -0.13456524908542633, + -0.02907385863363743, + -0.1418074071407318, + 0.17192599177360535, + 0.697289228439331, + 0.38130325078964233, + 0.6507576704025269, + 0.005012795329093933, + 0.09296613186597824, + -0.21925926208496094, + -0.2505892217159271, + -0.34688031673431396, + -1.4939807653427124, + 0.1576516330242157, + -0.26408883929252625, + -0.07235299050807953, + -0.06471963226795197, + -0.11286158859729767, + -0.059493839740753174, + -1.4128203392028809, + -0.2227497696876526 + ], + [ + -0.12860457599163055, + 0.04124286025762558, + -0.11609110236167908, + 0.10706081986427307, + 0.7368475794792175, + 0.262528657913208, + 0.23853906989097595, + 0.07235468178987503, + 0.018470266833901405, + 0.014972291886806488, + -0.1195920780301094, + 0.08717256784439087, + -1.4545354843139648, + 0.1598609983921051, + -0.04632589593529701, + -0.014307182282209396, + 0.054265186190605164, + 0.019666001200675964, + -0.1396884322166443, + -1.1443967819213867, + -0.05688692629337311 + ] + ], + "generated_utilities": [ + 0.20941472053527832, + 0.24078069627285004, + 2.0, + 0.16757088899612427, + 0.20940162241458893, + 0.22608521580696106, + 0.8135863542556763, + 0.8580727577209473 + ], + "group_id": "ms-bc2d3b425d9163a62dbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8266608119010925, + 2.0, + 0.15905821323394775, + 0.13648143410682678, + 0.5621776580810547, + 0.18381908535957336, + 0.20885342359542847, + 0.18184903264045715, + 0.19253113865852356, + 0.5574395656585693, + 0.10362428426742554, + 0.0659959614276886, + 0.09997051954269409, + 0.2210000902414322, + 0.00010728836059570312, + 0.12723565101623535 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.059883326292037964, + -0.16710972785949707, + -0.04027438163757324, + -0.23492489755153656, + 0.34960758686065674, + 0.16551673412322998, + 0.0, + -0.27484941482543945, + 0.07304387539625168, + -0.38491085171699524, + 0.03056102991104126, + 0.16416949033737183, + -1.5270280838012695, + 0.0, + 0.01279987208545208, + -0.07740817964076996, + 0.07519636303186417, + 0.14440757036209106, + 0.0022157281637191772, + -1.3374013900756836, + 0.0 + ], + [ + -0.6122239828109741, + -0.0734642744064331, + -0.4064018726348877, + 0.11902926862239838, + 0.34960758686065674, + -0.07764682173728943, + 0.0, + 0.37407442927360535, + -0.08740249276161194, + -0.20959049463272095, + -0.37996697425842285, + -0.304497092962265, + -1.5270280838012695, + 0.0, + -0.9113190174102783, + 0.3964778184890747, + 0.4811243414878845, + 0.2385387420654297, + -0.11476142704486847, + -1.3374013900756836, + 0.0 + ], + [ + 0.3877759873867035, + 0.6924911141395569, + -1.0402743816375732, + 0.005685340613126755, + 1.3496075868606567, + -0.2760518193244934, + 1.2840001583099365, + -0.028063178062438965, + 0.0023769063409417868, + 0.03187502175569534, + 0.04654876887798309, + 0.3263346254825592, + -2.5270280838012695, + -1.7457892894744873, + -0.012772172689437866, + -0.008647367358207703, + 0.019657021388411522, + 0.3050904870033264, + 0.19037143886089325, + -2.3374013900756836, + -1.738369345664978 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.568000316619873, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.4915785789489746, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.476738691329956 + ], + [ + 0.1989155411720276, + 0.7169155478477478, + -0.9921301603317261, + 0.22762086987495422, + 1.516188144683838, + -0.23096826672554016, + 1.0414973497390747, + 0.0012522991746664047, + -0.0518387071788311, + -0.23540246486663818, + 0.08515512943267822, + 0.5060473084449768, + -2.5231378078460693, + -1.4593229293823242, + -0.09671962261199951, + 0.09134317189455032, + -0.07944301515817642, + 0.18403387069702148, + 0.2437569499015808, + -2.270580291748047, + -1.9640144109725952 + ], + [ + 0.256519615650177, + 0.7805030941963196, + -1.3583617210388184, + 0.3310847878456116, + 1.2735480070114136, + -0.4127388000488281, + 1.2665152549743652, + -0.19997301697731018, + -0.11129427701234818, + 0.1647017002105713, + 0.3767858147621155, + 0.4831879138946533, + -2.752143621444702, + -1.6790910959243774, + 0.2554209232330322, + 0.3251851797103882, + -0.12605378031730652, + 0.696884274482727, + 0.29535919427871704, + -2.726069688796997, + -1.704551100730896 + ], + [ + 0.49772778153419495, + 1.0026791095733643, + -0.5513590574264526, + 0.008379313163459301, + 1.7768750190734863, + -0.6102496981620789, + 1.7048192024230957, + -0.29564282298088074, + -0.20420490205287933, + -0.23934513330459595, + -0.34791648387908936, + 0.0690477192401886, + -2.602295398712158, + -1.8138929605484009, + -0.38315677642822266, + -0.16421082615852356, + 0.17502839863300323, + -0.13996875286102295, + -0.30929630994796753, + -2.544666290283203, + -1.2411268949508667 + ], + [ + 0.07701912522315979, + 0.9529868364334106, + -1.497549295425415, + -0.3016125559806824, + 1.8504154682159424, + -0.3136579394340515, + 1.6619524955749512, + -0.28860214352607727, + 0.20993559062480927, + 0.6072397232055664, + 0.04243222624063492, + -0.133962482213974, + -2.0948100090026855, + -2.1691184043884277, + 0.44940999150276184, + -0.4183289408683777, + 0.5662978291511536, + 0.5016863942146301, + 0.7054548263549805, + -2.282663345336914, + -1.8892805576324463 + ], + [ + -0.2981111705303192, + 0.038420140743255615, + -1.687281847000122, + -0.37934571504592896, + 1.1300733089447021, + -0.6208293437957764, + 1.8144867420196533, + 0.16701459884643555, + 0.5508663654327393, + 0.4797254800796509, + -0.09421360492706299, + -0.3499920070171356, + -2.5030596256256104, + -1.5871793031692505, + -0.33045580983161926, + -0.29071635007858276, + 0.5385350584983826, + 0.9102991223335266, + 0.3121117353439331, + -2.2699851989746094, + -2.081907272338867 + ], + [ + -0.28236815333366394, + 0.18060839176177979, + -0.32270634174346924, + 0.033799149096012115, + 1.128151774406433, + -0.7577929496765137, + 1.6937967538833618, + -0.813525378704071, + 0.48178502917289734, + -0.6699209809303284, + -0.03810972720384598, + 0.8273124694824219, + -2.2768704891204834, + -1.317199945449829, + 0.0046694036573171616, + 0.5900158286094666, + -0.12453534454107285, + 1.046316385269165, + 0.2770566940307617, + -2.0448076725006104, + -2.4180057048797607 + ], + [ + 0.26116815209388733, + 1.305420994758606, + -0.5357515811920166, + 0.6523149013519287, + 0.5250536203384399, + -0.6821175217628479, + 1.2918037176132202, + 0.3275931477546692, + -0.8845832943916321, + -0.6987056136131287, + 0.10023711621761322, + -0.2849085032939911, + -1.7303776741027832, + -1.0117536783218384, + -0.06847482174634933, + 0.6783800721168518, + -0.04633811116218567, + 0.0702061653137207, + -0.3159322142601013, + -3.004882335662842, + -2.0691566467285156 + ], + [ + 0.5418367385864258, + 1.1724635362625122, + -1.717900276184082, + 0.9217556118965149, + 2.316706657409668, + -0.2411489486694336, + 0.720655083656311, + -0.6330894231796265, + 0.03703225776553154, + 0.38938266038894653, + -0.8305367827415466, + -0.2152116596698761, + -2.614929437637329, + -2.045497179031372, + -0.9044661521911621, + 0.9391879439353943, + -0.2323634922504425, + 0.8392186164855957, + -0.6889055371284485, + -3.0831263065338135, + -1.0035808086395264 + ], + [ + 1.0110303163528442, + 1.0131109952926636, + -1.3200907707214355, + -0.433234840631485, + 1.9466626644134521, + 0.46447479724884033, + 1.528633952140808, + 0.40949776768684387, + 0.07322453707456589, + -0.12349443882703781, + 0.24514153599739075, + 0.005379408597946167, + -2.487516164779663, + -1.1890358924865723, + -0.026332437992095947, + -0.6102214455604553, + 0.6559669971466064, + 0.4371780753135681, + -0.6651540994644165, + -2.8161611557006836, + -1.4271454811096191 + ], + [ + 0.9640108346939087, + 1.0122193098068237, + -1.9809117317199707, + 0.40601760149002075, + 0.9296187162399292, + 0.601173460483551, + 2.229055643081665, + -0.4972975254058838, + 0.6548057794570923, + 0.6165774464607239, + -0.7309268116950989, + 0.697066068649292, + -3.0948128700256348, + -2.4828031063079834, + 0.16930773854255676, + -0.11851751059293747, + -0.8277174830436707, + 0.8252183794975281, + -0.4829612374305725, + -2.922985076904297, + -1.1981642246246338 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03792491555213928, + -0.040427446365356445, + -0.04027438163757324, + 0.02325565740466118, + 0.34960758686065674, + -0.05261008441448212, + 0.0, + 0.018791399896144867, + 0.05303370952606201, + 0.01128687709569931, + 0.01692008785903454, + 0.06139722466468811, + -1.5270280838012695, + 0.0, + -0.004229618236422539, + 0.017936373129487038, + 0.06901376694440842, + -0.04424762725830078, + -0.033029720187187195, + -1.3374013900756836, + 0.0 + ] + ], + "predicted_scores": [ + 3.066279649734497, + 2.8866214752197266, + 2.7994003295898438, + 2.7754523754119873, + 2.6756951808929443, + 2.520692825317383, + 2.500575542449951, + 2.316096305847168 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007", + "selected_index": 0, + "state_hash": "58b3d0c1cafc199ad58508912becbaef59038a655cef0caafd263ade5c9b53fa", + "stored_base_utility": 0.8266608119010925, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20032641291618347, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20032641291618347 + }, + "base_progress": 0.20032641291618347, + "base_success": false, + "base_utility": 0.20032641291618347, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.20836615562438965, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20836615562438965 + }, + { + "contact_quality": null, + "progress": 0.21678751707077026, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21678751707077026 + }, + { + "contact_quality": null, + "progress": 0.22490769624710083, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22490769624710083 + }, + { + "contact_quality": null, + "progress": 0.21572625637054443, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21572625637054443 + }, + { + "contact_quality": null, + "progress": 0.2215946912765503, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2215946912765503 + }, + { + "contact_quality": null, + "progress": 0.20464813709259033, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20464813709259033 + }, + { + "contact_quality": null, + "progress": 0.21880853176116943, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21880853176116943 + }, + { + "contact_quality": null, + "progress": 0.22272410988807678, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22272410988807678 + } + ], + "candidate_progress": [ + 0.20836615562438965, + 0.21678751707077026, + 0.22490769624710083, + 0.21572625637054443, + 0.2215946912765503, + 0.20464813709259033, + 0.21880853176116943, + 0.22272410988807678 + ], + "candidate_source_chart_ids": [ + "ms-95028134c6e3670b89d5", + "ms-b8492a3a812095651d75", + "ms-95028134c6e3670b89d5", + "ms-ee2fcbc94d41385713d0", + "ms-016c5de39ed3c111889a", + "ms-c8706d0aa14a930236de", + "ms-9ae5c6c4132218ecf49e", + "ms-5ac1bf2d7d319955d45d" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank6", + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-d1960368ef76bbb10669", + "generated_tangents": [ + [ + -0.2732497453689575, + 0.10623323917388916, + -0.5341019630432129, + -0.52424156665802, + 0.38650238513946533, + 0.11353766918182373, + 1.2099891901016235, + -0.2784939110279083, + 0.29466575384140015, + -0.1788938045501709, + 0.012427084147930145, + -0.3214123249053955, + -1.5456247329711914, + 0.4183884859085083, + 0.17805370688438416, + -0.06384320557117462, + -0.3878546357154846, + 0.13299499452114105, + -0.2544509172439575, + -1.7263820171356201, + -0.864246129989624 + ], + [ + -0.21580372750759125, + 0.1762966513633728, + -0.6667286157608032, + -0.2037983387708664, + 0.8459116220474243, + 0.08843044936656952, + 0.694096565246582, + -0.11482015252113342, + 0.08122444152832031, + -0.2178671956062317, + 0.07332239300012589, + -0.40485692024230957, + -1.284013271331787, + 0.5618535876274109, + 0.2824944853782654, + 0.0454351082444191, + -0.4142775535583496, + 0.21625806391239166, + -0.06423258781433105, + -1.5438718795776367, + -0.5496693849563599 + ], + [ + -0.2495129257440567, + 0.0910559594631195, + -0.49450036883354187, + -0.3702200949192047, + 0.5745404362678528, + 0.018759846687316895, + 0.8391996622085571, + -0.12035444378852844, + 0.14337728917598724, + -0.07287713140249252, + -0.05631840229034424, + -0.06944793462753296, + -1.3841220140457153, + 0.22425532341003418, + 0.18419459462165833, + 0.013678228482604027, + -0.2958596348762512, + 0.21296192705631256, + -0.14534270763397217, + -1.3982222080230713, + -0.3495023548603058 + ], + [ + -0.2663615942001343, + 0.14154016971588135, + -0.38258078694343567, + -0.4201826751232147, + 0.55460125207901, + 0.7031106948852539, + 0.6285395622253418, + -0.11529606580734253, + 0.13880234956741333, + -0.016407139599323273, + 0.008387640118598938, + -0.3320184350013733, + -1.4349403381347656, + 0.11576896905899048, + 0.10303384065628052, + 0.06527340412139893, + -0.2843952178955078, + 0.33774620294570923, + -0.18962779641151428, + -1.386549949645996, + -0.11612573266029358 + ], + [ + -0.0685390830039978, + 0.10524150729179382, + -0.47070881724357605, + -0.5226870775222778, + 0.56330406665802, + -0.07589875161647797, + 0.6408642530441284, + -0.23171091079711914, + 0.12112893909215927, + 0.007567659020423889, + -0.08336234837770462, + -0.33656448125839233, + -1.3480051755905151, + 0.020533017814159393, + 0.1174524575471878, + -0.013492795638740063, + -0.3253559470176697, + 0.21753256022930145, + -0.12693777680397034, + -1.4755010604858398, + -0.41111454367637634 + ], + [ + -0.46328067779541016, + 0.14907579123973846, + -0.46756860613822937, + -0.3170608878135681, + 0.6231703758239746, + 0.03959271311759949, + 0.9835152626037598, + 0.017133131623268127, + 0.10476913303136826, + -0.14825591444969177, + -0.054740145802497864, + -0.347636878490448, + -1.119163155555725, + 0.571229100227356, + 0.2677895128726959, + 0.0102913286536932, + -0.21753139793872833, + 0.15240247547626495, + -0.03422674536705017, + -1.409219741821289, + -0.11958730220794678 + ], + [ + -0.21189619600772858, + 0.07958297431468964, + -0.490036278963089, + -0.42205339670181274, + 0.5753673315048218, + 0.016161277890205383, + 0.7361229658126831, + -0.1892039179801941, + 0.13392025232315063, + -0.025577999651432037, + -0.018562987446784973, + -0.21219003200531006, + -1.445809006690979, + 0.18592366576194763, + 0.17200149595737457, + -0.014207066036760807, + -0.28478527069091797, + 0.2607681155204773, + -0.1231447160243988, + -1.417062520980835, + -0.33946332335472107 + ], + [ + -0.23061718046665192, + 0.1084190160036087, + -0.44667258858680725, + -0.40127190947532654, + 0.5795594453811646, + 0.4705345034599304, + 0.6456956267356873, + -0.2400834858417511, + 0.17875860631465912, + -0.05076836794614792, + 0.0344834104180336, + -0.1540597677230835, + -1.8119797706604004, + 0.1843666434288025, + 0.07184919714927673, + 0.0648496001958847, + -0.23051045835018158, + 0.43014973402023315, + -0.2404172420501709, + -1.547011137008667, + -0.19852378964424133 + ] + ], + "generated_utilities": [ + 0.20836615562438965, + 0.21678751707077026, + 0.22490769624710083, + 0.21572625637054443, + 0.2215946912765503, + 0.20464813709259033, + 0.21880853176116943, + 0.22272410988807678 + ], + "group_id": "ms-d1960368ef76bbb10669", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.842734158039093, + 2.0, + 0.1342049241065979, + 0.18085119128227234, + 0.5091985464096069, + 0.016112878918647766, + 0.5080842971801758, + 0.19944033026695251, + 0.5132148265838623, + 0.5265648365020752, + 0.11437654495239258, + 0.04028162360191345, + 0.04675377905368805, + 0.13603349030017853, + 0.12877747416496277, + 0.05102705955505371 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2770041823387146, + 0.08894228935241699, + 0.07033950090408325, + 0.12426833808422089, + 0.16520383954048157, + 0.023725181818008423, + 0.0, + -0.25569218397140503, + -0.057803913950920105, + 0.06703128665685654, + 0.1732010841369629, + 0.07705201953649521, + -1.0374658107757568, + 0.0, + 0.04586680978536606, + 0.028830496594309807, + -0.0374501496553421, + 0.10374182462692261, + -0.007100850343704224, + -1.7975506782531738, + 0.0 + ], + [ + -0.4268343448638916, + 0.13483470678329468, + -0.06075429916381836, + 0.19817759096622467, + 0.09922409057617188, + 0.6547578573226929, + 0.0, + 0.15026617050170898, + 0.002472013235092163, + -0.31349992752075195, + 0.5119487047195435, + 0.571077287197113, + -1.0374658107757568, + 0.0, + -0.5828977823257446, + -0.20166385173797607, + -0.7097158432006836, + 0.8187870979309082, + -0.0694284737110138, + -1.7975506782531738, + 0.0 + ], + [ + -0.23462048172950745, + 1.088942289352417, + -0.5585125088691711, + 0.10042184591293335, + 0.4514904320240021, + -0.12210102379322052, + 2.176525354385376, + -0.10348781198263168, + 0.19529865682125092, + 0.12132670730352402, + 0.7056414484977722, + 0.12544797360897064, + -2.037465810775757, + 0.6289246678352356, + -0.008351176977157593, + 0.0010435802396386862, + 0.017314812168478966, + 0.6334809064865112, + -0.3720659911632538, + -2.797550678253174, + -1.5395829677581787 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.353050708770752, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.2578493356704712, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0791659355163574 + ], + [ + -0.09280626475811005, + 1.1662368774414062, + -0.8228501081466675, + 0.24497312307357788, + 0.5648186802864075, + -0.03522863984107971, + 2.259786367416382, + 0.14498579502105713, + 0.2030920535326004, + 0.39727938175201416, + 0.8023015856742859, + 0.3385764956474304, + -1.8738088607788086, + 0.7300182580947876, + 0.20887598395347595, + 0.23107871413230896, + 0.3028872311115265, + 0.7558343410491943, + -0.5975744724273682, + -3.0329887866973877, + -1.8127071857452393 + ], + [ + -0.13783538341522217, + 0.9487118721008301, + -0.3492928743362427, + -0.2956767678260803, + 0.2793389856815338, + -0.16702969372272491, + 2.558225631713867, + 0.17572781443595886, + 0.570004940032959, + 0.27895650267601013, + 0.6778483986854553, + -0.27156615257263184, + -1.8747575283050537, + 1.0240070819854736, + 0.27303546667099, + 0.33387133479118347, + 0.1442956030368805, + 0.24661725759506226, + -0.5561860799789429, + -2.9813332557678223, + -1.833754062652588 + ], + [ + -0.18553051352500916, + 1.0909820795059204, + -0.27226096391677856, + 0.1659364104270935, + 0.7108361721038818, + -0.27768880128860474, + 2.239938974380493, + 0.019483201205730438, + 0.21263465285301208, + -0.09839647263288498, + 0.9676799178123474, + -0.36985379457473755, + -2.133316993713379, + 0.3758644461631775, + -0.47750571370124817, + 0.17470555007457733, + -0.17035381495952606, + 0.2707958221435547, + -0.5672953128814697, + -2.394361972808838, + -1.668647289276123 + ], + [ + -0.3557406961917877, + 0.9564932584762573, + -1.0014276504516602, + -0.38222843408584595, + 0.41349664330482483, + -0.05637756735086441, + 1.6011608839035034, + -0.31295540928840637, + -0.15234358608722687, + 0.4648304879665375, + 0.2744198739528656, + -0.44805359840393066, + -2.1544065475463867, + 1.166503667831421, + 0.014861157163977623, + -0.23438170552253723, + 0.1278134286403656, + 0.5812121033668518, + -0.7571046352386475, + -2.5030438899993896, + -2.100955009460449 + ], + [ + -0.16500982642173767, + 0.7556273937225342, + -0.988593339920044, + 0.6213532090187073, + 0.15840065479278564, + 0.5245341062545776, + 2.452624797821045, + -0.5693200826644897, + -0.32241708040237427, + 0.49458175897598267, + 1.3341540098190308, + 0.059507817029953, + -2.304706335067749, + 0.7334492206573486, + -0.20702646672725677, + 0.32367274165153503, + -0.5396079421043396, + 0.9809319972991943, + -0.6172815561294556, + -3.197409152984619, + -0.9358728528022766 + ], + [ + -0.07666900753974915, + 1.8442692756652832, + -0.6603910326957703, + -0.40882760286331177, + 0.22092406451702118, + -0.23551997542381287, + 1.6105436086654663, + 0.09144749492406845, + 0.8141599893569946, + 0.8528898358345032, + 1.0663814544677734, + 0.3283998966217041, + -1.7124712467193604, + 1.2278027534484863, + 0.7039424180984497, + 0.3645748496055603, + -0.2968631386756897, + 0.4998055696487427, + -0.9362069368362427, + -3.461055278778076, + -0.8184676766395569 + ], + [ + -0.1520652174949646, + 1.405963659286499, + -0.7959750294685364, + -0.39229461550712585, + 0.46590372920036316, + -0.8267906904220581, + 2.952258586883545, + 0.5923218727111816, + -0.6996292471885681, + 0.8352785706520081, + -0.15220212936401367, + -0.1307106763124466, + -2.849807024002075, + 0.4983477294445038, + 0.5902999639511108, + -0.5390225648880005, + 0.13897128403186798, + 0.21545273065567017, + -1.1186240911483765, + -3.1762759685516357, + -1.0118725299835205 + ], + [ + -0.564094066619873, + 0.4751617908477783, + -1.3977731466293335, + 0.8189605474472046, + 0.17298781871795654, + 0.2476775199174881, + 2.8234164714813232, + -0.4518245458602905, + -0.7904537916183472, + -0.39086827635765076, + -0.05634337663650513, + -0.09099806845188141, + -1.9690911769866943, + 1.1869969367980957, + -0.5615661144256592, + 0.32865890860557556, + -0.8041903376579285, + 1.3324832916259766, + -0.6190662384033203, + -2.4903852939605713, + -2.3098156452178955 + ], + [ + -0.2328718602657318, + 0.9724385142326355, + -1.5535115003585815, + 0.7688977718353271, + 0.4409410357475281, + -0.8374075889587402, + 1.2719230651855469, + 0.653096616268158, + -0.2887840270996094, + 0.4892227351665497, + 0.9943453669548035, + 0.5566720962524414, + -2.496413469314575, + 0.6299278140068054, + -0.08689186722040176, + 0.024782167747616768, + 0.5747005939483643, + 0.023068726062774658, + -1.111369252204895, + -2.187150001525879, + -1.2047184705734253 + ], + [ + -1.1679872274398804, + 1.9997228384017944, + -0.45194220542907715, + -0.8861345648765564, + -0.5369991064071655, + 0.46434593200683594, + 2.175499439239502, + 0.7599186301231384, + 0.38914406299591064, + 0.6877046823501587, + -0.004208564758300781, + -0.1321776658296585, + -1.3474420309066772, + 0.8496907353401184, + 0.4218248426914215, + -0.9354817271232605, + 0.945868730545044, + 1.60379958152771, + -1.195396065711975, + -3.597724676132202, + -1.9041136503219604 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04476936161518097, + 0.08894228935241699, + 0.06436485052108765, + -0.0775555968284607, + 0.04825553297996521, + -0.060782790184020996, + 0.0, + -0.07688627392053604, + -0.06308497488498688, + -0.03861912339925766, + 0.1235572099685669, + -0.0017572194337844849, + -1.0374658107757568, + 0.0, + 0.03638586029410362, + 0.0556478351354599, + 0.0387292355298996, + 0.044096171855926514, + 0.029627740383148193, + -1.7975506782531738, + 0.0 + ] + ], + "predicted_scores": [ + 4.358102321624756, + 4.097344398498535, + 4.023711204528809, + 3.9058315753936768, + 3.8040666580200195, + 3.7624096870422363, + 3.7143890857696533, + 3.3574204444885254 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "416183106", + "selected_index": 0, + "state_hash": "cd187560c1c39db9e6bf7d9a611a8b7108dc88766de58fd62b4317f0bbcc1b50", + "stored_base_utility": 0.842734158039093, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8350887894630432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8350887894630432 + }, + "base_progress": 0.8350887894630432, + "base_success": false, + "base_utility": 0.8350887894630432, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.814424455165863, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.814424455165863 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.22760358452796936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22760358452796936 + }, + { + "contact_quality": null, + "progress": 0.2006266564130783, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2006266564130783 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.21153436601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21153436601161957 + }, + { + "contact_quality": null, + "progress": 0.19750897586345673, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19750897586345673 + }, + { + "contact_quality": null, + "progress": 0.15966159105300903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15966159105300903 + } + ], + "candidate_progress": [ + 0.814424455165863, + 1.0, + 0.22760358452796936, + 0.2006266564130783, + 1.0, + 0.21153436601161957, + 0.19750897586345673, + 0.15966159105300903 + ], + "candidate_source_chart_ids": [ + "ms-5dfbb16e173e1a3e2bd6", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + true, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-efdc07c985935d871adf", + "generated_tangents": [ + [ + -0.013332024216651917, + -0.09877345710992813, + 0.26450932025909424, + -0.02948884665966034, + -0.12492042779922485, + -1.7737995386123657, + 0.08859455585479736, + 0.06688976287841797, + 0.06304284185171127, + -0.1261521875858307, + 0.024119026958942413, + -0.05646693706512451, + -1.324194073677063, + 0.04585892707109451, + -0.01973208412528038, + -0.07470755279064178, + -0.0056900084018707275, + 0.1738128662109375, + 0.2990410029888153, + -1.220249891281128, + -0.2839846611022949 + ], + [ + 0.03280780464410782, + -0.05737597495317459, + -0.022029787302017212, + -0.001036912202835083, + 0.004179731011390686, + -1.5749638080596924, + -0.024465858936309814, + 0.08278200030326843, + 0.05197286605834961, + -0.028704054653644562, + 0.06959202885627747, + -0.03036804497241974, + -1.372532606124878, + -0.12132520973682404, + -0.057227808982133865, + -0.02337384596467018, + 0.1442265510559082, + 0.03886638581752777, + -0.016652077436447144, + -1.168245553970337, + -0.1577155888080597 + ], + [ + -0.0032540112733840942, + -0.04752686992287636, + -0.04314276576042175, + 0.0059797316789627075, + -0.07244718074798584, + -1.5057730674743652, + -0.11828851699829102, + 0.07665592432022095, + 0.05501272529363632, + -0.07879000157117844, + 0.04016869515180588, + -0.004581347107887268, + -1.195197343826294, + -0.1345781832933426, + -0.0619640089571476, + 0.03739229217171669, + 0.11475878953933716, + 0.04048973321914673, + -0.04752197861671448, + -1.0660927295684814, + -0.18573719263076782 + ], + [ + -0.008654564619064331, + 0.0036018006503582, + 0.02833312749862671, + 0.14265021681785583, + -0.0014200806617736816, + -1.5085525512695312, + 0.048266440629959106, + 0.008562147617340088, + 0.1230761930346489, + -0.09674461930990219, + 0.07801411300897598, + 0.12203509360551834, + -1.2971980571746826, + -0.08735552430152893, + -0.08992095291614532, + -0.06161929666996002, + 0.06254541873931885, + 0.043771058320999146, + 0.004117995500564575, + -1.1442360877990723, + -0.1357654631137848 + ], + [ + 0.12913350760936737, + 0.08720725774765015, + 0.0193556547164917, + -0.2315005660057068, + 0.18963700532913208, + -1.8492833375930786, + -0.20759832859039307, + 0.2419825941324234, + 0.03273481875658035, + -0.0850699320435524, + 0.08927270025014877, + 0.14676442742347717, + -1.4844624996185303, + -0.6773268580436707, + -0.14903348684310913, + 0.12158150970935822, + 0.35294899344444275, + 0.05146056413650513, + 0.13797593116760254, + -1.037107229232788, + -0.24696126580238342 + ], + [ + 0.027051463723182678, + -0.06046345829963684, + 0.03760448098182678, + -0.012343600392341614, + 0.021400943398475647, + -1.4144668579101562, + -0.09972888231277466, + 0.12906959652900696, + 0.015865160152316093, + -0.04949576407670975, + 0.13024920225143433, + 0.09889984875917435, + -1.251424789428711, + -0.09391428530216217, + -0.07430841028690338, + 0.009700924158096313, + 0.03600558638572693, + 0.15948447585105896, + 0.010383576154708862, + -1.0674681663513184, + -0.19397443532943726 + ], + [ + -0.19073574244976044, + 0.06492596864700317, + 0.09132397174835205, + -0.2691749930381775, + 0.500572919845581, + -1.6856186389923096, + 0.4800916314125061, + 0.283923864364624, + -0.1709602326154709, + -0.35143622756004333, + 0.08284296840429306, + -0.07589542865753174, + -1.6209856271743774, + -0.32679450511932373, + 0.06518667191267014, + 0.04909523203969002, + 0.07296597957611084, + -0.12730474770069122, + 0.23832252621650696, + -0.8187363743782043, + -0.5636305809020996 + ], + [ + -0.2372923344373703, + 0.07363962382078171, + 0.15948879718780518, + -0.17178823053836823, + 0.1485951989889145, + -1.4346777200698853, + -0.2271597981452942, + 0.04705031216144562, + -0.034805335104465485, + -0.37829411029815674, + 0.014230944216251373, + 0.33765649795532227, + -1.5487416982650757, + -0.5470280051231384, + -0.2538008391857147, + -0.015747789293527603, + -0.04476819932460785, + 0.1644936501979828, + 0.24060587584972382, + -1.043912410736084, + -0.5292702913284302 + ] + ], + "generated_utilities": [ + 0.814424455165863, + 2.0, + 0.22760358452796936, + 0.2006266564130783, + 2.0, + 0.21153436601161957, + 0.19750897586345673, + 0.15966159105300903 + ], + "group_id": "ms-efdc07c985935d871adf", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8197344541549683, + 2.0, + 0.8125, + 0.8125142455101013, + 0.7847581505775452, + 0.7794750332832336, + 0.5920823216438293, + 0.5697357058525085, + 2.0, + 0.5847549438476562, + 0.09384849667549133, + 0.07791829109191895, + 0.011395931243896484, + 0.09286168217658997, + 0.10894183814525604, + 0.0616525262594223 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2827143669128418, + -0.12206736207008362, + 0.1543778032064438, + 0.27268126606941223, + -0.17288367450237274, + -2.5338973999023438, + -1.7615662813186646, + -0.19161763787269592, + 0.1886080503463745, + 0.31686434149742126, + 0.5112922191619873, + 0.14046527445316315, + -2.17134952545166, + -1.8021173477172852, + 0.11115366220474243, + 0.25490811467170715, + -0.025668112561106682, + 0.4787765145301819, + 0.46609628200531006, + -1.711870551109314, + -1.5312864780426025 + ], + [ + 0.03046044334769249, + 0.042622197419404984, + -0.12103251367807388, + 0.7814357876777649, + 0.5093902349472046, + -2.648094654083252, + -1.5764049291610718, + -0.16382060945034027, + 0.15227201581001282, + -0.29746392369270325, + 0.394713819026947, + 0.044284895062446594, + -2.150869131088257, + -1.853789210319519, + 0.24474957585334778, + 0.16074210405349731, + -0.30960795283317566, + 0.986261248588562, + 0.4512072801589966, + -1.9782602787017822, + -1.814382553100586 + ], + [ + -0.47424501180648804, + 0.4193490147590637, + 0.31025299429893494, + 0.436926007270813, + 0.07080172002315521, + -2.2750933170318604, + -2.139326572418213, + 0.19072748720645905, + -0.05066118389368057, + -0.38944289088249207, + 0.11913672089576721, + -0.4728909432888031, + -2.630631446838379, + -1.9070930480957031, + 0.5720394849777222, + 0.1898895502090454, + -0.0872373953461647, + 0.3081148862838745, + 0.6700773239135742, + -1.8990029096603394, + -1.655537486076355 + ], + [ + -0.112115278840065, + -0.5390006899833679, + -0.27749133110046387, + -0.08230990171432495, + 0.18304404616355896, + -2.5003390312194824, + -1.177594542503357, + -0.37152859568595886, + 0.1827065348625183, + -0.1805598884820938, + 0.8884490728378296, + 0.3141748607158661, + -2.7738473415374756, + -1.8585447072982788, + -0.6611859798431396, + 0.5365979075431824, + 0.024639926850795746, + 1.0166820287704468, + 0.6261310577392578, + -1.3820469379425049, + -1.1859028339385986 + ], + [ + -0.013153300620615482, + 0.6059781908988953, + -0.7563084363937378, + 0.6436030864715576, + 0.4828413128852844, + -1.8755569458007812, + -1.896214246749878, + -0.445474773645401, + -0.6578570604324341, + -0.21610267460346222, + 1.3274874687194824, + -0.6251860857009888, + -2.005418300628662, + -1.463393211364746, + -0.40788042545318604, + -0.5932772159576416, + -0.6501284837722778, + 0.1491221785545349, + 0.8418278694152832, + -2.578186511993408, + -1.628711462020874 + ], + [ + -0.051258739084005356, + 0.6121965050697327, + -0.4303005337715149, + 0.378423810005188, + -0.4187525510787964, + -2.0054633617401123, + -1.5839252471923828, + 0.38306933641433716, + -0.8548648953437805, + 0.5034631490707397, + 0.3682137727737427, + 0.867935061454773, + -2.5900886058807373, + -1.8283436298370361, + 0.3810080289840698, + -0.6684208512306213, + 0.5362142324447632, + -0.17851078510284424, + 1.083348035812378, + -1.9181195497512817, + -2.306365728378296 + ], + [ + 0.5487861037254333, + 0.6947635412216187, + -0.3694673776626587, + 1.3571128845214844, + -0.20053730905056, + -2.9099156856536865, + -1.493881106376648, + 0.9321863055229187, + -0.355974406003952, + 0.6561991572380066, + -0.29231804609298706, + 0.4913230240345001, + -1.8409770727157593, + -2.0504229068756104, + 0.14528872072696686, + 0.483799010515213, + -0.32654285430908203, + 1.6123273372650146, + 0.03987070918083191, + -0.8449068665504456, + -2.3494603633880615 + ], + [ + -0.3259732127189636, + -0.5385614633560181, + 0.7304226160049438, + -0.12314724922180176, + 0.03945803642272949, + -1.5919837951660156, + -1.907224416732788, + 0.3930681645870209, + -0.8835994005203247, + -0.3352096974849701, + 0.17923185229301453, + 0.17222784459590912, + -2.512176752090454, + -1.859424352645874, + 0.536429762840271, + 0.11454571038484573, + 0.3300909101963043, + 0.5491204261779785, + -0.04762554168701172, + -1.1976318359375, + -0.5251960158348083 + ], + [ + 0.33097749948501587, + -0.04192434251308441, + -0.34520047903060913, + -0.04110339283943176, + -0.5535940527915955, + -2.2057979106903076, + -2.5934860706329346, + -0.9023091793060303, + 0.5658158659934998, + -0.6125177145004272, + 1.095580816268921, + -0.05609257519245148, + -1.606997013092041, + -2.0733447074890137, + -0.5881220102310181, + 0.8953573703765869, + 0.9258373975753784, + 0.27281180024147034, + 0.7141706943511963, + -0.9142226576805115, + -1.3141751289367676 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05978316813707352, + 0.057473838329315186, + -0.09624473005533218, + -0.011721253395080566, + 0.028141841292381287, + -1.4311916828155518, + 0.0, + -0.024008113890886307, + 0.07049623876810074, + 0.03528204560279846, + -0.010002851486206055, + 0.010910652577877045, + -1.1452367305755615, + 0.0, + 0.020785929635167122, + -0.03374411538243294, + 0.11616832762956619, + 0.05772918462753296, + 0.01671847701072693, + -0.828272819519043, + 0.0 + ], + [ + -0.13237658143043518, + -0.3393171727657318, + 0.29588890075683594, + 0.8627858757972717, + -0.1666169911623001, + -2.520320177078247, + -1.6265922784805298, + -0.031582750380039215, + 0.11019003391265869, + -0.2699315845966339, + 0.9513242244720459, + 0.15523536503314972, + -1.7896702289581299, + -1.2776744365692139, + -0.3942914605140686, + -0.30590808391571045, + -0.3098926246166229, + 0.8258905410766602, + 0.29909753799438477, + -1.5711073875427246, + -1.5744072198867798 + ] + ], + "predicted_scores": [ + 3.636545181274414, + 3.3324379920959473, + 3.0866990089416504, + 2.9894821643829346, + 2.8972675800323486, + 2.6028425693511963, + 1.7788119316101074, + 1.3280467987060547 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "706479600", + "selected_index": 0, + "state_hash": "b482f902b03ca480898798a8b5f19263d7cbb18c066305330b0796b607427fa7", + "stored_base_utility": 0.8197344541549683, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03719887137413025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03719887137413025 + }, + "base_progress": 0.03719887137413025, + "base_success": false, + "base_utility": 0.03719887137413025, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03183290362358093, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03183290362358093 + }, + { + "contact_quality": null, + "progress": 0.019060388207435608, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019060388207435608 + }, + { + "contact_quality": null, + "progress": 0.1717282235622406, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1717282235622406 + }, + { + "contact_quality": null, + "progress": 0.044950783252716064, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044950783252716064 + }, + { + "contact_quality": null, + "progress": 0.012144356966018677, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.012144356966018677 + }, + { + "contact_quality": null, + "progress": 0.032083868980407715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.032083868980407715 + }, + { + "contact_quality": null, + "progress": 0.03983277082443237, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03983277082443237 + }, + { + "contact_quality": null, + "progress": 0.2203868180513382, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2203868180513382 + } + ], + "candidate_progress": [ + 0.03183290362358093, + 0.019060388207435608, + 0.1717282235622406, + 0.044950783252716064, + 0.012144356966018677, + 0.032083868980407715, + 0.03983277082443237, + 0.2203868180513382 + ], + "candidate_source_chart_ids": [ + "ms-0a4d4909900fb0bc4a9a", + "ms-0b1dd0e538b8df15a6e4", + "ms-ee2fcbc94d41385713d0", + "ms-bb299baa34e49eafb9fc", + "ms-07b479748a91a9c92b20", + "ms-2fef2f43af8009bb88f1", + "ms-d4c5d7eea19d94314e2e", + "ms-5ac1bf2d7d319955d45d" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank7", + "ctt_transport_rank6", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "generated_tangents": [ + [ + 0.16383254528045654, + 0.42179369926452637, + 0.16461342573165894, + -2.1338515281677246, + 0.08883903920650482, + 0.5972244739532471, + 0.10003271698951721, + -0.2376829981803894, + -0.26638832688331604, + -0.07205671817064285, + 0.9734528064727783, + 0.11627578735351562, + -1.5066206455230713, + 0.2632649540901184, + -0.2858303189277649, + -0.019674409180879593, + 0.1757156401872635, + 0.048682138323783875, + -0.12139785289764404, + -1.718052625656128, + -0.2075815498828888 + ], + [ + -0.10916204750537872, + 0.6658870577812195, + 0.12393003702163696, + -2.1848275661468506, + 0.17044630646705627, + 0.4360089898109436, + 0.26449263095855713, + -0.34011539816856384, + -0.2549479007720947, + -0.1827726662158966, + 1.1058545112609863, + -0.21791580319404602, + -1.6534463167190552, + 0.33526065945625305, + -0.292739599943161, + 0.03876979649066925, + 0.13243404030799866, + 0.24716481566429138, + -0.20260515809059143, + -2.3194258213043213, + -0.5518152713775635 + ], + [ + -0.17379315197467804, + 0.6714427471160889, + 0.12935209274291992, + -2.3667335510253906, + 0.142299085855484, + 0.7797820568084717, + 0.14695042371749878, + -0.3271960914134979, + -0.2474137544631958, + -0.12848272919654846, + 1.0801074504852295, + -0.19133707880973816, + -1.5834693908691406, + 0.47572988271713257, + -0.2721030116081238, + 0.0743202418088913, + 0.14934219419956207, + 0.2220892310142517, + -0.18336063623428345, + -2.328981399536133, + -0.32035452127456665 + ], + [ + -0.17327190935611725, + 0.648026704788208, + 0.23139452934265137, + -2.241300106048584, + 0.21182551980018616, + 0.4038086235523224, + 0.2869844436645508, + -0.2783063054084778, + -0.3269444406032562, + -0.2556919455528259, + 1.1562069654464722, + -0.09216046333312988, + -1.6788058280944824, + 0.34088200330734253, + -0.28028857707977295, + -0.09964674711227417, + 0.2195674479007721, + 0.3082142174243927, + -0.1626938283443451, + -2.2618508338928223, + -0.5362296104431152 + ], + [ + -0.14865826070308685, + 0.6718940138816833, + 0.1439131200313568, + -2.2066173553466797, + 0.1279124617576599, + 0.664729118347168, + 0.14038732647895813, + -0.2893095314502716, + -0.260088175535202, + -0.1551385521888733, + 1.1699298620224, + -0.03954972326755524, + -1.5320545434951782, + 0.28058522939682007, + -0.30633148550987244, + 0.01826954074203968, + 0.2276003062725067, + 0.2617988586425781, + -0.11420491337776184, + -2.2845611572265625, + -0.3470449149608612 + ], + [ + -0.13937751948833466, + 0.6060163378715515, + 0.1937028467655182, + -2.2179930210113525, + 0.23789609968662262, + 0.5366073846817017, + 0.35657814145088196, + -0.3390844166278839, + -0.2873803675174713, + -0.1804806888103485, + 1.081243634223938, + -0.0493759959936142, + -1.562054991722107, + 0.36232563853263855, + -0.28738126158714294, + 0.010354973375797272, + 0.2523888945579529, + 0.2718321681022644, + -0.20946475863456726, + -2.313732862472534, + -0.53669273853302 + ], + [ + -0.07790698111057281, + 0.39966386556625366, + 0.16148176789283752, + -2.022827625274658, + 0.0058796703815460205, + 0.6123775839805603, + -0.19460445642471313, + -0.18917885422706604, + -0.21152812242507935, + -0.24591153860092163, + 0.8451730608940125, + 0.22218969464302063, + -1.4132885932922363, + -0.14920301735401154, + -0.13171988725662231, + 0.02767074480652809, + 0.02690739929676056, + -0.03900305926799774, + -0.3170877695083618, + -2.0181307792663574, + -0.5251866579055786 + ], + [ + -0.19375617802143097, + 0.6010558009147644, + 0.05264982581138611, + -2.406672954559326, + 0.336072713136673, + 0.7744350433349609, + 0.5523248314857483, + -0.468593567609787, + -0.2784287929534912, + -0.22781330347061157, + 0.9399668574333191, + -0.0835561454296112, + -1.832580327987671, + 0.4588161110877991, + -0.3360155522823334, + 0.054078880697488785, + 0.1387353241443634, + 0.21493403613567352, + -0.30240166187286377, + -2.459296941757202, + -0.6023001670837402 + ] + ], + "generated_utilities": [ + 0.03183290362358093, + 0.019060388207435608, + 0.1717282235622406, + 0.044950783252716064, + 0.012144356966018677, + 0.032083868980407715, + 0.03983277082443237, + 0.2203868180513382 + ], + "group_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558897972106934, + 2.0, + 0.04248148202896118, + 0.05394089221954346, + 0.057141318917274475, + 0.03495943546295166, + 0.08633947372436523, + 0.046462297439575195, + 0.043458908796310425, + 0.023027777671813965, + 0.061922311782836914, + 0.034134700894355774, + 0.010651648044586182, + 0.01158277690410614, + 0.02622869610786438, + 0.012083128094673157 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.4298704266548157, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + 0.09995961934328079, + 1.162198781967163, + 0.0, + -0.00648990273475647, + -0.3028862476348877, + -0.0050805360078811646, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + 0.23185759782791138, + -0.11529958993196487, + -0.058992013335227966, + -0.010845988988876343, + 0.06423944234848022, + -2.0730173587799072, + 0.0 + ], + [ + -0.48041507601737976, + 0.6002404689788818, + 0.5398462414741516, + -2.198392152786255, + 0.1505453884601593, + 1.162198781967163, + 0.0, + 0.32028457522392273, + -0.264798641204834, + -0.27968189120292664, + 1.143568515777588, + 0.623285710811615, + -1.6758570671081543, + 0.0, + 0.3762620687484741, + 0.20924779772758484, + 0.033039893954992294, + -0.22763806581497192, + 0.8496624827384949, + -2.0730173587799072, + 0.0 + ], + [ + -0.4385460615158081, + 1.6002404689788818, + 1.1335551738739014, + -3.198392152786255, + 0.11579504609107971, + 2.162198781967163, + -1.8792641162872314, + 0.44874686002731323, + -1.264798641204834, + 0.08758684247732162, + 2.143568515777588, + 1.074564814567566, + -2.6758570671081543, + 1.4318934679031372, + -0.01953941211104393, + 0.012724362313747406, + 0.02134235017001629, + 0.39346790313720703, + -0.15033751726150513, + -3.0730173587799072, + -1.573193907737732 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.758528232574463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8637869358062744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.146387815475464 + ], + [ + -0.4582313299179077, + 1.465004801750183, + 0.9327377080917358, + -2.900977373123169, + -0.12230202555656433, + 2.23574161529541, + -2.1357157230377197, + 0.42294827103614807, + -1.506612777709961, + 0.29363107681274414, + 2.034971237182617, + 1.023054599761963, + -2.559215784072876, + 1.1742838621139526, + 0.026679620146751404, + -0.0811251625418663, + 0.16039283573627472, + 0.3527735471725464, + 0.041033580899238586, + -2.8834660053253174, + -1.2895927429199219 + ], + [ + -0.8261920809745789, + 1.8162921667099, + 1.1455696821212769, + -3.5306012630462646, + -0.15104833245277405, + 2.447983741760254, + -2.128190755844116, + 0.19818204641342163, + -1.146593689918518, + 0.03477605804800987, + 2.4299705028533936, + 0.8797258734703064, + -2.331338882446289, + 1.5516352653503418, + -0.19862954318523407, + -0.2736409604549408, + -0.21770769357681274, + 0.47925806045532227, + -0.15124429762363434, + -3.0052826404571533, + -1.2108670473098755 + ], + [ + -0.2760573625564575, + 1.6254181861877441, + 0.8188510537147522, + -3.0690624713897705, + -0.22635167837142944, + 2.051481246948242, + -1.417292833328247, + 0.026889145374298096, + -1.3347617387771606, + -0.15878626704216003, + 1.9778507947921753, + 0.7640928030014038, + -3.1165366172790527, + 1.862673282623291, + -0.451316773891449, + -0.15708383917808533, + 0.26080578565597534, + 0.6509285569190979, + 0.23991608619689941, + -3.50675892829895, + -1.297291874885559 + ], + [ + -0.8321853280067444, + 1.9553383588790894, + 1.5495907068252563, + -3.790785312652588, + 0.3716217279434204, + 2.5994434356689453, + -2.1380205154418945, + -0.10642725229263306, + -0.7562419772148132, + -0.442053884267807, + 2.3244006633758545, + 1.161316990852356, + -2.699451208114624, + 1.6399294137954712, + 0.31003186106681824, + 0.5301566123962402, + -0.16996291279792786, + 0.9281869530677795, + 0.3683878183364868, + -2.650294780731201, + -1.1098875999450684 + ], + [ + -0.6997023820877075, + 1.4820023775100708, + 1.3613333702087402, + -3.8556365966796875, + -0.5036565065383911, + 1.4896399974822998, + -2.322740077972412, + 0.03597790002822876, + -1.1858596801757812, + -0.018024958670139313, + 2.0489397048950195, + 0.8914362192153931, + -2.3395535945892334, + 1.1835731267929077, + 0.5559698343276978, + 0.5892979502677917, + 0.4496755301952362, + 0.6100684404373169, + 0.22263532876968384, + -3.0098915100097656, + -1.455836296081543 + ], + [ + -0.594323456287384, + 1.846432089805603, + 0.8105549812316895, + -2.6211562156677246, + 0.2629232406616211, + 1.4025108814239502, + -2.3828163146972656, + 0.530456006526947, + -1.3874722719192505, + 0.49274298548698425, + 1.5545053482055664, + 0.4927254915237427, + -3.1490466594696045, + 1.5643255710601807, + 0.5331913232803345, + -0.53307044506073, + 0.7433652281761169, + 0.6546258926391602, + -0.74479740858078, + -3.721141815185547, + -0.9948303699493408 + ], + [ + -0.4409070312976837, + 2.4138050079345703, + 1.3223544359207153, + -3.3717033863067627, + -0.03033369779586792, + 2.350304365158081, + -2.112061023712158, + -0.22452247142791748, + -1.420605182647705, + -0.026168949902057648, + 1.4026775360107422, + 0.9532885551452637, + -2.6276464462280273, + 2.1231069564819336, + -0.49286600947380066, + -0.3450928032398224, + 0.5427718758583069, + -0.03872644901275635, + -0.6895387172698975, + -2.357243061065674, + -2.3816022872924805 + ], + [ + 0.5156354308128357, + 2.2980997562408447, + 2.0945849418640137, + -3.690303087234497, + 1.0696191787719727, + 2.266188144683838, + -1.0399069786071777, + 0.012727499008178711, + -1.2163764238357544, + 0.5646419525146484, + 1.1664416790008545, + 1.563025951385498, + -3.425304412841797, + 0.5126914978027344, + -0.18885818123817444, + -0.5277795791625977, + -0.1741226613521576, + 0.2763793468475342, + 0.8411282896995544, + -3.2598915100097656, + -2.2270073890686035 + ], + [ + -1.2896747589111328, + 2.3828718662261963, + 1.191500186920166, + -3.9747776985168457, + 0.7604506015777588, + 2.298020839691162, + -1.0116384029388428, + 1.177309513092041, + -1.6719520092010498, + -0.4176469147205353, + 2.709542989730835, + 0.26378321647644043, + -2.119570255279541, + 0.7668893933296204, + 0.6772738099098206, + -0.5338667035102844, + -0.7434203028678894, + 0.2167283445596695, + 0.03269544243812561, + -3.895913600921631, + -0.9124676585197449 + ], + [ + -1.323930263519287, + 2.338547468185425, + 1.4313098192214966, + -2.960175037384033, + 1.1079072952270508, + 1.5746114253997803, + -1.8946435451507568, + -0.4213535189628601, + -1.922096848487854, + 0.6645004153251648, + 2.1448850631713867, + 0.2334994077682495, + -3.4603066444396973, + 1.5447293519973755, + 0.25094738602638245, + 0.7163193821907043, + -0.10547596216201782, + -0.42469942569732666, + -0.49689921736717224, + -2.8712027072906494, + -1.6389087438583374 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027082443237304688, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + -0.03311736881732941, + 1.162198781967163, + 0.0, + 0.043807268142700195, + -0.264798641204834, + 0.0069312527775764465, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + -0.01150114368647337, + -0.007074322551488876, + -0.018292216584086418, + -0.035625845193862915, + 0.05120308697223663, + -2.0730173587799072, + 0.0 + ] + ], + "predicted_scores": [ + 4.430130481719971, + 3.745222568511963, + 3.388216495513916, + 3.2002310752868652, + 3.1686899662017822, + 3.1340255737304688, + 3.0641114711761475, + 3.0597243309020996 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1919198188", + "selected_index": 0, + "state_hash": "6b9e4c034c70f8e93116064dbcb48aa5fea37958e04fa30734f597c38aefea8b", + "stored_base_utility": 0.8558897972106934, + "task_id": "StackCube-v1" + } + ], + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "source_index": "data/cil_charts_rgb_refs/train/index.json", + "source_split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_index": "data/cil_charts_rgb_refs/val/index.json", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381" +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..e419fd06b69d608df190e361b3de03dae9b7568b --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics.json @@ -0,0 +1,3767 @@ +{ + "epsilon": 0.0, + "k": 8, + "mode": "measured", + "num_rows": 69, + "rows": [ + { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "chart_id": "ms-484ea653bbca2de3a4a2", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07419277505792353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "seed": "1231780997", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.2729938626289368, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.0031174421310424805, + "chart_id": "ms-4b4cd12d70c0615e0930", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14400158741031796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2148684412240982, + "proposal_oracle_utility_gain_over_base_at_8": -0.31350521743297577, + "seed": "1025740212", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21175099909305573, + "selected_utility_gain_over_base_at_8": -0.31662265956401825, + "selector_regret_at_8": 0.0031174421310424805, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7851315587759018, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.7882490009069443 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-6e268d1e2915740816a1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0862090980213101, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7181311249732971, + "proposal_oracle_utility_gain_over_base_at_8": -1.2818688750267029, + "seed": "1475440973", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7181311249732971, + "selected_utility_gain_over_base_at_8": -1.2818688750267029, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2818688750267029, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2818688750267029 + }, + { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2779860496520996, + "chart_id": "ms-8864796e31bbe93d7c5d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.23627606487156766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "seed": "551504635", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7220139503479004, + "selected_utility_gain_over_base_at_8": -0.001958608627319336, + "selector_regret_at_8": 1.2779860496520996, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2779860496520996 + }, + { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.06326162815093994, + "chart_id": "ms-b04b489f4fbad93b65ca", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06911736513585261, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7256770730018616, + "proposal_oracle_utility_gain_over_base_at_8": 0.3033645451068878, + "seed": "1113268940", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6624154448509216, + "selected_utility_gain_over_base_at_8": 0.24010291695594788, + "selector_regret_at_8": 0.06326162815093994, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2743229269981384, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.3375845551490784 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.008421123027801514, + "chart_id": "ms-c663f7e73e7e8376a05c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026961937751403514, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7255622744560242, + "proposal_oracle_utility_gain_over_base_at_8": -1.2744377255439758, + "seed": "1528740417", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7171411514282227, + "selected_utility_gain_over_base_at_8": -1.2828588485717773, + "selector_regret_at_8": 0.008421123027801514, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2744377255439758, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2828588485717773 + }, + { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "seed": "1273247072", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "task_id": "PickCube-v1", + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + { + "base_success": 0.0, + "base_utility": 0.050487834960222244, + "branch_car_at_8": 0.09754210896790028, + "chart_id": "ms-0345d8ac530f88cefd61", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1798098385334015, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.27579031898477024, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.11701449006795883, + "proposal_oracle_utility_gain_over_base_at_8": 0.06652665510773659, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.019472381100058556, + "selected_utility_gain_over_base_at_8": -0.03101545386016369, + "selector_regret_at_8": 0.09754210896790028, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.06279534846544266, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.16033745743334293 + }, + { + "base_success": 0.0, + "base_utility": 0.07743936777114868, + "branch_car_at_8": 0.005068322643637657, + "chart_id": "ms-0819eb89e627595e7e1b", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.14572584629058838, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2986384171869234, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.013550917617976665, + "proposal_oracle_utility_gain_over_base_at_8": -0.06388845015317202, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.008482594974339008, + "selected_utility_gain_over_base_at_8": -0.06895677279680967, + "selector_regret_at_8": 0.005068322643637657, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1321749286726117, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13724325131624937 + }, + { + "base_success": 0.0, + "base_utility": 0.19400137662887573, + "branch_car_at_8": 0.07661644369363785, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.30484520189300646, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.1163921132683754, + "proposal_oracle_utility_gain_over_base_at_8": -0.07760926336050034, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03977566957473755, + "selected_utility_gain_over_base_at_8": -0.15422570705413818, + "selector_regret_at_8": 0.07661644369363785, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8836078867316246, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9602243304252625 + }, + { + "base_success": 0.0, + "base_utility": 0.04583283141255379, + "branch_car_at_8": 0.0687156543135643, + "chart_id": "ms-126eb8b36769f9ba4671", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1985912322998047, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.22659482311753865, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.17300955951213837, + "proposal_oracle_utility_gain_over_base_at_8": 0.12717672809958458, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.10429390519857407, + "selected_utility_gain_over_base_at_8": 0.05846107378602028, + "selector_regret_at_8": 0.0687156543135643, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.02558167278766632, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.09429732710123062 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-16661b2fd2c85f57a66c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.001000205724267289, + "chart_id": "ms-20e1efbb745fd02abc95", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16573502466028908, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.001480837701819837, + "proposal_oracle_utility_gain_over_base_at_8": -1.9985191622981802, + "seed": "1375492786", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.00048063197755254805, + "selected_utility_gain_over_base_at_8": -1.9995193680224475, + "selector_regret_at_8": 0.001000205724267289, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2419701317558065, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2429703374800738 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-2dc5138251afd26a040d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565621826", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.08252689987421036, + "branch_car_at_8": 0.08448734506964684, + "chart_id": "ms-387e6771a55be98f1a59", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20744429528713226, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06284898813868309, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.14589862525463104, + "proposal_oracle_utility_gain_over_base_at_8": 0.06337172538042068, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06141128018498421, + "selected_utility_gain_over_base_at_8": -0.02111561968922615, + "selector_regret_at_8": 0.08448734506964684, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.06154567003250122, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.14603301510214806 + }, + { + "base_success": 0.0, + "base_utility": 0.03917040303349495, + "branch_car_at_8": 0.0, + "chart_id": "ms-38f7298184a2c5005e94", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1767096221446991, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.11490267487254319, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.11718571186065674, + "proposal_oracle_utility_gain_over_base_at_8": 0.07801530882716179, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.11718571186065674, + "selected_utility_gain_over_base_at_8": 0.07801530882716179, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05952391028404236, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.05952391028404236 + }, + { + "base_success": 0.0, + "base_utility": 0.009124159812927246, + "branch_car_at_8": 0.01312979031354189, + "chart_id": "ms-482c5c8d208b5907a2c2", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1834145486354828, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24422123975177484, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.021352291107177734, + "proposal_oracle_utility_gain_over_base_at_8": 0.012228131294250488, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.008222500793635845, + "selected_utility_gain_over_base_at_8": -0.0009016590192914009, + "selector_regret_at_8": 0.01312979031354189, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.16206225752830505, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.17519204784184694 + }, + { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 1.9076288938522339, + "chart_id": "ms-4da03300d92420e80ed0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1055876219626688, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "seed": "1747178030", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09237110614776611, + "selected_utility_gain_over_base_at_8": -0.054260119795799255, + "selector_regret_at_8": 1.9076288938522339, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9076288938522339 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.009623387828469276, + "chart_id": "ms-57be9ae181c077e7821f", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15538118252826771, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.01863894797861576, + "proposal_oracle_utility_gain_over_base_at_8": -1.9813610520213842, + "seed": "1549324805", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.009015560150146484, + "selected_utility_gain_over_base_at_8": -1.9909844398498535, + "selector_regret_at_8": 0.009623387828469276, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9813610520213842, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9909844398498535 + }, + { + "base_success": 0.0, + "base_utility": 0.1544450968503952, + "branch_car_at_8": 0.0, + "chart_id": "ms-6388b5d2356c556c3e55", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13157149394108555, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07292797416448593, + "proposal_oracle_utility_gain_over_base_at_8": -0.08151712268590927, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07292797416448593, + "selected_utility_gain_over_base_at_8": -0.08151712268590927, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.927072025835514, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.927072025835514 + }, + { + "base_success": 0.0, + "base_utility": 0.06173701956868172, + "branch_car_at_8": 0.028241891413927078, + "chart_id": "ms-638d2ba1917eff4d09e0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.11176212877035141, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12990558981115807, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08538073301315308, + "proposal_oracle_utility_gain_over_base_at_8": 0.02364371344447136, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.057138841599226, + "selected_utility_gain_over_base_at_8": -0.004598177969455719, + "selector_regret_at_8": 0.028241891413927078, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.026381395757198334, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.05462328717112541 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00025266362354159355, + "chart_id": "ms-652ad552ba98d2736620", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2367449700832367, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.11085562899340784, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.007886429317295551, + "proposal_oracle_utility_gain_over_base_at_8": -1.9921135706827044, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.007633765693753958, + "selected_utility_gain_over_base_at_8": -1.992366234306246, + "selector_regret_at_8": 0.00025266362354159355, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.22885854076594114, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22911120438948274 + }, + { + "base_success": 0.0, + "base_utility": 0.03784912824630737, + "branch_car_at_8": 0.01684868335723877, + "chart_id": "ms-674fad0e7c042d23629e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.15573063492774963, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2935561879950145, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.057549476623535156, + "proposal_oracle_utility_gain_over_base_at_8": 0.019700348377227783, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04070079326629639, + "selected_utility_gain_over_base_at_8": 0.0028516650199890137, + "selector_regret_at_8": 0.01684868335723877, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09818115830421448, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11502984166145325 + }, + { + "base_success": 0.0, + "base_utility": 0.12791956961154938, + "branch_car_at_8": 0.059114933013916016, + "chart_id": "ms-728b865677134d3788e3", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.22658847272396088, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24450025940895065, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.13260090351104736, + "proposal_oracle_utility_gain_over_base_at_8": 0.004681333899497986, + "seed": "750448296", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07348597049713135, + "selected_utility_gain_over_base_at_8": -0.05443359911441803, + "selector_regret_at_8": 0.059114933013916016, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09398756921291351, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.15310250222682953 + }, + { + "base_success": 0.0, + "base_utility": 0.20529556274414062, + "branch_car_at_8": 0.0617663636803627, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.27989473938941956, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.38411828214949667, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09449902921915054, + "proposal_oracle_utility_gain_over_base_at_8": -0.11079653352499008, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03273266553878784, + "selected_utility_gain_over_base_at_8": -0.17256289720535278, + "selector_regret_at_8": 0.0617663636803627, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.185395710170269, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2471620738506317 + }, + { + "base_success": 0.0, + "base_utility": 0.18213583528995514, + "branch_car_at_8": 0.0027207134990021586, + "chart_id": "ms-7f57045c114ddad153b5", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.227622389793396, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10807032783608049, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.003711601020768285, + "proposal_oracle_utility_gain_over_base_at_8": -0.17842423426918685, + "seed": "278008524", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0009908875217661262, + "selected_utility_gain_over_base_at_8": -0.181144947768189, + "selector_regret_at_8": 0.0027207134990021586, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2239107887726277, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22663150227162987 + }, + { + "base_success": 0.0, + "base_utility": 0.0056487321853637695, + "branch_car_at_8": 0.0, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1631626933813095, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2438992767980762, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.020783662796020508, + "proposal_oracle_utility_gain_over_base_at_8": 0.015134930610656738, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.020783662796020508, + "selected_utility_gain_over_base_at_8": 0.015134930610656738, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.142379030585289, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.142379030585289 + }, + { + "base_success": 0.0, + "base_utility": 0.00698717450723052, + "branch_car_at_8": 0.02401920035481453, + "chart_id": "ms-8a72b73ac452bc190aeb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.10329639911651611, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15532500254873144, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08651334047317505, + "proposal_oracle_utility_gain_over_base_at_8": 0.07952616596594453, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06249414011836052, + "selected_utility_gain_over_base_at_8": 0.05550696561113, + "selector_regret_at_8": 0.02401920035481453, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.016783058643341064, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.040802258998155594 + }, + { + "base_success": 0.0, + "base_utility": 0.08285123109817505, + "branch_car_at_8": 0.015108168125152588, + "chart_id": "ms-a96bca1ef0b74f686018", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1572066992521286, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.03601989971092138, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09714335203170776, + "proposal_oracle_utility_gain_over_base_at_8": 0.014292120933532715, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.08203518390655518, + "selected_utility_gain_over_base_at_8": -0.000816047191619873, + "selector_regret_at_8": 0.015108168125152588, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.06006334722042084, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.07517151534557343 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00563424825668335, + "chart_id": "ms-ac4fb33f3ab4db853995", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16883301005240778, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.14365969598293304, + "proposal_oracle_utility_gain_over_base_at_8": -1.856340304017067, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1380254477262497, + "selected_utility_gain_over_base_at_8": -1.8619745522737503, + "selector_regret_at_8": 0.00563424825668335, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.856340304017067, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8619745522737503 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "2054899126", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.037537217140197754, + "branch_car_at_8": 0.013502320274710655, + "chart_id": "ms-c832439976a37f52d094", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2594742774963379, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22009063480440055, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.025354644283652306, + "proposal_oracle_utility_gain_over_base_at_8": -0.012182572856545448, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.01185232400894165, + "selected_utility_gain_over_base_at_8": -0.025684893131256104, + "selector_regret_at_8": 0.013502320274710655, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.23411963321268559, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24762195348739624 + }, + { + "base_success": 0.0, + "base_utility": 0.09138330072164536, + "branch_car_at_8": 0.0, + "chart_id": "ms-c96b415f083f60857334", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.26426148414611816, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14208489100702013, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.011022846214473248, + "proposal_oracle_utility_gain_over_base_at_8": -0.08036045450717211, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011022846214473248, + "selected_utility_gain_over_base_at_8": -0.08036045450717211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2532386379316449, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2532386379316449 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.12736602361488303, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "534717141", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.14809735119342804, + "branch_car_at_8": 0.0008638898143544793, + "chart_id": "ms-da142025733b53fcf7eb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2643269896507263, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18818951741396497, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.001536289812065661, + "proposal_oracle_utility_gain_over_base_at_8": -0.14656106138136238, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0006723999977111816, + "selected_utility_gain_over_base_at_8": -0.14742495119571686, + "selector_regret_at_8": 0.0008638898143544793, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.26279069983866066, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.26365458965301514 + }, + { + "base_success": 0.0, + "base_utility": 0.1362423449754715, + "branch_car_at_8": 0.005128046497702599, + "chart_id": "ms-e2bad6e99e8a281207cc", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20825156569480896, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15502450434938486, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.018407067283988, + "proposal_oracle_utility_gain_over_base_at_8": -0.1178352776914835, + "seed": "1549324805", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0132790207862854, + "selected_utility_gain_over_base_at_8": -0.1229633241891861, + "selector_regret_at_8": 0.005128046497702599, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.18984449841082096, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.19497254490852356 + }, + { + "base_success": 0.0, + "base_utility": 0.09695577621459961, + "branch_car_at_8": 0.0, + "chart_id": "ms-e30fd80adc089e9f697a", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18920521438121796, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1335740334444898, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.023269018158316612, + "proposal_oracle_utility_gain_over_base_at_8": -0.073686758056283, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.023269018158316612, + "selected_utility_gain_over_base_at_8": -0.073686758056283, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.16593619622290134, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.16593619622290134 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07539165019989014, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18186684363707276, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.14950889348983765, + "proposal_oracle_utility_gain_over_base_at_8": -1.8504911065101624, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.07411724328994751, + "selected_utility_gain_over_base_at_8": -1.9258827567100525, + "selector_regret_at_8": 0.07539165019989014, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8504911065101624, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9258827567100525 + }, + { + "base_success": 0.0, + "base_utility": 0.09514755010604858, + "branch_car_at_8": 0.029236994683742523, + "chart_id": "ms-ec013396e4aff6170ee1", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24502670764923096, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.06092325242684257, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07131137698888779, + "proposal_oracle_utility_gain_over_base_at_8": -0.023836173117160797, + "seed": "750448296", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.042074382305145264, + "selected_utility_gain_over_base_at_8": -0.05307316780090332, + "selector_regret_at_8": 0.029236994683742523, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.17371533066034317, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.2029523253440857 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.004942953586578369, + "chart_id": "ms-ee30ebaf98622759d888", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.45862923715160336, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.015581388957798481, + "proposal_oracle_utility_gain_over_base_at_8": -1.9844186110422015, + "seed": "636680137", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.010638435371220112, + "selected_utility_gain_over_base_at_8": -1.9893615646287799, + "selector_regret_at_8": 0.004942953586578369, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9844186110422015, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9893615646287799 + }, + { + "base_success": 0.0, + "base_utility": 0.026694953441619873, + "branch_car_at_8": 0.004920683801174164, + "chart_id": "ms-f3f1d07cfd59b823da47", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.17124073207378387, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.30358571851076954, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07510259002447128, + "proposal_oracle_utility_gain_over_base_at_8": 0.04840763658285141, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07018190622329712, + "selected_utility_gain_over_base_at_8": 0.043486952781677246, + "selector_regret_at_8": 0.004920683801174164, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.09613814204931259, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.10105882585048676 + }, + { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.934090182185173, + "chart_id": "ms-081952a63087fc8f83cd", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.40989236974714055, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "seed": "2141802816", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06590981781482697, + "selected_utility_gain_over_base_at_8": -0.0048974305391311646, + "selector_regret_at_8": 1.934090182185173, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.934090182185173 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-121bc2e6676c1ac10911", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "77147264", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-22f153f0d4fa656018e6", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1271718007", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 1.964881807565689, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10190418918720931, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9679730087518692, + "seed": "1881223807", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03511819243431091, + "selected_utility_gain_over_base_at_8": 0.0030912011861801147, + "selector_regret_at_8": 1.964881807565689, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.964881807565689 + }, + { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 1.9264162927865982, + "chart_id": "ms-2867deaca7e98582620b", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10709427493066431, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "seed": "1189392917", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0735837072134018, + "selected_utility_gain_over_base_at_8": -0.03483493626117706, + "selector_regret_at_8": 1.9264162927865982, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9264162927865982 + }, + { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "chart_id": "ms-365d8c6de6de16f66744", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3553584127845171, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "seed": "1997090716", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.03280790150165558, + "branch_car_at_8": 0.012138530611991882, + "chart_id": "ms-3c326d05fec66e1d6bea", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10448389878723242, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0349019318819046, + "proposal_oracle_utility_gain_over_base_at_8": 0.0020940303802490234, + "seed": "278008524", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02276340126991272, + "selected_utility_gain_over_base_at_8": -0.010044500231742859, + "selector_regret_at_8": 0.012138530611991882, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9650980681180954, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9772365987300873 + }, + { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 1.9626595228910446, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3619001242329895, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "seed": "794181626", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03734047710895538, + "selected_utility_gain_over_base_at_8": -0.14361557364463806, + "selector_regret_at_8": 1.9626595228910446, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9626595228910446 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-58950fed04d203b97ab9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.015047211698777683, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1113371261", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "chart_id": "ms-651bc0395d78d2819158", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06079377324000063, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "seed": "1483645336", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.953145906329155, + "chart_id": "ms-9fad82fd288e4296e354", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15442455751472706, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "seed": "1176943004", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04685409367084503, + "selected_utility_gain_over_base_at_8": -0.10775963962078094, + "selector_regret_at_8": 1.953145906329155, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.953145906329155 + }, + { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24035198055524792, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "seed": "998761162", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9461991488933563, + "chart_id": "ms-fc8084d5cd7da937fc02", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.44645064197830664, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1991414747", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.05380085110664368, + "selected_utility_gain_over_base_at_8": -1.9461991488933563, + "selector_regret_at_8": 1.9461991488933563, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9461991488933563 + }, + { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.007853284478187561, + "chart_id": "ms-3afd8959363e917591d4", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.34224193319263285, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24087366461753845, + "proposal_oracle_utility_gain_over_base_at_8": 0.0381220281124115, + "seed": "75070641", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2330203801393509, + "selected_utility_gain_over_base_at_8": 0.030268743634223938, + "selector_regret_at_8": 0.007853284478187561, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7591263353824615, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.766979619860649 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-3ff1aedb72f65a182b58", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15812100047218913, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1055567618", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.6509099900722504, + "chart_id": "ms-4048f8f6e047422e3c79", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32609963351937027, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.812659502029419, + "proposal_oracle_utility_gain_over_base_at_8": 0.6161754280328751, + "seed": "1055157354", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.16174951195716858, + "selected_utility_gain_over_base_at_8": -0.034734562039375305, + "selector_regret_at_8": 0.6509099900722504, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.187340497970581, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8382504880428314 + }, + { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.03571280837059021, + "chart_id": "ms-5b26c33feed6992386b1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24182565394435904, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24029652774333954, + "proposal_oracle_utility_gain_over_base_at_8": 0.0418534129858017, + "seed": "1522272686", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20458371937274933, + "selected_utility_gain_over_base_at_8": 0.006140604615211487, + "selector_regret_at_8": 0.03571280837059021, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7597034722566605, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7954162806272507 + }, + { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 1.1816681623458862, + "chart_id": "ms-63ac245579b1188935ad", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.036206191308278124, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "seed": "211245842", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8183318376541138, + "selected_utility_gain_over_base_at_8": -0.020502567291259766, + "selector_regret_at_8": 1.1816681623458862, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1816681623458862 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.1815597414970398, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.05717674118582694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1665739511", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.8184402585029602, + "selected_utility_gain_over_base_at_8": -1.1815597414970398, + "selector_regret_at_8": 1.1815597414970398, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1815597414970398 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.6129319667816162, + "chart_id": "ms-72777024349c2aacd5e0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3193203145842822, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.8128684163093567, + "proposal_oracle_utility_gain_over_base_at_8": -1.1871315836906433, + "seed": "256770874", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.19993644952774048, + "selected_utility_gain_over_base_at_8": -1.8000635504722595, + "selector_regret_at_8": 0.6129319667816162, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1871315836906433, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8000635504722595 + }, + { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3404468297958374, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2708447347116597, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5798611640930176, + "proposal_oracle_utility_gain_over_base_at_8": 0.376139298081398, + "seed": "957101489", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.23941433429718018, + "selected_utility_gain_over_base_at_8": 0.03569246828556061, + "selector_regret_at_8": 0.3404468297958374, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4201388359069824, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7605856657028198 + }, + { + "base_success": 0.0, + "base_utility": 0.8431589603424072, + "branch_car_at_8": 1.7759137749671936, + "chart_id": "ms-9a14552b7db695dee92c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06758506233539574, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1568410396575928, + "seed": "77147264", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2240862250328064, + "selected_utility_gain_over_base_at_8": -0.6190727353096008, + "selector_regret_at_8": 1.7759137749671936, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7759137749671936 + }, + { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 0.0, + "chart_id": "ms-9f5d57fe2153fd267774", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2544258391757567, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "seed": "1607773860", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1673271656036377, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.007964074611663818, + "chart_id": "ms-b0fee26e1db967f90b81", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.04701404433315851, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8245300054550171, + "proposal_oracle_utility_gain_over_base_at_8": -0.0026850104331970215, + "seed": "1358421325", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8165659308433533, + "selected_utility_gain_over_base_at_8": -0.01064908504486084, + "selector_regret_at_8": 0.007964074611663818, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.175469994544983, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1834340691566467 + }, + { + "base_success": 0.0, + "base_utility": 0.8307622671127319, + "branch_car_at_8": 1.7905852794647217, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21421799686366766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.169237732887268, + "seed": "1271718007", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20941472053527832, + "selected_utility_gain_over_base_at_8": -0.6213475465774536, + "selector_regret_at_8": 1.7905852794647217, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7905852794647217 + }, + { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.01654154062271118, + "chart_id": "ms-d1960368ef76bbb10669", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1925298309196861, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22490769624710083, + "proposal_oracle_utility_gain_over_base_at_8": 0.02458128333091736, + "seed": "416183106", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20836615562438965, + "selected_utility_gain_over_base_at_8": 0.008039742708206177, + "selector_regret_at_8": 0.01654154062271118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7750923037528992, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.7916338443756104 + }, + { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 1.185575544834137, + "chart_id": "ms-efdc07c985935d871adf", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15129914778803955, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "seed": "706479600", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814424455165863, + "selected_utility_gain_over_base_at_8": -0.020664334297180176, + "selector_regret_at_8": 1.185575544834137, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.185575544834137 + }, + { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.18855391442775726, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2569381375337174, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2203868180513382, + "proposal_oracle_utility_gain_over_base_at_8": 0.18318794667720795, + "seed": "1919198188", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03183290362358093, + "selected_utility_gain_over_base_at_8": -0.005365967750549316, + "selector_regret_at_8": 0.18855391442775726, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7796131819486618, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.968167096376419 + } + ], + "summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.46099290780141844, + "low": 0.19680851063829788, + "mean": 0.3085106382978724, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 1.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3176470588235294, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.4057971014492754, + "low": 0.18840579710144928, + "mean": 0.2753623188405797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.0808477015729916, + "low": 0.6013251688004412, + "mean": 0.8163502178883101, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8163502178883101, + "per_seed": { + "1025740212": 0.528373658657074, + "1055157354": 0.19648407399654388, + "1055567618": 2.0, + "1113268940": 0.42231252789497375, + "1113371261": 2.0, + "1176943004": 0.15461373329162598, + "1189392917": 0.10841864347457886, + "1231780997": 0.7270061373710632, + "1271718007": 1.415381133556366, + "1273247072": 0.15342965722084045, + "1358421325": 0.8272150158882141, + "1375492786": 2.0, + "141984615": 0.09381213908394177, + "1475440973": 2.0, + "1483645336": 0.01641605794429779, + "1522272686": 0.19844311475753784, + "1528740417": 2.0, + "1549324805": 1.3787474483251572, + "1607773860": 0.8326728343963623, + "1665739511": 2.0, + "1747178030": 0.14663122594356537, + "1881223807": 0.0320269912481308, + "1919198188": 0.03719887137413025, + "1991414747": 2.0, + "1997090716": 0.13927479088306427, + "2015956957": 0.06711978986859321, + "2054899126": 2.0, + "211245842": 0.8388344049453735, + "2141802816": 0.07080724835395813, + "256770874": 2.0, + "278008524": 0.10747186839580536, + "416183106": 0.20032641291618347, + "534717141": 2.0, + "551504635": 0.7239725589752197, + "565369392": 0.7997656464576721, + "565621826": 2.0, + "585904130": 0.07529209802548091, + "626486813": 0.5054400166263804, + "636680137": 2.0, + "706479600": 0.8350887894630432, + "750448296": 0.11153355985879898, + "75070641": 0.20275163650512695, + "767747424": 0.5475241877138615, + "77147264": 1.4215794801712036, + "794181626": 0.18095605075359344, + "957101489": 0.20372186601161957, + "998761162": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9058472741949395, + "low": 0.4133020142714182, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.923898630168127, + "low": 0.5448560892550302, + "mean": 0.7180729379567007, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6866153551741166, + "low": 0.2764395795932208, + "mean": 0.4850921972540409, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4850921972540409, + "per_seed": { + "1025740212": 0.0031174421310424805, + "1055157354": 0.6509099900722504, + "1055567618": 0.0, + "1113268940": 0.06326162815093994, + "1113371261": 0.0, + "1176943004": 1.953145906329155, + "1189392917": 1.9264162927865982, + "1231780997": 0.0, + "1271718007": 0.8952926397323608, + "1273247072": 0.42682039737701416, + "1358421325": 0.007964074611663818, + "1375492786": 0.001000205724267289, + "141984615": 0.03783805563580245, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.03571280837059021, + "1528740417": 0.008421123027801514, + "1549324805": 0.004917144775390625, + "1607773860": 0.0, + "1665739511": 1.1815597414970398, + "1747178030": 1.9076288938522339, + "1881223807": 1.964881807565689, + "1919198188": 0.18855391442775726, + "1991414747": 1.9461991488933563, + "1997090716": 0.0, + "2015956957": 0.005367537960410118, + "2054899126": 0.0, + "211245842": 1.1816681623458862, + "2141802816": 1.934090182185173, + "256770874": 0.6129319667816162, + "278008524": 0.0074296220554970205, + "416183106": 0.01654154062271118, + "534717141": 0.0, + "551504635": 1.2779860496520996, + "565369392": 0.048005685210227966, + "565621826": 0.0, + "585904130": 0.004500773424903552, + "626486813": 0.02813516021706164, + "636680137": 0.004942953586578369, + "706479600": 1.185575544834137, + "750448296": 0.04417596384882927, + "75070641": 0.007853284478187561, + "767747424": 0.04542438860516995, + "77147264": 0.8879568874835968, + "794181626": 1.9626595228910446, + "957101489": 0.3404468297958374, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6847154450722229, + "low": 0.18113008173041084, + "mean": 0.4454944344309131, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4454944344309131, + "per_task": { + "LiftPegUpright-v1": 0.2254643738269806, + "PickCube-v1": 0.42682039737701416, + "PullCube-v1": 0.07680898695852033, + "PushCube-v1": 0.8999639531740775, + "StackCube-v1": 0.5984144608179728 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5186884595309823, + "low": 0.2226778654005392, + "mean": 0.36328783334001724, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9148936170212765, + "low": 0.7432624113475178, + "mean": 0.8358156028368794, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8358156028368794, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.782608695652174, + "low": 0.5507246376811594, + "mean": 0.6666666666666666, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.850041654515774, + "low": 1.5397213380149701, + "mean": 1.7062028111638965, + "n": 47, + "num_samples": 200 + }, + "mean": 1.7062028111638965, + "per_seed": { + "1025740212": 2.0, + "1055157354": 2.0, + "1055567618": 2.0, + "1113268940": 2.0, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 2.0, + "1358421325": 2.0, + "1375492786": 0.24345096945762634, + "141984615": 0.20044784247875214, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 2.0, + "1528740417": 2.0, + "1549324805": 1.4027505218982697, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 2.0, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.5298813670873642, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 2.0, + "278008524": 1.113811194896698, + "416183106": 2.0, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 1.4266315797964733, + "565621826": 2.0, + "585904130": 0.23764699200789133, + "626486813": 0.6124684102833271, + "636680137": 2.0, + "706479600": 2.0, + "750448296": 0.23580759018659592, + "75070641": 2.0, + "767747424": 0.18863565661013126, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 2.0, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 200 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.6080652458080347, + "low": 1.1926357444861662, + "mean": 1.3999846732054932, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7060283687943263, + "low": 0.4152482269503547, + "mean": 0.5695035460992908, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5695035460992908, + "per_seed": { + "1025740212": 0.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.6666666666666666, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.6, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.75, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.5, + "75070641": 1.0, + "767747424": 0.75, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8384615384615385, + "low": 0.476078431372549, + "mean": 0.6615987933634992, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6615987933634992, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 1.0, + "PullCube-v1": 0.38235294117647056, + "PushCube-v1": 0.6923076923076923, + "StackCube-v1": 0.7333333333333333 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6376811594202898, + "low": 0.4057971014492754, + "mean": 0.5362318840579711, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.20999720226447066, + "low": 0.14758246146515436, + "mean": 0.17512578895825348, + "n": 47, + "num_samples": 200 + }, + "mean": 0.17512578895825348, + "per_seed": { + "1025740212": 0.14400158741031796, + "1055157354": 0.32609963351937027, + "1055567618": 0.15812100047218913, + "1113268940": 0.06911736513585261, + "1113371261": 0.015047211698777683, + "1176943004": 0.15442455751472706, + "1189392917": 0.10709427493066431, + "1231780997": 0.07419277505792353, + "1271718007": 0.10710899843183383, + "1273247072": 0.1410368025527166, + "1358421325": 0.04701404433315851, + "1375492786": 0.16573502466028908, + "141984615": 0.16666657870321888, + "1475440973": 0.0862090980213101, + "1483645336": 0.06079377324000063, + "1522272686": 0.24182565394435904, + "1528740417": 0.026961937751403514, + "1549324805": 0.10346856229255086, + "1607773860": 0.2544258391757567, + "1665739511": 0.05717674118582694, + "1747178030": 0.1055876219626688, + "1881223807": 0.10190418918720931, + "1919198188": 0.2569381375337174, + "1991414747": 0.44645064197830664, + "1997090716": 0.3553584127845171, + "2015956957": 0.2284508985012672, + "2054899126": 0.0, + "211245842": 0.036206191308278124, + "2141802816": 0.40989236974714055, + "256770874": 0.3193203145842822, + "278008524": 0.10627711331165646, + "416183106": 0.1925298309196861, + "534717141": 0.12736602361488303, + "551504635": 0.23627606487156766, + "565369392": 0.2859321646983037, + "565621826": 0.0, + "585904130": 0.1652498530853035, + "626486813": 0.20632809068391378, + "636680137": 0.45862923715160336, + "706479600": 0.15129914778803955, + "750448296": 0.1527117559178966, + "75070641": 0.34224193319263285, + "767747424": 0.13255125751519692, + "77147264": 0.03379253116769787, + "794181626": 0.3619001242329895, + "957101489": 0.2708447347116597, + "998761162": 0.24035198055524792 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.18285887146206709, + "low": 0.133470413515452, + "mean": 0.15974686671965427, + "n": 5, + "num_samples": 200 + }, + "mean": 0.15974686671965427, + "per_task": { + "LiftPegUpright-v1": 0.10612647137472923, + "PickCube-v1": 0.1410368025527166, + "PullCube-v1": 0.1744863267265361, + "PushCube-v1": 0.18136164881975486, + "StackCube-v1": 0.19572308412453462 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.20414160361743686, + "low": 0.14859038870843547, + "mean": 0.17396925269606012, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6382978723404256, + "low": 0.35460992907801414, + "mean": 0.49645390070921985, + "n": 47, + "num_samples": 200 + }, + "mean": 0.49645390070921985, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6765912518853694, + "low": 0.12549019607843137, + "mean": 0.37402714932126696, + "n": 5, + "num_samples": 200 + }, + "mean": 0.37402714932126696, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.9230769230769231, + "StackCube-v1": 0.4666666666666667 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4782608695652174, + "low": 0.2753623188405797, + "mean": 0.37681159420289856, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3386524822695035, + "low": -0.005319148936170213, + "mean": 0.18794326241134754, + "n": 47, + "num_samples": 200 + }, + "mean": 0.18794326241134754, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": -1.0, + "1549324805": -0.3333333333333333, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.36923076923076925, + "low": -0.058823529411764705, + "mean": 0.14699849170437407, + "n": 5, + "num_samples": 200 + }, + "mean": 0.14699849170437407, + "per_task": { + "LiftPegUpright-v1": 0.0, + "PickCube-v1": 0.0, + "PullCube-v1": -0.14705882352941177, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.21739130434782608, + "low": -0.057971014492753624, + "mean": 0.10144927536231885, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.384272303660103, + "low": 0.9012552351477814, + "mean": 1.1504715986059424, + "n": 47, + "num_samples": 200 + }, + "mean": 1.1504715986059424, + "per_seed": { + "1025740212": 0.2148684412240982, + "1055157354": 0.812659502029419, + "1055567618": 2.0, + "1113268940": 0.7256770730018616, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 0.46439141035079956, + "1358421325": 0.8245300054550171, + "1375492786": 0.001480837701819837, + "141984615": 0.07189804397057742, + "1475440973": 0.7181311249732971, + "1483645336": 2.0, + "1522272686": 0.24029652774333954, + "1528740417": 0.7255622744560242, + "1549324805": 0.6790153384208679, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 0.2203868180513382, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.06726333405822515, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 0.8128684163093567, + "278008524": 0.019306766451336443, + "416183106": 0.22490769624710083, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 0.11818361282348633, + "565621826": 2.0, + "585904130": 0.01988216955214739, + "626486813": 0.06953954696655273, + "636680137": 0.015581388957798481, + "706479600": 2.0, + "750448296": 0.10195614024996758, + "75070641": 0.24087366461753845, + "767747424": 0.10304383677430451, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 0.5798611640930176, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.431180309034311, + "low": 0.5387795139021951, + "mean": 0.9849799114682529, + "n": 5, + "num_samples": 200 + }, + "mean": 0.9849799114682529, + "per_task": { + "LiftPegUpright-v1": 1.0640398189425468, + "PickCube-v1": 0.46439141035079956, + "PullCube-v1": 0.3505374649334151, + "PushCube-v1": 1.8488386101447618, + "StackCube-v1": 1.1970922529697419 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.0718660958118036, + "low": 0.6817222138600207, + "mean": 0.8805534762053656, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.600968998716814, + "low": 0.008772604939600467, + "mean": 0.3341213807176323, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3341213807176323, + "per_seed": { + "1025740212": -0.31350521743297577, + "1055157354": 0.6161754280328751, + "1055567618": 0.0, + "1113268940": 0.3033645451068878, + "1113371261": 0.0, + "1176943004": 1.845386266708374, + "1189392917": 1.8915813565254211, + "1231780997": 1.2729938626289368, + "1271718007": 0.584618866443634, + "1273247072": 0.3109617531299591, + "1358421325": -0.0026850104331970215, + "1375492786": -1.9985191622981802, + "141984615": -0.021914095113364358, + "1475440973": -1.2818688750267029, + "1483645336": 1.9835839420557022, + "1522272686": 0.0418534129858017, + "1528740417": -1.2744377255439758, + "1549324805": -0.6997321099042892, + "1607773860": 1.1673271656036377, + "1665739511": 0.0, + "1747178030": 1.8533687740564346, + "1881223807": 1.9679730087518692, + "1919198188": 0.18318794667720795, + "1991414747": 0.0, + "1997090716": 1.8607252091169357, + "2015956957": 0.00014354418963193893, + "2054899126": 0.0, + "211245842": 1.1611655950546265, + "2141802816": 1.9291927516460419, + "256770874": -1.1871315836906433, + "278008524": -0.08816510194446892, + "416183106": 0.02458128333091736, + "534717141": 0.0, + "551504635": 1.2760274410247803, + "565369392": -0.6815820336341858, + "565621826": 0.0, + "585904130": -0.055409928473333515, + "626486813": -0.43590046965982765, + "636680137": -1.9844186110422015, + "706479600": 1.1649112105369568, + "750448296": -0.009577419608831406, + "75070641": 0.0381220281124115, + "767747424": -0.44448035093955696, + "77147264": 0.5784205198287964, + "794181626": 1.8190439492464066, + "957101489": 0.376139298081398, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7216604295513223, + "low": -0.11880971740830834, + "mean": 0.31460225471957737, + "n": 5, + "num_samples": 200 + }, + "mean": 0.31460225471957737, + "per_task": { + "LiftPegUpright-v1": -0.002904328207174937, + "PickCube-v1": 0.3109617531299591, + "PullCube-v1": -0.30070200398810865, + "PushCube-v1": 1.1716741495407546, + "StackCube-v1": 0.3939817031224569 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.3720206405370213, + "low": -0.11932498876315852, + "mean": 0.16248053824866487, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3617021276595745, + "low": 0.13120567375886524, + "mean": 0.24113475177304963, + "n": 47, + "num_samples": 200 + }, + "mean": 0.24113475177304963, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3337858220211162, + "low": 0.05333333333333333, + "mean": 0.1758371040723982, + "n": 5, + "num_samples": 200 + }, + "mean": 0.1758371040723982, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.11764705882352941, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.13333333333333333 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.30434782608695654, + "low": 0.10144927536231885, + "mean": 0.18840579710144928, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.08687943262411349, + "low": -0.2304964539007092, + "mean": -0.06737588652482268, + "n": 47, + "num_samples": 200 + }, + "mean": -0.06737588652482268, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": -1.0, + "1549324805": -0.3333333333333333, + "1607773860": 1.0, + "1665739511": -1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.04564102564102565, + "low": -0.15254901960784314, + "mean": -0.051191553544494714, + "n": 5, + "num_samples": 200 + }, + "mean": -0.051191553544494714, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.17647058823529413, + "PushCube-v1": 0.15384615384615385, + "StackCube-v1": -0.06666666666666667 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.014492753623188406, + "low": -0.21739130434782608, + "mean": -0.08695652173913043, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9067608354204905, + "low": 0.45753479555867627, + "mean": 0.6653794013519014, + "n": 47, + "num_samples": 200 + }, + "mean": 0.6653794013519014, + "per_seed": { + "1025740212": 0.21175099909305573, + "1055157354": 0.16174951195716858, + "1055567618": 2.0, + "1113268940": 0.6624154448509216, + "1113371261": 2.0, + "1176943004": 0.04685409367084503, + "1189392917": 0.0735837072134018, + "1231780997": 2.0, + "1271718007": 1.1047073602676392, + "1273247072": 0.0375710129737854, + "1358421325": 0.8165659308433533, + "1375492786": 0.00048063197755254805, + "141984615": 0.03405998833477497, + "1475440973": 0.7181311249732971, + "1483645336": 2.0, + "1522272686": 0.20458371937274933, + "1528740417": 0.7171411514282227, + "1549324805": 0.6740981936454773, + "1607773860": 2.0, + "1665739511": 0.8184402585029602, + "1747178030": 0.09237110614776611, + "1881223807": 0.03511819243431091, + "1919198188": 0.03183290362358093, + "1991414747": 0.05380085110664368, + "1997090716": 2.0, + "2015956957": 0.06189579609781504, + "2054899126": 2.0, + "211245842": 0.8183318376541138, + "2141802816": 0.06590981781482697, + "256770874": 0.19993644952774048, + "278008524": 0.011877144395839423, + "416183106": 0.20836615562438965, + "534717141": 2.0, + "551504635": 0.7220139503479004, + "565369392": 0.07017792761325836, + "565621826": 2.0, + "585904130": 0.015381396127243837, + "626486813": 0.041404386749491096, + "636680137": 0.010638435371220112, + "706479600": 0.814424455165863, + "750448296": 0.057780176401138306, + "75070641": 0.2330203801393509, + "767747424": 0.05761944816913456, + "77147264": 1.1120431125164032, + "794181626": 0.03734047710895538, + "957101489": 0.23941433429718018, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7867360686720947, + "low": 0.26201372464497885, + "mean": 0.53948547703734, + "n": 5, + "num_samples": 200 + }, + "mean": 0.53948547703734, + "per_task": { + "LiftPegUpright-v1": 0.8385754451155663, + "PickCube-v1": 0.0375710129737854, + "PullCube-v1": 0.2737284779748948, + "PushCube-v1": 0.9488746569706843, + "StackCube-v1": 0.598677792151769 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.7215508813974009, + "low": 0.3420067752565534, + "mean": 0.5172656428653484, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.09389386805202087, + "low": -0.4070899225106786, + "mean": -0.15097081653640856, + "n": 47, + "num_samples": 200 + }, + "mean": -0.15097081653640856, + "per_seed": { + "1025740212": -0.31662265956401825, + "1055157354": -0.034734562039375305, + "1055567618": 0.0, + "1113268940": 0.24010291695594788, + "1113371261": 0.0, + "1176943004": -0.10775963962078094, + "1189392917": -0.03483493626117706, + "1231780997": 1.2729938626289368, + "1271718007": -0.3106737732887268, + "1273247072": -0.11585864424705505, + "1358421325": -0.01064908504486084, + "1375492786": -1.9995193680224475, + "141984615": -0.05975215074916681, + "1475440973": -1.2818688750267029, + "1483645336": 1.9835839420557022, + "1522272686": 0.006140604615211487, + "1528740417": -1.2828588485717773, + "1549324805": -0.7046492546796799, + "1607773860": 1.1673271656036377, + "1665739511": -1.1815597414970398, + "1747178030": -0.054260119795799255, + "1881223807": 0.0030912011861801147, + "1919198188": -0.005365967750549316, + "1991414747": -1.9461991488933563, + "1997090716": 1.8607252091169357, + "2015956957": -0.005223993770778179, + "2054899126": 0.0, + "211245842": -0.020502567291259766, + "2141802816": -0.0048974305391311646, + "256770874": -1.8000635504722595, + "278008524": -0.09559472399996594, + "416183106": 0.008039742708206177, + "534717141": 0.0, + "551504635": -0.001958608627319336, + "565369392": -0.7295877188444138, + "565621826": 0.0, + "585904130": -0.05991070189823707, + "626486813": -0.4640356298768893, + "636680137": -1.9893615646287799, + "706479600": -0.020664334297180176, + "750448296": -0.053753383457660675, + "75070641": 0.030268743634223938, + "767747424": -0.4899047395447269, + "77147264": -0.3095363676548004, + "794181626": -0.14361557364463806, + "957101489": 0.03569246828556061, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.07167863700634394, + "low": -0.29535206382421947, + "mean": -0.1308921797113357, + "n": 5, + "num_samples": 200 + }, + "mean": -0.1308921797113357, + "per_task": { + "LiftPegUpright-v1": -0.22836870203415552, + "PickCube-v1": -0.11585864424705505, + "PullCube-v1": -0.377510990946629, + "PushCube-v1": 0.2717101963666769, + "StackCube-v1": -0.20443275769551594 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.021096388483335657, + "low": -0.4195698232493748, + "mean": -0.20080729509135237, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6866153551741166, + "low": 0.2764395795932208, + "mean": 0.4850921972540409, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4850921972540409, + "per_seed": { + "1025740212": 0.0031174421310424805, + "1055157354": 0.6509099900722504, + "1055567618": 0.0, + "1113268940": 0.06326162815093994, + "1113371261": 0.0, + "1176943004": 1.953145906329155, + "1189392917": 1.9264162927865982, + "1231780997": 0.0, + "1271718007": 0.8952926397323608, + "1273247072": 0.42682039737701416, + "1358421325": 0.007964074611663818, + "1375492786": 0.001000205724267289, + "141984615": 0.03783805563580245, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.03571280837059021, + "1528740417": 0.008421123027801514, + "1549324805": 0.004917144775390625, + "1607773860": 0.0, + "1665739511": 1.1815597414970398, + "1747178030": 1.9076288938522339, + "1881223807": 1.964881807565689, + "1919198188": 0.18855391442775726, + "1991414747": 1.9461991488933563, + "1997090716": 0.0, + "2015956957": 0.005367537960410118, + "2054899126": 0.0, + "211245842": 1.1816681623458862, + "2141802816": 1.934090182185173, + "256770874": 0.6129319667816162, + "278008524": 0.0074296220554970205, + "416183106": 0.01654154062271118, + "534717141": 0.0, + "551504635": 1.2779860496520996, + "565369392": 0.048005685210227966, + "565621826": 0.0, + "585904130": 0.004500773424903552, + "626486813": 0.02813516021706164, + "636680137": 0.004942953586578369, + "706479600": 1.185575544834137, + "750448296": 0.04417596384882927, + "75070641": 0.007853284478187561, + "767747424": 0.04542438860516995, + "77147264": 0.8879568874835968, + "794181626": 1.9626595228910446, + "957101489": 0.3404468297958374, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6847154450722229, + "low": 0.18113008173041084, + "mean": 0.4454944344309131, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4454944344309131, + "per_task": { + "LiftPegUpright-v1": 0.2254643738269806, + "PickCube-v1": 0.42682039737701416, + "PullCube-v1": 0.07680898695852033, + "PushCube-v1": 0.8999639531740775, + "StackCube-v1": 0.5984144608179728 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5186884595309823, + "low": 0.2226778654005392, + "mean": 0.36328783334001724, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3723404255319149, + "low": 0.13829787234042554, + "mean": 0.2553191489361702, + "n": 47, + "num_samples": 200 + }, + "mean": 0.2553191489361702, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.34947209653092004, + "low": 0.050980392156862744, + "mean": 0.1981900452488688, + "n": 5, + "num_samples": 200 + }, + "mean": 0.1981900452488688, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.029411764705882353, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.3333333333333333 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2753623188405797, + "low": 0.10144927536231885, + "mean": 0.18840579710144928, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.4716312056737588, + "low": 0.2347517730496454, + "mean": 0.33936170212765954, + "n": 47, + "num_samples": 200 + }, + "mean": 0.33936170212765954, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7487179487179487, + "low": 0.2147812971342383, + "mean": 0.49067873303167414, + "n": 5, + "num_samples": 200 + }, + "mean": 0.49067873303167414, + "per_task": { + "LiftPegUpright-v1": 0.6666666666666666, + "PickCube-v1": 1.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.07692307692307693, + "StackCube-v1": 0.5333333333333333 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.37681159420289856, + "low": 0.17391304347826086, + "mean": 0.2898550724637681, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7404255319148936, + "low": 0.4702127659574468, + "mean": 0.5946808510638298, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5946808510638298, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8866666666666667, + "low": 0.43122171945701354, + "mean": 0.688868778280543, + "n": 5, + "num_samples": 200 + }, + "mean": 0.688868778280543, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.20588235294117646, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.8666666666666667 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5942028985507246, + "low": 0.3333333333333333, + "mean": 0.4782608695652174, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7543874885621051, + "low": 0.39060924925332796, + "mean": 0.555731212557954, + "n": 47, + "num_samples": 200 + }, + "mean": 0.555731212557954, + "per_seed": { + "1025740212": 1.7851315587759018, + "1055157354": 1.187340497970581, + "1055567618": 0.0, + "1113268940": 1.2743229269981384, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.5356085896492004, + "1358421325": 1.175469994544983, + "1375492786": 0.2419701317558065, + "141984615": 0.12854979850817472, + "1475440973": 1.2818688750267029, + "1483645336": 0.0, + "1522272686": 1.7597034722566605, + "1528740417": 1.2744377255439758, + "1549324805": 0.7237351834774017, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 1.7796131819486618, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.46261803302913906, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.1871315836906433, + "278008524": 1.0945044284453616, + "416183106": 1.7750923037528992, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 1.308447966972987, + "565621826": 0.0, + "585904130": 0.21776482245574394, + "626486813": 0.5429288633167744, + "636680137": 1.9844186110422015, + "706479600": 0.0, + "750448296": 0.13385144993662834, + "75070641": 1.7591263353824615, + "767747424": 0.08559181983582675, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 1.4201388359069824, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.1683035775687656, + "low": 0.3937598540945341, + "mean": 0.7714844558916235, + "n": 5, + "num_samples": 200 + }, + "mean": 0.7714844558916235, + "per_task": { + "LiftPegUpright-v1": 0.9359601810574532, + "PickCube-v1": 1.5356085896492004, + "PullCube-v1": 0.4317843718659681, + "PushCube-v1": 0.1511613898552381, + "StackCube-v1": 0.8029077470302581 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6671761653750486, + "low": 0.35016158437255124, + "mean": 0.5194311970001275, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.2788291653615742, + "low": 0.8215172549003754, + "mean": 1.0408234098119948, + "n": 47, + "num_samples": 200 + }, + "mean": 1.0408234098119948, + "per_seed": { + "1025740212": 1.7882490009069443, + "1055157354": 1.8382504880428314, + "1055567618": 0.0, + "1113268940": 1.3375845551490784, + "1113371261": 0.0, + "1176943004": 1.953145906329155, + "1189392917": 1.9264162927865982, + "1231780997": 0.0, + "1271718007": 0.8952926397323608, + "1273247072": 1.9624289870262146, + "1358421325": 1.1834340691566467, + "1375492786": 0.2429703374800738, + "141984615": 0.16638785414397717, + "1475440973": 1.2818688750267029, + "1483645336": 0.0, + "1522272686": 1.7954162806272507, + "1528740417": 1.2828588485717773, + "1549324805": 0.7286523282527924, + "1607773860": 0.0, + "1665739511": 1.1815597414970398, + "1747178030": 1.9076288938522339, + "1881223807": 1.964881807565689, + "1919198188": 1.968167096376419, + "1991414747": 1.9461991488933563, + "1997090716": 0.0, + "2015956957": 0.4679855709895492, + "2054899126": 0.0, + "211245842": 1.1816681623458862, + "2141802816": 1.934090182185173, + "256770874": 1.8000635504722595, + "278008524": 1.1019340505008586, + "416183106": 1.7916338443756104, + "534717141": 0.0, + "551504635": 1.2779860496520996, + "565369392": 1.356453652183215, + "565621826": 0.0, + "585904130": 0.2222655958806475, + "626486813": 0.571064023533836, + "636680137": 1.9893615646287799, + "706479600": 1.185575544834137, + "750448296": 0.1780274137854576, + "75070641": 1.766979619860649, + "767747424": 0.1310162084409967, + "77147264": 0.8879568874835968, + "794181626": 1.9626595228910446, + "957101489": 1.7605856657028198, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.6199673717984786, + "low": 0.817705367841226, + "mean": 1.2169788903225367, + "n": 5, + "num_samples": 200 + }, + "mean": 1.2169788903225367, + "per_task": { + "LiftPegUpright-v1": 1.1614245548844337, + "PickCube-v1": 1.9624289870262146, + "PullCube-v1": 0.5085933588244884, + "PushCube-v1": 1.0511253430293157, + "StackCube-v1": 1.401322207848231 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 1.056229973670797, + "low": 0.6653890719201139, + "mean": 0.8827190303401448, + "n": 69, + "num_samples": 200 + }, + "n": 69 + } + }, + "thresholds": [ + 0.2, + 0.4 + ] +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics_by_seed.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..f043b4bd8f082a11390732855bdbb4598d4ebd0f --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics_by_seed.json @@ -0,0 +1,1083 @@ +{ + "1025740212": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.0031174421310424805, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14400158741031796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2148684412240982, + "proposal_oracle_utility_gain_over_base_at_8": -0.31350521743297577, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21175099909305573, + "selected_utility_gain_over_base_at_8": -0.31662265956401825, + "selector_regret_at_8": 0.0031174421310424805, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7851315587759018, + "total_car_to_hidden_at_8": 1.7882490009069443 + }, + "1055157354": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.6509099900722504, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32609963351937027, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.812659502029419, + "proposal_oracle_utility_gain_over_base_at_8": 0.6161754280328751, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.16174951195716858, + "selected_utility_gain_over_base_at_8": -0.034734562039375305, + "selector_regret_at_8": 0.6509099900722504, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.187340497970581, + "total_car_to_hidden_at_8": 1.8382504880428314 + }, + "1055567618": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15812100047218913, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1113268940": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.06326162815093994, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06911736513585261, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7256770730018616, + "proposal_oracle_utility_gain_over_base_at_8": 0.3033645451068878, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6624154448509216, + "selected_utility_gain_over_base_at_8": 0.24010291695594788, + "selector_regret_at_8": 0.06326162815093994, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2743229269981384, + "total_car_to_hidden_at_8": 1.3375845551490784 + }, + "1113371261": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.015047211698777683, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.953145906329155, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15442455751472706, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04685409367084503, + "selected_utility_gain_over_base_at_8": -0.10775963962078094, + "selector_regret_at_8": 1.953145906329155, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.953145906329155 + }, + "1189392917": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 1.9264162927865982, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10709427493066431, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0735837072134018, + "selected_utility_gain_over_base_at_8": -0.03483493626117706, + "selector_regret_at_8": 1.9264162927865982, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9264162927865982 + }, + "1231780997": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07419277505792353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.2729938626289368, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1271718007": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.8952926397323608, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10710899843183383, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1047073602676392, + "selected_utility_gain_over_base_at_8": -0.3106737732887268, + "selector_regret_at_8": 0.8952926397323608, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8952926397323608 + }, + "1273247072": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + "1358421325": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.007964074611663818, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.04701404433315851, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8245300054550171, + "proposal_oracle_utility_gain_over_base_at_8": -0.0026850104331970215, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8165659308433533, + "selected_utility_gain_over_base_at_8": -0.01064908504486084, + "selector_regret_at_8": 0.007964074611663818, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.175469994544983, + "total_car_to_hidden_at_8": 1.1834340691566467 + }, + "1375492786": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.001000205724267289, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16573502466028908, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.001480837701819837, + "proposal_oracle_utility_gain_over_base_at_8": -1.9985191622981802, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.00048063197755254805, + "selected_utility_gain_over_base_at_8": -1.9995193680224475, + "selector_regret_at_8": 0.001000205724267289, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2419701317558065, + "total_car_to_hidden_at_8": 0.2429703374800738 + }, + "141984615": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03783805563580245, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.16666657870321888, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07189804397057742, + "proposal_oracle_utility_gain_over_base_at_8": -0.021914095113364358, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03405998833477497, + "selected_utility_gain_over_base_at_8": -0.05975215074916681, + "selector_regret_at_8": 0.03783805563580245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12854979850817472, + "total_car_to_hidden_at_8": 0.16638785414397717 + }, + "1475440973": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0862090980213101, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7181311249732971, + "proposal_oracle_utility_gain_over_base_at_8": -1.2818688750267029, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7181311249732971, + "selected_utility_gain_over_base_at_8": -1.2818688750267029, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2818688750267029, + "total_car_to_hidden_at_8": 1.2818688750267029 + }, + "1483645336": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06079377324000063, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1522272686": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.03571280837059021, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24182565394435904, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24029652774333954, + "proposal_oracle_utility_gain_over_base_at_8": 0.0418534129858017, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20458371937274933, + "selected_utility_gain_over_base_at_8": 0.006140604615211487, + "selector_regret_at_8": 0.03571280837059021, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7597034722566605, + "total_car_to_hidden_at_8": 1.7954162806272507 + }, + "1528740417": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.008421123027801514, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026961937751403514, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7255622744560242, + "proposal_oracle_utility_gain_over_base_at_8": -1.2744377255439758, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7171411514282227, + "selected_utility_gain_over_base_at_8": -1.2828588485717773, + "selector_regret_at_8": 0.008421123027801514, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2744377255439758, + "total_car_to_hidden_at_8": 1.2828588485717773 + }, + "1549324805": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.004917144775390625, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10346856229255086, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6790153384208679, + "proposal_oracle_utility_gain_over_base_at_8": -0.6997321099042892, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6740981936454773, + "selected_utility_gain_over_base_at_8": -0.7046492546796799, + "selector_regret_at_8": 0.004917144775390625, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7237351834774017, + "total_car_to_hidden_at_8": 0.7286523282527924 + }, + "1607773860": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2544258391757567, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1673271656036377, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1665739511": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.1815597414970398, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.05717674118582694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.8184402585029602, + "selected_utility_gain_over_base_at_8": -1.1815597414970398, + "selector_regret_at_8": 1.1815597414970398, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1815597414970398 + }, + "1747178030": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 1.9076288938522339, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1055876219626688, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09237110614776611, + "selected_utility_gain_over_base_at_8": -0.054260119795799255, + "selector_regret_at_8": 1.9076288938522339, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9076288938522339 + }, + "1881223807": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 1.964881807565689, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10190418918720931, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9679730087518692, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03511819243431091, + "selected_utility_gain_over_base_at_8": 0.0030912011861801147, + "selector_regret_at_8": 1.964881807565689, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.964881807565689 + }, + "1919198188": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.18855391442775726, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2569381375337174, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2203868180513382, + "proposal_oracle_utility_gain_over_base_at_8": 0.18318794667720795, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03183290362358093, + "selected_utility_gain_over_base_at_8": -0.005365967750549316, + "selector_regret_at_8": 0.18855391442775726, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7796131819486618, + "total_car_to_hidden_at_8": 1.968167096376419 + }, + "1991414747": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9461991488933563, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.44645064197830664, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.05380085110664368, + "selected_utility_gain_over_base_at_8": -1.9461991488933563, + "selector_regret_at_8": 1.9461991488933563, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9461991488933563 + }, + "1997090716": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3553584127845171, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2015956957": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.005367537960410118, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.6, + "pairwise_causal_calibration_ece": 0.2284508985012672, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06726333405822515, + "proposal_oracle_utility_gain_over_base_at_8": 0.00014354418963193893, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06189579609781504, + "selected_utility_gain_over_base_at_8": -0.005223993770778179, + "selector_regret_at_8": 0.005367537960410118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.2, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.46261803302913906, + "total_car_to_hidden_at_8": 0.4679855709895492 + }, + "2054899126": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 1.1816681623458862, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.036206191308278124, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8183318376541138, + "selected_utility_gain_over_base_at_8": -0.020502567291259766, + "selector_regret_at_8": 1.1816681623458862, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1816681623458862 + }, + "2141802816": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.934090182185173, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.40989236974714055, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06590981781482697, + "selected_utility_gain_over_base_at_8": -0.0048974305391311646, + "selector_regret_at_8": 1.934090182185173, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.934090182185173 + }, + "256770874": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.6129319667816162, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3193203145842822, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.8128684163093567, + "proposal_oracle_utility_gain_over_base_at_8": -1.1871315836906433, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.19993644952774048, + "selected_utility_gain_over_base_at_8": -1.8000635504722595, + "selector_regret_at_8": 0.6129319667816162, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1871315836906433, + "total_car_to_hidden_at_8": 1.8000635504722595 + }, + "278008524": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0074296220554970205, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10627711331165646, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.019306766451336443, + "proposal_oracle_utility_gain_over_base_at_8": -0.08816510194446892, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011877144395839423, + "selected_utility_gain_over_base_at_8": -0.09559472399996594, + "selector_regret_at_8": 0.0074296220554970205, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0945044284453616, + "total_car_to_hidden_at_8": 1.1019340505008586 + }, + "416183106": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.01654154062271118, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1925298309196861, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22490769624710083, + "proposal_oracle_utility_gain_over_base_at_8": 0.02458128333091736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20836615562438965, + "selected_utility_gain_over_base_at_8": 0.008039742708206177, + "selector_regret_at_8": 0.01654154062271118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7750923037528992, + "total_car_to_hidden_at_8": 1.7916338443756104 + }, + "534717141": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.12736602361488303, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2779860496520996, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.23627606487156766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7220139503479004, + "selected_utility_gain_over_base_at_8": -0.001958608627319336, + "selector_regret_at_8": 1.2779860496520996, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2779860496520996 + }, + "565369392": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.048005685210227966, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2859321646983037, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.11818361282348633, + "proposal_oracle_utility_gain_over_base_at_8": -0.6815820336341858, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.07017792761325836, + "selected_utility_gain_over_base_at_8": -0.7295877188444138, + "selector_regret_at_8": 0.048005685210227966, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.308447966972987, + "total_car_to_hidden_at_8": 1.356453652183215 + }, + "565621826": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.004500773424903552, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1652498530853035, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01988216955214739, + "proposal_oracle_utility_gain_over_base_at_8": -0.055409928473333515, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.015381396127243837, + "selected_utility_gain_over_base_at_8": -0.05991070189823707, + "selector_regret_at_8": 0.004500773424903552, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.21776482245574394, + "total_car_to_hidden_at_8": 0.2222655958806475 + }, + "626486813": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.02813516021706164, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20632809068391378, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06953954696655273, + "proposal_oracle_utility_gain_over_base_at_8": -0.43590046965982765, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.041404386749491096, + "selected_utility_gain_over_base_at_8": -0.4640356298768893, + "selector_regret_at_8": 0.02813516021706164, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5429288633167744, + "total_car_to_hidden_at_8": 0.571064023533836 + }, + "636680137": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.004942953586578369, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.45862923715160336, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.015581388957798481, + "proposal_oracle_utility_gain_over_base_at_8": -1.9844186110422015, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.010638435371220112, + "selected_utility_gain_over_base_at_8": -1.9893615646287799, + "selector_regret_at_8": 0.004942953586578369, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9844186110422015, + "total_car_to_hidden_at_8": 1.9893615646287799 + }, + "706479600": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 1.185575544834137, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15129914778803955, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814424455165863, + "selected_utility_gain_over_base_at_8": -0.020664334297180176, + "selector_regret_at_8": 1.185575544834137, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185575544834137 + }, + "750448296": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.04417596384882927, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.1527117559178966, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10195614024996758, + "proposal_oracle_utility_gain_over_base_at_8": -0.009577419608831406, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.057780176401138306, + "selected_utility_gain_over_base_at_8": -0.053753383457660675, + "selector_regret_at_8": 0.04417596384882927, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13385144993662834, + "total_car_to_hidden_at_8": 0.1780274137854576 + }, + "75070641": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.007853284478187561, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.34224193319263285, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24087366461753845, + "proposal_oracle_utility_gain_over_base_at_8": 0.0381220281124115, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2330203801393509, + "selected_utility_gain_over_base_at_8": 0.030268743634223938, + "selector_regret_at_8": 0.007853284478187561, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7591263353824615, + "total_car_to_hidden_at_8": 1.766979619860649 + }, + "767747424": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.04542438860516995, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.13255125751519692, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.10304383677430451, + "proposal_oracle_utility_gain_over_base_at_8": -0.44448035093955696, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.05761944816913456, + "selected_utility_gain_over_base_at_8": -0.4899047395447269, + "selector_regret_at_8": 0.04542438860516995, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.08559181983582675, + "total_car_to_hidden_at_8": 0.1310162084409967 + }, + "77147264": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.8879568874835968, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.03379253116769787, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.5784205198287964, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1120431125164032, + "selected_utility_gain_over_base_at_8": -0.3095363676548004, + "selector_regret_at_8": 0.8879568874835968, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8879568874835968 + }, + "794181626": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 1.9626595228910446, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3619001242329895, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03734047710895538, + "selected_utility_gain_over_base_at_8": -0.14361557364463806, + "selector_regret_at_8": 1.9626595228910446, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9626595228910446 + }, + "957101489": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3404468297958374, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2708447347116597, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5798611640930176, + "proposal_oracle_utility_gain_over_base_at_8": 0.376139298081398, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.23941433429718018, + "selected_utility_gain_over_base_at_8": 0.03569246828556061, + "selector_regret_at_8": 0.3404468297958374, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4201388359069824, + "total_car_to_hidden_at_8": 1.7605856657028198 + }, + "998761162": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24035198055524792, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics_by_task.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..e5ab43c2cd385e0902e327f97b1defce6119af35 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.2254643738269806, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10612647137472923, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.0640398189425468, + "proposal_oracle_utility_gain_over_base_at_8": -0.002904328207174937, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8385754451155663, + "selected_utility_gain_over_base_at_8": -0.22836870203415552, + "selector_regret_at_8": 0.2254643738269806, + "success_selector_gap_at_8": 0.16666666666666666, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.9359601810574532, + "total_car_to_hidden_at_8": 1.1614245548844337 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.07680898695852033, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.38235294117647056, + "pairwise_causal_calibration_ece": 0.1744863267265361, + "proposal_oracle_success_at_8": 0.14705882352941177, + "proposal_oracle_success_gain_over_base_at_8": -0.14705882352941177, + "proposal_oracle_utility_at_8": 0.3505374649334151, + "proposal_oracle_utility_gain_over_base_at_8": -0.30070200398810865, + "selected_success_at_8": 0.11764705882352941, + "selected_success_gain_over_base_at_8": -0.17647058823529413, + "selected_utility_at_8": 0.2737284779748948, + "selected_utility_gain_over_base_at_8": -0.377510990946629, + "selector_regret_at_8": 0.07680898695852033, + "success_selector_gap_at_8": 0.029411764705882353, + "success_support_gap_at_8": 0.17647058823529413, + "success_total_car_to_hidden_at_8": 0.20588235294117646, + "support_gap_at_8": 0.4317843718659681, + "total_car_to_hidden_at_8": 0.5085933588244884 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.8999639531740775, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6923076923076923, + "pairwise_causal_calibration_ece": 0.18136164881975486, + "proposal_oracle_success_at_8": 0.9230769230769231, + "proposal_oracle_success_gain_over_base_at_8": 0.6153846153846154, + "proposal_oracle_utility_at_8": 1.8488386101447618, + "proposal_oracle_utility_gain_over_base_at_8": 1.1716741495407546, + "selected_success_at_8": 0.46153846153846156, + "selected_success_gain_over_base_at_8": 0.15384615384615385, + "selected_utility_at_8": 0.9488746569706843, + "selected_utility_gain_over_base_at_8": 0.2717101963666769, + "selector_regret_at_8": 0.8999639531740775, + "success_selector_gap_at_8": 0.46153846153846156, + "success_support_gap_at_8": 0.07692307692307693, + "success_total_car_to_hidden_at_8": 0.5384615384615384, + "support_gap_at_8": 0.1511613898552381, + "total_car_to_hidden_at_8": 1.0511253430293157 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.5984144608179728, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.7333333333333333, + "pairwise_causal_calibration_ece": 0.19572308412453462, + "proposal_oracle_success_at_8": 0.4666666666666667, + "proposal_oracle_success_gain_over_base_at_8": 0.26666666666666666, + "proposal_oracle_utility_at_8": 1.1970922529697419, + "proposal_oracle_utility_gain_over_base_at_8": 0.3939817031224569, + "selected_success_at_8": 0.13333333333333333, + "selected_success_gain_over_base_at_8": -0.06666666666666667, + "selected_utility_at_8": 0.598677792151769, + "selected_utility_gain_over_base_at_8": -0.20443275769551594, + "selector_regret_at_8": 0.5984144608179728, + "success_selector_gap_at_8": 0.3333333333333333, + "success_support_gap_at_8": 0.5333333333333333, + "success_total_car_to_hidden_at_8": 0.8666666666666667, + "support_gap_at_8": 0.8029077470302581, + "total_car_to_hidden_at_8": 1.401322207848231 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/report.md b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/report.md new file mode 100644 index 0000000000000000000000000000000000000000..0f1d66b7deaa45aab2c8d01ea311dc9b238bd947 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/report.md @@ -0,0 +1,27 @@ +# Metric Evaluation (measured) + +K: `8` + +| Metric | N | Micro mean | 95% CI | Task macro | Seed macro | +| --- | ---: | ---: | ---: | ---: | ---: | +| base_success | 69 | 0.2754 | [0.1884, 0.4058] | 0.2270 | 0.3085 | +| base_utility | 69 | 0.7181 | [0.5449, 0.9239] | 0.6704 | 0.8164 | +| branch_car_at_8 | 69 | 0.3633 | [0.2227, 0.5187] | 0.4455 | 0.4851 | +| hidden_chart_oracle_success_at_8 | 69 | 0.6667 | [0.5507, 0.7826] | 0.8647 | 0.8358 | +| hidden_chart_oracle_utility_at_8 | 69 | 1.4000 | [1.1926, 1.6081] | 1.7565 | 1.7062 | +| outcome_ptr_at_8 | 69 | 0.5362 | [0.4058, 0.6377] | 0.6616 | 0.5695 | +| pairwise_causal_calibration_ece | 69 | 0.1740 | [0.1486, 0.2041] | 0.1597 | 0.1751 | +| proposal_oracle_success_at_8 | 69 | 0.3768 | [0.2754, 0.4783] | 0.3740 | 0.4965 | +| proposal_oracle_success_gain_over_base_at_8 | 69 | 0.1014 | [-0.0580, 0.2174] | 0.1470 | 0.1879 | +| proposal_oracle_utility_at_8 | 69 | 0.8806 | [0.6817, 1.0719] | 0.9850 | 1.1505 | +| proposal_oracle_utility_gain_over_base_at_8 | 69 | 0.1625 | [-0.1193, 0.3720] | 0.3146 | 0.3341 | +| selected_success_at_8 | 69 | 0.1884 | [0.1014, 0.3043] | 0.1758 | 0.2411 | +| selected_success_gain_over_base_at_8 | 69 | -0.0870 | [-0.2174, 0.0145] | -0.0512 | -0.0674 | +| selected_utility_at_8 | 69 | 0.5173 | [0.3420, 0.7216] | 0.5395 | 0.6654 | +| selected_utility_gain_over_base_at_8 | 69 | -0.2008 | [-0.4196, -0.0211] | -0.1309 | -0.1510 | +| selector_regret_at_8 | 69 | 0.3633 | [0.2227, 0.5187] | 0.4455 | 0.4851 | +| success_selector_gap_at_8 | 69 | 0.1884 | [0.1014, 0.2754] | 0.1982 | 0.2553 | +| success_support_gap_at_8 | 69 | 0.2899 | [0.1739, 0.3768] | 0.4907 | 0.3394 | +| success_total_car_to_hidden_at_8 | 69 | 0.4783 | [0.3333, 0.5942] | 0.6889 | 0.5947 | +| support_gap_at_8 | 69 | 0.5194 | [0.3502, 0.6672] | 0.7715 | 0.5557 | +| total_car_to_hidden_at_8 | 69 | 0.8827 | [0.6654, 1.0562] | 1.2170 | 1.0408 | diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/table.tex b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..94565401b483aab7b2a0aba694119f99763f1852 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/measured_metrics/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 69 & 0.2754 & 0.1884 & 0.4058 \\ +base\_utility & 69 & 0.7181 & 0.5449 & 0.9239 \\ +branch\_car\_at\_8 & 69 & 0.3633 & 0.2227 & 0.5187 \\ +hidden\_chart\_oracle\_success\_at\_8 & 69 & 0.6667 & 0.5507 & 0.7826 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 69 & 1.4000 & 1.1926 & 1.6081 \\ +outcome\_ptr\_at\_8 & 69 & 0.5362 & 0.4058 & 0.6377 \\ +pairwise\_causal\_calibration\_ece & 69 & 0.1740 & 0.1486 & 0.2041 \\ +proposal\_oracle\_success\_at\_8 & 69 & 0.3768 & 0.2754 & 0.4783 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 69 & 0.1014 & -0.0580 & 0.2174 \\ +proposal\_oracle\_utility\_at\_8 & 69 & 0.8806 & 0.6817 & 1.0719 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 69 & 0.1625 & -0.1193 & 0.3720 \\ +selected\_success\_at\_8 & 69 & 0.1884 & 0.1014 & 0.3043 \\ +selected\_success\_gain\_over\_base\_at\_8 & 69 & -0.0870 & -0.2174 & 0.0145 \\ +selected\_utility\_at\_8 & 69 & 0.5173 & 0.3420 & 0.7216 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 69 & -0.2008 & -0.4196 & -0.0211 \\ +selector\_regret\_at\_8 & 69 & 0.3633 & 0.2227 & 0.5187 \\ +success\_selector\_gap\_at\_8 & 69 & 0.1884 & 0.1014 & 0.2754 \\ +success\_support\_gap\_at\_8 & 69 & 0.2899 & 0.1739 & 0.3768 \\ +success\_total\_car\_to\_hidden\_at\_8 & 69 & 0.4783 & 0.3333 & 0.5942 \\ +support\_gap\_at\_8 & 69 & 0.5194 & 0.3502 & 0.6672 \\ +total\_car\_to\_hidden\_at\_8 & 69 & 0.8827 & 0.6654 & 1.0562 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..0dab826507980775347e2b6fbcfb7a9e7240ef77 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics.json @@ -0,0 +1,1918 @@ +{ + "checkpoint": "runs/ctt_residual_base_context_obj_seed1/model.pt", + "k": 8, + "measured_metric_summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.46099290780141844, + "low": 0.19680851063829788, + "mean": 0.3085106382978724, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 1.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3176470588235294, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.4057971014492754, + "low": 0.18840579710144928, + "mean": 0.2753623188405797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.0808477015729916, + "low": 0.6013251688004412, + "mean": 0.8163502178883101, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8163502178883101, + "per_seed": { + "1025740212": 0.528373658657074, + "1055157354": 0.19648407399654388, + "1055567618": 2.0, + "1113268940": 0.42231252789497375, + "1113371261": 2.0, + "1176943004": 0.15461373329162598, + "1189392917": 0.10841864347457886, + "1231780997": 0.7270061373710632, + "1271718007": 1.415381133556366, + "1273247072": 0.15342965722084045, + "1358421325": 0.8272150158882141, + "1375492786": 2.0, + "141984615": 0.09381213908394177, + "1475440973": 2.0, + "1483645336": 0.01641605794429779, + "1522272686": 0.19844311475753784, + "1528740417": 2.0, + "1549324805": 1.3787474483251572, + "1607773860": 0.8326728343963623, + "1665739511": 2.0, + "1747178030": 0.14663122594356537, + "1881223807": 0.0320269912481308, + "1919198188": 0.03719887137413025, + "1991414747": 2.0, + "1997090716": 0.13927479088306427, + "2015956957": 0.06711978986859321, + "2054899126": 2.0, + "211245842": 0.8388344049453735, + "2141802816": 0.07080724835395813, + "256770874": 2.0, + "278008524": 0.10747186839580536, + "416183106": 0.20032641291618347, + "534717141": 2.0, + "551504635": 0.7239725589752197, + "565369392": 0.7997656464576721, + "565621826": 2.0, + "585904130": 0.07529209802548091, + "626486813": 0.5054400166263804, + "636680137": 2.0, + "706479600": 0.8350887894630432, + "750448296": 0.11153355985879898, + "75070641": 0.20275163650512695, + "767747424": 0.5475241877138615, + "77147264": 1.4215794801712036, + "794181626": 0.18095605075359344, + "957101489": 0.20372186601161957, + "998761162": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9058472741949395, + "low": 0.4133020142714182, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.923898630168127, + "low": 0.5448560892550302, + "mean": 0.7180729379567007, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6866153551741166, + "low": 0.2764395795932208, + "mean": 0.4850921972540409, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4850921972540409, + "per_seed": { + "1025740212": 0.0031174421310424805, + "1055157354": 0.6509099900722504, + "1055567618": 0.0, + "1113268940": 0.06326162815093994, + "1113371261": 0.0, + "1176943004": 1.953145906329155, + "1189392917": 1.9264162927865982, + "1231780997": 0.0, + "1271718007": 0.8952926397323608, + "1273247072": 0.42682039737701416, + "1358421325": 0.007964074611663818, + "1375492786": 0.001000205724267289, + "141984615": 0.03783805563580245, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.03571280837059021, + "1528740417": 0.008421123027801514, + "1549324805": 0.004917144775390625, + "1607773860": 0.0, + "1665739511": 1.1815597414970398, + "1747178030": 1.9076288938522339, + "1881223807": 1.964881807565689, + "1919198188": 0.18855391442775726, + "1991414747": 1.9461991488933563, + "1997090716": 0.0, + "2015956957": 0.005367537960410118, + "2054899126": 0.0, + "211245842": 1.1816681623458862, + "2141802816": 1.934090182185173, + "256770874": 0.6129319667816162, + "278008524": 0.0074296220554970205, + "416183106": 0.01654154062271118, + "534717141": 0.0, + "551504635": 1.2779860496520996, + "565369392": 0.048005685210227966, + "565621826": 0.0, + "585904130": 0.004500773424903552, + "626486813": 0.02813516021706164, + "636680137": 0.004942953586578369, + "706479600": 1.185575544834137, + "750448296": 0.04417596384882927, + "75070641": 0.007853284478187561, + "767747424": 0.04542438860516995, + "77147264": 0.8879568874835968, + "794181626": 1.9626595228910446, + "957101489": 0.3404468297958374, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6847154450722229, + "low": 0.18113008173041084, + "mean": 0.4454944344309131, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4454944344309131, + "per_task": { + "LiftPegUpright-v1": 0.2254643738269806, + "PickCube-v1": 0.42682039737701416, + "PullCube-v1": 0.07680898695852033, + "PushCube-v1": 0.8999639531740775, + "StackCube-v1": 0.5984144608179728 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5186884595309823, + "low": 0.2226778654005392, + "mean": 0.36328783334001724, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9148936170212765, + "low": 0.7432624113475178, + "mean": 0.8358156028368794, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8358156028368794, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.782608695652174, + "low": 0.5507246376811594, + "mean": 0.6666666666666666, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.850041654515774, + "low": 1.5397213380149701, + "mean": 1.7062028111638965, + "n": 47, + "num_samples": 200 + }, + "mean": 1.7062028111638965, + "per_seed": { + "1025740212": 2.0, + "1055157354": 2.0, + "1055567618": 2.0, + "1113268940": 2.0, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 2.0, + "1358421325": 2.0, + "1375492786": 0.24345096945762634, + "141984615": 0.20044784247875214, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 2.0, + "1528740417": 2.0, + "1549324805": 1.4027505218982697, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 2.0, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.5298813670873642, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 2.0, + "278008524": 1.113811194896698, + "416183106": 2.0, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 1.4266315797964733, + "565621826": 2.0, + "585904130": 0.23764699200789133, + "626486813": 0.6124684102833271, + "636680137": 2.0, + "706479600": 2.0, + "750448296": 0.23580759018659592, + "75070641": 2.0, + "767747424": 0.18863565661013126, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 2.0, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 200 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.6080652458080347, + "low": 1.1926357444861662, + "mean": 1.3999846732054932, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7060283687943263, + "low": 0.4152482269503547, + "mean": 0.5695035460992908, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5695035460992908, + "per_seed": { + "1025740212": 0.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.6666666666666666, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.6, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.75, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.5, + "75070641": 1.0, + "767747424": 0.75, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8384615384615385, + "low": 0.476078431372549, + "mean": 0.6615987933634992, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6615987933634992, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 1.0, + "PullCube-v1": 0.38235294117647056, + "PushCube-v1": 0.6923076923076923, + "StackCube-v1": 0.7333333333333333 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6376811594202898, + "low": 0.4057971014492754, + "mean": 0.5362318840579711, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.20999720226447066, + "low": 0.14758246146515436, + "mean": 0.17512578895825348, + "n": 47, + "num_samples": 200 + }, + "mean": 0.17512578895825348, + "per_seed": { + "1025740212": 0.14400158741031796, + "1055157354": 0.32609963351937027, + "1055567618": 0.15812100047218913, + "1113268940": 0.06911736513585261, + "1113371261": 0.015047211698777683, + "1176943004": 0.15442455751472706, + "1189392917": 0.10709427493066431, + "1231780997": 0.07419277505792353, + "1271718007": 0.10710899843183383, + "1273247072": 0.1410368025527166, + "1358421325": 0.04701404433315851, + "1375492786": 0.16573502466028908, + "141984615": 0.16666657870321888, + "1475440973": 0.0862090980213101, + "1483645336": 0.06079377324000063, + "1522272686": 0.24182565394435904, + "1528740417": 0.026961937751403514, + "1549324805": 0.10346856229255086, + "1607773860": 0.2544258391757567, + "1665739511": 0.05717674118582694, + "1747178030": 0.1055876219626688, + "1881223807": 0.10190418918720931, + "1919198188": 0.2569381375337174, + "1991414747": 0.44645064197830664, + "1997090716": 0.3553584127845171, + "2015956957": 0.2284508985012672, + "2054899126": 0.0, + "211245842": 0.036206191308278124, + "2141802816": 0.40989236974714055, + "256770874": 0.3193203145842822, + "278008524": 0.10627711331165646, + "416183106": 0.1925298309196861, + "534717141": 0.12736602361488303, + "551504635": 0.23627606487156766, + "565369392": 0.2859321646983037, + "565621826": 0.0, + "585904130": 0.1652498530853035, + "626486813": 0.20632809068391378, + "636680137": 0.45862923715160336, + "706479600": 0.15129914778803955, + "750448296": 0.1527117559178966, + "75070641": 0.34224193319263285, + "767747424": 0.13255125751519692, + "77147264": 0.03379253116769787, + "794181626": 0.3619001242329895, + "957101489": 0.2708447347116597, + "998761162": 0.24035198055524792 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.18285887146206709, + "low": 0.133470413515452, + "mean": 0.15974686671965427, + "n": 5, + "num_samples": 200 + }, + "mean": 0.15974686671965427, + "per_task": { + "LiftPegUpright-v1": 0.10612647137472923, + "PickCube-v1": 0.1410368025527166, + "PullCube-v1": 0.1744863267265361, + "PushCube-v1": 0.18136164881975486, + "StackCube-v1": 0.19572308412453462 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.20414160361743686, + "low": 0.14859038870843547, + "mean": 0.17396925269606012, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6382978723404256, + "low": 0.35460992907801414, + "mean": 0.49645390070921985, + "n": 47, + "num_samples": 200 + }, + "mean": 0.49645390070921985, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6765912518853694, + "low": 0.12549019607843137, + "mean": 0.37402714932126696, + "n": 5, + "num_samples": 200 + }, + "mean": 0.37402714932126696, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.9230769230769231, + "StackCube-v1": 0.4666666666666667 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4782608695652174, + "low": 0.2753623188405797, + "mean": 0.37681159420289856, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3386524822695035, + "low": -0.005319148936170213, + "mean": 0.18794326241134754, + "n": 47, + "num_samples": 200 + }, + "mean": 0.18794326241134754, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": -1.0, + "1549324805": -0.3333333333333333, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.36923076923076925, + "low": -0.058823529411764705, + "mean": 0.14699849170437407, + "n": 5, + "num_samples": 200 + }, + "mean": 0.14699849170437407, + "per_task": { + "LiftPegUpright-v1": 0.0, + "PickCube-v1": 0.0, + "PullCube-v1": -0.14705882352941177, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.21739130434782608, + "low": -0.057971014492753624, + "mean": 0.10144927536231885, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.384272303660103, + "low": 0.9012552351477814, + "mean": 1.1504715986059424, + "n": 47, + "num_samples": 200 + }, + "mean": 1.1504715986059424, + "per_seed": { + "1025740212": 0.2148684412240982, + "1055157354": 0.812659502029419, + "1055567618": 2.0, + "1113268940": 0.7256770730018616, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 0.46439141035079956, + "1358421325": 0.8245300054550171, + "1375492786": 0.001480837701819837, + "141984615": 0.07189804397057742, + "1475440973": 0.7181311249732971, + "1483645336": 2.0, + "1522272686": 0.24029652774333954, + "1528740417": 0.7255622744560242, + "1549324805": 0.6790153384208679, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 0.2203868180513382, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.06726333405822515, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 0.8128684163093567, + "278008524": 0.019306766451336443, + "416183106": 0.22490769624710083, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 0.11818361282348633, + "565621826": 2.0, + "585904130": 0.01988216955214739, + "626486813": 0.06953954696655273, + "636680137": 0.015581388957798481, + "706479600": 2.0, + "750448296": 0.10195614024996758, + "75070641": 0.24087366461753845, + "767747424": 0.10304383677430451, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 0.5798611640930176, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.431180309034311, + "low": 0.5387795139021951, + "mean": 0.9849799114682529, + "n": 5, + "num_samples": 200 + }, + "mean": 0.9849799114682529, + "per_task": { + "LiftPegUpright-v1": 1.0640398189425468, + "PickCube-v1": 0.46439141035079956, + "PullCube-v1": 0.3505374649334151, + "PushCube-v1": 1.8488386101447618, + "StackCube-v1": 1.1970922529697419 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.0718660958118036, + "low": 0.6817222138600207, + "mean": 0.8805534762053656, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.600968998716814, + "low": 0.008772604939600467, + "mean": 0.3341213807176323, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3341213807176323, + "per_seed": { + "1025740212": -0.31350521743297577, + "1055157354": 0.6161754280328751, + "1055567618": 0.0, + "1113268940": 0.3033645451068878, + "1113371261": 0.0, + "1176943004": 1.845386266708374, + "1189392917": 1.8915813565254211, + "1231780997": 1.2729938626289368, + "1271718007": 0.584618866443634, + "1273247072": 0.3109617531299591, + "1358421325": -0.0026850104331970215, + "1375492786": -1.9985191622981802, + "141984615": -0.021914095113364358, + "1475440973": -1.2818688750267029, + "1483645336": 1.9835839420557022, + "1522272686": 0.0418534129858017, + "1528740417": -1.2744377255439758, + "1549324805": -0.6997321099042892, + "1607773860": 1.1673271656036377, + "1665739511": 0.0, + "1747178030": 1.8533687740564346, + "1881223807": 1.9679730087518692, + "1919198188": 0.18318794667720795, + "1991414747": 0.0, + "1997090716": 1.8607252091169357, + "2015956957": 0.00014354418963193893, + "2054899126": 0.0, + "211245842": 1.1611655950546265, + "2141802816": 1.9291927516460419, + "256770874": -1.1871315836906433, + "278008524": -0.08816510194446892, + "416183106": 0.02458128333091736, + "534717141": 0.0, + "551504635": 1.2760274410247803, + "565369392": -0.6815820336341858, + "565621826": 0.0, + "585904130": -0.055409928473333515, + "626486813": -0.43590046965982765, + "636680137": -1.9844186110422015, + "706479600": 1.1649112105369568, + "750448296": -0.009577419608831406, + "75070641": 0.0381220281124115, + "767747424": -0.44448035093955696, + "77147264": 0.5784205198287964, + "794181626": 1.8190439492464066, + "957101489": 0.376139298081398, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7216604295513223, + "low": -0.11880971740830834, + "mean": 0.31460225471957737, + "n": 5, + "num_samples": 200 + }, + "mean": 0.31460225471957737, + "per_task": { + "LiftPegUpright-v1": -0.002904328207174937, + "PickCube-v1": 0.3109617531299591, + "PullCube-v1": -0.30070200398810865, + "PushCube-v1": 1.1716741495407546, + "StackCube-v1": 0.3939817031224569 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.3720206405370213, + "low": -0.11932498876315852, + "mean": 0.16248053824866487, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3617021276595745, + "low": 0.13120567375886524, + "mean": 0.24113475177304963, + "n": 47, + "num_samples": 200 + }, + "mean": 0.24113475177304963, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.0, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3337858220211162, + "low": 0.05333333333333333, + "mean": 0.1758371040723982, + "n": 5, + "num_samples": 200 + }, + "mean": 0.1758371040723982, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.11764705882352941, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.13333333333333333 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.30434782608695654, + "low": 0.10144927536231885, + "mean": 0.18840579710144928, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.08687943262411349, + "low": -0.2304964539007092, + "mean": -0.06737588652482268, + "n": 47, + "num_samples": 200 + }, + "mean": -0.06737588652482268, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.0, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": -1.0, + "1549324805": -0.3333333333333333, + "1607773860": 1.0, + "1665739511": -1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": -0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.04564102564102565, + "low": -0.15254901960784314, + "mean": -0.051191553544494714, + "n": 5, + "num_samples": 200 + }, + "mean": -0.051191553544494714, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.17647058823529413, + "PushCube-v1": 0.15384615384615385, + "StackCube-v1": -0.06666666666666667 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.014492753623188406, + "low": -0.21739130434782608, + "mean": -0.08695652173913043, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9067608354204905, + "low": 0.45753479555867627, + "mean": 0.6653794013519014, + "n": 47, + "num_samples": 200 + }, + "mean": 0.6653794013519014, + "per_seed": { + "1025740212": 0.21175099909305573, + "1055157354": 0.16174951195716858, + "1055567618": 2.0, + "1113268940": 0.6624154448509216, + "1113371261": 2.0, + "1176943004": 0.04685409367084503, + "1189392917": 0.0735837072134018, + "1231780997": 2.0, + "1271718007": 1.1047073602676392, + "1273247072": 0.0375710129737854, + "1358421325": 0.8165659308433533, + "1375492786": 0.00048063197755254805, + "141984615": 0.03405998833477497, + "1475440973": 0.7181311249732971, + "1483645336": 2.0, + "1522272686": 0.20458371937274933, + "1528740417": 0.7171411514282227, + "1549324805": 0.6740981936454773, + "1607773860": 2.0, + "1665739511": 0.8184402585029602, + "1747178030": 0.09237110614776611, + "1881223807": 0.03511819243431091, + "1919198188": 0.03183290362358093, + "1991414747": 0.05380085110664368, + "1997090716": 2.0, + "2015956957": 0.06189579609781504, + "2054899126": 2.0, + "211245842": 0.8183318376541138, + "2141802816": 0.06590981781482697, + "256770874": 0.19993644952774048, + "278008524": 0.011877144395839423, + "416183106": 0.20836615562438965, + "534717141": 2.0, + "551504635": 0.7220139503479004, + "565369392": 0.07017792761325836, + "565621826": 2.0, + "585904130": 0.015381396127243837, + "626486813": 0.041404386749491096, + "636680137": 0.010638435371220112, + "706479600": 0.814424455165863, + "750448296": 0.057780176401138306, + "75070641": 0.2330203801393509, + "767747424": 0.05761944816913456, + "77147264": 1.1120431125164032, + "794181626": 0.03734047710895538, + "957101489": 0.23941433429718018, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7867360686720947, + "low": 0.26201372464497885, + "mean": 0.53948547703734, + "n": 5, + "num_samples": 200 + }, + "mean": 0.53948547703734, + "per_task": { + "LiftPegUpright-v1": 0.8385754451155663, + "PickCube-v1": 0.0375710129737854, + "PullCube-v1": 0.2737284779748948, + "PushCube-v1": 0.9488746569706843, + "StackCube-v1": 0.598677792151769 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.7215508813974009, + "low": 0.3420067752565534, + "mean": 0.5172656428653484, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.09389386805202087, + "low": -0.4070899225106786, + "mean": -0.15097081653640856, + "n": 47, + "num_samples": 200 + }, + "mean": -0.15097081653640856, + "per_seed": { + "1025740212": -0.31662265956401825, + "1055157354": -0.034734562039375305, + "1055567618": 0.0, + "1113268940": 0.24010291695594788, + "1113371261": 0.0, + "1176943004": -0.10775963962078094, + "1189392917": -0.03483493626117706, + "1231780997": 1.2729938626289368, + "1271718007": -0.3106737732887268, + "1273247072": -0.11585864424705505, + "1358421325": -0.01064908504486084, + "1375492786": -1.9995193680224475, + "141984615": -0.05975215074916681, + "1475440973": -1.2818688750267029, + "1483645336": 1.9835839420557022, + "1522272686": 0.006140604615211487, + "1528740417": -1.2828588485717773, + "1549324805": -0.7046492546796799, + "1607773860": 1.1673271656036377, + "1665739511": -1.1815597414970398, + "1747178030": -0.054260119795799255, + "1881223807": 0.0030912011861801147, + "1919198188": -0.005365967750549316, + "1991414747": -1.9461991488933563, + "1997090716": 1.8607252091169357, + "2015956957": -0.005223993770778179, + "2054899126": 0.0, + "211245842": -0.020502567291259766, + "2141802816": -0.0048974305391311646, + "256770874": -1.8000635504722595, + "278008524": -0.09559472399996594, + "416183106": 0.008039742708206177, + "534717141": 0.0, + "551504635": -0.001958608627319336, + "565369392": -0.7295877188444138, + "565621826": 0.0, + "585904130": -0.05991070189823707, + "626486813": -0.4640356298768893, + "636680137": -1.9893615646287799, + "706479600": -0.020664334297180176, + "750448296": -0.053753383457660675, + "75070641": 0.030268743634223938, + "767747424": -0.4899047395447269, + "77147264": -0.3095363676548004, + "794181626": -0.14361557364463806, + "957101489": 0.03569246828556061, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.07167863700634394, + "low": -0.29535206382421947, + "mean": -0.1308921797113357, + "n": 5, + "num_samples": 200 + }, + "mean": -0.1308921797113357, + "per_task": { + "LiftPegUpright-v1": -0.22836870203415552, + "PickCube-v1": -0.11585864424705505, + "PullCube-v1": -0.377510990946629, + "PushCube-v1": 0.2717101963666769, + "StackCube-v1": -0.20443275769551594 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": -0.021096388483335657, + "low": -0.4195698232493748, + "mean": -0.20080729509135237, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6866153551741166, + "low": 0.2764395795932208, + "mean": 0.4850921972540409, + "n": 47, + "num_samples": 200 + }, + "mean": 0.4850921972540409, + "per_seed": { + "1025740212": 0.0031174421310424805, + "1055157354": 0.6509099900722504, + "1055567618": 0.0, + "1113268940": 0.06326162815093994, + "1113371261": 0.0, + "1176943004": 1.953145906329155, + "1189392917": 1.9264162927865982, + "1231780997": 0.0, + "1271718007": 0.8952926397323608, + "1273247072": 0.42682039737701416, + "1358421325": 0.007964074611663818, + "1375492786": 0.001000205724267289, + "141984615": 0.03783805563580245, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.03571280837059021, + "1528740417": 0.008421123027801514, + "1549324805": 0.004917144775390625, + "1607773860": 0.0, + "1665739511": 1.1815597414970398, + "1747178030": 1.9076288938522339, + "1881223807": 1.964881807565689, + "1919198188": 0.18855391442775726, + "1991414747": 1.9461991488933563, + "1997090716": 0.0, + "2015956957": 0.005367537960410118, + "2054899126": 0.0, + "211245842": 1.1816681623458862, + "2141802816": 1.934090182185173, + "256770874": 0.6129319667816162, + "278008524": 0.0074296220554970205, + "416183106": 0.01654154062271118, + "534717141": 0.0, + "551504635": 1.2779860496520996, + "565369392": 0.048005685210227966, + "565621826": 0.0, + "585904130": 0.004500773424903552, + "626486813": 0.02813516021706164, + "636680137": 0.004942953586578369, + "706479600": 1.185575544834137, + "750448296": 0.04417596384882927, + "75070641": 0.007853284478187561, + "767747424": 0.04542438860516995, + "77147264": 0.8879568874835968, + "794181626": 1.9626595228910446, + "957101489": 0.3404468297958374, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6847154450722229, + "low": 0.18113008173041084, + "mean": 0.4454944344309131, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4454944344309131, + "per_task": { + "LiftPegUpright-v1": 0.2254643738269806, + "PickCube-v1": 0.42682039737701416, + "PullCube-v1": 0.07680898695852033, + "PushCube-v1": 0.8999639531740775, + "StackCube-v1": 0.5984144608179728 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5186884595309823, + "low": 0.2226778654005392, + "mean": 0.36328783334001724, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3723404255319149, + "low": 0.13829787234042554, + "mean": 0.2553191489361702, + "n": 47, + "num_samples": 200 + }, + "mean": 0.2553191489361702, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.34947209653092004, + "low": 0.050980392156862744, + "mean": 0.1981900452488688, + "n": 5, + "num_samples": 200 + }, + "mean": 0.1981900452488688, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.029411764705882353, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.3333333333333333 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2753623188405797, + "low": 0.10144927536231885, + "mean": 0.18840579710144928, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.4716312056737588, + "low": 0.2347517730496454, + "mean": 0.33936170212765954, + "n": 47, + "num_samples": 200 + }, + "mean": 0.33936170212765954, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7487179487179487, + "low": 0.2147812971342383, + "mean": 0.49067873303167414, + "n": 5, + "num_samples": 200 + }, + "mean": 0.49067873303167414, + "per_task": { + "LiftPegUpright-v1": 0.6666666666666666, + "PickCube-v1": 1.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.07692307692307693, + "StackCube-v1": 0.5333333333333333 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.37681159420289856, + "low": 0.17391304347826086, + "mean": 0.2898550724637681, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7404255319148936, + "low": 0.4702127659574468, + "mean": 0.5946808510638298, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5946808510638298, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8866666666666667, + "low": 0.43122171945701354, + "mean": 0.688868778280543, + "n": 5, + "num_samples": 200 + }, + "mean": 0.688868778280543, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.20588235294117646, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.8666666666666667 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5942028985507246, + "low": 0.3333333333333333, + "mean": 0.4782608695652174, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7543874885621051, + "low": 0.39060924925332796, + "mean": 0.555731212557954, + "n": 47, + "num_samples": 200 + }, + "mean": 0.555731212557954, + "per_seed": { + "1025740212": 1.7851315587759018, + "1055157354": 1.187340497970581, + "1055567618": 0.0, + "1113268940": 1.2743229269981384, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 1.5356085896492004, + "1358421325": 1.175469994544983, + "1375492786": 0.2419701317558065, + "141984615": 0.12854979850817472, + "1475440973": 1.2818688750267029, + "1483645336": 0.0, + "1522272686": 1.7597034722566605, + "1528740417": 1.2744377255439758, + "1549324805": 0.7237351834774017, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 1.7796131819486618, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.46261803302913906, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.1871315836906433, + "278008524": 1.0945044284453616, + "416183106": 1.7750923037528992, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 1.308447966972987, + "565621826": 0.0, + "585904130": 0.21776482245574394, + "626486813": 0.5429288633167744, + "636680137": 1.9844186110422015, + "706479600": 0.0, + "750448296": 0.13385144993662834, + "75070641": 1.7591263353824615, + "767747424": 0.08559181983582675, + "77147264": 0.0, + "794181626": 0.0, + "957101489": 1.4201388359069824, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.1683035775687656, + "low": 0.3937598540945341, + "mean": 0.7714844558916235, + "n": 5, + "num_samples": 200 + }, + "mean": 0.7714844558916235, + "per_task": { + "LiftPegUpright-v1": 0.9359601810574532, + "PickCube-v1": 1.5356085896492004, + "PullCube-v1": 0.4317843718659681, + "PushCube-v1": 0.1511613898552381, + "StackCube-v1": 0.8029077470302581 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6671761653750486, + "low": 0.35016158437255124, + "mean": 0.5194311970001275, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.2788291653615742, + "low": 0.8215172549003754, + "mean": 1.0408234098119948, + "n": 47, + "num_samples": 200 + }, + "mean": 1.0408234098119948, + "per_seed": { + "1025740212": 1.7882490009069443, + "1055157354": 1.8382504880428314, + "1055567618": 0.0, + "1113268940": 1.3375845551490784, + "1113371261": 0.0, + "1176943004": 1.953145906329155, + "1189392917": 1.9264162927865982, + "1231780997": 0.0, + "1271718007": 0.8952926397323608, + "1273247072": 1.9624289870262146, + "1358421325": 1.1834340691566467, + "1375492786": 0.2429703374800738, + "141984615": 0.16638785414397717, + "1475440973": 1.2818688750267029, + "1483645336": 0.0, + "1522272686": 1.7954162806272507, + "1528740417": 1.2828588485717773, + "1549324805": 0.7286523282527924, + "1607773860": 0.0, + "1665739511": 1.1815597414970398, + "1747178030": 1.9076288938522339, + "1881223807": 1.964881807565689, + "1919198188": 1.968167096376419, + "1991414747": 1.9461991488933563, + "1997090716": 0.0, + "2015956957": 0.4679855709895492, + "2054899126": 0.0, + "211245842": 1.1816681623458862, + "2141802816": 1.934090182185173, + "256770874": 1.8000635504722595, + "278008524": 1.1019340505008586, + "416183106": 1.7916338443756104, + "534717141": 0.0, + "551504635": 1.2779860496520996, + "565369392": 1.356453652183215, + "565621826": 0.0, + "585904130": 0.2222655958806475, + "626486813": 0.571064023533836, + "636680137": 1.9893615646287799, + "706479600": 1.185575544834137, + "750448296": 0.1780274137854576, + "75070641": 1.766979619860649, + "767747424": 0.1310162084409967, + "77147264": 0.8879568874835968, + "794181626": 1.9626595228910446, + "957101489": 1.7605856657028198, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.6199673717984786, + "low": 0.817705367841226, + "mean": 1.2169788903225367, + "n": 5, + "num_samples": 200 + }, + "mean": 1.2169788903225367, + "per_task": { + "LiftPegUpright-v1": 1.1614245548844337, + "PickCube-v1": 1.9624289870262146, + "PullCube-v1": 0.5085933588244884, + "PushCube-v1": 1.0511253430293157, + "StackCube-v1": 1.401322207848231 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 1.056229973670797, + "low": 0.6653890719201139, + "mean": 0.8827190303401448, + "n": 69, + "num_samples": 200 + }, + "n": 69 + } + }, + "num_rows": 69, + "report_type": "ctt_generated_measured_rollout", + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "source_split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "success_summary": { + "base_success_rate": 0.2753623188405797, + "base_utility_mean": 0.7180729379567007, + "hidden_chart_oracle_success_rate": 0.6666666666666666, + "hidden_chart_oracle_utility_mean": 1.3999846732054932, + "max_restore_error": 1.4901161193847656e-08, + "proposal_oracle_success_gain_over_base": 0.10144927536231885, + "proposal_oracle_success_rate": 0.37681159420289856, + "proposal_oracle_utility_mean": 0.8805534762053656, + "selected_success_gain_over_base": -0.08695652173913043, + "selected_success_rate": 0.18840579710144928, + "selected_utility_mean": 0.5172656428653484, + "success_selector_gap": 0.18840579710144928, + "success_support_gap": 0.2898550724637681 + }, + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381" +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics_by_seed.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..f043b4bd8f082a11390732855bdbb4598d4ebd0f --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics_by_seed.json @@ -0,0 +1,1083 @@ +{ + "1025740212": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.0031174421310424805, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.14400158741031796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2148684412240982, + "proposal_oracle_utility_gain_over_base_at_8": -0.31350521743297577, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21175099909305573, + "selected_utility_gain_over_base_at_8": -0.31662265956401825, + "selector_regret_at_8": 0.0031174421310424805, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7851315587759018, + "total_car_to_hidden_at_8": 1.7882490009069443 + }, + "1055157354": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.6509099900722504, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32609963351937027, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.812659502029419, + "proposal_oracle_utility_gain_over_base_at_8": 0.6161754280328751, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.16174951195716858, + "selected_utility_gain_over_base_at_8": -0.034734562039375305, + "selector_regret_at_8": 0.6509099900722504, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.187340497970581, + "total_car_to_hidden_at_8": 1.8382504880428314 + }, + "1055567618": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15812100047218913, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1113268940": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.06326162815093994, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06911736513585261, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7256770730018616, + "proposal_oracle_utility_gain_over_base_at_8": 0.3033645451068878, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6624154448509216, + "selected_utility_gain_over_base_at_8": 0.24010291695594788, + "selector_regret_at_8": 0.06326162815093994, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2743229269981384, + "total_car_to_hidden_at_8": 1.3375845551490784 + }, + "1113371261": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.015047211698777683, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.953145906329155, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15442455751472706, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04685409367084503, + "selected_utility_gain_over_base_at_8": -0.10775963962078094, + "selector_regret_at_8": 1.953145906329155, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.953145906329155 + }, + "1189392917": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 1.9264162927865982, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10709427493066431, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0735837072134018, + "selected_utility_gain_over_base_at_8": -0.03483493626117706, + "selector_regret_at_8": 1.9264162927865982, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9264162927865982 + }, + "1231780997": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.07419277505792353, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2729938626289368, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.2729938626289368, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1271718007": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.8952926397323608, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10710899843183383, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1047073602676392, + "selected_utility_gain_over_base_at_8": -0.3106737732887268, + "selector_regret_at_8": 0.8952926397323608, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8952926397323608 + }, + "1273247072": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + "1358421325": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.007964074611663818, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.04701404433315851, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8245300054550171, + "proposal_oracle_utility_gain_over_base_at_8": -0.0026850104331970215, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8165659308433533, + "selected_utility_gain_over_base_at_8": -0.01064908504486084, + "selector_regret_at_8": 0.007964074611663818, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.175469994544983, + "total_car_to_hidden_at_8": 1.1834340691566467 + }, + "1375492786": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.001000205724267289, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16573502466028908, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.001480837701819837, + "proposal_oracle_utility_gain_over_base_at_8": -1.9985191622981802, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.00048063197755254805, + "selected_utility_gain_over_base_at_8": -1.9995193680224475, + "selector_regret_at_8": 0.001000205724267289, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2419701317558065, + "total_car_to_hidden_at_8": 0.2429703374800738 + }, + "141984615": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03783805563580245, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.6666666666666666, + "pairwise_causal_calibration_ece": 0.16666657870321888, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07189804397057742, + "proposal_oracle_utility_gain_over_base_at_8": -0.021914095113364358, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03405998833477497, + "selected_utility_gain_over_base_at_8": -0.05975215074916681, + "selector_regret_at_8": 0.03783805563580245, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12854979850817472, + "total_car_to_hidden_at_8": 0.16638785414397717 + }, + "1475440973": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0862090980213101, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7181311249732971, + "proposal_oracle_utility_gain_over_base_at_8": -1.2818688750267029, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7181311249732971, + "selected_utility_gain_over_base_at_8": -1.2818688750267029, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2818688750267029, + "total_car_to_hidden_at_8": 1.2818688750267029 + }, + "1483645336": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06079377324000063, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1522272686": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.03571280837059021, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24182565394435904, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24029652774333954, + "proposal_oracle_utility_gain_over_base_at_8": 0.0418534129858017, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20458371937274933, + "selected_utility_gain_over_base_at_8": 0.006140604615211487, + "selector_regret_at_8": 0.03571280837059021, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7597034722566605, + "total_car_to_hidden_at_8": 1.7954162806272507 + }, + "1528740417": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.008421123027801514, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.026961937751403514, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.7255622744560242, + "proposal_oracle_utility_gain_over_base_at_8": -1.2744377255439758, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7171411514282227, + "selected_utility_gain_over_base_at_8": -1.2828588485717773, + "selector_regret_at_8": 0.008421123027801514, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2744377255439758, + "total_car_to_hidden_at_8": 1.2828588485717773 + }, + "1549324805": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.004917144775390625, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10346856229255086, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6790153384208679, + "proposal_oracle_utility_gain_over_base_at_8": -0.6997321099042892, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6740981936454773, + "selected_utility_gain_over_base_at_8": -0.7046492546796799, + "selector_regret_at_8": 0.004917144775390625, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7237351834774017, + "total_car_to_hidden_at_8": 0.7286523282527924 + }, + "1607773860": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2544258391757567, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1673271656036377, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1665739511": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.1815597414970398, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.05717674118582694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.8184402585029602, + "selected_utility_gain_over_base_at_8": -1.1815597414970398, + "selector_regret_at_8": 1.1815597414970398, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1815597414970398 + }, + "1747178030": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 1.9076288938522339, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1055876219626688, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09237110614776611, + "selected_utility_gain_over_base_at_8": -0.054260119795799255, + "selector_regret_at_8": 1.9076288938522339, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9076288938522339 + }, + "1881223807": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 1.964881807565689, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10190418918720931, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9679730087518692, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03511819243431091, + "selected_utility_gain_over_base_at_8": 0.0030912011861801147, + "selector_regret_at_8": 1.964881807565689, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.964881807565689 + }, + "1919198188": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.18855391442775726, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2569381375337174, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.2203868180513382, + "proposal_oracle_utility_gain_over_base_at_8": 0.18318794667720795, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03183290362358093, + "selected_utility_gain_over_base_at_8": -0.005365967750549316, + "selector_regret_at_8": 0.18855391442775726, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7796131819486618, + "total_car_to_hidden_at_8": 1.968167096376419 + }, + "1991414747": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9461991488933563, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.44645064197830664, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.05380085110664368, + "selected_utility_gain_over_base_at_8": -1.9461991488933563, + "selector_regret_at_8": 1.9461991488933563, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9461991488933563 + }, + "1997090716": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3553584127845171, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2015956957": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.005367537960410118, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.6, + "pairwise_causal_calibration_ece": 0.2284508985012672, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06726333405822515, + "proposal_oracle_utility_gain_over_base_at_8": 0.00014354418963193893, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06189579609781504, + "selected_utility_gain_over_base_at_8": -0.005223993770778179, + "selector_regret_at_8": 0.005367537960410118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.2, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.46261803302913906, + "total_car_to_hidden_at_8": 0.4679855709895492 + }, + "2054899126": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 1.1816681623458862, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.036206191308278124, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8183318376541138, + "selected_utility_gain_over_base_at_8": -0.020502567291259766, + "selector_regret_at_8": 1.1816681623458862, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.1816681623458862 + }, + "2141802816": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.934090182185173, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.40989236974714055, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06590981781482697, + "selected_utility_gain_over_base_at_8": -0.0048974305391311646, + "selector_regret_at_8": 1.934090182185173, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.934090182185173 + }, + "256770874": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.6129319667816162, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3193203145842822, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.8128684163093567, + "proposal_oracle_utility_gain_over_base_at_8": -1.1871315836906433, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.19993644952774048, + "selected_utility_gain_over_base_at_8": -1.8000635504722595, + "selector_regret_at_8": 0.6129319667816162, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1871315836906433, + "total_car_to_hidden_at_8": 1.8000635504722595 + }, + "278008524": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0074296220554970205, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10627711331165646, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.019306766451336443, + "proposal_oracle_utility_gain_over_base_at_8": -0.08816510194446892, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011877144395839423, + "selected_utility_gain_over_base_at_8": -0.09559472399996594, + "selector_regret_at_8": 0.0074296220554970205, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0945044284453616, + "total_car_to_hidden_at_8": 1.1019340505008586 + }, + "416183106": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.01654154062271118, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1925298309196861, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22490769624710083, + "proposal_oracle_utility_gain_over_base_at_8": 0.02458128333091736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.20836615562438965, + "selected_utility_gain_over_base_at_8": 0.008039742708206177, + "selector_regret_at_8": 0.01654154062271118, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7750923037528992, + "total_car_to_hidden_at_8": 1.7916338443756104 + }, + "534717141": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.12736602361488303, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.2779860496520996, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.23627606487156766, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.7220139503479004, + "selected_utility_gain_over_base_at_8": -0.001958608627319336, + "selector_regret_at_8": 1.2779860496520996, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2779860496520996 + }, + "565369392": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.048005685210227966, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2859321646983037, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.11818361282348633, + "proposal_oracle_utility_gain_over_base_at_8": -0.6815820336341858, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.07017792761325836, + "selected_utility_gain_over_base_at_8": -0.7295877188444138, + "selector_regret_at_8": 0.048005685210227966, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.6666666666666666, + "support_gap_at_8": 1.308447966972987, + "total_car_to_hidden_at_8": 1.356453652183215 + }, + "565621826": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.004500773424903552, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1652498530853035, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01988216955214739, + "proposal_oracle_utility_gain_over_base_at_8": -0.055409928473333515, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.015381396127243837, + "selected_utility_gain_over_base_at_8": -0.05991070189823707, + "selector_regret_at_8": 0.004500773424903552, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.21776482245574394, + "total_car_to_hidden_at_8": 0.2222655958806475 + }, + "626486813": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.02813516021706164, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20632809068391378, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06953954696655273, + "proposal_oracle_utility_gain_over_base_at_8": -0.43590046965982765, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.041404386749491096, + "selected_utility_gain_over_base_at_8": -0.4640356298768893, + "selector_regret_at_8": 0.02813516021706164, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5429288633167744, + "total_car_to_hidden_at_8": 0.571064023533836 + }, + "636680137": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.004942953586578369, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.45862923715160336, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.015581388957798481, + "proposal_oracle_utility_gain_over_base_at_8": -1.9844186110422015, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.010638435371220112, + "selected_utility_gain_over_base_at_8": -1.9893615646287799, + "selector_regret_at_8": 0.004942953586578369, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9844186110422015, + "total_car_to_hidden_at_8": 1.9893615646287799 + }, + "706479600": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 1.185575544834137, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15129914778803955, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814424455165863, + "selected_utility_gain_over_base_at_8": -0.020664334297180176, + "selector_regret_at_8": 1.185575544834137, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185575544834137 + }, + "750448296": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.04417596384882927, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.1527117559178966, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10195614024996758, + "proposal_oracle_utility_gain_over_base_at_8": -0.009577419608831406, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.057780176401138306, + "selected_utility_gain_over_base_at_8": -0.053753383457660675, + "selector_regret_at_8": 0.04417596384882927, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.13385144993662834, + "total_car_to_hidden_at_8": 0.1780274137854576 + }, + "75070641": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.007853284478187561, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.34224193319263285, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.24087366461753845, + "proposal_oracle_utility_gain_over_base_at_8": 0.0381220281124115, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.2330203801393509, + "selected_utility_gain_over_base_at_8": 0.030268743634223938, + "selector_regret_at_8": 0.007853284478187561, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7591263353824615, + "total_car_to_hidden_at_8": 1.766979619860649 + }, + "767747424": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.04542438860516995, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.13255125751519692, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.10304383677430451, + "proposal_oracle_utility_gain_over_base_at_8": -0.44448035093955696, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.05761944816913456, + "selected_utility_gain_over_base_at_8": -0.4899047395447269, + "selector_regret_at_8": 0.04542438860516995, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.08559181983582675, + "total_car_to_hidden_at_8": 0.1310162084409967 + }, + "77147264": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.8879568874835968, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.03379253116769787, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.5784205198287964, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.1120431125164032, + "selected_utility_gain_over_base_at_8": -0.3095363676548004, + "selector_regret_at_8": 0.8879568874835968, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.8879568874835968 + }, + "794181626": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 1.9626595228910446, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3619001242329895, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03734047710895538, + "selected_utility_gain_over_base_at_8": -0.14361557364463806, + "selector_regret_at_8": 1.9626595228910446, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9626595228910446 + }, + "957101489": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.3404468297958374, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2708447347116597, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5798611640930176, + "proposal_oracle_utility_gain_over_base_at_8": 0.376139298081398, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.23941433429718018, + "selected_utility_gain_over_base_at_8": 0.03569246828556061, + "selector_regret_at_8": 0.3404468297958374, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.4201388359069824, + "total_car_to_hidden_at_8": 1.7605856657028198 + }, + "998761162": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.24035198055524792, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics_by_task.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..e5ab43c2cd385e0902e327f97b1defce6119af35 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.2254643738269806, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.10612647137472923, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.0640398189425468, + "proposal_oracle_utility_gain_over_base_at_8": -0.002904328207174937, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8385754451155663, + "selected_utility_gain_over_base_at_8": -0.22836870203415552, + "selector_regret_at_8": 0.2254643738269806, + "success_selector_gap_at_8": 0.16666666666666666, + "success_support_gap_at_8": 0.6666666666666666, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.9359601810574532, + "total_car_to_hidden_at_8": 1.1614245548844337 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.42682039737701416, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1410368025527166, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.46439141035079956, + "proposal_oracle_utility_gain_over_base_at_8": 0.3109617531299591, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.0375710129737854, + "selected_utility_gain_over_base_at_8": -0.11585864424705505, + "selector_regret_at_8": 0.42682039737701416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.5356085896492004, + "total_car_to_hidden_at_8": 1.9624289870262146 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.07680898695852033, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.38235294117647056, + "pairwise_causal_calibration_ece": 0.1744863267265361, + "proposal_oracle_success_at_8": 0.14705882352941177, + "proposal_oracle_success_gain_over_base_at_8": -0.14705882352941177, + "proposal_oracle_utility_at_8": 0.3505374649334151, + "proposal_oracle_utility_gain_over_base_at_8": -0.30070200398810865, + "selected_success_at_8": 0.11764705882352941, + "selected_success_gain_over_base_at_8": -0.17647058823529413, + "selected_utility_at_8": 0.2737284779748948, + "selected_utility_gain_over_base_at_8": -0.377510990946629, + "selector_regret_at_8": 0.07680898695852033, + "success_selector_gap_at_8": 0.029411764705882353, + "success_support_gap_at_8": 0.17647058823529413, + "success_total_car_to_hidden_at_8": 0.20588235294117646, + "support_gap_at_8": 0.4317843718659681, + "total_car_to_hidden_at_8": 0.5085933588244884 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.8999639531740775, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6923076923076923, + "pairwise_causal_calibration_ece": 0.18136164881975486, + "proposal_oracle_success_at_8": 0.9230769230769231, + "proposal_oracle_success_gain_over_base_at_8": 0.6153846153846154, + "proposal_oracle_utility_at_8": 1.8488386101447618, + "proposal_oracle_utility_gain_over_base_at_8": 1.1716741495407546, + "selected_success_at_8": 0.46153846153846156, + "selected_success_gain_over_base_at_8": 0.15384615384615385, + "selected_utility_at_8": 0.9488746569706843, + "selected_utility_gain_over_base_at_8": 0.2717101963666769, + "selector_regret_at_8": 0.8999639531740775, + "success_selector_gap_at_8": 0.46153846153846156, + "success_support_gap_at_8": 0.07692307692307693, + "success_total_car_to_hidden_at_8": 0.5384615384615384, + "support_gap_at_8": 0.1511613898552381, + "total_car_to_hidden_at_8": 1.0511253430293157 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.5984144608179728, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.7333333333333333, + "pairwise_causal_calibration_ece": 0.19572308412453462, + "proposal_oracle_success_at_8": 0.4666666666666667, + "proposal_oracle_success_gain_over_base_at_8": 0.26666666666666666, + "proposal_oracle_utility_at_8": 1.1970922529697419, + "proposal_oracle_utility_gain_over_base_at_8": 0.3939817031224569, + "selected_success_at_8": 0.13333333333333333, + "selected_success_gain_over_base_at_8": -0.06666666666666667, + "selected_utility_at_8": 0.598677792151769, + "selected_utility_gain_over_base_at_8": -0.20443275769551594, + "selector_regret_at_8": 0.5984144608179728, + "success_selector_gap_at_8": 0.3333333333333333, + "success_support_gap_at_8": 0.5333333333333333, + "success_total_car_to_hidden_at_8": 0.8666666666666667, + "support_gap_at_8": 0.8029077470302581, + "total_car_to_hidden_at_8": 1.401322207848231 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/report.md b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/report.md new file mode 100644 index 0000000000000000000000000000000000000000..39791f79e8e91b2c1708c6179a45841b056a14f7 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/report.md @@ -0,0 +1,13 @@ +# CTT Generated Measured Rollout + +Rows: `69` +K: `8` +Checkpoint: `runs/ctt_residual_base_context_obj_seed1/model.pt` + +| Metric | Mean | +| --- | ---: | +| OutcomePTR@K | 0.5362 | +| SelectorRegret@K | 0.3633 | +| SupportGap@K | 0.5194 | + +These are measured simulator rollouts of decoded CTT candidates, not PPTC proxies. diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/run.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/run.log new file mode 100644 index 0000000000000000000000000000000000000000..3d65976412deaee2f45e25b919d53e6822b5a280 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/run.log @@ -0,0 +1,766 @@ +start +importing gymnasium/mani_skill +imported gymnasium/mani_skill +loaded checkpoint=runs/ctt_residual_base_context_obj_seed1/model.pt +loaded charts source=301 target=69 +generated proposals rows=69 total=552 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=PickCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PickCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +rollout complete rows=69 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_data_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..1ef49748c0932e9ccb1c83a065667e7a4032f7db --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_data_hash.txt @@ -0,0 +1 @@ +67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_index.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_index.json new file mode 100644 index 0000000000000000000000000000000000000000..21f76a4f847a39cd7631c9675f06c81edf567240 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_index.json @@ -0,0 +1,4179 @@ +{ + "audience": "train_retrieval", + "base_candidate_types": [ + "policy", + "policy_residual", + "anchor", + "base", + "expert" + ], + "candidate_type_counts": { + "expert": 2044, + "near_miss": 4088, + "no_op": 2044, + "random_negative": 20440, + "wrong_direction": 2044, + "wrong_gripper": 2044 + }, + "content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "dataset": "/scratch/knguy52/dovla/experiments/six_task_h16_collection", + "deployment_candidate_excludes_expert": true, + "deployment_clean": true, + "embedding_content_hash": "00859bfac7590abb24a3e7cc4c26fbb17122ba52d9f63369b8bb71a6ad1bc302", + "epsilon": 0.05, + "format": "cil_charts_npz", + "group_hashes": [ + "001fcaa1974d30d8f6edbd3e9117a10a084af107360ce1b7899dba37d093d073", + "00526cba701d9c5d7399d0436d2718fe144013e2e0277c3e00f247c3f2b026e2", + "007da68284552fa257c5929a32c040913c93b9f31569bc4a9e0e5f469972c3f5", + "0081224c84c2faa89c82c842265c41240c850ffc35d5400b57531935df3d704d", + "00bbd346a4f8658a96b2167b8ed3b4a4275cd535f5b46ff6a2ba6fe55ab5af22", + "00c41e13d6af229bd31262983b05d48413360e477204b62ba89278a110632a3c", + "00ccbf9e23e7dd8cc6ea5d91197bab1ee5d7e26f1c2e9dff0db7c48332e262d7", + "00df496b285db96e866529484da8bed3b079234ab8ed99faf38a9b105ae136d5", + "01335787d62e2c1bfa20e6a53a53239d19087e703ff07e2bb0666862536259b8", + "014ef3543741d318e6b6f9a3427e4423b1594cc26a73fbfa8266a1dadffc051b", + "015b93c295196814c0cf417d49ecf1d82d00e68458e42d2a0b3a1179d6d4c135", + "01a286ab7524d1d907099802c01da4995ebb8381d71bba0460100499c3c622f9", + "01e693c8677801418926281fd44938d17c1d275ed367253f31d1208676259a5a", + "023bdae7b5ddc517f49ec900a0e48dc698bfe1ec6f1a7cc7edfac21b46f37333", + "025c271db3f4ffb8e484ec4ac3f7244b0380bbf904037af516580e8d7ebf605a", + "0267094d9bc010468e6f684ab19365e65b31b38da78897277def7201c91bac27", + "0294a908f74ea0b1e54e4fb6d547fe5ba986ca4c6881572213732a0c867699ba", + "02fc97ab4aa793222f215b66d0ffc5c856706ec647c7482ee66344e066456af2", + "03020ce876d6f83841e56c51882dc8e6be8bf33ec9c9c836ab42ab8e4a586538", + "030b87c34937f69c95bb61bf59d9773b5a365a86fa73a60f619ccd0256d2968b", + "036bd56b347b3ea03e19a9a1e1aa865021d7b541fbdca003409a10e34912b3c7", + "0388ca1306e31d2ebd07b30a8d5366063523b4cb18d426c93351e7497cdf2721", + "03b05cc0e617e7ab7149752551af4303fa838be521fe4b22a15389558dfc0d60", + "0414958360a2c4e04be7534d9347e289e132993e773f7518bc8ee75a02ede570", + "041af35d269bba29a9d94019ad4c8e433bd9b11e3d2c12af4922829649df041e", + "042cc293ccfe917dac796cd6101f1d2f4fd0fc5c67a050c391399bb3c89b83e9", + "043d323f81c1208341028b4176ed8c4c56e69abb8c08aec31ddf093056d9db95", + "044f47ab5be1af5d16cf8dc96c3201970c551380fc2e97dcce7ae2f5265658b3", + "04529d1ba49a38cd27f3ffca329e63ab856f5c1e9cadcf7599c074ea64d26acb", + "045cb769e1cc19f043df64a32a05dc520442e9dd62fe0dcaac93c048c8a53589", + "04a1476f8f2cbb1493afb3e1849003222903e15a7cc7e519dfabb48bd1bfb57a", + "04c63f785efcf0d3591dd2051455fa4878b473bc2cbeee626ec7e549bbf46a14", + "04d6d326619cb05f115813e476605dca0485b6bcbb256765822fd1be38def668", + "04efaf4eb209e619534ffaa3bbb08fdc7a05759f77c96ede295435e899557617", + "04f4ff1016421b38f73be280be75ca4fea5c372f3a8d1cda4da185de229da025", + "04faed0a7dfd3ebc89a4bd2e8e075c9a81c861e396b860cf1172c5ff89245a11", + "05240853bb100723195b299a8e993517d7636f79d04d39f3d60ebe025863c321", + "054f169f2c20ec618c2805cc410f661edd54db72ff4e124ef2e8e695f5c41483", + "057423d91c0d82d8f62f114440e3d5800ff93a36d789dee2cb1558189230b3f5", + "058273a8b1145d6860b07cd133fbfb6439dadbc9337d32c27c9b6e353b9938c6", + "05b7a5da0c35c1c6c9dd3e767ba83f3a5d8c65cdc688f7c7583b086d2f5e578f", + "05c0e14adca2fb1afd50c9c82effa52f4a3c036d865a6ef07cf43ab6cb7cf9c1", + "05c0fa198643aa9b11a16b11f508c0f3e4876d76adb60edfd63e5b22da4f6096", + "05d812f28f8fa0fc86ae7e47fee0b87e82a8c147cd2c20a18efca3e4d1716ed0", + "05eff262f0c51434f1a2717cd888ac132d5418a211f5be302c76204c694c9c22", + "05f242687ca564fcce8802518ff330054ba4aa6fd52dc16b739af6e2cd73265c", + "060071dc4d719b0172224dadd17bbcad4074d9a025bd9753dc25bcaa82a1b731", + "0612f705ce60a6e5719459f4ed2ae70cc12c6f610553af267bba19f1079e159d", + "062f09156bfa5ecc2c90411a5c30cf7ab44cfdab8fdf6bba8dbf0d0264398669", + "0637e894caaf7866508328256b86a787acb4b59279b4bc4d77fa920669620b0d", + "06646b414506091dc9562eec9f7ef5a47fc11f0d46fefd5f17118e104334ad20", + "069838f6f7f1d064c02323989485d308d36f8650e6062785a461e2b6cff0b0b0", + "06b2ec0364fcc116b85d61aa2b8f007fc319e48756acfc228765b7aa0e79db87", + "06f3fd02d9a8cbf4fa564d32546c328fc318d993b9d2d1ef685324463c69ddbe", + "0713a51c5892c5558579ce4dffdc60b974477b43e53c3fe0ae4345b45549ec17", + "07202bb06fd690678da08362e77ce653818eba36201fadb49d49d1e56ec9c228", + "0749514f298a42cfe924e24c240e25f5ff48f47a898f8586b200cfa0701c7a37", + "0768ebbabf013c2a4b7f1aad3a2099c8c13cc1b0b6a083b446a17336f2d17d04", + "07721a0faa4a8ca8a5af0f07c002538292f7cf23086a796ac71f17d92c68ccd5", + "077f12bea73da5c959c3558c431a5137f203c4468023cd9bb80ed7aa72c5cb15", + "0781ab28ea0865deb62cdf28ff45c8db767f6acac236094d1ac9671a621c0b10", + "079f2e4745cdf1e6fe28c5fd9c44ff1154b2317c9670dce3df3e46c5956b6266", + "07d1cf8fa045404915eaa0bc582d7c1f1ea3e5417ebd7d9b076c0106cf937b49", + "07d78d133b041c1b387a66dde85c2e8d619868aadbe25df65adf9a537733fbe2", + "07d981b9235d6f62b04fe940d3b60888c7324180a1eaf5258b8b7d28dc9e09e1", + "082bc6e23d9f8848668b5e2bcf5d214b05d642fe9247181ac7efd87c1152c4dc", + "0844b2faa86b75616ad9e61099a8546dbe2ce19734eb93507a1c294eed6904a7", + "084e99784f5d3144472f2bb4c6ad302124d7f238dae7d6cb5c48e30e490e2b0c", + "085f9fcd72c44e1ec1f4ae40b1d183e6854e606726d8358ac37b9f360c0358c3", + "0884cb0cd7bfeda804f3351dfee94fedc6468594260bbe98c0e2414ec08781f2", + "088693fbb45b889e85ff958da47452272c49d1720c61f8a24e4ad652bc1fb716", + "089bd7ee69056cfb1908280fe2f2b1bac9fe708220e479097c6dd18fb259a24a", + "08bd69953da3b8045ffd02deb5f986a11f2d120614be22b53347d8514e5658fe", + "08ee326bc033bfde4df8541283f59dede6da6662c31cd3826c8938218f7a90a4", + "0905b1f6e979661d55c733e49ea636ad4829039c2f8327ff1f261409b6f9c8e9", + "090d54a2871e4dfca2aefefbc2e9663d0a7b710e7957d20c00c361dc43a1d2e1", + "09c524d9908967bff722776dacc5e2dc1841dd82e85f2eac79be2986be4727c7", + "09c52593507b34845144191b4a8edc5cd9b4808dcdc1593b396fe166b29ac84b", + "09da6fc458cc6cf1e4be0ceef803120255238b8962070c2cbffe715551717085", + "09e9d938e278c8cc5f8322c610fcea1283b7b77bafe842eb6449215d49394e02", + "0a9312ec31c472903cdb28026d86b90d551127135683773d122e17b6e0cfb857", + "0a9bc7df156ecfad97d28ee1227663785ce40cc737168364932f1a2e30cb90b1", + "0aa1a4571153dba5c1748f018cab757e1ece943ee4ea64a0f26bdae4f936d3e6", + "0aa4cb3d3b2e041fa4253c1cb23a914b8a5c644b56967d3741516fb17d3631f9", + "0ab83ad36b03e69d64d2ae4ef8c556097be124061af0360d32848dc41bdfb7ea", + "0abf91fd3685960f7b6e0812b4d8ce52b6a31224b4e739a506304cb46cd29fea", + "0ae69b83b02e7dbc4eb264be79de0da7185863d38bf366a9be1ae5d745b9a7fb", + "0af0a212eca005fb212d32eff4d5c2ce71ca2de8e9970ee416416ebe5d936f06", + "0af0b65f74f5a6edff26999f485e115fb04396168b31c5e77ef731b8b413e366", + "0b1ff010af5f1418d0f838f4f21a71c1025a797e50f0dfce6fdb578b4bdac377", + "0b41ad1882d3407ba74fd9527ddba492ebdb1356a231d41d3b97bea671cc902c", + "0b660aea784794a286ecd4ad09d7536cde2eb65a19280c8ba81d9112849a05af", + "0b9d988ac32a9d5138800792db6436da932ecaba29485cb95676d018b8c4cc66", + "0bb16a236a2dbbbc53df7f8f514f71076a1408dbe89e9b64f56ff6308149a5aa", + "0bbb7041a0e42c9d69fb059687967f4f5cf4dcec5a48a233d3a1c1b6ae117304", + "0bc9c1dc25638927adbf62edb1f44c41fc6468564c0c0bd07f151256e70edcdf", + "0be11c9bfdda8d9d43bf40a9567a2a7acc8b09343e6411ddb55a6b0097406f58", + "0be26fe12af26d5427c1b23a13e9a0ab66560bd0bab91a5023987f9a483a5abf", + "0bfafacef2b0488487659ba6e9f07942ddf7a9734f49746e14e68e8b903a5e57", + "0bfffb14c5e767e6b6a0d073a8c38b271abb71ec60be03aa3c6676de8620d1e1", + "0c0015ce0828d91018c21a25c3a988ae23e9a116b77ccfee298c2d770fcbc1e4", + "0c0136f75efec62c8234e61675eb5d7cc92a005887e1daf40961dda217effdf3", + "0c26b22b6f73a920acd2c3eb853dd13714c7d1a15cb7f01189612964543e2bc1", + "0c5ebfeb8ce3384fdf5c7466d9febb871c0a55b3dce9196adbf45beeb3c3a8dc", + "0c729263a7865648102e96eb98f764199f7e3f9be15d9852bb832d72aaed3cfb", + "0c8ad825fb1b90dad15342b1a38a73230557030fc6cb483d074998a391017af5", + "0c8edffaf99aaa4628cfc8af78d3b08d78e69e645baeb1ed83bfd23d9be43a5d", + "0caf685769ea343e0714af61762ed28dd8a3619c999cf2d503f590ec404494d5", + "0cb26bcd10a801c61924e70d10b27e1c10e58ec84a1f7fd1ea0fd4691d2d577f", + "0cb8a1e195b091af4c1ab6264bdadcf5dea204e12810c9f9674a810717bccb54", + "0cd782853dfd7e50a17e062e2466a0c2834bb6ae09ccca30173972814b444d92", + "0cdbd13e6ddc48ca28d0018a7b2a880b226489f679aee1a6ac782581c3894de8", + "0ce23e2fa8a749821680c196c1169b01e7e30af9b335e1cf59b8b62c58e04fa6", + "0d54c732a07f0fe8a6fc5458c2fec6b0e153a40bb33677bf852c815add211b7c", + "0d70dcc638707ec9f6834ffbcb4630e07abdccb780a1609d713a970f3f0ac1cb", + "0dc1128ba4c2992987f1080cfbf80add951244c5512109367ac2ff6dd8481f04", + "0dc6b7748f769c6f4fb5b5e760d69c37a5c91fa538ed1dd31a3d5ba4cbc5d607", + "0e292618af8adc8bb13df6fdc700564538050caaf00643122064b7a3376a1d0f", + "0e321b35d16bc03a5586ec91b97acf8c107f0b79c2825450f90f35f4fa3eaa0a", + "0e54f1bcdc7d87ecfc6e74d6de3e4c43d13474e1bec0b95138e7167710390990", + "0e846791b18e5014d92e8a138b09f6aa1edfa36549c1ef12a8ca4dc7e1a7e90f", + "0e95a43b5c9a6e453d36ad6f7cadaa8af2df3f11f0ca06fc4ebaa43fec26445e", + "0e9cfc6bb4e6178112967c91b2e35b0a3b38e389cc2aac0aab443dbd63f6b02f", + "0ed2c0dfa1163e363e7e1d89889e7f49e18528658208c0a068ca52e82d3159b5", + "0f1330ac1eb5b8485604be1a622a26c53c19cb305a61c61fc348020b7eeeec9b", + "0f207722dc163c7f09f47a4b114cde2e4d675c7399a3b2941f8c22954ad5df5d", + "0f298d9056aff724d852e7d30dbb93cff797cf382b8fefb1a23884389ba72ae2", + "0f4ded9465d1adb2948c4ebf574aeacac709555b4cba78e2f7ab0514f3d1dc60", + "0f6e062f0fbb5aad354b50ca9695824a01b4b1a2b6650fe534dcd517aae3a3d7", + "0f97e4c29d2ee4474f3e91f9728b59c2ff6ecc3ba0fe4709bea263555e4eecdd", + "0fc1fff21404a7c4c3bf43d0a946b349cddd81fd91394830c215687fd29635c4", + "0fc42e7c404cbbb29ebc6b271746a16e9226ff4270cd13ef194d66d9385f98e4", + "10bafd54576f39b903e32333fa9de033d34630504b3ccfe756933c7cd139fb22", + "10bc4cc7a43e48b9fd9fd2b4dae3488801ea0e043959d8838a5b6f2a485f0756", + "10cb9fea43a85b51aa089d8184d5e7ead3b732a471283229eb053f86a7b89c64", + "10d3721833fbb982ec10f2e41634a06e94f3e11be89b76c449dda794bf4afb03", + "11012a866289284e636f856a871945d4089e1218b25714ddb7e14e3cab1c2d05", + "113f5145a36a62fa140f4eda8be653dc498c33132e8440ba8bf91149e5bc1abb", + "1148a49a8e54f0192c8e4ffb72c997dd10918c28363d1dff62fb9176225d3e5e", + "115434f9bc4c1998ab24ed4071f8513198a71ea8711b65bd911d2283855eeef3", + "1169947cec8d4834b14ef01a1b21b27715efabed8c282612bad37c32b70a4cec", + "11757f25e1971b1949f99f53e1aec63fe120316d0d794ff04c5e3bf3e1e4469e", + "1177abdc80e9dd30dd06522f2a39b4b22ae70c1ca17a29414e0cd1c9015f2340", + "11bb2f5d4191b2c5785176893b48e085f0bb8944d6d257b64647ef8831eaf362", + "11cba72504d167af81d43fb4f4aa447fc4dc1cb8a873cd123fc80c00a4bf605a", + "12044591849b00fdcaee3693c012680c8dbe3ffe120fe6e771ec86954622880e", + "1227a4aa8cd9896b3e02709120920bc948bef60870f206080881533008607752", + "124103b3e01c126d1d7c42ab8999b7069a649f26afe76483bbe75e0516fc9092", + "1272ddae5bbaa8cee4a5917fa083c82b10d9280d0a6dc8f85968f6df3b822eb4", + "129467e910b6d1d31fdf7405d1aee0adacf2b45bcec803f8e1ae3926c3ecc432", + "12a1492fbc2c5986df9c059a33a7a0b9fc5445555bdf672b5fae0a0f34da8a86", + "12bfa8c65887fb9fb257aab37ef0aca1a2f50dda0836afe9a3845fa66295a1d7", + "12cfb3f21f142c20b8b75fe51a500470a46848c69674b4592ceaf5aad9d976e6", + "12d2cda4fddfc7694bbf706182fd2c9ab83964cef1cc1c7b2e3abe84dc6d910d", + "12e49f621d373e186d124b525688c4d1219e8a2b7dbed956835d6e64aefeb8e9", + "12e61e6a086c54726212dd8ae40498e7bf40a4e6fd9ab5b677c950d02eb7cd52", + "12ee15c35b3f1356c065b1161d2a63d7bf8bf083bf7485034dbdc0b40f23c525", + "1311e81322d103c2c43a03f83aca27a945245bf2a02de6eabc767baa61aa2442", + "1313d324d3e798a357662678e05bcb4c9a763b4c397bcf7816611f318b2ea4ae", + "135b237070f72bb9bc691fa076a6343c512900564e119ad27281243d2af0e42a", + "139e442c52e6ca3ad180fe14f1acb72016fc4c630405d7dcd2b8f848edd43a9e", + "13c92688765abb9394668fe90e9843b50fe58a60036450bfc308769592840eac", + "13c9558f5a69f6784d34d046906d2759d6c77af2104091f964afaf7bce5b13ba", + "13ccedcc5114c00535afd5d9e128b3f7011a2926486054ea868bd4a7b6e4b46f", + "13d043c134ac85a633c31260192367eabc7cf1767299124779b9f01a21ba59d5", + "13ee615d8033d8a0e64e589fad80b9efa6d9939c3a95a006300263b5fc7bf19a", + "14b218a5081e90244f5425b45e7d278a74ce673dd2e49343e0a82f4874fecd56", + "14c0c9817f4f22d5f2251efcc78b8ffd67926fba73fb45e17293b59cdb3dd535", + "1501199487ec32e664d5f69df0b7da5186e71defb0c64229e946c7c5f0a0cb2f", + "1507748f87930160bae3bacbc4b87aeafb13ff0584c0b3457b0b15e33af846a5", + "150ecef476986ef9da9b97b6f1ffa83c74fc8ea3d8d57213831b3566d328b61a", + "15128cce41835053eda7e79d8a527734f51a5f94f0e71bc4f7c30c82bbd7381d", + "1590a2e329844ef537803416fa313ec171f54da63051be0a66528df467f45636", + "15c10dd97eff18ff9c11efeaac85032c3db028f8a024a25b5732401570b5de2b", + "15ce55e979b5b57f83c2c79f86f4ac3685778340aeb72db247ade562c3dc9fa4", + "15d91dbb5bb6cbcd2971a2c9005a7d1591a02de0780530cdd09301babf1d4295", + "15db89c2bd9b6360edcfd4126de0d04ab42d10be7e7c9eaba23f1184493696b2", + "15eafc044f81072b73a7e4978efb8a5053684d910f074bb9de0900cee1bde5ec", + "164faca0fe5fbd3466a4751b72dd20c69f8349f0f6b35367b75dcd27d79fde6e", + "165aed8e4463db7858a18e57d902eb84591607731c33d0799fe2e1dd9943a16a", + "166ad8c3cb7f8dd902042e4720f289b39d64cc34e240f516c37a552b98ddc204", + "1680fbae27ac70b8a104e58a59fa2ba47adba09377dcac5e76e68153151a1a24", + "16d7a66ecb74238e0d120dac82df1f13ff15fcd44325e4de897be2612471637a", + "16daf6296ef9184c97ba0e67164997e16876f7a16f3feb490c704ec7bb942f47", + "17242b256da38439da805284116ccd22b6dc0b4f1162e966c58d2699b88ed142", + "1728fffcde610b0ea864ffe387cb258be47affaa5fa2444d7271475cced07a00", + "1730561bf6306e61c4609c774a86ca11b4e49a8d1b01c3425f94ee9ebf718042", + "17794ad07b32b794fba99b4aa198cab80c73a617c80388cb3938f1bc448c33b4", + "179a8d46e8da7103776cc9d41954954f9a30cd4455377cc6af1bf76a0704580c", + "17c358ed8931a3e41bc39832f8024ab4cf2ecebd3070060890e8381a61c7b344", + "17c67143df844a780a50ec041cfb21740ed75142c30dd9a778cf56d46c5e2f5e", + "17e01942d367267f255ecec7a0d0756a47a18c02e6a868ccf4709c4bb7e5d75a", + "17f64daf3c523045868995b9b5893aa69c6406419d51d0d2d5d6dccc95768fb1", + "1828cbf1e501bd910192ca5f0118ad162ca6e99b3d11a588966650e20e5194f0", + "182b16a6d5297ed49a61456d81e924731b627fa701a33bebf113b5d1dbac34a0", + "182f7365b61944dec5bb0919a2d802ef89258842fe129664444ed82a58cfcca8", + "18657e4bb7bcc144077e48065c56afbfaffd0001a13ffdeaf8f6bb2326d8c483", + "18d4f30d97e86421c7a2f9bd716a3944dc4745651cfa54bbbca295c0ed56a933", + "19b08945d0d2787fc572f103b8f4bec38912b933f1489b4c73c41ea3b26e141d", + "19de7f92ee293142b40d8214a494dfd201964ef788d8663ece4c2ab5d3c8612c", + "19e0efc3b92b86a743d8a25d8228e7a35b8767c51244a2940bf325a900680e95", + "19fd5cb3bbb4d83eb86d436bf9786e970c339b1d854e91ea35e9a144e4155203", + "1a12204deff18038d5b27ab69ab157c6feaedb6adfe5560c46e31174eeff0f13", + "1a169940b1033a1daefd265afd89c6491a7a05d9c479667ea58efe6168771027", + "1a644d661b36094b489ea6e2ed67bc9fed3a3f5adbc01583591fef5eb9704bb1", + "1b261a0ce7ec407cba1565a8e4c24f3219233746509f716cab8c4747ee3537c3", + "1b2f9895a0fffaf04836d9581bb8bb6c2529fedf2283da978f875c2860d61f2b", + "1b42c5fb7e97213a533064bfe6bdad601e007166165a8f29f63e7e7999362c42", + "1b47810cd051a297cd61db10f6fb32c434e35242afbb75ebb728a1a9fca6c7de", + "1b644336b4b6654215dbcd04f4fff2f80f003d27531a1a7d5ae344a96223ec0c", + "1b779aa158ed431977a27764618e2a4e373f3ce3c5e20f75c8938781881d0e68", + "1b7dcea7adab5f8b915a848e6415ef988dcb9c01cf49cf61c573955f0845a4a4", + "1b8a6a6fa1b179d5d5437229af8793f416cd6ed3492b1fa08b9883757ba91bde", + "1b945f3f2ba81f701ae6363cb35f8805bef6fd083cb7f96fc28b60bd59b2a42e", + "1bf373e4138ede234792ca2a31f7cdf7d8000b81213d183c592c57810e5f860a", + "1c7c2472654cefecab0ce73cfd71bdce6ba3723a2d1b0e60db8c98aade0214c0", + "1c8a863bb173fb7383178f4791d07dab6d7e61deafe0533713f1dbe165714347", + "1ca29fa47c93e63575ec16caceca3cccb1433ddf46316e5a0e516a370aba3c98", + "1cb925d6fff01c6d651d84259d1a8ac66ef3e62bbe6edbb2dfbf8a32ba6e254b", + "1ce64787a72f626f6a88392499a9255a86c5383a749e004594bb8c1bd28ab033", + "1d31a051b2ec1fc2654e62a20c125ab46a8257c375a84eed7028b7e4e52d89c0", + "1d520b036adf88c23e2c7b29f0332b8b2e0145a89df01b7704c4dad4a4b7a26e", + "1d5844a8d16fd769c49e5ec5a2dbda1f912441f777969d005b10b7a54ec6f222", + "1d61e39cdd0ec95fd2b45881a6ae2dcabf25bc7b199d93850c9cea4689561fe2", + "1d71ce873b57f889c07cbca6e061d4c9a7e9b09f7a5080257809e5ccce1ef3f1", + "1d7833e75164b04c32bacecf88e1adbce107ef962491f630fa7e7995e7dfd250", + "1d7cc58ad59e6b600b4bd9ad4df955100e8d941433168c6cc049e26e843bde9f", + "1dbf59ec48199ba97c18205303d128b76b9c663595181fd12274398a2ca084e3", + "1dc1fd9b8eaa6883f2faca7699c9f508d1f6a8a77686902ac177131cf2899099", + "1e069441905d0a241eb511339e9dbb1fa46aeddbb8297488c2f610ee338cce43", + "1e43c66eca65528152bfe4159ecdb59354ee36c34b54eb62e985b212b25acd13", + "1e50848ce7dc42b4b42718f33d0c415ab9a46775c69ca5a4b82276908bb943eb", + "1e76599c93125e60681aa1bd6ae72f50d0a47a9c49cde9c6756909e98464d2ce", + "1e79355b42972c791c37b68852c99930c49365068ee1a5b7da7692d2aad48a32", + "1ecd0b4f77b2aa35440911addeb9c567c92d0871fc1fa9100e21b2be14139d18", + "1ecd4ac9f9621863890d5f786017185d6fef36d9a90002909562bd6a95eca2aa", + "1f4c8d73bf3a9ac57f9e2b43d427baf173b127d556762c817091febec2e02b83", + "1fa770f0d73df883caa5caf32e572447960a6f4a39cb919b7e5ef0ba5d8d4264", + "1fa94044a3bd20aff19e7c24bf541dfb9f85059bda026eee808d37fcf149bf33", + "1fd4988ce77729f824275c64a1736a759fd53bf3a9802c5fb4320f94889edac9", + "1fff8991748bca60d71af9cc9b2b3827ab3180ae705ebd62243c072721f8f094", + "203223582c0e7fca69776a87bd032ce7acdc2a2294698ff3a98d3b434f009347", + "20c786ab915d0509301693df90eae1ebf07cd0291a9238b3c107325d77b631f6", + "2106a3207e94eb944903c4fba0c66e02ab7f70389b6fd2655e6a934365c53a28", + "211894e79d6b7bc3345ad4e7706cd6c2e18970084a6b2e493b65bc8d59a5c727", + "2158667fab766e543795e7d22e30a62e6b02552493d9ae5d40764e724deb48ac", + "215af1c5f11ad99c5e43328eb8f930dd88cef637a51840abef90e51f91cbc371", + "215f4b47964817c247e28ca440642fd20f53f922013de4b009d90cde4c9d82cd", + "218195c1668cc7deb5f978b763d886317fec3a8ba0cc600b515d689330f959bc", + "21852b6057ee5fa7766c8f5498de18cb6a9b2ff18129ef78c35ccb43cb3a41d5", + "2193db36c24d0b68f9a562323f3455e60bac32e17e92c25108afeba03c0b8d83", + "21ae1c762c3fff7bc2ff0b1334fa52ec4290d14699b4ad762802e086427e6ea9", + "21bb9b4fcfff8d963d835e84fd8b0a3f0e3bb00da0c31b3ab99f7eda3b9d1879", + "21df15768401d6a240f92e69c17482210d8decca91922111b82544359047545e", + "2208bc75cce9607aa11acf9f64f8a37a4d2f1f8f9d3dcd59328f72e9f2f573c0", + "22243656978707a34f369f92c925fbe6e807c42c9200857afa1712beabec982c", + "223348e6289ba2185e679f0cda88bb6a8d7a8f5f81325fb7199f27c88ddd5192", + "2244cf8a582b801857e5074afda78acceda9fa1730380526087424fef042006a", + "2266cacc0b7ff3f0808e75c671cbfcaae5ac8f347d0c0c809779a3d3dbfa5fe0", + "226c86fbe1e36d2904f345f76cfaba39a6d71be0bd735335cb0c503e3b5ca3e5", + "227faa5ea2706a8fb5dc63022bd1338300485f9df8f7fdbf1733b170fc67397a", + "228107ee33b2c2e9e1e2aa7156e4cad7d4313d56cbcc7245c6b6f0b5ab6ee0df", + "229f12e885bb5e4ac5d830c3548c117169f2ddf5f11b730ccd852fb076c32ad6", + "22a71995d6c30b1889e0ca873a9de38aa25197eaba39464831d78c8132c2aebd", + "22c123126020665ad6ff2e62a8ac0395674089bcfd7a2a5e95879a0f3f42ba66", + "22f8bee7eb3aa2f15cc3a50fde768180b27cff6da04fd3ac602d0588349fccb9", + "2354b73626bcb881817c40eb612de6c984ceea635873d56767fd7d27cf72f337", + "2366155c840f1847f61784e44b578172919a4fb17e789c5e9993adff27f20ac1", + "2393fc9fe882416a98301134c8f1bc40959cbb0df4a788c18d6ba97227dab57f", + "239edc4db69e5c6c2ec386800ed2a16c1d14f40c5296dc5b474fe02f303e2932", + "23a19cdcdb11c480c19b5e826ffa9ae9a41f75356503c4c189b952e7e7229c10", + "23e7778305fd2fcf3d8f178ff8b0542c668ef7e7e7ef8c1a1719742ed4205899", + "23f90b26c36b6a00f084465c27149865452885e7b6741d29261be0963c5addc7", + "2405dad17d1d80f91da8f5654687782f23179182fa8ea95d0e7942f024d88ff2", + "240e6c1a1da07b1182d1f9c34e2585ca561d75e0c7d9c467e9f668638e3d4769", + "243ba124b326bfb985fc9db69b613aa893389059ec2b6d5083705fbf4996f0aa", + "244a86be38dfd9d932f23b9d27a9416b40b633a754cc528b81f7987672c3ccb0", + "2470bfaedd598e75adcd25e04758049ee6770a8f94a68950d5f90509fadc948f", + "2483f6c4d3c44d993895b6443f371d662e7e0d212307b47f3ab729db8d59a20a", + "248838a33696f03df582c2fdf2f6079a214d6ff05333caa5f95f138dc9af2b42", + "24ae887a86eb0636d75da5749f2d58725e00b258a152666689851b7ada661760", + "24aff802e6017c90e6244f7ded77aed1fcbb545953cb200d04972648eced96ea", + "24bde8b59e05b32bb2b477d9106f41f2782c9842a470e70bf08bb527d24132e3", + "25029d6aa49a43a7e8643cdbe81f4689c354b4913f90703da4256b6d93cbe73d", + "2584bdcca36c2871dd912988d49fe1f921aa8ad95ef0c60409c774bec286b379", + "259297aebc704d7d5569506071b5d478f9e3b879717b63b09ab00e4608e514fa", + "25938b7d578e370d915f65c0643c37264dc75c7c2c5a4e86211fc222ee78798f", + "25aaa260d7f61482d0dece1a6dc0ebddda72f625bd38bba89694a83b834d7a71", + "25d303833c68d194606fe985b8b39f09cdc60f97efec4e116be8787bb23f98fb", + "2605fcc76d52f90b98dfa94952e2a390477323091d8d0651c1227a952c2c191f", + "26273a075d3f2fa5551130d04d65fa7cdbec6f2c4770ba144eced4ae719cbffb", + "267564c04ba8e005d6b23639c895d35cb0591e0291a9e263cf5cbde05cfef1bc", + "267b0a39e499f0278a58eca2a6205c39a1503c08694d52754c5d8a487ce64c1d", + "26e4de7e4ae0c7104c8c6aedae934d81d97dfc89169ebf415a3b0219eb8fa9cf", + "26e71a515501d52e4c0fae7edfadaab7638fd7152ac373d88be078abeb2d001e", + "26fa8b7a8f9dbb96c9ea366cbbece69d993514b5ad6be8d8522c7f65a4fc2b4c", + "26fe2653ac2789f9ff3ce5dcf164b2c7f228b648e61805e3855ae9c8bf59526f", + "27427c19368afa942253ff0a473755aacfe1f98982af2f540963275b55a49d1d", + "2771ffdf18dfdfd65d02bf1b8bb36aa56570b790a173759cba7442958b7c8e59", + "27794ef63ff9fb7240d2bda2254462fb9094bdd72226a306977f6b7d82821b01", + "27c5d39a379b325588db6be3aae94fb88d53d4cb5159be340b12466159c224ad", + "27cebfb6abc8671e279fff311fab47dda886f886c26118a8a17132cc55ffd2d0", + "280c1a759d77de7a2eb914d964e2bbd88d32640e4fc9885ea5ad6e97b7926ecb", + "283124debec6334f87ecad97e22eb29412f665347953741d545a1362d50cf700", + "284292effe9290e071b49536e7676e25b1f2b51403501d067e35c4cfd930cb7d", + "285dc9617d8a64ad63c678ff0ff60dfd6c2c395a2084e2dd1b423511b99f452f", + "289e975996c5a4d8054e12f8cd26151ffa576b2fdcbc04ce39d0b8e3853ae395", + "28ce9db3376e8b1aadaf10eacf458f510bc957fcdcfedfd029bad2cb1b07bc73", + "28dca848a9bb9643b1b2d9a4a8f2f7a044c5a1994acb722baf9f46915260dcbd", + "28dd980e1372e6396aed2934d45aeeef28430910a7ed953e7c0152bf548e9c91", + "2921ea55f352623ff28dd31ac8b6126a8df490e5433d55abfbc4c3bc45d04047", + "2924d65dfedd8cff2e2d9db5f0f5f65c545597e7aef61c5eee9215374b1c97cd", + "292f2b9e562927a46924f407c14ef193343db63063425a32603740d594b02d7e", + "293f27e5ab477371a20a6fe6c65b647c259a6a98c3b813fd186b7a356bb63c54", + "29462899b56ca8dc69f31a04da863dfb0aa826f16e099bfc2d5978c938316a6b", + "2969c04005b15ee1f9eda734cb9eeaa861702af06deef784dcb9b8fd06f97282", + "2974f03b745944abd7ccd676a1f3750e8f16ae68c9c074bdc65e10b5623990f7", + "29a08c002de030c471f68e1e749966abc348ebb2ceb422612ad8c9ff3fdff258", + "29b3e33c27623bb4d5e4c858238a1d9fd530fd43dd87173f95456244872a5ec2", + "29b6050cc305b3bb5e1a40aba11155d89c2967f36eef823d4bf93474dc862356", + "29c8e5d1c664155ba776984de03294cdd6d46e2a2481599278cb12b9c042b93f", + "2a08bcf84f783530d696add19ea25a3fcc1226584bd2dc1ea0bd38a805fed8be", + "2a550c026e9ff65d75443586c017ace88635598a8faf177cb70fdec7b4761cb1", + "2a621087236e8404b3c0e45a264692011e1e30e8be64b0555a45885758aab52f", + "2a74d3e3f8eaea1675636298b1992c4b487c22866d70d483ea9d44873dc8f085", + "2a7965e1a56d537f2a2fc4bc7c9a7c4ef69a5cad08ad8411b497598b73fd9715", + "2a84af45206416f82ec91b76b8453b31a3ad892082d4ebaf0ba02647725db5d3", + "2ab5a2e4a8ad61ab47ed5ff6fa9b024652368e9fc35e4dfce3a004ba7169f077", + "2b2568383c501e9b66e98687135285a10376864a191834e8192a47db47ada0d0", + "2b2a0e3b2e9a54e95583639e8663921d0c0dc95ad6e1eab5776c4c85f3d3c519", + "2b3ebd6abe46a3e2d257c221c2521e4f2fb92b20c830f3370929106fdf462276", + "2b4a13516d94a909b64523740d2a7b42d960130275ed885b8476e5fe1a053ea8", + "2b519f4c71b588b9255394601a55032d4137587d9b4bc84ad87311564d4573e6", + "2b5fe3cbaa0557413f47e9904a66c0bb44c8e85dfb076eb68dcdc7195c587738", + "2b694c3919c53658c7115697287ec18707f91c178535e1b93a839ea7236674f6", + "2b6fc7dd39006b19ab1ad37d025d46563503e4d3b475dfc3944e4f30cc0edb9c", + "2b72a5c938c4ead2c17c0a0fa02caf08ffba9cd0c24c9f3dcc5af30e259af482", + "2b72e11390e9cb9502e0c03612bf3462bf0762abbc3c5ae2adfc7cee8e60ce57", + "2b738c4c7c9fb5a740202f324f24154482ea4a7b6f828354fa6daf638cd02e60", + "2b7a77bebff0ce005faa414e6e225e537967e9aca9e489090bd0c7c109c0c497", + "2bd3904c8bb216a19bed7b0a6f91e6fd12654312bad26d9b7e117d21a837f039", + "2bfdfb35553a281bbb7b5dbffa56a58f3d0c6c59d30d0597d4504feba177384e", + "2c69a32e259e132f0c383c6cc4674ab062850bbe9240405338f47b5392091a66", + "2cbb005c3d38bcd89685e0838a6145254e733833d7df151efeb9fce97f445636", + "2cbf2e5dbd2bff53014a66a4cbd93f69f92f886c5b4ee02353e4ba9c94e3826f", + "2cf03c960330a31b32f11d7bb56831aaf8e883f1e62b05cb3f4453fd80e40fbf", + "2d1adbbeeaa8811ef41d9bbd778507b51163ef3021fa808f767943b465b8ce16", + "2d2323f574328014c75e32fea3155f729fde3861ecf369ab6f337d310c90c1b1", + "2d2d9b13157d8f72554f0f880795745f9b6ff6461dd073d584e756caddc4cdf2", + "2d3a44eee75068d618b9bdd36b116e8e40ccbad1c16156b8bd0b06369ae81ee9", + "2da33335bb58839ce28b97fdecbf093fa0213f9d9d79801e050957efd4d4b0ce", + "2db74393909f5acfc2d5e00ffa646cc924b6ea3dabfabba9329b260a560dc354", + "2dc05339260685d20603763b5ca9b7ce8dfbe566b95e3c0979f5898dc39922b4", + "2dc306af4f727cee39b8ec93cdeede9dd9b08460753a2365f828c08333c8639b", + "2dd3adb43e3b61853b8a30448e12a24e24618eb60be5ac6a6e6310cd608eec63", + "2dddc0fa022e4dcd9424b09b16f89f0f61255a55b0748ab2a22890e87878a0bd", + "2defa16f507f490fb59bb70e071d85414ef07276dfa21b5317f8d06c4e3b1a73", + "2df21916ed241f489d69e97d370e4d6d6941211f36a925a3b86d7ed1c0fc1cec", + "2e053f8b5edaeb383ad6e73c2388a18a8f9a6d6869b237fac4325d5a2245a564", + "2e07bf8f9debd974bcd7d7f5721df3d08df6e4a9a3c891ed518cf783df0c9e3a", + "2e2ddfb1b4872831c5c5ee20abea9046923a7fddcc7462c975f79d71af030ad5", + "2e3166ed803467db2f1e7955b14d163daa016118a0240832622a3a89e31cc290", + "2e34ef045128d6d9abf7e7bbb3b7d655a355c932450e8c216830216a55f45bf6", + "2e4abffd42d065405d5c50a7e89b60b4635129858fd422bd8a5bc56debfa3d1f", + "2e7dc207e0ec7256ce878bee56d6eb7e89851dfb15739e987b5154450714819d", + "2e8b45b76496d343b60a3084f4c983916d31c5b7eed40d45909452d83f749012", + "2e992008ae3b92ac0e6c56a979e6e97376f6ab01ba1657b26eddc2f1b774eb4a", + "2e9e3a0dc28d9cc455e426bedca616fdfab0fcb4b07db71afc3a3cc55de065b7", + "2eb409f84a30521f4d270324a38a35757203f0aa3cbe84a002bb76d075c4ee95", + "2ecc79683b3b1ee11cf498155da25ab786ee469d99f7bc0348d55409a9e59f67", + "2f04d8c5bb5359ce1f9de574cc90c3d40d4e3623368871c574c3dc5419ce88d2", + "2f0a059d5a5edfb859aec1f0e51a86a767e10f8debc5af8b1d705bac6c820659", + "2f20d184ed355324c87d927888e73b46809427944c6c76926b221a1ff10fc4aa", + "2f74cccff6a3b3582d3d5c8b80bb2a826f8e8338d1baea38010dcd660f90b724", + "2fd6faf46ffe17059af06ec0c9ba806ced076ca905ac45a24879deed1c45afd2", + "301f4176d152f5ca05d3a3b212924e746ad4e088b3b8f3cb9b468e8c75c4eadf", + "304b4b93752f54c39c79148e2f4fe81ea04595e2ca127ee9162dba7a49c5be9b", + "306722a747e9b40790bb9dfc3277629b283df3204757190dfa3c9a77bb82b6c3", + "3067e3bef680d92ca05a1b10a1efaf2313dc1a700c292d07ac763b0717807e72", + "306ae438448f671a8b163290a896bea20caba868a7989a4115c607936c5a23db", + "3085ad2c20aa1751700c2a52e56a31582262e6d33490a2d863b49ad42eedf490", + "308d8a8b4f266866938940a106851d72fe3e6c09d9a246b1d339a7eaed04a3ad", + "308fd860e7b20a5003f598b1560f3d3d9ca6c1ed6ca172b94d89d8c0a2a81f2f", + "309094bdd0171c5d9013e448ee826e8683edc9caa3b05cdb830474e451b2967b", + "309ea352f1fa69f0db7582c33dae60a6e94af9b27e5b520c5d904f944fa68126", + "30a504f78e25c60f73e949b01851fa600f1748f14eb937da25d60e34f8914834", + "30ccbbd11c3360af33dc437ae8648c6f844ac6c6ad3d9a4bc066f5be76a23793", + "30e05421984dc87342b7a0c5e0f5ada02174ed62e5ddd94515fdcb0562a23a7c", + "30f332c1a28575705598386d0a16c103751abdfede5f2248d79bf7fc1b502292", + "30fe071a03f9190a7b9bf43e6848989931924b3ed57feac707a887d398225ace", + "31321c89060394789fa478b8e5e55158d305a2ecdc1d3e1736ad1fc2da960561", + "3155118baf245eb3638c6b00a0dac1335b2908925cc9ea929080acc31b1d4692", + "3156405fdd7823ec3d73009465ded9eacca77ce01ad522b2686e3f0a53afd15d", + "31904cf5060e463ae0df2858d4ffe8c5e6f88f1b4fc97583a9942f4e39461bc7", + "31c34949f3dd67ded3e6234af496cdc55c1601ffb2e2f9566e0901cbc42104a4", + "31f19b0f89f217f06f53495c011828494e379517edb22220d1e08076081d2fd8", + "31fa2d2738687e84aeb791d1da281c35e009d609352b79e938b035c718a47530", + "32188bec2e0424b77ea0f3633c8be658cc35a739de5842dced47bb4cf29b2e85", + "3226aef36f5280e1a5f7aa6921eed13bc6f959316ac2e0b64006c092639f7580", + "3282223e993dd22d7998813c953ad2266e639d3d9f861168eb41a0e6accb55c8", + "32bd77fac7231b667834724f6fc99fb379885d7c0b217fbd5f741a8e40904b96", + "32d78508833e9de23b051b65fac7099f8a8582b13576e65ddd950cdba06bb3db", + "32eacb2a02b9c5c57ac5384a2c6670c7c1f2164a9895398fcc42d78f3ef0cacf", + "331915366ec39333ff7c232f1838bf30be30e90aebc2e357e9d36d7024e1131b", + "33452c95909e9c0cdfa89c3cc04a7b27102859779a85dd966559b993eb94a7b1", + "3351793a8c5db83f2f1877a1cc8bb57f7d76f3e614c55dbd58e8489491e24af5", + "337ea7d073d3cd9be0463afe0eb37c755c15240f54d10ed971cdfd4f09618d8b", + "337f9f4695b3e8a6d6ef006ebf5b1cde828e91ea4237ecc5992d6291292ee7cf", + "33ac16e402f3f2824708a8187c19404f952acb49f1ee8b4bbe5312ad585fb131", + "33ad56c3d64bd864c3a927b12fb95ac99c6dee0e6abc244fbf4b61149b16be74", + "33b0a6867562a918e8702903b476fa7100a1b356b6613d2e9c25035e31d0791c", + "33bd5cc6473cc48632de8dd7f0ae4c6d99d772cd07e29e5bf3b37f4f9e142d66", + "33c5f3455ea8abbe73312f89915385361981af36a329a1a357c3d7274c2a1d2f", + "33c7e4b1a1be597198141e9f4506cb9c9e0955b01e6fa2103b428cd8d187f5b4", + "33e3575cb0a63713d10b724814597d6160c2549a3ac1947125c10dfc02d66286", + "33f483a510152f19c6c4220525d2227535ac5ceb9fca7483f95ec74122dc0b73", + "33fc9890af9071f76d49957ca4596b2255dac4564c3238d02f369030ee0acde1", + "34169e0b266acc6516cf0abecb99fbf15d96010e9937ccd43726762a9716ac8b", + "3421114015b80b306d6ff05cda48ef6742e935d3e9f1fbd4a79c6eb378665f10", + "3432acb18a5419f76a291dc59126ca2a71d0ded8c0c5e6da08d58ac040d5a264", + "347682ce1cb3ed264f55a6f66f7908b2a5b4aced184463812ad49a7a785bdde7", + "34b271c48e0c0b2b2b29ac13d525b4a4fba844419fb6b88cd0c1b78cfca8c3a4", + "34ccab9180bf8852ae598aae302da522f7e2366d54fde419f9e990b4abcf1722", + "34ccb47ee7a64b14318d84d641a7046684f71ec4d01aadc54c3203a1085ac817", + "34f3ead3029e94a2a25c940365600dc03afde42ba6da925938431506431d3585", + "3543fb202207aa6628c47078d7f21f7590defca0b9eb7af4ba12ed5aa445c4e5", + "355cff0ae44e80d74b235b926e1e414860320ad4a101ec70fa99f9cc2481b626", + "35905154d89fd0df546a823190c2754e6d51f79cd30024c9c1acb9f2c7eda1a4", + "35a7d85bb77a51ffe29fa15e956a970b54810c1f618adbd8547a4efba9df46e5", + "35da11f528bc82d5f9ebf9f5bb9bc25c766cff532216af0646b1d8f15afd2baf", + "3619b67cc0937e9f89964936c4e2ef1897f48ad30b83e4224812c1222882511d", + "36253466aa8a53fc51bb8fc037d21924520d59aa36259be54b53f8d8c67015e5", + "36714da17a776533a53fdb0fac7fc0b051a1bd0bf85320ee3bbb09ff68da5b9a", + "36bfdfc702c50ddc4995779357499347962e08280e108d1f67fc6377ee56e225", + "36c654f26c8c4b5e4db51a5c1f2213b8206a5e4c88d84d5dcec4a7268b42265f", + "36d2173dcb129d98a1fbb9a25ecff8613706c046e4ccf8b49c07d0f8043f6d9a", + "374d2651979eeadfad2699f5a557d7db7174073b84eec55c9704fc171d768e12", + "376cff84d5eaabcd84de92e8037c98ff298ac1d372866331c49efed4d8ad0e15", + "37b247ed57b009b125d27738b5bec0372247fb76cb1cf225c16dff94019aa8fb", + "37c6a25a9fefc754d6d25c419a6485068c7a1dc1af5d07280145cbf7e849fa8b", + "380d3341ffe5b7eb9332109706e251b68211796ad2714d403a50e8809939f4b0", + "382e7932bbe71318dee082bd946788f610fa948c114d3a3bdd763553fb10aa59", + "3837c28ddc9543e2beff4e98d311d4a7ad2e675019dcfa0f38bc54dbdd00e1fe", + "384b8311b0fd3f43ae9e3c803c42ce0b5ac4f23f5dc1c1785def1cd01f34e99f", + "38680b605ac4b6e46bda8a39889e50dd8df12cc82c303e31928a16cf3e310cc5", + "387200a627634514bb8321770d460b668c9516375226ff1b129c54e00e743267", + "389b17e5ec1ac64f3b51977b3ebf9279409572fbbd1cc5779e5986f10ea0eaf6", + "389c9edea933534cebf8dccb4b7e03b96ebac870ab1c83bac3193688e5596152", + "38adf5a5620c39b5b3cd62b452de47b44487adbfcd855ff75b37eaa331277cb0", + "38be75538254cfe4d5e535b5bfeb08d320e36d2fc09e9d3fe0dbc369a03c6f2e", + "38c5e9c98332a91abd7c6654227b1eabf1717bc4c48eae5ba54d4250a1b8faae", + "38d86b76bfad976d136d0a0a6ebf1220b57bb58fc6d0c6bb1161474740643564", + "38f011dfb9503dbb7d7c5547e05c955e684a5de2a8a2431a40cde7c23e1b30a1", + "38f2f6a1c0398f9a45a2c8c4efe37de7ac56d3031102eede259aba10d6ae3fc1", + "392d57bd5fbc504434787a303f8e1c56064fd33fbb145766cf7e7f1a0088ba1b", + "393003d60574390abc414e5bad9f145b5a4382b3a34606d19e0f2612fce42834", + "396066036fb59e21a23d3dcd772be7f1834dc79369b98db5aab76fac639557ce", + "39907f7d1ac19d644f06417cd795e3ebcccc940383f1bd078e1b5c839f79057b", + "39943f932a5282c29a1719b4bd14cb195599345a3df6889c657702b127783e38", + "399da7528776a819912236825acf69f6cdf7b3ab904d49f16e970f0aed4053db", + "39aaf3de02157ee50731d29a7af009637f923a488b32a28213893cde9d796982", + "39cb6cbd40abd3a3b830ff12a87b423447419a54aea2df12ae424db1928a3aea", + "39d727d8064649b6d18281804dcfcc9d26bd27ff747b7a74252c7bfa62c6c01f", + "39dc7dc27988f1ec49f46b721feaabc2b63126a44c585f447c54abf33be9616d", + "39e5e558b78627797fa5fc55aab54c3353435867baecad912dc8c54062da8458", + "3a5343a13cc9fc4fe5e37203776a45663a5e988c50955ba5ee1ef78bc4f77623", + "3a573bba76d4f43d6e64238fa21e38aac63f8a2c511db774ae118d40c509ca29", + "3a8e9ea5bc01a4d394a2b7a91bf89b579395bd802683f24914a6c822969a7417", + "3a9ad0d2857e8b0dfac3bd237dfceb08c1db01b79a0169a868b8d588a589d35b", + "3ae9e28b8f68bb9676cac4f93d99c9704ef82ed42794bb2a9aca59f52833ede3", + "3ae9e7b2eb9044e1be1281a28177b8dc4ea1b4eca35f88a11157bc8f68e257f5", + "3aead832e9e872bc3a1d9324fd90910c8a904255c92159b2cb3202241d1c50b5", + "3b12a56b380ae79ca6c9a061b7e7902544a9cf51eea1b627206e44334fb3fc10", + "3b16dd05c50b6534be4523f8f34e0778455ad90ace4091604f18356a14b8edac", + "3b1bca7064e1eb773d2a2a55e2151392dbe3a9a44ae12a14017d258edb8842ed", + "3b29e3d5850f9099ae05d5171456f0a735ea8f07c137492557a93f425ef7e7c9", + "3b3b58ffa394f68cd450f57e00369649704aac9e4e954fe2d0c89f7ae75b634b", + "3bb376999dc724ca9695e469e5db3da657c83841343a498132d5849bf8e1db46", + "3bd236b7cd8f3bf3461f6e6eedfeb97ef0b0fcadfbc4a42164eea6546bfe0c72", + "3bea6795b923ffc0354dd3e3c427b242af16597e2c327e0bfd164026f1cc8c8a", + "3bec1a06ba82fe284d099ae5f195379ed8b870dc62a4ffbed1a67437805e3eaf", + "3c0d4da7c7013e037a799b975326ceb560a905a9381dce02c6540d46404e6dcc", + "3c3768c794292f1cb9a8e23c221abd3cfb5483be5893be5302dda43c86dd70f7", + "3c529806ad8b1c37819898031c2e8ac6a27d47b259acb7cf1691f51689c86651", + "3c611a1bddfd25ae3b7170d90da70818ca5e5327cefe4454b6628da9a08a6db4", + "3c7cc070b7fe2fada929c8cf26fb697e3c58062a6fe3ca348e1a1eda76b37625", + "3cb6873e95612d0fb931a822a8b55afed47585c707ae3f6136a682b244ea0f9a", + "3ccefeaa917898a3cd386dfe4d65199796e1cfc8bb4a73985af3f1e4ce3a26a1", + "3cedc03408a8b3da093700161a478a062762373e3a952e36a2e11992c6e089e0", + "3d2ff9c749ff902f5bfba3fd3f7a269b5aa37487001a7318e50f09fc043bb09f", + "3d3102629f40f0f6fc575a5a87ae7d73b895a8960e8746b056fba7f368a2595b", + "3d32a74525e557244497297607dd36cbbf4777e0778888152e1603d600b26564", + "3d60580449a07508fa4162d6c188e21c4dcea4014ae77b7b4908ff786d9456ab", + "3d74f4db788aa8bfd1a108c3cf1bed0958332e64df508f002310fdd13fb57e1b", + "3d74fce3a56854f176b3c1660203d4424bd3557c09ca2c224a3403946da2cc94", + "3dc736a1700bbadd6eb95f52a70b1d397f49aca10d5c94a3fe6f9ce946fedad3", + "3de589be6e9c38b0b10243bd7c2bc3055d643f29c3944339db80ab405b3a1c32", + "3de99f1d97b00c3baa52658450c06e846b07ea78cbe8a3338b7be49b03fbc357", + "3dfd24ce8083825ac0413bd174d48661f4e7a666faa6d6f54592d49fc244ce81", + "3e18849ab3807d887c0f4fa62a2bbf6e76a41dea1ac7950ab5a1ba4cf96af931", + "3e77980932066d7cbe8263a65005bcc889b865516454120c1f48f94c2fef9dbc", + "3e960f77007ccc87387c15f7aafe210ae38438f8f6c8ff64551033770046c8e1", + "3eda1adf6b8f49d33323ae8e6bc96a1cca229571f5927a804a6b878fef00e3c1", + "3f0c251e7b67fb6858967a27754f29ad110c89c0b158c4f2e5a96c05fd81b888", + "3f420e76b844336a91049960db17157980dfa4d5916d0b814d78cddbf57c4f86", + "3f5d876571875f965c438a8a58c96b7f32234911546f2c1590b83a5198acaa3d", + "3f6e99c36c36dd12dacfbe23c7ac87082c419842dc12bec4f74829f764a8e6f9", + "3f8f5d84da330a6c7016ad72d234c174aaf723b45cbe5d3a9145e675c2575568", + "3faf0136d7affd01be02bf5a9698317ee8a6ff6f6a0ba75779670d18c66a38b4", + "3fd34303b59e12e06bd772fc1b9792580da8a50e4d69dbf78702fe7bec9b8c0b", + "3fe4f6a0da2df15f272d64d4f9ea68425ad01fa3f0f6ee452ab9cffc98c34b18", + "400a9724d93c3ba1b67e2e8cec7225516e819676e4733d50ed33edafcab0a707", + "4012b33f2ecf8d1baf925426438baf8632a4fdb1f52dd767a572496a457e5ee3", + "4018bc333ed65ada31d42f42e5e51b29c6e927157e58cd60f037c1f26c49599b", + "404ef49e9e6d25710b2c12929318e60fd4601328688785d65844608044de48bf", + "40c9410e7b8ade1cb3a6fa71487e22f7892b5f85da1fbd14f433fabbc48cb583", + "40ebe8d29f30b9acd3993e9d72f36d2511b3a81c8d50a25b6304d3ebc4504eb9", + "40f4b6329e19c0ad22fb492fc79edcd242a2183f9fa3aff583dfcff16beaeceb", + "4119243479f2a04dfa7843c8a9357b64a7bbc4bb9378d364ba66938fa29993b3", + "41582150bb4e6f0252ef9eb9b8d9ccdc6d9111291411798bdca84198a1d29e79", + "41602d2b2ae7b9d18ce632b67e369cba36d272d2b0f412cd30a481cfa0cd49d1", + "417be49aa3741eb0120073ae71337f11595f05d93525afe5dcdcec42bcf24df5", + "41beec4ea2210c90956299942b14f90f8c6c450035b05f56a7b5a8fdd54db85a", + "41dfdd7f8a8debeeb472a7afc5a6a55d1139a39985f07eab8effaa20a412af3d", + "42090d888f6cf0a9bb0956166fbf65d0d16b4ee84bfb7975afa821ad8a10a869", + "421469c5402db04f1276760314af84edb5f2c5ca93e0d77a31f31cebacf0f96e", + "422872dac53957684f5fb702d6e11b39f4108487bb848c0d26b7b4376cef69a5", + "423003117279ba621fa3481afa37e09c1966b3fa7e49608f95ba53db066c4bdd", + "427ee0af0d810082509973505dda1b7b1befe244be465f40b3e642047132adf6", + "42a0a9a7c4bb7b6e66ec7f68c0905d2a4554f50cd078ea3465b2c3e4b6a7e085", + "430794193b81c68d824dc36a195b9570fab05a4a0b5b0be3c5ae5698cc82b6d4", + "434ba4341c13fc1fc6d71d9eee95ea2a39c06771630b215effea797ced42e9df", + "43536f1ba3748225251c8d65c5a5ba97747caf695d420f85e4b62457c8a9db9d", + "4356382bf08cc0c2d865bf6a0e018cbb2b90415c5dd37f5207c5a874e7e1b0c8", + "435933e580253d372d941f3a6b5be5352dbcf61dcd2355007b7f8369c5dc3728", + "43977f363939808102fc0d611a5e27cc924308b2a5a88939a46be95452a0f070", + "43e04de95dfed0fbb2c29a1c7485d3d6c66cc66ea7ad695e830915f1358c3a4b", + "43f0895baa6de45c86e086854700cc2ab2841f2afe5911496244f45ab3818109", + "440e2bcaa12caf38dd81cbd60439f72067e1767478c6b77e3613f76edb11e8c8", + "441a8a6e8fdc88b18e5a5174eab8a9f9b5110d15a3c5329ba01b9ed9da185aa7", + "441b9b6bb84b083a33a31a436fb9dca500ec5d7ca0051a1b7813eb2684be624f", + "4441befd5fd6304f1a65bf2406aa25ad86a0f30ca7bd4f5cd4deae4d384f1ca2", + "448596ed1a8fb48786ed62faa09ff8a9cc1ddb31179cfc0ba5eb60ee00400112", + "4494dfcb21b68387a1e0da5394813e325547a389598b224ce8b10b7bdb7ec945", + "44c16b61c03f9214c769001b2444ea84acbc91960c125fec67b7596862bb4bf3", + "44c9620d8eeeff6ccccd98d825af7354ed4eb4400131e03b5ea04409113b39e6", + "44da5db5b8e8740a38435d426e4eb9f855beeafa03098faaebc33ccea9d0b744", + "45244adc20436d2f5a427b80f4f1c15a73d8579ea5d2dbfb4bfc72e5f997d115", + "4557c4cd811e0166e218978bdb775fb6ea299c9603e3c20de6afb68e24706d9e", + "456b221337cbc952c55988375ec557286728cd771ef9408f1c764a4073a1e8f2", + "457bffb709cb5ccf6977d714e33b256783469ec035f37c071395e36e4cf4debc", + "45893369e05b3cf063d77f451ff8a6417a1f97bd6b4838c8dec22bef43bac523", + "45989e843b597dec4790098482c0063a6acd65d1c01bba9e8131bc813df32ad6", + "45dbc8cf1ef6e1c9ffc1c8b79f3e7513ac92f1e46154a0dbb7edf3d03ee898cb", + "46382d94897e05a802f9482b1e10fa0ade6053254dcabfc957022abbe14e500f", + "464151651c3adc93f0644b3f9b94b0e98dbb04136e92735dbbfe83b18658074d", + "4696efbc5ce832da96a9ef1c6f515d0e944264dd070cba698e9e89323ca114ae", + "46cd422c309d4f4394c38d7bcae4ce458613793042ad31626cca4a303cde1a51", + "46e704548e226c14dea88e9ff1acaf4de243139614902fdf3f5997d9c13fb34f", + "4783f6f77bb4629c69d6822d64cb2da730348eaa8a136a24194d24d53b83f559", + "47c6f0de97d395643f179a9f3247d2768707b5964c336eee0765202a603e393c", + "47ed9d7edc63708dae6e1f25927c5231c86b6ba6717983516b994ed544bb2e8a", + "47fefb31541d1c4cce72568f220febded0b5c4ec858cc6c1147050d8878b8083", + "4801fda1d5dd623aa273a8369de75adfae8657152a36650b6e8002da433117c6", + "48126690b86a0aecf87d8e7251173c0020ff593b458133ae8b560d6d9f5370da", + "4828a99d5491c1b8f564b661ac866eef5afea099c7f7ad2d67924e2c735a004e", + "484af435548059ee71cf025a6b8895e8576d14575677ef293037221f0624cc09", + "4856458e1a9f38fa310eba2ca517513c74fdbd8d62cf29196f5656460f42e268", + "4871fb0f89facb7ca44af55e031344953f8ef45c7b7585984f6fdf9e5946fe74", + "48a465de4fa18a9eb99967acba19b560edf8a061be1926210d5e96c837f18b11", + "48c30858be8459db0ca75bf71f5121e2f0b6ad18a328108363184efb2fecb713", + "48fa46445e65a643e29937c69e0538384ba7f1feef2088a73a6c39990ba26549", + "49253c445ce10685f3fef55e41d69dab439b9c9c07ad1d7fa96bfcfbb343e1e8", + "49371386ec950ac3831816f28cb358301a13abbdb3c56f5fb42003916cd59796", + "4938be3383313071b1269b37d6459e6a87242f58492b02dbc44f335d1a83c5ca", + "496d3f9c41547b2fa460c2054fb7c381747424a75117b3e471d5f5ced0067c3c", + "4979a34d6cf19c97162debc3a657c79cb50dabd39c747623d71f2c520c93f3f2", + "497cdb2183fd3f04cc5a0c73dee40295e9bf0a15a9d91234b7bf617280cba693", + "49817e31b53b5d8068fdda4e2e63369d63d548bed45065c03e04df508046367c", + "49b173f5c0aaed30183ede6f3fe27ac6e96db679cb07d35802d9576bcd672d88", + "49d89c167a538636e2cd7a049b633cc8c7f28d0e86860f2c30b3dbd3de4c8c92", + "49db0c2118ffe8b7f171b29e2e464253a1b10c551ae05ab473386a050c8cbcb6", + "49e4287b6ee26db52e0b2ba0229cb0cf295899071855dc3f665150df4a689ced", + "4a207c0a8a8e584fa8a3df1ad6d6a7572faec70672511c3f9fed0b5d608eed50", + "4a42d21f25dc546223c6113b33a861a36837ebe67506cbe9b0c1b75c8722d517", + "4a5e398567c199f6d294a82583b892fb3b0d3b5c40fded337ee09b634132d9ee", + "4a64db3dc71ce8b6eca7ff5091a0277e2b80ad862567edf4210832e7010a5521", + "4a77ffbfbd81da55bd8c5706cdb05b1dd3120466e1ac2face12b1206e0805a6f", + "4a7f6bac80765327bb0a049c8fc3cfce86f18cae7c8a1c6a57cd7dabec934d52", + "4ab9fc7611e72acf1d3a55fd6ebeb87ffd5b1efedee167da17118bb5705faa3e", + "4b09b1b44158cf0cc3de2907ced21de6de81b27ee094308de5e889de05fd66a7", + "4b0cc785d418234ca91e5b634d702589cd328168efca29e8cca374b3f0806b0a", + "4b0de9893a759494abf37b80d8c8b14487ba7c1f4c2933e3684cf0522d3b2e96", + "4b17ff76e6f9846ad966ec619bbfecec4c15f6155b12d82b97d286994927448f", + "4b5befafa28e84c8bb80aec49d60255781ed80956037db55bce16eaf8dcd7587", + "4b6543e98732849e1e2ba3b993f7da9340803e9e91a1f5f4c030ca972f720c1c", + "4b65ae471b05d20e97ee83c2c32fdc74e406e3019d0e9361b025dd1c2bff3584", + "4b82bb99d206dbe881b684309528b9a233f7c01aa2de42ada61be3497547eadd", + "4bb7a372a1e1de11644233db31b82c7d11e50c0628e7c8f58b2e21f988d4333d", + "4bc0b35b5c19fe1e718251ebaa8fcc726b3d0bcaf4dcf4552ddaf4bd822f97ac", + "4bd9eb8a050cdd74cb9e1348106cc1978b8607f0d34304f7bb856d86f99deae4", + "4befd84c3ba8fea7a5c64dd302955250b0d3ec2e3db7dbb7a1c2d2123865f273", + "4bf3d35df938a7a2900e0f5cfd4a174643b3752b74b69f762a579433444cb9d5", + "4c1549462902d97cee0fafa319ff25c0acc9a9d8b0f0c2fc046123b53e16f107", + "4c32c99d4808fc50a5c7d89b8a1fa32f8dad7f0ff8e06e748a1e4be8199b375e", + "4c33528daaa8c4791b76b49410b4081aa6bdaf0ed2db5bc7dfcd25f31ebc6a06", + "4c8d5cdb0ed963ff600d8701fc929d22a60812685d3b69868d4f42c9fea1bf6b", + "4cd852ab8d9d6c1307b5d037a8c2cbca1b9f0f651b9a4a8d75869ec89910a4e2", + "4cdaf3234c626e8930f9660042f269725faab25c1d6eac7714cf28ada00149ca", + "4d2c2a2036be8d1337f8f878f270dff29bd50ee2cabe82f4ebae082de71208a6", + "4d607073270ea17f497483ffd6baa03ae8fd6e770bf0e0094fc5f3ad9b1b38eb", + "4d747008088d0d38ae9c13e763ea7e35a3fab5859e0169b563aa077c1f5652a0", + "4da2ccad66b0320dfe85a3a91f9552177cac3735266fe014d2037a5a67257796", + "4da980d9ac346c38717f4a7e2361c05f0a5cfab7ede3d665368b67a7bddda4d5", + "4db5086479c285af9c641e7f93dfc4d3a7ecded77e9e29e377b7a654640e87c6", + "4dc96a99b46cd33924f46cb16d8a924dee4bcda345895a6c522eeead47208432", + "4dd7e0eec3c93e06a0151da75caa5a8dd09225d92f887f216671c589cc2f27c4", + "4e06b20920cb157f51231aa2aaa4b05d984392e7e0feac54112621758cabc3cf", + "4e129f2f9c65a1628d79caceade78b6ea056984fcc2e274f68ab01aebfe2140a", + "4e199e45884bb64c5eea327a917cf04e6de8e285df60592eb2b4afaa33729c43", + "4e26a5a289e257ad3df1d62ef154be1e724e71f19bbdc4d0ef54b0b5eb18aa8d", + "4e379fc475c786d5b1be61d19965483d11829f54164b0010d4283a33a8f60845", + "4e3ed35ca52c2e01a7cdd729e7068c1c03a5dc841390d426b04070b750adf764", + "4e4872b564cc0e5f969c3249956cf8aabcc79b0438edd7b02c65391e915fb6d1", + "4e7ae1e39495741171f3aaf74402143b8cf31d863924de41c80a69bb6186b89a", + "4eae55f012e712d2e10100b46eb90c597c8a0ce62bd167ee2f286e161434e0de", + "4eb8937b123b44f09c2574a572e384c7511a3ea950eac45d4aace87c2f44c391", + "4ed094cb2bb666fd3da8853a36964758d91dfe66d1bdd347e0f7c8b8c718ea0d", + "4ed6453dd0eadf7321ebfe6d030fefe0bb7daf1e52cf195ce43db0ff2fa5f4ab", + "4ed9df471fa4c9d9b7d6c29eaf127e1743ef80322b706f10008711fbdfad772d", + "4f186bd841dbf7ddaaca22aa1552a1e191844e3b80866158fcc37236159b7531", + "4f3d06d0b1335a13cb225acca00e85b351b3a5637c9f84e70bf434d52be4e93b", + "4f42fbaea85d8740f2a250daa88a9ad7dae53a07ab210746cc1526de742265a9", + "4f4bb1a8cf80e2c0847873006a5ddecc6871ee627588f1afb032a56779eb2cbf", + "4f865a399d2d50af188ffeb535c333d2017387ebf230f0b13e0e741dd35a350c", + "4fe42b5a3e9d4c0f6a2be6eacdd1a0cf45affa7cbb687c50b143da1179fe9c18", + "4feb47d1ff7a9cb4022ec5fc0914c6fbce0dd9f1a1a7af70a36c7c83949ddda3", + "500325e34d1afe2d2b79d83e7ebcc7e4b1f140c8113bc29fe712cf9298392906", + "504d7971b807bdc6ff779b6c274db38dafda22ac1a2b7d50becee1b3b6a2dad5", + "508a89c82aa71579d982948d197a6967e9bc64f5ec0b224467c4cfed96b88785", + "50cc531f76a921df9b6e2eef827c5921580930c58434a2778b01083e55967446", + "510d7c189bbf686a248aa8172d9553649b5d82cc389ec42c7f1c95bc3af9cb7a", + "511b16f4f474db69694757b57b5805f47decbecabef68ad4050812a972718198", + "5121cfd3d97b25de8e92fa91c182282af8d11d6a9035fd522bd2c9cfa1c3c075", + "5147c91d7c07fa96291abe56b71abe33fe5c8260d37c54b7f4c2008e802fd36f", + "5169e603e9ff88c8b5c21b97f23a0a4f96ddcb40ba25754c332c7cb0edb1640c", + "51ecd39f98794af8a8842cd2d4b41bc9cf9117825ff9915efd474464b2e5faa7", + "51f34f7fceaa4c434accbd8bd8596ac00f4a92642a437f46750ef45e718b69db", + "51fc880d84d3d8acb797768c4ebebbae41bdfc303777b4e52ec7ad95c08ed39c", + "522748f349598f8a5b5af35825819ab6ee2456f91aeccf1083dd7f6353480edd", + "5266828a56701e05b3c0ebfe65103f9d484b036c59b4e1eb0bda970bd5cf63bd", + "527e94eb60c278adb854c2849e5e1d508b6ef13f651ecd7aac0916528851f8be", + "52aa00e24b3fe442d1dd5d535a6b8e2287ee8a5e2bd85b334c377bd26b7bfca6", + "52b2e38e8e13617822a9291ee6c5365b5626ebe01d4272f39f75a7c9335c848e", + "53129650eca54ff4f0a67bb3cd93ed89c64d5dc87d945e9e8ac09ce72f8650ac", + "53214dd5fd650a6d93dc2d3ec89c65cc7d4503cdf665e7ae2b2bfb4895fbb112", + "5326f5cb0bfc01d08b7c17fea34cbdddb4d1a4fa5d907435c15a747b0750af20", + "532f79785cd9091700d6a0ec6a83d5d023e71e56ce704f91e3f5d92ec12a9af6", + "5335fd622f89353e5f12da72d7f570a680ea20fde15a4d030cc3e682917f3e08", + "535c9f3b9205d8d835cdf757b11cd0fe848c1d8bac1c660daa0e710abfac44c3", + "5368d271fa15e996184438f729026a741ceafa7b702e2f5319a02ed4a7537096", + "536ca7234857569d044bce93cd8058dab7eab3b19f4c4a56f2ac347a811fb435", + "53d325afbbaf0a085c8f4dab65d0ad4f6f3e48ad455df4b92efae664e3ed30cb", + "53f7214b828aa5afaf391ef2c4802d1f8d0c782a1d9bb71ff86c023134cda878", + "543c4fd0c392645155f627609c7d430a4e4d06e576863c514b20e1a564381e21", + "54a84ebc0507a9a9e843dc592c639f3b43c7d8e94c6019d71029c2d8e15dd930", + "54cc4fd24af0cacded407bd4f3bc683e25a9e969d89a010d36a0dbd586a2fc3a", + "54dc85bf612eafb02c47ebf4d67017c8de63bb7ced7344de32de836ea5551797", + "550f94d77469ebf1c5b125a72fed8df79c24a28f669fd339d74aa22ddac9b5a6", + "5513af533ca18fd5629a153368b994ac91b0449602a26297479018ab1f926071", + "552e2500e0c22bfc971e5fae6456053307211e2511a4b3f4edf67248e9e60c7b", + "55a8869fa9dc2191f7aad76c572cd5fbb72b826f2aa1566f0ede2fa89ec2a987", + "55fec48f1e3e713fd61d8afafc86d0bf0bb63b88212d7a3524d42201e4363d47", + "560da590c9a2d82030a253d044f3622ecb732576a98175b89f80efcf56d029ef", + "56330f94ec1211784277ffe3d0e6eb9d8570375c1502885ac9d0353bbe9b92ca", + "564b347329fe24e1e6550303be041361ae32520e753adcb0fa3b70d24f42953a", + "565a04ee80c7f76dd537e36950a9760676142d67f41fd998fc8295a59495246a", + "566a331cdaa86a9e637b843b124b48b2e492baf6e9a0856458f6192486912efc", + "56a30765776309f19a9e9cdd1f6d9456a23a1363636db79a0cf650bb7e31c547", + "56a94d11f9dd24ddc273a377cee421a7041081158351923c2bcdf1c180d3eae2", + "56afdf35be6e9e568b089419fdc113e430ed702e03f627833150a9b0f8b359fc", + "56b07e2d192eb70cc5a1dd8717ea1d104734dc3bf0247a85ec610a35ca8f5461", + "56bc76794589d321520fbcf73773be5152d5026d6117d231d5b49aac792836bb", + "56d9a85095e43f325d13a495567d69055af77d9eb0564a401e32cb3bc89336a5", + "56fd2f352e422edf3479dfdd0d0730885bcdc6e8140a93fb29efa33ce2e1e847", + "5750d0299820d644772be4cdb8b00a3dd45729a401c65134cb2a76cab7e4fb2e", + "578c2107d5f26305eed742fdc8f309810093277f54a5d831cb9d62431ff3761b", + "57aeaa2185be958180717afd2d67c85c071eac34214811e3478263c4d376c152", + "580347991ea4f30e4f531a31f6c561c15bde59a529de296194e4a35c2460692c", + "5810e162ef494be2cf77abb454b744a4f104dfc64a8bf30d81d8e065b3a0d7ce", + "58146c983a0dadd6f68056f5d0b5b285c1296cea6662566ad16257119665a177", + "581e0f0f3b40618fd8ef5e2f13555c891e6ad26e779700fbc9e7ac7a74370e85", + "585214c1720d13b39d0c41df6a0222e8eba17741364312ff3dc9aa1dab87743b", + "585e3e07fc2ef6b1dcbc173166d50ab12c9d23ab5b1ac9b6d2755fce6f4873b9", + "58773571cee830e082df51f368c9a63a7f20d290046b62633fdd995ede99418b", + "587db8c7d804f20dea3cd62a9c01057cb3b49601e4589440d2017bd02d638cc6", + "588a1e302469a7df1104d8bb094ca75222ee8da70b0d7f5466871c8247366fcf", + "5890e575bfe1cbd1718521af938e9e6042bf386075a3f2d18fa875cafb6bb992", + "5896dc528d2b7eb93675b4b1cc555efee4ff2eeccd4c3dfc8d36bceb18cf0e6e", + "58bafb2f5bb8a3b1eb354ecb07e3453d00501c6b5baed70aaab8391c3f400a2a", + "58dd3c4e88e304f5b06967264379c278b8fc6f530fe74a4d3aeeb00507badb80", + "58e1f86fa5ce3520e7d8f58889ee0d0c51e0157b76f12db19c7380abae59ccb0", + "58fd614ae697249681124a8c8fb27476cd7ac07bcb9bb5bec862323be835e426", + "59097dc61bdc1cb165e5fe9ef02fa705888f8737f2832e5e5355c1308efb9e82", + "591dee517c139be11a013a596aab5e3d4319d2d056f49a8cd9eee65859af7c6f", + "594c5ef422269e28c70f7684ae1395e7912c61532d21daa7e6c1372555afe0ae", + "596fbf2cced5eaa06817fe6cf08e2573bc73576833d15c973378433a3530b2c6", + "5973d3dab148e37f2b2ff8453d4c0db021866575b4278e4c53e4df8a700ab04a", + "59827f3a93b7f8823af0a16a43d17644aef2f7086b6138074a2df2a6ce5e6a48", + "59c53eacce4fab10ac3a923492063f77745b6b08badcc44256d82f74886e9e3d", + "59e170ca4b004d729ec102c5c05d2ddc53d12c725e025d112037eeb9dd788f7b", + "5a5d038adcb164227b02c45d95d545b73dc5aaf5c29f34d2dbae3866d12182ff", + "5a5faac5b9dd300ae710f7742a8f72e16d8ec858b153f397e45af70731cb6d43", + "5a65be1f3c040bd2c6e2fb583752e5dad7987e11e451d59e690f90ef14dee23c", + "5a665b818961b4ca6b32bf841165108602b3e27012b836b0f85d7561852d6d56", + "5a9e2a164309d31ea17ff845570e206871eb9f4a062edca50fa000b56818d599", + "5abe82d5f6eb9f97fc34764a7867c2f48f6ba3bef63e0c0db0283cb233d970db", + "5b077baa508db040e7bf3405264cf235c7d95ae59f33c54ce4fdd41a2297badc", + "5b0803ecdd5ae32f9940911e057431a19da47f172667396a50231caee56138dc", + "5b09088a32a4d31b4fa61cd7be14176699d042b759c867afe3664b6d4a3c8d54", + "5b29442d8070ea3f34d021c2229abbb1203ce47102ccc375e500ee28380a4999", + "5b48e419b56f1e0ac888f78d01256d7d9792181d244f7cf160bc6b33b86fa0ec", + "5b5d08a56b25f363a8b8eb54f1b66af6fd8d5e1d37ff19b56f08dde5868ba810", + "5b739a71f1b3b09bbf3d54543b662b0ea4eb22fe35809aa0ac14f2fdc343b6cd", + "5b8782b98afbfb414e51f003c063aca6dcc358ce27695ea52a0cb111b7ee3863", + "5b8ff653c10153620887a41370f6f1f7a862d06f9ff05002391c76ae0c7a7287", + "5b98921008beb016b9637d68f80eb8ad495391b3f2dbaa77ad64a705c315cbbf", + "5bc1788d1e6b4a285d30916d2f3846a50e57f7ca7a018eedebd0cfe875df9aa0", + "5c03f2b3a7c6f501261291db173abefc3d9d73a4094af49d78fcf6b76e505c5d", + "5c18ab98cb1bc02ac71d430fb9236c4b59f142d9d3ed4edac095b38ad1e6ce6f", + "5c24a8958cb0c33ff7617e4fd9770cb9e5b1f0d9932017bf0b7e9c7322516d53", + "5c25781bf806586340bf9b17c1f1dfc990e6dd21cadddd271448866f44d1c684", + "5c3b624fba16f6088eddcb0c9b9422eaa1aab3175b5e89732601d788440d1c16", + "5c4ed7d029b580c166984451d8c1fd42460bbba0a224c77b4e69f70cf1428a69", + "5c59eb61bb8b5953a21997aba7dd19476888f96424d4aa3e9f2eee253bfe9b25", + "5c7c610d7cc5465f9cbe4714c35ec928553493d0a16c9a63a0e57f8e4878fc6c", + "5c8b92dda4de3de4816c98e1d58444dc30f0e79493330a36d857be85bc4d3b79", + "5cb6260be3c98ae3ea566ba38130c44a50bf48304db36059d4c05f345458f0fb", + "5cd356062f8e1ada45e2ab1fed44d13c8b551331f1d485e4b6860466f78a8372", + "5cd993f6b096c7825ff9dac935488ce03f8dad2b060d67bf2505f22491292f5c", + "5cda3ced22b8cbf4998a18a9577a5ca62a34e37b38d118367d9d11d2d7d9b3b5", + "5d11328833f680ff28da140865138c3ba6383e972a7d95425a7f199a499017ac", + "5d13e695115cf2fd4c780da5bd73aa6774b7ceb334ebcd1bcede0e7716870565", + "5d1dc41467ed9f448b1dc703c862c1f6d51ed29037aef1e74964b8341f96034a", + "5d349e1d596ad3e2b74a6bbf355fdc4259476cd6977ed29600cf27ae50b32f2b", + "5d4597dfd3afff704c7d40fd84648673786f0d4a6776311ccac43be8832138f2", + "5d5c5ed5ef38e292d47d4e9f0849a6a550f03eaa18bf6f50d757cffe8d5a306e", + "5d8923c6ca6ca759ca9b471a2e08fc501c0ca65a8f38fc4e08fdbbf9b1c37702", + "5d90f1c2ea5e0973c16286af4fee863e69bd2c5dabf11e68efa4b30d15a38ba9", + "5daee84cb06225feda64408ceeac2f96556185059251680e18b092e894f6bf16", + "5dafd93880dfb3436f97a5eea1ecaa129a2b5b0a2765fd61b7ad752f4f45a8ef", + "5db84722f01e750446f95efb0f7b1e3e5be4e972df48183ef6449dbc33831e54", + "5dc3312ffcdd9b0392d350a016568b35465c467f4a8af3f8df8a01bd8d0b1a99", + "5dc49e102663cc053f4489ca3edcf207447454b65efbdeaa94f23a019e50b17d", + "5dc9d6774b9e06240c24fea6ac9ad4ab0a1c0fdae3b9236eb445bf7e05eb303c", + "5e58ca69ef681e85b55a5ee44016c5a277003ffb6fb80d76e1924d7911355b36", + "5eb24ff067b2d15897c338e09f972b4278a6a86b9167986594fcbfa1c4bc9c5a", + "5ebf7cc08cdfafc0c31b22d4f9c6765865ff9e63d72971016abc8c4dcec766e8", + "5ec3e6a888e606119f06059312103a6c70d1ddd2765dc399093457245bd07189", + "5eded2b78b90beb97074676bf47a1f3243b84ecd855a25aabc030a5a81102b55", + "5ee5a0f04e5de073e8b2e1ea283e8811305fb610276f297c8280b061019e3738", + "5eec32ec10e48bfc7f746c5560512e381d9c44eede3873a4d40f2854efd57476", + "5f9c8bf3d60bc1a4b796feff5e5340f352d4b70eeee37d3acd503fade03e1acf", + "5fde1897d8bbba084ff2a87e8fc4c103418060c56244f796f8b102d3ef6b70ae", + "5ff67526f77dbeabeb3eb9af58aae32ed2bd0af1c859deb6830c92b283cb720a", + "60072d766b1383035d005e1516a319b9c435c4e3ce59b69aeb5342f07dc380bd", + "604d392d36d6214c60fba0281725ef2125e573177d58d54d6a6f8183b73c3f66", + "60537dcabb3d7bdc1e312753525dd233fa8a7745c77ee26c77241a7886d0209c", + "60589d3cafbed181eecd4cc288e853fca6bcff092197b3b42995411ac6e3a2d2", + "6059c8c920478b870ae974d4422cba0ca49af2bdbda2fbbf10838cdcfa5c0387", + "60709160950e949aaf4b06a81bb046990a4a2699414d23ff7390998bf98867d7", + "60f873f907ee5d4f835f6bf079ea80950c32b0b885402692574ffa1bf340a0bd", + "612acbf37c5e1db127331fb6a7a9faa48b65756a36cccb51315bdd2663362227", + "6137250e31cccb6df9aef545f0af9e6f425d29f57df8c0bd59adde97f2a35066", + "6162e12b54d9c1dd7d764333543b0c1ffcf5660c585575595c04636beadcbd55", + "6171a6e20e268325d4a4a3fcf435febf5bd2572239d5e36dbc1ac4b091e86dc6", + "61ae93a705ef2553ecaf50ee273b55ca0e859f6be1b5501cc15ec1f78750f7f5", + "61bffd76ee2fc12fe529aed3fea7b656429f720be71e72d304b05ad787ffa1aa", + "61ecf85f0dcb3ce79c049cbf4246d31c3e9ee4fd97924b4d9786ae87ca6b6a6e", + "61f9efcef6f5be426991f087dd75332705677487880af8042569287517516522", + "620ade8ce0fc6e5781a57ab85ae405c9f62ea0360ec2a02d441ab4b6fb9f39d0", + "621d7c8e6248697e85db92cb715da91464290c722f306015957949842a1c820b", + "624d16569d89bc9da4b8f535ce8ed165b63ec7b7a290e116bdaaff37d0a09f2d", + "62a21d830ac26c364928f82f927d53233bfc87386ff2e97163fb981bdac82a0d", + "62db198b6f4c3f1a462a174b3d5d5a6078bc2dc15790360bdd7116e0f1e7f31c", + "62f9affb54a325d03ea5e8311b6fa62e04263251010f8f1e0545824b5250fc9e", + "62fbc7ac4cabfc232dfe0d8a78c55fc656073e2f6642396082245c9c98dd4998", + "63157ebe4e96307820a0d937d2a37bae3f9c14cca6cb651150156ec5c6d36448", + "6335cb3ff64c3cf8541dab770520a7cbcb6e3c8bec899c10393eefa9814f6ac6", + "6363f95dace9764ebb6dd5e4cd446dfe73be55be5f9e0b063ab5406af571075b", + "637340653f23ca873db36e16f9e98d6a1a1d7bd96baa07cf2afcad5123c182ee", + "637ccf26a58ec061ad2ee59d1f87c95f71f0bf8fa029b7ee9797b2a909c36ccf", + "63e61f2694503b1cd61ab7dc4c33759d884c3137d7e4f2a61709c562a01790cc", + "64642476b64edf3a64d69b2bf3231a5175d46c8a7db906ccb95271a787a799e1", + "646c682297f626e5d120f89acc519b581e8778b3b1dc6ffe16476a9835f3bcf0", + "6489ad49812acdb7da4187f73f716b78e1cd0b589e05fac85e5c75b1ec5cb1f1", + "6490ba8c17967e7a7203bce09753747dbdc0480c3f827fd4db746591b610a1d6", + "64d4e42816228f1b393c4571a5ab6e78acdc6e1a207df3bb942305c9de8d1c61", + "650b7cd2d1d95428a6e9ccf0dd9a40a72f1c636182446dce691c3169a007a87c", + "651182a33f678ada4b45eb1079ef7e8e105ae14b5bad8cb3952539de77cd7d05", + "6530adeb56d3e3fff647f3ee8d34065bed5ae533ea00772bce081e753c861291", + "655231a11c2e677666b6d7ba200414a8446e912ca60e5924b9ce1c6f8998f5d7", + "656f782ba203277a60d4a4c5a65a83feb7b139b96c5d5b0ecd132f21fb365f8a", + "65714992f01962d6e80f7afbd4c8e6332b419e436e0cd5d985958d03a754d787", + "6578e518dddb95d215dd3a7dbd1577616364d9359162715ab59552d3ec77d308", + "658b002d9661970ccaf80be9e81500d4a55990b813b813fa0846edece61b92b3", + "65a71290538408a4d8e1f694b3148e0f09b74528fe161cfe3f2778d148f23b26", + "65b2ebfcd8434bc57234573a03281c5ea42211945505d4bc7926620e7c037883", + "65f741be093e64aa33aea625d6cb2a0da9d0f66794d345d4d36a64c194f88a89", + "66106dad07085d8716cd3a57c38719985cfee9d7da7b9e75d42d653fde524a29", + "66278a5f3ea6f31ed36c22b8fffa8d927eda8d7f17f75bdcdcb3e476a900b092", + "666560495c0db35d8963004e3610db8147cba9b15b48e2b5298340a7a82411eb", + "669c686bd276c775b5e02b1268da9fb8d07c792351d0016576732ee0c9bbe7be", + "670869367ca4532f46d9214993c0f31081562026087e6d64941a9c4a7901875b", + "672bcec3d5308dc7eb9a921e5ef64fecce84cb150510afea304e61326daca37b", + "674b60cbcefefa4501172470e2073dbc4946967875c5451bbb39242b91c6ce02", + "675b54560835499f03023d08441c0ebd1e0d08f93c67df4d03476181c60717e2", + "675ee443873b67c1a0d4acf4fa7d50e28b2fb70531bf3af261d528478e59c0a2", + "677a7bf5d7402745cef77d25c358406a21a994b948b84b9627004b3eae0ff40a", + "67ae8e470a9c09f61434cfe21c78698058e3d595b0f68b7edb31c359c990f1c4", + "67b7367d7b14561a50011d56a056e3b8e4a46f32d6e44f48f335474600119352", + "68492785665af90395b524536599ba8101495f429eeed201c267d1a9486b8765", + "6873449ea7587bd681061ea00fb7a11653cc4d9611d6544a074bd95e1bd04434", + "68738cbc5f45b159196c3cb22db18e4c8daf229c4980efcb501427372dff8f66", + "6874e93c2da9e538c7b3817ab71c77b8523fc4ac9d4abc4ef62b46fca1afb8e5", + "688a3beeed114db0cc09136ff3e1f826a4c0009eeb0a5dba639d70c5b8479d10", + "68a2fe7ed3dbacfc51faf72da293adfba3d7aa87c52353002c82da7f71ad400b", + "68af249b1ba166e2336cd9e42a28de6d46fe383904832c4d6537e7352321d073", + "68b27de105169ec72e0d2b178ba1eb0c3fc0a98b4a7995b2b3e517c32a96c4bd", + "68db72e7e93af0434222b65801ab46dce3761afe35c0a144fc7954013456918d", + "68dfd709de674c13150cc384e52feef725a9bf0783546e621a96fbaa2bff3dfd", + "68e32d09a1873503fcd8c4473df014183b75fd59524c232428413f0e7c26069c", + "69247df2ed55b582d2560bfb3db2e758123d594cd88f2dba1e0fc4bfc33b1d29", + "69249f79f5aaaeac200aabe1636937412dfe678a99cd79bd6de302a2c889fb94", + "696e2f5c8d58f45c1879a88969d911893c391ec6cc625afbc119c5d31c1137ad", + "697e0f9df442d5d8ef90a2bdb81ddf81ecadbb8870c89f7021d46763fd8aef76", + "69a3819da4c4f0c147398fb5b7da7d804192caf89f06956ee2348b2ae023d499", + "69acfb9fbe3cd486ef3b93739328d90eb35b2431b15e512b081e8eae896d1e08", + "69ba2a070b08ab1d9e2b171e5ff96da782a6b2abe6d78847e82441fd8f6e48b3", + "69dad4799c84f69e502f17319717729db7b879694541dd198f51c79060f2b2f9", + "6a01220adc0cb957e87d4781fd8dd89b108d1e507b27296ad28773a0840567f9", + "6a0e6e4014dba325b39185dbf1a7a08e2cd10ea7c02bae337c795d68fa6ffafe", + "6a2a3bf1cda8da3cfd63ae6e9f6fbb55fd2ec6385749e7ca09f27edcede57584", + "6a2e45123f24f5b2a79c78a7eb96ab498b77a0247d3cb1d3a1086ba6c026e1ec", + "6a79de837ad6b4d5b78063a389d5a30efc754182c9a450f685bbab55fff0ebf5", + "6a7cea629eb5f5f3bc04f856a20ad4252f2ff11364e698f2bccc18891c9c0ff3", + "6aa8fa03e2650e4784898ef4e27a212426010d0cca9f9cea722b5a9d050197dc", + "6ac918e60536affa2c356c46bc2689ba51862f52e0c4da168f637655fd085de3", + "6acda15ef961735cc7d69ca607a4e11cb962c535ad6adab5bc2953ff51d6e9ef", + "6b166244dfacb745cd227432fd643b7259f9cddc517b05245f62ea70a95cf934", + "6b228ebec266d57dcaa49b58b1db767784f6486d45ee95dc131450a0479c9afe", + "6b27ba831dac670539efbe9131002e9213375a9a26af267c308e5f855c2486bb", + "6b4cf277783c75898a2559ae6e129f1a5af61eea9943b9d6a7d1e2248194141d", + "6b5c7b0afa8e4b320afd0b89e313c9b5fa62fa33bdeb7cdefc1673f9c4acfd4a", + "6b79946dea25c3685d5237e236c8931782449933fce6917c0c7204d6d7133fe6", + "6b937b0554365fa001d5016d0af872c629c510ab04a6e9e8580bccaae07047ce", + "6b9d661d4eb6199238cc5cf77affbd3616aec0b9a794418f256a14003b87c11a", + "6bcf7405aaf5b98665c658334cd647d60106a8b76717466a9c504d28cbc31c61", + "6bd312d9d400813835b84759667398a5c0b8fc2e05b5461da95fd5babc3ee785", + "6bdc16be5569ecceab477b89fe2586151da5c490333f5e0a1f596c4bf83f6b39", + "6c0b4e79c200c629e640557990dd39fd1b5fbf70f4ae153c137d6f35a563bca2", + "6c2f06a98f5f92eda14193ff996f38782ebae82f94c856ccfb148c4c5bf5f7a0", + "6c7fa58045d8afeadadb92380fa79952b72c66c3ca4a199ffd05b405009fe5e1", + "6c870298d4b3b9a594029a58de126c7d3912bd4c8aa8aa0956f96e0c94214237", + "6c93ceb690805df915a0d11307c3e898862ac2a8e2f03545d9f9f31333d97b9d", + "6c9d4b7e0d99d0168eda469e87ddcf684c3ff33d2a120784b3b6669089ff107e", + "6ccc1357aa0f377fa2486a55ec540541ef6f9143b4215c14f2533188603b0fd0", + "6cd5081e4647c16bfaa05928de0b79d48228f7163b2c097a78b43258081f0489", + "6cdd98c3d0ee8d4a939eee49d26bbc9744fc9e22e88563d636ad5b1d4572b211", + "6ce8a8829a4467d50859e5d48868ef249c31b4281ecface940b240804498a9cc", + "6cec9c740ba668e1bc5979a2f406be9d4326eaa440b49583caff5fff16a4a974", + "6d024da7f213afb6dddd051098fce2e2fc219009d7621833e4e0bfb3d459e93a", + "6d3a64c3c7d6bdcd11e121de5b26f8bb5b8ec0e16dd8e0a9614abb6279689583", + "6d3d79a21ed625cf8c2bd6fa5233f3b5eb112cea3b896a466208ff48db23837d", + "6d8429ee4d3409cf09dd4c1e1ee8d17ec622f3f65f2d04d2d5b46d3a1e19ce10", + "6d8b1d53d160411812b2e63ed22f7fab8a2865cc200ec6dd7a103a121405c5a9", + "6db6f17d30a1e7d929b1574447e0a58ae84fb52e785dc8ffeef72988e7b9918c", + "6de68bf5dbfc8d0a38db7653c7b28a6172424c2f8bb2e31a5318bd0636a34832", + "6e46fa4e2fa1ecc23219527e1ee220eb785973a0f8b404956297d5d6594a3847", + "6ec6d1488d2dade7f32eb4be6868b7ce41161dc4b88344dfb788047806eb3fcc", + "6f3203130b9b3bd29b81708bfc5fcaf3d48926c9fdf3e98eb330c02a99d4d702", + "6f6226cd7173c25f9283a26873c340a53ed94d4a734a6c38366fff4c82cbe894", + "6f8a46c31c1aceac05f0039bf6e9a36b209475ef18a584f1cb8d5e3bda48094d", + "6fb212de4b5bef6fc983bddb206fcb4978547681a9542de68b5abd7f51546fd7", + "6fd4ed214af38642cc6bb474abd126841f82ca4516af7d676fc525f3107eb4de", + "7005b0152b7e62e70a4ec3548246bc1b1d5729feb4958c2fd3688c137f69893a", + "70068c35f76f137bcbffaf69bfcf8103099b381ee31719b508515abb965fd586", + "701bec823b7e553bae03ac497ebec8a40592fea30cca8d91cf9efad75e6ad7ef", + "7064f9086f3605fd49bff512a1172dfbd0ceaf58a88410d163a8cd1a0ad34f94", + "7072448eee55c0872253c8a40a5bfdc2c36b8f7b1be17bebe825468540882db9", + "707966f3f0d9b4a67d3379ecf693b83b2a1e1c217358494af3504b95c137fe17", + "709835aa30c7d0a66f14b43475d8db201bf2bd39510d9ebdfad8848b84fc42cf", + "70ab03aa506b299df87e64cd28986c66d4a8dd1afa44066323e9cc1d9f89a6bb", + "70bcf0249de578d517dcd2d9ce720351b3aebc2b3408f92ffc31c3f264797cb1", + "70e8eb1a392cd7468a49fe73f3aaa99481ab7019d354ff555dbfa1901298e9d7", + "70fe15efe7ac3eafb95ce5f3f89daeaf76a65535fb98d50b60a029e4484de29f", + "71024b56d0f6a22d5fe9e7acd8ee1e7ec9a651721f305a2c7f3e4d5849781f5c", + "710f89d19eea86719bec11c771e48def42e5fae229de5f1f094639beb3c85efd", + "713f861df41f12f45b212350544849e022cd8665397f31300942f1ef32f1bc18", + "716c7dcc84d0813272c889fa45fdc626ef28b96df90f083f0f95e51b5909b7c3", + "7170b2876d55cff7bbbadde1cf60918fa82c96b47a419d0dddfe68656bb15f7e", + "71ccfa1efe119f4b3400c25f08c0cb253d6e84a3034595291688d0390362e7ef", + "71d64e8f87b796850cfde842ddfcc5397530526668b4758e95e3ff352ffb808a", + "7208d392e8466c55bd4db2b793267d4f537812981de57808548a24d575c00802", + "7241dc8b47fc985afcdf062800aaa0e4b3288f68ae23e6420d4174936ce1ffc5", + "725960adbfacfff83b82b7df54c856df7d95600c14505b43941eb4a624f2d833", + "7261079205d5dfa4e29d586c9b6d03ca1ee3e2225ef85043a4a4eaf8583df9cc", + "728722ded8b3e23f89f3fe1884bdbf69cb04c6f5b843bb307b4a62e2a2964762", + "729dfd823bb3d1363bbbe0f6beea353b4b9cdf5e47c3b6af17ffc74b854a2d88", + "72aabf4eaf51267a223ceecc011f0e24ba882cb2833bfb09f99d7000e4ce2935", + "72d78fd5e3c79370db8423179341cd9bba72ff6bd98a54b2a915a698196bef03", + "72e1b778da03bbf91d47c4c0fdb930d7ad38f76cf0490408e8b4df6eedac1ea6", + "730722ba5721418ddd857592947efd2f0e6f8166eaa20bf9cbab54a8f4b0a3cc", + "73546fed85b1e74894df9b8f32aa7bd56c478286997a8c92b08c67af0e5335f8", + "7355830e37a3f3593f8f485125738b6e7caaf9813042a1aa65a78148f98393e6", + "739d4dc3543c4be6ec96158cd1f81024d14489153b712afb0a7baa9a136af44c", + "73e3407bdb5f83b252fcf7c2ccd03c3a15690c91e77ab9a390e9d6e133770587", + "73f1a153831e1cab13cc9cb150325007b4b7ed7d147a5aef39665059d0d87327", + "73fb5ecd247e8ee18252483813c02d62106af8bc146cd4e3bd3a321f20cc866a", + "7461c878ab37114c8fa861bede453b9f180844dd0024a1031f8d0d5d4350be43", + "7485d46799828ad55580d2370456706f56e6b531b241d1c09d6e60fe61680738", + "7493553d2dbed8c03679691050369d2cbd955cfeb2e09469b115893f92f81f1e", + "74b8a267ac18fcae42bfc59b17bbfc54cb50efbc4983b4fc0de89fdbec45f26b", + "750b82983cca089581424966a9da92809d4739ff8a84fc81448b6c2144f0c151", + "750f7f9b8560ab8aca11e03d8825159f4a02c97263c4b164cc6b6cf8c627a367", + "7512074a62a9ea08ed848461d4b771a087bf6d0806d9d8e9de9f109de0b62713", + "754eeedfe7b3dcb1fb680ffd21ca550bea5bc1eed3c387b5889d69c739bf82dc", + "7550a0b2274ac6eadda85f132246d97cfdd1fbc17a8c60247c93e8a7d6d42352", + "7552868610e7a0492a47d7bb17f915b243f75b08fafc0ec2122db8ca25d02ba1", + "7574a2bed3690cf7b377627edac03b504ff654934a0e4df77254cab0678b3da3", + "758fbeca0e118842f28a52b4c62c1fbb9828f2e0a7c78c41db50bdc603bb82db", + "75a4c50f6b7b5f9416c27fabfc24bfce47505c2507c9e5c40689f1cf8edbe796", + "75b0312a7fe659ec2619fc0a00188c1c25648195d6e7d9a6f7064d3accd1b897", + "75b3017f16c079d88f1448c1f4b9665aeff8c770ee0174a59a6dac4af0945054", + "75c9c4a1f7af09bd21062dfb98f95b17143fb282d7ecc42f082167e36209e5b0", + "75dae78bf5c4fcc772bd1ae3d1e61f73ca0fd329342829d93701ecdb05cbe776", + "75f25b711155c22abd881f62cbdbe0bfe8236697ac644ba420b88a50799d6682", + "75f96d7235913b25c9433b1a0ec68104bc222192d7a8813010730f2a0659b3c6", + "7622d99087890010662d5c0ded17c8d5a71587f4e39fa1d389c45a24b04cbed9", + "762512c3c71425a88180aeb09010834265222f4af4988ca83e9d4b453058e003", + "763be34aaf034de4d21129b44404ebdc6f95c5e5aa07140a0215b3a831c0c975", + "7658c48e7502d6d1d0b8fbffe97f4df627e7974f75cdbb1b9ab6a31495a94ea2", + "765a03507d35544d425b3d04d0b5f8af819630c18ec83638bd7ca85fab92f351", + "765ea782be299b543c8481425b1da923aaa737acf2b736c47584089744724891", + "766672d016c0c6eb36997037b67867ab2c694726af1d88c375228194dfb90a87", + "767a47a19834085dd8e10c9d3d66ac5964793f8a9513a35b2dde4d1429df7a48", + "76a179bb27b3e317f8d4e66765cf407ff74fe9839e80bbd0d6de766e8bc90823", + "76d19a21140ea79bc1b758ce772ade48566e5a967ba98fc6319541f9e4cab3c5", + "7703f063aec2da8a8985f6e719c2a68d00f1ac73cb75800e295741fa4d40ea37", + "770e2608d147738abd66ba9367f4d31123a567da4e4237c08a58d1b357f6ba26", + "774be2ef21c84a0f0b67221f61609824f9e43ae746ceaa6ffdd72ddd8da88c49", + "774ec128960025ac079c5cc7860ba8866b328c0a910e4ee97d9cbc3948293a74", + "7771b23e1a872781d3c859508c8f81f5f0145aadcfd51d4937479e8e767e7326", + "777e1e2ad048ec24134ae9e9a2cc09bdb9e65db4a33aaf26a5571378fb576c86", + "779ed248c556b5bf3b3c773212e4390243e14167ee6adacbdc126e00057141df", + "77d7f1daec219ca893d2724e410ba5179573dffd22ac21b1ab2a67fc65159508", + "77f2ffd9c903d1cb59f3bb79ddb89b412ca1ee3529d555074ed4cd2e575d330f", + "77f65063c77913e76134b4c37ff835f373526c410e5b367c54fe4d8d613e473a", + "781ac47130a2dc741d2bec0037bbbf1faff9e21ed424934baacace67e827e599", + "7834f835213ed6839d9b2abd93cf8ac01ebe5bdca61c9855dcdb515765e61e2e", + "7841c519c3dac274e9c8c618fa6201314f47dd892aa879e0f41619ece8f58376", + "7847d17d7e5ba94d471bb710e54d9e653488c9753dd1d12661e44c04cb4793e7", + "78c796b942fe136b1056c697af7179df8ab8dd78e2c2bb385d9774710d9816e6", + "78c9fec65bf1ce348f0f3a33687b29f95d7056f34c580a0e4292702501694c01", + "78d4919ce6c9044423da92ac281f463a4de59ff03f9a54382fef3465b64f2aa1", + "78e0d7e8b4413c0759ce0b6c2bbb6a132b7ff8aed71e0ec46ca0d3669080c781", + "78f5bfe6f58ca23b2d85c54acea293e7e19ba60b63768934d1b65d1b44b31364", + "7910445c446ded88f3ae81db2004fe916506fadf83017f1d6c0481764fbc726d", + "791e0feef999d39967656eb542ccd5a5172cea4650d58f23bcb8fa8622551d0d", + "79493b75a4fdce4eaaf946864585719108489ee5f6dbb8440a70987d7ab3a14c", + "794f1d793b6807ddb2da2933851557a4ff3326c21d6a717897a475e7c8305e11", + "79665c355e938b7c9a5085d02e1f4454bbd1fa42d99ff76d56a0d37eede5505a", + "796cc42123f9a58eaa22cb3b3388e40160491dca862fb0a2aa17db03b45174be", + "79851d82ad4f83e06d475cd976b34d79a0884bf77624a6443337c585f5e7d88d", + "798ab8eda49c99f52524fab0e6c29e2fcea9c25bb7eb29e55d24f8bd928f5885", + "7997158c039d6cb8a0206f33a7fac43f12b6c05c50ab9523c5d319a07c9f47a4", + "79a8920832ba7837cd6d4b63760c7db8963bc9c4d5cc84b02d1656de6861cd11", + "79e246581d6965e0b6397f31780e76d0e681e72a94a031bd976890872e8c682f", + "79f7f98c4c91e9507352044f016d9373175a6410538d1d74765f5013de13efe9", + "79fe72616f57da92fbc0c265358909235cd86d5107d0e7d613adf8a67a868b91", + "7a0e515bd7dd1fc037d2793b5bb09d987967b38c7b0cdbaf484bdeebb6fffdc1", + "7a0e65a743023d8e6f5162be55768a3b850f64e76af441e3a5b99bf0528ec159", + "7a111a1887f6e3391d63b0141a7175207f02679e816cea6845b3486565b6cb17", + "7a1a59f3aa4d6387e5a5a2d61a4f4409428633616e29fa29d50ea0fc53bcd339", + "7a29256a151c5dfa124e5c339bf9e5bb74c2125c128b04d6fe46abf2a758f712", + "7a5b2cda69206c48b9e8d535cd1e96ea9cb9185fab4ce335a2956bcc05b0b115", + "7a70990914d64abb8b318d3bb2a06651e1832f21937c00651a6df6f061cd4842", + "7afef4f1d28f68d33ac3ffb659441e0102096da753d5c20969c536ea0dfed7e5", + "7b2cf8dd9d56b7ca7a7bfd1fa949e72969257010fc1d480bd54e043796eb182d", + "7b421ee486724cf4f7e562d916b247b7648a152d31428856c35c2c2db4813bd2", + "7b689b58e64f9ac6bee1eb14628884451cee03f41fdf2178916537c2b56133a0", + "7b6c5f005c475b264b2a7bc64a14240082a0ca96eaa41b4f9d59a0ed65e7d381", + "7b92e26c917b401a6baaa8cecdf723c0cd3d59666c778dca685bbc12ab9c6f5e", + "7bb5aab6218927cd29574273c5cdd63d39849a57c0ec9b59340302dac1ec5c5f", + "7bc0ebc0825f60d79a197e4d27689643d65798e598057d0ae33f72cd39a8d1ea", + "7be40af82f79492703783517086251c2be0511bcf1dd2b1659e4d627055acad2", + "7bf300a78229f582b67dce564df3158f02c03a7d17064a78d4b041ce41ffc3a4", + "7c13771c61816ecda59ab2026f2d2a85c882cfba9b967d5d41c31e338602eca1", + "7c249a595ca1f06717edfa3b35b2b487e2c2a47c29f75e4d52f4ac7625e24b41", + "7c5f4b2df36cdc780578afdd9ddf6d84a6cdf997977d0351ac0ff19bb403551c", + "7c671c670ce130f0d602bd6d0573ba85c6e754639d122ac4993bcea07d5e1b47", + "7c7f4330d058eff280e31afa750d2679d4e686f89f57a402ae5851cb33eaef95", + "7c809ee513fae51056cf753009b516a7d1c46af0c8b9d53761cde8467416e843", + "7c89620ab5745c5c096fed9a789fb4ef9a7038f3a03ac2a6aa033a960cea0759", + "7c90ac3e8b713181be6385e25e948a2bfe21ed28b0c882153787eb8be239c2d0", + "7c9aa9f7348291d8af6529b509632b76f898e46070c8b8cc5a74209f1f8e384c", + "7cf13a99136bc97bc57c8cbd7faf6c461f688084df1ec3a9d65f79c11c600491", + "7d4aeaf8a6c226ade1ca11ee3ae0217ebbe29c768e83b525129fd74895c0830a", + "7d653fa63d81af93ef7779035399a4e53411ec8a7f651256d7f721c1da327da5", + "7d6d2253f232f8dd1a23a77557f48814e176bb670dced172ff3ff7270115e2b4", + "7d83af9e9367d2ff00fe5848cfdb4ba392479eb3f1fe223f6828c6a735ce24e1", + "7d83e4f7e37ec9b9962eaf3c010021be13d664653d07999ca6c6a81e4fa3c427", + "7d9c7b07ccc1daa04385683f46ed47f8ff248fceaf24e67db501c065e8cb21b8", + "7da2911323411aaa29a9a089c7b549d57f54086f1831e8d11e614fb17e786326", + "7db20772098831748a7981a5ced53fd95f5e03e737349f7ac3900188cdd22f5a", + "7dd2da493ec5174b6763f9a0162334bf09572691c2d8908df6fa21d18cc67fa4", + "7de9b973552d85e3f6badbb2ee3721a640a18b97840ad0259039d6e6df85ff05", + "7dedf5136a89643bd0641830635e4f5a6ae5d68d3ecd6e4abc658a39da276f96", + "7df266244513d7042584a4a8e1b07a928fdcfd2eb2568be855f438b14502c66f", + "7e256ccc54272dfadec975e2192dcc8b899784a82c8eaafafc2320af6270edc1", + "7e4083c02abf3d1745e121c2404d91b27f2b0b972ee71c98dfc2fabc838eab8c", + "7e53515a2846c4ac3e3005a0d705ff1bc47fd08a4186c5bf3a5d4d342b7b76f3", + "7e648858b5dc20e69179df0316b44b36ba017448ff6bec6310da2a37e48093fd", + "7e6a0e320a76b2d7c4571302adb31af0cfc155f957166d39d44d4ad842d38c82", + "7e7078f9ba22a164fc61c762fb5db6d955ca1ca86a3b50b8961e9e5a3627e79e", + "7e80a3b28e467688135d12d509131fde13a4f17f81614a7b1f9d19379793e765", + "7ec7e941978523b6bb23d2fd3d55d6fe33879c6aebbd84fbb185117091c12b72", + "7ee3852676020e282bda9cef1c1045e25441647116ab309a214ec1f5f6f1610f", + "7eeaba0caf143763754abd5a641189db220b4f37abce91f1c66ea992dce96336", + "7eebdb6297c64378e8304231ece5291bd19ec690a37b2b58c90880cf89ad3bd8", + "7f175640c764b7df9f0cbc95bae421d5f9297fcf62160d65949b5b100557f9eb", + "7f792c5911a60663e0e9fa68a8f6be943a925cee621084cf3956ca175b27749e", + "7f823f60346810266ccd7ba91ea8984ca742aa3baf849349c8e0475b749dd68e", + "7f87c627f7f7412fee152b6c47e6eb8b6a13bd38fb9397d50a10f0be461e214f", + "7f8834cbc4e365c22d0e3918efe4d32714772b2c52fc2f836ba60f52662caeb8", + "7f98e2cfd1b02cbf7445f2f3c3fbe301e820dd82abfe4177d51c3285a507f3b2", + "7f9d4060748a069f6786af1e4860c6880d39e038135439b0b2458ad5d3c595a9", + "7f9df7d75ebce46939549aeb0324983263bb1d809264f4903d434452c7026514", + "7fb3d3b49f404b6314e29baa782768af91c6415ebd255ea3eded9dc3f584abad", + "7fbcaaf24c8235d541a5609f45fd7144db69a5defadf4bf6b518f9ef5f163e5e", + "7fcba24680c7ad95062aeb55090e10028546a5fd4f4a697d538593deeadabef7", + "7fd23e3b2cb3db7f81e7730037a1d3e53fc9774948a8b85613600590ac7d0f92", + "7fe0b89997665c24edc4173f17d379fb0241804761719324b77a3ad19cb134dd", + "7fec9ff3d09a0bcbe3db33b8066c3bacd1d09326c89e6e453e8c42ea49fda27e", + "7ff731d055995f2c7355f7a59174ff09d3ad70234c6ee73b5b29c2a0622d13df", + "8014b101a0e5de562df78d8629d92a6b367ecfce7c3f7eb968ca1178dcc94883", + "803fca0d27e485372ff299d0138f27c97487af7708745803e2eb557e40c71172", + "806f014e6df16a1654bf06eaf08ed3b60c299fc7b5d1c37d433b3a0ecaa81655", + "8096a2f2dcbbff46a37e430b9e9b6894d5f6b5c01c93a8187900dd7909ea0126", + "80b0d0b784604d32fdce1b467680d1630259424aaa829573fa50c6444cc0b1de", + "80f52225f3eeef206f805d640ed82231bf7c01b9f7cfee7ed524b1b299ce1197", + "80f9218c5e9118528da5dfd6780556e8443856b35e876c1685d2d25887b92367", + "810aa4c9218c0639ea1d9d7fd3811415d7444dc97f831a0dcaf75ef802242f74", + "812c2ac233a568a4da77e7b7487e4472579ac9bbd8ba095177caf7f1e2406b9a", + "814f2af21544b1b03d045f29be5b5d6defb02be970b8a4d85036944c3a408843", + "819ca88d90bf6c86b097fe09a8704530168f1b8931116e9e55affbaafd0ace33", + "81acd0d4f293b61cb77a0cfa090543beac2ab5792ca1e88447b16374fd1dc4cd", + "81d76fcd53d510136dca6c06bdd90619de32a48b1caa2da941bdcb661bf8af8b", + "81fe898b8769901447fb50c71d7d0627025ad9f7d533d59cc9f7d17eaaeefa60", + "82489daa4c05ab621452941f901dd0a684e640632f76836541fa3f45bdc27782", + "825ff15f0782c3ed5bdbb6412e3bb77aacf35c880840c8273142230c0df2c2c6", + "827fcbbece238ab1a8bfc63dc4ae4a456a40d215b9bb3f7c9383d49c1460d217", + "82826c41b67c800350648444e150c3a009db7301fb4c693f84f848977dd3e9fc", + "82d78f23b734d512517876ace50d7e7e19d9dcad4db80774ed3609ed8d3e49e6", + "83171042fc2b4dfc78224fb2b00fa8f1d00a038680000ea14462f14c604eba46", + "833133f91df4b418393445068bc2e9e93a065c463c8c8a25d3521e84f6414a93", + "834597c90083fb02ca00b90a38092f814fdb011897ed0cb4ee5c25bedb69d940", + "83aa7ee537812e09eb16fef58bf15fbc5efb929c298c5d5e1be5682bff8efd68", + "83b61aaa99d449b594ef00dfaa7d958c36d90dab01db14b2e04ee0db9c365445", + "83cbe1bf2b5a688a257ca8d424d4384d53a97da158822b9f086511e145839c23", + "83f7029e77724c8160a53346097015aeacaaaabb5e4e8e3930f21c82663783be", + "842402ca8d019c79cc2980fc26018ecdc2d715015f164225eb0112cf8b619587", + "84363142eb13b6d6a8c1ca2c282aeac21af2d69006b75733eea7b1ade7666d27", + "8463f76ac8b1f2ca1f2d0968a0dd799a8bb3d66f09eecc92341b45b3838840a5", + "8472c08437e9d065a8dfe76e302935c88138b65679974121599273edf2cd8d41", + "847cc4a4ff20a7cdb636264448eb18745c302f2ab0a10f1f61e1345cbe0bd5c2", + "847ef669e72c32f996d6a8245081aed7da92401654351337adeceac90899c4ab", + "848273eda2ed1190523b8ae401df6071e574af543b7d4237f4819ea8e795fb91", + "84bf01b039a60611ae8cb61ae6361a304f17ba7063425fa40f9836d7e76fd51c", + "84d4126224dd508c633f787902a15d111b6faca854ad91c470b008b84c466704", + "84d6d9ce5417edfffb97ce2f1947dd3b28381572a4bf02b041311b8a2ae6d3de", + "85026b06797489a856e2a5c1101cc0dcf0eaaa1b44ecc856f7df064abaed8e4e", + "8504c3f425c74ff723cb3a7f5c7e8df33f66e1eaa213cd2e71dca4f35491e68a", + "852359a7965660c3aa1296b56e4e98ecf4eb217a50e89ef12718539d31e0dfad", + "85374924744d63a016f40be7fbfbfcc00ef1baa374f4e9c31f35a039e4916d9b", + "856321fc3f1b3564e2631eb636aa5ece8bff0a96516788fb7ae44a68de39d799", + "856a22bd189c9505240da1131d3d09c44441396b58ee0031f0e69abecfbe5784", + "8571d2f43d6f338aa2691f71db7d7e10ead6c44764d7acfb62593f92afe32be0", + "85c19a7874cff0defb3daa6ed2411c6a300d40630127db54c7b1d72ab218ec8f", + "85d06bc43fa85e3a54b821385cf84b32ac25b95ae86a0d6448ec3a097436bae1", + "8602cd81454b46b26250823f90f178accf605c3cde02e5f940e4dd2ac634f131", + "863cb5c4d023cbdad947ac7cab27045fb066171a906913f4b8ff04ab9458217f", + "864c42307df55fcbe1f520b716e132e19f1ee4c0e4581b8fb51d1c8e0a2be428", + "86645cf08cbfdbba0c35ec11fccf41629fcc698746c542a776d96f1809aeafef", + "869892d5fe6eb6d2dbd806e2aa54b95f3e4087c76fed7f496ad8da9151917325", + "86c2413964f42e8db6d102682b3527c1d54fd1f36fcd167c75d1783ec9b2c682", + "86d5aa48fabe29b7086c8c48b33733fb5086507c03ce152f3dcff551fe403cfb", + "87251660f1c93538bd393e443bdea252c12178ad7b1fe3717c4cb456da1d5633", + "8726a1f04292bdbc4d137c58d3322127e3c73f977afe4f13da68584a409fab08", + "874aff72a30aa094c623b419ad6367bc9469095f544350d65af2c75f8fd78c60", + "8772efce2dcbd728dc3e6be593fcef89602e115e612aa50ede25ec3edab73f66", + "87a7443e2f33778a9f446b2e6bc81415c501381ad2f80caecd6cc5dee29a9e07", + "87aa334d871a911f774f07e7097a23960868f396ec4ef24f2d2a4d40a9d09150", + "87c60d890ad8c5c2a38be11978989d57c185a25969e84bcd8ba976a7f25070b4", + "87d1b82dd094c4acb9ad58a0ffbea7a5563a9936cd78f6e7fee778ebdcfcbe70", + "87dab3f34020532d6b44f52e4b7a035fd33bc5e4bab4c286faa547834a63b77d", + "87e20d968cd2fe30891243f403b3252a266e8db5fc90396caca386e302d0b8b0", + "87f4c982994ca15c1fc3d4d333ccf487b6548242feaab2b0dfdafda419bd0067", + "880779342113d7f59f2ac7f35283e3fa21c7f5635b19f6b2cfffd6bf6c9af087", + "881aa0facead55798fc83556e0e59977b06d1ec3cf3c254d86462c2dfce80736", + "884553201eb494d42a1993226344515c35c0d161c21d4a665a4239cfd45e005f", + "886ecdf27bbbdf53c2139328d18c174afcec3341a5aa4bd5ddc67e42a87c00c0", + "88a8b7550d896043b38333c24505d28737df4e73ec5fbc816795af28d9295048", + "88a9bdd66fc2344f8aff0b0d71c502b15dcc7bbe157324d6e40dd3058cc364e6", + "88b630562d5d85aa3ff2a6669b4bbfe5192067fc50d1cbcd58374f5d9d48c6f2", + "88ba13465bce1bb05049f99681cd787431e3a53903f0d3f51cfae12f95db0902", + "88c5f45e6084c9eb35b4377db489699d8f8fa81ced9a5c74341ae8f5217a0bdc", + "8918af679922da6f209ec9c23cf601bcf108176e52ff7fe2fcc20eb375fac7af", + "893c0f99b45c6e1ac223a4b470273b1a89431fb4ab140984aef93375e78e5e77", + "8981937b36eb34f0d585e60606ce56e8e67173cc3f04a22f46618d9f6e682e25", + "89875aece432ca420311ff4a23c57fd5b4c505e3c3b559165275845ae5e0e76b", + "89aae336d8ab2ca0ffe06a15efc2abde66c39f2d3f40fce5e85f111606b5e2a8", + "89b8caf01caefe1f5e873dee97904df01c28959a164af479fed1407c92add483", + "89c455c362afed63936ab400f5655351e685606805fec2d47dc36c40e15b971e", + "89c4b6d6bf0f071af66ee795ff42407c4fe150cb8e301997b721860501ab9a7a", + "89c84ad21dffffbd89fd8391e4a68536cd65177f9024c0fbb33ed0aa1988e9a9", + "89dc9547f563493b4c062ca9c1b56e277c0b973a53b4e11dec83468a4af1aab5", + "89e82b72fe11cbdc6c4685419428e65f1b96655d4eafa2343e891fd1f6429ea2", + "8a1f12fc3045a1ae803dd949931b6c477310b8636ed7c23b39af03b3dda64076", + "8a6baf027cbf0e268695248bd5dad15399913f37908c4a678622ca58b25ca568", + "8ae1576dcbfb6e37484071b60abefbee4a2d4183f8e8b21abcda98d615be1d51", + "8af1a4cecf6072ba4dfe2e726448b9d79c9350f80a171fc963e58b7fffcb0fd5", + "8af5596c49e5933241edd73e1c0349b84b28d9a9ef0bdb0b3f08dadcb3592d24", + "8afe556c423b4d558efaf132ca16b5fcd8ec07eab04614e72fcb67a700eedfe8", + "8b142c5aeb5e5c2ba168580db094b7b6da729837abe6199de738f104555e63df", + "8b181304e3a183cff73e26e12f68004c48cfbc7f66edb371622561c9ffb6d175", + "8b7f3e0da8bbb4bd381dd5faf22f3506c22b01f117a180f56f2060a15f7f3128", + "8ba5cc41b621f68b1da7a950361afac2ebd406bb12f8f21e1571be2680cea395", + "8bb3960b9aa49d3c7daf93c0b27a2e82677fe70954a5b1b8b40ff51e01f7b651", + "8bb4e371cb956240c12c6824fc1c42f113160973b79c22572e5140a5a29d4542", + "8bb8b135de8b7901b1b456ea52976b19d15090a7de2bc5afc5cf0e408102dbd6", + "8bbdea659991c41fc41e5d01605860cd53b22535b761954b847e34eb452c1bb0", + "8bcf7f0413754b850c9b24fa87068f48e73c559327425e1bfc82cae6fc4d6b89", + "8bd8787b5b3659b6315c160145c30b3aa5f799d10b61624b530ff90396a5b7ef", + "8c26ad6b9984963d835deae72d8b24856d38fef889bb05c10c85b3fd7a25cd6a", + "8c379436af1581c0c376678bb99b148d4756e39e9446cd804858c12009b2d373", + "8c4099078c2c0f6c69528c976e9308f5104108e2c6cfe95c3d2399672f5279b3", + "8c45266d075cdcfe5d59cd03fb88bc25a7a6ff511ef0b0cf9ae47fe5363b7d53", + "8c4cd3870bc9dc20ba127862a0ab3aac857ed1a4d0e2340dc7937eda0d115c21", + "8c4feea60b5188293e370c26dd06528cf6bdfb30518b3869007d3f80bdc3ef13", + "8c7148ff82aafe024eba8a2ebb64d6fd836fd75cbaac428abba1d92627fb07e0", + "8c7bdd8a6c39e9ac5222fb2b16861c1b24367708dc596f1bcc76c2c7e9c2b578", + "8d432dfd008cd8ced880729bfc45f3d548bc54922e5e6d4bacf680bcdc0a480f", + "8d5d02f66be9555b666419f16215b79e6929aa14a678c08d09d2480bc2a82b6d", + "8d717570e2a44975ba3d53e6575d4a295717dcf2564d14d62a2089a372a8fe34", + "8d72bf24d3835df975bd47aace23e91fe60bdb5cb0ba997ca78e52fbad343414", + "8d78008b2455a61d192635e692cf968000425c1b87e8ff46fd02b3816dcfae23", + "8d878504470b79be572a45051abba37a9ece730557e77120508eeb24db203851", + "8d96dec83d0411426e3245574e8c35f5b73c8df79c1390b684cae3a7950d76ae", + "8dc3568da85a0ea8b8b2d7c0bdf1cf458129cff09d04cbcb860575bd5d82ed95", + "8dc80b3bac6de50377d73b41145b48bb3ca6d23e99fc278cc28c1dba42bc1c5c", + "8ddd2ff6afd3e415f3a1316579c35833de765f667b9501e7e42da54f9eb4c7e5", + "8ecf4a9b224c936daa0053ab30a1858c67e74b17f2c82725389198ba26733706", + "8efc955a5a8e460ec1b481cef6a4a6f51552e6e1e49355f9feb912a0650757a4", + "8f030cf10304c8a821ff762ece320e3489870be3b40569b6663cbfafbe45d90d", + "8f03736ef3ed40b3dd7126f1c5b63059ec0a5b2ae71f67fb7d97de2fc16d2cf5", + "8f145b6dc0b3004631db47f710f7ab74a258adabbec957e85f67f15a99e5ef7e", + "8f4cb1bd07998f4bcc079c05285b81793fed07c094cb52e506b5fe2d654575d5", + "8f529f8b797d5cddfb9dbd137631316ec57cc23c6b3cd5249562e7fc8d8d80bd", + "8f91ee16ae905b7524533a6fcf40137005c080e4608becccd2c865d1e0933420", + "8f9d44973e1d625174356a6be2636e7f56339aa695ddc592dc65cd79f2355954", + "8fbead8e6f6dad8d2bd48456546b24c66bb9735ca70566daafb9118cdc7c83b5", + "8fdd16b6d541b7b81911d14d7f6430046b8069fda31d5c512e78f371f7df215f", + "8ff40cf299ee3337cd7fba77cb92fad05cbf3430ba4891c9545793a70b3a0384", + "9041d1ecb1c173f04808b57a9819a53b3156c1fe87df224597680c170297208a", + "9097a466fefe8548f577da17fd853422773485880e2d7c6085e8950694a316b1", + "90d310d3cc0848220d1011df2ecb0ef6045971926c923b6fa9ac2e8fbf8b2b50", + "90d7b9d3d86e904e19237a6f574643863bbc9146092f09a1b63fb18bcde427ff", + "90dbc96061612277e1e925b9beac596373152e74125ae7a9213bd1a1f8585e08", + "90ee3c873af25ebd77ed889cf4ef6410bb4b9eb00b32572f4651e93bd3b4743f", + "90fbcad54d4abeb3e68dbd7c4b923eabb630fc065a70246b8796627b3e3a1e96", + "911b8543dac7382e9e25b4dee5ce3774927cf78f57cb05eabdf33b3cdfa41e65", + "914f8bfa81a2c9fe0b168c645bbe45399099ace3ca439353229216f9a1b99587", + "91518c76c3bd6245cf1b051b958d8270e9d81f0373bfc56467dc896de7fa856d", + "91aeab64cf94aa7690f78bb5bb2edf91d64e5d8a037094b19a9c3e5feb183a60", + "91cabeda5e16e572b89e53f6ba3988fd8416fa7b03653f06969c126752dbed64", + "91d65e83e4fa2e40a29660e8697d3dedb1894b25601c3bb80df559389a9ca07c", + "927b077b9994f92cc17262f57066e2731c19fdf59f1ad23e9a187ad8a8e6a21e", + "92b4ffc72368d645dc6a5d181ec6cccaa6b2d5733d289a6fbd0d49da064ad778", + "92bacb8e4cb433020247552fb9c6cabd6455018488f8992ac6c202820aebe864", + "92c9c9caec17b174b853604267805f1caaaced45888eeb894cb5f07c3da60bf9", + "92d6eb7919388ebad69f0cdc179024407f2b0f10ade265a6bb82ceb3dd533319", + "92e08317b5264a5c75af6bd32f8f9d19d35dc2ae77a8fcebaee74b6349e2053d", + "9317c21bf5f13c7615b404074d4bdcd514250758aa7dec348b461e3ba7de2ea1", + "93717b7c08182de46766bedabbdd152710832ce74aa1e15d30d2d66748a74660", + "93d4e4a8d37483366afce2294a63b8e94031c2eb2595f06809f3fa154b8973bf", + "93da984a0282e86731dfcd961a8ce43b862a14c42d5b5d1e8874ffdd200d30d5", + "93f71412244e33e67e82e17efea5d58a9ecead2928ca36ac1e8cef2457f09296", + "93f7370ce73039faa1adee0d0cab9177772da8cb361d27ac1254cfdf74dfdd57", + "9433d8a34075bb592cf9c36bb1be21da08bdec9f9ef0718a0b61943b321b2098", + "945c821f819b42bed83f5450d180a86c23e6ce00695f7a18a105e2980b2ac2a2", + "948738b6923f5c6f6af94eaf47ada5f24436125e7b2d7532f2cb9b3f8a1ac64d", + "948b1fb3a88a247eacfa7863b50f8933ec9f89a59ce7cc3a17589ca825f5ad76", + "94a3567ffc300b6b0f7aa8e3de115ea0f5ada4d64e2184d2fc236c39056d500f", + "94ca3253fac86271dc0564b9de6cbeff335102bf817c744eb6d6434596879179", + "950c886241c044f7515eb00ccc25b1e9603899a3dfb4abf94727a1476bf37b05", + "9540e5430f9fac88e79b1e21bc8252affe0dc5893218218b2ff1839999223005", + "95473d6a2079786547cc70ab7f130c85e58839a865eebbcc11b0b75730e53b25", + "955f30c3eaf38525cd9d5503c4b1bea7b23c299cd6c8a6eaa6a6caa3748b24cb", + "95b77329d4972e58cce72f2ea588d614e3cf39cb424cd628e90e58fa2868a4f0", + "9620c1973d15d900e0cdfc235abab14254b5d29f240b8382ae9425fd47d18dcb", + "96290f319d1f2eb52a9f427d1df9bc5e5c149094606c64f51a172bb8111e1d96", + "963991dc96009d33a16333cd8dfd1995a7d274957508b25bb9f3c00683af0a85", + "9648696e71b0f84e436a6dd018c574b57d85ef948f6d5cc577a4964b9da64f68", + "9657dcc5c71ffa5ec11af1b35558bacbc681beefaf10373f74dfae9c13687f6f", + "96848b6b4616088854f302f4c39f7879232f208212d4816b6390ac11fb0eac1b", + "96af2b09a6a25be1f7f654be30ff50d473192cce28c69a5e89a1016ef5371093", + "96deb6e25f941a15a80a373e2ecb92809fa08772ea70db16466a16debe9842fd", + "96eb4c60b1ed6d1cdbb6d0a6e9df4e6d3072c037f5bf8fbb08d6f49fc0e96791", + "970c5fae48051e2e51273d468c630a6bb8abd4abf92cdc1e040ab2e5fe641145", + "97104c8814d8d54be64a2effe279e2de76ece7d9946d17f2bc721fc9ead045fd", + "97af82b0ce5128d252a339103965d1c99eca7a42ebbc19da35325e2cd716023e", + "97e800caad83fd0ea5bfc639fd4764cfe8a7a0fbb1268176e05015bdcff4ba19", + "9807f31242666c19189d7cd3d7007e6c617b49dabb548b216832e72d253341c9", + "980a7b92dfc3cb82bfde2b7660855cdff3a4960d863818224580dccbcb82a403", + "9811203c2c5bfc2d0a16b56c0aca2650534579d776a9cf0d2a949816af975180", + "981c07276056bc24a9bc32b733dedf96f088faf23bae6f053a5381b357f4a639", + "983888a1b90bcbf0f9188f2c89e0871aeef8a39bebfeac066c11418959c43089", + "98701e77e0b6c6887b96faf4f415863fe540bf5ea720db6c1df7da2bd60475f5", + "987defec81908223dae92da9b57746a6942845b340ba3f79cf8538dd83eda1fd", + "98889046817a4b80a4f39a5cb29ee9aacda0898af6ea7c02b82cbf7c1a75cf85", + "98b46bca82a90ea11dbd8c72a45f3697ba61059d391cb02b2cfbd98331ed4db1", + "98c28626cf398fb8e30aaf97dcc9b33e1eda33d6581088f76fdc2c5cbec9c375", + "98d35a6606cf800ca8622e8df0beb098445ed83ac11f1a14eebf21dffcf368d5", + "98ffdad2141e58ba3f1d928a7521c82d40aea01042acccab30f527e31d6d0c9f", + "996f5a33720e295a48e74c5316d54bbee57d280916362ca8f77e5d519db411fd", + "99984f7515c7d2a04b908a483c70a7cb0f2ca25605858592104a41e709665b30", + "99b4c019bfc6e68eb07326258763d2a30e257474a0da57b9791e6531c62e9e25", + "99bb9b16e809868a3c0e8195dfca2107f4bb888cf7a569b8e62a1dc8813d7323", + "99d4f33215908f8993f4002c40e20672c3831f527217db39469b57ac7bff8861", + "99e21fb94cc1441231170fed489593e95f264396012ce6d9f1887a4c693559b2", + "99e9ecd54eee0b8c28f53c9887b0f7f6e76a07fc2bc3defce7b09032c47fd533", + "9a528a5fe792a2afe2443a7f889a837a87c2f143271855c941c898e5bcb0b671", + "9a5f1226f926f45bcea2f48d1bbd21fb46b8607b63fdd464c5d429e41685a80e", + "9aed6498f33b8d9b4030e31861866f063c68284ff77e0d52c533eb19676a51e1", + "9aef46bf4b917c607a06ea8f84a33fc09b03e46d4161aafbccb836359b1cdf20", + "9b03128ac34991380aaf809fa4f6008fca9964dca37bb9a141af5be04b2dfd19", + "9b1c963738e61cb6be8b2320ec1f5d55b632d42de6852b77a1bdd26fa590f2ec", + "9b72c6281f55aa675ab785fbecd32f58bfed0a6b427b612b0872230734a2f6f4", + "9b93401d713707902cc42358969ae2e8a3f67a7abf682e6c9c662b6a4cd02c8e", + "9baaeff6a1950b08fe74b5224b5c12994587023cca5d9ccd79f2d5b9a9ed5961", + "9bd0c6c55c6ff96e5f018636211870a2aa45bd2f53041e5d6103e05afdbcadd7", + "9bda8160ad9e098a3db0011e26abcf4ae8137edffc8c7484befbf9c861f034ef", + "9c60ce7d1e20540bf17f23636fc186c7084acb8d696a47803d3a5667bfa20589", + "9c8ab72ebb2efef086b21855d5a4ae3f5292616e5a9c876f8bec8e28da0fddcf", + "9c91b3f38a7c6d1d242212504e68ba0a67a684a0d485f15dcd357bd19b411040", + "9ca581e36ee1174533857b9d926754b77009f9080246afaa63d9748d2d066095", + "9cab822d1fe9746b3beac1f5de7e24436882ed4a39ae207f3c9a51cf5b5097f0", + "9cf21be9a626e2ec27da045de4d7f4cce775ff1474fb7b4b1d361407f15bbc18", + "9cf6e80299c6ecc47c8d79c3c5f98265011381bcbf756a12c629ace25d71e6dd", + "9d20f9cd2e12a69400d1753f325f197a34cde2feaa0a6a41cbc8d114cb24d407", + "9d4d421526e0ce02fe621aefed63966502e7a6f1aefd76f27d7c5e7b57760626", + "9d5312a67b24be3aa7949b355c9627bb9982a017b95e263fb65e40ffabbbef35", + "9d5fc103de859786dedacc04713afa5f5ec908af5a8d4b6f654e4f35332f53fd", + "9d86dcec27b7472f83a0109dc136502848b7506e05e3d2b05a983f635bc617e7", + "9d91de5aac37435138b7a0fd208877e7f22c0f1ec931edd49824d81790758ccb", + "9e090e14941f35c1426fb43e188ac85fbf1d1a199e057b80de662be34937b5bf", + "9e0b3c8611c3983453fb5d666148abeb23e5f988cd09a0e5a34f61fb059d6898", + "9e0e34f721896560c995500d5251f2fac8c4562403cfb13ec4a2abf496d8e1a9", + "9e14ec38f8398efaf34b5b15bba9e47ab2019fad693dc51b513bbe274d9d160c", + "9e22c3ed476b2c040f1d9357094d17f92f041d539117569d75326ca8959e42dd", + "9e34c2de58f482b4d4e2a962ca938c8513d8555258088d2815b729ac114a5d10", + "9e397f028c0b7df416cfdbdab53e0b8a607d2348dd807343849ccd811e6dd429", + "9e513c1e436f575aaf5bd5c7df54304d8390ae88da87efdd3c9df0dfe3b395c9", + "9e81082b39b55ed18042c2c5371a652ed32337f6383d9e714cfeeadf99dd5851", + "9ea47633f98cf0baacd4966b055be5d3e71087b274b18821abfeefa6d304550a", + "9f06acce477a36f418f3cc826c27ccb555ee6a836ddcb8452cd76a766d8d4565", + "9f23be74bb52beb0da75166d46ee912a3ee0bf66954efee53a92ff558b0603ab", + "9f73734372aefdeadfafabfcff94725115e0cf4f9b15c18b6d671c93bc45d4b6", + "9f965d1380ea525701e2cc137a76dbf657d135908edb119b5ade89d517c642c1", + "9fbbc8610042a490eccef51f5d910410f875d56ea7ec41b455ce0d7f6fb76e1e", + "9fdb058c747884260689eae9bda96e5499101e7c85685c7441d62133692bf1af", + "9fea119b522607162bd0849202a5af02fbbcfcc9630713404eb24acf7fe44770", + "a038947bb5d061ac5c76c52746b1f3a3177cac57b6990c86694207fde7c6d52e", + "a0b1daea0fa36875362462eb688f7f3fdc0d602feb9df26f1ef223d31cc2da74", + "a0cc9e51bc6b5aac329313a6901cf0320d67cdc2ade26f4db2d279ec352b42f1", + "a0ea2d1b59920fdbf1565c05bf05ea3eb49b8a893dee51b93bf7d443d947b85e", + "a1240a45d1455232215af2dab413d40b707bb262e4183f3068350332abb33b22", + "a16a139ef9ca152e81fa85305fe2b5cb805153c6bdd2ea89b48a0f247ebbe12a", + "a178707a23e01e9e0d753982ef537e8ebd27f10881be88d117ebffd125c84112", + "a1828553aaf5d9ec4efdfa0a925fb03449f6db4b13c376691b34a6f4775b439e", + "a18640480013082dd7e7a22d73e66f35cc59f1fd1b93f5ed3de1cec43413bd5f", + "a1cb72da3133be47ced5d3fcab95755ef24d418f76f4fb0b4d47e6e8ff89a470", + "a1d812357c1be25f45e129a8b8635d6a2e8d8b2eeb0248984bbd64459918b28c", + "a21d104b5dd167736bce9902303d4ba1b1f02237c9b4a5f6eff0a7439042895e", + "a23825f174104e351285c36f589dc37770f5bc835d7d0e920e09573b704e62cc", + "a24336324a1e3800247283a001ffbf0c66ba08c1bfd2194aaa88e91da4c2a16a", + "a2456c53189fb0ad5fa63355cc5c3041e80729e753cad05922be0d65b48d2fe1", + "a25f880fefbc75dece8b8cfcab410d6383a6e87ba0d63b18cacf501ef8d0f4fb", + "a271158686e7f14f011abf7ccbfa93f0b0a444e3b356c8f623aa73fe8013d82c", + "a2a7b5c40a898db6db90eca16c9f41edde6feaadb445b78161abbd523d4ee84f", + "a2b0b4b8b334464890d9a5d9d6f39b05260fab4f43ba90faab4ee969594848a2", + "a2c6b407b6b42eaf49eb9077bff37c976cec72ddf64de83b95d34b75d5e09718", + "a2ec6a6102628055650d617239939fcbe2324cf862641123946607ee1eea542b", + "a335b1d36fab56c1b1ccfd69f872f9d3b702b312e2f8d64b3b63990e2a02b32b", + "a36b7eae29896149c7696386376724ce04884f2cf9ae685a42133a31bded0fd5", + "a38fe3c04414577e28a405c41a2ff11ccd80b808dfeef1bc20e36264da1368c7", + "a42c6de83dca6e8a8483d1725cd6a3f78811ae75be16ab8afa4d9a95965f30c6", + "a440cf910bbfe95dc9a81c122775b244009e5fbea638563e5760e4e482038dcc", + "a4776777ba6678085120b1ae5cbcf2d7739848f9ebfe6cb6d7b7bee59653964c", + "a49914e2c5326aff76aa60d7f5ee3c3640d622c9e8b4c68b37c3ccf680fbd7c7", + "a4bb7e0ab08ef465f0b663f20a013ec927c25d17ee3d4111eed4044b1d1c2980", + "a4c7c3bcb82cc25073907469db3e203766cb08f4cb9e2396165243e0708db374", + "a4c80a95b8f850c2dfbf410aa2f795d233320f9714d5237e5120063e6b5ee2c0", + "a4cffc4bf7e71400877b3811d34b1038e95f6053ee7be10b8990b485c7112140", + "a4ee41ba2c8e777a9555f962d285865998c7df97062acfdb9ce6caa79aa76bb1", + "a4ef3eea01ae764762e35970478dc48efcb323c6465751ac59d5101dea270504", + "a546c967ddb57c8307ccb939b6a2be51ec7639d4678662bb377661aaa3c80b38", + "a56d066f7b27d828d979ac0852165457388eb9fb7ccf509ce55be322a79336eb", + "a5def6fa74e09ff25a2be14c68cb7685ae5b85c27241afd2204da0810134086e", + "a5df13d7ca9fd83e65a0a5795994ed24336471126925fb292b760d5e8bc7ea98", + "a67c76e319bee136016fec9557c156c4c8715121226474fb21005392a92ad059", + "a682b5600e66458787495ec446edb276970f001498e62f2f7416d7445f85c8cf", + "a690a4c2865eaa72e8d5f2985a25fb8a77c688b977fe7f670d5ec3f1e41a9285", + "a69ca4c0ce5b896dcce10da086405f8d109c21fdc48bc2f174d6da28a516739e", + "a6ce6ff75d73dd15b9755db511202f12f64f6cc578ae887d28c1b5429c60df8d", + "a709d08dbcf24d3d86edf9333fb4b2f3767c49860619eb5844abebd7e8995130", + "a70dead3d5c69f30999bd47228b9a51d8b6a03009aa88510f1862af51e3d29f5", + "a71fff06653ad8cf199d4e6f37db51a88391f423f5ff498188361c1e7b6d34ab", + "a75e488dc3e774199bc944e1bb796138ca794094f25e6cd6fcc2697ff39752e8", + "a7acb1e2a2622b4f46a3c77fecb918f935fbee00f4698ddf64ca13299e0d779c", + "a847b4f5daa6a2730017bed869d6fbb7592a82e92d83e337a9f379a7bf071aea", + "a84ba5d88853d9f39f58293080d7675c8a20280d0950ed4b34f403b07fc94294", + "a864d70f399f5f45dec5d6850ee6cac2e405af6d81e08a0167230b795711345c", + "a8b1d5189fb760726fa336137e4278c0dbdd9c6b263f24be466d4d2e04af99a0", + "a8c190caef1cd42699635f6ab8c273c11a9c540afd96e14745854499dc90747e", + "a8cc6da58014750451c1aba1cc1b8ae1eb1f439ee203d1462972e4cd1238194d", + "a8ebc50ebd06ac561c84209d42e3395d57b7a927aed1790db3217dbb6d261ab6", + "a90aeee371c90fa42d40c1b41c630ec063f748040bc5006fd9c2b88372430d46", + "a90ee73b0f090b36a3b906fc3ba254d3d921f539e4fdf8178fa2764c3c515ff7", + "a951575db1c5632e10bbf2f3012328ad5e9db87c8c3d6642165a5e59c3790092", + "a953dae70fc62946d83ceeb303e8ae3838b0b0930ee3daec62514a2859a0f5d7", + "a953eb424669375abde80755e7260407805606f07e0a2475802425430ccfa20d", + "a981bdeb4c49cc8f338c986c9c80e5b0fb4179c88f512d472dbc4d32b6ddd837", + "a9baa63d7dab29028f641775aeabe1073f5bd65eafa0395caa9dc2b473acb87b", + "a9c76a3cd3ac63866e2fa0680a392bcc7fe67d652a27ab62c2f3fe50d07e90c3", + "a9d5e8a3bb7e612f9263323961dea5f8e066e61fea3ec027af77816c7545414c", + "aa055331e79144b7c663468f03e6fe569cd36baf77fa333e08bd6a7e3d4f242f", + "aa1d01d96bd3a988acec7c407d567aee6f731c70a4f449796468e2883a1a20c2", + "aa2c77c6f377cdfb433d54423d96748858541f03da3d8943586a1913416fc9f5", + "aa5b0cf7fa87a46134f260b548a74341fcecab639ad38e324f09dfa171c272e2", + "aa5c9f8b4c37a2dc2f29ee3e60e2b49487c0561c5a698c0fac5ef89eafc02d61", + "aa64551893a506054432e9e661b5efb425769dfbd9965568d21ad1eb10dff7d3", + "aa6ce1d0818f17de327f41b9a0c29e1cb143e49f5a039965bc42659ea7492197", + "aa97ca3097e3987f4519063fc8728c5b3b01365ccaf191553fcd10197670106d", + "aae9480c83f3cc084dca70e76f618f82b87bf2fb2e617143edf01665ef506093", + "ab62a849fcc9de65ce3e9f885d959a2b3454622150816ab94e10c0d8d5f0af14", + "aba3af84c0c932cb2ac57217c3c01c65cab49ec15f8773ffe5bbba1c7fdb4051", + "abdd082537018e878916cd25555617da6b91fa387c23f45b53fd80da215f15c7", + "abe660e96896609144d8f5994cae6cd441200a09e16a5a9080469f3023b91e86", + "abfc23b6077fd0a9b69141b3798a24ae1bf70e8a6e985057ad09c3c1c9e17246", + "ac2b9a3dcc45274eb9f2cd32498118f539bacfdf5705c3a9a60d68b8be68cebf", + "ac4a30459d90b1ab88ed2c6d96d1e6afa2aa1a59e708ff7350de36b260e4bd64", + "ac91bb18dc539462084798d093ebabc3c38ecdc88db10582b97bd1ec4063b1d0", + "aca5ebdeb25a6839fdf7d99c3c40094535dcc69fd098c51ea7e86f870a768388", + "ad1fea799f9b928ae94a7a9f820465a3d54a164276dd68f1fbc771e2458ea95d", + "ad4aa588253fd0b75effba32d3f3f62d2a0b20499b4b2ff20183c9e9dce8921c", + "ad4d7e703b7b024c40df24238902ab1f66c74d3982da912c354dbdb120b00a18", + "ad545207d62ac1a22cdee8252b85b32924c5e16627dd921a96a9c327c21ee4c9", + "ad9016fbde8ab97a81bc34a102503eb570e2d1cf3aa49de6d37cb6eb272261df", + "adb558ba9f031272bd864ed4c02cb66079249969b4a84714d8e904450bad301f", + "adf7df02e5ca3584edd4f5cdbcb27252efea4db7d58d67f833789d016a911c43", + "adf817abe34fd96eb5bb476d4ecd70d5838d3d1535b1a1fa7f89e48deba3f691", + "adfc6ca3cf146e6b70ec610f433048f8f2f61e4b0a9a61b00bcbd548aeac7c68", + "adfe8570209520592870637d0c6d89dfb53ddbf0d5162c38e7ffa79c154e6823", + "ae282a9feba3cb183bfc8ae5f3029dea7f85af44579e140d29111a5ba56a8a8f", + "ae4a88c7e31198956db2ffbb52fd170f8391acb7af4b75f0f7e17c52ed28be38", + "ae5753a98c376c9e7d29d50ddc0e0a6c9eaa15a6fa983e15fd5bf6a70dc1cbb2", + "ae61bafda0632145d2c840cbf5b9ee880f5d515b1dfdf84255024d3aa9ebb866", + "aeb84e1cc03533df9a567948e3096a4651622ddb8c65d273ac6ed9cacb76b27e", + "aec8370a06a5d1b6bc64be94fcffea3bbcb31bd7c3f87072f6eabe7c2f74353f", + "aeda0d224f68d7d2271fb38d4cdea0bab20c6d678bac9e7415be86e1428a8f2f", + "af0eab4a7ff3a9f565c5d441ee48c6728a8d2572bb47ba29bf054ad05c830fa2", + "af3d43fd362bb702008257f845c0fbd7fb0b247038bf11c8577d02b448ab9644", + "af40469b4be5f0f237a06204c6410d570c9b1713dd294125dca3fd71e5e67947", + "af93c5d79d26229544af7194357c4a6874a024c4f513e73ffb9b6fff55d8f800", + "afb9f2bf7e8bd33d310cdc0db5a756e3a4783e435816d67a201b8ed3f797b1ef", + "afc38038f2d082d7566ca44715c132c0d531f317a1a3d54a88178e187124b18b", + "b02963656a19f9f7dbd1c3065b907b214dac7cec8ae7b0ca8bc882a31112e43d", + "b02f7b28daa1a2f7aab7632e16b8cac778bc70c3e8a1b92d2d9bbf3721e3a4ca", + "b03c695f732f3d794a58bd6ccb30774a7e9f77e6335038a1b81997cfd39be8ea", + "b0641b7d679fce235e06223f2fc03382a110fda99b2624b408fdf9edd18c9a02", + "b067b6ae3257a455bfa37d6317ea11da0285343065480c68b88fdbf1f564c456", + "b088b8fb0d23433e44bfb8495d0bcdf14a90a8688a40a33d77ffa1434e4bd021", + "b08f6cb49d8f2c0cf7fc4529717ef050a3d5ac48257e43bcbb21ffdf9ef56721", + "b09d4968b098e1be873a624d4735318198e28acc81516d23ff2a718d5e05da0e", + "b0c9a587407aff5c3b96b28cb92227db0849d7b1fb4dd54ed01e9e194748fda6", + "b0d272d65e86b83427688ceb352a31af5bcabcac115cbeda72ce0cd8df37c3c1", + "b0f9d0a383327c41c70b0d99cc1e17d9e4b0f1a1766c4b1c410a847baec0eaab", + "b1188a1e16f91b3e25c4437f118bede028ee084b2d52ef9cc226ac04a33076f2", + "b148ce2a75b2aafba33d67b9414f1a8b5a355f84f16d4c0b6438da02ff71d9dc", + "b14c6e982b47a9f7cbc83e2178db724a8dc1a43b96d54f4d28ccc5f2a03de985", + "b164746f375cbce5e956785bb3571c970f97f0e3ea06b58f63e6f031ba2b1d58", + "b16a445cba1d8945f315d29452fc1bb6b36006f64c9dd3a71818a89102c48dd7", + "b16c7b13be12df7a0a17561e577053f14140827fbe7f4e220b1d3db6df56a2f0", + "b17984a1dea67fbaa09b778d2318f797f0b0096ae071c7380d13b2913befd546", + "b186db261318ce9d0eec255ee4aa33caad5d23c3fbfcb7b0cc74a01c7ccfec43", + "b1940f4e9449beaa5e5553b1396d1bf7ae7cc18512cb8927bd44996302e44010", + "b19d1adb2abaf3a1939ab6fdd40020c0fe19361f943dfba64dee935859ee905f", + "b1a5bac13634f77e22d4f4d18e98ea341b72743117f0720cb05cfeb06188a979", + "b1d03318fe70818bd4a120e56e58c65af108ca8089ca38b33aab4d2f49666b22", + "b1e5ef44a5efd3a383ae8ccbbae9e741e221ea1fbcc5d05bd23ed1f2fc3e5fa3", + "b1e84bde7f97e4c7525cdf0971c26f570dfa7821392c3414da9bf6d49989be9a", + "b1ff72519a4f541cf01e4ae32345cede5aaf9d36c67d965e8981c26a139c0460", + "b260bbe4a2dff2585030152513fbd0eecfa57c138379e0087ea6f906c03201a8", + "b286fa0eb10cb2daf7436686cd9582632439c280b04b6b7539e1cf79a9f11f93", + "b2ce9e4d70b36c0cd47252751ed47b8c7765839750ab046b270f863a82f19fa5", + "b2efd303d551237f7c1248e6619bbf925479058a16a716f102d67b94e494131b", + "b31cae085781532bef6eef09b4bfa31dacf2bc02dc20fa2d8d922a7e9f985f43", + "b322dac84aec62b55e85728718e2a0d9ddab14bfebcbde23f2eb6b949dd4759e", + "b341b490f5f8c125d57f9d9633f8af89f022d09083e79b096a593a6b23336d13", + "b34596b6121b6f5302625bfc13325f97a351ee3824e7b08171554f043e7213e0", + "b37d030e04976ad9717464b22b3854a50114d068ed51509060bc73376a484116", + "b37d200ac7c0c826ddce7614f9224f2365e8ab7732483874f4ca181394c2a5e5", + "b390d84397dacfca64a5a0e664c32603b4064baa103beea8c699d807793ee9f7", + "b3a19406b80a96b58ffb2186de2d9ecbcb3ea04f0079be0b038fc3676d2f5763", + "b3c4733af59558e62a4d1f9f5ac676e225e448d7cf70c381d9a3927d5f49c450", + "b4090022ae5f781ec0846b887e8576989db3ba34f51aa66d8b619eb4ff73f665", + "b464834e6ccec4397ee4c4f488c26779ddf6a067b7258b25fcd31bf1d61c77f4", + "b46d29e256ffda81cd2812cbeb60119d4c3521ed84384a74970b74876f81bc7b", + "b46df5a380426793e33ecd6a5cf0ef9cac8fab77a13a0d9a1dbf80ad98ff4f2e", + "b4d3a904bae484b4816a592f36be4cfc175b6af30db3d5fdbc11d3692a42ceef", + "b4e84424287c53a682e4e2c85b57e730c130cbdcaeee4c5f08e481b0c9767469", + "b5731d35376fc114009c9ecc6feb8061e4326f8127de14c332ae7fd0f1c8e8f9", + "b5740ca18f0bc9a3f6fa9ca025e89a4f4ae9c571ff67d75890cb3d6414463e69", + "b57d18041db98a5c8a9f58f983b9b0d8771bbfad555450164542942faa0da287", + "b590eea2c394badae79c7964c2752de3e07cd42725af5153ad6e0f4f0f81560d", + "b5af2eac889304f3f2106cc0cd4e4269657876909cf0b1a6a5e1422caa69c026", + "b5bac383570e16b15b2e55caec37e1761c6ec382a8b744174d29687b4508a94c", + "b5c30c652903550a6533882aa07c3c99df4e03ad64056ba2df1245cd8d2556f2", + "b60f4d4880ebd0615405671fadd4969804cd9ce9c249ff4af97d235871d45324", + "b6266da8eded1619177f4f8b08f28b6dc348b362741465078fe9e3f05d516da9", + "b647017f986b484639ada0c1521f4cbf3d09bb6d852f17be1892fb2e22bad569", + "b657cd7a60e964d2f7b07dc6eaf7e1a4e91f6fb08c90180d3dfa1792260fffc1", + "b67925a01180149949620a0dddcc956c5af9cc1579fb8026c717ac2c2d6f81d8", + "b68f5c89a885438b835515c63cfc0cf094b9b480cbf4067fabc35fc73e54ae70", + "b69f06e4f6f273d7a8a6150e56c64e6ea920641fb286e654a6d548a0ec5e3ce7", + "b6afb900709a13953b9833e507e5b0c53761bf68ba514642a45fcf3fe7dacea2", + "b6b8789681c6078777268a6ff1098d67a2176a210b052275e8bdfd378f0b7be7", + "b6bb805f019769e7c22c3099958cc7079d8261a3ceb6760f2307b824170a06b0", + "b70a4c5b5ad5ef61ce3dfe870e9a3f04dce28b863c72efd2302aaf9d0284dabd", + "b712f04ad27bfc82a95a0a9539c5cf93b2ac2b0f938ef593d5301645a3dd8232", + "b7358990749f89a239d0c35e4454c1f2256f2696f43c0b9a6dfca0a01e8c234d", + "b73736da3712e2ac62d7296642cc9b095a26ff278454b9c800c8e88ac19b0c68", + "b75225cc5620623a5f0aeb1ae56a99714cea6aac6ad220343d4ac6fe3681e22b", + "b75f3b57a8761038dc34e8c06242932cd7a3a42a4684ad04c3d0aac1c8e01f35", + "b79f8493c9f22103dc29fb01c9606f726684a35e81826d67290a251a494b2d6d", + "b7d02801cf1eaa121d3e3e958f782edd778ca42dd1acb0d63030f50b6ecae0f4", + "b8089da48eb95d0fa7aa7178c148f878c61fc5c425e163276050d66769b39570", + "b848f9c1cacdfec2ca9ec77091fa926373901a81e89893ffd44b8e34e813c6ac", + "b8a81aac98d4107d71a94ed1e7dc03918b99566b6fef297c43daa713c1e0aec9", + "b8ac6150c2c004604580621d18eb3af49871dc2626c1d951334225bc65d1c438", + "b8b5a6b49726a5e7522829273581631f66139462cee12dbd41ca7b93d8012a3f", + "b8b736ff78f5dd8bdae19eefa05ac7204f4f07187d163341e9316f2caa7921aa", + "b8c3bf17a916c8272eec662a19fcd3ba15885deb7013e3165443147be8624afa", + "b8d7cedc5d15d681707f2f38e63c1b38836456b98577792d229443c26cffebe8", + "b8df782b5cc955e789adf0cc2dba357393d352da099418073e9a9c255f0f9797", + "b8fd7e25bc04fd03c01b2c376dcd64328a543ebcd9d53564ac8113be74b1aac4", + "b928ed0ade3ad7ae4d974356691db8dd8a7fa18ed135a9aef8d746184aeda3a9", + "b93fba82189b4ca8c32a2557f46a130f50b57171e1951004e5378a2d8fad05fd", + "b97305ccba8b3ff42187b7db6c2859328d1799cd55cd581fd3cda2ab303cfd93", + "b9a967c30fe2122901176bf823703e243ef604bc34f75753fe0d8161c46578ef", + "b9acce70b7e12e8b34844749eea5419deb55261a739d1519432c2cfb66dc567d", + "b9c540741090fe4388183904f3f19738a7b727103ccb50bdcaa87abe0a812dec", + "b9dc3d032fec920e4b54a5f3e44c3d2ecf6c8c09745da579b3fe1a22eefa9bbe", + "b9ef19b21ea3253daab567caafe43363eafd77cf8df382c906231547b7d1eade", + "b9f2cddfcd21ccd881f6178c8388173f4997370ec9a2a5b3a3f10309af4cea8f", + "ba3e5e077781c9e12718fd602c426becd35555b333346d5c8b8d0e4b8d59fb75", + "ba3ea31cba469ba9522e17bd900bea020ef196b15bb4f6082ba322d71c7cd2aa", + "bab819a3345cc38eb9d2f4b3d594f7040b75114388442241d3dcc83ce9289ff7", + "bacfed8aa647656b317f21fae6db862e24fc59481a66338f040977139dc3d059", + "bad11986a7cf2e25876f9ef3449c835f2a40ce7ea2f8bbe7da9bdd9beda4a3d8", + "badae713e39d2bb3121a95ff0de7e6fd013bdb9dc86119b49f7edf04222f175a", + "bae48bf75fa4d9c777766969cea7efa83e9175b9735a27008fe457060dac51ab", + "bb17ee93889768038584160ae37403acb9401651630ab3a405abd65214f8e1dd", + "bb1a561eeef5a67d08fa6850904216b1d7593e365271505e36cee5c56e6e886b", + "bb53e6ce9e0343c8e4a1a689015ffcbb21f3db895b2ad5ae51472dfae8f98759", + "bb811974b2088e8fdc2606f0c8c076e55e48972c5213cf3234945c63bb9aa8d3", + "bb82c9ba73073bc7bf00d3315367a6e01d136bcd9a9f0677b9ee642fd2a59227", + "bbb54ad3f3e4d91ef75bc242b35a9a28625963d91f7c123d99d53c2ddd58ba60", + "bbe0f13c903510cd9b65c6bcbe40c141847bd3289e8d27a17c494d98104f1ad6", + "bbea53187321d8577bf4de25cbaedc339cd30cab7900e2d0f9dfceff92e30cd9", + "bc491ccaea000e7928611c69f1b3e7ab5f24f03ada9c29fcbb69f7fc05aec117", + "bc65a7f288310c5ea632964514744d5054e03a6862fb478ff41f6ebfb5e4d749", + "bc6adec90a6dd989815d2ebd2d3d122e5472c349eaab762ddfac58eaf23a9f57", + "bc8cb19dfa4512030c9cfd5171d50b53c2524193b48d715df8bb2f349d24a0be", + "bcb180173212c62ba2392972845d7d66e70d6efbf4d99f19703185b405d8d6b3", + "bcd8b31f715818910c72709e4e53ec79e715058b3b8b20b88d421485b142705a", + "bd2121f0f8916cb944dc0b9cce499d8c8201bb9ed855484806832f0e3774cb14", + "bd55dfb70701260beac95f7ae6c29b0cc675b33a56815d4368ba5a5a9b2fb960", + "bd68f00870fa2e09f15aaa7d6f7f5e3c0e0c8e9df9fd13caeb5ba5ec721129c3", + "bd6bf91e43218326db970a15eb8c67d056a452ea307dfa1826cb6633acf7da0c", + "bd6e313ca24faa2d2310f0608762f6c5fac8ed5041e62e6e8b5c7c80edfddeef", + "bd932b7f64314837ebadb99027ac33363e1ee0729674016f754c0ccad8e8f101", + "bdc0a195aca3787839f81cfb8e1b323702fa9650364a46c5fcd573a2c49f60c7", + "be0b46e7fc8a373d20a3fcfbafb196c04fad15e6cb4292c552dc243ae1323f09", + "be4265c2119ae94dd3dd6269a67b63bac95797b3cba30b31349625b1607919a6", + "be543b6873587292c6e6a41172e329c02e95e4a64e504f19ea40c4cb61acaeaf", + "be6c45e7f69bc3a50da61647200e2ac1ae02bbb1a8ced24f59103c3044bf24a3", + "be7d2336c310249ab90834b1f81317af638e6dadbaf11073b38092b368151dbe", + "bea496e9c1a81e100601aac156c4c56a930441a8a23838954bbb3323a21047fd", + "beacab588382cbada4b9001299ebb9fc2af37622555922c641cccb25aec00506", + "bed8a7542d9679d11feae206a51d62c627da9221a91b6835876f1600176d7dc3", + "bf0cd750da6ba9752ef29d82afc70701c2f466f4f7762efb7c587c973e66d8ea", + "bfb2e72e247a7f021efea6c2ec93634f2e549a87d06a6d518c04b2b8ced37d7b", + "c00ac129f5040ff62e1ed55dc195170aed57ceecf978448290f4c98b6f2b7cda", + "c00e8564f343e9d6d7bbf1516dbba91b91abf3c89dd7b0637417f310e9e51245", + "c01a0385e94fc2fcccf46f244ca591e67737a37030caf872033f60323dd42c4a", + "c02899f51d77c003d5e83858d1c9b0f4f01b04de3f7642167221b4f13a0b2982", + "c094c64f759b68c09c223f21b1cfd965b766db99af89b76efe2170546a2ecf6a", + "c0c3bc99b53fd5e09e26d8814ddf1ba8920932745eef252fc09e240d1c0e0d3e", + "c0dc7ebe001f9d7d104c791c9de10c31255c5a850e14fc31dedffa78e6541e83", + "c0fc70c7452a8d99076484534e581dba98bd1e867317810feb0a103d2e5c37b1", + "c10514926b6b320fbab45c471f278074787252ffec0aee89aa55d78ee98abc39", + "c116eb782fbedff0cc22f419af954749c075c7af985c9eff7d0c165d76144ba8", + "c16d20b8b99da3786748ec64c23f1d4514e948d12aa8f5bc5187a38454282226", + "c16e7d672eaf324069aa23d5d6a1b5f138a706dcdb9d093c6ef42bd08d2b890f", + "c1791b84f1d8438c1ca830aa9d7176dd9115d3ceb6fd23aa7b204fcb05165ce3", + "c1a4323c6bd7607433d6ddf4c76e4502e8b7a6b3b72fa385c906cdef3699cc93", + "c1ad7d9b7be6927a9c31647b1f69ebb2da21c369c6a817099e6976b981c10deb", + "c1c21f7564d5ef802a6cb2a5fc333b10067d5814640df797701eb8fe4b114e7b", + "c1c2b04639ef27293b03468107a7b2017f793a7c5ce17232db08ad396c23fcdd", + "c1c84ed5e9f2228c43bd94eb64102cd7d02ed700cf4f712be746eec31807506c", + "c1d0a5bb9e5792eb0903e793a7e924bafd953d7e53982e0f31e650162658cfbd", + "c1e5ec301d21dd885bc19330ff1cc10a1e028bc22cc866015a3e550ae1762066", + "c1ed275f0d09590a03749fbf16e0767add13f879a89ca1c55950f460feeaf205", + "c215aada505a3b2c249a1aa1e3e6b5ae1817942bc390c1e2366096e8e438f996", + "c221e076bc9d99365c242a0b180ee5895b305510d85fe5b1e1e4fca94ecef40a", + "c2456af35e9845a2a973de93fb3bbf8031c8f2e22dba10d9496fa138065731ca", + "c2461e62faef7d19e61d6a84bddd561ee9746eb30d0b28779b0f24b0ca84dba0", + "c25b6a2ca778cb99b613fe51208ffc8ef01a48f9516645701c248300d6ceceab", + "c273d409d18c86f08b74523ff515897dfce4c5bf272ef773512fa8328eb3e808", + "c27aaa5652f29d382f2e42082cb18850ddbe91d2c142d8cf62d16fa36e6dc829", + "c299e5c291e69611dad567130400548bbca22948ac9a2be5d86cf0be1ed59c8c", + "c2bb18b10dc4435b7225f86e6c9ddeb04669f2b2a062cb6d28fcb9fe61e7d29f", + "c2c199ea4cefaead9e23e15c44d8f94855398419bc748403163652b18e0f14cc", + "c2da835ea1010e60a621f955df541bb0d11ad051b342041497657f9d3a0f44bd", + "c2f750075e3b223f783bba59ca6cb7537feff873a77554742e170652df62a787", + "c2fa1db7cc0f17f1b740a1d3da1b0c2ce7e1cfb7abc7f92eb342cbdc0336983a", + "c308697a46022bb28dd37829dd583edbc414f1646d6ba88d8e788d74fe75e3f4", + "c33f6e1d953c2e2459176f207c5837fb768cc7eddbeadf4619aa2777546de62e", + "c35b5d06952680ddbc500436b29ba9576bc8f11bcca1f5977d6714edc5510ceb", + "c35b835d08530179fdaab703ae0f98b31eb83ff1b1a9707c341029c0516d955d", + "c37bc89dca3af800f44aadf1d78e4f7e3a3e89af77ea633d31651e1a95eda347", + "c3af48670ea6f3200c2ff85ace910f488e272e926055891d86b862e0b353bfe3", + "c3c41e5552308fa4970fbda82afa59b66f489c27a7db4d5d94a60d0db8c8039c", + "c3f8ff3e207767558db61456c7f127af9d79c2d0eb50a8270bce5ea45db2c543", + "c40d78ed2ee68ff20485aef5e34579f5f4054a6fcf5d9b305e7291b701bad3e3", + "c44cb714ef7ca78ac4a00b50760b3abec03706a7545d5031a3a5ab948f7e3830", + "c4736ad4146e4335178db8a9741c652d5d05833fd419a4d44301e034289f86d8", + "c52d67b0644f775244057009d1d807cf959238e8b094a325b89130b66fb13549", + "c5935c567c338bdd710e01da38b97eb94bf8a373c524ffa90f06e141ca21e0bb", + "c5bd03645c8290866a1d39f3b34ff2bb3e804bf5a0d19d1c38b6f3327ead98aa", + "c5c447a38bbbb89458230f11f0b3732d0fc3dfa2e7dc93aa8177bd348c3d8b13", + "c5d45b34ecba22e99f79cd23ae74c5a3af84174d5379bcda1564d7fa6fec5495", + "c5d4d80494eec80c6ed832e4cd5132266d42a0076ac5053fafe48ddc173abaf0", + "c5f95ddaaf9299738b4281a94832b4558835d454e6f79f15b9b41df9e4f5d760", + "c5fe2f8f0295e4674f673532c94ee14bd9cb9a81e0bc9cde32814bb3576865fb", + "c611664dfe03fc5ee5c7fbe647d0d25e57afaa76bfcf90ec9e82f3331b41dcb5", + "c61963f555a95cfc758ce53612e8f0e4cd2841cbb614aff60605dc6ed944617b", + "c620c3c9379df967de6fd8f4c84db0679e9d2dad81cbfb3e2d77d5ce454aa3a2", + "c638f92723f379d1dc48135cdd1a27c1edfddbc74c7ffaab10cd4879227f346b", + "c63f67dffc9e3aa8d60e13154e76d0db63e3fa987ef4c910f9912ca63a0f6f04", + "c64430ee7118581e5bde54d8e9856c15054d73a04f4899f64ab8b59a320e9365", + "c646b12f66f51279cd7c736b218ad1f413585472da68bb60bb92369f7cc66a03", + "c660922239aca421a5cba165941bf8cf071cd9b118cdeec290fe7f5095a38663", + "c67c22645c881371accb498fdc5160ca3f3a69cebea7d152970ae608de1f480a", + "c6c3f5e34e82d2e364d88b418291b2cc75a632eab80e23a502411656fe5e2af8", + "c6d7bbf143e0a344523a1d435ff82552e0ea7c64c6eccff80f9bbe7d56107045", + "c6e87c32c76b5a2e213fde365de66d9f0fc396743bd3d759388be531c3299d42", + "c6ea608314b0fb56168c2fcf3fe90f112a36997db1f7f52b767255a85dd2de9d", + "c6f1e298f5c323a4991b06bf766f36b810701cb0522045b27f73aa8a2d8081d5", + "c71638e6645944d5989bff886d6604db71549d3e91a8138277d2f9d85c9bfd26", + "c729c560656c7822b69406e8dbfddec6ac00bab0a37680eeca66c468bf163a39", + "c753105e888b26ab8eb96284b0c6ae789b38ec094f3a428b913dab070eb0451e", + "c76ecb80f592ab61cf9933db4a4ead1e171011c713a0d4b7de404783b40f05bc", + "c774bd3b040c5bed6d13003b638a72d4e9a2ba11dcabb1ab4a80a641d5d7d37e", + "c7835c562b64fa359cc458c940feb654af46e8b10e2e84fea0e25f077b9f1e41", + "c7848b1fee9b082918121acfb53913b4a8ab7103e792b429e76c9657150e070a", + "c78bf4ee459dd9e2a4acd4a83fb6178ae4f3ec991c0571dbaf28b6cd57407261", + "c79a23775dd5de96daa4f02e7aa240eb03a1da8f63b61e81bdcb891e29416905", + "c7dec169ce85e8407d5039e463a0fe4a5686cbd6f9167dda51caac44cabbf1a1", + "c7f25797d0ee04f41ae508ba50790f03ae751892f9e24bef0a3e447fa25822d5", + "c8bb562f8f805383903a5574b9f84445414378205aab1d3d22b5559a7aeef6cf", + "c8d9cd0644ce09573c71564657ee8e0a78a66162fc7d207a8567710c8f7d7fda", + "c8e463592713e1c7727569c6e667d8918152fc4cc28c15fa75ce9b41239475bb", + "c909c842b7d32ea12f44d92ac041cbdc2c64248e9522ae465168aaff9e899909", + "c91c361212e050943a81e7bafb3d4f2b3a17c63f4373f240593f76233b48e5db", + "c92bfca22b81581e51629b216c9389ede9575af63c0cb28097310c7035952bd4", + "c94ee22263a5ae8a3bab787be129a555d8c2b662564b238c6d11cf1d88fae542", + "c9785645bf1b8f42290c3c10e0036dea320a18d1bbd8df0d8b57f974565bc139", + "c9d61d4310b1e84c3208b667dba9ae044e0acfc1fe96b96d8e2a11fd1f4d9322", + "c9ebe28a3b00f9cecc4ec7d71ff046f07fb16d4377fe33f68711a4145243b523", + "c9f778d144174b667b4657826699683e72d081916ab3d4443ea5cf1de63b5ebe", + "c9fb44774055fb5761f738df4c8bbda5c59121dc85ed27811ee16a7b17b8a234", + "ca02950b9ed42a6cd522865e22c5d5da36b6f4cd1206ce8f36fec5c2a9fb7100", + "ca0ecdc8c99fe496f1a0728042f34bb8477aac8d6f6eb988220200fa60ba3d8a", + "ca894012f7454a07dc52cb1a1f0f3ff3df6ac6c9066f13c331ed5c1d73eb0f3b", + "ca8b4797bb2a647ab2051a4a299d4e1ffbed14988e084b66330bac78a06aca3c", + "caea21412875078c8e0d019066c50138b67a46835a5cc2c25945dae6a099059a", + "caeb0797f58b1448e6f64d2636587d63efbb6e27607c26a6fa3b660296288487", + "cafff4262ddd03cd199ebce7acff792b8e0f906c2f90c0f7b556f8faaff4228b", + "cb1e6c8197a549f105abbef457a379bd4d71aa5c35d63d780733cf1f77e3e5f1", + "cb308fddcab3858dfba8e2d1705b9ca9aea23cdaf9dcbfddf4b77d6db5db9040", + "cb3d92ef090fd329dceea69ac6d48aa42aa88149701769b1a4625ba80cbceefa", + "cba2b38f8e1d3963539ceaa0303e8669a015599a0e01062fbe4e82e8f359c667", + "cba925939b83f0d297ea329100f6dd9ec95d783625be4ba508391646ad22388e", + "cbc4cbe38d4e3f010efbfc9d4a6d021783b5f8d66a9edebb9e6afc046ef0ee4f", + "cbd9adb58482cd85fc5c54d0a13ebb0e0be278c3edcfe7a42b8b246cc8b46e27", + "cbdfe9dad04b6df20ab06dc513414ca71c92b6b01d7e74b79a13ab76ae590474", + "cbe87cae84854bc4177b85a47b953f39523e61561c2488db4b4d64374cebf4f1", + "cbf3c36ef8b5885ced1792a28616aaabddeaaf368413906e9f7ff6ac103c9ad0", + "cc1299ddca29d3f47b825ac93c64b361242fa25803f328fca13441684a4c7a63", + "cc1b30d08252f2224567abf1482c833f629a31afede28893f45868a3122d025c", + "cc8b9c23a4766d262667973e449c8e1fb6f5ea9182359fe848983de8e69420f5", + "ccae31e6c91561c34ae01567e9f442e6ff4ae66e8b2ff665c8fb8199b0222253", + "ccc80694b03de71ba7eb7837ce36fd2c8ff6cc6a7e121f1a0010e21394ec7d28", + "ccc9952cb786577ce9a48ec50a19331a7e3b5aeaed521227fe6b36ca1650e8a4", + "cccab9452697a5822f7228d9ab3198a4e3f2b73810aa78705005edf91c8b68ff", + "ccccbbabccb218f9dd35e3245a04c9f58c86e3d11edef57aa70d17ed0a47ca5a", + "cce04483aafe15033058fc57a6adae63474c8886b7fbd33c2a214c02b64b66c5", + "ccf474053b489d3012904f3d2c634e04ed67f79fb7f258d12174ceffe4d581ee", + "cd03e2df6e42a8883dfbe44e1c0efc58bf63aa6f6d32d47fedd8bba1e23a973e", + "cd1847ee00a06fcac86360000d06315bdcb62abf7dac02e82799277175c37a03", + "cd1c9e3e32e853d95f9f87241c0907ac60f999a01c96947e80829a4cb472a2f8", + "cd1de18adf0a6ef2f23a850f3b25be3407533a2b7cce889d7ebc5ef5c137a3df", + "cd456c7ce5577a4d33433324f28e893d9257b69bbcb020525de57e17f7ca2379", + "cd6590458cb77b5c7aa0bc28bb00582e6f5b6693f59af5d38df77b4db73b7aa3", + "cd90dd5b59b902f716500ba7c07bdcc5f9d0265757f2d1367a47af2c143adb32", + "cdf4874cc5a4273cab51aa533b5fb08d7eb14cab1179572e46c68715013abf23", + "cdfc4db314c0d71b8350c0022310e352a396e01c94d446739e8fc33f6603a4c7", + "ce80b2a6b2332251556de83f456f5be22ddf5c98131073cfb4db9469f6c06eb0", + "ce8347d62ffbfe3ef6cc58ae9ad45b4a527df7b40b899e0d7e09e5cdfb0d254f", + "ceca7e2e8199bad3a54ecbfb618ba5292f78d1f3731bd38a7f8f9999ff8bb941", + "cee38d8376a08f3ca2660337ae409abce9f7f89102199b7b55be928111c3e34a", + "cef24ac7588db91521430146ed719c060006d2d6ed9315fb63fb24975dc094e4", + "cf66b4aae4a945bbca0bab0f912f0a584ac11475c3caa7c65afc461052f2d415", + "cf6b951dda214e04ac328b5e572449c55180815430fcb0c181cf61da6e998975", + "cf844beaf2e6c685aaeeffb989285275203a16da7b6c92a4478a82de33bb6982", + "cfb6ab02ca975d55b0a6c1878cc4a3f22ee15f7f9ce6642e1490eaac8fb8da77", + "cfc5f785a585166a29ef45f8f3887658907612b4704cad2606fdd86580d9306e", + "cfdd01086c7ef1d3ec816295c00994d3133a0c9fc0f097c579a38f67ec965436", + "cfddea7938adca95988dc306017212e307a7d9d29300a49e14781cfe582b3cb0", + "d01fe0cc347b04c146e981c9f3c86510cb9b4ac16c75439afcafa961284857eb", + "d070e8c8a5b0b356d8fbc0d5e50dfbc8cba88939108ff3bb17c8cee06ef92dba", + "d073ee715c3c64a67b1dc52f02c0645f2e16dc4b8934d870046d611511240593", + "d0a2f11cdc874d7797789d80bb38d398145d118b2f161f08d6f171b571673b78", + "d0b0069247d86164502ffe2c678701ba41e1ab19cfdaa75461bc56b6b257a399", + "d0b354e0d29108ea1816cefb8c4aef012ef0c74a5dcea0fb7683de54eaad24a6", + "d0db6137e2be2f61916b26e69df32c0a1525d91867ec2a6baccd1b05dbf733b7", + "d0f579e298d6c00b884b463527e40a31c703d780772924526fc4601c3ef12a72", + "d12dd9f2c736de3dbf0ab05b0c4715204d65d682112f01d6f0a881719086827e", + "d164afffcdfbad0c8b8829a9dfeb78cc90d6a4245e4744a2f4f9462af84ab382", + "d192fa159a7984c4b8d688747b547f3c310c85249445eda2a9bac352ade8b47a", + "d1a0d5cce1543295d9ac16a559390ca2ef9625fbec0a148a88662b3bdbc07566", + "d1a395ca68b59f7acdb1ae4b11b826a5ce25f79c9184abf7dbdac2d87d2a53de", + "d1ab005534a41df9b1eb7c7af7275121d5cac159e60361088178022ec008693b", + "d1b5feafb908fc6394d1e094bef4e084d8003e475c7314265c8e0fa1bfc63484", + "d1c84b48656978636a6b54ab2d8c5a289df23fe8ba0cd8d8f4e1dfc7873112c2", + "d1f34f69500b5ea583cfcdb117ef5cda1352b29fff00113740607e752e8d522f", + "d219e8bb14cc94caa1833f01f066fa4bd963a3624286e9fca56ed00866356b39", + "d23cd9664d80088a3a9a96950b1d0859a1e198edbc456bbeded1a83db91be9b6", + "d240fafaf43c653236bbf6eebbf58cb3b479c5f0bb18c1552eb55949f8abc091", + "d24203d0614be5b8da34b8412d279a17f0441bd4381ace8178a1c044142b48b0", + "d25a40be63ec05dd9772fbfbc282715edcdea3bac2a6050a637367a6be37f0d6", + "d26c279d4047b77c1a4a9a95383b7487e2e22bc12aee632ae36466fb6c02a0c0", + "d28ac0816f3c9f0bf18e3d08d90ebd4a752239f4f23d06bff4f47cc0c8423ce4", + "d2cc4fbd2e43a98674c6c4e68b970c0ddfc139b3d0db1f70db40e8210e8402ad", + "d2d2ec5406caaa6d416c4acbf9dba6eaa4c259b8a97dd1203fc90bd92cff0bf7", + "d2eddfd7392636028a12000fe3b0d06120fbc56d0b5bb92f34a7a52ec0feaa82", + "d33d3d89031950904ea761667310076f6b95a4cc5ca930b7b31b91131f7121a1", + "d3638aaebe3f77b329913705ada57e8ecbfe424f447bb858b96a4aea55f7cd29", + "d37f403d2e1a6ad27ffa735476b4bc9cc58d1031e7a6ee8c42ae69932c9c56d6", + "d3da0f35aa5303b3ababfc1859247dd0328001748311a88cd77376fadaea8a70", + "d41d6abf63e47839422bfe192daeb310402091b9e7e024b25ec534131766d17e", + "d41f562e5632185f7696701cf7294f8deb89416449143d857d247de6eabd1101", + "d436ffd00330337a12dc5021ca62dd16010db019b10d98dbc3d88d878eab6fb4", + "d43d1f6f23de682b557ae10b04fb03b9578a38684ebb2c75ad53680a39bc6b42", + "d4887e482e42cb9ce2d5a0350d934526145985b1e312e711792f24c38a951ed9", + "d4af0e0f77184ac84939d282b93b1018614e0fb91a40d6a0aaea1109f37d8301", + "d4c681632a45ce471ed9a1798c14e9f6bc335eb80162e5f1371611462d639d14", + "d4f514dc89a34da7c37a834886b1bd0144061a816f77fe6cef2bfd066955d183", + "d52bed0de5961ce7653d028389a47d304bbd5ec2fb492688d88e01c20c9e9751", + "d53949e80f745b4d64e9553b036572189079413f781fcd75d9c956d7a09aaa53", + "d55253c49dd42e9a2dec3363f86c4c8c1184894b8f07fe730a51c6660b1edbb9", + "d55b36cf5c50bc67d8bb6b130ab1cf86a08e9317ea19ea7ea9a3df4645d6646d", + "d5737cab82e5a56e1ace27cd0dfecf5e81a85a5b80e246956891a90b2ae92e5f", + "d581b656c72970c3bde09f7167783111a25b8fbc77dbad63a9a5042e6e1c02d3", + "d586cc1e5ffecd95fdc88554b909d8221f17d16a4726358d2bc327a2d254b8df", + "d5c3519f7ae4e585e1c3cd23634b0d912b7ef200686b7ab800b31be4b3e272fb", + "d5ccdc32319e9be322f989a297b46a9a0b100979c009b6ba6deb1bec2f3365f7", + "d5ee28369b14bfb5029dccae76c41cc3188b947e4f7c045d5945febeec161b14", + "d60003a162be418f023b13145c3b1429932f426df639d42f7eda9c12b25f09bb", + "d61fe033fb6964bad0a578ee4c4536c6634c299ac42a84d963a3743a618558ac", + "d62433ee8b0120631f7f4af20bdee7f25e9625184b009a790d3cbd85e25cd0d3", + "d624e46ef452e34d883a49229c0136009bea51e568c553540b303a7d2c94cf82", + "d639559a862fb3e47517bc78973650a49c35c11e75d16771260d838e65ce4854", + "d6a34c88ccf7607bfea26640f4fedb3197571406ef9225dbebb65252b12dc1c4", + "d6ad863a975be3ebe83c4ad6f1ce0c26beff8b91b3fe36887fd13ea576a6e568", + "d6ef592700ea7d33fbd82279ac402a40bf172cbe8b4903576bba1299252fb198", + "d6f318978802318d0e5ac779d37df16f5a0d831be6d654aa57f1e88c33c4417e", + "d71f2212c6c3b2416020d45b3e01d88c807edcf43f6b83e95b4cea0de13befef", + "d728f827d33c65af123c6671f18997b4e8f26289ed3f940faa60b4eabac80ea0", + "d74727b5d722908bbbb3ecbc7cc5b5d356a3c09635fdb78e3fb23a8d9a090622", + "d76911df66f2f4125ce433021ad75a37b8f11dc1e98310a3a5eb00bd2585c467", + "d7691a888b5923650a28fe3758cff2345bb1ae8d80ee8f44bc43de702259da8c", + "d783b3620e8f7e8de20c14c7c66a742cde0bde86556180ee97689ab3626c19de", + "d79918e84e2867f0b93095aa13d9df173a1306205e2c62a26c95d5e75af9a4f8", + "d8051dc72cec1c8f55d0163c4f6606d459ff34a268c1c639323160148604f86a", + "d8117061146d01c9377d0a4142990c2a3eb36399b490ad643275ebfac9e53933", + "d86a6cb10f44ae323616539741800437205a848404368791f18793c8697c27c0", + "d88c67aaeecd2123733c147b75b18a7e23d67ac8abaf7cebc51f004b001cbf94", + "d9064a1bb6a2b88b73173f4e5258feae5b2d2022e91639f3ddeee2c6adc6597f", + "d90c3aae92540da66419e78719f2313129851b552e2f3d21effabdbd0b311e5a", + "d93cae2b91c10e8b3f169f19ce95c444cc9c8b2b46c47e4556d2830ace7381c1", + "d93e2156fbfa90b936ae9334eb4dfda6fbd1b597cab22cb2c90be37a77932af2", + "d9406b1fc98b36f27aaae17128b74ec926f8ad01aa3dac817410f4cff97acba3", + "d956c4f0e28406e4c7b6bef2d310b7df8fa0cbd440eedb9c06f3ece7c5e4ee42", + "d957c2a65445dae0b2e0f0bc24514a284d28a55cf39ee1cee5a8d3e688b925bf", + "d9931641acc262693d83fef4e7c13905f928c7442a51ad8fdf6e1d6f74646982", + "d9a77f4a690ed3e06d58633a85543fd0c80cc41136e7f258c47f1552fb075343", + "d9f3a7eb5261064dfb2740a6bb3e8dfe8cabce064c91099800bf73db39f00fe9", + "da09f8de280cc5727564880afd5046632dc2d4650e1576b3569c862785995645", + "da11ecebd84b29778efb2c763b45ba7e11e55c26ea665e136dc6148fc81385a7", + "da151d715afef7c4222446b54520107722f3c1294f727eec51cee471cd982e01", + "dab4e81d25783c9bd0f74aee16ee181acf0af223526b364ec8945928d7c673e9", + "daf8e49ac3283bbbed06fe7c1cb3f514bbcccf5e32dd189c4cab86c74f013636", + "db1e47ccc5090a6d2281a7bb9d33077e0659d7bb7c697b21a9e649b6a3e3745e", + "db290975bec0205404d595bae0d1f9f9ee1fd7bc26f9175121ad300bcb31bd59", + "db2e86a479922463332e6278dbf55e367557fcab64dda06d3cc7de5c22906df6", + "db4b658fe727a99f255ea3957d2d3529947b6caac5ec48fe25c1708db1fb984a", + "db5d60dbab7a234266c0956d863a5f5c5a100ae4067a4d46e833133108272f8c", + "db91903e6de419db6c737b68d92f7f85721221ebe4c27813f1cd08cef5ecce5b", + "dbb5bbcf389e31bf48d994286a8395a089e7a6de641dc06567efc06a8125c08b", + "dbc1a9a5ff51b0d3d41c72676706adef865ab3e45d26ff2a05a993487a3a8029", + "dbcf50070e709dee8be0cef48068aa573818c8de9d5b7c440bd01c1719eef6a6", + "dbd36f5d6918f0b3f987d0d18bde8833f41024e3019ed93025db2b78afaf7f49", + "dbed23923526ee3ad9c1ba501c4c7150bfad61ab420e5e711e2b4d4e76192d25", + "dbf0ff60e08b430fcdaa06cedddef05cdb2fd205061012a0ae4113d9e22c7c18", + "dc1e5c4c9a22b0020c807905cfd648879658bf757e481c49d677f7ab315ccb3a", + "dc234840669526b8354dedff55c633a89fae838cdce0ae3c777b20bd0ff8a5b0", + "dc48000c38c5cdb704442f68e2b1faee6e061401cb9df35454212f46001bc316", + "dc6c45d3fb6024fbec0d0c553ceb9182658161d39902d52c837631db86229ae4", + "dc700ef65fb407d53f0af7dc467f57549ed1c4b6ccf8f743f380ada78301915c", + "dc78b2e1c869e9e4cef3a00177f73ecaf699a511a4bfb691b0be761977d35758", + "dca104eb57c18b8fbeb17dd08fd35b458a0cd4c870e1202653e4452fc46ccd00", + "dcc4579b164b16e503e32fc81f19470ff3982c05d4f61a41a063d1c25adf8e71", + "dcd3bae5bf55c08d3de86830ddb7571d60fcd64bf47134a6cd8fa7d15d10fc4e", + "dd29a8e572daee4a427ebdfcfd9398d6ff0384d4d2fce00c2eeb5055d110b081", + "dd8cb0abc314ebe78432a219bbfb7ada743bb081249088447dbcc4e4cc70694d", + "dd938aa5437ea65e7b2e1a8ab074abc3fc74de10e486f9256553eee63262b54b", + "dd95113e1f59e4c6968eb8bffa51d996a80a07bceeafedf95a1c8c58b302715d", + "ddd7e5361fe63c2df295a8cc68e8dd7678892867ff1cd46dd837175f30fe4175", + "ddd82ffd7398afcd1b42a4859b65af105c55dfbe21564d87addedf163028a75f", + "dde71388ecf1192c2b1cf1b0bb1341e5800ccf19e54417b3d6a3a791e60e97f9", + "ddeaa5c8e0724599daca62a8952f9f3dbdcabb48576783d4a5921d67bf6290a4", + "de1f40bae42ab53dc5a1a9154c96b94109cb63db2939d268d8c27be0f50c72aa", + "de2f03f07e8f5c21b5d86803f1c512267e8cdfc5580d1f7767383bc7dfc14501", + "de484ca0333fb6a1426bc6e8423a3ecf0e8cc6176a92c3d55155189b9e5bd488", + "de6da3d412c132de0603ee71fbf57148663c9652ca08988d366a2bd38032211e", + "de79493ba0e7a9bee56efd8a6291d858b5a59c282990fd67bcd24b8bb3e8529a", + "de7d61196c0267bbc0f2e2350157db8607f5ca2005af62e78b6c492a53e627d1", + "de8b3255316fb1ba2eab755aa341fb1645b172862215080cdbe07d595f6e1a67", + "dec04ecea85213d947af9b3d88ed75d20fef74ebdb0a5b319477e7f40a2b019c", + "dec68eb79dc49e5ebef9247cdb8ca00020536d911cdb3d504a44903e0f13813a", + "dee4c4ad7cce6ba20f17b691f3829bab814297ad16f25a705c017156787705c0", + "df72209a247f2662c87a21897664b840207d054f5a609083c4e4bd58c913dede", + "df97baa5c6e3317a4b7199a23efdf9739b610abcf987760302ca7749e6bee48e", + "dfa0964ccd4294a4786d98030d7f786a51de585ab9fcc66e9c5af18d7d61fa22", + "dfad806f4668af0fdeb83f2124e6049759bfc6a1ba71467f70a3e241d50db23a", + "dff55f5278fbef8d13bf51671d73b47fe02ae344308f737e751d224de5ae0c9d", + "e00b6adec3f29f19699b8ba00228d2c78f2d72bb996d0c4ca64e232b18bc36ef", + "e0348e9295bee7097b7872c7f50ca11c038e81dc92e7f76edf179e8ce64728bd", + "e09ce2a126107b366b8f8967eb0e8acd23e46f2100aae5d70dd3320ba973d21f", + "e0c8e621f083e33474b8b839fa19f52a709ad11cef6da76b1000469c3a4d84e4", + "e0cc2ed2a8b47079fd14310f898e4a16dbc27fddb31b8855b89f3d9ecc5cb42f", + "e0ee1ac0a2e7f6e414418734e3f8accbc6f9f09832a454b5ead8ab7eff5bd3a4", + "e11e96d715757aae37c340799218673e837289d9b80fae6534ec3f91dcb01699", + "e13160d8f2421179342e179055d54da9de069a1ccd13c16dc04928481d8d024a", + "e13c156e3c6b33114561f9a03066a2d1f474760c197ab3e22dda33996abc6225", + "e1490b6ff8322a5aa02caa25ee79e0e8b6dbd41368f32b055a83b38b3f20b072", + "e155b99821af5d977974bf58e1807c82d232156f53041f13339bcac76da04221", + "e15d71bda0132aa953c779dc21d8eaec2160b70dfe978c4a64c59cfaec344854", + "e1833ccd30ff1b270164c4ca86a26f362a7b4d378abe951db23af2437292a7a5", + "e1a0656684ec308cae1e1c9973c7db7ac78b8891d318efa3854fd0ccd6b55223", + "e1e126ccf4e5b5d3f928fb86c7c249c071384cfb6d6c49f59748fe062b89270f", + "e21ce0dc2ee47f468c8e7b8a13c61223d7379531ba486d121feec867fd8e6e7f", + "e22bc8f83c92edc88ee5e4dae3d64c162633b5e1e73d21dbcc85adfaf90b809b", + "e22d4e35d578d3bc10e8c0bb073103c053b3ae33c0f42e5f40ee6f843acff811", + "e2306bf5516952d77c9a0b4a41ab36ceabd3bd43413b14a2dceddb4c0342d351", + "e27557f997a9172b8decf36332ec7af376eb8cfbcce1577279b2a79db50b74ac", + "e2b6036fc7df03dd49b347b097e61a41f1608ff15d2268190f460d5d4d17eaa7", + "e2cca3b592eb4f08c768a27df03111493aa29246e2f896a08bbd1316dcc9c8b5", + "e2d431394164bd0c41c89f9b8adf6088435011a64d396a874e55abdd7269e4bd", + "e2e4de26dd09f5f661a27182517d173c2b5784b62b363fb94489ff795bbce5af", + "e2e71862d3b63a9440de0b65100f8f3a5f868f4685a2a8f514805033a88c9ef6", + "e3161696c0ff6c0c240ab5c8a39baf2cee8329d591aa4219abb2e6a8f339c0af", + "e31ec9fbd684cebfe22f4fd79d494c1940868bbb1b706892044aea02326b6703", + "e331d60266d5fb71a2cf5b4e342c841d9e08c449c00bbf5f21b41061d5e7916f", + "e3d29931e874f9fb5e99c63c1da162bd8be6723ba3b15e2a3d115353a095984b", + "e3ed4db7194f77d1944709b3080b212e4d1148881725a441b6a89a66747d152f", + "e401536f4e5869d0aafdcee910b572aed4bc9214aeb90a0b3ac026b6f2a36b15", + "e42ef36206320b49147f2d4545be3c521ffca988fdb68b93de1e150d74f45d80", + "e437847d2cd237d56bb0f4e94d08b96d54a346efbf5e00bd4403001bca402b3e", + "e49a197326a94c87b190f6dd10eeea7819b2bc890e12060ed234afdc5c549f21", + "e4b7acbe1a91e932a5ff62c651e8bbce6f578675db5c9f073bc0f4f595a4ba7e", + "e4c8ffb54e9731092c01d80eb64f15c244287cd8ce2755042e43812f9ae308dc", + "e4eb7f4d1ddbcc4ffb54b2bb8cf2f0a3d25bb5bc69d96ce8d49ca9aaa1346e6e", + "e4fbe2e7051ce629dbf11bec69934aa615a86ce36f050d38530722b082d96f11", + "e4fc8dec6fb6f8ce4464735036575318c30469a8841f3421203662d728c7863d", + "e5560868dd89e19db66b810054778ecfdfb79dbee842c17cd59d5f9259c8e745", + "e57a10a2ec65c306f28602a78f921856b4da3dde464448a376161c9207ad5642", + "e59cfb000b5e37220df6031d4b75f9caa5064d8e250d4c71555ccb917f9f4f53", + "e59db623511c79fa44e3331011355c450de3267fdee43f73a6e1c3be6d535c58", + "e5c3d31414b03e9ebffec47d7e6539abacca63883994e7797d0449e759904e8d", + "e62dc0358bd61478c2d074d0fd346a78581416f6da7f5badf1b14638501df1b3", + "e65253c864af09523087a95d6e93bb100f4d23a89d0cf9e7e149ca7d75796ebb", + "e663a99cbff1eedab7d4eefdf715b20b94288ea6aeea064d3fa5652427a79223", + "e6f99b65c2153acbba6bfceaedb81e6022862e7ca651aab3f2a6dfadd0b50d1e", + "e705fd902403d9cc9857a3cb79418b2309a2a73617d28b1ebfd9afb8affd6903", + "e720b0907b7c559e2ad11daf498bce7e81b90e04f6b7daeb902fcb445315791d", + "e7304b63fa61c5bf44279793451cce774c87dea397e7ebcb2db6415df9b98818", + "e74b7a55a37a76b850f7df1fdfbd4007b58480460e036b16cd2aba06a8170201", + "e74d07886141e7b7f17aacef7846cb3851eca347bbeb9bf9e12115209ea7a89d", + "e77a1c97ce579a6463489cdeb9d291a1deef4981ed63886f079a0cad1b71b24c", + "e7cc860c5ee1d94b6ddb825db86998db190ac87293ce08f59ce1f18720460690", + "e7efe271fb083d4aa855fc07a557b0ded89c5af326a13937164f32d4cd5ef697", + "e7feeb46423d16a543c678b1d9e622de494d4ec17a1d51340eac3b6ad0f8af20", + "e80068e7a72ce50a0bd03b687439a393a446208f30c8e9a042f4987eb1995c8c", + "e81a7edda92f777c9606500d272f71dcaad22047e5043f9760d290a7b9f5c210", + "e8314d44fdc7f77c3837fb3f8a00072e6957542d09afab98e04c0078a54f0de2", + "e86b8058966437966248ca46b07426dfb0a3f053336043edb21475056a20fde7", + "e88c64b97523bd0fcd353a58a965d2d49cb5d8d4ff848a2984bfe33ce0ae52aa", + "e893a900a6014aae7750c958e2ec9b4cff8b442c2d7330cad9ca387b4541587a", + "e8a7b252b4bd1f8edb0ab2702106d8186b92de10c4669bd7c2edc7a6e162c5ef", + "e8e85111e0a93a445476d3004291721ec6f144962efea2a6c2ba30b1aba3569f", + "e93bfc5bc342a1682e229484069d4cde0420f1092212a91dd88b20618aef5f3e", + "e9748c12d086fff748c634103440f42822b4d30d330de751fb7ebd16fbb65fa5", + "e97a1a46b82cf13f9847dfbc47322620100f5113275f800ed2440577f989165f", + "e9a667208b6e17c44ae2a228803f68bc443fd9890863921ba73faa43e0531ed3", + "e9c70d4e8fb45ba63db4e9e130bfe555bebb46a30dac84bc65efcccd5bcf74fc", + "e9d4b30aace05a47643d8a3306b7607b465c9af32f343e6c27e15afbe2ed9ac8", + "ea7322d351765ee91211e726f2a2e71b148614ff858476f2d314faff22537095", + "eaa227c5a58d2552cdc53b0ea798ef846e669c58bf6e1236a9f3ac8df3d5bad7", + "eaa547e3196dbfc5bf312fd6ea2d6860d9ae18fce3872e91d94a259e4146bc55", + "ead919499710acc08c0afeceef3fbf1e194220008ff82f204e997b21f0c4d1ae", + "eaff850268079777284976516c8b4e741a24febc416e9616a9b0ce64d0a39bc2", + "eb078772e8c9314f42cf175c06ec13c2a59fa5156144670cd43b0cac5ff3a7d1", + "eb17893ad911455c681dd6323aa29fba6306391cc375a157a2febb36171a11f9", + "eb5f7c73c14782faf0db5f533399f067787dc7c0042fd8b4dc6117e4b7dd4488", + "eba58ab888533f8846fdc9933d6d36e7325f463e132129ed5e040d70375a2393", + "ebba08751abc0f9c4f5074fae111f889f1cbc096694fdcca5011e5d753d5c630", + "ebed887eb371af718ef561d2d7488b49d834caadd3af8c5a0544b68efb7b7526", + "ebfb11f55ace4943dcd0b4c8c78d8e3316e1d4d22c73aae77880a891d2273e12", + "ec0c3c07db268251f8cdc4756dd27ebdd9a37526cbb44d0cf5a9c1f2ed52ed7c", + "ec4d703860817d225317b3ca6fe99f767d67bf59c14201c1109b89fbc2703fbe", + "ec4de8153a1f4115e7e6eb09ca88b9a38349ac2993c137e62bbf7d638e1e5415", + "ec54410110150cd6c1ebd1d382d132d0627034a939e7b6ef93c2170db5a5ec62", + "ec59c3922f577bac3bb7132377d65a763a014b6cac1269973bef2f8541a63075", + "ec9081187fd745bfa1c3c7f9d05a46149def1200bbc2772183179385df2525fb", + "ec9da9a275700cd08b7f6cc22ad718bf73621761924b7366ddbcbf6a783c85a5", + "ecbb1070a795551da662c74f7809c8ca608ac26e70e1ddca1c73c026143465d7", + "ecbb17cf37f94c9c7b3f89231851aafcbe0925e7e4d7eb96d5ff54427e014aa5", + "ed0b2d5e9d81368ac6029b17c0b2126f43dd2eec9201c76bef086c3a4f0cfb17", + "ed1e78181721b3f5aa283d9ebf81462ef18abe21419874ced31934518045a5f0", + "ed545e76d727586c418f0744d339402358a3aa5540da6a397c659d86a0bc38b5", + "ed616c04d4a39d94cc2066556580c2263bd90b6fe017b8fe86b8ca13f019ae4e", + "ed7f80b722d4c5d6cdee6e8c5c621f956149df086e3723cd4064ba0d5be6838f", + "ed82fe48d398b7d2cf5fd18e30e272c93df532b8c2c1467ebf9ca356c33f909a", + "edc1ef0e343b63c4b1fc0ac740deb67667e878d75a060dd098ed0e06050f649c", + "edc5ed01c736565a07399504b2f911ad7a8b02670da0dc1eeb2ca5a780cf8d09", + "ee3750871ccfbf83bb92d4c931e88d78a6e592eb39409134c1a18c8461722172", + "ee9448da8478284de63644ea593656cd7c9fd513177eb3e58283ba84a8c1079b", + "eec253d9e8deb02ffa7e8a78467d48c67b5e593e36f46b2cb74fd3804823d5bd", + "eed982158217d01a1d5b5c628a6aee7f4aa9758aa44e35ae6b48ae4b0b36bc23", + "eee65bd6251f9ce7acacc87488126d67cca954487a386830d5e4f3304bfd5b70", + "ef0211b2a54bc7071af61659e4a0487f611c1ffb44d722cf8b6a7f47a266559d", + "ef0aeed7749cc3cdf075c187cf83633e54450383a329c6e2c0dd9d9c893d7271", + "ef4c5f83618c0f55d9ec021f80e2c8feed4a3044076ab4825cdec43b091da1d1", + "ef588ad2795460519841155c165a062f282045f733d1ed6d5cb1d548a306fbf1", + "ef8a439b7f1d5e93bc57ef1607c0aeb59f791fe38a8a8280daad941542de8cf8", + "efad264184bd7aa09ef2eb1b312f42ac1954241db19463bb7ce5f07e3b9c7419", + "efd01beab68ff2d4fcb1e4f0286207fcbfb74427295d34575d8ceb391c9c5806", + "efdf1bc52ff67b6eca8933310fe35f6e2d914fe396c9ea7b65cc8eada41c3fa6", + "f021fbaabe1c089409a58f516da03fd2e19c31b074e05be889fca1151e50fe49", + "f037cd9b21c0dbb33b365f4bd160389d80fa5373a748b3dbf9be255f0fe64338", + "f03eaf76ad7edd7350374cba202dc95389556f56bf4eb3c02a32c61551fb8921", + "f04338b6f05255d1a4efc150f3a1a2d5ba1533e874fba4247596b3442faba88f", + "f057e1afc93790016ca44dc2cffdbf67c7d98134488954fe16fca88479c0f422", + "f0611536592f851e763da6c07e4f7b6fe767250d886b1bc1f5ac527edf740d29", + "f06b9f2044daa583f152be142881eb31a0b7caf424e1e2596359a7d2eb00524b", + "f071dd23de9824926063c391a86f4129f95dbaf9034032c70f807bfee403f15a", + "f0750574f2b36357f51ca81b9248c6b2d7154de18878f944a4a438a7d26e26ce", + "f082fca71ee6941d5c60d24a7c7358b281461c8c0a2e2c55e4373f220c7a25b3", + "f0a1e26d253c2e1370acaf11fcd05ada4585b63dc572ea53813e7f0870337be9", + "f0c004c62fcb2c23aece72277c417f7d33656d9d9e0e05187494ef6f29056018", + "f0cc7ffae79b48422c342ed7ee7ab2f9c117c7d9b92102ea48b05a11e47402e7", + "f0cd108d4d1b11a4bed5645b1b295f9ff0699a6c07434ee79fb7764684c53617", + "f0d7c76e820802f368a015e8589209c0b1ff010af501316b0e4a82e7590cf821", + "f124e0aab0ff513e104eb3ba2ff2c974505d10ffd7bfb3087781fca6d1f2c852", + "f12c64024ab41f66b085ec4528a3d78c82c6dd83c61812ecf475a1e0f12f9004", + "f13265389b401334fde5646bc13b7e7c4f8a05c630878615034f77541b1396f3", + "f14f547e4ae201c702c05d04b1024a226baab1f7f20e74559c94c8b5800e0359", + "f167fa0d6df35de3a5ba0b255a2ab45436efeb3b143a7cdaf0e8a6a68ec77f6b", + "f19f0f821f7cddeb39e9fce9ac5bc78d6a24e64275b28b2290779ea1bf99fd19", + "f1b00f410ccaef0d7ae613b337569f38d35d6520b51cfbe13970ba7263367342", + "f1c038bbd103b1de60b0ffa944d940168af7f5cb28ad2f619f58a53c5b63eacc", + "f23baa5741bccea2fb140328e4e00ab0f5cb1e1e424701bcd5a4c4be33f6e098", + "f23e67c6b6e6c08a3e110eaa2331c1068330ed9eea335a54606d1d2e156ed26f", + "f25aed755465da7d5c3b4e3c95a25600cc4bc635e9640e7ef993d1c38040b344", + "f2669e86800b3022fd2a15123b693b94b45af55681b255b85a85dbada47af1da", + "f2a404f19cfe1b22fe2af0d23235a5f7146d784a3b1a0692d6bb8bdb3044b552", + "f2b1b88aa102690a9c394f9214a04cfddc9c153861459fb689f46bcc63f8f4e2", + "f2b5f26abaeded0cdbbbd330a97f60a2cb3af54e0e4014f0cb1c0251a2989be3", + "f2f5a1978daeeaf25c8ffa085e09762e27630f6001497d9ad2be7923124031a3", + "f3aae264c5324b53d7c9dfa3c93a217cc9acb8b59786172b2436713fb7bf04db", + "f3acee3ba45918475e3cb3f57aa38f918024778585bb85d7ad778e5e8ff12046", + "f3b1ea2e2365db574a8289dfbed5fb65424c09de311a0297f2229dcc65e52243", + "f46ad557fe453b456d94d7591a9482f084e3c502e334a445ca0cfab1d465c943", + "f46bed6638a9631ec01f7450a8edffdfd1cd597efa1e3c18a7b971ec9b56d5c4", + "f4826ae6b3ed964c98c870304a5985692a6401ad5d2bd799fbf648f3e455a44b", + "f48b5f5117b250997fe6c64af116867753b003ba870eb9e70cc511a7d6cc8830", + "f4b1501caa6b19f21b015ab1cfb44af7a77d9cf11436e9b8e74ed73802fd4274", + "f4d2e4d7dd527388994c2677651db31a41e812bde44eef9c6683a97ba479129c", + "f4ee92a108e08b0836423cb6772b501eb452a4ae5400192dc88bce99518f91c7", + "f4f047a3a53e130dee3f20a27f71637e3b13c6437223c8a641b6b4a9f6df27ef", + "f4f81e3dda331425e5f951652c77ee7deda8d3fb5cbc5a2bc1e9ea53db2b9320", + "f51a262120089fda895a9f80697ca8b686e8c057f19ee736daf479b7826271d5", + "f55b877df863c6b4a5a3e233892e11c00368a8a49ffd2dfd7a1feada89702cd1", + "f573ff54d3b07aaf198fef31cf632c82a409ed59eba85845ff96b0367d964b1e", + "f5750ff4545c2651debea6d7c4464bb4d92721eae8fbecea7c011af94b084fb5", + "f58cceafea23ddd305ba490e7f8cdd8ae1df5b02f66e07bd698b2d1353ec578d", + "f5939607dc859b81f8b9eeda2fbb4bf58e9ac3f3f259f2141284fa44fe350cc7", + "f5deaac65043d57651d7e4530e62761b8014a54771fb2fed110eb0b30e86f3d3", + "f6041f907c89e38602fc4f723a7ba880ba6ba6293494427f433675fc139c5740", + "f63d5f0e4107f691719d2c7e1b0aa0fa40a7fd868acce8040031abdc046fe06d", + "f63f72c986c0997616af03a1c7a6a2a01ce412229e7cd8f2592de4b60789a350", + "f65732cbd65b8f3f8b4b3257ae9be69f496d828b288a643c54070577e23694c8", + "f65b8d423e8c5a5869491f91860120e2ad932f87aa646d51ccc5c97e478e8b74", + "f67204bc7998220c42535161b3da945d9522fb4f66d804b7bda97391d18ed41e", + "f67a153cc00c83debd8241b7b91eb4ebc6efda81127a3143614b4ec202362dbe", + "f67c79723180993631bcd816141ea11c780d74ce1f167dfd20d64eee34020ee3", + "f68e1fb5c0a10c18c1b436659a676060f2ff347549074643cc376516187d5e18", + "f6b57a0cddb03aed35a73ccd10e928ba35eec1f593614a9621e383a15a4bbc59", + "f6fe4d7f7d890ab3e96cd6deecace45ccf2b7703dfe4d7cbd19fc38bf96a4fd4", + "f72103979549dc42fd342428606c57e0056e1ad7aa6367c9ad53b6eae3ba84b1", + "f73acce0d0caaee1fbb643dfb565a921a94617a3349d93a8469b9760fb7d2dd2", + "f77aeecf0782ee72b26dbd0bc2afdd9474c7a16608b383e35e8e7247ca1719ca", + "f7869b4843638e7f3dfae398dda8e0a7560cd9c5aee80270a5267ae8866cf734", + "f7897905419d70b23db05fc0a1001d0b1a3b0af0f832117479edb4e0143bf2fd", + "f78b1994b24b4fdae392f7b46573346d468ed74676c110875db6860536411972", + "f792dd2b474132905f755c2f31fdc6d04573311b6838f293d104ecbba7ef9bf1", + "f7a7d978a948f36568666db2f266897d1d6db97c881afa967813a1d90c861ed0", + "f7ac2ef367d2595d4316d92f2f5a1e0b3e3ef0c52b13bc6903ab626e0768daef", + "f7c9b0a8be9f9fccfb26c2cd67badd6c4baae6fa4720cb9b9ce32718d0d42ce0", + "f7d15ee450b14cfad3a63d8a40a182d6cf2d729e5427d1bd43b1d514fdb2ac18", + "f844dac31b49b7e2be2afc4507c6835c6275992bed428d5c2811406eb0636ffd", + "f8856132a7ee942f9935d819d8bd1ac2e8db73d3ad1e3504231ec44eac8b4b2a", + "f88d7b87fbb90b5eed6107d4ca238347cbf7bd9960d11e6dbd6202336ba47a48", + "f8bff3207a9c4d7acba4b30d1fe9159776e6dc1cfb0a684cc73660264474adcd", + "f8df2bc65afbba35fc3a1f266114766feec630ea2a193325a0610fdde899e85b", + "f8e8e4500b2f9425a8c19279d62346ba5015130418ab023656960a73308e39c8", + "f8f3be4f9c461671164909f3a6a4360a0a18664a5783a29b2f869345d28689ba", + "f9044f70f67651a5074910319f059bd29ba17d5aaa643cd022af04092ec5b8fb", + "f9373e58869e091a50e8fca190323e20ad36dad195a8bf0aafb84d405ffb41e6", + "f9426611ad2ed31b5fab523266149e5a7076e35454e8b4b21b69761164b56c46", + "f952fb8637baad559eb2a8d8541a65dbdd0453b111a4f459b15c12e741274545", + "f963862beab529f79ec7450689199006366b9f9eb602dcf88829b30549b0dff9", + "f9b818c580b690669d02210dfef753cf6d8cdeee736dd43edf3e3873be4f3593", + "f9bf5accfa2f7af0cd3c16bc75bdbecfad15455e969c43a2386ea5e0dc164b76", + "f9fd406f361fbc59236ff4e5db9fb03bf34b0d52a314e50b9a06463060c39b9a", + "fa03e828df8c0572b8a53b3ede30b1093736785e63a68fbebcab58688c8d25fd", + "fa03f44e759f0f2b45d4ae60b6bdf1f883c7b28c8e584f4c9dbdfd0da8acf021", + "fa2d7063043b250978556d2efd5c69280ac146530785e68eded6c8c4a2272f2a", + "fa3375c4d494fbde579c72af92d9b836e215537068065aa024adda70b3e2fb87", + "fa44d75d52a573658e4fb168bff6591313105d2b215ec101d87c8003e5ae1ee3", + "fa601c703bdf0f05901fda4215366afbd6b8da68a13a1acb1ae5a7d0ca551dad", + "fa86d37d13aa868ffe88887c502f54675183fb72a6bf0f0eda73353cc93a6f1c", + "fad28332415da4ed66e0a02710033fae4e7974bd895649061f6fe1447aa48eb7", + "faeecb415d0620338cffc4971adac667836da581e69d97a32e8cff554889bb23", + "fb0c91e2f07bb12e03c91114d97b28f384dacc22f55c9c2534fbb5341594ad79", + "fb1c4b9dce86b632b78ffea1501e361fcff7a4d914c99eeb5646a1483c17a75a", + "fb1ca8b33a54a6968735c70926614093f25f1485371b0b5c94a50866f1561c02", + "fb4006f12e6539562628afbdffc1114c6109fcef4021e01a31272d54f7973643", + "fb61451de49728bbd653e2fb66435c9d7551b5716e4d795fb8ce5de400c31628", + "fb63b0298dea67ca91f29157c90022fdd2f6b692c615cd54213b93b435c04a68", + "fb87ea8200382d0ee7b1a4ddf3ca0d7068d3f3967fbc1cdab44b22301b06a094", + "fbaa467be2afcb06c9ee22987e7ea16fa3e79c192296b6c4129f07c8f60714be", + "fbb08add215e5c4520b18356ee5e2cdcfb7397916a93d3df4d9e0dd0232af675", + "fbb5155bf20f9145c10c42fe56c7cf1d1c6b7a8d5f7c748184a431c101113f0b", + "fbcbd4a4a21904d22d4d54896b1429e206a079dce30c8eca0933af05232a14a4", + "fbfd191d086e9d8c6ca4a41d248079bd285124897f997b424966b625f3a67a1a", + "fc31a936db1402433f3384dfdab6f361b46d3556e4c7d95f52bb844018d491b1", + "fc4ddb1b80c7669514ad1ff5d62406a0fe7e8688782d5189a850dcff081340d0", + "fc6c4e67ebdb78dcb98619014a4419b074e164bd5e992991575dfd2a66291e4b", + "fc72d6536dfdeb226151f1905bd74a43e8f04d2e7c4b12c36f6c3433c50d0ce7", + "fc7dd06302e49c42e86e4e8a4e9e093e918964e38a9e35ce206109c4b92b5391", + "fc8a7c5bbd30a4fe7b08b44f2057a67dfcf5588a1ffc249d405f6bf9756da974", + "fcc5e54f7c00f05f6f78b3cf6c1daeafcd911a02240265d44cbc2b853d6a5ab3", + "fcc6d55972e8e70cf5ea47dab7fc1b7efa3f4dc786c5a33ba46e4ae892947a72", + "fce6f3c6682f3b46233f31f7424fe98ca8855762fc84632c4019464a3c06caa5", + "fd057104993516baf2087ccaf9a5f36fb95373e33a94a82810c7a7ad048306f9", + "fd18b02e97f3f2538c2c21b3d618c0f0d47678fdd8494ad14a0fcc5ad1bd95f5", + "fd2c9a9f34990dbe30ab307f0950e8a255c1d575db5e24ca54618ecd92be3263", + "fd2debd3f338a433e73fa8321ffd1581b173f088dea2590a4850810f55616bf5", + "fd44d05b0be2cf03d5487ef0eacdbb3b76e5c00fd547db7623d7d26cda7fcd3f", + "fd4bc5136625be7774541ac903c7c4f174833235a42a9853adfd8371097f6fc1", + "fd4ee2313d81fb52f0930d955c668ed97fe3b85bf1a1c92c6d28c7253c84fab7", + "fd68aad1a332f4bca84e4a4c75d469604e8f51a2bb6e02f734349ba870efac86", + "fda15d0579879a85bf0495c5862956f75fb23e793ba45977c389721b079c6c77", + "fda896bad99a7e6f63b12edb5586f732b73ba6187afddea5b9f2c1b9deec86cb", + "fdc4083896df70525d6cb8838de7de6e510385561045aae9b6a11d9eb7174957", + "fddf26cb81896c40e3033f1d43c911b706fb58b681757d1c607e5cd8ba997df5", + "fde50489ec700e54054b2c5a63ee1b596e1dd9d25e1178b9d97bda5cadfb99c0", + "fe0d45c84237f46dbcd90a2d90b7d7103a5b1ae79d9641dfaa7e7b5d7c155382", + "fe2db61e327fde7d349bff423eedb5664b2d8d268a1050d50d700e44a4a4ec84", + "fe45796d92ed2ffa9edadee0a8ed0baf06a01ea06f7e2b3b280a3660399707a0", + "fe4834ee67e3dc86f371db6861ab5514c8451744449c22c80ddfe0855b2441a4", + "fe5b72d2a8a29732c5307f43f7212a492ff5720c9eec8e98b31c01b57293e447", + "fe7e015b2a149f7e3addde43798f0860be1f92ef8e0beb0032e9282e858f1a2d", + "fe7e12db4ca7b495ab3de2b8e812ea23219baf38291e7a5836944eee383c1362", + "fe860d68dbfdccebe089b378ebc4e3beaea70d1a8b24d2334c47799a5edff5a7", + "fea5a8461835b1382a1110a77224df1ce8a067bed704f1a6076904647f068b1d", + "fef252c6261533f1b9784314052f8e1e48ac6bf10dfc302c24b718ea18b32274", + "ff22d9c91fd4d74876b2b41c3a653963d348c8f2040039d5bd0af665a5b59af5", + "ff3168092a52292e5e43e7db6331bc539b4b37f1df12d19b1c616ff3c2dc0ea0", + "ff3542e42ba107e85406f63a9d670e4d53f34d98f13b7e53baa3de8f44b05879", + "ff3f03ddfca7aae413d0716ae39a988d7964e752108c348a4a4423616d85f81a", + "ff55ea16c630a7b728e6616bb5427410845da8046007564c6258d7d02b986859", + "ff6427309ced42e9287b4540496c6ede902122ee4ca6763c4267c75792f11ef5", + "ff65174fde7ad22bd4838e4c6a501c904ea8406aaa9a233f1f15b266abb8e1a1", + "ff7e5550a6afa9804c92139375dbd8bdb6930b456f9266869cd42c7384cbc6d2", + "ff88a6da61a4a0fc99ea252508a180d48600f13212246fe266b5a02b64ac14d3", + "ffc2588c1509c663c1406c1494a663b87978bc0c3620feedae2857f5c22ca4d7" + ], + "include_outcomes": true, + "label_counts": { + "negative": 24912, + "neutral": 6636, + "positive": 1156 + }, + "leakage_contract": { + "deployment_must_not_read_outcomes": false, + "nontrain_outcomes_are_evaluator_only": true, + "train_split_only_for_retrieval": true + }, + "max_flat_action_dim": 112, + "num_groups_exported": 2044, + "num_groups_scanned": 2873, + "num_groups_skipped_by_split": 829, + "num_rows": 32704, + "object_embedding_content_hash": "fe97ac428d6b0ed4da7060eb6f697c1392b4a8a09988a10c0f633b1e457bd568", + "object_embedding_manifest": { + "dataset": "embeddings", + "dim": 64, + "extractor": "rgb_object_layout_v1", + "num_embeddings": 2044, + "path": "object_embeddings_rgb_layout.npz", + "reads_outcomes": false + }, + "observation_embedding_manifest": { + "dataset": "embeddings", + "dim": 32, + "extractor": "rgb_jpeg_stats_v1", + "num_embeddings": 2044, + "path": "observation_embeddings_rgb_stats.npz", + "reads_outcomes": false + }, + "retrieval_index_allowed": true, + "schema_version": 1, + "shard_content_hashes": { + "charts_00000.npz": "43e04927549fe3aadafe63067bb82c16c1167c54d0c8ebd47d4114a814f338f1" + }, + "shards": [ + { + "num_rows": 32704, + "path": "charts_00000.npz", + "sha256": "6f3211ef91cac305045b540e71fa40156ee697abff9db3c8f994149a2fc1b663" + } + ], + "skip_counts": {}, + "split": "train", + "split_fractions": { + "test": 0.15, + "train": 0.7, + "val": 0.15 + }, + "split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "split_policy": "stable-hash", + "split_seed": 0, + "state_hashes": [ + "000b25ed5631161d0436064f18bf2927ebfd97feec3b027cb7a3fc7f802faf32", + "005c6ef90461f4461053d2f248598e4dc200fdb34d95079ba9b6ee9cdc796282", + "006bca42e3c79ccd3a2cfe015bf7c713f69f779e9f9b6e05e6d60e05adc1ef2d", + "00889249a42766e52451f1930d859b4e9c6a18eb2d2a23e4a466065e1ec6cd8f", + "0091b5c64f92215ad5e12cb639483bcb463e589fb1442461b3b7f2e1f99bd18f", + "00a20929cfedf805498d877fc9a08bf4e18093022ea54d5e205ecd22b0aff85f", + "00bf989afb844173147aa217cda23c2d540be3549b8d54cee51524018f0886d4", + "00c9e85ebf618703709c8e915feb24d5f9fc6ef4b1ba77a7fb4bbd07f931b588", + "00f7d5c91383370fa6a6cf7e0da5227d76d4148a8d74c28136bb2e2c32e9d5b7", + "010ae081b71f7c0e58868fcb7737f45102882974c383ff4d8c1b02ce1d895661", + "0117fdd7ec26170b0745e65b08ea88008b09508b62e77d62a69473f535c25715", + "012a634e5771527516fd013df470d5b27c8a795d224f217fa9655c378cbdc6ae", + "0147c0991e5aec13d38c9a12625b9824d5e7a497734845173f31878b6daf3369", + "01642e4ace471a0ee60cd4a110c921359d11750c72c4a4a17274dc285f8de946", + "016859c711e2d334a59ec1acb53b154655e7875d1c095191ac681a0d6f2d459b", + "01a740cfda480c25d931fd14305d6b661829ab5ebeb80ab3efb4bc1b7d400958", + "01cabb034d28efce276c7c37f7c4d1b3fa2c9d00b140b109d619358e1a99ac3d", + "01cc39c8b7c4511f037106757c86d8a9147e5306a4d30e1bae3b41bde0455895", + "01d46a34d74e86012c60ccbcd0cf56dbba216277d41a455b8f04dd1367dbf373", + "01d93a2edcd9393b7d5f2543c37638ee18a7f731976a7f58e144de69747df1fd", + "01f9e4c4ee47d37a28ff9581e3c433583b0b46f8edea284260559029e7c04bf4", + "020d8719f66fdc0b0c47e07ef57b3ac2a06173c2c743a699b972fe3996758e9a", + "02251ce01393ea7b7a290137bfe09e465ff090cee97d821901fb100038b3fb16", + "0251eba2dbfe3f1be9cbbb27321cef00827b19394ac802bde83bcce72d7be6df", + "02752960d18784921c0f4b6022bb7a034bce292b03909ebbd0353c12d48ab314", + "02860b143f2e95695756174fceb8ddb331e7510aa0454f124250edb7863fd5c5", + "02af5467f98ae64c370ed93e1b882b5259adf7e1a41a00e36d0b0a64bb6ef756", + "02df6abd276ead475e81d9a264ff6c4187b0505a8c8d36e0300ee9fde4d4c731", + "031cb9954afaa630fd3a550eb9641912cb237d06db40d1acd81a7f6cc7ebdc31", + "032090600396eef932cda799b7e4ca43ec002031b155583d41ddf5f884b0e066", + "03422e256c0166427e92805dd77424b2025b7be0dfa3d2cacdd17e844c91a80a", + "035290871ce5028fce982cb3000b928e26a7daddacfa891677740136e5f415c6", + "0393682c39f9ad90feafe5ccadfe0e5cd11db16f15286b404bc4ff620d0a9164", + "04016ddff5f00fa3ef76c2246256ff48d5c7bf896cb3563f358e0ca9b5cad5d4", + "04042fe5cd41177bde33eb3dfc7c022d68426c8bce2b386b6c3d744b3b6c8ba9", + "0413d5ba033ddb1d8be7f3fd905b3a3fea63495b9e2bc6636af73a0cb799b926", + "04463ccaaca5d28ca99f63f4f53e070efa4ad2299568bd63a9105702336b5ab6", + "04553237c84ff09a6543e606ab9df00f3fdd3359b381e9cfcecb9f0cf8f66c58", + "04736afd75d9b70defe4c540c6b91495c1d483916acab8b9cb88954faaf5d7fb", + "04891991eaac4f5fe2628582b59594f435c2f81d561f2c42ec1dd98ea57408b9", + "04980e08e9f08128a95235e4570f5c3b26e9900930f8e1901b1cc805055d835b", + "04e020a7f9b5a6b03c4d0cf02f1a5226b30f77918560b372a4a86342886926b3", + "04f06ba196bb7367e0483cf316dadeaec8bee82115844252a7139a70a7c202a3", + "050edccb9a8c424bb0da9dbe686f19e78196f374d0a0a6d8f47b92cb2f8766fe", + "05165e6acce39716961beb3bb0ee00c1158ebf8fa3658dc70048d10cd99093cf", + "0518bcd40edaebdb67c915e851d4938473f26ce6dca50a21e69f83fc32288c89", + "054a1bb649b430a86f7802a9457cc6a83d15cd5b8ab6bf8adf410843ce7aa13b", + "05550c96fe8ba235ebbc76dd6570b63df19358382aafb3937726a87d268713b7", + "0583a22e745ea93e01a9d2f9b53642289d9a05b222a73c0fecf8e710fa1b59e7", + "05b5cf5f903d65c126181a2bf230af182068934b33d8184849b6b8981f68ebdb", + "05c6d0e8769e7b9de479e7e02edf2eeda58b5ebddebf2e1f15c962e257d49e40", + "0602af5175c035dfbe398a4014adbeedfe194d75ed41a0b7eb8a79b48ff6bccf", + "060d8bf103ac5fde82e3703ca8e2578b4f8c7bb7c5cb738f5d1209b919c61dba", + "06257bd3fa90f0afcecc239680999406ab79980eb303d45a2d1fafea670e4a38", + "069f55a960c8724588c1ffd62afe2fd0bd83efd2e34cf6a5168032643e6352f2", + "06eb82c273f65f490d85d60a39406f6cf2318d14ee9596ecb1cf7fcc3ac9f8da", + "0743154e958abc52c160fce6065de25d968ab60c7c7a536d400d2295bfc27255", + "079b15a218441cffd25d9f3aa501b9143a3292bb7005f9eae05704e55f3b38ec", + "07de76536f57dde25b6235024f5a41e215d6a13d9f823e2ef50e4e84c87fb739", + "07f15fafa5887d8328b676cfb87c6f8f7dcae9cb187b8b6969e2f3a4ef37e8d0", + "083bb24051300e75dbd52e0202170794bc727994f278922c2d158af777a3f2cd", + "086bbd948427f350f1d964b288a729ae80da38291e409fb7e0f0c688d99562d8", + "08713adb8a3215938b380fbc0c63f235e063847962fe02dca3f6d20113fea1cb", + "088e20d6cc294aded49228f0ac9a7191e7cc9fb848bc19a6ec9d03fb7258748b", + "0890c32a99d90761a2f31733eedbd91b1980913ef735752d889c0710fe572764", + "08b122b9a1c8a40b5965898017846a28d6377690db7128b7b31417f45d1e407b", + "08b58c89333a6a06c7714c86704b3f06bd2ad23a18360ca5e8ac7a2bde0649ce", + "08c6791c2cfa2927363fe34b6af41498531aa6816ad3edd85c3f9835a1ac1014", + "08db846838425971e9024677f1cc8e070c502bcbeabaf05f6f3631dbed7a3ed6", + "090f2f8b58a9dc0ae63008a8fac34a18b2aa93522542bb472a1202fa81e69465", + "091953502d8a89e9db47809dba001d5756c622e3aba8318722f8da0054bf92f5", + "092e168f994875a90490b287f3aeb7ad7ec27428bfd07d0e1ec1645dd2e01af8", + "093675a243ffec08a1c179a6757e916d5cd8dd9cc46921469e62bbf3bc12fba1", + "0947a97e05cd96908d88dd2db1de6be0175c37c2a635e5506ab0ef60723e9244", + "09a45c5c82ae74f3fa1dbfdd04e9c9cbb68ed2af9f96dbf6cbe731b22c5f00c6", + "09f192983656d6b34d005d8824dd443c5aa2a86212e1ef363d3ca398a61127bc", + "0a2105ebcf3307835abd4a7f5a7ea78b1b8061a118837a41879beec8ac9c4333", + "0a39ffbc1c4821c186d48f34e505d1dd889f98280e19b2967c7296fdbd4686d9", + "0a7a2bb9c6b1a50d1a9522163191dde71a243bd834f093cefb2ff1481dc637bc", + "0a7da33713c8db7bd3bc44ae27bb34e21ee7301152b2bb9d76626ec934e57f5a", + "0a994de647582a2ed0d6ce05c5ee895783f3d0a242425bd51bce5a205f5e0c58", + "0aacc576c7717acf5a36d239d777b8220079ff790301e35589802e502b65cc6e", + "0ab3580b189b821099bf7214360927a818a8e415f10d4ee7006a7d6ed518446f", + "0aec7bce1ce582391a1644d3411acad983fd582a91af97f199631e487d483d9a", + "0afa74ff678a87361267b42d66b59fab3466e8dbf8b95a5f891c25ee4a0a6ee5", + "0b034ef0b1596741e3285665c5bab78be118eeca05fac3b0685c0e405e60b01a", + "0b856ee1f7054bfe985a8a4929618c2f8c241b97d548c3ba2edd0ce3c8636aba", + "0b8bf874669ae24a086caf5364a73cc92e86e1baa7da60ddd21cc9dc58cffe54", + "0bedc58061fd753c26bcf63e803d240be5e7f5a044b0628b5d5e456b35345fad", + "0bee130933c311df93b7e72ecb227b6894a91763ffc616339264184411326529", + "0c0ab38d6a39a42e4cbcecdc53231d83e1b1e645f5b24b1a6fdc8ed2213d108b", + "0c3ffc0a5ed70915980abb6a49e16622554ee49a5730d559dddfab9115b6ab85", + "0c7a98ae4b33c672b70b431ab840bf764156ad0a3143c8df83ae1af6ffea6551", + "0c95367a82ef1977f91b7bdece3b7d452ac5947ac67445d20680d4324e6d70e3", + "0cb834d2ef4c99da95a871846baf987a939a014b4cca2f156974445a21c89196", + "0cc76f5c6a58bb1c614293fa3e9294f64fd03eb893cbd05576a9e93ee282ae61", + "0cdbd10f984a65865896ad1dc53188dacf141bf7ef001a0762752abc4207d819", + "0cf838444d8848a8b327d97bc22b1e3ecf4c14bffb554ebe9dbcccf35078d516", + "0d2cf77cb168280d8d9d3b9ca4d212baf17223c4da3f1f0dbedaf7d95bbfb42b", + "0d31cf5b5687ceab9361200aff34004ff93663803b4644b27d1d959f457d6090", + "0d77208f3083852c88c631f5a4102ad7684253ee7a8f2ecb5105725de98a9f22", + "0d84d509f5df6bc0a68efbeece7432b092fa13b52609b417755a8db67632970d", + "0d95a334465d12b48b4882fcb17e7bcf3c9c1e4dc69e753f044d91f7525654e0", + "0dc074e26e6b013f64ab01a46d120a9f9ddcffb756692e9dd7105039c88af908", + "0de8d03b14fedb9561b9015c19f8ecaf855b222a2423b0fc9a502374ce4a6c2a", + "0e0cd8063d131f6694b924dbd28dc10dfaf98b2f7df5ff6bb011ab411cc528ef", + "0e33d93a2b330cc4879f36904e581a044e9934fe4c2f28734201782301bb9a40", + "0e55cd1f05f691f5f7316ced762325ce7b77786edb3e9e65486d53c6e34d123a", + "0ea0a077924dd4cecbe0a3cf6e56b97055a69138eabc98f73d668cdc55a8d342", + "0f5b30575f5d8d9fbb772dde91794d51ae97b2f4a84b8986885d9ce9982bd455", + "0f60d64a0247253787e778e9e05d247a67c0c1e6ad497d8d596c13a8515015f1", + "0f76c338852c84ba09db812d81c87051f18e3a4175f866f1748dcbf8051d58b3", + "0f87489e07b3fd86800dc412d9dba47d153dca7d2a2bf1a2bd499bdf2c6ad911", + "0f9b2fb89c06a7b3f998a4e878ac7359a60ddb43ce1d9d46c5298eb2ce78ae10", + "0fd61f330351d6e64c0b9d1e6e17e73760f6e839bce6dfd46aa8958b0a00e0d7", + "101dfa78c8e779ffd065d635677a806d1220e5d870d64ac8bd95376840c9e7c0", + "101f400f3a1a701669887d3522d66c35e1dd66e64a55542c144f74dd4c7228a1", + "101f8a873e2ff5f5f2422c858ee11adcadb2eb241c8e329c5ca8d6b156feca65", + "102bd69c2a0b6d60b3e093cef71d13b91187ab631524b3fde873a75a795f82a1", + "1030bb1d5219a2c2d7b4560f0b8fed5a0b7645b178bf2a25224f3e8a1620a5e4", + "1047bd5df24c19ad178a6ba3b0e1108087b995bcf277520efa1ba27511aca823", + "1057678ed417e6ca1ca396a9143da3da3c7a44d7824c00178707a929a980993b", + "105fb9809fe0bc910f4fa8fb60b5a9bca2a517ed9781d2c99525cef6bd7a7c33", + "1076382ed2d4fc79f71805538631478e0cbb956e3884829ea3582c5ee8511b31", + "109679dd2218dfbb038bcb415aa96ae23b53c35a1a9cf156656633fd33b12051", + "109be5cf5ff9399f14ee012b0e509b867c597216a88f62b615f0fa4f79cb313d", + "10e050af1016ad6490f688299853f4c6a26b49fc52e36d8e84bb16b1d6242f63", + "1106d1fdacb747420db88c70e23d0522e849e8ec815670c0ff7c826e4a75d53f", + "111c5243a9942e1524b878a9931a8bab82b9f22dcb1a804a118b7d5745e1c557", + "1132f5177070b804b78222ed58670c88c119a10914c7cd820a71b18061fccc4c", + "113618a22be4bee42ee2bb850f8ccc18363063f6b3cdc8a30a8ca58b9b39fd3e", + "1142609193b022cf38eec3ec0870d6c51b2c09eaaf6138a8d55deb11a6c5f22d", + "11450674cb59ca3534c525d225161770cb93022c728f4835a7f52574e959525d", + "114707d425d986244f3e56284f7bd171563cf48974a42fd7a9280980c67a3f95", + "11773ee6fd7210cf7c79880cb39fa7cf7ab2829df19080e82cad70c285ae4268", + "11b4798cc26f0599eecdc28b4d71317e23c4acf33bb37093c50a39a188631945", + "11b56f2db4d76ba9adf943bc17191c2cb6e5a85d85a1208593cb7c1be386ed44", + "1209427eab050c56945e46d87c5c97c8037c55ddae6e4f0aa97a12c04b38226d", + "1233ac6385a62a731f93e267599b5c5169646ecc236c05196bb6256a4601e223", + "12a42cd1df46b7dcdbe7e2b9a63a24bf047a07cb453c4064a2f92be8e58e7a95", + "12b0cf2d082125418884cb235be8f0cf114690e28e8e68ff8de681b5a0a801a0", + "13149b1fd7056e439febeb35a6867dcbb149e9081d37fa40e83b1f2918ea5e6a", + "131b8be76b5625f24c2b90ef450e09681b8105c48bb24612d260f7abe159b85d", + "133e83462cc2534113fba38b5377d44f087fd0aa62fb17119a2d775c4d36ae13", + "135c96d100c2bbb210c05b0fd0ef4847d85bce2f7db80ff53a6db6a34388ecb8", + "13600885db220c1356adecf1f18bea3b6548495d5a2dd92685a3405e6dc32601", + "139065da5be130bbc91483d942c99736367a5b8e88fed0352cea387a7a8dd805", + "13ea6dc51f604aa5829da023023854a2495048e1f0760c9b3693d1b6d6cbc295", + "140b360113e9fb4ba876e619c0f710422a72e52423a3a5c80a7ed758f1681024", + "1443c885727a11619a79c4d8beb93b4f70e2eb0d682dfd1cf245d1e66fa67beb", + "145fb05e3a2ae5ce368a34f6fcd96b7ef31dbf82c770652fdb4a93e1309f5cad", + "146051089e993bfdffdd37eaaa68c9bede30409ab45c27b17c19c9bdfdfcc447", + "1480b5f9e56ed421dfa2682a888b5340df9e3417b2bf8c26eb628970dec5a891", + "14a571926fb18de1ad06566225635e5562408c6b1e2f78c46b959033de658e70", + "14b5b48bc2a11d15c666669958c19a316df7da7ae93585e4fc29782003f22093", + "14eb14bc234531012f5a646c2481f6f876e53d1549adc80ef13030c6ca20ba62", + "150e06f72127d24afa4e0520537a06b3c6948119d290c73c8bce75462e52b706", + "151401539a88f6fd7ef8230cac5454f2ba76c8e3413bbb677dafdf138979eadf", + "152b07d8b0784d39ddbc1d8033cb660d667cda442b82103cf2cee4bc344b62e1", + "1589857b2270441a120ba8e1402c97cb04438de24c09d023d2255f6916f012e7", + "15c85017940438224999372bf58525a0aa05fae056b9b62153189e1b19774ea7", + "15d9ef11aa04e54d29e0e84e71fe641dc54224f835c497ea74c18a6c2f710e53", + "1627d9af029612789104cebb56c6f6ca76795364d6ea2d2bb6eb4911ae636d1d", + "1652f6d3ba2e0a8818b2449312190f6e266553fe636d1f9ab834dd7984a01386", + "167ef625245b0fba78daf7d0bbbc5e6fe0aefaa9dbe1e57dac61fa631b475138", + "16946e5f2cb7e92c9b2e2be8f3e6e03e7c4a7b884986bd8db1b427600a1e6591", + "16a262c3f0d6c6f5df9156d5b413d2ac9c076056df7e3b059c0bacfd32a02c74", + "16ed36710a71acf4e83a0c735591a8d8c90197e92a38efcc8f5ecf64a1868ec0", + "16ef1975b3bdb478bc236271efdc9cffab09408902a5f97ce16f5d71136edb63", + "1705a03558836eb72c3179c8c3f78886201b56975c7b1ee7ef61a9cadeb4a57b", + "1711dc5e3e5570e0e338a8747c3c62bcafd93fb89e20f4596ccdf0da8c1a31ce", + "17218e8f2c3254089caec4db721323a772301788f1ae0e1d540fb82101af6777", + "1729956f439d4baafa124ad35906905bce52a38cee105d0384caf2bf4b1b8c16", + "1748ee905ebf6e5544ca8a20a2dc34e69272edfde8c325294b5a6e7feaed3445", + "174d8e6fd793f2b7032fbf0d980e5f4d8bd766c85f9873782481450c8baf2078", + "178a0f21298df6295ec0adc89eed35c2c7b0010375ade0b3ea639cb06e4c7c34", + "179c1cca46f3cb63e3f75c021f41dee206bc6a554eed8b17a49522eeb93db65d", + "179db1683a5d69775c9698822868220d9908f07495335ad2473ab81fd62943b6", + "17b9ff51808a80a062bbbf618fd89a0d21a707ddfeadc23243cb82a6aa2f9241", + "17bdbd2ced1b33326fd60ffe6f5af3a474d9323baf21761a637897088221a48f", + "17caa6df22d23dbeeba5faf91e1d00809f676ec09c65139d493616baa2d814c7", + "17cb801862c13d2966d77f9841a7a3e020ec4eaf3b76f14adb6aa1816047b308", + "17e6e2f1435e3d01d6654206b99090f419461dbfb37993d5a7fc128169279cfa", + "180c43a6c0de8a43420887064895b3aa42bc535816446275fee6a6a6af523c30", + "184ce76e1968d8a3369ccd694d266187a1189d85515eef1759b0c1c6f784d838", + "1859ef068a39acc92b320dadbb60f49dcec9bb2799dbc8c6f25f54151240f0ff", + "18ab98247de6b45481ea7b0934e111b3fb070f46173f5f586bb9d45f4cd3c700", + "18c879052385c74614d1a44ec0ca146ff4413a852dc5ed816189f45047c61ce5", + "18df2522938daae00fc6bf415f08a5503d5d4b600b060b5e514628b10a8363dc", + "18e88569169f27453c3a17a88edbe98bddf953e9022234f2e5a0bcba4849d111", + "18fff60f2e7239cd15e46ca71f35847df2b47d7e499474cdd1d3512894750bbd", + "191f38fba735695d000d80a4521af73f630fea434a5676d87d1b27c544bd10cc", + "193ddaf83e3bfa7913045010dc99c78a7900f686f81bb1a64c72c8402131b494", + "194d8b0abc3141bb77465d1a491f4c23166105fef6a0ec0b46dbab0fa0b27a97", + "197409c1266c07fe3c0307fa9dbc526ccee630d158390c766a97a8b3b521d16a", + "19abf2d41c0502ea57f2d87647da2fd9096befc28cb60052e5c1f9a03815c5fc", + "19b6783b167f943186601698d9096375a33996a58185b4fd8f6a1ba54222b021", + "19de583fa8e8a4aa2ba3b2362b6f4a4d5139dd4c45718c1b6a13f34cf5614017", + "19e44efb9c3709c75976aa81dae9336fabd9f3a16efd14724881cb3730964579", + "19f460b02fce9592cf785408a8e17e51bf35598a9f123262c56d9cbf144a1aa0", + "19fa18cc19f1a47e94f37519208bb6dcefc4b91981884f101182e1025c76433f", + "1a296224e9477d2b02c8f3f9abdb975a926f5b16f09a262c00a1db29c7e29127", + "1a5a2327dbf13f4d8062a3a676c7c3be2acc4469f771f49f385b6f608303f885", + "1a5ca891c3e8863c73d87d409285913f3ff31d7d816ff7d769b18424406d0b72", + "1a6fe7be0ae3c5a1eddbdea9cdb59ffec5cbe937bad1daea304c4a4c33ffdfc8", + "1a770631048156b7cb85f7f1c3604ba29f06fb0194113c0cdebb9741cb21cf0d", + "1a9e514e8ebae8faf4aa9b2561e82c4da462ac342967fd23eeb62d69c788c37f", + "1a9fad98a2bc4349a33356f65471b768c20daab285e0ec8926cc2cbd64519950", + "1adf55444a134beb42228bb9fdcea16e6ed4ea5514f059ad5f241bec3c07e58a", + "1b44657071336edcebbd0778f6a189a4c5d407483596e41b82cf2caa223f7184", + "1b4d8ab0d77e3a7b204f36bd251a684a81f307e3e5c353299c03e46527bcbe57", + "1b692f8a75a468d9d73d47e69bed36a362534ac7439e4b7ffb46a28ca6ae4300", + "1b938aacb3f82d2eba748f4d3ce039643df540aa52b8a226b1c47559dfd95300", + "1ba01ade66bbd22f55c0b6c4fa5d5ad80ebf7c31f10ff208dea06bfa5acb145f", + "1baf7fdda1317cc8e1aa739628f9f87d48b7e0fdf26e88fcad20f570602d58d6", + "1bd4b7607ffc276d7dea3a901f5c49dd25aa8450579c89f9983848e7d9b68b8b", + "1c260a74d7a7e4a86538fd40f497ddaa09cf0bdb87d3cc4a2f491d1b76cad2e5", + "1c2f381834ffddbaa77f1c0985c687d61ef5d1d87723ba0026aac8c53f6c61a0", + "1c4b6cbb1a2a0dd5e7445c1927f918866bc9ea76cea0d3c46c79421831ed4b09", + "1cb958ce0de35812bd01a62f72785aeaf1d2a0b942e3ea7be0b3893287d0bcfd", + "1ce18ff05a935eb586210f789946ff86af6512f5626fd0f6f3d90aa823c59b26", + "1cf5eb085a340488bf274e08c6abbafb32b94ff4456da6286a373409848f1373", + "1d53c386b644f1386934ba661826894ad14674b148d6199883619500cbfeac72", + "1d693019454f375a0ee1f16e9a0cc473cbf941f780b4eab01c210c6ab7d1ddc3", + "1d78802c9564ceb0481995708138e6af5daf67039ec5fffaa7b6e62c2a09b7af", + "1d8b9fdf05159ad613bdf0714f03e8b12dc33b3f78389bb315d13c336f8497e6", + "1d9d0ad3ae1bd0b3c2584d50c9f535b95f6f981c0c306a903ab3a14622fa4e83", + "1da17175ee9ae5d0b54fbb2a6afe84ab8f750228fec9acbe63af0fdc34e86a27", + "1dddb86c43db3223ceeb167535187e1dc4f77f558311137ab9598e173f42fcf4", + "1e37d61f66212b48b8f89e47c6a666b81600b191dbf1b93cd77301e19b59bc1d", + "1e60e9b00d53547071ebd296021a97f135965fc4d14108f39b2422776efadafa", + "1e898b25e792d44792dae03c892d9509da8138095c2ee7ff05f747df251ef63d", + "1e9370a197078bb8a937eac632798a8fc73f985b22a08e3e4f4418a7d252a53f", + "1e9a4d1e09c1d1eac8f0ddfc3732c0a6625aed1cfb4792cc9f0c35b226bea6d3", + "1ecff6afd06d6caa8afee5203626181994051d7a71897efddc9be4c42af91989", + "1ed9801a7d77ece07e35f4a3dcb5ae794b3fcb8d8b30472ba56281fb661950d7", + "1f1974c9c42bcb251ec0a6fd11a37ba070cba50c94b94d4d8173f17c7b3a8669", + "1f2197f0f714f26bbceeadb235929047fa69008d79e878abd900f1599e4eaaaa", + "1f5676917e8606c554ef1e53c14771da295818cbcc218eb002cdd7f1ed92cac8", + "1f93422b066c8f014f661708d7b347338edd9f76fe8a7e7b2318c15128416d7d", + "2006d578e24eebea02053b839228dc47b6caeacf097a56d9d747b1819df472ae", + "20403264c1f7c368497fb1b777303827fa9983e06820d7281639af0bf3f4e96e", + "20646ba67096da4fb1ab927a11101408c675c2eb98f47805aaa769a8b3db67f9", + "20bb27f8873c4d93c946b6891f3e513ec84ca2b6f09a73049952615b0248e599", + "210a7a5467b03c8bc0c2cf65ac6d5db5d76a98b285bc0a1e9c9c117b1768021d", + "212234d9d3b54e23a153e6222de1af78eabd8b93df9f6436078ee4b2a3223f68", + "2126cfceae5b8e513d078433eef8fe23109167b12bbff8d1c360ccee15bd0e61", + "2149243c6ba95fedbe578ddf8ac61ff2325b3ebc36705d3e278142f3806a062f", + "2151099bc42497224ddfdf04bd012b865f8565cf17b37ec2ccd7a4a73403b1fc", + "2163bf387c2bf5b804e987bf19844e67db7a90704c25bcad773aef04d4d859a5", + "21c4f19658aba33d1fb5c99d6e0d6c0c139f5218e44d78a77ffa5284896141b9", + "2207b41b5d1d5d2916d616e2af7d8aefecafb390937d39fa6e2a540a3bd10e4d", + "220831fdb2b9d540590561a6fa4d2b97ce6d6482f2ce58f567f9089ea8387831", + "22280f10fa1c1437d2664ada33ffba110c92a720bd7d5d657803207dd18535c7", + "22a36a44c5a57339ad24d7023c86fa5588a6300e8637b51351dd7ac95da29064", + "22f355bcedcb413a3a1dc60fe601ecca2584fcb94b8f2b56491c6361d23791f2", + "22fe153e2c7dc2a356f3551ddab83540615177d8066b248bf72168113be476cc", + "2314f1f7d20b72aa7d1d275a77ff5e154b3ca6e5beecce0cff9cba24fb95f873", + "23523c218d0de27396382b1ed9debbe4d895135feff96a92af88cafe23eee549", + "235c8c8b46ccec4d668a3c9c9acf7e1691674e6d76bd31cdc5b8919d9c62d4bb", + "23750121de6757ae56f097ccf73f62973ab859693341fc53986837e6cdef1a50", + "2376eb1a205defae31db9d21736c3a9981730d3e8de1c237e084b331f56f0226", + "23c8cc053a3e3937b369c9a82b97dbab7c1b3418079d83bb808c8ee6a8cb977f", + "24037ecc9e35cbc55aaec3c4beac71aa1b73ed68c84ba0882c32d53a156835d3", + "24063a84bea566065e3eb027a021d7321f6073ab346602abf8bbd590a787e463", + "24418099cd07ceb506972b9e4e1787eea959ca7655ea273a189bdd687bb39f54", + "2442a7acfa983a1ea38aa0f9d72e26e644620ea256c96537a9ea9146d9453022", + "245709bde24a4091af4962982e892c9e60ecbeebb4467db6ce16bdaadcfb3d9d", + "2459c1f49eef7a04b4e30c35ab426b182f335c3502bff32a32b15b42f73ad032", + "24894d48044a516cc510afc78da561f4baffc729562753188a040e7378af3d26", + "248a9d49901fe119fb30582e47717afe7ae45690000a45fcc8ee35518753d89f", + "249ecfe12202bdf2985659df13fd59ccb8802ae2dc6188499f1386d7362c0cab", + "24a0451be2da0b8e65249852523eedd2ec51ed37745039663151334f2023c326", + "24c84ffd078a3121a4145a43c56e4f1a2bf0af2f3cfdb189f98c838731066f04", + "25063678270b59e0dca5a6da6eba3292f23b73d217c9cac51605b15560e77120", + "2518b9c1010b8d9b2d7d1cd22c1483a017a24ce6480e7d11b44e414ef44b0765", + "25348dd365bc036dac3ca8c5d136c534990cb2faa13ed65b9beaab47203d8084", + "254464dd9570b6db1fd50422d4c43c65e00e79e73b7f67805753d87663fa78c1", + "254ab0473645a7d33ea1c7177f62059d16b3aed7d778d712fa260af6b41fe5ed", + "2562e4795a7465ef02f84e915ca1b58067581b7131b29252f9dfdea193c17fbc", + "25b468e54479d43367920591e3ed6aeebd9abf6f10e1eea7317a9385008d1170", + "25c0469c9130572e4564ba51afc94384fe426a6bb26817c98e57d5bb2226ffda", + "261a4e8be9e066d28fe884861cda7b2f9157f907f5fbc1e367352c42f3878753", + "26212fe5d8453ae051b0ac57fe878c9227f645900a62cf21bcbfdf6a0fc34c7b", + "2656e30609ebc0b05b195b9c0e09ddca9d3dcf58b9c2407eb1baa979794c909e", + "266db8f2f7eb2cb3582b4069583e6e96c68f8b4c2111cac4214cd4572c1bf692", + "2679142ca668cd2ed57827268096b51b059693540395cb5346e1c31aaa903641", + "26b4a45547ce31c9abe7b9e18b8dde2b06083198e899247c49f3da64f310c18e", + "26b9354f82f2e5c5d5264dd5e3d91b597a90830ed119946f4e086ecbb69e4c1c", + "26d40782ae484f44ff1d3a6c9a760c457b48f9cc56194998c017ca391b7d597a", + "26db33a4e5b448543a2a7852a03a797297f34982049f1bd0dc7ebf3807f9f981", + "26e92dd8552c48b677811eca3aa2246840723432f17775fafe537fdd1fb61302", + "26f397e39bd00131ad71a7aad609c68e156ba48c68b264166234d946ab4419d2", + "27066882d04770d7e9e64e700a19a6f9370b9a34ad3802ca1f9bdf19c37770bf", + "270b00f071e7989acc5fce6eb464501e6b57d9c336002964e9eb65617383d72f", + "273a74693860dfe7fcfb511a287decae91dd4c8a72b71af522017505694a6794", + "27407481c77c04104695dc23de6a69977aef5457e2ad988c1644523701f13d7e", + "2740a0e42da39c2fa8c568cbe32098b1c3d225756665665556f3f7f6a43a0495", + "27858d3f5cb16b93823d8914d2f8c3372e7c686c9e54b1ffd81c5b934ad1e5d8", + "27a7460340351ea1d4644e1db88e17f3eb72666eb67d741f28010308ab3a7a41", + "27b0bb9643a61edfdc28887735cbc0983b19ca2d152c614359ca281fa5d5d57a", + "27b242c5b4cf58073dc81620e1d34d802768844a3a6b1f6175330bcd5dfd74fd", + "27bcda081c48a50079e57660ce90ed1035693e751717c2460abc3724fce99a8a", + "27e7476fd5be3d2d690ae3d918652b34fb880dcbdf5868fdc723b92538497555", + "280189bb83c0b59ea0a19cc763241246d9c875cc395a2a44b8981b58913503ab", + "28124203f106909f1df4f1eb9f24106733b41600a23bc9ee10f3500647c78219", + "2828d558dee61df8b392e18e255063d3cc328b0bbf2bee4245c4378cf3452423", + "2855125525b095570c5c67a3e29341c7acc8d003430f0eaa7c9ffd3b62a4b82d", + "285ebf730d8e2e3c9de075c89cd76833a4604d7bf38e55e1f232514825053166", + "286613917a09de3d887c18c08de2c8318e746292f5960f14629b6837fcea83e4", + "287ab60d93d0337301bcbfe0c86095fc2efa96ec49f298a840203dfa6a6811cb", + "287eeaf0dc4a3c7d7ea6d5a3dc340ad31e89a936ea03e1418e7d334500e9bac8", + "2885cd50b962d924a12f2dce415a974cef12dd736a2d3d6808497099ad4242fb", + "288a9d61d75d3a9ac82165408ab2f83d773bc6976d0a596343e0115f37baef8f", + "288cdd186e53aa4e0eeda60339143adc8cf1b20a7568a5275edaeb2ac5d44d30", + "2892fb34d6fdf1048f8348e29c1e2c3eaff561a648a9016ecac7390198e0a8df", + "28a2333a43a7143735131c539910d2b9a76961757224329d64832322fe34de18", + "28b23e9ea4c919838a761bb97144a6472f73bd6f7713209efcee7f30f2735306", + "28b9b8824154fa65a6140ea02b8b19d23f258f3981b58eeb1157aa0157d12a55", + "28efb908bbddbda776159898adba8ac6c589dc6b3d37829b590c807ed2eac206", + "292c695b9d20625d27207381f8a3e70898aee3771622f38b71e090a86afc0a58", + "294a4a2c1a832c0388fe80f5beecfaf4ca644cd360c07309573cc1577bcd972e", + "294e7b801a7eb160b3a4e5a424080598ba1f1b3f73400f5138a60c3ef0bccffc", + "29744e52041dfdb69de822316a881dc2e91ac164a8483c53c3f8cea36de54686", + "29b0d55d68bd9dfa81634efce57079b22ef6fc8199b081b7c1381731de5c48a6", + "29c4bc894cc82a5ab9c0b81405b35d21f9f9cff75b7f5f33413a1fe01a7a7943", + "2a2c87339df141c33a723fbfdd435a731a0d6852dd65ca14e95e6abf6d92e75f", + "2a4492b8f60864d826a76eed372bf588abf7407050f4fcdbb4b3ccfb73559183", + "2a5f58badc336d55b3dc5815488b62b0667ec6fb54d6bae91243f46c9062b24e", + "2a943b3c5c254092702cda0c505dc040cbabdc7e04bd7658b35481f0c5f699bc", + "2aaaf70ea612431d001d3466f0b41627abfe991e7ecf768ebe8734785812e243", + "2aceba1f4c30f8131dc9ffac25a01aebf23b15c34b19f5fa8926aa4e63a7cd8f", + "2ade1798df72258a18c346a6051b1ee1779e06d9dad7c27137403468fc630bac", + "2aea7980b8e26c8d434644d2b1a4c65ccffc39ba85b00eb8de41675c42756667", + "2b0881118dff8870ad09e53fcd20f0e4663e45c8ca5f38df1c4bf17361a2a13e", + "2b08e0a8c9d17fd5c672d90351625b6dd1ab26e2a77f68ae04ef065c3a3b8758", + "2b360994ca527b6f16c2aecba4370306e7afbd6c7d20995e38f96bba71da6be0", + "2b38841187c474862deb3f9e0eb5065ada3046739c7ca2b8f62f0c85a62fc8da", + "2b42c4ee1c3d0d4d83dab44e08dd6ede637b0ddd7c16b8190b5b08988715b45e", + "2b4f44a83e139d6234b2609433bf8436c4460bcfe236addcfbe63595c8ec994f", + "2b621cec625ced5ee26b5a268b34a664e21be930eaacf81e052434a037cdeac3", + "2b7cc30d5340a2b66dcf09fdd5c6e5ca85bd388bfbd3345630990eed07578f80", + "2b9cf96df99a58f4ebce65d21e097d819c00d44ad2fd27a98d3308aa88e6b3d1", + "2ba4807fa727358531b1957300c9abdfb1ba797fabb7a7dec22a4b2dbb8c8171", + "2bc9be0fd0d56c399fd79f919ba90ce52faaab5bc69a62519ebd214e17489472", + "2bcc96d40c2a72958ad61b4a1aa2123c8d356bee6aa1ab4a661135a5bc90569b", + "2c1654b1ef1308a13bfe2ff712e531a32297032794072730a00ece77f458fe61", + "2c27838dad793f315a486df8361278039fbb7199008c026422040c445f2b9f86", + "2c387f23eb35c3d4a52f59dc80cd87a9fbd1ddf6314944a60e7fba59d0765ed3", + "2c43cc1bbee25412c64d941f66df407bfbf6b4fa6dec9128336bb1f301c6d7a2", + "2c4432fe90d8817433832a4800d0bf40c932a4692404b1fa73eab43bfade88a7", + "2c70cf437ff6f42edeb8f76651a33fbf7230d6f0b672cdd08bca19916c1e30e5", + "2c7e589a02095c96e26ff938206dd582151dabadacfc5a8721ef8cfa86527b94", + "2c956b88706537a241524c1abc8ab57a320c8711f8503cb8aaa4bb17f3200cba", + "2c9add30016eb8b582370afb233e756a2c0e71441505cd96142f27e36e88c1d9", + "2ca3401452d5d5866ec65739ab2665c8b5f0be1c614dd7059de089fec3a711e6", + "2cb99fea0cad751b8a4a1b941ed87172d52d55fb43fb83638e31b9efbf4233fb", + "2cc0152d812d1e4a0d511ff3c3a9916e776aa614a366f1f73d3106e6897bc52e", + "2d1bb608b747735c0b5a69c4e51fd79ed6b5cb826d2a61cfa642cd13f8989b41", + "2d2dadaec7a8758c8dc8309497a471fa2e019e60f9d6cd3aa19b4ec0a51d2b99", + "2d4cd5e46d3aacbf9b82a6edbe2411a74c4a93212246c1bb2372372e4bf3274d", + "2d683c2c3b49680b6de9b50e2276b7bdcc137261f37cabba58a7748d299a5494", + "2d7a92c2c3fa714388b8ac5806366ceb8121ad8f5d2f8b9984b2fd1ec6b06e37", + "2d89f2626ffa73c6be327d17d2a6db0779eae4beca522ce7f24665f70ccad02d", + "2dbf667179319df46d094d753e44d5dfaad646777e08b29b4c51d58c6447496a", + "2dd0949bc9ed9322b494caa15ea82863a6a116f20e0e6651c1d5e6b6eb43f52c", + "2de26fa321cc0f2e464c49e500b33d39797c3b6be17a60668063ba26dfa76e42", + "2ded610c1de810d2c462f51208ee2d8f471d9a3995a6bc6932c5fac629560bdf", + "2e119b1f0a1c7fb0f1b2c3f5ec1db7e89f7657c009a48b309538b396e36b2ad4", + "2e28d2c5a315902609c458005876a85506dafbdf01751fdcaab44a68d0a3bfa2", + "2e6dd85f441bd65e0fc4220329cafb0408430e5d8de2ab07a5990095ff43e78e", + "2ea77ca16c79e921ab5851baac1f0fb33529d7c76a7d2526fe3e403761fde1bf", + "2ecfa522280f5eac8bdf12c4ddf4c4a33c5b60f36f306a8e2561bd7c05d3f148", + "2eecb84249abc69da89a65bc6cafef171db054f0399e30b7cd2778512941fae8", + "2f53841fd202a63ce5944e746c15fa26e17302e631b041f1b64022545e851d77", + "2ff450770fb2fcaa0867b408f1a5e66c2b65770311e4c4ca0eaf4e68c6e04a3f", + "30511fbb595277baec4c896231d4e9295ae744a425e66949fc1c9f745e951a7f", + "30653eee6ae7bbc39c131176c37c2b9fd5ec2c50008eb84f5ae21627c1fc12c5", + "306a71235602fe4889afe2a1efc2fec8907396abc8edb9b814c710abda504325", + "30aae25405ca1260a5434470f4cf866a9039e557139dc0260f75f35dcba741bf", + "30d6674f194cb2b76d9e31310c49eed068a97c0f27598c46d930a2f6fa59ea10", + "30e628d6fa4953917a5c581772f8ddee9fbc06c63a4d2956bed54b6ea7323c73", + "310e11e670166746bec5b4b32994cc60083db42de75f51d351e07fcdc7d3de46", + "31121904f1bcb9dca43ffd7e0ba17e8b4553fefb8385245b890a54e572f8ed32", + "312ce46a276e976e2336873ecf88d8e6ce8da47e9d1c280fe7934b1cbce52834", + "3133263be7c1529f939a42c98f9f5cf54339ab8cc8ad88ecf24b653c2ab2e526", + "31619efc33f3642822abcc8df062b55ed10485df2a6b91649e86018ab2f2c9f8", + "316d5b71d81c1d3b43e88a63aa20bfca9a2ae50618a4dbd9c3338f5eacbbbe25", + "31c8c284b98bdd6e6b6e76e48688b98041b67fde8ed080bb821b723063f24866", + "321231bad3b78e54ae2e36b015490b90756bb489d54c3f4a474665fa9db95b23", + "32136646f2c328cdea70a654cc9aabf770a36f339acbeae5f8948e40c0e2d3da", + "3229ed80b09d389cd3ab365025abc3e5a0541efccf5213b41200a8dc12d24ffb", + "3253500b87ad1014ba69e19395cd95459110bc8b1dc37e4a906778e19d71d04d", + "3257111be2381ffac0f0e668d5a3ba794eb8c4c2dd03e57bf13bc6d4471b88b7", + "326c201a620cc66c4a4d90735a7009ad601df58ca76002c49a0cb39831f37d71", + "32a616312a1e0a3a8b2b427320a861a432451b405d7ba02a7158d76b055f39a9", + "32b93c82ffe40795f963cc8a2f0b18d9ceae277f114b027f79e4d36fe9b085ab", + "32f8abe41933ba53748abf536e0e0ccf4cc4e79bd65f4c33f1df78f52b45d970", + "32ff9812853c478ec4ba6d031cd30a38f47f295d2103f6bdd1e28450958fc14a", + "3338ae2f7ba490655d53c884258f45d848437fd173c216a38837ca2cc09f8e5a", + "333988cd2d484d4f92a17e109724085c52ad8c70ce1759c53553210e50d0f0c9", + "3355c5148bb66e9b9638e671d428c271469a8ca2de794275c94af852e8209424", + "337b9dab25d59bbfeac2ddf9ff3b194f0ed6c876b99b071cb42fb2574a58a0d3", + "33b73f56f58baa737f8591faf7618590c2ecd86bcba29259e69585da9dbc521a", + "33bcda29c685106c1c0f44fdb5946a975f15b59275e8ae3d7c0ad7f8d404987c", + "342b09937c0ecbdf5b01afe4d178961d86950c31daa9d90fbb5f9bfebb5d8a3b", + "34373b6e9907806a7985134899c6979611d5c9e7e176d41b97e5880ecbed2fa4", + "344a203506196c9215b5e9f57aacda02e90cadd3a9430befb19349d13afd9178", + "3472ab26eca549d514b26dde208c03dcfe5a7a764be1c5514b9bf3aecc6a66f1", + "3480bb22f27a0d18b2df530a5dce8893367e0f3855fe65da9a1be6638f97e27c", + "34937198be44c29e1ed3159e7ec17dacb6df05fc6cc9c0b1663beef0e95462f1", + "349c29d22cba205fe77921efacec0ac6ef8b71ea131f273e91dcc5d042771f75", + "34b6365aa65685adfb11f115ccaea4ad1bebf026fd9e7b0318313a7d75a27c18", + "34c35b63588dfdcb6be0ce3eb3398ccb276afe2f2ea8a6ac3a30921c858d9b53", + "34d179a8f3ec05d380cf2667f5bc24b504fe9a125ec62f5759bf76225d5e50c3", + "34e69d9035c99f06a5aa13492a356a90693976f53d37de5868016b5519877481", + "34f74afd2a016440c205ac27a027ca4ca31ae30eb360a5241462ff6cfde32dd0", + "3540be5c5eb3cd046b586a603ddd6b954bfae55c076a97a22f77e4dd40ac9493", + "354ff8174aa039d4a22a628025b8a9a6ec415f7a6a6cf5874a722c1f36b7da40", + "360027a341c08574c0477f28fdfc05e786916e797b7559af6348a68273739d25", + "362be82bc53aeba93a56f0c4cd5f2a371fcac8f45d5e7240b2866865e8a03804", + "3690398863a71cbf8c4fca9137a29a0e6e15b71470a2f0fa1cdf06a5d0986bd0", + "371b1838c0b76c4959eb92dbb7a645db0e915ffa25c3906b2959d5cc627c3fde", + "37337b4f60f4e996080e8d99ab02621bc21eab68b26607ed1e8ff6a7797b0ea2", + "373f72cd6fdabdb9446a1a73ffcec5022c36c185aa4036ad5429b02f7f6ed666", + "37522879f3d15105cda2d3ab6b48fed2900198e74e844df6fe086b385d0a9122", + "3790c32a444c5c6f3edb16980563d016559ab40aa9a48700e292b413223ea56e", + "379eec9aab24cee73bdca8e950ba019a1f57af3f88bf83f14540496a223e8420", + "37ecce1d750a93d7ff5e6ce79171495cc95a596a892a565921fba2898fdd868b", + "3827b9b917f9e9c1e0c92f935c31ae04e70a8ecc430e1a17df84688c499264cc", + "382ed00337d1740c1abcba4e6b069e5c19fae28ca9f551087ad3d8223b10e821", + "384398950404d022e3d937bfbeea616ed984f4800284a2548af95bd875e718e0", + "3859548b787c8da0ffb479bccf43817cec3aa1e6ae9c065952140c89add2b017", + "385bde3762f16b48d40ffca0f7e13cb61a3685cac72681c1ea6d46b8565fcbe7", + "386620005ea0a9aeca9e2115e73f6a315f651d1e6a33c570438104467964ff23", + "387f75f652efc68830d25b8fa3e05b3e67b26413579c7ca31092a97da03ebe90", + "389fe21426476539bc015e368b4b1db1d44dd895995e1aa3a9f082fdac2b995b", + "38ae7ecb7169e273a8ff2ace53eec67dd77941dfab4ca3610f84cf489ac06c1e", + "38f13a2e254e38adef39a763b20eb8dd3ff8fdc529bae0429891336ee448c286", + "39000ae05180499f9212cda76d3251fc437ba01b55d5cc9387822f0673bfd14b", + "390f26525ac002e5a9812985150e0b5a53bd4bc13d39d33dff76c58c5943ed1c", + "3922d60da591994cba312d2f2a8fc8d8117c09b20b198274129a070f113cd5c6", + "392d57bad566f771464837d462a4f940f6bb8b307ab19c5e6cd0b93b94c90d10", + "393395504d82649ea066d408f61d423ade8334b7d7bc95f2949b1a91a3c21898", + "393400bd5410c8a61b80eb939fd77272f5baad15e273c7401838185b5fb8d548", + "3937272e73fb81158ba96b14584d01423a8d789a1c439c3ac19085358e4d605f", + "3940d2f105eaadd7273423da9474f33a55787dc9d4e3c4c334ec2fe28f4a568c", + "39516f23294ca9fbc6114cb20c160c6b7f0c205d63889193cb85f834e1bb329f", + "39570eea5b1d6207bbfc9b1792bf673b1c6aaf0e8934cf232a771fd4eb0b42d5", + "395a62303d9fbcbf1a19506d29fd1a95337a3e998115b8f45602dd04f19f27d1", + "39764a681f8e76c170113a5ffb08d4678cefde18fbe999acd5134bc1333c4a86", + "39b3d97686cdbb4c091fdd041c7ce589bab27b9a794f798052cb0cd13fe8873e", + "39bc2f74fa976e69d6dd2a8c599e695dd3ce6bd2a28fb764ace7acc236da158f", + "3a0a35b88e4e046bb4809979d6c0638f0e673a23342bd0f845602f9d68d83446", + "3a8799a5aeb66e6b62a8f9d1971a49d558032ef93454ef38af18d6ce59d58c64", + "3a9d4a48fd9eff24b30053f0563e2cd0a8fb066a1f20f131840a902aa8d29440", + "3abcf8119ef5d704352ac5c3393f8905d420e5cb6e269df9355c046d33513644", + "3ad8e387b6b41cbd255f061e746d929211b17dfa1dc4c2bb3f988a731a6a1467", + "3adc8c039c613b98ff8c20e0670f7a758ed2e9e39feb135d9f5365a08ad47615", + "3aef058e6d5fda406e7e9d808a96e70f18ba8e13162edb2908661f7cf00ce7f9", + "3b072f9113509c9d2cbbf1c8844cdef49e2a497e972c6f6671e89870fdcf7012", + "3b2fd03b31505badf3cc0c0ae855f48bd2783c9893c35e9ccc88f55110271fae", + "3b76d52cda8f218ad86f297985c5938c24a4b6b632854f3011494b7cb744268a", + "3b9e0b44fe5d97d8001018fd4570ff07c06d80ba961d940a98c4472ed3ae83f8", + "3ba52aff6dc34cf2624a72e08e61cd95f3de7295b25b0cd669b267c9bab081b9", + "3ba52cf7bc2fb85c021648f84774c9837dc9612bd7a619c517c1c66e4ef44875", + "3bb56255d36bc07beb145539c98a47d0d3aa1ba617524b5948e3b2c22ad1645c", + "3bb92d37f525b8e3c1af2c617cc6ce9c7780bb4041eadded976f54ceed6cd655", + "3bc4dbf30cb70d551f8355bcca87cecc0d215b5744e0b50c9b0c783ced0a21da", + "3bc883d49816b88701ebaaebcba750fba11e2680392e23ee31f486c0c5862b68", + "3be2384dee2306d6a9284b55449b778ce79b220658a9b1bd21dd9118ab14ee21", + "3bf059c7df21bfa5f8b729db597f2ccfa845b0b3b1e732fc1e10f4425bcd3782", + "3c780b30a7f1aa34eef3a076aeb0d0c9e0b6233d652829a80d6d09464475a2ac", + "3c7aeffc56ec06c0d657ea706dbf66038fb53b3ab45ce57abd52c188a56adc31", + "3c9c25ac5614ff2f35703f3c8b16d214c19cfaa98be57f16c62419a05248f1ec", + "3cae26f451b61e3273296c01d01d60a12c16719d2ba21ec35e56f42cb37196b9", + "3cb0441e03264e1f033bcbc9a5bbb0d4410cde112281ac6549a918354325e2fb", + "3cfd4dace103cd378e4de22dadcac1e856aa3297ba31185bf3324ad0de33c9ea", + "3d178d2959ab3643134b553b2bf9240b0faadd3169023160903ed01a928021fd", + "3d2baf4da5720c05a85d5458cab3caf75b88dbc812e96696877dfea74760ecee", + "3d5b80db97a27e956b860cef66224a4e13774242ec0cbd808cf605e22dff6ed4", + "3d6e35262a8b1c59a282867c635be36e6d3a19a8d926f243f8811ec445b36782", + "3dbd487c90e62aeb025e3e17bf1dada719a29d9d02eb890b08f8d335ce9bfa82", + "3df597c5e716d2d5c9df79f49d6d278fa9cb40e3c3dc9b62d0d570a8f6e844ea", + "3e8457255e410c29054eb03ecbe8b264f1b3c5f74cb676fae8febbe706f1ac01", + "3e9459ca0a452caa4ffe6c08f088f780b56014a8fceb4475633d52b79ec71102", + "3e9cc0fb1dd20c269828064dca21f3594ddb7a2b2de3befbe6a5aa7d122210f1", + "3ea36a3f2725b618cef04969dd397b1631dda55ae703a926f78e6ce73bacd4ce", + "3ec85c9d41a20e1fbd27a3d3a341d28dee5281763720a6e6198b3bee1f336d37", + "3ee3d21edd0432f918f2b2f2da8d028916a311c6d1db676955e5b985ead589b4", + "3ee6a26718544fff1c7ecf9b145d04616e2405948378f9ec342251f56d9c233e", + "3f006610eb2c7a2e531da7e7e997d21202e3510dc0ac56a508ea567c3b7f863f", + "3f39376f90046200bff93b447e0e6a0dceac5d3c783811289e0b98cac2c429ee", + "3f836c4a17f0b9d0af547785829a9330ea4702b83d86098ff7dd612c3fc4f642", + "401436e97b2f59480c5c391ab5bcaac864876386e2bf617497cf181fe45690fa", + "4021fc9ac7e29c383c2a74fcecbb0bbffc07306dd5ad62445cdc087135e45ad2", + "405ce23ce6f135491c13e994d6051c77b74bd5133bf7b92fdd05083a78a306d2", + "40a61127f13657828dfbb6be83c8d0ddc2ab8b6e6ea8b0d23ecceaaed43f9250", + "40bc5caf332cfa6dff82a65aba8c88b49d8d9258240b7dce0a0a208ea44e2397", + "40fd462b084290e729d6e367d8e05017492546885d3dadadacc0e1c5d045131a", + "412893375eef842497f0b60c9a438f557df0632085d4bc926c66b080c40673e4", + "41b4b99877e41a0d93cb88c9bc81db9ef7d14c83c6b9357b01090b103131406b", + "41cf7462c3ae41845da13e06bc518236acd8fd18f38cd7e1b279c38678392caf", + "420435e1d25b95bf6ab1c9a55c923e409d5da847168091b3760226e243ba2c14", + "420eeea9117f59559bd541412bcb29cf8554b2ebb1d4b8638371e8d73b15ff2b", + "42218ce98ad3d8d50bc9cb306062f0baa439662373548602415acc7e59d6e9fd", + "42558ca06ed02105c1166604d705f4f767a1e36c7a78618437fc36f0fc7ba0ac", + "4255e9bfe477e94d2520a86c29f52e6041a4b05683c78288689f0d3254db0d02", + "42649b4f1d4cfb99397aaa33778874f4012567a6f2dd991393b094ae0596857d", + "4265b41d4e437989877b223b245e1063734921b5400fd04ba45ac53c5ffe90c6", + "4267a36f64ce7838ad9e10bfdba933172a098d263bb63c86ad57267027942326", + "426b2a61d885d5483224791a3c349ddbaee80a35e9c40f80bfa17fa1f7ddf578", + "42777dcca97c89d7e5dab61e5d9d50f5296e32579e2cb8063baf1f68b8921d90", + "428f274d9a39516b33a8ce19bbf358eea69640c134c81028375e2d49aaa891a9", + "42922521409bdb49b86404d39136e93ea75eb0580955c13fe8cd2cba85756686", + "42d0312b0a32fbd94ca1b9eaa4b8e3129fa96947832f97a6a44d15675e697585", + "42f5ca82146746b778bfe5ec97acedd150aea7e3294c9898623fb86a53b70300", + "430d0e82480ebb89b687f5648879a0c26589aa9c9f887d3327b279c330b2b591", + "43295a9e46b5e2ec8e671c5a130fdd7f942b7efb434d66f7b7835620e2a859a5", + "4349b4d1462a48bca23750e495a84a3b950f705dfcb3d11c0cffe385eaa19ee3", + "436a928e4fe1f3e129a24d447dd62821a8b2a5c9055189f0dea60828b7f361a3", + "438e8de1b84bb27d2c051c461ba76ebc5de760814e0988577684197b0005b3f5", + "43bb7b981ac7954e20a941c170ae59219c91c77a2966d776942db5e4028f24ee", + "43cc161ca50b1b7d86a7e025187296c87c6d289f8e9e7bb86f8b6d1edaf7f026", + "441586d4cbf2db66b066c6a372442092654c3d87158fd9e46e6091fb5feb634c", + "4424a4bc86c0f46c8093e3bed4442f576579feacfc43f2366f36e4fe6edec869", + "44650da1115b47ef24e431eaaebc63ce3482b9888382be4309e9e08ae417f4e6", + "447de9ca3a4c845fd9fff0d19c05e240558368ea14d8a481281c42c1b3f12984", + "4498af1a55b9b415d066ded464919d2a55edb6969b8cc00092adbc52edba84e4", + "44cb0c73cd673eb2469fbeb0f283e7eeea30e78fd242c7fee547887a1acc4f92", + "44cc9e16828fa11b5d0a4541d3a29383c8f97c4eaad6f8b8cd7ad9b804f77951", + "44d383d0c482930c758e9abf75ef1b4145b7b91a7f3ba4a8e41563c30cc60009", + "44d43be322fe6f463b76c7e48e789ff01e07d3712606fd6155204d9e96fa7f53", + "44e9bcea8279515cc4de38c7c1b47cf7db5b1d5fe7b3373f99f31a334341f2b3", + "45092edc1b76ae550294388a7008a4673b601988e5d4dd47e0d5e42fb3e1ef01", + "45607e5b779450299b4a4e3d2bec1d1bfaa5de408b81221fbf109d782b8dcc06", + "4591e8241665cbe9eb8e1e475f97037849633d40bc1e3dbec237d87d292913fa", + "45c3b0d13cc47a16729ce2bdc9c4463a2b73f299ec743fb2c3cd93dfdab96286", + "45d6fe5461af10acda5c405e9d1a098a709cdbd59c89c6da0b41ab7b9565c360", + "465d1b63766e37d26df675c644396e902d587b2ed23cb7654dcb05fa05543133", + "46844362e00bd617fccf5dec58b106fc8b40c69054f5ea96fd3f08a6f7106d10", + "468917e21826e30fbf98f2b87d396d0472fe5b98c216eb12819fc588e2288f32", + "46affc777d8595127e09bdbabb975693f690868625ca609955ac8bf61669c912", + "46c916e7635bbf037b90447dde28310890f8b755c054927867966849863981c8", + "46d8a3840800d6114f9e23ff4c6920318183c011fae7580a86042feb881ebfb9", + "46dba5eba3ae9cab894e7240ac8ea3ee6475750d6c22bc1ab75749ad11c90516", + "477dea60e11636b609f77bf206708ea7dbf025d7729d90f692a3e732b08b21fc", + "478481dae1cc617790b53d41d5783e8000b51a433ce3bd0979e2fd5512f0324e", + "47966500228dc154f1e5184b4eb5c1d7079d2cf0ab6d6873751954f85ac2fdf7", + "47d6af26ce42dec6de81734d68bf4d2e12153415a0048b9645858e67a46e57d0", + "47de3552f64938c412f636cf0c628e63813bc8464b3d15d6b39b25bc509f718b", + "47e7e6462074a1a7dbe81ecfe3c03772620597901fc0063e09b85f360053f888", + "47edf18219d7e7f42ce9b3fe479cfcadf04f52ae02a7cd02586b965f5a9163d3", + "47fadbe243f65705a8d6f991f805a499136f721c1165f8e6f93f186f0a0007c7", + "48130f01e7241c2a0cc995b602940e1a52ee6d6e4bbd1f64580f9ed0817685e6", + "482d446437c7d574e8059cb27f6f258acf28bb2570996ad698ace431b7bb501f", + "4844fa3bfab190bfb8f62b622ca7923aa2473fcb339f49964d2043046948edf5", + "48873a484e5925c760df63afa2c3c1a66198254db1ca5717cdd081b5c79c0516", + "488c31db771d6fb4a209ccc928baea074f090e7fb56ff863b2e569d9ed1bc372", + "48a37c04e31111c927f2a066db22f491afe23df1eb94d9d9288c999f471aa982", + "48e31de5c3115f218c5eae6d25a2d4cd097cc97480aa61a93b876eeae03a1871", + "491364a39d23fd232b902a1bad2e3b9aa7884909a16969eaecaca24c4d125257", + "4938fbb14d7f0a430b584a9a5d28a53667e267d6669b4c6419dd6ec9bb6b77f4", + "4983a313916042ed6a0c6c2bd9d48cb8a2904585e99db84a6a7d9bbbb105f99f", + "499aa49d7834b33ab4bd44e7f51e7fca36bb850bfc76b3b5b8542db1df09e628", + "4a2726cdf7b53daa50ce0d8f82988880cb1ab67aac8e3eb096e386b06ee377eb", + "4a2f05baddba7f86c20f9bd95ec3a75ba9e604c10ddcb3c0b46f0926feba4bc5", + "4a459a84cefab2ae16c9dd41d5a5e38abbdddf9e88763fbea8a358e8f0a45450", + "4a4a6d458d69c78b42eecc5d11a1565e210edac4cc7754485a4f7dc2ec353a10", + "4a6ef7f11a9b59fff7083b81620a26bbf8bd849ccea64eac497b37a58ae69826", + "4a7ffb8d73d336b41de75ae007d05cb60d78fa29f979886dc1f4758cf66a14ff", + "4a871532448ee3c6d421c8a462db8e257039a1dd1547315b004671a4a3b79eec", + "4a8761e3fbb633cf41fee33db760dcf10f1c1c281fe188bdce901883371631ef", + "4ac00b172d90a8f879b320d58f5ce6f177b4f0f1f2fdc4896d8513121e1bc444", + "4ac2003f5ff908998db16efd8639d53cb3d5d26d28fa7221ed0f8163f6c953d9", + "4b01d0ffe4419a55d185fc6a50f1b80d629d87d5d44caa68a4f002cad0331eb7", + "4b10518d510ef77e3c7337bf0761660350d2350b22cea604bf6da5cfc569ecf7", + "4b20bff40caf59eaf8e5b8468dbb031f2239f249f8eeb09eea3a2c5aab036f8f", + "4b3210a8d9fa69888af9d8c56c0def3bb79e5e05f0a8be419e0bf58797864eda", + "4b5b4b5c0af753edf62720b09c8461fbf6917106b875aa95e5429c8b66e08c76", + "4b6a1d6c205a38fe3b7c533c282dd7b29ac4b5ff6864409396020a569999b374", + "4b7f80cc29961955ea40c7d3567b35a02164cd1533d958d883be82c4ba37b31d", + "4b8a1def4d36441651de8f179889ad3bd052bb0becb9605941116db4018c95f8", + "4bbf36b7c2c14caba6070bbae9458af2c813b599c8f6f3c6e2cf9705b21a99fd", + "4bc646e0147b8c2f1d49768a352f413b830a2332df659f02e944e942d47a4786", + "4bd9d910a8041060f92d96bf5eb568ce0140fe65e0015b82973300cf0e5b0688", + "4be8d922a5de73ec325c9248fc1416187e5c829799430a4dd333e4a7553150ea", + "4c1afc535fe4498f8e6ac334816ed0b9bbbabf835f590d477284915804ec6b02", + "4c54b6a006e39b19e32a114c684e4687e5bdb997e3c1d32299df1a6d821e51e4", + "4c6311c9c35a3420cdf7020ede4be8fd5f7b42235f8f7691677857ecd3987a48", + "4cfd7bdd44bea3204d1c2a9061be31965670616e9683866ae44294f5376a6f40", + "4d04a412a956ca16c52fc241daa05d7607215f93580007a531b0d254fd9f52d7", + "4d052ee0eb223728de2c5566ab2bc2532d03ad930b5f2cc92f0a40c1e75dc478", + "4d6bfda867e611caabef7d76612f0b5aaa6418d8225f4e6d42fa953a950d17e4", + "4de79945a00baf1aa259d5875bfcc7727bf66a46dc8fa9bc2231d72b821d48bd", + "4de923ffbb46a3f8e6ac63c98a68521fd21c1ab0f77649c95a30bacabead4e5f", + "4dee6a3056ebdf8fadc76c651ada138068fced8ecb685dead613ff9f1ce21912", + "4e1e2a65b95720debed07bccc2ab2fb4530f46961a49a58b3d06121dba5ac69d", + "4e39cb23a75019301edf20fc52aafa4cd6af401443571086d35b2bd0578b742f", + "4e5a8e5c4554d8fa8a888d419e03dedf35272f60c09c06494bdd6025f1aaf47a", + "4e5e55b9e0c37032de19375ef4f580467a7d16c24c117a936a64b1acedfbe1f9", + "4e96aab0b69cd5652b2226c538170f17d2c99dd6fbe6558086d131f1ab426fba", + "4ec8791a63530bde19bd54031d055d324e66610201287ff0a15c2b1a2fda1534", + "4ed68cecaf66306974ee0a1c02ac6d5522fd388a6ebf63f1260ed6bd778e6ccb", + "4ee615c9b08f46dde44466f10ad5597cb8da99157dcf86430e3409d29ee2e6e7", + "4f2bfd951868f8f07f40b14079b16caf8ad4ff9cb426f1f444e36016d29c6d3d", + "4f44da2e6a0ee3611a2f59dcbd74e92f9003581abbe0ffdf0eaa4441c485f5cc", + "4f5acdad32a85f1f0913249599e5c961a1bceeca6258089074bd6c79525915ff", + "4f7ba8c30d185c5a7dfa3e424b3c9e8f2a2bffb63293f27438fe741526458cb2", + "4fbfc62a8da5a39510020c9ce45669045efe132b191c2e611b5db80f6ff1911e", + "503aa9adefdf6b5f8bc9fabe338b915ccb08a00434799b4b2a9887bd2475433a", + "507197a8f0ca6fbc5ea59f39fb0f1f93f463b87ccef674d37f216214cf048ec6", + "508d052b57de3a3b4e3c114fd09f3c0e1390e61331e018ee6636a39d34c9ea42", + "508ee9e55ec34d55b104c6fc261d9eb746e4e70512ad86cc0b4309420b47ab66", + "508f1063fa711383380384576cf4b882743780c40f5b068d19ca20f21a78498f", + "50dd754d35675a097c9ac8e548f50f62d860771bebba8ca0c32e8db8da81bc38", + "5106b9ea61a0c1ad92726963b125faff828e1c21cd1872565f50b1cd3aec3886", + "51164d3f4bf9e63d7d12f2557f54de4f38cfd15f84d3c9ac6aba4e7b4f7cf00e", + "511f1c32d909dba225053747092b92f90309b832ca3b77266befcaaa1127124b", + "5125d348c3b07f6c3178ec0d4f202d1651e134a8c5c54b12c2a51f703328f066", + "513fcc34df2dbdeeff9e169cbb3da37e3f749e7fc1015c015a9cd546b0044572", + "5145f5c1bbdd99afaa2fd7f44f5e9e1f65aa81e02190f562903a48943cbf81d6", + "5177307c8c4ff23aecd65de6866fb2638d3d7ec38857cfa69a6b1735d3ae4ceb", + "519a433335a476427b4e44e79c849f394218055b0b3d55bb5d827d57404235f0", + "51ebce2f08cf13324964a1a686b9d497861cc807cf4f352c649a44836c3e94a5", + "520c9cec4ebc993350ca357fcfb6a8c3f6b94ad865e20f46ddcb517c414bf9dd", + "5220fecc9698d8c201b67957faed62daa1a60bacef7cf5654cddf7e2b7ccc75c", + "52259a41e170c1c126230665c1e0e2b9fcd62ebc41309bf079f2d8372c1e6d95", + "522e80a35e589f1d0ad9f4f7b6b6cd0b718e84a27b5001690f1e62f9b932360b", + "5238327c3ec5fa63a5a4cb5dbadda9d28c0486b41c53f91661a2130bd715baf8", + "5244e6ef0b5ed9f3207115544961ddc1ebc1911fd6964bacb01d8bf76c00d659", + "524c7192963314b4237bce27dd2894d568e7fc3372c9b7d76e772e53bed3f6b2", + "5255f663fa401436723b5d5f7894ed1097b9ff37dfaf3d2b746e8f5ba8467982", + "52764263ad81241602654110f7beef81da96da4ec7b8c60799edbd5857587e37", + "52a60872f2deacda62eae94debfa7f6940e2d58712570cab942848334ec35422", + "52f3bca068bbdcd19eaff06f35b06f7e535d88eec057180ef79c1d408b3b6630", + "52fa559775b9d6bc1a7a600084774c5960ed4a3bb7fc494ebf021e5cd1cbfdfb", + "52fad5a8f3e0c1a5efe527864537eb8c3ee9ba1d18e21bd0553925d1a4dac3cb", + "52ff1f7716b9f49065fbbdcc8ef2ce9ddb14b91daa85fdb2729237d0e59f6cc5", + "5305de227271bd209a44b2db1b715e7d261f4c402556d75a55e0f1b527f60567", + "530b04bc8910bae2df31a59dc8ac9b7ef4f4dc854b82f552a23044fb168e24c0", + "5330727dddfd3cfbd27b98ab33714e4c1acd7aefc2386acb5fd788a7657ce576", + "533ba29ec6475d2a3d97751dcf7cd534b0ac20aa01c93b7bb6fbf637e84f4acc", + "535739cdc04feef32fb71d37d31105fda85e8465082bf605b5441435aada8542", + "5387d01cc44757a81ae1ccf1111f3a77fe8c210cdf6ed1a991192a80f4cdb0d0", + "53b018be0e50f8ad61778e2330ed74ec2ffc8bec9b5d06a8cba967046f7fe86b", + "53f26ef79ca484c2bff9c31ed2c0b2d6ac2fc5a437d9b29fb1c5ab9dca002e8e", + "542faf6ec88e75b833dc99ce6d5fd4d30eb3373afc85f9f535fcabc38d0ccf1c", + "5468da1bfe41c940866fee9927d6945071614195f680f2b64b83c55bba3192f9", + "5494e181e6577586ec41dd3e5abc372a33e765cb03daf56d3b20db748dc79b80", + "549cc016d0d8b032362d4d1649408bfffa31446c2590b2e12804ebbdc47fb2b6", + "54bf2c3e0ed21b66938eb7623c57deac2f09ee563dea39c3e14d86a9217d2654", + "54d8b11fa7465c7082792100e111f8e33492e254f3892a41af6f78fc123d2454", + "54e5ea5fe2853c502e96c870d0111a15365a5f6bce1f3193a7b9c348645481c0", + "54e9dd90dc1debc55834f9788aaded500adec9d7619cc66817eb3b8eda688b94", + "55043e55212e6205cba548efc9313fdbb51cfa8cca57640fbff9dc8aab528853", + "556445fd46eda083b9309061f7e2fa74f036037f632b9b970eeb6722d23303ff", + "5573a9b44b03589da0cabf94c4a8e8937abc3d157356d864293a721fffd6b64c", + "55bf14f4a1a1eb840ef9a1784a65ba2d04e37319e45a2acf8349860cbef6a232", + "55d1286538ca7f5a4fd5356054a3b8f1dc9526876d81e36a0bdf741c802f1650", + "55e77f7587c4d4a56c639ef0c76bf425495c87ba7fe9cc3ca0c1cf436418d585", + "55fa2fdacb594e28831ed04b9b0fa4a047abd8e550f7d5eb724679036e528d53", + "5613fb98363341a6104fac31488fe8fcf907baf29f54278bbd9df7342c0f8685", + "566626d5c211e1d702e139dc08be33384e6d823d5cec75a6cb7b3310c6998a0b", + "56c1062c0c0ed6a0ba04664b78a6355cd2fcbcdac3aa93bc06819def28aa37f9", + "56e70514de14b66605e2b2e23053ed771e60f1e0de950cc1fd0c510a9a5301c2", + "5706b028c6b3503ed4ec4b5f490a5ccd746f48f3a8520d31d1d83cca2b8594c4", + "5709be9646cf910153f3f75a496597712cb7411e2f9447c7fef7fd492b32286e", + "571925f0b66c738b52105644b24558c21771db1f4432842dcb87a1ab468a74a9", + "571d8a20fccfe7b2de32adddf80c3e3f437341d84ae5fea77e5c55d8d9201dfc", + "5725a7914b447c22de18d20518bfd925addee19a17d889f3625836ebcadc972e", + "5740788c353fc6c81713fd4349e0db8b199ae0e1e8e4d5adc84dac9087e53e47", + "5743c10d43e919f0b522f535e9682357ae759de4f2044f5913aa11483dff0881", + "5743f3b3ae3820ce68338256058963f97e55e5242640d1dc30f974297d9391a3", + "57441d4c49333dd837ecafa0038bebcee2d990af8ecb55d4ff3b34ee9e0fe376", + "5792777bcf215d5f29e1271849674a435ed2a491f59bc498a24f761716bc833c", + "57bf1e0123dac8c3ff51ddd5c0e66620bcc27ae225e8c35927529b2fde62383f", + "581c0ff964ecde7b34cfd519ffec1d7846dacb2d96dcaab573231b5e824f341f", + "582bf2770b2a7a438b46966f406f82fa7f23dd8ad0f4c516837964af31808117", + "583ff7d92c90bc9811ad96d829f25bd68db1d0f5baefc02207c3c55179ffbd1e", + "58587d94c44401d22c7570b425d78588aaa439b5ba8a02532cc32fafd64b790b", + "5861d48f3a22385c21966caf1cbff9e534b2bc6d9b24b04a1c2bc92a175101cc", + "586df6b16d0371f939f01dd0fca066135a548ec526170a270879a775c4d6e8a7", + "58953bd77119c5a75dc8db52eae11aaf841ce9f6acf9c724aac634e0916c98d9", + "58af007dbfbb09cda0c25a22146bde68c6101750a61cc2bb16773de4fdd0288e", + "58c399854348d4943780f0593aa64371481e164045bc1e02af6cd5435b68a781", + "58dbdaab4d64ff0407fed71b3944f8a45d07694f2a39bb72cad56d94606c61e6", + "58e1c39129cfeeb249160e0272d7a56800521b08204d103bdce6b8867569c0ae", + "591de6a6a561511570150c61e0dc026f7babbe07e022131910a92f7523704d93", + "593124d7a97e611c01bcd92a663ff46fe9cf2d26cb22366fe35be7232ea1908c", + "5951167c8577ebc5e40b7323822b0b24a5da3268fbed0121456adcd5546090b7", + "59958e12c7427964002f36c93678bd6093ddc70364b2a735f592ac7b53d303e4", + "59a558b41fee06ccd7a1c840b603c99b1de92ec6ea6deed3152c01066a533212", + "59c8aee345f6da52cfba936abbdd140b1f318a80b616fa7778d12bab59819e3a", + "59f012671896e2a38807e6186a07c1111855296e31327c5fdd2a1dba5b785f3f", + "59f0eebad0170449246bb9826bc605a71fb122133ad7d4e7d99c69603f8629e2", + "5a07dac7bd8ad5b87afa1bc38b0276e6480da48f5e6d17b7825c8098bf88c1ec", + "5a0ba0d9ac4d2cd1f66309b325bff32bbe54816c4dcfc42064244df39d65a2a1", + "5a28a5206f5de7aa53c740e1b5b5b6cbbed409faebf4eb9f803d7c63a5198e79", + "5a3bfba80d88b4a3978f734acd65bebd13219893a477685dd8a71d513d02f424", + "5a7cf8dda296956ca9d978266b4cba91b8c4923c9b21ea98e740a35cb1901b50", + "5a82c9e9b43a93b1d64933552ec8836d741ca1406b29f8ff0c121ab3447c795a", + "5aa9412ceba584bc77c6bbabf4fd18ed16253403928e49524ed59535670e5e1c", + "5aaf3e59356111f7d87a5be93f33806c428219dfa4ab46dc5790f8503e31c581", + "5abfec56f23ffc8b44eee5e710ce21cef1efa1b0987dec1ecf800891da7b1001", + "5ac17fd232e43b44cbc0abeb3705d2538a647767056a2973c6dc82f24ed377fc", + "5af847ef35663184ba6a26712cb6a7344a96f2fcf273fa3a292b4bb46701bd02", + "5b2ff51ee5fc993256b0d9a247a5fef9874fab06a2fb561da7741d48c98c4700", + "5b4f43cff4bb797638bfd67b131632468a2c82a8f46e8028c4b2f70e5c374e5f", + "5b5cee99d8eef512becf6b4c6f2adc28a7674b2a5b2b034f6299ef87e135cc1a", + "5b81733f6c2d4875de6d68334be907addfee1ee916eb178a17b4a3532e57ce79", + "5bb74799899c9c13b3de7b14145ee9a36e517b3255b98fc68ed551e3f168eaca", + "5bf76a6863797b85f12c1e9657dbb98cbfeda76a85e5649366bb51c2e25ae0ea", + "5bfd333e5ee50ebc3ef7343214119e6485fcfe62177243c9ed44ca94ab862a42", + "5c0660c0a889c03272b8cb05ddefc3bb0f3208eceb119af380c167b29c1152a9", + "5c12ed807c668a70cf1e43c34b6976fa59d4a75b94cf5f46e6a66553dc551001", + "5c3800b2100255c46b68a1c6b3949bf252472c389be1176689bbe9072892db81", + "5c7875a957f4a01842286acce36973d6f1b43f0b7be8cf309ce3d3fc1b8a576a", + "5c81f028f79f4c7834047b6f0e6a9099cddec1698b588f49fb095cf79ce66096", + "5c89cbf0a0308716fa318e4b90d30aa624a3420ecbcdbab00885ded6dec88f6e", + "5c95d9702aa1067cb6ed8b2ca35dced65dd135880926fe90f6723d32342ddbfc", + "5cb3005a1e6be72d09b1fae86de8b44c7aef9f7b3fbcaf7be1314471d533b087", + "5cc8076479ea7c2283aa6ca570696a1ff9830912ffa176e470a8360a0b94b1e8", + "5cdfefce4177c0db9df5fe05c4d9838a299082541dc99c4a2b2ca62eb09973d0", + "5d197d6f8601540b20d9c8650ac6c7db604ea308f7ebaa6455f4f4f22bb9095a", + "5d5711143100eee894c191c7308a6310af56b08efd5d96764e4633c39d472f9c", + "5d77ac5abf8d323e7a23e3e0fb820cbd184050de53e1f6592102f3ce1888782f", + "5d8d05965144cc8ed052b1ebea298aeaa8acc790f297bf6b360d8799ffefe1a5", + "5da330a2d7539cb8079584a52f6d704f1dbed9de147c04d8001ff5e11b0536ea", + "5daa16db0f4094f7ea1436e6b4bc0ff8eb07efc22c3b63675509a52a235c51a9", + "5db712157f9dc54dead72fde0ed3dea5df4cdc2f9572b76c471c5c22b9232bef", + "5dd622ce281dfa0b24d79e8399b00c68bc086ea44ce4326920fb148b0e6f4d78", + "5df83d897e2924cb350b4a3305db4d44e1d066c990aed69b29e43e0f6698f533", + "5e1afb6bc141daf303dbab37f66ee10ebbe8b31ab3c5656946238e2eafd6340d", + "5e24f0ff25337bcf32fac07d4030e897af5dabefcfbeb5f1cf2e5403ba82b9c2", + "5e3432bde574450d89842f92daf06262e8cc64d281cc107ce3f103dd32cc0eb9", + "5e4b5c42a003ceaea591f2895a3e0f3102897dd8b1ecfe91a8bd5b5f562e96f2", + "5e589de637f00be39babfdfb626967ed63a1cd173634ed66cd240dfd436de90f", + "5e685214a2659cfab9e0ef45c26ecf4f5698b852eac4238043d62e60fa539519", + "5e6eb52c3e99332abab43be7928ca3633f011753547397f5a314b3e166035a48", + "5ea7348a56ef47dcb20a2c685063b693a67edab31a66863d7db871acf8dc477a", + "5eb9a73a975f07b91f8b3e72ba5c8456bc036afa6e319978295073db5afc4a6f", + "5eb9c1eaefa61748f03a84a8985aa3574ff723f4ded3b96cc6fd312de9f478a7", + "5ef8af7db8c8467bc40fa36bd555fafe62a1ab4e9946b732bd14b8b7785fda5d", + "5f250cb3bcadecde0bd95597f5d4fba1b26740d4d34d012595438f01455e9e59", + "5f2f285781ede7bd7228d4bc4cd74e2e679af4ab6f79188c80ea37525786638c", + "5f3490442150aef643b464dc288c4a064717afa7e6fe82a0db06d12bb4376fc5", + "5f3ade9c6ab3e134e04cd4a11338b71299eb710e0b2aebeb923e7dbe832020e3", + "5fe9694c951479112476e396fb24b6701c41954c967143688ecf85829914c8c9", + "5ffe2d07115394912d1f6396ab7085e45e9aa33eb85d986ecc24d1b720cdaa1e", + "600952a30695601677ef7a4b490fbcca600890c72bb1fcba30e6608f52a3b048", + "601938fde46de7f2ea79924da1579b5c5776505e76b41d86957a714552446738", + "6050db010826953bf2c017835d673e3201d38873160548b2efd17911823f3310", + "607db073c05e934b6c553b86c9b277d3ff247c7d7a6b8cfb70158246d56eb0be", + "607e3ebbfd8fa181ff70f397c7b22d0ad49597cb5cc85ead3b09821969d1e139", + "609f5fcd87f8b7224c173b7705321238e7406b3a54298777d7fec85ae6eb2ada", + "60e15063301376c03a8acf67435bef4242d6ae4b1836722c25a338b8c1bf4024", + "618d56cecf230518ff456c9888a33aaa90bf6b63edbca25f220851a7d2047900", + "6192d9e645ce38581a2ceafa43b9f5a92a38ea33439bd627797e16f54d0d2792", + "61a51010912747df01da5cd08f06e107a55b3e04a0ca7f99169432a3c2ff7872", + "61cd4801f08d19367b08eefc34e54ca6118b7c9e45e4e40bfd123134f8dc6425", + "61ce3e015ff0053f90457530820a7a8d46fe13e094e36b34c455a9c5ba1259b3", + "61f7a10997a6de054614fb67050e4ba86dafac0e7e82cac440206469de7de82b", + "62188821e56e67a9095da01261d799a8f4303257116295a6d51560bf7f96d272", + "621d8c4f8a40c4cc92b77b97df8bcc633317af6589211de5485863c573897f32", + "625750077e5ef43a7436d8a4e067f2bc09fc9c037e0a37b9c6116697c9af3c53", + "62702c6a819aeb5926918fc55115d3ad2a7667b0465dde4335e9f15cc5e91e33", + "62767847449284fce1f687e9c8015a80ae44b4e5b3d741177f2d5323e438f32b", + "62b1c59113149af010c492e62e5ccb43ec4adc764b9536cd0852fe9bcf2c897f", + "62dc61fd40b554612b1ba15c8cfc78a3a034eab498edbb67758f37664179fc37", + "62df329d815d6957eb334e427fe52d6ce5d18863f4e2e812c982004f24bd1d3b", + "631fc475f1c091e56e359a9aa4471130ab9a5895582c35da8b378c747967ba26", + "633a4fd35f046f06b4a4dc0e5a90b1e477763a2124535e3e425c78e25527bd4a", + "633bf032214ab02575b9908ade4b36342e8801f3eb6dda731f807b6aa89547c8", + "6341a2b834df92c235865fc9924d178335a4744ac558f4a2f96bf9b25441bd0d", + "63606c5d2195afd67a85725058a3dabb0855cc4b56ed7d66048433b523200118", + "6376c53f85f5ef69d5b59e136d2ad58d7b045fe50d0696662cc988f8bf145673", + "637b7b47f8f66722655cd95433ff31ae247dc7285c3348a7abf5dda931b84f1d", + "638daf7eb92d77f62d09ad78bb2c3f915e9809db388c9262deffd339d42f8b10", + "63a27ddfb2656a6745bb08f49faec53d3e559269161b2ee647134804ec18305a", + "63a986aaa6d340794b5a30bfeef97f8fb7b8fb91e641a447f129bf1dcd635f55", + "63e0e5c2a713106bccfd18bd22f26b4403229956df862889834518e440e6416d", + "63ef9a580cb94305c24269d0542a6ec21cd0f1e2c91e57f55ffbac99bc7163a6", + "6423ea5b8363fe772ca1e62ed51591ef0e2c7cad699b917253275cf8a71321db", + "64365e6dd5353b5442d4700932937991b7775997dd01d8fe6453abc7cd9a3e0b", + "643db93d793e3e730baa3209418e81e3f268dbb1e7a481ddcf59300a95b2d8d7", + "647a28aad4b02ff76bf2259ce3e1c17ddc7b444c3ac52705e681c64cb175291d", + "647f41c5b6dfa99ea202bde2c72d94b72b96e1e16cb297d7e00557a37c227154", + "64a7a3087e7a42aa66ea37486727c3874a65ac8611e49acf44d804965ea352b4", + "64d351d40a34f7884df2f840b7fa21727f8fed6df9e05e5b1fa6bdddbec49ebf", + "64d5c7ace5d90a493707fde2ba69c96975472d619423de17d258a14a73d6c04c", + "64d807ab260eb9d668450a8f1935359de7f4df518fbd924d220a81c6329301f6", + "64f76ea78bd84b0f135d3ede3c9285c77bb30e844a15dcc5eaf6301ba843cff7", + "6513fa0432f4384c96452465cc6c3482864a8e8fd9e7805b5a6a162e049ae062", + "653a93f89a8da3751e456c49059224b46698b894b64350b7e3c63f44658e0d8f", + "65647f03bd50562243692b63cc1fdfc4a754a814a21bf410cf4e0c48540d95c0", + "659c2ec601cdf969de0359d6b82058e778b563e99581d841af4c429ddaa1edb4", + "65a83ee09df22d92c5b76d1ed8e2e1f8d63cc02c4abe190763340d3d8313eeab", + "65ade03bf0d5ee974f9c8bb9d4c4a00a66b48e1bbfc7386d34c7a4170d16c49e", + "65cdab08c1e4a155c6f8e5617b93d8648b3b6a5912d4b790d02857d3507be9d6", + "65e5649aea03c20cd090502514db112dc347491b82ab8eca8b6f394a5c1b51fc", + "65ef0031ba9a2d6b0ec1fe5f29e47b3450c348c8d790e5e4d4900697c304f440", + "65f80b4528d187f9bcb15ee1794c0f43689c5d0b08bc2a75eaf2a31c52bd341c", + "65f9260cf8b1be768366e28a641a648f4769425ce0b37774259be274d7a64038", + "6605f42fbdae3b25ab1530fdb2e5d131f1b1ad59ed024da45c2da15fc8fce44e", + "6620b6219ad7c9bb82b4fc14b69fdbe3e0bb3f0a843350dea575f2a68352f8bb", + "662575859843a090be178ef11c3ff949b26975089fe6a0dc0efe041da3363c95", + "66477eb8101c0eae9ca8d6e00fc5af48b1ebe634380b7a368bc9887036d9edc9", + "6656cf871958da5c32aeceafe2572dd475124aaaf701cdc309f577d418044b5e", + "667f496aaae9c55dc80a3241ed1680bdd57dfda585610cc577356107c47a67e9", + "6681e3c1384378849a7a9295a3c4a4753b4b65445095201402636da3729f2322", + "66f368c851503c59694f9ee32a9f32927b8fe9fa1e80dbbb701f87bcc048e57b", + "66fcec4db0d060ad0ae50ac16bcf92257fb5cebf0a8cacaf0097b4a3cf590463", + "67252f580d6b891d16bfaf4180aed1b01ebb7ab5d6ffe0ff6dbcc3f315c1cad1", + "675c6a52ee8c1097c94aabe8f89da07c84bfb1729b4fc83d082035b4098cbf39", + "67aa1cb881d30869a3568440aebfd1ff0c8e52743ffdf4deb1f648dfa9ec27da", + "67aedb4ea431c92855d75899e571ffc8a0566485bce9ba162205149c951b08fb", + "680b5d1f7823434a26a54fa967b9243361a3bc8505f6d3014aad56e8255fee8e", + "683d4185c296a570ac27cc4108e8c1b7a14bd682b0b04879cddc4786448ff178", + "68516ac9f1a2c06694609dbe17e755798c4972d137779cbbed90bda9ab0dbe94", + "6873b65c87de9d43a3be6c7f6c3dad94b85dd07d3063f127fb8ff9736ed74855", + "687dd21c9c339af790cc61e3520fcb729bdd986e40acb3cbac5b21882b0a7d7a", + "689c596a1a402a711028033b1e00b31e7a30d93605654124841b30283edbfde6", + "68d2c928a05ee93db92e1161dd420921dfc98e3bd055cea5383ed8c450cb9a10", + "68f0dda42d4f45fe70c134d627e1ffa69479dd40159fd5d8164b14649e82b2c4", + "68f2c4b0e3f60bf7f398c9b6aa28bacf6d2f5e7234b3f0857e3d06920c214c04", + "691a1ea063bea59e2ea787fd37e06c333998f93705c02f8c88badb80f0e08abf", + "69296a5c433d9a8b385e82a2c65bee4a2de9a80b4f77441559a9ea83f3f6f1a8", + "6974bf3ae3d564d5bcc545a797bc0b2e11f69be43b840119bcf58374566c858b", + "6974f12d69e3af683d1ab7a7d20a865486783d23b4e1c282b154cb6faaebac71", + "698c4b79a847e21ce2dfceb683173bb2517d401de64c50844cd4ff796bbd98cc", + "698d53ba96b04fa3bbc0ff889ce2ec3398df89fe1d09022d4c1870a6360c9074", + "699c232eb369996e6e1955d8755aa4df666ea7adbd9171521c1106deaf28a2b9", + "69d39be858ab5a0d1731a1a0de016d5ef8d400c011d243b6a5182f1adf354ee8", + "69e7227a17f559f45e0e53f484042ed1b12edea43a4f1c9115d0134f7dda7465", + "6a3a2abc2148efeb99cf92d43659fe4437ad3f381fc5cec5cf18bc297c6fec96", + "6a690eb87882ab82e8202172c9cd7f6e3c9fb672069eb0ab8dac0e7dc7d05de0", + "6a75bcba06bd24c12169079e32622867190ffaf07100424ed00e1858c69822db", + "6a91e9529558fbf167a7cf0893a07bd8e16faed710a86e06af068fa15bb67873", + "6aa592fba84736f361ddcf66707fe15babd6f50b5f1fd49c21c272ecfd7f205d", + "6b4719b2a5a8835e0740c73c2bbf6b94213866794b174ecfece0f737f1cd001d", + "6b51b99bbc348bee5c4e4604da974fc566d70867a4d0e011d55a252e06cb0db2", + "6b7822407423f5266f91eba2227e33c75eb6cb85dc67efc76b07fc06ddfad88d", + "6b9663fe2428b5109a5721f7e8866e9e69d8a62b1f270019ecc36a347603e821", + "6b9c5aefef9b58f1e8e192fff34cf257b08e466aeaf65a2853c9a0d88a19502f", + "6bcf9d9355d514c522b60268466703f3338beddfec4380bd24d7c022b8ec87c7", + "6bd01a73d20d03563f91657146f0a6e11c082e8da4d74ab6f76bd146ad9610e2", + "6be5b5b85ec4edb66ee377e4682a06b3882e7b7c6ea4b8b7f7d3fc74337e2dbe", + "6bfccfa455cc73d12acfec2372639c0a80e7e809e43f3e9aa536677a4497fb60", + "6c06419d83f639d799195f586480be6d1f65ad715cb88b9f2700086f6b70e594", + "6c2f8a4d7b49e4bfd26d82b1779e7fd4b6ce7f78de2e47b7005a63132718bc98", + "6c66f8b2ea659391d439527556839fdf14b43d7104cfaceaa2c50e54b7bbf966", + "6c7909f2a90be8e7963c586b202e174f445d8d3d7dec3b6e864e136502484008", + "6c7b914983c75cd033d8e9266cc5df962bf7329113b58dbd8b29830e93ad4e7c", + "6c8284565dfd154013854f88ef41a73491447a92b69ee0313f8266d2007737f9", + "6c86c884635b1713342b8a2104ab5ffcf17c5f5e388a2f1396f36280c8135b66", + "6caeaa904146cf8358cb8ccf09c6a44c64e1942acd24fba6432ebfe674209788", + "6ce3537f885d05ca6480438c7ce3e87d1f7d2521eb0f31f5c08e47cf301be28a", + "6d36f8a775d3eaa6a2c0b0ad390e01f35bb6b8ea75409807ec9b9a3c93be592c", + "6d4fe2a0608ef4ef22a00655ca12540eb6636334c5202261a66dbb6d10a58ea4", + "6d896096e52d1cccf599dc7fa94fcde94fff526220ca8c4ffc626c42495df1a2", + "6d96d09c7e5a6a9112a35e96e7b57c65de7883a1f51f9a5db07146ce857e07cd", + "6db2b190ba0fd5fc752e7285ae272f7a391803e52660e928273987805c9ea2ba", + "6dd7c52cad4d13afdcda3ca78630bf63ac86ee745524929fa91c1cc9ccb544cd", + "6df1495b93ab4d7cb31c85976c88767a2906f21144a7a8b856ed57c5d584700d", + "6dfeb5a052c7866d1798360e88d5714fab6693772b66e0417d45c74b5763c33d", + "6e21c0b9482031bc9f387ddd33a53ab464c4a00c1eae99f69ace97054c81abae", + "6e428659f64733b176f25f790389d3f4347ce2862ab67d3ae1fd4ca9a42cd330", + "6e586147e1f0d42aeeee54bdb5236d097e305ab7aae76514ee6f4acd0bceed63", + "6e6347ede261653f4adc6b01200384e013ba4103de13e31f7cb82cb17eb3e109", + "6e63daf41296594baabea559f145029b9fc5364946a47ad7a4899ae6f6c8f9f7", + "6e90c6bd75b3a980e10388fdefdce25fc783e51729cf6f00c8f652b0f4355365", + "6ea1ebde2882b97883e62dfd80d3918f6c8de7406f0de4ce9619c25b2438755a", + "6eaf4819e64e8f094c8f1964c77bb2aeb952e2691b520f2ec2fa7e11f52fdcb3", + "6ef70efbf5db1a44d48e3738de90d485aea621d09b3efc0a541198a5d5b20bc0", + "6ef9b67e2d1d4d36228c7b5218e96042ae7ad65312c2022c52b4b14c67686487", + "6f25c4816728e39049eb9e3a648e2ad713e25e33507c05f83c84126d5507052f", + "6f3ab3548e1e543697f0eb90ee19d20ee9fb9bbd8b940606a4b0fd8ee6595c0f", + "6f3b4625f41a1f8b908a4f6c900231e22e702083803b05f1de408a1b58cc987d", + "6f3e158f32511ccc6690cd472aa9f02321cf7d353e167241a7c6ff92134c5994", + "6f4218c3d3934cb40b30b14df5a72de35d8dbd1146159083cdabe234471186bc", + "6f79af20b759c197b9bd3b1b63a504fabf62e15f570d70c5a8afc28aee5559aa", + "6f89168841ee0bd7f09a457fddb74638c7bde20dbcc26695b6a702ce2126e883", + "6fa265f656f2ee4d7a2b8c55b477ded670ecf48496fb9c4afd98bc0d32d39ebb", + "6fa315dac5b32cf7abf9acfe27c648859b2ccaa3b6e161adeff396ade3723978", + "6fbb766aa08fb955aa7577e9a7f2ef8d9fafa9dd0fc8d3521ebf5f460301cfda", + "6fc5414ef35c32d41c91fbd549f5bdbc0d8b0db216280c83544ab0b8738b6f00", + "6fcd3d40f63cbd59389972fb750ee27e1adcdef6a5274943948f40f56116fb14", + "6fe35d8f5fa325e35f4cde71cda545655b1c653f1772bfd8ce8fcdc280162ada", + "6ff0db043be5147c89ea98552ecdb3d43eb872a907cc4797c01bb226d980f5b2", + "701eca67391efee74e0293d67cfd39e615baa695474d81970834e9e62c7d3cf9", + "70333759946dbc2f084534f94f76d6604dc5dc3ebde87e79bf4bbdc12a7dfe9a", + "703768057979fb6f20c571ccc72439a6595a20f77c4b01d4b2d1bed855e1cd42", + "703b9ba161ba6e11b97f64d87fcdb54c9f53272699dbefe94990d83d2271590f", + "703f7f554bbbfe686479fb152fef9e4e0f8310a827e86b8515adf24b07a620ed", + "707e3480e3adf85aac5025238159eb94804c76fd671e28e23176d75048a511b3", + "709f5003d5a255e258b1cb7a6b18c421a2e1826b448c72ed8847eeac5497ef3e", + "70b3a8211a06b1d0caa06b96b49926e57109a461918cc02186fe43840ed9cc3e", + "71088e9aae4a8185d5d27950295b872a98ea093428f97864c6ae60a04f332a98", + "714c7d5079aa57ea41eaf89e238ecf1f20918147887e41bcb8d6b2f6dd05d8f7", + "71673963d8dad3c8041c1c46fef6c23da0c49ea1df404712d1d2cfec197b0574", + "717caea1d37df06e06b5cb5b9cbee7081e65491abdddc40c5f073f499eb97c89", + "71826c795a682894dabef3ff7ce3208029ce95212b8b8149010dd01522a124a3", + "71853c6e61e8a444c03052320348c810c0073bfb0d31ebddcfb9c4e9c007c02c", + "718c33f21779928837d659c7614a844276b6e76a5fcc9b62a092a25fa2c29184", + "7192b4a737df2076f40c10e6893c2ffa904b42694f1938960401a11cf8738d68", + "719712ae6554824342c77f11e57208d5fa78d40528df8a18e61d48b54430ed47", + "71be758ab30f48eba0225f88b5e4677fd61309234c44078db44801c879ba3074", + "71d9b63e6b32b3a79ceab2553a5d4b8d0d060c02034ce041a0d86081287d851e", + "721f9f634bf6ebd5c7ebed59495abec3cd11c9b72631477995ecba11e5dfd8c2", + "7233fc7cf525a3643d7abe89bfc1f9b2cd0a11f9936a54d7797967f646c07116", + "72462d383c86b76753eb6817608dd89ec609c406cbae7ed4e717b595909de462", + "725ff03054935ab9923ad90ecaca69238e39046d3fbd14000e0b698401754d8f", + "72695a3c8e402351d00ad77e266b6ba8a282aca36289e88e36c61709877919fc", + "72e78cfdeeffe0bd2597a166d4da21981b87977df5f8c29d250cb10e51cb6581", + "72f4d67ba20569c6b7596ec75c0e033971e148698de7f4bebcdae8fa8554db17", + "72fa79b3fa115e9812888bf78b82495c5f4cc6a234ce5adbc3eb386984c3184c", + "72fc5100b1654ec387007afe56d8bf5eca6072396db1f7d944936eea41695f21", + "73024882c66bcc50e1eb8fa5534a49ae5dfef16fa8869944b961cfab6364c0af", + "730b69a152e1b93d05dc8d4dd430cf48831c4edf53558e148161613762d5e6c4", + "73298093213dcc7424d426cb063933c34faad99e3ecc54f6a1f9da12e0757b16", + "73466bffa17e0f61131673c325dde5e475e2534dced253fa6c2c041060fe8d16", + "7359f0b367c18c3514982a300af59a3715a3cfbbf68ffe9619532be27eddc830", + "7364b82fd2ea76068008093d85b78ebb04a31672269cf9b25c4fd1d1a4c6190f", + "739ed04dab3d45687f8a69040dd7d3d9193876342abc769ea1f83d4cf177934a", + "73a4760cfd30333ca79502c85b198c3776db9749fa4c4b38edca375db671140f", + "73ab62d6f93a31d65bf4b47d0752b58e6bca8ee092e30a0b75ed176165770b60", + "73e19802a19244d6ba391c14e7e910267e01e9d9a1823acff209361dba4f418e", + "742111f5094433a5f24f30d74366d717136d0c7f310c03cff6e4c7e4d2f16972", + "743c7e0bcaeefc11b5879252821ec7ba7f7e617480ba2989bfda8db33d43b8a9", + "74684ee19ac68b9d5652c36f29d803ac7fb131c41bba960ba12cbfb38f420374", + "749d83f2bb09989fe35c1b6eabec5bc4c5730e2dc87ca1f2cf173ea5fea2034a", + "74a3878c4403073afe2c4a2779a4460b958fb4687381f2f40792a88a808cff1f", + "74b8ff4e4480e118d6a2f4918ed91f2bc0585219cdbbf6062ce8fb1996f647d1", + "74ba9ba206af46de2e61639e2df0374a8248861b38702c513176fff43a393c41", + "74d5ad78c8c335d3eb3e4b4be59efe582982aa2f607c15de98d25794595b3139", + "75049ffb0430bb1c3b413f523d00860d9ebe79608c5b96638715b44457d8580f", + "7521f48d4093fd7a9f6f645199fcf1745851467131ac37581781ef7d2b225df6", + "7540f3ca514bcf1ec786b4b7d11ba9f68eee79fdf891ed8c4f711c40e5ba8b25", + "7579cec0a34fdeefe775b1c85d11e936c39b6429e30189253c52d8a9230aff9f", + "75926f87d7473582ce80f378332a95c7c20b6a27ae07575eaafbd3203b23322a", + "75b7d995c7b34a8667be75ed3691324e218b44f94f772c4b2c3765401f01d106", + "75e79e1f3af52e503798047cccc630e2d91cf5525851c0884290c03dfde1aa80", + "760dbbc813ccd9ae23ce4af764de33059e888bf038d7a65e5d6816892f9cc103", + "764089293fede6ed51e409246c5cb35248fd9f2538c7ef10c37b36cda222fcad", + "767475c5471db706e5873690e0ce6b40160e0cfffae596f2e0609f971f41f849", + "769b6e89a2fbebc067461c2c54e8f79f242c729bd7206183ba670dc65b275f82", + "76b0bc810534bba4a859942e88a604edb3d394b8421d516298302ff23557409d", + "76e965b06bd3d10aa463f44812e8ea04eb8e67c95f382595a70be22c42dbdd4b", + "76fd68580002a202777d8bee3d257cc28cffeee5de1799bec497fac4427e0c04", + "77250960a3a38b0e46d4366dd3387196ab80ef8e5c4e224587156f973a55e094", + "773fc1d055664538e2caff1408b4786eceb0348027158c0ce749ee031220ac93", + "776acd210e8fe45c59dcc77d5c4c9249019230746124fd78e363d5842cce43d4", + "7794ba7ff588975da27457e5de81fd2021a7d5f8657c33672e36b38cd7bd13a0", + "779804956eb5a224028889be5dffaaea3d890ee8403e346b8a68051f8c122260", + "77cffae22072a98ce40bd10afed90a8a4ef52833566e3b14c4fe5c3603f0555d", + "78196725b339be6d51593df871fa995b35e14ed2dd70cac976afa3cb7330c05e", + "78435438eadc53e4b65b80e6f52e7d3b8f989b81bad7b49fb4b06e6fed2ce3db", + "7858336ed2c00e5618549fbbd572a335c911c09a4976bef5f8ac1f08a5933920", + "785bf60336e1b28f1892dcbfba28bfa6898b93802fe99093ce45e8fc308c7916", + "78622374cece0a3dd71545fd5579fa5ba742fb0e275d452adedbe3bb821a5edb", + "78698ce70fc4b006e2d8f6374ba1cfcf74c0a9c4b29b075c957d92f942cd73d0", + "786a54e1510ecf3e0c15b22d8b1cf54d50d94297cb6bae026b522534374c78af", + "78a73a8ad0f98bc4a4d8638ad881b815a3e32bf2a2949f180602e7875134aa1c", + "78a8a9c1a3b7908c79c7f78d726116cc70a98e29b738ac171e11b2fb25e1151c", + "78afb450b01a4b9fceeeb74d3b40965be61cba7e43cd68c7dbd7028e89268bb3", + "78ee6e15729bc5127e4e4fa362dc21a2482f39a6fd3cecbb9de2fff3ad884382", + "78f740cf7bf686664e07a6dfd745854f5b16eb9d211acefbbccf1f903b3c7fcc", + "79393d7ca4a928730d3ba7c7b33d3ac19a72629becdbfd5266baa2058dc936ec", + "794bb3b63d8687eb22490f6255abac5d7e1a4eb7f1852766700090be9c4ed871", + "7970e3c540590f3d367a29d9b5fb6c0fd23c809dfc652e359680d9e03bd1c86b", + "797f3cafd57e7c848b365be2a79eabc1cadf42857f5a2abc1eb2850b082a1f42", + "798f00477a084973c5c1f4a93cde9839dce6ca25634a4bb83e78e5e3ce9e703a", + "79a445fe8117b072375d8cb67d2ce686e79cca3d029ce8a703255513edd715c7", + "79b2565cd36a5aa7332f4cd279edc77d9daf30771d5d7b961967603af3d770df", + "79b9c50866c5fa2e695540ce7a267c0889d0f5f3f30abfb820d44a407009c1fe", + "79d537183284063b3c119f5bf642b045592f50620b0feb4e444beec840a003e9", + "79f36210ee715b0f9867bf90584758664f37847083647139d14935ead9805adf", + "7a1e81189da79ee94e93d47aebd5bcd6fd85b716169e9561f213678523a4a17c", + "7a51f2d1ce1a72560afb60675ff504b47fa01bb657a0dccf975a371525239d89", + "7a88199756be7bd36308fdb0db2319a89bad7be1a4336c7eb15f92eadede9be0", + "7a9442e42d1113539ae1654ed1ecbe54c67bf72e55e467c57688c0f4e4d31e43", + "7aa0646914320a193ed7cb20bbbe65d900bc659a2a581f5a44055c60197ed130", + "7aa4be1cc1fb76f5ee2d307f24110c84d565971c030a848da542be8edf937c07", + "7aa850a2ca05f925ad97095285f06535102b93bf507e85dbbd53f1ca3d000952", + "7acaf7a8d5f3879c46553fec4f100fb2a1dc9f8ed0c0d8a883489427ef2d28ea", + "7aefadb4ddffb69dd946a207c21f412da4554397a8b06865e239c602d2f41a55", + "7af5155fdd59640ac809c59916c3620598a785fae44a3e702aae488c119c5198", + "7b16ac75723b6f9894eb0599c963725531588337e41ad546eb8395327d5e0697", + "7b21c9201cb94c0510eadce9d4cd9a069c097f584a50c41e7968ce3fabf34d6e", + "7b85738369d19b76ee12a33138b6fd71f12b61c541322bd99b6e8c76f28d8f84", + "7b9501a5c4baca2b8e964d4fd1e9d6ac724d7a0a1c0160cfbc089920f900fcf1", + "7bb52dacdb2687faf5c22439851c2c834dd54c485d7543d0aed875ee4a1a394c", + "7bbfca7a917b961e83f03bf91996b05f5668bc93facc9cb94ed1fef301a115b6", + "7bc8cd8c3a7c5107c3c2bcbea68a070cf1ac231a74057404dee584fdb70c0a03", + "7bdb0e3427bab33b42fe14d28182c3273838199c7b439da9e1bf2fbbe30ce74a", + "7c01f13b6ffeaa36f3d6754644c6b5f5f62ba7d3adff55ea78ee05281a7151b1", + "7c13410bc3b2cafdecde4586c74704a26f911d86a07f147af9af91f24fac60fd", + "7c15efa53621d248accaee695bbc7adce989c445911c33070ed5377af5a738df", + "7c276fdf802f10307b08c02b3597e7ddca1ff05d81e58426241b978b1295bd4e", + "7ca8ff06f6624f3dd7194560b088908c42e11b69a2b015b252e84e391519fcdf", + "7ccc8fa8248956bd43d77506a4c23697e30fa61f2f5602063e885c278657066a", + "7cd938fabf4f322c80e458c82e4e29ba87807a0d77af5714dbf740c14d4096d2", + "7ceb188274648cdcac071879cdf812555a194a99b60c088d389b7db82149584e", + "7d1cb0401330cbba5c232aab8f71fb185f52d939d5a474c481db009167520432", + "7d26d34a80b40971e91ed54ebe1c94d874fa06f7ca0cc1710112faf2d02410eb", + "7d28016ba3f18a075f25cd72c23c8aa3248df56228fb1a52a723d9a24e726368", + "7d4978222a424efebe70733bbf7ebf56ab35a773b50ccb82a690b07d65576549", + "7d4da588cc4a42eb2dbf5920937d1af624734e7b94c9e9fd35e000de9b19958f", + "7dce8a0adb7f8202cfa2bb850fd97ff64521cadc0ca4a4d564ef4b6abe260b69", + "7e14c223156ea6e79def806060fd0c9888732cc9959313d42a56e6a384e1828d", + "7e23d0dd4cd6bd622866dec150a1d38b77187d2895e0dda5229305902ab23be7", + "7e7c44b1e6c90932b9d8b3496d700cea042dc79e088873bd16584bfb0065d21c", + "7e9b8e888284ef44482249468bcf1d3c05e248d8f83365adfa032d7155b7c39c", + "7eaf4730cb6b13e2ac7db67cf61b6a29f5715cbd294685f743fe5b1545a2e173", + "7ec7ac968403f21ebc8bac18aeae134a206b12c7524c4057e2b47a42fecef24c", + "7efe2a5615bc2ac81c2a8c56a439dafbb7e04bc721a6857129b0dbb7156a49a8", + "7f02c41794b097eede7d73b9d380ee74f7df101cb0fcb2c5ed18268fa95dcf87", + "7f4eeac34b3358cd586794fe9fb121217a86464deb4b1493bfd413c57ae77af2", + "7f6c61ff00f23fd6ae6b96d61a303f6e44363ff4d1679b13b9cef244e814cc84", + "7fd7e3e665311b4248ea1af06a835d00dae5a427922abe1f898fe66a958e5382", + "7fdaf4bdab2643b67df88ccbdd1cf0d5878361070c6374a12939e7cf93bc733b", + "80313a3c133d5918239dbaf1248d1b2abb392f39b294c81ea689af6ee0707fd2", + "8031bebb0d9126c9be5daa6beeca271b8f7879957fe35e0e47c17060eb9b7811", + "8074859bcf62c319d896a8f29d98216f8991ab9bdf49f879ab36d33dd155d9d2", + "80ad51c06c23fceb63beb5462345dcb9a2d1b67054015609ca1189c2308e8f8a", + "80ae9c3044117ffd27c3f375f0e29c9e75d2a3edba632de71adfb25ce66b0771", + "80e4cae673b060511ceade38c0a68359b9e878300c9d5d0259b6552deaa5a1d6", + "811b18091fbf1a6e5d1452fbbdbc2e9ecab73191974e0214e38bf70cbd0010fc", + "8124f366da84fb287245aca463572ef89c274a6360cc0e9bdee7d477ef041215", + "81518eb16b5d44c22df9334eb65fac0dcc85031b0c946172485819306c58aa8e", + "818dbba72cafd429131e635b3ef6f3dcd5f3d40c359d6f615235f11eb769bd27", + "820c4004f3ca9ed9556f3e68fdcdd8ab6e394dba0f50e532d682344629184ee0", + "827972c6d14855d08db1c34677e97fd79a18c0bd9fb2b9e9e45f6a0a05a2952c", + "82d226725b39f5e86eb5bbfdca67c52fa239ee867160c45a87c90194f9c3e509", + "82f4be5f4a60f1413f24d360f20b179fded451bb7f109f1f6b7a15a7b5393f80", + "83134fa9b0a26a50c2fcdf580f8ab7160a8404bdfb887cf727d7588c080b4ce7", + "835422aa07b758d79e0f7f35f6e5db7bf413b872aa8a71f2abaa82a96bc8c799", + "836ad77eb86cc6612922f0fbd4a848187aa5c64219811459d4bbebfe01ecc345", + "837b550ef7d135d2c0dcd5984d065228dd5442d3351d9cca94321162d8b14af1", + "838f6d15062b30326ad7199db9d071f7507bccf1fceb33f3e8a204d285382faa", + "83d185f09165dc5d972254bea55079367c28a0fb300ebbf3fe92bd74fe2d93c4", + "844a4ccb1843663785075b9e9f858bfd930d305b1a5c84753cf9c0a2a0295da7", + "845d12f4cfa04b863ab3301a97d26363b521787bc09ed660cb283520b35e9b9e", + "848898dbe0c01cc84e63e5f2095e35255b43bc886f0e27eaa81e8b8260a297ed", + "84a94da8a7ac516f98b806874e3ffaf05d4dbc506126b3dbfe613fdead407985", + "84bd9538257b7814607343164e7ed9b5e4342b8b4f7cd3545f72fc1ddd06896a", + "84cf3d3a21806b2e0a25f2137a8df5a2838f1b0b747657386e8e4018a9f7009b", + "84e808828a69df041c29bac0e30230b711a94e906ac9820f06655213f786b6e7", + "84f7e204b791c99dd098aadcde86e776a5ceabcfd3061aa4c654a7635de22f9c", + "856769850310c7f661ab3a987a49aac3f19b580db8f96ab79daafa718e247f79", + "8570530e03ba5bd517b643180a182c36fe9c233ee4f5cdcfda9adf6544d97e5b", + "8574f58478a831ff1c012fef160d321b0e4a0c4e0aa1bfd4f02ce1a1fe3c9b43", + "85a6f82c5d501f4f85312ee5c78761f3711c14fe917f5141fea49f97adb7aa27", + "85beba3967198e7e0def9d27d36611e722d7660db0c44da4dc75932d501c438f", + "8631f5eb9a74ac31db1a05ed767754d8ec31f089a80f38116a71014bda213882", + "863bb5e180baa89cd2770b88f31edf3988b20dbbac613a409311b9465e593c5c", + "864a2260f794a96c454c1e311409f8b7eb71278f8cef9f6f1b610de80c6f347b", + "865f0cf27ae5e60a2195c621c0ac35b3bcc9324ec41b8986fbc714ba480f9ea5", + "8699aad8ce4823f923740806ea40269c4ef238e140c811865958e98d665cb08f", + "86d85370fe7dd74a8023657798682dda13199fab06e13bd306d4a19dd8193a0d", + "871cb4c9c8f611b49d7743fd17eb92d14640bbdee058490611bca68dda174f3d", + "872c8acb3d12e47c416d8c271212920cc4654d7339111e4d89065ae2f6b15285", + "873f4d11489712ec090a4451aa926f41fd3800f396ab4613c7055f9710a41565", + "874f46d9052509cd07edba8888b99776024653a9dba3aa2ac48a63f232f7eff2", + "87afdca89f997e5f096f51341578d2ab8b8c5f1f091e2011d0df87b28570cbd2", + "87d36a47ee715c077744344ee6e94618940f64b34d8f232fa15248e96ec679e6", + "87dc2cc17041f5f8857c27415fac432301a74c3f7cc6e3ab1e6b10a0f124ad34", + "87fe37b344acf82bdc4a7d88fea87ed6570374d3511bdab793b90e0d2e1f2a0d", + "883ace1beaaea3781f22637789a1a7fa17bfb1ce7542e941d81bb468a7d4c6e5", + "8846f36ac79147eb10014d75f6fd6585d01f3490d72496a0bdd74fc5b1e7f181", + "884db55185577f00049d89caa0ffd4f57d3fec4fde57b2baec680c6e4ff4836e", + "885febe6854c1e5acb517005f8a6da8f35ed7099c45193832ca4fa5e3ce91848", + "8882fa13167d65377a7ba90a7a4dcedcfac1e80b54c84a5b946296074674ba31", + "88bc8e9c23670d5bd20a398979d863b137b267aa1fd7d8c84678d3f72a163be4", + "88cc71bd445254f00b73120853a0b61cfd04c7a6f912d657198109ef8deb7020", + "88da3f334bdf2598c22e1c4cfc7e4f27813fab04b71a1c94fbd394963104f5ac", + "88ef35c4215e6661b6b13b887413f6cfb5fe007fbe0eb7c31ab8f8667d49f939", + "88f24ed049e9cfdc5e6e52664281db32ee6d605f8880ff9795f339c8b8cb6ca7", + "88fc7fb8d44b1eb7909f8ee97234e419796e3b2262957bf06855040d53b30e20", + "8906b268b99b4ac3c7eade7be1ff2265be2c7fd00c92ab2c1ef2a3a4fd4d2cdc", + "8909455ec2ed4207fb2252286af435945ef7f4441b14dcdb2233b2c7e473c62c", + "892da162016430553dc0ebf569bcde69a9d968cfb7fd52167ed6e243a99af167", + "8981887e94552109b3cd72aff3dfa3c9d76eaaaabebf0254c890e8a58ddcab8c", + "89a3a2ac4e3d523b93cf14975ce15e5fd39483055aea8eb9754967463ef4bb22", + "89d52a436b5a0a0f5caf3ec0dae813d38b2ec6f90f9b6ad984155b2ef628451a", + "89fffa58748f397976e7995342db915a9ef79590640577796c7d99ee674e0eaf", + "8a12f9756e56fecb2d2654523cc2b63ea0219362fcd3ab082bb95812968ec31b", + "8a258b19bfa17e45608be2a3076e02f86b2bebced13facefc1f28a5c503e3958", + "8a38325262b8f4fab2f9586fcb48caa450546bbf8881b2512fc7ae56f0463327", + "8aa00ccd9ec106af3f8d6269121ff1dbd381673b740a26106da23ba717c5267c", + "8b09e0958549cdad4bbcb5072d88bb28b2d16887b3580879796753f2ca73dbfa", + "8b1b7db85dfb029764987b5d104bb3bd23bce31c4c15285536218025bcde4bc3", + "8b2e208352c7bf5089566c2285fb25c19a7e4c68d62a1a45ba3eb0642f5a44a5", + "8b3e1ea8341e7a38998e66b52af92276c2ab7d089fe710bbebe40c37fdf9c210", + "8b3e415c88f3f697e9264f31cec88cb364776b3b2174b6ec9d2c9ae8ab41bb2a", + "8b455bed9daf8f050877d9ad9a1d0008e1c5b5b47af5f2608ff6d63d7f2778f4", + "8b4dcf783f047800dfce66ac900496e13d11389aa43f11d0ba0aa17a53d35900", + "8b4e9df3726ccc0a874a7657bea1c03e50a1e63855a8fa4aec0ca050d16c5cf8", + "8b91b7882239b55dccd3f2a3b699b870aa6e3cf9820fb32f7f64fe0975716466", + "8bc15315ec593223c023b73d8e47bc4fa17ec93d3eb420598028076918e9e59a", + "8bd6765985b355a8426754aae723dea54f2e1ac544ff0a1a3442941e6febaac8", + "8be8a2bdaa02f79e7cdb4c7c3906826fa6bf90a20bcf9285cad4b3b105aeb2ba", + "8c02e41c60007b69416eb7689404b46186087ccfe0f5989f20784c2c196b7b1b", + "8c8d4af696736585ce185cc8af41d0b6109a2ad22d4bfa782ea7ef9a3834e55f", + "8c8e11c3bd2c532170df6ca8a165d44f1e707806299fe6ffcab13a6f2e122521", + "8cb3a08d1ccf19c8725c6a98ed7b3386e5799fab11ace8108c2883579f682714", + "8d177f26ef02fd353303673b91e46d8390241c1c279dbdb22c991b3c4e9f51e7", + "8d28413eeca9c997dac5d6ec5ab6ab12517e44b8c4a51ca3ba7c859491df3952", + "8d9ef2d42ea02122ee5863b202c05351599d62885d2b98f96bb737e4ee6ebc33", + "8dbb7ce9aa0f47e558051a724cd235cdeda295cef4ae6af454076079bdff514a", + "8dcfd3fb6ca84c1f95efc8f897513707d70ebce78c2518aa8f65e814e71f3545", + "8df113b565bd3cc6cec8965e630711bb168b1cc181f53bce4c6b7ce61ba56aac", + "8e22151cba0ac0a1e46d5ddce156fd63cca4eae19b77238cca06c8c8ce821eb8", + "8e2eb253feab70a0380720d136c671a52ebd609d5c1272ccdc6e3ad461435156", + "8e4f60fbc9c964d0ebbd10f8b9265be69a3a3edc2119ea12426ef43119d4ad5c", + "8e5ceea749d717624820f70c22b33842bca091f4ae50be442a7ec0f09c83303a", + "8e6d9e83a4e8ecd5052dfac7c65bafb828ac22f1788885c36a58ec7aca9d27d9", + "8e9c2d035ed8eff3f545b75e289b7e582c7968100ea73e6fa33b0be1d5bb50b8", + "8ead263dc6ed57e40bc9a47ef5e9f5c3e646b986f2ea12f7e6892ead5de2ab8c", + "8f170427867d4318616b12da898037de0f181a5431d3acded4af696cf1326370", + "8f4222b557e6df356279166322bbd1705e8a56c35891f90c1530cf4bb2eb54a0", + "8f44f3b78ef1575f9b970421c721350dc71b5db2af4dec9abaab2ca7578652e0", + "8ff84b817ee44a102d307bb8083ced2d3cdedf4cf0079e6c459c7b677de3a329", + "901a8a97dbb978e7d127091a6fba78c0400b75a12bbf3d417b28fb0af80b6782", + "905bcd129bbf0751e06533ac956befd3daaaa9ee2e6645cc6fa3ecf71fbb9a89", + "90695b391187c27e2685fb7e84f2652a507fb2a02449a190d3267bf7cb3f5b45", + "906b270dd3e6ce8f729128dd2cb3031beb593dc7429e4dd36b13f401cdf792b5", + "908adfeb33ccf16b1cbda5b092c58b268e734268ff6c60cfc466cc856adf2baa", + "90a9b33fee83fcf0446eb2bd3672676120fa0145c84b4a808785d5cf67fb307d", + "90b17242f7b1bf044dcb0c8b348a6b20c4f43458b8a4263562653701f136c4ef", + "90cde577fd11b1b1735407d95de549f9c0a53404b7f2d12fcd870cb927e9f059", + "90d2284908e605264442186b3da69f670bf446ef3616ef6e730a093e090e2b3a", + "90ef7fa7bf2f7675c592189ef0b9aee190f6b142d6df567bc67852a3fe46d9af", + "91436958d8b110e882c5c4a525733c39f6c910e196f4deb9f42603dd7c4603ad", + "91629fdb7a0c28df18abce3897b3b96b2554f0a3b724004d415f8cd7a3a47e92", + "917dd26021510d66f517c0d401ecb5d7db01a25a1195fc4ecb9a7a363814268b", + "919a549f4845967ac2f847626e2b00056eecf902c91d9a4be8378a41640fbf34", + "91bd8da2cffc4096f89a36c40aa3d00916cdcc73f8a681efa0222941be36543e", + "91ffe0cf3888e203f9d71c38b09f1fa3d34ba073342f26cc49bdd72e92ab0206", + "920c0664db9a83dcfaf66ae2e13aef9ade367d3221902ff79efbec849060b80a", + "922aa008e201c6d77a55d168be1f336b56911fc60a133a2ed900f3cf13d73431", + "92563ce948515f1da8aceedf199f6ed7dd85ecc6de1f26c98b99655deafe03cc", + "92697ea9bfbcfec599c6d210f730325aaf05166191250d1f162d162258961af0", + "926aa9c880645612620c26ddcd7a444ffabb7b7edf31b3bc2d22b50819698712", + "9291a90d8529ecbfa4905aa368ef4026cb7a430e3e6714b9b8f334dbe5cb96d2", + "929700cddcbccce723799f822366c39243dbc4b470d2164354f1168a9b7c97a3", + "92bf7e3f8a5589cf3497c2a9dc4b945e169b87e44a1c01240873eb41bb77ee8d", + "92c5af8113a4129e4e6b5e6a9e16a45471fdde0ec55ab2cf1f64aa1fba547d5d", + "92c8bc3fc53f871322fa08f65bd33d640fdc555892939d08208ae59cbc888a18", + "92e75868a799ff640fc178d95d09d447e721af90c051b2ce8d970f9037c04670", + "931e6ab771d72e7d47b63fc46fccaea8b91fcc80aa97a456bee36974cac43006", + "932f61eac4752e0bd55dae596f5fb82c6a5655beed2cc0f209ff157dbe89e180", + "9363a4d2b67faf2bb77955660010ced2a615c568d3761c57a3bf2ff6404ec4a6", + "94a44eb8c02107750fa3c557560e6bec0aef76e28851d236b2d6cf85565a07f5", + "94d38a120b004f5a6b66229add7e74e72dbd074772ba15d1742d4a647d2713c8", + "951e91b14a12ad616ca8991f1132fa4f763d1858822d0157ebffc429b49cae4c", + "952cdfdd6d836ab796722cb133ce2ee16b14348de138f70a79f1c6af33ad9c38", + "95312bd603102ebca1809ff72dbbf8e073586e5c0a4c68cacc6199d1e1224e6d", + "95406e941d31e39578bd8abbdb76f1cb2b3eb7bcc360261d849b45b641718be8", + "9568476e7af8a3f709e95241c456daeb7c9b80accea85bb5f7f71ad6b4387f33", + "956ff6130d82abbacde17e05a5bf2923b8a4a993b65c424d4f9c95a30427714c", + "9593d227ea9654f8734f7c51145c6d3eee4179603c050ccd5642cb50d2f790d1", + "95a05b8ab460184690fc31a8a13698b286f0e92bdcb33fb5cf6af66a29b3ab47", + "95a5f3efbe93d44838d1f96f3621c477d24e786415f4c81c9879dd403e4b4278", + "95c254bc14997a2e0c244f9213f7a2230d543948e2c41e38937cb69235ce7eb8", + "95c4df01b8d01f9506b8814f2631c677d1604853eb37cc30abd19bd33be912c5", + "95ca30ba70b76786b6fbc3b5aad9ca496be3168819291372dae2eb2b6e70e234", + "965257f692454bf6a792cebadc934b730ecfa2a179d3fe46df2232fb32d57d91", + "96d82f098fbb8db5b98ec2e8f8ec63a3641b4c660c41aaaab61cb09ac4ae15ba", + "96f19408441d972335469bf0b4364ad2cd39f3e435ba1b0e02972576769b7970", + "96fa526e45620e358f25a3d8752067f54c93554cca27cf49eb56c7913c4e8f7b", + "97320818d86e303e534a01a6dcf3774bfb866340af2a33897cdeb7ddd3f8de06", + "9747edba14159f5fb25629223ff12e9bc0d216302826344d592c5105e428919c", + "975bf5d76668602e6ba980cee7f109c9434a3a37c1b8b35d93fb5e8e7a608a18", + "977685b7cffb73f1ef26fa6a4f2b7fb5bd358a4f8f839052a2c89dbbb8a09670", + "97abf8d14bff7ab218a3cfe00a321ac0e109c828bf806129aa3b2c33ee6fae09", + "98002f7653792d2b2c41892baeed0c7871b7856cb26b78caf2195b83d11d135d", + "98302df238b9b63340092368d30b1e5290ffdca36d5a5b3d24a79f63f7dcce21", + "9834e883bee15cbce032514b9ab9335f702664589143d62a1fc71a032af3d901", + "98722a51369c5ca8848ee4144a01909eaaafb0722e5b8c04ce0c87d9aa096dab", + "9875cd5eccda046323b2e00db35a097674bff9ec0541d5b3eed73f5f8e39caf1", + "987eb90a6d108de4e939e23419ffe5a45a8f0caba5219dc6d14f049b0393ce76", + "987f394e337fb104b903dda46a5301aaff287730007f46ef25afadae5b7dbc78", + "98b3c272f364d556fa622ce5c522e9fc0f060dc4aef276bb44dd1d46d5e9db10", + "98d225867299292f12703bffec008805ea807aee642be225324659d309d6b226", + "98d302541fa8f33ca7ca098bd107aea4ed97518af5859ebb10c8a6ea039bae2f", + "98e07240600a5cea93c946519efac38d57c0b3858ae6a47638bea60c21879471", + "98e654baa488ccf9b9007a817167c6973be3bce500d7f03d1fdd73298cae5e1f", + "99428849c4d6e31bebde27e9099ab532683054ce8c153a9d10a7b65e56f932bf", + "9945f0c167dffd618b6f84eaf7a8709781a93caa6e0ba0a6c2393c54e5b7f3b7", + "9958c8980b2b1b12f3715a6f85c7aca1486f43cc2ce26172440b894f9fcbac94", + "9982a84e51afd9d4e847ca4dec6ad6d8fe88c8849074f780cc82a921f363473c", + "999257b0754b17852fb928aaf87ed897706aaa3588e087cfe4f1eea6960450aa", + "99b3b41376a8c953826ee94c16fdb7a8043cacc75e44148bc4d48cad1c6758b2", + "99c3c79104bcaaffc5c38c1c85b4b96bc953dbaba19994863b50bab0a91c4dc2", + "99e82a068e68ff5b9d9b6de848bb2669cee8923cdf9256a0794faac08625f004", + "99f302b695c73ae896aa6a57bb5c781cd38940e32e98672cd1c6387ca00a73ea", + "9a42dbb9f9aa960515c14ac7992fa86c323fd97df6deb622daf0e340b30e4f93", + "9aabcb855d31dc16f8bed9fc53fd312eaa100f5b252bbc59eacfa0743e7f12c7", + "9ad7db7364cd7d7d8697de9aea722bfec1955477728fef1bd63114bdb4e42224", + "9adcd005b974106a7f456d51d6b603fbd89c076c1d74fb6965712e9c35e073b6", + "9ae8a425fd5b7d3f65a6939b75afa5f607f45380d5fd4a54146672e940a05b1e", + "9b4eb0e364d3349b3d3f10bbe399fd8ce1368111f97d521c7636a349572fc063", + "9b572af176a0ce08ca4766f2fcfae2a2a6f44ce8138c941be9aef566beb694ad", + "9b804167e135aac643a0d5144bc61a7c188d47eb0a6e17f0ccfb5cb08183022e", + "9b8670857d068299982e3e12da1e040cbc4089da59f6652f7b8ae34584c276c0", + "9b9cb54e75f99488e11db80a28db4063de7fb107041291028130bb6637a82dac", + "9b9ddee0c23e3340dad578eaf73967b63092832f46e26829f7d05b493ff48473", + "9c0f001259d0ecae9b91379fc580c00e436b58d9715f51e6e39ddf6459606997", + "9c22c606bc416d61f094cc2dc17efd03777200b2037a27f5d485c97884b231e3", + "9c304d4792c3378127ba41e6a5ce0730ceb9098676fe4bbd4dcc5c55be621260", + "9c4b420bda3ad85c9610c1ddb0930fdae31c0509c31a65d65f48ac32d72eb2a4", + "9c4ed10cefd936aa1c795de9324409d690e05d54899d1a9e1bfc432bc45e9b31", + "9c5428b74d0734c1798659a5c2a60cdd104a631bc5c2cd482108ede09800926f", + "9c58dd0f52e8ade322cd6875f3d98cff28ff26657432258b9248a9b60400f9cb", + "9c6eb67983bac3503d1fa4f3e747f02c7a25d3de09fbd646cfff2d010810f726", + "9c97222a640a39803482f4e08a69e01a0450a3be06fe4fc9641cf0131b0e86c6", + "9ca67f9e97473ad9f858f9fd62d6c2f9c7709049980088db569a4728f7623aca", + "9ce73e107b7cba29d5ad4ac9b3a920127a0d4b2f7c81ad090d2152bd4f1bcd48", + "9d14045957382f9a922548f9dbe983f021f3d102ac0b4d2c2658dc4700d3b952", + "9d1592120d27fe3d51e75420ae1269de3b705afc3dbb5b89cce8de4ccb52b16e", + "9d3c066c345a544216a3e13515026d70b30b2bf65442999ada7ee95942fda828", + "9d80dd48429037b3827ba214b30643adf392eec87a0c3ba9d2167e12557a58a6", + "9d80ee70666e9432e5b6b7c04bf60f050ef6ef8ddc40d02486ab6eadc0cc1191", + "9dbd67c8b94fe422989baf694e2aa381769117e3664a310407b683265fa032ad", + "9df49ef86b0558c59f03245b2744e5ec1ed8f24add5adda97f61534de4fe4c5f", + "9e2772494f3e3a4d63bea115187ea31220252ed837a201930c9015ad51d4c0b7", + "9ec4ac3f75b6b18803e497d20e16b30f6b5a98d905e8afb47e895ed64aead6cc", + "9edd451a1be159f01f6432f0ad82ef62570ee660cdced89909882eb0d78c5a90", + "9eee7d2578410d51fa92385eff3d2357afc8f8e200163f99102ed21acfe7839d", + "9ef26a47cf3da4e812aabe3aa2cd7e1e6b73276f12f82643281967d5dc7bbd45", + "9efbab6a3cec41e6dc9c76eed55e3b427f7b4926803c4373ff88ebac5b4a1d87", + "9efd4b51ad36d81e214f27d70fb6e36ecc4c3566a1b659f78d6259742fcd9bb4", + "9f1009087c7b37ae089117668f728aaa5f3ecb22baa8c604cbb56eda5096b1f0", + "9f438393cc1d673b507b5163f7f99fa21fc9af792a0578aa7c117d8ef2d9614d", + "9f4608d4eab9ae10ac70d874f81e00f0c06ca749edd729478bc739c0a8fc1976", + "9f537e23b1e63ebdaeba8c83d6a910e290c782791d0652b8bed9fadc802ae38b", + "9f720ffc8c64a4886030d06ee66ddba984162e4e90567fdcd1e371052c71ce7d", + "9f83ec16c6dfdc98808e6539b3af01ce77f36771064307578895db61f041d296", + "9fa6a3c0a80ac376f44c6818436d1ba91c105d80f58bd984524b3801c57d4033", + "9ff10d17184c153e9069f5b824aec53bd5f435325952615bf79a6e0a430dd425", + "a03e7717c89a3aaa577413130e453605801d85ca172be688e483096d54d40777", + "a05f659e06fb5ca28fc989c75b2092aa41efc477340b3143c980c624439bd579", + "a070b405d69bb48b537162f88d038381a8b8253fdf90a8f080ec52c83da09f8c", + "a078011b24a347ae2c5be1d725a3d4c6ffe933a58d37fe7c5e5085315a9793ed", + "a079e506e625b7b14bf23dfc1608663d4d16ea2a8e7aa8b9e50662ba862a93f1", + "a0a32a724d874bd58f98b5f7fb50b1d157a6ebc89f6efede46e9142b9fd5d16f", + "a0b6c02e901895db03c5ea136c311331a2982addeabca9117e552b623ea8646a", + "a0bf65f9ebed8f1cd09c82107f1c208997cf8b2a17c9e2b6b588e0d3f1b3f609", + "a0c09e25ef17e1692554bdd8e22ba9a13b8919a359ae77ca15e82a818f9cc228", + "a0df1828ea87abce9638cfc85257ec619ed709cbe2192fc222708c13c256e38e", + "a0ea59161b4c689a3ecaf735f546c10d007594c544fc010f836cb7d01ea5d197", + "a122c64266b92c9dee0fa5ad5d81772d0180f88416ca5ce41e7fddbfa8ca4cd6", + "a123e568191dbccc159dbf0e2610a7dd4d4cd1d901d5fd255d84b36e93171dd3", + "a1279e59cdc47708df9e5014384f1f7824738edf0986e296526bc7763f6dface", + "a12b0d2bd7e72eb43a9219cbb0b6c5c593d94ddbcd01820f06a7a12d7a30c575", + "a14884b94402c0efadd3fc5e9dff09bfed8621c78ac78a3e50340e6e06b2a36f", + "a1854a9e9f442c7bd612070ce1950c4ed7094fee3b31beeac51ea5f545fd0c57", + "a18bd1bd8c4c4b16779ad1695fb61ebaf9d343672af48bee35b47fe1414968a3", + "a1a5267ab45977532a38fbff6777b9544779982f6f4e83c549de66b63a4ba068", + "a1ac8f2d5688e6cd4c46638539fd8e8fc274b8077ef84fd47e3cb3e148864cfe", + "a1b96a861d0b7be41db93fdf76b9ca4b90e24e451e2895b9d90cbdec14cfca30", + "a1bad290aaa74df21c1058d796b601398ebb6a0223dc2ab35831eb8d222fb5ae", + "a1dbe74c9cf79fc19bee00cae7b280cbeca083839382f8b8ffd335846c4f1ef1", + "a1e8da8c76d26d280439cb1705749ba35d2ed1cad5321c1556c45d65e5e0e26f", + "a23a151a9824adf8b8e54ea8f026762ceb74574168c081a84ee58f86948adada", + "a251b51e2297b9639a578d470f6813079c90edb65aec82753a9a1ac005dec569", + "a267aff604611134b035bdd7e8d744e6a463af69a6b4f9083a31b23d584e7946", + "a2ab534a12a3231a13d0028ae76a9672c44afad2b9b1e01ee05c49d375ae442c", + "a2b9e6fc6089a192910ffa79508000f4f459f5533cdbf6dcf7a23ceb3a745c6c", + "a2bf76422d88c525965efcecea741bd1f94a6c86c8608a6843ae2601f56aaa5d", + "a2f189b2f7404dd679f481e41b8d61fecf5da0d92409edef587cd6463f037895", + "a341833440fd48aea26699fdc1624a52eb3db99cd103932100d6a55bb6b0ee78", + "a366b08e727752b0261a79be508a76bf912873a09755df37d9d550f16436e359", + "a37b642ebec2bc4f64baece4a7c71adcd4d4cfbf1a2220e14058b3edc38fa6dd", + "a3803b601e55ef8bf2797ab8276df011c7eeef666383810ff7ade10167c04019", + "a3b25f99314074318a875968b4626d99a5b42dcf7b8c402be4c2698bead84d08", + "a3cc1097f0e358e7aaa41eccc5cc0001745cbbac0ca383a6d0185cc111c9d6ed", + "a3ccd3ee0afb191d916a5727318209623918e7cd970493710189c20786956af1", + "a3d40ca71837fabb3666dcd6bb36d375d4b6ff612a79e53c51ea31b9d7ec3dee", + "a3dc406694c180921d703ab6353d32c97a8e7222eea4e4d4c163f6779b381129", + "a40af89757b4e1f7e4c4fadcded4a6cc724a18d86d2c57296c12458f1d762eea", + "a4354136ce6b985a5d3d980778b5fddaa5ba5f445b7ce6270a9c5f85963a1a7e", + "a43cab807744dcf2bbd2d088e8cd43af87096233291eae48711afa8040b8249d", + "a49d7dc6f8218a62df895bbdb45910dc182f25ac2b7344427198f0218d0438e8", + "a4b8b9783603e6964a3fe485be7d515cdd6f9cd92baab45ae4563ce8a445995a", + "a4fca0e1b8fa35e1450019470bbacdef5dc45aac4feed5e002547d9d6f59476b", + "a5351a3819981a736641c3c50a4d6ca3faabe5ae8e32edc28bc1cef141c40d6b", + "a5617e451323f4ddb657cc1ac0c9b2b08ba01d0c9ee6b5bb90adef7d4c333568", + "a5776d9e11b439905583492e705b91a1b42b34abc508d79ce0e0722f6ca6acef", + "a583abb75ff1fa4bb7d52db7b9109127fe822d281aa9003149444f944021bafe", + "a5b422afc53594c54b4c4b16be01e923e9c79f8b593736f49ac225c951cd8c14", + "a5dfd9da32550e164a6c63eb08c1f6a2547987d2b33969ff7b0215f8ee622dc6", + "a5e995f954f1646970f15c779b6fe6be94c0ee52144ff5a2e9d72eff33ffeb2a", + "a621ad9f22c1fec373c9baffcff0c941a8d7f792f0a18586dc5125a627878116", + "a6683640657870fe56c4af8333bb373efe582acd241443c5de17ba3453619bb6", + "a6791df4f079d4614a3a1f216f1d86eaa5f3bdc98bd273f56b69b0592b41dd26", + "a6eddb8ff9858cd674acb7cc225eca94121cf50c2a9de3ef2ac05ee0adf5b0f0", + "a6f6c8f1dd7d6a28fe80567947e1c77305fdfb7190e3f94f79b2cee4b8449d3e", + "a702a62a0ce17078c88d1416a10b81c9aec85b78d2a706cbf77272e2efc1a4f7", + "a708819bb0384b19c9ebcba75702ae96f641ee3cf79fd712077e727912c0457a", + "a724449ef878facb7ee0b008e220ea864963f0e7e39cdd31a43864ef9921025c", + "a74224ae36d9293a32a0505281ad4f1733bbe621873bb0c984f8954d316daab3", + "a743bd88fce22643ef830c23b68506dd375abac6465ad60b4cb8351040e62454", + "a750263eddf945283109f003cd8046cdc153df71899d09b30556ca8fff014fa1", + "a772cf9ca1bf588c36712a36ddad337b29af5ab0401c3264dae1a0bfd0a8bf0f", + "a7d7fdd7df2d66a787622eb9a27212a1dd3f1f8a07c7e77c3be9fc959d771e69", + "a7eb99092abaf7f8b9d0670b5906ddf805a1773f1b4f62f6dc9fb5538132a693", + "a7f0ca54f2700303978ec97d9f365ad1f917ce94f606d2595f0a539985bd6e1b", + "a841679522059f62ce5a950cad182a3ef1ff961be3332d2f38192bf05624b9ab", + "a86b3699691acf7e426f5c0ce4272cfac4d634a3a106902dbf1938e2435dbc2d", + "a89081c801b7308c86904b0f441fea806e7acceed9c420f90078295498919d44", + "a890bfd6614b9ff75e4acf074b26d9f7875caf6d281bbceda0e42ecf78a7455d", + "a89c9af7c6d1d0818e2fdaaf3dd920dd20d4afb730e80b8d2640bdf1fba75ea4", + "a8ce83c637cca3cd88eda23cec113036b92b12ea61ee6bec7f5a00ccb2f946ec", + "a8f018689fafbdf00bab10aff5d77035210177e8a86a94d7ac6f631271851cb3", + "a90d7bdac4e8db9bd85268ea2b0601d32b28138dbb1af43ea9ed8475656fc803", + "a914075c60372981b697d3f25a5d0303feb5707b71cbdb81aa59e20afa38a69c", + "a933b42767b675f23c55a3bd0888051a90a6db5c12e528a1109f45c86d265b9c", + "a93d49bda71d2aadc8ab5ebc10c9136812a71ddd0e0e3a257930a06d9ad120a1", + "a950359abacff930799b98035d2dee1a0c84943487c21065b25c417a480728e3", + "a956b733a72c588a886dd80f07b9a50a7c50c71eb090eedb9813bca88fcd414c", + "a9d932a62fa73f6d6a96d146fe28c041b9452dccd5a6d0131f996ebeab7ab8f0", + "a9e1f90410ff4001727d240e543b1da201dca7540cdbd5720fbabc52d55555c5", + "a9f0c0de38ec436a4af0aeb96a04249d84df15ccc080c0225546d5006fccf5ac", + "a9f7ffb91f44bd27ad011c0721a6697ddf77dcd7e05a2473c2b2ae4b45d6ed67", + "aa49fbe535345917bb469c8f07a4bb19b263320a1b878247504ace3b668429ff", + "aa634009f9ffa363413b6b8490516ec76b3e37545a792226271e5dae9333653b", + "aa66f54b57d2ce29fd90750a393ddad20927e731f383c2bde590741963435ceb", + "aa685e32c88aac051f95e9663d74d92536871bb1570ca2f880e3649d72dcb4df", + "aa6cbf5104804f5cb7e86fd753e4e5369ca2299c248fa5386b2c6441ecf79107", + "aa89b61df7e81d444be2c2c9b7d86a55f27d70841a4009d08bf2f6f8809b7426", + "aa8f9285a8e65fce85ef3357eaa84a104d40ef1658045a2b95c913c664d90598", + "aabd8f2fd6a25e53262d8c8cf76418c49fbe2de5ec7f69b813a9013e64b2a608", + "aac35f875f1806cd3f3547362515bcda9d6151c2231d41c0f8bb62ffaaa40e8f", + "aaee3bf077a76386aa71f808b32df60a5b57cf0ad84413c35999abfdaed06797", + "ab30982785dbf8b2be3b1781cc58d1e85fcd0e061889e46d23b7abe964202dde", + "ab8664776033525115237381c15dfc31c621d66c93c0d28c58952644d0bfca09", + "abb567bcb73b46d0668540edfb8b70a816a17fc2d26e8cce5978631a3c0a4b71", + "ac2020054cd25ce5be47042209efe44a780470610210e47a07bae9b1d75c6d51", + "ac59fde9d43a11dd89fedeb2d49a2ad8e722b2be68b9fd5c5ac9520fb0cadcb0", + "ac6dabee5b5707147eb79e3bb6b790df357dc26f77e37e5b6af43845234e82a1", + "ac7a9078774e79a56683c947338c4740b4d0218ce1296b660c91fd8fae06a0d4", + "ac908e26fed79816dea69ab9577341edd806b565fcd971f8a86fff8e3db4b932", + "aca04b419e208d72cb2769900f22462f9df8b0502bcefec22556bdedef44a110", + "acc4ea38d065d2f3336ed088cc991b07e581d37456d4370fcd40d81f6f36b7e6", + "acc5c7ac1c8bc43afd9fc0d870d8a65432604c53294a77ee31eb6f60637d130c", + "acec02d51cbb9f79b85881f375b122f7fd1bd7483964f86729ad4aa8cbd922ef", + "acf9d606aa42cdce69f9bf465a11ddefbfb50d2b8800efae9644276aea08b2a4", + "ad0bf4a8f205d5339e85a4212cef2b20b7ced6280c2e9f5f45ccb004b6af322a", + "ad511b083a403a2b6d901119d09bc310afbe220f66b4cbc997ce6e1b43923b62", + "ad7bc98c757fe2eef34f27ca38ebf39cc7560a6475ea5fb65b53a6ac62361296", + "adc0bc79f7cdebfda01aefe2fa44fb0406fca002c0afe4f286022b704074625b", + "ae16ab0f91f2f878129865aca27304a11882aaf09f7a1f6c9c886216601fdec6", + "ae21a5986a959789da9807444a242b62794e9551de2b3e33d472a60a891e9181", + "ae39a4ac22a55fcf1a8a78ebdc58fb8d1fe5f7a6b4a51f0ea36e65cea0c9c868", + "ae39ab3cbe928abdccc9f6eec045dd500e343938bad19e3ef945b06d5fa16e60", + "ae454bc89861d4a5c6244640aa99db82c11f4196f6386e5a66739bc9a458e534", + "ae5bfe44bf7768585ce234c04d1d0494bddf10ccd26964ba59f0dd47f0c2b268", + "ae8e2940d3dc04d7a6a8a6aa07aea6b1c7e943d11ade25e6dd5e489dbe0ee8d3", + "ae8fdce92af832a981dea810c831d8c267879922314f42c4e9f6275776c154d7", + "ae936a3462d7e22809bd94ae5931ae70430f00031ae97dd2bd0e7ff78343e728", + "ae9ec3f3a37a86ce3e8f4605da3f65632e534875fb5bd4ade1db25c1beb1e1fa", + "aeff9c08f60704690cebdcb3f6ec1b9095c9e7d75504a384fbc0bbc11c9d5341", + "af1db02874a387fbc5f94838d3a0b80d3a940e21e95730f6b7d4cd6bcb6dc022", + "af40ef6e683b838b9d629021d698846993a024b916555f7af5ceebcc0dcb73d0", + "af5d9abbeb59f0510b4d1f5d5b4bebfafabafbc03e030c467c68fc1026a60db1", + "af6ac547c28b21459f56a8506afb797e43f8186c9c08519b19983ba48a8e48e3", + "af94252639de288cd696f79c779ec18bdd6f087b485d465a9a2d53be647e4bb3", + "af9874ea784ef6a5422de0f475430314d5c5c80f2d627c9c4d5b101f2dbd5c95", + "afb8867def2977d96ab00821322d5f342c123bee85e33c9530001026a22b1fc2", + "afcee626e7e0ddf98780a26e9ce0627de9b45db75e8bf1d50f86844c0c8e9018", + "afd476b9cf674144af007dc146cf82e7dbc8fe486b80888bfc9d82f70588a23b", + "afe06523c00a6e5b945484aebfad43ef059ad8d648cb1bcc680ca26c034a6f80", + "b02acd3d0ec221ad7b8893f16b79fcd8bbd831f74409baaa2184b623a4c63829", + "b03763967523aa5f431d80ce741e8418d8764ba8316afaa6bd804d51f9247553", + "b078f6c1ab0b1f8866eb77d939d2d70f9ec748e9a45cc09fee00a1b9f8cc7898", + "b07c85be90171841365bd23640932d9a41511e389f7ffb9adfcc5ab66d89bf57", + "b0ac14cdfcd1f096709f5e82ca65c67c2e0a27f2ba6d688a954c356501cb7d20", + "b0b04d5848dc77d7e5908eb8c0a6df6887c743bfe1078b13b14571ca3269ccfb", + "b0b999be0229df105d8b877277e75f63a392daf57752c18b3fc14a6c1ff674be", + "b0c43e875e30694fc93cfee195abe7069714ae7655347bc779694bc60694b637", + "b0d3870ec2a569f0f320dd68d4c728fa7c82c1423ad214286e3e5f9de19086cb", + "b0e33fe309f08f0aa5750e1184da66b702ddef2b9ecbbcf2e3fa3d35b465a72e", + "b10d1de22a03af66a12caeede346d3b183929a809d5beda7f13a42e69eb190ec", + "b11f3bcb872439562b3233ed7bc52f2786a0b449a41d6701b74666208f092e5a", + "b12ad0ccf78825d7347c3a7bf4aa5d5ca460dec3ee1584db25f09518fe20baec", + "b156c635011a93a648f076dd68f26a0c9aee6e5e674ee4a207653d65f451338c", + "b1628145ee10ce0474906869897ccda3c70dcc5334d33223d950aeb2ff9f0f57", + "b18d03055f8f39c8a459af611514a947e6b08e51d17cec2d74f2159cb22c50ec", + "b1f012120595870ca5d56c46ee5882430dd91ac6be210da64182088b15236bfa", + "b2366b6bca134f7a8b7347dac3bb0b100db6d3cf1de9ea0eeb417e1a324b9538", + "b24726902e3146d4ab97bf67178ed77840fe5d9ff6942d105a48c79f6deaa0c2", + "b288350c5709578836c834d4955a2fb3faacf583df46a9edfc32c44c116a8ba9", + "b29ee483be358080d8f6db6047e42600fd7481adc4b86dbee6facc4199b39ea6", + "b3088bbd2f0cb560ca5edd8269eefde6e52e98807dcd2daaaa04ff531f49a983", + "b30afabdba5dc3dd57cfc03f8c8142972748ebe769127fb481171ec84d5e6782", + "b30f9b407fb8ab04e4108164a282def889f0c6a659a6925051dd82835f739858", + "b35a6351bae968e768fa415340252ca2358af96c3181ec6a2b414656302165f1", + "b3a4be33968e42a4b8851312c7d9c996e078c7ddb25aa484a34e559c65d17fb0", + "b3dd9456f5fd8e73e0a9c7b0ef5bbade876a54e0b1f91868efbefaed8e6b62c0", + "b3fcd7c5df285b09027b8ad2356c12e05736373385da2ba556796de7e43b3094", + "b417c0e7dac8894fe8b58e4a9b2ae13525ef8065d68181dba35457570e7e19b8", + "b44002ae51ac10d1d0851b77a8c5e5109e7369fae9d73b459e7b3aa3f0f6cbdf", + "b45ee5251159a5e01dc599e6e1a30a304859484cbd066d9ecb6d8cc0e714ec27", + "b4782cfa6af28c51d1a53555e241f4f68589f6e89c84b7246261344db3bfae0f", + "b47f6e0b37f9793776fa983dba2dd994a42c32346b4b1f4c9278ffa6ce3529fa", + "b48314748c45c8dff55a2316ed448bd72f20d231778e80368376bbe483a2b1b8", + "b4a51cfb2f14fac383167432544f2b82e7e120014acf948aaadd33ae1ed331ca", + "b4ad08fb9090c4ab98399dbb2e98a5cf0b334bd88580b38250cf13c9d386b4fb", + "b4b38ab0899c091d3ddfc7a88603261cc3446a70c5bc09d3ebf651f3f0d6832c", + "b4ccdc6ff79881b8e5ce94394ba1e8000be7611793c98624c09b7737325617e2", + "b4e174aaa52c030cf6b034355335fdc549f44681908979ff6500ea6dde0cb67a", + "b513c611ee93c3033d681ac34dc5334fcae74e64dcdd1cdd16ca84269a4cdb6d", + "b5326045d5af7216b123b3f45417dccbf6e9cd74ce36508cb1801793f1ec789c", + "b58a15be3759b35178f2a230d09976799f0bfd2bd4b47f730ffc5133c8ab0d37", + "b59a22535f3efb4fbb1a9e3cb7a414a382ee70835a3a34bfe3f7304879a0076c", + "b5b5a06c87e9b6185f039dc51b297bcb413f09f577fd54dabb2522256e0d95cd", + "b5bbbafcf9f6b175e3dff8d0aaca59059cf2299247cd7d825304a26a13e25b13", + "b5c1a87db5d6e07258cff72927b1820ab5e83b10c0029456d43476447dfa473f", + "b5c242e5c5c2dd850f6448fcf141f42aee95261f7cc5ec287991dbc845ee0cec", + "b5c7c8e294414ef68009c0ae31352d550b47a4ef468a29e034a81acff91f6c40", + "b5d318dd57a9c80b3e8c3fb510c8109ad38bee6a3fe0e4a495619be1758227da", + "b5de3dfc01a42bd025e966c95808dde6a80c364b73cbf8f0d42f1f5a4e5eb023", + "b5ebd74e821ac2afbf6df49f7d9d4071750dfa86cfb91dba56c403af731dccd9", + "b5f2d01087a437e4ad8e7ed4a37db82ed01e39b8ce4fc4e5213cecd480371678", + "b66b4005cbfc9b8c8548e0348990af5da72fdb3bfe9fe1c0e01852ae0ee02eb8", + "b684f330e18b73f11336a3ff76bab18dbdbbdcbbb4e9401707118fb8d319ccba", + "b6b886286302a6c5faba0e67d898b1b416706ef980e779930e8673699c7811ce", + "b6d1a49654eb4109c9a1426b33c4b3eee473f744c7dbb6f0fd0bc859d4d07ff7", + "b6d8eddbc4f2611621906fd72f117fb090909ae263c6583654380efda492570d", + "b75ec2550dcf9a890403ef27db99f24849cc1f172d08d68e57e5aa2639b6db3e", + "b7fe798ebcf5157ee4dd11701414ff41e15d63f74ca55895e53db3a3e6b7d68c", + "b82d43dc022095090b121186bf6029625117111d18c836eb0b4d463a2a00d70b", + "b84de002e997f9f9e0df1349dcd3238bd91f30dd163180a17dd7cddaa2df57ed", + "b922630a7c7fd034f00cf9d2f940f32862470390df5e6b97858a32e3b3411290", + "b98ea5c34a50ce0e0dd1cdfd6075f7b1832f0d5d1252b7459feaf0a9ddc56dea", + "b998048a5255c631eb43e8cc0c8b892e29b7b9de408e4d99fb3447862c734e7a", + "ba0e2abe57d4e8976f7b6dee8c399cc76d672974a63968bb22e74fa3f3c07c42", + "ba17117fe128ec4754d6f6d360227770b44ae0d3c0ccd6cd676c15e61a959b6e", + "ba51369f3f4aa59d6bf7f75cb9359705bc8da0df0c6d68c8608f4286d76d333d", + "ba73c3a2d34ebf5fb4faf99ec8b0d20104f805520fcc4242316692e239584e31", + "bb0abbf276e17a783bcb113938c42f3c3c47a263a3bdf13e3efe2881ed0cebd9", + "bb14ff83d06e928142c58f803627b6b6067f7d7010725f231d3bc3991537325a", + "bb679d31a017ae04a18eaebff3f6fb8f429b92f73b9da58c65f8f662af751fda", + "bbb6568d2f337cb7194aa2b2aec0fb6a188c46d06a714fd27f0eb79f524c8315", + "bbd3c5a145a404a237f0d5d6900cc4b3c1767160954322a286491c874a50da5d", + "bbeecbe54c37f58d527baf903067f9dad4708779c45cd500d58c01f62b7f3b32", + "bbfcf175ab1c5bc358e67c71a7e7b68a7ff599eec1846eadc93113b45f15f036", + "bc2bd8f50cf3c34d2cb52e4acb44cbccfcf12768a96e14c39f8a117dc8d1f120", + "bc547c6f717ee21173258f4c941a3a3c4acd7c8578ebf8d35ef8b98dff04ddfd", + "bc5f61fb37b1c01c737ab760249b7f12da334e3adaa471ecf20116794b71c1e4", + "bc64c4c70b7cd5ff2acec0c8eb0b7de2b95c0f95df1107612c6c86aacd750b75", + "bc6eaf5d6e55fd6755fd47c42ba065cfd837ee952d1baba7caebd5b18b5a04b4", + "bcb3fc371ffa17c1c073cc2accd53923efeb08a01d24ed7d806532ed460eeb2f", + "bcb499ca031752491bf542f8ec41e8be53b4ade5d1955f0e25156181d3bdc13d", + "bcd9ad96c094d2b37d619be8f3fafa21e1d7c541e355a5dd8aba1427e2c5da0d", + "bcdd90d4e12d43067f3bed7dc8774068e4f42335771892bdb17e8a69afcee82e", + "bce86da49ca3b3e13bce9c8a69617ebb5e761edbaed7ac9cb914e3ea681d14d6", + "bcf87d42810cf1d06f5b7e1c63f715ac6407cb2fa729488030612a8c5be72cf7", + "bd23a935c7370573992718d3062157b935bdba9e90d5ce38a2f039792ac7706e", + "bd36a8c360935cbe0fbdf5b58237bb8eec29a2620100a7ea3363f1b977e044c9", + "bd4742ece96e7508c7fe3019146b9e886d39ffa7c03cbc486edd29e807e97ac3", + "bda5ef41f329a3daca0dcc206bf1fa8e743c267226364a383065e9b453d7a5a8", + "bdb0721795731acec21268778b65231fab27a672cf0c89d144ce4de017dd3da9", + "be02f86b8ffa1a1090f50d83e4592e652fab03b76eac2de1d3bd1461630612b3", + "be267196ec200de80b7d4ffa4fbee39772cf74936647dd847851289092f42f62", + "be6cc9db2102eeccb24879afe2971e6fe5aaeda2ef1042fbeeeba79a1a1c5635", + "be77bcf22d386e03355561d906a4d01f536230205251150ad805e06b9e4bcced", + "be9678da0d814cb4b23b38dcc097182a377fd1c709439cae4d9d8549ede3e1ef", + "bebe2a929aaac911b350d2d921386029b225bf1e3d3d08da214a9a5dc3bb62a0", + "bed10168e4ddf079981c1b85076f28a3ce061584175feec439e0f737b950865b", + "befdb6fb0fb17e561e51bee3afca24285c8055bdcd5ed793ac12cc49939185e6", + "bf120a9a0f7df85c1ba2da3658c8978748e3c3b1bebaa82fa8d8439ecb4da32e", + "bf12504a200f2bc6c6502eb732bbf3eea946be65c0837729ef477e1ede70ce20", + "bf21233f56d276bfa502ad634f5e8c7f7ea04421db5d894da274b2652d65eec4", + "bf3eb231d9cb0dca6d3f37766e4f5ce99a62ac72526d638dd7f28a8923834afa", + "bf7e9e164281f29ef0eb79a1e142065d86ce32e9c927c9d94c079ffa98174eb3", + "bf90af2b98795bd827adf4c4b2d0b7a583b37f29240a8f5ba4edcf20c5a9f3c1", + "bf97fb94799f55df9377e1a428aea35a9aafafaa53a5b8d65d902542d8c6fd23", + "bfd67afb7e0d59083ae71a16f8540a54890e5fb890350363b0fe33e344e8879e", + "bff20b1fcfb36e27b079aae822ec04b4b25a2a5809a1be3387eb49542ebf8735", + "bff67f7214a8e1eb41e7bd601e76092330f41d8ac24b5824972c1a4d6c050be3", + "c024a64ca09b458d19a05c552070c1342763c6d68337bca48f0097432fc3595f", + "c034c03b47b63c202ec02eed6420878b1c4bb0e2ac260d91df82c54f1ca190de", + "c04ed340bae75e79cdd78942e9335de2837e00b273efa911e62f7e53eab5ef25", + "c05e4185c5bc92b4a260dfc81e9faee47539d118f13474933e5ccfcf0be3f740", + "c06524578ce761107a87557d97f85990be11337eaf5e3ca90fdc4b260baa75ff", + "c06549de3238c0c6e7b2f07f003b2f0ffb4c9c27edff4bfe101a5795e0bc6f5f", + "c072202f9b3b28a346cde65ce651963430c942590739690c3084bca8119015f3", + "c07233637a353018d3f7bea24ae21554a01430c39ee202d62a458c13b199c5af", + "c072353b9e9217c2abaed2c476d9a771b759c59578cceb93d264740aec6a4384", + "c07921530aa4faf4287314001c48ba740df2388ca8ec94dfbfb74a3eb50d58bc", + "c08c7e43682e8e4c87d8be637f9d82d4b82429e4aa372e6a2e30d373e93b4d32", + "c0a1f61328f349ecf9f32d154d912097d236ca5200c509253c56dbb2ae7fa413", + "c0b5635ded2ca1c72e4ae38072bf0bb69a5cbbd13eb59acd42599e3e50aa4066", + "c0c4e1754896994da712d47d8507b7f875e99a4d5f65910d989c73aa800d109a", + "c0d55523dbf8dc16c804f877dc889ddebcd2a5359ee7148c21d78935c2fac7f9", + "c0e43d2d0c4f96d5a4d1d3ab1f1469c31d8efd1c05132a75461cc6e8c826f0b9", + "c0ed6130b0bdd0359e918cb5d3b2f6423a42fb6efc4af3d556d1cf4e984c0cec", + "c0efbdbb716f87ad84e409e078f1919460f3de01f2b33e53ce0ffae28b8941c7", + "c0f652e25c47ebc5fc2e18598b76e83c656643bdb75602a92289fb01fa0066af", + "c102e2f2303f130e73f2b2dd75b6603fdaaba996a1ca4ba043e9c136f7c15420", + "c1253ef0ffbce0d7f76f114b7184a12dc245ef7cdf7eb3ea2d51332101a4ea08", + "c16f94d277ce05d085f72444894a40c6ec0479c0a49398e2da51073effdf2fbb", + "c16f9bcd2c4d8f79169ad9955e892aad835f6ced928d1579087faf42955bdc19", + "c180609592cc3870f3dc650c79360da83f5102115b4df9aadd7023e0b7691554", + "c18677c93d36734e09db302b680fad865fdad625dd42ac98a56afe0bb23c7b79", + "c1b11ab212a92c06204de8eb8e33db59c06fe2cc9350b692f91238af02b77270", + "c20c660029c5bb5c25ab60ffa6cbd63eb50e1ea43dac36926bd32c87e0d191c9", + "c21d65e6dabab9487a553c295e2149e194dd0b17a36b209dda8c6acbc49b691c", + "c21fcd5859d67db5a7fb511fe21e68f37cf6d0ae1a7eb3695e3e46e74d1caf67", + "c2288e2a9b6d9e23c664e5a238815c204fc92379ca4b1de8e369d491015f3fa6", + "c24a3fcab5943b4336cb25e2c4496b62055967aa48d8876577b9c7bd67cf7c80", + "c2a1d0fefbd6321505cb03d69d49cdef50ab1cb792fec9cf4a23a2dd0ff2cb62", + "c2cfee3f872b61bf99bf2bd71381f5c7cd279e249bf804b7b8e2861ccafc7776", + "c31106c0aafcb04e0f97c35994ecb7e89348b7948861d5261adfa6579c5f4f1f", + "c33032b249679e0f5113a3448b152fc582acce16b6c145398ea5da4b13027e68", + "c34b9d242a85f0dd41f34a88dafc8faf1b3ac03488731cd6131f4083819dfbb8", + "c354687dcc12700beaa4c12307ed810a27f3ce079cda07e476c5f80acdf9d8be", + "c3612869eed6d45631154807b57745bca7ce167ebf45787d9bd5bcf8b234fc74", + "c383a9a9768dc0b25b1259b736264f763ef1b1c651cdf24a475bd267c3b951d6", + "c399984d6b0c9d1999e2805b521bfe0a27488982249c821889da7727d760f1ac", + "c3b28e516f3b74a15b151adcf82fd9aed69d90a91aea048fb03ebf3ccccd7de3", + "c3d2ab670f1ccd44a266fff4bde9c07cd7c7911142e4a8abdd2e23a07d822b3f", + "c3da57acf36595c800970f7880c8852e7c04d7daad8a0f27105cefb8068eb5e2", + "c41af0d05d256ace803806241254b43a9ce52c0fa512ac9a1a5027c6a26a6bb2", + "c450137ddf1ac2940dcec86a56e99ff42ee74a420d86eb28cdebbf2a25960012", + "c4536a2b42d40e58686bed1dc5c422da58fe3cc6dc8e7e9e6f9ab1e9120019b2", + "c48a0a0c5647f069fb67e33e9475256d4f7edffebed6d1472825ee72753aba88", + "c4aaaf90719f53cd17a3187436bb383c505c1ecdbe6907c43f5513461b3acae4", + "c4cab42c78c297b8fcc09523dd25ae6a8a8086e5b4263ac26e18f1a333ea1787", + "c4d45104076f6520102d542848213e0889b2d563c9777c13fd4389c7929f0a70", + "c4da74fd5cb8e52860b27008effe3da41e6360d9934a66a52020437cb0c0a153", + "c4f814cc6ed768d0e56a13d19679631906b07418c3e033ea24f143d827a40193", + "c529c725b71da442936bb8ab052712caa8812c1f3176170f460642223274a589", + "c542024eaaaa752bfc798493ee4945c876146979711fb9cc49c9b71d84a508ce", + "c552a359e5e1a4bd1ff4d6a5313e086d4fc9382444d7bbdc6372d25052301ce1", + "c55839974c8b040c7bbf1adf0077afb3d0e699044c74be927217f2c445f7091e", + "c55c2a1121a5d3fe7144f9a737652ea0e0ccac440b34ca5ae6f739322c14971a", + "c5616f4c2c8e787dc1a217841c54667eccaff4d25a6f203811b31b8171a283bf", + "c59ee61269705faaf361a3fa88eea141e108dfe28a427c6e488b13ae88de7165", + "c5a5a6d1f10a0a1d8d7eb22f171e311c031144da9c8e1aaff5225089cec00163", + "c5a6a1c441fb287fc9d2416bddfa134666d87296bc9079fdb40ef3dd88f834af", + "c5b2951e583e3f5d7bab20aa1f620dc8e62071cb0d9fcf36011cee5c79f24f0a", + "c5c0af94a126c1b7d8be21886530273bf5db83f4ecbe624034dd36af5038a16e", + "c5de598be6e1b4db13e787c719b2852ab2988caa4d1909389f5c60a3b4839232", + "c5e4cdc0aa0d84e16a372597c6261b520cad72f635271c0942fcfac0a3aa6849", + "c65f6259756a3f7330763fc5b04a03c6a7bfa4d7adba7b24cb52bf25f8ccf582", + "c667a9f939ab7cbc24f16808a496913e96c956f1a30dcaf190ddebbd2e2c37c3", + "c685c7deda7bee11ae26a52bd9c2e0bc69725db3da8c11ca31de05126b59444c", + "c6a6f1bcce0906960d0db892e8a31402332a77c0a34f30af2d1a1fe4577f1ca5", + "c6e52e8fc9cc1326987043c2e5cd275d9739742e759dc500b48dc1e1afb9990e", + "c6ff80d4c29f48b524d61b8d7637ee883a639940f373acaca91985a8fd2760fb", + "c723c0f98c601d0f562a1069910741b66a215bf266cf88a2bf46d39e54c8337c", + "c73658bc27f27a413c5b379e75960bb121ee0ffb6e20209bb8ba3822d0d7b73c", + "c76e6da2382e79e7b091a92fe16a4b1a4329533e5ccf782dfcac91a092585722", + "c79078deea0cabe9b6cda04bbd16a271bf55231129a32d3923b5d61f0e164493", + "c7b43b191278a9fbe9cca46d7250b30521923e3c969a1c377a266770d2994618", + "c7ce0c2ae2170931ae88ed3aa666e043a9571ac1c725ad769b055a7249bc1fae", + "c7ecfe011cdb48ac32b4a4ffc380ffcd844620329f25b3eb275dec9548fdb13f", + "c850c40819cf6be6039abc709e34a90a41d0299aacd5432bde96a32b187c7939", + "c8d5f6a03cf5578ff3c27f20418a5fd7e2f75936fdf843e7c6dde8119b2202b1", + "c8d631c5dd8e4329af5d97d73eb7d0bd2a0c7f18ce291731c4ae9b5540a300ba", + "c92d8d90bb4f444c193024326c23bbf6e700afa1564e9212e1c39529ed08c2b6", + "c932e5515c72dc46b1695007001ce1032a77d3a8b1b313c2c892ef4b2c9d7d4b", + "c98f48a896e4f421facf2c022912144fffe504e3db694226092561f802f22830", + "c995710e45ca54270ba28255ecf88460be706305607354f0ea067c3843aa43ae", + "c9ab4df781c5b5c2ce6e3521532cdf914185234fe0f0a4d8efe2f882a53ed368", + "c9b3e6759a8e4bd06b705333df6b1e39d0f285fe2f389f5aff84c1d543ba0512", + "c9bc72a44e5f32adc8d7f3408a983a192e4fe97bfa408126b3074c0b759499fd", + "c9cba066b29ed7b13272cd5ed7ba87332d37b1ca861a14523b68537a4bcfcf91", + "ca0dfd82bd822da0e32efd6402e176a98e724020726654e2317af944b1ae8471", + "ca79360ba5ff1285e65649a6d3a83a99010576e08d618eceb90ad103aaf03490", + "ca88c8f7dc5e0b1dc4b8f5000cab018d8a2c7b1c1f160311bafcb5e388a9452f", + "caa3bfb83ed58e30949a863b79a2af63f9e16ee16b7d7c311d14362e907fbe9f", + "cacc91e38cde889f3a3ed7ed8b914d6ffbb1231c0a4ed13e2bcff10fc3464b43", + "cafd41841a6971c11257fb7e6189c7785498e20635721574650fc381c539681e", + "cafd9a430506f2cbc372666b6543a890bc6934870dd4b5ee9040a47032390b94", + "cb0517ee94da12b8eee80c94ff7346510d18d7e285f951a87e78762cc47d9e58", + "cb072729d8819401aae11fca56357df42a34fb91fd5b902cd006d7fc36af2f24", + "cb10006e3b9263921ecadde5362e86bdf3a8146c22db36e458f8f4bcc6e526f4", + "cb1312d8be0cbdb89d8cb7c9e4971edd45995484af46b1efd4fe7f1825e4031a", + "cb15b5de90f1f391f397471faf423fd6e70259c74cc4209a792986e4e9d57bbf", + "cb1da3e4cfac0866b4f42bb6104d1ca463379e9a1b15b2ad7a1669d70f457ca7", + "cb21511ac20b769340b179d9e5b6f1683b3a13fb44093f68ca225be514b20b83", + "cb8904eaf404eca0238f1f2c1532feb5746bd94ed9d7577383ad11a8f4e62d77", + "cbddf4858f884f84126efe1c43b65fb1269d916d0f94ee73cc4b3b5dc2949e7e", + "cbf5706c38795e3efdadf678690e074e3f2512ff9868d8665655811d1053a5dd", + "cc170a33d8290a975c2c965f0893eb728ac611d55b3b06b87be576b85bb57877", + "cc25f4b88d039fa902ba63f48ac72b27a1c82c7a1552040ceeab9dc788052d3d", + "cc30c9c182b14d50bca7900e9af56708d83adfe54179fbcae40b2db471db46e2", + "cc4f1dea9120114b4b3bfabbc5d96c080ce5ec16d50501bd4543a2817e6b834b", + "cc68e4b42102346f64fcfc28a5407d758d87cad2b058b6b500587e60b1d54188", + "cc88dd027865c2036edfe3895b017b69977f1cbf6a90cb1ef8596f5e0d886f78", + "ccacaec81b3af38e6b70ce193c830d1081aa02a06cf99dbd5c13e035afc0a526", + "ccd0ebc51296dfcf4e63cf0560dda61e542e6aa4183958b1595f30a01057778e", + "ccf399adea2bbc343bd217129eb44c3eb6516916108c3b4e8d126179356fb07f", + "cd371cb03771295a33e5ad744fd245c02d34de6b06609f3923a6c95554b66775", + "cd380281327a8079b9eec2f202ec9f51f4008f9ce6d096f523dc9901015bd901", + "cd3a0a64425073cb18a40113b21de741a01166932eac1b1b8bd358dc85832af5", + "cd456c636492018ca447218c02d8fa52ef443b088cc6a57a3aaa83aebd71c370", + "cd4b0b156adb451ce45ef6f93d44353b81d1ae76ee0d8eaacab68832c26ba075", + "cda8fef741223993341e4ed5d553000635a1bbb0c66a2b0a5723542077dcdfc9", + "cdac002519a0b137114e4761cfa75f82e189ce7ed319655f15c37ef04e4896d5", + "cdc0e13996861748790ed8d99cf83a43a8bf8c1d2063dc89ab890325a5eefc15", + "cdcfcbeb98cd99aa567ab0c7bff236b3572cc9a2500507b189f6f28de7425860", + "cdd0ae1676d3f8a072fc5218723f6e2e515a8258b82f51bacdc2a3f78c9b2348", + "cde33a13b4c099644749d2facbf2d4341a9e9ee10354095f774310cdcdefabb1", + "cdf8d5aa0a6cc5b51a2944f3c570de8794e9e945317c3e1a5aecd87cee420299", + "ce005b7688f239258d6e6714763b3a37aabfbc9ec585ac5fb624e228c2a102b3", + "ce0823ac2f688885cf9e1a6a3f45844332ea6a428d685c1cb041feeec764ca32", + "ce7855544d63ef0544126ec17880120aeb4cd2b834c5bec78f33e409a1b117c3", + "ce796c3e11929e06bda4cf091d8108ddbbdb1f16b95cbb86dbbe3950b36dc8b8", + "cef2f0750274e9d6fb19a80350a41255bf66e58e100a4bfb1fff6833308eb661", + "cf0491fdfbf94a1fdabf6bc700e5bae8a8aa4ede01b3174cd79f8c8fdefa8fb6", + "cf1206186d32ff4c5fcde159ed0322a4a16ee0be90a79192f80830cab670ca56", + "cf1421b73484081f4683ac57aba50831a901313c0e4010c12485e5b4d2d54445", + "cf1b6ca2a327b7a0553e51527285f213016fc8fcdb2841895a5350141066bbd7", + "cf596932c32af9f99e3a9d9c4a434c18d49b0698a62ecc760ceda02606a8e77d", + "cf6e1977d9fc2dc361ce318e15d9e0200f80299349b480efbc05bbbd8fc53c85", + "cf82c6faacff4d071925e08e101145a1d140a21a95c246e6a0c221837cbdf464", + "cfb37033c2cc4ff84000d6587e152ab3bc37bf44d06a3b8d8371e3ec36b0b7d1", + "cfb3713401986e4be228d6139d98c1b4b5059d8eaea055cc4817c965d0827347", + "cfc5981766f3757e4f015d9cb9c905266e1e5a068d91cc23c9fb54d0f4626177", + "cfccd1f89b646ed289d68e41563e44cefc6d903526af3b51e003dd0e6ea2eb4b", + "cfe2005d6603a710aad7afa82c13c9b27c382f62cd602d371e2505dde16fa1aa", + "cffb93ee92c901b673855d2fc800de2344a16785b8e750abffa64b12b4643e81", + "cffd8cb1754770684de7bdb190219a8c81d077ab861f6f1a2838d5b8caa24af1", + "d016b007e6a831837ffe35c8ab482f5c1220e3a96fdb81c493576dc94498bb0c", + "d03d5a0fd96b5ac4ed689dd49336670aacb7b3d0f727347766b71119d421b34d", + "d04f3c5d101b4e1c5c63a1f09bbe3af3cb7a1e3acbef5b6ad46ba8af4059b743", + "d07053f7c2fe9a0bc31c4bf4223236226952198b195504b958c6f67bc3e24a0b", + "d07e1e4b8e401e7a91201526e70e8f8b9c894db55cd961c81fe855d3053bc5b8", + "d09c34f46a32f80b3780e18e54292a344cb74eed88945a3da07a69d7d3dca009", + "d0b47688297a825cc4a6d72cfb45e24d17a5da9d8f3d43cba2af8f13dbf3621c", + "d0dd6364b3886aa40068a8537aa6719ecb9d3a7e185ca5338b3bd4eae43aac31", + "d0f14e441af05c93061d2b8b45ec72b5191e1c998991ab103ee48902537dafef", + "d1667d0fdb73ec24000d778bb95c383e345dea0315c43ae462e510731b674b18", + "d1695227ba09c29e9e60c5d5305e2b1812134ef5bb04656b7d54846e970c51b4", + "d1831fdbd3721ac4d291560f54ff189aaf13adb759d344b8360495a5436f49f6", + "d1ca52ea943499cb88d83ed1a1d4696727e30fcf9e1c1a888f1f8661de1e5b64", + "d1e292907c03ea6dc5f98dea5f5e8d8d514fbf4cdafcd28ab4a8755c637cfe34", + "d21907d45a7fe36bb446dea4c790889f472b6de896800ed0d68c0d73ec23edff", + "d22d1baeb2fd07a953539f8ffe95fa404fe31cba8058f2234ec48007e0ac9cb9", + "d22f899fae63fc221efb42c1b8c03c110d1d9b365c3f17267261e5c445454f4e", + "d2309680eac9074c3d554b75c8477b7466b29c0ca6140b4bb61dcca09c877a6b", + "d254fb11bcd49612eb4d2189ed13085213ca8f113c9dab52eb93979adb7dcffd", + "d2771a1ea6f906469c07a1e8fef04e6e643d707d7125a38478f997778c1cc90d", + "d28acf5fbcf2510e976669e6ab05fb86db09a1dfd1eaed44fb96c5363dfe8733", + "d2addf95aa41f5e13a6ad590b2713f0ea7670624ceb3df45f5e6f0847fe73863", + "d2d9d70590b8374c968d8d982113565843967802241d93306f007f9b79c1a99b", + "d302274ec5e8a3e1aeaec35db97550937668e42bdbf98e53948dd4b834171dea", + "d3138688b8d2099d4601de9ffed504cedb79976e47aad6522e2d4ff6f13d5994", + "d3169979e682be12dd5305e08dbc513429285768617d1b7a98efa20286501d58", + "d31b09e18c9a5c15705119822e09ed574aa647bc38e21d57355968a65fbc8a05", + "d322f0d4bf8d5e22bc3835516eac087d05a568838effc1b703114422b44d5cb4", + "d34180ef4ca91e938c1245aa42202fd82b50fb53a8a2c42e4fc8eb478c24b02f", + "d3524da45fa4191acb576840d36223f45e1bd7e25e406b6150005897291bf4e0", + "d365b706c8193737aa79d0ce71d6d7489ea46780b17df89ad02a66c13d9c1eb8", + "d371916d7c47430779a55070128d0311441547566cd9f18afdfea10e69191ba9", + "d37493a4cf9bae160adeda969501f311b2c616a56327b492b027721b48801afc", + "d382ea48424cfc817aaff993d824152802559459d71465f25a13e90436043cf8", + "d388e45a663cbf86578fe0cb8d19c0166b4b11e0d1a3b48103390561e7c0acad", + "d3d1dedb04ba701168bcf97bfff1d5589fafcda8ea1153f361a0bc94e3689996", + "d3e4dfff94666f5375e8da4d0a1c4167909eecfdef85ee5f7f85900e08797095", + "d3fd3eb99a4b1e0cd42b805bee67fa353a11bd39c51935e07233d751e01800fd", + "d4177cb874b7a7307336b8774dbec5db48ba5bb281861f49f45ba104c609c56d", + "d42724b6e4f8209ea7139fb53c761b47e923fc360ab6c2c93727d2880c4d2ec4", + "d4445cb8041006a06f30940f452fb25ab5718c99054d263aec03e37ffa554a0b", + "d45f77a1c72f28f05ad7a0d26f74ab8e263d381cfc8e330920e7ff92de4e7e78", + "d461cf05c001b85297342bec480204f28d5573fb3c2c7a0f7588bb8e9aad4dee", + "d487e8e2502e45a634f6d83a3d784e66e203476dbdac51742edae02202a73ccf", + "d4911ee2e307625b1af25cd1ce9e9e42e8f1013a53e9c0e6c5ed1aa6c762879f", + "d498da45a43990513ae88b937d74bfa2a2eaa5d6c20d4fbd8aebd2a4a1a77feb", + "d4cb4c626dc946ed93e1e15283bc68a6b313b2449ec60051b1d25589277b8bc7", + "d4ec74e5fac26adc6ac7afb3b1fd3ed6ab7a2288d882660a95d0d949844e3eac", + "d51c08ca2193e5449fe9d63168b50130e9700ef88c377c6b79f8eff85ce9bc18", + "d530bb300a81c9bcf846ef44ba7d1bf309be17f5647b366e0b321990d6566e8d", + "d532cd7d6f2171be22137ed4b733dd0c79efcd8f560663a4e6530051b69dca1f", + "d54c515b8daaac7db5022b06c73f56eb455768823e71a8c7105c70702c9f0def", + "d55a55aef96c54b8234b8c52f62e44e93728af52d16af7818b952416007eee1d", + "d56f0479216784c4f31c98511368936d9ec46f53ef36c30779033bf833716bc7", + "d5866744aa5485039257eec813b3c798ae708920b48c50c55f3ac91a7d8b313d", + "d5963fa825935ade10ebeb3a5ab89f7c29b5a89859b6e4d9a05b99f3fe76a6e6", + "d5c42ad099d946dd538d27751c569e77104260c88208ebb17303e037b33c8de2", + "d5c64604cb597fa9233354ad4c6184f50e075e3f608fc4e066e39ce41485133f", + "d5cfbba01fec87614efa807630f78b2d40aa00c3f54ef03d5b3d8313eb964564", + "d5e76dc3ce2665b4abb00a96203625c578cfece64c99dff710b28b6ffa184fe4", + "d5fceb5d50d5e30dee026f099317ac53d4a1641564148f8994f97a7fd3794a1e", + "d604760d8544d6548d357c8fb7f7b56b51452ba1d9d21e3374f8135977800802", + "d61e5c37f01be9e876f3202371ec9eda45c423b52980e16b0f15058ec2bf9e8b", + "d6629213250691aec35c30304183005f2177b5b7930ae4a0264ac795944d6779", + "d66418fe345340cba51a03ec40d1dbfa622275dc1fb4fb4839071b8bad8531ac", + "d6937eadac2b6c3d0375e46e5492df69273ab4945242e12f5628df05457158a0", + "d6db4c02d1721ac2784f2c309b97d099d69a2c73b80f2b53af0e478f6ead5828", + "d6deced34e316b96c5ad0ee3041f14f090a8115b76ff4765e1ef0b1d1c5936c1", + "d70eee734906d8157ca2a4adc428a0279e887ea9df38850873a991d40b363b08", + "d74097f6558874f34dc121b870a042ae2798ef27e12dc8a8ce49fa834817d81e", + "d75e0996adcd6c44d978b2b56b517d28d52b277a00ea557b949d71eb6731c1f9", + "d7812ecddd19c0888a7ddc1b19504b9d13ce3b9f08594e8af96f539716bb12c8", + "d786281abd5086e355ca25841d15069947ee83ff1febee9f91a3a6b985e3529d", + "d7aa736b01c8f04fdf206f169aeaa16b5f11775467f659105030ea3a635f66e3", + "d7c5ef08eb5c2c55507d0554d06da42f076ba5dd8c4cd7ebe95eb9cdd72ecff7", + "d7c808f88a1ee2dd7e0508779b7296f465c97a790a1525c38f27d2956730c8f5", + "d7ceec1687b45dda147c887a9a1747e8b29eed1ab2736335e4f941c40bc36f50", + "d7f0bfacc9d7918ef2c50b51f2760b9399f1a087a00a26d12551f63f6c4f3bcf", + "d8041796f4cbba48d6f3e468f182eea71c719e1b4515082b170adb67fa5640c2", + "d827cdcb1f4a28085009d79166e0766be4ceadbec5654943377567edd6d1a9ab", + "d88522c4f93e5c1e6670d82ea79ca300ffbc0f894ffc9430224000ef782f8fa1", + "d8a1213d2b8a725dad7dd1f6c94e46c5402c708b811d1964ff09ed5640b8e29f", + "d8b9c5e74d8d3e00148b98b8ba7b62cf980c8d0d59a576bc372d6c95ffa25b5a", + "d922ae2fa9fce5b9a8bca376cb078bab270b7d38baf9d2f30e3f9f5740432dc0", + "d98c7ec8189d9bdbd3096d624214e8490e275e1e14130031e146652aeb1de3af", + "d9a8b49b226594f5688d633f92b6dadbdfe670ae73c2ed8b8c6f8f11a74a50b8", + "d9cf4ab91413fcca5d5ebc76de93248aa95deb050b70ba1ef5329cfd9813c918", + "da06ca9f1bd4c1116e446737bc45bc24139610e4d86cc12c49912575f0ad3702", + "da0b0dc816f0a9044f295db801b007b657e1bc9119c0d2973d864f992479dbf0", + "da1c6e9277c0a19b8158e84dd9580a3164df4c2766a9866dbf11776c1196b07b", + "da2aa0974c90a4b92ac0847661095b9ac66d42c72563d0764baa12494a398183", + "da372dcf7884fd6dd5878f012626f38cd2c0a0c00ede4fe1e9b5d1c627f025d7", + "da40f051387657cb3841435ef1eb8de999d0b481a498c56ce69e405499948e52", + "da459a3bd11b2eb7dc03c679865c777fa390bd595961a61937a5994056aaddcf", + "da46003d68d66486d0dfad911aaa7dc805d34d411b78d471ada75cfc3ac403aa", + "da78a8e6bed8c5080da3ee344b34a450c3dc3c8b397ae24a8f6f1073e4000786", + "da97b31bae61c6b47bcbd6df3e428cb672e7ce25e615dba3d3e786113d88b8b5", + "daa90216b4b743a3cf2c9ad5b266a3c73c54207705060a7a9012f0e30317e131", + "daf3c47d33f1186ec51d15b5c6543abf982cc1557a3a6e1744e911a07e164b0d", + "db0e91755ff469e050b088764aff3c0ebbc90339907aa7bb9a5bcfce8c41b185", + "db1ac06f5945913376b6e0738efbb5dece43cad713af671cb01ead48ff4320e1", + "db2b2e354461be6b9fd1d9f65f83f608c82f37dc34b4123fc46a7170b0aa320e", + "db4d2acf1804f056caa6338a0c1121ebe7e872eb623820d2faf28f33033e589b", + "db8af8c419ca5c9e68fefafcc963f225091d9a50843113db6693d9e4bb4e0d2f", + "db9e83e985cacab6d1fdbca4aeae0719315ed7812438c1614b407f86dee3e6c1", + "dba3f95e581a67af2cc56bcd034a08321d9429a6a4a6558e0dc45e8d6e49c4c5", + "dbf89cef1ba7324ab6b407e149ac280eecac0c8b5820bc3d7448391ca59aaf70", + "dc06ce6ab156ba6a1b0200c9b2b3da835da67acd9a2a0d949d2d99d4b4bacf22", + "dc5f6880b7ab6efc2987eb81ea52bd648018d5f2dfaa1793f37640af324f35fe", + "dc9f64fe826159e80dc4a3b3e431c2ebfb89813de20dd3728fef9075dddb7180", + "dcbf7df88c53f2241c6d065d9706914e797856c427a9a7748f1a63714e035b57", + "dcc17cdd0fe848571eb98a651b77b0fca689d676576e396cef0e1e6213309cc6", + "dcd9c20f60c91d021109231b65061a5110cef9adfabc1381bf85aa06f3e4cb7b", + "dce77d78d4180ba19dd613af265e892ce7f35409a0e0d06331d101ab388d5d67", + "dcf195c7d41770f7c6c6a8f9869603f5d971e42f6322417ea4f804c457e858c5", + "dd428ea6b43db62554dc7f571fc9c9f4d735af775c0220921bb872acf95b9a21", + "dd5fb7c5f41c05f16a82beafa5bbf35b793c75a6cbf7c55f9bb6e1d78d5f13c4", + "dd61cece503090b07d75c50d1586fe3b5e7d7244216f79adb88ae8f631c0b5c0", + "dd814e024eef95fbbd404fe77becdd1b85b1cd84a1e6669e45198a45a7581e73", + "ddd1bb658f9157c87bd8b8e37c73e56b5fd70d5776d39e89af71a6b5fa75a5f9", + "dde0cb6bb333a5d275d2a42260faffe1f3c35a21cd71f9e6ce2dc65c5c364dd3", + "de2b5031bb88c517a74c9c7ce89d5663604cad24204589e5f04990665617bb19", + "de3c9542ce9158d9b846bca4c0e919fdaaeedcf781e0f3a532fbbdb0ef4c5225", + "de3d5a79d11855a00ba7a1fcf7b77a1f53fc6e6e64832cd9eafa469d0488c4c0", + "de4e63972c76033624e557a1ebe022edb003eae1549e6c00e897c55e547c34de", + "de5e4e21a02c52d22cbf382a6a968e4a68f0be6d26a80e997135747fbbac0e3b", + "dea0a83b2388d405870ab8d20e6701d12f7770a0ef891d35e8a6c60a24f549db", + "deb7bdd29c4ebab19db23e28bdd715a7e08efc63f1bd6c99937993271d26bec2", + "deea6967477e2d07a943fa1612a56e40712723c4cac464676ba95d03745c5712", + "df22cfdbe476a57a3a5b02c2ce8942fbbdfa2bef3af34e3ef92dd087d7848fb9", + "df32dadc5ea31fbf05bfc17765453c4b752ce748ea5c11f3eaf461cb4c4cf86e", + "df36ee603a248f177b0f7faf15812cc77475edc5e0eb4b3c2b90630d54211fe8", + "df3fdb2f1e2b9712dba895c5dba6d01ecfc53b1dfdd4a63a1db80df373135f3b", + "df762849fc9890946bcc0b5797d4d924e430849dd3c2859dff2c4ba7f009172d", + "df79c31a3880ff4606e683dea00a9f2d9c667074a3e15c7cbf315e51a8d47919", + "df9e0382e541f41c49bbf53abb33be62a5d85dd0901b3364dda0956337788d3a", + "e01fb81986221bb00578cc81023e8ae9d432c72ff0177fdde4057d96bcceb463", + "e04134fc94dbef26800f264b5643d7502849a1f4b1fc6af5ddc23a1a8f7af260", + "e04cd88d4542f2cfd01ba7a165a772991dd8df58dc9876b6a6633cbf5204dd32", + "e053ce301796e47ad5b2eed409cbe269442cbb8169c7c3135f9905161c621b07", + "e0756dca1abef5b526f8ae94b26d938328865bb3d80245bf87f8b4fe291612cf", + "e07f02c55b571705efbd7fd1acc36170409c3da0abbc1324f2f87179e4100de5", + "e0b00d3bedb2f4d54817338493a959e7725e0dadf958f41949ba385709a3b716", + "e0fb21f8466f8b028c0a8ec4ce48a6991ce3f73b80c1a300fabf49ea1f11b5e2", + "e0fd0bab853b2375c1458b581b9c4eb10a892740899b2730f77d5cf452315f50", + "e1116deb7a312c0773f7504fa68643e7642d7e07edc6f76ecf0e7f2317e4eaae", + "e116e702614e7e8dffd26029d472791b380e98b24796b9b0729c1431cb11073b", + "e13b045d58d8eb0173956697314814b0818b090fe63a02b4d383f050bf947302", + "e149e8d8cc8840f38238c52e2e4e095923ba0c1ba192fbd4115df6c612e69238", + "e15cf52cd356a8e4b4c394beab2276b76fddd40d600f4979bf671b8a8ae8d908", + "e165291d79e8169ad9ece304325eab1ff728bf614fd944a40d06ca1213341f43", + "e17bd7410c31d2059a4638b5d8798f1863143f7a099d1b60e03308a8c0065fda", + "e1e001d88d85d034c5aee96eb55f261c5318176bfb9acfdbeaa8c16c5302c6b9", + "e1e228481bbac4845368d96e9979b2e28a4d9c96046d94d4eb75ee8add763555", + "e1eda798f2f368b8857b491ca6731a72e832ea80fd29033e752239e12597c627", + "e1edaacf2d378c94eb4fcfdf03a1f080e3a7f2f5498718719b203f7e3f5b1abb", + "e1ef250815b1426c96147c4072ab677ef792e1109910427d9cced32834e50a4a", + "e20c328c0bad99a61bd6801575f9d5f9e8e1f31f4d6567cb9ab4ed4cff110771", + "e21d30dd0a17f4e3a48c59368fcc31959cdd1081319e280121522a0cffeb886f", + "e231c038fed93023890dd0e30ba99e3d0bd4f43d22f14c82756c138840a31774", + "e2422f3fc47eed98843f471f11ee45a4d5017d6ba68f5977ab3108eb2cda62a0", + "e249962c602b987e4520b848a2433337a2615e537a00dca6df1ba86146614d09", + "e299be1a347f7ee34b86aa0acd5d1251813ceda172c868b32c74c927067b830c", + "e2a8a71bf005884953ba2c79222dce333654a560e088e9cf38dac85680b1f735", + "e2b5b8cd854bdecc517cbd0a199160bebb1e56907278546f752c1d5a0e93a57c", + "e2b8f68c5ba84769f8c704bf4157ccd3e99b01b121dd3a4e8cf3a6792b8d6d39", + "e2c05ca53adfa0f9b3e5c432b2d6aa14c7825cf879d6befc9b384a02a1128624", + "e2cce80dae5af088e4d5a2f938be265a32d40a5aa29e216adc94c6af85769342", + "e2d6766f2fc6adec1f494b3c97ada10bc6b3dd89fffaa6b9c7774c0140e702e8", + "e305c6624bc5303875d34fbb385598ae3418b4f5036fe9763d6c5ba5b3261fa2", + "e328ec3ce32b204b4b8d6bf0f63bb2937b8070ab8158d792b746a4cf5bef0112", + "e3290eebc099a02b3bda9c30ab81e7a91f11b4635d3d68ba488795fceb7292ee", + "e354482685966bcfaa5ccd17d666dec791abb29017bed1cff35c1f3b48427196", + "e371f2ecf7e74a3b1a21a0d2dbd76ff061ede68511f0e504ca9f3b0562f666ff", + "e38810c68a7a67fdc05d198102f932ec2302ed507f9268ba85bbf84940be2bb5", + "e395caeba922f8b90effac45dc8e59be8078bce39141d8faf98ac3b4a2be832d", + "e3ad990bf0964b8d707c91fa546956dae31678ed77f2bdb827b9cc7659ce9718", + "e3bc95a739f1260d432bccbf2bb6e8e588e957d1128bca21b8175ad944922069", + "e3ca4bb6e4d1ee23cc72e89ad926308195c8cb17ab4fa4e240ccba8b99bde1e0", + "e3e53335a806f53b0df5e5632c0a5ce39d062736641aa05d81be66bece91cdee", + "e406fde4c19b80fa9ab6dd1a9dc4482b579dda8e387196c8218a5e2d36cf767f", + "e45054295a867fd9f601fb78e919ee099ae8bd87be5bd91f9055d9b5c95ba1c7", + "e4833559ec5fce3dee483ba72c851d1ead07e7a693163e5ec4d14afd7d3e4ada", + "e49d798fd16d767825c4ad445843bc037f30c731303cf07626052611103e69dc", + "e4afd66f886e9c1db0d2fc427c08f765d3f5ea8a57a76c1f3f675dcd1693b797", + "e4b88f8bc22ac0755f21e7e57a5daa1242c722dd27e5d88218a71fcd377044ea", + "e4bc88414306bf55b62cb428f40f2eb464a9b27b3d65dbe712f85ead7604bfcb", + "e4cbf1b81022ebb6a3f4ca228bd6e1d533e051087cb3be4dae37a6b335b64b27", + "e4e28808ae8dfd46216a2fb9252d8f9afafa858d11cf775076b653c004cb32db", + "e4e2bb9879bfece542a719a3d80a69b09404c4de19ea69c7c27e2d2f73c6ce92", + "e4f68bc2988442906fb3146a0c313b75bf7d98c78b03fbba3449eba222b4e2ff", + "e4fbaaedb205db953e8d24af46d3937040d565da17dd3c1a6358598ec4065910", + "e5154a7d17607655f0bdfe3160a78ccf8e0ba51ea39acae0376350886f8e1845", + "e55299d0bc052cb970f3e49f110d9d50c45c8c5aac7b7ded84020228bf1ede03", + "e556d85a21c5f358d7d57d3c4126a0476dd687aed40bfc633440a5f9114c0a62", + "e5e8e61a6acf4409b3b546bc34c4c5c3df6f8230d7e4ccfc33a2679da4ba8940", + "e608c7c92c378f3a308d3f279f0eaa97c8fb7e7d8dd97ea445b1c690362a3a38", + "e60b254147e2585fbbf72d47ea2954b9a673d8717c7d3e910bc513246e5089c0", + "e61fac6a1d5695f0b9010ab423fa50e035f5a29cc33ad776963371df7620b0b9", + "e6a517e9dfa6af4da7ac8a6d5ec19af46219981a80f523afa4e93b8a807c69e6", + "e6c72ed82e109f3797ee8861b619ab6088818c184d9ad5e96af3cb2381b7e95e", + "e6d6496eb3909c504ba87b96fb20e29c2a2cb1d6d15222570dacbdb4d6e0b239", + "e6daf10fc8aacdfb63cebf93b94bec2053e8fea782c7e9e02034387db2cab032", + "e6f45d4600f715028e9eefe9ca9ebeb9e34923defa01abcb5ebe3a997d8f077a", + "e758b129c07dbf4795ff3c35ddfb08b94af172a15cd90c21805ddd3269795d7c", + "e75ca95dce785b8977f2b24392174c21ce5b772035b87b24fd8f7df2894e8b04", + "e7669041b406fe2fb9605a2dea02b57ed708712921edaf3049e30c15c0e49870", + "e769fd9bd29296b03b3bfa821ce720a82c7fad4f6c3545defa203fb70f3f5890", + "e778fc043354b52215651eee4314565d3b8a52b3060f76014c43e768a0723ad5", + "e77fa12e38c90a278fb7af1465517449accfa94a16aed341a3c9f7c8a19d2acf", + "e7b9e659a56a75a9cc66abaf2878026f9ec9ea7b6753b744fadc05e8b4fe2e49", + "e807a1dae7d3b43b44646da39c2f93eac68f75c5d408bb4c7df269b2c31bfa6b", + "e809ef944112bcb401a996b7c48e9c60d7c951635b878b53a9a4e8e7198d5771", + "e8210d2b3a53f28d90cd4c0d5d563c4974afb154e58a950de60a6ce1e94b98a0", + "e83f1bc3ef5cdb9898f71848e2455310e173d9e625f6f62622ee9ca1f067d6cb", + "e8a3e432bb9a9a7a777dda70ead4e914e3852169e92cc1213435d2800a4b0d15", + "e8ab701f9c746b2050930d13cd14c1a4da09470e84a4af00182b240e0190f2c2", + "e8ddd4a423f53d9e73e89994910905d4177eba6ba48a22b6e94fb46c2eedbdc4", + "e8e065640856dc967ec7df4f09627d3df869baf4768b8b9f52d5455aa3824b98", + "e8e555f7ad29c9b6cf48afe7e613a9e7326b0ed49d8c4fa98574dce237cc8bd6", + "e8f7b249bfc5fbf13cf71c70f52c4cfe7ba312f9b2eda18ed1b85f95947dcf0b", + "e931fc764175ccbc37d92970ca353c3ef516278cde0fd8302140552eab5eee68", + "e935cfce5873192c80a1489d0905c0ec0a2c93bf66fddb4525e4ba1a261bad2e", + "e94950b99f9e3a0dccef7b2aad372368c1d0aa60ed9d46f44440f91152bfa9d8", + "e9c3fced5b9ee5c1b0564c4b0e97bdf8036c915928d6d71306c6a8153a3dbf52", + "e9f20a5843a9b56c9f60c76fe1a4a20502d3ac52f6a79e6bb0fddfe57c734f4b", + "e9f7e71b5ac971ee7edd8954535c7bb37a9b4423bf2b4c334a4d837a719d6940", + "ea5df06840f024216c7060de9e0e27e831a22648e6cc40f3cb3c39f9ca632790", + "ea677fca7f169e12f3c208c112812333ff2c7a322418d181c9a173155006659c", + "ea73e0364115fda77575b697cebee340ad79f5d70bb0c3e1a84fcf9366041a13", + "ea8c900045b92ce2c1fcfbde002b9cd3fba951e0930ffa5506af460aa540aad3", + "ea9fb8c996a5e0d3355b766f1236bcd7a1fa9071da553f229d72821e270d5241", + "eaa4e3b998e89bd22997baed140e1377accf6b6bae9a9d1e86054d583885d929", + "eada10e544c804df305e002ae188e023f385ec4ec1aad6532c21a721981d28c3", + "eadf398b6f49fcd70b7893a706e3ca3cec3931e2f79f1bf0478725d835df8094", + "eb2184aec299ddbafd802d22f3c13fd5f78cc0b39aa17d20217480ea1b539150", + "eb21f51a46305dbcfafcae086fbd51bbd1bba56b0c8cea89d67ea33f61e8e540", + "eb49069e0d0803d055f3d2498d02c39396653693a7bad1d32dc458e3118a2218", + "eb715472371cabefba3465b6b77fbff6aa0fc2b4f00c8fe7e55da7cdb1e57c16", + "eb829ab015a5470d658bfc7cf8d8a4f9ed7b0649fc04477878c2c5bb086ac465", + "eb919932d5bcb710486119df62f73e823b2662eb21355d6a3229976093c579e3", + "ebf1af91b52cd8ca4efe45784949e3cc2e8f246d7f4360202ae6f3749a725c3b", + "ebfc8c6d358667cd32e57e33ffbf9aa00db477c85e0e78f25cd2889ffe8a0fb2", + "ec128b250d230602ce125413a544982ece6e917ebe41620a35b3c2da7e64751f", + "ec248ceff6e47b10f0d67f19204dde463c9a1408cc04ba680abb10619a4edd29", + "ec34aae96b40eb7e90848804ed9ceccdb00855bf550a25ea89d1e9387475d256", + "ec3ff85db7bc96ce057455dac92b0d1e8b1a53813902595a4ecfb7eb248527e4", + "eca600fa3e3f7716c80a84047650bb56523b5eef746421cfd879f7de4e393677", + "ecd38846f9a886c1c8ced9a9bccb141c92951cf2c1a0762d75b46a4e27ac1979", + "ecfeccdc4944cd60927341295292880891b0047b9767a3067c38a6a88fc892e4", + "ed3411e5dbd97935027bab83e0de5f1844b0203fdb471007197c8fd12abcbb75", + "eda0be2d9847f3bed6e1f0897845672968add857c9059f415357dbd6a513948c", + "eda11580ce1dcbae0093fc69bea154e1b5e2c429c40a29a07cb6fa005413095f", + "edcf7090faa657a30800ba1effb6f5336ce6c6da7d67f34fdd7bdef9ae1799cb", + "edd3144bce78365312149662b177843b7be7ebc94852e60db48883321cd1c28d", + "edd8157eaf2fabdfa0228ccf1bbdf8da438b5c85aad7e5e2b16415e03225fc9c", + "ede58af4da98df112c7e8d3e3bf12d1dc95cd6a2c2133c5eeb854eeb92549e48", + "ede62eb7c52421c015c5c55624a6f4eeb50adf742ced0cb0e8103793174b4336", + "edea1d001bd33c4521b4976f8c90c447ceb5f39377994d58cbe80b20760855a1", + "edeefba63f3b8bdd4193ebd0f4bde202512ec7186db9680f83ea3503d2bf2221", + "edf05e34b8627e131cc9b0011ee2814775b3ef1f15e7df2a994799b77349e766", + "edf49d0ccc646cd8e65b71c90acbe0577f1e70a44488387d10d5af0ff0ec4ac1", + "edfe570ddea7c44dcf50e5d209f1d0858ac2f6d1f5f4314a47b9291c1e7761eb", + "ee27d30b6be2213a9955755bf53c689e22a1d36d6d937c6e5ce30f033baf1ff9", + "ee2eba2147ebe6e0be37840a004b67948c629fdba8fdfa4717838f8f14fe6655", + "ee58e5f3069e7a722ea63d9f4adc2a70363f44578c99b653e9242f51966d3557", + "ee67b0345ba3a7220dc499ea6cd43247969026767eca25819111f4437406e5f5", + "ee81040064f573d55bc892a32c0a6a9633ecb54f5e92dbf3e5250168a9ff7f8e", + "ee9ebd625b437d3e95188418939dc28c7bf4ba90a218a9ad07adda258499360d", + "eeabdf9f8b0db1bceb0bae0890b5fdfb9326eedbc40dcb85c7f26c6e14f3d37c", + "eed3e97400c4186ba00d90ef1ef61a2add556118dc8721a9d0cf4fd1677821ab", + "eee22c4fc66a21e9250f8e48c48b4b20312e15ad4dc92eda8fd1a8e84ec3b28d", + "eef0e5551b49791fa467c9827c9b6e3bf48fb58d15608a863f3b54531d6e7c2d", + "ef014ca51bfe5dce1a5cadcb691a09e845e1f189e197edcb1b057577e7db2b09", + "ef21ffe1a20987ae754772d981827616c2430fb2ee65fc446dc12b890ac44691", + "ef5bc23227ee57f961995d4db74011fa4e4f67326381286a17660c10f2bdaf2a", + "ef5e8925c07e682028717a42aaf87e36955876bcf1b2cf90c11c9f31047dc205", + "ef650facfe06596e2790017a7de7284ae386555ce70df6ac6792510b0c7782d0", + "ef651f38f75bb4454622f634118195c8fa2f380a02c85e542d0d5e8d40a312d3", + "ef98d4518adc923a900d7f16d0eee156539b83cf1eed9b777c661648296c8ac5", + "efa37ee575b79f5cfe55a72d9df54837d8ad4596af2cd6c9be85fcc147f607ee", + "efc630fb564e71a9a422d6fa7b95119128cb8e883fc59f15dbecfe36e9ecaa44", + "efdc2d00f2661a9477844c248a90bf38774d9ac165b55caa12814e13ffda9f6b", + "efe84d821bfa612112cbb36d005691aaf4429ed7d43863abd6028e2628aed9e8", + "efed2f5dff554dffd1f0016ade25144b7bfc1b05491f22dfd977473d08d90243", + "f002dc29bf45877bccf6944d2b2f85ca3bb5e62003414b4dd8d0da1a8f0239f0", + "f039cb6e4f3b43eb5fbd395eb0922df5deff0be36ebe5339171e5fda1a175bc0", + "f064a324d8450b5bf59a974ff9ca07478be793974aadacac4ba283d522ce1031", + "f0653049808872abb6486f7a0237281fea4cdbf633c2b9b7377b97c036c5b636", + "f0878d7d54e80a2f4cb870d77044819820cec0639b0c3ed55d68060e02aa2534", + "f0c51e190f9e1d34a145c340fcd2f3702770119d1504da2e90ee8cf61ebc98ce", + "f0dbdfdffab87118e1549236292638f7846f0346115debc135c9f7ab76481624", + "f0f6100f7f043d80e2d8ce1d779db6b497b549634415cf66ddb07b5cfc32a9ba", + "f0f910c3e7bdc180f5d4bdcf9969badcd6f9ce1ad0021aaac14c0e6268b93593", + "f113925bba3c99674527e3a1674af50ccc53272dfa59bb7e3ac95713e0263cc9", + "f11a0117829cfa3cba8005d6f7853eb1b972bd0c946a92175a4e94725e03721e", + "f11fa2ba5323f3df29ad79ad699aa5f4a6de4b5a44843d15393588dec3aecd42", + "f12815e47d204a27bd5b1f5f7d006bbb61f7bc8ebc81a5a238e3ca323bf50fe9", + "f140b3f6e23c614a37a424499d2195d9390aebede5e7d7249528f8c97ac4f8d4", + "f141613c3556bc991572da8b83f7e173bc25c96339e883c0419ef6909a1b4c9a", + "f176f0da7b059ac9e4eaf82d45200a273674b1a3c7762d988c8b45f72a9f5c9f", + "f18f388ab015171d6aab7c42b2d0b7460f2051e823cc7209f11a2a158eb9e69f", + "f1e8698545df73640275657660f6d64d39e679ad11a8e51db2beff9efd7d575d", + "f201a6a126befab7f6ecc164e97bd97e7fb96d41c449d5afed4e70bc38dca2ef", + "f219e244f6d65c5d2d342f118c7f3b86f8d17e9fefb358dc2bac5517a93f4403", + "f232edac8808e595c7660ed3940acc5757889c36fc23cbaf94c0256c0ca0e4de", + "f235b5c48056676e8b72a6ab386dffb67c99cd2e777a366a7bf2271009947a0f", + "f275219cb79a11c529c59fff67cbe509a3b4b494baa631231dc2ebdc434cd4e7", + "f297043452426d4badda87fe216cfaf3218081193df564a8e509b1e88436183a", + "f2eb09aa74a3916a82b93cbb45e64b6987b4b0db2bb41508039da083373192a4", + "f3202ff50ba57b89a223afda873dc154246a315d02cabd86d92e79aac87b43ee", + "f36947f97e0c076049cf7bbfc4b1f9e29a44ccd257ba819c8a4390ccd8c104f4", + "f36fa28a0d82ff6459afb0cc4c214a3f00a2f2c05ce33a7bac83708b92e8f790", + "f370cfc1ec34d5682a22d438f262a6466af86ebb6075f28a95852a8a9e1368fe", + "f3bf79df83a9498bef664e804ba1ac9abb0540d1aae420d363aca076e898e2c5", + "f3c7e653d4b7b48da3d6827590d2571ec55a0e2fc93089bbf0729b5118932cf5", + "f3e69d248ea724fffe205c18afedb24684d9e9732f7123bb52cb080c0813276a", + "f424781bd9b11fb67fcdee09cb5e383ad7eb8e73c9549f7cb2da216a5e597c38", + "f451ebfabdb362efd297cbce185efb7a1fbcbcb0ee813e81eb06cdc8ab9241e9", + "f4607d690baf013d7a802204f90eff95d514e819f345a9ca28a1ce98f372d49b", + "f46093cd26aaa07510e0b7bb6ea87df4989f67d720ede606e1ab1efe80d63d76", + "f4af0630de6af54e875ca3b5d161441a92c31a3d2755c4679296ec56ed1a4208", + "f4d75bd491dfcf2f2794dd3ade4da77c7f9d4ae201ba391c79c3885bd2a3a032", + "f4d97df423579ef674f017107340bb087391536562fd7949514ec8a0d6f3457e", + "f510d227beb77e05cc85eeb4cbaf7a037ab4696ddd28337cb3d066a514e20344", + "f5811d175868d36ea16bbf4edfaa39742b67911bc911f6407c853d4a60c2004b", + "f5a72bbabf09277fb14b7d4c26a8ad14fb18e35b770b84c63adff036b1130211", + "f5a9086e74abd4849d1138c889e6363e1ad753a23eea3b336c9b5098dcbbbbf0", + "f607d317a2120ccef46ceca3f4b01107e96dfafbf7e59b9eaac61518a9384894", + "f648007ae7f6484e0bbbb8540f09dd3799d97892127e3719cf24202000f58eed", + "f657f2650b495c498d64853a352dbddc540ef5fbf88c00612eb41fbff9b443c0", + "f687ac299deaec19007e4a2e760d551d30b34298db875526723ccc4fb29c4543", + "f6919a9c489ddc3392482baf57b77b9d6dcf57701f91d52f39622b411357b59c", + "f700762310adf887ff5ccb9d8345a9b8c70fb579f323066c2a784120612038a3", + "f729f479bac37a44de358818074337e1df23ac866b68c37ce94bad896108f756", + "f74f1b83529ecd86b42df89fb3db019bd37dbdfc0cebc44c1a8295617836aacb", + "f7601fe05dbc775a3669d8cb7056b902646da54214fb50515b5e46c2409e2c6e", + "f7691fdfdb3293b4544fc7a352f0f0bb36f9dc5f9f7be92351ec0713b440fba3", + "f7702d266606741d021f0a71da9cf4742074f901e150b84f94d792303db58e6f", + "f785b6dd3ff60c5fe59dd1c191c094844203febfd64e047f37c5d77173f3f66d", + "f7ee42dc5ffe55dbbe926bfb15e73fc7856a879741e98cc2dde8d0d93d5c3952", + "f7f4125d2f2282786b18a12631ecfdc43b84d6831f40e667c2e29307177a24df", + "f80da66a27af19b2789125c6b3f6d7c1761be79cf443343933c1e9dcf40fc996", + "f80fca2b74648131a8a080a1e8ab8a6f7d611bcbfb2c96206e11f4c6ccedcb35", + "f82c04cf08a5d7ffbc2052d4f95bed1564bfb469e14b3d1abebd310baa7a318f", + "f847a0c8222272968b74e8818e2021323d0ceb7e2e2a59f40e6ce7259b249359", + "f85696718a674949094404ab65331bcf680c5db6db8bbce2d0ca66cb795c2937", + "f87e78a9e384348c22fd07fef02a503472337a3cbed6fe3b0592793fda93d178", + "f881585742358979d9c7078d9838a7cbab4cf29046bd22840a529cf9a7687a10", + "f896d7cc6906c2c999a594df63f5db5abca03fd217c6d22b4c083c8f1a847dd8", + "f8bd539030c47ef4ff3a14d114494cdb38bab3c1e3bdc3265b293fa311d6370d", + "f8d90aaeeabd0645344faedbe4a1f9b051f2bb897fc5fb001e02a6de84f45a3e", + "f9323ea2196bfa6deea220e7416a5fffa89909695221079f04b0259bea505bca", + "f95f0181a9ef304fcf345420a6136c5dd128dedf7d29ea48d056f24cdb990061", + "f987345695896ce10e780add318656a40f3036d2dfa95daf70776c2d89909d70", + "f9c47c28d51431a91e70491c1ca8cab97e592279a94f9607368228549ad57936", + "fa07c7f20c27c4bf9145d52efbe65dbec7fe7e8e0a74543437b056b2fa40b98e", + "fa1b71dfc12a4f385623fe52b9f2454dd162217414070904c282e74116447700", + "fa31e969c67e0e4953325d3a91af4672b1f3d68ee721c1f819f040838966c625", + "fa39b77c7e175c3622da909e0c48dc141bc2421906f477bcdf99ba3ca389d679", + "fa47b56854afae0bb9ae5678b80137f64bbfa32e3a177583c97dde0caae50788", + "fa8721a6a99ad2ee7c2e31f26f5a28a03aa2e6cedb9f099c81b1947da8c079de", + "fab9244a98d17007ca287be76d375653d42376c8a40c9837d697b5a9d54442e5", + "fae85d58a94ee8ac87350af04247eb429dc13eba424d783babdb3b40962959fb", + "faf20ae2f6ea21bc12fcba0fed95fc2d2c66fa525eccb574b82d2e775d33e72c", + "fb0104a9ad8c9302afa9844b799df77114a1f599ae73fb70ca182c18657746d6", + "fb06e5d2ecd8c36cadbee98f013d650a6c07bb2040955789da0d930f95d97e87", + "fb6c478eee898b673291ae1ccaec9d0c1e3118308994e6e9ed615548ea682490", + "fb7a06547c76adb0113a91ae5e50d2de4241269eb736261a6943d7cf576ec5ef", + "fb8086cbe71654a8393479e76bc7865b3813caf6dfeb169c9b65234afdf316ea", + "fb8db1967a1bbbdcde29dbce41454d1cb692aa293ac147ebbc0ce88f94357194", + "fbae6636795b7b141ee220b0c648196fbf00d711eb5b0730b7981fcc79d98e0c", + "fbddd6b3a7a38eb467e4dcd74d53870097e4244fe2869fecfca9e44780db9ab1", + "fbe4c736b1b9641c14842d2f9c2d6a4a10e6838033b53bf5bbabfab58c8932d1", + "fbe89b5c637a113631eb2ebfc096d50b3df29a2732f57a74aa8b237b547a04f6", + "fc288e5b6716568bfc9909a0e3645678a20e33954d8cf3222dbbee1033550e88", + "fc32e8d507aa88822efa6300d76556838e30a182d9b8084a59c05ad83a83a15f", + "fc437094c0ca48af5d90e3736f836efc88679b90be80bfbcf782d4459d7b089d", + "fc501b523438dab044b23b812eda04e81412320fabdf2c7dbdc8830909c2d660", + "fc66d4b1c09bd0e559c579f228d3c38a078e45e9d1305d4cfff09a47329a0bbb", + "fc86885e0b1a0de5f04adcfaf51758add43ef7ff78fe43cc43a50cf664176586", + "fccd668b17136cdbb99e09013df6c6dc0a5a91f1eb1bf22f989a21cefa20ae82", + "fcd1b5d1a225fba957d02e346ad9878be14a6204b601bc7f2d3e9a6b06216294", + "fce089f68c0ae00d5da1e754906c73f1c3dda9140a37617dd6eb83c4b2c2ba48", + "fce84ffad30fafc2e31b69dcd740f4d6d675a37c7639473101c780d5061b1a03", + "fd17ec469ece96894cbad6f7c7e391c4a4c110283bcce36d923414a824f835ac", + "fd386d3ba151f051475e843a9e93e392fc7cd8f3193b21c2a78870ef190a8629", + "fd56a8f2722ba03958b443a558c4e13465f2984cd852cdcf87dab011f33b24a8", + "fdaa56189cd67b46adeae1059d510dbfb12edbcdc518bfa2e400140fe3b09bcf", + "fe19349e611215277bbee9b125024dec258002ca7dfcd3a7d6e2b5935dcfed50", + "fe196574c75b0c689806a6ec1f4ea29cb90df6513c167c4c8f168a3cc4af9676", + "fe1c600d5df592d4c32d16ff5e494006f5c98b172fc2b4f6e1697a713ff561ee", + "fe2b7d4e8a2279775c1366b1447b7c2ca7b9c78f80250778eeee02afe18fb26a", + "fe35b9a5ab367c6ce9b369d900c2e296de0d65106a374f90da0fb7339e6a1f8e", + "fe605b386e9ad2f29da2d05a1ecbeeddca2f4e96e997dc7f936f2da629381fc3", + "fe79db37640f9939ca2efc575b9ea6f0ada86bcfc2fa03c6d33decbd4adea4c6", + "fe8a0a9d5dd6d92788187afa795eaac740e028d09f80a76682e4f0c33a5f136b", + "fe94f8c3fc0116c93a617302afdda03a1964b143983d60e257509a690fedca37", + "febc1c3d860db8a9fbe1fee5d8c1f502d19797294cac207b19f790661312b229", + "fee93db0be8b667fcad98db04b454a25912e56cf54f3fb8637fdbc0eabdcf2a6", + "feeb03e7148982cbe914b41c73f9e6f399ccdfccdac69269fb3a92d214b253c2", + "fef7239ff6ce720d7a669ff227efa115fdeec7720a1dd0c35e5b1fdc58e9bd6a", + "ff07c472654ffc93b1e5cc2d73137b9c00da43f91f81c163d8f1ce6e3549ce46", + "ff1b6d0ed5dc56d68521316910c4d5f1e9e4c149199a4fc4ea61f27bfc7292d0", + "ff25cc50d0ec9a6667cb3562ea5a4db8311bafb7b6da50189ef2b88e8f792eed", + "ff3206f9c2acccdd3717c968f3a263cab72d9bd0ae23a9d3457d0129e01f6af9", + "ff6826aba61a52110343bc8400371d982003dfba256edac36ebd3559a2b924b0", + "ff79f3a026549cc2971adfd50a9b5308ea8e872e0cd3b1f88f9923d18d9ef983", + "ff7b6aaafe05fb6bfae3c4b5d4d2085bc210bf47117d02702d8b3aeabb31f0aa", + "ff9132bfcc7b97ca2285f63b6b98c9e5ca470341af195fa9ab9b33e5597cfd77", + "ffb1eb364e359c1ea440fd1aaae3ace5524bcb3e61da03b813c249d7f77de6f7", + "ffd5b2e7c3e69af9156ecbf8478ef0d2911bb1ceb72ff6c5dc2207e71ce09a82", + "ffde31f75b901ba39b1a965ac5540655b4a238fbff179c2e51a2ead30aada872", + "fff33bb40f0d5856f295bd86e969edec5147c0aac5672ff4f959b4dfad835dfd" + ], + "task_counts": { + "LiftPegUpright-v1": 5664, + "PickCube-v1": 11264, + "PullCube-v1": 4240, + "PushCube-v1": 5872, + "StackCube-v1": 5664 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_split_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..26988e766c6751325c7f72d33b199bd133350cde --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/source_split_hash.txt @@ -0,0 +1 @@ +867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/split_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8533e80f235606cef4ba0119a3ac3806d63a0983 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/split_hash.txt @@ -0,0 +1 @@ +9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/table.tex b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..94565401b483aab7b2a0aba694119f99763f1852 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 69 & 0.2754 & 0.1884 & 0.4058 \\ +base\_utility & 69 & 0.7181 & 0.5449 & 0.9239 \\ +branch\_car\_at\_8 & 69 & 0.3633 & 0.2227 & 0.5187 \\ +hidden\_chart\_oracle\_success\_at\_8 & 69 & 0.6667 & 0.5507 & 0.7826 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 69 & 1.4000 & 1.1926 & 1.6081 \\ +outcome\_ptr\_at\_8 & 69 & 0.5362 & 0.4058 & 0.6377 \\ +pairwise\_causal\_calibration\_ece & 69 & 0.1740 & 0.1486 & 0.2041 \\ +proposal\_oracle\_success\_at\_8 & 69 & 0.3768 & 0.2754 & 0.4783 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 69 & 0.1014 & -0.0580 & 0.2174 \\ +proposal\_oracle\_utility\_at\_8 & 69 & 0.8806 & 0.6817 & 1.0719 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 69 & 0.1625 & -0.1193 & 0.3720 \\ +selected\_success\_at\_8 & 69 & 0.1884 & 0.1014 & 0.3043 \\ +selected\_success\_gain\_over\_base\_at\_8 & 69 & -0.0870 & -0.2174 & 0.0145 \\ +selected\_utility\_at\_8 & 69 & 0.5173 & 0.3420 & 0.7216 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 69 & -0.2008 & -0.4196 & -0.0211 \\ +selector\_regret\_at\_8 & 69 & 0.3633 & 0.2227 & 0.5187 \\ +success\_selector\_gap\_at\_8 & 69 & 0.1884 & 0.1014 & 0.2754 \\ +success\_support\_gap\_at\_8 & 69 & 0.2899 & 0.1739 & 0.3768 \\ +success\_total\_car\_to\_hidden\_at\_8 & 69 & 0.4783 & 0.3333 & 0.5942 \\ +support\_gap\_at\_8 & 69 & 0.5194 & 0.3502 & 0.6672 \\ +total\_car\_to\_hidden\_at\_8 & 69 & 0.8827 & 0.6654 & 1.0562 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/target_index.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/target_index.json new file mode 100644 index 0000000000000000000000000000000000000000..bf98fd3b10c4105d20658db53fde359eba827cfc --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/target_index.json @@ -0,0 +1,929 @@ +{ + "audience": "evaluator_only", + "base_candidate_types": [ + "policy", + "policy_residual", + "anchor", + "base", + "expert" + ], + "candidate_type_counts": { + "expert": 419, + "near_miss": 838, + "no_op": 419, + "random_negative": 4190, + "wrong_direction": 419, + "wrong_gripper": 419 + }, + "content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "dataset": "/scratch/knguy52/dovla/experiments/six_task_h16_collection", + "deployment_candidate_excludes_expert": true, + "deployment_clean": false, + "embedding_content_hash": "e4374b56912fdb6814eb0320518733d1602a90e0dd2f9268f65c8a148751f963", + "epsilon": 0.05, + "format": "cil_charts_npz", + "group_hashes": [ + "008d30588ad015db473bad0a71c13039cb1d7725fd6345f85f61b4f135a5914f", + "010f7c7c4e75115a818b44fcca59b1b036ccd9e3bef5c336a53008f8a7b0c90c", + "01349bd14e96a28b27485fa7b4152088fee51e14509318f34e89cccdcd369c86", + "026a5501ef47551172cabd8769a56ef391320107d1c27757256c6804a0bc52d2", + "029379cca8b2cc44b70649722a5e1a91cb6e609ca2ca13f242e01ba12bfca76d", + "029627d900d662439e83a0985be181083aee1388afd0887f6e0fbe802d8d33f0", + "047b5299a16717c2aea07ceeb278229f0b4993fd8fac84c7c7added14a47ee77", + "04d0793c098cc4e382d401ded0ea3d56c9677d2ad834b5901da53ccda9e7d606", + "05989b6549b3ff8e8e6c76d5ac972447f33600ec31b946a59777fd915ee136f6", + "059f2165f0f92f81ec8c873a978423ce310464a3cc27b1b08eb88fa7d5a5b3ed", + "05ec227fb1c3e112625c754c648e95e326449767ace3e1602902db3ad71e378e", + "05f2d0050690209c8abcbbf38499e0e195f5fe9078d5964a2fa4d351dc340a4f", + "072314f777216049af237bb018df2e562ad21accca2928fc4a1728c4c0a70dca", + "0746e4881ae94e6843c1d7e85f889ac5196b2a58ca2f1cf5e2c731735ccd70f8", + "086e7180f0fc674f6b011339a3ab06feb8a4d914744385734c4af6938d63ff60", + "08bce62400dec82cb7f3de6cfdd7e831fc3733bb7d920e8a13144702403468d2", + "0a68ad44f481250311d0f54d46938b1fd557893af5d9efca367ded1c61e3c2e8", + "0b50e4bf134f6df63b79deb0d5c1e8428d36b52ac3b97260c5d2d3f3286c025d", + "0b5309f59e3b116aaafffd79932bf558afe148cfa503ee100797c472adbb66dc", + "0b677f82a30861323128c400c43186f364d115c3646f77e0b5b6fbd9aba93af8", + "0b7b725b4ba7064a7598c6b283e2bea2f81f95b08e5390df39bdca14cf174397", + "0be3d37ad2ea7b9c4768dc6be2b505a8569caafe7e3c850bfc5f466cb00ac715", + "0cd039b28bc0a289515b0552105237c97f010ea8f52f1d79d15ac23b2c5f51b5", + "0d02743f28fb0fc84f8aa9b502d34a15c82714c2afb9c8ea8a0e2731a134cba8", + "0d5ad7cb6f339dd2b38a88d486f273f01a85efad9cf16205236872fce6299e6e", + "0f0804f0ceaa4f105cabcbe15bcf37c28c8b2deb76adbc77d10f106b31d1a19d", + "0f64d8c709a8f757bc9a972ac48f24f5f9f618bc9dc10c00bdced3ea5995cffe", + "0ff07d6f8629a231fac814ae3f8a9d18d9f2d5a5d3144b16a20ae7d1aa202cf4", + "115cc5a336dcfd42aaf2e70e91399dcaea0c67511c2d2776f0583d533f3ab1a2", + "1214e59fe6c7a0a2c552d75d76556b94ed956f9483f43c23c9c918ef74bed5c4", + "13bdb78c493f25aaa60ae00be517b8d81a019edd3fed9db1d110730c682cf077", + "152c1154221df1c61379262f4431935a6643a8b056939e6067a24277dbd95144", + "1736a3e41589f068c5583aa47361c5a65a38c756d7c83dd2b291e533f8ccf29b", + "185899d72412376f441583214f89c8b4f5919141ecc7d26040d421ce38aa827a", + "1859b39dcf1792ba35aae7f6525f1c5bebabaa6cf96764b6d91efd065b7c5b72", + "19164061e9741ae8144c089d22914c6687147fb1eb9769ac6138ef3ff3aa3bd6", + "1985928f34b5eb98e58fc7b2e0ba76f219326d56aae7e44d5e05b7c0e37a6a47", + "1a6b5ec8a4bba5d6cc47c74786b709c3f7fcc745cecf70a73dab8b5e1196081a", + "1af3466db9afc8d2081db54c173835b8ca6499309fe4f3f3ab21c9e73e73c360", + "1bd01f63cd4031a48713788ec00bd2f31261029118e648e055cf33c285d3ed15", + "1c59c34bb7ec1e57a6c4cb6b99b155a268a5905621800c9d03e32626d2aaedc3", + "1c84f9a3ba87babf7017b4310f0995d45d5e2a6ec3856e30afa5792152a078df", + "1e0dd390e3e7b388f54330562460b0c813e82ad862d26c66b42c6c3d2e46e2bb", + "1f7225e5a9e00d9ab87b51faeb0874623ac983f56dcdf4a855d0da29f861cbda", + "1fa325165da53abe38062a07aba9d719b26a7e890f8663f320ba2eae96072d14", + "21707fe0167d5227f7f416747410e40ae3f78c6c2b825361ff3f3091d56f42f2", + "21b0c38799dbd91817654ead09968002f251beb305f2a0dd488e70665cb691e0", + "222b477e56cdc6dd529442698720ed23b2fb6089d0a934ed3cb97104669ad1be", + "227321345173cebc9198ba533e27dc7f68bfd22fe075783a60ec7c32a37ed68d", + "240fa7babba75db5977aa68dce1f0d77015a455839915d71ebc76b252acf7399", + "24bf7cb1b06503ef7d2d0d33b11d92f096a2d9f043ec3454dd24190c2f447410", + "26ac77dfd2a9ee1f8346cc862a4c76e04a552d4bf09a95dba0fd72e86c22d544", + "26faf0b54827c513fd7411133f3d655c69be1762be7262b7c95a59a53c8e5376", + "28ffdf18cab5b40f176891bfad5f86b67c78fe0a164a773f0ea4c9e4eb3e2e89", + "2974927c1ec6cdb00ca053ab7064ccfc1528d35084141b0b55baf42bd1679e94", + "29c243497d2fe43220d59c5689f78e8fc25be720880c7c96fe3bb83cee571730", + "2c519a92b9c61870a25bd465e387ddd56cf785dac01d3d799f1336071a688cf6", + "2cc11b7818e043aaa695b6cb39e23ca678666972630ac06f4f9f233d1f7f0808", + "2d3b45195ad9548f6d927e45edd80c0472fe6b99444d7bae2078279d94f38dcf", + "2dd4376fd28fc3d5be4efdb5e4d0788a73c5e2e017433df7afcdf4e295073e71", + "2eda514c8c3130822ecfb887b782233433591d277b4d5544f35869951575a3d5", + "2f62b63771280281f0bd7b76aeaf94b18feb3d9e77d5f4c3e6c941bd1b24f682", + "2f6c7e63a96faafb2769f3bad09af0dfb334b7572bed6de64bf9e9fe48a46bb2", + "2fc4bcded9e3b5488db04aa8d76d91801cd086ca4f9b1297717a52bc367e9b16", + "2fd10e31d0173bf7895979bebccc703d39b3bac009853985801006d7d6e2005d", + "300fe4259c0457ea26c5c5ddcc0ddd771457c628008b83e328aad35d009a7988", + "31a5cc9dd9197d83dbbd3d3ca4142fbf28b2fc8201bb741270773cb1894cfc7b", + "320f6a196e59b2b4edd608ebf7ee79f17c8ad5466630843e775035c4846b90c7", + "3228ee331db1a462dda0f3f48e0e61c214e7605e1108cce578ea5f2fd595f1d8", + "333b07c20959eea559a91f453fed7eb811df3829c14be6ce7a0f2cf0d2b43920", + "33555b978758146c6ef5db44724a5c7d0c6bd220b6f59563564431729312103e", + "33e88827861239c6dd5fc396fc3565daba3dba151859cb16e84bb12f27cdecf7", + "3404ffd881c45722234ea44705c34a255fd4c7bcb8bf6bac83c63d67405866bd", + "3419d427c11b72a992314179016bb0a9148e4e87233c09d9a2670db8fcca245f", + "34fe8be6a6881aede4d578671114f9ecebff8022ff512590dcb98d7eb56136a7", + "362adb9e2f87f785eea8ff421cc49108735a1156b118db43f7dfb3640de23566", + "363bb1c2cc2d0815667febd486acfcfbd569db438fa75e5e535e1533c735964f", + "36b9c0d8e8dbf42d0ecf7b80842f7ece7e817229b8d9d81b9abaf789f2c84bad", + "38d12b23358dd923d0dea27b84fe3be71eb3da4d43c60a7437fd0d79429cc151", + "391a987cfdc35a7003584b41e687fcf13b3ea16a0bdea3579e38b9b68283dbff", + "3975173592d19589ee2483b3a9c729a1b09e357dcb9b99289e91e364a55d34a4", + "39e42a7c1b6df133a041cae19e34e0f2b67091d7aaa9fdc857e8141d1a02e0b2", + "39ea289587d7c4a9fd58e23fb6d50bf46589a1a9d30d37f385ad74547d61262a", + "39f53df98841de6d36ebe7d912d78c71b7a4880cc7bc08db0e52dab1678006fe", + "3c6ca95fd406cdcabd3371a318249e933f024d2d96c627aa2735a8cf7fae7804", + "3c90d5d7da13b39be123adb5d07405cd5957e4bfb7c514020dcd16c358ab80c3", + "3cad974195456ee5a2db5145a90ca0f2558e212a4c893a0616803f4b58326e77", + "3d546da4e682f1e5e2e378874bb802a821fcf1a2cd8f3ad02fb4c2c2539ed504", + "3d7dd030d76915bc67fd843d4a0f7f16e93fe56d1ed2dfceb77e3c1e846aa2a4", + "3ea9916cefe474f99143e9d69e0363de2625d9ce34d093043f1731a1a36ed598", + "4085a47a2f6b662206c0c7fe411d7b713e52f429bdefb47b0aee4f7f7eb40065", + "40fd8700ab3b156b4b0f82ea40b48dd00341dac9dea27e31a89bb53587a81fc2", + "4177ad99c83f31a03223c2c57ad0633ee8d930604b5da6143288bb9d95a073c5", + "4199a7885060607ed8db3b1218328efd16d4d2ccf80bc1a81d0bc590ebed3232", + "426e498c237a053ceccae427cb0541c4b237d508b4464f772fa89f78625b4120", + "427b4bd0585c94622bc045b12e05ac66ac352697c428ca2962489b2e099e099d", + "434cf031465e32cc99d0be0df1be0e209bf206de47292393be87ae541ffaecc0", + "438a37d6b4bd0e2fd1adfd430acdf90e336993bd959f79d5bf71fb37a825150d", + "43ceee1b0eca37c6537d26a22ae452abb5881e096dcc2413a056be46ffceddf4", + "43d3ff2bed6137b1430b5b5a3b10e54c39ecddcccbbd9a41e9a16ec6604be4cc", + "44632cc411cf104f311f9118f831b8b096458e112ba5644c00e7c4c230cfcca6", + "44bb557346fda02484ec041ca21fb14dfb3de5c9721d5d129ef0e9ba51181358", + "450ebccf76e662aaf43e9bde1076c30ef05eb7a7ca03cc5e40d242d4088010ab", + "459c03ca4b12a9927ec189194a6dc2732521761d69a41490909503ed15d0be51", + "45d4b64e2aecad1a79fdcc592484933cf1a922e6ef3db9e9aa626ffe9e4607fc", + "46c00e829542ab0f4441761c7a1c22ac053712d38c1f1cd57bc6bed307e62feb", + "47416d5350e161048d0d62abae130ee6e13e620c1453601bb68f12566d3a8aa8", + "4884708d82754bea1caf4baf09354f1013b3932daeecfa07713ee2ebc17ad8b5", + "48e3602d53ab14882fb5a3fc964efb696a97b153329b4be01765b62164b31cc5", + "48fdfb473480e244ab50a0aac797379a1d447f455569fcb32d54fb3eea07a388", + "49c4c3c498d85c4b459939a083610984c215e633e361f0ccd2d6e11fa3fbce39", + "4aa7582e97431f30c9db15f9ed5bb7568ca31cbc5861c2c64abc20827ae986fc", + "4b63bed5359a285c67e9ff7d84280f7b0a40c641104fa0a1a50af5ce0ca419d7", + "4bf3a488c5402697c040893291a578005cb34907d802b68844fc5774842462ab", + "4c6d7902a1e22b2bf6f7f4167a860be5ed39be0aed7c909f23dc649a728b3cea", + "4d0c7ab2ff619a787e1e947cfcf1c100528c47c16c90aebe19e3281d5747a55e", + "4dba02fb52c247ab7473ed24861f38cfb281456a57a480e4d64457180ebd2f70", + "4f81b09cefc1d9a5f91ae4cc1d5b2788c7c0e0b384e17512e82d4bf023c24999", + "506d3395d1567f02c0c44436838ea9889d35976ecbcb7ea30f9a50adca34ae3a", + "50ab237de0f3ad4f4d676158d6f474657d272f8ae0dba7ca4950f917c0233b85", + "50b8547d0dfb7ea95945ad82c54b630cc3ab54e0ab01033a9f19166fa41f58d9", + "50dd63169fcb6480776acb6cf689077b9fe565bd6acffec4e86bbe8b12411424", + "5196ffdffe946a62f4500745d1912f3fdb667cf7cd0cafa5ae7a035b896db048", + "51d45b6d1badda9da12208d1b9a1ff775403485fda5a8652a0e8d7d499f4939b", + "51f1b9dd8bd0de616f527f8848f95818854511884d434a2b394115e1bcc7a070", + "52479f35626dbd059b00c6c86e23379ab399d11ce8042b4399b3bc9c099fcc2b", + "52fee2dc2bd87e3c1a6c0620c5c627b2cc30490936440fc6122e0432b983f5c4", + "53d530ffa92afed3ce280ed6285c4888ba3bb59ff1cbd07fac523a5318f00ec3", + "5480d25e04a50a517a951793e91afcdbc3ea583236d08af15ca500be17218b43", + "54b739816771589b437a20ea18ad6c027b5bc534aa88a3be253cce7a37acd3f1", + "56137914bc27ccc2862b7a1a1452b01a33e6fa0d263c6afa6ab0607d40a8f099", + "5741844eeabbf8c6c21ca90af591fd3c50da313baa957a4053557992b8cc4866", + "5747a957d9972c2d47c89fd759afc4a8531a338971ce57954a785f61761f57c7", + "57acbd0e674472d2028036494f2e7e3ef2d052f4edac55fed900c1dc89f1347d", + "5897e01933db895c1bcadd473c11310268cbf5caf38c818f81d4bc96e93a4f10", + "5910b96219ff6c27133fcd192fdbc1c14b2930f1eba4593f03a31db8b7e3291f", + "5987281f5dfa9bd2089a2ca5da16d8ab94bbc2b908f1d04bf6548d9d79d6f0b3", + "59d30ed358c5b684b4eb3b6a32c5c62986761e80d85b519e1a48f6b639ccbbce", + "59d9238f47488e2bdd936a0d1c63ea4e763e511fb9b5c903862e239c35475ed4", + "5a678af69532034429641cbeabdcdcd84eade06b83dac385dd6d8d7ea36195b9", + "5bf1b42a54729b348c61228315ecb19a7b2ceede4d0e7b8dc9afa8891bc81d58", + "5d0f3f38df807dd181beb9335c96aec831ddabc4d9a634aa7c54334ab16c1d75", + "5d4e9ddcebff2ec81105e8ae79f57de3c984200f9994e88c466a0aa07c8e1320", + "5d515949f2580c7204d7e0ea416bb67a32d528e4ff3041f37bd94c08b56c5357", + "5e418b7e9d172a9813cabe7b9bf34c168db366015d182770fae70f0ac41422c0", + "5e78b744add77bebb64bcf18bf09a83d5597e058afec7410d5da6eb169100c30", + "5eac061f4ceb6a0fc89ae4a1cee37ca22bf27174233561d82042b577f736242b", + "5ffa8307514fd765cae152d50388c2481438d15691116a76f78c1fcef9d2ba57", + "600b24bbb9011d209e230efc0651ecbfdf9192844de83e795d5993ee034c8f06", + "602956f2c532b51b46cdd0c64db43929f489e5804e760c69806aed0e26b3514f", + "60c343e6f949a3098f6f33740971af76511890cd47b68c88f7ce1894d6d46b0e", + "61497e92cfa672eff759375d67829981565e5be07bb87bda0caf2525f38171c4", + "6214f701d647152e08612cd72bd1e15b64329676d053324533b8c928d6e6c9d4", + "62f0b0da66a12c0b2bb3b7f1324906246d8aa8c22dcf738728d169df29b85cbb", + "6349e990f1235293d2dbad51467abd02bf7dab9fe63a77e1f5e661bb5e1a1672", + "63b494c4eaafcc2da75d4d1235d198b3c0b0c06f6bf762a6a5fc7baeb1a9e385", + "64010a7e425884a51a7a5195beffa3367cb63dc05d7d9af85e6089f959e7e589", + "64bd5fbc7e70465034494d5dd5147ece3426224ab802b7051041606c9af60ea3", + "6518e953250447419c08c59d2ee155fd52ff507b28eeb3a83433bd89432a37ea", + "651c0e51e5e9e00565b3824a7f42082fd052a2df3b8df4e1caee0cd8e50e90ce", + "65afb3b6725976a6ee42025db81e84c935fdc10361f657ebb33d9cfdc442d502", + "662a3ab296db484aed6b385a16f29bce8467cf9e45bd85b7d4603ef7aa75a911", + "665000f91e6250f222e1d20d8821ac20395c23127d25b29b2a18d824efe5d95c", + "665b59a56dfe89b21ce7f3e1078d4070fc5d6c6aa9c32fc101782b87c94da10e", + "66db6cd423a630f043ba70653d9cb2d3912d39d74ba20aba42f4ed49be24515c", + "6707e8af129b5c01f0227ea9337cc805a44e08594231bff59fced200904b68ca", + "6762618ec7b24c3b7302187e22d53c4c8fc36c5479d927e7b22f82df0096ab81", + "68aa926879c32f5120a16375df788268f6d466b722c653eae0cb489b64c2715e", + "68c7948e1397aae7ada0e711538dbde0b894aeb6c65a2a28b789072222c7d04b", + "6978f93cc00cdf51bfb5d79aef73aa1e6d9cefa78fa0452fbd6c7d536692e095", + "6a4016ddc183de9be99d891de4af7b52506be085bdbd7b623afe6ae67d380743", + "6aa36251dd6981548c9da087d57cdd769ef61ed00e6b91d9ef9ad66fd8042151", + "6b317d5548424aa5a1e695f983622fffea91e6839ad1998a13000c2510316cd6", + "6bed9461397e0217ff6532c954a833dc28fece0cb46e647554a189e66524cb97", + "6c583b7ca612a308e259dfa946fa8cf45fc8b8ba31262221f833ff3f44783095", + "6ef52217b53a0484ef0bc7e133620ce5236c4268111f9f9e6562732ddb3c1bf4", + "6f019701fa2e5327cdf14bbfefca64df0f5aeecb6928fb52e5a01acae8633fe1", + "6f25c0d6d8191dec23c8d22c518212f1d2507ad4e3d4e4ac34869a5e3d988cf5", + "6fb7eb6b8b430c8ef441afd0ace15a4c42dfd23b3ac134c399abfa06390058ea", + "7258d7dd2d99620e2f9563f387392f056353d9cd0feaa9065c2a8be194a95f91", + "730b0ee7d1a69be505cfa2d8f82ef12619218566820fcbf2c2049d781cedf6ad", + "7345a52d3dfba26a5528debf07f380ed05b6e5a1dd313e7e108226d74edb2724", + "75038a8ab21048d173a77df2a2d9410cf256f6a587ef333f82d545a2b0da5495", + "772ecbeaf77547ccf7b918354bc234fffe749626c7bc4b6644544711300e3795", + "77a5e2b24fd4ffc2325efccbe53d2d0af89831ac3081665825a5d19b0f99bf58", + "77b31ca7d2196bab4fb31060a8c79365e306514ebcdaadc089a12ad03b7ab734", + "77da10edb61dd80d381746bfb7929ee08eaf96f64b65f535695e5f076fce2861", + "792e6b2c4a8865ef31fdb6e497bf4c8299a0cefa2fe47777c8b20dc4be86fae9", + "7a19124e06706c046acb04bc05c52c019ab35a5c0bc2d9d48e752d4ae0942009", + "7b750fab861ef3d12d2b387ed46455aa8b65ad72ade5ca7d253bfc8d8408c160", + "7bd6f159ff7920fed389c26281943a38f556af30c29d4aa402d5d1ad808a39d4", + "7d913b1ad623536ac5812120629dd8cb0455511983d5c0d1d26277b9aa0b5105", + "7e010a7eadabd6135dcbc0a81d71a0c4a40d9ec4d8e4a41b9bc887d64fbf8c8a", + "7e030ac9c9322183567b1955bd4924366422f8ba5246e22a232e5932702f2069", + "7f64fa79eb60a5f4b50f1b33da9b9b3d99737cef21da808c825c33092615cbc8", + "7f9ff6f99c5de72a9908c3fadfde6298fadfdb9eb36c32219e4a7fa10892f642", + "7fdaabf2a4be809033161efb0bbc2828cbefe75c439216450b11d705a706a1e6", + "80bda934d0265c977e22f84b9a47bc985de3740895abeee0e54cad76fd791093", + "8196baac48edec4fd98a232dd60880cfc89bfb76b772d9ce7c857a4051429a44", + "81afc453735d484ff81d6fcb46b6738fc9de062504b255a5d366d15833b2b93c", + "82ba96b0db417c596fbe2c6f05edd0aeb2e82e0903016792d0bbcfe6551bd17e", + "840446c401008c1f4f1b1f82237092cbc1a020e55cc02ce94792c3fd4c4f4ee0", + "8488122d93a4ca2429fe5228db45ba7185a2290931adc5c1c99e8cbab534ac39", + "860f89295df9fcc82bbdc1b569404fbc284cd0db6eb4aedac0d72ceae19e27aa", + "865a93b7d07abbf7233f13edf40ac7ac9c8013b44bb5b1d8f56f6628c9db00e6", + "86d9accc836046ab5fe18c08db4a4d0d5d65eef66c487b81d39befa68a892c69", + "8744f7b069ab27db815e0d8ddafb255992a61c37084b201454809d668506fd34", + "8755482d237160251dbe35fe0020ba85c14e541289f1dec4f6494305fbd8df89", + "87a181cd2f9e280ca7cdc7f0fde3e07112a363891302d8682860c081c35921b4", + "87f6acac31dff38cd80e58f8371fff6142712d897de20092d418c4a430a80633", + "88a012cdc526b418b4aa88816d251083f4853e870482f8e924e8b6a8af8ba731", + "893c75484cb41d8965549dea2b8a2bc089cb463a1e6f8469546dc92c8705de39", + "89d95882eaacf6f7cb28c46b23ba22c3ca11986ad0365890f461a800c7d087d4", + "8a41d9b49b6fb8253d7df35219e7867110656805953c6d665f128bc74765ef89", + "8a6dc00f6e7e5cd26234fa4eeb4d15a8547fa593afc36997459216137d1300b7", + "8aab254e85254d5cf1f1103bed51c9cdf8d6e287dda1b267b7d2adf0d513eb2e", + "8b7b0c0c36a755ab9fdc1c810ccf6abe54e24c1fd381c8da44fe1fefd0d05d76", + "8d11dd1ec76d295ab531cef7f05a62c5e138f2084880886a78c6855f907a19f4", + "8db7ec8b19bdce5f9886dc235e95546452a592b45a0e0377fae405b80564eef6", + "8dd973f2ade114309f025349bd4852d7fb3fed114871ab48e19583c48260626b", + "8e2579fa847f97c9029eda36fde5da04309c4e42188364edd4ade1cafdbaa213", + "8e94f3617db242fa9ca8ec1d20a20134c46fb1301f1c585863055e5e1ccfc87f", + "8ef7802d4d36504e28d179e70878185878721e7aa7b791aeb9f4510ae926e73f", + "90287d800986804804b7ef7bcdeaa94888d09b6329918ddc0136dbd31c7ac8c0", + "913df90c98f124b2ac5f5565e5e86e1043f79f34f1c6410e7843886c4bfd138a", + "9143d67f035c7c08bf56e3bb65f4d597492147e3bd06e575d05b9617fd5a2e6b", + "914a0ddd825254d368ed88a4e4c0e1492b62b685b54d43a74eb809c21c0b057d", + "9155e1827763664471baf371a8b288f53632986039fe13fe8bc519af27b7448e", + "9180decef7665bdc3dfc930e38fdaa742261568d8a323e5ebcbb5dda83ab7435", + "92c3f4c9a5c73d021484d9c22b11dc3f0a5938d92e8b4a3ac29a540daf62cdae", + "93308310bdac1dc182ac074cf0f452d5857412a3dc381d6adab000413862c294", + "93fe1b775c9d5a005b4c24ed4cdcbeace7c443fd96acde340cf37531e0d958c8", + "94adae68e8f80732228ddd487310402d7af461be7ebb5f483fcc14fad5c84368", + "954022aeea5a16063fab63a3400d1e5ae2e4c1cf3af56c94d437dcb613d6eb9e", + "95848d52f9d2b349485284334c4efe48bc0c9d0e4f3993dc3fa0b93917c0c794", + "970ae009f20045d56e65f7c177ebc783dc82bf777b5dec048d9ffb62c2e7073d", + "975adc9af6b12272dc8484adb4210f1137470d6c77b2982cf4809f2cff9b712c", + "980d048a07e2ed128f0cfc5768babeb12f7b8c1102dd264d40304becd711ada4", + "982679e3e46a917a3cff3891b491fba40130ce6ec338858450dd68b6dd6ee16a", + "98d2821446cb8cf7f5342382fc409bf4e1531234751095148028095c4d68c0b9", + "997b672b7ce7473d2751343cde3d2aaeaf840cb344d368ebbadd3d2205a3f079", + "9a0729ed13e288c93f267bb3557f4698a73ed5aa31a55a8e84b558b94825f262", + "9ab25a8cc539aeaf87f23200ed1c3f9667d44963b7f6da1e903f16736bc509e3", + "9ada1587db68d41c577e46d5046862a766502fb2dafa18a139e5f797b035c295", + "9b6021119181becda810b14ab492087af39bbb14c7b7a73d51f6e9f274aa83eb", + "9bc128b3db4a990e7db2c97b5c7761eb5452183da0e8b815fe35c85de239251e", + "9c72c21529d08c5069a521af5f213b99dc9741ab154c07c3bd0b4937801e6cfb", + "9d09df5c56e375bc9454202fa81f76636c4357c73e2f5000368c4300c30c9d8a", + "9e25bb1895a025f06794aba14ab0cdbbc2d4c900b747d42d48957a823e0bab5d", + "9e71e5498441b4b84fda9d4a381c841cdf9dda4efa3e0911ec8866e7fb9a9d6b", + "9ed736060546283eaf7a9e6d7f2be31ab3cfcce0d01a7602dc9bf3311dc84567", + "9f30c7f9e60ba8535bf2d90500801f28f4177b66ea3a2366eb50ef7ff7f9b9db", + "9f3b2d807346a6cb4c758c408fef30a978ae8b2b82ea419f13d447daad058f5e", + "9f9a2aec986ef7f8a6fdf55160dcae5c969717a28c50a5bd8a3b46bc0ad415fd", + "a00ac37fd929bfad607c2c6799f93375b96a8f3011cf2f49ff9f47201ca01987", + "a12758b2b9df7c5e40f379dbc2c5374e2fa0508e5f21c7b5cc38dda34dcc7102", + "a20b63cd6f96f12ec818d2ee4b0e6f2017b73aab05981d5a46d2c466e7d82432", + "a2727388face2b4a7efc7b6c58838a198a9e1f420e0b8852b925386e77a83f70", + "a2cd9da5ec1623c7aa739bda183c8101f171d8713dcd39db1ac5871ff56b4017", + "a5119d9b97da13e0e5999ef6c91227ce8a3b17de9bb77acf4abec2e00259b822", + "a544ddf0f57c78a44f54dc946cb6885693ed7372a89f15bd914569f1d9849ee3", + "a54c68ec203d07c2a967d02825037471086ba238263be088d59d9c7f00612ed6", + "a5ba2e61cefc84d8908c6b2bb830d91790177360ad02b3a744c640ff5b45d6df", + "a72c67526742bf14cf400ad3515ed8b87c2fc7bb4f95070a0faea29c8ea984d9", + "a775848264bf7b4a3777a556d11f38e2994d92714c0c7734b5a4d46162c84226", + "a83bf42daf636e7a77f77955757dcca4224440cff075bf7eb0370795f9d8663c", + "a93f33032986ce24e99eb38f10fbd9a62d75462d5adeca5b4e236cef0836bd90", + "ab78cf2c57454d46dbd7fea49f3293f01e7c6dc9ae10e2eb2b4fb20a3648ecdd", + "ab8b5ff6eae18b911f6e8dc11c2a34f65b1c9b6d116c91a00b8a8792096688ed", + "ab9e518e9c2bd1aa4fa1e395553620371f0c79804c89f5daeb6ce18d5b66af23", + "ac5ecc054864495a34f0b9a16a5473b786d7497474773dcb84947ed204009aad", + "acca44ec9bfa0a02833526adbaf86c7041a0ac281b28cb7ee91f18fd09b078f9", + "ad8f425b29b9b12876e4fc5fd18c3f867046ae41c81ec55cf0b6d208e85dd742", + "ae3fc0f559f37778cfdf27f640bd35a16aa6b76c4ffecf8d42baf37e4a5a7905", + "aefc7849c632d4044a926f20f1d94b82af1b7ed6dc0167d2cc249b218a343a93", + "af28d652f179b9718c6a9a9e74f9e8e7f12c437a5ad54a75d80a82c23dc8e644", + "af82611bf1b54c6df4bc9285d7264be4ed86af1711891cbdf484def25a7d98ef", + "b0ea0f0cf25530508c4a9f67de55afe8038355b7e0b860af7ba9f04aa31a3e06", + "b1bc36c06168dda8d850b543cbd2d870694b4869230cd985b20dc8b6ca23c1d1", + "b20791ea51ec80c181aa6a54ebc7d1383974764c55ee7dd3cb3cca64948266b9", + "b34c178c6a7c97b598af258fa1a791d9c0c61e8432517e3d91953c0cde131d7c", + "b40f7a9d22cda22a063bc22dd3b4db5c3598a98609594494193ed48dc8192cb0", + "b461e54387e0a47ece2b22c3dcc65391d86517ead678ec33a5d901fbbadc0926", + "b4ccc21e5ea8003a945bf634d6b800fc44eee0aad713f12e57e1984c525a65f9", + "b714c7af15a1b9b8b710eb5c7611595076321e58f55c377e69c9121093522bcc", + "b8004aa52a284d2556c496f43d30dc6fb1cdd091695e1e50c1a90f0d65f92b09", + "b81b7e5043ba8065513ae1e47dd72ba75abd98d33e82a7ca7e78e051cbe43cf2", + "b8876098ea9a407eb70ae6ec08951a4f55579c2e0c08716a7c6cc43156beba4d", + "b9285dbe600a0d883bb1a8106ab84908e1daec78d16497c41f6d64424cb98313", + "b97e8fd40a8dbf94cb8e2011dac90a9480e3f112d431fc70ab75ac357644f8e2", + "b9b5303b9443950e596764e3ab43ba388dc88bd9e9fa4db7feef060ce896c62f", + "b9c2abe9f6ecb2b7b77d64e7191d9ed09a36554e9421caf1b2e65c5463365230", + "b9fefd5cdff2e6c28088cc0a53e346a2c097998df8b6fc297222153e1daf65df", + "bb2bbefd2a016c0ad8e9e3e8858cd69d918f0eca2138b4f2289c7afd8ac1f4d6", + "bb30b5791e4307912aa9adbb690455c076e533c6285272da8d12e78f51db283c", + "bbf53aba9adeea04e3f958a95feecb13ac2d29b45a69478899314a996c78e544", + "bc6192bc3b2826b4e1b0b4d086536b6565c66d2c8bd66378de084b53625943dc", + "bce154fde165db7793c519eb7cd7b398ff52e9cd42578142e5c8ee8fb0656547", + "bd1dc7793ae70f7d999d0e13a56b492cda6f3ab0de6301699294ad15b001ac7b", + "beb32919cb900c53a3953442a5d071436449a4b9ec5b2bce6056f3d0a1f87714", + "bf6f5ac70503dd191546b89ffc869bc3e28efae51bf05ec330535f6e5286e301", + "bfc5863c1a54513bc8256de6b55beb115b2fa384a58f23530fcf523dcbb52e1d", + "c0b1165b54a3e58bae71e154a65337aed1918c1d065439d9a3f9d65f7d1324a3", + "c10afbedc26936296433deb12073cbd34317fe0ab7899c916c8cfe4001642ac8", + "c14e6d6cb9f0b03afcc8cc196c3a29e3e9e2306becbe004eb2313e306cd0a74c", + "c17f9ccd9fd2ea3d7d9230a42289690c9decb88e0a880a42037a79e94b100e02", + "c309e10ae79aa8821732639fddbc5af8d7480dcab2d0316dd331ad859a03eb98", + "c390b96eed7402670db4562842a2aaa9581848dee1538a66faef88134835c045", + "c39690ec883cace98ebcccef51fe48fc75b323082a2c0ad951b40531383a9717", + "c403e6f15abb1ca9d51a883ecc88cfcee37d64058e19559d87cc8e9659369ddc", + "c628c9f4b7dd0a2931ccd0a68cdb70335c49d125adc341c75ced526e0b5d7439", + "c66503618c5c0194e190365bfdd6f18fc85f6e8b14f248c0724575689225af85", + "c669ba0197f300ec35c9cda7a239e8a69e73488459a625ab9476b09624287715", + "c9ab26239629a397da5a94679f176211063e4fabfd3c41a2481405a5f70b2e71", + "cae0fe58a9afff0750883faf68e9d4ed511d4adf9f799828e6c7c331d0b2733d", + "cb0f5ddc19e8db59d7affe16f76ef2585cc9a4c992ec4aa144bd25fbaae42eb0", + "cb42cf2a13ecee4f5c3585d182f1a5e0172acde5cdb5dca20df348c61d5707e5", + "cb7117229c57ca9d821d8ec24d4a922d8348a5fe3325939bf84dfb18f11eb5a9", + "cbde24769f686efa9b3c354173679e351c5ea0d76fadb3e69593a7ead04a1ee7", + "cc70ac956671e95234876cc83614c2f88db9110ecce9381fd0ee64803b299429", + "cd36b417bafdabd7721c88c17d2c29c33e9a11cc0594f52a54950906d4a44fbb", + "cd47f108a726fc44a0eaaf3d4bde604dbdf8c41861f2c9a64ef11c7a36c8d6a5", + "cd4b3470071fd954fce44c84264d4bc9e61790b37d2b5e9374efef01316e6dfc", + "cda1f2569cdb637d0550db64ff59e80ba7330792ac95484c37cb0e3d113eccf8", + "cdf79c0142aaa8c5353ce2d676a01a9865903051dc987c8b73cdd73c76ec9c23", + "ce43ed1b892a3a7b12e0bcadb397a918718cd979e9e0795395c986f56bc2a750", + "ce7c329ebd2e1b1d9bbc17a9b9b74bcd914df56e03745413b6dce38eb4772bcf", + "cf2039c1d46672c9f9a5515c3d60575e6be00fcb93c2cbc1477459ff50955ef5", + "cfc4f4161ce90cb06c135ded875919f4675c107e786fb6adaa9070e9fe56d34b", + "d0ac396c83185c55d031ae1fbfa33bbb017d14a316cabb55503b6fe772fdee36", + "d11329eb550769a78aa2d9f8be77708dd9ed7527d706dad9fa2ed230fa38fe5a", + "d140b4148ccabf6f483642b7ce61825956b6e0a84ca2bd32a08a98f84a200a0a", + "d152d1675da0f33cf86b1664536034d4c9f4ddfc901e51c018b87280284756df", + "d2d52cebb51643d7f8ad3b2e52987b7c7f68e66e172418fb42636767f927ffb8", + "d42152e189ac72bdfc7e818137ba41cb5265a008aa3cdcacfdc560f5af5341e0", + "d49d739f974de58a57bc4f74207ae2bc8d8e66361adeaf55e961a27aa1eaae65", + "d563f93f87d17243f00fe477d8e78157ae7edf218e5c88e941cde003bea36a6f", + "d5a907cbc19ff057e4dafd6cbd075d983f7e71085148ddff5920871bc2167575", + "d5e7f6b0de3d88f2eef5cb8276e5f45caa1fb6e902025627b04b490b5d98ade1", + "d65210aa0d78bb32b64e066dcb64b6f8f71f71779a5d499354ddfeab22f1332d", + "d6f1ed8d805236ccf9f98fd65500bb4aa9ad8c41f8a071ddc29ca8e752929ae7", + "d7ad91e423e48f64697d40ac5d829b3386ed420ea0fd9841676704f0a062dbb0", + "d87661fbf955f5f80d07effe80080355aaeddb42c6cb0409ebf62ecb5393b674", + "d90b629f237efb49c3ab6d0aff93c838b4db299e6d7feee7636508d12b5c5fdf", + "d911da1cc1aa1a02712d5b1a762cf0b5d5ab389492c1a4b0559299f05493d3ec", + "d99afab46c17b26a07d8f71fedfadaca28767a8f350a2d9589736ce7da6cc3bc", + "d9df49c2c7600b22a4c93bb4b1f9dc9b1157f0117b215b7d1c00e26770ed5582", + "dab08cc34ed2fcdd470d03536ca642ff4624b381eaf9d536195bbe17ebe9c17b", + "db064204080f8555787bb6221183aa561add66a3383b4ba439f45c6ee5a2d96b", + "dccbc6279f3222dbde7759e12a7dda491c55472378491e24482e4ba527959a06", + "dd5269687a6c1f75d0234677f460ecef52a58ff78ab2f9ca6822a1316b36c5c4", + "dd87ae7fff40771e69e0a6180d1305fc6d1d0ce432e9aced610fb3f49194e7c6", + "def0de3ee663dc1c9d1b8df1f0127da4720d8dc16adffce81c060c330dda769b", + "df1f96b7a4242177ecb40fbebf6ad7091106e10dcb7b8be68cdf62ae9532d7ad", + "dfaa421efe0204df2a51dd0f34cc72c3310aab26426b107e3973b954cebb730c", + "dffea2882ee92b26abce07a5376e46308bc5e7a44749f72858eadc0aaf862fa3", + "e0077b61aca56e8aa2a018e68311f0e6aefb5d030952537cb7eef012f41369a1", + "e031d45f43b36e3034b2966c9f75218e710ec72dcc2a5e970c1d4244e4913e31", + "e0de0c7672d9da979ef118f4983eb9ad6ef0402fc8e6f5935c75326e26231f68", + "e1b16c59c51132e39f175f624b134609dc6280d818f8c47d9b6eea77e7b193bf", + "e2679dd5d22d82f7095519aa46615e702dcdf209163df9cdb734d295f70ed8cf", + "e2fa296a21818b123e3d7f367826c5e997b1371c858beb73610b110ca8ce4839", + "e50781f70c95585a91804012aa888669166226a18d9fd8d104cf901f565bb067", + "e579dd010621e89c430a10b2996eb3b216029e121a77237418f7280ad0f5d088", + "e61eb554944b8e39f620309e492c5ac7361b46cd9706619aa516e1c87ab2f11e", + "e6ad5e3ada47a44524ddd841faee42a564c8b8fff2eb255362771dd9f3cd5a55", + "e76923fb355d375ab50ff13eeb1ffa782eaae18ccabfa627716742e9c1390653", + "e7afca6e54dddeefdb0e7f1b02cc0cf2abf00561ecc8086fb882ed8ced4928f9", + "e84d29a0132a44780d83de149ff7581d54b4bc7acba7241efd3758dc613e3cc4", + "e906c0d38c20ca1f640313df9905bb9a39b48d7fe15c57a2ab75f1565a191fe5", + "e9173b067eb363df353546f8567b60b5122ced586ee8c343ccbb3bda6ab73592", + "e9795c286533eb785b99b4ec8b8d78f7a1eb8bb725e0d7916cafbeee1ca8003a", + "ea4b1dc02232d972aefbc800b4156284095112baf01168056ddc24b8a247f512", + "ea7664f177fe6c9fea15443fbf74a78eaed5618ae0f37bf189a0b726d17a5d4f", + "eb3537c28c26f4f61dee000695de989b41525c029c7d92eaff101a6c58fbbfdb", + "ebba25c528191213226e54e837bb727c0dd9fb48ecfcbfce91412f7fddee51a5", + "ebd7b1abe0d22ca80aa8f56e1d62c8a38ff410749cc212d3da44eb580195e0f9", + "ed4e3c8d856aaa25e7adb8ef0ab63276a38f2de8d211ddff05b794283c206470", + "ed82a193afed8645087d6ca4289eb20157008afd83272002a5669d25cfc51400", + "ee15e6d1acea63bf964c93eee42be5dd3029058f8770dd9caf4d5337a2c01bd9", + "ee5d303b876c4dac62384687b039f9658fff97c84fd87b08ff59b99a50a2c6cf", + "ee623702315a339ba49bed2fe17471ae117b38e2e55d2469e7cdd4c8ab88565a", + "eeea07e1f63be94a7ee57285c6c517fd3ffefdb1f48e02d1318b7d0d204db5f2", + "ef727e08aa8670e19394b9f923ac237625629c01368ef4a89e9cb565aad5d2ff", + "efc515168087302956545ad66a1b2923d107e4142ad40dc14c43ebfbc769e247", + "f08cf650a03124cb4ff3d44fd4a70abe0342eb048dca6644323b335770a9a083", + "f1176ced474ef2f0ade0eea28d45b69b1c172cb97124465766a696984397688a", + "f2143c8de49b21841348de9bd75cac7ffb13b71d118249abb1917698450f638b", + "f28ef034d38ffdec076a212d91fa453834eeab284db013d48370f45b23ac1e1e", + "f2f034448bd3ed4bf77478619b5c901fcb70f6a8f3ecab9dbbaa143a6953d9d0", + "f4d897d2739e313ae9a7f4fab2d18e96b751d08263efa31b92c65c39b569a099", + "f5636406724c3560b13585d91d6a1b58716ad37a776edc88af285d7b05f70d76", + "f56756762723fb4248c7c9cbc4288058fd1bf9a5b92dddc39c7454bdac885124", + "f582d63ab10b23f5177f0615badaa3359f0a0f7a56fb4165437c0ccb3bcceccb", + "f661f3a1841fceb3f780a7b40257af20ed3e704c463eeecf783b2c749d7a916a", + "f6685d129fc3f57ba35b71ce97af43735c428fee02c6c50fca3b8dfcce32da38", + "f72fe1265f2add1e7058bae000d66be9f4ec18762d6413e0bc58548f043a59ab", + "f731dbd87c85ffe281c1d8740aa242fa9a7b1d7210e0b1ca69592d11f6f11f8e", + "f7d9e96925c19e11d9a4329b0ff009a383d758857a44287115687ddc7d098d93", + "f7f300fb09d9d92cf906384f012ebe09835fb0d03b902e2a230c001b302b9274", + "f8fac33ac4e76d8146004ac03838c9e4977f31db2e3226693233e411ae4c5cdc", + "f919f6e81d3e9733aa6e62846129ac10370ca19d3e72ff8594876e9e77334efa", + "fa49239daee14b0d44eed2285a78ae6d7ea97346a5f71a9439f394afac870c63", + "fb3c26f9e5d25c1414b4eab1076ffe73023df0950ccb8798d2b2359e7060ff69", + "fb7d518eeb067ac77240f19ac6cece874b2af904d69198f83fab2f6f7de203f9", + "fb9f9805796702942ff8ce23f4b5f2e2abcecba678a82eaa95eb2f35d3d37cc6", + "fc3e326d60df78352c6da66ac491963b7c457797ebcd1e96c2436594fe59a6c7", + "fc6bb05f32006e045e36a3300044ffd7205edd7a2ace65ddb7fc77df71abd6f6", + "fc96267aab2baa5d3ae73bb0adc76accb567533bcbf6a69ddf358b7ece8e51de", + "fd03ea8635c19ccd8b6496bccce844e72697d6a638a7fe75234441e839b7615c", + "fdbb4ed5157a5396e2fb421e74d4d4cf4bf3131cca117026ea1f51d77a857994", + "fe4209d418bd8745b7e03e0450b65aa434d749e4f0b0f3793a1849ebbe1f93cb", + "fe54a9ab1415ed784e1c9d3aaa035570a6cf495a610171edc863f5743524e7f6", + "fe578fe58231dea95dce9a9201b55208564b4a59479d910d2144111d16e809eb", + "fe62a651f7a3b22cfc8af16729766ce8b0dba209d70e68d12dea3c7ff7c5d3c9", + "fe895f4166f609e50dcbadd349f7534f86356f5dc12f47d6511059ddd8fcb4d1", + "feaa87eec12904d90fed4e3ae7cd9189e9adf521af0c660ccfc811e39398c52a", + "fee98f221151bb980cdb862306da346bf3b0d9cec0b9cbdd9c2ba13ef635201a", + "ff1366cc917bc6dcdc630665d80888e067a67153e0291eb502ce1c061598ceb9" + ], + "include_outcomes": true, + "label_counts": { + "negative": 5134, + "neutral": 1325, + "positive": 245 + }, + "leakage_contract": { + "deployment_must_not_read_outcomes": true, + "nontrain_outcomes_are_evaluator_only": true, + "train_split_only_for_retrieval": true + }, + "max_flat_action_dim": 112, + "num_groups_exported": 419, + "num_groups_scanned": 2873, + "num_groups_skipped_by_split": 2454, + "num_rows": 6704, + "object_embedding_content_hash": "411ee7d4c96458879f14fc9460c0e10d5309d83aacd5552c6bcbe089e81838bc", + "object_embedding_manifest": { + "dataset": "embeddings", + "dim": 64, + "extractor": "rgb_object_layout_v1", + "num_embeddings": 419, + "path": "object_embeddings_rgb_layout.npz", + "reads_outcomes": false + }, + "observation_embedding_manifest": { + "dataset": "embeddings", + "dim": 32, + "extractor": "rgb_jpeg_stats_v1", + "num_embeddings": 419, + "path": "observation_embeddings_rgb_stats.npz", + "reads_outcomes": false + }, + "retrieval_index_allowed": false, + "schema_version": 1, + "shard_content_hashes": { + "charts_00000.npz": "6f6ea9dd2f6809cbed119598842a1dd6baaf5db099587a2748a00264aafae4e8" + }, + "shards": [ + { + "num_rows": 6704, + "path": "charts_00000.npz", + "sha256": "538fac2468d1bde33b9e76f5233e331040a8c76ff5fd224727bd8140686f4071" + } + ], + "skip_counts": {}, + "split": "val", + "split_fractions": { + "test": 0.15, + "train": 0.7, + "val": 0.15 + }, + "split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381", + "split_policy": "stable-hash", + "split_seed": 0, + "state_hashes": [ + "00febbd1a7ad126d88b65aac55a35a9c4c63bc11e0f6e45344ddd4969242b950", + "0161713e96d5c5e1a45422cbbd0367210454a00f7d6e9840771090f4849fcb5b", + "0271bb52312dbcc4b57060d87dd5b3a514bb13bb1e20b235ad8204d61ad50a24", + "0544a922f6c9c662f2c9238466411bd1942870f7f182a508fd124141ef777c01", + "054fa57f5f9a67e3839a4c9674f9146dda13d27aa43c99ab5f87dedb1c0c2712", + "069933a0983331f68db2455b6a2b4fa216a97e119e1fb5b42a65adc252aca505", + "06a95069882817d6afbcaae8ec6bc12ed361c0a9446c56639880a8b9223c0c74", + "06d2eae8a8bb4c4503348959d65191f1f652edf6267e750861de79a6e0f1b1be", + "08353e08c94f126c924237211ac6e55a78461a8ff88eb8a1dbddbf844b6f2ec0", + "0905ba70e353c374ec760960c11b7a7914e3a0f401bf94a7772225dc39a94cc7", + "0a48350b21b9cbc27742e61775d9a3cd611daa5905f96a2cc3c59625d4b081ed", + "0b7be6179dacba118f3fdbe0e890ffc2d4ae1d15a87a215f41574bda0b24907a", + "0c4be6df74c82811154373ce6212cb0d7766cfe37020a987e1c1a553fc82aba4", + "0c5278be095a7a37efb891610364a48e4a4d84d044f63fab14e15e9351d2a9d8", + "0cab5c0c4fd5dd2183a8bc5891151fb5e5697bb605a5f7aa3c85f87f0503a804", + "0cf52a33f7d5f568029fdd8a8a3b634cdc7443ec30e9c127a73f25543ccd2c30", + "0d7cc8eda8b418bc8993e1fdf3fd64353a35fafd2501660fe5d87c6f7ef3d692", + "0d873ad4e7f5c34b9002a03754159bc9f2641e80bb46292920d135468141f582", + "0d8db9d71f59cad390736c77c29a6ee9f9f24de0c5250e1b8115164f4bc33f66", + "0dc29c4c945813a6553cd08d2dddb3982abec790f14fca7f70dfab255302960e", + "0dcfcf107cd9dfedc537d2da4969654c2c4fc3d04b1176f8e1426633edc8fae3", + "0dff2cb77a11d1888d2a48a76c8da04fe4e0daa54b5194bdb5ea100e2eb48ab2", + "0e23f36f5e66b1e4a3a7c074a6757653fecb8ebf1beebcd1e78033f5c2656a35", + "0e3910690cb8e214f63534f41f2acc8afec60992e30a2ed4d2fb61fcd93604a2", + "0edee0a0681248fffae8e5677c3658aa6a8b59d9eb3e33d90ea3027923b85216", + "0f0d99b44408bc284f59809b1266764117c2426549ba6b7e22b897e4c24cdfd1", + "0f19062a067ca2c041986c3de45c4e8d4e0f32838adcf5b52c29dac097394b47", + "1023ebe09fd0537cc30774636aa5c98411ae6ed1c2dc259a2624bbd063eaf23e", + "1113eef231093ad70c840dc75559d4ca74bd3d8dc4aaefa1c0b04354fe6776af", + "11f96b224bfcd397ee4f98e388350e3b65418b00a9ab4d2746128b42b12b43d5", + "158c57e861f5dbf4fafa3370948c03172a7354fd34511a04af24d744cca2d11e", + "15a9a5593a8b8ab430be4f23e1279949867e92ff352f3e2033c69495d6f636f1", + "16a1c922cc898f22123974a540e044a4ab649945807f078770480f7297e8e90c", + "16ddcd38b720cafaa913dab508ceefddc7620486bb42e9e3b0a9b81e7155543f", + "16e7bf765913827062088fe3aab96fcdad44b4615afb788a2a07e434d272b389", + "171fce66778284ce4d146eac1ca6327cab37d27bbc4be27a2a9fff473038245b", + "17547f1d97d7d63690d4f5ae56430531c2bcb8c2289abdea53e0151d4bd9b79d", + "18d69dcc611026a63a4602aa482606f68eb3ce416d10ba04f9fadd6b3622a991", + "1921b5d9fe877551d561618aa96a78ddc9480216ba450d537d5e4b27206c4999", + "19cb279a5068d0754add0249c6c2577e07f80aa33c56746853c796f5418fc7d8", + "1a43f6367e6d71af41f099f5216e60ec305d43486fe887af0c2f19849f406b75", + "1a5d3ae36c55dc60075de4acb795d4258bd60953aaca4786039280f763703323", + "1ad053ad6219459720e464e61ba1165cbbe5210a308e3f80962cc0e4376e38fd", + "1c665a2c6b863f8415c790f358f9138f56958247ed05638712d6357c208708da", + "1da1e71bbc20c3fee4c7494196a98750f0081b55357d61ee7cdc53945bb7965e", + "1de189fddeff2424f521f5026da163632e4417fac1f9336255f0e62614ee5bb2", + "1e6a33aa31538f769c69f26afd8c3fb7c5f0290023735688823d00e5e34f22db", + "1ed8594ffd73dc1116a58995bf378e36dbf065e9af2b003e643eb9933e7422dc", + "1f7544e2474c3c34d37aa4fd972618a6f29b1e84a054ebe0dffc58c8a2e7a71b", + "1faf9206f8f302ab744b3d89effef3a2a055b0e600c874ec05045043e1dfbfa7", + "20191c4fcda51e9e83b7e86251c8a03607d12daf307afbc00d51d85e4e7def84", + "2053f765d67e1e99fa2f2d67b7c8dbdc0edda26a58e6602fd99f9153ec1e826c", + "207798bd670f5ca13a6fe29e77f180432de0fe8e09512311636a9c49b6d27523", + "20cc15006255a0f0abb3e7f4e59313cbaee4bb4c70b8fda8bf24a8429d7bfeff", + "213e926b1edf4eb61706e283cd09447dbb4e7e529420a94918b6dcc6593f0b29", + "2144d1c886bb5f05a02f9f782ab4ef7451d251b76cf8774d23862aaedf517c32", + "2171371427e17c1995dad1d3526082f7b27faa7ee8f8d115d3e1f5a29fe1bba6", + "21736847d7537ba2fbee1d21f9e7c6f55dd5c9df2239977d889770ddf12484b2", + "225fc26bfe12be03306bfd0e7e321879c3e0515185c5941d4162e432fe6ec2a3", + "22af0dbc179d12978f8a50de641f892441c0fa6ff71eca94e9a7d4fb6d805446", + "232da4dad7749216ed41234b7490922221df235896962a2155fcb14e975705d9", + "23466412690e3c55214d93f7681b36635a8e1359f0ede7506e7c8801e12a3fb7", + "23b377bbdaf00ea187f5864295d60d63f998e9923abd69ee7308b5224d0e37f7", + "23c0c827d7f0d4742515d5227c112062f3eea5b8a374fae7b9f04a34b8a1ddf8", + "25438255a311c51c782bd1ca4591dbe9229bc3be0160d5ef35b016acb0850b61", + "256f6415a7e248845e28ac5e7d542da9867ae0f5a8765a3eae7809291bec1e3a", + "279cef5e2ebb26ddcc87c670075598b35cbc4366b39e9fd22c16533fd9f6f79a", + "28a15d04c873a079c1d06c470d2d77b8e96e025c30b859f176dba529fdcc514e", + "28e0ea4b46d79646462b01eaa2c675b5872098bba0bf3d66a75d1c1a750261ce", + "290401966fd54d78073a1984e84efd046b150ad5b91413e6f5bbcfd35a27c19a", + "294c656a8b36af0bf9249985a8fdac65c4530e7c81d78a0b226f721ad6f9d2fe", + "2a018927dc86ad3fbecca897ae6ec63485d334434829d3670fe94776f8444bea", + "2aba3b30872a625774dc16ad32767e78fcc9e0a1e8e800f976967682c2fabf89", + "2b68052385906b5436a650c00b9c3ffe0253943c3a86a40187fb964d4fb2870c", + "2b7e6e58aaa56a83b488a80cf01f40bfd6fbd5941855bb29c2ce21221ef78374", + "2bc8e606bc7f0a2875ad9a588f3e7a0aa32df858fe5542ff8f7c102cc9c9f14a", + "2d0fab9e61b78cd62d1ca7d76326481635d04d61812c74ab9cbcc2b0e0a3ab1a", + "2d666e1b77c3039e2574d2c71967860a070f2166462f049a3245013d8bb9ec71", + "2d71b5cd66d133bb09dab6f0f35dd703104df4b77255a9ee1a8315d1ca6f59d7", + "2e247e7348b1c46bfa9b0068a40df1f45a4ab4d9c7ac366e14d0a9f99e7d63ab", + "2e7618499962df0d0c566b843a23f7f4fb31301bc9cbab802d427206f496cce8", + "2e89f06988ec2b92a1c8b18b2b3697ee0404bde77298adca7ce981dc1b1a7417", + "2edbff51b1020474d5e181f95bfd43c6fbfed555f6b5d5687917c0817d4ba0e5", + "2f08ab47ee8e608031042183ba73f76856f60fb0262a8adabbdd29700e91da9d", + "30928509ef611054e0fec9b07ce84529aa3afb560aab8a0afad741c493c3efc8", + "312adebbcbcb2b79ef60eb52a89d93e0dbced13464e3bf66ecfbec402be9c7af", + "32bb5ed54dbb72237062117185d38803a2821325ae9186773df5dbd2ee73155e", + "3301879024b9c79869a02253edec7b100d7677fcb2023acd6e9ae7c9cf46308f", + "333fc8cfc5f76fd98905846b73548b0023cac4f5f10cf49d3ea5efd8e0c15fff", + "3376c21147b72dc0474a3899c9eb74d175f9b78d228913dd7c3a27bc18f6af3b", + "34a2ee666ee8be188582c8f7c239b6ff96df48562fe53f34fbc9006c61b87b23", + "35f48e0a0b32a3966c94cbbb83fac82d9d6c89b6ae9194c446e085b265ef05b3", + "36178548038fbcdee704bc77af6a77590b37e6a3915480091ac1d77d9fe92a59", + "3622d0ee6c6d00a746bda15fb39d7044c31bd30e627a4a7ba5737477cc2c788f", + "3655a761324c8dc8844e8ba8b9a4772e2453ea55f7c1520925ef154dfd43df4a", + "36c71852489c6888128d3ff5c2e62d599759c400acb473bcc0a06b2f9acf066b", + "36f646f44b39a2775ef7f08cdd6256eb68b045ff9d2ef373e8826165c2fbe98d", + "36fa82aa1f5bee9ced8fb9c2bde5246529f5735b6aef2abfa3398721a222afcd", + "3718724484b8054be4ba240b10122a71068067812d05e7b9963932051445168b", + "371f27874971dd4d10050be4bbbc1e38e9afe1d8bdf5de5bbc6b3b480334e7ce", + "37487957af8670032f1ebd2698dffcfc99b5e2931094bbce10a1f5a41d17e5b6", + "376260250744115f58dd61526c1195fa32865622d2bf0daab219ef3491f00a8a", + "377d211e4bed8b5fe1de5384be5ba8ac62161b150784f0e7d2b6dc3568e4d508", + "37fdb61efd2db8d7ee146b537977800e6c42f93a35f1b452a8a76ae50aa2e1a5", + "3802165b910c46febd084599f3ad3bb68e7f94946aa316b3be4077dc64d6892c", + "380464d744f9af313aefefee8de18cddcbd32aa3e09aaf4ae13b60c0cdf2c535", + "385b3e8838fe0639b71bdb89d173cde82ce32e8218e8a6900f7d544855d3402c", + "3864afe56d299c4e93a7b4b270e10fb22a03024afefbbc4676456fb2bf09ee2a", + "386a215a2dedbc0166ce9e574929a13416acf7edf161c0c281a7d3752c1664f9", + "399ab40ccb7266269c21475263d0e7264b2b983c333db7aa09ca0c3ff532101e", + "39e3ae6e36f8d2befbcd1d844052de9ec49b87d041c1152f23ce51b5fe4a7365", + "39fc414f3d1216defb30c0764ca78e922de5fae956c8f6fcc0c6cd3ff3c03c5d", + "3a3c4ce053b1d397bbba192d6cfcf15b8c08f6e43f2286971a9321c4a1b9f783", + "3add4fdf06b87cfe5ea9fbb1f1a6fdb2b2f53181cb4f4819abaa511222c728a4", + "3b1c9478132c0351de3a4efd39668453b42aa1eb060e7f6a1def9379b74c4012", + "3b76ae93b0a9d50a284a8556fe33d6bd5255ea98d1c84822d166a4e5a2380a90", + "3c8e3b6a6980d85319a45619c74f75689a10d57455f60ad363b0db859fe33cb4", + "3e44df029d4120641875fb004a2a937c49fe4779a7ce5b1c15df1c7c1e3393a0", + "3e7dbcf1d6545bf86b069dd68aa26c7b2d4deaf965dca2f37a2c7b327ec007a8", + "3f27b12be9a2fd631200a9fbd83b29deecde1e760ec54035723fe0793de1e445", + "4136b9be1dde7532b54d1ce1f96598e153c15c3a76ec32b95813cc38b070b954", + "41498d0adc8e023aca47897192364c2ce5745b089f33013b8b9deb69e24b942a", + "4167b16627d6e167a2dbd1be8484490ec0c6da9de79b106634ad88b9b65e93b9", + "4171cf812123e2b6f87fb20b97a1992b79181a242809805cb68c7ec70913dd6e", + "41919df4cca85f0b531324cfffd6584be4830bc8bd9a988fdf7881d30a8dde56", + "4236092dcd909a41dc844bb5007d916aaf2142ba0c7332fbf96947b5b8ee61b1", + "426d33af3a135919ef5d2b06dabf99d1a1c15ed3b1bb2b9e4c2978a5a107b0cf", + "4372ac740d1727b2385f90171db7c748a03a6fa3849940c98f4b9754a7b29d43", + "442548c3a3f961d677d15b34974fe1729a5c922853972634cd4ef9a66a130b3e", + "446f95aee72b7659374dd227a9ffe3150e198e66df6434c3b4d8c9e563c35ffd", + "449d06ce2a53a15c8070a0555ccfe750c1f25fed1317c69aa040aa181a24353b", + "454c2f13e99ab69415a54b5e6b0d4d5f934892e1b5083e568be81894e1ca2a9b", + "464f92220100eb1bbd1b22e9d03626667f349d7ea6f74c23f1bcef7fc88d5ba0", + "47a9c4f96c6872687900bf48c444e916d883ae0147879a5136565d9f781aaf1e", + "4843e199e6ede40f1655b18465b9caba2d2d1afb569e977520703101f30088d7", + "49417479651902941b3e22068b8f19676e4f484938e9068df2d418739e62cc18", + "4a6cab077475699ab9b8df7a8e6207d267d0d2358da13cf6493010703c398c6c", + "4aa88d7007545b593c65410ddba0b240f986229a5b3eaf101d52da62d1d145f1", + "4ab7c6c04df7d5502cd7a523e550cff7e0e5cd589b392491a7331b95ee9da323", + "4b81a042ffa2df32f8f83b5604e4a00759eda66d89a404a2558eafba6d4fee6c", + "4bf18fcd7c9afb43f6ae51b4dc792f1c53af66240e23b0fccb23b7c3d8aebc1c", + "4c03b46384e7d52ff10e415873ecfa8c5c15e6a35dcad5767afed802c3d1200c", + "4c9e5cab65487fb267c99ba75713cfb3a2a030f49e6cfcfb4f49ba84f398714b", + "4d0131e19ef58b57be98408e41c9a58aa60c41ab4dd830f587cf209bd3972720", + "4d25e2d4328ed58c8f62e38711768a25b57fdc929210950785171937151217ba", + "4e5b04961304eba5fdde7a3060e1a8a98a4a72f1f260a71090f499a79769aa8b", + "4fbaa3eadac6794630ca6e6754e7f252387e419761876d548179007e6e70bdb5", + "5079ae092765e2fb5b517d123bd809195e2b2d26b55db1509aeaeefaac437113", + "507eb82aa5d9d9c694ec304b6dfc7e339efd292c0f36d8ad69de93022801fd97", + "50f8c0bafafcc8b5c0908d4a4ce21c87bf5b8dc7524cf3513aec538e4efde930", + "52bc6084f0ae006eb0c52083a34981493ce7918c8d02d4b5467b92a16cad52a9", + "52cf2ba1f84ccfbf1812ba17714ffb951d5a5503465df2fb7e14fd81930c82b9", + "53e0f8565233b4daba347cd65931b0d03d3be14b00b81f53eb19e8732f7fad75", + "54276b11b5dc801246bf871ed20acea86c35bc3e48d65668292c54894cda707d", + "5573e534e62463125b164ba4d53d7ff101f2040bc3e43ea9f8e66e8a78a66432", + "55c17cc62f79dcc544b49163b3e763a294a0b661482408b35ec48b2afc4eba4e", + "5797212ef801c231eeb5e6157caeb769c9f1c954c945c6e41f6b1de27fe6ea5a", + "58157eae91b262404e5660758e41c875ce3d2823b2e4e5fc30e8a3e1384561ae", + "5842b9664210226005965140e501df8cfd1193dc1def74032e3fb54e6b9217ef", + "590d891c164938e5e47d3dd069e1c81dced1f8a6e52a3020aa35ddce011f6dfb", + "599decd9be71c7b255562b1d9623c90819211305e579d8ad4a3af74c5ce8c4d6", + "59f04a0707f4139930d13192eb42ef9cf0a5193fa98f46e64ec927d3bd34f662", + "5a53a1e8f981c816e78f88933c5d2cf7aba4d9a9a28e953315d817abf3e937cf", + "5a8eec221fecdba8dfeba990ba2ef824a40da43c4167f70dac07ffd3335e8fbf", + "5b231a214ee5db29e6857bfe09c6a36d9dbd7dd2b85aa25528d47902099bd8c1", + "5c670f5657e3227b5ec3bc5199ed81e653deb046b4f6fb539c40281da5887418", + "5d8536a1f3f9d4054766e67598a6212a995f62d36f57fd8241d4432b800d429b", + "5d95de9ba01f727e8fad9c3476b8010bc5753d93b9f775ec94e307aeaab0488d", + "5dd35ce96ea763c051ec4c81442ea8b847f0d357a99fd385ca5b09e9cce36a22", + "5e1aee289d761d9dc1711bc6b3c97c0eeddc16b630c815839a156e42e7f39195", + "5e4eff8ba21f97afac6f8d2f6467b6e285e21dba4b21c6bc0f58d2f086edc2ab", + "60a111176ae35f70f9585afa440fe7f3d18977376c267480fa3094c3393adf62", + "60c84890cf7803f6ae595c3566b6aa4176405f2a8ba83c0f6cd6e000247ca271", + "60f630ba7d00adbbbe73b85429e665bd3c041464321cb346d27774365cab4682", + "61aa27300c1a31db1051074f03440d13ac491b0a5577f0b3de363e5a3b189bd9", + "6206882c83f707aa1f3259cd8cbc1f890f45bc4a85124cc602dc3318eeb6d3d8", + "635c92d494d7796b0c2f484147287f53a89a765e464d054cafb688b9d96655cc", + "6361a57a00da67d6102113b3f3ba0bbb7634c08cbd2096db4289f234380f6778", + "657d6b2557b1397c99b24a9e5777a3e28bb62b4a942ccf9da164c08e6a655e0d", + "65b892cbc5957bc2cc57375a544114a6475b8d9c274b5204d6e0d9c1b9cae15e", + "65de868c437f8dcddad0e37f6533940fde34b39148b9f63104f41d8145610798", + "65fc6b04f85a036c86076adfc04b741aedfe77a79a90db37b4023174ed4604da", + "6728ff8f1d45daa81b955edc59625766ed7b9e5b26f65edeb3dcd646675cdda1", + "69a10964b21da0fecd5c463c6816f544cb1efb299cfcd7a50a4b3a8ae59a9e0d", + "69b1edf50126e222eedc15ef65e9aef983fb9d340ccb13cb2740f509cb7019a9", + "69b58c84600892b4fa2cef6512cc93f04e14415058a2e2275bfdbb6ffece1495", + "6b71c1274c1625f9d19ef6d1f7044365d9dbff6bb252a6445648ff98c073baed", + "6ba83bb33eb0852b8a304b335ab26c5716a00bafeb0b1619146263c25ee0966c", + "6d4eb328e7293fa796088ce8a44db34d6e177e36d4bbb103f369859a08842c5b", + "6d9ecd81dd1ed4c5e6fc7bcec4e40d9097f9f818422e122794aaddfe260780eb", + "6dab1cd77a4ef3a9aee45a8537f58ab30d455d55700428a5ffab4022c2ee9d7d", + "6e29a3f512c9a33928ddfaeb68dae4c9ff946678744704d45ee053726b09e5e2", + "6e2d61579abd8f5acab73369a191a425e369300e3de717d6b5e8b1c79cfb2fae", + "6e6db234d66d1aab39325523447331e08d1255a7d81c67c10348b3abb03f24b3", + "6eea19a0e64ae3f4ec5dc0e5178341635ff858ec9c2f935976c054dafcdc05e4", + "6f76d0be63f1094f195d03f468ab82e36ae3f5f846d96d70812668b5d8e2f7db", + "6fcf3251ef3b9a95d9d2917e3604c3e2c1c05d2f4004fbe0686920abf3574575", + "71415a4b8c2ece02ddf3e9f1abe357c30c11029627c5510ec937582f36eb2939", + "71de47b052ee81350c5ad7224bc7dcbb5be7c4fd375f0821a479c31d82c9bdbc", + "727ca5e770a26fb5fa53155067cd05cc54bec3392d6a1856765f966ca660f9a4", + "72a6b851451517c6bdda780093ffb0e1f713db1580fb466d1dd390c2603f1d23", + "73e64ca24a747c2550b8a82dd413d5fe30e26f03fa95771d660c4b21561b7877", + "7498d1733ee3bfc7e7cdb11d6ad5a65369420da5ed5db9c4d752ae710f7e6e00", + "74e57ea458f6a044861dd23b053a31372df73e0b47e139b1c30d126edf775ee3", + "761134143d39da9361ac778266206266d961c8a76d60b063ff66903a0c51bb87", + "77e531ba2e0fdfb9d13b55ac7c023bec7acdbc08a87322fbec534cdc5020410a", + "77fe4e7237e2a146bc81019079776768d43ca9d55e19baf25f6db7ead3cb3fa7", + "7a56f9770759b68564c8bdbb38baa964b4acce93a3687488b18bf4a82561be32", + "7c419f28cac41a890964171a7ede35e4df22f5166d04b34ba2f1f5c530e895cb", + "7cb35abb83b346ffac1f0a58212082119a81f64f5cceedbd7b8af7810f2ebc07", + "7cfc2744e8e2c90dec0a973312a07f72b33ccff72595d6531f66c31ed099c6e1", + "7eb1f194992237cecaa642db7c5140b22a58ccf39c3aea4a21afb463b58b728f", + "7f60c45a4a714bf1066d53bf7d2b72abac5194fa5979d3aa68b93b64c9facecc", + "8243a06a68ff61d79c805f2c45bcd25fea70bea8fad91df198b432ab470ce9f8", + "8350ee6948c386f427a68e9cce71b828dda126548c1704c5aca0d040de3e1ec5", + "839775ebcdb6e011391fc3c3a66abd24907fcccd0d2c939b5cb390a65fb76f07", + "83d122ef72fc3ac252eefea7659e06a26f887d24374c04a5d750299f76aaed7c", + "843281e77ecc5a5210dbcd0b0b98edf7b95537fb6cdacbfb74fea2f8b11d8a51", + "85044cb38abf6ea983bce7170966ab3062fcdf01b28e25048481770a3418b82d", + "852cb696bf9bcb0a2c8c25bb5d111528a92006f3345eb8a47b3eef0efa343d72", + "85654a7b2d601c501d06ccec3bdcba969240c6862c59a34fa4ee9e3216a6f978", + "85797e1a9396533183d3551f5f62e960350b2cc8692881077242461722204027", + "86f1519103b9098184c2719a2553fcb4f24abf9635e7db4f4e09808b8af84f2c", + "86fde29349d8159afd645bd5b7c5556bf74a7563f5d27f5252cce6523681f6c8", + "87100034cff240e23384afe44efb0b09e90603f382896bd58c186faf0fe4aeb5", + "8751179f0094283c5cf1ddede063340f3a84784459a23264adc86421baf930a9", + "87d27f35acbb6f4734a61bf3a13edbe579fbb5d4ccf246ab2378e24b7558253b", + "88b7ccfb5fa2d3d4bec816f96e0a2dc5a15ce4ea602df297725a58dcb0806732", + "896df52c27a5c01769e1e6e9fbfbb795f43e7de78ac34f34a40bb37c147de197", + "896e19fbf4b3372093f28200f82ff64bedf1d38b1f7fe349d998496342162082", + "89e41d06c9b85a5e85183168ae099df4f324852eed8ec1314e347767563fea96", + "8a69885fdc86627e9cf7ec2880a8c8348b0cd02df31962c0c4d2c389a1a066df", + "8a8bdd653602f2f158019a5f439e3a899cf0a85921aa14cca082d6e603560909", + "8ae4bf13a85c96359e899a85b465784a6bdc84f8f36315f706c34d955beabd29", + "8b6ae722b7977d64014fa0af8f179df23ff208fd2bfae650d6c2277dde6979a3", + "8bc2f96c2f5cfec2923d02f631b6adb72273f0596f19d1de7b39d20acf0b6b12", + "8bc5b83371b1c90418b00374bfd06f8db036885a988bd6833ab0c06fb1ea98df", + "8bca1c6617cbbfd5ce3cfa106987cf26130f8539c26044e1a22a2f51fb1aca1f", + "8c05750c1ca939baf32d25d1f2d92a61800b49a76426ec28c1559d59606a61eb", + "8cb8ad276815c315802c4d97c5820cee5b1077c684875ebebd549432c2c4a812", + "8d091dc0975a50221f3b19a0569128ca17b1a52d01fe047e380b0aaf0e63b52d", + "8d314a3c1bf7b6c300dfecf79320d58bf6d2c23fb174f92d5c64d5e8542a96d7", + "8fd961b8dda83744a40dc807e6d7d2a3ea9d5d1bd799c1e1d5b2fefa666d6ec4", + "8fe2c0bd9c12919205e8c8b29ea61bd96d89616e9b7498b2ee8013446d5dd10f", + "902b91e3a5080e9f94da80889ceaa51c5f31d964846008622b6ddbeb07e28392", + "90e57c604e72df20a9694ebcd97e3fd9f00c5e5d8ec239e84a7b8a0c431e66f8", + "92928d4b346d02eac80d60c98e94f9f1b113f7d01134383db932754c4c5bd3d1", + "9393b510d59ccd16774e17789b3a01f90a64554dffdb9eac197d050c0bf5d09a", + "954890c63eef6f575f68e4f37adcaf21f4a2e343bb291d0b5585d0ea6d46a6b3", + "954c283eebad88edcd070cdb36ab55e457424f3fcb9210f59386cd3e4ee4e74f", + "956fbfd3ce94d4619ffdd9194deb85dc6d82e89fa199cc978e0cfca1d3e8ad6b", + "9588b1651ce73d10cae6b77ac6b991868c806a63979dc3b4be971f686e9723b8", + "95af0aec0162909a0addf5f0002455abb2bd61577e28562a5ed00c2260cb8fbe", + "964083a52374c2723dcb95bfed5636587a298497bf58e1183b60af055c46f397", + "96909c33a461e46a0d938d888e4371d1295f04535166bc16c9dfbb35c9d552da", + "9bd4c58f5a4d057257cae54e018f4eb88f9816f3cf7f88949345453a70512ff9", + "9c674f43e5ec1a37fb7eeb8264a5b4d785f17f716f1897c41616ed37333ba70a", + "9c6faadf417db194a1a2e68cfc6a4792339ca759edfe44c747fb4c516d7a66c2", + "9ca9acceac4275ff088b9bcd89d78019a7f71f284365c0681882194a8af11390", + "9ceaf7a3ba12f73c742a6f86a81c708c03240515b9cc5b89cae86ec4601a744e", + "9d8ede4ea688ed3e8334867c8f34e34e5c070ac993ae9bcf04489e7a25aec07c", + "9dee4ee61296b82b98e8972d47e802ee8ac8a5008b5e03cd5b130d0edfa15c77", + "9ef46448570d1e27b98284d64a08cec2d25a8118fad8017d44a99f4f6c061c47", + "9f1bd954340e3d26d52465ccbcd4b72baa30a3ce520e2a6e4358b5617b176963", + "9f585ccfb9a92e299ff0f76a9a777b4c5eeec4189f4fa9ab50c1cf3d74c21e05", + "a17f4334476d179113e2db7a6dbff71ecae86b10dfb2ad7fe2d8cc9135b4f2a5", + "a3170ffc83f7e0cc197350a8d74d85500d474673998ef1c62f1ddc3f364eb53e", + "a31eca43deeec05441c35933fc20b48f45b191042da018d7f6d97232c1ed9ce3", + "a322d6d1f0b71546ad4e9f746bc3fbf1742df0ecc431dca06584e44f29328e43", + "a35f0920ead6068375bb19cceff473cc971e9b578a0bc963fe7e617e2fdea9d3", + "a43f30a7fdf9c7a1561c6eb9aaca66890cdec58295265ab8d305ff076e36e0a8", + "a467e1194c7dc803eec26bc9535be71b58444bb5748d179e2a4e5e0662d3e0de", + "a49644cf53180202b00609f3cddc005a01e2086bb63369c711ff0950c8f8c214", + "a4b7c79b9897333ce6ceb761dc6854a40225ee9211a54101ba64dcbc92e0a6fe", + "a52945e1d0442f75f9a004ecb76d0e2b17e6137b124e73953d23c99aefa7749b", + "a52d52c4694251b5798e3478578d4d39fe1e964a70548f6388cd3c5e5cdb0fa7", + "a53fbb7223f7fdf1f7b641c51f0f8005ec0f32cfce74d9276001d8d8ea1151dd", + "a59577e4bbf52734f1931725d54881da6d66231bc9fe4c238d840b8c5444fa4b", + "a59ba23421ebc70cc23935fec64df1f948ef7e38c9ae6ae5d51491aea195046b", + "a939547a17e2245da625a539382110eaadcb4aa5b5e0821c15d6ae3a017f55e4", + "aa1da523736c9af8a5c7af126aef2900c7e06c0ba501113ecaed22f8982f7cda", + "aac283530a6c5a5e222a951421b18af4933a1f893ddc009d2e500340dd5d5abc", + "ab4a571a4eae6c30751bdee0aed400dc75363f94dc0f4b7782cb4556e83c23f3", + "aca0c3bd9fdcf931da31abcc3e4ec560120c83ab86deee1e627b5fe8e06b21b6", + "aef1e30af94b53299662a893e7d3a08099e1a9e2431e39096bd55d3eefc3953c", + "af28f08a766c0dfa0650194c27fbfb2b9b057e676e959dcde8c0dc04ba0372a6", + "af680a60c99d6ed6f75252add7208bbac1f27c4263be29008866b314963f9933", + "af8587819a0c45ae5a25878cc8bedc571a5a14d719ff7ad25bd3b088af294b3b", + "afb57d77ba6a109f6435059dac6617930c19567b3a9a5c690932e04ef7b7d45a", + "aff6e17d88bd7ef94395eaa52ed7f8bfa1a84a322d28c1535cdbaeff3fe137c8", + "b015523bce4598e2b2d077d075a5eed6fae4235b2851862ec3edf2a2a454b89b", + "b018d81f318adafd8aeaf9997b18e1e23da168b5fc28e640cf50d570dcc9e315", + "b10f856b83f430458887304c2f3609fc5bf70132d9546fbbf763183d9c8ec5b6", + "b12d970acf0fb01d206f2547177e9c4008abb0d81d20a8df2faf566176cb042f", + "b1daedac8c8e33c5708b2cb66bd406e57633cf05b6bbd95c11f9e752b35da4b5", + "b246439be3988b28f0fee9316443e3a92a37bf97ce2be5c748f04ddf800418ff", + "b3ae5672ebbcf36d12467c9fefcd760018e4b5733a34c41bce09e760bfaa9715", + "b50ce258f9c2c3a1b1284b7a573a2fa4a11ebb787929d39011a47002463ea7df", + "b50fca44807c252ef01dd95698262033154f8707905f779c0a20abc6439ca0d9", + "b5a7e7146a98035523ef6490183cdde177444103df52e1217bf8e72ada4d6328", + "b6324c2d34799565940dcccdd23e1360e974a46b44c209a6a7a9fdabf790d044", + "b66654f95bae5b30f1a326c90dc930b954368834385eaa21725571afe94b354d", + "b677f0aee329cf1b061cc0e2601a9c33218c5c10c0cace5820236313a0825d3e", + "b679de5b8df5d6b74ce1c8331418e05575117dfb8903a5381b6e92d9eb0b6a4a", + "b68d92aa8fccb4f75dc0ad09e9b6d567f7d40d421ae5321b36bb0a48236e598e", + "b72fa8f582c7fcc689962e8ffd5f8fe3f837aba4cd11ce7687532498c11fede9", + "b7761c8e81ab553d5b43ef3e570e6639d8d4fd3d3b217198f7bcc86b0078db4d", + "b7f2c74cc6970aa943740ba5c7fa947e4dc991233fd35e51412ced790d20298d", + "b8c14ae75cbc0d0dcb2d256815e6e4fab110ea29a3582201c0df5cebe47544d5", + "b8c87db8cf709cb8f4bbd8e684279f796def8980d809fd174a79690113de7207", + "bac19e11a6db7315a718dcf8236ca104336de292ca5553afe82f9418239b4271", + "bae7c6cd0a3c6e5f44834d64879b260dece1a4232f3fcb1f01756b03391c8831", + "bb5d7e079e9ab21431f60dd6c4495a3984001039620203c0962965e5f42de079", + "bbd5c9fa1463d0d789c66b421b274e424b1289de879bc2095a39924521dcdf06", + "bc107f5bfc824a5872fb29eb153fe68133664f84a0bda0c1f040629cc5722e55", + "bc28efd0ad227d98655ebf16a86d39e0eeca17d2f7e4a1bb14562b9772777332", + "bd570017cc62932d88f56095bb4fc99aa0d3f718317c089808b4649309f57322", + "bed8189cf521818dded6793b749c12a91df30fa033b1d0d628b74b078aa3d17f", + "bef655041d427966b37d503ec3ee6a608a397e34e50b3faf19c7c8010d32e9e9", + "c0b19497f20c96ff5a998cdfd8fe901b2636548e67f7521e17d8c1c3a344b119", + "c1fe0dcfc510b321f89d86e9e28d959be4b9f7191bbdfe75d94aa00ba907fecf", + "c29fa01c7d7c2300dc9f248122d101f2ae7096dbdff573efbb51b271db18a237", + "c2c4d1a16f5c13da6ead578d5ea39e79319c2fbe4b950d9a9eef447f0d885985", + "c3207049b4fe55b2130b223f9d327b21f11dc51c5ca20eca2a8d75c13f6000e5", + "c4c4cd450d60c5748c8fc36d10c75f182cfe952711617dacb76a0b0e7391a35b", + "c53776402fe96a74a9aa02b2ae732f279a3ed258aebd35e7f7347da28491856d", + "c5608d269923a2a3ee69d360c52f71f1eea591a603304c908ddb465f737c0782", + "c56cf2d189daaefc764ff1bbcf667d4625be73dbcdfe9c42e0f503d59b5ccd23", + "c60a678a37f48d744f7ba785c4adc7de65317982199801ebd603090315c85ca6", + "c6244a7b5686ad0e553cd0fb349481c14e4390ec7a65d414da7dead452a535d2", + "c6dba1af7f4d12fe2720b67f0ba6ce4984cf9d47c6d288f2467681d02f5af6d4", + "c7418675ff86e0833d60ea80a8c8a0f9b99d239421ca68164c6824a99a276f54", + "c77af3500dc3ee06cb5dd5ef1e7cffd70a3c7bca08b9c00196884cc3dae8db90", + "c9feb7939ea3febccdec91c170527060563ee0dbf0dac5b75092ef5c5bcc9e6f", + "ca67eca42b424c6374979d63df4eeae00381446c7c7eea0934a3a21b9352cd9c", + "ca83a160910b077d3ca4b6f0185b4a99549d8287d187a4c204ffcb6867514887", + "cbddadeb108dc36a2cf5e686e2cb3a143946afe47ceb15dcc185af4b272e3777", + "cc47d275990836af665da5f4dcb0c118d3d4897d376ebeac53ba6f207dbb8a64", + "cd015dbf90e521f2443eb714d30785c1d21240e44dc10e1f8464ff146866f6d1", + "cd7d0bcd5fbfa262877644a5d88d8e2b6016e6cb9a880761c1cc1823c4e6c621", + "cdd8100bc160c8c866913e8a033cce5f402efb1fc1538247d4cbcda3e1216659", + "cdef8f05ddad0db32cc966def25db6898a05c42bd15391b77c619eaf5ab2a904", + "cf4dd171afff5c96a512d37c25ea9e292f1cca25a2d267b2aef735d51605de17", + "cf99daf73630fb1d3448b7bbf0b4ba0558c886dd8ae1fd77a38a0b610fa1ea3a", + "d0a9e8772c316ac795de39d315b5ce38b72472b4df16d6ea6ae473c94c853a0b", + "d0c4b2351028946098f41c3fe28981d2aee8e3f5caf4ba4fcb12462d0c2fa12c", + "d0d11e3470945260027f6fefa6a4d0a3cbd29d2ecb9b9420a14ff6fa952c957d", + "d0f6f9c2fb93b869433b409b0093f0e15e226e6d750632a90a0e6e970a3464b6", + "d1ac7a2471d394057a588ec47c74a0b78e7cc2f495bb1585b184eb72dfd4ccce", + "d2615c76b309b1437f0d1f0294b8e88fc5fe3e9259a41a9ae4cdf137ef73609f", + "d3723031d231e59cd40ba8632e01ec7604f5197a48a45898cd0e314541331c87", + "d440f8f0db5b04f2ada14de102a3b3a7e3e774ab487a6bde0275a4344d62dd16", + "d584d77a56c70ad17c27a192efd074d4f5f74452f2c62ebae6b627d80ae063f6", + "d5b1bf93014f200928b481640c3ae57597cf73a0f1668a77e07ddd8683a81508", + "d652ef33b9ffeeff77ae17aeeea979ab8c1d403431bdf245867abf4566b54689", + "d66bff5b50f3ca994a03faa77246bf9e3a8193a4903fec25d5f56f7fee31611a", + "d66e373e83c0d5bf391ef83eb47b2ea4971409df16bf2d2d3f90e4819c641cc6", + "d723c5e330147c034be5d384e3c539f323c50b01326b901b3f78ec8a29d13b41", + "d842c9b772dbd601bfad1c3d830df5aacbb685507247666a80d8ef7ec0ab4db2", + "d92d42c9d643429671fe1f7384014ca15ae89ce49be0c5d524d2172938585986", + "d9efd2aa3c0a57f3bb93916653cfee963e4b1141917d703e61509d8d6dab7c78", + "da5e94dc767915969e1c8188d29035197a22ba0b9d8be1cebc04e5a8203646a0", + "da96925490c07aa86ee90d9b7c31bd59fdc56d27c891cb6728059322add4260b", + "db258468bee6ca4d5ca59334c4e45ca958aa6df89ceadebead6c04ec77feca35", + "dd2f29960e09d45097d7c7324130b96c27ee8483c4dca1b0045dd58a158b4838", + "dd794a7a3fcf1fe4a062e9664df3eecc6f45f0f15bb1672fe552f117acc1f646", + "dd97f6d3742257341118baa40d95ac216a80d20d39d47ec9c17e37234151d783", + "ddcb6f80de7e1ee656dabcf2566d3a78fb97806bbb6077a66d0b1e9d8a3e251c", + "de0beb18c2c3a2c895ca5f7a01665041577ef8373f8037591a1b1bd9b906e13a", + "dedc28fe961870e02089cb419fe3be918a27dd6568af8724f7e6879f749cd029", + "df67a5c802fc30ab72aeababbb0e93dd8e5b42cd9646ebc54ff9f47240c6f361", + "dfaa69ca87d3288942dcaf896c9ba8b8a6b0ba61006631efac7e521e8b543c9f", + "e128aa827a20a9ca1a057cf1991d49872999d78244f9f9d126399e8aa70b4841", + "e170828ddffedb42c27cf62dc347fca5ebcde643e3b4585691189c321cfda9c4", + "e1bb15a4b9aa87ea3327bde96322b226d9e407fee96824e7281a4142d9c32592", + "e1c5715b0a722ae7aa244fc75ea30dcfa239f6af640e6ba5d4937d22c85cb982", + "e24cb331e947114a105313fec3abee7bba02e65792962b11f789ff65354bca55", + "e34faf75bfbb6ec5d6fea460d34a300f33d61d5544249ce0adb97e3c9a8bf469", + "e417c01c804a4b846fb4ab0bc5f5e635822dd469814b6f454344a371a0c54846", + "e4a3df05b8fa77155ebe3f0e59b48ed9727f70e8624623cb69596dd0ccb2acff", + "e55e578cf3630b4265a38473546efd00faeaccae3b69cafbd46c38b43810159b", + "e58a39b6c5e754b1bfd881b5e6b3072eb678e7909a173dbe589b1f1046cc3c61", + "e614a0919bca602cb5416a1b69b025e7679766f28a630f8fe36a044c9b6c19a2", + "e730a01ae3f75be9423d32d2fc782f0250689840a9f35f552f9234956b91b651", + "e7cbfefa2b400f6c2a4088c06e774c341f1fec283d7e6de39d6aa38f8fb16eff", + "e83f6805a42db291bb19df80219ebf6257f4ca9965f7c96acf9c84c19845466f", + "e8b62f0abf8484fd663882a6423adc003b27b151b2e081544cdb328480b0efec", + "e968c0a922d5fbb0063602dfac63455d45924717d817299133a56a73fa9c85d9", + "eb68932633b2c16e1761b398a52354406182ae85cbc65ae7e27dbcfdb667d6af", + "eb961efc66c3dda1fceeffc0f5dcb270b6f354484fe5ff983e7252533d2db5b6", + "ee27e25f902e654569ef7a8ded0e028eea1f7b48cce472d02537f9c58b90f31b", + "ee9c47b6572c11e77808aa5278255d5baaa53327776a2eb1f3c02ac21706667e", + "eec2faad73968e9d8df0cc9171b24b238365533e490c3c1da68f2a01b3a58dac", + "f01042ebfb2d85430a7264450902c9d7f8f208343cbfa398de5efe062158cb10", + "f0262b9ee515bf10133044931f0078e678627bee08a7021dae6532b1d10652eb", + "f07a5ec90031146f8a1b92c877a9ef88d90b67725b687daa47850afcb8a4f5bf", + "f137e08187e31a78bf93686f27df66e24ba62fb370115cf099ab3452b3e84934", + "f18c1737e19dbb670db3262115d9d640533cf10ba73f8196ddb5a7d582744feb", + "f194576d56014f50ba3f94c92baf120c3dbcebe176b661154484c2a92b7b40c5", + "f3088bbb09667126643e781a17542b06cf97833d29b414ff29cdfc0c249b0025", + "f36b21cc9f9651ab59717c67501c154929fdb69d2124fcf87beb5cb4eddc385b", + "f37badc006085509e7f393cc62d362f41a6300fe2a54c46c6e0763b4afbfe7f7", + "f3bc1c4b3902b647d6660ae83b6a272dbf80b85cbcb7b57d74afbe94d6d754b1", + "f54e7d6b2f7b5aaa89a466187ae089784a605b946d50d221e21185b14ad2e5a0", + "f5579f35655c4c99b49931e63782e64494470ac21c1693fecacad3d3d54291bb", + "f70bac72a9d4055c60922c63e2f2cb3b77aa90250cc071a96f67d3ab658a4a12", + "f7899c890521825ae7f31b8c107f2ac9cf8e50097d4c06b92e226a772ba7b925", + "f82052e29e6ca52c7c633334e5c939674ca671cf5a592db59f882e88c5bbfdac", + "f937b82c14d253a7eecc1fc8effc545517af158c38d181727dcb8205e42a27bf", + "f9576ef7044564daed5e3bd01e196df3a69109244971a106c44d8108f286f08d", + "fa65af636e0484758f08f422fe84c5ccc340fd7f33a96fe5bd63e2a9c110fe6c", + "fc969058a53f8827440a2d1750091bc41aaded65e487bc033c81f2140b997321", + "fd18f20b9ea23bcb6fc5091c74383ba65a8b58c19f12da01c74b82ce5f821843", + "fd5336c29326b4bcacb4c55cedbe14a1d5e61ef269102110f643f665d1f8f03e", + "fdb70a2a5773acfdd00811abbadb690df8422aaaed7bd93893f576e2f5704bac", + "fe2303f90cb10eb75f13d551f88d2103bfb4f0c83682f807942f4a768b6852d1", + "fe6e280d14d7dd0fe23a487c684330c060896eee257e80121bc188a4f6110c09", + "ff5cbd0b1fdca636b746e9d39e0c2e984969c249bd093bcdd886290622f87775", + "ff6bbc3315c35a873ae49a7aea0911da2039eb0bdfd754beec5b119d5ca46b29" + ], + "task_counts": { + "LiftPegUpright-v1": 1056, + "PickCube-v1": 2384, + "PullCube-v1": 928, + "PushCube-v1": 1008, + "StackCube-v1": 1328 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed1/train.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/train.log new file mode 100644 index 0000000000000000000000000000000000000000..47930a55ca59c56491acc59fee66b8df9e8d74b4 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed1/train.log @@ -0,0 +1 @@ +CTT checkpoint trained separately; rollout used checkpoint=runs/ctt_residual_base_context_obj_seed1/model.pt diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/command.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/command.txt new file mode 100644 index 0000000000000000000000000000000000000000..7c9c6d10593a23c9cb0f1fa5d88ebc1e459cfee6 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/command.txt @@ -0,0 +1 @@ +python scripts/eval_ctt_generated_rollout.py --checkpoint runs/ctt_residual_base_context_obj_seed2/model.pt --source-index data/cil_charts_rgb_refs/train/index.json --target-index data/cil_charts_rgb_refs/val/index.json --out-dir runs/ctt_residual_base_context_obj_rollout_val69_seed2 --k 8 --pool-size 0 --neighbors 8 --max-target-charts 69 --group-batch-size 1 --device cpu --sim-backend physx_cpu --render-backend cpu --restore-tolerance 1e-5 --delta-scale 1.0 --bootstrap-samples 200 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/config.yaml b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/config.yaml new file mode 100644 index 0000000000000000000000000000000000000000..22d2cea0a482b87cadab53347ff24e234f048a3b --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/config.yaml @@ -0,0 +1,18 @@ +bootstrap_samples: 200 +checkpoint: runs/ctt_residual_base_context_obj_seed2/model.pt +delta_scale: 1.0 +device: cpu +exclude_self_source: False +group_batch_size: 1 +include_targets_without_positives: False +k: 8 +max_target_charts: 69 +neighbors: 8 +out_dir: runs/ctt_residual_base_context_obj_rollout_val69_seed2 +pool_size: 0 +render_backend: cpu +restore_tolerance: 1e-05 +sim_backend: physx_cpu +skip_metrics: False +source_index: data/cil_charts_rgb_refs/train/index.json +target_index: data/cil_charts_rgb_refs/val/index.json diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/data_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..49b12b50537d95327f9c62ddd41910531a6c14a4 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/data_hash.txt @@ -0,0 +1 @@ +1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/eval.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/eval.log new file mode 100644 index 0000000000000000000000000000000000000000..b467e3ff72579d608e7dee787e89f4d012cd270b --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/eval.log @@ -0,0 +1,766 @@ +start +importing gymnasium/mani_skill +imported gymnasium/mani_skill +loaded checkpoint=runs/ctt_residual_base_context_obj_seed2/model.pt +loaded charts source=301 target=69 +generated proposals rows=69 total=552 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=PickCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PickCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +rollout complete rows=69 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/git_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/git_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8b137891791fe96927ad78e64b0aad7bded08bdc --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/git_hash.txt @@ -0,0 +1 @@ + diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_candidates.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_candidates.json new file mode 100644 index 0000000000000000000000000000000000000000..7f50960b15facae65337337e60c225ff3ee56553 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_candidates.json @@ -0,0 +1,44442 @@ +{ + "candidates_evaluated": true, + "checkpoint": "runs/ctt_residual_base_context_obj_seed2/model.pt", + "decoder": { + "delta_scale": 1.0, + "lossless": false, + "name": "linear_keyframe_decode", + "source_code": "spline_tangent_code stores start/mid/end residual keyframes" + }, + "exclude_self_source": false, + "k": 8, + "neighbors": 8, + "pool_size": 8, + "report_type": "ctt_generated_measured_rollout", + "rows": [ + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7270061373710632, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7270061373710632 + }, + "base_progress": 0.7270061373710632, + "base_success": false, + "base_utility": 0.7270061373710632, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.725460946559906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.725460946559906 + }, + { + "contact_quality": null, + "progress": 0.7120364308357239, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7120364308357239 + }, + { + "contact_quality": null, + "progress": 0.723896324634552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.723896324634552 + }, + { + "contact_quality": null, + "progress": 0.6907892227172852, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6907892227172852 + }, + { + "contact_quality": null, + "progress": 0.7161080837249756, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7161080837249756 + }, + { + "contact_quality": null, + "progress": 0.707915723323822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.707915723323822 + }, + { + "contact_quality": null, + "progress": 0.7119319438934326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7119319438934326 + }, + { + "contact_quality": null, + "progress": 0.6006918549537659, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6006918549537659 + } + ], + "candidate_progress": [ + 0.725460946559906, + 0.7120364308357239, + 0.723896324634552, + 0.6907892227172852, + 0.7161080837249756, + 0.707915723323822, + 0.7119319438934326, + 0.6006918549537659 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-efbde8652e75274fbd69", + "ms-df961fe24a43e56d1716", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-484ea653bbca2de3a4a2", + "generated_tangents": [ + [ + -0.04396830499172211, + -0.08343648910522461, + 0.6609010100364685, + 0.3912830054759979, + 1.2738001346588135, + -0.22729763388633728, + -0.41784465312957764, + 0.12113326787948608, + -0.1098494827747345, + 0.16556432843208313, + 0.14648115634918213, + 1.5712038278579712, + -0.33531808853149414, + 0.0032642632722854614, + 0.25243619084358215, + 0.09833438694477081, + 0.2740958034992218, + -0.08344472944736481, + 1.2964844703674316, + 0.07571494579315186, + -0.11336424946784973 + ], + [ + -0.17958395183086395, + -0.09769926965236664, + 0.3501026928424835, + 0.030126169323921204, + 1.3105278015136719, + -0.3103365898132324, + -0.5545125603675842, + -0.061156630516052246, + 0.028589868918061256, + 0.07429073750972748, + 0.156553715467453, + 1.4471940994262695, + -0.296586275100708, + 0.0018916875123977661, + 0.08276046067476273, + 0.09654374420642853, + -0.014714613556861877, + -0.09888015687465668, + 1.5025501251220703, + -0.20295250415802002, + -0.008486762642860413 + ], + [ + -0.173442080616951, + -0.10092069953680038, + 0.5204766392707825, + 0.47542688250541687, + 1.328404188156128, + -0.3512614071369171, + -0.35438400506973267, + -0.12318980693817139, + -0.0074684638530015945, + 0.15452751517295837, + -0.05886925756931305, + 1.3658742904663086, + -0.009514480829238892, + 0.05464161932468414, + 0.18910977244377136, + 0.17883265018463135, + 0.16515031456947327, + -0.11293287575244904, + 1.421081781387329, + -0.26805827021598816, + -0.08112901449203491 + ], + [ + -0.2532610297203064, + -0.10047822445631027, + 0.46251213550567627, + 0.2273329794406891, + 1.3086507320404053, + -0.01985238492488861, + -0.5974724888801575, + -0.06729739904403687, + -0.006806142628192902, + 0.16290780901908875, + 0.02308567240834236, + 1.3942630290985107, + -0.2884600758552551, + -0.05897437036037445, + 0.14326632022857666, + 0.056614089757204056, + 0.13427653908729553, + -0.014556601643562317, + 1.436126470565796, + -0.11865103244781494, + -0.22375035285949707 + ], + [ + -0.2011590152978897, + -0.07051295042037964, + 0.4562249779701233, + 0.23362600803375244, + 1.2472186088562012, + -0.25279319286346436, + -0.5657718777656555, + 0.0708199143409729, + 0.03144624084234238, + 0.25414907932281494, + -0.13450783491134644, + 1.3995730876922607, + -0.28993117809295654, + -0.12428523600101471, + -0.006003204733133316, + 0.044804804027080536, + 0.05686257779598236, + 0.06919267773628235, + 1.4648306369781494, + -0.08783349394798279, + -0.191087007522583 + ], + [ + -0.19128648936748505, + -0.09508658200502396, + 0.4334104359149933, + 0.28650733828544617, + 1.3387606143951416, + -0.07222752273082733, + -0.6596245169639587, + -0.11855903267860413, + -0.004214378073811531, + 0.23318347334861755, + -0.0008311532437801361, + 1.3847993612289429, + -0.26629433035850525, + -0.08290413022041321, + 0.22168152034282684, + 0.07008364796638489, + 0.1424579918384552, + 0.05860574543476105, + 1.4799671173095703, + -0.20615199208259583, + -0.1873224675655365 + ], + [ + -0.14408321678638458, + 0.014559552073478699, + 0.25759410858154297, + 0.10650721192359924, + 1.38962984085083, + 0.3713410496711731, + -0.6855922341346741, + 0.006837889552116394, + -0.05902540683746338, + 0.07595857232809067, + -0.11012532562017441, + 1.2790130376815796, + -0.41708293557167053, + 0.016162410378456116, + 0.1911972463130951, + 0.2482377141714096, + 0.2648971676826477, + -0.3591464161872864, + 1.7145140171051025, + -0.20835676789283752, + 0.2266080379486084 + ], + [ + -0.27097994089126587, + -0.1293676495552063, + 0.2697438597679138, + 0.07886222004890442, + 1.445467472076416, + -0.12780547142028809, + -0.556123673915863, + -0.11891081929206848, + -0.056843534111976624, + 0.18678805232048035, + -0.06028033047914505, + 1.3125343322753906, + -0.36576956510543823, + -0.07183168828487396, + 0.1435416340827942, + 0.07444605231285095, + -0.03573937714099884, + 0.13283783197402954, + 1.7500474452972412, + -0.3045712411403656, + -0.04804620146751404 + ] + ], + "generated_utilities": [ + 0.725460946559906, + 0.7120364308357239, + 0.723896324634552, + 0.6907892227172852, + 0.7161080837249756, + 0.707915723323822, + 0.7119319438934326, + 0.6006918549537659 + ], + "group_id": "ms-484ea653bbca2de3a4a2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7242962718009949, + 0.7209752798080444, + 2.0, + 0.19684189558029175, + 0.21780464053153992, + 0.19326990842819214, + 0.22509369254112244, + 0.20856516063213348, + 0.21041584014892578, + 0.19898515939712524, + 0.20721960067749023, + 0.20830264687538147, + 0.19554394483566284, + 0.22549311816692352, + 0.19481703639030457, + 0.2172507494688034 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.4380870461463928, + 0.4186934530735016, + 0.6294705867767334, + -0.08003801107406616, + 0.5882754325866699, + 0.006411761045455933, + 0.0, + 0.27439412474632263, + -0.43702489137649536, + -0.26345914602279663, + -0.3102925717830658, + 1.1419475078582764, + -0.05133828520774841, + 0.0, + 0.11964668333530426, + -0.04283810034394264, + 0.021698474884033203, + -0.3213534653186798, + 1.444929838180542, + 0.058529436588287354, + 0.0 + ], + [ + 0.18277202546596527, + -0.31313952803611755, + 1.5374140739440918, + 0.9199619889259338, + 1.58827543258667, + -0.42588773369789124, + -1.307411789894104, + -0.1648276150226593, + 0.17999683320522308, + 0.7365408539772034, + -0.2825929522514343, + 2.1419475078582764, + -0.44205278158187866, + 0.3557547330856323, + 0.11787930130958557, + 0.04347073659300804, + 0.5306029915809631, + -0.6201959848403931, + 2.444929838180542, + -0.7078790068626404, + 0.7410526275634766 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.614823579788208, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7115094661712646, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4821052551269531 + ], + [ + 0.2415536344051361, + -0.2574644386768341, + 1.3600168228149414, + 0.8914169669151306, + 1.311052918434143, + -0.13483268022537231, + -1.4109176397323608, + -0.007070749998092651, + 0.30199557542800903, + 0.867316484451294, + -0.0111103355884552, + 2.228789806365967, + -0.26305240392684937, + 0.4046628475189209, + 0.25534987449645996, + 0.29946526885032654, + 0.777583122253418, + -0.535997748374939, + 2.2955121994018555, + -0.6670190095901489, + 0.8460665941238403 + ], + [ + -0.212661013007164, + -0.6653347015380859, + 1.8750245571136475, + 0.7373600006103516, + 1.2695744037628174, + -0.7584145069122314, + -1.695502758026123, + 0.05721297860145569, + -0.04310476779937744, + 0.4696756601333618, + -0.2954420745372772, + 1.9897286891937256, + -0.6554491519927979, + 0.5805841088294983, + 0.17026324570178986, + 0.3433569669723511, + 0.3734014630317688, + -0.8102566003799438, + 2.792973279953003, + -0.8463170528411865, + 1.0759955644607544 + ], + [ + -0.039009034633636475, + -0.7773740291595459, + 1.064732313156128, + 0.712108850479126, + 1.270837426185608, + -0.5491334199905396, + -1.7919384241104126, + -0.5782418251037598, + 0.4058297872543335, + 0.5231752395629883, + -0.521438479423523, + 2.2066850662231445, + -0.5258626341819763, + -0.09100466966629028, + 0.24944378435611725, + -0.2271452248096466, + 0.9009795784950256, + -0.8140718340873718, + 2.6494152545928955, + -0.5153427124023438, + 0.671539306640625 + ], + [ + -0.3804699182510376, + -0.5808351039886475, + 1.8389031887054443, + 0.7141245603561401, + 1.0997881889343262, + -0.1678517758846283, + -1.884540319442749, + -0.3411717414855957, + 0.4830858111381531, + 0.8356180787086487, + -0.8238832950592041, + 1.743321418762207, + -0.3751307427883148, + 0.5275082588195801, + -0.08622422814369202, + 0.23666565120220184, + 0.802552342414856, + -0.8389871120452881, + 1.9817423820495605, + -0.4874984323978424, + 0.4390762150287628 + ], + [ + 0.4497724771499634, + -0.7880887389183044, + 1.4436085224151611, + 1.083756446838379, + 2.2298290729522705, + -0.03008332848548889, + -1.0250177383422852, + -0.6266056299209595, + 0.3751387298107147, + 1.1566712856292725, + -0.6914000511169434, + 1.953734278678894, + 0.08129626512527466, + 0.9812837839126587, + -0.4981238543987274, + 0.6697847247123718, + -0.05150604248046875, + -0.06941354274749756, + 1.9621039628982544, + -1.345557451248169, + 0.43397778272628784 + ], + [ + 0.03950639069080353, + -1.037135124206543, + 1.1041697263717651, + 1.109439730644226, + 1.144017219543457, + -0.07355684041976929, + -0.8942177295684814, + -0.16419151425361633, + -0.12125559151172638, + 0.36935141682624817, + -0.7298048734664917, + 1.4954023361206055, + -0.018926143646240234, + 0.4787553548812866, + -0.3127076327800751, + 0.6351315379142761, + 0.5993886590003967, + -0.5143972635269165, + 2.429173469543457, + -0.6674217581748962, + 0.5048977732658386 + ], + [ + 0.9763806462287903, + -1.1549197435379028, + 2.0378684997558594, + 1.2418556213378906, + 2.427785634994507, + -1.1326193809509277, + -1.5491721630096436, + -0.8937685489654541, + 0.2583051919937134, + -0.10687285661697388, + 0.2628750801086426, + 1.6517728567123413, + -0.21639958024024963, + 0.7581590414047241, + -0.29044294357299805, + -0.08927425742149353, + 1.3009071350097656, + -0.9239004850387573, + 2.5070056915283203, + -1.3246500492095947, + 0.24751555919647217 + ], + [ + 0.9781984090805054, + -0.13759100437164307, + 1.8698725700378418, + 1.4942535161972046, + 2.5587658882141113, + -1.1335766315460205, + -0.6910277605056763, + -0.1434302181005478, + 0.4021325707435608, + -0.12275522947311401, + 0.6571680307388306, + 1.9964357614517212, + -0.1442476212978363, + 1.3201656341552734, + 0.2586786448955536, + -0.38222962617874146, + 0.9950553774833679, + -0.8371140956878662, + 1.8977141380310059, + -0.795723021030426, + 1.505234956741333 + ], + [ + -0.6250126361846924, + 0.4231366813182831, + 1.701646089553833, + 1.6233667135238647, + 1.0861127376556396, + -0.3387698233127594, + -0.9825164079666138, + -1.1511213779449463, + 0.11523068696260452, + 1.5823121070861816, + 0.5840664505958557, + 2.145885705947876, + -1.2145874500274658, + 1.1558246612548828, + -0.5131542682647705, + -0.4715161919593811, + 1.0824741125106812, + -0.7667124271392822, + 1.8457963466644287, + -1.228658676147461, + 1.2112116813659668 + ], + [ + 1.1640653610229492, + -1.1697030067443848, + 1.6035369634628296, + 0.2161661982536316, + 1.4819366931915283, + 0.2569611370563507, + -1.3529595136642456, + 0.41208329796791077, + 0.03121529519557953, + 1.731370210647583, + -0.8131548166275024, + 2.996675729751587, + -1.1173620223999023, + 0.7474988102912903, + 0.21459203958511353, + -0.2133086621761322, + -0.22307389974594116, + -0.6306187510490417, + 2.0490896701812744, + -0.49808698892593384, + 0.8432145714759827 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.014628380537033081, + -0.18207009136676788, + 0.5374140739440918, + -0.04374772310256958, + 0.5882754325866699, + 0.25251278281211853, + 0.0, + -0.23516394197940826, + 0.12562739849090576, + 0.3381730020046234, + 0.033050984144210815, + 1.1419475078582764, + -0.3290551006793976, + 0.0, + 0.04625481367111206, + -0.0026901662349700928, + 0.0754578709602356, + -0.1481514573097229, + 1.444929838180542, + 0.049897611141204834, + 0.0 + ] + ], + "predicted_scores": [ + 6.357418537139893, + 5.67083740234375, + 5.565855503082275, + 5.436758041381836, + 5.376450538635254, + 5.213481426239014, + 5.070798397064209, + 4.92073392868042 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1231780997", + "selected_index": 0, + "state_hash": "771fb752a186163f05a49a9b404284bde55d89838769a764fca1d6d1a9419652", + "stored_base_utility": 0.7242962718009949, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.528373658657074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.528373658657074 + }, + "base_progress": 0.528373658657074, + "base_success": false, + "base_utility": 0.528373658657074, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.6627864837646484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6627864837646484 + }, + { + "contact_quality": null, + "progress": 0.7146708369255066, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7146708369255066 + }, + { + "contact_quality": null, + "progress": 0.2057957649230957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2057957649230957 + }, + { + "contact_quality": null, + "progress": 0.20426128804683685, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20426128804683685 + }, + { + "contact_quality": null, + "progress": 0.7040143609046936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7040143609046936 + }, + { + "contact_quality": null, + "progress": 0.1966126710176468, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1966126710176468 + }, + { + "contact_quality": null, + "progress": 0.6997706890106201, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6997706890106201 + }, + { + "contact_quality": null, + "progress": 0.6655330061912537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6655330061912537 + } + ], + "candidate_progress": [ + 0.6627864837646484, + 0.7146708369255066, + 0.2057957649230957, + 0.20426128804683685, + 0.7040143609046936, + 0.1966126710176468, + 0.6997706890106201, + 0.6655330061912537 + ], + "candidate_source_chart_ids": [ + "ms-8f3d4e1d07bb5dcc7c57", + "ms-df961fe24a43e56d1716", + "ms-e1b6bcb41532a800d3cb", + "ms-0660d5f65df5fc4b6132", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb", + "ms-13d8bae55d3c3710138f", + "ms-502dba1003c09ec60ba0" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank6" + ], + "candidates_evaluated": true, + "chart_id": "ms-4b4cd12d70c0615e0930", + "generated_tangents": [ + [ + -0.300254762172699, + 0.053849607706069946, + 0.7199182510375977, + 0.5696301460266113, + 0.37969574332237244, + 0.132772296667099, + 0.06084561347961426, + 0.275797963142395, + 0.015088420361280441, + -0.04985090345144272, + -0.09546753019094467, + 1.8002060651779175, + -0.07629215717315674, + 0.005778871476650238, + -0.0030481629073619843, + 0.04588718339800835, + 0.15246544778347015, + -0.14120404422283173, + 1.568716287612915, + -0.19701126217842102, + -0.03143902122974396 + ], + [ + -0.1923649162054062, + 0.05242970585823059, + 0.8130223751068115, + 0.5171482563018799, + 0.28202661871910095, + -0.2814216613769531, + 0.06353339552879333, + 0.1968238651752472, + -0.02700730599462986, + -0.2716432511806488, + -0.1141243502497673, + 1.760290503501892, + -0.165985107421875, + 0.12487167119979858, + 0.08824179321527481, + 0.19057968258857727, + 0.31001168489456177, + -0.1510704606771469, + 1.5847454071044922, + -0.10264299809932709, + -0.22121229767799377 + ], + [ + 0.02517171949148178, + 0.022989556193351746, + 1.021344780921936, + 0.5466411113739014, + 0.21092377603054047, + -0.07494795322418213, + -0.29042571783065796, + 0.4393842816352844, + -0.0019921306520700455, + -0.1441338062286377, + 0.04217243567109108, + 1.7934508323669434, + -0.4576164186000824, + 0.17146851122379303, + 0.11672595888376236, + 0.024392124265432358, + 0.1892388015985489, + -0.156822070479393, + 1.6405470371246338, + -0.2620435059070587, + 0.035926759243011475 + ], + [ + -0.20293591916561127, + 0.054996930062770844, + 1.0815178155899048, + 0.7565701007843018, + 0.39579933881759644, + -0.12166270613670349, + 0.010693728923797607, + 0.2836955189704895, + 0.033144909888505936, + -0.19373682141304016, + -0.02088361233472824, + 1.722155213356018, + -0.22261345386505127, + -0.040947794914245605, + 0.22898752987384796, + 0.12912122905254364, + 0.2327134609222412, + -0.19339342415332794, + 1.43467116355896, + -0.09849818050861359, + -0.06645791232585907 + ], + [ + -0.33093082904815674, + 0.08323894441127777, + 0.7557040452957153, + 0.6216579675674438, + 0.2939087450504303, + -0.051970675587654114, + 0.17446112632751465, + 0.19948093593120575, + -0.029680700972676277, + -0.03263171762228012, + -0.1835663914680481, + 1.7952388525009155, + -0.47616440057754517, + 0.1938922107219696, + 0.18788287043571472, + 0.19288389384746552, + 0.16428688168525696, + -0.15009815990924835, + 1.4842219352722168, + -0.1300111562013626, + -0.18514320254325867 + ], + [ + 0.09189243614673615, + -0.051281724125146866, + 1.1037657260894775, + 0.6742314100265503, + 0.07488898932933807, + 0.19092682003974915, + -0.10823947191238403, + 0.3776037096977234, + 0.13911443948745728, + -0.05855753272771835, + 0.10154823958873749, + 1.7754346132278442, + -0.15086999535560608, + 0.20505103468894958, + 0.2659764289855957, + 0.06166105344891548, + 0.1542041003704071, + -0.23311443626880646, + 1.5005040168762207, + 0.006253182888031006, + -0.16563260555267334 + ], + [ + -0.12978534400463104, + 0.07043270766735077, + 1.0013962984085083, + 0.5269284248352051, + 0.25870266556739807, + -0.24719852209091187, + 0.04909998178482056, + 0.234236478805542, + 0.008548240177333355, + -0.20292186737060547, + -0.09823883324861526, + 1.694366455078125, + -0.059740275144577026, + -0.0781109631061554, + 0.02889179438352585, + 0.0746571272611618, + 0.07848958671092987, + -0.18098969757556915, + 1.4529037475585938, + -0.02616429328918457, + -0.2298535704612732 + ], + [ + -0.16542251408100128, + 0.1198706179857254, + 0.6873865127563477, + 0.6309127807617188, + 0.1625741422176361, + -0.0017057806253433228, + -0.25552499294281006, + 0.14617007970809937, + 0.057443417608737946, + -0.007251046597957611, + -0.1327020525932312, + 1.8942509889602661, + -0.1863330900669098, + -0.14612554013729095, + 0.06294852495193481, + 0.042082853615283966, + 0.18045394122600555, + -0.08017493784427643, + 1.432365894317627, + -0.16892123222351074, + -0.009519800543785095 + ] + ], + "generated_utilities": [ + 0.6627864837646484, + 0.7146708369255066, + 0.2057957649230957, + 0.20426128804683685, + 0.7040143609046936, + 0.1966126710176468, + 0.6997706890106201, + 0.6655330061912537 + ], + "group_id": "ms-4b4cd12d70c0615e0930", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7225138545036316, + 2.0, + 0.6951258182525635, + 0.18915215134620667, + 0.20867900550365448, + 0.20041285455226898, + 0.20335882902145386, + 0.20815956592559814, + 0.2024221420288086, + 0.1940193623304367, + 0.20689067244529724, + 0.19995209574699402, + 0.21901433169841766, + 0.19443893432617188, + 0.20502614974975586, + 0.1959807276725769 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.09018868207931519, + 0.3819495439529419, + 0.9586174488067627, + 1.0247726440429688, + -0.179018035531044, + 0.125870943069458, + 0.0, + -0.1166897565126419, + 0.07711493968963623, + -0.24461698532104492, + -0.005436837673187256, + 1.737370491027832, + -0.4425608515739441, + 0.0, + 0.24725346267223358, + -0.03395397961139679, + 0.643236517906189, + 0.05422210693359375, + 1.2738127708435059, + 0.18164724111557007, + 0.0 + ], + [ + 0.6095315217971802, + 0.35729360580444336, + 1.9586174488067627, + 2.0247726440429688, + -0.2820541560649872, + 1.044631004333496, + -2.239265203475952, + -0.14250506460666656, + -0.4363628029823303, + -1.244616985321045, + -0.32413747906684875, + 2.737370491027832, + 0.24581307172775269, + -0.053983256220817566, + 0.2877395749092102, + 0.18538890779018402, + 0.45247530937194824, + -0.9457778930664062, + 2.273812770843506, + -0.8183527588844299, + 0.5866778492927551 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -4.478530406951904, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.10796651244163513, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1733556985855103 + ], + [ + 0.8415398597717285, + 0.17323988676071167, + 1.9669679403305054, + 2.188286304473877, + -0.386857271194458, + 1.3415166139602661, + -2.06968355178833, + -0.38555508852005005, + -0.6971946954727173, + -1.0110756158828735, + -0.19943451881408691, + 3.0229673385620117, + 0.03575621545314789, + -0.26961493492126465, + 0.5202459692955017, + 0.0921676978468895, + 0.7439858913421631, + -0.724668562412262, + 2.116668939590454, + -1.0585715770721436, + 0.7851569652557373 + ], + [ + 0.6909375190734863, + -0.018118590116500854, + 1.677350401878357, + 1.9505902528762817, + 0.11725521087646484, + 1.2839276790618896, + -2.0543243885040283, + -0.38843441009521484, + -0.6865814328193665, + -1.5103397369384766, + -0.2952713072299957, + 2.384809970855713, + -0.10558199882507324, + 0.29018354415893555, + 0.33942633867263794, + -0.19196103513240814, + 0.35257238149642944, + -0.8657878637313843, + 2.5445473194122314, + -0.706183910369873, + 0.7214455008506775 + ], + [ + 0.8977934122085571, + 0.7311913371086121, + 2.203964948654175, + 1.686698079109192, + -0.338996022939682, + 0.631161630153656, + -2.3877103328704834, + 0.07195566594600677, + -0.6892462968826294, + -0.9634823799133301, + -0.5621621012687683, + 2.479537010192871, + 0.675940752029419, + -0.047649919986724854, + 0.14082925021648407, + 0.19517391920089722, + 0.15626585483551025, + -1.3974462747573853, + 2.759931802749634, + -1.166107416152954, + 0.18018442392349243 + ], + [ + 0.7796773910522461, + 0.06440207362174988, + 1.5159449577331543, + 1.6007665395736694, + -0.12459242343902588, + 1.5781673192977905, + -2.5707991123199463, + 0.034798189997673035, + -0.44772639870643616, + -1.3837177753448486, + -0.5199699401855469, + 3.1566030979156494, + -0.22805768251419067, + -0.5881848931312561, + 0.49021050333976746, + -0.08376060426235199, + 0.5395568609237671, + -1.3038995265960693, + 2.37483811378479, + -0.615712583065033, + 0.7884383201599121 + ], + [ + 0.4830939471721649, + -0.12824741005897522, + 2.133068561553955, + 2.42085337638855, + -0.1641906201839447, + 1.6598048210144043, + -2.5325045585632324, + 0.06623834371566772, + 0.14292877912521362, + -1.85994553565979, + 0.01158285140991211, + 3.2622194290161133, + -0.017848491668701172, + -0.08653604984283447, + 0.2199593186378479, + -0.0463244765996933, + -0.04547742009162903, + -0.5160569548606873, + 2.1955723762512207, + -0.364714115858078, + 1.0323584079742432 + ], + [ + 0.7637265920639038, + -0.4020084738731384, + 2.4714059829711914, + 1.33060622215271, + 0.1464400589466095, + 0.7656213045120239, + -1.9972842931747437, + 0.42333751916885376, + -1.118815302848816, + -1.5639967918395996, + -0.32944348454475403, + 2.0334420204162598, + 0.37165334820747375, + -0.2861292064189911, + 0.6376298666000366, + 0.8521975874900818, + 0.6845490336418152, + -1.515796422958374, + 1.5925071239471436, + -0.4112323224544525, + 1.1011840105056763 + ], + [ + 1.456933617591858, + 1.0030901432037354, + 1.1997551918029785, + 1.7652757167816162, + 0.03292572498321533, + 1.302201271057129, + -2.748408317565918, + 0.7148768305778503, + -0.5089412331581116, + -0.5701506733894348, + -0.40410247445106506, + 3.586395502090454, + -0.37424683570861816, + -0.6850852966308594, + 0.9945629835128784, + 0.5821471810340881, + 0.8666700124740601, + -1.1607441902160645, + 2.1460094451904297, + -0.7912135720252991, + 0.6301633715629578 + ], + [ + -0.09475094079971313, + -0.28330767154693604, + 1.8353716135025024, + 2.3373594284057617, + 0.7110240459442139, + 1.9665697813034058, + -1.8502212762832642, + -0.40577346086502075, + -0.03506562113761902, + -0.7167142033576965, + -0.39153769612312317, + 2.3472883701324463, + 0.08497899770736694, + -0.9910531640052795, + -0.32254326343536377, + -0.11388416588306427, + -0.4873260259628296, + -0.8018304109573364, + 2.295731782913208, + -1.325164794921875, + 1.2513983249664307 + ], + [ + -0.32657402753829956, + 0.8198520541191101, + 1.211392879486084, + 1.974273443222046, + -0.11595214903354645, + 1.954041838645935, + -1.5547080039978027, + 0.06433123350143433, + -0.079216867685318, + -0.5887003540992737, + -0.6015424728393555, + 2.756274938583374, + 0.641853928565979, + -0.8642687201499939, + 0.07965350151062012, + 0.11391039192676544, + 1.10663902759552, + -1.506749153137207, + 3.222418785095215, + -1.0799858570098877, + -0.12158209085464478 + ], + [ + 0.9412136077880859, + -0.5255860090255737, + 1.9096710681915283, + 2.113077402114868, + -1.0082788467407227, + 0.1505374312400818, + -3.0413341522216797, + -0.24721917510032654, + -0.20225940644741058, + -1.6247544288635254, + -0.33320704102516174, + 1.8443769216537476, + 0.7569764256477356, + -0.5270920395851135, + 1.1813421249389648, + 1.0764858722686768, + 0.01533237099647522, + -1.4125463962554932, + 1.82404363155365, + 0.054037630558013916, + 0.45250433683395386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.056152522563934326, + 0.005679905414581299, + 0.9586174488067627, + 1.0247726440429688, + -0.054595887660980225, + 0.044631004333496094, + 0.0, + -0.09190673381090164, + 0.09592300653457642, + -0.24461698532104492, + -0.007907181978225708, + 1.737370491027832, + -0.029382914304733276, + 0.0, + 0.06305502355098724, + 0.03764393925666809, + 0.0785532295703888, + 0.005711078643798828, + 1.2738127708435059, + -0.04058480262756348, + 0.0 + ] + ], + "predicted_scores": [ + 4.831881999969482, + 4.637731552124023, + 4.629426956176758, + 4.454403400421143, + 4.3109450340271, + 4.067284107208252, + 3.851375102996826, + 3.8106584548950195 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1025740212", + "selected_index": 0, + "state_hash": "5b394fca1712c0667d126198527a4f0ba81893d536a5fcbf7b90f5b13fa703dc", + "stored_base_utility": 0.7225138545036316, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.7040655612945557, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7040655612945557 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7247136235237122, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7247136235237122 + }, + { + "contact_quality": null, + "progress": 0.717224657535553, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.717224657535553 + }, + { + "contact_quality": null, + "progress": 0.7159295678138733, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7159295678138733 + }, + { + "contact_quality": null, + "progress": 0.40124979615211487, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.40124979615211487 + } + ], + "candidate_progress": [ + 0.7040655612945557, + 1.0, + 1.0, + 1.0, + 0.7247136235237122, + 0.717224657535553, + 0.7159295678138733, + 0.40124979615211487 + ], + "candidate_source_chart_ids": [ + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-9ced92cccacfe4d157b0", + "ms-dfc38c43e9a87f99ef7f", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-6e268d1e2915740816a1", + "generated_tangents": [ + [ + 0.026772454380989075, + -0.1296411007642746, + 0.3895333707332611, + 0.1391119360923767, + 1.9083693027496338, + -0.08746890723705292, + 0.11865496635437012, + 0.18726663291454315, + -0.15920095145702362, + 0.10431478917598724, + 0.06600998342037201, + 1.333777904510498, + -0.43381696939468384, + 0.159135103225708, + 0.17815954983234406, + 0.10608826577663422, + 0.3377605974674225, + -0.08688642084598541, + 0.8666390180587769, + -0.21693208813667297, + -0.4241900146007538 + ], + [ + -0.1682554930448532, + -0.12048175930976868, + 0.19391068816184998, + -0.04517236351966858, + 1.7990427017211914, + 0.08498160541057587, + 0.16223889589309692, + 0.02773110568523407, + -0.09519593417644501, + 0.006628185510635376, + 0.005198635160923004, + 1.3731099367141724, + -0.032558292150497437, + 0.06638997048139572, + 0.04230092465877533, + 0.09292496740818024, + 0.1456332504749298, + -0.1668664664030075, + 1.0234153270721436, + 0.023253291845321655, + -0.09180307388305664 + ], + [ + -0.15021051466464996, + -0.16460613906383514, + 0.22778600454330444, + -0.052551835775375366, + 1.9887595176696777, + -0.11002078652381897, + 0.09651431441307068, + 0.08139576762914658, + -0.059139251708984375, + 0.1783117949962616, + -0.09237655252218246, + 1.2773078680038452, + -0.20209002494812012, + 0.15842197835445404, + -0.1090216189622879, + 0.10600344836711884, + 0.19155451655387878, + -0.016486629843711853, + 1.0678369998931885, + -0.3012024760246277, + -0.1768263578414917 + ], + [ + -0.27181482315063477, + -0.1355568915605545, + 0.38721534609794617, + -0.14984719455242157, + 1.761793851852417, + -0.27559980750083923, + 0.09324458241462708, + 0.09733743965625763, + -0.02516706846654415, + 0.24463962018489838, + -0.013168446719646454, + 1.2979891300201416, + -0.6586089730262756, + 0.2973225712776184, + -0.01377873495221138, + 0.136238694190979, + 0.17095285654067993, + -0.0614003986120224, + 1.0419929027557373, + -0.18981415033340454, + -0.013761118054389954 + ], + [ + -0.18508736789226532, + -0.12962208688259125, + 0.25904661417007446, + -0.10958153009414673, + 1.9822971820831299, + 0.016127049922943115, + 0.0933694839477539, + -0.022003725171089172, + -0.0713600441813469, + 0.07369425147771835, + -0.03565383702516556, + 1.3851419687271118, + -0.3443012833595276, + 0.1290782243013382, + 0.023380320519208908, + 0.11314325034618378, + 0.2110779583454132, + -0.08936159312725067, + 1.0043904781341553, + -0.2254841923713684, + -0.17989367246627808 + ], + [ + -0.11984036862850189, + -0.15093642473220825, + 0.0017314553260803223, + -0.21155759692192078, + 2.0077171325683594, + -0.21363839507102966, + 0.008436620235443115, + -0.030026763677597046, + -0.03960853070020676, + 0.19732311367988586, + -0.1784546971321106, + 1.2282062768936157, + -0.1912350058555603, + 0.11905762553215027, + -0.0025265179574489594, + 0.05166539177298546, + 0.2459152638912201, + -0.14281965792179108, + 1.1461100578308105, + -0.2580501139163971, + -0.030434131622314453 + ], + [ + -0.14233838021755219, + -0.11305731534957886, + 0.23599496483802795, + -0.03712493181228638, + 2.0280590057373047, + 0.026946067810058594, + 0.008028149604797363, + -0.08196964859962463, + -0.054160602390766144, + 0.12734714150428772, + -0.08615278452634811, + 1.3986033201217651, + -0.39400535821914673, + 0.09593387693166733, + 0.05878238007426262, + 0.11810895800590515, + 0.2561825215816498, + -0.027173668146133423, + 1.033850908279419, + -0.2784793972969055, + -0.1574062705039978 + ], + [ + -0.3736599087715149, + -0.12925270199775696, + 0.10042649507522583, + -0.15230914950370789, + 1.8361225128173828, + 0.1269616186618805, + -0.20964395999908447, + -0.18080934882164001, + 0.034028373658657074, + 0.06390392780303955, + -0.17515945434570312, + 1.378110647201538, + -0.4343094527721405, + 0.20561745762825012, + -0.1421905905008316, + 0.05440641567111015, + 0.0006457120180130005, + -0.36248528957366943, + 1.0916004180908203, + -0.424134224653244, + -0.06055821478366852 + ] + ], + "generated_utilities": [ + 0.7040655612945557, + 2.0, + 2.0, + 2.0, + 0.7247136235237122, + 0.717224657535553, + 0.7159295678138733, + 0.40124979615211487 + ], + "group_id": "ms-6e268d1e2915740816a1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7257392406463623, + 0.7165254950523376, + 2.0, + 0.72465980052948, + 0.7087006568908691, + 0.2306610643863678, + 0.7143489122390747, + 0.7045385837554932, + 0.5905889272689819, + 0.1919487714767456, + 0.1914421170949936, + 0.2230607271194458, + 0.20312398672103882, + 0.19757536053657532, + 0.19724789261817932, + 0.19653666019439697 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.44810131192207336, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0264720916748047, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.9882198572158813 + ], + [ + -0.4707597494125366, + 0.09937268495559692, + 0.6395220160484314, + -0.40935325622558594, + 2.4514670372009277, + -0.02397046983242035, + -0.21874001622200012, + 0.16799744963645935, + 0.0360843725502491, + -0.09719255566596985, + -0.8619975447654724, + 2.2499821186065674, + -0.4933375120162964, + 0.35259634256362915, + -0.34735018014907837, + -0.517992377281189, + 0.6015943288803101, + -0.7677568793296814, + 1.5267367362976074, + -0.16888000071048737, + 0.3771851658821106 + ], + [ + -0.8530820608139038, + -0.6707327961921692, + 0.7970488667488098, + 0.34902632236480713, + 2.1393990516662598, + 0.5651285648345947, + 0.113044872879982, + 0.2987540364265442, + 0.4139185845851898, + 0.41691792011260986, + -0.4906405806541443, + 1.5687955617904663, + -0.8899085521697998, + -0.01883232593536377, + 0.24473188817501068, + 0.554391622543335, + 0.23424841463565826, + -0.10042615234851837, + 2.1214542388916016, + -0.15780393779277802, + 0.5807479023933411 + ], + [ + -0.6562768220901489, + -0.059059903025627136, + 0.24766850471496582, + 0.06101328134536743, + 2.5052053928375244, + 0.5204365849494934, + 0.1595638245344162, + 0.47176462411880493, + 0.3036114573478699, + 0.13197286427021027, + -0.2551935017108917, + 2.185537099838257, + -1.0984127521514893, + 0.3213496804237366, + 0.49875718355178833, + 0.1223822832107544, + 0.2575722932815552, + -0.7017092704772949, + 1.7495591640472412, + -0.19501343369483948, + 1.1214004755020142 + ], + [ + -0.45427173376083374, + -0.041353583335876465, + 0.8143320083618164, + -0.39680930972099304, + 2.4542014598846436, + 0.382559597492218, + -0.9993512630462646, + 0.5202391147613525, + 0.5282149314880371, + 0.4165073335170746, + 0.43153735995292664, + 2.5625123977661133, + -0.721943199634552, + 0.6765530109405518, + -0.6697012186050415, + -0.0637422502040863, + 0.6919896602630615, + -1.033813714981079, + 2.317282199859619, + -0.13503669202327728, + -0.07009828090667725 + ], + [ + -0.548885703086853, + -0.609938383102417, + 0.7181543707847595, + 0.11203774809837341, + 2.478057861328125, + -0.6170704960823059, + -1.0383654832839966, + 0.3524467349052429, + 0.5748242139816284, + 0.42987847328186035, + 0.013119548559188843, + 1.3954555988311768, + -0.6403790712356567, + 0.025414258241653442, + -0.7780460119247437, + -0.23124903440475464, + 0.48347827792167664, + 0.05266502499580383, + 1.692553997039795, + 0.362108051776886, + 0.6330842971801758 + ], + [ + -1.3070502281188965, + -0.947432816028595, + 0.4198402762413025, + -1.1768442392349243, + 1.7843327522277832, + 0.7587301731109619, + 0.7447960376739502, + 0.9411643147468567, + 0.3013244867324829, + -0.3862171769142151, + -1.0263384580612183, + 2.97957181930542, + 0.1849566102027893, + 0.08799406886100769, + 0.1984272301197052, + 0.5277765393257141, + 0.2931878864765167, + -1.1385071277618408, + 1.884846568107605, + 0.23083388805389404, + 0.6359583139419556 + ], + [ + -0.5375385880470276, + 0.16159841418266296, + 1.260543942451477, + -0.6744770407676697, + 2.1646244525909424, + 0.6190755367279053, + -0.746461808681488, + -0.38674360513687134, + 0.8897944092750549, + -0.594306468963623, + -0.05647379159927368, + 1.7390328645706177, + -0.037059009075164795, + 0.470071941614151, + 0.010003047063946724, + 0.08421775698661804, + -0.4613104462623596, + -0.22854508459568024, + 1.772307276725769, + -0.29165884852409363, + 0.6179300546646118 + ], + [ + -0.6489957571029663, + 0.4526708126068115, + 0.5115790367126465, + 0.31459903717041016, + 2.9655239582061768, + 0.07586031407117844, + -1.1667817831039429, + -0.7146965861320496, + 1.0295621156692505, + 0.1932714432477951, + -0.2703063488006592, + 2.9046523571014404, + -1.5882594585418701, + 0.3030351996421814, + 0.6543201208114624, + -0.448961466550827, + 0.4115087389945984, + -0.4087216854095459, + 2.6432881355285645, + -0.6797424554824829, + 0.2669999599456787 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.22510427236557007, + -0.26737168431282043, + 0.10375632345676422, + 0.12666502594947815, + 1.379244327545166, + -0.01088210940361023, + 0.0, + 0.23721180856227875, + -0.10981421172618866, + -0.021816372871398926, + -0.071512371301651, + 1.0360438823699951, + 0.09019589424133301, + 0.0, + 0.03181244805455208, + -0.06466590613126755, + 0.2068846970796585, + 0.14229199290275574, + 0.7610219717025757, + -0.03211131691932678, + 0.0 + ] + ], + "predicted_scores": [ + 6.741085052490234, + 6.448919773101807, + 6.381951332092285, + 6.351556777954102, + 6.2756218910217285, + 6.065993785858154, + 5.817976474761963, + 5.585904598236084 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1475440973", + "selected_index": 0, + "state_hash": "38ff5a264c3c39ff1ec8756d84e622963a11ff1ac61568ba57fbea0c114c3140", + "stored_base_utility": 0.7257392406463623, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.7239725589752197, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7239725589752197 + }, + "base_progress": 0.7239725589752197, + "base_success": false, + "base_utility": 0.7239725589752197, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.712515652179718, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.712515652179718 + }, + { + "contact_quality": null, + "progress": 0.7240610718727112, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7240610718727112 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7171879410743713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7171879410743713 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.712515652179718, + 0.7240610718727112, + 1.0, + 1.0, + 1.0, + 1.0, + 0.7171879410743713, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-dd32ce9859c365c535b4", + "ms-dfc38c43e9a87f99ef7f", + "ms-dfc38c43e9a87f99ef7f", + "ms-9ced92cccacfe4d157b0", + "ms-9ced92cccacfe4d157b0", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4", + "ms-dd32ce9859c365c535b4" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-8864796e31bbe93d7c5d", + "generated_tangents": [ + [ + -0.10783638060092926, + -0.09325660765171051, + 0.2980384826660156, + -0.33395111560821533, + 2.0316874980926514, + -0.6613283157348633, + 0.13582468032836914, + 0.30387037992477417, + -0.18794894218444824, + 0.021270275115966797, + 0.05030153691768646, + 1.67851984500885, + -0.704618513584137, + 0.8027870059013367, + 0.21700423955917358, + 0.010395519435405731, + 0.34987202286720276, + -0.39984625577926636, + 1.33915376663208, + -0.44861865043640137, + 0.2573627531528473 + ], + [ + -0.012575432658195496, + -0.15628373622894287, + 0.14575940370559692, + -0.3477315604686737, + 2.1024715900421143, + -0.21605506539344788, + 0.3728860020637512, + 0.04174472391605377, + -0.1807873398065567, + 0.2590184509754181, + 0.023569080978631973, + 1.4081878662109375, + -0.41216424107551575, + 0.3406746983528137, + -0.03466382250189781, + 0.05362289026379585, + 0.3251451551914215, + -0.2605082392692566, + 1.3668866157531738, + -0.5721875429153442, + 0.18283721804618835 + ], + [ + 0.12038832157850266, + -0.07027330249547958, + 0.09921270608901978, + -0.39877015352249146, + 2.0046935081481934, + -0.3416796624660492, + 0.400138795375824, + -0.002096042037010193, + -0.13842645287513733, + 0.07764597237110138, + -0.0947338119149208, + 1.4713890552520752, + -0.31413087248802185, + 0.4171268343925476, + 0.0804351419210434, + -0.03479379415512085, + 0.3605607748031616, + -0.21419651806354523, + 1.3085267543792725, + -0.5157297849655151, + 0.22035014629364014 + ], + [ + 0.012208536267280579, + -0.017652850598096848, + 0.2961108684539795, + -0.2688570022583008, + 1.965496301651001, + -0.2678772807121277, + 0.1856738030910492, + -0.002307325601577759, + -0.19012604653835297, + 0.00589403510093689, + -0.03486165404319763, + 1.8909276723861694, + -0.4045448899269104, + 0.5490383505821228, + 0.14483818411827087, + 0.06391429901123047, + 0.4477684497833252, + -0.2726069688796997, + 1.3530051708221436, + -0.508605420589447, + -0.12967249751091003 + ], + [ + -0.005118951201438904, + -0.008849266916513443, + 0.2604236602783203, + -0.19897401332855225, + 1.9821579456329346, + -0.406417578458786, + 0.2835600972175598, + 0.15547087788581848, + -0.18190845847129822, + 0.10572440922260284, + -0.11708914488554001, + 1.7370109558105469, + -0.3137834668159485, + 0.4488229751586914, + -0.06477105617523193, + -0.006240908522158861, + 0.39373844861984253, + -0.20722119510173798, + 1.36594557762146, + -0.561301052570343, + -0.1171288788318634 + ], + [ + -0.11272348463535309, + -0.02308821864426136, + -0.028663665056228638, + -0.248068705201149, + 1.9404525756835938, + -0.24219614267349243, + 0.588707685470581, + -0.0058912187814712524, + -0.1755552589893341, + -0.00034246593713760376, + -0.04061531275510788, + 1.8202155828475952, + -0.2156980037689209, + 0.37749236822128296, + 0.08020981401205063, + 0.024505749344825745, + 0.2673306465148926, + -0.2767093777656555, + 1.2514865398406982, + -0.3545854687690735, + 0.15575112402439117 + ], + [ + -0.290314257144928, + -0.09416916966438293, + 0.040746986865997314, + -0.402606338262558, + 1.936187744140625, + -0.27139273285865784, + 0.023540079593658447, + -0.05686098337173462, + -0.09082868695259094, + -0.010172031819820404, + -0.06936090439558029, + 1.7161526679992676, + -0.6132203936576843, + 0.6554793119430542, + -0.07010607421398163, + -0.0399882048368454, + 0.30135107040405273, + -0.5384387969970703, + 1.3447628021240234, + -0.642231822013855, + 0.1492292732000351 + ], + [ + 0.03728649765253067, + 0.0138520747423172, + 0.28487300872802734, + -0.2022971510887146, + 2.0122439861297607, + -0.30780452489852905, + 0.18874788284301758, + -0.010944128036499023, + -0.19370464980602264, + 0.053227126598358154, + -0.09578505903482437, + 1.8564132452011108, + -0.3929198086261749, + 0.5121991038322449, + 0.1994069665670395, + 0.054055098444223404, + 0.4637346863746643, + -0.22313715517520905, + 1.37990140914917, + -0.5531796216964722, + -0.16263440251350403 + ] + ], + "generated_utilities": [ + 0.712515652179718, + 0.7240610718727112, + 2.0, + 2.0, + 2.0, + 2.0, + 0.7171879410743713, + 2.0 + ], + "group_id": "ms-8864796e31bbe93d7c5d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.728268027305603, + 0.7257949113845825, + 2.0, + 0.7287554740905762, + 0.72548508644104, + 0.7011849880218506, + 0.7089968919754028, + 0.689879298210144, + 0.722068727016449, + 0.7046384811401367, + 0.1933039277791977, + 0.629345178604126, + 0.18965840339660645, + 0.18740439414978027, + 0.2032846063375473, + 0.20379847288131714 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.3859826922416687, + 0.1119421124458313, + 0.6443181037902832, + 0.11836245656013489, + 1.863237738609314, + -1.166780948638916, + 0.8025261163711548, + -0.5342166423797607, + -0.2057022899389267, + 0.8987585306167603, + -0.525823712348938, + 1.6212486028671265, + 0.25674968957901, + 0.4037417471408844, + -0.4561460018157959, + -0.18429945409297943, + -0.1778680682182312, + -0.22491087019443512, + 2.095893144607544, + -0.3484030067920685, + 0.47628670930862427 + ], + [ + -0.446367084980011, + -0.19229409098625183, + -0.03271818161010742, + 0.39123907685279846, + 1.1786221265792847, + -0.16883885860443115, + -0.3337137699127197, + 0.7789723873138428, + -0.12953178584575653, + -0.028340458869934082, + -0.871502161026001, + 0.7702060341835022, + -0.645424485206604, + 1.0858956575393677, + -0.05413459241390228, + -0.14592209458351135, + -0.03685358166694641, + -0.686826229095459, + 2.084512710571289, + 0.11898508667945862, + 0.20845119655132294 + ], + [ + 0.09377387166023254, + -0.03508002683520317, + -0.29283326864242554, + -0.9559528827667236, + 2.0994086265563965, + 0.02743762731552124, + 0.6819101572036743, + 0.607413113117218, + 0.4378519356250763, + 0.6497513055801392, + -0.5086554288864136, + 0.9556822180747986, + -0.8518581986427307, + 0.31636691093444824, + -0.4196639060974121, + 0.6866374015808105, + 0.5175505876541138, + -0.572692334651947, + 1.651871681213379, + -0.7227057814598083, + -0.33861052989959717 + ], + [ + -0.42610836029052734, + 0.10144555568695068, + 0.7105271220207214, + -1.291037678718567, + 2.2307684421539307, + 0.17725014686584473, + -0.08436906337738037, + 0.9681323766708374, + 0.5482062697410583, + 1.368379831314087, + -1.021532416343689, + 2.5197110176086426, + -0.3508763909339905, + 0.7584415078163147, + 1.0978648662567139, + 0.2626119554042816, + 0.6115584373474121, + -0.7777479290962219, + 1.7278802394866943, + -0.15240566432476044, + 0.4239119291305542 + ], + [ + 1.0207308530807495, + 0.8557561039924622, + 0.4489574134349823, + -1.0117379426956177, + 0.8262714147567749, + -0.630092442035675, + 0.09812727570533752, + 0.21663640439510345, + 0.23771141469478607, + -0.47456812858581543, + -1.3189021348953247, + 2.269665002822876, + -0.7066116333007812, + 1.1826255321502686, + 1.0058563947677612, + 0.1694127470254898, + -0.5649446845054626, + -0.32043421268463135, + 0.46698564291000366, + -0.9052557945251465, + -0.12436902523040771 + ], + [ + -0.5202209949493408, + 0.30583277344703674, + 1.2163622379302979, + 0.4305659234523773, + 1.7276779413223267, + -0.1535550355911255, + -0.5289345383644104, + -0.632349967956543, + 0.9393473267555237, + 0.38391292095184326, + -1.1998018026351929, + 1.9739618301391602, + -1.3405330181121826, + 0.048939138650894165, + 0.6369897127151489, + 0.629721999168396, + 1.388192892074585, + -1.065996766090393, + 1.9119879007339478, + -1.0432264804840088, + 0.9627590179443359 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.13860583305358887, + -0.06957411766052246, + 0.1504676342010498, + 0.1671028435230255, + 0.5675687789916992, + 0.16587841510772705, + 0.0, + -0.21137550473213196, + -0.21542736887931824, + 0.2565074861049652, + -0.00923699140548706, + 0.5251069068908691, + -0.06964048743247986, + 0.0, + -0.19547803699970245, + 0.10278619825839996, + -0.19206219911575317, + -0.1487000733613968, + 0.44575393199920654, + 0.0725468099117279, + 0.0 + ] + ], + "predicted_scores": [ + 6.7557806968688965, + 6.573371887207031, + 5.993346691131592, + 5.651432037353516, + 5.6416802406311035, + 5.634435176849365, + 5.356122016906738, + 5.133599758148193 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "551504635", + "selected_index": 0, + "state_hash": "ab6587e0b2a3b5f21f23c56daf8894249774912a3b20794bd49dc1b78ec736a7", + "stored_base_utility": 0.728268027305603, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.42231252789497375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.42231252789497375 + }, + "base_progress": 0.42231252789497375, + "base_success": false, + "base_utility": 0.42231252789497375, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.21454350650310516, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21454350650310516 + }, + { + "contact_quality": null, + "progress": 0.2053624391555786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2053624391555786 + }, + { + "contact_quality": null, + "progress": 0.4355689287185669, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4355689287185669 + }, + { + "contact_quality": null, + "progress": 0.20295369625091553, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20295369625091553 + }, + { + "contact_quality": null, + "progress": 0.6990174651145935, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.6990174651145935 + }, + { + "contact_quality": null, + "progress": 0.19902725517749786, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19902725517749786 + }, + { + "contact_quality": null, + "progress": 0.4670216143131256, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4670216143131256 + }, + { + "contact_quality": null, + "progress": 0.513749361038208, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.513749361038208 + } + ], + "candidate_progress": [ + 0.21454350650310516, + 0.2053624391555786, + 0.4355689287185669, + 0.20295369625091553, + 0.6990174651145935, + 0.19902725517749786, + 0.4670216143131256, + 0.513749361038208 + ], + "candidate_source_chart_ids": [ + "ms-13d8bae55d3c3710138f", + "ms-e1b6bcb41532a800d3cb", + "ms-8bc7aa419d17368d81b5", + "ms-0660d5f65df5fc4b6132", + "ms-c090bce47703a4c4bb0b", + "ms-e1b6bcb41532a800d3cb", + "ms-8bc7aa419d17368d81b5", + "ms-df961fe24a43e56d1716" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-b04b489f4fbad93b65ca", + "generated_tangents": [ + [ + -0.10732845962047577, + -0.07998903095722198, + 0.9089713096618652, + 0.43404635787010193, + 0.5116224884986877, + -0.4192449152469635, + -0.0853036642074585, + 0.21688437461853027, + 0.060581207275390625, + -0.10935025662183762, + -0.08304091542959213, + 1.5341674089431763, + -0.2520782947540283, + -0.30165374279022217, + 0.21617785096168518, + 0.03804491087794304, + 0.029590457677841187, + -0.28844237327575684, + 1.558558702468872, + -0.1732308715581894, + -0.41648069024086 + ], + [ + 0.10789267718791962, + -0.025463012978434563, + 0.9054484367370605, + 0.5132200717926025, + 0.43973252177238464, + -0.28184157609939575, + -0.4851565361022949, + 0.35337552428245544, + 0.020470203831791878, + -0.13234728574752808, + 0.09148307144641876, + 1.60129714012146, + -0.8142393827438354, + -0.14194037020206451, + 0.31978920102119446, + 0.0017017032951116562, + 0.15934930741786957, + -0.2463037520647049, + 1.7043859958648682, + -0.3596044182777405, + -0.21034398674964905 + ], + [ + -0.2914814352989197, + 0.0723748505115509, + 0.6198295950889587, + 0.5085612535476685, + 0.7717130184173584, + -0.44099000096321106, + -0.5913452506065369, + 0.42901328206062317, + 0.023957092314958572, + -0.10156137496232986, + -0.03212863951921463, + 1.4908665418624878, + -0.13424307107925415, + -0.479592502117157, + 0.20987318456172943, + 0.0969122052192688, + 0.03166326880455017, + 0.1798076629638672, + 1.7433905601501465, + -0.05927984416484833, + -0.09309232234954834 + ], + [ + -0.1918175369501114, + -0.03785164654254913, + 1.0073039531707764, + 0.5900051593780518, + 0.622347891330719, + -0.3132612407207489, + -0.18197572231292725, + 0.24078305065631866, + 0.11385473608970642, + -0.11136939376592636, + -0.00994226336479187, + 1.4678353071212769, + -0.3760998547077179, + -0.24490900337696075, + 0.4038201570510864, + 0.09390346705913544, + 0.18737035989761353, + -0.3093724846839905, + 1.491455078125, + -0.20497477054595947, + -0.24525514245033264 + ], + [ + -0.34775400161743164, + 0.06690829992294312, + 0.6236593127250671, + 0.6907037496566772, + 0.5310370326042175, + -0.2572833299636841, + -0.08047932386398315, + 0.17149080336093903, + 0.013812781311571598, + 0.05431506037712097, + -0.21567025780677795, + 1.558121681213379, + -0.4854121804237366, + -0.08453099429607391, + 0.30467668175697327, + 0.17470823228359222, + 0.04356248676776886, + -0.2433718889951706, + 1.6160027980804443, + -0.17445962131023407, + -0.3500150740146637 + ], + [ + 0.17549237608909607, + -0.11710525304079056, + 0.9921681880950928, + 0.5986083745956421, + 0.24832230806350708, + 0.100563645362854, + -0.4224139451980591, + 0.34710606932640076, + 0.13641080260276794, + -0.015819929540157318, + 0.10837534070014954, + 1.5618746280670166, + -0.31603893637657166, + 0.0254497230052948, + 0.4129064381122589, + 0.02422311156988144, + 0.10954248160123825, + -0.30140072107315063, + 1.6172378063201904, + -0.12455418705940247, + -0.3618107736110687 + ], + [ + -0.19824378192424774, + 0.02349727600812912, + 0.5157075524330139, + 0.477207213640213, + 0.5088697671890259, + -0.05654667317867279, + -0.30953019857406616, + 0.22353291511535645, + 0.0600614920258522, + -0.09159446507692337, + -0.03458709269762039, + 1.5768359899520874, + -0.34343600273132324, + -0.5828196406364441, + 0.11493048071861267, + 0.004861665889620781, + 0.13417184352874756, + -0.06164027750492096, + 1.639733076095581, + 0.25797146558761597, + -0.29030829668045044 + ], + [ + -0.2250225991010666, + 0.035239078104496, + 0.6645710468292236, + 0.5496208667755127, + 0.50779128074646, + -0.4300542175769806, + -0.11587768793106079, + 0.10986512154340744, + 0.04473678022623062, + -0.07210289686918259, + -0.10999809950590134, + 1.5273667573928833, + -0.27252569794654846, + -0.44839179515838623, + 0.16335031390190125, + 0.12081894278526306, + 0.19093842804431915, + -0.22147516906261444, + 1.8027544021606445, + -0.18689005076885223, + -0.3028786778450012 + ] + ], + "generated_utilities": [ + 0.21454350650310516, + 0.2053624391555786, + 0.4355689287185669, + 0.20295369625091553, + 0.6990174651145935, + 0.19902725517749786, + 0.4670216143131256, + 0.513749361038208 + ], + "group_id": "ms-b04b489f4fbad93b65ca", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7198216915130615, + 0.7247374057769775, + 2.0, + 0.22730140388011932, + 0.21130704879760742, + 0.2076549530029297, + 0.2006116509437561, + 0.21285679936408997, + 0.19614991545677185, + 0.20440611243247986, + 0.2015489637851715, + 0.19511334598064423, + 0.20091164112091064, + 0.19681888818740845, + 0.19675713777542114, + 0.20853900909423828 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.30706143379211426, + 0.2797066569328308, + 0.4565920829772949, + 0.7342244386672974, + -0.28428468108177185, + 0.012420237064361572, + 0.0, + 0.19500797986984253, + -0.1289721131324768, + -0.3524337112903595, + 0.07460063695907593, + 1.7684156894683838, + 0.11335399746894836, + 0.0, + -0.5014548301696777, + 0.4537479281425476, + 0.1963198333978653, + -0.025568604469299316, + 1.6286182403564453, + -0.0963512659072876, + 0.0 + ], + [ + 1.3070614337921143, + -0.7202933430671692, + 1.456592082977295, + 1.7342244386672974, + -0.15130555629730225, + 0.9519147872924805, + -1.4541058540344238, + -0.6432010531425476, + -0.26959508657455444, + -0.3406839668750763, + -0.4419664740562439, + 2.768415689468384, + -0.3271861970424652, + -0.13685251772403717, + 0.23842881619930267, + 0.20406432449817657, + 0.3414191007614136, + -1.0255686044692993, + 2.6286182403564453, + -0.9051820039749146, + 0.6850019693374634 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9082117080688477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.27370503544807434, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.3700039386749268 + ], + [ + 1.0634682178497314, + -0.8590207099914551, + 1.6810569763183594, + 1.8525642156600952, + 0.06716938316822052, + 1.0774534940719604, + -1.177553415298462, + -0.7412493228912354, + -0.3193308115005493, + -0.4253617525100708, + -0.4890587329864502, + 2.624048948287964, + -0.109145388007164, + -0.10380380600690842, + 0.20137116312980652, + 0.20110605657100677, + 0.06929725408554077, + -0.9908366203308105, + 2.5783159732818604, + -0.7248227000236511, + 0.919882595539093 + ], + [ + 1.2083125114440918, + -0.9112091064453125, + 1.1255995035171509, + 1.9678500890731812, + -0.3424606919288635, + 1.350451946258545, + -1.5799542665481567, + -1.014689326286316, + -0.3427739441394806, + -0.24094250798225403, + -0.4992668032646179, + 2.403038740158081, + -0.29080134630203247, + -0.1569681167602539, + 0.4877905249595642, + -0.16464389860630035, + 0.41957730054855347, + -0.6997033357620239, + 2.80957293510437, + -1.0509651899337769, + 0.8937826156616211 + ], + [ + 1.2133954763412476, + -1.0171141624450684, + 1.1572445631027222, + 1.8050156831741333, + -0.29570114612579346, + 0.7672240734100342, + -0.976697564125061, + -0.8219587802886963, + -0.746895432472229, + -0.346658855676651, + -0.7942779064178467, + 3.0403196811676025, + -0.6172577738761902, + 0.3615778088569641, + 0.546725332736969, + -0.06415437161922455, + 0.43834370374679565, + -1.332592248916626, + 3.029940605163574, + -0.5720348358154297, + 1.1660562753677368 + ], + [ + 0.7937494516372681, + -0.45201122760772705, + 2.0367026329040527, + 2.1590728759765625, + 0.38935917615890503, + 0.5424834489822388, + -1.391097903251648, + -1.0657228231430054, + -0.44411367177963257, + -0.3797546923160553, + -0.03612968325614929, + 2.867069721221924, + -0.26322364807128906, + -0.32477179169654846, + 0.007963955402374268, + 0.053216055035591125, + 0.36466139554977417, + -0.7477234601974487, + 2.170649766921997, + -0.46266868710517883, + 0.7669195532798767 + ], + [ + 1.0920023918151855, + -1.0678586959838867, + 1.4391454458236694, + 2.155820608139038, + -0.7347199320793152, + 0.6208673715591431, + -1.6380563974380493, + -0.6589961051940918, + -0.8267081379890442, + 0.011062979698181152, + -0.1169244647026062, + 2.743579626083374, + -0.41342735290527344, + -0.5980290770530701, + 0.49618369340896606, + -0.09863747656345367, + 0.2179805338382721, + -0.6006474494934082, + 2.2965025901794434, + -0.97348952293396, + 1.2447597980499268 + ], + [ + 1.8286218643188477, + -1.486896276473999, + 1.8610713481903076, + 1.792364478111267, + -0.06209900975227356, + 0.7138283252716064, + -1.0023046731948853, + -0.8588749170303345, + -0.30936315655708313, + 0.33444294333457947, + 0.2771298885345459, + 3.0452492237091064, + -0.6618144512176514, + -0.747363805770874, + 0.19540274143218994, + -0.05051566660404205, + 0.5057645440101624, + -0.3770519495010376, + 3.028423309326172, + -0.19366812705993652, + 1.1766406297683716 + ], + [ + 1.057653784751892, + -0.15160071849822998, + 1.1646175384521484, + 2.517653226852417, + 0.037323400378227234, + 1.4844036102294922, + -1.2659661769866943, + -0.9694168567657471, + -0.792635440826416, + 0.15484732389450073, + -0.6592323780059814, + 2.426063060760498, + -0.8063763976097107, + 0.37280529737472534, + -0.1984816938638687, + 0.3488052487373352, + 0.36773377656936646, + -0.9600323438644409, + 1.801730990409851, + -1.1618295907974243, + 0.526383101940155 + ], + [ + 0.8403785228729248, + -1.7131881713867188, + 1.7484346628189087, + 1.3915244340896606, + 0.038706690073013306, + 1.7859117984771729, + -0.6996253132820129, + -0.12596702575683594, + -0.07181461155414581, + -0.8556877374649048, + -0.6567700505256653, + 2.935109853744507, + 0.3808797299861908, + 0.1733608990907669, + 0.9097857475280762, + 0.7001722455024719, + 0.14087507128715515, + -0.32659411430358887, + 2.278188943862915, + -1.87721586227417, + 1.438266634941101 + ], + [ + 1.259635329246521, + -0.5244767665863037, + 2.201894998550415, + 2.2337045669555664, + -0.333431601524353, + 0.7420244216918945, + -2.0896377563476562, + 0.00931471586227417, + 0.6261725425720215, + -0.49785688519477844, + -0.9404669404029846, + 3.756554365158081, + 0.5252982378005981, + -0.3158212900161743, + 0.4053819179534912, + 0.8204222917556763, + 0.5041900277137756, + -0.43809741735458374, + 3.519717216491699, + -1.0316743850708008, + 0.3069072663784027 + ], + [ + 0.9530733823776245, + -0.011007547378540039, + 2.396453380584717, + 2.6382436752319336, + 0.29263433814048767, + 1.6715352535247803, + -0.819398045539856, + -0.5916767120361328, + 0.006437480449676514, + -0.3942071795463562, + -0.5187541842460632, + 3.677276849746704, + -0.4376804232597351, + 0.3331298828125, + -0.3693196773529053, + -0.3432707190513611, + 1.104022741317749, + -1.0780245065689087, + 3.0859882831573486, + -0.26956111192703247, + 0.8377041816711426 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.30706143379211426, + 0.024911105632781982, + 0.4565920829772949, + 0.7342244386672974, + 0.01698923110961914, + -0.04808521270751953, + 0.0, + 0.10375595092773438, + 0.23247671127319336, + -0.12021404504776001, + -0.19280612468719482, + 1.7684156894683838, + -0.23127278685569763, + 0.0, + -0.1010780781507492, + 0.00952194631099701, + -0.16307592391967773, + -0.14496707916259766, + 1.6286182403564453, + -0.28656232357025146, + 0.0 + ] + ], + "predicted_scores": [ + 6.294991493225098, + 6.1743950843811035, + 5.736081123352051, + 5.545506000518799, + 5.456851482391357, + 5.102646350860596, + 4.967200756072998, + 4.937523365020752 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113268940", + "selected_index": 0, + "state_hash": "76e9a962970055fa7565c007db1b35e003cd352908be4cf573066d8b58e2d443", + "stored_base_utility": 0.7198216915130615, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.7267584800720215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7267584800720215 + }, + { + "contact_quality": null, + "progress": 0.7201066613197327, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7201066613197327 + }, + { + "contact_quality": null, + "progress": 0.7173653244972229, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7173653244972229 + }, + { + "contact_quality": null, + "progress": 0.7081647515296936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7081647515296936 + }, + { + "contact_quality": null, + "progress": 0.7245497107505798, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.7245497107505798 + }, + { + "contact_quality": null, + "progress": 0.722693145275116, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.722693145275116 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.7267584800720215, + 0.7201066613197327, + 0.7173653244972229, + 0.7081647515296936, + 0.7245497107505798, + 0.722693145275116 + ], + "candidate_source_chart_ids": [ + "ms-b483cac145e16ccdf42a", + "ms-baa36ee75ec2de550b9b", + "ms-efbde8652e75274fbd69", + "ms-062addb9da32a0c2ea97", + "ms-baa36ee75ec2de550b9b", + "ms-b483cac145e16ccdf42a", + "ms-efbde8652e75274fbd69", + "ms-efbde8652e75274fbd69" + ], + "candidate_source_task_ids": [ + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1", + "LiftPegUpright-v1" + ], + "candidate_success": [ + true, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-c663f7e73e7e8376a05c", + "generated_tangents": [ + [ + 0.10946595668792725, + -0.11374206840991974, + -0.30370935797691345, + -0.09413708001375198, + 2.146193504333496, + -0.004189819097518921, + 0.13687065243721008, + 0.10518065094947815, + -0.06227639317512512, + 0.10917890071868896, + 0.008878372609615326, + 1.1038117408752441, + -0.07490143179893494, + -0.17087791860103607, + 0.06763358414173126, + 0.003116860054433346, + 0.17291629314422607, + -0.11192534863948822, + 1.142543911933899, + 0.18380320072174072, + -0.16685497760772705 + ], + [ + 0.026685722172260284, + -0.04553455114364624, + -0.26302388310432434, + -0.10152976214885712, + 2.136355400085449, + -0.057701557874679565, + -0.14895033836364746, + 0.05181865394115448, + -0.0992811992764473, + -0.043570078909397125, + 0.07364282011985779, + 1.3353840112686157, + -0.15444502234458923, + -0.2545003294944763, + 0.05045922100543976, + 0.047208771109580994, + 0.06242549419403076, + -0.21155910193920135, + 1.0452136993408203, + 0.030841559171676636, + -0.11804291605949402 + ], + [ + -0.009969472885131836, + -0.03685017302632332, + -0.30377474427223206, + -0.10030913352966309, + 2.1355435848236084, + -0.07478314638137817, + -0.06573319435119629, + 0.05628681182861328, + -0.10190877318382263, + -0.05741732567548752, + 0.06106235459446907, + 1.3288439512252808, + -0.16746309399604797, + -0.25997209548950195, + 0.036219522356987, + 0.04209299385547638, + 0.09307558834552765, + -0.21978159248828888, + 1.000388503074646, + 0.03648659586906433, + -0.03894805908203125 + ], + [ + 0.020938843488693237, + -0.09490247070789337, + -0.29992005228996277, + -0.04530210793018341, + 2.1611111164093018, + -0.014455854892730713, + 0.15765157341957092, + 0.04537287354469299, + -0.0998525321483612, + 0.04522865265607834, + -0.07850425690412521, + 1.185364007949829, + -0.22974291443824768, + -0.13400767743587494, + 0.02067471295595169, + 0.03210567310452461, + 0.13150949776172638, + -0.12193818390369415, + 1.0589673519134521, + 0.04473540186882019, + -0.00128251314163208 + ], + [ + 0.03843197971582413, + -0.08653861284255981, + -0.33079013228416443, + -0.13347841799259186, + 2.185880184173584, + 0.05989789962768555, + 0.04211077094078064, + -0.02038167417049408, + -0.11486528813838959, + 0.08138403296470642, + -0.056508637964725494, + 1.2871387004852295, + -0.15371173620224, + -0.24662600457668304, + 0.018292300403118134, + 0.04085967689752579, + 0.1069001778960228, + -0.16666091978549957, + 1.0928897857666016, + 0.0013755261898040771, + -0.18731218576431274 + ], + [ + 0.05528000742197037, + -0.07219599187374115, + -0.3663255274295807, + -0.06928510963916779, + 2.1292848587036133, + -0.02273339033126831, + 0.10645395517349243, + -0.008280232548713684, + -0.11774088442325592, + 0.01741180568933487, + -0.07084622234106064, + 1.049666404724121, + -0.18437302112579346, + -0.20236153900623322, + -0.06443154811859131, + 0.012843076139688492, + 0.04830491542816162, + -0.16940997540950775, + 1.1128853559494019, + -0.009312480688095093, + -0.10608768463134766 + ], + [ + 0.05965904891490936, + -0.07622204720973969, + -0.4074612557888031, + -0.04590828716754913, + 2.1456429958343506, + -0.13582226634025574, + -0.030505597591400146, + 0.02432785928249359, + -0.19222700595855713, + 0.09118318557739258, + -0.08913388103246689, + 1.0548746585845947, + -0.37534233927726746, + -0.14381365478038788, + 0.12527123093605042, + 0.07386000454425812, + -0.015574052929878235, + -0.09169892966747284, + 1.1436439752578735, + -0.09258164465427399, + -0.10136809945106506 + ], + [ + 0.15269577503204346, + 0.010219737887382507, + -0.3038608133792877, + -0.04885582625865936, + 1.9410953521728516, + 0.5958267450332642, + -0.08891606330871582, + -0.0011883378028869629, + -0.1737256646156311, + 0.043074771761894226, + -0.1666845977306366, + 1.0102267265319824, + -0.41516172885894775, + -0.16439305245876312, + 0.0007879920303821564, + 0.17655529081821442, + 0.23835942149162292, + -0.4307820796966553, + 1.1399189233779907, + 0.07694810628890991, + 0.14124073088169098 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.7267584800720215, + 0.7201066613197327, + 0.7173653244972229, + 0.7081647515296936, + 0.7245497107505798, + 0.722693145275116 + ], + "group_id": "ms-c663f7e73e7e8376a05c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.7233090400695801, + 0.7198824882507324, + 2.0, + 0.7120869159698486, + 0.27145418524742126, + 0.23090600967407227, + 0.2526220679283142, + 0.2656436562538147, + 0.264312207698822, + 0.27572086453437805, + 0.296668142080307, + 0.2128777951002121, + 0.22965002059936523, + 0.2196153998374939, + 0.22027263045310974, + 0.19939398765563965 + ], + "instruction": "Lift the peg and hold it upright.", + "negative_tangents": [ + [ + 0.038699887692928314, + -0.3685438632965088, + -1.3628361225128174, + 0.0675731897354126, + 2.4906270503997803, + 0.8971554040908813, + 0.06863358616828918, + 0.14689946174621582, + 0.0500560887157917, + 0.5852231383323669, + -0.5345680713653564, + 1.7433128356933594, + -0.5520187020301819, + 0.36285334825515747, + 0.06837204098701477, + 0.04091557487845421, + 0.26646411418914795, + -0.3870337903499603, + 1.7901486158370972, + -0.46956348419189453, + 0.5412080883979797 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.13726717233657837, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.7257066965103149, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0824161767959595 + ], + [ + -0.09956050664186478, + -0.6439464092254639, + -1.5331499576568604, + -0.2275703251361847, + 2.2570624351501465, + 1.145668387413025, + -0.19933581352233887, + 0.15889883041381836, + -0.07524599134922028, + 0.6580387353897095, + -0.5754004716873169, + 1.6105921268463135, + -0.25637149810791016, + 0.37256744503974915, + 0.17931225895881653, + 0.286465048789978, + 0.07884949445724487, + -0.35266172885894775, + 1.548264980316162, + -0.2917717695236206, + 0.3307930827140808 + ], + [ + 0.13072630763053894, + -0.5927069783210754, + -1.4328954219818115, + 0.1252535879611969, + 2.5439467430114746, + 1.119347095489502, + -0.12134809792041779, + -0.09485669434070587, + -0.040146272629499435, + 0.4683093726634979, + -0.6791773438453674, + 2.1314187049865723, + -0.4368498623371124, + 0.5876930952072144, + 0.27027207612991333, + -0.1808907389640808, + 0.037005022168159485, + -0.6130290627479553, + 1.60811185836792, + -0.8427046537399292, + 0.5648807883262634 + ], + [ + -0.019807856529951096, + -0.40178078413009644, + -1.5710430145263672, + 0.3568149507045746, + 2.8693275451660156, + 1.2080440521240234, + 0.45903119444847107, + -0.07479061186313629, + 0.2587376832962036, + 0.37667450308799744, + -0.5607666373252869, + 1.9825769662857056, + -0.7874732613563538, + 0.48894837498664856, + -0.04161303490400314, + 0.2237108200788498, + 0.12043289840221405, + -0.8541218638420105, + 1.3131444454193115, + -0.8318716287612915, + 0.7653946876525879 + ], + [ + -0.039324238896369934, + 0.18316024541854858, + -1.2052042484283447, + -0.020045489072799683, + 2.441659927368164, + 1.3116326332092285, + 0.04364071041345596, + 0.617156982421875, + -0.5366188287734985, + 0.5837841629981995, + -0.6157102584838867, + 1.946446180343628, + -0.44197604060173035, + -0.12030342221260071, + 0.2635759711265564, + -0.34203875064849854, + 0.3788418769836426, + -0.01673591136932373, + 1.7612254619598389, + -0.8283271193504333, + 0.44542378187179565 + ], + [ + 0.020479684695601463, + -0.3728349208831787, + -1.2566280364990234, + -0.4903675317764282, + 2.7149407863616943, + 1.1410136222839355, + 0.18438707292079926, + 0.15751159191131592, + 0.5994289517402649, + 0.7441410422325134, + -0.5757432579994202, + 2.025078058242798, + -1.1044142246246338, + -0.24153441190719604, + 0.7623711824417114, + -0.07102327048778534, + 0.7491155862808228, + -0.853058397769928, + 1.1346031427383423, + -0.4604499638080597, + 0.3380756974220276 + ], + [ + -0.0438276082277298, + 0.21775567531585693, + -0.9180121421813965, + 0.2164689004421234, + 1.7669851779937744, + 0.7701845765113831, + -0.4171431064605713, + -0.44690096378326416, + -0.6473215818405151, + 0.5592721700668335, + 0.1537061333656311, + 1.2201651334762573, + -0.02862715721130371, + 0.759445309638977, + -0.08159622550010681, + -0.038715001195669174, + 0.8136870265007019, + -1.129607081413269, + 1.5099153518676758, + -1.2117259502410889, + 1.2496943473815918 + ], + [ + -0.6884663105010986, + -0.7670764923095703, + -1.3375780582427979, + 0.42161574959754944, + 2.4925031661987305, + 0.18111640214920044, + -0.4373469054698944, + 0.6228322386741638, + -0.3739543557167053, + 1.3811917304992676, + 0.05884122848510742, + 1.0582032203674316, + -0.38174134492874146, + 0.7282494902610779, + 0.9550073146820068, + -0.6478474736213684, + -0.467768132686615, + -0.22179017961025238, + 2.1456754207611084, + -0.39938730001449585, + 1.08339524269104 + ], + [ + 0.7042056322097778, + -0.4828507900238037, + -1.914047122001648, + -0.7084186673164368, + 1.87782621383667, + 0.42005008459091187, + 0.511604905128479, + -0.2043081521987915, + 0.9001545906066895, + 1.083567500114441, + -0.9796027541160583, + 1.3277345895767212, + -0.5564201474189758, + 0.6932777166366577, + -0.5934457778930664, + -0.296670138835907, + 0.14519014954566956, + -0.8612968325614929, + 0.8968887329101562, + -0.5584146976470947, + -0.434501588344574 + ], + [ + 0.19210004806518555, + 0.5024477243423462, + -0.4993329644203186, + -0.826042115688324, + 1.6430644989013672, + 0.9877098798751831, + 0.49057987332344055, + -0.13611945509910583, + 0.7228546738624573, + 1.4937405586242676, + 0.18012672662734985, + 2.188062906265259, + -0.6084107160568237, + 1.0874407291412354, + 0.5744171142578125, + 0.9375161528587341, + -0.43115919828414917, + 0.19787153601646423, + 2.652085781097412, + -0.529538631439209, + 0.03626328706741333 + ], + [ + -0.014053341001272202, + -1.351341724395752, + -0.7293765544891357, + 0.9410406947135925, + 3.139275074005127, + 1.3463419675827026, + -0.728021502494812, + -0.6353646516799927, + -0.19685329496860504, + 1.5315749645233154, + -1.2550475597381592, + 1.7411813735961914, + 0.14418578147888184, + 0.7038239240646362, + 0.1004495620727539, + 0.7590892910957336, + 0.9287852048873901, + -0.23726919293403625, + 0.8381044864654541, + -1.2906644344329834, + 0.9584261178970337 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03644278272986412, + -0.12137487530708313, + -0.3628361225128174, + -0.027813076972961426, + 1.4906270503997803, + -0.10284459590911865, + 0.0, + 0.15174832940101624, + 0.015595663338899612, + -0.10680150985717773, + -0.14723241329193115, + 0.7433128356933594, + 0.25961190462112427, + 0.0, + -0.23801246285438538, + -0.10784421861171722, + 0.09500086307525635, + -0.06586915254592896, + 0.7901486158370972, + -0.10978007316589355, + 0.0 + ] + ], + "predicted_scores": [ + 7.598120212554932, + 7.074145793914795, + 7.03989839553833, + 6.809096813201904, + 6.70497465133667, + 6.377159595489502, + 5.469299793243408, + 5.32899284362793 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1528740417", + "selected_index": 0, + "state_hash": "c02fdfdd2339e4215d60b2221be925e066a7763f9e3bba3ff33a15f20e286f5a", + "stored_base_utility": 0.7233090400695801, + "task_id": "LiftPegUpright-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15342965722084045, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15342965722084045 + }, + "base_progress": 0.15342965722084045, + "base_success": false, + "base_utility": 0.15342965722084045, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.07777997106313705, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07777997106313705 + }, + { + "contact_quality": null, + "progress": 0.04109710454940796, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04109710454940796 + }, + { + "contact_quality": null, + "progress": 0.1038902997970581, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1038902997970581 + }, + { + "contact_quality": null, + "progress": 0.4708559513092041, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.4708559513092041 + }, + { + "contact_quality": null, + "progress": 0.07253576815128326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07253576815128326 + }, + { + "contact_quality": null, + "progress": 0.09191600978374481, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09191600978374481 + }, + { + "contact_quality": null, + "progress": 0.05067390203475952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05067390203475952 + }, + { + "contact_quality": null, + "progress": 0.04289213567972183, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04289213567972183 + } + ], + "candidate_progress": [ + 0.07777997106313705, + 0.04109710454940796, + 0.1038902997970581, + 0.4708559513092041, + 0.07253576815128326, + 0.09191600978374481, + 0.05067390203475952, + 0.04289213567972183 + ], + "candidate_source_chart_ids": [ + "ms-23a355317920e192147f", + "ms-a509a572b3871e591d26", + "ms-5df54a8baaceb8c73c8a", + "ms-0541a144afe57b5a7ae6", + "ms-4015c296239564108da2", + "ms-307f9067d5b66f55b3a6", + "ms-b7e7fe66a6c86898eec2", + "ms-b7e7fe66a6c86898eec2" + ], + "candidate_source_task_ids": [ + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1", + "PickCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank5", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank6", + "ctt_transport_rank6" + ], + "candidates_evaluated": true, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "generated_tangents": [ + [ + -0.2633914351463318, + -0.24415768682956696, + 0.9743947982788086, + -0.1303619146347046, + -0.024884283542633057, + -0.2420523762702942, + 1.0717332363128662, + 0.314116895198822, + -0.2180466204881668, + 0.030273988842964172, + -0.08761095255613327, + 0.18791615962982178, + -0.20732125639915466, + 2.459271192550659, + 0.02998794987797737, + 0.13088786602020264, + -0.004302307963371277, + -0.07890604436397552, + 0.2167949378490448, + -0.3225422203540802, + 2.8667795658111572 + ], + [ + -0.35856717824935913, + -0.21494071185588837, + 1.2262903451919556, + -0.038692787289619446, + 0.4024645686149597, + -0.17845800518989563, + 1.2016874551773071, + 0.05131341516971588, + -0.14427171647548676, + 0.06735620647668839, + -0.0521039217710495, + -0.01999160647392273, + -0.05861958861351013, + 2.786029100418091, + 0.14884191751480103, + 0.09314198791980743, + -0.001286521553993225, + -0.20260174572467804, + 0.23968689143657684, + 0.03133571147918701, + 2.820800542831421 + ], + [ + 0.0325254425406456, + -0.32269322872161865, + 0.6115252375602722, + -0.09641175717115402, + 0.4406045377254486, + -0.28701019287109375, + 0.6941764950752258, + 0.3170878291130066, + -0.17988522350788116, + -0.05719827860593796, + 0.02314738929271698, + 0.11381135880947113, + 0.12774470448493958, + 2.5892746448516846, + 0.22388124465942383, + 0.12328775227069855, + -0.062124744057655334, + -0.09468741714954376, + 0.36773669719696045, + -0.011422485113143921, + 2.8251781463623047 + ], + [ + -0.13648687303066254, + -0.30327659845352173, + 0.4851149916648865, + -0.1762879192829132, + 0.756318211555481, + -0.2686004936695099, + 0.7468767166137695, + 0.29756754636764526, + -0.1784353405237198, + -0.07621937245130539, + -0.04210872948169708, + 0.18523074686527252, + 0.2125585377216339, + 2.610225200653076, + 0.23728817701339722, + 0.07998001575469971, + -0.08203758299350739, + -0.011189937591552734, + 0.38997745513916016, + 0.05735740065574646, + 2.957305669784546 + ], + [ + -0.2623583674430847, + -0.19964829087257385, + 1.0483981370925903, + -0.1308472901582718, + 0.09816092252731323, + -0.21537232398986816, + 0.6463452577590942, + 0.27233392000198364, + -0.24594703316688538, + -0.041606657207012177, + 0.05039536580443382, + 0.15449047088623047, + -0.17688804864883423, + 2.6832849979400635, + 0.2820209264755249, + 0.0677497386932373, + 0.0134086012840271, + -0.07833032310009003, + 0.3621862530708313, + -0.11108329892158508, + 2.829379081726074 + ], + [ + -0.33549773693084717, + -0.17868132889270782, + 0.929031491279602, + -0.14533010125160217, + 0.22347202897071838, + -0.2668790817260742, + 0.6549648642539978, + 0.38585883378982544, + -0.18775269389152527, + -0.058350615203380585, + 0.06588498502969742, + 0.1194341629743576, + -0.018949836492538452, + 2.5024490356445312, + 0.2588111460208893, + 0.1296253502368927, + 0.008959144353866577, + -0.09163670241832733, + 0.31655794382095337, + -0.19999781250953674, + 2.834423780441284 + ], + [ + -0.4173704981803894, + -0.3724336624145508, + 0.5605266690254211, + -0.24761976301670074, + -0.15740084648132324, + -0.6621396541595459, + 0.8614896535873413, + -0.1276450753211975, + -0.2848299741744995, + 0.07669876515865326, + -0.24686670303344727, + -0.06723156571388245, + -0.17518436908721924, + 2.8572006225585938, + 0.10023102164268494, + 0.15556290745735168, + 0.09022867679595947, + -0.2503125071525574, + -0.06601586937904358, + 0.09898245334625244, + 2.4181342124938965 + ], + [ + -0.5222467184066772, + -0.3916386365890503, + 0.7598212957382202, + -0.19166620075702667, + -0.2371518909931183, + -0.6868388652801514, + 1.0427273511886597, + -0.13800588250160217, + -0.3128302991390228, + -0.07054481655359268, + -0.2948596775531769, + 0.01922476291656494, + -0.39983636140823364, + 2.7168216705322266, + 0.04087206721305847, + 0.24478109180927277, + 0.12932133674621582, + -0.46600866317749023, + -0.08169755339622498, + 0.08273890614509583, + 2.473327398300171 + ] + ], + "generated_utilities": [ + 0.07777997106313705, + 0.04109710454940796, + 0.1038902997970581, + 0.4708559513092041, + 0.07253576815128326, + 0.09191600978374481, + 0.05067390203475952, + 0.04289213567972183 + ], + "group_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.6040759086608887, + 2.0, + 0.14582239091396332, + 0.5723002552986145, + 0.10770338028669357, + 0.10128920525312424, + 0.08499570190906525, + 0.14736615121364594, + 0.07181884348392487, + 0.11773071438074112, + 0.06549974530935287, + 0.06568789482116699, + 0.10681070387363434, + 0.05432777479290962, + 0.0625491738319397, + 0.06991603225469589 + ], + "instruction": "Pick up the cube and move it to the goal position.", + "negative_tangents": [ + [ + -0.072060227394104, + 0.2043294906616211, + 0.8692872524261475, + 0.020746350288391113, + -0.004742175340652466, + 0.05779099464416504, + 0.0, + -0.0498790442943573, + -0.20218299329280853, + 0.41556206345558167, + -0.2306707501411438, + 0.32090064883232117, + -0.2089032530784607, + 0.0, + 0.05318737402558327, + -0.04637153446674347, + 0.04412045329809189, + 0.08670947700738907, + -0.10918038338422775, + -0.059010982513427734, + 0.0 + ], + [ + -0.5282122492790222, + 1.204329490661621, + 1.8692872524261475, + -0.8456907868385315, + 0.42858073115348816, + -0.3134727478027344, + -2.5308754444122314, + 0.1655428111553192, + -0.12024828046560287, + 0.0010203085839748383, + 0.2592625319957733, + 0.2679504156112671, + 0.25922709703445435, + 2.851924419403076, + 0.0039860643446445465, + -0.014717570506036282, + 0.01173020526766777, + 0.10578173398971558, + 0.05138798803091049, + 0.3216937780380249, + 2.9243662357330322 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.061750888824463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.703848838806152, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.8487324714660645 + ], + [ + -0.6558206081390381, + 1.3553937673568726, + 2.057647466659546, + -1.0405915975570679, + 0.2629950940608978, + -0.28233209252357483, + -2.589219093322754, + -0.06961745023727417, + 0.02406928688287735, + 0.2341437190771103, + 0.49766814708709717, + 0.4930538535118103, + 0.370309442281723, + 3.111422061920166, + 0.06151486933231354, + -0.2028706818819046, + 0.12007936835289001, + 0.2164750099182129, + 0.16723006963729858, + 0.26430171728134155, + 3.0643396377563477 + ], + [ + -0.18174728751182556, + 0.8890945315361023, + 1.5388343334197998, + -0.4532086253166199, + 0.3864310085773468, + -0.45410382747650146, + -2.623199462890625, + 0.5360933542251587, + -0.04706605523824692, + 0.11160072684288025, + 0.5460608005523682, + 0.39992716908454895, + 0.17028246819972992, + 2.8760411739349365, + 0.29450932145118713, + -0.3028962016105652, + 0.06744809448719025, + 0.1687500774860382, + 0.42097336053848267, + 0.3965923488140106, + 3.28550386428833 + ], + [ + -0.4313088357448578, + 0.9752410054206848, + 2.0068275928497314, + -0.5783756375312805, + 0.5761726498603821, + -0.5289677381515503, + -2.321643590927124, + 0.0023015737533569336, + -0.07372284680604935, + 0.05664224550127983, + 0.03747282922267914, + 0.4900178611278534, + 0.31588616967201233, + 3.018609046936035, + 0.12529893219470978, + -0.2872275412082672, + -0.2500917911529541, + 0.0006694421172142029, + -0.39133700728416443, + 0.6382719874382019, + 2.810041904449463 + ], + [ + -0.5560051202774048, + 1.4042110443115234, + 1.662226915359497, + -1.4059122800827026, + 0.8242433071136475, + -0.10998553037643433, + -2.7556281089782715, + -0.26875895261764526, + -0.4201813042163849, + -0.371376633644104, + 0.7718684673309326, + 0.6452833414077759, + 0.027421817183494568, + 2.3879575729370117, + -0.47588983178138733, + -0.045049820095300674, + 0.0838315486907959, + 0.10257662832736969, + 0.4181908369064331, + 0.2650004029273987, + 3.006909132003784 + ], + [ + -0.4896228015422821, + 1.0570447444915771, + 1.408416509628296, + -0.7269805669784546, + 1.023213267326355, + -0.22378163039684296, + -2.1348509788513184, + -0.4423682987689972, + 0.26834720373153687, + -0.5835619568824768, + 0.3675343692302704, + 0.3307996988296509, + -0.01537257432937622, + 3.4422543048858643, + 0.3622148334980011, + -0.26806333661079407, + 0.4238347113132477, + 0.2583734393119812, + 0.10431534051895142, + 0.2077663242816925, + 2.446356773376465 + ], + [ + -1.1077957153320312, + 0.4289678931236267, + 1.9882663488388062, + -0.3021383285522461, + -0.006195276975631714, + -0.22295325994491577, + -1.8697566986083984, + 0.9590625762939453, + -0.0024877265095710754, + -0.3088582456111908, + 0.6517632007598877, + 0.4271944761276245, + 0.34532302618026733, + 2.704721450805664, + 0.7557594776153564, + -0.8073300719261169, + -0.7479258179664612, + -0.19068273901939392, + 0.14240311086177826, + 0.2233867347240448, + 2.1774275302886963 + ], + [ + -0.9354162812232971, + 1.5971527099609375, + 1.0168101787567139, + -0.7164653539657593, + 0.519247829914093, + -0.7477964162826538, + -3.338376045227051, + 0.7212944030761719, + -0.6477807760238647, + 0.38193029165267944, + -0.1568622589111328, + 0.47457069158554077, + -0.30742186307907104, + 2.6848907470703125, + -0.4251534044742584, + 0.6623334884643555, + 0.03348249942064285, + -0.2996619939804077, + -0.8319172263145447, + 0.7883892059326172, + 3.5870752334594727 + ], + [ + -0.44698429107666016, + 0.3201255202293396, + 1.5405346155166626, + -0.9956957101821899, + 0.8944320678710938, + -0.4196317791938782, + -2.010188102722168, + 0.7199084758758545, + -0.49276289343833923, + 0.4512734115123749, + 0.48276829719543457, + 0.22657281160354614, + 0.8205416798591614, + 3.6653342247009277, + 0.6818231344223022, + 0.4660317897796631, + -0.14916014671325684, + 0.5328372716903687, + -0.5979922413825989, + 0.3124805986881256, + 3.110726833343506 + ], + [ + -0.5630171298980713, + 1.934722900390625, + 1.4954721927642822, + -1.684873342514038, + 0.133146733045578, + -0.4359630346298218, + -2.681238889694214, + 0.8904907703399658, + -0.48221057653427124, + -0.7635119557380676, + -0.23804441094398499, + 0.5691602230072021, + -0.23042798042297363, + 3.760842800140381, + 0.3684825003147125, + -0.20475278794765472, + 0.04838913306593895, + 0.17439094185829163, + 0.07042267918586731, + -0.1653938889503479, + 3.844882011413574 + ], + [ + -0.3705383241176605, + 1.5082495212554932, + 0.9002141356468201, + -1.3291171789169312, + 0.15676641464233398, + -1.2286558151245117, + -1.7006758451461792, + 0.34438908100128174, + 0.32962289452552795, + -0.6556114554405212, + 0.3502740263938904, + 0.7518025636672974, + -0.49080890417099, + 2.10355806350708, + 0.36426910758018494, + -0.4822288453578949, + 0.7496572732925415, + 0.43558254837989807, + 0.34650692343711853, + 0.9977220296859741, + 1.9853770732879639 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.005563676357269287, + 0.2043294906616211, + 0.8692872524261475, + 0.08163481950759888, + 0.05995750427246094, + 0.012857973575592041, + 0.0, + 0.01583634316921234, + 0.04724548012018204, + 0.018815450370311737, + 0.02742627263069153, + -0.03381502628326416, + 0.007930099964141846, + 0.0, + 0.01774722896516323, + 0.08064591884613037, + -0.06814983487129211, + -0.002621002495288849, + 0.05759798362851143, + 0.05164051055908203, + 0.0 + ] + ], + "predicted_scores": [ + 3.0987119674682617, + 3.048224449157715, + 3.0443451404571533, + 2.829008102416992, + 2.3250162601470947, + 2.161208152770996, + 1.4783743619918823, + 0.9284471273422241 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1273247072", + "selected_index": 0, + "state_hash": "237fafab6bfdb561d66c862026f803fbd19d4007f6313bdb3d79b6dd74f69327", + "stored_base_utility": 0.6040759086608887, + "task_id": "PickCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.050487834960222244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.050487834960222244 + }, + "base_progress": 0.050487834960222244, + "base_success": false, + "base_utility": 0.050487834960222244, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12168914079666138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12168914079666138 + }, + { + "contact_quality": null, + "progress": 0.08366147428750992, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08366147428750992 + }, + { + "contact_quality": null, + "progress": 0.15701176226139069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15701176226139069 + }, + { + "contact_quality": null, + "progress": 0.12812411785125732, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12812411785125732 + }, + { + "contact_quality": null, + "progress": 0.18499283492565155, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18499283492565155 + }, + { + "contact_quality": null, + "progress": 0.08666101843118668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08666101843118668 + }, + { + "contact_quality": null, + "progress": 0.09535380452871323, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09535380452871323 + }, + { + "contact_quality": null, + "progress": 0.09223812818527222, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09223812818527222 + } + ], + "candidate_progress": [ + 0.12168914079666138, + 0.08366147428750992, + 0.15701176226139069, + 0.12812411785125732, + 0.18499283492565155, + 0.08666101843118668, + 0.09535380452871323, + 0.09223812818527222 + ], + "candidate_source_chart_ids": [ + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2", + "ms-ccc79c981e3a67ebeaf2" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0345d8ac530f88cefd61", + "generated_tangents": [ + [ + 0.45177316665649414, + 0.053055040538311005, + 0.5662141442298889, + -0.08378662168979645, + 1.0378338098526, + -0.4903506934642792, + 0.11517664790153503, + -0.18985208868980408, + 0.536932110786438, + 0.612867534160614, + -0.4678453505039215, + -0.2846616804599762, + -0.7690434455871582, + -0.24230216443538666, + -0.723652720451355, + 0.37185996770858765, + -0.23070524632930756, + 0.42627060413360596, + -1.3476972579956055, + -0.23582014441490173, + -0.3316645324230194 + ], + [ + 0.15758034586906433, + 0.08929029107093811, + 0.27810442447662354, + 0.12750515341758728, + 0.8122290968894958, + -0.23212140798568726, + 0.21684962511062622, + -0.2672390043735504, + 0.5321968793869019, + 0.687800943851471, + -0.4824269711971283, + -0.4698670506477356, + -1.2611265182495117, + 0.31366878747940063, + -0.3113158047199249, + 0.4353826642036438, + -0.19363166391849518, + 0.43276938796043396, + -1.3187494277954102, + 0.3054482340812683, + 0.15071073174476624 + ], + [ + 0.09606397151947021, + 0.01897987723350525, + 0.27905046939849854, + 0.1527319848537445, + 0.9234761595726013, + -0.33946576714515686, + 0.2668255567550659, + -0.21642044186592102, + 0.5691895484924316, + 0.6546608209609985, + -0.41415712237358093, + -0.4193723201751709, + -1.2848857641220093, + 0.2762640118598938, + -0.41601526737213135, + 0.46642565727233887, + -0.1210404485464096, + 0.32488104701042175, + -1.2147209644317627, + 0.19869518280029297, + 0.1881680190563202 + ], + [ + 0.10475245118141174, + 0.043515369296073914, + 0.16247892379760742, + 0.2928580343723297, + 1.027008056640625, + -0.20103126764297485, + 0.316720187664032, + -0.2060665786266327, + 0.5696927905082703, + 0.7628860473632812, + -0.3807542026042938, + -0.24436303973197937, + -1.3558554649353027, + 0.13071060180664062, + -0.4661105275154114, + 0.49856120347976685, + 0.032164350152015686, + 0.17239071428775787, + -1.2636163234710693, + 0.018511801958084106, + -0.017703816294670105 + ], + [ + 0.041100695729255676, + 0.08698694407939911, + 0.10086056590080261, + 0.06992438435554504, + 1.004431962966919, + -0.6041911840438843, + 1.1277751922607422, + 0.41241931915283203, + 0.47663602232933044, + 0.6032533049583435, + -0.413040429353714, + -0.8885858058929443, + -1.3979250192642212, + 0.21834880113601685, + -0.4339330494403839, + 0.4801313281059265, + 0.1524370163679123, + 0.4601544141769409, + -1.3526644706726074, + 0.5546669363975525, + -0.05564048886299133 + ], + [ + 0.28570207953453064, + 0.26462239027023315, + 0.41819876432418823, + 0.200236976146698, + 1.0851836204528809, + -0.38349828124046326, + -0.07097262144088745, + -0.35254666209220886, + 0.48069003224372864, + 0.5254405736923218, + -0.6000253558158875, + -0.6434926986694336, + -0.9392081499099731, + 0.11152574419975281, + -0.4522472620010376, + 0.4310823082923889, + 0.01918680965900421, + 0.39273834228515625, + -1.1843786239624023, + -0.08674001693725586, + 0.24136638641357422 + ], + [ + 0.22639581561088562, + 0.21364231407642365, + 0.29420721530914307, + 0.29290685057640076, + 0.8009777665138245, + -0.5039989948272705, + -0.1242033839225769, + -0.18591883778572083, + 0.4670332074165344, + 0.5588225722312927, + -0.6637930870056152, + -0.48225724697113037, + -1.1685916185379028, + 0.20823800563812256, + -0.4058948755264282, + 0.3993051052093506, + 0.032337650656700134, + 0.4755861163139343, + -1.4540271759033203, + 0.04864993691444397, + -0.015121623873710632 + ], + [ + 0.15841281414031982, + 0.04041200876235962, + 0.412304162979126, + 0.35304316878318787, + 1.0737528800964355, + -0.2817305624485016, + -0.140988290309906, + -0.246922105550766, + 0.5877202749252319, + 0.7453543543815613, + -0.5220662951469421, + -0.5889569520950317, + -1.1438002586364746, + 0.05441661924123764, + -0.4762595295906067, + 0.5195172429084778, + 0.03120456635951996, + 0.3142508268356323, + -1.433992624282837, + -0.1491376906633377, + 0.13459615409374237 + ] + ], + "generated_utilities": [ + 0.12168914079666138, + 0.08366147428750992, + 0.15701176226139069, + 0.12812411785125732, + 0.18499283492565155, + 0.08666101843118668, + 0.09535380452871323, + 0.09223812818527222 + ], + "group_id": "ms-0345d8ac530f88cefd61", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.007898053154349327, + 0.0058197579346597195, + 0.052657388150691986, + 0.016109248623251915, + 0.1698339879512787, + 0.008177022449672222, + 0.12333428859710693, + 0.1150994747877121, + 0.15141737461090088, + 0.08515926450490952, + 0.1523766964673996, + 0.04073335975408554, + 0.027175527065992355, + 0.1798098385334015, + 0.08658158779144287, + 0.0618208646774292 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2244938164949417, + -0.708204448223114, + 1.4058912992477417, + 0.19480958580970764, + 1.1751772165298462, + -1.2265020608901978, + -0.8636159896850586, + -0.07021944224834442, + 1.2708500623703003, + 1.0246398448944092, + -1.7615737915039062, + -0.44495686888694763, + -1.2927910089492798, + 0.27074551582336426, + -1.6349679231643677, + 0.9748592376708984, + 1.0210484266281128, + 0.36906859278678894, + -2.528050661087036, + 0.7326674461364746, + 1.553327202796936 + ], + [ + 0.35595428943634033, + -0.8224175572395325, + 1.1192705631256104, + 0.4673205614089966, + 1.1115962266921997, + -1.4676826000213623, + -1.057816505432129, + -0.008461203426122665, + 0.9729399085044861, + 0.8019976615905762, + -1.4944581985473633, + -0.21817980706691742, + -1.3722625970840454, + 0.3557305932044983, + -1.8202308416366577, + 1.2552388906478882, + 0.7855625748634338, + 0.5709130167961121, + -2.4140331745147705, + 0.9063631296157837, + 1.282220721244812 + ], + [ + 0.19927366077899933, + -0.7607958912849426, + 1.4928451776504517, + 0.38265493512153625, + 1.2970150709152222, + -1.2964622974395752, + -0.5547049045562744, + -0.2585906684398651, + 1.3020356893539429, + 1.0737547874450684, + -1.5593467950820923, + -0.1617565155029297, + -1.4868489503860474, + 0.15243440866470337, + -1.4100472927093506, + 0.7991768717765808, + 1.0339381694793701, + 0.6228882074356079, + -2.8132376670837402, + 0.7685560584068298, + 1.8755284547805786 + ], + [ + 0.6393168568611145, + -0.2793915271759033, + 1.4690577983856201, + 0.4745514690876007, + 0.9989874362945557, + -0.7359552979469299, + -1.2171094417572021, + -0.39386802911758423, + 1.1646839380264282, + 0.6661151051521301, + -1.9162347316741943, + -0.43809595704078674, + -1.3973355293273926, + 0.7664861679077148, + -1.553026795387268, + 1.1995127201080322, + 1.285676121711731, + 0.822955846786499, + -2.574894666671753, + 0.3542059361934662, + 1.5986168384552002 + ], + [ + 0.26509445905685425, + -0.5443360805511475, + 1.2502005100250244, + 0.6112543940544128, + 0.924185037612915, + -1.0554168224334717, + -0.7927505373954773, + -0.07152043282985687, + 0.9228929281234741, + 1.545881986618042, + -1.6973015069961548, + -0.496639221906662, + -0.8727940320968628, + 0.5596445202827454, + -2.1939897537231445, + 1.239556074142456, + 1.4699029922485352, + -0.182618647813797, + -2.7263355255126953, + 0.976334810256958, + 1.0402796268463135 + ], + [ + 0.1353418529033661, + -1.1921336650848389, + 1.6913124322891235, + -0.43279919028282166, + 1.1331026554107666, + -1.8503069877624512, + -0.46699997782707214, + 0.5785945653915405, + 1.5516390800476074, + 0.38185298442840576, + -2.192314386367798, + -0.6304612159729004, + -1.6237419843673706, + -0.06383657455444336, + -1.2674099206924438, + 0.7967972755432129, + 1.2453874349594116, + 0.01532512903213501, + -2.6341590881347656, + 0.8366512656211853, + 1.9056174755096436 + ], + [ + 1.0916199684143066, + 0.23665857315063477, + 1.7737994194030762, + -0.8046588897705078, + 1.944507360458374, + -1.3606563806533813, + -0.022103488445281982, + 0.5769228935241699, + 1.0605480670928955, + 1.3100652694702148, + -1.9243974685668945, + 0.43040379881858826, + -1.9046732187271118, + -0.49132758378982544, + -1.9095335006713867, + 1.8850300312042236, + 1.366606593132019, + 0.37651723623275757, + -2.86037540435791, + 0.7753443717956543, + 2.072996139526367 + ], + [ + 0.7277938723564148, + 0.23822063207626343, + 0.4208102226257324, + -0.18717050552368164, + 1.1951282024383545, + -1.1622966527938843, + -0.6617333292961121, + -0.6339871883392334, + 0.8901646137237549, + 0.9977425932884216, + -1.8568902015686035, + 0.3291373550891876, + -0.9377886056900024, + 1.0705609321594238, + -0.90934818983078, + 1.7013394832611084, + 1.5952999591827393, + -0.05327540636062622, + -2.546459197998047, + 0.15792781114578247, + 1.2277774810791016 + ], + [ + 0.9283657073974609, + -1.421413779258728, + 1.6497737169265747, + -0.1984737515449524, + 0.9705085158348083, + -0.8917888402938843, + 0.026184558868408203, + 0.09181642532348633, + 0.6335604786872864, + 1.6531281471252441, + -1.4918432235717773, + -0.01735416054725647, + -0.5425058603286743, + 0.7866935133934021, + -0.7011347413063049, + 0.9797741770744324, + 1.6596866846084595, + -0.13051137328147888, + -2.692556858062744, + -0.05617690086364746, + 0.9771407842636108 + ] + ], + "predicted_scores": [ + 3.8818323612213135, + 3.315857172012329, + 3.1084160804748535, + 3.026110887527466, + 3.0173468589782715, + 2.8973727226257324, + 2.8117752075195312, + 2.2416369915008545 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "1ca7c651b2dc4638488078318fe8aa293aa6f6da78672d440752daa9f2d4e8a3", + "stored_base_utility": 0.007898053154349327, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07743936777114868, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07743936777114868 + }, + "base_progress": 0.07743936777114868, + "base_success": false, + "base_utility": 0.07743936777114868, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.010246972553431988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010246972553431988 + }, + { + "contact_quality": null, + "progress": 0.009836931712925434, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009836931712925434 + }, + { + "contact_quality": null, + "progress": 0.010462522506713867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010462522506713867 + }, + { + "contact_quality": null, + "progress": 0.016623934730887413, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016623934730887413 + }, + { + "contact_quality": null, + "progress": 0.016291439533233643, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016291439533233643 + }, + { + "contact_quality": null, + "progress": 0.008464436046779156, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008464436046779156 + }, + { + "contact_quality": null, + "progress": 0.007733523845672607, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007733523845672607 + }, + { + "contact_quality": null, + "progress": 0.007662097457796335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007662097457796335 + } + ], + "candidate_progress": [ + 0.010246972553431988, + 0.009836931712925434, + 0.010462522506713867, + 0.016623934730887413, + 0.016291439533233643, + 0.008464436046779156, + 0.007733523845672607, + 0.007662097457796335 + ], + "candidate_source_chart_ids": [ + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-61bde0225d6f4175b7cb", + "ms-02311c7783bc67e188b4", + "ms-02311c7783bc67e188b4", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7", + "ms-d6da14bffe59c0265db7" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-0819eb89e627595e7e1b", + "generated_tangents": [ + [ + -0.27776890993118286, + 0.23108194768428802, + 0.7111798524856567, + 0.7763466835021973, + 0.9968110918998718, + 2.906764268875122, + 1.5565147399902344, + -0.4989220201969147, + 0.23751190304756165, + 1.8172210454940796, + -0.07352765649557114, + 1.018977403640747, + 2.1709094047546387, + 2.8995978832244873, + -0.5208884477615356, + -0.09437926113605499, + 0.8423108458518982, + 0.5564112663269043, + 0.42147862911224365, + 2.1392533779144287, + 1.0801944732666016 + ], + [ + -0.1273861676454544, + -0.1396070420742035, + 0.6843858957290649, + 0.24298560619354248, + 0.9184566140174866, + 2.1182150840759277, + 1.0224272012710571, + -0.6168533563613892, + 0.18282200396060944, + 1.6481122970581055, + 0.15509067475795746, + 1.137547254562378, + 2.268160820007324, + 1.603983759880066, + -0.5772895812988281, + -0.2041473090648651, + 0.9118913412094116, + 0.704179584980011, + 0.5581766366958618, + 1.6132352352142334, + 0.335203617811203 + ], + [ + -0.07269053161144257, + -0.07543421536684036, + 0.7156150341033936, + 0.1343604326248169, + 0.7264622449874878, + 2.192913770675659, + 1.2316503524780273, + -0.6132524013519287, + 0.22511693835258484, + 1.69596266746521, + 0.17755138874053955, + 1.3399709463119507, + 2.240403652191162, + 1.60073983669281, + -0.4946681261062622, + -0.20121850073337555, + 0.8695274591445923, + 0.6954562664031982, + 0.5624851584434509, + 1.67295241355896, + 0.40039920806884766 + ], + [ + -0.19697685539722443, + -0.05964571610093117, + 0.7484403848648071, + 0.49418261647224426, + 1.358931303024292, + 2.1272594928741455, + 1.354171633720398, + -0.20623332262039185, + 0.1664104014635086, + 1.4757176637649536, + 0.10749824345111847, + 1.0062272548675537, + 1.3730510473251343, + 1.345031499862671, + -0.4678349494934082, + -0.004985216073691845, + 0.6688669323921204, + 0.34790584444999695, + 0.3010469973087311, + 1.613962173461914, + 1.203932523727417 + ], + [ + -0.29352378845214844, + -0.030778557062149048, + 0.6847835779190063, + 0.4548744857311249, + 1.434896469116211, + 1.9511553049087524, + 1.2712209224700928, + -0.2169855535030365, + 0.16551634669303894, + 1.4247666597366333, + 0.042481109499931335, + 1.1290748119354248, + 1.3033252954483032, + 1.440265417098999, + -0.4396582841873169, + -0.036126572638750076, + 0.7003827095031738, + 0.4048020839691162, + 0.24394062161445618, + 1.579622745513916, + 1.2323073148727417 + ], + [ + -0.3463971018791199, + 0.010657303035259247, + 0.5432380437850952, + 0.30100831389427185, + 1.5649447441101074, + 1.5867369174957275, + 0.6540951132774353, + -0.5391755104064941, + -0.02725127898156643, + 1.5252676010131836, + 0.06962070614099503, + 1.1831287145614624, + 1.8866065740585327, + 1.6422253847122192, + -0.4391603469848633, + -0.12437775731086731, + 1.1717960834503174, + 0.9279109239578247, + 0.4423338770866394, + 1.3115346431732178, + 1.173444390296936 + ], + [ + -0.6493239998817444, + -0.0025205612182617188, + 0.35217803716659546, + 0.039193183183670044, + 2.0393121242523193, + 2.1324617862701416, + 0.8784822225570679, + -0.5305458307266235, + -0.0757417157292366, + 1.5512365102767944, + 0.19312569499015808, + 0.9525071382522583, + 1.7539006471633911, + 1.5624492168426514, + -0.5603730082511902, + -0.012362434528768063, + 1.032532811164856, + 0.4476777911186218, + 0.521888792514801, + 1.6565124988555908, + 1.1062722206115723 + ], + [ + -0.434856116771698, + 0.03118390589952469, + 0.4904160797595978, + 0.24183911085128784, + 1.500748872756958, + 1.5910472869873047, + 0.7564234733581543, + -0.6446245908737183, + 0.06703783571720123, + 1.3886953592300415, + 0.05164824053645134, + 1.3714869022369385, + 1.9783786535263062, + 1.5633913278579712, + -0.3470684587955475, + -0.16040073335170746, + 1.1508047580718994, + 0.7564806342124939, + 0.5159573554992676, + 1.4810285568237305, + 1.4211665391921997 + ] + ], + "generated_utilities": [ + 0.010246972553431988, + 0.009836931712925434, + 0.010462522506713867, + 0.016623934730887413, + 0.016291439533233643, + 0.008464436046779156, + 0.007733523845672607, + 0.007662097457796335 + ], + "group_id": "ms-0819eb89e627595e7e1b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07638250291347504, + 0.11176031827926636, + 0.139996737241745, + 0.06282445043325424, + 0.08466696739196777, + 0.0763862133026123, + 0.05953681468963623, + 0.06854937970638275, + 0.06834080070257187, + 0.09309522807598114, + 0.14572584629058838, + 0.07182137668132782, + 0.020047685131430626, + 0.05106111615896225, + 0.03756682202219963, + 0.061605654656887054 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.04522436857223511, + -0.3834587037563324, + 2.32004976272583, + 1.6386330127716064, + 1.508955717086792, + 1.681371808052063, + 2.9060468673706055, + -0.5984516143798828, + 0.36428770422935486, + 0.7442611455917358, + 1.163742184638977, + 0.4921322464942932, + 1.9586588144302368, + 3.335318088531494, + -0.02929070219397545, + 0.5442408919334412, + 2.4529502391815186, + 0.17471881210803986, + 1.804263710975647, + 1.5498225688934326, + 1.9392809867858887 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.19627264142036438, + 0.015629824250936508, + 0.723259687423706, + 0.0313839316368103, + 0.02149224281311035, + 0.7142634391784668, + 0.0, + -0.29767167568206787, + -0.10304851830005646, + 0.6168417930603027, + 0.0271989107131958, + 0.369077205657959, + 1.5924487113952637, + 0.0, + -0.11617732048034668, + -0.1419914811849594, + 0.9400725364685059, + -0.04139697551727295, + 0.7166774272918701, + 1.092864990234375, + 0.0 + ], + [ + -0.18197131156921387, + 0.30914804339408875, + 1.4557228088378906, + 1.4375734329223633, + 0.9600651264190674, + 1.7448713779449463, + 3.031123399734497, + -0.6073551177978516, + -0.4749099016189575, + 1.3016932010650635, + 0.32392996549606323, + 0.6961254477500916, + 2.1961381435394287, + 3.730100154876709, + -0.6154190897941589, + -0.3908892273902893, + 2.0994153022766113, + 0.18937554955482483, + 1.5752670764923096, + 2.2457275390625, + 1.5629570484161377 + ] + ], + "predicted_scores": [ + 3.1131200790405273, + 1.5541642904281616, + 1.505118727684021, + 1.397347331047058, + 0.9729986786842346, + 0.37713712453842163, + 0.31507235765457153, + 0.16730530560016632 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "582daf047d78a1b01018d54f5798143c4be6129fd7b6cbc025fc85818e519f3c", + "stored_base_utility": 0.07638250291347504, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19400137662887573, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19400137662887573 + }, + "base_progress": 0.19400137662887573, + "base_success": false, + "base_utility": 0.19400137662887573, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.047912776470184326, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.047912776470184326 + }, + { + "contact_quality": null, + "progress": 0.16342134773731232, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16342134773731232 + }, + { + "contact_quality": null, + "progress": 0.09078226238489151, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09078226238489151 + }, + { + "contact_quality": null, + "progress": 0.11114299297332764, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11114299297332764 + }, + { + "contact_quality": null, + "progress": 0.14748775959014893, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14748775959014893 + }, + { + "contact_quality": null, + "progress": 0.13230924308300018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13230924308300018 + }, + { + "contact_quality": null, + "progress": 0.11984594911336899, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11984594911336899 + }, + { + "contact_quality": null, + "progress": 0.0701046958565712, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0701046958565712 + } + ], + "candidate_progress": [ + 0.047912776470184326, + 0.16342134773731232, + 0.09078226238489151, + 0.11114299297332764, + 0.14748775959014893, + 0.13230924308300018, + 0.11984594911336899, + 0.0701046958565712 + ], + "candidate_source_chart_ids": [ + "ms-25ab20bb3da8ba34be95", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "generated_tangents": [ + [ + 0.9212105870246887, + 0.2495684176683426, + -0.7139983177185059, + -0.26197585463523865, + -0.16777950525283813, + -3.125133514404297, + -1.1343600749969482, + -0.845574140548706, + 0.1403491497039795, + 1.0603828430175781, + 1.0307753086090088, + 0.25075656175613403, + -1.175748586654663, + -0.15695838630199432, + -1.2099212408065796, + 0.036964427679777145, + 0.6747310757637024, + 1.717119812965393, + -0.639168381690979, + 0.5787088871002197, + 0.5156751871109009 + ], + [ + 1.031951904296875, + 0.10456676781177521, + -0.0990132987499237, + -0.8342064619064331, + 0.048051148653030396, + -3.0087099075317383, + -1.678978443145752, + -0.4555343687534332, + -0.05653058737516403, + 0.6307278275489807, + 0.35930153727531433, + -0.1259935200214386, + -1.54985511302948, + 0.01895079016685486, + -1.026986002922058, + -0.02540828101336956, + 0.022382229566574097, + 1.5241296291351318, + -1.186220407485962, + -0.3008204996585846, + -0.03713391721248627 + ], + [ + 0.9142172336578369, + 0.28473961353302, + -0.7407100200653076, + -0.4550410211086273, + 0.16757476329803467, + -2.450864791870117, + -1.1916437149047852, + -0.7311481237411499, + 0.011652294546365738, + 1.0108829736709595, + 0.9055538177490234, + 0.27949410676956177, + -1.4278541803359985, + -0.7882205247879028, + -1.1272212266921997, + 0.030480921268463135, + 0.27419817447662354, + 1.558565378189087, + -0.9363412261009216, + 0.057448357343673706, + 0.1634758859872818 + ], + [ + 0.8624707460403442, + 0.14740881323814392, + -0.5647199153900146, + -0.3894729018211365, + 0.13196495175361633, + -3.027218818664551, + -0.9061381220817566, + -0.737486720085144, + 0.03615472838282585, + 0.8685611486434937, + 0.8174701929092407, + -0.1609852910041809, + -1.6961089372634888, + -0.34921181201934814, + -1.0282783508300781, + 0.009532948955893517, + 0.2734566330909729, + 1.357501745223999, + -0.7216119766235352, + 0.01734524965286255, + 0.07203369587659836 + ], + [ + 0.9260347485542297, + 0.17288383841514587, + -0.5587950944900513, + -0.7025783061981201, + -0.01387980580329895, + -2.9452710151672363, + -0.9583986401557922, + -0.6184329986572266, + 0.008266683667898178, + 0.7364510297775269, + 0.6600156426429749, + -0.0791592001914978, + -1.6208281517028809, + -0.3753531575202942, + -1.1459007263183594, + -0.022931182757019997, + 0.1371820569038391, + 1.3399425745010376, + -0.8164740204811096, + -0.11073678731918335, + 0.2146928310394287 + ], + [ + 0.8769559860229492, + 0.06960363686084747, + -0.3903687298297882, + -0.33665645122528076, + -0.08544418215751648, + -2.8250210285186768, + -1.2471399307250977, + -0.6057186126708984, + 0.0977524146437645, + 0.8803013563156128, + 0.4490353763103485, + 0.02901233732700348, + -1.8310905694961548, + -0.4052550792694092, + -1.017613410949707, + 0.02303798496723175, + -0.03266887366771698, + 1.3069043159484863, + -1.017390489578247, + -0.5154598951339722, + -0.33940544724464417 + ], + [ + 1.0058937072753906, + 0.21089066565036774, + -0.47070208191871643, + -0.6376264095306396, + -0.10214352607727051, + -2.9263365268707275, + -1.0027015209197998, + -0.6247806549072266, + 0.056414879858493805, + 0.8938659429550171, + 0.6290707588195801, + 0.04730623960494995, + -1.6837979555130005, + -0.3781541585922241, + -1.1397086381912231, + -0.018741093575954437, + 0.14122837781906128, + 1.416974425315857, + -0.8748225569725037, + -0.10672138631343842, + 0.0648472011089325 + ], + [ + 0.9055119752883911, + 0.2079230099916458, + -0.10746070742607117, + -0.46652254462242126, + -0.21936234831809998, + -3.202359437942505, + -1.061741590499878, + -0.6140661239624023, + 0.07765284925699234, + 1.0384798049926758, + 0.6780832409858704, + 0.13602565228939056, + -1.3173234462738037, + -0.7119632363319397, + -1.2493627071380615, + 0.1700039654970169, + 0.18064826726913452, + 1.6471635103225708, + -0.8899497389793396, + -0.11035288870334625, + -0.1744442582130432 + ] + ], + "generated_utilities": [ + 0.047912776470184326, + 0.16342134773731232, + 0.09078226238489151, + 0.11114299297332764, + 0.14748775959014893, + 0.13230924308300018, + 0.11984594911336899, + 0.0701046958565712 + ], + "group_id": "ms-0dee962a5901fb8b5bbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23161475360393524, + 0.23611454665660858, + 0.1380448043346405, + 2.0, + 0.25444933772087097, + 0.25831863284111023, + 0.22753827273845673, + 0.2483263909816742, + 0.21622240543365479, + 0.17534737288951874, + 0.23816657066345215, + 0.09714888036251068, + 0.05844185873866081, + 0.12679120898246765, + 0.0472835898399353, + 0.03277532383799553 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7612967491149902, + 0.192471444606781, + -0.2699308395385742, + 0.017076104879379272, + -0.14400714635849, + -1.9032442569732666, + 0.0, + -0.28746497631073, + -0.2857179641723633, + -0.09024286270141602, + -0.03538453206419945, + 0.06432074308395386, + -0.2535901665687561, + 0.0, + 0.06180626153945923, + 0.16256888210773468, + -0.031329914927482605, + 1.563896656036377, + 0.04795600473880768, + 0.02485179901123047, + 0.0 + ], + [ + 1.2132649421691895, + 0.536715567111969, + -1.4273607730865479, + -1.0538015365600586, + -0.5170869827270508, + -3.4203505516052246, + -2.3889083862304688, + -0.7717514634132385, + -1.1465628147125244, + 0.8393621444702148, + -0.4419952630996704, + -0.23069629073143005, + -0.0637764036655426, + -1.5193647146224976, + -1.0273863077163696, + 0.39812424778938293, + -0.02874515950679779, + 2.7962026596069336, + 0.2370568960905075, + 1.5884850025177002, + 0.9987253546714783 + ], + [ + 2.1451404094696045, + 0.09629477560520172, + -1.8500688076019287, + 0.012462526559829712, + 0.13087880611419678, + -2.4062256813049316, + -3.303264856338501, + -1.3855981826782227, + -0.0292893648147583, + -0.17000514268875122, + 0.0005041323602199554, + -0.7487195134162903, + 0.06868582963943481, + -2.107095718383789, + -0.5890907645225525, + 0.1777246743440628, + -0.5576009750366211, + 2.767899990081787, + -0.3420543074607849, + 1.1357430219650269, + 1.8840694427490234 + ], + [ + 2.032343626022339, + 0.4649246335029602, + -0.5597869753837585, + 0.37935349345207214, + 0.14823919534683228, + -3.389478921890259, + -1.972822904586792, + -0.0014220476150512695, + -0.6147069334983826, + 0.9483420252799988, + 0.641323983669281, + -0.7033213973045349, + 0.4629128575325012, + -1.6774024963378906, + -1.1420938968658447, + 0.5891244411468506, + -0.712251603603363, + 2.1924612522125244, + 0.18694229423999786, + 0.2795591354370117, + 2.4723148345947266 + ], + [ + 2.299381732940674, + -0.6742787957191467, + -0.886760950088501, + -0.1637115478515625, + 0.32663029432296753, + -1.9267126321792603, + -1.9424545764923096, + -0.3586595058441162, + -1.1000827550888062, + 0.45532336831092834, + -0.4314403235912323, + -0.12869995832443237, + -0.032513588666915894, + -2.046614170074463, + -1.2942235469818115, + 0.3962012231349945, + 0.797533392906189, + 2.202794313430786, + 0.4938191771507263, + 0.15863806009292603, + 1.745660662651062 + ], + [ + 2.107815742492676, + 0.963295042514801, + -1.2979395389556885, + -1.0134333372116089, + -1.319197177886963, + -3.176518440246582, + -2.4782180786132812, + -1.3465216159820557, + -1.0473096370697021, + -0.22631770372390747, + 0.7881518006324768, + -1.4033093452453613, + 0.3855947256088257, + -1.3816941976547241, + -1.1257284879684448, + -0.7653472423553467, + 0.6845007538795471, + 2.9796371459960938, + 0.1967305988073349, + 1.4200057983398438, + 1.2843091487884521 + ], + [ + 1.7303649187088013, + 1.0728741884231567, + -1.1676844358444214, + -0.20355769991874695, + -0.8773125410079956, + -3.2701919078826904, + -2.1479928493499756, + -0.044036924839019775, + -1.252698302268982, + -0.01393243670463562, + 0.6447213292121887, + -0.7088868618011475, + 0.11592492461204529, + -2.1431162357330322, + -0.8284967541694641, + 0.4642922580242157, + -0.5207862854003906, + 1.8997660875320435, + -0.6910213828086853, + 0.2906065583229065, + 1.4081878662109375 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7612967491149902, + -0.5331565737724304, + -0.2699308395385742, + -0.20421931147575378, + -0.22613269090652466, + -1.9032442569732666, + 0.0, + 0.09110367298126221, + 0.009229481220245361, + 0.004811525344848633, + 0.656868577003479, + -0.28485387563705444, + -0.31901538372039795, + 0.0, + 0.08918565511703491, + 0.3355104923248291, + 0.026827186346054077, + 1.563896656036377, + 0.13480885326862335, + 0.02485179901123047, + 0.0 + ] + ], + "predicted_scores": [ + 7.601155757904053, + 6.7186150550842285, + 6.455691814422607, + 6.337220668792725, + 6.190089702606201, + 5.931434631347656, + 5.5402512550354, + 5.225919246673584 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "73f2826502af62b561f13c32614947588918c3deb631165739e66d787008fa98", + "stored_base_utility": 0.23161475360393524, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.04583283141255379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04583283141255379 + }, + "base_progress": 0.04583283141255379, + "base_success": false, + "base_utility": 0.04583283141255379, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06292174011468887, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06292174011468887 + }, + { + "contact_quality": null, + "progress": 0.06575056165456772, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06575056165456772 + }, + { + "contact_quality": null, + "progress": 0.04468737170100212, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04468737170100212 + }, + { + "contact_quality": null, + "progress": 0.05525680258870125, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05525680258870125 + }, + { + "contact_quality": null, + "progress": 0.05402831360697746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05402831360697746 + }, + { + "contact_quality": null, + "progress": 0.05949234962463379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05949234962463379 + }, + { + "contact_quality": null, + "progress": 0.06155836582183838, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06155836582183838 + }, + { + "contact_quality": null, + "progress": 0.07069015502929688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07069015502929688 + } + ], + "candidate_progress": [ + 0.06292174011468887, + 0.06575056165456772, + 0.04468737170100212, + 0.05525680258870125, + 0.05402831360697746, + 0.05949234962463379, + 0.06155836582183838, + 0.07069015502929688 + ], + "candidate_source_chart_ids": [ + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-38dff128761905bb1507", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-126eb8b36769f9ba4671", + "generated_tangents": [ + [ + 0.655849277973175, + 0.30569660663604736, + -0.2553529441356659, + -0.263614296913147, + -0.017318665981292725, + -2.6994242668151855, + -1.2797112464904785, + 1.3210679292678833, + -0.20580926537513733, + -0.34731701016426086, + 0.0599541999399662, + 1.1947097778320312, + -0.9926650524139404, + -0.22014300525188446, + -0.5412349700927734, + -0.006252258084714413, + 0.7201115489006042, + 0.35588404536247253, + 0.7991909384727478, + 1.6356327533721924, + 1.4817169904708862 + ], + [ + 0.5567323565483093, + 0.2999644875526428, + -0.22638460993766785, + -0.2857460081577301, + -0.12410396337509155, + -2.6662635803222656, + -1.1602342128753662, + 1.3528891801834106, + -0.19935408234596252, + -0.3467772305011749, + 0.14726582169532776, + 1.185856819152832, + -0.9297693967819214, + -0.17500300705432892, + -0.5039250254631042, + 0.00262271985411644, + 0.7704614400863647, + 0.38718703389167786, + 0.7173773050308228, + 1.7797887325286865, + 1.5975384712219238 + ], + [ + 0.6345928907394409, + 0.22035647928714752, + -0.3399869501590729, + -0.5475633144378662, + -0.6624675393104553, + -2.795095443725586, + -1.7817482948303223, + 1.4510209560394287, + -0.07520416378974915, + -0.5234764814376831, + 0.20408061146736145, + 1.3866987228393555, + -0.9252206087112427, + -0.6048867106437683, + -0.4693869352340698, + -0.05610787868499756, + 0.9850830435752869, + 0.2763737142086029, + 1.0204551219940186, + 1.0851173400878906, + 1.6064753532409668 + ], + [ + 0.6709209680557251, + 0.22418354451656342, + -0.5388002395629883, + -0.38371536135673523, + -0.5098652243614197, + -3.042370080947876, + -1.8117797374725342, + 1.4727284908294678, + -0.1018194705247879, + -0.40387922525405884, + 0.344486266374588, + 1.4169539213180542, + -1.0792994499206543, + -0.6196965575218201, + -0.34013456106185913, + -0.010105160064995289, + 0.863153874874115, + 0.366731733083725, + 0.9567697644233704, + 1.240022897720337, + 1.4136368036270142 + ], + [ + 0.7780305743217468, + 0.15041577816009521, + -0.03117486834526062, + -0.3764033913612366, + -0.3695783019065857, + -3.130204916000366, + -1.807060718536377, + 1.5753087997436523, + -0.17455875873565674, + -0.4455745816230774, + 0.33693787455558777, + 1.3635772466659546, + -0.7788095474243164, + -0.10962678492069244, + -0.20459851622581482, + -0.09904170036315918, + 0.6942819952964783, + 0.9156771898269653, + 0.57056725025177, + 1.858638048171997, + 0.898651123046875 + ], + [ + 0.5313800573348999, + 0.1547086536884308, + -0.4544283449649811, + -0.299678236246109, + 0.184084951877594, + -3.0529234409332275, + -1.6618046760559082, + 1.3184638023376465, + -0.1364424079656601, + -0.1288485825061798, + 0.4071817696094513, + 1.2905830144882202, + -0.7680494785308838, + -0.5091203451156616, + -0.3352624177932739, + -0.1254398226737976, + 0.7647642493247986, + 0.38689929246902466, + 0.3632575571537018, + 1.2556016445159912, + 1.437532901763916 + ], + [ + 0.5951058268547058, + 0.18803085386753082, + -0.6277573108673096, + -0.3599866032600403, + -0.4260461926460266, + -3.027994394302368, + -1.897934913635254, + 1.447753667831421, + -0.15206478536128998, + -0.3560253381729126, + 0.16618674993515015, + 1.4100892543792725, + -1.074182152748108, + -0.5514003038406372, + -0.37114202976226807, + -0.009665049612522125, + 0.8715071678161621, + 0.30882224440574646, + 0.9878675937652588, + 1.1838805675506592, + 1.028620958328247 + ], + [ + 0.4412482976913452, + 0.1250564008951187, + -0.45928481221199036, + -0.23636017739772797, + 0.05779939889907837, + -3.066941738128662, + -1.807917833328247, + 1.2976337671279907, + -0.10770668089389801, + -0.17227691411972046, + 0.4780229330062866, + 1.4018090963363647, + -0.8124645352363586, + -0.39200007915496826, + -0.3194437026977539, + -0.08693481981754303, + 0.6708048582077026, + 0.3947564959526062, + 0.3704075813293457, + 1.3500699996948242, + 1.2606252431869507 + ] + ], + "generated_utilities": [ + 0.06292174011468887, + 0.06575056165456772, + 0.04468737170100212, + 0.05525680258870125, + 0.05402831360697746, + 0.05949234962463379, + 0.06155836582183838, + 0.07069015502929688 + ], + "group_id": "ms-126eb8b36769f9ba4671", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05304628610610962, + 0.14190539717674255, + 0.1985912322998047, + 0.1210482120513916, + 0.18570420145988464, + 0.05259696766734123, + 0.1827520728111267, + 0.15291261672973633, + 0.07772710174322128, + 0.15551692247390747, + 0.08802877366542816, + 0.07544025033712387, + 0.08058679103851318, + 0.10843702405691147, + 0.023751219734549522, + 0.00727109145373106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.7799564599990845, + -0.0863567665219307, + -0.04370701313018799, + -0.04603701829910278, + -0.003975361585617065, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.02617853879928589, + 0.009054642170667648, + -0.050068795680999756, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + -0.007249712944030762, + 0.0760425329208374, + 1.0404269695281982, + -0.04351449012756348, + -0.010379016399383545, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + 0.005863763391971588, + -0.04370701313018799, + 0.010433673858642578, + -0.05825299024581909, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + 0.22243881225585938, + -0.007246788591146469, + 0.19961372017860413, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.11177664995193481, + -0.1665668487548828, + 1.0404269695281982, + -0.09226274490356445, + -0.18212640285491943, + 1.0522968769073486, + 0.0 + ], + [ + 0.7799564599990845, + -0.7482962012290955, + -0.04370701313018799, + -0.2370874285697937, + 0.3763168752193451, + -2.3430333137512207, + 0.0, + 0.8326367139816284, + -0.441777765750885, + -0.44057023525238037, + -0.6443575024604797, + 0.7451434135437012, + -0.6306462287902832, + 0.0, + 0.008161306381225586, + 0.22021520137786865, + 1.0404269695281982, + -0.09226274490356445, + -0.35497862100601196, + 1.0522968769073486, + 0.0 + ], + [ + 1.7799564599990845, + 0.04087676852941513, + -1.043707013130188, + -0.8482787609100342, + 0.3640388548374176, + -3.3430333137512207, + -2.8715221881866455, + 1.8326367139816284, + -0.4197058081626892, + -0.040397901087999344, + -0.37471577525138855, + 1.7451434135437012, + -1.6306462287902832, + -0.7785218954086304, + -0.5121169090270996, + -0.7797847986221313, + 2.0404269695281982, + 0.9077372550964355, + 0.645021378993988, + 2.0522968769073486, + 1.2749258279800415 + ], + [ + 1.6519862413406372, + 0.1747584044933319, + -0.9129671454429626, + -0.6869257092475891, + 0.12357090413570404, + -3.2319328784942627, + -2.704270839691162, + 1.7347972393035889, + -0.40612348914146423, + 0.23568139970302582, + -0.3912501931190491, + 1.8630532026290894, + -1.8548930883407593, + -0.9794526100158691, + -0.25177696347236633, + -0.6884964108467102, + 2.237109422683716, + 1.1246912479400635, + 0.7490227222442627, + 2.320941686630249, + 1.2039268016815186 + ], + [ + 1.7142105102539062, + 0.3941279351711273, + -1.021533489227295, + -1.1548089981079102, + 0.5052449107170105, + -3.714951992034912, + -3.04028058052063, + 1.529634952545166, + -0.22215746343135834, + 0.19690367579460144, + -0.12262976169586182, + 1.4847477674484253, + -1.490212082862854, + -0.7427945137023926, + -0.7506002187728882, + -0.9221256375312805, + 1.9211885929107666, + 0.5114957094192505, + 0.30887648463249207, + 2.4437777996063232, + 1.3625030517578125 + ], + [ + 1.3140592575073242, + -0.1129835918545723, + -0.9130957722663879, + -1.1089756488800049, + 0.7560886144638062, + -3.3889732360839844, + -3.1182656288146973, + 1.7436503171920776, + -0.9962512254714966, + 0.43109408020973206, + -0.8453629016876221, + 1.2057833671569824, + -2.194112777709961, + -0.22479140758514404, + -0.7258806824684143, + -0.9702881574630737, + 1.8160195350646973, + 0.649415135383606, + 0.6846981048583984, + 2.1997463703155518, + 1.7655411958694458 + ], + [ + 2.6430320739746094, + -0.7696453332901001, + -1.765415906906128, + -0.8281408548355103, + -0.1440155804157257, + -3.3335886001586914, + -3.398141860961914, + 2.4138498306274414, + 0.47537559270858765, + -0.9514593482017517, + 0.2674855887889862, + 2.6307411193847656, + -0.7876039743423462, + -0.35013434290885925, + -0.7511636018753052, + -0.04397457838058472, + 2.123702049255371, + 1.1604151725769043, + 0.7323322892189026, + 1.88338303565979, + 1.5078264474868774 + ] + ], + "predicted_scores": [ + 6.690057754516602, + 6.604052543640137, + 6.222714900970459, + 6.147878646850586, + 5.996226787567139, + 5.941082000732422, + 5.827487468719482, + 5.642694473266602 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "2390fc23ce8b50bdcdcbb1ab5f0583d2dad2b94fb871db73b1a8401265f59987", + "stored_base_utility": 0.05304628610610962, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.025880932807922363, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025880932807922363 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.025880932807922363, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-1f19fd22562f87bdb776", + "ms-babaeb20129c46551281", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-067cac2e4dbeddb12239", + "ms-c1620854ba52f09e24d5", + "ms-067cac2e4dbeddb12239", + "ms-cad6645f8175e38ad155" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank5", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-16661b2fd2c85f57a66c", + "generated_tangents": [ + [ + -0.6824389100074768, + -0.42353397607803345, + 1.9156477451324463, + 0.27319207787513733, + 1.1869573593139648, + 1.6918898820877075, + 3.035088300704956, + 1.200953722000122, + -0.2849206328392029, + -0.3664309084415436, + -0.02527526021003723, + 0.13283467292785645, + 1.7509948015213013, + 2.0090372562408447, + 0.9902731776237488, + -0.09048666059970856, + -1.5413249731063843, + -0.017665475606918335, + -0.3971679210662842, + -0.03746393322944641, + -1.81447172164917 + ], + [ + -0.5094736218452454, + -0.4551224112510681, + 1.1852335929870605, + -0.39674949645996094, + 1.1378552913665771, + 1.9335203170776367, + 3.5333211421966553, + 1.1739822626113892, + -0.08422532677650452, + -0.17456355690956116, + -0.05542765557765961, + 0.5721486210823059, + 1.4768439531326294, + 2.1573987007141113, + 0.47684183716773987, + -0.41097113490104675, + -1.140610694885254, + -0.003322198987007141, + -0.5198103189468384, + -1.5106821060180664, + -2.2797093391418457 + ], + [ + -0.17949147522449493, + -0.27099424600601196, + 1.509683609008789, + 0.11790081858634949, + 1.112421989440918, + 2.195878028869629, + 3.7887556552886963, + 1.2573126554489136, + -0.15526361763477325, + 0.2515209913253784, + -0.28459781408309937, + 0.695488691329956, + 1.9764295816421509, + 2.660031318664551, + 0.7410562634468079, + -0.15210482478141785, + -0.8240321278572083, + 0.12183477729558945, + -0.057398468255996704, + -0.7678194046020508, + -2.11991548538208 + ], + [ + -0.2508142590522766, + -0.21315686404705048, + 1.3031139373779297, + 0.30783143639564514, + 1.0800400972366333, + 2.267934799194336, + 3.4985358715057373, + 1.4066815376281738, + -0.21535561978816986, + 0.1340244561433792, + -0.3501300811767578, + 0.6035999655723572, + 1.8388158082962036, + 2.672978401184082, + 0.7043142318725586, + -0.11049807071685791, + -0.7912341952323914, + 0.2669382095336914, + 0.012259066104888916, + -0.8488905429840088, + -1.793541431427002 + ], + [ + -0.09856274724006653, + -0.5983659625053406, + 0.6540148258209229, + -0.5010594129562378, + 0.6528372764587402, + 3.1899161338806152, + 3.608482837677002, + 1.4343825578689575, + -0.012374132871627808, + 0.0454079732298851, + -0.42775359749794006, + 0.2738773226737976, + 1.513808012008667, + 2.3734359741210938, + 0.9223951697349548, + -0.2623382806777954, + -1.3865463733673096, + 0.2606728672981262, + -0.2553842067718506, + -1.0965392589569092, + -1.938814640045166 + ], + [ + -0.3856390118598938, + -0.46329641342163086, + 0.7654943466186523, + -0.19061198830604553, + 0.8493849635124207, + 2.530032157897949, + 3.7486729621887207, + 1.2035999298095703, + -0.2593596875667572, + -0.452076256275177, + -0.05605132877826691, + 0.05742599070072174, + 0.48679158091545105, + 1.7778207063674927, + 0.5767878293991089, + -0.3285136818885803, + -0.5391584634780884, + -0.20521382987499237, + -0.0013042688369750977, + -1.6672983169555664, + -2.5346899032592773 + ], + [ + -0.08828580379486084, + -0.7060015201568604, + 0.8980893492698669, + -0.5621912479400635, + 0.8083421587944031, + 2.847586154937744, + 4.094493389129639, + 1.091495394706726, + -0.12258994579315186, + -0.29970160126686096, + -0.511314868927002, + 0.5406684875488281, + 1.621792197227478, + 1.9818919897079468, + 0.7899641990661621, + -0.22454890608787537, + -1.068435549736023, + 0.565302312374115, + -0.02269512414932251, + -1.276261806488037, + -1.887265920639038 + ], + [ + -0.15478308498859406, + -0.167904794216156, + 1.6011892557144165, + 0.09527513384819031, + 1.1605334281921387, + 2.3821661472320557, + 4.203315734863281, + 1.4099047183990479, + -0.20276464521884918, + 0.10758756846189499, + -0.07489912956953049, + 0.6418052911758423, + 1.7774180173873901, + 3.06317138671875, + 0.3030961751937866, + -0.20969876646995544, + -0.37359678745269775, + -0.010742828249931335, + 0.18264299631118774, + -0.5141030550003052, + -1.8507068157196045 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.025880932807922363, + 2.0, + 2.0 + ], + "group_id": "ms-16661b2fd2c85f57a66c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.23635749518871307, + 0.1864171028137207, + 0.18280422687530518, + 0.16576561331748962, + 0.2967458665370941, + 0.2833873927593231, + 0.2587948143482208, + 0.2695174813270569, + 0.2239263951778412, + 0.15686756372451782, + 0.2505500912666321, + 0.12250886857509613, + 0.16453635692596436, + 2.0, + 0.13450095057487488, + 0.11914549767971039 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.12099093198776245, + 0.06886856257915497, + 0.2952747344970703, + -0.1764048933982849, + -0.12411242723464966, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + -0.046127498149871826, + 0.12581980228424072, + 0.042072445154190063, + -0.06942430138587952, + 0.7517287731170654, + 0.0, + 0.690804123878479, + -0.06023237854242325, + 0.049633681774139404, + 0.01943105459213257, + 0.013327360153198242, + -0.8525059223175049, + 0.0 + ], + [ + 0.3599843978881836, + -0.34296315908432007, + 0.2952747344970703, + -0.3797188997268677, + -0.41771262884140015, + 0.8606208562850952, + 0.0, + 1.3532578945159912, + 0.06634362041950226, + 0.138022780418396, + -0.2342309057712555, + -0.36017337441444397, + 0.7517287731170654, + 0.0, + 0.690804123878479, + 0.10550708323717117, + -0.35431206226348877, + 0.16382712125778198, + 0.06674730777740479, + -0.8525059223175049, + 0.0 + ], + [ + 0.567882239818573, + -0.4729353189468384, + 1.6256722211837769, + -1.5372763872146606, + 1.0487556457519531, + 1.5327277183532715, + 2.2847697734832764, + 2.6703484058380127, + 0.4013517200946808, + -1.3629915714263916, + 0.6385155916213989, + 0.2664203345775604, + 1.2691267728805542, + 1.2942520380020142, + 2.2084813117980957, + 0.6982719302177429, + -0.8422143459320068, + 1.070263385772705, + -0.9558742046356201, + -1.3996182680130005, + -1.4242117404937744 + ], + [ + 0.6830785274505615, + -0.5584439635276794, + 0.841848611831665, + -0.5293771624565125, + -0.14354348182678223, + 2.097932815551758, + 2.9733188152313232, + 2.603226661682129, + -0.1639159619808197, + -0.918163537979126, + 1.002030849456787, + 0.40132570266723633, + 1.1751198768615723, + 1.455662727355957, + 2.3202755451202393, + -0.18584474921226501, + -1.3483099937438965, + 0.0660635232925415, + 0.4330971837043762, + -1.1989169120788574, + -2.0896859169006348 + ], + [ + 0.44418102502822876, + 0.07469291985034943, + 1.6960139274597168, + -1.329486608505249, + 1.3067576885223389, + 2.2694292068481445, + 2.0299551486968994, + 1.6317954063415527, + 0.7472026348114014, + -0.12518703937530518, + 0.3035789430141449, + -0.5889836549758911, + 2.345867395401001, + 1.9470158815383911, + 0.8398547172546387, + -0.6090301871299744, + -1.5060018301010132, + 0.5695335865020752, + -0.6025367975234985, + -1.8703374862670898, + -2.728482246398926 + ], + [ + 0.31904691457748413, + 0.04995492100715637, + 1.3823881149291992, + -0.5106721520423889, + -0.2236303687095642, + 0.9921484589576721, + 1.5933295488357544, + 2.741320848464966, + -0.21808990836143494, + -0.7775652408599854, + 1.224307894706726, + -0.08168366551399231, + 2.692474842071533, + 2.269430160522461, + 2.3270838260650635, + -0.2901480495929718, + -1.2098140716552734, + -0.07570821046829224, + -1.1340655088424683, + -1.9999847412109375, + -1.6802332401275635 + ], + [ + -0.6257733106613159, + 0.3840647339820862, + 0.9012271165847778, + -1.248098611831665, + -0.1650761365890503, + 2.7645747661590576, + 2.7588071823120117, + 3.0172548294067383, + 0.4120163321495056, + -1.6110656261444092, + -0.10423526167869568, + 0.5334746241569519, + 1.7809263467788696, + 0.5690644383430481, + 1.5881801843643188, + 0.8860509991645813, + -1.1502506732940674, + 0.7821243405342102, + -0.549727201461792, + -1.326378345489502, + -1.5100535154342651 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.2773973345756531, + -0.13796104490756989, + 1.2952747344970703, + -1.0323810577392578, + 0.5605664849281311, + 1.8606208562850952, + 2.439758777618408, + 2.353257894515991, + 0.24642756581306458, + -0.861977219581604, + 0.4799116551876068, + 0.264199823141098, + 1.7517287731170654, + 1.4282559156417847, + 1.690804123878479, + 0.1215878501534462, + -0.9322434067726135, + 0.8536378145217896, + -0.35758620500564575, + -1.8525059223175049, + -1.910990595817566 + ], + [ + -0.4916617274284363, + -0.5934253931045532, + 2.074575424194336, + -0.9512085318565369, + 0.3705800175666809, + 1.691048264503479, + 2.2907681465148926, + 1.532329797744751, + 0.8991785049438477, + -0.7733803987503052, + 1.3577321767807007, + 0.9844847917556763, + 1.7379451990127563, + 2.050845146179199, + 0.7804086804389954, + -0.8400509357452393, + -1.6339564323425293, + 0.9922879934310913, + 0.6006971597671509, + -1.0612030029296875, + -1.3134464025497437 + ] + ], + "predicted_scores": [ + 7.575928211212158, + 5.902347087860107, + 4.982975959777832, + 4.682953834533691, + 4.231492519378662, + 3.7651405334472656, + 3.3235254287719727, + 2.9474380016326904 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "74b116fd9ceddd5b2261edd96a14934a4b4a78a40191a654d6743661b819f407", + "stored_base_utility": 0.23635749518871307, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.0008456110954284668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008456110954284668 + }, + { + "contact_quality": null, + "progress": 0.0010655125370249152, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0010655125370249152 + }, + { + "contact_quality": null, + "progress": 0.0008172591333277524, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0008172591333277524 + }, + { + "contact_quality": null, + "progress": 0.0007147590513341129, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007147590513341129 + }, + { + "contact_quality": null, + "progress": 0.0007733504171483219, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0007733504171483219 + }, + { + "contact_quality": null, + "progress": 0.0009190241689793766, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0009190241689793766 + }, + { + "contact_quality": null, + "progress": 0.000718792260158807, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000718792260158807 + }, + { + "contact_quality": null, + "progress": 0.0012095769634470344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0012095769634470344 + } + ], + "candidate_progress": [ + 0.0008456110954284668, + 0.0010655125370249152, + 0.0008172591333277524, + 0.0007147590513341129, + 0.0007733504171483219, + 0.0009190241689793766, + 0.000718792260158807, + 0.0012095769634470344 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-8e12f7ad446482627dd3", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-20e1efbb745fd02abc95", + "generated_tangents": [ + [ + -0.2970411777496338, + -0.13891279697418213, + 1.6107505559921265, + -0.6531276106834412, + 1.6390912532806396, + 2.8007149696350098, + 1.9549026489257812, + -0.2721187472343445, + 0.03811945766210556, + 1.6218369007110596, + 0.4281376898288727, + 1.174232840538025, + 2.360860586166382, + 2.097445249557495, + -1.3558846712112427, + -0.16812588274478912, + 1.9791476726531982, + 1.03844153881073, + 0.5798172950744629, + 2.5662693977355957, + 1.8120908737182617 + ], + [ + -0.1011519581079483, + -0.2186785191297531, + 1.675442099571228, + -0.8823689222335815, + 1.8577234745025635, + 3.0255486965179443, + 2.3365561962127686, + -0.33499643206596375, + 0.13097485899925232, + 1.560232400894165, + 0.5682781934738159, + 1.2047090530395508, + 2.4729018211364746, + 1.7730436325073242, + -1.165023922920227, + -0.33508002758026123, + 1.8437622785568237, + 0.5554348230361938, + 0.1301388293504715, + 3.2331323623657227, + 1.0670866966247559 + ], + [ + -0.3184179663658142, + -0.06403876841068268, + 1.6588443517684937, + -0.5006476044654846, + 1.9847056865692139, + 3.2442283630371094, + 1.751053810119629, + -0.53663170337677, + 0.056220643222332, + 1.5616052150726318, + 0.4458616077899933, + 1.088004469871521, + 2.0531327724456787, + 2.18585205078125, + -1.1377735137939453, + -0.2569707930088043, + 1.9332804679870605, + 0.8343563079833984, + 0.5231003761291504, + 2.9470410346984863, + 2.1021714210510254 + ], + [ + -0.4304850697517395, + -0.07702029496431351, + 1.6036697626113892, + -0.7226977348327637, + 1.712554693222046, + 3.1905558109283447, + 1.81992769241333, + -0.4713207185268402, + 0.07778530567884445, + 1.6159635782241821, + 0.3994346261024475, + 1.1715067625045776, + 2.044454574584961, + 2.1256802082061768, + -1.3056920766830444, + -0.23062705993652344, + 1.9890633821487427, + 0.8633118271827698, + 0.4869081974029541, + 2.961604118347168, + 1.9759396314620972 + ], + [ + -0.3146146535873413, + -0.06958557665348053, + 1.778191089630127, + -0.53082275390625, + 1.1891772747039795, + 3.3179545402526855, + 1.4995636940002441, + -0.42483940720558167, + 0.14092330634593964, + 1.46672523021698, + 0.5188098549842834, + 1.4428155422210693, + 2.558320999145508, + 2.306028366088867, + -1.2591758966445923, + -0.2865239381790161, + 2.061889171600342, + 0.731976330280304, + 0.6196022033691406, + 3.5407614707946777, + 1.8295236825942993 + ], + [ + -0.1450331062078476, + -0.11107157170772552, + 1.6518384218215942, + -0.7693350315093994, + 1.1015067100524902, + 3.1857244968414307, + 1.6807351112365723, + -0.011720404028892517, + 0.1923666000366211, + 1.7378132343292236, + 0.44404587149620056, + 1.0362420082092285, + 2.450160503387451, + 1.815553069114685, + -1.4958609342575073, + -0.4214487671852112, + 2.145357608795166, + 0.8884361386299133, + 0.7274243831634521, + 2.626995801925659, + 1.2838307619094849 + ], + [ + -0.5335362553596497, + -0.12909740209579468, + 1.5401071310043335, + -0.6026504635810852, + 1.5971753597259521, + 3.320101022720337, + 1.6288726329803467, + -0.4474392235279083, + 0.07987426966428757, + 1.5799193382263184, + 0.5128104090690613, + 1.0947915315628052, + 1.9190994501113892, + 2.4076898097991943, + -1.3900490999221802, + -0.28468674421310425, + 1.9677813053131104, + 0.8986243605613708, + 0.6235958933830261, + 2.8599469661712646, + 2.1499834060668945 + ], + [ + -0.38367998600006104, + -0.203634575009346, + 1.4452296495437622, + -0.5168579816818237, + 1.2648968696594238, + 3.1302900314331055, + 1.5878973007202148, + -0.13402852416038513, + 0.06269306689500809, + 1.4342036247253418, + 0.7396007776260376, + 0.93980473279953, + 1.8660436868667603, + 2.7143394947052, + -1.0735108852386475, + -0.21826890110969543, + 2.111849546432495, + 1.0680543184280396, + 0.5848070383071899, + 3.2526438236236572, + 1.787776231765747 + ] + ], + "generated_utilities": [ + 0.0008456110954284668, + 0.0010655125370249152, + 0.0008172591333277524, + 0.0007147590513341129, + 0.0007733504171483219, + 0.0009190241689793766, + 0.000718792260158807, + 0.0012095769634470344 + ], + "group_id": "ms-20e1efbb745fd02abc95", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13665315508842468, + 0.1696699559688568, + 0.049180448055267334, + 0.05942712351679802, + 0.22736775875091553, + 0.0799722671508789, + 0.17261533439159393, + 0.21202586591243744, + 0.18980810046195984, + 0.24345096945762634, + 0.08349347114562988, + 0.11624568700790405, + 0.17245106399059296, + 0.07060472667217255, + 0.041495781391859055, + 0.028585990890860558 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.08645056188106537, + -0.060760222375392914, + 0.6493666172027588, + -0.016124725341796875, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + -0.2971414625644684, + 0.023367643356323242, + 0.6914786100387573, + -0.1027369350194931, + 0.1733318567276001, + 1.2948026657104492, + 0.0, + -0.30476057529449463, + -0.10394972562789917, + 0.9822354316711426, + 0.055961012840270996, + 0.004969358444213867, + 2.122485637664795, + 0.0 + ], + [ + 0.046016186475753784, + -0.24098780751228333, + 0.6493666172027588, + 0.05737954378128052, + 0.18658411502838135, + 1.8734135627746582, + 0.0, + 0.22233793139457703, + 0.3896067142486572, + 0.6914786100387573, + -0.061455681920051575, + 0.17170286178588867, + 1.2948026657104492, + 0.0, + -0.4717581272125244, + -0.5722612738609314, + 0.9822354316711426, + 0.3690842390060425, + 0.33333829045295715, + 2.122485637664795, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.590132236480713, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.670657634735107, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.27303409576416 + ], + [ + -0.4654892086982727, + -0.6659128069877625, + 2.053840160369873, + -0.17420679330825806, + 1.1238669157028198, + 2.773953914642334, + 2.9706993103027344, + -0.9652835726737976, + -0.6603021621704102, + 1.957632303237915, + 0.30636119842529297, + 1.0932379961013794, + 2.4988603591918945, + 2.1735024452209473, + -0.7229697108268738, + -0.6565687656402588, + 2.1097376346588135, + 0.79427170753479, + 1.1421128511428833, + 3.200641393661499, + 3.3294520378112793 + ], + [ + 0.8783296346664429, + -0.6790766716003418, + 1.3789844512939453, + -1.578566551208496, + 1.2084856033325195, + 1.906846046447754, + 2.7278928756713867, + -0.8570022583007812, + -0.3114888072013855, + 2.660332202911377, + 0.4615887999534607, + -0.16199523210525513, + 1.9830265045166016, + 2.4224376678466797, + -0.9260885715484619, + 0.5555675029754639, + 2.5474588871002197, + 0.0070996880531311035, + 1.627825379371643, + 3.9487287998199463, + 3.03330135345459 + ], + [ + -0.1274794638156891, + -0.26015421748161316, + 2.5414626598358154, + -1.0220305919647217, + 1.4708960056304932, + 2.0982167720794678, + 3.1457836627960205, + -1.0992621183395386, + -0.9075721502304077, + 0.8330968022346497, + 0.15631377696990967, + 1.3095502853393555, + 1.8582203388214111, + 2.101317882537842, + -0.9184049367904663, + 0.4676415026187897, + 2.784534454345703, + 0.4529607892036438, + 0.6403366327285767, + 3.1566896438598633, + 2.5268735885620117 + ], + [ + 0.9289348125457764, + 0.1545984447002411, + 1.6792887449264526, + -0.6400681138038635, + 0.5956984758377075, + 2.5183756351470947, + 1.6261346340179443, + -0.019961506128311157, + -0.2638699412345886, + 1.310987114906311, + 0.4996453523635864, + -0.06606829166412354, + 1.9830868244171143, + 2.392476797103882, + -0.3871336579322815, + -0.7797489166259766, + 1.927225947380066, + 0.4438522458076477, + 1.0257132053375244, + 3.211881160736084, + 3.374328851699829 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05862678587436676, + -0.042836256325244904, + 1.6493666172027588, + -0.7465556859970093, + 1.1865841150283813, + 2.873413562774658, + 2.2950661182403564, + -0.48127153515815735, + -0.11381363868713379, + 1.6914786100387573, + 0.06965510547161102, + 0.8288453221321106, + 2.294802665710449, + 2.3353288173675537, + -1.3047605752944946, + -0.39009055495262146, + 1.9822354316711426, + 0.9993376731872559, + 0.81731116771698, + 3.122485637664795, + 2.63651704788208 + ], + [ + -0.045653510838747025, + 0.00397084653377533, + 1.3591042757034302, + -0.6414353847503662, + 0.8377937078475952, + 2.8277816772460938, + 2.203674554824829, + -0.1951642632484436, + 0.12272481620311737, + 1.7411555051803589, + -0.004818864166736603, + 0.8769108057022095, + 2.243906021118164, + 2.4328396320343018, + -1.6330946683883667, + -0.4807296395301819, + 2.2988247871398926, + 0.8021057844161987, + 0.6970767378807068, + 2.8728103637695312, + 2.288278102874756 + ], + [ + -0.4273069500923157, + 0.16743427515029907, + 1.538269281387329, + -0.7002415657043457, + 0.7622614502906799, + 2.805328369140625, + 2.267277479171753, + -0.6541028022766113, + -0.2980780303478241, + 1.456940770149231, + 0.5105806589126587, + 1.040317177772522, + 2.4321720600128174, + 2.696331024169922, + -1.1555191278457642, + -0.025772064924240112, + 1.6384800672531128, + 1.4557092189788818, + 0.6594157814979553, + 3.1446406841278076, + 2.249678134918213 + ], + [ + -0.6431475877761841, + -0.24224388599395752, + 1.3122833967208862, + -0.16711556911468506, + 0.9232780337333679, + 2.850522994995117, + 2.645284414291382, + -0.379825621843338, + -0.6750738620758057, + 1.1236441135406494, + 0.4938291311264038, + 1.1075668334960938, + 2.203859806060791, + 2.799032688140869, + -1.359815239906311, + -0.5104091763496399, + 2.0970702171325684, + 1.057171106338501, + 1.1167995929718018, + 2.8190994262695312, + 2.2184524536132812 + ] + ], + "predicted_scores": [ + 1.0665324926376343, + 0.9473126530647278, + 0.6300321221351624, + 0.44677799940109253, + 0.2111024409532547, + 0.19814617931842804, + 0.05749352276325226, + -0.8892081379890442 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1375492786", + "selected_index": 0, + "state_hash": "56acb365a02ebac0ac4893531e168df3c430b486416156feeea40eeb178bda3e", + "stored_base_utility": 0.13665315508842468, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2dc5138251afd26a040d", + "generated_tangents": [ + [ + 1.6630635261535645, + 0.3364940285682678, + -1.1061028242111206, + -0.5212184190750122, + 0.6414185166358948, + 0.31022578477859497, + -0.9027884602546692, + 0.8580968379974365, + 0.018770955502986908, + -0.2807011306285858, + -0.08123781532049179, + 0.2720161974430084, + -2.3702707290649414, + -2.2326834201812744, + 0.6190675497055054, + 0.5045605301856995, + 0.8924949169158936, + -1.502112627029419, + 1.245079517364502, + -1.888390302658081, + -0.6553448438644409 + ], + [ + 1.4316965341567993, + 0.1998540759086609, + -1.1474429368972778, + -0.5519155263900757, + 0.22836831212043762, + -0.3073066473007202, + -0.3848276734352112, + 0.5623452663421631, + 0.14031925797462463, + -0.14810025691986084, + -0.30554065108299255, + -0.1590932011604309, + -3.0235400199890137, + -2.117691993713379, + 0.45407184958457947, + 0.37112584710121155, + 0.6169575452804565, + -1.341200828552246, + 0.9203458428382874, + -2.283836603164673, + -1.25898015499115 + ], + [ + 1.5809606313705444, + 0.3620300889015198, + -1.0180070400238037, + -0.8809640407562256, + 0.6343622207641602, + 0.7183874845504761, + -1.0426461696624756, + 1.1562525033950806, + -0.0012709833681583405, + -0.3069444000720978, + -0.26884499192237854, + 0.3377997875213623, + -2.921963930130005, + -2.3162052631378174, + 0.40847247838974, + 0.453095942735672, + 0.9378368258476257, + -1.4076868295669556, + 1.0142264366149902, + -1.9853324890136719, + -0.5875377655029297 + ], + [ + 1.580847978591919, + 0.3325806260108948, + -1.0962201356887817, + -0.6875804662704468, + 0.42038458585739136, + 0.5779291391372681, + -0.9634775519371033, + 0.9584397077560425, + -0.020952241495251656, + -0.20275816321372986, + -0.16259431838989258, + 0.35765987634658813, + -2.7152206897735596, + -2.415917158126831, + 0.557228147983551, + 0.4680039584636688, + 0.9834054708480835, + -1.5287296772003174, + 0.9148144721984863, + -1.8712751865386963, + -0.8346226215362549 + ], + [ + 1.592742681503296, + 0.34411555528640747, + -0.8949155807495117, + -0.6642281413078308, + 0.12804077565670013, + 0.6749277114868164, + -1.140320062637329, + 0.9702752828598022, + 0.03766300529241562, + -0.13944312930107117, + -0.32787424325942993, + 0.3410719633102417, + -2.911372184753418, + -2.625809907913208, + 0.4719639718532562, + 0.4915294051170349, + 1.0420522689819336, + -1.4377121925354004, + 0.8342892527580261, + -1.9092910289764404, + -0.8492064476013184 + ], + [ + 1.3482401371002197, + 0.23607315123081207, + -1.1230732202529907, + -0.791126012802124, + -0.015790238976478577, + -0.6648694276809692, + -1.387589454650879, + 0.7008053064346313, + 0.05333154648542404, + 0.018860504031181335, + 0.07875882089138031, + 0.06073237955570221, + -2.0448343753814697, + -1.5583157539367676, + 0.6191694140434265, + 0.2761877775192261, + 0.8809552788734436, + -1.0871162414550781, + 0.9719794392585754, + -1.7854135036468506, + -0.671933650970459 + ], + [ + 1.5167537927627563, + 0.3650404214859009, + -1.3342314958572388, + -0.5989227294921875, + 0.4047403037548065, + 0.6936545372009277, + -0.270136296749115, + 1.1136995553970337, + 0.04881930351257324, + -0.187240868806839, + -0.02180282026529312, + 0.333523690700531, + -2.9222593307495117, + -2.319345712661743, + 0.3447681665420532, + 0.42478007078170776, + 1.274926781654358, + -1.4496917724609375, + 0.9142850637435913, + -1.9181504249572754, + -0.8424738645553589 + ], + [ + 1.5288500785827637, + 0.28309112787246704, + -0.9288069009780884, + -0.3774929940700531, + -0.06359642744064331, + -0.5223523378372192, + -0.5713916420936584, + 0.33994153141975403, + 0.1605323851108551, + -0.13476330041885376, + -0.49160775542259216, + -0.204166442155838, + -3.1072325706481934, + -2.0889811515808105, + 0.5006641149520874, + 0.34123772382736206, + 0.6568778157234192, + -1.4446017742156982, + 1.1540147066116333, + -2.175333261489868, + -1.2963474988937378 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-2dc5138251afd26a040d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1373138427734375, + 0.1398601531982422, + 0.11154793202877045, + 0.07755142450332642, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7500379085540771, + -0.32870030403137207, + -0.3281364440917969, + 0.020404458045959473, + 0.03884005546569824, + 0.26651138067245483, + 0.0, + 0.2614295184612274, + 0.02006034180521965, + 0.6256309151649475, + 0.10119366645812988, + 0.0365104079246521, + -1.9807136058807373, + 0.0, + -0.4040032625198364, + -0.3390212059020996, + 0.444367915391922, + -1.3744113445281982, + 0.3090496063232422, + -1.6372854709625244, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7500379085540771, + 0.1347455233335495, + -1.3281364440917969, + -0.3984496593475342, + -0.6640693545341492, + -0.7334886193275452, + -1.63351309299469, + 0.4696621894836426, + -0.03721961751580238, + 0.0896599143743515, + -0.8988063335418701, + -0.047963276505470276, + -2.9807136058807373, + -2.542311906814575, + 0.5875393152236938, + 0.41857796907424927, + 0.32660895586013794, + -2.3744113445281982, + 0.9575246572494507, + -2.6372854709625244, + -1.9586280584335327 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.26702618598938, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.08462381362915, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.9172561168670654 + ], + [ + 1.7928142547607422, + -0.046942055225372314, + -1.3733142614364624, + -0.4212188422679901, + -0.5566414594650269, + -0.9966021180152893, + -1.7926092147827148, + 0.37284746766090393, + 0.18181614577770233, + -0.056899696588516235, + -0.8793746829032898, + -0.06206277757883072, + -3.2539288997650146, + -2.3651227951049805, + 0.543649435043335, + 0.4969414472579956, + 0.3953157365322113, + -2.6153955459594727, + 1.239845633506775, + -2.9289329051971436, + -2.1138057708740234 + ], + [ + 1.4313925504684448, + 0.3287010192871094, + -1.4217545986175537, + -0.11546039581298828, + -0.38684383034706116, + -0.3866080045700073, + -1.4945378303527832, + 0.4180656969547272, + 0.08673545718193054, + -0.30675017833709717, + -0.8370212316513062, + -0.3124745488166809, + -3.2861690521240234, + -2.5144338607788086, + 0.7133253216743469, + 0.5272455215454102, + 0.6288780570030212, + -2.7024707794189453, + 1.0730262994766235, + -2.603350877761841, + -1.8104162216186523 + ], + [ + 2.0841901302337646, + 0.18348857760429382, + -1.4159170389175415, + -0.5868262052536011, + -0.3290694057941437, + -0.8189557790756226, + -1.811798334121704, + 0.8927561044692993, + -0.4510294198989868, + -0.1373635083436966, + -0.46978816390037537, + -0.2017078548669815, + -3.1228857040405273, + -2.619961977005005, + 0.7154320478439331, + 0.4623824954032898, + 0.8127838373184204, + -2.5683043003082275, + 0.8921399116516113, + -2.799823522567749, + -1.6220401525497437 + ], + [ + 1.5681514739990234, + -0.11813957989215851, + -1.066882848739624, + -0.4835556447505951, + -0.4833664894104004, + -0.1345365047454834, + -1.872452735900879, + 1.059093952178955, + -0.3537854254245758, + 0.5130321979522705, + -0.7769420146942139, + -0.4462882876396179, + -3.249842882156372, + -2.2144932746887207, + 0.6907999515533447, + 0.14545467495918274, + 0.8816275596618652, + -2.4357430934906006, + 1.424092411994934, + -2.656961441040039, + -2.3529839515686035 + ], + [ + 2.0935373306274414, + 0.20671290159225464, + -1.0638595819473267, + -0.37751641869544983, + -0.5913090705871582, + -0.3702518939971924, + -1.3611384630203247, + 0.8432458639144897, + 0.4395832121372223, + -0.19578908383846283, + -1.2028591632843018, + -0.07535962760448456, + -3.2909016609191895, + -2.9216225147247314, + 1.1581575870513916, + 0.9284459352493286, + -0.3347962498664856, + -1.9277539253234863, + 0.8971017003059387, + -3.245394229888916, + -2.1499102115631104 + ], + [ + 2.5108895301818848, + -0.2977973222732544, + -1.458537220954895, + -0.5705423355102539, + -1.4453105926513672, + -0.4854125380516052, + -2.383378028869629, + 0.39735814929008484, + -0.3523370325565338, + 0.4586251974105835, + -1.2388068437576294, + 0.1082226037979126, + -3.5858566761016846, + -1.8809643983840942, + -0.18688076734542847, + 0.0871247947216034, + 0.0062812864780426025, + -3.012958526611328, + 1.0266033411026, + -3.1649136543273926, + -2.6460556983947754 + ], + [ + 1.2948591709136963, + 0.21422013640403748, + -1.3904463052749634, + -0.13111025094985962, + -0.1661812961101532, + -0.4983506500720978, + -2.0006139278411865, + -0.4048727750778198, + 0.10130663216114044, + 0.1955089420080185, + -0.27839744091033936, + 0.8026529550552368, + -3.2792744636535645, + -1.6839708089828491, + 0.6468846201896667, + -0.2772523760795593, + 0.45975640416145325, + -2.612403631210327, + 1.6571701765060425, + -1.8971503973007202, + -1.2593979835510254 + ], + [ + 0.9154124855995178, + -0.2073056846857071, + -2.1218466758728027, + -0.4230979084968567, + -1.2478010654449463, + -0.5439814925193787, + -1.185287594795227, + -0.06685006618499756, + -0.7776275873184204, + -0.015365853905677795, + -1.3256038427352905, + 0.46163398027420044, + -2.8853814601898193, + -2.387786865234375, + 1.2300693988800049, + 0.9541484117507935, + 0.29864931106567383, + -2.931433916091919, + 1.3643410205841064, + -2.001633882522583, + -2.6548502445220947 + ], + [ + 2.035454750061035, + 0.03935282677412033, + -1.9080866575241089, + -0.17657285928726196, + -1.4843900203704834, + -0.4207392632961273, + -1.554489016532898, + 0.1909814178943634, + -0.5061874389648438, + -0.420978844165802, + -0.7634562849998474, + -0.7818551063537598, + -3.374903678894043, + -2.5324442386627197, + 1.2213574647903442, + 0.9676649570465088, + 0.5052346587181091, + -2.0974624156951904, + 0.9588823914527893, + -3.6367201805114746, + -1.953636646270752 + ], + [ + 1.4856661558151245, + 0.3259274363517761, + -2.1635096073150635, + -0.8793277740478516, + 0.2686290740966797, + -0.340938925743103, + -0.8516338467597961, + -0.3830271363258362, + 0.5376158356666565, + -0.2721732258796692, + -1.2276334762573242, + -0.38624608516693115, + -3.346294403076172, + -3.093522071838379, + 0.6735774278640747, + 0.5564011931419373, + -0.5450864434242249, + -1.5815953016281128, + 1.4068235158920288, + -2.7531042098999023, + -2.515698194503784 + ] + ], + "predicted_scores": [ + 3.835603713989258, + 3.6783268451690674, + 3.3198025226593018, + 2.9785282611846924, + 2.8413028717041016, + 2.7241344451904297, + 2.5705950260162354, + 2.5176258087158203 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565621826", + "selected_index": 0, + "state_hash": "f6edd849b0b51788d7be25e402fb514cb777ccf220797faaf769000814df02cb", + "stored_base_utility": 0.1373138427734375, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08252689987421036, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08252689987421036 + }, + "base_progress": 0.08252689987421036, + "base_success": false, + "base_utility": 0.08252689987421036, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.09265488386154175, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09265488386154175 + }, + { + "contact_quality": null, + "progress": 0.08510921150445938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08510921150445938 + }, + { + "contact_quality": null, + "progress": 0.09707490354776382, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09707490354776382 + }, + { + "contact_quality": null, + "progress": 0.08394956588745117, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08394956588745117 + }, + { + "contact_quality": null, + "progress": 0.0661265030503273, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0661265030503273 + }, + { + "contact_quality": null, + "progress": 0.0821652039885521, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0821652039885521 + }, + { + "contact_quality": null, + "progress": 0.04405144974589348, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04405144974589348 + }, + { + "contact_quality": null, + "progress": 0.09674423933029175, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09674423933029175 + } + ], + "candidate_progress": [ + 0.09265488386154175, + 0.08510921150445938, + 0.09707490354776382, + 0.08394956588745117, + 0.0661265030503273, + 0.0821652039885521, + 0.04405144974589348, + 0.09674423933029175 + ], + "candidate_source_chart_ids": [ + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-4aed2a49597da545e551", + "ms-4aed2a49597da545e551", + "ms-206099bab6bfe7c84fce", + "ms-4aed2a49597da545e551", + "ms-45fbd749d5ed3fe37c13" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-387e6771a55be98f1a59", + "generated_tangents": [ + [ + 0.28549665212631226, + 0.05311489850282669, + -0.38248977065086365, + -0.018529057502746582, + 0.6787716746330261, + -1.2279969453811646, + -0.6420841813087463, + 0.26857051253318787, + 0.17115972936153412, + 0.2551596462726593, + 0.7053962349891663, + 0.04692859947681427, + -0.5527177453041077, + -0.18881969153881073, + -0.23941057920455933, + -0.0982455462217331, + 0.029637441039085388, + 0.07620451599359512, + 0.23607471585273743, + -0.3256145119667053, + -0.5287892818450928 + ], + [ + 0.36704954504966736, + 0.05758073925971985, + -0.3474738299846649, + -0.011212378740310669, + 0.6217342615127563, + -1.2112942934036255, + -0.6362608075141907, + 0.2833932638168335, + 0.15666010975837708, + 0.2438482642173767, + 0.6734858155250549, + 0.020701885223388672, + -0.590171754360199, + -0.22812683880329132, + -0.17236584424972534, + -0.11846224963665009, + 0.06505389511585236, + -0.08531375229358673, + 0.1952979862689972, + -0.2484874427318573, + -0.582443118095398 + ], + [ + 0.27893614768981934, + 0.09527479112148285, + -0.3688504993915558, + 0.03827756643295288, + 0.5562700033187866, + -1.3607341051101685, + -0.5660457015037537, + 0.283571720123291, + 0.17398253083229065, + 0.26349055767059326, + 0.6594426035881042, + 0.0042448490858078, + -0.6312653422355652, + -0.34405291080474854, + -0.21913892030715942, + -0.10617777705192566, + -0.0103473961353302, + 0.024394690990447998, + 0.08682183921337128, + -0.16082853078842163, + -0.3717314898967743 + ], + [ + 0.22916582226753235, + 0.07972680032253265, + 0.1921221911907196, + -0.23081974685192108, + 1.1584932804107666, + -0.9987024068832397, + -0.8276074528694153, + -0.07720938324928284, + 0.1346161812543869, + 0.09562423825263977, + 0.9233322143554688, + 0.06993602216243744, + -0.8096259236335754, + -0.38820624351501465, + -0.15911449491977692, + -0.0007902458310127258, + 0.5113657116889954, + -0.10122306644916534, + -0.15443506836891174, + 0.20267784595489502, + -0.26166316866874695 + ], + [ + 0.6438100337982178, + 0.1824950873851776, + 0.2209223508834839, + -0.2092418521642685, + 1.2713840007781982, + -1.1816667318344116, + -1.134199619293213, + 0.0350564569234848, + 0.1381925493478775, + 0.15791894495487213, + 0.9445805549621582, + 0.15191148221492767, + -1.0084794759750366, + -0.5793180465698242, + -0.4575043320655823, + 0.04685113579034805, + 0.6560973525047302, + -0.08546553552150726, + -0.3179672360420227, + 0.302046537399292, + -0.7222995758056641 + ], + [ + 0.2856890857219696, + 0.19474300742149353, + 0.2752723693847656, + -0.36156928539276123, + 1.2245597839355469, + -0.6177957057952881, + -1.2531189918518066, + 0.02007262408733368, + 0.1324319690465927, + 0.15256844460964203, + 1.0360097885131836, + 0.4375746250152588, + -0.4120018184185028, + -0.3471463918685913, + -0.1916099190711975, + 0.049049533903598785, + 0.3769719898700714, + -0.09059213101863861, + 0.020839035511016846, + 0.5283606648445129, + -0.2865541875362396 + ], + [ + 0.17219120264053345, + 0.09193360805511475, + 0.38803911209106445, + -0.22901012003421783, + 1.144775152206421, + -0.8856706619262695, + -0.7789334654808044, + -0.2274218499660492, + 0.1517324447631836, + 0.09137210249900818, + 1.0625745058059692, + 0.21263983845710754, + -0.8271968364715576, + -0.4312806725502014, + -0.0942372977733612, + -0.02930457517504692, + 0.5402745008468628, + -0.14919717609882355, + -0.3878049850463867, + 0.1922987699508667, + -0.4372692406177521 + ], + [ + 0.17109063267707825, + -0.07279525697231293, + 0.4865674674510956, + -0.41254833340644836, + 0.8830982446670532, + -1.1640024185180664, + -1.1009790897369385, + 0.42098212242126465, + 0.25346437096595764, + 0.01581348478794098, + 0.504300057888031, + 0.5966982841491699, + -1.5934369564056396, + -1.4469102621078491, + 0.01507195457816124, + 0.049132492393255234, + -0.02747724950313568, + -0.361056387424469, + 0.44712918996810913, + -0.7449579238891602, + -0.9258217811584473 + ] + ], + "generated_utilities": [ + 0.09265488386154175, + 0.08510921150445938, + 0.09707490354776382, + 0.08394956588745117, + 0.0661265030503273, + 0.0821652039885521, + 0.04405144974589348, + 0.09674423933029175 + ], + "group_id": "ms-387e6771a55be98f1a59", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.03822460025548935, + 0.18199902772903442, + 0.20744429528713226, + 0.05679210275411606, + 0.07646497339010239, + 0.038853906095027924, + 0.0824488028883934, + 0.05751838535070419, + 0.02475176379084587, + 0.040490686893463135, + 0.027167975902557373, + 0.06515108048915863, + 0.04604649543762207, + 0.014139334671199322, + 0.0258652176707983, + 0.034187138080596924 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0643576979637146, + 0.02293895184993744, + 0.03272289037704468, + 0.02799379825592041, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + 0.05611860752105713, + -0.010578691959381104, + 0.03396539017558098, + 0.754948616027832, + -0.12030506134033203, + 0.05740344524383545, + 0.0, + 0.01674991101026535, + 0.0017660856246948242, + 0.1110122799873352, + -0.29033517837524414, + -0.015073239803314209, + -0.062492311000823975, + 0.0 + ], + [ + 0.03256732225418091, + 0.029803231358528137, + -0.11524808406829834, + -0.03975573182106018, + 1.7051892280578613, + -0.20410704612731934, + 0.0, + -0.11635321378707886, + -0.12916040420532227, + -0.10934318602085114, + 0.754948616027832, + -0.10127544403076172, + 0.10553282499313354, + 0.0, + 0.2124219685792923, + -0.13985675573349, + 0.20122885704040527, + -0.29033517837524414, + -0.09104716777801514, + -0.33053046464920044, + 0.0 + ] + ], + "predicted_scores": [ + 6.3780999183654785, + 6.203186511993408, + 5.9015960693359375, + 4.977954387664795, + 4.771164417266846, + 4.232055187225342, + 3.805530309677124, + 1.5832422971725464 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "6b753a027f9378ff308946331f6bb658d452d8e77220ab0cd6fc813be0ac75cc", + "stored_base_utility": 0.03822460025548935, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03917040303349495, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03917040303349495 + }, + "base_progress": 0.03917040303349495, + "base_success": false, + "base_utility": 0.03917040303349495, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.06579937785863876, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06579937785863876 + }, + { + "contact_quality": null, + "progress": 0.08081644773483276, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08081644773483276 + }, + { + "contact_quality": null, + "progress": 0.0637521967291832, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0637521967291832 + }, + { + "contact_quality": null, + "progress": 0.04907931759953499, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04907931759953499 + }, + { + "contact_quality": null, + "progress": 0.044412415474653244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044412415474653244 + }, + { + "contact_quality": null, + "progress": 0.060444433242082596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060444433242082596 + }, + { + "contact_quality": null, + "progress": 0.11467790603637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11467790603637695 + }, + { + "contact_quality": null, + "progress": 0.05245332047343254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05245332047343254 + } + ], + "candidate_progress": [ + 0.06579937785863876, + 0.08081644773483276, + 0.0637521967291832, + 0.04907931759953499, + 0.044412415474653244, + 0.060444433242082596, + 0.11467790603637695, + 0.05245332047343254 + ], + "candidate_source_chart_ids": [ + "ms-397e502908d16b7804b0", + "ms-98eb81debb5186ebbe38", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-397e502908d16b7804b0", + "ms-be219977a9db8dbb1e58", + "ms-7dedf60ad41ea2d92ee5", + "ms-397e502908d16b7804b0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-38f7298184a2c5005e94", + "generated_tangents": [ + [ + 0.06392338871955872, + 0.09199774265289307, + 0.39832690358161926, + -0.08481396734714508, + 1.1780062913894653, + -2.1410186290740967, + -0.026175349950790405, + 0.3576704263687134, + 0.007041775155812502, + 0.39361920952796936, + -0.20811009407043457, + 0.917994499206543, + -1.023305892944336, + 0.15454500913619995, + 0.17341533303260803, + 0.46468600630760193, + -0.13702644407749176, + -0.036796897649765015, + 0.2083837389945984, + -1.7337701320648193, + -1.0185716152191162 + ], + [ + 0.3518551290035248, + 0.07209115475416183, + 0.1243547797203064, + 0.07106101512908936, + 0.917472243309021, + -1.8660422563552856, + -0.34445130825042725, + 0.344399631023407, + 0.08186738193035126, + 0.19340568780899048, + -0.36229076981544495, + 0.7980875968933105, + -1.5588971376419067, + 0.033204786479473114, + 0.22450880706310272, + 0.3089137077331543, + 0.04057250916957855, + -0.17583157122135162, + 0.3140011727809906, + -1.8366100788116455, + -0.6990377902984619 + ], + [ + -0.011062920093536377, + 0.1393205225467682, + -0.058338314294815063, + -0.4927695095539093, + 1.4945178031921387, + -1.5698870420455933, + -0.9513618350028992, + 0.5369454622268677, + -0.04390205442905426, + 0.23241496086120605, + -0.3265971541404724, + 1.3889678716659546, + -1.312913179397583, + -0.11061471700668335, + 0.41059115529060364, + 0.45313361287117004, + -0.0007749646902084351, + -0.01357463002204895, + 0.10118094086647034, + -2.1971399784088135, + -1.5030590295791626 + ], + [ + 0.26104041934013367, + 0.16586053371429443, + 0.3561619818210602, + -0.24365101754665375, + 1.4226458072662354, + -1.5604000091552734, + -0.5354271531105042, + 0.3232274055480957, + -0.018214862793684006, + 0.24432046711444855, + -0.4937252700328827, + 1.5809727907180786, + -1.190437912940979, + -0.013494908809661865, + 0.19259680807590485, + 0.3032260537147522, + 0.012787118554115295, + -0.08537150919437408, + 0.2886306345462799, + -2.011969804763794, + -1.5524606704711914 + ], + [ + 0.24262487888336182, + 0.09299619495868683, + 0.44020742177963257, + -0.3339966833591461, + 1.3947012424468994, + -1.4245266914367676, + -0.5730263590812683, + 0.34654945135116577, + -0.015343474224209785, + 0.21485567092895508, + -0.5876930356025696, + 1.55846107006073, + -1.2206151485443115, + -0.006257236003875732, + 0.11838798224925995, + 0.28561991453170776, + -0.11307172477245331, + -0.14989586174488068, + 0.4280414581298828, + -1.9923348426818848, + -1.4128297567367554 + ], + [ + 0.07177262753248215, + 0.11817136406898499, + 0.19908684492111206, + -0.2735249698162079, + 0.8937707543373108, + -1.2653834819793701, + -0.20989418029785156, + 0.3359907269477844, + 0.03906693682074547, + 0.3878709375858307, + 0.18974530696868896, + 0.7361236810684204, + -1.6000571250915527, + -0.15636850893497467, + -0.060112934559583664, + 0.2737217843532562, + 0.5054886937141418, + -0.06787629425525665, + 0.2340533435344696, + -1.4242932796478271, + -0.5128096342086792 + ], + [ + -0.20007582008838654, + 0.2851930260658264, + -0.14037975668907166, + 0.19749146699905396, + 1.5596563816070557, + -1.0835773944854736, + -0.9457593560218811, + 0.7596969604492188, + 0.16349643468856812, + 0.32024094462394714, + -0.2900964915752411, + 0.9833406209945679, + -1.6681772470474243, + 0.20751577615737915, + 0.13381241261959076, + 0.47939008474349976, + 0.10001221299171448, + -0.04324066638946533, + 0.14821138978004456, + -2.220677375793457, + -1.3216150999069214 + ], + [ + 0.0864458978176117, + 0.14882588386535645, + 0.27863213419914246, + -0.4068523943424225, + 0.6527861952781677, + -1.1175181865692139, + -0.11312580108642578, + 0.2875620126724243, + -0.031250230967998505, + 0.2120424211025238, + -0.65817791223526, + 1.217991828918457, + -1.2307050228118896, + 0.47097188234329224, + 0.07097706198692322, + 0.3608599007129669, + -0.020377397537231445, + -0.4141119718551636, + 0.3017750382423401, + -1.4808273315429688, + -1.7473108768463135 + ] + ], + "generated_utilities": [ + 0.06579937785863876, + 0.08081644773483276, + 0.0637521967291832, + 0.04907931759953499, + 0.044412415474653244, + 0.060444433242082596, + 0.11467790603637695, + 0.05245332047343254 + ], + "group_id": "ms-38f7298184a2c5005e94", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.1194220781326294, + 0.1767096221446991, + 0.0505807027220726, + 0.026383202522993088, + 0.09035396575927734, + 0.11899666488170624, + 0.059947751462459564, + 0.08427222818136215, + 0.041255753487348557, + 0.05034176632761955, + 0.02779809758067131, + 0.011037330143153667, + 0.020422041416168213, + 0.049474477767944336, + 0.03517866134643555, + 0.02796018123626709 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.08923250436782837, + 0.05675226449966431, + -0.1250607967376709, + -0.07566308975219727, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.001595139503479004, + 0.026048749685287476, + 0.1681501865386963, + -0.11568224430084229, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.18507611751556396, + -0.15127414464950562, + 0.07637521624565125, + -0.15284772217273712, + 0.0320390909910202, + -1.8774652481079102, + 0.0 + ], + [ + -0.2749255299568176, + 0.2578675150871277, + -0.46759605407714844, + -0.5041444897651672, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + -0.13614141941070557, + 0.11120480298995972, + 0.15478569269180298, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + 0.01827770471572876, + -0.38639169931411743, + -0.18168723583221436, + 0.3532869219779968, + -0.5437159538269043, + -1.8774652481079102, + 0.0 + ], + [ + 0.4466920495033264, + 0.07532927393913269, + 0.8145812749862671, + -0.963495135307312, + 1.3846542835235596, + -3.0133423805236816, + -1.0476078987121582, + 0.6759306788444519, + 0.5485053658485413, + 0.634345293045044, + -0.7682492733001709, + 2.2281322479248047, + -2.7100830078125, + -0.8660445213317871, + 0.7648183703422546, + 0.607989490032196, + -0.6848516464233398, + -0.06598936021327972, + 0.2030458152294159, + -2.9672060012817383, + -1.4293488264083862 + ], + [ + 1.0058056116104126, + 0.3426892161369324, + 0.6375484466552734, + -0.5828341841697693, + 1.7963712215423584, + -3.3760194778442383, + -0.9086123704910278, + 0.20509907603263855, + 0.10472571849822998, + 1.1158980131149292, + -1.0710231065750122, + 2.203950881958008, + -2.4466300010681152, + -0.3184358477592468, + 0.43084120750427246, + 0.32133060693740845, + -0.6005205512046814, + 0.07880236953496933, + 0.52874755859375, + -2.7805917263031006, + -1.6972103118896484 + ], + [ + 0.48786461353302, + -0.007949680089950562, + 0.41629794239997864, + -0.2277727723121643, + 1.0953291654586792, + -2.7825636863708496, + -1.7178279161453247, + 0.8412191271781921, + 0.6389099359512329, + 0.7100303769111633, + -1.1701807975769043, + 2.8546230792999268, + -2.570579767227173, + -0.2383948564529419, + 0.4282584488391876, + 0.27555936574935913, + -0.4073549509048462, + 0.49247825145721436, + 0.6483681797981262, + -3.3716540336608887, + -1.891394853591919 + ], + [ + 0.17139381170272827, + 0.831203818321228, + 0.1966383457183838, + -1.5011540651321411, + 1.811257004737854, + -2.6416001319885254, + -1.8791863918304443, + 0.03218984603881836, + 1.0182924270629883, + 1.5643268823623657, + -1.2954871654510498, + 2.249680757522583, + -1.9383416175842285, + -1.4303685426712036, + 0.48937559127807617, + 0.2848057746887207, + -0.6113857626914978, + 0.6453920006752014, + -0.13643476366996765, + -2.514732599258423, + -1.8046306371688843 + ], + [ + 0.1380026936531067, + 0.5165722370147705, + 0.9639596939086914, + -0.3682268261909485, + 2.2811193466186523, + -3.5902557373046875, + -1.047336220741272, + 1.3064022064208984, + 0.7264143228530884, + 1.3356540203094482, + -0.41242116689682007, + 1.8184869289398193, + -3.016441822052002, + -0.8501555919647217, + 1.0147737264633179, + 1.0417438745498657, + -0.24422766268253326, + 0.3441629707813263, + 1.075059175491333, + -3.178764820098877, + -1.7388081550598145 + ], + [ + 1.0805304050445557, + -0.5906903743743896, + -0.32329434156417847, + -1.7024246454238892, + 1.3012806177139282, + -3.3655173778533936, + -2.0100932121276855, + 0.28595322370529175, + -0.42309290170669556, + 0.9014679193496704, + -0.11985284090042114, + 3.033172607421875, + -1.890408992767334, + -0.049424707889556885, + 0.8836197853088379, + 0.07151538133621216, + -0.6898007392883301, + 0.6614744663238525, + 0.7956362962722778, + -3.344180107116699, + -1.4918344020843506 + ], + [ + 1.4192681312561035, + -0.5825785994529724, + 0.11329725384712219, + -0.08465814590454102, + 1.0504729747772217, + -2.438842535018921, + -0.9858216047286987, + -0.35852980613708496, + 1.4301284551620483, + 0.37676453590393066, + -1.3725404739379883, + 3.114079475402832, + -2.0786290168762207, + -0.5835205912590027, + 0.1985606551170349, + 0.260464072227478, + 0.41686397790908813, + -0.104665607213974, + 0.6382055282592773, + -2.7474825382232666, + -1.663126826286316 + ], + [ + 1.0832164287567139, + 0.11555392295122147, + 1.2108526229858398, + -1.5446858406066895, + 0.9112894535064697, + -3.3027408123016357, + -1.2419135570526123, + -0.057880282402038574, + -0.07349961996078491, + 0.026286423206329346, + -0.897603452205658, + 3.2857306003570557, + -2.232046365737915, + -1.2509676218032837, + 1.1900405883789062, + 0.8446143269538879, + -1.339659333229065, + -0.07882180064916611, + 0.1166829764842987, + -2.662961721420288, + -1.0079293251037598 + ], + [ + 0.43863287568092346, + -0.2024492472410202, + 0.0567094087600708, + -0.3029903173446655, + 1.612892746925354, + -3.26739764213562, + -1.7898839712142944, + -0.1649491786956787, + 0.2927369475364685, + 1.020188808441162, + -0.10171866416931152, + 2.3230679035186768, + -2.615206003189087, + -0.08625847101211548, + 0.9057100415229797, + 1.4201092720031738, + -0.3173455595970154, + 0.09508261829614639, + 0.9565507173538208, + -2.2954390048980713, + -1.2229193449020386 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.03360384702682495, + -0.028338372707366943, + -0.12341004610061646, + -0.04655510187149048, + 0.6840170621871948, + -1.9024429321289062, + 0.0, + 0.032517433166503906, + -0.02922782301902771, + -0.033792078495025635, + -0.005206942558288574, + 1.2953410148620605, + -1.4925868511199951, + 0.0, + -0.06465893983840942, + -0.026740312576293945, + 0.046701133251190186, + -0.06953387707471848, + 0.02321198582649231, + -1.8774652481079102, + 0.0 + ] + ], + "predicted_scores": [ + 3.8450777530670166, + 3.747302293777466, + 3.028522491455078, + 2.623569965362549, + 2.476485252380371, + 2.088772773742676, + 1.6912859678268433, + 1.608293056488037 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "dee046e5a3c9b5e78be27f1df1f4f235344ec818a1a7439afe5f4cb30139cf34", + "stored_base_utility": 0.1194220781326294, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.009124159812927246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009124159812927246 + }, + "base_progress": 0.009124159812927246, + "base_success": false, + "base_utility": 0.009124159812927246, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.021712401881814003, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021712401881814003 + }, + { + "contact_quality": null, + "progress": 0.01253493595868349, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01253493595868349 + }, + { + "contact_quality": null, + "progress": 0.020592987537384033, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020592987537384033 + }, + { + "contact_quality": null, + "progress": 0.024399837478995323, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.024399837478995323 + }, + { + "contact_quality": null, + "progress": 0.015089293010532856, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015089293010532856 + }, + { + "contact_quality": null, + "progress": 0.01447206735610962, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01447206735610962 + }, + { + "contact_quality": null, + "progress": 0.030993500724434853, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030993500724434853 + }, + { + "contact_quality": null, + "progress": 0.023590564727783203, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023590564727783203 + } + ], + "candidate_progress": [ + 0.021712401881814003, + 0.01253493595868349, + 0.020592987537384033, + 0.024399837478995323, + 0.015089293010532856, + 0.01447206735610962, + 0.030993500724434853, + 0.023590564727783203 + ], + "candidate_source_chart_ids": [ + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-35a50d2822c2c3cc4516", + "ms-6758deb06644a14aa722", + "ms-35a50d2822c2c3cc4516", + "ms-6758deb06644a14aa722", + "ms-397e502908d16b7804b0", + "ms-35a50d2822c2c3cc4516" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-482c5c8d208b5907a2c2", + "generated_tangents": [ + [ + -0.10324154794216156, + 0.11459678411483765, + 0.6651431918144226, + 0.08507943153381348, + 1.1882154941558838, + -0.012950196862220764, + 0.2219807356595993, + 0.3474201560020447, + 0.18798601627349854, + 0.6566392779350281, + -0.31874650716781616, + 1.4696160554885864, + 0.3617602288722992, + -0.10195180773735046, + -0.05230840668082237, + -0.021728409454226494, + 0.1829734444618225, + 0.44701093435287476, + 0.18698903918266296, + 0.3523678779602051, + -0.7169893980026245 + ], + [ + 0.09427937865257263, + 0.24168075621128082, + 1.1515212059020996, + -0.15542587637901306, + 1.1397836208343506, + 1.445318579673767, + -0.3245232105255127, + 0.2744519114494324, + 0.28566601872444153, + 1.0274202823638916, + -0.4358368515968323, + 1.6342661380767822, + 1.3807165622711182, + 1.3334944248199463, + -0.37339451909065247, + 0.19507186114788055, + 0.357954204082489, + 0.7042507529258728, + 0.24645519256591797, + -0.6367019414901733, + -0.6903630495071411 + ], + [ + 0.20592395961284637, + 0.020406574010849, + 0.8932121992111206, + 0.12083536386489868, + 1.3073530197143555, + 0.9037048816680908, + -0.3148799538612366, + 0.3076699376106262, + 0.2580004334449768, + 1.004162073135376, + -0.3212444484233856, + 1.5189355611801147, + 1.0994170904159546, + 0.5334300994873047, + 0.08030927181243896, + 0.16071689128875732, + 0.10693933069705963, + 0.5786799192428589, + -0.053774625062942505, + 0.05243697762489319, + -0.7388463020324707 + ], + [ + -0.1731804460287094, + 0.17591097950935364, + 0.5804473757743835, + 0.0004608035087585449, + 1.1744091510772705, + 0.0935523509979248, + 0.020604103803634644, + 0.34948986768722534, + 0.1699536144733429, + 0.6689888834953308, + -0.44994327425956726, + 1.1837468147277832, + 0.39836356043815613, + -0.1453128606081009, + -0.06979362666606903, + 0.01856297440826893, + 0.17380034923553467, + 0.3868778944015503, + 0.2002110779285431, + 0.3215237855911255, + -0.5445191860198975 + ], + [ + -0.11170406639575958, + 0.024100452661514282, + 0.914171040058136, + 0.016485899686813354, + 1.3674840927124023, + 1.1694116592407227, + -0.2093716859817505, + 0.2524982988834381, + 0.25639545917510986, + 0.9274079203605652, + -0.5004943609237671, + 1.3227384090423584, + 1.0618607997894287, + 0.43705642223358154, + 0.1531822681427002, + 0.11158479750156403, + 0.2071995735168457, + 0.5246315002441406, + -0.16538608074188232, + 0.07106602191925049, + -0.6931731700897217 + ], + [ + 0.05488795042037964, + 0.005637887865304947, + 0.9165982007980347, + 0.2725606858730316, + 1.513552188873291, + -0.292318195104599, + -0.3103640079498291, + 0.22820740938186646, + 0.1727409064769745, + 0.66246497631073, + -0.25479790568351746, + 1.590884804725647, + 0.3123844563961029, + 0.11982061713933945, + 0.12594442069530487, + 0.15583494305610657, + 0.14659681916236877, + 0.321718692779541, + 0.1898215413093567, + 0.32241296768188477, + -0.4913976490497589 + ], + [ + -0.22871951758861542, + -0.021601146087050438, + 0.4148894250392914, + 0.21790257096290588, + 0.9094608426094055, + -0.362530380487442, + -0.25824183225631714, + 0.525179922580719, + 0.21581538021564484, + 0.6320717930793762, + -0.6137592196464539, + 1.2848787307739258, + 0.17660120129585266, + 0.06205551326274872, + 0.05705440044403076, + 0.2466651350259781, + 0.06993600726127625, + 0.5956662893295288, + 0.3316118121147156, + 0.24304735660552979, + -0.030683904886245728 + ], + [ + 0.20705199241638184, + 0.009793613106012344, + 0.8095183968544006, + 0.12078338861465454, + 1.2094385623931885, + 0.6669921875, + -0.5426391959190369, + 0.49290749430656433, + 0.19917532801628113, + 1.019526481628418, + -0.38820043206214905, + 1.4178555011749268, + 1.0817474126815796, + 0.33618342876434326, + 0.022878531366586685, + 0.2310313880443573, + 0.10935914516448975, + 0.7442690134048462, + -0.14142760634422302, + 0.16419675946235657, + -0.6085910797119141 + ] + ], + "generated_utilities": [ + 0.021712401881814003, + 0.01253493595868349, + 0.020592987537384033, + 0.024399837478995323, + 0.015089293010532856, + 0.01447206735610962, + 0.030993500724434853, + 0.023590564727783203 + ], + "group_id": "ms-482c5c8d208b5907a2c2", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0359329991042614, + 0.1834145486354828, + 0.04271410033106804, + 0.03202331066131592, + 0.12089761346578598, + 0.03575414419174194, + 0.07773216813802719, + 0.1062050461769104, + 0.10849495977163315, + 0.08369352668523788, + 0.14167599380016327, + 0.008830646984279156, + 0.014355997554957867, + 0.02518286369740963, + 0.07873731851577759, + 0.015250584110617638 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.007817894220352173, + 0.01831631362438202, + 0.14601469039916992, + 0.10826835036277771, + 1.6591682434082031, + -0.005238652229309082, + 0.0, + -0.02753499150276184, + -0.009624123573303223, + 0.5762790441513062, + -0.3171614408493042, + 1.4374253749847412, + 0.7921857833862305, + 0.0, + -0.037753358483314514, + -0.025671005249023438, + 0.027724146842956543, + 0.4680846929550171, + -0.08415359258651733, + 0.033902376890182495, + 0.0 + ], + [ + 0.4543180465698242, + 0.20962828397750854, + 1.14601469039917, + -0.33832865953445435, + 2.659168243408203, + -0.7141264081001282, + 0.3707960247993469, + 0.21857939660549164, + 0.6796972155570984, + 1.5762790441513062, + -1.3171614408493042, + 2.437425374984741, + 1.7921857833862305, + 1.7551628351211548, + -0.19566796720027924, + 0.6842412948608398, + 0.11985036730766296, + 1.468084692955017, + 0.29799214005470276, + 0.3715361952781677, + 0.2624160647392273 + ], + [ + 0.2115812450647354, + -0.09577503800392151, + 0.7509078979492188, + -0.6239187121391296, + 2.4163239002227783, + -0.5409671664237976, + 0.16154196858406067, + 0.5615253448486328, + 1.0765573978424072, + 1.4198161363601685, + -1.430187463760376, + 2.5212466716766357, + 1.9029841423034668, + 1.5550287961959839, + -0.5099657773971558, + 1.0103299617767334, + -0.21028786897659302, + 1.4441221952438354, + 0.5605546236038208, + 0.6799722909927368, + 0.5554409027099609 + ], + [ + 0.46792101860046387, + 0.08014003932476044, + 1.5437835454940796, + 0.08789932727813721, + 2.3046298027038574, + -1.1965231895446777, + 0.8527317047119141, + 0.22510167956352234, + 1.0473172664642334, + 1.3720369338989258, + -1.2376489639282227, + 2.066061019897461, + 2.033992052078247, + 2.166865825653076, + -0.46199846267700195, + 1.1675353050231934, + -0.22657811641693115, + 1.7576167583465576, + 0.766140878200531, + 0.1856600046157837, + 0.008163660764694214 + ], + [ + 0.9431964755058289, + 0.6587716341018677, + 0.8381389379501343, + -0.8407833576202393, + 2.3666741847991943, + -1.05813729763031, + 0.1429125964641571, + 0.10640819370746613, + 1.0277029275894165, + 1.193630576133728, + -0.7695048451423645, + 2.9484994411468506, + 1.1866973638534546, + 1.5341145992279053, + -0.1225738674402237, + 0.9121334552764893, + 0.7116963863372803, + 0.8198207020759583, + 0.6562460660934448, + 0.1118304431438446, + -0.2320157289505005 + ] + ], + "predicted_scores": [ + -0.16841058433055878, + -0.28825217485427856, + -0.3040606379508972, + -0.41618430614471436, + -0.43267136812210083, + -0.6537924408912659, + -0.8287419676780701, + -0.8810359835624695 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "903f78d5baa541d73612cac5e063bb553fba566c0d4adcb9603aa055d710fb15", + "stored_base_utility": 0.0359329991042614, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14663122594356537, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14663122594356537 + }, + "base_progress": 0.14663122594356537, + "base_success": false, + "base_utility": 0.14663122594356537, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06835349649190903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06835349649190903 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.06835349649190903, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4da03300d92420e80ed0", + "generated_tangents": [ + [ + 1.5912787914276123, + -0.036956414580345154, + -0.21241113543510437, + 0.6516801118850708, + -0.19399788975715637, + 1.5948033332824707, + 1.271910548210144, + 1.4476734399795532, + 0.1724436730146408, + -0.9798520803451538, + -0.42552649974823, + -0.6116935014724731, + -2.8600406646728516, + -1.6242611408233643, + 1.367811918258667, + 0.11179615557193756, + -0.8350984454154968, + 0.33912092447280884, + -0.6248295307159424, + -1.3573098182678223, + -0.7161210775375366 + ], + [ + 1.7093102931976318, + -0.196620911359787, + 0.02189430594444275, + 0.641255259513855, + -0.4094357490539551, + 2.530367612838745, + 1.4680109024047852, + 2.0426712036132812, + 0.1967797428369522, + -1.2756084203720093, + -0.5933140516281128, + -0.6144430637359619, + -3.5528340339660645, + -2.1312100887298584, + 1.4805469512939453, + 0.2696048319339752, + -0.6461998820304871, + 0.41171813011169434, + -0.28442347049713135, + -0.7976076006889343, + -0.9309910535812378 + ], + [ + 1.4498106241226196, + -0.29270243644714355, + 0.04120686650276184, + -0.13904261589050293, + -0.21596965193748474, + 0.7187117338180542, + 2.4622488021850586, + 1.5556423664093018, + 0.1405021697282791, + -1.037904143333435, + -0.42656487226486206, + -0.9309213161468506, + -2.8924334049224854, + -1.6524156332015991, + 1.009226679801941, + 0.010251995176076889, + -0.44311296939849854, + 0.07730083912611008, + -0.2179584503173828, + -1.8193836212158203, + -0.6662688255310059 + ], + [ + 1.5806620121002197, + -0.14825692772865295, + -0.2096545398235321, + 0.583511233329773, + -0.5389400720596313, + 2.4316601753234863, + 1.4776484966278076, + 1.8064268827438354, + 0.19802790880203247, + -1.2530491352081299, + -0.5538384318351746, + -0.795089602470398, + -3.2339212894439697, + -2.5375781059265137, + 1.4592887163162231, + 0.23937417566776276, + -0.48405611515045166, + 0.3261455297470093, + -0.24709457159042358, + -0.6881916522979736, + -1.1293952465057373 + ], + [ + 1.5368729829788208, + -0.09165816009044647, + -0.11731317639350891, + 0.554122805595398, + -0.27068716287612915, + 2.2091636657714844, + 1.6179258823394775, + 1.8708558082580566, + 0.1728523224592209, + -1.3351114988327026, + -0.556918740272522, + -0.8974283933639526, + -3.302716016769409, + -2.4506871700286865, + 1.5876867771148682, + 0.22924140095710754, + -0.5252518653869629, + 0.2545866072177887, + -0.24682819843292236, + -0.8035078644752502, + -1.14055597782135 + ], + [ + 1.5098063945770264, + -0.22725751996040344, + -0.26270297169685364, + 0.5413416624069214, + -0.4311066269874573, + 2.0036556720733643, + 1.6241755485534668, + 1.839166522026062, + 0.2298468053340912, + -1.2273516654968262, + -0.7428346276283264, + -0.8687328100204468, + -3.3517966270446777, + -2.3812711238861084, + 1.5311939716339111, + 0.17737452685832977, + -0.405849814414978, + 0.26439476013183594, + -0.2840878367424011, + -0.978763997554779, + -1.39325749874115 + ], + [ + 1.6119011640548706, + -0.15155303478240967, + -0.4006286561489105, + 0.7292817831039429, + -0.4522017240524292, + 2.1315112113952637, + 1.7339651584625244, + 1.7694288492202759, + 0.27131545543670654, + -1.235857367515564, + -0.46351078152656555, + -0.911089301109314, + -3.1760571002960205, + -2.4977662563323975, + 1.5386873483657837, + 0.20270687341690063, + -0.3883051872253418, + 0.14286354184150696, + -0.23281234502792358, + -0.8912167549133301, + -1.1589937210083008 + ], + [ + 1.4282817840576172, + -0.2259243130683899, + -0.2913416922092438, + 0.6201746463775635, + -0.37133270502090454, + 1.4765207767486572, + 1.910367727279663, + 1.6940661668777466, + 0.316834956407547, + -1.439162254333496, + -0.5653513073921204, + -1.072232723236084, + -2.8804492950439453, + -2.411958932876587, + 1.7964717149734497, + 0.17439192533493042, + -0.6093809604644775, + 0.0989665761590004, + -0.2696952223777771, + -1.2158225774765015, + -1.27056884765625 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.06835349649190903, + 2.0 + ], + "group_id": "ms-4da03300d92420e80ed0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18184101581573486, + 0.1803790181875229, + 0.09926676750183105, + 0.20791122317314148, + 0.26338279247283936, + 0.2025860995054245, + 0.2575649917125702, + 0.23894403874874115, + 0.2174234390258789, + 2.0, + 0.1918075978755951, + 0.10365168750286102, + 2.0, + 2.0, + 2.0, + 0.15933696925640106 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.1168947219848633, + 0.13667210936546326, + -0.051466986536979675, + 0.03360617160797119, + -0.19173425436019897, + 1.7646396160125732, + 0.0, + 0.9886866807937622, + 0.19114632904529572, + -0.3927978277206421, + -0.21108940243721008, + -0.031822919845581055, + -1.6683034896850586, + 0.0, + 0.9908627271652222, + -0.05301591753959656, + 0.08875280618667603, + 0.04224884510040283, + -0.03718748688697815, + -0.06952071189880371, + 0.0 + ], + [ + 2.1713812351226807, + -0.41443270444869995, + -0.23275701701641083, + 0.6524178981781006, + -0.25828492641448975, + 3.331055164337158, + 2.47529673576355, + 1.4984867572784424, + -0.396874338388443, + -0.725897490978241, + 0.1870076060295105, + -0.4014764428138733, + -2.239036798477173, + -2.218061923980713, + 1.6540865898132324, + 0.013689309358596802, + -0.22782719135284424, + 1.2475574016571045, + 0.7231295704841614, + -0.281164288520813, + -1.4593302011489868 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 2.1168947219848633, + 0.045049574226140976, + -0.25911271572113037, + 0.6112266778945923, + -0.5530642867088318, + 2.7646396160125732, + 1.9622430801391602, + 1.9886866807937622, + 0.28067120909690857, + -1.392797827720642, + -0.5793421268463135, + -1.031822919845581, + -2.6683034896850586, + -2.812993049621582, + 1.9908627271652222, + -0.28380459547042847, + -0.7305920720100403, + 0.7978808879852295, + 0.1574108749628067, + -1.0695207118988037, + -0.9408642053604126 + ], + [ + 2.228377103805542, + -0.19656506180763245, + -0.23844876885414124, + 0.6676552295684814, + -0.7639804482460022, + 2.937652111053467, + 1.882341980934143, + 1.8207013607025146, + 0.27702635526657104, + -1.549763798713684, + -0.5955427289009094, + -1.185685157775879, + -2.8421716690063477, + -3.054234266281128, + 1.691805362701416, + -0.37051287293434143, + -0.43653741478919983, + 0.514695405960083, + -0.0132378488779068, + -0.899186372756958, + -0.8349848985671997 + ], + [ + 2.425011157989502, + -0.03216857090592384, + -0.010275468230247498, + 0.4265761971473694, + -0.7737585306167603, + 3.1123812198638916, + 1.5686779022216797, + 2.112421989440918, + 0.3475198745727539, + -1.3452436923980713, + -0.6067132949829102, + -1.1150503158569336, + -2.461210250854492, + -2.8643558025360107, + 2.185579299926758, + -0.6223857402801514, + -0.8183724880218506, + 0.8074819445610046, + 0.536148726940155, + -0.7826522588729858, + -0.6682058572769165 + ], + [ + 1.6831411123275757, + -0.19312052428722382, + 0.010244876146316528, + 1.1032377481460571, + -0.4994945824146271, + 2.9000508785247803, + 2.187715768814087, + 2.200220823287964, + 0.4744641184806824, + -1.6715843677520752, + -0.493664413690567, + -1.459894061088562, + -2.90458345413208, + -2.3254852294921875, + 1.4726771116256714, + -0.21338894963264465, + -1.293912649154663, + 1.3146823644638062, + -0.09375743567943573, + -0.5040683150291443, + -0.9596792459487915 + ], + [ + 1.6738543510437012, + 0.6350066661834717, + 0.19669359922409058, + -0.1396545171737671, + -0.9625160694122314, + 2.207883596420288, + 1.7678128480911255, + 1.319273829460144, + -0.267609566450119, + -0.5747214555740356, + 0.19376236200332642, + -1.4490230083465576, + -2.6397883892059326, + -3.2567262649536133, + 1.2646806240081787, + -0.14523190259933472, + -0.050948500633239746, + 0.6943628787994385, + -0.1503342241048813, + -0.7313232421875, + -1.7930638790130615 + ], + [ + 2.139432668685913, + 0.34125545620918274, + 0.6370852589607239, + 0.8591548204421997, + 0.14890235662460327, + 2.6835381984710693, + 2.324920892715454, + 2.0091233253479004, + 0.13480046391487122, + -0.47407835721969604, + -1.102479100227356, + -1.0710523128509521, + -2.3226847648620605, + -3.6534740924835205, + 1.8767409324645996, + -0.01776304841041565, + -1.167153000831604, + 0.693484902381897, + 0.563054084777832, + -0.17544519901275635, + -0.8315125703811646 + ], + [ + 1.1254388093948364, + -0.03027276322245598, + -0.8753920793533325, + 0.3578681945800781, + -0.4256642460823059, + 1.7686536312103271, + 2.253190517425537, + 2.8274970054626465, + 0.3938461244106293, + -1.4084759950637817, + -0.7252857685089111, + -0.07288885116577148, + -1.7749395370483398, + -2.1030378341674805, + 1.92497980594635, + 0.20398634672164917, + -0.03521168231964111, + 1.4702520370483398, + -0.36331063508987427, + -1.5291211605072021, + -0.8369957804679871 + ] + ], + "predicted_scores": [ + 5.25183629989624, + 4.327092170715332, + 4.227852821350098, + 3.780285358428955, + 3.7225940227508545, + 3.6847755908966064, + 3.204503297805786, + 2.865238904953003 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1747178030", + "selected_index": 0, + "state_hash": "87c74ddf97c53ddeeaae98467b4a13b1e0e499bd94eddb617759d39164cab542", + "stored_base_utility": 0.18184101581573486, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.009399374015629292, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009399374015629292 + }, + { + "contact_quality": null, + "progress": 0.006204982753843069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006204982753843069 + }, + { + "contact_quality": null, + "progress": 0.007237812038511038, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007237812038511038 + }, + { + "contact_quality": null, + "progress": 0.008653719909489155, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008653719909489155 + }, + { + "contact_quality": null, + "progress": 0.010276774875819683, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010276774875819683 + }, + { + "contact_quality": null, + "progress": 0.006190141197293997, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006190141197293997 + }, + { + "contact_quality": null, + "progress": 0.0069203972816467285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0069203972816467285 + }, + { + "contact_quality": null, + "progress": 0.0071913800202310085, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0071913800202310085 + } + ], + "candidate_progress": [ + 0.009399374015629292, + 0.006204982753843069, + 0.007237812038511038, + 0.008653719909489155, + 0.010276774875819683, + 0.006190141197293997, + 0.0069203972816467285, + 0.0071913800202310085 + ], + "candidate_source_chart_ids": [ + "ms-7eaea428dfdef7255a66", + "ms-dc26b448cce4164a23e6", + "ms-e84163d9ef2e7014daca", + "ms-6a3add21affa4796f9be", + "ms-6a3add21affa4796f9be", + "ms-e84163d9ef2e7014daca", + "ms-f85a653af630821f2572", + "ms-0e592951fa32139b6dc9" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank4", + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank5", + "ctt_transport_rank1", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-57be9ae181c077e7821f", + "generated_tangents": [ + [ + -0.30926334857940674, + -0.3646051287651062, + 1.0946054458618164, + -0.36321312189102173, + 0.9840645790100098, + 2.676570177078247, + 1.7332475185394287, + 0.04148532450199127, + 0.08790934830904007, + 0.9474417567253113, + -0.09922204166650772, + 0.810888409614563, + 1.813698649406433, + 2.016425848007202, + 0.8269513845443726, + -0.5040432810783386, + -0.19824446737766266, + 0.20947657525539398, + 0.08509057760238647, + 1.264101505279541, + 0.2216612994670868 + ], + [ + -0.6527650952339172, + -0.3242778778076172, + 1.444452166557312, + -0.5342669486999512, + 0.7839994430541992, + 2.1463117599487305, + 1.489823579788208, + 0.2299799919128418, + 0.1252458691596985, + 1.0040754079818726, + -0.11908742040395737, + 1.0013298988342285, + 1.8913668394088745, + 1.497053861618042, + 0.5943560600280762, + -0.5696547627449036, + -0.1311655193567276, + 0.21365685760974884, + 0.25112485885620117, + 1.3706295490264893, + 0.27272772789001465 + ], + [ + -0.7989112734794617, + -0.21156653761863708, + 1.441413402557373, + -0.3115503191947937, + 0.6862883567810059, + 2.3581361770629883, + 1.1852996349334717, + 0.18688702583312988, + 0.030488146468997, + 0.7526087760925293, + -0.0653768926858902, + 1.264310359954834, + 1.640783429145813, + 1.4627430438995361, + 0.5899918675422668, + -0.5287672281265259, + -0.2672613263130188, + 0.24378249049186707, + 0.20653939247131348, + 1.1917920112609863, + 0.09985701739788055 + ], + [ + -0.7405605316162109, + -0.1526736468076706, + 1.1547361612319946, + -0.19872139394283295, + 0.8518969416618347, + 2.5984432697296143, + 1.2750420570373535, + 0.3957311809062958, + 0.031122282147407532, + 1.0674495697021484, + -0.36115434765815735, + 0.8102352619171143, + 1.6482597589492798, + 1.3360583782196045, + 0.5427156686782837, + -0.40142983198165894, + -0.15791146457195282, + 0.26739078760147095, + 0.3148157298564911, + 1.5067071914672852, + 0.7017356157302856 + ], + [ + -0.7004825472831726, + -0.2538937032222748, + 1.2259334325790405, + -0.03816375136375427, + 0.7258093357086182, + 2.302774429321289, + 1.3373589515686035, + 0.5013123750686646, + -0.050833381712436676, + 0.5012713074684143, + -0.3792320787906647, + 1.0458316802978516, + 1.6910203695297241, + 1.5447372198104858, + 0.5399075150489807, + -0.6167660355567932, + -0.14431537687778473, + 0.20329181849956512, + 0.2862829864025116, + 1.4421494007110596, + -0.006353422999382019 + ], + [ + -0.8840819597244263, + -0.2599353790283203, + 1.60691499710083, + -0.1055542379617691, + 0.7680267691612244, + 2.445167303085327, + 1.0817859172821045, + 0.23314587771892548, + -0.04322775453329086, + 0.6557232737541199, + -0.260415643453598, + 1.2362195253372192, + 1.4674233198165894, + 1.7542859315872192, + 0.6350005269050598, + -0.4188421368598938, + 0.003112286329269409, + 0.23604613542556763, + 0.3358215391635895, + 1.2808003425598145, + 0.0012314319610595703 + ], + [ + -1.0495582818984985, + -0.22637467086315155, + 1.338814616203308, + -0.38610005378723145, + 1.3288910388946533, + 2.4673244953155518, + 1.1249277591705322, + 0.10093340277671814, + -0.008707791566848755, + 1.0432755947113037, + 0.36337634921073914, + 0.5582226514816284, + 1.5481916666030884, + 1.3156152963638306, + 1.004977822303772, + -0.19249799847602844, + -0.3012973666191101, + 0.14856944978237152, + 0.4736841022968292, + 1.6065986156463623, + 0.3558976650238037 + ], + [ + -0.5317566394805908, + -0.08665710687637329, + 1.5755513906478882, + -0.02031444013118744, + 1.0695438385009766, + 2.639113664627075, + 1.1690127849578857, + 0.27002424001693726, + -0.02944600023329258, + 1.0525678396224976, + -0.10891043394804001, + 0.8228455781936646, + 1.7721060514450073, + 1.7008202075958252, + 0.6315513253211975, + -0.1996202915906906, + -0.004995107650756836, + 0.22394779324531555, + 0.10063542425632477, + 1.4173269271850586, + 1.517505407333374 + ] + ], + "generated_utilities": [ + 0.009399374015629292, + 0.006204982753843069, + 0.007237812038511038, + 0.008653719909489155, + 0.010276774875819683, + 0.006190141197293997, + 0.0069203972816467285, + 0.0071913800202310085 + ], + "group_id": "ms-57be9ae181c077e7821f", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.28031715750694275, + 0.19804009795188904, + 0.15024888515472412, + 0.23104076087474823, + 0.21580509841442108, + 0.013450047001242638, + 0.20318475365638733, + 0.22562944889068604, + 0.14178292453289032, + 0.15763869881629944, + 0.27735981345176697, + 0.1909358948469162, + 0.06666654348373413, + 2.0, + 0.060366373509168625, + 0.05085722729563713 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -0.06739383935928345, + 0.06319573521614075, + 0.8779754638671875, + -0.03682136535644531, + -0.07093041390180588, + 1.3382306098937988, + 0.0, + 0.07168813049793243, + 0.01971740648150444, + 0.4866671562194824, + 0.07402461767196655, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + 0.03574584424495697, + 0.021435141563415527, + 0.0021790266036987305, + -0.06565424799919128, + 1.1393508911132812, + 0.0 + ], + [ + -0.17397063970565796, + -0.08753994852304459, + 0.8779754638671875, + -0.09028738737106323, + -0.17196941375732422, + 1.3382306098937988, + 0.0, + 0.022144019603729248, + 0.1154828891158104, + 0.4866671562194824, + -0.09212043881416321, + 0.36222100257873535, + 0.8966265916824341, + 0.0, + 0.48864150047302246, + -0.2021142691373825, + -0.022606968879699707, + 0.0021790266036987305, + 0.03272634744644165, + 1.1393508911132812, + 0.0 + ], + [ + -1.0144902467727661, + -0.023572498932480812, + 1.8779754638671875, + -1.0368213653564453, + -0.03799626976251602, + 2.338230609893799, + 2.239948272705078, + 0.16168542206287384, + 0.07221447676420212, + 1.4866671562194824, + -0.3557073175907135, + 1.3622210025787354, + 1.896626591682434, + 2.0382778644561768, + 1.4886415004730225, + 0.1997920423746109, + -0.4386618137359619, + 1.0021790266036987, + -0.3174663782119751, + 2.1393508911132812, + 1.9118322134017944 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.479896545410156, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0765557289123535, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.823664426803589 + ], + [ + -1.0723698139190674, + -0.05779706686735153, + 2.0670831203460693, + -0.830497682094574, + -0.17302533984184265, + 2.2277159690856934, + 2.0435242652893066, + 0.08191673457622528, + 0.08711377531290054, + 1.416495680809021, + -0.5633984804153442, + 1.3216067552566528, + 1.8660295009613037, + 2.2198705673217773, + 1.6887608766555786, + 0.06658947467803955, + -0.4010601043701172, + 1.1803714036941528, + -0.15144284069538116, + 2.308220386505127, + 1.8460347652435303 + ], + [ + -0.8901733756065369, + 0.3378032147884369, + 2.2360360622406006, + -1.207388162612915, + -0.34048303961753845, + 2.460798978805542, + 2.100795030593872, + -0.1847335249185562, + -0.06362440437078476, + 1.2987924814224243, + -0.11972835659980774, + 1.170393943786621, + 2.0620977878570557, + 2.4285032749176025, + 1.3666276931762695, + -0.01825574040412903, + -0.4620283842086792, + 1.1138218641281128, + -0.5787107944488525, + 1.78761887550354, + 1.740113615989685 + ], + [ + -1.1796774864196777, + -0.12369386106729507, + 1.546891689300537, + -1.0500564575195312, + -0.026454759761691093, + 2.177122116088867, + 2.0003931522369385, + 0.13697585463523865, + 0.37715619802474976, + 1.9734747409820557, + -0.16338464617729187, + 1.3775254487991333, + 2.3054146766662598, + 1.5546050071716309, + 1.2645983695983887, + 0.3812609910964966, + 0.02191823720932007, + 1.0944219827651978, + 0.09844094514846802, + 1.8145865201950073, + 2.0624771118164062 + ], + [ + -0.5514534711837769, + -0.4441075921058655, + 1.8971614837646484, + -1.6112515926361084, + 0.1306135058403015, + 2.736365556716919, + 2.270737886428833, + -0.35948747396469116, + 0.03968408331274986, + 1.061530351638794, + -0.29366230964660645, + 1.5066968202590942, + 2.34443998336792, + 2.3445637226104736, + 1.2286460399627686, + 0.13919776678085327, + -0.21014492213726044, + 0.9723342061042786, + -0.5822539329528809, + 1.8932968378067017, + 2.159864664077759 + ], + [ + -0.9470903873443604, + -0.7433359622955322, + 1.798203945159912, + -0.5850675106048584, + -0.7611241340637207, + 2.2761964797973633, + 2.0114798545837402, + 0.7724847793579102, + -0.23453289270401, + 0.7384847402572632, + -1.022956132888794, + 1.0059813261032104, + 1.652310848236084, + 2.6376113891601562, + 2.148331642150879, + 0.9844839572906494, + 0.18531352281570435, + 1.6280735731124878, + -0.06812705099582672, + 2.2684829235076904, + 1.1641287803649902 + ], + [ + -1.729402780532837, + 0.15382106602191925, + 1.0736894607543945, + -0.5141006708145142, + -0.6476962566375732, + 2.0120980739593506, + 2.8703343868255615, + 0.5578619837760925, + -0.8154823184013367, + 1.4325131177902222, + -1.0468122959136963, + 0.9670607447624207, + 1.973939061164856, + 2.7364890575408936, + 1.8303502798080444, + 0.8089333772659302, + -0.41138362884521484, + 1.8146085739135742, + -0.11676108837127686, + 2.3541221618652344, + 2.3875224590301514 + ], + [ + -1.8409323692321777, + -0.2118261456489563, + 1.5354409217834473, + -0.3873547911643982, + 0.7043789029121399, + 2.2068612575531006, + 2.5864486694335938, + 0.8632195591926575, + -0.10360705107450485, + 1.6340630054473877, + -0.017105937004089355, + 1.90720796585083, + 2.608898639678955, + 2.3387808799743652, + 2.3744726181030273, + 0.13692857325077057, + -0.7716406583786011, + 1.3477450609207153, + -0.9811362028121948, + 2.15514874458313, + 2.298004150390625 + ], + [ + -1.2484530210494995, + 0.5977066159248352, + 2.8256869316101074, + -0.3418053388595581, + -1.031638503074646, + 1.7023751735687256, + 1.4487309455871582, + -0.3013373017311096, + -0.3682725131511688, + 2.4404053688049316, + -0.8263542652130127, + 2.1105899810791016, + 2.5769248008728027, + 1.6751525402069092, + 0.8367876410484314, + -0.1554209142923355, + 0.17985862493515015, + 1.0017495155334473, + -0.3176954984664917, + 2.6737403869628906, + 1.7722500562667847 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -1.0015538930892944, + 0.43126925826072693, + 1.2786648273468018, + -0.7182899713516235, + 0.06109866499900818, + 2.323270797729492, + 1.3882524967193604, + -0.2783380150794983, + 0.35085979104042053, + 0.5870303511619568, + -1.0134913921356201, + 0.4675567150115967, + 0.9185858964920044, + 1.9441449642181396, + 1.3693488836288452, + 0.05759717524051666, + 0.19058758020401, + 1.9360744953155518, + -1.0018632411956787, + 2.043489456176758, + 1.1226990222930908 + ] + ], + "predicted_scores": [ + 5.009902000427246, + 3.7094740867614746, + 3.506354570388794, + 3.256587028503418, + 3.1261518001556396, + 2.2112104892730713, + 1.8244775533676147, + 1.5559664964675903 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "9c4de49d50d74c8c571620759fff1d43be23093a5f66e946f7ae6d75cb26ade5", + "stored_base_utility": 0.28031715750694275, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1544450968503952, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1544450968503952 + }, + "base_progress": 0.1544450968503952, + "base_success": false, + "base_utility": 0.1544450968503952, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.009421487338840961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009421487338840961 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08336275815963745, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08336275815963745 + }, + { + "contact_quality": null, + "progress": 0.07322987169027328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07322987169027328 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.08531469106674194, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08531469106674194 + }, + { + "contact_quality": null, + "progress": 0.08539849519729614, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08539849519729614 + } + ], + "candidate_progress": [ + 0.009421487338840961, + 1.0, + 1.0, + 0.08336275815963745, + 0.07322987169027328, + 1.0, + 0.08531469106674194, + 0.08539849519729614 + ], + "candidate_source_chart_ids": [ + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-68b43d83b97196e0d27f", + "ms-ff129e39095c2861b522", + "ms-ff129e39095c2861b522", + "ms-ff129e39095c2861b522", + "ms-68b43d83b97196e0d27f" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-6388b5d2356c556c3e55", + "generated_tangents": [ + [ + -0.1346358209848404, + -0.290164977312088, + 1.3188178539276123, + 0.1854286789894104, + 1.2796549797058105, + 3.007685899734497, + 2.3563530445098877, + 2.484987497329712, + 0.026990480720996857, + -0.27481046319007874, + 0.751373291015625, + 0.7147144079208374, + 1.8042527437210083, + 2.37459659576416, + -0.26678499579429626, + 0.02226560190320015, + 0.9388956427574158, + 0.8101158738136292, + 1.034219741821289, + 2.466493606567383, + 2.2822017669677734 + ], + [ + -0.2568861246109009, + -0.19353245198726654, + 1.4529776573181152, + -0.1390744000673294, + 1.0377912521362305, + 2.9880504608154297, + 2.1059212684631348, + 2.381761312484741, + 0.15299169719219208, + -0.21691611409187317, + 1.122802495956421, + 1.1129459142684937, + 2.4177308082580566, + 2.2433297634124756, + -0.39117854833602905, + -0.04049832001328468, + 0.876803994178772, + 0.7658475637435913, + 1.1124186515808105, + 2.7858569622039795, + 2.717148780822754 + ], + [ + -0.28434789180755615, + -0.17466460168361664, + 1.3416450023651123, + -0.20311586558818817, + 1.3057668209075928, + 2.56211256980896, + 2.5722076892852783, + 1.9682852029800415, + -0.0061507876962423325, + -0.18780440092086792, + 1.0633792877197266, + 0.8869252800941467, + 2.2866291999816895, + 2.264634132385254, + -0.6698980927467346, + 0.10091784596443176, + 0.6638089418411255, + 0.7630418539047241, + 1.1821287870407104, + 2.280217409133911, + 2.386282205581665 + ], + [ + -0.26237136125564575, + -0.23905599117279053, + 1.6208723783493042, + -0.19124546647071838, + 0.9796238541603088, + 2.8906607627868652, + 2.228334665298462, + 2.2495615482330322, + 0.1718578040599823, + -0.1611553430557251, + 1.1272178888320923, + 1.2065633535385132, + 2.335747003555298, + 2.2670536041259766, + -0.4416409134864807, + -0.01251941453665495, + 0.8514349460601807, + 0.6165656447410583, + 0.9599085450172424, + 2.9152796268463135, + 2.509885549545288 + ], + [ + -0.3158658742904663, + -0.17014434933662415, + 1.6859221458435059, + -0.25803977251052856, + 1.0942418575286865, + 2.8559882640838623, + 2.141686201095581, + 2.1352880001068115, + 0.0933176651597023, + -0.07159272581338882, + 1.4386317729949951, + 0.7763835191726685, + 2.158827304840088, + 2.4083712100982666, + -0.6753755211830139, + -0.09084516763687134, + 0.6954506635665894, + 0.6088263988494873, + 0.6035493612289429, + 2.703592538833618, + 2.6240713596343994 + ], + [ + -0.22186796367168427, + -0.04749760404229164, + 1.3944339752197266, + -0.17063748836517334, + 1.1214823722839355, + 2.5736377239227295, + 2.551818609237671, + 2.207057476043701, + 0.15786957740783691, + 0.08558044582605362, + 1.23016357421875, + 0.5588364601135254, + 2.169656991958618, + 2.2912497520446777, + -0.6412237286567688, + -0.07000379264354706, + 0.8803773522377014, + 0.5812371373176575, + 0.7415778636932373, + 2.634361505508423, + 2.1934618949890137 + ], + [ + -0.17021532356739044, + -0.08369076251983643, + 1.5253565311431885, + -0.14208582043647766, + 1.0116651058197021, + 2.639970541000366, + 2.589531660079956, + 2.143213987350464, + 0.16520118713378906, + 0.16542699933052063, + 1.3231823444366455, + 0.6546212434768677, + 2.240913152694702, + 2.1633529663085938, + -0.6388358473777771, + -0.04554488882422447, + 0.6752522587776184, + 0.5876330137252808, + 0.8056936264038086, + 2.764847755432129, + 2.196819305419922 + ], + [ + -0.015666082501411438, + -0.14996975660324097, + 1.6243469715118408, + -0.026797041296958923, + 1.1090970039367676, + 3.2096641063690186, + 1.8211236000061035, + 2.043275833129883, + 0.2226421982049942, + -0.3239434063434601, + 1.5672627687454224, + 1.3945112228393555, + 2.06219744682312, + 2.4981837272644043, + -0.3383021056652069, + -0.005654589273035526, + 1.1494463682174683, + 0.8538839221000671, + 0.9946694374084473, + 2.952598810195923, + 2.3358447551727295 + ] + ], + "generated_utilities": [ + 0.009421487338840961, + 2.0, + 2.0, + 0.08336275815963745, + 0.07322987169027328, + 2.0, + 0.08531469106674194, + 0.08539849519729614 + ], + "group_id": "ms-6388b5d2356c556c3e55", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08186312764883041, + 0.11667267978191376, + 0.09751813113689423, + 2.0, + 0.27431026101112366, + 0.11760994046926498, + 0.20386962592601776, + 0.19832774996757507, + 0.22735612094402313, + 0.1356993168592453, + 0.1713169515132904, + 0.1792232096195221, + 0.06848804652690887, + 0.1094626635313034, + 0.11555596441030502, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.0018978863954544067, + -0.4448504149913788, + 0.7108399868011475, + -0.33157044649124146, + -0.062200844287872314, + 1.4758033752441406, + 0.0, + 1.4153923988342285, + 0.14116239547729492, + 0.00777280330657959, + 0.5541017055511475, + 0.07029426097869873, + 0.26837193965911865, + 0.0, + -0.015542566776275635, + 0.3573532998561859, + 0.6168417930603027, + 0.3612897992134094, + 0.369077205657959, + 1.5924487113952637, + 0.0 + ], + [ + -0.09549225866794586, + -0.20593886077404022, + 1.7108399868011475, + -0.1554051786661148, + 0.9377991557121277, + 2.4758033752441406, + 2.4156031608581543, + 2.4153923988342285, + -0.2459496259689331, + -0.9922271966934204, + 1.5541017055511475, + 0.4461512267589569, + 1.2501436471939087, + 1.7566912174224854, + -0.8014360666275024, + 0.04551727697253227, + 1.6168417930603027, + 0.5393297672271729, + 1.369077205657959, + 2.5924487113952637, + 3.1126015186309814 + ], + [ + -0.11610143631696701, + -0.3961976170539856, + 1.9081053733825684, + -0.14575888216495514, + 1.1381605863571167, + 2.7434439659118652, + 2.5136890411376953, + 2.6401467323303223, + -0.45453745126724243, + -0.9589858651161194, + 1.4083189964294434, + 0.5800129175186157, + 0.9940754175186157, + 1.7450767755508423, + -1.0145976543426514, + 0.3312625586986542, + 1.4729199409484863, + 0.2520747482776642, + 1.3623151779174805, + 2.5670394897460938, + 3.3589630126953125 + ], + [ + -0.2318952977657318, + 0.024853751063346863, + 1.829875111579895, + 0.09877045452594757, + 1.1515556573867798, + 2.2239503860473633, + 2.153250217437744, + 2.8057901859283447, + -0.1800437569618225, + -1.2225371599197388, + 1.421208143234253, + 0.6627710461616516, + 1.3895055055618286, + 1.856442928314209, + -0.5928603410720825, + 0.4364123046398163, + 1.7852745056152344, + 0.3345714807510376, + 1.1167365312576294, + 2.7676005363464355, + 3.041402578353882 + ], + [ + -0.2227625697851181, + -0.012646213173866272, + 1.7988159656524658, + 0.21457992494106293, + 0.8877373933792114, + 2.9257700443267822, + 2.2101094722747803, + 2.2874886989593506, + -0.09066365659236908, + -0.49909964203834534, + 2.042389392852783, + 0.6287553906440735, + 1.074569582939148, + 1.7805378437042236, + -1.1840007305145264, + 0.3728531002998352, + 1.165173053741455, + 0.3054528832435608, + 1.4924153089523315, + 2.5540966987609863, + 2.664964199066162 + ], + [ + -0.6864494681358337, + 0.119636669754982, + 1.2460956573486328, + -0.6367122530937195, + 1.4080986976623535, + 2.5262017250061035, + 2.508194923400879, + 2.1323020458221436, + -0.43302130699157715, + -1.2678899765014648, + 1.0870052576065063, + 0.46296897530555725, + 0.765267550945282, + 2.3093044757843018, + -0.46595194935798645, + -0.43870145082473755, + 2.028986930847168, + 0.26284897327423096, + 1.5542359352111816, + 2.530392646789551, + 2.7435970306396484 + ], + [ + 0.18175412714481354, + -0.478390634059906, + 1.286731481552124, + -0.8066962361335754, + 1.4787592887878418, + 2.1033878326416016, + 1.750453233718872, + 2.047861099243164, + 0.3163147568702698, + -0.30588728189468384, + 1.0766479969024658, + 0.5822455286979675, + 1.673675775527954, + 2.039386510848999, + -0.62891685962677, + 0.615638792514801, + 2.018416404724121, + 0.2530737817287445, + 1.34544837474823, + 3.2287912368774414, + 2.9459850788116455 + ], + [ + 0.5059412121772766, + 0.17952178418636322, + 1.3319568634033203, + -0.8850483894348145, + 0.6336387395858765, + 1.775376796722412, + 2.3280270099639893, + 1.7305657863616943, + -0.02617873251438141, + -0.7963138818740845, + 1.9142115116119385, + 0.6635866165161133, + 1.3493314981460571, + 1.9258644580841064, + -0.3795176148414612, + 0.7861210107803345, + 1.0032029151916504, + 0.8491684794425964, + 0.5918128490447998, + 2.399027109146118, + 2.3932981491088867 + ], + [ + -0.34448128938674927, + -0.2996852695941925, + 0.9233196377754211, + -0.22890916466712952, + 1.6932814121246338, + 3.465762138366699, + 2.2005038261413574, + 2.6472361087799072, + -0.8981531858444214, + -0.2658765912055969, + 1.8755900859832764, + 0.2962568402290344, + 1.8239326477050781, + 1.0176585912704468, + -0.7912051677703857, + 0.6727061867713928, + 2.199978828430176, + 1.015731692314148, + 0.43087130784988403, + 3.5876548290252686, + 3.893789052963257 + ] + ], + "predicted_scores": [ + 6.715732097625732, + 5.764956951141357, + 5.601903438568115, + 5.531642913818359, + 5.302748203277588, + 5.134200096130371, + 4.9023871421813965, + 4.442611217498779 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "38d539e46140304e3f157cbc207aa0d9489e5470633c216a0379906af445a1ec", + "stored_base_utility": 0.08186312764883041, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06173701956868172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06173701956868172 + }, + "base_progress": 0.06173701956868172, + "base_success": false, + "base_utility": 0.06173701956868172, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.109224833548069, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.109224833548069 + }, + { + "contact_quality": null, + "progress": 0.057921767234802246, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.057921767234802246 + }, + { + "contact_quality": null, + "progress": 0.06627502292394638, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06627502292394638 + }, + { + "contact_quality": null, + "progress": 0.021286645904183388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021286645904183388 + }, + { + "contact_quality": null, + "progress": 0.06679177284240723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06679177284240723 + }, + { + "contact_quality": null, + "progress": 0.04627428576350212, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04627428576350212 + }, + { + "contact_quality": null, + "progress": 0.054965514689683914, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.054965514689683914 + }, + { + "contact_quality": null, + "progress": 0.14829117059707642, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14829117059707642 + } + ], + "candidate_progress": [ + 0.109224833548069, + 0.057921767234802246, + 0.06627502292394638, + 0.021286645904183388, + 0.06679177284240723, + 0.04627428576350212, + 0.054965514689683914, + 0.14829117059707642 + ], + "candidate_source_chart_ids": [ + "ms-3a4e2d080c9c4b279895", + "ms-7e88428497a6b13936c4", + "ms-3a4e2d080c9c4b279895", + "ms-a2518826fa0a816cc46f", + "ms-d535b38d768633f6d922", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-d535b38d768633f6d922" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-638d2ba1917eff4d09e0", + "generated_tangents": [ + [ + 0.19399428367614746, + -0.19429710507392883, + 0.7175668478012085, + 0.10569876432418823, + 1.6889584064483643, + 1.2522661685943604, + 0.3853796720504761, + 0.30492085218429565, + -0.23217259347438812, + -0.04009782522916794, + 0.34059959650039673, + 0.7768262028694153, + 0.19477424025535583, + 0.8821943402290344, + 0.8789517283439636, + 0.07258066534996033, + 0.0825379490852356, + 0.24814879894256592, + 1.0100672245025635, + -1.0790424346923828, + -0.5989418029785156 + ], + [ + 0.31291431188583374, + -0.44760996103286743, + 0.5468348860740662, + 0.04534590244293213, + 1.4834082126617432, + 0.4475386142730713, + 0.15500012040138245, + 0.3648453950881958, + -0.1622944325208664, + 0.283906489610672, + 0.05420593544840813, + 0.6349453926086426, + -0.5642687082290649, + 0.5123119950294495, + 0.3089844882488251, + 0.13765224814414978, + 0.023821890354156494, + -0.1964070051908493, + 0.7212094664573669, + -1.5697035789489746, + -1.1024484634399414 + ], + [ + 0.21944445371627808, + -0.26180827617645264, + 0.7513684034347534, + 0.07701483368873596, + 1.6302812099456787, + 1.209617257118225, + 0.567581832408905, + 0.43779635429382324, + -0.13354693353176117, + 0.07422672212123871, + 0.13548581302165985, + 0.7683057188987732, + 0.16806039214134216, + 0.6201119422912598, + 0.6791425347328186, + 0.0793917179107666, + 0.247845858335495, + 0.19680571556091309, + 1.0756046772003174, + -0.9696177840232849, + -0.5063502788543701 + ], + [ + -0.004198238253593445, + -0.374566912651062, + 0.6863826513290405, + -0.3365015983581543, + 1.32783842086792, + 1.1231145858764648, + 0.16173768043518066, + 0.12974990904331207, + -0.10361332446336746, + 0.23197752237319946, + 0.2207789421081543, + 0.8557438254356384, + -0.06281271576881409, + 0.5347269773483276, + 0.2829627990722656, + -0.15411359071731567, + -0.03339408338069916, + 0.1670663207769394, + 0.570730447769165, + -1.079788327217102, + -0.38740941882133484 + ], + [ + 0.2365972101688385, + -0.41906940937042236, + 0.5717867016792297, + -0.32165977358818054, + 1.379390001296997, + 0.3509386479854584, + 0.7487069368362427, + 0.21619491279125214, + 6.029754877090454e-05, + 0.09556954354047775, + 0.30910229682922363, + 0.43290406465530396, + -0.3340233564376831, + 0.2862071990966797, + 0.5412895679473877, + 0.047336529940366745, + 0.22724208235740662, + -0.0628570169210434, + 1.2245359420776367, + -1.1665306091308594, + -0.6084585189819336 + ], + [ + 0.06348957866430283, + -0.22858506441116333, + 0.7147878408432007, + -0.23703420162200928, + 1.6729516983032227, + 0.7785142660140991, + 0.6724982261657715, + 0.34640511870384216, + -0.11504130065441132, + -0.07103797048330307, + 0.1612311154603958, + 0.614039421081543, + -0.08057567477226257, + 0.7951291799545288, + 0.4771057367324829, + 0.01192605309188366, + -0.029168590903282166, + 0.09965971857309341, + 0.7977463006973267, + -0.8623479008674622, + -0.6617336273193359 + ], + [ + 0.26776623725891113, + -0.2583177089691162, + 0.7229114770889282, + -0.25329554080963135, + 1.555058479309082, + 0.5885607004165649, + 0.5584661364555359, + 0.18812701106071472, + -0.018099995329976082, + 0.23198270797729492, + 0.2221827208995819, + 0.5615720152854919, + 0.22216364741325378, + 0.3671315908432007, + 0.30829712748527527, + 0.032752685248851776, + 0.15349799394607544, + -0.044431447982788086, + 0.9087569713592529, + -0.9487636089324951, + -0.48602381348609924 + ], + [ + 0.20732782781124115, + -0.4252946972846985, + 0.5844835042953491, + -0.27273496985435486, + 1.3516948223114014, + 0.6736023426055908, + 0.6423986554145813, + 0.35052746534347534, + -0.009159471839666367, + -0.15334001183509827, + 0.24608778953552246, + 0.47937795519828796, + -0.3758259415626526, + 0.36896395683288574, + 0.5268030166625977, + 0.0503540076315403, + 0.11380910128355026, + -0.07216577231884003, + 1.3559410572052002, + -1.0476348400115967, + -0.6348272562026978 + ] + ], + "generated_utilities": [ + 0.109224833548069, + 0.057921767234802246, + 0.06627502292394638, + 0.021286645904183388, + 0.06679177284240723, + 0.04627428576350212, + 0.054965514689683914, + 0.14829117059707642 + ], + "group_id": "ms-638d2ba1917eff4d09e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.028925320133566856, + 0.11176212877035141, + 0.057868462055921555, + 0.09960909932851791, + 0.07349967956542969, + 0.029011588543653488, + 0.06443297863006592, + 0.05166193097829819, + 0.025834282860159874, + 0.056667208671569824, + 0.04477008432149887, + 0.019685089588165283, + 0.018628716468811035, + 0.08587563037872314, + 0.02232462167739868, + 0.010265171527862549 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04120570421218872, + 0.038339823484420776, + 0.5979268550872803, + -0.05306033045053482, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.3407015800476074, + -0.21580028533935547, + -0.036036908626556396, + -0.004389047622680664, + 0.5364620685577393, + 0.026709645986557007, + 0.0, + 1.0126078128814697, + -0.10034149885177612, + -0.16565871238708496, + 0.07125610113143921, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + -0.5276790261268616, + -0.19392354786396027, + 0.5979268550872803, + 0.05016906559467316, + 1.9043312072753906, + 0.6329002380371094, + 0.0, + 0.6852816939353943, + -0.21580028533935547, + 0.04531368613243103, + -0.09276384115219116, + 0.5364620685577393, + 0.021915271878242493, + 0.0, + 1.0126078128814697, + 0.29255303740501404, + -0.16565871238708496, + 0.14600259065628052, + 1.3883657455444336, + -0.547389030456543, + 0.0 + ], + [ + 0.5910290479660034, + -0.038985610008239746, + 1.5555552244186401, + -1.028314232826233, + 2.3103959560394287, + 0.8163888454437256, + 2.064336061477661, + 1.168483018875122, + -0.21718227863311768, + -0.19770365953445435, + 0.11310696601867676, + 0.6850457191467285, + -0.06804794818162918, + 0.33076921105384827, + 1.7529149055480957, + -0.562712550163269, + -1.6352436542510986, + 0.1224215030670166, + 3.354771852493286, + -1.1317338943481445, + -1.7348618507385254 + ] + ], + "predicted_scores": [ + 4.312126636505127, + 4.290807723999023, + 4.064854145050049, + 3.963754892349243, + 3.7331457138061523, + 3.612520694732666, + 3.518197536468506, + 3.428798198699951 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "5d6f4227fab3b4bb36756c9a15c32b6fe6c9a0ae4b07b3f20a2bd65c2594df13", + "stored_base_utility": 0.028925320133566856, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.018323838710784912, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018323838710784912 + }, + { + "contact_quality": null, + "progress": 0.017151853069663048, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.017151853069663048 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.023069700226187706, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023069700226187706 + }, + { + "contact_quality": null, + "progress": 0.005640605930238962, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005640605930238962 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.019837161526083946, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019837161526083946 + }, + { + "contact_quality": null, + "progress": 0.016323328018188477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016323328018188477 + } + ], + "candidate_progress": [ + 0.018323838710784912, + 0.017151853069663048, + 1.0, + 0.023069700226187706, + 0.005640605930238962, + 1.0, + 0.019837161526083946, + 0.016323328018188477 + ], + "candidate_source_chart_ids": [ + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-2e40499cb146da6f89e7", + "ms-a6a6de64c39abad45f86" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + true, + false, + false, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-652ad552ba98d2736620", + "generated_tangents": [ + [ + 0.3993667960166931, + 0.01467999815940857, + 0.8236827850341797, + -0.701335072517395, + 2.074022054672241, + 1.8071799278259277, + 0.779463529586792, + 0.3570999205112457, + 0.06147110462188721, + 0.7161964774131775, + 0.3467694818973541, + 1.2415695190429688, + -0.12027537822723389, + -0.23912478983402252, + -0.42852506041526794, + 0.04579297453165054, + 1.4412566423416138, + -0.21752147376537323, + 1.143805980682373, + 2.98844838142395, + 1.2782543897628784 + ], + [ + -0.024377495050430298, + -0.027264267206192017, + 0.7988369464874268, + -0.7997981309890747, + 2.4030120372772217, + 2.5693132877349854, + 0.9931743741035461, + 0.28386300802230835, + -0.00976945273578167, + 1.0466086864471436, + 0.6274899840354919, + 1.215024471282959, + -1.2369223833084106, + 0.47804349660873413, + -0.31641435623168945, + 0.056881461292505264, + 1.430560827255249, + -0.3863064646720886, + 0.6832652688026428, + 2.42386531829834, + 1.7024511098861694 + ], + [ + 0.09451492875814438, + -0.017048465088009834, + 0.9759140014648438, + -0.8719127178192139, + 2.0689077377319336, + 1.8039824962615967, + 0.901330292224884, + 0.24651363492012024, + 0.0975688025355339, + 0.6806550621986389, + 0.5312308669090271, + 1.3876440525054932, + -0.11206024885177612, + 0.10132507979869843, + -0.3278767764568329, + -0.131260946393013, + 1.5182933807373047, + -0.17465271055698395, + 0.9838205575942993, + 2.670842409133911, + 1.2546682357788086 + ], + [ + 0.026769131422042847, + 0.0005880407989025116, + 0.7986657619476318, + -0.8370130062103271, + 1.9655966758728027, + 1.8510187864303589, + 0.7054927349090576, + 0.31965190172195435, + 0.08110027760267258, + 0.7890376448631287, + 0.48895692825317383, + 1.372125267982483, + -0.3302253484725952, + 0.14341162145137787, + -0.32688868045806885, + -0.07599435746669769, + 1.526108741760254, + -0.12525631487369537, + 0.8992928266525269, + 2.496145248413086, + 1.4744231700897217 + ], + [ + 0.027667418122291565, + -0.06151444464921951, + 1.2060891389846802, + -0.7992005348205566, + 1.8079657554626465, + 2.1099348068237305, + 0.6640889048576355, + 0.24634325504302979, + 0.04415634274482727, + 0.7627803683280945, + 0.47449788451194763, + 1.4929237365722656, + -0.38216859102249146, + 0.27360230684280396, + -0.3836110830307007, + -0.047117680311203, + 1.5105971097946167, + -0.23858605325222015, + 0.7320762276649475, + 2.231227159500122, + 1.9689866304397583 + ], + [ + 0.06250117719173431, + 0.04385526478290558, + 0.9026212692260742, + -0.6674528121948242, + 1.7355897426605225, + 1.8075567483901978, + 0.394543319940567, + 0.4861884117126465, + 0.09448187798261642, + 0.6392885446548462, + 0.38521555066108704, + 1.336188554763794, + 0.06261160969734192, + 0.19100789725780487, + -0.4509143829345703, + -0.05264287814497948, + 1.441964864730835, + -0.00014866888523101807, + 0.7572662830352783, + 2.3709421157836914, + 1.1847805976867676 + ], + [ + 0.09632676839828491, + -0.009599275887012482, + 0.7571297287940979, + -0.7809529304504395, + 2.08193302154541, + 1.2948377132415771, + 0.44271188974380493, + 0.16933895647525787, + -0.02538822777569294, + 0.8536251187324524, + 0.3307485580444336, + 1.5156704187393188, + -0.5239653587341309, + 0.05099817365407944, + -0.2650001049041748, + 0.08685488998889923, + 1.6421197652816772, + -0.18257303535938263, + 0.7593610882759094, + 1.9531049728393555, + 1.2956956624984741 + ], + [ + -0.12293921411037445, + 0.1826166808605194, + 0.9262709617614746, + -0.9620940685272217, + 2.1618926525115967, + 1.9479553699493408, + 0.44185671210289, + 0.18182949721813202, + 0.023798175156116486, + 0.9044660329818726, + 0.4397014081478119, + 1.205376386642456, + 0.04302605986595154, + 0.4440864324569702, + -0.33897510170936584, + -0.12255626916885376, + 1.4983183145523071, + -0.13862715661525726, + 1.0010534524917603, + 2.302675485610962, + 1.759432315826416 + ] + ], + "generated_utilities": [ + 0.018323838710784912, + 0.017151853069663048, + 2.0, + 0.023069700226187706, + 0.005640605930238962, + 2.0, + 0.019837161526083946, + 0.016323328018188477 + ], + "group_id": "ms-652ad552ba98d2736620", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08378833532333374, + 0.10458396375179291, + 0.19086095690727234, + 0.04039820283651352, + 0.15980230271816254, + 0.0428888201713562, + 0.2367449700832367, + 0.18185830116271973, + 0.17593276500701904, + 0.16103637218475342, + 0.15250858664512634, + 0.05226989835500717, + 0.1701633185148239, + 0.11590366065502167, + 0.05217729136347771, + 0.03575938940048218 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.09009888768196106, + 0.08798432350158691, + 0.718157172203064, + -0.6351244449615479, + 1.405301570892334, + 0.6596384048461914, + 0.0, + 0.060225605964660645, + -0.0894029438495636, + -0.10323715209960938, + 0.052752673625946045, + 0.5354657173156738, + -0.013395614922046661, + 0.0, + -0.3450838029384613, + -0.11172826588153839, + 0.7912200689315796, + 0.33702391386032104, + 0.5586241483688354, + 1.8093180656433105, + 0.0 + ], + [ + 0.557296633720398, + 1.087984323501587, + 1.718157172203064, + -1.6351244449615479, + 2.405301570892334, + 1.6596384048461914, + 1.7430553436279297, + 0.3360595703125, + -0.2521040737628937, + 0.8967628479003906, + 0.26587578654289246, + 1.5354657173156738, + -0.12294083833694458, + -0.022678710520267487, + -0.416314959526062, + -0.26016199588775635, + 1.7912200689315796, + -0.16825227439403534, + 1.5586241483688354, + 2.8093180656433105, + 2.2589924335479736 + ], + [ + 0.2635665237903595, + 1.0005124807357788, + 1.973160743713379, + -1.7548720836639404, + 2.176497459411621, + 1.5416926145553589, + 1.9556171894073486, + 0.2215275913476944, + -0.5432531833648682, + 0.8434366583824158, + 0.3559061288833618, + 1.6657240390777588, + -0.3025258779525757, + 0.04401832073926926, + -0.17062894999980927, + -0.1509227454662323, + 1.7901219129562378, + 0.0010261386632919312, + 1.726505994796753, + 2.9778714179992676, + 2.0530292987823486 + ], + [ + 0.23987123370170593, + 0.8904718160629272, + 1.467047929763794, + -1.990342378616333, + 2.7685346603393555, + 2.0486786365509033, + 2.013002395629883, + 0.006436288356781006, + -0.25436922907829285, + 1.1101897954940796, + 0.4448293447494507, + 1.3105798959732056, + 0.07367905974388123, + 0.23779401183128357, + -0.34580594301223755, + -0.35948312282562256, + 1.4435756206512451, + -0.35472720861434937, + 1.801603078842163, + 3.0683224201202393, + 2.2330009937286377 + ], + [ + 0.6738064289093018, + 1.3973212242126465, + 2.0960373878479004, + -1.735988736152649, + 2.5216798782348633, + 1.1975955963134766, + 2.234865665435791, + 0.4602587819099426, + -0.7169049382209778, + 0.9412583708763123, + 0.7081141471862793, + 1.1730585098266602, + -0.016052812337875366, + -0.10474976152181625, + -0.03458032011985779, + 0.044317394495010376, + 1.6339548826217651, + -0.5292693376541138, + 1.5412750244140625, + 3.018833875656128, + 2.6051149368286133 + ], + [ + 0.6305910348892212, + 1.540043592453003, + 1.626009464263916, + -1.8823078870773315, + 2.2431485652923584, + 2.1866872310638428, + 2.3003249168395996, + 0.25838330388069153, + -0.2936764061450958, + 1.2413102388381958, + -0.12014040350914001, + 1.743309497833252, + 0.3618801534175873, + -0.37178292870521545, + 0.11553955078125, + 0.04856392741203308, + 1.2029554843902588, + 0.04960852861404419, + 2.0190422534942627, + 2.5883281230926514, + 2.7636377811431885 + ], + [ + 0.5004369616508484, + 1.4795310497283936, + 2.325191020965576, + -1.7464677095413208, + 1.9271103143692017, + 1.2483245134353638, + 1.9852193593978882, + 0.24291110038757324, + 0.4463852345943451, + 0.5267151594161987, + 0.32179588079452515, + 1.2356117963790894, + 0.03201058506965637, + 0.2881847023963928, + -1.0345205068588257, + -0.4171831011772156, + 1.5080864429473877, + 0.11479203402996063, + 1.9481834173202515, + 2.840816020965576, + 2.427428960800171 + ], + [ + -0.3133848309516907, + 0.43174320459365845, + 2.305823802947998, + -1.657350778579712, + 2.010749578475952, + 1.9222235679626465, + 1.3648147583007812, + 0.1938892900943756, + -0.6084526777267456, + 0.7855725288391113, + -0.07715931534767151, + 0.9140809178352356, + -0.9242425560951233, + -0.4681842029094696, + -0.07827043533325195, + -0.23733587563037872, + 0.9328996539115906, + -0.6088451147079468, + 0.8040639162063599, + 2.620466947555542, + 2.5148611068725586 + ] + ], + "predicted_scores": [ + 2.189335346221924, + 2.0589840412139893, + 1.8798794746398926, + 1.8045814037322998, + 1.5384629964828491, + 1.338186502456665, + 1.3312212228775024, + 0.7763898968696594 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "767747424", + "selected_index": 0, + "state_hash": "8c1ab306d1e82f81085196c60b3c846d2e2eb20ebc538597af359a6252623e7e", + "stored_base_utility": 0.08378833532333374, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03784912824630737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03784912824630737 + }, + "base_progress": 0.03784912824630737, + "base_success": false, + "base_utility": 0.03784912824630737, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.03371499106287956, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03371499106287956 + }, + { + "contact_quality": null, + "progress": 0.04782070592045784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04782070592045784 + }, + { + "contact_quality": null, + "progress": 0.03829113766551018, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03829113766551018 + }, + { + "contact_quality": null, + "progress": 0.12104523181915283, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12104523181915283 + }, + { + "contact_quality": null, + "progress": 0.07092549651861191, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07092549651861191 + }, + { + "contact_quality": null, + "progress": 0.0719817504286766, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0719817504286766 + }, + { + "contact_quality": null, + "progress": 0.05774464085698128, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05774464085698128 + }, + { + "contact_quality": null, + "progress": 0.028163930401206017, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.028163930401206017 + } + ], + "candidate_progress": [ + 0.03371499106287956, + 0.04782070592045784, + 0.03829113766551018, + 0.12104523181915283, + 0.07092549651861191, + 0.0719817504286766, + 0.05774464085698128, + 0.028163930401206017 + ], + "candidate_source_chart_ids": [ + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-7a044cb6c7bb03f3ea10", + "ms-25ab20bb3da8ba34be95", + "ms-48178f52a96de3d1ead0" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-674fad0e7c042d23629e", + "generated_tangents": [ + [ + 0.9901873469352722, + 0.12360922992229462, + -0.2232363522052765, + -0.38938653469085693, + 0.3337686061859131, + -2.008176326751709, + -0.3517991304397583, + 0.46582189202308655, + 0.04784628748893738, + -0.33971720933914185, + 0.017600134015083313, + -0.5129214525222778, + -1.0531117916107178, + -0.4508397579193115, + -0.203139066696167, + 0.0007231589406728745, + 0.6066197156906128, + -0.7301870584487915, + -0.27302658557891846, + -1.1774073839187622, + -0.44787415862083435 + ], + [ + 0.8991358280181885, + 0.19254790246486664, + -0.17443087697029114, + -0.44649723172187805, + 0.23242536187171936, + -1.9363666772842407, + -0.1544741988182068, + 0.4052850604057312, + 0.044559232890605927, + -0.370149165391922, + -0.07202247530221939, + -0.4836416244506836, + -1.3371537923812866, + -0.23648254573345184, + -0.21802955865859985, + 0.05119294300675392, + 0.6496686339378357, + -0.6800534725189209, + -0.19558092951774597, + -1.0557403564453125, + -0.46343186497688293 + ], + [ + 0.9861155152320862, + 0.14955280721187592, + -0.3322644531726837, + -0.46376001834869385, + 0.4665031135082245, + -1.6896737813949585, + -0.16477763652801514, + 0.3933957517147064, + 0.07449617236852646, + -0.4259939193725586, + -0.03261926770210266, + -0.4738824963569641, + -1.3679347038269043, + -0.43129962682724, + -0.2540896534919739, + -0.12142802774906158, + 0.4131118953227997, + -0.7374284267425537, + -0.3372821807861328, + -1.2552692890167236, + -0.47657260298728943 + ], + [ + 0.7735786437988281, + 0.22672899067401886, + -0.22413203120231628, + -0.7112962007522583, + -0.16793420910835266, + -1.765476942062378, + -0.6376448273658752, + 0.27754539251327515, + -0.07475936412811279, + -0.7416353821754456, + -0.32008084654808044, + -0.44644373655319214, + -0.9784222841262817, + -0.6154751777648926, + -0.38817328214645386, + 0.1537492871284485, + 0.7908267378807068, + -0.8680008053779602, + 0.06949670612812042, + -1.153836727142334, + -0.06420813500881195 + ], + [ + 0.7969854474067688, + 0.22821016609668732, + -0.2962935268878937, + -0.6233872175216675, + 0.05773313343524933, + -2.0004169940948486, + -0.5777955651283264, + 0.28147614002227783, + -0.13770993053913116, + -0.5338562726974487, + -0.24591869115829468, + -0.5146559476852417, + -1.2625700235366821, + -0.49143052101135254, + -0.3756565749645233, + 0.14758989214897156, + 0.6155756115913391, + -0.7990389466285706, + -0.022716522216796875, + -1.0248379707336426, + -0.2633824050426483 + ], + [ + 0.7150876522064209, + 0.19802415370941162, + -0.3352034389972687, + -0.4889489710330963, + 0.09766247868537903, + -1.9784432649612427, + -0.5443668961524963, + 0.3809836506843567, + -0.12667275965213776, + -0.5288137793540955, + -0.30938172340393066, + -0.47195154428482056, + -1.2304800748825073, + -0.241909459233284, + -0.34431222081184387, + 0.12775352597236633, + 0.730070173740387, + -0.9150301814079285, + 0.06360238790512085, + -0.9554764032363892, + -0.44829538464546204 + ], + [ + 0.9162852764129639, + 0.1627006232738495, + -0.5613288879394531, + -0.6743629574775696, + 0.2753516137599945, + -1.476357102394104, + -0.49550431966781616, + 0.11746828258037567, + -0.007133401930332184, + -0.24751639366149902, + -0.10520651191473007, + -0.10798293352127075, + -1.2416572570800781, + -0.41993898153305054, + -0.6537002325057983, + -0.12407141923904419, + 0.3091234564781189, + -0.7746067643165588, + 0.16305281221866608, + -1.2652614116668701, + -0.30002978444099426 + ], + [ + 0.7665120363235474, + 0.003784533590078354, + -0.741275429725647, + -0.7520970106124878, + 0.4740793704986572, + -1.833933711051941, + 0.7798678278923035, + 0.7178219556808472, + -0.09947560727596283, + -0.48181402683258057, + -0.5541530251502991, + -0.373829185962677, + -1.8973037004470825, + -0.7944494485855103, + -0.4560420513153076, + 0.35093244910240173, + 0.08538462221622467, + -0.8632031083106995, + -0.3773888945579529, + -1.253408670425415, + -0.9948717355728149 + ] + ], + "generated_utilities": [ + 0.03371499106287956, + 0.04782070592045784, + 0.03829113766551018, + 0.12104523181915283, + 0.07092549651861191, + 0.0719817504286766, + 0.05774464085698128, + 0.028163930401206017 + ], + "group_id": "ms-674fad0e7c042d23629e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07829316705465317, + 0.080319844186306, + 0.11335399001836777, + 0.15264075994491577, + 0.14950451254844666, + 0.07767044007778168, + 0.12988285720348358, + 0.10878288745880127, + 0.15573063492774963, + 0.10951457917690277, + 0.06582474708557129, + 0.02477661892771721, + 0.07745532691478729, + 0.03348777815699577, + 0.10050909221172333, + 0.033770203590393066 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 1.5533217191696167, + 0.5758762359619141, + -0.6480545997619629, + -0.10892429947853088, + 0.7943102717399597, + -2.7399702072143555, + -1.8395006656646729, + 1.447904109954834, + 1.4495785236358643, + -0.055675506591796875, + -1.3537942171096802, + 0.38944748044013977, + -1.6196211576461792, + -1.2710431814193726, + -0.9744676947593689, + 0.057668983936309814, + 1.8350472450256348, + -1.2618876695632935, + -1.2385435104370117, + -1.7976703643798828, + -1.9491409063339233 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9072847366333008, + 0.06923278421163559, + -0.14729690551757812, + 0.15558937191963196, + 0.40412187576293945, + -2.342223644256592, + 0.0, + 0.2589508295059204, + 0.35182610154151917, + 0.10176032781600952, + -0.3489869236946106, + 0.13150231540203094, + -1.3123579025268555, + 0.0, + 0.10376164317131042, + 0.0909266471862793, + 0.1919306516647339, + 0.28928786516189575, + -0.4625590741634369, + -0.30475670099258423, + 0.0 + ], + [ + 1.9072847366333008, + 0.05373973771929741, + -1.1472969055175781, + -0.47107377648353577, + 0.3914180397987366, + -3.342223644256592, + -2.4980990886688232, + 1.2589508295059204, + 0.8455543518066406, + -0.7174267172813416, + -1.0694091320037842, + 0.33346641063690186, + -2.3123579025268555, + -1.1108808517456055, + -0.33852219581604004, + -0.7232105731964111, + 1.145590901374817, + -0.7107121348381042, + -0.5049349069595337, + -1.112295150756836, + -1.244770884513855 + ], + [ + 1.808228611946106, + -0.18174676597118378, + -1.2057973146438599, + -0.6115378737449646, + 0.11874309182167053, + -3.3499772548675537, + -2.3861043453216553, + 1.2540870904922485, + 1.1210845708847046, + -0.953934371471405, + -1.3015953302383423, + 0.6089286804199219, + -2.1655566692352295, + -0.8375923037528992, + -0.26793593168258667, + -0.998324990272522, + 1.2481704950332642, + -0.8320856690406799, + -0.376935750246048, + -1.0882012844085693, + -0.9857641458511353 + ], + [ + 2.291844606399536, + 0.05574151128530502, + -0.9760321974754333, + -0.5710926055908203, + 0.8478215336799622, + -3.22356915473938, + -2.591245651245117, + 1.2184754610061646, + 0.855300784111023, + -1.0267595052719116, + -1.3318730592727661, + 0.4197496771812439, + -2.0368778705596924, + -1.3685163259506226, + -0.582176685333252, + -0.8082455992698669, + 1.3435251712799072, + -0.44022125005722046, + -0.06775334477424622, + -0.7692400217056274, + -0.8863754272460938 + ] + ], + "predicted_scores": [ + 2.433502674102783, + 2.286835193634033, + 1.6041948795318604, + 0.9942966103553772, + 0.9331410527229309, + 0.6499817967414856, + -0.26208561658859253, + -1.5038988590240479 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "6802b2d1fb8a5afa54b7321e2bb938176de48d74f28e9b2a9c741c23547f0ef1", + "stored_base_utility": 0.07829316705465317, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.12791956961154938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12791956961154938 + }, + "base_progress": 0.12791956961154938, + "base_success": false, + "base_utility": 0.12791956961154938, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.034705858677625656, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.034705858677625656 + }, + { + "contact_quality": null, + "progress": 0.10312310606241226, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10312310606241226 + }, + { + "contact_quality": null, + "progress": 0.06007760763168335, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06007760763168335 + }, + { + "contact_quality": null, + "progress": 0.07922307401895523, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07922307401895523 + }, + { + "contact_quality": null, + "progress": 0.0422377772629261, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0422377772629261 + }, + { + "contact_quality": null, + "progress": 0.06801291555166245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06801291555166245 + }, + { + "contact_quality": null, + "progress": 0.0445721335709095, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0445721335709095 + }, + { + "contact_quality": null, + "progress": 0.028874794021248817, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.028874794021248817 + } + ], + "candidate_progress": [ + 0.034705858677625656, + 0.10312310606241226, + 0.06007760763168335, + 0.07922307401895523, + 0.0422377772629261, + 0.06801291555166245, + 0.0445721335709095, + 0.028874794021248817 + ], + "candidate_source_chart_ids": [ + "ms-1ba37e30b90a3b28e62f", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-728b865677134d3788e3", + "generated_tangents": [ + [ + -1.0635509490966797, + -0.144166499376297, + 0.802090585231781, + -0.2630089819431305, + 0.42815130949020386, + -0.6543326377868652, + 0.8752983212471008, + 0.20988425612449646, + -0.0268863532692194, + 0.2100597620010376, + -0.7060477137565613, + 0.38852477073669434, + -0.16292265057563782, + 0.12568438053131104, + -1.2479095458984375, + 0.9765241742134094, + 1.52181875705719, + -0.2224280685186386, + -0.4899129867553711, + -0.5188641548156738, + 0.4392372965812683 + ], + [ + -0.8478046655654907, + -0.0586041659116745, + 0.553169846534729, + -0.13006587326526642, + 0.22915710508823395, + -0.8670809268951416, + -0.4086799621582031, + 0.5121471285820007, + -0.12022474408149719, + -0.1769028604030609, + -1.2220101356506348, + 0.8367915749549866, + -0.6423728466033936, + 0.6282032132148743, + -1.4395055770874023, + 0.9174424409866333, + 1.0084816217422485, + -0.7866349220275879, + -0.2790687680244446, + -0.2328687459230423, + 0.18835793435573578 + ], + [ + -1.079932451248169, + 0.02097637951374054, + 0.42843595147132874, + -0.06362952291965485, + 0.5173932909965515, + -1.1870826482772827, + -0.24006062746047974, + 0.6398480534553528, + -0.09183204174041748, + -0.22705382108688354, + -0.8776602745056152, + 1.1680071353912354, + -0.47220680117607117, + 0.8978961706161499, + -1.3510977029800415, + 0.6759516000747681, + 1.1731407642364502, + -0.6885026693344116, + 0.30227115750312805, + -0.28838708996772766, + 0.6158943176269531 + ], + [ + -0.8748584985733032, + -0.047565214335918427, + 0.7239023447036743, + -0.6460307836532593, + 0.46451473236083984, + -0.8489879369735718, + -0.4247792363166809, + 0.8628618717193604, + -0.0037170033901929855, + -0.08119147270917892, + -0.8511540293693542, + 0.5110795497894287, + -0.6595770120620728, + 0.7682916522026062, + -0.9892342686653137, + 0.7929928302764893, + 1.4224151372909546, + -0.6785783171653748, + -0.4034343361854553, + -0.4234212040901184, + 0.35307013988494873 + ], + [ + -1.0285038948059082, + -0.09682001918554306, + 0.5330713987350464, + -0.41781651973724365, + 0.594927191734314, + -0.7877529859542847, + -0.3395764231681824, + 0.7509548664093018, + -0.1473914235830307, + -0.10875154286623001, + -1.1968668699264526, + 0.8369057774543762, + -0.7651267647743225, + 0.8366407155990601, + -1.185888409614563, + 0.7464142441749573, + 1.060364007949829, + -0.8087634444236755, + -0.18946054577827454, + -0.5167063474655151, + 0.14220881462097168 + ], + [ + -0.9473214149475098, + -0.13021314144134521, + 0.5214983224868774, + -0.3412332534790039, + 0.4770852327346802, + -0.9695706367492676, + -0.3040820360183716, + 0.7984873056411743, + -0.14576475322246552, + -0.07656867057085037, + -1.1646051406860352, + 0.8167257308959961, + -0.8126498460769653, + 0.8298033475875854, + -1.285340428352356, + 0.7093576788902283, + 1.071486473083496, + -0.9461115002632141, + -0.1984189748764038, + -0.32209935784339905, + 0.33333319425582886 + ], + [ + -0.9799249172210693, + 0.09466944634914398, + 0.7084234952926636, + -0.5850423574447632, + 0.565342128276825, + -0.40946367383003235, + -0.6522653698921204, + 0.62880539894104, + 0.06187302619218826, + -0.0017405301332473755, + -0.925563395023346, + 0.8420876860618591, + -0.6774611473083496, + 0.8070276975631714, + -0.9017876982688904, + 0.7470946311950684, + 0.9803131222724915, + -0.7930443286895752, + -0.2944353222846985, + -0.5137248635292053, + 0.0745958611369133 + ], + [ + -1.2287569046020508, + -0.055298492312431335, + 0.5617462992668152, + -0.7970031499862671, + 0.9275137186050415, + -0.7675570249557495, + -0.7011186480522156, + 0.6776913404464722, + -0.2211901992559433, + -0.19782641530036926, + -1.2233498096466064, + 0.7510000467300415, + -0.41770225763320923, + 0.9504981637001038, + -1.2038949728012085, + 0.7601245045661926, + 1.2908552885055542, + -0.7631882429122925, + -0.4973359704017639, + -0.44955307245254517, + 0.174300417304039 + ] + ], + "generated_utilities": [ + 0.034705858677625656, + 0.10312310606241226, + 0.06007760763168335, + 0.07922307401895523, + 0.0422377772629261, + 0.06801291555166245, + 0.0445721335709095, + 0.028874794021248817 + ], + "group_id": "ms-728b865677134d3788e3", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.05257302522659302, + 0.05559448525309563, + 0.11835493892431259, + 0.04238405078649521, + 0.16897141933441162, + 0.02001136541366577, + 0.1609041690826416, + 0.14248299598693848, + 0.14091376960277557, + 0.17822234332561493, + 0.22658847272396088, + 0.1736571490764618, + 0.18659374117851257, + 0.12753915786743164, + 0.07164359092712402, + 0.0063618822023272514 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04065108299255371, + 0.05943702906370163, + 0.0039327144622802734, + 0.10760968923568726, + -0.3465921878814697, + -0.6843758821487427, + 0.0, + 0.3034271001815796, + -0.38217779994010925, + 0.019724853336811066, + -1.1125431060791016, + -0.016437113285064697, + 0.09826856851577759, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + 0.055904269218444824, + -0.2721228003501892, + 0.07830148935317993, + 0.0 + ], + [ + -0.9075424671173096, + 0.06243014335632324, + 1.0039327144622803, + -0.6845037937164307, + 0.050797268748283386, + -1.6843758821487427, + -1.3340954780578613, + 1.3034271001815796, + -0.29618290066719055, + -0.035879287868738174, + -2.1125431060791016, + 0.9835628867149353, + -0.9017314314842224, + -1.01132333278656, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233 + ], + [ + -1.117501139640808, + 0.23503510653972626, + 0.9669153094291687, + -0.8317459225654602, + 0.06331060081720352, + -1.5474717617034912, + -1.3840656280517578, + 1.240980863571167, + -0.053158730268478394, + 0.10333722829818726, + -2.2084054946899414, + 1.1802411079406738, + -0.8987206816673279, + -1.136216402053833, + -1.9180536270141602, + 1.4563629627227783, + 2.625094413757324, + -0.1296394169330597, + 0.3340006470680237, + -0.7212131023406982, + 0.9384214878082275 + ], + [ + -1.2640053033828735, + 0.2415706366300583, + 0.9424604177474976, + -0.8472410440444946, + 0.35185110569000244, + -1.8022713661193848, + -1.2391949892044067, + 0.9333417415618896, + 0.03309231996536255, + 0.22933605313301086, + -1.7521884441375732, + 1.0360857248306274, + -0.6365935802459717, + -0.7614871859550476, + -2.103987216949463, + 1.5381183624267578, + 2.11069917678833, + -0.5904175639152527, + -0.07874824106693268, + -0.3884509205818176, + 1.0602728128433228 + ], + [ + -0.7705078721046448, + 0.18350359797477722, + 0.579454779624939, + -0.5434049963951111, + 0.2274666726589203, + -1.7208446264266968, + -1.5971848964691162, + 1.6145182847976685, + -0.6915329694747925, + 0.20122811198234558, + -1.8584421873092651, + 0.9026989340782166, + -0.5741626024246216, + -0.5299967527389526, + -1.544856309890747, + 1.7236231565475464, + 2.1288580894470215, + -0.7592856287956238, + 0.3358893394470215, + -0.2929669916629791, + 1.67848801612854 + ], + [ + -0.3686429262161255, + 0.3043383061885834, + 0.7294905781745911, + -0.8736650347709656, + 0.3122289776802063, + -1.5051836967468262, + -1.8593363761901855, + 0.8389579653739929, + -0.8785254955291748, + -0.5520821213722229, + -1.8517990112304688, + 1.3417646884918213, + -0.32390326261520386, + -0.42145586013793945, + -1.6530441045761108, + 1.7202519178390503, + 2.180105209350586, + -0.4262692928314209, + -0.1161729246377945, + -0.08550512790679932, + 1.1130309104919434 + ], + [ + -0.9971213936805725, + -0.4085577130317688, + 0.9404422044754028, + -0.9977414011955261, + 0.4963066577911377, + -1.4445728063583374, + -1.8557989597320557, + 1.3813822269439697, + -0.9350934028625488, + 0.6622016429901123, + -2.0982701778411865, + 0.5712680220603943, + -1.3476169109344482, + -0.8056140542030334, + -2.2595674991607666, + 1.7601569890975952, + 2.5158329010009766, + -0.8607662916183472, + -0.010314732789993286, + -0.9802933931350708, + 0.587103545665741 + ], + [ + -0.538765549659729, + -0.2632336914539337, + 0.574018120765686, + -1.3315669298171997, + 0.23096206784248352, + -1.4029772281646729, + -1.2746275663375854, + 1.674896478652954, + -0.6276504993438721, + -0.06763898581266403, + -1.44939124584198, + 1.1256837844848633, + -0.6540690064430237, + -0.7850072979927063, + -2.472240686416626, + 1.201231837272644, + 2.056978702545166, + 0.4725829064846039, + 0.7320148944854736, + -0.47291314601898193, + 1.2244172096252441 + ], + [ + -0.5474913120269775, + 0.3537777066230774, + 0.1717698574066162, + -0.9464929103851318, + -0.24061335623264313, + -2.5145106315612793, + -1.9605224132537842, + 2.171875476837158, + 0.5462924242019653, + 0.2324821650981903, + -2.4616305828094482, + 1.3455290794372559, + -0.703314483165741, + -0.6289631128311157, + -1.848944902420044, + 1.755210518836975, + 2.649005174636841, + -0.931267499923706, + -0.3109248876571655, + 0.21283608675003052, + 0.6994417309761047 + ], + [ + -1.4273285865783691, + 0.48591291904449463, + 1.4724129438400269, + -1.1116547584533691, + -0.04224221408367157, + -1.1567587852478027, + -1.0686161518096924, + 1.7429871559143066, + -1.1814582347869873, + 0.07730802893638611, + -2.1004433631896973, + 0.45253968238830566, + -0.7293854355812073, + -0.4477439522743225, + -1.986203908920288, + 1.1158549785614014, + 2.0316240787506104, + 0.5763667821884155, + -0.48857587575912476, + -1.07709801197052, + 0.5611822605133057 + ] + ], + "predicted_scores": [ + 1.380915641784668, + 0.013786926865577698, + -0.30374717712402344, + -0.4565553069114685, + -0.6123114228248596, + -0.6671327352523804, + -1.0171722173690796, + -1.4761658906936646 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296", + "selected_index": 0, + "state_hash": "62e00728e448102460275bc2c35150dbf429ab0d5670a6706fa41256273bac3c", + "stored_base_utility": 0.05257302522659302, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20529556274414062, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20529556274414062 + }, + "base_progress": 0.20529556274414062, + "base_success": false, + "base_utility": 0.20529556274414062, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.14004097878932953, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14004097878932953 + }, + { + "contact_quality": null, + "progress": 0.1419321894645691, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1419321894645691 + }, + { + "contact_quality": null, + "progress": 0.12569941580295563, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12569941580295563 + }, + { + "contact_quality": null, + "progress": 0.19780242443084717, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19780242443084717 + }, + { + "contact_quality": null, + "progress": 0.19119806587696075, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19119806587696075 + }, + { + "contact_quality": null, + "progress": 0.09375361353158951, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09375361353158951 + }, + { + "contact_quality": null, + "progress": 0.07512229681015015, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07512229681015015 + }, + { + "contact_quality": null, + "progress": 0.05006565526127815, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05006565526127815 + } + ], + "candidate_progress": [ + 0.14004097878932953, + 0.1419321894645691, + 0.12569941580295563, + 0.19780242443084717, + 0.19119806587696075, + 0.09375361353158951, + 0.07512229681015015, + 0.05006565526127815 + ], + "candidate_source_chart_ids": [ + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-4bda8533405961380322", + "ms-6ddecc7f074b9e267409" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "generated_tangents": [ + [ + 0.10673574358224869, + -0.14959247410297394, + -0.12463840842247009, + -0.37258410453796387, + -0.05057141184806824, + -2.4426519870758057, + -0.9322618842124939, + -0.7199194431304932, + -0.041174888610839844, + 0.6258273720741272, + 0.6073430776596069, + -0.09425163269042969, + -0.4072318971157074, + 0.28364187479019165, + 0.015309281647205353, + -0.19833293557167053, + -0.35566574335098267, + 0.3385075330734253, + -0.6021514534950256, + -0.8059977889060974, + 0.08888347446918488 + ], + [ + 0.20093709230422974, + -0.08732785284519196, + -0.07632574439048767, + -0.3199525475502014, + -0.15303108096122742, + -2.4377336502075195, + -1.001490831375122, + -0.7903403043746948, + -0.04812772572040558, + 0.6852710247039795, + 0.5423687696456909, + -0.0874888002872467, + -0.49707430601119995, + 0.2025846242904663, + -0.013522166758775711, + -0.1688818633556366, + -0.32763415575027466, + 0.36862581968307495, + -0.6373065710067749, + -0.7562669515609741, + -0.18745648860931396 + ], + [ + 0.22022023797035217, + -0.06436137109994888, + -0.053831905126571655, + -0.14377784729003906, + 0.011305093765258789, + -2.408085346221924, + -0.7647324204444885, + -0.8291052579879761, + -0.10818691551685333, + 0.7213743329048157, + 0.6900634765625, + -0.2628686726093292, + -0.41571927070617676, + 0.12526550889015198, + -0.062162432819604874, + -0.13022896647453308, + -0.3306620121002197, + 0.28726160526275635, + -0.4696381092071533, + -0.6505988240242004, + -0.2831553518772125 + ], + [ + 0.1806195080280304, + -0.12329690158367157, + -0.2678345739841461, + -0.0991961658000946, + -0.11933040618896484, + -2.370558023452759, + -1.0490424633026123, + -0.7236278057098389, + -0.03582095354795456, + 0.7128385901451111, + 0.2698477506637573, + 0.031962111592292786, + -0.6503012776374817, + 0.1851949691772461, + 0.036946482956409454, + -0.09476342797279358, + -0.48858946561813354, + 0.23446667194366455, + -0.6592259407043457, + -1.1064987182617188, + -0.2510225772857666 + ], + [ + 0.4808974862098694, + -0.14640262722969055, + 0.30818068981170654, + -0.7484868764877319, + -0.08179956674575806, + -2.536876916885376, + -1.3846843242645264, + -0.3886636793613434, + -0.09180399030447006, + 0.3166411817073822, + 0.2011575996875763, + -0.1790960431098938, + -0.826603889465332, + 0.3920462727546692, + 0.03207596391439438, + -0.10321949422359467, + -0.39817750453948975, + 0.41908925771713257, + -0.7173893451690674, + -0.977794349193573, + -0.27252867817878723 + ], + [ + 0.2641885280609131, + -0.09089608490467072, + 0.36994171142578125, + -0.2175145447254181, + -0.31745749711990356, + -2.5401532649993896, + -0.7345007061958313, + -0.6528854370117188, + -0.023025771602988243, + 0.8325459957122803, + 0.5940406322479248, + 0.21302993595600128, + -0.2803461253643036, + -0.42173969745635986, + -0.2120581865310669, + -0.005919860675930977, + -0.2557860016822815, + 0.5652105808258057, + -0.603173553943634, + -0.7599316835403442, + -0.4067913591861725 + ], + [ + 0.2427782416343689, + -0.04562993347644806, + -0.22766199707984924, + 0.03796195983886719, + 0.3980022370815277, + -2.0137979984283447, + -0.9746316075325012, + -0.7912865877151489, + -0.218749538064003, + 0.814417839050293, + 0.8569574356079102, + 0.25053051114082336, + -0.2706298232078552, + 0.26941999793052673, + -0.021201658993959427, + -0.24368491768836975, + -0.09445913136005402, + 0.6419159770011902, + -0.6443300843238831, + -0.5719162225723267, + -0.09823915362358093 + ], + [ + 0.347103476524353, + -0.11219358444213867, + -0.23887714743614197, + -0.2464659959077835, + 0.11277837306261063, + -2.269803524017334, + -0.8880805373191833, + -1.2211774587631226, + 0.053966037929058075, + 1.1723911762237549, + 0.7669907808303833, + 0.04349309206008911, + -0.39687860012054443, + -0.07752034068107605, + -0.18468192219734192, + -0.1400359719991684, + -0.10476179420948029, + 0.2363874614238739, + -0.7008696794509888, + -0.6770836114883423, + 0.04924654960632324 + ] + ], + "generated_utilities": [ + 0.14004097878932953, + 0.1419321894645691, + 0.12569941580295563, + 0.19780242443084717, + 0.19119806587696075, + 0.09375361353158951, + 0.07512229681015015, + 0.05006565526127815 + ], + "group_id": "ms-7ea98b8f4d2220347ee0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18600693345069885, + 0.07475636899471283, + 0.27452313899993896, + 0.20920448005199432, + 0.18411576747894287, + 0.27989473938941956, + 0.19416475296020508, + 0.10948207229375839, + 0.123555026948452, + 0.1468026340007782, + 0.12693151831626892, + 0.11466284841299057, + 0.07585643231868744, + 0.12908481061458588, + 0.22501108050346375, + 0.06431269645690918 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.06974434852600098, + -0.02351950854063034, + 0.04567508399486542, + -0.07042121887207031, + -0.038941264152526855, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.08617860078811646, + 0.9112892150878906, + 0.7374391555786133, + 0.06539928913116455, + 0.03787863254547119, + 0.0, + -0.021528124809265137, + -0.04566385596990585, + -0.03331831097602844, + 1.6466763019561768, + 0.04444807767868042, + -0.27494049072265625, + 0.0 + ], + [ + 1.1160619258880615, + -0.10018755495548248, + 0.19327251613140106, + -1.3990073204040527, + 0.672177255153656, + -3.303851842880249, + -2.832740545272827, + -1.902563214302063, + 0.39503830671310425, + 1.542332410812378, + 1.6850805282592773, + 0.6842766404151917, + 1.1400834321975708, + 1.670598030090332, + 0.631821870803833, + -0.26601675152778625, + -0.2725638151168823, + 3.003178119659424, + 0.23009130358695984, + -1.0388810634613037, + -0.19119825959205627 + ], + [ + 1.031497836112976, + -0.2943698763847351, + -0.11846622079610825, + -1.3361037969589233, + 0.6225074529647827, + -3.545761823654175, + -2.5078811645507812, + -1.4024183750152588, + 0.6736462116241455, + 2.093475580215454, + 1.9168713092803955, + 0.6377919316291809, + 0.5105260014533997, + 1.8172619342803955, + 1.1300766468048096, + 0.1616329848766327, + 0.03408849239349365, + 2.2435190677642822, + 0.735376238822937, + -1.5760226249694824, + -0.7514586448669434 + ], + [ + 1.7542359828948975, + -0.7351009249687195, + -0.854544997215271, + -1.223644495010376, + 0.07327532768249512, + -3.150663137435913, + -2.5486767292022705, + -2.110039472579956, + 1.0206904411315918, + 2.1492340564727783, + 2.187993288040161, + 1.2547765970230103, + 0.8607165217399597, + 1.1150517463684082, + 0.9828771948814392, + 0.5030426383018494, + -0.4691219627857208, + 2.2485060691833496, + -0.14985057711601257, + -1.9480454921722412, + -0.05157876014709473 + ], + [ + 0.6040669679641724, + 0.5665496587753296, + 0.24913550913333893, + -1.6600215435028076, + -0.10813313722610474, + -3.8031911849975586, + -3.226392984390259, + -1.5098689794540405, + 1.0684192180633545, + 2.1557064056396484, + 1.545849084854126, + 1.011389970779419, + 1.113816499710083, + 0.9937302470207214, + 1.372757911682129, + -0.4055754244327545, + -0.23403912782669067, + 3.0650274753570557, + 0.1330145299434662, + -0.7907523512840271, + -0.6428619623184204 + ], + [ + 0.30788952112197876, + 0.6807127594947815, + -0.035938769578933716, + -0.8788743615150452, + -0.30191969871520996, + -4.111589431762695, + -2.432647705078125, + -1.7479791641235352, + -0.321926474571228, + 1.9823087453842163, + 1.6617395877838135, + 1.3319180011749268, + 1.5937504768371582, + 2.0502309799194336, + 0.63725745677948, + 0.7232187390327454, + -0.23885302245616913, + 1.8472312688827515, + 0.3300834596157074, + -0.43588966131210327, + -1.2869892120361328 + ], + [ + 0.3636443018913269, + -0.4284004867076874, + 0.2675324082374573, + -0.4861376881599426, + 0.2239738255739212, + -2.813422918319702, + -2.4891929626464844, + -1.165071725845337, + 0.6442201733589172, + 1.1830427646636963, + 1.177597999572754, + 0.33577069640159607, + 0.6362329721450806, + 1.5409071445465088, + 0.29718804359436035, + -0.5603961944580078, + -0.8089720010757446, + 2.6271419525146484, + -0.5456377267837524, + -0.2824941873550415, + -0.7960987091064453 + ], + [ + 1.1001086235046387, + -0.9762114882469177, + -0.00277864933013916, + -1.704659104347229, + 1.2373628616333008, + -3.385840654373169, + -1.6910454034805298, + -1.470743179321289, + 0.9354362487792969, + 1.9378679990768433, + 1.515415906906128, + -0.3002503514289856, + 0.21796274185180664, + 1.542279839515686, + 1.7353785037994385, + 0.23501873016357422, + -0.16601301729679108, + 2.1989498138427734, + 0.8671687841415405, + -1.6252647638320923, + -1.2141560316085815 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06974434852600098, + -0.15391214191913605, + -0.22159501910209656, + -0.07042121887207031, + -0.09706336259841919, + -2.2828922271728516, + 0.0, + -0.8026905059814453, + 0.1400253176689148, + 0.9112892150878906, + 0.7374391555786133, + 0.13415807485580444, + -0.10778343677520752, + 0.0, + -0.08470147848129272, + 0.1231045201420784, + 0.08352774381637573, + 1.6466763019561768, + -0.005736887454986572, + -0.27494049072265625, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -5.125826835632324, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.152101516723633, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.8942345976829529 + ] + ], + "predicted_scores": [ + 5.43532133102417, + 5.334425449371338, + 5.021456718444824, + 5.008562088012695, + 4.660686016082764, + 4.276692867279053, + 3.5957794189453125, + 3.458228588104248 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "a2b9f3b3ea6458c6734081cea861cf54026796a0a594bf9be3657d28f12e9b3c", + "stored_base_utility": 0.18600693345069885, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18213583528995514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18213583528995514 + }, + "base_progress": 0.18213583528995514, + "base_success": false, + "base_utility": 0.18213583528995514, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.002675553085282445, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.002675553085282445 + }, + { + "contact_quality": null, + "progress": 0.0032987594604492188, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0032987594604492188 + }, + { + "contact_quality": null, + "progress": 0.0033065874595195055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0033065874595195055 + }, + { + "contact_quality": null, + "progress": 0.0031580727081745863, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0031580727081745863 + }, + { + "contact_quality": null, + "progress": 0.00375221180729568, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00375221180729568 + }, + { + "contact_quality": null, + "progress": 0.004366755485534668, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004366755485534668 + }, + { + "contact_quality": null, + "progress": 0.004318694118410349, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004318694118410349 + }, + { + "contact_quality": null, + "progress": 0.0034571688156574965, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0034571688156574965 + } + ], + "candidate_progress": [ + 0.002675553085282445, + 0.0032987594604492188, + 0.0033065874595195055, + 0.0031580727081745863, + 0.00375221180729568, + 0.004366755485534668, + 0.004318694118410349, + 0.0034571688156574965 + ], + "candidate_source_chart_ids": [ + "ms-8e12f7ad446482627dd3", + "ms-f2b2b2a1aa2863c3676f", + "ms-f2b2b2a1aa2863c3676f", + "ms-02311c7783bc67e188b4", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3", + "ms-8e12f7ad446482627dd3" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-7f57045c114ddad153b5", + "generated_tangents": [ + [ + -0.39032959938049316, + -0.508104145526886, + 1.5406129360198975, + -0.4155612587928772, + 1.9475791454315186, + 3.158402681350708, + 1.9311771392822266, + 0.34415990114212036, + 0.07478859275579453, + 0.9082386493682861, + 0.5083922743797302, + 1.3319385051727295, + 1.284725546836853, + 2.5255703926086426, + -0.13781119883060455, + -0.1745506078004837, + 1.0435887575149536, + -0.3424244523048401, + 1.1724116802215576, + 1.4357433319091797, + 1.6907014846801758 + ], + [ + 0.1280473917722702, + -0.5491660833358765, + 1.4465504884719849, + -0.41509976983070374, + 1.7438325881958008, + 3.0281577110290527, + 1.7826979160308838, + 0.4565412700176239, + 0.16918210685253143, + 0.9650174379348755, + 0.3661339282989502, + 1.0957837104797363, + 1.8078123331069946, + 2.0525758266448975, + -0.269488662481308, + -0.16108833253383636, + 1.2963379621505737, + -0.1321779042482376, + 1.051758885383606, + 1.5588188171386719, + 1.7583426237106323 + ], + [ + 0.08456126600503922, + -0.5221831798553467, + 1.2618119716644287, + -0.40739238262176514, + 1.727027416229248, + 3.1535515785217285, + 1.8114442825317383, + 0.527355432510376, + 0.1626051664352417, + 1.0738208293914795, + 0.546379566192627, + 1.0603299140930176, + 1.7653990983963013, + 1.6606471538543701, + -0.12099452316761017, + -0.21441476047039032, + 1.4798492193222046, + -0.15774168074131012, + 1.2178139686584473, + 1.6470692157745361, + 1.750726342201233 + ], + [ + -0.06606529653072357, + -0.6003207564353943, + 1.4119372367858887, + -0.36464112997055054, + 1.7714884281158447, + 2.7030832767486572, + 1.7662653923034668, + 0.4509817659854889, + 0.10357790440320969, + 0.8971320986747742, + 0.442639023065567, + 1.3400689363479614, + 1.4598724842071533, + 2.1251139640808105, + -0.09128360450267792, + 0.008713070303201675, + 1.4488658905029297, + -0.06104664504528046, + 1.2843379974365234, + 1.388801097869873, + 2.0064051151275635 + ], + [ + -0.12615616619586945, + -0.6678372621536255, + 1.198927879333496, + -0.37412479519844055, + 1.5052831172943115, + 2.952237129211426, + 1.7415893077850342, + 0.46397364139556885, + 0.11576055735349655, + 0.9424229264259338, + 0.23245862126350403, + 1.0981528759002686, + 1.8414663076400757, + 1.9273345470428467, + -0.24184110760688782, + -0.032975923269987106, + 1.1227922439575195, + -0.34808146953582764, + 1.25648832321167, + 1.3888342380523682, + 1.6293386220932007 + ], + [ + 0.1342376172542572, + -0.48477381467819214, + 1.4279762506484985, + -0.3248389661312103, + 1.0237741470336914, + 2.874401330947876, + 1.9266736507415771, + 0.7250481843948364, + 0.20822665095329285, + 0.9497660398483276, + 0.49012768268585205, + 0.8773930668830872, + 2.1183817386627197, + 1.8961174488067627, + -0.31126612424850464, + -0.08741752803325653, + 1.3766226768493652, + -0.1429089456796646, + 1.5382955074310303, + 1.7073025703430176, + 0.9260591268539429 + ], + [ + -0.14159391820430756, + -0.6119739413261414, + 1.0168184041976929, + -0.47098153829574585, + 1.4086096286773682, + 3.124621629714966, + 1.6730246543884277, + 0.46711400151252747, + 0.06346733868122101, + 0.9438840746879578, + 0.0648728460073471, + 1.0331757068634033, + 1.8586443662643433, + 1.8924436569213867, + -0.2553775906562805, + -0.02012600749731064, + 1.0731415748596191, + -0.23178528249263763, + 1.2510342597961426, + 1.3793447017669678, + 1.6255412101745605 + ], + [ + -0.17344127595424652, + -0.5800595879554749, + 1.0790690183639526, + -0.2571268081665039, + 1.5217797756195068, + 2.6276838779449463, + 1.5147156715393066, + 0.3386792242527008, + 0.056403473019599915, + 0.9924086928367615, + 0.21902933716773987, + 0.8074168562889099, + 1.6257332563400269, + 1.9361677169799805, + -0.17142485082149506, + -0.02326297201216221, + 1.1998100280761719, + 0.017527207732200623, + 1.040473461151123, + 1.4633429050445557, + 1.76557457447052 + ] + ], + "generated_utilities": [ + 0.002675553085282445, + 0.0032987594604492188, + 0.0033065874595195055, + 0.0031580727081745863, + 0.00375221180729568, + 0.004366755485534668, + 0.004318694118410349, + 0.0034571688156574965 + ], + "group_id": "ms-7f57045c114ddad153b5", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.11277659982442856, + 0.1479199379682541, + 0.07541074603796005, + 0.042439721524715424, + 0.15517297387123108, + 0.03584214299917221, + 0.1660110056400299, + 0.12549421191215515, + 0.227622389793396, + 0.09924932569265366, + 0.18198737502098083, + 0.11130931228399277, + 0.10854844748973846, + 0.01617596670985222, + 0.08437560498714447, + 0.04182587191462517 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.013166621327400208, + -0.12399804592132568, + 0.48011910915374756, + 0.4925293028354645, + 0.6256281137466431, + 2.213102102279663, + 0.0, + -0.4044480323791504, + 0.1722363382577896, + 0.3389366865158081, + -0.08761294186115265, + 0.21791136264801025, + 0.9396321773529053, + 0.0, + 0.06717276573181152, + -0.5456919074058533, + 0.722014307975769, + -0.051606684923172, + 0.7784175276756287, + 0.5338386297225952, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.9143741130828857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.455221176147461, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.263336420059204 + ], + [ + -0.47406601905822754, + -0.37954479455947876, + 1.123671293258667, + -0.8903511762619019, + 1.5515481233596802, + 3.7488584518432617, + 0.9699987173080444, + -0.23581115901470184, + -0.2149372398853302, + 0.8883978128433228, + 0.5803861021995544, + 1.7047908306121826, + 2.149672746658325, + 2.2741973400115967, + 0.2513730227947235, + 0.6982267498970032, + 2.1354174613952637, + -1.151021122932434, + 1.9550246000289917, + 0.5981679558753967, + 1.338004469871521 + ], + [ + -0.1836935132741928, + -0.42300617694854736, + 1.7664438486099243, + -0.5576877593994141, + 1.0626983642578125, + 2.5276265144348145, + 2.3329849243164062, + -0.4834023118019104, + -0.25722599029541016, + 0.5209662914276123, + 0.24240870773792267, + 0.8533134460449219, + 1.5531291961669922, + 2.304234504699707, + 0.7609332799911499, + 1.0344312191009521, + 2.270921230316162, + -0.926997184753418, + 0.9014018177986145, + 2.259265422821045, + 2.172292470932007 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3910996615886688, + -0.9467402100563049, + 1.4744372367858887, + -0.2748112082481384, + 1.753147840499878, + 3.3321292400360107, + 1.7170560359954834, + 0.09920699894428253, + -0.11250317096710205, + 1.5213767290115356, + 0.0891318991780281, + 1.0193004608154297, + 1.7396382093429565, + 2.202378988265991, + 0.1176290512084961, + 0.5433868765830994, + 1.6113250255584717, + -0.2701912522315979, + 1.7789868116378784, + 1.246854305267334, + 1.6763029098510742 + ], + [ + 0.3669140636920929, + -0.8853410482406616, + 1.1054010391235352, + -0.008406088687479496, + 1.683157205581665, + 2.8090970516204834, + 2.1524839401245117, + -0.1140800267457962, + -0.16068093478679657, + 1.3338439464569092, + -0.34659242630004883, + 1.4494925737380981, + 2.0485284328460693, + 2.244007110595703, + 0.4862309694290161, + 0.6211890578269958, + 1.5708913803100586, + -0.22552455961704254, + 1.7964838743209839, + 1.386763095855713, + 1.3543281555175781 + ], + [ + 0.7718826532363892, + -0.5710152983665466, + 2.151604175567627, + -0.5648711323738098, + 2.1273765563964844, + 3.657623291015625, + 1.9845168590545654, + -0.1599704772233963, + -0.6097310781478882, + 1.135812520980835, + -0.09587362408638, + 1.555975079536438, + 2.3309824466705322, + 1.9832960367202759, + 0.46482378244400024, + 0.6467846035957336, + 1.7182108163833618, + -0.0750286877155304, + 1.9485927820205688, + 0.959426760673523, + 0.9474185109138489 + ] + ], + "predicted_scores": [ + 4.125182151794434, + 3.808912754058838, + 3.6503946781158447, + 3.3113675117492676, + 3.2645554542541504, + 3.13165020942688, + 2.9304094314575195, + 2.3975682258605957 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524", + "selected_index": 0, + "state_hash": "4885f9acf2601b3d2c201bc8e00ef617845ad2a6d08f96d585249345f457be89", + "stored_base_utility": 0.11277659982442856, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0056487321853637695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056487321853637695 + }, + "base_progress": 0.0056487321853637695, + "base_success": false, + "base_utility": 0.0056487321853637695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.011842290870845318, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011842290870845318 + }, + { + "contact_quality": null, + "progress": 0.01331732701510191, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01331732701510191 + }, + { + "contact_quality": null, + "progress": 0.007131179329007864, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007131179329007864 + }, + { + "contact_quality": null, + "progress": 0.007257858756929636, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007257858756929636 + }, + { + "contact_quality": null, + "progress": 0.00787546206265688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00787546206265688 + }, + { + "contact_quality": null, + "progress": 0.008778492920100689, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008778492920100689 + }, + { + "contact_quality": null, + "progress": 0.004853228572756052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.004853228572756052 + }, + { + "contact_quality": null, + "progress": 0.003935039043426514, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.003935039043426514 + } + ], + "candidate_progress": [ + 0.011842290870845318, + 0.01331732701510191, + 0.007131179329007864, + 0.007257858756929636, + 0.00787546206265688, + 0.008778492920100689, + 0.004853228572756052, + 0.003935039043426514 + ], + "candidate_source_chart_ids": [ + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-0b93907b2a7261815f63", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895", + "ms-3a4e2d080c9c4b279895" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "generated_tangents": [ + [ + -0.12132076919078827, + -0.2878642678260803, + 1.0866169929504395, + -0.4333801865577698, + 1.3068206310272217, + 2.5597479343414307, + 1.9493379592895508, + -0.0816757082939148, + -0.505718469619751, + 0.20882830023765564, + -0.3755462169647217, + -0.05416959524154663, + 1.8078550100326538, + 0.6881452798843384, + 1.1576682329177856, + -0.7253053188323975, + -1.2206331491470337, + 1.11386239528656, + 0.8785176873207092, + -0.0019288063049316406, + -1.526033639907837 + ], + [ + -0.6971748471260071, + -0.16849307715892792, + 0.8937058448791504, + -0.6516115665435791, + 1.1871507167816162, + 3.069403648376465, + 2.0086045265197754, + 0.004958391189575195, + -0.38325098156929016, + -0.06779725104570389, + -0.28374800086021423, + 0.0900740697979927, + 1.780896544456482, + 0.854914665222168, + 1.1015669107437134, + -0.7410902380943298, + -1.6288267374038696, + 0.8072232007980347, + 1.0519955158233643, + -0.6125564575195312, + -0.911697506904602 + ], + [ + -0.3046749234199524, + -0.40183591842651367, + 0.9257275462150574, + -0.8784635066986084, + 1.2770981788635254, + 3.3158912658691406, + 2.2451343536376953, + 0.3959258794784546, + -0.26736778020858765, + 0.06612849235534668, + -0.3355943560600281, + 0.23737362027168274, + 1.8543399572372437, + 0.6302992105484009, + 1.222571849822998, + -0.7062807083129883, + -1.4489601850509644, + 0.8391095399856567, + 0.8286699652671814, + -0.21063582599163055, + -0.9838241338729858 + ], + [ + -0.48727017641067505, + -0.27816566824913025, + 0.897692859172821, + -0.6824548840522766, + 1.1272952556610107, + 3.0164794921875, + 2.324998378753662, + 0.1511741429567337, + -0.2782328724861145, + 0.03229042887687683, + -0.5024949312210083, + 0.5358070135116577, + 1.8460942506790161, + 0.8555551767349243, + 1.201119303703308, + -0.7147851586341858, + -1.1396129131317139, + 0.7666560411453247, + 0.9762827157974243, + -0.0225905179977417, + -1.1771522760391235 + ], + [ + -0.6487288475036621, + -0.2438424974679947, + 1.4727966785430908, + -0.48905354738235474, + 0.5148495435714722, + 2.9174039363861084, + 1.5262482166290283, + 0.10794110596179962, + -0.351114958524704, + 0.47189971804618835, + -0.6558436155319214, + 0.661037027835846, + 1.795812964439392, + 0.45406192541122437, + 0.8320794701576233, + -0.7480214834213257, + -1.1061779260635376, + 0.7629175186157227, + 0.5486004948616028, + -0.443786084651947, + -0.8721569776535034 + ], + [ + -0.6348989605903625, + -0.21784470975399017, + 1.2891793251037598, + -0.4872874319553375, + 0.6194880604743958, + 2.8650362491607666, + 1.8282687664031982, + 0.12518739700317383, + -0.4802975058555603, + 0.18316322565078735, + -0.2056957185268402, + 0.45549896359443665, + 1.0011792182922363, + 0.20390743017196655, + 0.8425129055976868, + -0.8140647411346436, + -0.958462655544281, + 0.7885424494743347, + 0.5988937616348267, + -0.2641720473766327, + -0.8488433361053467 + ], + [ + -0.5075993537902832, + -0.20654913783073425, + 1.2304483652114868, + -0.4762396812438965, + 1.3018770217895508, + 3.0701406002044678, + 2.494612455368042, + 0.3038874864578247, + -0.5493683815002441, + 0.1495422124862671, + -0.33732831478118896, + 0.7177326679229736, + 0.8174186944961548, + 0.5500130653381348, + 1.3581563234329224, + -0.7364436984062195, + -0.8602816462516785, + 0.9882403016090393, + 0.9251771569252014, + -0.3310457170009613, + -1.1166590452194214 + ], + [ + -0.464261531829834, + -0.23492270708084106, + 1.3735148906707764, + -0.43437132239341736, + 1.0023789405822754, + 2.9266200065612793, + 2.208437442779541, + 0.33880719542503357, + -0.5210384130477905, + 0.4060251712799072, + -0.4580993950366974, + 0.5641075372695923, + 1.1459509134292603, + 0.366401731967926, + 1.2579251527786255, + -0.6666497588157654, + -0.7129340171813965, + 0.9066519141197205, + 0.9991755485534668, + -0.46294230222702026, + -0.8339345455169678 + ] + ], + "generated_utilities": [ + 0.011842290870845318, + 0.01331732701510191, + 0.007131179329007864, + 0.007257858756929636, + 0.00787546206265688, + 0.008778492920100689, + 0.004853228572756052, + 0.003935039043426514 + ], + "group_id": "ms-81f4e8858b2ba90aea2e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.01361372135579586, + 0.007307033054530621, + 0.021326780319213867, + 0.04093001410365105, + 0.15059307217597961, + 0.013598322868347168, + 0.09240063279867172, + 0.1631626933813095, + 0.05027441307902336, + 0.05712141841650009, + 0.1538398563861847, + 0.06023097038269043, + 0.020811915397644043, + 0.019836824387311935, + 0.1318017691373825, + 0.051341354846954346 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06500570476055145, + 0.046472612768411636, + 1.2439939975738525, + -1.446747064590454, + 1.7077040672302246, + 2.291205883026123, + 2.1016674041748047, + -0.6354803442955017, + -0.4465000331401825, + 0.9666666984558105, + -1.1514010429382324, + 0.45639365911483765, + 1.613925814628601, + 1.0568119287490845, + 1.3792463541030884, + -0.7852597832679749, + -1.576931118965149, + 1.1258286237716675, + 0.9614027738571167, + -0.9620904922485352, + -1.7528356313705444 + ], + [ + -0.23400194942951202, + -0.21453839540481567, + 1.458961844444275, + -1.701543927192688, + 1.4158955812454224, + 2.2648162841796875, + 2.2270467281341553, + -0.5239131450653076, + -0.44378146529197693, + 1.0904332399368286, + -1.223331093788147, + 0.45784294605255127, + 1.6057316064834595, + 0.9251953363418579, + 1.308751106262207, + -0.6351475715637207, + -1.5950077772140503, + 1.0428481101989746, + 0.6781296730041504, + -0.9007717370986938, + -1.880659818649292 + ], + [ + 0.2324782907962799, + 0.05129373073577881, + 1.1239540576934814, + -1.1853233575820923, + 1.3290393352508545, + 1.933064341545105, + 2.192976951599121, + -0.23718085885047913, + -0.279358446598053, + 1.3028839826583862, + -1.4676742553710938, + 0.16475489735603333, + 1.67984938621521, + 0.6600161790847778, + 1.5654886960983276, + -1.1545368432998657, + -1.5593464374542236, + 1.1976646184921265, + 0.5715436935424805, + -0.9047394394874573, + -1.9614591598510742 + ], + [ + 0.7452290058135986, + 0.614987313747406, + 0.7383754849433899, + -1.5856380462646484, + 1.264894723892212, + 1.7337512969970703, + 2.502504825592041, + 0.040616750717163086, + -1.1382783651351929, + 0.7095704674720764, + -1.014717698097229, + 0.5126742720603943, + 1.3401073217391968, + 1.4275505542755127, + 1.4739655256271362, + -0.7308655381202698, + -0.9416123032569885, + 0.661049485206604, + 0.9453439712524414, + -1.5968079566955566, + -1.715583324432373 + ], + [ + 0.15219944715499878, + 0.5165310502052307, + 1.0654370784759521, + -0.5756344199180603, + 1.2834129333496094, + 1.6370232105255127, + 2.414426326751709, + -1.1256691217422485, + -1.1171015501022339, + 0.5337101817131042, + -0.9850769639015198, + -0.35428309440612793, + 2.4350547790527344, + 1.8563504219055176, + 2.1929399967193604, + -0.1821107268333435, + -1.4600040912628174, + 1.5430631637573242, + 1.4937412738800049, + -0.8152608275413513, + -2.020042896270752 + ] + ], + "predicted_scores": [ + 10.417702674865723, + 10.037516593933105, + 8.708206176757812, + 8.181203842163086, + 7.666223526000977, + 7.613871097564697, + 6.922053813934326, + 6.808025360107422 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "562e8bc3804d1d310e0eaf612dbc110b8efc94514458dacf551ef33f4f0b8ee7", + "stored_base_utility": 0.01361372135579586, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.00698717450723052, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.00698717450723052 + }, + "base_progress": 0.00698717450723052, + "base_success": false, + "base_utility": 0.00698717450723052, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.029071787372231483, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029071787372231483 + }, + { + "contact_quality": null, + "progress": 0.06671091169118881, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06671091169118881 + }, + { + "contact_quality": null, + "progress": 0.005293230060487986, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.005293230060487986 + }, + { + "contact_quality": null, + "progress": 0.02160499431192875, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02160499431192875 + }, + { + "contact_quality": null, + "progress": 0.044261831790208817, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044261831790208817 + }, + { + "contact_quality": null, + "progress": 0.07134183496236801, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07134183496236801 + }, + { + "contact_quality": null, + "progress": 0.06265010684728622, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06265010684728622 + }, + { + "contact_quality": null, + "progress": 0.016331294551491737, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016331294551491737 + } + ], + "candidate_progress": [ + 0.029071787372231483, + 0.06671091169118881, + 0.005293230060487986, + 0.02160499431192875, + 0.044261831790208817, + 0.07134183496236801, + 0.06265010684728622, + 0.016331294551491737 + ], + "candidate_source_chart_ids": [ + "ms-9229ce141dd41a11fac1", + "ms-45fbd749d5ed3fe37c13", + "ms-9229ce141dd41a11fac1", + "ms-45fbd749d5ed3fe37c13", + "ms-45fbd749d5ed3fe37c13", + "ms-48178f52a96de3d1ead0", + "ms-45fbd749d5ed3fe37c13", + "ms-25ab20bb3da8ba34be95" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-8a72b73ac452bc190aeb", + "generated_tangents": [ + [ + 0.38898879289627075, + 0.048232778906822205, + -0.27594080567359924, + -0.4399867355823517, + 0.6052330732345581, + -2.289738893508911, + -0.8728967308998108, + 0.0841187983751297, + -0.12292179465293884, + 0.04051147401332855, + 0.9417217969894409, + 0.871377170085907, + -0.17005059123039246, + 0.38461577892303467, + 0.03485450893640518, + -0.18288835883140564, + 0.25484609603881836, + 0.3678521513938904, + 0.39994749426841736, + -0.0563485324382782, + 0.022962406277656555 + ], + [ + 0.42928653955459595, + 0.022410452365875244, + -0.38184383511543274, + -0.5375196933746338, + 0.31605851650238037, + -2.4681060314178467, + -0.7639077305793762, + 0.5468398332595825, + 0.012673158198595047, + -0.02565433830022812, + 0.9478268623352051, + 0.5630228519439697, + -0.7646524310112, + 0.34908342361450195, + -0.04627295210957527, + -0.35641101002693176, + 0.18813484907150269, + 0.4935140013694763, + 0.5122667551040649, + -0.23788228631019592, + 0.010100230574607849 + ], + [ + 0.23712769150733948, + 0.13500076532363892, + -0.36908820271492004, + -0.37255918979644775, + 0.5231672525405884, + -2.231074333190918, + -1.03863525390625, + 0.08374889194965363, + -0.05988902598619461, + 0.125088170170784, + 0.958392858505249, + 0.7723782062530518, + -0.29773563146591187, + 0.45239418745040894, + 0.08420394361019135, + -0.192107155919075, + 0.22629736363887787, + 0.4328534007072449, + 0.43861401081085205, + -0.14866802096366882, + 0.017510607838630676 + ], + [ + 0.4899980425834656, + 0.04783404618501663, + -0.45972272753715515, + -0.5446975231170654, + 0.25615087151527405, + -2.509995937347412, + -0.6396778225898743, + 0.4406871199607849, + 0.014913775026798248, + -0.009976811707019806, + 0.9618518948554993, + 0.5958398580551147, + -0.7680066823959351, + 0.21110044419765472, + -0.03699727728962898, + -0.366599440574646, + 0.2802734076976776, + 0.4948219656944275, + 0.4585062861442566, + -0.06579568982124329, + -0.020760580897331238 + ], + [ + 0.22369495034217834, + -0.06301255524158478, + 0.12384846806526184, + -0.8326464891433716, + 0.6406474113464355, + -2.646409273147583, + -1.1933045387268066, + 0.7598435878753662, + -0.022133609279990196, + -0.4027285575866699, + 0.6744118332862854, + 0.8667235374450684, + -1.349009394645691, + -0.4615873694419861, + -0.05838386341929436, + -0.1220080703496933, + 0.02892071008682251, + 0.3502132296562195, + 0.8222885131835938, + -0.5224537253379822, + -0.46730276942253113 + ], + [ + 0.47789353132247925, + 0.014254044741392136, + -0.3995330035686493, + -0.8906805515289307, + 0.41174107789993286, + -1.9399954080581665, + -0.6106557250022888, + 0.6918694972991943, + -0.14082100987434387, + -0.171038419008255, + 0.396634042263031, + 0.2174096703529358, + -1.219422459602356, + -0.17239926755428314, + -0.19276103377342224, + -0.0120627973228693, + 0.08096519112586975, + 0.29732704162597656, + 0.2129671275615692, + -0.7286126017570496, + -0.6878169775009155 + ], + [ + 0.48613131046295166, + 0.01594846323132515, + -0.3296055495738983, + -0.5491228103637695, + 0.2825898230075836, + -2.540189743041992, + -0.676031768321991, + 0.5068598389625549, + 0.03761497884988785, + -0.016039960086345673, + 0.9585782885551453, + 0.5539495944976807, + -0.7344855070114136, + 0.32664743065834045, + -0.007775496691465378, + -0.35730859637260437, + 0.2810666859149933, + 0.38950514793395996, + 0.45345014333724976, + -0.18090961873531342, + -0.1197081208229065 + ], + [ + 0.328957736492157, + 0.08033671975135803, + -0.1528473198413849, + -0.5800721645355225, + 0.4263259768486023, + -2.3949427604675293, + -1.0099446773529053, + 0.06037957966327667, + -0.16935035586357117, + 0.20834535360336304, + 0.8870474100112915, + 0.9208224415779114, + -0.3436633050441742, + -0.107276052236557, + -0.22534698247909546, + -0.29083573818206787, + 0.33226278424263, + 0.5033165812492371, + 0.3189806640148163, + -0.23628006875514984, + 0.1325584203004837 + ] + ], + "generated_utilities": [ + 0.029071787372231483, + 0.06671091169118881, + 0.005293230060487986, + 0.02160499431192875, + 0.044261831790208817, + 0.07134183496236801, + 0.06265010684728622, + 0.016331294551491737 + ], + "group_id": "ms-8a72b73ac452bc190aeb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.02298194169998169, + 0.01763375662267208, + 0.009361605159938335, + 0.02526181936264038, + 0.09214292466640472, + 0.022978028282523155, + 0.07704488933086395, + 0.056830525398254395, + 0.045355141162872314, + 0.06095697730779648, + 0.10329639911651611, + 0.05954315513372421, + 0.03003152273595333, + 0.008136491291224957, + 0.05599620193243027, + 0.011116823181509972 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.0913797616958618, + 0.3305599093437195, + -0.0778055489063263, + -2.223818778991699, + 0.9291321635246277, + -2.905466318130493, + -2.4575929641723633, + 1.1732350587844849, + 0.22933529317378998, + -0.8654288053512573, + 2.1122353076934814, + 1.5987088680267334, + 1.144788146018982, + 0.40604642033576965, + 0.10343833267688751, + -0.7513935565948486, + -1.054006814956665, + 1.396103024482727, + -0.16642586886882782, + -0.04196041822433472, + -1.2630513906478882 + ], + [ + 1.3690472841262817, + 0.2738865315914154, + -0.27599766850471497, + -2.431976556777954, + 0.7025187015533447, + -3.155372142791748, + -2.6898910999298096, + 1.1788215637207031, + 0.19074484705924988, + -0.7378551959991455, + 2.3089826107025146, + 1.3478807210922241, + 1.3458092212677002, + 0.3417958617210388, + 0.24155153334140778, + -1.004831314086914, + -1.3499912023544312, + 1.311987042427063, + -0.25646817684173584, + -0.3180008828639984, + -1.3290650844573975 + ], + [ + 0.9032705426216125, + -0.3011789321899414, + -0.21911907196044922, + -2.884286403656006, + 0.319735586643219, + -2.613290548324585, + -2.175532817840576, + 1.5185720920562744, + -0.1741684526205063, + -0.4879068434238434, + 2.721827507019043, + 2.1753995418548584, + 0.8214534521102905, + 0.7874165177345276, + 0.013686276972293854, + -0.7485443949699402, + -0.764183521270752, + 0.9452257752418518, + 0.07210072875022888, + 0.2788098156452179, + -1.8349425792694092 + ] + ], + "predicted_scores": [ + 3.9230997562408447, + 3.7890663146972656, + 3.765925645828247, + 3.3247056007385254, + 2.9679667949676514, + 2.929856061935425, + 2.90167498588562, + 2.408285140991211 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "722240dc5e14c5a30badc5fbec9f15ecf388cf6402fb77013c8f9dbdcd100846", + "stored_base_utility": 0.02298194169998169, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.08285123109817505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08285123109817505 + }, + "base_progress": 0.08285123109817505, + "base_success": false, + "base_utility": 0.08285123109817505, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.035203736275434494, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.035203736275434494 + }, + { + "contact_quality": null, + "progress": 0.032373785972595215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.032373785972595215 + }, + { + "contact_quality": null, + "progress": 0.05178316310048103, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05178316310048103 + }, + { + "contact_quality": null, + "progress": 0.0497889518737793, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0497889518737793 + }, + { + "contact_quality": null, + "progress": 0.05319662764668465, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05319662764668465 + }, + { + "contact_quality": null, + "progress": 0.07769393920898438, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07769393920898438 + }, + { + "contact_quality": null, + "progress": 0.03644818067550659, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03644818067550659 + }, + { + "contact_quality": null, + "progress": 0.06582687050104141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06582687050104141 + } + ], + "candidate_progress": [ + 0.035203736275434494, + 0.032373785972595215, + 0.05178316310048103, + 0.0497889518737793, + 0.05319662764668465, + 0.07769393920898438, + 0.03644818067550659, + 0.06582687050104141 + ], + "candidate_source_chart_ids": [ + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186", + "ms-20ca52baa214c31b3186" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-a96bca1ef0b74f686018", + "generated_tangents": [ + [ + 0.24377943575382233, + 0.32708078622817993, + 0.911333441734314, + -0.5903794765472412, + 1.3735504150390625, + 0.2285427451133728, + 0.3272901475429535, + 0.21760070323944092, + 0.19221735000610352, + 0.417921781539917, + -1.0585862398147583, + 0.4535292685031891, + -1.0655913352966309, + -0.31938672065734863, + 0.24255125224590302, + 0.28544744849205017, + 0.2651522159576416, + -0.6137786507606506, + -1.0221281051635742, + -1.125736951828003, + -0.4783599078655243 + ], + [ + 0.11985970288515091, + 0.3361794352531433, + 0.6141902804374695, + -0.5244864821434021, + 0.7764644622802734, + 0.04178490489721298, + 0.6964371204376221, + 0.22153443098068237, + 0.07246668636798859, + 0.5326949954032898, + -0.9732793569564819, + 0.36889153718948364, + -1.846948504447937, + -0.19115303456783295, + 0.4001021683216095, + 0.39801299571990967, + 0.30469444394111633, + -0.6587927937507629, + -1.259711503982544, + -1.1519087553024292, + -0.7216782569885254 + ], + [ + 0.09906940907239914, + 0.3210197687149048, + 0.8100796937942505, + -0.6630990505218506, + 1.3996472358703613, + 0.08193384110927582, + 0.5416862964630127, + 0.13100376725196838, + 0.24795670807361603, + 0.25795668363571167, + -1.0583336353302002, + 0.36826959252357483, + -1.3281937837600708, + -0.1057974249124527, + 0.15827549993991852, + 0.289165735244751, + 0.2950815260410309, + -0.49331915378570557, + -0.9946284890174866, + -1.0162885189056396, + -0.2692895531654358 + ], + [ + 0.046681515872478485, + 0.26552754640579224, + 0.8814612627029419, + -0.6127615571022034, + 1.3233678340911865, + 0.13734042644500732, + 0.3281581401824951, + 0.07143212109804153, + 0.2597976624965668, + 0.28958308696746826, + -0.9637115001678467, + 0.5178719162940979, + -1.0891590118408203, + -0.3096383213996887, + 0.16210781037807465, + 0.30034083127975464, + 0.2671850919723511, + -0.505286693572998, + -1.0878467559814453, + -1.2061097621917725, + -0.2321859896183014 + ], + [ + 0.036467716097831726, + 0.2845940589904785, + 0.42028266191482544, + -0.7000330090522766, + 1.2009313106536865, + 0.4338286519050598, + 0.6932967901229858, + 0.32800519466400146, + 0.2756168246269226, + 0.1253436654806137, + -1.1826969385147095, + 0.3525141477584839, + -1.5950883626937866, + -0.22825323045253754, + 0.2078842669725418, + 0.38955697417259216, + 0.3499182462692261, + -0.3336631655693054, + -0.959294855594635, + -1.1037397384643555, + -0.7263059616088867 + ], + [ + 0.3529552221298218, + 0.3237884044647217, + 0.9772035479545593, + -0.40606868267059326, + 0.7661637663841248, + -0.01265116035938263, + 0.10729837417602539, + 0.18035152554512024, + 0.2199239432811737, + 0.26751601696014404, + -0.8885954022407532, + 0.49449723958969116, + -0.48741045594215393, + 0.284982293844223, + -0.113868847489357, + 0.32704752683639526, + -0.24657611548900604, + -0.40503430366516113, + -0.8470893502235413, + -0.9366427063941956, + -0.9513434171676636 + ], + [ + 0.13152211904525757, + 0.25801020860671997, + 0.9706121683120728, + -0.727902889251709, + 1.9322187900543213, + 0.5594120025634766, + 0.07011374831199646, + 0.27577298879623413, + 0.18436232209205627, + 0.14370256662368774, + -1.4306739568710327, + 0.6487612128257751, + -1.2129435539245605, + 0.3180162310600281, + 0.25029003620147705, + 0.32358434796333313, + 0.02250918745994568, + -0.6521103382110596, + -0.8014636635780334, + -0.4835957884788513, + -0.5101519823074341 + ], + [ + 0.21362939476966858, + 0.2772570848464966, + 0.8402403593063354, + -0.3651801347732544, + 0.9455380439758301, + -0.15807780623435974, + 0.01500430703163147, + 0.27950620651245117, + 0.11633198708295822, + 0.20554964244365692, + -0.9478622674942017, + 0.2525849938392639, + -0.6278101205825806, + 0.14455139636993408, + -0.05825187638401985, + 0.2951047718524933, + -0.1235896497964859, + -0.4522935152053833, + -0.7448015809059143, + -1.2800843715667725, + -0.6439568996429443 + ] + ], + "generated_utilities": [ + 0.035203736275434494, + 0.032373785972595215, + 0.05178316310048103, + 0.0497889518737793, + 0.05319662764668465, + 0.07769393920898438, + 0.03644818067550659, + 0.06582687050104141 + ], + "group_id": "ms-a96bca1ef0b74f686018", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07923875749111176, + 0.1507580727338791, + 0.09708476066589355, + 0.1572066992521286, + 0.09626718610525131, + 0.0781460627913475, + 0.07252912223339081, + 0.04244875907897949, + 0.026137471199035645, + 0.03892441838979721, + 0.04886416718363762, + 0.10530619323253632, + 0.0477486252784729, + 0.0898323506116867, + 0.01968318223953247, + 0.0289560966193676 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.48996758460998535, + 0.9053895473480225, + 1.8985445499420166, + -1.3594893217086792, + 1.6097791194915771, + 1.132408618927002, + 1.8319320678710938, + 0.46463528275489807, + 0.13208985328674316, + 0.8792686462402344, + -1.868321418762207, + 1.1490275859832764, + -2.0193891525268555, + -2.0637927055358887, + 0.11448347568511963, + 1.2957178354263306, + 1.293219804763794, + -1.7789109945297241, + -0.4312591254711151, + -1.7441130876541138, + -0.06536102294921875 + ], + [ + 0.03654795140028, + 1.7676925659179688, + 1.048339605331421, + -1.366391658782959, + 2.5093846321105957, + 2.160305976867676, + 2.8675124645233154, + -0.309950590133667, + -0.5841768980026245, + 1.3030188083648682, + -1.2228018045425415, + 1.1373907327651978, + -2.831526041030884, + -2.2111129760742188, + -0.14044174551963806, + 1.660491704940796, + 1.5215414762496948, + -1.0137104988098145, + -0.40579986572265625, + -1.9571893215179443, + 1.241256833076477 + ], + [ + -0.2826732099056244, + 2.2520718574523926, + 2.418309450149536, + -1.7572941780090332, + 1.5832278728485107, + 0.6361246705055237, + 2.2009639739990234, + 0.9109495878219604, + 0.22027403116226196, + 0.7075455188751221, + -1.2762477397918701, + 0.8630691766738892, + -1.2986351251602173, + -2.567821979522705, + -0.4690782427787781, + 2.1363346576690674, + 0.1853562593460083, + -1.541505217552185, + -0.033681899309158325, + -1.0161254405975342, + 0.06877446174621582 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03534998744726181, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.3504420518875122, + 0.0, + -0.013733148574829102, + 0.021970629692077637, + 0.014389097690582275, + -0.7432765960693359, + 0.006671249866485596, + -0.9755387306213379, + 0.0, + -0.027152668684720993, + 0.2708500623703003, + 0.052558183670043945, + -0.7615737915039062, + 0.020878344774246216, + -0.2927910089492798, + 0.0 + ], + [ + -0.1400495320558548, + 0.3312731981277466, + 0.5862538814544678, + -0.5824885368347168, + 0.9715663194656372, + 0.5714048147201538, + 0.0, + -0.3271818161010742, + -0.04413123428821564, + 0.16598933935165405, + -0.7432765960693359, + 0.2282017469406128, + -0.9755387306213379, + 0.0, + -0.4382812976837158, + 0.2708500623703003, + 0.2094714641571045, + -0.7615737915039062, + 0.23577484488487244, + -0.2927910089492798, + 0.0 + ] + ], + "predicted_scores": [ + 2.423630475997925, + 2.00319766998291, + 1.951001763343811, + 1.8348560333251953, + 1.5207659006118774, + 1.333327054977417, + 0.9061217904090881, + 0.8724957704544067 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "61714505b9ac6cbb1c6deff698bb3e96fd54c7cdb8f0d4307bfdd236634525c0", + "stored_base_utility": 0.07923875749111176, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.10657280683517456, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10657280683517456 + }, + { + "contact_quality": null, + "progress": 0.08607447147369385, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.08607447147369385 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.10657280683517456, + 0.08607447147369385 + ], + "candidate_source_chart_ids": [ + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80", + "ms-37384bede4d1c03f3b80" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ac4fb33f3ab4db853995", + "generated_tangents": [ + [ + 1.6531691551208496, + -0.05526414141058922, + -0.1839403212070465, + 0.42755988240242004, + -0.09132128953933716, + 1.700735092163086, + 1.5142931938171387, + 1.6024315357208252, + 0.19293171167373657, + -1.11464524269104, + -0.5208871364593506, + -0.7540864944458008, + -3.1391124725341797, + -1.9721730947494507, + 1.2823563814163208, + 0.25687524676322937, + -0.9032511115074158, + 0.3206101953983307, + -0.6732171177864075, + -1.6390252113342285, + -1.0234016180038452 + ], + [ + 1.888071060180664, + -0.2116311937570572, + 0.10793489217758179, + 0.4411371648311615, + -0.3422536849975586, + 2.3450942039489746, + 1.5086116790771484, + 2.2642388343811035, + 0.20116852223873138, + -1.4900288581848145, + -0.7762719392776489, + -0.7539726495742798, + -3.886331558227539, + -2.482748031616211, + 1.4267284870147705, + 0.4085291624069214, + -0.6320739388465881, + 0.29684925079345703, + -0.3692408800125122, + -1.113175392150879, + -1.1301547288894653 + ], + [ + 1.6716148853302002, + -0.13889551162719727, + 0.007288455963134766, + 0.3261609375476837, + -0.17362120747566223, + 2.013519287109375, + 1.6892683506011963, + 2.1416587829589844, + 0.2149132788181305, + -1.5493266582489014, + -0.687951922416687, + -0.9009572267532349, + -3.6530354022979736, + -2.873899221420288, + 1.4981926679611206, + 0.37792372703552246, + -0.5116065144538879, + 0.08120601624250412, + -0.3444892168045044, + -1.060876727104187, + -1.3318877220153809 + ], + [ + 1.7461152076721191, + -0.21108204126358032, + -0.13795432448387146, + 0.32761266827583313, + -0.3691607117652893, + 2.3137362003326416, + 1.6223986148834229, + 2.0560142993927, + 0.24970006942749023, + -1.4734489917755127, + -0.7384984493255615, + -0.8993710279464722, + -3.650503635406494, + -2.9555695056915283, + 1.415787696838379, + 0.41653957962989807, + -0.4787940979003906, + 0.17419150471687317, + -0.32359981536865234, + -0.9217686653137207, + -1.3698807954788208 + ], + [ + 1.5240848064422607, + -0.3394169807434082, + 0.14871841669082642, + -0.32393109798431396, + 0.027644485235214233, + 0.5967182517051697, + 2.466012477874756, + 1.6800605058670044, + 0.18204478919506073, + -1.2379786968231201, + -0.6065686345100403, + -1.0351016521453857, + -3.2212986946105957, + -1.9640520811080933, + 0.9234760999679565, + 0.19026611745357513, + -0.473141074180603, + -0.0059905946254730225, + -0.41115814447402954, + -2.032818078994751, + -1.0137569904327393 + ], + [ + 1.647157907485962, + -0.21023760735988617, + -0.2447310984134674, + 0.34482625126838684, + -0.40069395303726196, + 1.971716284751892, + 1.8459899425506592, + 2.1531007289886475, + 0.22969676554203033, + -1.4864333868026733, + -0.9127892851829529, + -1.0147104263305664, + -3.628441572189331, + -2.7818243503570557, + 1.4673426151275635, + 0.3162531852722168, + -0.42331385612487793, + 0.1564280092716217, + -0.4080636501312256, + -1.2643910646438599, + -1.6785491704940796 + ], + [ + 1.6835044622421265, + -0.1711624711751938, + -0.322613388299942, + 0.4638591706752777, + -0.331248939037323, + 2.0816099643707275, + 1.8940846920013428, + 2.0946686267852783, + 0.26951080560684204, + -1.4435874223709106, + -0.6085704565048218, + -1.0365313291549683, + -3.5451056957244873, + -2.8089160919189453, + 1.4839354753494263, + 0.35859429836273193, + -0.43926113843917847, + -0.0053787678480148315, + -0.31296366453170776, + -1.1489890813827515, + -1.4108330011367798 + ], + [ + 1.649933934211731, + -0.24798466265201569, + -0.3144838511943817, + 0.3171953856945038, + -0.2297109067440033, + 1.523589849472046, + 1.9365217685699463, + 1.9921112060546875, + 0.2927779257297516, + -1.6297800540924072, + -0.6952162981033325, + -1.1927822828292847, + -3.184230327606201, + -2.780707359313965, + 1.660999059677124, + 0.3532601594924927, + -0.6409322023391724, + 0.05518341064453125, + -0.42419081926345825, + -1.48614501953125, + -1.4847630262374878 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.10657280683517456, + 0.08607447147369385 + ], + "group_id": "ms-ac4fb33f3ab4db853995", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.2651830017566681, + 0.22676818072795868, + 0.22454051673412323, + 2.0, + 0.31214088201522827, + 0.21535459160804749, + 0.18962346017360687, + 2.0, + 0.1830430030822754, + 0.15030252933502197, + 0.12050267308950424, + 0.17022761702537537, + 0.11389490216970444, + 0.07058519124984741, + 0.07076358795166016, + 0.18658748269081116 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 2.0962133407592773, + 0.057441189885139465, + -0.2733909487724304, + 0.4170292317867279, + -0.7772165536880493, + 2.858414888381958, + 2.1098427772521973, + 1.8366810083389282, + 0.017507493495941162, + -1.6127345561981201, + -0.39152991771698, + -1.017486333847046, + -2.815659999847412, + -2.787520408630371, + 1.6748408079147339, + 0.2986273169517517, + -0.6001855134963989, + 0.5475705862045288, + -0.28012242913246155, + -1.5382447242736816, + -1.6652274131774902 + ], + [ + 2.1338062286376953, + -0.07656055688858032, + -0.7531881332397461, + 0.48830646276474, + -0.36628106236457825, + 2.381558418273926, + 1.7315943241119385, + 1.838928461074829, + -0.050897032022476196, + -1.171897292137146, + -0.8350631594657898, + -0.8261303901672363, + -2.2635834217071533, + -3.2190167903900146, + 1.3416705131530762, + 0.44279617071151733, + -0.7299901247024536, + 1.2929750680923462, + -0.0032969415187835693, + -1.5000369548797607, + -1.6060972213745117 + ], + [ + 1.5521352291107178, + 0.03659695386886597, + -0.44382959604263306, + 0.21355122327804565, + -1.0471938848495483, + 3.194620370864868, + 1.9892739057540894, + 2.5084128379821777, + 0.531404972076416, + -0.9168282747268677, + -0.49052953720092773, + -1.1588059663772583, + -3.156189441680908, + -3.278008460998535, + 0.921509325504303, + 0.49181532859802246, + -1.0228296518325806, + 0.5866082310676575, + -0.4882396459579468, + -0.7946261763572693, + -2.119061231613159 + ], + [ + 2.7138938903808594, + -0.3406146764755249, + -0.32174432277679443, + 0.9962238669395447, + -0.558576226234436, + 2.049264907836914, + 1.8703808784484863, + 2.113438367843628, + 0.9615883827209473, + -1.9615708589553833, + -0.0062177181243896484, + -0.5808464288711548, + -2.1758205890655518, + -2.219890594482422, + 1.022050142288208, + 0.7201904654502869, + -1.3177021741867065, + 0.4089192748069763, + -0.0315951406955719, + -0.7215378880500793, + -2.069262981414795 + ], + [ + 2.5701913833618164, + 0.3137266933917999, + -0.5592924356460571, + 0.0026398897171020508, + -0.19283491373062134, + 2.2903807163238525, + 2.637770652770996, + 2.5494186878204346, + 0.9258015155792236, + -0.7146614193916321, + -0.686233639717102, + -0.7985042929649353, + -3.431988000869751, + -3.2178471088409424, + 1.319100260734558, + -0.15109702944755554, + -0.2694774866104126, + 1.0912322998046875, + -0.6380462050437927, + -2.0780839920043945, + -2.1209774017333984 + ], + [ + 2.6195321083068848, + -0.2646539509296417, + 0.15159690380096436, + 0.8749752044677734, + -1.1848887205123901, + 2.8789353370666504, + 1.6037814617156982, + 1.3343384265899658, + -0.2254078984260559, + -2.041776657104492, + -0.9811336994171143, + -1.3441768884658813, + -2.2502636909484863, + -1.9091612100601196, + 0.9959704279899597, + 0.42473238706588745, + -0.5330828428268433, + 1.6945891380310059, + 0.4069111943244934, + -0.5568764805793762, + -1.472819447517395 + ], + [ + 2.9485361576080322, + -0.6907311677932739, + -0.057036399841308594, + 1.2457408905029297, + 0.09488356113433838, + 2.6509125232696533, + 2.0378005504608154, + 2.278454303741455, + 1.1734973192214966, + -1.853393316268921, + 0.09614759683609009, + -0.35674989223480225, + -2.1597909927368164, + -1.9672894477844238, + 1.9760816097259521, + -0.47744297981262207, + 0.021782994270324707, + 1.6328153610229492, + -0.8569514751434326, + -1.0814869403839111, + -1.7780723571777344 + ], + [ + 2.1897926330566406, + -0.3883178234100342, + -0.6492674946784973, + 1.3223958015441895, + 0.30954307317733765, + 2.031817674636841, + 1.878535270690918, + 2.5030624866485596, + 0.6644855737686157, + -1.7382179498672485, + -1.1862120628356934, + -0.7967990636825562, + -2.8425841331481934, + -2.3635308742523193, + 2.0061349868774414, + -0.6015456318855286, + -1.429253339767456, + 1.2773160934448242, + 0.04122643172740936, + -2.0497689247131348, + -1.1791472434997559 + ], + [ + 1.586853265762329, + 0.06395213305950165, + -0.8944701552391052, + 0.5118889808654785, + -1.1310638189315796, + 2.865907669067383, + 1.3295278549194336, + 1.0862370729446411, + 0.6303088665008545, + -0.3754369616508484, + -0.04129624366760254, + -0.7044278979301453, + -2.923245429992676, + -1.7837438583374023, + 1.5932514667510986, + 1.2467985153198242, + 0.13401156663894653, + 1.6706414222717285, + -0.816297709941864, + -1.2893165349960327, + -1.0985345840454102 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.1383795738220215, + -0.1483764797449112, + -0.27178800106048584, + 0.26738643646240234, + -0.12845781445503235, + 1.6495440006256104, + 0.0, + 0.9713280200958252, + -0.050046056509017944, + -0.36543774604797363, + -0.022124826908111572, + 0.08494043350219727, + -1.671510934829712, + 0.0, + 0.5271386504173279, + 0.3898613452911377, + 0.21280205249786377, + -0.2029280662536621, + -0.01802508533000946, + -0.35994601249694824, + 0.0 + ], + [ + 1.7570452690124512, + 0.07137972116470337, + 0.02318662405014038, + 0.2907020151615143, + -0.2308352291584015, + 2.89892578125, + 2.537220001220703, + 2.238605499267578, + 0.16207315027713776, + -1.6996564865112305, + -0.8050198554992676, + -1.2014062404632568, + -2.5764517784118652, + -2.393839120864868, + 1.1276613473892212, + 0.31928181648254395, + -1.036102533340454, + 1.1266506910324097, + -0.1944657564163208, + -1.5998979806900024, + -2.151465892791748 + ] + ], + "predicted_scores": [ + 5.929937362670898, + 4.861267566680908, + 4.486452579498291, + 4.37121057510376, + 4.077110290527344, + 3.99444842338562, + 3.964893102645874, + 3.537386655807495 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "565369392", + "selected_index": 0, + "state_hash": "0b62c42c3beda0eb959692d55792514223681b522edd0a5e73247682cf0bfe95", + "stored_base_utility": 0.2651830017566681, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-456cdaaa617fc54fd766", + "ms-fbd73493bcf73542f735", + "ms-9681d9cb43cc26adb0fe", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766", + "ms-456cdaaa617fc54fd766" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "generated_tangents": [ + [ + 1.5821378231048584, + 0.3146262764930725, + -1.0519134998321533, + -0.4090140759944916, + 0.605135440826416, + 0.02337801456451416, + -1.0089218616485596, + 0.7230665683746338, + -0.012307297438383102, + -0.23266643285751343, + -0.09701662510633469, + 0.26082491874694824, + -2.365182638168335, + -2.036797046661377, + 0.4285762310028076, + 0.4792984426021576, + 0.8540741801261902, + -1.276024341583252, + 1.1826664209365845, + -1.9315857887268066, + -0.6276378631591797 + ], + [ + 1.3737164735794067, + 0.2679445743560791, + -0.841779351234436, + -0.30625125765800476, + -0.1751491129398346, + -0.6902062892913818, + -0.6044722199440002, + 0.224562406539917, + 0.1592528223991394, + -0.09754335135221481, + -0.464151531457901, + -0.27126649022102356, + -2.840402841567993, + -1.8718410730361938, + 0.28366222977638245, + 0.2886638343334198, + 0.589530885219574, + -1.0971307754516602, + 1.037682056427002, + -2.1236624717712402, + -1.1697248220443726 + ], + [ + 1.4693235158920288, + 0.33992689847946167, + -0.9559155702590942, + -0.7759301662445068, + 0.5938560366630554, + 0.4410706162452698, + -1.1516180038452148, + 0.9748616218566895, + -0.041813597083091736, + -0.2766180634498596, + -0.27089664340019226, + 0.37276214361190796, + -2.81288480758667, + -2.1215333938598633, + 0.2626761496067047, + 0.4437684714794159, + 0.8953127861022949, + -1.1623727083206177, + 0.9205278158187866, + -1.9004206657409668, + -0.5554968118667603 + ], + [ + 1.3242058753967285, + 0.21055619418621063, + -1.0986342430114746, + -0.46126535534858704, + 0.25072017312049866, + -0.5641969442367554, + -0.49803632497787476, + 0.47351670265197754, + 0.11856383085250854, + -0.09280801564455032, + -0.29051852226257324, + -0.15622341632843018, + -2.8529722690582275, + -1.9449175596237183, + 0.3076666593551636, + 0.3464489281177521, + 0.5473067760467529, + -1.043578028678894, + 0.785569965839386, + -2.2451817989349365, + -1.1477209329605103 + ], + [ + 1.2049038410186768, + 0.21534813940525055, + -1.0089812278747559, + -0.7239872217178345, + 0.017136886715888977, + -0.9327054023742676, + -1.4726097583770752, + 0.6058399677276611, + 0.057108648121356964, + 0.04002790153026581, + 0.07871583104133606, + 0.07699061930179596, + -1.9197403192520142, + -1.3473838567733765, + 0.4497959613800049, + 0.23396480083465576, + 0.8955276608467102, + -0.8793306946754456, + 0.9313246011734009, + -1.6798534393310547, + -0.5567028522491455 + ], + [ + 1.4960994720458984, + 0.31615322828292847, + -1.0203620195388794, + -0.572618842124939, + 0.3891475796699524, + 0.29192644357681274, + -1.0681753158569336, + 0.832166314125061, + -0.06805451214313507, + -0.1598108410835266, + -0.18521764874458313, + 0.36702466011047363, + -2.688307285308838, + -2.25980544090271, + 0.3495863080024719, + 0.4443596303462982, + 0.9426625370979309, + -1.2859559059143066, + 0.8251757621765137, + -1.8694875240325928, + -0.8118435144424438 + ], + [ + 1.498408317565918, + 0.34343862533569336, + -0.7902431488037109, + -0.576141893863678, + 0.07044373452663422, + 0.3306047022342682, + -1.1915507316589355, + 0.8400086164474487, + -0.03115631826221943, + -0.07947546988725662, + -0.3417731523513794, + 0.37364697456359863, + -2.825819730758667, + -2.354111433029175, + 0.27121174335479736, + 0.4421323239803314, + 0.9562599062919617, + -1.1478915214538574, + 0.7435473799705505, + -1.878373146057129, + -0.7719675302505493 + ], + [ + 1.3977625370025635, + 0.3278428316116333, + -1.282359004020691, + -0.5069842338562012, + 0.3946380615234375, + 0.42068952322006226, + -0.3734458088874817, + 0.9397799968719482, + 0.0006347186863422394, + -0.12525802850723267, + -0.017962239682674408, + 0.35881081223487854, + -2.728942632675171, + -2.072857141494751, + 0.18919141590595245, + 0.36938372254371643, + 1.256686806678772, + -1.2285515069961548, + 0.7937507033348083, + -1.8180181980133057, + -0.7821594476699829 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.149173304438591, + 0.06221481412649155, + 0.1385849118232727, + 0.06656398624181747, + 2.0, + 0.10397814214229584, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7398372888565063, + -0.02836926281452179, + -0.4292973279953003, + 0.028730571269989014, + 0.009982943534851074, + -0.3464317321777344, + 0.0, + 0.009800434112548828, + -0.06816069781780243, + 0.030653484165668488, + -0.00554502010345459, + -0.06216755136847496, + -1.9802618026733398, + 0.0, + 0.01717478036880493, + -0.05601567029953003, + 0.010390102863311768, + -0.8072932958602905, + -0.05775028467178345, + -1.9342114925384521, + 0.0 + ], + [ + 0.7398372888565063, + -0.1961192637681961, + -0.4292973279953003, + -0.0017886161804199219, + 0.06461763381958008, + -0.3464317321777344, + 0.0, + 0.42972221970558167, + -0.5797654986381531, + 0.19436834752559662, + -0.23844504356384277, + 0.1733923852443695, + -1.9802618026733398, + 0.0, + -0.27269166707992554, + 0.04145494103431702, + 0.12319430708885193, + -0.8072932958602905, + 0.4949256181716919, + -1.9342114925384521, + 0.0 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.7398372888565063, + 0.2242223471403122, + -1.4292973279953003, + -0.537040650844574, + -0.9353823661804199, + -1.3464317321777344, + -2.159137487411499, + 0.489885151386261, + -0.006211067084223032, + 0.09238272905349731, + -1.0055450201034546, + -0.04778099060058594, + -2.98026180267334, + -2.576570987701416, + 0.2992324233055115, + 0.34874510765075684, + 0.5165960192680359, + -1.8072932958602905, + 0.834364652633667, + -2.934211492538452, + -2.0790040493011475 + ], + [ + 1.9740948677062988, + 0.4187958538532257, + -1.2225801944732666, + -0.6112513542175293, + -1.0962133407592773, + -1.4875872135162354, + -2.1883857250213623, + 0.6603437662124634, + -0.20142723619937897, + 0.1697872281074524, + -0.8377580046653748, + -0.16444604098796844, + -2.87237286567688, + -2.4153482913970947, + 0.03898555040359497, + 0.5175161957740784, + 0.7254846096038818, + -1.9598647356033325, + 1.1053451299667358, + -3.172945976257324, + -1.9334450960159302 + ], + [ + 1.5938913822174072, + -0.018363401293754578, + -1.1851862668991089, + -0.20344951748847961, + -0.9226735234260559, + -1.3553601503372192, + -1.932102084159851, + 0.6446930170059204, + 0.30503973364830017, + 0.3522501289844513, + -0.9527953267097473, + -0.15980565547943115, + -2.9574379920959473, + -2.9056527614593506, + 0.6069583892822266, + 0.3722725808620453, + 0.7255810499191284, + -1.8503519296646118, + 0.5613387823104858, + -3.095244884490967, + -2.023747444152832 + ], + [ + 1.6626678705215454, + -0.02834661304950714, + -1.0125243663787842, + -0.5629516839981079, + -1.230878233909607, + -1.414373755455017, + -2.42856764793396, + 0.8520264625549316, + -0.1296539008617401, + -0.22410845756530762, + -1.0539721250534058, + -0.060277245938777924, + -2.740527391433716, + -2.1194698810577393, + 0.6826534867286682, + 0.7535779476165771, + 0.35573869943618774, + -1.9124289751052856, + 0.7101565003395081, + -3.326658010482788, + -1.8195855617523193 + ], + [ + 2.2881288528442383, + 0.6951426863670349, + -1.0978585481643677, + -0.008003771305084229, + -1.0754073858261108, + -1.7734596729278564, + -2.139570474624634, + 0.9205271601676941, + 0.29560548067092896, + 0.49158430099487305, + -1.2606573104858398, + 0.43956148624420166, + -2.831833600997925, + -2.960129976272583, + 0.6832801103591919, + -0.03894898295402527, + 1.036036491394043, + -2.127277135848999, + 0.6759494543075562, + -3.1690073013305664, + -1.9895020723342896 + ], + [ + 1.9020261764526367, + 0.6575945019721985, + -1.3994193077087402, + -0.5040701031684875, + -0.6857678890228271, + -0.7056208252906799, + -1.5344107151031494, + 0.7615087032318115, + 0.6105715036392212, + 0.19082021713256836, + -0.6303516626358032, + -0.4614318609237671, + -2.3375303745269775, + -2.188478469848633, + 0.6886799335479736, + -0.03237125277519226, + 0.483882874250412, + -1.3982734680175781, + 0.23996800184249878, + -3.3969180583953857, + -1.5116117000579834 + ], + [ + 1.666227102279663, + 0.6915197968482971, + -1.6392340660095215, + -0.9650112390518188, + -0.2591104507446289, + -1.5417896509170532, + -1.695828914642334, + 1.130251169204712, + -0.555443525314331, + -0.4872768521308899, + -1.0994479656219482, + 0.5934816002845764, + -3.5218958854675293, + -1.8239338397979736, + 0.7150859832763672, + 0.8003170490264893, + 1.0113730430603027, + -2.1905770301818848, + 0.8595032095909119, + -3.039506673812866, + -2.3675880432128906 + ], + [ + 1.59856116771698, + -0.036071136593818665, + -1.700652837753296, + 0.27602267265319824, + -1.608276128768921, + -0.875938355922699, + -1.6728798151016235, + 0.8851535320281982, + 0.7461313605308533, + -0.7497223019599915, + -1.0928030014038086, + -0.4735495150089264, + -2.223254680633545, + -2.5322465896606445, + 0.044906944036483765, + 0.6160990595817566, + 1.3697642087936401, + -1.6367510557174683, + 0.086475670337677, + -3.323115825653076, + -2.058189630508423 + ], + [ + 2.285855531692505, + 0.6314746141433716, + -1.2119637727737427, + 0.039707839488983154, + -0.9751565456390381, + -1.7796330451965332, + -1.5261995792388916, + 0.5450726747512817, + -0.8933275938034058, + 0.7903657555580139, + -1.5466060638427734, + 0.5994364023208618, + -3.370340585708618, + -3.1641077995300293, + 0.3225497603416443, + -0.39524197578430176, + 1.152822732925415, + -1.508494257926941, + 1.1221402883529663, + -3.361729860305786, + -1.9251339435577393 + ], + [ + 2.2599079608917236, + -0.005553290247917175, + -2.4053664207458496, + 0.39961689710617065, + -0.40980857610702515, + -0.9905421733856201, + -1.7244036197662354, + -0.06417369842529297, + 0.6376181840896606, + 0.5168956518173218, + -1.6740490198135376, + -0.1833474189043045, + -3.9368975162506104, + -1.7146861553192139, + 0.9366075992584229, + 0.701168417930603, + -0.23131346702575684, + -2.740941286087036, + 0.908729076385498, + -3.6231396198272705, + -1.3161351680755615 + ], + [ + 1.819408655166626, + -0.5504165291786194, + -1.302573800086975, + -0.6927691698074341, + -0.7548186779022217, + -2.1824703216552734, + -2.4921669960021973, + 0.6337187886238098, + 0.1691494882106781, + 0.7547447681427002, + -0.4371253252029419, + -0.6562469601631165, + -2.313655376434326, + -2.4974589347839355, + 0.05105690658092499, + 0.3473445773124695, + -0.4244192838668823, + -1.8233305215835571, + -0.1157217025756836, + -2.0408761501312256, + -2.2010600566864014 + ] + ], + "predicted_scores": [ + 3.6475484371185303, + 3.555715322494507, + 3.4646213054656982, + 3.458019495010376, + 3.0644209384918213, + 2.9289677143096924, + 2.7355568408966064, + 2.3974509239196777 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2054899126", + "selected_index": 0, + "state_hash": "b26c5f64bc7e877098323281f2b58e7850275b30405bc411ed7ffc7cf69f4e26", + "stored_base_utility": 0.149173304438591, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.037537217140197754, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.037537217140197754 + }, + "base_progress": 0.037537217140197754, + "base_success": false, + "base_utility": 0.037537217140197754, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.02533026598393917, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02533026598393917 + }, + { + "contact_quality": null, + "progress": 0.008884906768798828, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008884906768798828 + }, + { + "contact_quality": null, + "progress": 0.008443613536655903, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008443613536655903 + }, + { + "contact_quality": null, + "progress": 0.02165730856359005, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02165730856359005 + }, + { + "contact_quality": null, + "progress": 0.010604898445308208, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010604898445308208 + }, + { + "contact_quality": null, + "progress": 0.02483522891998291, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02483522891998291 + }, + { + "contact_quality": null, + "progress": 0.029142936691641808, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029142936691641808 + }, + { + "contact_quality": null, + "progress": 0.009114067070186138, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009114067070186138 + } + ], + "candidate_progress": [ + 0.02533026598393917, + 0.008884906768798828, + 0.008443613536655903, + 0.02165730856359005, + 0.010604898445308208, + 0.02483522891998291, + 0.029142936691641808, + 0.009114067070186138 + ], + "candidate_source_chart_ids": [ + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058", + "ms-d535b38d768633f6d922", + "ms-60f0d45b604940db0058", + "ms-6f26118aa94bacd82250", + "ms-60f0d45b604940db0058", + "ms-60f0d45b604940db0058" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-c832439976a37f52d094", + "generated_tangents": [ + [ + -0.8198245167732239, + -0.03920508921146393, + 0.5819593667984009, + 0.7463089227676392, + 0.10489299893379211, + -0.10580489039421082, + -0.20906049013137817, + 1.1903159618377686, + -0.34412050247192383, + -0.6929799318313599, + -1.9149105548858643, + 0.643068253993988, + -1.6121429204940796, + -1.0431427955627441, + 0.1031402200460434, + 0.24797122180461884, + 0.7003986835479736, + 0.023552045226097107, + 0.13818491995334625, + 2.3249471187591553, + 1.3970123529434204 + ], + [ + -1.0568962097167969, + -0.18154874444007874, + 0.4242200553417206, + 1.3580749034881592, + -0.2119625210762024, + -0.10483729839324951, + -0.6275549530982971, + 1.8155874013900757, + -0.3267662525177002, + -0.7155795693397522, + -2.3674774169921875, + 1.2270745038986206, + -1.9952422380447388, + -1.1826207637786865, + 0.12356767058372498, + 0.3143351376056671, + 0.4271289110183716, + 0.07336220145225525, + 0.4373856484889984, + 3.0269980430603027, + 1.3758344650268555 + ], + [ + -1.0864646434783936, + -0.16934318840503693, + 0.3293246030807495, + 1.4138600826263428, + -0.07199227809906006, + 0.03945857286453247, + -0.4595247507095337, + 1.8685837984085083, + -0.3098562955856323, + -0.6432785987854004, + -2.4197516441345215, + 1.0945141315460205, + -2.026355504989624, + -1.45381498336792, + 0.06001746654510498, + 0.32894641160964966, + 0.45864784717559814, + 0.0468917042016983, + 0.595192551612854, + 2.9091782569885254, + 1.4164118766784668 + ], + [ + -0.6667748093605042, + -0.06050927937030792, + 0.5117385387420654, + 0.8088277578353882, + 0.022689253091812134, + -0.14834624528884888, + 0.016691237688064575, + 1.2453981637954712, + -0.3827824592590332, + -0.5151144862174988, + -1.9905287027359009, + 0.7936994433403015, + -1.7512415647506714, + -1.3275130987167358, + 0.11352592706680298, + 0.2874607741832733, + 0.6238629817962646, + -0.0001649409532546997, + 0.25904381275177, + 2.1807234287261963, + 1.6636848449707031 + ], + [ + -1.0627790689468384, + -0.06425783783197403, + 0.45271599292755127, + 1.2808951139450073, + -0.07449811697006226, + -0.08740252256393433, + -0.3336034417152405, + 1.748331904411316, + -0.25657546520233154, + -0.6822872757911682, + -2.256725311279297, + 1.1384570598602295, + -2.145827054977417, + -1.134251356124878, + 0.12506824731826782, + 0.30707207322120667, + 0.5226618051528931, + 0.044191062450408936, + 0.6069962978363037, + 2.6754794120788574, + 1.2606077194213867 + ], + [ + -0.543549120426178, + 0.05135799199342728, + 0.04146483540534973, + 0.9642932415008545, + 0.2744615077972412, + -0.7055658102035522, + 0.03691324591636658, + 0.9951686859130859, + -0.32953909039497375, + -0.23412221670150757, + -1.3947173357009888, + 0.9031991958618164, + -1.835490107536316, + -0.7074829339981079, + -0.1214071661233902, + 0.1754102259874344, + 0.7325943112373352, + -0.28040844202041626, + 0.5575394630432129, + 1.8994114398956299, + 0.8361173868179321 + ], + [ + -0.911431074142456, + 0.015191037207841873, + 0.5205184817314148, + 0.7936104536056519, + 0.05123633146286011, + -0.14767053723335266, + -0.06205552816390991, + 1.1552459001541138, + -0.4112311601638794, + -0.5587905049324036, + -1.9563528299331665, + 0.9562231302261353, + -1.4852240085601807, + -0.8329577445983887, + 0.03854535147547722, + 0.253476083278656, + 0.5263214111328125, + 0.09186801314353943, + 0.6027330756187439, + 2.5320591926574707, + 1.1744015216827393 + ], + [ + -0.8819104433059692, + -0.004576355218887329, + 0.35864776372909546, + 1.2648749351501465, + 0.19551649689674377, + -0.12527060508728027, + -0.251573383808136, + 1.5137056112289429, + -0.12731441855430603, + -0.6814642548561096, + -2.018608570098877, + 0.9902058243751526, + -2.0675392150878906, + -0.812502384185791, + 0.24994364380836487, + 0.27510279417037964, + 0.5562025904655457, + -0.038785889744758606, + 0.4682663381099701, + 2.339139461517334, + 1.2594951391220093 + ] + ], + "generated_utilities": [ + 0.02533026598393917, + 0.008884906768798828, + 0.008443613536655903, + 0.02165730856359005, + 0.010604898445308208, + 0.02483522891998291, + 0.029142936691641808, + 0.009114067070186138 + ], + "group_id": "ms-c832439976a37f52d094", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0808674544095993, + 0.13129448890686035, + 0.11199682950973511, + 0.2594742774963379, + 0.17684690654277802, + 0.11511725187301636, + 0.17100229859352112, + 0.12906593084335327, + 0.12404869496822357, + 0.10653237998485565, + 0.15492436289787292, + 0.05800905078649521, + 0.04767048358917236, + 0.09533219039440155, + 0.03300365060567856, + 0.07704520225524902 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025296777486801147, + 0.11629299819469452, + -0.024434387683868408, + 1.0812468528747559, + 0.04022188484668732, + -0.01547396183013916, + 0.0, + 0.05167186260223389, + 0.07629208266735077, + -0.00870610773563385, + -0.5777420997619629, + 0.07687681913375854, + -0.6336902379989624, + 0.0, + -0.043449461460113525, + -0.029798656702041626, + 0.18944299221038818, + 0.03022998571395874, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + 0.05685463547706604, + -0.3914306163787842, + 0.4578048586845398, + 1.0812468528747559, + 0.10990457236766815, + -0.18159747123718262, + 0.0, + 0.05167186260223389, + -0.5002377033233643, + 0.04299694299697876, + -0.5777420997619629, + 0.02747189998626709, + -0.6336902379989624, + 0.0, + -0.08638033270835876, + 0.17561247944831848, + 0.37304359674453735, + 0.2538231611251831, + 0.7312084436416626, + 1.8623785972595215, + 0.0 + ], + [ + -0.3902576267719269, + 0.03852551057934761, + 0.16114424169063568, + 2.081246852874756, + 0.22516529262065887, + 0.8184025287628174, + 0.3388257324695587, + 1.0516718626022339, + -0.061541248112916946, + 0.22279328107833862, + -1.577742099761963, + 1.027471899986267, + -1.6336902379989624, + -1.6284003257751465, + -0.24836044013500214, + -0.3427097499370575, + 1.1894429922103882, + -0.15580247342586517, + 1.7312084436416626, + 2.8623785972595215, + 1.0248357057571411 + ], + [ + -0.5903857946395874, + -0.10628272593021393, + -0.07594148814678192, + 2.194340944290161, + 0.11632551997900009, + 0.942036509513855, + 0.06351110339164734, + 0.7966359853744507, + 0.1702205240726471, + 0.1549067199230194, + -1.439386248588562, + 1.3076335191726685, + -1.3458164930343628, + -1.90281081199646, + -0.2967699468135834, + -0.27783676981925964, + 1.0556988716125488, + 0.06510581076145172, + 1.6170521974563599, + 3.027111530303955, + 1.154527187347412 + ], + [ + -0.7559880614280701, + 0.694898247718811, + -0.3551136255264282, + 1.8615511655807495, + 0.9249891042709351, + 0.969862699508667, + 0.4452719986438751, + 1.5801937580108643, + 0.13303470611572266, + -0.13816028833389282, + -1.2399333715438843, + 1.5769362449645996, + -1.5071980953216553, + -2.0578060150146484, + -0.8449342846870422, + 0.3327537477016449, + 1.6781622171401978, + 0.4016150236129761, + 1.8172147274017334, + 3.1609320640563965, + 1.4916398525238037 + ] + ], + "predicted_scores": [ + 2.265834331512451, + 2.2305963039398193, + 2.0992085933685303, + 2.014530658721924, + 1.3508869409561157, + 1.0760185718536377, + 0.9567503929138184, + 0.7489107847213745 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "0f2810f22383259329f6e94fdc8384176c664f2ed476924e59db7cda4f4a4863", + "stored_base_utility": 0.0808674544095993, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09138330072164536, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09138330072164536 + }, + "base_progress": 0.09138330072164536, + "base_success": false, + "base_utility": 0.09138330072164536, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.015890737995505333, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015890737995505333 + }, + { + "contact_quality": null, + "progress": 0.006532192230224609, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006532192230224609 + }, + { + "contact_quality": null, + "progress": 0.0023473103065043688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0023473103065043688 + }, + { + "contact_quality": null, + "progress": 0.007760286331176758, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007760286331176758 + }, + { + "contact_quality": null, + "progress": 0.01693703792989254, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01693703792989254 + }, + { + "contact_quality": null, + "progress": 0.0026943087577819824, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0026943087577819824 + }, + { + "contact_quality": null, + "progress": 0.009350995533168316, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009350995533168316 + }, + { + "contact_quality": null, + "progress": 0.013496856205165386, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013496856205165386 + } + ], + "candidate_progress": [ + 0.015890737995505333, + 0.006532192230224609, + 0.0023473103065043688, + 0.007760286331176758, + 0.01693703792989254, + 0.0026943087577819824, + 0.009350995533168316, + 0.013496856205165386 + ], + "candidate_source_chart_ids": [ + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24", + "ms-d6da14bffe59c0265db7", + "ms-db2c4015a0690cc80d24", + "ms-db2c4015a0690cc80d24" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-c96b415f083f60857334", + "generated_tangents": [ + [ + 0.06174185127019882, + 0.16033563017845154, + 0.6671453714370728, + 0.40813764929771423, + 0.3197011351585388, + 2.89839243888855, + 1.3230814933776855, + -0.14131903648376465, + 0.006032740697264671, + 1.1513893604278564, + 0.23397895693778992, + 0.8938525319099426, + 1.2062853574752808, + 1.2772371768951416, + -0.12925788760185242, + -0.6825408339500427, + -0.03556711971759796, + 1.296842098236084, + -0.601061224937439, + 0.1869063377380371, + -0.16873237490653992 + ], + [ + -0.3362332582473755, + 0.09249238669872284, + 1.1323519945144653, + 0.7928755283355713, + 0.5456668138504028, + 2.4640860557556152, + 1.2390525341033936, + -0.4458146393299103, + -0.09833692759275436, + 0.85829097032547, + -0.14441698789596558, + 1.6865679025650024, + 1.6072865724563599, + 1.194542407989502, + -0.12387286126613617, + -0.673762321472168, + 0.00941266119480133, + 1.4972132444381714, + -0.4563497304916382, + 0.774008572101593, + -0.4810538589954376 + ], + [ + -0.5711472630500793, + 0.23895512521266937, + 0.8510761260986328, + 0.5206438302993774, + 1.3842408657073975, + 2.4938137531280518, + 0.8395487070083618, + -0.5246961116790771, + -0.07523845881223679, + 1.12064790725708, + 0.49252521991729736, + 1.1939893960952759, + 1.4841113090515137, + 0.8059262633323669, + -0.19115960597991943, + -0.4434705078601837, + 0.671130359172821, + 1.4036900997161865, + -0.39283162355422974, + 0.6669578552246094, + 0.3197353482246399 + ], + [ + -0.2605810761451721, + 0.06699453294277191, + 1.0132678747177124, + 0.5877721309661865, + 0.7528336048126221, + 2.762845039367676, + 1.0420910120010376, + -0.16084054112434387, + -0.08754749596118927, + 0.9618850350379944, + 0.16109144687652588, + 1.2569996118545532, + 1.3267520666122437, + 1.039302945137024, + -0.11500556766986847, + -0.6801729202270508, + 0.20641353726387024, + 1.3454126119613647, + -0.2619008421897888, + 0.619723379611969, + -0.4462575614452362 + ], + [ + -0.4345166087150574, + 0.005640070885419846, + 0.45308390259742737, + 0.7029827833175659, + 0.520183801651001, + 2.582667350769043, + 1.0958545207977295, + -0.09384369850158691, + -0.13915623724460602, + 0.6293283104896545, + 0.21264910697937012, + 1.0392800569534302, + 1.2485343217849731, + 1.30171799659729, + -0.03129736706614494, + -0.6041085720062256, + -0.1849183291196823, + 1.6694345474243164, + -0.1887371838092804, + 0.2625390887260437, + -0.36729738116264343 + ], + [ + -0.5511574745178223, + 0.26515454053878784, + 0.9038888216018677, + 0.540073037147522, + 1.4176926612854004, + 2.3262815475463867, + 0.6459638476371765, + -0.32197311520576477, + -0.1562633365392685, + 1.2329294681549072, + 0.43738022446632385, + 1.0444974899291992, + 1.4231468439102173, + 0.9694470763206482, + -0.19568341970443726, + -0.45007869601249695, + 0.7409229874610901, + 1.4327186346054077, + -0.35214728116989136, + 0.596210241317749, + 0.23459947109222412 + ], + [ + -0.15069086849689484, + 0.06990204751491547, + 0.9320263862609863, + 0.6539721488952637, + 0.7788692116737366, + 2.5048460960388184, + 1.065310001373291, + -0.2339310646057129, + -0.07993582636117935, + 0.8268597722053528, + 0.18042296171188354, + 1.3550728559494019, + 1.321607232093811, + 0.793192446231842, + 0.03388269990682602, + -0.6482014656066895, + 0.1964799165725708, + 1.4189894199371338, + -0.29010432958602905, + 0.7080780863761902, + -0.45979389548301697 + ], + [ + -0.15596266090869904, + 0.06414393335580826, + 0.7779709100723267, + 0.7256145477294922, + 0.7950412034988403, + 2.3781542778015137, + 1.0835585594177246, + -0.21336519718170166, + -0.12509968876838684, + 0.726881206035614, + 0.025465819984674454, + 1.324411153793335, + 1.5107316970825195, + 0.8208259344100952, + -0.08363127708435059, + -0.6538136005401611, + -0.021880105137825012, + 1.5258489847183228, + -0.2999498248100281, + 0.798958957195282, + -0.4104281961917877 + ] + ], + "generated_utilities": [ + 0.015890737995505333, + 0.006532192230224609, + 0.0023473103065043688, + 0.007760286331176758, + 0.01693703792989254, + 0.0026943087577819824, + 0.009350995533168316, + 0.013496856205165386 + ], + "group_id": "ms-c96b415f083f60857334", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15946325659751892, + 0.16313008964061737, + 0.2226802557706833, + 0.18744397163391113, + 0.21545825898647308, + 0.15985511243343353, + 0.16218391060829163, + 0.26426148414611816, + 0.184181809425354, + 0.24352896213531494, + 0.14179718494415283, + 0.1026589497923851, + 0.1274908483028412, + 0.05275428295135498, + 0.08544860780239105, + 0.07143914699554443 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.16191565990448, + 0.4561867117881775, + 1.1241415739059448, + 0.08947351574897766, + 0.32140541076660156, + 3.6895532608032227, + 3.6284518241882324, + -1.8666775226593018, + -0.22328677773475647, + 1.48662531375885, + 1.4737720489501953, + 1.35750150680542, + 2.507124423980713, + 1.9315069913864136, + -0.8371121883392334, + 0.5339290499687195, + 1.0061794519424438, + 2.149812698364258, + 0.08911214768886566, + 0.17410686612129211, + 0.3549031913280487 + ], + [ + -0.14804881811141968, + -0.5333454608917236, + 0.7481307983398438, + 0.22411420941352844, + 1.7204210758209229, + 3.908766746520996, + 3.3698034286499023, + -1.6747719049453735, + 0.2769082486629486, + 2.042428970336914, + 0.44748249650001526, + 1.9492043256759644, + 2.918121576309204, + 3.2413060665130615, + -0.6994609832763672, + -0.3854917883872986, + -0.6702244281768799, + 2.408869504928589, + -0.5672509670257568, + 0.5941196084022522, + 0.46011871099472046 + ], + [ + -0.46204906702041626, + -0.4236203730106354, + 1.8582539558410645, + 0.8416532278060913, + 1.2386386394500732, + 3.7320151329040527, + 2.242549180984497, + -1.8652616739273071, + -1.0259326696395874, + 1.3383660316467285, + 1.7068204879760742, + 0.7941814661026001, + 3.3389508724212646, + 2.7797679901123047, + -0.9363516569137573, + 0.17812880873680115, + 0.6111975908279419, + 2.066016435623169, + -0.28941091895103455, + 1.501695990562439, + -0.35273995995521545 + ], + [ + -0.7471081614494324, + -0.029650092124938965, + 1.019482135772705, + -0.30564722418785095, + 1.380143642425537, + 3.247985601425171, + 3.0548973083496094, + -0.5582134127616882, + -0.920264482498169, + 2.3859832286834717, + 1.1080355644226074, + 1.5716153383255005, + 3.089156150817871, + 2.043484687805176, + 0.1913408637046814, + -0.7557333707809448, + -0.6297664642333984, + 1.7829738855361938, + -0.6967166662216187, + -0.10757219791412354, + 0.229112908244133 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027880847454071045, + -0.2692106068134308, + 0.12925803661346436, + -0.1745016872882843, + -0.1231425404548645, + 1.9967715740203857, + 0.0, + -0.1433020830154419, + 0.2667914927005768, + 0.8867976665496826, + -0.057856738567352295, + 0.5789351463317871, + 1.7831475734710693, + 0.0, + -0.35565105080604553, + 0.293251097202301, + 0.012226536870002747, + 1.121243953704834, + -0.15060651302337646, + 0.03409510850906372, + 0.0 + ], + [ + -0.7738120555877686, + 0.08056297898292542, + 1.1292580366134644, + 0.4734484851360321, + 0.8768574595451355, + 2.9967715740203857, + 3.0509116649627686, + -1.143302083015442, + -0.1310054361820221, + 1.8867976665496826, + 0.9421432614326477, + 1.578935146331787, + 2.7831475734710693, + 2.6037895679473877, + -0.5597631931304932, + 0.11406822502613068, + 0.2593899667263031, + 2.121243953704834, + -0.1791832000017166, + 0.5773167610168457, + 0.41381677985191345 + ], + [ + -0.6024693250656128, + 0.29007619619369507, + 1.3939473628997803, + 0.3474765419960022, + 1.1848478317260742, + 3.127882719039917, + 3.3329083919525146, + -1.2855982780456543, + 0.20972535014152527, + 1.8329110145568848, + 1.267270803451538, + 1.709198236465454, + 2.927372455596924, + 2.9659509658813477, + -0.7555317282676697, + 0.015305742621421814, + 0.6009905338287354, + 2.188507556915283, + 0.07961152493953705, + 0.4547080397605896, + 0.20378917455673218 + ], + [ + -1.2702982425689697, + 0.06162632256746292, + 0.9447290897369385, + 0.837199330329895, + 0.5848468542098999, + 3.4801836013793945, + 2.8138537406921387, + -1.5083554983139038, + -0.45768383145332336, + 2.08561110496521, + 1.4951050281524658, + 1.8484808206558228, + 2.5617129802703857, + 2.942486047744751, + -0.562404215335846, + 0.17441512644290924, + 0.7880480289459229, + 1.8807158470153809, + -0.043379828333854675, + 0.39090824127197266, + 0.43096962571144104 + ] + ], + "predicted_scores": [ + 2.9776570796966553, + 2.588106393814087, + 2.4356861114501953, + 2.3696303367614746, + 2.3323519229888916, + 2.198922634124756, + 2.1822986602783203, + 2.0857701301574707 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "bf24d506cf320effed2cd7faf94d8618b0626bf8d5c3abc1f837513597c18231", + "stored_base_utility": 0.15946325659751892, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0703698992729187, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0703698992729187 + }, + { + "contact_quality": null, + "progress": 0.06225750967860222, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06225750967860222 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06159693002700806, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06159693002700806 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 0.0703698992729187, + 0.06225750967860222, + 1.0, + 0.06159693002700806, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-6d46dae312649c977b64", + "ms-ebacae0135af2832dfbb", + "ms-6d46dae312649c977b64", + "ms-003a2225c0892dcda683", + "ms-3d090ac2459c4b891dd1", + "ms-ebacae0135af2832dfbb", + "ms-003a2225c0892dcda683" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + true, + true, + true, + false, + false, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "generated_tangents": [ + [ + -0.0733477771282196, + 0.2440221756696701, + 0.15181493759155273, + -0.41763055324554443, + 1.9432990550994873, + 0.5350282192230225, + -0.27695566415786743, + 0.25723034143447876, + -0.06783943623304367, + 0.3047679364681244, + 0.3456987142562866, + 0.3823747932910919, + -0.4067792296409607, + 0.14749783277511597, + -0.8008304238319397, + 0.11215963959693909, + 1.2942795753479004, + 0.21942253410816193, + -0.0323183536529541, + 1.8373687267303467, + -0.3271423280239105 + ], + [ + -0.06810550391674042, + 0.06187015026807785, + 0.49137261509895325, + -0.3761376738548279, + 1.8276028633117676, + 0.30036407709121704, + -0.24131083488464355, + 0.08209671080112457, + 0.06700944900512695, + 0.12063291668891907, + 0.3343655467033386, + 0.6434705257415771, + -0.5821419358253479, + -0.26108187437057495, + -0.8045896887779236, + 0.19552746415138245, + 1.0714962482452393, + 0.08714357018470764, + -0.18119210004806519, + 1.9759488105773926, + -0.0006811916828155518 + ], + [ + 0.0842127725481987, + 0.17406752705574036, + 0.5474393367767334, + -0.3364284634590149, + 1.5966274738311768, + 0.23887129127979279, + -0.5160813927650452, + 0.2524411678314209, + -0.032035455107688904, + 0.11147579550743103, + 0.493985116481781, + 0.5396754145622253, + -0.6150932908058167, + -0.06750963628292084, + -0.9069730043411255, + 0.20672529935836792, + 1.2836986780166626, + 0.11550749093294144, + -0.04422783851623535, + 2.0371110439300537, + -0.03447459638118744 + ], + [ + -0.13922162353992462, + 0.059459343552589417, + 0.483210027217865, + -0.43105340003967285, + 1.721055507659912, + 0.39234158396720886, + -0.17947715520858765, + -0.006537705659866333, + -0.02469947747886181, + 0.14277306199073792, + 0.2891971170902252, + 0.6004711389541626, + -0.4498223066329956, + -0.2213577777147293, + -0.7869839668273926, + 0.197745680809021, + 1.1171355247497559, + 0.02183094620704651, + -0.21498942375183105, + 2.0113275051116943, + 0.01405133306980133 + ], + [ + -0.1677147001028061, + 0.025322608649730682, + 0.4403640031814575, + -0.4360009431838989, + 1.7552154064178467, + 0.3358663320541382, + -0.004117816686630249, + 0.104227215051651, + -0.0601222962141037, + 0.18684738874435425, + 0.27332305908203125, + 0.43872106075286865, + -0.3200552463531494, + -0.2029048055410385, + -0.8302770853042603, + 0.13667984306812286, + 1.139520525932312, + 0.011928841471672058, + -0.27651447057724, + 2.025801420211792, + 0.023210197687149048 + ], + [ + -0.07179400324821472, + 0.09151734411716461, + 0.48320209980010986, + -0.40990766882896423, + 1.9464759826660156, + 0.1472827047109604, + -0.2739640474319458, + 0.032065972685813904, + -0.07190819829702377, + 0.16847270727157593, + 0.33975619077682495, + 0.5009675025939941, + -0.35624414682388306, + -0.31571322679519653, + -0.789415180683136, + 0.13318884372711182, + 1.192352056503296, + -0.0014500021934509277, + -0.15416264533996582, + 1.934866189956665, + 0.005992040038108826 + ], + [ + -0.17290262877941132, + 0.07997788488864899, + 0.4743005335330963, + -0.4328017830848694, + 1.936750888824463, + 0.28863662481307983, + -0.15091127157211304, + 0.08707568049430847, + -0.025879545137286186, + 0.14880897104740143, + 0.29611441493034363, + 0.45149537920951843, + -0.4635292589664459, + -0.24268673360347748, + -0.7780963182449341, + 0.1647162288427353, + 1.183290719985962, + -0.004053756594657898, + -0.2848284840583801, + 1.9144017696380615, + -0.0437970906496048 + ], + [ + -0.2155911773443222, + 0.04945673048496246, + 0.4276820719242096, + -0.4204087257385254, + 1.80472993850708, + 0.3480533957481384, + 0.025602787733078003, + 0.07459354400634766, + -0.050504542887210846, + 0.14902879297733307, + 0.27028581500053406, + 0.3612027168273926, + -0.3037831783294678, + -0.20018760859966278, + -0.8478841781616211, + 0.1292649358510971, + 1.168738842010498, + -0.0031080394983291626, + -0.3887249231338501, + 2.0357699394226074, + -0.007598087191581726 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 0.0703698992729187, + 0.06225750967860222, + 2.0, + 0.06159693002700806, + 2.0 + ], + "group_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20931994915008545, + 2.0, + 2.0, + 0.2008315920829773, + 0.09476469457149506, + 0.05857282876968384, + 0.10009106248617172, + 0.07648500055074692, + 0.05889680236577988, + 0.014406224712729454, + 0.09262995421886444, + 0.15997587144374847, + 0.04682888835668564, + 0.09751574695110321, + 0.06459556519985199, + 0.20335909724235535 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.7537296414375305, + 0.4158197343349457, + 1.4326180219650269, + -1.4107877016067505, + 2.9579341411590576, + 0.9669660925865173, + 1.4339183568954468, + 0.5987398624420166, + -0.2018759399652481, + 0.7105501890182495, + -0.03802938759326935, + 1.3799947500228882, + -0.6568997502326965, + -0.37473973631858826, + -1.857657790184021, + 0.038770925253629684, + 1.931185007095337, + 0.6317195296287537, + 0.1380508691072464, + 2.954265832901001, + 2.9641003608703613 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8678367137908936, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.7494794726371765, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.928200721740723 + ], + [ + 0.6226351857185364, + 0.34334811568260193, + 1.492361068725586, + -1.5917733907699585, + 3.1334922313690186, + 1.0313001871109009, + 1.2075668573379517, + 0.3046378493309021, + -0.11550238728523254, + 0.4499495327472687, + 0.16646838188171387, + 1.5877203941345215, + -0.6488803625106812, + -0.5760968327522278, + -1.8349132537841797, + 0.14188142120838165, + 2.0770444869995117, + 0.5806879997253418, + 0.23913151025772095, + 2.8127381801605225, + 3.0649962425231934 + ], + [ + 0.5975998044013977, + 0.722756028175354, + 1.7548110485076904, + -1.166775107383728, + 2.6060047149658203, + 0.6675758361816406, + 1.125187635421753, + 0.3373170793056488, + -0.1842741072177887, + 0.8186168670654297, + -0.31481224298477173, + 1.2082113027572632, + -0.9804694652557373, + -0.14131735265254974, + -1.700500249862671, + 0.07075637578964233, + 1.5488730669021606, + 0.549576461315155, + 0.034750498831272125, + 3.227431297302246, + 2.9464566707611084 + ], + [ + 0.844982922077179, + 0.7132244110107422, + 1.4430484771728516, + -1.3331272602081299, + 3.1703803539276123, + 1.1851513385772705, + 1.355688452720642, + 0.6115275025367737, + 0.2616361379623413, + 1.2030918598175049, + 0.05625858157873154, + 1.6050300598144531, + -0.7155078053474426, + -0.021488159894943237, + -2.073806047439575, + 0.5387547016143799, + 1.6223118305206299, + 0.7140727043151855, + -0.06027664244174957, + 2.7503156661987305, + 2.6344075202941895 + ], + [ + 0.5821706056594849, + 0.78206866979599, + 1.8119354248046875, + -0.8252442479133606, + 3.038909435272217, + 1.3900940418243408, + 1.7462186813354492, + 0.005083680152893066, + -0.27262064814567566, + 0.867202877998352, + -0.5555376410484314, + 1.172202706336975, + -1.2016760110855103, + -0.06355220079421997, + -2.2925045490264893, + 0.6078628301620483, + 1.7895314693450928, + 0.8720078468322754, + 0.472573459148407, + 3.377012014389038, + 2.6039178371429443 + ], + [ + 0.3789674639701843, + 0.3270051181316376, + 1.056584358215332, + -2.0788497924804688, + 2.948819875717163, + 1.6022355556488037, + 0.8599810004234314, + 0.9570736885070801, + -0.6380544900894165, + 0.3103100061416626, + -0.42896610498428345, + 1.7150276899337769, + -0.8975871205329895, + -0.278381884098053, + -1.305256724357605, + 0.3637259304523468, + 1.8004443645477295, + 0.16446983814239502, + 0.19097819924354553, + 3.43778657913208, + 3.538135528564453 + ], + [ + 0.6903814673423767, + 1.1327197551727295, + 1.8523926734924316, + -2.0428731441497803, + 3.1870431900024414, + 0.5352929830551147, + 0.6004904508590698, + 1.2120530605316162, + 0.16280107200145721, + 1.119707465171814, + 0.7612010836601257, + 0.7890661954879761, + -1.3447637557983398, + -0.1703537553548813, + -1.1010353565216064, + 0.32831913232803345, + 1.5534476041793823, + -0.2119135856628418, + -0.562728762626648, + 2.4702467918395996, + 2.5528204441070557 + ], + [ + 0.9177048206329346, + -0.04751536250114441, + 1.4819984436035156, + -2.1542677879333496, + 3.522390842437744, + 1.8177759647369385, + 2.2213237285614014, + 0.8223017454147339, + -0.1973809152841568, + -0.053272128105163574, + -0.5862784385681152, + 0.6421516537666321, + -0.8713911771774292, + -0.9495017528533936, + -1.6454768180847168, + -0.38891172409057617, + 1.648432970046997, + 1.4951133728027344, + 0.5893608331680298, + 2.449934482574463, + 3.419659376144409 + ], + [ + 1.2956775426864624, + -0.3119873106479645, + 1.5930066108703613, + -1.7682520151138306, + 2.4385876655578613, + 1.4679458141326904, + 1.4536705017089844, + 0.31068816781044006, + 0.660710871219635, + 0.8104690909385681, + 0.483506441116333, + 2.1112046241760254, + -0.22927385568618774, + 0.17858651280403137, + -2.485126495361328, + -0.9187790155410767, + 1.2810213565826416, + 1.3727718591690063, + 0.44044917821884155, + 3.4018898010253906, + 3.39054799079895 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0012856721878051758, + 0.00795695185661316, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + 0.11055737733840942, + 0.0, + 0.02143383026123047, + 0.0025591105222702026, + -0.030314385890960693, + -0.0125969760119915, + 0.3799947500228882, + 0.0076233744621276855, + 0.0, + -0.857657790184021, + 0.013764806091785431, + 0.9311850070953369, + 0.07110422849655151, + 0.03423507511615753, + 1.954265832901001, + 0.0 + ], + [ + -0.24627035856246948, + -0.16095152497291565, + 0.43261802196502686, + -0.4107877016067505, + 1.9579341411590576, + -0.023127973079681396, + 0.0, + -0.24312204122543335, + 0.1389055997133255, + 0.11457735300064087, + 0.06618259102106094, + 0.3799947500228882, + 0.07616078853607178, + 0.0, + -0.857657790184021, + 0.09948430210351944, + 0.9311850070953369, + 0.006249547004699707, + -0.33439844846725464, + 1.954265832901001, + 0.0 + ] + ], + "predicted_scores": [ + 2.724388599395752, + 2.2137644290924072, + 1.8454216718673706, + 1.8319257497787476, + 1.8158799409866333, + 1.7394673824310303, + 1.7313783168792725, + 1.690064549446106 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "534717141", + "selected_index": 0, + "state_hash": "5318995ba859471b8678d6594274c7c0565a1ccfd83ea302378cc2884dc5cc32", + "stored_base_utility": 0.20931994915008545, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.14809735119342804, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.14809735119342804 + }, + "base_progress": 0.14809735119342804, + "base_success": false, + "base_utility": 0.14809735119342804, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.002109348773956299, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.002109348773956299 + }, + { + "contact_quality": null, + "progress": 0.0016142526874318719, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0016142526874318719 + }, + { + "contact_quality": null, + "progress": 0.001568953157402575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.001568953157402575 + }, + { + "contact_quality": null, + "progress": 0.0013637542724609375, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013637542724609375 + }, + { + "contact_quality": null, + "progress": 0.0013946691760793328, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013946691760793328 + }, + { + "contact_quality": null, + "progress": 0.0013481974601745605, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0013481974601745605 + }, + { + "contact_quality": null, + "progress": 0.0006553332204930484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0006553332204930484 + }, + { + "contact_quality": null, + "progress": 0.0024978716392070055, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0024978716392070055 + } + ], + "candidate_progress": [ + 0.002109348773956299, + 0.0016142526874318719, + 0.001568953157402575, + 0.0013637542724609375, + 0.0013946691760793328, + 0.0013481974601745605, + 0.0006553332204930484, + 0.0024978716392070055 + ], + "candidate_source_chart_ids": [ + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-0e592951fa32139b6dc9", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-84388e63928e1a30ffc4", + "ms-f2b2b2a1aa2863c3676f", + "ms-84388e63928e1a30ffc4" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-da142025733b53fcf7eb", + "generated_tangents": [ + [ + -0.16491813957691193, + -0.44812941551208496, + 1.522404432296753, + -0.29195141792297363, + 1.6155595779418945, + 3.036224126815796, + 2.4263293743133545, + -0.11942023038864136, + 0.156746506690979, + 1.2404955625534058, + 0.30534762144088745, + 0.9937582612037659, + 1.7859283685684204, + 1.3082846403121948, + -1.1007142066955566, + -0.2226385921239853, + 1.4320591688156128, + 0.6443809866905212, + -0.10069802403450012, + 2.5378925800323486, + 0.788082480430603 + ], + [ + -0.25906503200531006, + -0.4311758279800415, + 1.550649642944336, + -0.10462211817502975, + 1.5163681507110596, + 3.0287973880767822, + 1.9991042613983154, + -0.08224475383758545, + 0.13758185505867004, + 1.187612533569336, + 0.1380847841501236, + 1.0298229455947876, + 1.7279993295669556, + 1.705025315284729, + -1.2922234535217285, + -0.11821980774402618, + 1.7057580947875977, + 0.9846886992454529, + 0.40484148263931274, + 1.9227759838104248, + 1.367584466934204 + ], + [ + -0.3099903464317322, + -0.22394734621047974, + 1.6123816967010498, + -0.0927695482969284, + 2.086507797241211, + 3.3660175800323486, + 1.9148619174957275, + -0.24106869101524353, + 0.11848428845405579, + 1.1820400953292847, + 0.14725540578365326, + 0.9857961535453796, + 1.5172349214553833, + 1.7305700778961182, + -1.1125696897506714, + -0.21418099105358124, + 1.6485706567764282, + 0.9478051662445068, + 0.3282330632209778, + 2.321153163909912, + 1.6825212240219116 + ], + [ + -0.4207296371459961, + -0.34084856510162354, + 1.6467245817184448, + 0.06274494528770447, + 1.2211084365844727, + 3.5212364196777344, + 1.7665579319000244, + -0.32972094416618347, + 0.17988735437393188, + 0.9871339797973633, + 0.25226306915283203, + 1.225220799446106, + 1.9555734395980835, + 1.8543727397918701, + -1.2462942600250244, + -0.24397948384284973, + 1.7886381149291992, + 0.6570941805839539, + 0.3053135573863983, + 2.7333998680114746, + 1.375671148300171 + ], + [ + -0.4197947382926941, + -0.41934341192245483, + 1.5252076387405396, + -0.10965941101312637, + 1.530900001525879, + 3.3226230144500732, + 1.6510274410247803, + -0.26396510004997253, + 0.15492278337478638, + 1.2087863683700562, + 0.09346982091665268, + 0.9122577905654907, + 1.489465355873108, + 1.5815447568893433, + -1.225317120552063, + -0.16869446635246277, + 1.6537623405456543, + 0.7922667860984802, + 0.24121429026126862, + 2.372321844100952, + 1.6607167720794678 + ], + [ + -0.4949355125427246, + -0.41885125637054443, + 1.547715425491333, + 0.06952869892120361, + 1.609220266342163, + 3.6366617679595947, + 1.5721161365509033, + -0.20115438103675842, + 0.13943473994731903, + 1.2018177509307861, + 0.24125489592552185, + 0.9632784128189087, + 1.3610948324203491, + 1.7838205099105835, + -1.24807870388031, + -0.2722555100917816, + 1.6444717645645142, + 0.823582649230957, + 0.42304664850234985, + 2.1840851306915283, + 1.7703300714492798 + ], + [ + -0.4479713439941406, + -0.3628988265991211, + 1.6345138549804688, + 0.14003247022628784, + 1.279918909072876, + 3.654670238494873, + 1.973222017288208, + -0.5072991847991943, + 0.24864208698272705, + 1.4242440462112427, + 0.22237750887870789, + 1.0343232154846191, + 1.5170308351516724, + 1.8595339059829712, + -1.446240782737732, + -0.4530695378780365, + 1.9973170757293701, + 1.062402367591858, + 0.13227444887161255, + 2.6164281368255615, + 1.4536553621292114 + ], + [ + -0.2238340526819229, + -0.27119576930999756, + 1.3281177282333374, + -0.07311345636844635, + 1.2926609516143799, + 2.861276626586914, + 1.480619192123413, + -0.04855930805206299, + 0.1650649607181549, + 1.0754681825637817, + 0.6095993518829346, + 0.8284340500831604, + 1.1569561958312988, + 2.1539785861968994, + -1.074147343635559, + -0.17524687945842743, + 1.7277039289474487, + 0.9363681674003601, + 0.37941500544548035, + 2.6101033687591553, + 1.175939917564392 + ] + ], + "generated_utilities": [ + 0.002109348773956299, + 0.0016142526874318719, + 0.001568953157402575, + 0.0013637542724609375, + 0.0013946691760793328, + 0.0013481974601745605, + 0.0006553332204930484, + 0.0024978716392070055 + ], + "group_id": "ms-da142025733b53fcf7eb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.14843207597732544, + 0.1717694252729416, + 0.09356185048818588, + 0.23231086134910583, + 0.19341367483139038, + 0.04822012037038803, + 0.23024770617485046, + 0.16561973094940186, + 0.2643269896507263, + 0.12403462827205658, + 0.12689702212810516, + 0.16619956493377686, + 0.13481859862804413, + 0.07537531852722168, + 0.11867459863424301, + 0.04740651696920395 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.04848463833332062, + -0.18164527416229248, + 0.6558789014816284, + 0.038211286067962646, + 0.08492374420166016, + 2.265157699584961, + 0.0, + -0.09011825919151306, + 0.0018594712018966675, + 0.6884076595306396, + 0.027596401050686836, + 0.06117302179336548, + 1.278116226196289, + 0.0, + -0.3951960802078247, + -0.08061015605926514, + 1.07906174659729, + 0.3229694366455078, + -0.21076995134353638, + 1.8564467430114746, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.0903544425964355, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.570276260375977, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.736253261566162 + ], + [ + 0.08095964789390564, + -0.7926032543182373, + 1.571103811264038, + 0.07894467562437057, + 0.2701656222343445, + 3.806621789932251, + 2.203669786453247, + -0.042464032769203186, + 0.7168105244636536, + 1.8671371936798096, + -0.30838707089424133, + 1.7242767810821533, + 1.8847527503967285, + 1.305934190750122, + -0.42665380239486694, + -0.2973874807357788, + 1.1881210803985596, + 1.6921244859695435, + 1.070060133934021, + 2.067784309387207, + 1.916501760482788 + ], + [ + 0.7620636224746704, + -2.1605899333953857, + 2.1443262100219727, + 0.17886832356452942, + 1.6259945631027222, + 3.787412166595459, + 1.3663169145584106, + 0.49799659848213196, + -0.6301277279853821, + 2.275850534439087, + -0.6735645532608032, + 1.7504758834838867, + 2.7512195110321045, + 1.7462939023971558, + -0.6835377216339111, + 0.36623528599739075, + 1.405979871749878, + 0.6472548842430115, + 1.4791045188903809, + 2.7838869094848633, + 3.0856707096099854 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.29792410135269165, + -0.30587875843048096, + 0.6558789014816284, + -0.4171115756034851, + -0.1733894944190979, + 2.265157699584961, + 0.0, + 0.14607194066047668, + 0.05651267617940903, + 0.6884076595306396, + -0.0433836504817009, + 0.07522618770599365, + 1.278116226196289, + 0.0, + -0.41462820768356323, + 0.26706579327583313, + 1.07906174659729, + 0.44956159591674805, + -0.37072819471359253, + 1.8564467430114746, + 0.0 + ], + [ + 0.18432100117206573, + -0.9473621249198914, + 1.5389783382415771, + -0.041053857654333115, + 0.6282865405082703, + 3.276425838470459, + 2.1839330196380615, + -0.3612370491027832, + 0.23081421852111816, + 1.8094429969787598, + 0.10581490397453308, + 0.8865737915039062, + 2.5146288871765137, + 2.2134878635406494, + -1.6751070022583008, + -0.08268362283706665, + 2.303063154220581, + 1.3061904907226562, + 0.4770132899284363, + 3.109205722808838, + 2.509927272796631 + ], + [ + -0.5288646817207336, + -1.5067847967147827, + 1.376499056816101, + 0.13231416046619415, + 1.048660159111023, + 3.5048489570617676, + 1.9678237438201904, + -0.14848726987838745, + 0.05475667119026184, + 1.5529061555862427, + 0.1597936749458313, + 0.7318928241729736, + 2.17749285697937, + 2.0782582759857178, + -1.5473355054855347, + -0.6715520620346069, + 1.7043644189834595, + 1.1190917491912842, + 0.4214729070663452, + 3.3400330543518066, + 2.3147284984588623 + ] + ], + "predicted_scores": [ + 1.2798094749450684, + 1.2566577196121216, + 1.0953518152236938, + 0.5736859440803528, + 0.4687597155570984, + 0.43638235330581665, + -0.01489083468914032, + -0.2645830512046814 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "141984615", + "selected_index": 0, + "state_hash": "0026cc5b3762b520baadbda7bf138a2d24fffa5b2306f245ddbc1d1556cccc22", + "stored_base_utility": 0.14843207597732544, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.1362423449754715, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1362423449754715 + }, + "base_progress": 0.1362423449754715, + "base_success": false, + "base_utility": 0.1362423449754715, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01934593915939331, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01934593915939331 + }, + { + "contact_quality": null, + "progress": 0.018479665741324425, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018479665741324425 + }, + { + "contact_quality": null, + "progress": 0.014796634204685688, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.014796634204685688 + }, + { + "contact_quality": null, + "progress": 0.016739627346396446, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016739627346396446 + }, + { + "contact_quality": null, + "progress": 0.011937320232391357, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.011937320232391357 + }, + { + "contact_quality": null, + "progress": 0.009798427112400532, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009798427112400532 + }, + { + "contact_quality": null, + "progress": 0.04993142560124397, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04993142560124397 + }, + { + "contact_quality": null, + "progress": 0.04576842114329338, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04576842114329338 + } + ], + "candidate_progress": [ + 0.01934593915939331, + 0.018479665741324425, + 0.014796634204685688, + 0.016739627346396446, + 0.011937320232391357, + 0.009798427112400532, + 0.04993142560124397, + 0.04576842114329338 + ], + "candidate_source_chart_ids": [ + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-7e88428497a6b13936c4", + "ms-a2518826fa0a816cc46f", + "ms-cf4b63d109a88a4c42bf", + "ms-76f27d9a935b25e15c38", + "ms-76f27d9a935b25e15c38" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-e2bad6e99e8a281207cc", + "generated_tangents": [ + [ + -0.808625340461731, + -0.3958311676979065, + 1.4513057470321655, + 0.10015183687210083, + 1.8970415592193604, + 3.1691267490386963, + 1.6807830333709717, + 0.496963232755661, + -0.593037486076355, + 0.3745551109313965, + 0.04373528063297272, + 0.5483255982398987, + -0.059080034494400024, + -0.17493756115436554, + 0.0818648487329483, + 0.27815714478492737, + 0.11154191941022873, + -1.379705786705017, + 0.6501579284667969, + -2.061927080154419, + -1.9550199508666992 + ], + [ + -0.7248064279556274, + -0.4604642987251282, + 1.5598711967468262, + 0.005166083574295044, + 1.815105676651001, + 3.014575481414795, + 1.7203545570373535, + 0.42555975914001465, + -0.6206048727035522, + 0.3805982768535614, + 0.001539815217256546, + 0.6111949682235718, + -0.14247506856918335, + 0.04190334677696228, + 0.14704649150371552, + 0.2566608488559723, + 0.07439719140529633, + -1.3993266820907593, + 0.6737163066864014, + -1.9276411533355713, + -1.82131826877594 + ], + [ + -0.40650177001953125, + -0.49699872732162476, + 1.4829405546188354, + 0.13907760381698608, + 1.8399548530578613, + 3.485158920288086, + 1.5967557430267334, + 0.41830646991729736, + -0.5379894375801086, + 0.3360911011695862, + 0.09948419779539108, + 0.812816858291626, + 0.22150996327400208, + 0.03305993974208832, + 0.06606549769639969, + 0.23715467751026154, + 0.42798539996147156, + -1.5503917932510376, + 1.3109371662139893, + -1.9463622570037842, + -1.5908902883529663 + ], + [ + -0.6315448880195618, + -0.4405671954154968, + 1.6896828413009644, + -0.003717586398124695, + 1.7352089881896973, + 3.074761152267456, + 1.769089698791504, + 0.4281931519508362, + -0.517143726348877, + 0.36459216475486755, + -0.03713090717792511, + 0.5744054913520813, + -0.23416925966739655, + 0.06707154214382172, + 0.1249435693025589, + 0.22939813137054443, + 0.18297842144966125, + -1.3903321027755737, + 0.5762546062469482, + -1.9213354587554932, + -2.0142428874969482 + ], + [ + -0.6389257311820984, + -0.443348228931427, + 1.5659390687942505, + -0.24717013537883759, + 2.3528077602386475, + 3.363032817840576, + 1.5654137134552002, + 0.32284098863601685, + -0.2637955844402313, + 0.5328659415245056, + 0.2923922538757324, + 0.7081804275512695, + -0.3174516558647156, + 0.1322580873966217, + 0.31917423009872437, + -0.058363039046525955, + 0.27812865376472473, + -1.2366472482681274, + 0.4684528112411499, + -1.9222798347473145, + -1.19365656375885 + ], + [ + -0.538007915019989, + -0.3468935489654541, + 1.5260671377182007, + -0.30296847224235535, + 1.7538366317749023, + 3.3511900901794434, + 1.2871084213256836, + -0.11565133929252625, + -0.16260549426078796, + 0.7333114147186279, + 0.0004078671336174011, + 0.3794543743133545, + -1.0234380960464478, + 0.5007558465003967, + 0.5275300145149231, + 0.27768683433532715, + 0.3729734718799591, + -1.0702588558197021, + -0.10718375444412231, + -1.9243483543395996, + -1.7530497312545776 + ], + [ + -1.092842936515808, + -0.3846888542175293, + 1.2500170469284058, + -0.7387838363647461, + 2.329988479614258, + 3.2617266178131104, + 2.0534462928771973, + 0.7335795164108276, + -0.32580500841140747, + 0.10699981451034546, + 0.121836356818676, + 0.4568679630756378, + -0.5413341522216797, + 0.11316933482885361, + 0.13814303278923035, + 0.15308858454227448, + 0.33005163073539734, + -1.6212522983551025, + 0.009338468313217163, + -1.193364143371582, + -1.317259669303894 + ], + [ + -1.0184271335601807, + -0.3951433300971985, + 1.2153401374816895, + -0.7337980270385742, + 2.3822622299194336, + 3.210197925567627, + 2.015505075454712, + 0.7741807699203491, + -0.30517974495887756, + 0.10034145414829254, + 0.2205776870250702, + 0.3355855345726013, + -0.5925508141517639, + 0.2985408902168274, + 0.11383157968521118, + 0.17892028391361237, + 0.2655179798603058, + -1.67020845413208, + 0.06308495998382568, + -1.019197702407837, + -1.18149733543396 + ] + ], + "generated_utilities": [ + 0.01934593915939331, + 0.018479665741324425, + 0.014796634204685688, + 0.016739627346396446, + 0.011937320232391357, + 0.009798427112400532, + 0.04993142560124397, + 0.04576842114329338 + ], + "group_id": "ms-e2bad6e99e8a281207cc", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.13993772864341736, + 0.16482847929000854, + 0.20825156569480896, + 0.16188287734985352, + 0.13454246520996094, + 0.14753298461437225, + 0.14504572749137878, + 0.06676968187093735, + 0.09772863239049911, + 0.0585557222366333, + 0.12863430380821228, + 0.10292482376098633, + 0.07993738353252411, + 0.03641533851623535, + 0.04985213279724121, + 0.1516878604888916 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + -1.3249012231826782, + -0.9610954523086548, + 2.054325819015503, + 1.018629789352417, + 2.393625497817993, + 2.2159433364868164, + 2.098443031311035, + -0.49140021204948425, + 0.044472046196460724, + 0.8045110702514648, + 1.4511574506759644, + -0.3077920377254486, + -0.052587300539016724, + 0.036380887031555176, + 0.7875679731369019, + -0.3928021192550659, + -0.09456783533096313, + -1.4901847839355469, + 1.1711406707763672, + -2.1349995136260986, + -2.076193332672119 + ], + [ + -1.4271517992019653, + -1.114187479019165, + 1.618905782699585, + 1.0621737241744995, + 1.6005314588546753, + 3.017716407775879, + 2.3444156646728516, + -1.0368298292160034, + 0.6155713796615601, + 1.0049463510513306, + 1.3708372116088867, + -0.27396243810653687, + -0.19098177552223206, + -0.5735776424407959, + 0.863328218460083, + -0.4340485632419586, + 0.09502258896827698, + -1.8856427669525146, + 0.9514620304107666, + -2.3912711143493652, + -2.6571128368377686 + ], + [ + -1.0088777542114258, + -1.803802251815796, + 2.6037826538085938, + 0.763943612575531, + 1.7241847515106201, + 2.433204174041748, + 2.273603916168213, + 0.06359493732452393, + -0.17274422943592072, + -0.23299545049667358, + 1.7126038074493408, + -0.3797045052051544, + 0.7567765712738037, + -0.9981052875518799, + 1.0371832847595215, + -0.5120593309402466, + -0.059397101402282715, + -1.9757622480392456, + 0.7378945350646973, + -2.6195693016052246, + -2.178041458129883 + ], + [ + -0.09093856811523438, + -0.14735746383666992, + 1.350311517715454, + 1.477137565612793, + 2.134960174560547, + 1.7331886291503906, + 2.24615740776062, + -1.7148966789245605, + -0.3949584364891052, + 0.4433448612689972, + 0.9034961462020874, + -0.23445925116539001, + -0.6371403932571411, + -0.038229942321777344, + 2.010648250579834, + 0.14472149312496185, + -0.26303571462631226, + -2.3003270626068115, + 0.565727949142456, + -2.586710214614868, + -2.098479747772217 + ], + [ + -1.8506121635437012, + -1.4700038433074951, + 2.4313056468963623, + 1.6264371871948242, + 1.1548577547073364, + 2.9678268432617188, + 2.990251302719116, + -0.25485146045684814, + -0.36889538168907166, + -0.14847010374069214, + 0.6971698999404907, + -0.6752342581748962, + 0.44545215368270874, + 0.663374125957489, + 0.6998850107192993, + -0.6500043869018555, + -0.04876291751861572, + -2.446995258331299, + 0.7760334014892578, + -1.38918137550354, + -1.5372083187103271 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.053252458572387695, + -0.006605625152587891, + 1.0103552341461182, + 0.25736117362976074, + 1.010573148727417, + 1.4690883159637451, + 0.0, + -0.0840916633605957, + -0.24044206738471985, + -0.06801849603652954, + 0.061444878578186035, + 0.11697901785373688, + 0.03979811072349548, + 0.0, + 0.054485201835632324, + -0.11758939921855927, + 0.31161248683929443, + -0.602575421333313, + -0.021049559116363525, + -1.3257782459259033, + 0.0 + ] + ], + "predicted_scores": [ + 3.989610195159912, + 3.801568031311035, + 3.771143674850464, + 3.5158884525299072, + 2.6187055110931396, + 2.271235466003418, + 1.9595327377319336, + 1.9518026113510132 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1549324805", + "selected_index": 0, + "state_hash": "f054ce13f25dcda8808ebde2bee98ca84bfbdc3fc328a0eb831e21faada3fa4c", + "stored_base_utility": 0.13993772864341736, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09695577621459961, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09695577621459961 + }, + "base_progress": 0.09695577621459961, + "base_success": false, + "base_utility": 0.09695577621459961, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.027097145095467567, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027097145095467567 + }, + { + "contact_quality": null, + "progress": 0.02346259355545044, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02346259355545044 + }, + { + "contact_quality": null, + "progress": 0.04263921454548836, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04263921454548836 + }, + { + "contact_quality": null, + "progress": 0.030199527740478516, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.030199527740478516 + }, + { + "contact_quality": null, + "progress": 0.02752045728266239, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02752045728266239 + }, + { + "contact_quality": null, + "progress": 0.027290603145956993, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.027290603145956993 + }, + { + "contact_quality": null, + "progress": 0.021321892738342285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021321892738342285 + }, + { + "contact_quality": null, + "progress": 0.03452460095286369, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03452460095286369 + } + ], + "candidate_progress": [ + 0.027097145095467567, + 0.02346259355545044, + 0.04263921454548836, + 0.030199527740478516, + 0.02752045728266239, + 0.027290603145956993, + 0.021321892738342285, + 0.03452460095286369 + ], + "candidate_source_chart_ids": [ + "ms-206099bab6bfe7c84fce", + "ms-6e27d292a9cbea108fc9", + "ms-6e27d292a9cbea108fc9", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b", + "ms-b334dd4c101f3bd0476b" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e30fd80adc089e9f697a", + "generated_tangents": [ + [ + 0.3590228855609894, + 0.19683429598808289, + -0.27843567728996277, + 0.6162688732147217, + 0.5412493944168091, + -0.801476240158081, + -1.189948320388794, + 0.00704619288444519, + 0.04275801032781601, + 0.010225243866443634, + 0.9955396056175232, + 0.3697530925273895, + -0.13494029641151428, + 0.037109315395355225, + -0.3661476969718933, + 0.08389641344547272, + 0.3658115267753601, + -0.05796189606189728, + -0.08679118752479553, + 1.0497918128967285, + 0.28130897879600525 + ], + [ + 0.35214918851852417, + 0.19684278964996338, + -0.23120197653770447, + 0.6600933074951172, + 0.6702309250831604, + -1.220144510269165, + -0.4723138213157654, + -0.021933957934379578, + 0.15916015207767487, + 0.03964683413505554, + 0.8672034740447998, + 0.04386952519416809, + 0.056779175996780396, + -0.014775574207305908, + -0.3643971085548401, + 0.028458911925554276, + 0.15976694226264954, + 0.07386019825935364, + -0.35446876287460327, + 1.2123723030090332, + 0.36295968294143677 + ], + [ + 0.387315034866333, + 0.17117254436016083, + -0.18575212359428406, + 0.56825852394104, + 0.7309894561767578, + -0.774138331413269, + -0.5184234976768494, + 0.0943412333726883, + 0.03323892503976822, + 0.10794530808925629, + 0.9891995787620544, + 0.020437508821487427, + 0.3686356246471405, + 0.3178538680076599, + -0.6662771105766296, + 0.03811803087592125, + 0.28423410654067993, + 0.26532724499702454, + -0.41444945335388184, + 1.1595392227172852, + 0.3615068197250366 + ], + [ + 0.4227678179740906, + 0.3649612069129944, + -0.14785698056221008, + 0.3151308000087738, + 0.7339253425598145, + -0.6750684976577759, + -0.26398754119873047, + -0.16428720951080322, + -0.05128262937068939, + 0.5043823719024658, + 0.8338589072227478, + 0.3655779957771301, + 0.9708422422409058, + 0.8090587258338928, + -0.21247264742851257, + -0.0019111288711428642, + 0.06804615259170532, + 0.17872658371925354, + -0.47385716438293457, + 1.0519821643829346, + 0.6560294032096863 + ], + [ + 0.4932646155357361, + 0.2583072781562805, + -0.13222184777259827, + 0.2968616783618927, + 0.6814801096916199, + -0.9484403133392334, + -0.9983848929405212, + 0.011778086423873901, + 0.016037452965974808, + 0.31290170550346375, + 0.7360706329345703, + 0.3139982223510742, + 1.1989713907241821, + 0.7084675431251526, + -0.1079864501953125, + 0.12537041306495667, + 0.162700355052948, + 0.22526514530181885, + -0.40702927112579346, + 0.9055491089820862, + 0.9361089468002319 + ], + [ + 0.5312609672546387, + 0.18521516025066376, + -0.08911815285682678, + 0.43024638295173645, + 0.895797073841095, + -0.813001275062561, + -1.119739294052124, + 0.03544671833515167, + 0.04468908905982971, + 0.33778032660484314, + 0.656747043132782, + 0.20835548639297485, + 1.2784655094146729, + 0.8251712322235107, + -0.028265830129384995, + 0.13416336476802826, + 0.2592344582080841, + 0.29410091042518616, + -0.394891619682312, + 0.7114847302436829, + 1.0665563344955444 + ], + [ + 0.5456377267837524, + 0.3053399920463562, + -0.2471875250339508, + 0.2321479320526123, + 0.5717474222183228, + -0.9433281421661377, + -0.907658040523529, + 0.08428370207548141, + 0.011065405793488026, + 0.27069997787475586, + 0.6413487792015076, + 0.5000815391540527, + 1.2367584705352783, + 0.6818474531173706, + -0.11301298439502716, + 0.07140019536018372, + 0.11339274048805237, + 0.23209679126739502, + -0.4480735659599304, + 0.9108291268348694, + 0.8850172758102417 + ], + [ + 0.431556761264801, + 0.22277767956256866, + -0.146419495344162, + 0.18227934837341309, + 0.5816071033477783, + -0.9192342758178711, + -1.0169851779937744, + 0.11149691045284271, + -0.018793081864714622, + 0.2590688169002533, + 0.9602459669113159, + 0.4354865550994873, + 1.364359974861145, + 0.503301739692688, + -0.2617972195148468, + 0.09783723950386047, + 0.21418017148971558, + 0.13097012042999268, + -0.2826020121574402, + 1.146355152130127, + 0.9910222291946411 + ] + ], + "generated_utilities": [ + 0.027097145095467567, + 0.02346259355545044, + 0.04263921454548836, + 0.030199527740478516, + 0.02752045728266239, + 0.027290603145956993, + 0.021321892738342285, + 0.03452460095286369 + ], + "group_id": "ms-e30fd80adc089e9f697a", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.12557005882263184, + 0.11941087245941162, + 0.10894306749105453, + 0.060310762375593185, + 0.18920521438121796, + 0.11678676307201385, + 0.14083251357078552, + 0.12988364696502686, + 0.18350017070770264, + 0.1286114901304245, + 0.06744732707738876, + 0.04428015276789665, + 0.08399941772222519, + 0.0742691159248352, + 0.05598282814025879, + 0.09703926742076874 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.044197678565979004, + -0.12298870831727982, + -0.37176433205604553, + 0.5503368377685547, + 0.5411782264709473, + 0.6186751127243042, + 0.0, + 0.28674131631851196, + 0.08053566515445709, + 0.3295105993747711, + 0.8157306909561157, + 0.1715114712715149, + -0.43090736865997314, + 0.0, + -0.5639103651046753, + -0.48903688788414, + 0.7485904693603516, + -0.4474331736564636, + -0.18787053227424622, + 1.2062294483184814, + 0.0 + ], + [ + 1.3153167963027954, + 0.7455203533172607, + -1.0113484859466553, + 2.097999334335327, + 0.7455009818077087, + 0.3843202590942383, + -0.16713692247867584, + -0.8400754928588867, + -0.3845864534378052, + 1.065396785736084, + 1.712794542312622, + 0.803345263004303, + 0.6529266238212585, + 0.6022698879241943, + -1.1222561597824097, + 0.06260442733764648, + 1.101211428642273, + -0.133493572473526, + -0.8626402616500854, + 1.8406728506088257, + 0.8011860847473145 + ], + [ + 0.23532289266586304, + -0.516789972782135, + -0.3119599223136902, + 1.501338005065918, + 1.4930639266967773, + -0.419913113117218, + -0.6910054683685303, + 0.30246037244796753, + 0.041916340589523315, + 1.2685117721557617, + 2.5912232398986816, + 1.2864888906478882, + 0.3821069598197937, + 0.4677737355232239, + -1.84663724899292, + -0.8953573703765869, + 2.3256053924560547, + 0.18948839604854584, + -0.5580936670303345, + 1.528401494026184, + 1.4914395809173584 + ], + [ + 0.6903802752494812, + 0.3962534964084625, + -0.8207939863204956, + 0.5973623991012573, + 1.310564398765564, + 0.6411572694778442, + -0.2032768577337265, + 0.25504088401794434, + 0.6581551432609558, + 0.3938753306865692, + 1.0127826929092407, + 1.6796927452087402, + 0.4867250621318817, + 0.05769670009613037, + -2.026149034500122, + -0.17722094058990479, + 0.9414948225021362, + 0.6044259667396545, + 0.03290170431137085, + 2.9423327445983887, + -0.060306549072265625 + ], + [ + 0.47530579566955566, + -0.6085019111633301, + 0.12691906094551086, + 0.5673632025718689, + 0.08076304197311401, + 0.8255819082260132, + -0.9781221151351929, + -0.796129584312439, + -0.6745100021362305, + 0.7784937024116516, + 1.7613102197647095, + 1.4703174829483032, + -0.36175650358200073, + 0.16939619183540344, + -0.9399533271789551, + -1.2696547508239746, + 1.3314080238342285, + -0.3690394163131714, + -1.0323668718338013, + 2.305772304534912, + 0.27392566204071045 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.6770331859588623, + 0.09077263623476028, + -0.3141186833381653, + 1.5503368377685547, + 1.0133107900619507, + 0.12400172650814056, + -0.2399108111858368, + -0.4914711117744446, + -0.11917559802532196, + 0.7716305255889893, + 1.8157306909561157, + 0.7167991995811462, + 0.5690926313400269, + 0.6154549717903137, + -1.5639103651046753, + -0.5166093111038208, + 1.7485904693603516, + -0.15158627927303314, + -0.46638885140419006, + 2.2062294483184814, + 0.7902159690856934 + ], + [ + 0.9456368684768677, + -0.3964601159095764, + -0.5845282077789307, + 1.2133667469024658, + 0.5843623876571655, + -0.02685314416885376, + -0.006928771734237671, + -0.2159697711467743, + -0.3017333149909973, + 0.40500110387802124, + 1.9734654426574707, + 0.46303731203079224, + 0.33380845189094543, + 0.7758987545967102, + -2.060479164123535, + -0.955610990524292, + 1.942940592765808, + -0.23636659979820251, + -0.10507094860076904, + 1.8367226123809814, + 1.2234702110290527 + ] + ], + "predicted_scores": [ + 6.529160022735596, + 5.073872089385986, + 4.5338897705078125, + 4.301075458526611, + 3.954558849334717, + 3.819385290145874, + 3.693312644958496, + 3.690680980682373 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "585904130", + "selected_index": 0, + "state_hash": "ca8873454d7cf9723b1e743355bf255813544ae3212d80abc954fd347d7adf27", + "stored_base_utility": 0.12557005882263184, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1320568472146988, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1320568472146988 + }, + { + "contact_quality": null, + "progress": 0.12680597603321075, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12680597603321075 + }, + { + "contact_quality": null, + "progress": 0.09484513849020004, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09484513849020004 + }, + { + "contact_quality": null, + "progress": 0.10936561971902847, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10936561971902847 + }, + { + "contact_quality": null, + "progress": 0.11417748779058456, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11417748779058456 + }, + { + "contact_quality": null, + "progress": 0.06823223829269409, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06823223829269409 + }, + { + "contact_quality": null, + "progress": 0.1029265895485878, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1029265895485878 + }, + { + "contact_quality": null, + "progress": 0.0987953171133995, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0987953171133995 + } + ], + "candidate_progress": [ + 0.1320568472146988, + 0.12680597603321075, + 0.09484513849020004, + 0.10936561971902847, + 0.11417748779058456, + 0.06823223829269409, + 0.1029265895485878, + 0.0987953171133995 + ], + "candidate_source_chart_ids": [ + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-19cb10d10516bd0216a6", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-4de1f6d71208bf1d182d", + "ms-37384bede4d1c03f3b80" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "generated_tangents": [ + [ + 1.5705944299697876, + -0.06851592659950256, + -0.0494934618473053, + 0.141649067401886, + 0.3069194555282593, + 1.6204804182052612, + 1.7940168380737305, + 1.8458722829818726, + 0.12396884709596634, + -1.1453535556793213, + -0.4320434629917145, + -0.7400387525558472, + -3.26072096824646, + -2.046830654144287, + 1.4676625728607178, + 0.2676600515842438, + -0.8536955714225769, + -0.2154492288827896, + -0.33907508850097656, + -1.831056833267212, + -1.1493115425109863 + ], + [ + 1.8652468919754028, + -0.2339145839214325, + 0.17751935124397278, + 0.23989439010620117, + -0.0668235719203949, + 2.4834957122802734, + 1.7183148860931396, + 2.470745325088501, + 0.22536756098270416, + -1.5991404056549072, + -0.7004081010818481, + -0.5648285746574402, + -3.7447264194488525, + -2.515866994857788, + 1.5853447914123535, + 0.4201934337615967, + -0.581537127494812, + -0.1651209145784378, + -0.08740833401679993, + -1.26237154006958, + -1.3109184503555298 + ], + [ + 1.6519989967346191, + -0.22999686002731323, + 0.07272452116012573, + 0.17863675951957703, + 0.08630292117595673, + 2.0467116832733154, + 1.782813549041748, + 2.344526529312134, + 0.23404105007648468, + -1.7041015625, + -0.6769210696220398, + -0.682097315788269, + -3.5634939670562744, + -2.8932785987854004, + 1.603251338005066, + 0.3866955041885376, + -0.5121111273765564, + -0.3393757939338684, + -0.1352817416191101, + -1.216221570968628, + -1.4936425685882568 + ], + [ + 1.7114033699035645, + -0.24365893006324768, + 0.013003885746002197, + 0.09338992834091187, + -0.10904493927955627, + 2.304471254348755, + 1.6770527362823486, + 2.308021068572998, + 0.2794436812400818, + -1.54660964012146, + -0.7000381946563721, + -0.7109536528587341, + -3.6303679943084717, + -3.1097657680511475, + 1.5242034196853638, + 0.41823381185531616, + -0.5191966891288757, + -0.3860350251197815, + -0.07287967205047607, + -1.2147479057312012, + -1.6497353315353394 + ], + [ + 1.5470114946365356, + -0.19456952810287476, + 0.2033877670764923, + 0.06865578889846802, + -0.28358471393585205, + 1.8922683000564575, + 1.821535587310791, + 2.2622311115264893, + 0.1230999156832695, + -1.7442768812179565, + -0.6677217483520508, + -0.879165530204773, + -3.289560079574585, + -2.8239214420318604, + 1.560648798942566, + 0.2975308895111084, + -0.5076003074645996, + -0.11996276676654816, + -0.48666930198669434, + -1.3704237937927246, + -1.8300011157989502 + ], + [ + 1.7211253643035889, + -0.31464123725891113, + -0.25204816460609436, + 0.2328798770904541, + 0.4461529850959778, + 2.118340253829956, + 1.7182228565216064, + 2.352573871612549, + 0.29229557514190674, + -1.8145620822906494, + -0.6128995418548584, + -1.046897292137146, + -3.5329716205596924, + -2.6767611503601074, + 1.5114555358886719, + 0.43397367000579834, + -0.4783868193626404, + -0.34383583068847656, + 0.03508239984512329, + -1.3395237922668457, + -1.5731325149536133 + ], + [ + 1.5320279598236084, + -0.33009469509124756, + 0.29068851470947266, + -0.3612316846847534, + 0.2736901342868805, + 0.7711696624755859, + 2.45078706741333, + 1.858219861984253, + 0.18155181407928467, + -1.2958438396453857, + -0.677787184715271, + -0.911828875541687, + -3.1887199878692627, + -2.020345449447632, + 1.0366549491882324, + 0.27170655131340027, + -0.48277270793914795, + -0.322127103805542, + -0.4044927954673767, + -2.291778802871704, + -1.5043164491653442 + ], + [ + 1.6546272039413452, + -0.19025036692619324, + -0.1967904269695282, + 0.2030722200870514, + -0.020738422870635986, + 2.034555673599243, + 2.197772264480591, + 2.3154332637786865, + 0.26576685905456543, + -1.5636883974075317, + -0.7797420620918274, + -0.8226667642593384, + -3.454789876937866, + -2.9299166202545166, + 1.645264744758606, + 0.3283607065677643, + -0.43253177404403687, + -0.4816684126853943, + 0.002722412347793579, + -1.3148722648620605, + -1.615276575088501 + ] + ], + "generated_utilities": [ + 0.1320568472146988, + 0.12680597603321075, + 0.09484513849020004, + 0.10936561971902847, + 0.11417748779058456, + 0.06823223829269409, + 0.1029265895485878, + 0.0987953171133995 + ], + "group_id": "ms-e9ed5d488afdfd6449d1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.22621813416481018, + 0.26591962575912476, + 0.22478708624839783, + 0.074407659471035, + 0.3134628236293793, + 0.19173070788383484, + 0.2438776195049286, + 0.26689577102661133, + 0.24477621912956238, + 0.24051737785339355, + 0.2738616168498993, + 0.04947046563029289, + 2.0, + 0.04723116010427475, + 0.08354459702968597, + 0.06540697813034058 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.6524851322174072, + -0.023243099451065063, + -0.05794792249798775, + 0.1783304661512375, + -0.3767814338207245, + 1.4893875122070312, + 0.0, + 1.024657964706421, + 0.3163401186466217, + -0.49971193075180054, + 0.05114656686782837, + -0.13273626565933228, + -1.9796202182769775, + 0.0, + 0.818771481513977, + -0.13705189526081085, + -0.21289467811584473, + 0.3914462625980377, + -0.6477798819541931, + -1.0717217922210693, + 0.0 + ], + [ + 2.363477945327759, + -1.042197585105896, + -0.8369066119194031, + 0.17341986298561096, + 0.6040215492248535, + 1.9536304473876953, + 2.2788424491882324, + 1.5015904903411865, + -0.07916292548179626, + -0.6816228628158569, + -1.3893373012542725, + -0.2375274896621704, + -2.5662975311279297, + -2.7615673542022705, + 1.151218295097351, + 0.29418760538101196, + -0.4583754539489746, + -0.4555256962776184, + -0.6620294451713562, + -2.534846067428589, + -1.916935682296753 + ], + [ + 2.0104916095733643, + -0.539351224899292, + 0.13553066551685333, + 0.9507300853729248, + 0.6636125445365906, + 1.5649433135986328, + 2.0495975017547607, + 2.832026958465576, + 1.1830073595046997, + -1.6508865356445312, + -0.26773035526275635, + -0.3988863527774811, + -2.8235108852386475, + -2.2897520065307617, + 1.4448039531707764, + -0.5226297378540039, + -2.1615242958068848, + 0.8214395046234131, + -0.755327045917511, + -2.571093797683716, + -1.7457375526428223 + ], + [ + 2.166736602783203, + -0.3969564735889435, + 0.5171664357185364, + 0.12789201736450195, + -0.9176173806190491, + 1.8307013511657715, + 1.834255576133728, + 1.3594005107879639, + 0.13751044869422913, + -0.9923363327980042, + -1.136932611465454, + -1.2923126220703125, + -2.445897102355957, + -2.4779889583587646, + 2.6843042373657227, + 0.10015779733657837, + -2.0766196250915527, + 0.1269409954547882, + 0.16698843240737915, + -1.2709240913391113, + -2.7113101482391357 + ], + [ + 1.0476312637329102, + -1.2484432458877563, + -0.8652145862579346, + 0.8706620335578918, + 0.6091254353523254, + 1.9634168148040771, + 2.105522394180298, + 1.2906999588012695, + -0.3275601267814636, + -1.0110361576080322, + -1.6514918804168701, + -1.376278281211853, + -2.345550060272217, + -2.4931728839874268, + 1.00270676612854, + 0.04506785422563553, + -0.9420130252838135, + -0.6248484253883362, + -1.469360589981079, + -3.059006929397583, + -3.495709180831909 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.6524851322174072, + -0.3197403848171234, + -0.07008056342601776, + 0.38692954182624817, + -0.013657404109835625, + 2.4893875122070312, + 2.3340368270874023, + 2.024657964706421, + 0.35216206312179565, + -1.4125897884368896, + -0.8262880444526672, + -0.49936625361442566, + -2.9796202182769775, + -2.9234721660614014, + 1.818771481513977, + 0.09626653790473938, + -1.2128946781158447, + 0.10123217105865479, + -0.7020833492279053, + -2.0717217922210693, + -2.5944693088531494 + ], + [ + 0.9812780022621155, + 0.2612687647342682, + 0.8081930875778198, + 0.3689844310283661, + -0.16453352570533752, + 2.8411240577697754, + 2.2401933670043945, + 1.7904633283615112, + -0.04837304353713989, + -0.6139949560165405, + -0.8549875020980835, + 0.3574656546115875, + -2.8409454822540283, + -2.1011672019958496, + 1.382948875427246, + 0.6656897068023682, + -1.5090553760528564, + 0.4825518727302551, + -0.3345044255256653, + -2.26375150680542, + -2.561906576156616 + ] + ], + "predicted_scores": [ + 5.859316349029541, + 5.420124530792236, + 5.264007568359375, + 5.090690612792969, + 4.776496410369873, + 4.625821113586426, + 4.44064998626709, + 4.344982147216797 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "626486813", + "selected_index": 0, + "state_hash": "5608eea5b7e99b5fed6b316f742ecd34044df87286b2338705d8472043fc4375", + "stored_base_utility": 0.22621813416481018, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.09514755010604858, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09514755010604858 + }, + "base_progress": 0.09514755010604858, + "base_success": false, + "base_utility": 0.09514755010604858, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.05872911214828491, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05872911214828491 + }, + { + "contact_quality": null, + "progress": 0.05446813628077507, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05446813628077507 + }, + { + "contact_quality": null, + "progress": 0.06178152561187744, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06178152561187744 + }, + { + "contact_quality": null, + "progress": 0.006611684802919626, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006611684802919626 + }, + { + "contact_quality": null, + "progress": 0.006492753978818655, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006492753978818655 + }, + { + "contact_quality": null, + "progress": 0.006466865539550781, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.006466865539550781 + }, + { + "contact_quality": null, + "progress": 0.0066977739334106445, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0066977739334106445 + }, + { + "contact_quality": null, + "progress": 0.0056935749016702175, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0056935749016702175 + } + ], + "candidate_progress": [ + 0.05872911214828491, + 0.05446813628077507, + 0.06178152561187744, + 0.006611684802919626, + 0.006492753978818655, + 0.006466865539550781, + 0.0066977739334106445, + 0.0056935749016702175 + ], + "candidate_source_chart_ids": [ + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104", + "ms-8c0266b35fc59963795c", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104", + "ms-0093b06390890d181104" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-ec013396e4aff6170ee1", + "generated_tangents": [ + [ + 0.5101381540298462, + -0.05360936373472214, + -0.029661327600479126, + 0.20449179410934448, + 1.0406460762023926, + -0.030199676752090454, + -0.3574920892715454, + -1.022248387336731, + 0.23367558419704437, + 1.1467262506484985, + 0.24691137671470642, + -0.724727988243103, + -0.3094742000102997, + 0.3639238774776459, + -0.165083646774292, + 0.8841742873191833, + 0.1577814817428589, + 0.3289101719856262, + -0.22462034225463867, + -1.9921259880065918, + 0.06798835098743439 + ], + [ + -0.15466274321079254, + -0.03181654214859009, + 0.3227255940437317, + 1.1556735038757324, + 0.6858857870101929, + 0.19386076927185059, + 0.4094448387622833, + -1.6155661344528198, + 0.4407074451446533, + 1.43799889087677, + -0.06252732127904892, + -0.14964619278907776, + -0.9649289846420288, + -0.15493474900722504, + -0.6133165955543518, + 0.6074562668800354, + 0.06724636256694794, + 0.28682491183280945, + 0.1895415186882019, + -1.3882741928100586, + -0.38994941115379333 + ], + [ + 0.285857230424881, + -0.13716495037078857, + 0.5049164295196533, + 0.38094261288642883, + 0.7799856066703796, + -0.0754309594631195, + -0.5730307698249817, + -1.216273546218872, + 0.24912449717521667, + 1.2083938121795654, + 0.4422561526298523, + -0.48583900928497314, + 0.34284552931785583, + 0.6974050998687744, + -0.19796833395957947, + 0.9154776334762573, + 0.48387467861175537, + 0.39983516931533813, + -0.2657671570777893, + -1.5331840515136719, + 0.41048866510391235 + ], + [ + -0.21052299439907074, + -0.13993632793426514, + -0.10644039511680603, + 1.7271068096160889, + 1.2685949802398682, + 0.4705065190792084, + -0.2037225365638733, + -1.3093360662460327, + 0.12810085713863373, + 1.2043800354003906, + 0.1598292887210846, + 0.12638956308364868, + -0.8021771907806396, + 0.5791755318641663, + -0.189354807138443, + 0.5201027989387512, + 0.17081068456172943, + 0.34929633140563965, + 0.019526392221450806, + -2.0044541358947754, + -0.7854268550872803 + ], + [ + -0.155709907412529, + -0.09890305995941162, + -0.1439400613307953, + 1.3399819135665894, + 0.7990903854370117, + -0.03894232213497162, + 0.4093298017978668, + -1.4367101192474365, + 0.27560991048812866, + 1.5024744272232056, + -0.010531358420848846, + -0.12255418300628662, + -0.7647441625595093, + -0.08253826200962067, + -0.5598908066749573, + 0.640886664390564, + 0.31670981645584106, + 0.35831886529922485, + -0.11838579177856445, + -1.8318662643432617, + -0.2831522822380066 + ], + [ + -0.20471538603305817, + -0.07208540290594101, + -0.06225898861885071, + 1.1957406997680664, + 0.8960942029953003, + 0.2189735770225525, + 0.21606463193893433, + -1.4886016845703125, + 0.31738755106925964, + 1.547255516052246, + 0.05674219876527786, + -0.2351195216178894, + -0.8285989761352539, + -0.07602265477180481, + -0.5344630479812622, + 0.6355008482933044, + 0.1701146364212036, + 0.11661215871572495, + 0.1879451423883438, + -1.9480006694793701, + -0.3432569205760956 + ], + [ + -0.17921487987041473, + -0.08891166746616364, + -0.1476530134677887, + 1.390163540840149, + 0.9013686776161194, + -0.037880584597587585, + 0.22892099618911743, + -1.4280190467834473, + 0.26878079771995544, + 1.5264595746994019, + 0.051676854491233826, + -0.23780396580696106, + -0.9902065992355347, + 0.230289489030838, + -0.48221898078918457, + 0.6269107460975647, + 0.1803981363773346, + 0.30742016434669495, + -0.015006333589553833, + -1.7922697067260742, + -0.2591914236545563 + ], + [ + -0.3252200484275818, + -0.10510633885860443, + -0.05997195839881897, + 1.5174412727355957, + 0.9277306199073792, + 0.06104784458875656, + 0.3326023519039154, + -1.1899224519729614, + 0.2765958607196808, + 1.51645028591156, + -0.09933304041624069, + -0.21646061539649963, + -0.9433207511901855, + 0.010022200644016266, + -0.5805730223655701, + 0.6100038290023804, + 0.2636059522628784, + 0.1665084958076477, + -0.010278105735778809, + -2.134931802749634, + -0.2663840651512146 + ] + ], + "generated_utilities": [ + 0.05872911214828491, + 0.05446813628077507, + 0.06178152561187744, + 0.006611684802919626, + 0.006492753978818655, + 0.006466865539550781, + 0.0066977739334106445, + 0.0056935749016702175 + ], + "group_id": "ms-ec013396e4aff6170ee1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.010581731796264648, + 0.049839019775390625, + 0.06402494758367538, + 0.012959301471710205, + 0.11614581197500229, + 0.012509982101619244, + 0.09607907384634018, + 0.1605721116065979, + 0.1708059310913086, + 0.055588722229003906, + 0.2052522599697113, + 0.0690404623746872, + 0.11059975624084473, + 0.03047112748026848, + 0.24502670764923096, + 0.22897732257843018 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.16510963439941406, + -0.011389851570129395, + -0.16074393689632416, + 0.8547754287719727, + -0.09704409539699554, + 0.09471678733825684, + 0.0, + -1.0372710227966309, + 0.5171947479248047, + 1.4950246810913086, + -0.29076793789863586, + 0.0122823566198349, + 0.059903502464294434, + 0.0, + 0.13894814252853394, + 0.2961620092391968, + 0.025543689727783203, + 0.01881265640258789, + 0.052319109439849854, + -1.1815040111541748, + 0.0 + ], + [ + -0.6906574964523315, + 0.5966935753822327, + 0.1873793751001358, + 1.8547754287719727, + 0.16492335498332977, + 0.4949548840522766, + 0.624785304069519, + -2.037271022796631, + 1.5171947479248047, + 2.4950246810913086, + -0.3186037838459015, + 0.2194363921880722, + -0.6441555023193359, + 1.221185326576233, + -0.44174641370773315, + 0.866999626159668, + 1.0255436897277832, + 1.018812656402588, + 0.6033775210380554, + -2.181504011154175, + -0.4328075349330902 + ], + [ + -0.68684321641922, + 0.35380983352661133, + 0.41641929745674133, + 2.0589241981506348, + 0.20610027015209198, + 0.2780776619911194, + 0.6000017523765564, + -1.8711607456207275, + 1.71079421043396, + 2.484286069869995, + -0.12494567036628723, + 0.3767848014831543, + -0.59480881690979, + 1.3215627670288086, + -0.18650269508361816, + 0.5971667766571045, + 1.00047767162323, + 1.2905888557434082, + 0.7682754993438721, + -2.280778408050537, + -0.26339399814605713 + ], + [ + -1.0669870376586914, + 0.5963312387466431, + 0.24800772964954376, + 1.5324974060058594, + 0.049914538860321045, + 0.33535104990005493, + 0.9774697422981262, + -2.318913698196411, + 1.14731764793396, + 2.1673643589019775, + -0.3369846045970917, + -0.13161931931972504, + -0.7527784705162048, + 1.2647147178649902, + -0.5229569673538208, + 0.6999921798706055, + 0.654599666595459, + 0.9745514988899231, + 0.5995839238166809, + -2.3844704627990723, + -0.7404599189758301 + ], + [ + -0.660759449005127, + 0.560627818107605, + -0.19398994743824005, + 1.7401045560836792, + 0.006708189845085144, + 0.2270365059375763, + 0.9201371073722839, + -1.6917380094528198, + 1.8382139205932617, + 2.184737205505371, + -0.07137502729892731, + 0.13004565238952637, + -0.8376798629760742, + 1.3597201108932495, + -0.8258764147758484, + 0.521450400352478, + 1.4799309968948364, + 0.7742871046066284, + 1.007819414138794, + -2.398472785949707, + -0.7120757102966309 + ], + [ + -0.3511212170124054, + 0.6072589755058289, + 0.6547060012817383, + 2.306784152984619, + 0.03337647020816803, + 1.1676279306411743, + 0.48420602083206177, + -1.7057392597198486, + 1.7822117805480957, + 2.741852283477783, + -0.970782995223999, + 0.3188708424568176, + -0.1370975375175476, + 1.740339994430542, + 0.18519049882888794, + 0.2771400809288025, + 0.8003557920455933, + 0.8813821077346802, + 0.9829010963439941, + -1.9104408025741577, + -0.7000454664230347 + ], + [ + -1.1815872192382812, + 1.0524637699127197, + 0.5610368251800537, + 2.407832622528076, + -0.2576238512992859, + 0.1467430293560028, + 0.27881303429603577, + -1.6810011863708496, + 1.5921515226364136, + 2.929525375366211, + 0.19559642672538757, + 0.6552202105522156, + -1.3889155387878418, + 1.1560360193252563, + -0.4104887843132019, + 0.8887493014335632, + 1.283969759941101, + 0.7811828851699829, + -0.10630232095718384, + -1.4329519271850586, + -0.5646982192993164 + ], + [ + -1.4926873445510864, + 0.11535564064979553, + -0.6240940690040588, + 1.631784200668335, + 0.7930782437324524, + -0.35493338108062744, + 0.20104485750198364, + -1.5626122951507568, + 1.3110854625701904, + 3.0686309337615967, + -0.8419785499572754, + -0.0010939836502075195, + -0.49860918521881104, + 1.5949748754501343, + -0.938766598701477, + 1.762406587600708, + 0.625543475151062, + 0.20789378881454468, + 1.1513795852661133, + -2.3980860710144043, + -0.36970049142837524 + ], + [ + -1.0271471738815308, + 0.6133835911750793, + -0.3870598077774048, + 1.9299802780151367, + 0.9606215357780457, + -0.19876420497894287, + 0.29684266448020935, + -1.6164112091064453, + 1.3925114870071411, + 1.9474635124206543, + -1.2669438123703003, + -0.6821421980857849, + -0.7869906425476074, + 1.3602250814437866, + -0.14778050780296326, + 0.9938437938690186, + 0.6618988513946533, + 0.1858283281326294, + 0.8312947750091553, + -1.7043554782867432, + -0.23048581182956696 + ], + [ + -0.409736305475235, + 1.5572996139526367, + 1.0024399757385254, + 1.6385810375213623, + 0.83731609582901, + -0.1629825234413147, + -0.05952054262161255, + -1.0968945026397705, + 0.728961706161499, + 3.1370975971221924, + -0.44591856002807617, + 1.149456262588501, + -1.3439809083938599, + 1.738822340965271, + 0.4063607454299927, + 0.4372418224811554, + 1.276847004890442, + 1.4612212181091309, + 0.6917726993560791, + -1.914740800857544, + 0.5415292978286743 + ] + ], + "predicted_scores": [ + 4.882284164428711, + 4.643063068389893, + 3.9711496829986572, + 3.844956398010254, + 3.3679914474487305, + 3.356393814086914, + 3.28397274017334, + 2.771803140640259 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "750448296", + "selected_index": 0, + "state_hash": "f5a19cf07e6a6cc71c95ce01e976d34d1eeae68c8a93ecc985fd0a78cb0c1a5e", + "stored_base_utility": 0.010581731796264648, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.008193274028599262, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.008193274028599262 + }, + { + "contact_quality": null, + "progress": 0.007785916328430176, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.007785916328430176 + }, + { + "contact_quality": null, + "progress": 0.010366380214691162, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.010366380214691162 + }, + { + "contact_quality": null, + "progress": 0.009668509475886822, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.009668509475886822 + }, + { + "contact_quality": null, + "progress": 0.015532891266047955, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015532891266047955 + }, + { + "contact_quality": null, + "progress": 0.02954266406595707, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02954266406595707 + }, + { + "contact_quality": null, + "progress": 0.016943970695137978, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.016943970695137978 + }, + { + "contact_quality": null, + "progress": 0.036433976143598557, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.036433976143598557 + } + ], + "candidate_progress": [ + 0.008193274028599262, + 0.007785916328430176, + 0.010366380214691162, + 0.009668509475886822, + 0.015532891266047955, + 0.02954266406595707, + 0.016943970695137978, + 0.036433976143598557 + ], + "candidate_source_chart_ids": [ + "ms-003a2225c0892dcda683", + "ms-003a2225c0892dcda683", + "ms-0285f5815c210ac00bec", + "ms-0285f5815c210ac00bec", + "ms-be219977a9db8dbb1e58", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5", + "ms-7dedf60ad41ea2d92ee5" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-ee30ebaf98622759d888", + "generated_tangents": [ + [ + 0.27170443534851074, + -0.4520937204360962, + 0.6051304340362549, + -0.11098112165927887, + 2.2225608825683594, + 0.8262457847595215, + 0.51774001121521, + 0.22463487088680267, + -0.13016048073768616, + 0.7891191840171814, + 0.10605950653553009, + 0.4298771023750305, + -1.3047176599502563, + 0.21161803603172302, + -0.13798145949840546, + 0.6419728398323059, + 0.8496493697166443, + -0.619520902633667, + 1.1715009212493896, + -2.2415504455566406, + -1.6119660139083862 + ], + [ + 0.2660503685474396, + -0.4555244445800781, + 0.5446423888206482, + -0.0777994841337204, + 2.3126749992370605, + 0.9720523357391357, + 0.578887403011322, + 0.2177036553621292, + -0.14381539821624756, + 0.8266566395759583, + 0.17044782638549805, + 0.541135311126709, + -1.3385353088378906, + 0.13335537910461426, + -0.1603492945432663, + 0.6253682971000671, + 0.925575315952301, + -0.4906349182128906, + 1.1988892555236816, + -2.2153210639953613, + -1.8029555082321167 + ], + [ + 0.3739367723464966, + -0.42600834369659424, + 0.9287277460098267, + 0.03848761320114136, + 1.9865660667419434, + 1.2307316064834595, + 0.579405665397644, + 0.3883819580078125, + -0.14122119545936584, + 0.6310248374938965, + 0.22717157006263733, + 0.33508965373039246, + -1.7190955877304077, + -0.06860573589801788, + 0.08088497817516327, + 0.5474994778633118, + 0.6855216026306152, + -0.6447333693504333, + 1.0441217422485352, + -2.1500213146209717, + -1.3579318523406982 + ], + [ + 0.35994911193847656, + -0.43802374601364136, + 0.9374743103981018, + 0.043926119804382324, + 1.9804353713989258, + 1.2484631538391113, + 0.6148548722267151, + 0.3553198575973511, + -0.13248299062252045, + 0.6398996114730835, + 0.21147191524505615, + 0.36410677433013916, + -1.8008677959442139, + -0.013217300176620483, + 0.06951896846294403, + 0.5593108534812927, + 0.6968275308609009, + -0.6997720003128052, + 1.0882527828216553, + -2.1821072101593018, + -1.3579750061035156 + ], + [ + 0.41315406560897827, + -0.25528600811958313, + 0.5363587141036987, + -0.20223447680473328, + 1.8758914470672607, + 0.8883450031280518, + 0.5835685729980469, + 0.35880497097969055, + -0.09831319004297256, + 0.2971900403499603, + 0.08581723272800446, + 0.45772281289100647, + -1.966092586517334, + -0.04350966215133667, + -0.2585785686969757, + 0.3062516748905182, + 1.0503592491149902, + -0.7279158234596252, + 0.8477835059165955, + -2.0772640705108643, + -0.6477218866348267 + ], + [ + 0.23574531078338623, + -0.16467499732971191, + 0.23795995116233826, + 0.21440979838371277, + 2.8141818046569824, + 0.8763961791992188, + -0.12586015462875366, + 0.8820905685424805, + -0.0445159450173378, + 0.2997494637966156, + 0.0844811201095581, + 0.913824737071991, + -2.358003616333008, + 0.1929166167974472, + -0.08803212642669678, + 0.5527779459953308, + 1.0282915830612183, + -0.6101901531219482, + 1.1536555290222168, + -2.258113384246826, + -1.429984211921692 + ], + [ + 0.4009239077568054, + -0.10225620120763779, + 0.6916215419769287, + -0.14502769708633423, + 2.9053778648376465, + 1.6723735332489014, + 0.6771396398544312, + 0.7002356052398682, + 0.11819807440042496, + 0.24285200238227844, + 0.4466201663017273, + 0.5122247934341431, + -1.8524523973464966, + 0.2524349093437195, + 0.027714475989341736, + 0.8019974231719971, + 1.25539231300354, + -0.5372257828712463, + 1.086941123008728, + -1.4629790782928467, + -1.634588360786438 + ], + [ + 0.7704882621765137, + -0.32318365573883057, + 0.08198440074920654, + 0.25582268834114075, + 1.919844627380371, + 1.0956228971481323, + 0.9605214595794678, + 0.5610069632530212, + -0.04221829026937485, + 0.38827699422836304, + 0.08968988060951233, + 0.30052971839904785, + -1.791446566581726, + 0.552302896976471, + 0.2858082056045532, + 0.49666696786880493, + 1.0112940073013306, + -0.35014182329177856, + 1.0454986095428467, + -1.9807567596435547, + -2.004758596420288 + ] + ], + "generated_utilities": [ + 0.008193274028599262, + 0.007785916328430176, + 0.010366380214691162, + 0.009668509475886822, + 0.015532891266047955, + 0.02954266406595707, + 0.016943970695137978, + 0.036433976143598557 + ], + "group_id": "ms-ee30ebaf98622759d888", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.08405601978302002, + 0.09846439212560654, + 0.04595305770635605, + 2.0, + 0.08787640184164047, + 0.032481253147125244, + 0.08755072206258774, + 0.058936577290296555, + 0.10809408128261566, + 0.06723630428314209, + 0.0625009536743164, + 0.1415095031261444, + 0.08326160907745361, + 0.10944461822509766, + 0.0673408955335617, + 0.033725105226039886 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.4581542015075684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9406367540359497, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.470216989517212 + ], + [ + 1.6340367794036865, + -0.30366095900535583, + 0.2830277681350708, + -1.1627482175827026, + 3.58767032623291, + 0.6565636396408081, + 1.4321975708007812, + 0.8704822659492493, + 0.14327874779701233, + 1.0190284252166748, + 0.3078446090221405, + 0.7927982807159424, + -1.9305998086929321, + -1.7108646631240845, + 0.10729888081550598, + 0.9427294731140137, + 0.41770225763320923, + -1.7673753499984741, + 1.3209179639816284, + -2.8053033351898193, + -1.5330681800842285 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.00628507137298584, + 0.0595628023147583, + 0.39481037855148315, + -0.006641983985900879, + 1.8433213233947754, + 0.14830267429351807, + 0.0, + -0.24425631761550903, + 0.42130333185195923, + 0.41399094462394714, + 0.478841632604599, + -0.007461488246917725, + -0.5437495708465576, + 0.0, + 0.255215585231781, + -0.07331699132919312, + -0.1736500859260559, + -0.285358190536499, + 0.567956805229187, + -1.7103407382965088, + 0.0 + ], + [ + 1.0371580123901367, + -0.33467406034469604, + 0.9327473640441895, + -0.5396467447280884, + 2.1487879753112793, + 0.6294846534729004, + 0.6252679228782654, + 0.5140941143035889, + -0.20645928382873535, + -0.04528486728668213, + 0.14780870079994202, + 0.430325984954834, + -0.9293508529663086, + -1.5147639513015747, + 1.1104007959365845, + -0.05476301908493042, + 0.1755150556564331, + -1.3470268249511719, + 1.675830364227295, + -3.2944674491882324, + -1.9277915954589844 + ] + ], + "predicted_scores": [ + 9.16378116607666, + 8.605719566345215, + 8.50434398651123, + 8.490377426147461, + 7.924519062042236, + 7.78317928314209, + 5.880914688110352, + 2.4533824920654297 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "636680137", + "selected_index": 0, + "state_hash": "2de607a9c69005130e51ff2b551c0eeedc85c58fef796682e6527bb5f719ff7b", + "stored_base_utility": 0.08405601978302002, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.026694953441619873, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.026694953441619873 + }, + "base_progress": 0.026694953441619873, + "base_success": false, + "base_utility": 0.026694953441619873, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.020747045055031776, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.020747045055031776 + }, + { + "contact_quality": null, + "progress": 0.018429061397910118, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018429061397910118 + }, + { + "contact_quality": null, + "progress": 0.013478715904057026, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.013478715904057026 + }, + { + "contact_quality": null, + "progress": 0.015456080436706543, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.015456080436706543 + }, + { + "contact_quality": null, + "progress": 0.04147064685821533, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04147064685821533 + }, + { + "contact_quality": null, + "progress": 0.044700879603624344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.044700879603624344 + }, + { + "contact_quality": null, + "progress": 0.025082269683480263, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.025082269683480263 + }, + { + "contact_quality": null, + "progress": 0.039384882897138596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.039384882897138596 + } + ], + "candidate_progress": [ + 0.020747045055031776, + 0.018429061397910118, + 0.013478715904057026, + 0.015456080436706543, + 0.04147064685821533, + 0.044700879603624344, + 0.025082269683480263, + 0.039384882897138596 + ], + "candidate_source_chart_ids": [ + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-38dff128761905bb1507", + "ms-02fd0338e80ef19ceec1", + "ms-02fd0338e80ef19ceec1", + "ms-53a86042af3c0887e556", + "ms-1a4979e969aa331fb126" + ], + "candidate_source_task_ids": [ + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1", + "PullCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-f3f1d07cfd59b823da47", + "generated_tangents": [ + [ + 1.0349353551864624, + -0.06126919761300087, + -0.24474528431892395, + -0.4928235411643982, + -0.09823867678642273, + -1.5665614604949951, + -0.34603285789489746, + 0.6530897617340088, + 0.012888672761619091, + -0.3264999985694885, + -1.0608021020889282, + 0.3909127712249756, + -0.2603576183319092, + 0.6442807912826538, + 0.6143785715103149, + -0.01590779423713684, + 0.08504053950309753, + -0.449088454246521, + 0.19008608162403107, + 0.24809932708740234, + 0.5028657913208008 + ], + [ + 1.2439202070236206, + 0.016533263027668, + -0.4084916412830353, + -0.3623259961605072, + 0.3134339451789856, + -1.8045082092285156, + -0.16475188732147217, + 0.5584524869918823, + -0.12950576841831207, + -0.2849171757698059, + -0.8536432385444641, + 0.49415135383605957, + -0.3136178255081177, + 0.5371270775794983, + 0.34843894839286804, + -0.10000298917293549, + 0.18952998518943787, + -0.6882005929946899, + 0.31218549609184265, + -0.5124547481536865, + 0.5893510580062866 + ], + [ + 1.0175174474716187, + -0.0002964623272418976, + -0.15028533339500427, + -0.3534201979637146, + 0.12945501506328583, + -1.4312644004821777, + -0.27580761909484863, + 0.7022987604141235, + -0.09329473227262497, + -0.18243283033370972, + -1.097499132156372, + 0.6538478136062622, + -0.32707029581069946, + 0.74146568775177, + 0.35227170586586, + -0.11970359086990356, + 0.17694193124771118, + -0.5119301676750183, + 0.3433413803577423, + 0.23969638347625732, + 0.6604613065719604 + ], + [ + 0.9946953654289246, + -0.015589572489261627, + -0.17950591444969177, + -0.4158869981765747, + -0.03835874795913696, + -1.5828815698623657, + -0.3362371325492859, + 0.6675266623497009, + -0.024929681792855263, + -0.241695374250412, + -1.09584641456604, + 0.349558025598526, + -0.24058499932289124, + 0.5352067351341248, + 0.49236515164375305, + -0.019468313083052635, + 0.12109559029340744, + -0.5607954859733582, + 0.2185971587896347, + 0.376800000667572, + 0.48997700214385986 + ], + [ + 0.8608055114746094, + 0.26703953742980957, + -0.4005744159221649, + -0.2484826147556305, + 0.2338886708021164, + -0.9424741268157959, + 0.44213464856147766, + 0.6766794919967651, + -0.045276157557964325, + -0.07421816140413284, + -1.0170491933822632, + 0.5813264846801758, + -0.8028758764266968, + -0.21456409990787506, + 0.15495555102825165, + 0.07764653861522675, + 0.2836809456348419, + -1.0219577550888062, + 0.10980084538459778, + 0.4904273748397827, + 0.5579061508178711 + ], + [ + 0.9573588967323303, + 0.27570217847824097, + -0.3582111895084381, + -0.3116881251335144, + 0.3710542321205139, + -0.8759490251541138, + 0.30302301049232483, + 0.6119298934936523, + -0.09536349773406982, + -0.07157202810049057, + -1.150516390800476, + 0.6570000052452087, + -0.9844952821731567, + -0.21766610443592072, + 0.08414357900619507, + 0.0901758074760437, + 0.2130962461233139, + -1.0716454982757568, + 0.1417546570301056, + 0.25137341022491455, + 0.3846231698989868 + ], + [ + 1.204817771911621, + 0.09754139184951782, + -0.6386665105819702, + -0.062338411808013916, + 0.09314592182636261, + -1.5826767683029175, + -0.41071248054504395, + 0.521346926689148, + 0.25248655676841736, + -0.1181819960474968, + -0.40200352668762207, + 0.6560121774673462, + -0.41702309250831604, + 0.25525963306427, + 0.3502126634120941, + -0.22212518751621246, + 0.36393314599990845, + -0.7670950293540955, + 0.23357316851615906, + 0.18474698066711426, + 0.6281170845031738 + ], + [ + 1.2584664821624756, + 0.22888241708278656, + -0.6232999563217163, + -0.2170407772064209, + -0.11911964416503906, + -1.6025922298431396, + -0.021400898694992065, + 0.42525210976600647, + 0.42933526635169983, + 0.1212443932890892, + -0.6297640204429626, + 0.4663831293582916, + -0.8602383136749268, + 0.35413768887519836, + 0.31219640374183655, + -0.2646739184856415, + -0.16007615625858307, + -0.6059231162071228, + 0.18030160665512085, + -0.01707574725151062, + 0.014252588152885437 + ] + ], + "generated_utilities": [ + 0.020747045055031776, + 0.018429061397910118, + 0.013478715904057026, + 0.015456080436706543, + 0.04147064685821533, + 0.044700879603624344, + 0.025082269683480263, + 0.039384882897138596 + ], + "group_id": "ms-f3f1d07cfd59b823da47", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.07767709344625473, + 0.07153411954641342, + 0.08480638265609741, + 0.06087514013051987, + 0.15252648293972015, + 0.0776997059583664, + 0.1476559340953827, + 0.17124073207378387, + 0.12327007949352264, + 0.12037092447280884, + 0.08562581241130829, + 0.09917601197957993, + 0.07137986272573471, + 0.039922356605529785, + 0.04496842622756958, + 0.05561637878417969 + ], + "instruction": "Pull the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 1.9664052724838257, + 0.20301294326782227, + -1.1196715831756592, + -0.17260856926441193, + -0.5341777801513672, + -2.678128242492676, + -2.431448459625244, + 1.5603530406951904, + 0.03585217520594597, + -0.3520800471305847, + -2.473386287689209, + 1.0162299871444702, + -0.7868502736091614, + -0.9102090001106262, + 0.9442894458770752, + -0.41473180055618286, + 0.7349075675010681, + -0.8863289952278137, + 0.8796874284744263, + 0.028114937245845795, + 0.24920660257339478 + ], + [ + 1.900488257408142, + -0.00838443636894226, + -1.2311820983886719, + -0.361179918050766, + -0.3987593352794647, + -2.551008462905884, + -2.701385021209717, + 1.7117550373077393, + -0.20423506200313568, + -0.3429180979728699, + -2.6584055423736572, + 0.8018259406089783, + -0.856230616569519, + -0.901038408279419, + 1.2131764888763428, + -0.6892215609550476, + 0.6726197600364685, + -0.7798839211463928, + 0.6868253946304321, + 0.07731680572032928, + 0.30107203125953674 + ], + [ + 1.6257587671279907, + 0.275219202041626, + -0.7838636636734009, + -0.5118123292922974, + -0.3429456353187561, + -2.342111349105835, + -2.7158243656158447, + 1.7038744688034058, + -0.1860528439283371, + -0.11617843806743622, + -2.1894750595092773, + 1.2306283712387085, + -0.8203369379043579, + -0.5157363414764404, + 0.7951346635818481, + -0.464976966381073, + 0.8823708295822144, + -0.7671404480934143, + 0.9700374007225037, + 0.21184596419334412, + -0.11244213581085205 + ] + ], + "predicted_scores": [ + 4.140278339385986, + 3.720200538635254, + 3.6682474613189697, + 3.4919662475585938, + 3.44966983795166, + 2.959789991378784, + 2.7394957542419434, + 2.1507058143615723 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2015956957", + "selected_index": 0, + "state_hash": "3f477a29208b02558b99bd0f3305d0f1845c277d32e0b5afdafa71012673383b", + "stored_base_utility": 0.07767709344625473, + "task_id": "PullCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.07080724835395813, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07080724835395813 + }, + "base_progress": 0.07080724835395813, + "base_success": false, + "base_utility": 0.07080724835395813, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.04525895416736603, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04525895416736603 + }, + { + "contact_quality": null, + "progress": 0.06861761212348938, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06861761212348938 + }, + { + "contact_quality": null, + "progress": 0.07222916185855865, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.07222916185855865 + }, + { + "contact_quality": null, + "progress": 0.09644092619419098, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09644092619419098 + }, + { + "contact_quality": null, + "progress": 0.053700655698776245, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.053700655698776245 + }, + { + "contact_quality": null, + "progress": 4.214048385620117e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 4.214048385620117e-05 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.04525895416736603, + 0.06861761212348938, + 0.07222916185855865, + 0.09644092619419098, + 0.053700655698776245, + 4.214048385620117e-05, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-dab80e1bb7987577e33d", + "ms-bfe867e8d518b9f8333a", + "ms-bfe867e8d518b9f8333a", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68", + "ms-8278abbc47a6efcb5b68" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-081952a63087fc8f83cd", + "generated_tangents": [ + [ + -0.032365500926971436, + 0.10390760004520416, + 1.415757656097412, + -0.5925084352493286, + -0.8634079098701477, + -0.19858211278915405, + -0.2506406903266907, + 0.048071250319480896, + -0.11267983913421631, + 0.12202358990907669, + 0.15672841668128967, + -1.0689752101898193, + -0.06612500548362732, + 0.06195733696222305, + 0.04202529042959213, + -0.013784321956336498, + 0.016651272773742676, + 0.1811247020959854, + -0.5694010257720947, + -0.2003055214881897, + -0.03322325646877289 + ], + [ + -0.11432172358036041, + 0.08919510245323181, + 1.517054557800293, + -0.4753502309322357, + -0.8041483759880066, + -0.19328033924102783, + -0.35954850912094116, + 0.09516977518796921, + -0.0904659777879715, + 0.15756557881832123, + 0.20103389024734497, + -0.8708748817443848, + -0.07928246259689331, + -0.00747188925743103, + 0.08754106611013412, + 0.004732646048069, + 0.11909907311201096, + 0.17976027727127075, + -0.3799339532852173, + -0.008889615535736084, + -0.05170711874961853 + ], + [ + -0.11421684920787811, + 0.07243978977203369, + 1.5277025699615479, + -0.5288271307945251, + -0.8376646637916565, + -0.12296772003173828, + -0.3800280690193176, + 0.11622491478919983, + -0.10285645723342896, + 0.1337386518716812, + 0.3056662976741791, + -0.8803032636642456, + -0.049797624349594116, + -0.023733511567115784, + 0.04615781828761101, + -0.016816962510347366, + 0.13501235842704773, + 0.14333173632621765, + -0.36909955739974976, + 0.030240654945373535, + -0.10721567273139954 + ], + [ + -0.11995033919811249, + 0.060259461402893066, + 1.217244029045105, + -0.7226763963699341, + -1.1381659507751465, + 0.2173631489276886, + -0.1764562726020813, + -0.09369504451751709, + -0.0863342434167862, + 0.2310180962085724, + 0.16642993688583374, + -0.9087406396865845, + -0.24564912915229797, + 0.15980838239192963, + -0.023319657891988754, + 0.029917996376752853, + -0.04658149182796478, + 0.003416284918785095, + -0.4824352264404297, + -0.23991714417934418, + 0.09200343489646912 + ], + [ + 0.026846006512641907, + 0.03736086189746857, + 1.4077420234680176, + -0.7325681447982788, + -1.1262304782867432, + 0.045286305248737335, + -0.19949376583099365, + -0.04826843738555908, + -0.051994092762470245, + 0.1934489905834198, + 0.20467272400856018, + -0.8510723114013672, + 0.1608184278011322, + 0.023805618286132812, + -0.029557082802057266, + 0.018773630261421204, + -0.011426910758018494, + 0.12089154124259949, + -0.41795814037323, + -0.14041092991828918, + 0.010147333145141602 + ], + [ + -0.206819549202919, + 0.09014281630516052, + 1.5819236040115356, + -0.9215487241744995, + -0.7654125094413757, + 0.12386204302310944, + -0.337863028049469, + -0.05849051475524902, + -0.013000188395380974, + 0.02482248842716217, + 0.29215696454048157, + -0.6927528977394104, + -0.31674784421920776, + 0.11673110723495483, + -0.09033209085464478, + -0.06116269528865814, + -0.04351656138896942, + 0.21102769672870636, + -0.6393582224845886, + 0.10619005560874939, + -0.2805402874946594 + ], + [ + -0.34209907054901123, + -0.0786755234003067, + 1.4100205898284912, + -0.9066512584686279, + -0.8203497529029846, + 0.08083697408437729, + -0.3675681948661804, + 0.26753443479537964, + -0.11146403849124908, + 0.16693587601184845, + 0.4500954747200012, + -0.4960119128227234, + 0.08443322777748108, + 0.3699827194213867, + -0.09823426604270935, + -0.12029463052749634, + 0.10245548188686371, + 0.19952167570590973, + -0.6053216457366943, + 0.06697890162467957, + -0.2166258990764618 + ], + [ + -0.45381081104278564, + 0.18843740224838257, + 1.566582441329956, + -0.7463564872741699, + -0.7096806168556213, + 0.5056304931640625, + -0.2998039126396179, + 0.06751719117164612, + -0.1269555389881134, + 0.06923900544643402, + 0.1686798334121704, + -0.9188179969787598, + -0.040544867515563965, + 0.27317044138908386, + 0.021121639758348465, + -0.13543808460235596, + -0.08620311319828033, + 0.2963440418243408, + -0.6836603283882141, + -0.14094305038452148, + -0.08842611312866211 + ] + ], + "generated_utilities": [ + 0.04525895416736603, + 0.06861761212348938, + 0.07222916185855865, + 0.09644092619419098, + 0.053700655698776245, + 4.214048385620117e-05, + 2.0, + 2.0 + ], + "group_id": "ms-081952a63087fc8f83cd", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0006054937839508057, + 2.0, + 2.0, + 2.0, + 0.09041815996170044, + 0.03755016624927521, + 0.10023769736289978, + 0.08501018583774567, + 0.10316208004951477, + 0.13556814193725586, + 0.0687420666217804, + 0.16420090198516846, + 0.07440806925296783, + 0.03177754580974579, + 0.05512912571430206, + 0.05822136998176575 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05530714988708496, + -0.0015276968479156494, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.028481781482696533, + 0.0, + 0.07634823024272919, + 0.10118435323238373, + 0.01472395658493042, + -0.021184463053941727, + -1.3624117374420166, + 0.06702810525894165, + 0.0, + 0.064424529671669, + -0.13110561668872833, + -0.04677841067314148, + -0.0666094422340393, + -0.8611308336257935, + -0.02982822060585022, + 0.0 + ], + [ + -0.13999104499816895, + -0.16139763593673706, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.17549771070480347, + 0.0, + -0.15019088983535767, + 0.2207401990890503, + -0.06357991695404053, + 0.10177183151245117, + -1.3624117374420166, + -0.05549207329750061, + 0.0, + -0.1271420419216156, + -0.057942673563957214, + -0.279840350151062, + -0.11312717199325562, + -0.8611308336257935, + 0.1586582064628601, + 0.0 + ], + [ + 0.2317524552345276, + -0.2474955916404724, + 1.582733154296875, + -0.5283455848693848, + -1.530975103378296, + -0.2997032403945923, + 0.0, + -0.3180278241634369, + 0.1991673707962036, + 0.26833486557006836, + 0.005899060517549515, + -1.3624117374420166, + 0.032782405614852905, + 0.0, + 0.17653466761112213, + 0.5550464987754822, + 0.13015127182006836, + -0.11312717199325562, + -0.8611308336257935, + 0.020897403359413147, + 0.0 + ], + [ + -0.7016198635101318, + 0.312343955039978, + 2.582733154296875, + -1.5283455848693848, + -2.530975103378296, + 0.7002967596054077, + 2.0224196910858154, + -0.031223494559526443, + -0.07945594191551208, + 0.7471928000450134, + -0.05505431815981865, + -2.3624117374420166, + 0.4337144196033478, + 1.385794997215271, + -0.18244768679141998, + -0.22281835973262787, + -0.5003347992897034, + 0.8868728280067444, + -1.8611308336257935, + 0.19736243784427643, + 0.2646070420742035 + ], + [ + -0.41503679752349854, + 0.3199126422405243, + 2.3341803550720215, + -1.5432449579238892, + -2.6036009788513184, + 0.8186125755310059, + 1.889835238456726, + -0.20609386265277863, + 0.0827421247959137, + 0.8381537795066833, + -0.0944519191980362, + -2.4805798530578613, + 0.3916063606739044, + 1.2392640113830566, + -0.2266693413257599, + -0.33516526222229004, + -0.3536234498023987, + 0.9749420881271362, + -2.1439476013183594, + 0.3261662721633911, + 0.13869160413742065 + ], + [ + -0.6778035759925842, + 0.3674989640712738, + 2.3216195106506348, + -1.6129508018493652, + -2.3233790397644043, + 0.37060582637786865, + 2.3460428714752197, + 0.2269098311662674, + -0.269074022769928, + 0.9810001254081726, + 0.023842941969633102, + -2.478595733642578, + 0.46739575266838074, + 1.1671457290649414, + -0.41542381048202515, + 0.030661001801490784, + -0.48762595653533936, + 0.7803086638450623, + -1.7778995037078857, + 0.10219369828701019, + 0.08611755073070526 + ], + [ + -1.0292376279830933, + 0.05653715133666992, + 3.017642021179199, + -1.2965388298034668, + -2.1434991359710693, + 0.8112732172012329, + 2.3553943634033203, + 0.14745590090751648, + -0.42473679780960083, + 0.8983602523803711, + -0.5389888286590576, + -1.9595260620117188, + 0.8665074110031128, + 1.7057592868804932, + -0.36306142807006836, + -0.6694610714912415, + -0.3699086308479309, + 0.7089512944221497, + -2.2496888637542725, + 0.1759258657693863, + 0.13106243312358856 + ], + [ + -0.2738665044307709, + 0.13772070407867432, + 2.3146424293518066, + -1.9326081275939941, + -2.760838508605957, + 0.10491085052490234, + 1.9823331832885742, + 0.3494338393211365, + 0.3550114631652832, + 0.7471076846122742, + -0.06264704465866089, + -2.2210469245910645, + 0.5386381149291992, + 1.431454062461853, + 0.1620958298444748, + -0.00858105719089508, + -0.8883554935455322, + 1.1764272451400757, + -1.8280467987060547, + 0.30690017342567444, + -0.32479730248451233 + ], + [ + -0.8548018932342529, + -0.2078946828842163, + 2.740286111831665, + -1.6552648544311523, + -2.470123291015625, + 1.0607044696807861, + 2.4267592430114746, + -0.16535070538520813, + -0.2640931308269501, + 0.2447068691253662, + 0.5616287589073181, + -2.808340311050415, + 1.0820176601409912, + 1.2134929895401, + 0.2653927206993103, + -0.454708456993103, + 0.03163468837738037, + 0.8172736763954163, + -1.8861355781555176, + 0.45078086853027344, + 0.29641321301460266 + ], + [ + -0.7025277018547058, + 0.5556451678276062, + 2.3374006748199463, + -1.9256110191345215, + -1.7495018243789673, + 0.9721168279647827, + 1.8326293230056763, + 0.5355191826820374, + -0.5788277983665466, + 0.19061005115509033, + -0.6322742700576782, + -3.031383514404297, + -0.1651831567287445, + 0.8584551215171814, + 0.2280312329530716, + 0.27221786975860596, + -0.3728860020637512, + 1.102063775062561, + -2.5995376110076904, + 0.7578867673873901, + -0.3784477412700653 + ], + [ + -7.212162017822266e-06, + 0.552659273147583, + 2.0397679805755615, + -1.8501746654510498, + -2.5457160472869873, + 1.0514975786209106, + 2.5270917415618896, + -0.7367458343505859, + 0.7676748037338257, + 0.7428221702575684, + -0.538831353187561, + -2.7423243522644043, + 0.07969537377357483, + 1.0069079399108887, + -0.8100140690803528, + -0.9174093008041382, + -0.9411231875419617, + 1.0430617332458496, + -1.4683198928833008, + 1.0334573984146118, + 0.4721405506134033 + ], + [ + -0.41159626841545105, + 0.41878798604011536, + 1.6363914012908936, + -2.4960880279541016, + -3.049954891204834, + -0.09567022323608398, + 2.3269245624542236, + 0.11159682273864746, + -0.7885257005691528, + 0.44582536816596985, + -0.47802892327308655, + -1.5574463605880737, + 1.0153859853744507, + 2.258706569671631, + -0.36591836810112, + 0.49115514755249023, + -0.9523675441741943, + 0.5892280340194702, + -2.1512646675109863, + -0.7744256854057312, + 0.592117428779602 + ], + [ + 0.1343594193458557, + 0.45838433504104614, + 2.6735801696777344, + -2.005676746368408, + -3.340463161468506, + 1.0152086019515991, + 1.429718255996704, + 0.7929191589355469, + -0.2598758339881897, + 1.6483545303344727, + -0.7411167025566101, + -2.8978958129882812, + 1.3759692907333374, + 1.4385837316513062, + 0.2961248755455017, + -0.042098239064216614, + -0.6380948424339294, + 1.0523303747177124, + -2.1783251762390137, + -0.045405834913253784, + 0.7960561513900757 + ] + ], + "predicted_scores": [ + 4.203157901763916, + 3.707049608230591, + 3.626953125, + 3.5704145431518555, + 3.536304473876953, + 3.1239418983459473, + 3.1084041595458984, + 2.9858627319335938 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "2141802816", + "selected_index": 0, + "state_hash": "d1cf06bba4675ad6a6942dc4f6fbade890fbf21b5c3068f8bbe30cfc60c54d9b", + "stored_base_utility": 0.0006054937839508057, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.12855732440948486, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12855732440948486 + }, + { + "contact_quality": null, + "progress": 0.12833033502101898, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.12833033502101898 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.09257189929485321, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09257189929485321 + }, + { + "contact_quality": null, + "progress": 0.0656263679265976, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0656263679265976 + }, + { + "contact_quality": null, + "progress": 0.060852110385894775, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060852110385894775 + }, + { + "contact_quality": null, + "progress": 0.1262555718421936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1262555718421936 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.12855732440948486, + 0.12833033502101898, + 1.0, + 0.09257189929485321, + 0.0656263679265976, + 0.060852110385894775, + 0.1262555718421936, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-398ddbf730ed8830b08b", + "ms-2ba02dcefa15a9371706", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b", + "ms-398ddbf730ed8830b08b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + true, + false, + false, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-121bc2e6676c1ac10911", + "generated_tangents": [ + [ + -0.7257179617881775, + -0.03575143963098526, + 0.23788043856620789, + -0.9687948226928711, + -0.9038702845573425, + 0.10042782127857208, + 1.625234842300415, + 0.27071014046669006, + 0.1942325383424759, + -0.14601635932922363, + 0.39276736974716187, + -1.3444900512695312, + -0.041935503482818604, + 1.0646857023239136, + -0.3098151385784149, + -0.05423483997583389, + -0.6402682662010193, + 0.3935595154762268, + -0.45741546154022217, + -0.27093175053596497, + 0.4617878198623657 + ], + [ + -0.49222296476364136, + -0.02527921460568905, + 0.19154101610183716, + -1.0117971897125244, + -1.118886947631836, + 0.06761611253023148, + 1.7967050075531006, + -0.027564451098442078, + 0.2803365886211395, + 0.09631343185901642, + 0.2927677631378174, + -1.0953946113586426, + 0.08128461241722107, + 1.87663996219635, + -0.21061468124389648, + -0.16087670624256134, + -0.4095489978790283, + 0.2142687737941742, + -0.25936591625213623, + -0.0271415114402771, + 0.04426760971546173 + ], + [ + -0.6047983765602112, + -0.04153505340218544, + 0.033416807651519775, + -0.7612345218658447, + -1.0191209316253662, + -0.46741023659706116, + 1.0215144157409668, + -0.18060284852981567, + 0.2781020998954773, + -0.12955832481384277, + 0.24962148070335388, + -0.9737802743911743, + -0.004270076751708984, + 0.5479865074157715, + -0.11303319036960602, + -0.04215051978826523, + -0.5573393106460571, + 0.29055196046829224, + -0.832116425037384, + -0.6861945390701294, + 0.33294442296028137 + ], + [ + -0.7405837178230286, + 0.16284878551959991, + 0.13304874300956726, + -1.1297900676727295, + -1.0768933296203613, + -0.027847886085510254, + 1.5588839054107666, + 0.1773626208305359, + 0.229502871632576, + 0.03195039927959442, + 0.2792961597442627, + -0.8868368864059448, + -0.4169521927833557, + 1.0307973623275757, + -0.28355836868286133, + 0.039785824716091156, + -0.6305896043777466, + 0.32707977294921875, + -0.42981040477752686, + -0.300695538520813, + 0.6085741519927979 + ], + [ + -0.8752638101577759, + 0.15627846121788025, + -0.2642001807689667, + -1.4300812482833862, + -0.9086292386054993, + -0.40588781237602234, + 1.422707438468933, + 0.35890230536460876, + 0.32191863656044006, + 0.1711602658033371, + 0.07514350861310959, + -1.0588886737823486, + -0.037381649017333984, + 0.8639682531356812, + -0.3543846607208252, + 0.05986660346388817, + -0.5966168642044067, + 0.5338371992111206, + -0.4685141444206238, + -0.7181501388549805, + 0.8448859453201294 + ], + [ + -0.6440788507461548, + 0.26109427213668823, + 0.1002824604511261, + -1.1860899925231934, + -1.2110652923583984, + 0.2786216735839844, + 1.4428701400756836, + 0.049609556794166565, + 0.30572041869163513, + 0.013519495725631714, + 0.3673737943172455, + -0.9484045505523682, + -0.38821327686309814, + 1.107595682144165, + -0.11746107041835785, + -0.10040338337421417, + -0.514596700668335, + 0.2826034128665924, + -0.5153048038482666, + -0.17715111374855042, + 0.11471407860517502 + ], + [ + -0.7710386514663696, + 0.20785890519618988, + 0.002715498208999634, + -1.027133822441101, + -0.7781583666801453, + 0.05032403767108917, + 1.601724624633789, + 0.1300971806049347, + 0.20471708476543427, + -0.07448870688676834, + 0.30200162529945374, + -0.6621583104133606, + -0.41534000635147095, + 0.9123588800430298, + -0.32488465309143066, + -0.03074706345796585, + -0.612622082233429, + 0.38867878913879395, + -0.6351329684257507, + -0.41679513454437256, + 0.6185945868492126 + ], + [ + -0.18245281279087067, + 0.06815386563539505, + 0.19148990511894226, + -0.6779538989067078, + -0.4827861785888672, + -0.36563435196876526, + 1.8912129402160645, + 0.09653539210557938, + 0.28170448541641235, + -0.11424743384122849, + 0.30296146869659424, + -0.5772009491920471, + -0.19625002145767212, + 1.2267690896987915, + 0.05415017157793045, + 0.10929052531719208, + -0.21586258709430695, + 0.4771376848220825, + -0.36779356002807617, + -0.0263901948928833, + 0.815432071685791 + ] + ], + "generated_utilities": [ + 0.12855732440948486, + 0.12833033502101898, + 2.0, + 0.09257189929485321, + 0.0656263679265976, + 0.060852110385894775, + 0.1262555718421936, + 2.0 + ], + "group_id": "ms-121bc2e6676c1ac10911", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.15086030960083008, + 0.08354458212852478, + 2.0, + 0.1453719437122345, + 0.19646552205085754, + 0.10343040525913239, + 0.1443832516670227, + 0.13976123929023743, + 0.17431864142417908, + 2.0, + 0.12949708104133606, + 2.0, + 2.0, + 0.11397694051265717, + 0.1271887570619583, + 0.025320395827293396 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.39665424823760986, + 0.08244594186544418, + -0.026901081204414368, + -0.959688663482666, + -1.2648048400878906, + 0.05568990111351013, + 0.0, + -0.048488348722457886, + 0.03576555848121643, + -0.012020409107208252, + 0.08576416969299316, + -0.25812816619873047, + 0.03589746356010437, + 0.0, + 0.0016258209943771362, + -0.04025750607252121, + 0.010670006275177002, + 0.5142596960067749, + 0.06220144033432007, + 0.07906517386436462, + 0.0 + ], + [ + -2.15720796585083, + 0.13025060296058655, + 0.41441529989242554, + -1.4498568773269653, + -2.6386165618896484, + 0.5572659969329834, + 0.7093658447265625, + -1.0835764408111572, + -0.896172285079956, + 0.07744038105010986, + 1.1224509477615356, + -1.6321502923965454, + 0.877888560295105, + 0.4978562295436859, + 0.5774499177932739, + 0.8093571662902832, + -0.8521628379821777, + 2.054164171218872, + -0.6021197438240051, + 1.0350128412246704, + 0.823830246925354 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.39665424823760986, + -0.2284793257713318, + -0.027931194752454758, + -0.959688663482666, + -1.2648048400878906, + -0.17197783291339874, + 0.0, + -0.06679429113864899, + 0.1646350920200348, + -0.11601293087005615, + 0.08576416969299316, + -0.25812816619873047, + 0.1479925513267517, + 0.0, + -0.04833713173866272, + 0.22201494872570038, + -0.16177666187286377, + 0.5142596960067749, + -0.10178101062774658, + 0.21631890535354614, + 0.0 + ], + [ + -0.8824495077133179, + 0.0034109503030776978, + -0.5904110074043274, + -1.400491714477539, + -2.4447474479675293, + -0.00789790228009224, + 1.6208159923553467, + 0.26819390058517456, + -0.23575741052627563, + -0.7184416651725769, + 0.9072303771972656, + -1.2077033519744873, + -0.21496516466140747, + 0.8258882761001587, + -0.4575563669204712, + -0.3655202388763428, + -0.5908474922180176, + 2.0210399627685547, + -1.2132787704467773, + 0.9719676971435547, + 0.24625355005264282 + ], + [ + -0.6953714489936829, + 0.4026735723018646, + 0.6648998856544495, + -1.356613278388977, + -2.953742742538452, + 0.037713564932346344, + 1.53464937210083, + 0.027994811534881592, + 0.03435474634170532, + -1.1939163208007812, + 0.9877840280532837, + -1.180513620376587, + 0.4064505994319916, + 0.5743880867958069, + 0.1558205783367157, + 0.3796072006225586, + -0.6676555275917053, + 1.4665511846542358, + 0.04170459508895874, + 0.17645099759101868, + 0.2388516366481781 + ], + [ + -1.4095232486724854, + -0.06358731538057327, + -0.7774888873100281, + -2.806776285171509, + -2.44041109085083, + -0.8258867859840393, + 2.383592128753662, + -0.093393474817276, + -0.24701429903507233, + -0.5286431312561035, + 0.7741218209266663, + -2.111828565597534, + -0.31807881593704224, + 1.3003538846969604, + -0.9306488037109375, + -0.8708829283714294, + -0.3694953918457031, + 2.169797420501709, + -0.7153271436691284, + 0.029519766569137573, + -0.5629553198814392 + ] + ], + "predicted_scores": [ + 5.867988109588623, + 5.841307640075684, + 5.431830406188965, + 5.335111618041992, + 5.190425872802734, + 5.006448268890381, + 4.667802333831787, + 4.228028297424316 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264", + "selected_index": 0, + "state_hash": "d1280a9e9362f8da3290de84fdd0f0b78475d470cd0230e590ccaf8f5428548a", + "stored_base_utility": 0.15086030960083008, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.11496219038963318, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11496219038963318 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1255601942539215, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1255601942539215 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 0.11496219038963318, + 1.0, + 1.0, + 0.1255601942539215, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9", + "ms-6825a9112cb71ea137c9" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + false, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-22f153f0d4fa656018e6", + "generated_tangents": [ + [ + -0.7708300352096558, + 0.15459537506103516, + 0.6022915840148926, + -1.4455561637878418, + -2.0376839637756348, + 0.2948944866657257, + 0.34606489539146423, + 0.020299717783927917, + 0.050667427480220795, + 0.0030802711844444275, + 0.10662112385034561, + -1.3358653783798218, + -0.06966105103492737, + 0.8762516379356384, + 0.17116057872772217, + 0.029481906443834305, + -0.3574213981628418, + 0.29465121030807495, + -0.7650296092033386, + -0.015099853277206421, + 0.5462791919708252 + ], + [ + -0.559061586856842, + 0.14502456784248352, + 0.45190829038619995, + -1.0738792419433594, + -1.5723164081573486, + -0.5392160415649414, + 0.7303147315979004, + -6.195902824401855e-05, + -0.09901300817728043, + 0.10049861669540405, + 0.051869314163923264, + -1.1138311624526978, + 0.033548980951309204, + 0.33838987350463867, + -0.11065022647380829, + 0.03068166971206665, + -0.1765679270029068, + 0.17674095928668976, + -0.6770181059837341, + -0.363102525472641, + 0.18358556926250458 + ], + [ + -0.810593843460083, + 0.18474450707435608, + 0.5427998900413513, + -1.2668769359588623, + -2.039551258087158, + 0.170603409409523, + 0.5517785549163818, + 0.030628055334091187, + -0.008206840604543686, + 0.023059621453285217, + 0.11736996471881866, + -1.446105718612671, + 0.020194947719573975, + 0.6126370429992676, + 0.044161491096019745, + 0.0003640996292233467, + -0.31732720136642456, + 0.24351078271865845, + -0.7596177458763123, + 0.2315993309020996, + 0.34795206785202026 + ], + [ + -0.8680835962295532, + 0.12034939229488373, + 0.8356432914733887, + -1.1642320156097412, + -2.1295688152313232, + -0.17883101105690002, + 0.3872016668319702, + -0.05696237087249756, + 0.0003417814150452614, + 0.014565914869308472, + -0.0069030895829200745, + -1.2308298349380493, + -0.11796659231185913, + 0.8523040413856506, + 0.11972692608833313, + 0.021913524717092514, + -0.2314680963754654, + 0.33641567826271057, + -0.6810120940208435, + 0.09716376662254333, + 0.28725558519363403 + ], + [ + -0.7122457027435303, + 0.18187060952186584, + 0.5787478089332581, + -1.2160292863845825, + -1.988480806350708, + 0.17790736258029938, + 0.5816813111305237, + 0.05637797713279724, + -0.05884978175163269, + 0.09957218915224075, + 0.14835883677005768, + -1.6232458353042603, + -0.06337305903434753, + 0.6757876873016357, + 0.21428130567073822, + -0.025791537016630173, + -0.3279285430908203, + 0.24416449666023254, + -0.9319389462471008, + 0.5412918925285339, + 0.054402247071266174 + ], + [ + -0.8143599033355713, + 0.1726515293121338, + 0.6953363418579102, + -1.250404953956604, + -2.039469003677368, + -0.08825115859508514, + 0.5160762071609497, + 0.063111811876297, + 0.07397442311048508, + -0.04727152734994888, + 0.19868794083595276, + -1.1970100402832031, + -0.25815117359161377, + 0.4863189458847046, + -0.17198443412780762, + 0.06569875776767731, + -0.2151908427476883, + 0.3011682629585266, + -0.7800189852714539, + 0.37843602895736694, + 0.6221891641616821 + ], + [ + -1.1280444860458374, + 0.2126903384923935, + 0.3045387268066406, + -1.3631364107131958, + -2.1111292839050293, + 0.27852940559387207, + 0.5967617630958557, + 0.19003501534461975, + -0.06598386913537979, + 0.22563880681991577, + 0.1391657292842865, + -1.6498486995697021, + -0.7170576453208923, + 0.4736671447753906, + 0.08535799384117126, + -0.06540680676698685, + -0.17089809477329254, + 0.21565906703472137, + -0.568437933921814, + 0.5021947622299194, + 0.39886555075645447 + ], + [ + -0.37826770544052124, + 0.1162397563457489, + 0.7141335606575012, + -0.8773645162582397, + -1.4110991954803467, + 0.015673592686653137, + 0.07687386870384216, + 0.025286629796028137, + -0.06477708369493484, + 0.0015392526984214783, + 0.06454146653413773, + -0.9371962547302246, + 0.03644940257072449, + 0.24814444780349731, + -0.11165717244148254, + 0.0296492800116539, + 0.07360316812992096, + 0.026344329118728638, + -0.5505878925323486, + 0.08845961093902588, + 0.28262630105018616 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 0.11496219038963318, + 2.0, + 2.0, + 0.1255601942539215, + 2.0 + ], + "group_id": "ms-22f153f0d4fa656018e6", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 2.60770320892334e-06, + 3.129243850708008e-07, + 6.109476089477539e-07, + 2.0, + 0.20781123638153076, + 2.0, + 0.16136802732944489, + 0.15920132398605347, + 0.16996727883815765, + 0.14405608177185059, + 0.15874098241329193, + 0.08526891469955444, + 0.0828208178281784, + 2.0, + 0.034265756607055664, + 2.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.3937166929244995, + 0.17073306441307068, + -0.5225132703781128, + -0.8479964733123779, + -1.713785171508789, + -0.19215916097164154, + 0.0, + 0.8004336953163147, + 0.4925512969493866, + 0.3161412179470062, + 0.10414126515388489, + -0.9857386350631714, + -0.4383167624473572, + 0.0, + 0.05412057042121887, + 0.021832358092069626, + 0.5262433290481567, + -0.24631118774414062, + -0.20008409023284912, + -0.04257836565375328, + 0.0 + ], + [ + -1.3937166929244995, + 0.2384568303823471, + 0.3933030962944031, + -1.847996473312378, + -2.713785171508789, + 0.20405007898807526, + 1.2454378604888916, + 0.0006654821336269379, + -0.07783810049295425, + -0.002305123955011368, + 0.18527865409851074, + -1.9857386350631714, + -0.009703370742499828, + 0.7078995108604431, + -0.08121174573898315, + -0.031056255102157593, + -0.40927961468696594, + 0.5954965949058533, + -0.8039777874946594, + 0.05187072232365608, + 0.2283477634191513 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.490875720977783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.4157990217208862, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4566955268383026 + ], + [ + -1.5219146013259888, + 0.23501667380332947, + 0.5663670301437378, + -1.892876386642456, + -2.744781494140625, + -0.04844887554645538, + 1.3053655624389648, + -0.11823110282421112, + -0.02477371320128441, + -0.17061075568199158, + 0.22949494421482086, + -1.854048252105713, + -0.025548618286848068, + 0.7958164215087891, + 0.12320645153522491, + -0.23218455910682678, + -0.6995364427566528, + 0.8081986308097839, + -0.558222234249115, + -0.18823161721229553, + 0.04578563570976257 + ], + [ + -1.608424186706543, + 0.022089049220085144, + 0.15639334917068481, + -1.982436180114746, + -2.6509876251220703, + -0.01820056140422821, + 0.95000159740448, + -0.08617040514945984, + -0.46914470195770264, + 0.3453706204891205, + -0.020491018891334534, + -1.738605260848999, + -0.0859340950846672, + 0.443394273519516, + 0.09907735884189606, + -0.38721394538879395, + -0.3415156602859497, + 0.524202287197113, + -0.7343614101409912, + -0.22556106746196747, + 0.4535057842731476 + ], + [ + -1.7957028150558472, + 0.593864381313324, + 0.06846150755882263, + -2.0563552379608154, + -2.2595136165618896, + -0.12188448011875153, + 0.9899688959121704, + -0.14783401787281036, + -0.22327497601509094, + -0.32116562128067017, + -0.07331573963165283, + -1.910165786743164, + 0.264303058385849, + 1.0567166805267334, + 0.016188345849514008, + 0.159999281167984, + -0.22885318100452423, + 0.705758273601532, + -0.8778964877128601, + 0.4381847381591797, + 0.19897118210792542 + ], + [ + -1.499897837638855, + -0.2043466418981552, + 0.42094048857688904, + -1.430327296257019, + -3.1506710052490234, + -0.08664844930171967, + 1.4003938436508179, + 0.24064289033412933, + -0.35471564531326294, + -0.37839531898498535, + 0.47059693932533264, + -1.7703200578689575, + 0.101543128490448, + 0.8470064401626587, + 0.16171279549598694, + -0.0019259154796600342, + -0.6455138325691223, + 0.5694639682769775, + -0.9997703433036804, + 0.5366826057434082, + -0.11106891930103302 + ], + [ + -1.8847153186798096, + 0.21543154120445251, + 0.9309400320053101, + -2.1095237731933594, + -2.136512517929077, + 0.6948637366294861, + 1.233637809753418, + 0.577111542224884, + -0.5727867484092712, + -0.6055393815040588, + 0.8738593459129333, + -2.6654083728790283, + -0.171565979719162, + 0.8934992551803589, + 0.17394188046455383, + 0.6209473609924316, + -0.1149752140045166, + 0.6160256266593933, + -0.5519447326660156, + 0.7195507287979126, + 0.5175096392631531 + ], + [ + -2.138639450073242, + 0.36738795042037964, + -0.33481496572494507, + -1.9255597591400146, + -2.2420146465301514, + 0.31966179609298706, + 1.3172208070755005, + -0.48551350831985474, + 0.27687638998031616, + -0.41806119680404663, + 0.05362778902053833, + -2.3665032386779785, + -0.47635504603385925, + 0.081889808177948, + 0.542697548866272, + -0.7215265035629272, + -0.906980037689209, + 0.6977691054344177, + -1.4041197299957275, + -0.11211369931697845, + 0.5149455070495605 + ], + [ + -1.4869698286056519, + -0.30130720138549805, + 1.28191077709198, + -0.9839465618133545, + -2.2501890659332275, + -0.20459605753421783, + 1.3607591390609741, + 0.6349532008171082, + -0.8627415895462036, + -0.623334527015686, + 0.21189503371715546, + -2.2093186378479004, + -0.4655338525772095, + 0.7016672492027283, + 0.6916180849075317, + -0.0807163268327713, + 0.1465948522090912, + 0.05083376169204712, + -0.8381134271621704, + 0.015291888266801834, + 0.05565965175628662 + ], + [ + -2.365525245666504, + 0.45377761125564575, + 0.2984481453895569, + -1.3690096139907837, + -2.843494176864624, + 0.32462257146835327, + 0.5082458257675171, + 0.7033615112304688, + 0.31817564368247986, + -0.4890395700931549, + 0.09036337584257126, + -2.904804229736328, + 0.560937762260437, + 1.649811029434204, + -0.9795587062835693, + -0.46419358253479004, + -0.4739357829093933, + 0.34800490736961365, + -1.0712809562683105, + 0.9167320132255554, + -0.48252081871032715 + ], + [ + -1.2987136840820312, + 0.7370830774307251, + -0.49668365716934204, + -1.201794147491455, + -3.552318572998047, + -0.5406968593597412, + 2.1693286895751953, + 0.5925421714782715, + 0.7903083562850952, + 0.991620659828186, + 0.7936639785766602, + -2.3832802772521973, + -0.621701180934906, + 0.43542394042015076, + -0.14415577054023743, + 0.15921397507190704, + -0.395956814289093, + -0.2778439521789551, + -1.069305419921875, + -0.5347616076469421, + -0.26581722497940063 + ] + ], + "predicted_scores": [ + 4.533088684082031, + 4.359924793243408, + 4.246114253997803, + 3.96541428565979, + 3.6263058185577393, + 3.6202778816223145, + 3.4237430095672607, + 2.6584508419036865 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007", + "selected_index": 0, + "state_hash": "3139b3ac90f6fb958f14d875d082c95c8569f039086e0789e6c9e079d602b945", + "stored_base_utility": 2.60770320892334e-06, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.0320269912481308, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0320269912481308 + }, + "base_progress": 0.0320269912481308, + "base_success": false, + "base_utility": 0.0320269912481308, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.040287479758262634, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.040287479758262634 + }, + { + "contact_quality": null, + "progress": 0.0174645334482193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0174645334482193 + }, + { + "contact_quality": null, + "progress": 0.021355673670768738, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021355673670768738 + }, + { + "contact_quality": null, + "progress": 0.018107041716575623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018107041716575623 + }, + { + "contact_quality": null, + "progress": 0.018239781260490417, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.018239781260490417 + }, + { + "contact_quality": null, + "progress": 0.022067666053771973, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022067666053771973 + }, + { + "contact_quality": null, + "progress": 0.038215383887290955, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.038215383887290955 + }, + { + "contact_quality": null, + "progress": 0.019529670476913452, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019529670476913452 + } + ], + "candidate_progress": [ + 0.040287479758262634, + 0.0174645334482193, + 0.021355673670768738, + 0.018107041716575623, + 0.018239781260490417, + 0.022067666053771973, + 0.038215383887290955, + 0.019529670476913452 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-4e67cfd30a257b4bad0c", + "ms-ad5107846b45eae5a0ab", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-940cce8479900a244d54", + "ms-543339410c02ce2269d1", + "ms-2423891634e7b469eed8" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "generated_tangents": [ + [ + 0.14018268883228302, + 0.289065420627594, + 0.8483375310897827, + -1.7791224718093872, + 0.4814610779285431, + -0.18178966641426086, + 0.12895378470420837, + 0.069358691573143, + -0.059383705258369446, + 0.035258613526821136, + 0.13515865802764893, + -0.5167298913002014, + 0.050205498933792114, + 0.11525478214025497, + 0.0007116049528121948, + -0.15094910562038422, + 0.04490123689174652, + -0.037689417600631714, + -0.5086848139762878, + -0.211318701505661, + -0.05127803981304169 + ], + [ + 0.17016763985157013, + 0.34602171182632446, + 0.9436938166618347, + -1.9131704568862915, + 0.4846016764640808, + -0.19932904839515686, + 0.4778391718864441, + -0.1454114317893982, + -0.04693783074617386, + 0.11983165889978409, + 0.05753686651587486, + -0.5236656665802002, + -0.26676976680755615, + 0.40207332372665405, + 0.04501873999834061, + -0.08188727498054504, + 0.19682645797729492, + -0.2202015370130539, + -0.7720839381217957, + -0.12631845474243164, + -0.1130034327507019 + ], + [ + 0.1542426496744156, + 0.24253727495670319, + 0.9044508934020996, + -1.724196195602417, + 0.46095937490463257, + -0.05063405632972717, + 0.19632071256637573, + -0.022570163011550903, + 0.04484015703201294, + 0.12057716399431229, + 0.12402200698852539, + -0.47842562198638916, + 0.07320907711982727, + 0.05156547576189041, + 0.004217091947793961, + -0.1772926151752472, + -0.08882392942905426, + -0.1795724779367447, + -0.670177698135376, + -0.034486472606658936, + -0.07198132574558258 + ], + [ + 0.2664527893066406, + 0.23964805901050568, + 0.7832643389701843, + -1.741670846939087, + 0.2732876241207123, + -0.0017767548561096191, + 0.42830294370651245, + 0.06510612368583679, + 0.08872084319591522, + 0.15838471055030823, + 0.2198942005634308, + -0.402571439743042, + -0.006391555070877075, + 0.34281936287879944, + -0.12248881161212921, + -0.23056839406490326, + -0.10353462398052216, + -0.16413728892803192, + -0.4925073981285095, + 0.07817566394805908, + -0.022388413548469543 + ], + [ + 0.2701839804649353, + 0.24286682903766632, + 0.7585902214050293, + -1.760545015335083, + 0.40861284732818604, + -0.050659388303756714, + 0.45358529686927795, + -0.009043246507644653, + 0.10340894758701324, + 0.21402087807655334, + 0.1538773477077484, + -0.46348661184310913, + 0.06407955288887024, + 0.2675989270210266, + -0.03547931835055351, + -0.20605412125587463, + -0.11561249196529388, + -0.21442778408527374, + -0.6482141613960266, + 0.11941751837730408, + -0.1081092357635498 + ], + [ + 0.2603517770767212, + 0.29136765003204346, + 0.8377959728240967, + -1.7505241632461548, + 0.45684218406677246, + -0.02211475372314453, + 0.1808648705482483, + -0.02677808701992035, + 0.09574203193187714, + 0.0705883800983429, + 0.11149191856384277, + -0.38531720638275146, + -0.01217949390411377, + 0.1385795623064041, + -0.01819026842713356, + -0.1604638397693634, + -0.08709345757961273, + -0.17443491518497467, + -0.6340339779853821, + -0.24893814325332642, + 0.13137471675872803 + ], + [ + 0.10570760071277618, + 0.2851371765136719, + 0.8287855982780457, + -1.7305612564086914, + 0.5774431228637695, + -0.0024221092462539673, + 0.18119779229164124, + 0.023360982537269592, + 0.08462734520435333, + 0.09469969570636749, + 0.18955576419830322, + -0.4194323420524597, + -0.09402373433113098, + 0.10175512731075287, + -0.06348559260368347, + -0.12264516949653625, + -0.009652271866798401, + -0.08393912017345428, + -0.5841110944747925, + -0.0612044632434845, + 0.002477407455444336 + ], + [ + 0.22996777296066284, + 0.24743469059467316, + 0.7988656759262085, + -1.8694814443588257, + 0.2585720121860504, + 0.1517871767282486, + 0.6424685716629028, + 0.05439126491546631, + 0.15564747154712677, + 0.11952495574951172, + 0.09387843310832977, + -0.5468964576721191, + 0.1126844584941864, + 0.07026393711566925, + -0.02239188179373741, + -0.06617730110883713, + -0.18370158970355988, + -0.178851917386055, + -0.7413548827171326, + 0.10490772128105164, + -0.14298400282859802 + ] + ], + "generated_utilities": [ + 0.040287479758262634, + 0.0174645334482193, + 0.021355673670768738, + 0.018107041716575623, + 0.018239781260490417, + 0.022067666053771973, + 0.038215383887290955, + 0.019529670476913452 + ], + "group_id": "ms-27c9f70e97b8850cc3ab", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.09440004825592041, + 0.14095592498779297, + 2.0, + 0.09219714999198914, + 0.05765572190284729, + 0.02530825138092041, + 0.07676126062870026, + 0.06643831729888916, + 0.047928065061569214, + 0.07107676565647125, + 0.05561619997024536, + 0.02124820649623871, + 0.08948668837547302, + 0.10203869640827179, + 0.06119762361049652, + 0.04547174274921417 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.984761953353882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.7230467796325684, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.554086923599243 + ], + [ + 1.0343247652053833, + 1.287368655204773, + 1.867376446723938, + -2.4465248584747314, + 1.036996841430664, + -1.2664310932159424, + 1.4399157762527466, + -0.8265262246131897, + -0.28319495916366577, + 0.16331201791763306, + -1.5917816162109375, + -1.5816115140914917, + -0.1664504110813141, + 0.7195659279823303, + 0.29144588112831116, + -0.18517816066741943, + -0.08826595544815063, + -0.8266173601150513, + -0.7933371067047119, + -0.938305139541626, + 0.5359267592430115 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.3968086242675781, + 0.5359851121902466, + 0.7176156044006348, + -1.52370023727417, + -0.1481943428516388, + 0.10887271165847778, + 0.0, + -0.12331700325012207, + 0.14385993778705597, + -0.041522324085235596, + 0.16089874505996704, + -0.6189663410186768, + -0.01699581742286682, + 0.0, + -0.07216246426105499, + -0.15073662996292114, + -0.134138286113739, + -0.06049776077270508, + -0.26776671409606934, + -0.019470810890197754, + 0.0 + ] + ], + "predicted_scores": [ + 2.6910204887390137, + 2.5916354656219482, + 2.4882662296295166, + 2.3931052684783936, + 2.295912742614746, + 2.16906476020813, + 2.1331605911254883, + 1.6332793235778809 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1881223807", + "selected_index": 0, + "state_hash": "0f58a11c36c331c9abab00795189dccbc7e0aebf985ca9df67bf9da756f154e3", + "stored_base_utility": 0.09440004825592041, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.10841864347457886, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.10841864347457886 + }, + "base_progress": 0.10841864347457886, + "base_success": false, + "base_utility": 0.10841864347457886, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03331609070301056, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03331609070301056 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.03331609070301056, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-22cfd6a9a619133478b6", + "ms-22cfd6a9a619133478b6", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b", + "ms-663c7e7fdaea6a945a2b" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-2867deaca7e98582620b", + "generated_tangents": [ + [ + 0.16600991785526276, + 0.18908433616161346, + 0.33548200130462646, + -0.9058983325958252, + -1.540191411972046, + -0.1795315444469452, + -0.871669352054596, + -0.3932388126850128, + -0.15279380977153778, + 0.7445515394210815, + 0.05246046185493469, + -1.212005376815796, + 0.053169697523117065, + -0.07310402393341064, + -0.26925426721572876, + 0.06416788697242737, + 0.40964382886886597, + 0.3355107307434082, + -0.28585803508758545, + -0.3497779667377472, + -0.6202583312988281 + ], + [ + 0.07943939417600632, + 0.05913250148296356, + 0.6852797269821167, + -0.8930180072784424, + -1.4039890766143799, + -0.39491453766822815, + -1.018752098083496, + -0.18416669964790344, + 0.002063695341348648, + 0.6112512946128845, + 0.4535205662250519, + -1.0310211181640625, + -0.006541192531585693, + 0.08287917077541351, + -0.17422263324260712, + 0.04211186617612839, + 0.2024422585964203, + 0.3467860817909241, + -0.17872291803359985, + -0.18163183331489563, + -0.13720151782035828 + ], + [ + -0.04607692360877991, + 0.09997068345546722, + 0.6241083741188049, + -0.5203796625137329, + -1.1481754779815674, + -0.17407873272895813, + -1.150116205215454, + -0.22077441215515137, + 0.05634499341249466, + 0.5917983651161194, + 0.5587509870529175, + -1.1554293632507324, + 0.06377473473548889, + -0.16887415945529938, + -0.010603468865156174, + 0.09912288188934326, + 0.5224568247795105, + 0.060127824544906616, + -0.22322219610214233, + 0.021599620580673218, + -0.8569719791412354 + ], + [ + 0.09243831783533096, + 0.06287702918052673, + 0.7539478540420532, + -1.0097072124481201, + -1.5342717170715332, + -0.6428382396697998, + -1.2201895713806152, + -0.27795061469078064, + -0.02100360207259655, + 0.5988133549690247, + 0.28385302424430847, + -1.287218451499939, + 0.13356050848960876, + 0.13597771525382996, + -0.18391810357570648, + 0.07739007472991943, + 0.3036441206932068, + 0.30869847536087036, + -0.17749691009521484, + 0.09222644567489624, + -0.20486858487129211 + ], + [ + -0.08986543118953705, + 0.0525314062833786, + 0.6842445135116577, + -0.4866493046283722, + -1.1337988376617432, + -0.20357748866081238, + -1.1288089752197266, + -0.20260179042816162, + 0.05673578381538391, + 0.5497421622276306, + 0.568764328956604, + -1.12516450881958, + 0.026677489280700684, + 0.027614064514636993, + -0.026013080030679703, + 0.08397582173347473, + 0.5806421041488647, + 0.04677726328372955, + -0.19632792472839355, + 0.1386643946170807, + -0.8952764272689819 + ], + [ + -0.21484331786632538, + 0.35670602321624756, + 0.2919608950614929, + -1.321199893951416, + -1.2454345226287842, + -0.1845996379852295, + -1.170832872390747, + -0.4199824035167694, + -0.04239681363105774, + 0.45418572425842285, + 0.44296663999557495, + -1.0982940196990967, + -0.12045922875404358, + -0.027421846985816956, + -0.1989881694316864, + 0.27994588017463684, + 0.45071840286254883, + 0.3256455063819885, + -0.2464848756790161, + -0.6050560474395752, + -0.5589829683303833 + ], + [ + 0.13482598960399628, + 0.3827795386314392, + 0.5731130242347717, + -1.3230628967285156, + -1.481137752532959, + -0.6478264331817627, + -1.032254695892334, + -0.1686381697654724, + 0.022745568305253983, + 0.34474611282348633, + 0.3553159832954407, + -1.062491536140442, + -0.06841418147087097, + 0.030543625354766846, + -0.219338059425354, + 0.2924506664276123, + 0.3867171108722687, + 0.581658661365509, + -0.06119430065155029, + -0.4303423762321472, + -0.4755922257900238 + ], + [ + 0.10067836940288544, + 0.30324530601501465, + 0.6015622615814209, + -1.295541763305664, + -1.3804712295532227, + -0.4262353479862213, + -1.076366901397705, + -0.20897558331489563, + 0.02361627109348774, + 0.40963461995124817, + 0.35281074047088623, + -0.9959433078765869, + -0.1752685010433197, + 0.004814960062503815, + -0.17192019522190094, + 0.3385526239871979, + 0.4245295226573944, + 0.6542333364486694, + -0.09239929914474487, + -0.3336179554462433, + -0.453787237405777 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.03331609070301056, + 2.0, + 2.0 + ], + "group_id": "ms-2867deaca7e98582620b", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0924256443977356, + 0.05646410584449768, + 2.0, + 2.0, + 0.183049738407135, + 2.0, + 0.16824382543563843, + 0.1424575299024582, + 0.08230158686637878, + 0.15871968865394592, + 0.16382521390914917, + 0.05336849391460419, + 0.07674244046211243, + 0.10727545619010925, + 0.06594061851501465, + 0.045542001724243164 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.44511327147483826, + 0.05827039107680321, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.2619132101535797, + 0.0, + -0.37528133392333984, + -0.07968637347221375, + -0.07514983415603638, + 0.19317352771759033, + 0.03868463635444641, + -0.06793206930160522, + 0.0, + 0.13187909126281738, + -0.20905672013759613, + 0.030073702335357666, + 0.21597933769226074, + -0.0761370062828064, + -0.11662125587463379, + 0.0 + ], + [ + -0.15486939251422882, + 0.24427275359630585, + 1.2608249187469482, + -0.35779595375061035, + -1.1953308582305908, + 0.44790399074554443, + 0.0, + -0.37528133392333984, + -0.29508399963378906, + 0.7311322093009949, + 0.19317352771759033, + 0.09925723075866699, + 0.1767415851354599, + 0.0, + -0.5768744349479675, + 0.1300974041223526, + 0.21151533722877502, + 0.21597933769226074, + 0.04431194067001343, + -0.040631115436553955, + 0.0 + ], + [ + -0.29166290163993835, + 0.09681949019432068, + 2.2608249187469482, + -1.3577959537506104, + -2.195330858230591, + -0.13884210586547852, + 2.5595173835754395, + -1.3752813339233398, + 0.390796959400177, + 0.673005223274231, + 1.1931735277175903, + -0.41860872507095337, + -0.24474714696407318, + 0.3015621602535248, + -0.2400326132774353, + 0.14207373559474945, + -0.26512715220451355, + 1.2159793376922607, + -0.3237651586532593, + -0.7466862201690674, + 0.30001604557037354 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.119034767150879, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6031243205070496, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.6000320911407471 + ], + [ + -0.3580354154109955, + 0.03320832550525665, + 2.5546932220458984, + -1.138798713684082, + -2.260084629058838, + -0.04964215308427811, + 2.8354909420013428, + -1.6732244491577148, + 0.21277914941310883, + 0.5097208023071289, + 1.387328028678894, + -0.691602349281311, + -0.28483790159225464, + 0.35576167702674866, + -0.0605899840593338, + 0.3764685094356537, + -0.28031954169273376, + 1.063202977180481, + -0.03841540217399597, + -0.5410391092300415, + 0.5580100417137146 + ], + [ + -0.5578174591064453, + 0.3727092146873474, + 2.1929497718811035, + -1.577579140663147, + -1.8962993621826172, + -0.2802738845348358, + 2.1670117378234863, + -1.0752147436141968, + 0.16372516751289368, + 0.6087679862976074, + 0.8854888081550598, + -0.18080148100852966, + -0.18821784853935242, + 0.010591715574264526, + -0.12560874223709106, + 0.15850795805454254, + -0.1898641288280487, + 1.1823316812515259, + 0.06655246019363403, + -0.4014361798763275, + 0.4376043379306793 + ], + [ + -0.43790310621261597, + -0.12778332829475403, + 2.6857643127441406, + -1.6293541193008423, + -1.9850949048995972, + -0.7167636752128601, + 2.465301513671875, + -1.4102312326431274, + 0.2071499079465866, + 0.5742072463035583, + 0.9525313377380371, + -0.34116798639297485, + -0.04760183393955231, + -0.05569759011268616, + -0.4777999520301819, + 0.17186680436134338, + 0.0841393768787384, + 0.7708102464675903, + -0.2784478962421417, + -1.2458651065826416, + 0.46140795946121216 + ], + [ + 0.06170770525932312, + 0.45555436611175537, + 1.6506444215774536, + -0.9394457340240479, + -1.9117670059204102, + -0.6910014152526855, + 2.40145206451416, + -1.7531975507736206, + 0.10217079520225525, + 1.2514894008636475, + 1.3236911296844482, + -0.6504120826721191, + -0.6943472027778625, + 0.7029284238815308, + -0.7387814521789551, + -0.33777421712875366, + -0.8187699317932129, + 1.8448829650878906, + 0.00849190354347229, + -1.394152045249939, + 0.918946385383606 + ] + ], + "predicted_scores": [ + 4.198071479797363, + 3.129955768585205, + 3.086650848388672, + 3.0081777572631836, + 2.726970672607422, + 2.0165863037109375, + 1.782904863357544, + 1.744734764099121 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1189392917", + "selected_index": 0, + "state_hash": "e8cf9b95f09f32d8e7942b114792725bd89c392a97ff7e6468ebf09df1bb0d21", + "stored_base_utility": 0.0924256443977356, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.13927479088306427, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13927479088306427 + }, + "base_progress": 0.13927479088306427, + "base_success": false, + "base_utility": 0.13927479088306427, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 3.081560134887695e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 3.081560134887695e-05 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 3.081560134887695e-05, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-365d8c6de6de16f66744", + "generated_tangents": [ + [ + -0.21114064753055573, + -0.048341378569602966, + 1.5556553602218628, + -0.46712633967399597, + -1.2199385166168213, + -0.11526814103126526, + -0.16461020708084106, + 0.05351035296916962, + -0.026479950174689293, + 0.16385047137737274, + 0.03798053413629532, + -0.9300800561904907, + -0.2876429557800293, + -0.07136794924736023, + 0.08146791905164719, + -0.026538491249084473, + 0.2775648832321167, + 0.22740747034549713, + -0.16802898049354553, + -0.20614074170589447, + 0.10743649303913116 + ], + [ + -0.1791655272245407, + -0.02371925674378872, + 1.6634361743927002, + -0.5424115657806396, + -1.3346848487854004, + 0.08280134201049805, + -0.23706042766571045, + -0.022989988327026367, + 0.016705770045518875, + 0.11423534154891968, + -0.03222975879907608, + -0.9869319200515747, + -0.16671717166900635, + 0.026621557772159576, + 0.031675536185503006, + -0.012221254408359528, + 0.2166295051574707, + 0.12822693586349487, + -0.07551956176757812, + -0.2610858082771301, + 0.051014289259910583 + ], + [ + -0.22534425556659698, + -0.08131547272205353, + 1.5393378734588623, + -0.5683905482292175, + -1.2945749759674072, + 0.07191448658704758, + -0.19457030296325684, + -0.020548522472381592, + -0.002910829149186611, + 0.16843777894973755, + -0.030726857483386993, + -1.1023364067077637, + -0.06796851754188538, + -0.17109696567058563, + -0.029572751373052597, + -0.02294149249792099, + 0.20655836164951324, + 0.15574046969413757, + -0.038037896156311035, + -0.20111629366874695, + 0.032548144459724426 + ], + [ + -0.06677797436714172, + 0.0060897208750247955, + 1.807574987411499, + -0.5565886497497559, + -1.4104347229003906, + -0.1659129559993744, + -0.35193121433258057, + 0.03796319663524628, + 0.0001925518736243248, + 0.07528190314769745, + -0.019746683537960052, + -1.056929349899292, + -0.1898556351661682, + 0.018602512776851654, + 0.027445875108242035, + -0.054877590388059616, + 0.1442004144191742, + 0.10452555865049362, + -0.09021466970443726, + -0.16082943975925446, + 0.11316534876823425 + ], + [ + -0.09619469940662384, + -0.05190189555287361, + 1.875879168510437, + -0.7030974626541138, + -1.4674580097198486, + -0.010268077254295349, + -0.5517680048942566, + 0.09250915050506592, + -0.09007631987333298, + 0.2208840250968933, + -0.04731278866529465, + -0.9954752922058105, + 0.28448495268821716, + -0.19590722024440765, + -0.08933983743190765, + 0.04485812783241272, + 0.2518169581890106, + 0.20833410322666168, + 0.01020514965057373, + -0.22327697277069092, + -0.3381204903125763 + ], + [ + -0.11923225224018097, + -0.10508862137794495, + 1.6521183252334595, + -0.7320802211761475, + -1.3820405006408691, + -0.09266479313373566, + -0.9676738381385803, + -0.013583093881607056, + 0.007066716905683279, + 0.1558513641357422, + -0.04582802206277847, + -0.9558316469192505, + 0.15382614731788635, + -0.16842050850391388, + -0.04179012402892113, + -0.013198169879615307, + 0.14783084392547607, + 0.3141729533672333, + -0.07083505392074585, + 0.0677485466003418, + -0.05980193614959717 + ], + [ + -0.2758737802505493, + 0.04080096632242203, + 1.3296016454696655, + -0.49317029118537903, + -1.1362073421478271, + 0.2666260600090027, + -0.5547595620155334, + -0.1566736102104187, + -0.08262604475021362, + 0.11275874823331833, + -0.14412081241607666, + -1.1537481546401978, + -0.27469709515571594, + 0.10920702666044235, + -0.0987781435251236, + -0.08442221581935883, + 0.212855264544487, + 0.11175546050071716, + 0.001248687505722046, + -0.29110726714134216, + -0.05398714542388916 + ], + [ + -0.07298408448696136, + -0.07261212170124054, + 1.5537549257278442, + -0.6544216871261597, + -1.4192097187042236, + -0.346295565366745, + -0.33383774757385254, + -0.0756678581237793, + -0.027217773720622063, + 0.10626167058944702, + 0.041009142994880676, + -1.1146725416183472, + -0.11862128973007202, + 0.13409720361232758, + -0.10111531615257263, + -0.11558949947357178, + 0.06267830729484558, + 0.07639651745557785, + 0.033487915992736816, + -0.17422418296337128, + 0.029652714729309082 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 3.081560134887695e-05, + 2.0 + ], + "group_id": "ms-365d8c6de6de16f66744", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20231953263282776, + 0.12319359183311462, + 2.0, + 2.0, + 0.1590207815170288, + 0.019343823194503784, + 0.1322992593050003, + 0.09479346871376038, + 0.1768283247947693, + 0.1054161787033081, + 0.16278749704360962, + 0.08510464429855347, + 0.09564858675003052, + 0.07681193947792053, + 0.1334480196237564, + 0.08148902654647827 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.09058855473995209, + -0.022760659456253052, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.07837581634521484, + 0.0, + -0.05228453874588013, + -0.0679071694612503, + 0.026890814304351807, + 0.03908693790435791, + -1.4526653289794922, + 0.04786187410354614, + 0.0, + 0.019858457148075104, + -0.05534793436527252, + 0.17386651039123535, + -0.024165034294128418, + -0.05880159139633179, + 0.044484615325927734, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.063174724578857, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6561365127563477, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0078073740005493 + ], + [ + 0.05168074369430542, + 0.27186232805252075, + 2.0911388397216797, + -1.6812132596969604, + -2.3280625343322754, + 0.38023167848587036, + 2.5090978145599365, + -0.2604735791683197, + 0.04725922644138336, + 0.8121852874755859, + -0.6659450531005859, + -2.2462568283081055, + 0.49988994002342224, + 1.9786770343780518, + 0.02324119210243225, + -0.45050501823425293, + 1.1065142154693604, + 0.5724118947982788, + -0.6320602893829346, + -0.7826599478721619, + 0.47463753819465637 + ], + [ + 0.46382713317871094, + -0.06331050395965576, + 2.657292366027832, + -1.1560972929000854, + -2.765941619873047, + 0.7805119752883911, + 2.3117406368255615, + -0.3644295036792755, + -0.25628525018692017, + 0.7623568773269653, + -0.4176616668701172, + -2.646364688873291, + 0.40904325246810913, + 1.7654002904891968, + -0.23315654695034027, + 0.026929572224617004, + 0.8236149549484253, + 0.8322737812995911, + -0.7612741589546204, + -0.4725078046321869, + 0.5895571708679199 + ], + [ + -0.26226741075515747, + 0.3922410011291504, + 2.450082302093506, + -0.9020822048187256, + -3.028742551803589, + 1.0746917724609375, + 3.104015350341797, + 0.3752234876155853, + -0.009036362171173096, + 0.3652568459510803, + -0.7608495950698853, + -2.337242603302002, + -0.03944078087806702, + 1.972575306892395, + -0.32381975650787354, + -0.45855021476745605, + 1.6927499771118164, + 0.8358478546142578, + -0.8610806465148926, + -0.6741821765899658, + 0.4013170003890991 + ], + [ + 0.5790615677833557, + 0.8308188915252686, + 3.141986846923828, + -1.218785285949707, + -2.0300495624542236, + 0.4300342798233032, + 3.0737316608428955, + -0.09167198091745377, + 0.4237035810947418, + 0.7421274185180664, + -0.637946367263794, + -2.1245827674865723, + 0.4184945523738861, + 1.2377147674560547, + -0.4068492650985718, + -0.03615438938140869, + 1.5385099649429321, + 0.51850825548172, + -0.5737811923027039, + -1.472043752670288, + 0.45800793170928955 + ], + [ + 0.38714414834976196, + 0.25513744354248047, + 2.9400248527526855, + -1.8116681575775146, + -1.7500431537628174, + 1.403240442276001, + 1.9709672927856445, + 0.6755956411361694, + 0.49445489048957825, + 0.19490671157836914, + -1.1990764141082764, + -2.803361177444458, + 0.499578058719635, + 1.164682149887085, + 0.9588776230812073, + 0.5154257416725159, + 1.0328537225723267, + 1.1309876441955566, + -0.9365650415420532, + -0.9663279056549072, + -0.032733798027038574 + ], + [ + 0.7417005896568298, + -0.12406697869300842, + 3.0325844287872314, + -0.5656485557556152, + -2.5084800720214844, + 0.27374324202537537, + 2.2886011600494385, + 0.5997821092605591, + 0.9623891115188599, + 0.38451504707336426, + -1.4501185417175293, + -1.6908833980560303, + 0.6743149757385254, + 1.8593690395355225, + -0.018024832010269165, + -0.20444278419017792, + 1.364742636680603, + 0.34230074286460876, + -1.4427728652954102, + -1.5577378273010254, + 0.5589802265167236 + ], + [ + 0.3448556661605835, + -0.5947309732437134, + 2.3888885974884033, + -1.0376405715942383, + -2.218167781829834, + 1.568857192993164, + 3.0995821952819824, + -0.4682580530643463, + 0.3072889745235443, + 0.14532917737960815, + -0.567415177822113, + -3.085655689239502, + -0.6656016111373901, + 1.6447856426239014, + 0.8135664463043213, + 0.3758860230445862, + 1.399398922920227, + 0.11528515815734863, + -0.531513512134552, + -0.6104787588119507, + 1.4474445581436157 + ], + [ + -0.18966330587863922, + -0.7208725214004517, + 2.586766481399536, + -0.6973344087600708, + -3.2187445163726807, + 1.4117884635925293, + 3.0797414779663086, + -0.29149553179740906, + 0.09620378166437149, + 0.874613881111145, + -0.8004729151725769, + -2.0231733322143555, + -0.5948910713195801, + 1.5080900192260742, + -0.750367283821106, + 0.8155628442764282, + 1.6562600135803223, + 0.8091130256652832, + -1.1569771766662598, + -1.4121384620666504, + 0.3738836348056793 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.06428352743387222, + -0.015524566173553467, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + -0.1607142686843872, + 0.0, + 0.27928173542022705, + 0.05664636939764023, + -0.03914147615432739, + 0.1700872778892517, + -1.4526653289794922, + 0.05555404722690582, + 0.0, + 0.18240894377231598, + -0.05239863693714142, + 0.17386651039123535, + -0.21921080350875854, + -0.3728579580783844, + 0.11250120401382446, + 0.0 + ], + [ + 0.008915487676858902, + 0.540733277797699, + 1.373640537261963, + -0.4681985378265381, + -1.599449634552002, + 0.22902190685272217, + 0.0, + -0.3544532060623169, + -0.2348114550113678, + 0.04456782341003418, + 0.2907915711402893, + -1.4526653289794922, + -0.1788860559463501, + 0.0, + 0.07762694358825684, + -0.07040916383266449, + 0.17386651039123535, + 0.44059625267982483, + -0.19159233570098877, + 0.09149748086929321, + 0.0 + ] + ], + "predicted_scores": [ + 4.233409404754639, + 4.0162577629089355, + 3.88030743598938, + 3.4916210174560547, + 3.4586877822875977, + 3.002124309539795, + 2.914055109024048, + 2.882695436477661 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1997090716", + "selected_index": 0, + "state_hash": "98870eab9f394b355af9b8a6d689b3bcd574af94db5d1a9139d0aed0b633b959", + "stored_base_utility": 0.20231953263282776, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03280790150165558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03280790150165558 + }, + "base_progress": 0.03280790150165558, + "base_success": false, + "base_utility": 0.03280790150165558, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.01805901527404785, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01805901527404785 + }, + { + "contact_quality": null, + "progress": 0.022866547107696533, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.022866547107696533 + }, + { + "contact_quality": null, + "progress": 0.019091889262199402, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019091889262199402 + }, + { + "contact_quality": null, + "progress": 0.01995408535003662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01995408535003662 + }, + { + "contact_quality": null, + "progress": 0.019698604941368103, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019698604941368103 + }, + { + "contact_quality": null, + "progress": 0.019849225878715515, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.019849225878715515 + }, + { + "contact_quality": null, + "progress": 0.023350223898887634, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023350223898887634 + }, + { + "contact_quality": null, + "progress": 0.02442134916782379, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02442134916782379 + } + ], + "candidate_progress": [ + 0.01805901527404785, + 0.022866547107696533, + 0.019091889262199402, + 0.01995408535003662, + 0.019698604941368103, + 0.019849225878715515, + 0.023350223898887634, + 0.02442134916782379 + ], + "candidate_source_chart_ids": [ + "ms-4e67cfd30a257b4bad0c", + "ms-2423891634e7b469eed8", + "ms-4e67cfd30a257b4bad0c", + "ms-543339410c02ce2269d1", + "ms-2cbe83da02847008bf24", + "ms-fe38b164397dd669f391", + "ms-3f6cd47af5e9e6781b73", + "ms-2cbe83da02847008bf24" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank4", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank5", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-3c326d05fec66e1d6bea", + "generated_tangents": [ + [ + 0.19276104867458344, + -0.6077608466148376, + 0.9373341798782349, + 0.4416253864765167, + 0.2649969458580017, + 0.46473005414009094, + -0.03351306915283203, + 0.27680742740631104, + 0.34446781873703003, + 0.24360504746437073, + -0.15033817291259766, + -0.5768869519233704, + -0.4378374516963959, + 0.08635913580656052, + -0.16154083609580994, + -0.013578517362475395, + -0.08223603665828705, + -0.14786718785762787, + -0.074046790599823, + -0.4710044860839844, + -0.2585410475730896 + ], + [ + 0.30996179580688477, + -0.625074565410614, + 0.9759076833724976, + 0.3689461052417755, + 0.09832823276519775, + 0.756193995475769, + -0.7694090008735657, + 0.3742751479148865, + 0.5233266353607178, + 0.10121019184589386, + -0.186079740524292, + -0.33603256940841675, + -0.4210893213748932, + -0.0727536678314209, + -0.08566601574420929, + -0.054874129593372345, + -0.26122283935546875, + -0.2174142748117447, + -0.13606640696525574, + -0.1925308108329773, + -0.030809640884399414 + ], + [ + 0.3530561327934265, + -0.5663501024246216, + 1.0013673305511475, + 0.5904250144958496, + 0.2888363003730774, + 0.5083624124526978, + -0.3288395404815674, + 0.19990651309490204, + 0.3434184193611145, + 0.16889619827270508, + -0.09934317320585251, + -0.6318717002868652, + -0.1830020248889923, + 0.05097272992134094, + -0.24534952640533447, + -0.0378064326941967, + -0.14884088933467865, + -0.13635729253292084, + 0.16512149572372437, + -0.3850916028022766, + -0.03494034707546234 + ], + [ + 0.24989527463912964, + -0.6553949117660522, + 0.9435427784919739, + 0.471303254365921, + 0.3789020776748657, + 0.47036027908325195, + -0.6558746695518494, + 0.3336401581764221, + 0.4827704429626465, + 0.21932020783424377, + -0.16736462712287903, + -0.4875970482826233, + -0.32751625776290894, + -0.07440099120140076, + -0.24660229682922363, + -0.009617837145924568, + -0.18763534724712372, + -0.09053583443164825, + -0.1090857982635498, + -0.2659403085708618, + -0.1171875 + ], + [ + 0.41503775119781494, + -0.677285373210907, + 1.028478980064392, + 0.5291897058486938, + 0.22892804443836212, + 0.417287677526474, + -0.7241002917289734, + 0.2863876521587372, + 0.39558354020118713, + 0.04016329348087311, + -0.1694144904613495, + -0.4067331552505493, + -0.2850966453552246, + 0.17843478918075562, + -0.1777254045009613, + -0.01690167933702469, + -0.08967171609401703, + -0.1685670167207718, + 0.06325215101242065, + -0.23178468644618988, + 0.06850159913301468 + ], + [ + 0.2314331978559494, + -0.5535585880279541, + 1.000349998474121, + 0.38280996680259705, + 0.3449220657348633, + 0.41305145621299744, + -0.3596547245979309, + 0.43198442459106445, + 0.4153168499469757, + 0.052870988845825195, + 0.04645320028066635, + -0.5818946957588196, + -0.3390159606933594, + 0.17957070469856262, + -0.16262038052082062, + -0.019297663122415543, + -0.2034837156534195, + -0.03579661250114441, + 0.17443202435970306, + -0.13197749853134155, + -0.013713151216506958 + ], + [ + 0.10679545253515244, + -0.5216257572174072, + 1.1731065511703491, + 0.2876773178577423, + 0.2409408539533615, + -0.07592439651489258, + 0.08116781711578369, + 0.2347056269645691, + 0.3631366193294525, + 0.02719423919916153, + 0.10486897826194763, + -0.7611414790153503, + -0.33368831872940063, + -0.19680024683475494, + -0.2561682164669037, + -0.08563822507858276, + -0.008296146988868713, + -0.14190365374088287, + 0.03549067676067352, + -0.12387071549892426, + 0.13013368844985962 + ], + [ + 0.4368787407875061, + -0.6942615509033203, + 1.0927714109420776, + 0.4788568913936615, + 0.18266475200653076, + 0.31222712993621826, + -0.46198225021362305, + 0.31760334968566895, + 0.4469907283782959, + 0.07939180731773376, + -0.10216853767633438, + -0.4994339346885681, + -0.2430732399225235, + 0.25602757930755615, + -0.2264256775379181, + -0.017290612682700157, + -0.06057564914226532, + -0.19999904930591583, + 0.06390033662319183, + -0.13188783824443817, + 0.009188741445541382 + ] + ], + "generated_utilities": [ + 0.01805901527404785, + 0.022866547107696533, + 0.019091889262199402, + 0.01995408535003662, + 0.019698604941368103, + 0.019849225878715515, + 0.023350223898887634, + 0.02442134916782379 + ], + "group_id": "ms-3c326d05fec66e1d6bea", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.18246883153915405, + 2.0, + 2.0, + 0.11388173699378967, + 0.035728439688682556, + 0.05159728229045868, + 0.043176501989364624, + 0.04580867290496826, + 0.02321702241897583, + 0.040450215339660645, + 0.04055213928222656, + 0.024657413363456726, + 0.017660364508628845, + 0.044850513339042664, + 0.031234994530677795, + 0.06195336580276489 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.018136948347091675, + 0.10074883699417114, + 0.11574286222457886, + 0.0, + 0.1410461664199829, + 0.38714247941970825, + 0.2381301373243332, + 0.06927800178527832, + -0.6138286590576172, + -0.33484986424446106, + 0.0, + 0.012610852718353271, + 0.5407634973526001, + 0.07784794270992279, + 0.3537199795246124, + -0.1602279543876648, + 0.002093076705932617, + 0.0 + ], + [ + 1.9327397346496582, + -1.7429183721542358, + 1.8877019882202148, + -0.33001917600631714, + 0.49746543169021606, + 0.47390231490135193, + 1.7802670001983643, + -0.8589538335800171, + 0.37976425886154175, + -0.18104670941829681, + -0.8859321475028992, + -1.6138286590576172, + 0.34242650866508484, + 1.2023069858551025, + 0.555707573890686, + -0.35764527320861816, + -0.10027600824832916, + 0.4702257812023163, + -0.6294202208518982, + 0.1926015019416809, + 0.5916687846183777 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.5605340003967285, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.404613971710205, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.1833375692367554 + ], + [ + 1.7389304637908936, + -1.4791877269744873, + 2.158487319946289, + -0.3266654312610626, + 0.5907376408576965, + 0.43260636925697327, + 1.9136874675750732, + -0.5760036110877991, + 0.3071907162666321, + -0.4073694348335266, + -0.8863769769668579, + -1.8163083791732788, + 0.5969879627227783, + 1.196010947227478, + 0.7674136161804199, + -0.2368052750825882, + -0.12123071402311325, + 0.36782756447792053, + -0.8187789916992188, + 0.15793970227241516, + 0.43327897787094116 + ], + [ + 1.9560860395431519, + -2.008063793182373, + 1.6910814046859741, + -0.2576735019683838, + 0.647503137588501, + 0.5322431921958923, + 1.9268598556518555, + -1.2501428127288818, + 0.775363564491272, + -0.0965004414319992, + -0.5331029891967773, + -1.639766812324524, + 0.44450435042381287, + 1.5980249643325806, + 0.9402694702148438, + -0.1454993337392807, + -0.2541998028755188, + 0.6709543466567993, + -0.44338259100914, + -0.043697476387023926, + 0.7131086587905884 + ], + [ + 1.4609386920928955, + -1.6896755695343018, + 2.174467086791992, + -0.34207087755203247, + 0.962078332901001, + 0.28904104232788086, + 2.0044631958007812, + -1.184704303741455, + 0.8424319624900818, + -0.6237084865570068, + -0.9171007871627808, + -1.5616872310638428, + 0.3556978702545166, + 1.5233795642852783, + 0.7210954427719116, + 0.028062552213668823, + 0.21759141981601715, + 0.2823620140552521, + -0.588395357131958, + -0.12581703066825867, + 0.7275858521461487 + ], + [ + 2.087728500366211, + -2.0203187465667725, + 2.4627456665039062, + 0.09322953224182129, + 0.4771752953529358, + 0.2439241260290146, + 2.3431313037872314, + -0.7176190614700317, + 0.8298519849777222, + -0.47738587856292725, + -1.4635096788406372, + -1.2598087787628174, + 0.008061617612838745, + 0.9835944175720215, + 0.10656380653381348, + -0.1548040360212326, + 0.20115382969379425, + 0.4581896960735321, + -1.197885274887085, + -0.28882738947868347, + 0.17229679226875305 + ], + [ + 2.535038471221924, + -2.2647459506988525, + 1.280318260192871, + -0.8243153095245361, + 0.36149507761001587, + 0.01678559184074402, + 1.259200096130371, + -0.5383518934249878, + 1.0661393404006958, + -0.1134992390871048, + -1.0009663105010986, + -1.8629896640777588, + 0.5329263806343079, + 1.1952301263809204, + 0.5380017757415771, + -0.21264730393886566, + 0.33232176303863525, + 0.9111320972442627, + -1.0945085287094116, + 0.7546109557151794, + 0.8692140579223633 + ], + [ + 2.3966352939605713, + -2.3338406085968018, + 2.0661728382110596, + -0.1174439787864685, + -0.28071242570877075, + 0.8010929822921753, + 2.485050678253174, + -0.5360689163208008, + 0.9712552428245544, + 0.5487139821052551, + -0.46108368039131165, + -2.0631186962127686, + 0.6046813726425171, + 0.9991111159324646, + 1.299267292022705, + 0.3742367625236511, + -0.7156072854995728, + 0.7805436849594116, + -1.3091955184936523, + -0.5538330674171448, + -0.20684683322906494 + ], + [ + 2.614837884902954, + -2.24919056892395, + 2.2911505699157715, + -0.2939787209033966, + 1.1876089572906494, + 0.15644106268882751, + 1.7348854541778564, + -0.7635170221328735, + 0.5141214728355408, + -0.17918013036251068, + -0.8421157598495483, + -1.5084272623062134, + -0.18822845816612244, + 1.8389434814453125, + 0.24652135372161865, + 0.38509100675582886, + 0.5761538147926331, + -0.15904095768928528, + 0.0013433098793029785, + 0.6120359897613525, + 1.3977106809616089 + ], + [ + 1.5386989116668701, + -2.0177698135375977, + 1.8581953048706055, + -1.2861346006393433, + 1.0945217609405518, + 0.8363389372825623, + 2.708517074584961, + -0.9201098084449768, + 0.8546208739280701, + -0.058566831052303314, + -1.5932974815368652, + -2.015472412109375, + 0.2565937638282776, + 1.988462209701538, + -0.04372602701187134, + -0.8754178881645203, + 0.27551382780075073, + -0.29693683981895447, + -0.2824576497077942, + 0.40926772356033325, + 0.7658611536026001 + ], + [ + 2.8735361099243164, + -2.675261974334717, + 1.459324836730957, + 0.380817174911499, + 0.6231281757354736, + 0.03267902135848999, + 2.296022415161133, + -1.587423324584961, + 0.7833051681518555, + 0.6618785262107849, + -1.7110093832015991, + -1.436572551727295, + -0.14784568548202515, + 1.525026559829712, + 1.4909214973449707, + -1.0601696968078613, + -0.5454426407814026, + -0.09859517216682434, + -1.4344520568847656, + -0.058256953954696655, + -0.3444262146949768 + ], + [ + 2.0854334831237793, + -1.3952012062072754, + 2.651411771774292, + -0.7663257122039795, + -0.05179548263549805, + 0.8797686100006104, + 2.0904030799865723, + -0.8943268656730652, + 1.0378066301345825, + 0.08841629326343536, + -0.36119532585144043, + -1.236596703529358, + 1.0752983093261719, + 0.349973201751709, + -0.19527214765548706, + -0.8867040872573853, + 0.18460334837436676, + 1.454195261001587, + -0.6220338940620422, + 0.9919598698616028, + 1.229588508605957 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.009577363729476929, + -0.06759434938430786, + 0.05406871438026428, + 0.0, + 0.003663182258605957, + 0.0492824912071228, + -0.038241639733314514, + -0.02580159902572632, + -0.6138286590576172, + -0.056222498416900635, + 0.0, + -0.026217222213745117, + -0.051223963499069214, + -0.031640589237213135, + -0.010870158672332764, + 0.08728909492492676, + -0.020528942346572876, + 0.0 + ], + [ + 0.9327397346496582, + -0.7429183721542358, + 0.8877019882202148, + -0.06735062599182129, + -0.023807168006896973, + 0.1439460813999176, + 0.0, + -0.19209599494934082, + -0.14847975969314575, + -0.22449488937854767, + 0.09710657596588135, + -0.6138286590576172, + -0.1854294240474701, + 0.0, + -0.17072844505310059, + 0.09961000084877014, + -0.017846792936325073, + 0.22415585815906525, + -0.18760734796524048, + -0.060083091259002686, + 0.0 + ] + ], + "predicted_scores": [ + 2.515261650085449, + 2.362213611602783, + 1.9975978136062622, + 1.9524565935134888, + 1.9387346506118774, + 1.9376667737960815, + 1.6050832271575928, + 1.4873660802841187 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "278008524", + "selected_index": 0, + "state_hash": "6333fcf80f76890398820f27d1a99efe459cdb6ff795201d73e3bcedc966664f", + "stored_base_utility": 0.18246883153915405, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.18095605075359344, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18095605075359344 + }, + "base_progress": 0.18095605075359344, + "base_success": false, + "base_utility": 0.18095605075359344, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05759069323539734, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05759069323539734 + }, + { + "contact_quality": null, + "progress": 0.03104570508003235, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03104570508003235 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.036939144134521484, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.036939144134521484 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.0, + 1.0, + 0.05759069323539734, + 0.03104570508003235, + 1.0, + 0.036939144134521484 + ], + "candidate_source_chart_ids": [ + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323", + "ms-8a38529670b19d605663", + "ms-bad55a54ecb4349a5323" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + true, + true, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "generated_tangents": [ + [ + -0.9199861288070679, + 0.30872613191604614, + -0.11077669262886047, + -1.9874893426895142, + -0.8131400942802429, + 0.15791809558868408, + -0.5111090540885925, + -0.03193216025829315, + -0.3522268533706665, + 0.11017335951328278, + 0.3190152049064636, + -0.6767958402633667, + 0.15991482138633728, + 0.2653621435165405, + -0.5116323232650757, + -0.5092017650604248, + 0.15582670271396637, + 0.7595919966697693, + -0.08631685376167297, + -0.1302405446767807, + -0.29183804988861084 + ], + [ + -0.7484748363494873, + 0.24598805606365204, + -0.46471795439720154, + -2.0231847763061523, + -0.7788739800453186, + -0.009635865688323975, + -0.5818192362785339, + -0.005646020174026489, + -0.22873127460479736, + 0.4051384925842285, + 0.4068411886692047, + -0.30067235231399536, + -0.10813647508621216, + 0.42515963315963745, + -0.20543107390403748, + -0.3371945321559906, + 0.04766494035720825, + 1.0037654638290405, + -0.16063323616981506, + 0.03557148575782776, + -0.3789990246295929 + ], + [ + -0.731407642364502, + 0.30742979049682617, + -0.4756476581096649, + -2.045758008956909, + -0.7928981184959412, + -0.10356190800666809, + -0.4837225675582886, + -0.15372467041015625, + -0.24504807591438293, + 0.37941187620162964, + 0.416603147983551, + -0.35127514600753784, + -0.12188982963562012, + 0.5938593745231628, + -0.37876227498054504, + -0.36554548144340515, + 0.030406087636947632, + 0.9633036851882935, + -0.1396521031856537, + 0.10455045104026794, + -0.163976788520813 + ], + [ + -0.7403339743614197, + 0.29316914081573486, + -0.1711283028125763, + -2.079578161239624, + -0.9704251885414124, + 0.010900795459747314, + -0.6362676024436951, + -0.17625626921653748, + -0.16771242022514343, + 0.38578927516937256, + 0.5840837359428406, + -0.24930799007415771, + 0.18123987317085266, + 0.6146359443664551, + -0.6376851201057434, + -0.38402917981147766, + 0.2362058162689209, + 0.9600014686584473, + -0.09930005669593811, + 0.3097380995750427, + -0.1147318184375763 + ], + [ + -0.3282294273376465, + 0.16843962669372559, + -0.09394654631614685, + -1.9015899896621704, + -0.6381380558013916, + -0.18369367718696594, + -0.6958561539649963, + -0.11395466327667236, + -0.14993177354335785, + 0.28022268414497375, + 0.41690579056739807, + -0.2501760721206665, + 0.5187122821807861, + 0.45513468980789185, + -0.38766011595726013, + -0.45108339190483093, + 0.2503165304660797, + 0.7832396030426025, + -0.13590681552886963, + 0.15501433610916138, + -0.39501962065696716 + ], + [ + -0.38397884368896484, + 0.2456217259168625, + -0.005237668752670288, + -1.7133713960647583, + -0.6999824643135071, + -0.3198879361152649, + -0.528630793094635, + -0.24333840608596802, + -0.05804062634706497, + 0.4728117287158966, + 0.4355371594429016, + -0.2204369604587555, + 0.36551615595817566, + 0.25783246755599976, + -0.2590427100658417, + -0.5544210076332092, + 0.37955984473228455, + 0.5370518565177917, + -0.2108737826347351, + 0.16847515106201172, + -0.36978408694267273 + ], + [ + -0.985397458076477, + 0.37230902910232544, + -0.4251243770122528, + -2.2483575344085693, + -1.0317788124084473, + 0.0034407377243041992, + 0.1626998484134674, + -0.11172974109649658, + -0.2908857762813568, + 0.20559722185134888, + -0.13058370351791382, + -0.1935850977897644, + 0.33945032954216003, + 0.9497448205947876, + -0.35347533226013184, + -0.30820897221565247, + 0.14114263653755188, + 1.154399037361145, + -0.3007515072822571, + 0.3745461702346802, + -0.5210039615631104 + ], + [ + -0.3994852304458618, + 0.22625382244586945, + -0.08539852499961853, + -1.7654892206192017, + -0.5889594554901123, + -0.18787041306495667, + -0.8400595784187317, + -0.18711340427398682, + -0.042555905878543854, + 0.4458334445953369, + 0.4649679362773895, + -0.03487749397754669, + 0.28858545422554016, + 0.21199294924736023, + -0.21182990074157715, + -0.5362876057624817, + 0.4092741906642914, + 0.5948329567909241, + -0.22820013761520386, + 0.2394702434539795, + -0.30517613887786865 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 2.0, + 2.0, + 0.05759069323539734, + 0.03104570508003235, + 2.0, + 0.036939144134521484 + ], + "group_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.17512565851211548, + 2.0, + 2.0, + 2.0, + 0.10078734159469604, + 2.0, + 2.0, + 0.06518048048019409, + 2.0, + 2.0, + 0.05268773436546326, + 0.056524619460105896, + 0.07100173830986023, + 0.16304254531860352, + 2.0, + 0.07095202803611755 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -1.7685134410858154, + 0.4561518430709839, + 0.8554723262786865, + -2.9925076961517334, + -1.652122139930725, + -0.9696183204650879, + 1.440989375114441, + -0.1663060486316681, + 0.005464707035571337, + -0.08078178763389587, + 0.5969918370246887, + -1.7046172618865967, + 0.3472420275211334, + 0.7052499651908875, + -0.026682782918214798, + 0.013620201498270035, + 0.002864290028810501, + 1.5629743337631226, + 0.06437517702579498, + -0.20232005417346954, + 0.06808048486709595 + ], + [ + -1.571894645690918, + 0.49223509430885315, + 1.2358425855636597, + -2.7074813842773438, + -1.6053731441497803, + -0.9737302660942078, + 1.2407286167144775, + -0.41521748900413513, + 0.27269765734672546, + -0.08917579054832458, + 0.4081580936908722, + -1.3627166748046875, + 0.07880738377571106, + 0.7477485537528992, + -0.2180848866701126, + -0.13109977543354034, + -0.3875364661216736, + 1.8194886445999146, + -0.06430798768997192, + 0.07869948446750641, + -0.12621839344501495 + ], + [ + -1.7744590044021606, + 0.367733359336853, + 1.3591725826263428, + -3.3725903034210205, + -1.8434977531433105, + -0.8745664954185486, + 1.6967558860778809, + -0.44676536321640015, + -0.6177129149436951, + 0.03244173526763916, + 0.4171296954154968, + -1.4895254373550415, + 0.9296896457672119, + 0.15846145153045654, + -0.006281482055783272, + 0.6454329490661621, + 0.6225314140319824, + 1.6578954458236694, + 0.3061527609825134, + -0.41300463676452637, + -0.2527916431427002 + ], + [ + -1.7233139276504517, + 0.020191192626953125, + 1.0188184976577759, + -2.873849868774414, + -2.027033567428589, + -1.1650824546813965, + 2.0792155265808105, + -0.8799198865890503, + 0.6924232840538025, + -0.7650182247161865, + -0.04520058631896973, + -2.468881130218506, + -0.13257521390914917, + 0.6857031583786011, + -0.10177457332611084, + -0.7609032988548279, + -0.08249430358409882, + 1.7503252029418945, + 0.1621951013803482, + -0.22467197477817535, + 0.43942898511886597 + ], + [ + -2.3507251739501953, + 0.16114166378974915, + 1.5912489891052246, + -2.1838369369506836, + -2.061230182647705, + -1.7918524742126465, + 0.8407515287399292, + -0.3446531295776367, + -0.7727420926094055, + -0.2617931067943573, + 0.6622774004936218, + -1.05295991897583, + 0.13840271532535553, + 1.2008695602416992, + -0.509428858757019, + 0.7052024006843567, + -0.6614989042282104, + 2.0029079914093018, + -0.82997065782547, + -0.9390356540679932, + 0.7515151500701904 + ], + [ + -1.7504347562789917, + -0.13171464204788208, + 0.7620843648910522, + -3.4511537551879883, + -2.5023341178894043, + -0.8595067858695984, + 1.6361833810806274, + -0.4263937175273895, + 0.005345506593585014, + -0.8147202730178833, + 0.4142894744873047, + -0.8735606670379639, + -0.4809738099575043, + 0.9412479996681213, + -0.6222366690635681, + -0.8843234777450562, + -0.8842994570732117, + 2.4228873252868652, + 0.6301752328872681, + 0.727215051651001, + 0.5050253868103027 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.7685134410858154, + -0.04630154371261597, + -0.002802610397338867, + -1.9925076961517334, + -0.6521221399307251, + -0.013848721981048584, + 0.0, + 0.014648675918579102, + -0.0837627500295639, + -0.04761720821261406, + -0.024705946445465088, + -0.7046172618865967, + 0.0914837121963501, + 0.0, + 0.04683725908398628, + 0.048148274421691895, + 0.03868643194437027, + 0.5629743337631226, + -0.043637245893478394, + -0.08927876502275467, + 0.0 + ], + [ + -0.7685134410858154, + 0.1268729865550995, + -0.0878140926361084, + -1.9925076961517334, + -0.6521221399307251, + -0.08497345447540283, + 0.0, + -0.23278282582759857, + 0.1505891978740692, + -0.0029508620500564575, + -0.03223156929016113, + -0.7046172618865967, + -0.18528565764427185, + 0.0, + 0.009110454469919205, + 0.006383728235960007, + -0.012130538001656532, + 0.5629743337631226, + 0.23608635365962982, + -0.06009632349014282, + 0.0 + ], + [ + -0.7685134410858154, + 0.6076252460479736, + 0.2745842933654785, + -1.9925076961517334, + -0.6521221399307251, + 0.03038167953491211, + 0.0, + 0.33581337332725525, + 0.35011863708496094, + -0.06728862971067429, + -0.4030081629753113, + -0.7046172618865967, + -0.44776007533073425, + 0.0, + -0.01595483534038067, + -0.040477462112903595, + 0.28290560841560364, + 0.5629743337631226, + -0.42285358905792236, + -0.11823103576898575, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.881978750228882, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.410499930381775, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.1361609697341919 + ], + [ + -1.5711963176727295, + 0.5461994409561157, + 0.8384940028190613, + -2.960416078567505, + -1.9055238962173462, + -0.7765143513679504, + 1.4308912754058838, + 0.10734349489212036, + 0.011199884116649628, + -0.028424743562936783, + 0.5893871784210205, + -1.7020739316940308, + 0.48640453815460205, + 0.9950923323631287, + 0.20675025880336761, + 0.11047534644603729, + 0.09110014140605927, + 1.3974875211715698, + -0.21079741418361664, + -0.3005509674549103, + 0.09192514419555664 + ], + [ + -2.12906813621521, + 0.3435881733894348, + 0.7164700627326965, + -2.750021457672119, + -1.3681434392929077, + -0.8937081694602966, + 1.6475064754486084, + -0.1447279453277588, + -0.44103574752807617, + -0.5038806200027466, + 0.9006609320640564, + -1.8256494998931885, + 0.433420330286026, + 0.3506351411342621, + 0.4619242548942566, + -0.3905109167098999, + -0.01803053542971611, + 1.9505069255828857, + -0.339522123336792, + -0.20715753734111786, + -0.410531610250473 + ], + [ + -1.194382667541504, + 0.9652969837188721, + 0.41722428798675537, + -2.6203644275665283, + -1.139541506767273, + -1.5412108898162842, + 1.667648434638977, + -0.5256404876708984, + 0.17716272175312042, + 0.30056339502334595, + 0.09983009099960327, + -1.8219844102859497, + -0.23339274525642395, + 0.5763673782348633, + -0.4285883605480194, + 0.1476086974143982, + -0.3515554666519165, + 1.2458031177520752, + 0.3135184645652771, + -0.33711761236190796, + -0.09264840185642242 + ], + [ + -2.0614774227142334, + -0.4080510139465332, + 0.5476135015487671, + -3.0662269592285156, + -1.497420310974121, + -0.11606889963150024, + 1.03195321559906, + -0.3406473994255066, + 0.12844708561897278, + -0.4317864775657654, + 1.035879135131836, + -0.7729326486587524, + 1.0862895250320435, + 0.16315871477127075, + -0.14380013942718506, + -0.8498450517654419, + 0.9780882000923157, + 1.0358695983886719, + -0.5399289727210999, + -0.10386796295642853, + 0.8144621849060059 + ] + ], + "predicted_scores": [ + 6.941451549530029, + 6.583841800689697, + 6.470824241638184, + 5.723037242889404, + 5.598393440246582, + 5.495008945465088, + 5.4782795906066895, + 5.432579517364502 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "794181626", + "selected_index": 0, + "state_hash": "4043b3c43ca4f115e1ea4fbcd98c7d08f797df2efbfe6b67bcffa5592a8aa36d", + "stored_base_utility": 0.17512565851211548, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.2934207916259766e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.2934207916259766e-05 + }, + { + "contact_quality": null, + "progress": 0.05296194553375244, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05296194553375244 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.000760689377784729, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.000760689377784729 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.1980533599853516e-05, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 1.1980533599853516e-05 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 1.2934207916259766e-05, + 0.05296194553375244, + 1.0, + 0.000760689377784729, + 1.0, + 1.1980533599853516e-05 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-ec5ac79449116b0a30c1", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304", + "ms-630c7d4c0ef39da64304" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + false, + true, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-58950fed04d203b97ab9", + "generated_tangents": [ + [ + -0.31529414653778076, + 0.06719810515642166, + 1.3790019750595093, + -0.8087677955627441, + -1.5558207035064697, + 0.1901077926158905, + 0.1225493848323822, + -0.009450942277908325, + 0.020079508423805237, + 0.13775329291820526, + -0.03765364736318588, + -1.1909911632537842, + 0.10977646708488464, + 0.057752951979637146, + -0.037106942385435104, + 0.03026609867811203, + -0.009784594178199768, + -0.03841622173786163, + -0.46646183729171753, + 0.05364760756492615, + 0.2150888592004776 + ], + [ + -0.3955671191215515, + 0.08322340250015259, + 1.355778694152832, + -0.7844339609146118, + -1.5168399810791016, + -0.008477360010147095, + 0.18259158730506897, + 0.026051878929138184, + -0.010561231523752213, + 0.13007283210754395, + 0.0012517385184764862, + -1.1184790134429932, + -0.040822625160217285, + 0.0874737948179245, + 0.10217350721359253, + 0.027105841785669327, + 0.002047210931777954, + 0.07655707001686096, + -0.5974532961845398, + 0.05660775303840637, + 0.13720059394836426 + ], + [ + -0.27848654985427856, + 0.0682956725358963, + 1.5169649124145508, + -0.7754560708999634, + -1.5582163333892822, + 0.1750037521123886, + 0.14128366112709045, + 0.005196854472160339, + 0.04952076077461243, + 0.09669095277786255, + -0.02788574993610382, + -1.0915590524673462, + 0.0059854090213775635, + 0.13754324615001678, + 0.005132380872964859, + 0.04684419929981232, + -0.0276670902967453, + 0.00729253888130188, + -0.5461607575416565, + -0.038370281457901, + 0.07043653726577759 + ], + [ + -0.30860960483551025, + 0.1983822137117386, + 1.2585598230361938, + -0.8459683656692505, + -1.4227685928344727, + 0.4656536281108856, + -0.03908658027648926, + -0.18026328086853027, + -0.006392387673258781, + 0.050521716475486755, + -0.06917355209589005, + -1.3046241998672485, + -0.22434425354003906, + 0.23492836952209473, + -0.07044491171836853, + 0.015672385692596436, + -0.07792384922504425, + -0.16247378289699554, + -0.366665780544281, + -0.08924534916877747, + 0.047389835119247437 + ], + [ + -0.14354734122753143, + 0.12389950454235077, + 1.5886749029159546, + -0.7557408809661865, + -1.6064472198486328, + -0.07062575221061707, + -0.007238209247589111, + 0.051516443490982056, + 0.04593954235315323, + 0.0594705194234848, + 0.007216770201921463, + -1.180119514465332, + 0.002326071262359619, + 0.1234818547964096, + -0.020440172404050827, + -0.0020101983100175858, + -0.08072556555271149, + -0.026401802897453308, + -0.5160574913024902, + 0.08116337656974792, + 0.16265127062797546 + ], + [ + -0.1600053459405899, + -0.0019751600921154022, + 1.4490902423858643, + -0.806849479675293, + -1.5834367275238037, + -0.3417109549045563, + -0.04128307104110718, + 0.0046322643756866455, + 0.0322856679558754, + 0.057643599808216095, + 0.006958127021789551, + -1.2496333122253418, + 0.08490565419197083, + 0.046819135546684265, + -0.07280509173870087, + -0.018529821187257767, + -0.21667523682117462, + -0.07447965443134308, + -0.37272197008132935, + 0.08866557478904724, + 0.0742463618516922 + ], + [ + -0.2705276608467102, + 0.015920650213956833, + 1.833092451095581, + -0.874640703201294, + -1.805408239364624, + 0.04700533300638199, + -0.3806138038635254, + 0.11919692158699036, + -0.05380581319332123, + 0.1927267611026764, + -0.006913498044013977, + -1.1574627161026, + 0.4797649085521698, + 0.13377641141414642, + -0.09122300148010254, + 0.056952085345983505, + -0.06653980910778046, + 0.019200608134269714, + -0.36915457248687744, + -0.08732870221138, + -0.23738515377044678 + ], + [ + -0.26357823610305786, + 0.05911149084568024, + 1.5157943964004517, + -0.9628791809082031, + -1.8328700065612793, + 0.17975999414920807, + -0.5800464749336243, + -0.027612760663032532, + 0.04569070041179657, + 0.1539253294467926, + 0.03349337726831436, + -1.1308995485305786, + 0.23134228587150574, + 0.18589317798614502, + -0.031458284705877304, + 0.06552442908287048, + -0.27106863260269165, + 0.2178078144788742, + -0.4619292616844177, + 0.2917059659957886, + 0.10871528834104538 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 1.2934207916259766e-05, + 0.05296194553375244, + 2.0, + 0.000760689377784729, + 2.0, + 1.1980533599853516e-05 + ], + "group_id": "ms-58950fed04d203b97ab9", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.20183078944683075, + 0.18959736824035645, + 2.0, + 0.0385412722826004, + 0.15877258777618408, + 0.03365510702133179, + 0.17151890695095062, + 0.13845136761665344, + 0.11539244651794434, + 0.1388089954853058, + 0.15365298092365265, + 0.0967668741941452, + 0.0908290445804596, + 0.06094485521316528, + 0.0377749502658844, + 0.028643101453781128 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.028361663222312927, + -0.489355206489563, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.16713106632232666, + 0.0, + 0.5545767545700073, + -0.18274790048599243, + 0.2772042155265808, + -0.45601847767829895, + -1.3811633586883545, + -0.1571006327867508, + 0.0, + -0.004826217889785767, + 0.2916078567504883, + 0.34807145595550537, + -0.37882840633392334, + 0.08759653568267822, + -0.07658697664737701, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.797813892364502, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.654926061630249, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.4572639465332031 + ], + [ + 0.17793144285678864, + 0.008656054735183716, + 2.343496322631836, + -1.9455183744430542, + -2.4697093963623047, + 1.0036901235580444, + 2.5124316215515137, + -0.29308056831359863, + -0.02035735547542572, + -0.059014320373535156, + -0.5863867402076721, + -2.254472255706787, + 0.3354390263557434, + 1.381989598274231, + -0.27729421854019165, + 0.5023907423019409, + -0.8654648661613464, + 0.39588332176208496, + -0.5682056546211243, + 0.20400357246398926, + 0.48529666662216187 + ], + [ + -0.5227808952331543, + 0.12448178231716156, + 2.4855635166168213, + -1.6602094173431396, + -2.3857760429382324, + 0.323077529668808, + 1.9053025245666504, + -0.698577344417572, + 0.619890034198761, + 0.3500756621360779, + -0.1476593315601349, + -2.8024404048919678, + -0.21084021031856537, + 1.0928113460540771, + -0.3938775062561035, + 0.4228009581565857, + -0.39060449600219727, + -0.31665533781051636, + -0.9205541610717773, + 0.03774743899703026, + 0.3333621621131897 + ], + [ + 0.10275222361087799, + 0.08925776183605194, + 2.0934102535247803, + -1.379586935043335, + -3.3245811462402344, + 0.6494860649108887, + 1.990661382675171, + -0.021198302507400513, + 0.4400750398635864, + 0.39176517724990845, + -0.8639137148857117, + -1.981496810913086, + 0.01414397917687893, + 1.2019504308700562, + -0.7839368581771851, + 0.108054518699646, + -0.17177742719650269, + -0.36239588260650635, + -0.5078206062316895, + 0.4236416518688202, + 0.25084057450294495 + ], + [ + -0.8110343217849731, + -0.11216431856155396, + 2.8897132873535156, + -1.319732427597046, + -3.1141648292541504, + 0.416613906621933, + 2.3244988918304443, + -0.6579729914665222, + -0.39938226342201233, + -0.08646327257156372, + -0.3429824113845825, + -2.4165658950805664, + -0.6488300561904907, + 1.4793217182159424, + -0.8628695011138916, + 0.6845848560333252, + -0.7794754505157471, + -0.014068186283111572, + -0.27444708347320557, + 0.8030790686607361, + 0.08975183963775635 + ], + [ + -0.23147638142108917, + -0.26693814992904663, + 2.8227345943450928, + -2.3573813438415527, + -3.3905396461486816, + 0.38961973786354065, + 1.9831907749176025, + -1.2505497932434082, + 0.00624750554561615, + 0.9689428210258484, + 0.11981219053268433, + -1.5865073204040527, + 0.4901374876499176, + 0.6158997416496277, + 0.3767610788345337, + -0.025944411754608154, + -1.4681568145751953, + 0.48217374086380005, + -1.5973502397537231, + -0.7199738621711731, + 0.5119494795799255 + ], + [ + -0.19429966807365417, + 0.7059361934661865, + 3.2694599628448486, + -2.656050682067871, + -2.5502617359161377, + -0.23801833391189575, + 1.628230333328247, + 0.36624830961227417, + -0.08827590942382812, + -0.2098810374736786, + -1.1745270490646362, + -2.2196781635284424, + 0.44475334882736206, + 0.48411858081817627, + -0.25864526629447937, + 0.2009485512971878, + -0.9008907079696655, + -0.3571908473968506, + -0.16894888877868652, + -0.6256314516067505, + -0.4078131914138794 + ], + [ + 0.22205571830272675, + 0.6026778221130371, + 1.8579424619674683, + -2.404672384262085, + -3.728808641433716, + -0.19806396961212158, + 2.8292319774627686, + -0.2435564398765564, + 0.7903643846511841, + -0.376501202583313, + -1.0294175148010254, + -1.7672805786132812, + 0.5992995500564575, + 2.044847249984741, + 0.0011147260665893555, + 0.18739356100559235, + -0.952850341796875, + 0.8702673316001892, + 0.038269996643066406, + -0.4372258484363556, + -0.37835830450057983 + ], + [ + -0.3783646821975708, + 0.08899453282356262, + 2.721517324447632, + -2.3160932064056396, + -2.0679585933685303, + 0.549463152885437, + 3.141833543777466, + 0.09782099723815918, + 0.5830706357955933, + 0.7134941816329956, + 0.2986952066421509, + -2.9980335235595703, + -0.6391466856002808, + 1.075132131576538, + -0.7696593403816223, + 0.4343666732311249, + -0.911625862121582, + -0.31454378366470337, + -1.5935450792312622, + -0.8993679881095886, + -0.3811922073364258 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.1209426149725914, + 0.10675475001335144, + 1.4316291809082031, + -0.9109628200531006, + -1.7493720054626465, + -0.0875239372253418, + 0.0, + -0.0486987829208374, + 0.06580346822738647, + 0.24258655309677124, + -0.20229026675224304, + -1.3811633586883545, + 0.4185009002685547, + 0.0, + -0.1431063860654831, + 0.15154658257961273, + 0.17190229892730713, + 0.10405468940734863, + 0.036865830421447754, + -0.1988307237625122, + 0.0 + ] + ], + "predicted_scores": [ + 4.611660480499268, + 4.5243706703186035, + 4.38554048538208, + 4.170772552490234, + 3.9188783168792725, + 3.6644399166107178, + 3.558708429336548, + 3.06156849861145 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1113371261", + "selected_index": 0, + "state_hash": "1b9018834a77531260bf8dfa74a9832916fe215bda78b5367b061bb4e0ce2333", + "stored_base_utility": 0.20183078944683075, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.01641605794429779, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.01641605794429779 + }, + "base_progress": 0.01641605794429779, + "base_success": false, + "base_utility": 0.01641605794429779, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.039980679750442505, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.039980679750442505 + }, + { + "contact_quality": null, + "progress": 0.060836225748062134, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.060836225748062134 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06095251441001892, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06095251441001892 + }, + { + "contact_quality": null, + "progress": 0.031782910227775574, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.031782910227775574 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.0629233866930008, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0629233866930008 + } + ], + "candidate_progress": [ + 1.0, + 0.039980679750442505, + 0.060836225748062134, + 1.0, + 0.06095251441001892, + 0.031782910227775574, + 1.0, + 0.0629233866930008 + ], + "candidate_source_chart_ids": [ + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-9f61e09c05b4c8f74366", + "ms-9f61e09c05b4c8f74366", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1", + "ms-74704688fb63757619e1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-651bc0395d78d2819158", + "generated_tangents": [ + [ + -0.4790000915527344, + -0.2623399794101715, + 1.2292762994766235, + -0.9752191305160522, + -1.6180856227874756, + -0.2288983166217804, + -0.028735220432281494, + -0.3114524185657501, + 0.01069274265319109, + -0.275970995426178, + 0.022787142544984818, + -1.5272339582443237, + -0.40187370777130127, + 0.5987383723258972, + -0.22365739941596985, + -0.07965487241744995, + 0.23476973176002502, + 0.8301016688346863, + -0.360110878944397, + -0.5283201932907104, + 0.005559906363487244 + ], + [ + -0.3233262896537781, + -0.3249558210372925, + 1.278425693511963, + -1.119921088218689, + -1.674001932144165, + -0.6046465635299683, + 0.23673884570598602, + -0.30465856194496155, + -0.02057245932519436, + -0.2387981116771698, + 0.08232123404741287, + -1.4324522018432617, + -0.35365891456604004, + 0.6753489971160889, + -0.2134622037410736, + -0.07452715933322906, + 0.26820996403694153, + 0.7786791324615479, + -0.12941446900367737, + -0.523983359336853, + 0.2599738836288452 + ], + [ + -0.3779054880142212, + -0.238853320479393, + 1.2486516237258911, + -0.7021474838256836, + -1.4407782554626465, + -0.36793550848960876, + 0.3716639280319214, + -0.054011762142181396, + 0.09405867010354996, + -0.1169712021946907, + -0.16203543543815613, + -1.2887053489685059, + -0.4147561490535736, + 0.5722816586494446, + -0.2696605622768402, + 0.006031792610883713, + 0.17731794714927673, + 0.8135117888450623, + -0.21739381551742554, + -0.4056673049926758, + 0.23164701461791992 + ], + [ + -0.3505050539970398, + -0.2582804560661316, + 1.0538042783737183, + -0.6612803936004639, + -1.555436372756958, + -0.2776733338832855, + 0.14931350946426392, + -0.07758373022079468, + -0.01656237430870533, + -0.013535372912883759, + -0.21431154012680054, + -1.051558017730713, + -0.41920626163482666, + 0.4313759207725525, + -0.34845492243766785, + -0.01781424880027771, + 0.30896419286727905, + 0.7666733860969543, + -0.20654213428497314, + -0.48529598116874695, + 0.42542150616645813 + ], + [ + -0.25688695907592773, + -0.2868007719516754, + 1.15911865234375, + -0.7883217334747314, + -1.3453056812286377, + -0.4138046205043793, + 0.5186532735824585, + -0.19082722067832947, + 0.07793162763118744, + -0.15708866715431213, + -0.1384713053703308, + -1.2223492860794067, + -0.1700986921787262, + 0.46245038509368896, + -0.35558512806892395, + -0.06017529219388962, + 0.3921342194080353, + 0.7723162770271301, + -0.3470229506492615, + -0.48607730865478516, + -0.03092317283153534 + ], + [ + -0.383963406085968, + -0.2568821907043457, + 1.3227291107177734, + -1.033108115196228, + -1.7499196529388428, + -0.3432324230670929, + -0.03116506338119507, + -0.31188544631004333, + -0.0018440410494804382, + -0.2096678614616394, + 0.09987686574459076, + -1.330317735671997, + -0.504436731338501, + 0.7273666858673096, + -0.2382347583770752, + -0.09948363900184631, + 0.25238290429115295, + 0.7306241393089294, + -0.2710176706314087, + -0.43206655979156494, + 0.07764556258916855 + ], + [ + -0.2638291120529175, + -0.15891681611537933, + 1.5607041120529175, + -1.1172218322753906, + -1.5984916687011719, + -0.021221444010734558, + 0.09867265820503235, + -0.2503146529197693, + 0.03268471360206604, + -0.45578712224960327, + -0.021439678966999054, + -1.5354739427566528, + -0.1466650664806366, + 0.7854465842247009, + -0.25617313385009766, + 0.002685592509806156, + 0.40647679567337036, + 0.9339250922203064, + -0.1025136411190033, + -0.5770187377929688, + 0.46320152282714844 + ], + [ + -0.31497132778167725, + -0.4016556143760681, + 1.2052533626556396, + -0.5810573101043701, + -1.340848684310913, + -0.36131349205970764, + 0.23329539597034454, + -0.08051857352256775, + 0.01538036111742258, + -0.16655072569847107, + -0.028443746268749237, + -1.2804170846939087, + -0.26589950919151306, + 0.033651262521743774, + -0.4386689066886902, + -0.035484254360198975, + 0.4670451879501343, + 0.5834474563598633, + -0.16527092456817627, + -0.1602071225643158, + 0.022881165146827698 + ] + ], + "generated_utilities": [ + 2.0, + 0.039980679750442505, + 0.060836225748062134, + 2.0, + 0.06095251441001892, + 0.031782910227775574, + 2.0, + 0.0629233866930008 + ], + "group_id": "ms-651bc0395d78d2819158", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 7.152557373046875e-07, + 2.0, + 0.0, + 0.0, + 0.18490922451019287, + 0.0, + 0.18801423907279968, + 2.0, + 0.18292954564094543, + 0.06743006408214569, + 2.0, + 0.08718416094779968, + 0.0016145259141921997, + 2.0, + 0.0, + 0.0 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.13822472095489502, + 0.052123308181762695, + 1.2542681694030762, + -0.5262794494628906, + -1.515254020690918, + -0.022995710372924805, + 0.0, + 0.07829460501670837, + 0.056280869990587234, + -0.006563127040863037, + 0.00594717264175415, + -1.0464823246002197, + 0.028334707021713257, + 0.0, + 0.04214516654610634, + -0.0032968670129776, + 0.025241270661354065, + 1.0898823738098145, + -0.09987176954746246, + 0.009208589792251587, + 0.0 + ], + [ + -1.138224720954895, + -0.12241673469543457, + 2.254268169403076, + -1.5262794494628906, + -2.515254020690918, + -0.4036164879798889, + 2.5164785385131836, + -0.48240265250205994, + 0.09632497280836105, + -1.0002868175506592, + 0.6668505668640137, + -2.0464823246002197, + 0.5158075094223022, + 0.48990970849990845, + 0.02257649227976799, + -0.23748205602169037, + -0.2213379293680191, + 2.0898823738098145, + 0.1738651841878891, + -0.37470543384552, + -0.06145218014717102 + ], + [ + -1.0695267915725708, + -0.1775755137205124, + 2.1683619022369385, + -1.4577077627182007, + -2.7917678356170654, + -0.632304310798645, + 2.3911960124969482, + -0.4637458324432373, + 0.35316434502601624, + -0.8419650793075562, + 0.8253071308135986, + -1.9648017883300781, + 0.2602742314338684, + 0.4741981625556946, + 0.17542533576488495, + -0.15374207496643066, + -0.4857789874076843, + 2.119079828262329, + 0.44966256618499756, + -0.3760189712047577, + 0.18533895909786224 + ], + [ + -0.7955029010772705, + 0.23637625575065613, + 1.9978463649749756, + -1.8726613521575928, + -2.2562904357910156, + -0.3048732876777649, + 2.261690378189087, + -0.6620026230812073, + 0.06043636053800583, + -1.091504693031311, + 1.038455605506897, + -1.9805728197097778, + 0.7370907068252563, + 0.5352527499198914, + 0.26298987865448, + -0.2739749252796173, + 0.14720074832439423, + 2.0173659324645996, + -0.16336800158023834, + -0.5858949422836304, + -0.1647021770477295 + ], + [ + -1.6335437297821045, + -0.08274295926094055, + 1.833235502243042, + -1.758880853652954, + -2.0250279903411865, + -0.8149738311767578, + 2.4238271713256836, + -0.33182430267333984, + -0.16191768646240234, + -1.4929053783416748, + 0.8467401266098022, + -2.4791038036346436, + 0.2757382392883301, + 0.16471439599990845, + 0.18122689425945282, + -0.2995648682117462, + -0.34926944971084595, + 2.063732147216797, + 0.3726923167705536, + -0.259366512298584, + 0.36020517349243164 + ], + [ + -0.9912663698196411, + -0.19695568084716797, + 2.342681646347046, + -2.1077044010162354, + -3.031508684158325, + -0.6591198444366455, + 2.5516788959503174, + -0.8631293773651123, + 0.2481168806552887, + -1.107275366783142, + 0.5796921253204346, + -2.479954719543457, + 1.0449233055114746, + 0.40612131357192993, + -0.12985309958457947, + -0.18829946219921112, + -0.6196849346160889, + 1.6295442581176758, + 0.4197757840156555, + 0.02911210060119629, + 0.5378711223602295 + ], + [ + -0.6689683198928833, + -0.5626275539398193, + 1.738625407218933, + -1.919468879699707, + -2.6463799476623535, + -0.32590651512145996, + 2.287154197692871, + -0.9975759983062744, + 0.7224722504615784, + -1.427150845527649, + 0.572014331817627, + -2.2473490238189697, + 0.8538937568664551, + 0.0030456185340881348, + -0.1059482991695404, + -0.003734409809112549, + -0.2650519907474518, + 2.554786205291748, + -0.24461163580417633, + -0.6881535649299622, + 0.5301691293716431 + ], + [ + -1.5373890399932861, + 0.42636638879776, + 2.9180266857147217, + -2.126905679702759, + -1.825490117073059, + -0.6280819773674011, + 2.593716621398926, + -0.6465503573417664, + 0.5798410177230835, + -0.5376753807067871, + 0.5435326099395752, + -2.610151767730713, + 1.0700397491455078, + 0.7953996658325195, + 0.5072687864303589, + -0.02665230631828308, + -0.6953800320625305, + 2.0521392822265625, + 0.751386821269989, + -0.7565212249755859, + -0.5263289213180542 + ], + [ + -0.7768699526786804, + -0.18900753557682037, + 1.7192409038543701, + -2.2835047245025635, + -1.713951826095581, + 0.004747450351715088, + 3.2709672451019287, + -1.1866822242736816, + 0.38759592175483704, + -0.7857802510261536, + 1.539168119430542, + -1.3250088691711426, + 0.8453556299209595, + 0.5691420435905457, + -0.701140820980072, + -0.6115058660507202, + -0.8829987645149231, + 1.5742011070251465, + -0.3939879536628723, + 0.45414644479751587, + -0.23744438588619232 + ] + ], + "predicted_scores": [ + 4.050506591796875, + 4.038098335266113, + 4.001011371612549, + 3.959012031555176, + 3.8739075660705566, + 3.815523624420166, + 3.801790237426758, + 3.5321407318115234 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1483645336", + "selected_index": 0, + "state_hash": "857c28f61d13d7ee5a59956a87579f6c17f6c601c1680e574691e095e373d439", + "stored_base_utility": 7.152557373046875e-07, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.15461373329162598, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15461373329162598 + }, + "base_progress": 0.15461373329162598, + "base_success": false, + "base_utility": 0.15461373329162598, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.041229456663131714, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.041229456663131714 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03825731575489044, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03825731575489044 + }, + { + "contact_quality": null, + "progress": 0.03897048532962799, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03897048532962799 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.03536248207092285, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03536248207092285 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.041229456663131714, + 1.0, + 0.03825731575489044, + 0.03897048532962799, + 1.0, + 0.03536248207092285, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1", + "ms-1c4df81cce643ca4fb3e", + "ms-ec5ac79449116b0a30c1", + "ms-ec5ac79449116b0a30c1" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + true, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9fad82fd288e4296e354", + "generated_tangents": [ + [ + 0.12536628544330597, + -0.11647109687328339, + 1.2921411991119385, + -0.24000583589076996, + -1.2581112384796143, + 0.11144833266735077, + -0.32800525426864624, + 0.09118542820215225, + 0.097465381026268, + 0.37290751934051514, + -0.04069405049085617, + -1.1826680898666382, + -0.060546427965164185, + -0.01645784080028534, + -0.03844963386654854, + 0.16085399687290192, + 0.09477581083774567, + -0.31814146041870117, + -0.12133142352104187, + -0.14529256522655487, + 0.15311136841773987 + ], + [ + 0.018894150853157043, + -0.14952826499938965, + 1.329519271850586, + -0.2941533327102661, + -1.251598834991455, + 0.30027931928634644, + -0.3334652781486511, + 0.05567649006843567, + 0.08584731817245483, + 0.40141862630844116, + 0.0010401755571365356, + -1.2042847871780396, + -0.04038769006729126, + -0.016273587942123413, + 0.06102803722023964, + 0.17994867265224457, + 0.209195077419281, + -0.24125592410564423, + -0.2691193222999573, + -0.018109411001205444, + 0.12268800288438797 + ], + [ + 0.1202477291226387, + -0.15849904716014862, + 1.2376221418380737, + -0.19699864089488983, + -1.2386269569396973, + 0.18403024971485138, + -0.4259967803955078, + -0.021072864532470703, + 0.14224401116371155, + 0.3849940001964569, + 0.0711132138967514, + -1.222604513168335, + 0.028215348720550537, + 0.010118171572685242, + 0.17152467370033264, + 0.1558779925107956, + 0.13798077404499054, + -0.2491908222436905, + -0.3019210696220398, + 0.004700005054473877, + 0.0299319326877594 + ], + [ + 0.15748688578605652, + -0.07043987512588501, + 1.3349759578704834, + -0.20302537083625793, + -1.1472175121307373, + -0.14832288026809692, + -0.21017318964004517, + 0.16716204583644867, + 0.06764215975999832, + 0.316910058259964, + -0.018689855933189392, + -1.054552435874939, + -0.29433655738830566, + 0.025387123227119446, + 0.06591791659593582, + 0.10386522114276886, + 0.22614863514900208, + -0.19228409230709076, + -0.1799493432044983, + -0.1313530057668686, + 0.19895869493484497 + ], + [ + 0.06650558114051819, + -0.11055754870176315, + 1.0697321891784668, + -0.30425894260406494, + -1.0227339267730713, + 0.4965338110923767, + -0.7454113364219666, + -0.04227656126022339, + 0.1323845535516739, + 0.49749746918678284, + -0.08936873823404312, + -0.9663962125778198, + -0.22731533646583557, + 0.016360580921173096, + 0.0704846978187561, + 0.20341277122497559, + 0.16095131635665894, + -0.3076995015144348, + -0.2250744104385376, + -0.3158529996871948, + -0.22438421845436096 + ], + [ + 0.32339608669281006, + -0.2009962499141693, + 1.398557424545288, + -0.15897992253303528, + -1.1609511375427246, + 0.15981946885585785, + -0.26014435291290283, + 0.22751465439796448, + 0.08869170397520065, + 0.40087276697158813, + 0.3614153265953064, + -1.004456877708435, + -0.40240389108657837, + 0.13136352598667145, + 0.01480763778090477, + 0.2172158658504486, + 0.10615666210651398, + -0.38317227363586426, + -0.41943812370300293, + 0.10234701633453369, + -0.04620635509490967 + ], + [ + 0.04029425233602524, + 0.05161657929420471, + 0.9304510354995728, + -0.31075945496559143, + -1.1189072132110596, + 0.522032618522644, + -0.3789590001106262, + -0.04234050214290619, + -0.011937307193875313, + 0.3761950731277466, + -0.12726551294326782, + -1.2070890665054321, + -0.24522726237773895, + 0.29885485768318176, + -0.08305133879184723, + 0.14699284732341766, + 0.041526660323143005, + -0.41915464401245117, + -0.03320619463920593, + -0.2105114459991455, + -0.3015945255756378 + ], + [ + 0.19390295445919037, + -0.24859407544136047, + 1.6159428358078003, + -0.31207573413848877, + -0.9656159281730652, + 0.3233919143676758, + -0.939553439617157, + 0.18780739605426788, + 0.046040572226047516, + 0.5581344366073608, + -0.01026865839958191, + -1.2877377271652222, + -0.13225805759429932, + 0.1152028888463974, + -0.2035476267337799, + 0.33016154170036316, + 0.06611089408397675, + -0.2509678602218628, + -0.08846113085746765, + -0.478963702917099, + -0.016499951481819153 + ] + ], + "generated_utilities": [ + 0.041229456663131714, + 2.0, + 0.03825731575489044, + 0.03897048532962799, + 2.0, + 0.03536248207092285, + 2.0, + 2.0 + ], + "group_id": "ms-9fad82fd288e4296e354", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.19493231177330017, + 0.06623455882072449, + 2.0, + 0.15396031737327576, + 0.16184639930725098, + 0.025735855102539062, + 0.134642094373703, + 0.15606965124607086, + 0.12982505559921265, + 0.14038166403770447, + 0.09718823432922363, + 0.015769004821777344, + 0.09577034413814545, + 0.057760089635849, + 0.047366172075271606, + 0.035309627652168274 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + -0.0622546412050724, + -0.09277644753456116, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + -0.005287081003189087, + 0.0, + -0.005681931972503662, + 0.03343720734119415, + 1.0536105632781982, + -0.00917971134185791, + -1.333775520324707, + 0.03716959059238434, + 0.0, + 0.019381746649742126, + 0.026990801095962524, + 0.21505475044250488, + -0.029777705669403076, + -0.14875531196594238, + -0.048092663288116455, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.959449768066406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 5.103323459625244, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.591951370239258 + ], + [ + -0.13909977674484253, + -0.26507115364074707, + 2.099381446838379, + -1.432984709739685, + -2.215683698654175, + 0.18431386351585388, + 2.42065691947937, + 0.3679453730583191, + -0.24726781249046326, + 2.055530071258545, + -0.4911385774612427, + -2.0511183738708496, + 0.04109602048993111, + 2.784976005554199, + 0.005537793040275574, + 0.05149966478347778, + 1.124292016029358, + -0.6457813382148743, + -1.226535677909851, + -0.28760284185409546, + 1.0270601511001587 + ], + [ + 0.30531278252601624, + -0.4336698055267334, + 2.6217315196990967, + -1.5083816051483154, + -2.5322670936584473, + 0.06784650683403015, + 2.3866119384765625, + -0.09851980209350586, + 0.2774229049682617, + 2.157179355621338, + -0.9796892404556274, + -2.8201849460601807, + 0.08135517686605453, + 2.818830728530884, + -0.6848310828208923, + 0.37913888692855835, + 0.7420558929443359, + -0.22611916065216064, + -1.5158166885375977, + 0.0614812970161438, + 1.3468891382217407 + ], + [ + 0.27444392442703247, + -0.22534464299678802, + 2.616544246673584, + -1.2606401443481445, + -2.0848917961120605, + -0.23496872186660767, + 2.1798665523529053, + 0.7968885898590088, + 0.26037585735321045, + 1.5473439693450928, + -0.233917236328125, + -1.867544412612915, + 0.48288947343826294, + 1.9728566408157349, + -0.4071806073188782, + 0.5528238415718079, + 1.6528295278549194, + -1.0936471223831177, + -1.062675952911377, + -0.5311015248298645, + 0.9778987765312195 + ], + [ + 0.5435068607330322, + 0.24469684064388275, + 2.6045663356781006, + -1.284223198890686, + -2.6095967292785645, + 0.9732104539871216, + 2.741159200668335, + 0.6031707525253296, + -0.5612202882766724, + 2.4774889945983887, + 0.1548280119895935, + -2.5516507625579834, + -0.0651187151670456, + 2.3864798545837402, + 0.130750373005867, + -0.1318313330411911, + 1.6559979915618896, + -1.0673086643218994, + -0.5477704405784607, + 0.18809780478477478, + 0.7076419591903687 + ], + [ + 0.31294435262680054, + 0.0263526514172554, + 3.0396268367767334, + -1.3165022134780884, + -1.8981198072433472, + 0.5049136281013489, + 2.31935715675354, + -0.4664327800273895, + -0.56757652759552, + 2.7719295024871826, + 0.21425002813339233, + -2.3925602436065674, + -0.26789456605911255, + 2.008218288421631, + -0.1879691481590271, + -0.34721577167510986, + 1.6638673543930054, + -0.30173182487487793, + -1.8719977140426636, + -0.8951039910316467, + 2.094865083694458 + ], + [ + 0.533157467842102, + 0.308005690574646, + 1.7770951986312866, + -0.6794775724411011, + -3.2953453063964844, + -0.15337547659873962, + 2.5854313373565674, + 0.07320006191730499, + -0.24033141136169434, + 2.5622174739837646, + -0.9168426990509033, + -1.5317132472991943, + -0.5942863821983337, + 2.2703769207000732, + -1.0709311962127686, + 0.6969255208969116, + 1.5304101705551147, + -0.7959219217300415, + -0.9165043830871582, + -0.8046965003013611, + 0.638719916343689 + ], + [ + 0.7393907904624939, + -0.13299384713172913, + 2.4281864166259766, + -1.861674189567566, + -3.3330533504486084, + -0.18516618013381958, + 1.6492295265197754, + -0.6932566165924072, + 0.06020718812942505, + 1.1755363941192627, + -0.016285300254821777, + -3.0671546459198, + -0.6052039861679077, + 2.6829426288604736, + 0.7021514177322388, + -0.477348268032074, + 0.466697633266449, + -0.9403301477432251, + -0.8690205812454224, + -1.1888904571533203, + 0.5360406041145325 + ], + [ + -0.0713023692369461, + -0.9776724576950073, + 1.2895376682281494, + -1.6489125490188599, + -1.4475497007369995, + -0.6822370886802673, + 3.14093017578125, + -0.5356632471084595, + -0.7523763179779053, + 2.766268253326416, + -0.8215572834014893, + -2.753042697906494, + 0.10929550230503082, + 2.570046901702881, + -0.34730663895606995, + 0.8763631582260132, + 1.3356308937072754, + -0.14330342411994934, + -1.1863114833831787, + -0.30006271600723267, + 0.7264611721038818 + ], + [ + 0.35943418741226196, + 0.22802312672138214, + 1.7158434391021729, + -1.4605827331542969, + -3.3450541496276855, + 0.23581582307815552, + 1.759531855583191, + 0.7012510299682617, + 0.0034519433975219727, + 1.4857903718948364, + -1.2866599559783936, + -2.1787426471710205, + 0.8807055354118347, + 1.9066205024719238, + -0.9918697476387024, + -0.32221919298171997, + 1.1986194849014282, + -1.187239646911621, + -0.5752912163734436, + -0.015687286853790283, + 1.775393009185791 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16282694041728973, + -0.1309112012386322, + 1.2432053089141846, + -0.13616442680358887, + -1.4076197147369385, + 0.15446226298809052, + 0.0, + -0.15669959783554077, + -0.08878482133150101, + 1.0536105632781982, + -0.09701520204544067, + -1.333775520324707, + 0.10988402366638184, + 0.0, + -0.06573517620563507, + -0.2108776718378067, + 0.21505475044250488, + -0.16805344820022583, + -0.14875531196594238, + 0.12529203295707703, + 0.0 + ] + ], + "predicted_scores": [ + 3.67533278465271, + 3.6726293563842773, + 3.551239013671875, + 3.468717575073242, + 3.2934136390686035, + 2.26446533203125, + 2.1814773082733154, + 1.823534607887268 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1176943004", + "selected_index": 0, + "state_hash": "0d923438bf2b227e3521cb937d011f5bddf9b553a882853098b702ec31090614", + "stored_base_utility": 0.19493231177330017, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.06781657040119171, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06781657040119171 + }, + "base_progress": 0.06781657040119171, + "base_success": false, + "base_utility": 0.06781657040119171, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.05735775828361511, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.05735775828361511 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.06561818718910217, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06561818718910217 + }, + { + "contact_quality": null, + "progress": 0.06600728631019592, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06600728631019592 + }, + { + "contact_quality": null, + "progress": 0.06556455790996552, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.06556455790996552 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 1.0, + 0.05735775828361511, + 1.0, + 0.06561818718910217, + 0.06600728631019592, + 0.06556455790996552, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-07aa7d1f9c01e93d04bc", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40", + "ms-dec13f2e40baaf5c8d40" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + true, + true, + false, + true, + false, + false, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "generated_tangents": [ + [ + -0.35940051078796387, + 0.26138144731521606, + 1.1218804121017456, + -1.1519173383712769, + -1.8066575527191162, + -0.2927666902542114, + -0.29577481746673584, + 0.049288854002952576, + -0.04845111817121506, + 0.1943264901638031, + 0.16906771063804626, + -1.1960171461105347, + 0.19308754801750183, + 0.3298017978668213, + 0.19319206476211548, + 0.14853538572788239, + 0.37110635638237, + -0.5730489492416382, + -0.4156043529510498, + 0.05488809943199158, + -0.06755031645298004 + ], + [ + -0.2960370182991028, + 0.2934601902961731, + 1.109288215637207, + -1.2864618301391602, + -1.6074607372283936, + -0.09004327654838562, + -0.2562062740325928, + -0.030219927430152893, + 0.046365439891815186, + 0.19546306133270264, + 0.2563927471637726, + -1.0252492427825928, + 0.32666411995887756, + 0.3295997083187103, + 0.10318690538406372, + 0.217575341463089, + 0.38629478216171265, + -0.7123311161994934, + -0.23487460613250732, + -0.14324526488780975, + 0.11484400182962418 + ], + [ + -0.2867763042449951, + 0.26990586519241333, + 1.1454148292541504, + -1.254696249961853, + -1.574082851409912, + -0.16223374009132385, + -0.46713531017303467, + 0.06653812527656555, + 0.004242160357534885, + 0.17129941284656525, + 0.13043415546417236, + -0.985645055770874, + 0.23969587683677673, + 0.28597861528396606, + 0.05446753650903702, + 0.24525628983974457, + 0.4033745527267456, + -0.7307695746421814, + -0.2557576894760132, + -0.16481541097164154, + 0.21549968421459198 + ], + [ + -0.37504494190216064, + 0.18524841964244843, + 0.9497373700141907, + -1.0685830116271973, + -1.176116704940796, + -0.13801464438438416, + -0.1652403473854065, + -0.017291411757469177, + 0.057547666132450104, + 0.2606729567050934, + 0.11610789597034454, + -1.0282714366912842, + 0.2768854796886444, + 0.07643024623394012, + 0.11365670710802078, + 0.22004571557044983, + 0.3535291850566864, + -0.5654693245887756, + -0.17161616683006287, + -0.11329098045825958, + -0.05826231837272644 + ], + [ + -0.2096341997385025, + 0.3217547535896301, + 1.2635153532028198, + -1.2642682790756226, + -1.6292884349822998, + 0.010172441601753235, + -0.44962286949157715, + -0.15860790014266968, + -0.05424071103334427, + 0.1652822047472, + 0.4396679699420929, + -0.8116848468780518, + 0.19552484154701233, + 0.14938408136367798, + 0.18905597925186157, + 0.207344651222229, + 0.3720264136791229, + -0.5570352077484131, + -0.5578036904335022, + -0.439746618270874, + -0.07734593749046326 + ], + [ + -0.13951198756694794, + 0.2989269495010376, + 1.293318748474121, + -1.1962066888809204, + -1.6288855075836182, + -0.2245807945728302, + -0.5613883137702942, + -0.01875273883342743, + 0.01569196954369545, + 0.32219794392585754, + 0.38344618678092957, + -0.7402890920639038, + 0.3756786286830902, + 0.19320344924926758, + 0.10246355086565018, + 0.2723497152328491, + 0.3831385374069214, + -0.5127334594726562, + -0.5140840411186218, + -0.2359667271375656, + 0.0076765865087509155 + ], + [ + -0.1366761177778244, + 0.3460448980331421, + 1.2618845701217651, + -1.1698354482650757, + -1.6935830116271973, + 0.03249320387840271, + -0.46773266792297363, + -0.08201417326927185, + -0.07811436802148819, + 0.2002832591533661, + 0.3100554347038269, + -0.7904842495918274, + 0.18730613589286804, + 0.1492442935705185, + 0.2157992571592331, + 0.24433599412441254, + 0.45189207792282104, + -0.48394668102264404, + -0.5058633685112, + -0.3069969117641449, + -0.0896095335483551 + ], + [ + -0.31444495916366577, + 0.11777603626251221, + 1.0513482093811035, + -1.2007355690002441, + -1.4836351871490479, + -0.02403835952281952, + -0.6299536824226379, + -0.013205036520957947, + 0.01766916736960411, + 0.2846088707447052, + 0.32949161529541016, + -1.0364447832107544, + 0.11130443215370178, + -0.06628595292568207, + 0.11114044487476349, + 0.347646564245224, + 0.29851871728897095, + -0.36744821071624756, + -0.7297919392585754, + -0.12092496454715729, + 0.3485521674156189 + ] + ], + "generated_utilities": [ + 2.0, + 2.0, + 0.05735775828361511, + 2.0, + 0.06561818718910217, + 0.06600728631019592, + 0.06556455790996552, + 2.0 + ], + "group_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.0793772041797638, + 2.0, + 2.0, + 2.0, + 0.1144598126411438, + 0.04584880173206329, + 0.08543911576271057, + 0.1307348757982254, + 0.10944780707359314, + 0.07284757494926453, + 0.05719476938247681, + 0.09351089596748352, + 0.13337698578834534, + 0.039893895387649536, + 0.09353269636631012, + 0.08224260807037354 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.016482055187225342, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.0903974175453186, + 0.0, + 0.012217208743095398, + 0.010632793419063091, + 0.03624187409877777, + 0.04524834081530571, + -1.3283450603485107, + -0.00880381464958191, + 0.0, + -0.014520794153213501, + 0.5632922649383545, + 0.003262042999267578, + -0.11586570739746094, + 0.0014255642890930176, + 0.015446990728378296, + 0.0 + ], + [ + 0.04007059335708618, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.12278485298156738, + 0.0, + -0.0962398424744606, + -0.08766138553619385, + 0.29553186893463135, + 0.10519622266292572, + -1.3283450603485107, + 0.00032788515090942383, + 0.0, + -0.13836456835269928, + 0.5632922649383545, + -0.13551044464111328, + -0.11586570739746094, + 0.07183873653411865, + -0.03684847429394722, + 0.0 + ], + [ + 0.08037775754928589, + 0.4143911600112915, + 1.4473507404327393, + -1.0172343254089355, + -1.722773551940918, + 0.33122140169143677, + 0.0, + 0.20786377787590027, + -0.42098182439804077, + -0.17684945464134216, + 0.1062583178281784, + -1.3283450603485107, + 0.28051087260246277, + 0.0, + 0.7470210194587708, + 0.5632922649383545, + 0.03247350454330444, + -0.11586570739746094, + 0.4809897243976593, + -0.1744043231010437, + 0.0 + ], + [ + -0.4189682602882385, + 1.1229554414749146, + 2.8264856338500977, + -2.2389113903045654, + -2.4226768016815186, + 1.2001769542694092, + 2.1568119525909424, + -0.016906127333641052, + -0.24083517491817474, + -0.23574936389923096, + -0.13933053612709045, + -2.458132028579712, + 0.40251442790031433, + 0.870932936668396, + -0.28115370869636536, + 1.4463695287704468, + 0.9248121380805969, + -0.9938455820083618, + -0.680131196975708, + -0.051702484488487244, + -0.4663417339324951 + ], + [ + -0.6373537182807922, + 1.3288449048995972, + 3.1669085025787354, + -1.4171695709228516, + -2.242492198944092, + 1.6900161504745483, + 2.778874158859253, + 0.47899991273880005, + 0.3402186334133148, + 0.4507899582386017, + 0.8237879276275635, + -1.7865383625030518, + 1.1732149124145508, + 1.5826115608215332, + 0.4416889548301697, + 2.222722053527832, + 0.6443713307380676, + -1.8598005771636963, + -0.9690301418304443, + 0.3504742383956909, + -1.1371408700942993 + ] + ], + "predicted_scores": [ + 5.1568989753723145, + 4.8573408126831055, + 4.804030418395996, + 4.69545316696167, + 4.384824752807617, + 4.1982316970825195, + 4.097631931304932, + 3.475550413131714 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "998761162", + "selected_index": 0, + "state_hash": "78cc63bdeba9f4d6d6da2efd23c0f7f6631321321e917a70299a7c8025e71d47", + "stored_base_utility": 0.0793772041797638, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.023216769099235535, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.023216769099235535 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.023216769099235535, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e", + "ms-07aa7d1f9c01e93d04bc", + "ms-70e24231e9bb560abb3e", + "ms-70e24231e9bb560abb3e" + ], + "candidate_source_task_ids": [ + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1", + "PushCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + true, + true, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-fc8084d5cd7da937fc02", + "generated_tangents": [ + [ + 0.06674793362617493, + -0.017455171793699265, + 1.4833317995071411, + -0.5928583741188049, + -1.0986278057098389, + -0.07370790839195251, + -0.3972606658935547, + 0.10492485761642456, + -0.029081830754876137, + 0.08049751818180084, + 0.10041757673025131, + -1.2483304738998413, + -0.08490768074989319, + 0.040428973734378815, + 0.06932853907346725, + 0.09570556879043579, + 0.11309096217155457, + 0.1258828341960907, + -0.0645999014377594, + -0.05343492329120636, + 0.128771111369133 + ], + [ + 0.017478227615356445, + 0.04112286865711212, + 1.4941059350967407, + -0.5866529941558838, + -1.1760003566741943, + -0.08213865756988525, + -0.2005012035369873, + 0.01987043023109436, + -0.022918878123164177, + 0.1040123850107193, + 0.20422354340553284, + -1.2287273406982422, + 0.00930255651473999, + 0.05246828496456146, + 0.037923481315374374, + 0.11094716191291809, + 0.06123155355453491, + 0.10196759551763535, + -0.16752895712852478, + -0.012012243270874023, + 0.050198912620544434 + ], + [ + -0.03730447590351105, + 0.014132339507341385, + 1.4144362211227417, + -0.6733148097991943, + -1.1825482845306396, + -0.008287966251373291, + -0.35631752014160156, + -0.03764437139034271, + -0.012792257592082024, + 0.07346613705158234, + 0.19872477650642395, + -1.2307169437408447, + 0.023642033338546753, + 0.005198977887630463, + 0.0063120052218437195, + 0.06589984893798828, + 0.07383552193641663, + 0.0706728845834732, + -0.09465029835700989, + -0.0023286044597625732, + 0.0881691575050354 + ], + [ + -0.016812756657600403, + 0.04553462564945221, + 1.3833285570144653, + -0.6451879143714905, + -1.2300786972045898, + -0.14793860912322998, + -0.3728104829788208, + 0.04813587665557861, + -0.050586894154548645, + 0.08829276263713837, + 0.21551665663719177, + -1.2655410766601562, + 0.07566395401954651, + 0.14191819727420807, + 0.023812171071767807, + 0.08441734313964844, + 0.0654795914888382, + 0.11626894772052765, + -0.16875094175338745, + -0.17117637395858765, + 0.06423484534025192 + ], + [ + -0.2635042667388916, + -0.024782611057162285, + 1.2271443605422974, + -0.4885129928588867, + -0.8494554162025452, + -0.08545178174972534, + -0.2659471035003662, + 0.013631850481033325, + 0.004045146517455578, + 0.21345219016075134, + 0.2647266387939453, + -1.1775646209716797, + -0.03151434659957886, + -0.1833101063966751, + 0.1026906818151474, + 0.07188206911087036, + 0.20349493622779846, + -0.06375865638256073, + -0.2572023868560791, + 0.29140204191207886, + -0.397828608751297 + ], + [ + -0.15293361246585846, + 0.05725117772817612, + 1.328978419303894, + -0.7022752165794373, + -1.2217082977294922, + 0.010761305689811707, + -0.1359962821006775, + -0.019572749733924866, + -0.05365105718374252, + 0.02904101461172104, + 0.14346858859062195, + -1.4018242359161377, + -0.2438257783651352, + 0.16816318035125732, + 0.07734844833612442, + -0.01614791341125965, + 0.15640252828598022, + 0.13216665387153625, + -0.3579481244087219, + 0.14938241243362427, + -0.19346219301223755 + ], + [ + -0.3136376142501831, + 0.05857615917921066, + 1.4760576486587524, + -0.7016957998275757, + -0.9582225680351257, + -0.24509742856025696, + -0.5958195328712463, + -0.0006892681121826172, + -0.052986957132816315, + 0.14698907732963562, + 0.15362226963043213, + -1.0713621377944946, + -0.08330953121185303, + 0.13679146766662598, + -0.04393580183386803, + 0.07613711059093475, + 0.15425747632980347, + 0.12584447860717773, + -0.2639538049697876, + 0.09439608454704285, + 0.2678697109222412 + ], + [ + -0.3261988162994385, + 0.06708893179893494, + 1.3728224039077759, + -0.7918506860733032, + -1.060340166091919, + -0.18442299962043762, + -0.8321369290351868, + -0.04335425794124603, + -0.04527103155851364, + 0.16791337728500366, + 0.16491550207138062, + -1.0117523670196533, + 0.03638753294944763, + -0.009315937757492065, + -0.1257777065038681, + 0.1290484368801117, + 0.17142516374588013, + 0.18401315808296204, + -0.24615490436553955, + 0.025304079055786133, + 0.20221281051635742 + ] + ], + "generated_utilities": [ + 0.023216769099235535, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0 + ], + "group_id": "ms-fc8084d5cd7da937fc02", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.120394766330719, + 0.1662210375070572, + 2.0, + 0.0270967036485672, + 0.16474007070064545, + 0.014024153351783752, + 0.13986149430274963, + 0.17214345932006836, + 0.19527268409729004, + 0.08535367250442505, + 0.1354738473892212, + 0.11415943503379822, + 0.05034071207046509, + 0.10602898895740509, + 0.01804502308368683, + 0.09680917859077454 + ], + "instruction": "Push the cube into the goal region.", + "negative_tangents": [ + [ + 0.4742112457752228, + -0.6877930760383606, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.0012008249759674072, + 0.0, + 0.04633669555187225, + -0.307714581489563, + -0.22357922792434692, + -0.35245198011398315, + -1.486854076385498, + 0.15573161840438843, + 0.0, + 0.5157145261764526, + 0.243257537484169, + 0.2085912823677063, + 0.48896148800849915, + -0.06008961796760559, + 0.4603208899497986, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.883587837219238, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.6201469898223877, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.5597648620605469 + ], + [ + -0.8389496207237244, + 0.5873794555664062, + 2.327263116836548, + -2.175445795059204, + -1.844232439994812, + 0.15723112225532532, + 2.920079231262207, + -0.5335484743118286, + -0.6761462688446045, + 0.48151493072509766, + 0.2968217730522156, + -2.99812650680542, + -0.27805766463279724, + 2.0262699127197266, + 0.10151885449886322, + -0.7244771122932434, + 1.0176090002059937, + 1.278473973274231, + -0.2260074019432068, + -1.0207180976867676, + 0.7682583928108215 + ], + [ + 0.2503487765789032, + 0.20628705620765686, + 2.289511203765869, + -0.8669976592063904, + -1.9807379245758057, + 1.1820356845855713, + 2.8384368419647217, + 0.43699324131011963, + -0.688888669013977, + 1.0812339782714844, + -0.18395745754241943, + -1.6906647682189941, + -0.040969282388687134, + 1.254612684249878, + 0.24077603220939636, + -1.0562812089920044, + 1.2436273097991943, + 0.2388666272163391, + -0.5016235709190369, + 0.4570234417915344, + 1.355008840560913 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.10285450518131256, + -0.2634444832801819, + 1.3578369617462158, + -0.7960293292999268, + -1.423076868057251, + -0.12851905822753906, + 0.0, + 0.06062778830528259, + 0.006933145225048065, + -0.08638572692871094, + 0.32057398557662964, + -1.486854076385498, + -0.1531560719013214, + 0.0, + 0.14251592755317688, + 0.07826228439807892, + -0.24055713415145874, + -0.11116695404052734, + 0.200389564037323, + -0.07051187753677368, + 0.0 + ], + [ + -0.07637833058834076, + 0.6696901321411133, + 2.130587100982666, + -1.9006763696670532, + -2.6654670238494873, + 0.06729818880558014, + 2.65816593170166, + -0.0612010583281517, + -0.1226595938205719, + 0.9200000166893005, + -0.4352770149707794, + -2.266615390777588, + 0.48697614669799805, + 1.5162358283996582, + -0.014945849776268005, + -0.4459138512611389, + 1.1448073387145996, + 0.8031747937202454, + -0.6817312240600586, + -0.20262596011161804, + 1.1690938472747803 + ], + [ + 0.519248902797699, + 0.27624276280403137, + 1.911887764930725, + -1.8784401416778564, + -2.9212372303009033, + 0.7762739658355713, + 2.547428846359253, + 0.08362685143947601, + -0.1671719253063202, + 0.8596282601356506, + -0.17510351538658142, + -2.220954656600952, + 0.4918195605278015, + 1.357650876045227, + 0.3462192416191101, + 0.2476332038640976, + 0.6965686082839966, + 1.0915483236312866, + -0.6818809509277344, + -0.7465203404426575, + 0.724744439125061 + ] + ], + "predicted_scores": [ + 3.261988639831543, + 3.237738847732544, + 3.007798194885254, + 2.962484121322632, + 2.906874179840088, + 2.7258682250976562, + 2.4441936016082764, + 2.238560438156128 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1991414747", + "selected_index": 0, + "state_hash": "127cf975da326f174c815484811a6acc4855bf07460a2aa5b0d728cb2a1f41c7", + "stored_base_utility": 0.120394766330719, + "task_id": "PushCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20275163650512695, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20275163650512695 + }, + "base_progress": 0.20275163650512695, + "base_success": false, + "base_utility": 0.20275163650512695, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1694907248020172, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1694907248020172 + }, + { + "contact_quality": null, + "progress": 0.21781498193740845, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21781498193740845 + }, + { + "contact_quality": null, + "progress": 0.17819397151470184, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17819397151470184 + }, + { + "contact_quality": null, + "progress": 0.200613334774971, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.200613334774971 + }, + { + "contact_quality": null, + "progress": 0.0787760317325592, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0787760317325592 + }, + { + "contact_quality": null, + "progress": 0.1785118579864502, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1785118579864502 + }, + { + "contact_quality": null, + "progress": 0.19316479563713074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19316479563713074 + }, + { + "contact_quality": null, + "progress": 0.187234565615654, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.187234565615654 + } + ], + "candidate_progress": [ + 0.1694907248020172, + 0.21781498193740845, + 0.17819397151470184, + 0.200613334774971, + 0.0787760317325592, + 0.1785118579864502, + 0.19316479563713074, + 0.187234565615654 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-3afd8959363e917591d4", + "generated_tangents": [ + [ + 0.7377750873565674, + 0.26347267627716064, + -0.08314630389213562, + -0.9994946718215942, + 0.9082075357437134, + -0.19150912761688232, + 1.3097355365753174, + -0.16698706150054932, + 0.08360017091035843, + -0.1160506084561348, + 0.101993128657341, + -0.1766650676727295, + -1.296095609664917, + -0.5928234457969666, + -0.11710420250892639, + 0.13019615411758423, + 0.3176431357860565, + 0.09339398145675659, + -0.3347843885421753, + -0.9365300536155701, + -0.6035232543945312 + ], + [ + 0.24448202550411224, + 0.39407414197921753, + -0.3670208752155304, + -1.0298949480056763, + 0.5744311809539795, + -0.264547735452652, + 0.38746458292007446, + -0.03442789614200592, + -0.060970596969127655, + -0.03835376352071762, + 0.3443411886692047, + -0.061457276344299316, + -2.3562428951263428, + -0.35382968187332153, + -0.29803335666656494, + 0.06864343583583832, + 0.25068435072898865, + -0.1870059221982956, + -0.2646765112876892, + -1.807386875152588, + -0.28571200370788574 + ], + [ + 0.21231411397457123, + 0.32847559452056885, + -0.23204758763313293, + -1.045691728591919, + 0.6634618043899536, + 0.10659906268119812, + 0.27520033717155457, + -0.11260038614273071, + -0.06890711188316345, + 0.04264010488986969, + 0.22799637913703918, + -0.29159629344940186, + -2.414637804031372, + -0.26020193099975586, + -0.14008629322052002, + 0.08521687984466553, + 0.372461199760437, + -0.19722320139408112, + -0.409726619720459, + -1.4473233222961426, + -0.3482573330402374 + ], + [ + 0.17680992186069489, + 0.3088708519935608, + -0.4014383852481842, + -1.1345856189727783, + 0.6421109437942505, + -0.29996058344841003, + 0.4782603979110718, + -0.0847964882850647, + -0.058243148028850555, + 0.04311869293451309, + 0.22336465120315552, + -0.22831225395202637, + -2.349536180496216, + -0.2620731592178345, + -0.2297009825706482, + 0.0788859874010086, + 0.28485727310180664, + -0.14942477643489838, + -0.502982497215271, + -1.7022714614868164, + -0.32193323969841003 + ], + [ + 0.3183712959289551, + 0.1401023268699646, + -0.03044000267982483, + -1.1987438201904297, + 0.9590957760810852, + 0.3255043029785156, + 0.49064400792121887, + 0.11818192154169083, + 0.05391983687877655, + 0.2607717216014862, + 0.15072235465049744, + -0.08412399888038635, + -2.3221940994262695, + -0.5120459198951721, + -0.20447063446044922, + 0.17877575755119324, + 0.4177470803260803, + 0.06512308120727539, + -0.5801995992660522, + -1.6647918224334717, + -0.24401062726974487 + ], + [ + 0.12330439686775208, + 0.19448666274547577, + 0.08899796009063721, + -0.9709631204605103, + 0.6608500480651855, + -0.17747187614440918, + 0.20461422204971313, + 0.08426865935325623, + 0.05225510895252228, + 0.010891012847423553, + 0.3167150914669037, + 0.03324928879737854, + -2.2975637912750244, + -0.047919467091560364, + -0.1316448599100113, + -0.045330993831157684, + -0.006210550665855408, + 0.09347040951251984, + -0.2524777054786682, + -1.4940173625946045, + 0.09557963162660599 + ], + [ + 0.1730630099773407, + 0.2970139980316162, + -0.4695887267589569, + -1.067762017250061, + 0.6304346323013306, + -0.29939547181129456, + 0.44776809215545654, + 0.16391919553279877, + -0.0837835967540741, + 0.06573087722063065, + 0.22065696120262146, + -0.47530072927474976, + -2.380866289138794, + -0.3768603205680847, + -0.21038717031478882, + 0.1146782636642456, + 0.3076738119125366, + -0.07745186984539032, + -0.5000383257865906, + -1.6727688312530518, + -0.2857869267463684 + ], + [ + 0.2767746150493622, + 0.2590828537940979, + -0.013947397470474243, + -1.1364459991455078, + 0.6993545293807983, + -0.41169485449790955, + 0.053765684366226196, + -0.053581610321998596, + -0.06956262141466141, + -0.0026132240891456604, + 0.24177467823028564, + -0.0682813823223114, + -2.4299659729003906, + -0.08124825358390808, + -0.17382197082042694, + 0.06207389757037163, + 0.1271786391735077, + 0.07436180859804153, + -0.3390803933143616, + -1.6315560340881348, + 0.030501589179039 + ] + ], + "generated_utilities": [ + 0.1694907248020172, + 0.21781498193740845, + 0.17819397151470184, + 0.200613334774971, + 0.0787760317325592, + 0.1785118579864502, + 0.19316479563713074, + 0.187234565615654 + ], + "group_id": "ms-3afd8959363e917591d4", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8582140207290649, + 2.0, + 0.18031787872314453, + 0.07887399196624756, + 0.5568947792053223, + 0.16715207695960999, + 0.5635193586349487, + 0.18913890421390533, + 0.5682768225669861, + 0.10102970898151398, + 0.09163376688957214, + 0.12909311056137085, + 0.12299326062202454, + 0.09030237793922424, + 0.13531166315078735, + 0.06874948740005493 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.15699657797813416, + 0.6247961521148682, + 0.107974112033844, + -0.3284463882446289, + 0.22832846641540527, + -0.12728703022003174, + 0.0, + -0.2739208936691284, + -0.051639966666698456, + -0.2660524249076843, + -0.2539072632789612, + 0.09800386428833008, + -1.84090256690979, + 0.0, + -0.017676634714007378, + -0.018830953165888786, + 0.1583891659975052, + -0.07512116432189941, + -0.01770937442779541, + -1.5237011909484863, + 0.0 + ], + [ + 0.4675478935241699, + 0.6247961521148682, + 0.3345896601676941, + -0.5404787063598633, + -0.06834304332733154, + -0.12728703022003174, + 0.0, + -0.14999517798423767, + -0.2730908691883087, + 0.08837036788463593, + 0.45600926876068115, + 0.410434365272522, + -1.84090256690979, + 0.0, + 0.4969322681427002, + 0.22034725546836853, + 0.2921968400478363, + 0.5872196555137634, + 0.19443652033805847, + -1.5237011909484863, + 0.0 + ], + [ + 0.41189882159233093, + 1.6247961521148682, + 0.12157746404409409, + -1.328446388244629, + 0.9316569566726685, + -1.1272870302200317, + 1.276085376739502, + -0.029595039784908295, + -0.014941371977329254, + 0.004283940885215998, + 0.6188332438468933, + -0.49464237689971924, + -2.84090256690979, + -1.8046140670776367, + 0.0007966384291648865, + -0.006994657218456268, + 0.014263113960623741, + 0.6869955658912659, + -0.2918644845485687, + -2.5237011909484863, + -1.7481608390808105 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.552170753479004, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.6092281341552734, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.496321678161621 + ], + [ + 0.44720953702926636, + 1.4417266845703125, + 0.4182493984699249, + -1.547792911529541, + 0.990667462348938, + -0.9114025831222534, + 1.535681962966919, + 0.21791213750839233, + -0.30513259768486023, + 0.013669610023498535, + 0.7976462841033936, + -0.6395556926727295, + -2.5809314250946045, + -1.571488618850708, + 0.038753002882003784, + 0.04824339970946312, + -0.11262937635183334, + 0.9187417030334473, + -0.1308273822069168, + -2.59043288230896, + -1.9625922441482544 + ], + [ + 0.5162960290908813, + 1.2868248224258423, + 0.2058304250240326, + -1.4812198877334595, + 1.085485816001892, + -1.05157470703125, + 1.1480523347854614, + 0.2796477973461151, + -0.1532609462738037, + 0.12988577783107758, + 0.8581418395042419, + -0.6760798692703247, + -2.769819736480713, + -2.192568778991699, + 0.10523252189159393, + 0.05066283047199249, + -0.08061028271913528, + 0.7438116669654846, + -0.07841359078884125, + -2.3079848289489746, + -1.4200246334075928 + ], + [ + 0.4112767279148102, + 1.8745536804199219, + 0.5301340222358704, + -0.9917724132537842, + 0.8905197978019714, + -0.8391705751419067, + 1.3120794296264648, + 0.3723032474517822, + -0.19477176666259766, + -0.35689839720726013, + 0.2843710780143738, + -0.45066511631011963, + -2.812030792236328, + -1.588818073272705, + -0.09205310046672821, + -0.5067525506019592, + -0.41988417506217957, + 1.0423402786254883, + 0.1759319305419922, + -2.945967197418213, + -2.05134916305542 + ], + [ + 0.46365559101104736, + 1.105017900466919, + 0.466808021068573, + -1.5382812023162842, + 0.8287478089332581, + -1.4617033004760742, + 0.8807560205459595, + 0.2107943296432495, + 0.2954367995262146, + 0.45928943157196045, + 0.6575880646705627, + -0.5414218902587891, + -2.711836099624634, + -1.6827114820480347, + 0.3175691068172455, + -0.38371264934539795, + 0.34041139483451843, + 0.34132614731788635, + -0.8783347606658936, + -2.133782148361206, + -1.3974260091781616 + ], + [ + 0.13586324453353882, + 1.6158640384674072, + 0.24489957094192505, + -1.4942185878753662, + 1.1602565050125122, + -1.648895502090454, + 1.1836220026016235, + 0.17831489443778992, + -0.5400142669677734, + -0.42907604575157166, + 0.5951722264289856, + -0.7885013818740845, + -3.2272911071777344, + -2.208972692489624, + -0.4857475459575653, + 0.5623884201049805, + 0.02566027268767357, + 0.590702474117279, + -0.6581307649612427, + -2.7863473892211914, + -2.031818151473999 + ], + [ + 0.21801596879959106, + 1.0595136880874634, + 0.9040011763572693, + -1.1076470613479614, + 1.2744677066802979, + -0.7150566577911377, + 1.0042598247528076, + 0.5841938257217407, + -0.2832033336162567, + -0.45343834161758423, + -0.17037296295166016, + -0.5152367353439331, + -2.1963798999786377, + -1.5652427673339844, + 0.03223085403442383, + 0.07496664673089981, + -0.1515023559331894, + 1.233866572380066, + 0.4301607310771942, + -2.489328384399414, + -1.8620245456695557 + ], + [ + -0.15804454684257507, + 2.2168471813201904, + 0.9374236464500427, + -1.8158314228057861, + 1.114590048789978, + -0.9279311895370483, + 1.890993356704712, + -0.8653054237365723, + -0.5551189184188843, + 0.4295695126056671, + 1.1614964008331299, + -0.5840407609939575, + -2.900725841522217, + -2.579883575439453, + -0.1900041103363037, + 0.42368441820144653, + 0.38376376032829285, + 0.9654008150100708, + 0.10608989000320435, + -3.332181453704834, + -1.8879828453063965 + ], + [ + 1.0901068449020386, + 2.576328754425049, + 0.5122992396354675, + -1.9627916812896729, + 0.026246964931488037, + -0.3622438311576843, + 0.9286589622497559, + 0.644849956035614, + 0.9211093187332153, + 0.3528344929218292, + 0.3661169707775116, + 0.20941221714019775, + -2.582620143890381, + -1.6867890357971191, + 0.09869468957185745, + -0.8795469403266907, + -0.2239246666431427, + 0.5206944942474365, + -0.20716872811317444, + -2.8072235584259033, + -2.3385443687438965 + ], + [ + -0.0005125105381011963, + 1.8920408487319946, + 1.0018010139465332, + -2.2556979656219482, + 0.7338991165161133, + -0.4559611678123474, + 1.027335524559021, + -1.0165901184082031, + -0.14792552590370178, + 0.11212217807769775, + 0.8827763795852661, + 0.1683982014656067, + -3.2136244773864746, + -2.4300284385681152, + -0.9183067083358765, + 0.10752613097429276, + -0.2955099046230316, + 0.31867098808288574, + 0.06515705585479736, + -3.416110038757324, + -1.5137704610824585 + ], + [ + -0.39508435130119324, + 2.55599308013916, + 0.128604456782341, + -1.8821169137954712, + 0.6769157648086548, + -0.7477259635925293, + 1.6593644618988037, + -0.005859369412064552, + 0.22320973873138428, + -0.2906110882759094, + 0.22343486547470093, + -1.488371729850769, + -2.81829833984375, + -0.939697265625, + 0.9867495894432068, + 0.2955887019634247, + -0.11908379942178726, + 0.473803848028183, + -0.11059392988681793, + -3.471217155456543, + -2.3480191230773926 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.018576741218566895, + 0.6247961521148682, + -0.017397306859493256, + -0.3284463882446289, + 0.044771432876586914, + -0.12728703022003174, + 0.0, + -0.005512619391083717, + 0.03525955602526665, + -0.005881692748516798, + 0.04493492841720581, + 0.056808531284332275, + -1.84090256690979, + 0.0, + 0.11075251549482346, + -0.05448189005255699, + -0.041690655052661896, + 0.010008454322814941, + 0.056893616914749146, + -1.5237011909484863, + 0.0 + ] + ], + "predicted_scores": [ + 7.24985408782959, + 5.471922874450684, + 5.066809177398682, + 5.065141677856445, + 4.941882610321045, + 4.642519950866699, + 4.432784080505371, + 4.277013778686523 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "75070641", + "selected_index": 0, + "state_hash": "bd1bbc057450972b9fa6e08a54ebd8c6d82d09997dad16cecf9584632b705b7b", + "stored_base_utility": 0.8582140207290649, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.15755623579025269, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15755623579025269 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1525605022907257, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1525605022907257 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.13584548234939575, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13584548234939575 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.15755623579025269, + 1.0, + 1.0, + 0.1525605022907257, + 1.0, + 1.0, + 0.13584548234939575, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-4cab292bc65fa59274ef", + "ms-0949097d19e597fc7f63", + "ms-4cab292bc65fa59274ef", + "ms-6bd5870eb5ff0dd76919", + "ms-4cab292bc65fa59274ef" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + true, + false, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-3ff1aedb72f65a182b58", + "generated_tangents": [ + [ + 0.08186054974794388, + 0.006009191274642944, + -0.06694689393043518, + -0.0055119991302490234, + -0.010643720626831055, + -2.132002353668213, + -0.25076591968536377, + 0.0505671352148056, + 0.01362932100892067, + 0.14865826070308685, + 0.19644054770469666, + 0.04079253971576691, + -1.9053442478179932, + -0.19569317996501923, + 0.11837344616651535, + -0.027462827041745186, + 0.019821152091026306, + 0.037378594279289246, + -0.1200321614742279, + -1.6845331192016602, + -0.12577423453330994 + ], + [ + 0.13234175741672516, + 0.04501605033874512, + 0.13640928268432617, + -0.14431041479110718, + 0.037565574049949646, + -2.147111654281616, + -0.19054502248764038, + 0.0752481073141098, + -0.031138623133301735, + 0.13148614764213562, + 0.2956600487232208, + 0.05605921149253845, + -1.8617624044418335, + -0.10655191540718079, + 0.02225549891591072, + -0.050534848123788834, + 0.028844401240348816, + 0.060540422797203064, + -0.08961048722267151, + -1.5277156829833984, + -0.10681098699569702 + ], + [ + 0.05637793242931366, + 0.062057361006736755, + 0.09954336285591125, + -0.11781985312700272, + -0.06153351068496704, + -2.009679079055786, + -0.3176780343055725, + 0.06290937960147858, + -0.09404604882001877, + 0.12751705944538116, + 0.13278409838676453, + 0.1133994311094284, + -1.8079689741134644, + -0.23078136146068573, + 0.049533046782016754, + -0.01197347417473793, + 0.07555678486824036, + -0.18356020748615265, + -0.24197518825531006, + -1.5916109085083008, + -0.15395718812942505 + ], + [ + 0.07001128047704697, + -0.006341617554426193, + -0.013905733823776245, + -0.10054901242256165, + 0.004002317786216736, + -2.089357376098633, + -0.29093819856643677, + 0.07939032465219498, + 0.0024054748937487602, + 0.12677901983261108, + 0.24757447838783264, + 0.05934283137321472, + -1.8960217237472534, + -0.24146471917629242, + 0.1240246593952179, + 0.0006272131577134132, + 0.07136332988739014, + -0.013259604573249817, + -0.03865545988082886, + -1.662118911743164, + -0.1441478133201599 + ], + [ + -0.10122169554233551, + -0.014751335605978966, + 0.04853445291519165, + 0.018408209085464478, + -0.0015732795000076294, + -2.041250467300415, + -0.17305952310562134, + 0.21826361119747162, + -0.05995109677314758, + 0.15702185034751892, + 0.09051486104726791, + 0.06831461191177368, + -1.9522912502288818, + -0.027929797768592834, + 0.01918385550379753, + 0.04103182256221771, + 0.19815915822982788, + 0.033082395792007446, + -0.08686968684196472, + -1.602992296218872, + -0.1569117307662964 + ], + [ + -0.30795198678970337, + -0.09053993225097656, + 0.28962182998657227, + 0.0344066321849823, + -0.08195352554321289, + -2.2240450382232666, + -0.20861703157424927, + 0.2981802225112915, + -0.07621319591999054, + -0.11032906919717789, + 0.157115638256073, + 0.015329763293266296, + -2.1129181385040283, + 0.04135496914386749, + 0.09659168124198914, + 0.0655207633972168, + 0.15245279669761658, + 0.04563286900520325, + -0.24892079830169678, + -1.642627477645874, + 0.08329165726900101 + ], + [ + -0.013811811804771423, + -0.03245372697710991, + -0.31161126494407654, + -0.16774164140224457, + -0.24508851766586304, + -1.9709912538528442, + -0.17202073335647583, + 0.26413002610206604, + -0.07094306498765945, + 0.14067888259887695, + 0.3410402834415436, + -0.3525315523147583, + -2.244873285293579, + 0.23459428548812866, + -0.2386588752269745, + 0.031863175332546234, + 0.38297897577285767, + -0.11049143970012665, + -0.40620607137680054, + -1.3109962940216064, + -0.4964732229709625 + ], + [ + -0.039339467883110046, + -0.10939320921897888, + 0.28396040201187134, + 0.07240608334541321, + -0.12615758180618286, + -2.052049160003662, + -0.2889990210533142, + 0.2383739948272705, + -0.035172559320926666, + 0.02148519456386566, + 0.29923754930496216, + 0.002026692032814026, + -2.1962761878967285, + -0.027928397059440613, + -0.053601328283548355, + 0.14903222024440765, + 0.4132443070411682, + 0.1425221711397171, + -0.08159804344177246, + -1.587170124053955, + -0.0466499924659729 + ] + ], + "generated_utilities": [ + 0.15755623579025269, + 2.0, + 2.0, + 0.1525605022907257, + 2.0, + 2.0, + 0.13584548234939575, + 2.0 + ], + "group_id": "ms-3ff1aedb72f65a182b58", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.851725161075592, + 2.0, + 0.8126396536827087, + 2.0, + 0.7855525612831116, + 0.7821746468544006, + 0.6115735173225403, + 0.5732660889625549, + 0.7798006534576416, + 0.09109212458133698, + 0.8456447124481201, + 0.16636842489242554, + 0.0955686867237091, + 0.0822453498840332, + 0.0018809139728546143, + 0.08579078316688538 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.09319563210010529, + 0.18433167040348053, + 0.03857926279306412, + 0.20899039506912231, + -0.19732166826725006, + -2.7716264724731445, + -0.9089346528053284, + -0.012062903493642807, + -0.001710914308205247, + 0.02447224222123623, + 0.6588084697723389, + -0.43759873509407043, + -2.7224881649017334, + -1.560755968093872, + -0.002114389091730118, + -0.004199527204036713, + 0.015975257381796837, + 0.7779304385185242, + -0.31706690788269043, + -2.4826252460479736, + -1.5341289043426514 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.8178693056106567, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.121511936187744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0682578086853027 + ], + [ + -0.10748189687728882, + 0.29221633076667786, + 0.12325333058834076, + 0.21938443183898926, + 0.017790749669075012, + -2.485055446624756, + -1.1110891103744507, + 0.062413301318883896, + -0.2567306160926819, + -0.15453597903251648, + 0.549060583114624, + -0.18322163820266724, + -2.5428390502929688, + -1.8537542819976807, + -0.07324761152267456, + 0.28765758872032166, + 0.16924789547920227, + 0.6205734610557556, + -0.4268453121185303, + -2.6385459899902344, + -1.5760513544082642 + ], + [ + 0.44683152437210083, + 0.3713763356208801, + 0.1084144338965416, + 0.2590586543083191, + 0.1997859627008438, + -2.787759780883789, + -1.1921976804733276, + 0.33878061175346375, + 0.019987009465694427, + 0.021556159481406212, + 0.36814823746681213, + -0.2135106772184372, + -2.416769027709961, + -1.3776189088821411, + -0.3500843346118927, + -0.26303720474243164, + 0.10322260856628418, + 0.9827795624732971, + -0.3747403025627136, + -2.6419029235839844, + -1.4116339683532715 + ], + [ + -0.2883707880973816, + 0.02873578667640686, + -0.11175497621297836, + -0.106661856174469, + 0.03059367835521698, + -2.3899149894714355, + -1.0789995193481445, + 0.2610729932785034, + 0.4800889790058136, + -0.45478329062461853, + 1.0867195129394531, + -0.7417076826095581, + -3.09529185295105, + -1.730720043182373, + -0.33508142828941345, + 0.10841497033834457, + -0.38902348279953003, + 0.44348275661468506, + 0.0854053795337677, + -2.3334603309631348, + -1.7851121425628662 + ], + [ + 0.16367590427398682, + 0.05515189468860626, + -0.3197028636932373, + 0.5501840114593506, + -0.3621746301651001, + -3.0582237243652344, + -1.135798454284668, + -0.15054848790168762, + -0.5578444004058838, + 0.3691062331199646, + 0.9388304352760315, + -0.19103051722049713, + -2.7536559104919434, + -2.093773603439331, + -0.037153881043195724, + 0.19014233350753784, + -0.36205345392227173, + 0.24469107389450073, + -0.5916419625282288, + -2.7417092323303223, + -1.637229323387146 + ], + [ + -0.3646707534790039, + 0.6696967482566833, + -0.44303250312805176, + -0.4559982419013977, + -0.6542845368385315, + -2.9630653858184814, + -1.4721050262451172, + 0.5010843873023987, + 0.5162685513496399, + 0.42702627182006836, + 0.6571387648582458, + -0.08207055926322937, + -3.278073310852051, + -1.7742136716842651, + -0.6448181867599487, + -0.45361432433128357, + 0.3905653953552246, + 0.7770582437515259, + -0.19482973217964172, + -2.094736337661743, + -1.083003282546997 + ], + [ + 0.9472348093986511, + 0.6904511451721191, + -0.2936761677265167, + -0.34556156396865845, + -0.7593099474906921, + -1.9670193195343018, + -0.7599589824676514, + 0.04467743635177612, + 0.6557444334030151, + 0.2765021026134491, + 0.042136549949645996, + -0.37541040778160095, + -3.161308765411377, + -0.8803613185882568, + 0.38967010378837585, + 0.4423010051250458, + 0.019825218245387077, + 0.9605399370193481, + -0.874176561832428, + -3.0791730880737305, + -1.3198856115341187 + ], + [ + 0.8605663180351257, + 0.04459291696548462, + -0.8886744976043701, + -0.38333380222320557, + -1.0364235639572144, + -3.0104973316192627, + -0.889693021774292, + -0.08674328029155731, + 0.20159615576267242, + 0.1069621816277504, + 1.4727201461791992, + 0.09883639216423035, + -3.129849433898926, + -0.8773520588874817, + -0.2628254294395447, + -0.3415844738483429, + -0.35313117504119873, + 0.3155616521835327, + -0.556646466255188, + -2.0992307662963867, + -0.7895610928535461 + ], + [ + 0.39962780475616455, + 0.3406272530555725, + 0.05363208055496216, + -0.06018832325935364, + -0.5703235268592834, + -2.5974044799804688, + -0.6783060431480408, + 0.7815009355545044, + 0.35641810297966003, + -0.00838596560060978, + 0.6228271126747131, + 0.25608590245246887, + -2.5914058685302734, + -0.8502807021141052, + 0.726007878780365, + -0.6774719953536987, + 0.6261520981788635, + 1.149846076965332, + 0.6782730221748352, + -3.391972541809082, + -0.7993651032447815 + ], + [ + -0.29702335596084595, + 0.6166410446166992, + 0.6319320201873779, + -0.6542804837226868, + -0.8892693519592285, + -1.8599720001220703, + -1.3999924659729004, + -0.3017314672470093, + 0.9357973337173462, + -0.5596981048583984, + 1.3143360614776611, + -0.30643409490585327, + -2.028008460998535, + -1.3395459651947021, + 0.6212120056152344, + 0.6739580631256104, + -0.5840877294540405, + -0.06664055585861206, + -0.23446696996688843, + -2.0225906372070312, + -2.438906669616699 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.0649428591132164, + 0.09363124519586563, + -0.014398600906133652, + -0.012044146656990051, + 0.03117428719997406, + -1.7716264724731445, + 0.0, + 0.05836518481373787, + 0.012665675021708012, + -0.013142907992005348, + -0.06252282857894897, + -0.06281104683876038, + -1.7224881649017334, + 0.0, + -0.06448972970247269, + -0.03696601092815399, + 0.030633779242634773, + -0.0031400322914123535, + 0.0327644944190979, + -1.4826252460479736, + 0.0 + ], + [ + -0.04005520045757294, + 0.2738151252269745, + -0.05346817523241043, + 0.20275527238845825, + 0.10101960599422455, + -1.7716264724731445, + 0.0, + -0.36066195368766785, + -0.21821273863315582, + 0.22022001445293427, + -0.0596311092376709, + -0.17951729893684387, + -1.7224881649017334, + 0.0, + 0.3205486238002777, + 0.25526633858680725, + 0.17392843961715698, + 0.5170043706893921, + 0.43821561336517334, + -1.4826252460479736, + 0.0 + ] + ], + "predicted_scores": [ + 4.094995975494385, + 4.043515205383301, + 3.7610859870910645, + 3.6654560565948486, + 3.6402814388275146, + 2.4322822093963623, + 1.8528903722763062, + 1.8338117599487305 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055567618", + "selected_index": 0, + "state_hash": "027cc529c99c00db47a01ecfc12ab432557fbd911e8ff3c8a56adf2d3404549f", + "stored_base_utility": 0.851725161075592, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19648407399654388, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19648407399654388 + }, + "base_progress": 0.19648407399654388, + "base_success": false, + "base_utility": 0.19648407399654388, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.8651304244995117, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8651304244995117 + }, + { + "contact_quality": null, + "progress": 0.17065170407295227, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17065170407295227 + }, + { + "contact_quality": null, + "progress": 0.21422582864761353, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21422582864761353 + }, + { + "contact_quality": null, + "progress": 0.213461235165596, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.213461235165596 + }, + { + "contact_quality": null, + "progress": 0.09443560242652893, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.09443560242652893 + }, + { + "contact_quality": null, + "progress": 0.18556243181228638, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18556243181228638 + }, + { + "contact_quality": null, + "progress": 0.18798279762268066, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18798279762268066 + }, + { + "contact_quality": null, + "progress": 0.19312432408332825, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19312432408332825 + } + ], + "candidate_progress": [ + 0.8651304244995117, + 0.17065170407295227, + 0.21422582864761353, + 0.213461235165596, + 0.09443560242652893, + 0.18556243181228638, + 0.18798279762268066, + 0.19312432408332825 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-9ae5c6c4132218ecf49e", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-4048f8f6e047422e3c79", + "generated_tangents": [ + [ + 0.030008889734745026, + 0.17263352870941162, + 0.06290218234062195, + -0.35769036412239075, + 0.20301605761051178, + -0.013823091983795166, + 0.05073705315589905, + -0.011901825666427612, + 0.0407995879650116, + -0.06939861923456192, + 0.30223992466926575, + -0.13431647419929504, + -2.0956919193267822, + -0.06421901285648346, + 0.017468735575675964, + -0.025348056107759476, + -0.003789648413658142, + 0.11182623356580734, + -0.14155647158622742, + -1.6103920936584473, + -0.09692034125328064 + ], + [ + 0.47444403171539307, + 0.17549239099025726, + 0.01300850510597229, + -0.4280071258544922, + 0.4995863437652588, + -0.09945118427276611, + 1.1715638637542725, + -0.0569804310798645, + 0.010068010538816452, + -0.11193756014108658, + 0.2464638650417328, + -0.3236076235771179, + -1.5485386848449707, + -0.3787398934364319, + 0.03685591369867325, + 0.21671059727668762, + 0.2907164692878723, + 0.11012721806764603, + -0.14242395758628845, + -1.1806422472000122, + -0.5797140598297119 + ], + [ + 0.03376307338476181, + 0.40081632137298584, + -0.3041814863681793, + -0.25909775495529175, + 0.23389090597629547, + -0.28211724758148193, + 0.3170156478881836, + -0.08350473642349243, + -0.13009127974510193, + -0.04976027458906174, + 0.36134809255599976, + -0.3108815550804138, + -2.242213487625122, + -0.3403996229171753, + -0.11681632697582245, + -0.0013635903596878052, + 0.20280736684799194, + -0.11909632384777069, + -0.2411510944366455, + -1.9621775150299072, + -0.14653849601745605 + ], + [ + 0.01651117205619812, + 0.2903990149497986, + -0.2700941264629364, + -0.4617370367050171, + 0.33837172389030457, + -0.20104432106018066, + 0.19775182008743286, + -0.041856229305267334, + -0.08482728898525238, + -0.0354887917637825, + 0.31226062774658203, + -0.34998375177383423, + -2.218757152557373, + -0.29290342330932617, + -0.07298333942890167, + 0.044787146151065826, + 0.24641327559947968, + -0.11413340270519257, + -0.4040852189064026, + -1.803842544555664, + -0.2802719473838806 + ], + [ + 0.14583849906921387, + 0.10028946399688721, + -0.008981138467788696, + -0.4491742253303528, + 0.49456340074539185, + 0.320042222738266, + 0.535348117351532, + 0.08054546266794205, + -0.02265104465186596, + 0.18600349128246307, + 0.15243756771087646, + -0.22849059104919434, + -2.1810052394866943, + -0.567722737789154, + -0.006112057715654373, + 0.20684833824634552, + 0.33710867166519165, + 0.06878407299518585, + -0.5710155963897705, + -1.7927601337432861, + -0.26154419779777527 + ], + [ + 0.011184021830558777, + 0.29185396432876587, + -0.15513935685157776, + -0.44751620292663574, + 0.369235098361969, + 0.11543236672878265, + 0.1479061245918274, + -0.06622764468193054, + -0.097317636013031, + -0.04419920593500137, + 0.3350147306919098, + -0.38822662830352783, + -2.342780351638794, + -0.2594625949859619, + 0.002361048012971878, + 0.05833374336361885, + 0.3517112731933594, + -0.1215195506811142, + -0.3438400626182556, + -1.5425746440887451, + -0.2135961651802063 + ], + [ + 0.15981324017047882, + 0.2047366350889206, + -0.013066917657852173, + -0.49522680044174194, + 0.3404083251953125, + -0.04098178446292877, + -0.11504268646240234, + -0.02589324116706848, + -0.09764020144939423, + -0.028640396893024445, + 0.23497521877288818, + -0.16871348023414612, + -2.2523000240325928, + -0.08155752718448639, + -0.06333911418914795, + 0.08381980657577515, + 0.10687589645385742, + 0.0644548237323761, + -0.31258511543273926, + -1.6810500621795654, + -0.011160194873809814 + ], + [ + 0.09823539853096008, + 0.2795335054397583, + -0.46369877457618713, + -0.6231623888015747, + 0.29014328122138977, + -0.18661874532699585, + 0.34255513548851013, + 0.09270450472831726, + -0.11617457866668701, + 0.031556062400341034, + 0.24025917053222656, + -0.5258431434631348, + -2.278263807296753, + -0.4086526036262512, + -0.08707267045974731, + 0.10760253667831421, + 0.3566898703575134, + -0.00210687518119812, + -0.5319662094116211, + -1.8286168575286865, + -0.1939733922481537 + ] + ], + "generated_utilities": [ + 0.8651304244995117, + 0.17065170407295227, + 0.21422582864761353, + 0.213461235165596, + 0.09443560242652893, + 0.18556243181228638, + 0.18798279762268066, + 0.19312432408332825 + ], + "group_id": "ms-4048f8f6e047422e3c79", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558006286621094, + 2.0, + 0.1986379623413086, + 0.03769078850746155, + 0.5361635684967041, + 0.18274834752082825, + 0.17300468683242798, + 0.12688571214675903, + 0.13836368918418884, + 0.07454231381416321, + 0.0899321436882019, + 0.10402637720108032, + 0.1711854636669159, + 0.17255032062530518, + 0.039613887667655945, + 0.10826331377029419 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1422826200723648, + 0.21033799648284912, + 0.1978604793548584, + 0.059162259101867676, + -0.2529524564743042, + -0.15161612629890442, + 0.0, + -0.2264404594898224, + -0.2209387719631195, + -0.025619791820645332, + 0.06467145681381226, + 0.07422274351119995, + -1.7048513889312744, + 0.0, + 0.15056702494621277, + -0.06247536092996597, + -0.1033044084906578, + 0.1334916353225708, + 0.2784581184387207, + -1.4811434745788574, + 0.0 + ], + [ + -0.44999396800994873, + 0.271686851978302, + -0.4267011880874634, + 0.2730966806411743, + -0.01798880100250244, + -0.04680868983268738, + 0.0, + -0.09472770243883133, + -0.06993720680475235, + -0.06752754002809525, + -0.13884133100509644, + 0.24737784266471863, + -1.7048513889312744, + 0.0, + -0.09056920558214188, + 0.4132460653781891, + 0.04097161442041397, + -0.25072288513183594, + 0.49742037057876587, + -1.4811434745788574, + 0.0 + ], + [ + 0.17392832040786743, + 1.2103379964828491, + -0.3708428144454956, + -0.36093437671661377, + 0.744896650314331, + 0.23929360508918762, + 1.4388959407806396, + -0.04149632900953293, + 0.015302501618862152, + 0.023577755317091942, + 0.5954978466033936, + -0.3274606168270111, + -2.7048513889312744, + -1.646976351737976, + 0.0006552413105964661, + -0.008917607367038727, + 0.022643378004431725, + 0.7492771148681641, + -0.26030659675598145, + -2.4811434745788574, + -1.684959888458252 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8777918815612793, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.293952703475952, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.369919776916504 + ], + [ + 0.1792769730091095, + 0.9774190187454224, + -0.5587559342384338, + -0.5885956883430481, + 0.9904338717460632, + 0.41647347807884216, + 1.4140102863311768, + -0.2462461292743683, + -0.1271664798259735, + -0.2736872732639313, + 0.8877567052841187, + -0.6213628053665161, + -2.937077283859253, + -1.3732452392578125, + -0.0015218837652355433, + -0.27707284688949585, + -0.16698139905929565, + 0.4978373646736145, + -0.479316771030426, + -2.7234396934509277, + -1.6759265661239624 + ], + [ + 0.1658722460269928, + 1.3070863485336304, + -0.4893108606338501, + -0.04209986329078674, + 0.7449368238449097, + 0.3420809805393219, + 1.4472918510437012, + 0.08132945746183395, + 0.13181470334529877, + 0.22494959831237793, + 0.22593030333518982, + -0.49233078956604004, + -2.7544617652893066, + -1.7597702741622925, + -0.365922212600708, + -0.17204320430755615, + -0.1821347028017044, + 0.6334611177444458, + -0.33006516098976135, + -2.2188992500305176, + -1.5452594757080078 + ], + [ + 0.18608316779136658, + 1.23563551902771, + -0.5727517604827881, + 0.10049733519554138, + 0.9073957204818726, + 0.5650919675827026, + 1.8946568965911865, + -0.17719107866287231, + 0.34025683999061584, + 0.44525983929634094, + 0.6176816821098328, + -0.2832425534725189, + -3.1802117824554443, + -1.8703548908233643, + 0.07433810085058212, + 0.1340292990207672, + 0.497944176197052, + 1.0071699619293213, + -0.44033437967300415, + -2.7992849349975586, + -1.8935935497283936 + ], + [ + 0.40270811319351196, + 1.5102664232254028, + -0.8070739507675171, + -0.16390679776668549, + 0.3215266466140747, + -0.16522419452667236, + 1.0900052785873413, + 0.07780961692333221, + 0.5627153515815735, + 0.5386761426925659, + 0.2949375510215759, + 0.059314221143722534, + -3.2417359352111816, + -1.899662971496582, + 0.4160631597042084, + 0.44336366653442383, + 0.47819244861602783, + 1.162381649017334, + -0.7831735014915466, + -2.625349521636963, + -1.2278814315795898 + ], + [ + 0.8529313802719116, + 1.5106499195098877, + 0.19478559494018555, + -0.6338159441947937, + 0.7529481053352356, + -0.24222904443740845, + 0.9723314046859741, + -0.14078867435455322, + -0.5563283562660217, + 0.5440018177032471, + 0.004017293453216553, + 0.019366413354873657, + -3.1521430015563965, + -2.0841636657714844, + -0.2760852575302124, + -0.5531485676765442, + 0.22684869170188904, + 1.100138545036316, + 0.046634942293167114, + -2.429711103439331, + -2.280839443206787 + ], + [ + -0.4760962128639221, + 1.8579951524734497, + 0.2322864532470703, + 0.19953304529190063, + 0.48002323508262634, + 0.7484984397888184, + 2.073361396789551, + -0.04567806422710419, + 0.04004429653286934, + -0.5635936260223389, + 0.5135918259620667, + 0.3908633887767792, + -1.9335949420928955, + -2.442507743835449, + -0.6620182394981384, + 0.4175874888896942, + 0.7711085677146912, + 1.2790143489837646, + -0.5464571714401245, + -3.2030673027038574, + -2.3768434524536133 + ], + [ + 1.0231709480285645, + 1.1804075241088867, + 0.11513572931289673, + -1.23623526096344, + 1.4848686456680298, + 1.1120561361312866, + 0.9504602551460266, + 0.8234015703201294, + -0.04407603666186333, + -0.37310001254081726, + 0.24512693285942078, + 0.4242170751094818, + -2.4985997676849365, + -2.1206297874450684, + 0.5509956479072571, + 0.8686066269874573, + -0.5043656229972839, + 1.3957195281982422, + -0.07662208378314972, + -2.371795654296875, + -2.2404346466064453 + ], + [ + 0.49735015630722046, + 0.3600291609764099, + 0.3103446364402771, + 0.6051294803619385, + -0.15457379817962646, + -0.5570405721664429, + 1.2383109331130981, + 0.9059417247772217, + 0.7879754900932312, + -0.31492647528648376, + 1.5871094465255737, + -0.8906484842300415, + -1.8698136806488037, + -2.401073694229126, + 0.3560212254524231, + -0.6580788493156433, + 0.9037240147590637, + 1.614617109298706, + 0.3788090944290161, + -1.6909503936767578, + -0.8765208125114441 + ], + [ + 0.10883396118879318, + 1.8993752002716064, + -0.7673929333686829, + -0.0936923623085022, + -0.15508884191513062, + -0.1517968773841858, + 0.465425968170166, + 0.43258219957351685, + 0.46641960740089417, + -0.3528187572956085, + 1.09894597530365, + -0.6569450497627258, + -2.025486946105957, + -1.2820806503295898, + -0.21141839027404785, + -0.9515489339828491, + 0.7230286002159119, + -0.21062928438186646, + -0.32822558283805847, + -3.289642333984375, + -1.7125959396362305 + ], + [ + 0.37358880043029785, + 2.052347183227539, + -1.212618112564087, + -0.6321570873260498, + 0.7031511068344116, + -0.3477393090724945, + 1.2687753438949585, + 0.8023800253868103, + -0.031013749539852142, + 0.35525181889533997, + 0.5262036323547363, + -0.9288194179534912, + -1.7686026096343994, + -2.128396987915039, + -0.7001423239707947, + -0.12872236967086792, + -0.012955078855156898, + 1.0237239599227905, + 0.03636348247528076, + -2.0397119522094727, + -0.9761244058609009 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.060203760862350464, + 0.21033799648284912, + 0.0356404185295105, + -0.03044089674949646, + 0.024639129638671875, + -0.032671064138412476, + 0.0, + 0.016544528305530548, + 0.04633430764079094, + 0.005854343995451927, + -0.014876246452331543, + 0.0028293728828430176, + -1.7048513889312744, + 0.0, + -0.05953434109687805, + -0.04181046783924103, + -0.0018185675144195557, + -0.09216523170471191, + 0.019338399171829224, + -1.4811434745788574, + 0.0 + ] + ], + "predicted_scores": [ + 7.490676403045654, + 7.170323371887207, + 6.697402000427246, + 6.384732723236084, + 6.295480251312256, + 6.164147853851318, + 5.90757942199707, + 5.689966678619385 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1055157354", + "selected_index": 0, + "state_hash": "ae21d2b71722381a530d3a9259d02ea6728eb8a4a84fafc40ccd653d62ad3968", + "stored_base_utility": 0.8558006286621094, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.19844311475753784, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19844311475753784 + }, + "base_progress": 0.19844311475753784, + "base_success": false, + "base_utility": 0.19844311475753784, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.17567642033100128, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17567642033100128 + }, + { + "contact_quality": null, + "progress": 0.1609627604484558, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1609627604484558 + }, + { + "contact_quality": null, + "progress": 0.23198044300079346, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23198044300079346 + }, + { + "contact_quality": null, + "progress": 0.18044036626815796, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18044036626815796 + }, + { + "contact_quality": null, + "progress": 0.21981436014175415, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21981436014175415 + }, + { + "contact_quality": null, + "progress": 0.15481720864772797, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15481720864772797 + }, + { + "contact_quality": null, + "progress": 0.2075604647397995, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2075604647397995 + }, + { + "contact_quality": null, + "progress": 0.22741466760635376, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22741466760635376 + } + ], + "candidate_progress": [ + 0.17567642033100128, + 0.1609627604484558, + 0.23198044300079346, + 0.18044036626815796, + 0.21981436014175415, + 0.15481720864772797, + 0.2075604647397995, + 0.22741466760635376 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-ecb37ebbab21a506d8ca", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-5b26c33feed6992386b1", + "generated_tangents": [ + [ + -0.003956928849220276, + -0.06990145146846771, + -0.16480055451393127, + 0.30755630135536194, + -0.012048155069351196, + -1.086521029472351, + -0.14643651247024536, + 0.07404845952987671, + 0.050366371870040894, + 0.03993061184883118, + 0.13770818710327148, + 0.24159541726112366, + -1.9209805727005005, + -0.0966176986694336, + 0.08004353195428848, + 0.06546422839164734, + 0.08362357318401337, + -0.14493761956691742, + 0.2019823044538498, + -1.9210615158081055, + -0.3066258430480957 + ], + [ + -0.0462736040353775, + -0.05885855853557587, + -0.09201386570930481, + 0.23123934864997864, + -0.021240919828414917, + -0.785561203956604, + -0.19830679893493652, + 0.1271599531173706, + 0.04168267548084259, + 0.030569881200790405, + 0.15824760496616364, + 0.2760062515735626, + -2.0482239723205566, + -0.05982495844364166, + 0.1003795936703682, + 0.06336382031440735, + 0.11962166428565979, + -0.16470323503017426, + 0.1422899067401886, + -1.9973554611206055, + -0.2114710509777069 + ], + [ + -0.19363166391849518, + -0.14713603258132935, + -0.39566370844841003, + 0.4984795153141022, + 0.11760872602462769, + -1.262765645980835, + 0.09189155697822571, + 0.22310979664325714, + -0.0947994515299797, + -0.11231277137994766, + -0.017672404646873474, + 0.22374528646469116, + -2.1181323528289795, + 0.038686588406562805, + 0.005642283707857132, + 0.044049616903066635, + 0.29121851921081543, + -0.08374150097370148, + -0.028894901275634766, + -1.9262609481811523, + -0.36327096819877625 + ], + [ + 0.010901883244514465, + 0.0019717514514923096, + -0.20607814192771912, + 0.25729432702064514, + 0.029973581433296204, + -0.811781644821167, + -0.17005163431167603, + 0.1184350997209549, + -0.028967997059226036, + 0.03754618763923645, + 0.07278949022293091, + 0.33260172605514526, + -2.0846428871154785, + -0.07179608941078186, + 0.113896444439888, + 0.03850396350026131, + 0.11058761924505234, + -0.13422997295856476, + 0.1795283406972885, + -2.0686228275299072, + -0.23739710450172424 + ], + [ + -0.2527129650115967, + -0.0534171387553215, + -0.11055079102516174, + 0.287568598985672, + 0.007939234375953674, + -0.7499806880950928, + -0.17638403177261353, + 0.23254576325416565, + -0.10218941420316696, + -0.1576124131679535, + -0.07019532471895218, + 0.21945291757583618, + -2.081728219985962, + -0.04562319815158844, + 0.21058905124664307, + -0.01752379722893238, + 0.1355130672454834, + -0.0845472663640976, + 0.021035194396972656, + -2.333874464035034, + -0.1841607689857483 + ], + [ + -0.14197538793087006, + -0.10166645050048828, + -0.13611909747123718, + 0.3405918776988983, + -0.017250031232833862, + -0.8338378667831421, + -0.10513591766357422, + 0.09952886402606964, + 0.046714864671230316, + 0.09271503984928131, + 0.11503598093986511, + 0.416715145111084, + -1.8861693143844604, + -0.15263445675373077, + 0.1018257886171341, + 0.10959236323833466, + 0.09383639693260193, + -0.18258638679981232, + 0.10749277472496033, + -1.9266455173492432, + -0.3445923626422882 + ], + [ + -0.27726882696151733, + -0.06937313079833984, + -0.08735474944114685, + 0.3757733404636383, + 0.11187510937452316, + -0.7334935665130615, + -0.10866284370422363, + 0.2559641897678375, + -0.10003504902124405, + -0.07836895436048508, + 0.10444165766239166, + 0.14157049357891083, + -2.126008987426758, + 0.013141870498657227, + 0.04199111461639404, + -0.004544961266219616, + 0.1588364690542221, + -0.10955263674259186, + 0.033526286482810974, + -2.068049430847168, + -0.2482026219367981 + ], + [ + -0.2805596590042114, + -0.08894135057926178, + -0.2837352454662323, + 0.3582393229007721, + 0.25944972038269043, + -0.7279287576675415, + -0.1078721284866333, + 0.16095273196697235, + -0.11950705945491791, + -0.044760726392269135, + -0.0387117937207222, + 0.05619433522224426, + -2.170316696166992, + -0.10741914808750153, + -0.06863324344158173, + 0.025339357554912567, + 0.005009979009628296, + -0.028363361954689026, + 0.016270101070404053, + -2.294799566268921, + -0.22589710354804993 + ] + ], + "generated_utilities": [ + 0.17567642033100128, + 0.1609627604484558, + 0.23198044300079346, + 0.18044036626815796, + 0.21981436014175415, + 0.15481720864772797, + 0.2075604647397995, + 0.22741466760635376 + ], + "group_id": "ms-5b26c33feed6992386b1", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8326650857925415, + 2.0, + 0.15131878852844238, + 0.15007886290550232, + 0.5293327569961548, + 0.06490398943424225, + 0.5425760746002197, + 0.20163774490356445, + 0.15259261429309845, + 0.23924465477466583, + 0.12766429781913757, + 0.08033241331577301, + 0.07788355648517609, + 0.13886235654354095, + 0.09716498851776123, + 0.11682072281837463 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.06662726402282715, + -0.14569896459579468, + 0.04974943399429321, + 0.539892315864563, + -0.19018621742725372, + -0.5884796380996704, + 0.0, + 0.12748029828071594, + 0.11038697510957718, + -0.03454907238483429, + 0.08700031787157059, + -0.13105249404907227, + -1.9082190990447998, + 0.0, + -0.0033895689994096756, + -0.15730009973049164, + 0.15846982598304749, + 0.11786291003227234, + -0.02387581765651703, + -1.5966379642486572, + 0.0 + ], + [ + -0.06662726402282715, + 0.059319257736206055, + -0.19934123754501343, + 0.539892315864563, + 0.07902508974075317, + -0.5884796380996704, + 0.0, + -0.3428957760334015, + 0.08558038622140884, + 0.28825634717941284, + -0.32335156202316284, + -0.3631777763366699, + -1.9082190990447998, + 0.0, + -0.3765835762023926, + -0.5554376840591431, + 0.3468284010887146, + 0.875933825969696, + 0.11397302150726318, + -1.5966379642486572, + 0.0 + ], + [ + -1.0666272640228271, + -0.6187554597854614, + -0.8818012475967407, + 1.539892315864563, + -0.428971529006958, + -1.5884796380996704, + 2.0647666454315186, + -0.03600159287452698, + 0.032059766352176666, + 0.030456876382231712, + 0.14377687871456146, + 0.20562076568603516, + -2.9082190990448, + -1.332404375076294, + -0.014788046479225159, + -0.0006958844605833292, + 0.023885300382971764, + 0.5519534349441528, + 0.1840958297252655, + -2.5966379642486572, + -1.4879226684570312 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.129533290863037, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.664808750152588, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9758453369140625 + ], + [ + -1.2971861362457275, + -0.7044790983200073, + -1.134492039680481, + 1.5821952819824219, + -0.7159937620162964, + -1.7615629434585571, + 2.031708002090454, + -0.25998222827911377, + 0.10180196911096573, + -0.24491730332374573, + 0.2074550986289978, + 0.2793378233909607, + -2.752415657043457, + -1.3360638618469238, + 0.03524862602353096, + -0.1854405552148819, + -0.1696440726518631, + 0.7172544598579407, + 0.0028989464044570923, + -2.680950164794922, + -1.2071388959884644 + ], + [ + -1.465846300125122, + -0.34887754917144775, + -1.046374797821045, + 1.703241229057312, + -0.19744430482387543, + -1.2604204416275024, + 1.9822474718093872, + 0.18854136765003204, + -0.146264910697937, + 0.059789031744003296, + 0.39534950256347656, + 0.29636046290397644, + -2.6612584590911865, + -1.5028973817825317, + 0.059081435203552246, + -0.11711112409830093, + -0.07238245755434036, + 0.25233787298202515, + 0.12625277042388916, + -2.2362401485443115, + -1.7053974866867065 + ], + [ + -1.1892943382263184, + -0.20831987261772156, + -0.8013461828231812, + 1.7197901010513306, + -0.39134201407432556, + -1.5160430669784546, + 2.4594008922576904, + -0.1669330894947052, + -0.337530255317688, + 0.15848031640052795, + 0.05738166719675064, + 0.41757503151893616, + -3.1393940448760986, + -0.9257140755653381, + -0.5068305134773254, + 0.1864723265171051, + 0.45369866490364075, + 0.6036283373832703, + -0.018199875950813293, + -2.8028440475463867, + -1.3847744464874268 + ], + [ + -1.1111165285110474, + -1.191490888595581, + -1.361661434173584, + 1.197826623916626, + -0.29219919443130493, + -1.7355475425720215, + 2.0579750537872314, + -0.08019472658634186, + 0.5140348672866821, + -0.5474883317947388, + -0.03311571478843689, + -0.060686588287353516, + -2.3427765369415283, + -1.8552747964859009, + -0.36303937435150146, + 0.01966947503387928, + -0.5651369094848633, + 0.22617560625076294, + -0.16322728991508484, + -2.421419143676758, + -0.921785831451416 + ], + [ + -1.012363076210022, + -0.6483663320541382, + -0.7951939702033997, + 1.2563079595565796, + -0.5323562622070312, + -1.5913673639297485, + 1.406038761138916, + -0.39036238193511963, + -0.12943506240844727, + -0.4373972415924072, + -0.20462383329868317, + 0.12806488573551178, + -3.570098400115967, + -1.8897531032562256, + 0.6369156837463379, + 0.2483828067779541, + 0.6809477210044861, + 0.8138958215713501, + 0.6604747176170349, + -2.717491865158081, + -1.9953765869140625 + ], + [ + -0.378162145614624, + -0.23438718914985657, + -1.2589956521987915, + 1.844052791595459, + -0.8936926126480103, + -1.230797290802002, + 2.5580525398254395, + -0.21314577758312225, + -0.05935201793909073, + 0.46811148524284363, + 0.38594305515289307, + 1.0049593448638916, + -2.491497278213501, + -1.3096134662628174, + 0.15605805814266205, + 0.7682130932807922, + -0.4010901153087616, + 0.8638691902160645, + 0.5755372047424316, + -3.324826240539551, + -1.6860322952270508 + ], + [ + -0.6920164227485657, + -0.5060986280441284, + -0.5146759748458862, + 2.3206489086151123, + 0.26810598373413086, + -1.7696658372879028, + 1.4922571182250977, + -0.6155052185058594, + -0.5581206679344177, + 0.4068271219730377, + 0.5812678337097168, + -0.2056950330734253, + -2.227752923965454, + -0.7079744935035706, + 0.3153572678565979, + 0.019053615629673004, + -0.10691303759813309, + -0.18750637769699097, + 0.5490041971206665, + -2.7611911296844482, + -1.0861625671386719 + ], + [ + -0.9623568654060364, + -0.821997880935669, + -0.0017323493957519531, + 1.8667385578155518, + 0.39793866872787476, + -1.3673040866851807, + 2.8636348247528076, + -0.739641547203064, + -0.4186078906059265, + -0.21357670426368713, + 0.6820193529129028, + 0.09538085013628006, + -3.242215633392334, + -1.023385763168335, + -0.2367737591266632, + 0.9248462915420532, + 0.04536557197570801, + 0.2294020652770996, + 0.15174128115177155, + -2.509904146194458, + -0.5901334881782532 + ], + [ + -0.565070390701294, + 0.37745314836502075, + -1.7464253902435303, + 1.4869437217712402, + -1.187619924545288, + -2.3802757263183594, + 2.5987753868103027, + 0.5777139663696289, + -0.04772312939167023, + -0.49332448840141296, + -0.515097975730896, + 0.751485288143158, + -2.7030012607574463, + -1.5739474296569824, + 0.8266244530677795, + -0.9454089403152466, + 0.7678747177124023, + 1.0557427406311035, + 0.0584852397441864, + -2.8590199947357178, + -0.8677505850791931 + ], + [ + -0.2763708233833313, + 0.28772038221359253, + -0.7008944153785706, + 1.3402774333953857, + -0.78607177734375, + -2.1210508346557617, + 1.4908761978149414, + -0.4155166447162628, + 0.40930652618408203, + -0.1769891083240509, + -0.20092366635799408, + 0.5321645736694336, + -3.2144649028778076, + -1.2440845966339111, + -0.247700035572052, + 0.7174559235572815, + 0.11071103811264038, + -0.023232340812683105, + 0.6976174116134644, + -2.9463140964508057, + -0.5927516222000122 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.06662726402282715, + -0.05619156360626221, + -0.02257448434829712, + 0.539892315864563, + 0.053197890520095825, + -0.5884796380996704, + 0.0, + -0.04219311475753784, + 0.010949447751045227, + 0.027476036921143532, + -0.02041013538837433, + -0.01791653037071228, + -1.9082190990447998, + 0.0, + 0.01256529800593853, + 0.014024228788912296, + 0.03545428439974785, + 0.019800782203674316, + -0.022738635540008545, + -1.5966379642486572, + 0.0 + ] + ], + "predicted_scores": [ + 6.3688836097717285, + 6.3467512130737305, + 6.275082111358643, + 6.2109055519104, + 6.087492942810059, + 6.081385612487793, + 5.703493118286133, + 5.19040060043335 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1522272686", + "selected_index": 0, + "state_hash": "10034ca87ce4b5c491c0aa23d2a30389ba8265fa695d65198bf719c54565cf2a", + "stored_base_utility": 0.8326650857925415, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8388344049453735, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8388344049453735 + }, + "base_progress": 0.8388344049453735, + "base_success": false, + "base_utility": 0.8388344049453735, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8181180953979492, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8181180953979492 + }, + { + "contact_quality": null, + "progress": 0.1479516476392746, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1479516476392746 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1400405466556549, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1400405466556549 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.8181180953979492, + 0.1479516476392746, + 1.0, + 1.0, + 1.0, + 0.1400405466556549, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-5a456ddf97d294548cf1", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-0b226017db7f07c5307e", + "ms-dc26a7037003b94b9f93", + "ms-5dfbb16e173e1a3e2bd6" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + false, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-63ac245579b1188935ad", + "generated_tangents": [ + [ + 0.0875767171382904, + 0.06667950004339218, + -0.11912426352500916, + 0.02909502387046814, + 0.04533493518829346, + -1.8368381261825562, + -0.1082448959350586, + -0.0030595362186431885, + -0.006928380578756332, + 0.10766997188329697, + 0.13685259222984314, + 0.1791902482509613, + -1.384962558746338, + -0.08337503671646118, + -0.057299669831991196, + 0.07471176981925964, + 0.12037551403045654, + 0.0058998167514801025, + -0.016657859086990356, + -1.1310760974884033, + -0.027680858969688416 + ], + [ + -0.03461998701095581, + 0.030188776552677155, + 0.02041885256767273, + 0.0791589617729187, + 0.04111059010028839, + -1.769657850265503, + -0.13236629962921143, + -0.0026993602514266968, + -0.012054651975631714, + 0.063559390604496, + 0.23516583442687988, + 0.23861247301101685, + -1.475093960762024, + -0.0302799791097641, + -0.031195703893899918, + 0.1027858704328537, + 0.029262766242027283, + -0.056897953152656555, + -0.06885898113250732, + -0.9965713620185852, + -0.056251779198646545 + ], + [ + 0.13037967681884766, + 0.044371433556079865, + -0.023115307092666626, + -0.055332034826278687, + 0.10363438725471497, + -1.8522902727127075, + -0.11580264568328857, + 0.09957293421030045, + -0.029794400557875633, + 0.06683573871850967, + 0.16266819834709167, + 0.13251268863677979, + -1.3987717628479004, + -0.11576487123966217, + -0.0020797140896320343, + 0.10281313955783844, + 0.16798318922519684, + -0.029749050736427307, + -0.02250218391418457, + -1.0928770303726196, + -0.019477322697639465 + ], + [ + 0.1336415708065033, + 0.021478980779647827, + -0.0920698344707489, + 0.06146487593650818, + 0.058310478925704956, + -1.9416826963424683, + -0.16801291704177856, + 0.011562362313270569, + -0.0352933332324028, + 0.09769804030656815, + 0.19700577855110168, + 0.0953979641199112, + -1.4475812911987305, + -0.13870875537395477, + 0.011281915009021759, + 0.10560320317745209, + 0.08078952133655548, + 0.02909603714942932, + 0.04007375240325928, + -1.1974456310272217, + -0.09815460443496704 + ], + [ + 0.14251402020454407, + 0.07695221155881882, + -0.014767199754714966, + -0.052157893776893616, + 0.11734548956155777, + -1.8062281608581543, + -0.11383187770843506, + 0.15381109714508057, + -0.07094687223434448, + 0.09402982890605927, + 0.22719672322273254, + 0.20944231748580933, + -1.2605376243591309, + -0.07241888344287872, + -0.04573053494095802, + 0.08019408583641052, + 0.1890329122543335, + 0.13442978262901306, + 0.09435175359249115, + -1.0764597654342651, + -0.06758096814155579 + ], + [ + 0.03332400321960449, + -0.02650948241353035, + -0.027606159448623657, + 0.028628617525100708, + 0.1964508593082428, + -1.9185587167739868, + -0.10465216636657715, + 0.027949780225753784, + -0.03177535533905029, + 0.07444071769714355, + 0.19828975200653076, + 0.2741793990135193, + -1.389540433883667, + -0.1669108122587204, + 0.105293408036232, + 0.10851116478443146, + 0.14594432711601257, + -0.08073602616786957, + -0.14245057106018066, + -1.159231185913086, + -0.1667010486125946 + ], + [ + 0.14103801548480988, + 0.010021902620792389, + -0.05186501145362854, + 0.01830558478832245, + 0.1603812575340271, + -1.7513757944107056, + -0.13741683959960938, + 0.09187138825654984, + -0.001362161710858345, + 0.08333832025527954, + 0.2154909074306488, + 0.19489426910877228, + -1.264728307723999, + -0.15727905929088593, + 0.007760878652334213, + 0.13821762800216675, + 0.19939735531806946, + -0.008028343319892883, + 0.012169510126113892, + -0.9980083703994751, + -0.16007345914840698 + ], + [ + -0.2055303007364273, + -0.07026727497577667, + 0.19995340704917908, + -0.1151643916964531, + 0.23799385130405426, + -1.6392040252685547, + -0.04348176717758179, + 0.1457993984222412, + -0.12437643110752106, + 0.05682114511728287, + 0.28148573637008667, + 0.23318122327327728, + -1.691398024559021, + 0.042029380798339844, + -0.0035445429384708405, + 0.08843211829662323, + 0.2996772527694702, + -0.03317791223526001, + -0.17194655537605286, + -1.0370934009552002, + 0.09168896079063416 + ] + ], + "generated_utilities": [ + 2.0, + 0.8181180953979492, + 0.1479516476392746, + 2.0, + 2.0, + 2.0, + 0.1400405466556549, + 2.0 + ], + "group_id": "ms-63ac245579b1188935ad", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.855198860168457, + 0.8138954639434814, + 0.20312510430812836, + 2.0, + 0.7862945795059204, + 0.7843606472015381, + 0.6004383563995361, + 0.15506848692893982, + 0.15924105048179626, + 0.132452130317688, + 0.05381816625595093, + 0.09561765193939209, + 0.07140937447547913, + 0.10806754231452942, + 0.023168057203292847, + 0.07957816123962402 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.007633227854967117, + 0.02389371022582054, + 0.2181050330400467, + 0.026324495673179626, + 0.1209479570388794, + -1.3937149047851562, + 0.0, + -0.2493116855621338, + -0.17256586253643036, + -0.06812826544046402, + 0.019445210695266724, + -0.2657848000526428, + -1.0657541751861572, + 0.0, + -0.04306507110595703, + -0.07170329242944717, + 0.035314902663230896, + 0.06211268901824951, + 0.18158195912837982, + -0.7849693298339844, + 0.0 + ], + [ + -0.011555496603250504, + 0.0016381663735955954, + 0.019193550571799278, + 0.26171422004699707, + 0.08122420310974121, + -2.3937149047851562, + -1.53977632522583, + -0.00580335408449173, + -0.0020052569452673197, + 0.01787502132356167, + 0.4615197777748108, + 0.1938573122024536, + -2.0657541751861572, + -1.4660639762878418, + -0.00267898291349411, + -0.0038005562964826822, + 0.0179074015468359, + 0.5644202828407288, + 0.2690354883670807, + -1.7849693298339844, + -1.3308851718902588 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.07955265045166, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9321279525756836, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.6617703437805176 + ], + [ + -0.16036809980869293, + 0.11436319351196289, + -0.14857570827007294, + 0.09414434432983398, + 0.02274160459637642, + -2.426297903060913, + -1.7686878442764282, + -0.26862773299217224, + 0.16241224110126495, + 0.24208402633666992, + 0.4166663587093353, + 0.07841069251298904, + -2.2318098545074463, + -1.2228643894195557, + -0.08888694643974304, + 0.20051127672195435, + -0.14729085564613342, + 0.5799955129623413, + 0.33039355278015137, + -2.00492262840271, + -1.4948352575302124 + ], + [ + -0.3820815682411194, + -0.3679526746273041, + 0.18845602869987488, + 0.4168931245803833, + -0.007936321198940277, + -2.700572967529297, + -1.795093059539795, + 0.1203194335103035, + 0.19610334932804108, + 0.2749679982662201, + 0.8331882357597351, + -0.06589585542678833, + -2.333136558532715, + -1.1008005142211914, + -0.02170141413807869, + 0.24186654388904572, + -0.3734443485736847, + 0.5144574046134949, + 0.3134346902370453, + -1.5088386535644531, + -1.1978046894073486 + ], + [ + 0.13625949621200562, + 0.47836095094680786, + -0.05790846049785614, + -0.09614765644073486, + -0.14281350374221802, + -2.522264003753662, + -1.0473889112472534, + 0.4792231023311615, + -0.4869813621044159, + 0.17644789814949036, + 0.8425257205963135, + -0.28899794816970825, + -2.5508456230163574, + -0.9762126207351685, + -0.16381770372390747, + 0.1586131900548935, + -0.3460317850112915, + 0.6056831479072571, + 0.7327044010162354, + -1.494289755821228, + -1.4671605825424194 + ], + [ + -0.40018847584724426, + -0.028111951425671577, + 0.31324630975723267, + -0.22690656781196594, + 0.2902344763278961, + -2.9841673374176025, + -1.4654184579849243, + -0.1155615746974945, + -0.4125809073448181, + 0.6098153591156006, + 0.8544195890426636, + 0.22702345252037048, + -1.7853796482086182, + -1.3474400043487549, + -0.454882949590683, + -0.3122820556163788, + 0.22399993240833282, + 0.05299121141433716, + 0.07802119851112366, + -1.3896000385284424, + -1.894580602645874 + ], + [ + -0.6587541699409485, + -0.6905457973480225, + 0.10385943204164505, + -0.2785833477973938, + 0.2321375161409378, + -2.565701723098755, + -1.1605603694915771, + -0.6775721907615662, + -0.30887091159820557, + 0.1186944767832756, + 1.0924456119537354, + 0.5009081363677979, + -1.8667830228805542, + -1.2491121292114258, + -0.11290804296731949, + -0.03346216678619385, + -0.28778207302093506, + 1.2255009412765503, + 0.6856998801231384, + -1.1723419427871704, + -1.1220875978469849 + ], + [ + -0.5019480586051941, + 0.07311037182807922, + 0.5101989507675171, + 0.7159117460250854, + 0.005071915686130524, + -2.6556949615478516, + -2.321233034133911, + 0.6035679578781128, + -0.13785676658153534, + -0.4309522807598114, + 0.9664275050163269, + -0.020235419273376465, + -1.6894025802612305, + -0.7511875033378601, + 0.5713916420936584, + 0.24962951242923737, + 0.40067923069000244, + 0.12209588289260864, + 0.03222101926803589, + -2.27744460105896, + -1.9113109111785889 + ], + [ + 0.7552750110626221, + -0.8085685968399048, + 0.20896124839782715, + -0.6043995022773743, + 0.733860969543457, + -1.610467791557312, + -1.0368051528930664, + -0.02292630262672901, + 0.6320546269416809, + 0.14528857171535492, + 0.8372507095336914, + 0.41523146629333496, + -2.339751958847046, + -0.6185142993927002, + -0.5750091075897217, + 0.12324154376983643, + 0.2804809510707855, + 1.0503896474838257, + -0.15392154455184937, + -0.9555137753486633, + -1.517809510231018 + ], + [ + 0.8853163123130798, + 0.09475164860486984, + -0.13681088387966156, + 0.08429434895515442, + 0.13555681705474854, + -1.5859344005584717, + -2.1957859992980957, + 0.9337366819381714, + -1.0016963481903076, + 0.11365362256765366, + -0.31175100803375244, + 0.7412776350975037, + -1.0797133445739746, + -0.8527018427848816, + 0.6110302209854126, + 0.006565961055457592, + 0.38363590836524963, + 0.5695288181304932, + -0.2547180950641632, + -2.6268043518066406, + -1.748514175415039 + ], + [ + 0.513801097869873, + 0.7078304886817932, + 0.7309613823890686, + -0.32746708393096924, + -0.36263394355773926, + -3.3305888175964355, + -1.2108653783798218, + 0.5104080438613892, + -0.26447102427482605, + 0.6361017823219299, + 0.5986918210983276, + -0.48598653078079224, + -2.4424209594726562, + -1.4694169759750366, + -0.5649402737617493, + -0.22020848095417023, + 0.6405193209648132, + -0.0698668360710144, + 0.7155715227127075, + -2.352860450744629, + -0.7502654194831848 + ], + [ + -0.9911022782325745, + 0.9677192568778992, + 0.5195661187171936, + 0.6733791828155518, + 0.7589308023452759, + -3.3654959201812744, + -0.6360958814620972, + 0.5926312208175659, + -0.06773044914007187, + 0.3785405457019806, + 0.5934506058692932, + 0.6434512138366699, + -2.891094923019409, + -0.8924227356910706, + 0.8036434650421143, + 0.6109268665313721, + -0.6607317924499512, + 1.0173940658569336, + 0.2546094059944153, + -1.4653143882751465, + -2.0709643363952637 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.16686837375164032, + 0.1354454904794693, + -0.11150697618722916, + 0.7247087359428406, + 0.2886054813861847, + -1.3937149047851562, + 0.0, + 0.6050093770027161, + 0.20291854441165924, + 0.5827622413635254, + -0.2687305808067322, + -0.25171223282814026, + -1.0657541751861572, + 0.0, + 0.29065752029418945, + -0.023413484916090965, + -0.24587245285511017, + 0.4600236713886261, + -0.4278547465801239, + -0.7849693298339844, + 0.0 + ] + ], + "predicted_scores": [ + 5.630728244781494, + 5.463387966156006, + 5.200445652008057, + 5.182462215423584, + 5.181726455688477, + 5.047179698944092, + 4.899623870849609, + 4.412124156951904 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "211245842", + "selected_index": 0, + "state_hash": "fb52e84b6addb8bdddf02a80d5c8567e0fa27195729e8440271b710441061b7f", + "stored_base_utility": 0.855198860168457, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.13187798857688904, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13187798857688904 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.13499042391777039, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13499042391777039 + }, + { + "contact_quality": null, + "progress": 0.16052719950675964, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16052719950675964 + } + ], + "candidate_progress": [ + 0.13187798857688904, + 1.0, + 1.0, + 1.0, + 1.0, + 1.0, + 0.13499042391777039, + 0.16052719950675964 + ], + "candidate_source_chart_ids": [ + "ms-875a4668671c02054a3c", + "ms-20f3cfcf183c69853b1e", + "ms-875a4668671c02054a3c", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + true, + true, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "generated_tangents": [ + [ + 0.05264825373888016, + 0.03665849566459656, + -0.04196348786354065, + -0.12311732769012451, + 0.32306361198425293, + -2.348386526107788, + -0.04350852966308594, + 0.05847747623920441, + -0.1053842082619667, + 0.12492304295301437, + 0.06711933761835098, + 0.211573526263237, + -2.1858420372009277, + -0.1017637699842453, + -0.007234741002321243, + -0.01731230691075325, + 0.31457260251045227, + 0.10490778833627701, + -0.11348345875740051, + -1.5930464267730713, + -0.1564388871192932 + ], + [ + 0.07825403660535812, + 0.058583058416843414, + -0.1342998445034027, + -0.03148752450942993, + 0.08402667939662933, + -2.2371621131896973, + -0.27078020572662354, + -0.024682804942131042, + 0.00805166456848383, + 0.17025572061538696, + 0.1683911830186844, + 0.07624384760856628, + -1.8365434408187866, + -0.17505960166454315, + 0.0820261612534523, + 0.018710482865571976, + 0.018737733364105225, + 0.022886931896209717, + -0.05467510223388672, + -1.6140875816345215, + -0.19445520639419556 + ], + [ + 0.05307849496603012, + 0.029461383819580078, + -0.10224887728691101, + -0.11858844757080078, + 0.0649922639131546, + -2.0333945751190186, + -0.2658430337905884, + 0.06256981194019318, + 0.036147210747003555, + 0.12818849086761475, + 0.22078293561935425, + 0.17197996377944946, + -1.9191186428070068, + -0.13697020709514618, + 0.04940533638000488, + 0.05206349864602089, + 0.027456894516944885, + -0.00018197298049926758, + 0.027761712670326233, + -1.6916732788085938, + -0.16476592421531677 + ], + [ + -0.057636141777038574, + -0.01742929220199585, + -0.036404818296432495, + -0.01711226999759674, + 0.09378434717655182, + -2.0185699462890625, + -0.22933721542358398, + 0.13657210767269135, + -0.010317999869585037, + 0.1696648895740509, + 0.01408969983458519, + 0.18841473758220673, + -1.9807454347610474, + -0.1428220421075821, + 0.006691664457321167, + 0.09006409347057343, + 0.2312215268611908, + -0.051600679755210876, + -0.07336241006851196, + -1.722533941268921, + -0.19763889908790588 + ], + [ + -0.021733596920967102, + -0.022700006142258644, + -0.09785762429237366, + -0.0392765998840332, + 0.02021634578704834, + -1.9918321371078491, + -0.29967331886291504, + 0.15010276436805725, + -0.006304899230599403, + 0.15463489294052124, + -0.00865880399942398, + 0.0901183933019638, + -2.0479912757873535, + -0.11742065846920013, + -0.005247395485639572, + 0.11192092299461365, + 0.2463504821062088, + 0.038363367319107056, + -0.07147014141082764, + -1.624755859375, + -0.32979658246040344 + ], + [ + -0.08303648233413696, + 0.005722474306821823, + 0.02876332402229309, + -0.063916876912117, + 0.09928908944129944, + -2.1052942276000977, + -0.22953367233276367, + 0.1969548612833023, + -0.052117086946964264, + 0.1749492585659027, + 0.05964469164609909, + 0.15526777505874634, + -1.8762308359146118, + 0.0742640495300293, + 0.003120254725217819, + 0.08843435347080231, + 0.21593104302883148, + 0.009374931454658508, + -0.040671706199645996, + -1.5572540760040283, + -0.12304079532623291 + ], + [ + 0.06068043410778046, + -0.07272287458181381, + 0.016918838024139404, + -0.23076801002025604, + 0.06407099962234497, + -2.143833875656128, + -0.17938441038131714, + 0.23395749926567078, + -0.08170146495103836, + -0.0024092718958854675, + 0.03844539448618889, + 0.12899602949619293, + -2.111145257949829, + -0.1777956336736679, + -0.059261541813611984, + 0.09445318579673767, + 0.30387645959854126, + -0.05118764936923981, + 0.024333566427230835, + -1.648310661315918, + -0.1620248556137085 + ], + [ + 0.18324795365333557, + -0.05526398494839668, + 0.017617106437683105, + -0.2363443225622177, + 0.20967961847782135, + -1.349843978881836, + -0.7425796389579773, + -0.05657553672790527, + -0.0720028430223465, + 0.012850567698478699, + 0.15367190539836884, + 0.1685241460800171, + -1.4467493295669556, + -0.41098105907440186, + 0.024837851524353027, + 0.1823609322309494, + 0.1010756641626358, + -0.011001542210578918, + -0.14043891429901123, + -1.534637451171875, + -0.16784745454788208 + ] + ], + "generated_utilities": [ + 0.13187798857688904, + 2.0, + 2.0, + 2.0, + 2.0, + 2.0, + 0.13499042391777039, + 0.16052719950675964 + ], + "group_id": "ms-6c8d06bace98a28a8e4e", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8583254218101501, + 2.0, + 0.8125, + 0.12893107533454895, + 0.8582976460456848, + 0.15346482396125793, + 0.18961156904697418, + 0.13901273906230927, + 0.10113435983657837, + 0.11161506175994873, + 0.09923169016838074, + 0.16662254929542542, + 0.04211682081222534, + 0.024710848927497864, + 0.05722464621067047, + 0.8584086298942566 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.10696407407522202, + 0.3957799971103668, + 0.15132969617843628, + -0.32772913575172424, + 0.005261177197098732, + -2.2650365829467773, + 0.0, + 0.3958877623081207, + -0.5120954513549805, + -0.4317946135997772, + -0.3970779776573181, + 0.5317854285240173, + -1.8659565448760986, + 0.0, + 0.1198807880282402, + -0.11760073900222778, + -0.6048114895820618, + 0.15180528163909912, + -0.532695472240448, + -1.6115882396697998, + 0.0 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.120756149291992, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.070481538772583, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0495758056640625 + ], + [ + 0.30105674266815186, + 0.4776385426521301, + -0.22267001867294312, + -0.13300693035125732, + -0.11095686256885529, + -3.138023853302002, + -0.793931245803833, + 0.16969743371009827, + -0.16339385509490967, + 0.02508675307035446, + 0.654399573802948, + -0.22830897569656372, + -3.1443004608154297, + -1.4291013479232788, + 0.005021120421588421, + 0.005560946650803089, + -0.1901366114616394, + 1.0211858749389648, + -0.09353887289762497, + -2.3457469940185547, + -1.406853199005127 + ], + [ + 0.4160586893558502, + 0.13316196203231812, + -0.21056225895881653, + -0.25006812810897827, + 0.060951750725507736, + -2.8735690116882324, + -0.9062713980674744, + -0.28707224130630493, + -0.2396206259727478, + -0.19859735667705536, + 0.29039445519447327, + -0.24077445268630981, + -2.644789934158325, + -1.8330551385879517, + -0.23438936471939087, + 0.04620177671313286, + 0.1472887098789215, + 0.9629149436950684, + -0.020509399473667145, + -2.7985036373138428, + -1.2463163137435913 + ], + [ + 0.2486386001110077, + -0.022333532571792603, + 0.14628465473651886, + -0.8381847143173218, + 0.21282373368740082, + -3.585387945175171, + -1.2156991958618164, + -0.08670558780431747, + -0.3178575336933136, + 0.4322021007537842, + 0.6421236395835876, + -0.33922863006591797, + -3.004714012145996, + -1.19637131690979, + -0.015640778467059135, + 0.10077434033155441, + -0.449941486120224, + 1.0431067943572998, + 0.2899783253669739, + -2.500624179840088, + -1.6148788928985596 + ], + [ + 0.46770766377449036, + 0.03902164101600647, + 0.2556130588054657, + -0.45893940329551697, + 0.08939866721630096, + -3.163339853286743, + -0.7599828243255615, + 0.3327450156211853, + 0.3252364993095398, + 0.3067358434200287, + 1.1394517421722412, + 0.472858726978302, + -3.1504058837890625, + -1.9780522584915161, + -0.26712900400161743, + -0.262444406747818, + 0.20269782841205597, + 1.336982250213623, + -0.06957009434700012, + -2.0651021003723145, + -1.8484716415405273 + ], + [ + 0.48348987102508545, + 0.19676931202411652, + -0.20689666271209717, + -0.21521174907684326, + -0.5048230886459351, + -3.491732358932495, + -0.7390058636665344, + 0.18321916460990906, + -0.1509065330028534, + 0.27166980504989624, + 1.004181981086731, + -0.5531666874885559, + -2.531594753265381, + -2.0243897438049316, + -0.44593361020088196, + 0.49786314368247986, + -0.006632372736930847, + 0.24423635005950928, + -0.6331305503845215, + -2.7707719802856445, + -2.1364808082580566 + ], + [ + -0.5684993863105774, + 0.0623701810836792, + -0.10449839383363724, + -0.7858190536499023, + -0.057080868631601334, + -3.981018304824829, + -0.6721940636634827, + 0.10191962122917175, + -0.7216448783874512, + 0.3870577812194824, + 0.5508396625518799, + -0.6811783313751221, + -3.6131227016448975, + -1.7484180927276611, + 0.34220433235168457, + -0.16193144023418427, + 0.16876991093158722, + 0.6509957909584045, + 0.2548854351043701, + -2.0779964923858643, + -1.5782216787338257 + ], + [ + -0.1224658265709877, + 0.22953394055366516, + -0.41326335072517395, + 0.19802969694137573, + 0.6866055130958557, + -2.6415462493896484, + -1.049781322479248, + -0.6788480281829834, + -0.8187242746353149, + 0.6141759157180786, + 1.3649951219558716, + -0.36378854513168335, + -2.2142889499664307, + -1.3759111166000366, + 0.583359956741333, + 0.6860965490341187, + 0.8285356760025024, + 1.4575884342193604, + -0.4101440906524658, + -2.218319892883301, + -1.5390111207962036 + ], + [ + 0.3368571698665619, + 0.4528505206108093, + -0.2985517382621765, + 0.37766069173812866, + 0.2064778208732605, + -2.8070714473724365, + -1.5598305463790894, + -0.9032005071640015, + -0.48420336842536926, + 0.639835000038147, + -0.3801422119140625, + -0.1376074254512787, + -3.369077205657959, + -1.9231767654418945, + -0.16211053729057312, + 0.53387051820755, + 0.6583098769187927, + 0.7611914873123169, + -0.46404945850372314, + -2.7554476261138916, + -1.4307246208190918 + ], + [ + 0.6002402901649475, + -0.14213827252388, + 0.30183425545692444, + -0.20118463039398193, + -0.16981711983680725, + -3.4212875366210938, + -0.2252577543258667, + 0.34123149514198303, + 0.003176456317305565, + 0.7433648109436035, + 1.1403348445892334, + 0.5709706544876099, + -1.8894051313400269, + -0.7200812101364136, + -0.7235198020935059, + 0.2147149294614792, + -0.28744715452194214, + 1.0323235988616943, + 0.5815067291259766, + -2.582993984222412, + -1.2160475254058838 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.019069351255893707, + 0.012574762105941772, + 0.07413499802350998, + -0.04097828269004822, + -0.03991875424981117, + -2.2650365829467773, + 0.0, + -0.06082560494542122, + 0.05821063742041588, + 0.07896332442760468, + -0.037061095237731934, + -0.08879614621400833, + -1.8659565448760986, + 0.0, + -0.022465867921710014, + -0.026843247935175896, + 0.04789978265762329, + -0.019972383975982666, + -0.04040021449327469, + -1.6115882396697998, + 0.0 + ] + ], + "predicted_scores": [ + 4.905849456787109, + 4.465726375579834, + 4.1869683265686035, + 3.873480796813965, + 3.8334813117980957, + 3.511204957962036, + 3.3798365592956543, + 3.3796608448028564 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1665739511", + "selected_index": 0, + "state_hash": "0f77663511063f17b0962024501b863c62b881513459d8bfa3ac00bbaed67de4", + "stored_base_utility": 0.8583254218101501, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + "base_progress": 1.0, + "base_success": true, + "base_utility": 2.0, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.11944694817066193, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.11944694817066193 + }, + { + "contact_quality": null, + "progress": 0.16878563165664673, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16878563165664673 + }, + { + "contact_quality": null, + "progress": 0.17180444300174713, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17180444300174713 + }, + { + "contact_quality": null, + "progress": 0.17166458070278168, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17166458070278168 + }, + { + "contact_quality": null, + "progress": 0.19043110311031342, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19043110311031342 + }, + { + "contact_quality": null, + "progress": 0.1885138899087906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1885138899087906 + }, + { + "contact_quality": null, + "progress": 0.19852638244628906, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19852638244628906 + }, + { + "contact_quality": null, + "progress": 0.1698688566684723, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1698688566684723 + } + ], + "candidate_progress": [ + 0.11944694817066193, + 0.16878563165664673, + 0.17180444300174713, + 0.17166458070278168, + 0.19043110311031342, + 0.1885138899087906, + 0.19852638244628906, + 0.1698688566684723 + ], + "candidate_source_chart_ids": [ + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1", + "ms-5a32c809ed679663dcb1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-72777024349c2aacd5e0", + "generated_tangents": [ + [ + 0.46157407760620117, + 0.174520805478096, + -0.03888651728630066, + -0.5337043404579163, + 0.5942518711090088, + -0.2038794755935669, + 0.9634844064712524, + -0.15029233694076538, + 0.050802119076251984, + -0.046997345983982086, + 0.08813080191612244, + -0.10318443179130554, + -1.1648684740066528, + -0.15636833012104034, + -0.06540808081626892, + 0.2142813801765442, + 0.36294233798980713, + -0.014899656176567078, + -0.2761247158050537, + -0.8063332438468933, + -0.4218128025531769 + ], + [ + 0.1354047954082489, + 0.1665385216474533, + -0.09189078211784363, + -0.5567952394485474, + 0.4742484986782074, + -0.36955758929252625, + -0.047621965408325195, + -0.15154656767845154, + 0.0809565931558609, + 0.06719428300857544, + 0.19103571772575378, + 0.06663753092288971, + -1.800724744796753, + -0.04190605878829956, + -0.1072264164686203, + 0.0981844812631607, + 0.07242068648338318, + -0.10282446444034576, + -0.16536816954612732, + -1.1918113231658936, + 0.11890645325183868 + ], + [ + -0.029327720403671265, + 0.1477586179971695, + -0.07216331362724304, + -0.41563108563423157, + 0.41079404950141907, + -0.08326615393161774, + -0.04381251335144043, + 0.024007931351661682, + 0.09756569564342499, + 0.0832497775554657, + 0.24985820055007935, + 0.06584241986274719, + -1.7538155317306519, + 0.1450558453798294, + -0.03608652576804161, + 0.03242557495832443, + 0.061501696705818176, + -0.04694490134716034, + -0.12731868028640747, + -1.2788195610046387, + 0.11673453450202942 + ], + [ + 0.12438970059156418, + 0.14528720080852509, + -0.12435182929039001, + -0.5101529359817505, + 0.47771918773651123, + -0.32751479744911194, + -0.1798877716064453, + -0.0757099986076355, + 0.061697252094745636, + 0.07973569631576538, + 0.2250921130180359, + -0.01808471977710724, + -1.8440006971359253, + -0.08621540665626526, + -0.2479267716407776, + 0.07480540871620178, + 0.03405578434467316, + -0.07880456745624542, + -0.061933428049087524, + -1.2551229000091553, + 0.19224265217781067 + ], + [ + 0.01854190230369568, + 0.2946150302886963, + -0.3474480211734772, + -0.3782707452774048, + 0.42811042070388794, + -0.283975750207901, + 0.007112085819244385, + -0.08178913593292236, + -0.05934348702430725, + -0.011700280010700226, + 0.24963107705116272, + -0.017295151948928833, + -1.9101580381393433, + -0.0944548100233078, + -0.21573540568351746, + 0.0623256154358387, + 0.23162756860256195, + -0.27955859899520874, + -0.11699864268302917, + -1.5937581062316895, + 0.09011290222406387 + ], + [ + -0.028540179133415222, + 0.24281413853168488, + -0.19422301650047302, + -0.5394165515899658, + 0.5835269689559937, + 0.06668397784233093, + -0.047837793827056885, + -0.0931469202041626, + -0.06707969307899475, + -0.021994300186634064, + 0.17776334285736084, + -0.17167231440544128, + -2.0242831707000732, + 0.004569120705127716, + -0.06618009507656097, + 0.0867065042257309, + 0.32150667905807495, + -0.2761147618293762, + -0.2576178312301636, + -1.239419937133789, + -0.07899683713912964 + ], + [ + -0.03803916275501251, + 0.2134903520345688, + -0.3332861363887787, + -0.5010786056518555, + 0.48019707202911377, + -0.2275649607181549, + -0.011795222759246826, + -0.053437620401382446, + -0.04079996794462204, + 0.050588078796863556, + 0.16244184970855713, + -0.09413272142410278, + -1.840475082397461, + -0.056180715560913086, + -0.17065294086933136, + 0.08698076009750366, + 0.2513794004917145, + -0.3058536648750305, + -0.34668129682540894, + -1.406508445739746, + -0.022539496421813965 + ], + [ + -0.01849667727947235, + 0.21552197635173798, + -0.4834817945957184, + -0.5902851223945618, + 0.3741875886917114, + -0.27450865507125854, + 0.10583284497261047, + 0.1401326060295105, + -0.05798898637294769, + 0.10467419773340225, + 0.11828957498073578, + -0.26767292618751526, + -1.872161865234375, + -0.09646837413311005, + -0.18259958922863007, + 0.17331455647945404, + 0.345462441444397, + -0.1606987565755844, + -0.43198955059051514, + -1.382481336593628, + 0.18469150364398956 + ] + ], + "generated_utilities": [ + 0.11944694817066193, + 0.16878563165664673, + 0.17180444300174713, + 0.17166458070278168, + 0.19043110311031342, + 0.1885138899087906, + 0.19852638244628906, + 0.1698688566684723 + ], + "group_id": "ms-72777024349c2aacd5e0", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8314920663833618, + 0.1595238745212555, + 2.0, + 0.16971935331821442, + 0.5892879366874695, + 0.1617850512266159, + 0.1877371072769165, + 0.5782577395439148, + 0.12074722349643707, + 0.11237785220146179, + 0.08792310953140259, + 0.1209893673658371, + 0.13739405572414398, + 0.0773405134677887, + 0.09497407078742981, + 0.047447383403778076 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.002835560590028763, + 0.3854118585586548, + -0.06168194115161896, + -0.16570186614990234, + -0.05051702260971069, + -0.43052077293395996, + 0.0, + 0.024141468107700348, + -0.011737043969333172, + -0.05502588301897049, + 0.013536334037780762, + -0.000911470502614975, + -1.6284780502319336, + 0.0, + 0.030459173023700714, + -0.05656937509775162, + 0.03393230587244034, + 0.06894856691360474, + 0.00450655072927475, + -1.2762084007263184, + 0.0 + ], + [ + 0.2191334217786789, + 0.3854118585586548, + 0.5048428177833557, + -0.16570186614990234, + -0.20026534795761108, + -0.43052077293395996, + 0.0, + -0.026082046329975128, + 0.03163425251841545, + -0.04950854927301407, + 0.1969367265701294, + -0.05762401595711708, + -1.6284780502319336, + 0.0, + 0.002695085946470499, + 0.34764212369918823, + 0.6051180362701416, + 0.9295560717582703, + 0.38285934925079346, + -1.2762084007263184, + 0.0 + ], + [ + -0.04624907672405243, + 1.3854118585586548, + 0.05864633619785309, + -1.1657018661499023, + 0.7997346520423889, + -1.43052077293396, + 1.1980153322219849, + -0.023269198834896088, + -0.026228107511997223, + -0.009256521239876747, + 0.6858966946601868, + 0.03280947729945183, + -2.6284780502319336, + -1.7920862436294556, + -0.0004090145230293274, + -0.00536443293094635, + 0.007614931557327509, + 0.7401981949806213, + 0.11622340977191925, + -2.2762084007263184, + -1.6813035011291504 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.3960306644439697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.584172487258911, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.362607002258301 + ], + [ + -0.13329119980335236, + 1.1171538829803467, + -0.19378040730953217, + -0.9341158866882324, + 0.8714147806167603, + -1.659934163093567, + 1.1658833026885986, + 0.23766514658927917, + 0.21810689568519592, + -0.05730185657739639, + 0.4946695566177368, + 0.17369553446769714, + -2.559515953063965, + -1.5605767965316772, + 0.08475297689437866, + 0.22485606372356415, + -0.2591644823551178, + 0.9637633562088013, + -0.08890970051288605, + -2.1015453338623047, + -1.5709789991378784 + ], + [ + -0.3507236838340759, + 1.1737966537475586, + 0.10341192036867142, + -0.7761039137840271, + 0.47493064403533936, + -1.2297213077545166, + 1.4181653261184692, + -0.1981543004512787, + -0.18248644471168518, + 0.17768341302871704, + 0.843591034412384, + -0.07443197071552277, + -2.6062240600585938, + -1.750958800315857, + -0.15503963828086853, + 0.18218432366847992, + 0.002181168645620346, + 0.8641887903213501, + -0.026101917028427124, + -1.9398812055587769, + -1.8561092615127563 + ], + [ + -0.22363319993019104, + 1.533349871635437, + 0.4454386830329895, + -1.5271000862121582, + 0.8945828080177307, + -1.105146884918213, + 0.7171626687049866, + -0.20207461714744568, + 0.31458696722984314, + 0.20214562118053436, + 1.0593571662902832, + -0.4655032455921173, + -2.8809642791748047, + -1.580764651298523, + -0.22098299860954285, + -0.08731096982955933, + -0.15165574848651886, + 0.89030921459198, + 0.611029326915741, + -2.2792654037475586, + -1.798964262008667 + ], + [ + -0.3702698349952698, + 0.9771865606307983, + 0.27564477920532227, + -1.7643829584121704, + 0.7298657894134521, + -1.875503420829773, + 1.4598875045776367, + 0.47600361704826355, + 0.13439032435417175, + -0.23905222117900848, + 1.2192013263702393, + -0.44705769419670105, + -2.603621482849121, + -1.3898506164550781, + 0.16782721877098083, + -0.2559890151023865, + 0.2965841293334961, + 0.8750308752059937, + 0.1521819680929184, + -2.3417282104492188, + -2.2663588523864746 + ], + [ + -0.31705379486083984, + 0.866703450679779, + -0.09125347435474396, + -1.5959886312484741, + 0.8041162490844727, + -1.7037310600280762, + 1.6304850578308105, + 0.11580469459295273, + 0.2654762864112854, + 0.12200001627206802, + 0.6904033422470093, + 0.19859254360198975, + -2.7615463733673096, + -2.059410333633423, + 0.5591834783554077, + -0.4801292419433594, + -0.3343847393989563, + 0.8787070512771606, + -0.14748375117778778, + -2.0334713459014893, + -1.1611082553863525 + ], + [ + -0.11324095726013184, + 0.8777624368667603, + 0.8215147256851196, + -1.5343353748321533, + 0.881126344203949, + -1.0607860088348389, + 0.5398488640785217, + 0.6020828485488892, + -0.5065795183181763, + 0.02798180840909481, + 0.33880186080932617, + 0.320717990398407, + -3.3552045822143555, + -1.1435439586639404, + 0.19279444217681885, + -0.5245973467826843, + 0.30647164583206177, + 1.3726601600646973, + 0.3123217523097992, + -2.9675674438476562, + -1.7007204294204712 + ], + [ + -0.6432619094848633, + 0.79607754945755, + -0.49864423274993896, + -1.0710605382919312, + 1.0709084272384644, + -1.612175703048706, + 1.2555015087127686, + 0.2873077392578125, + -0.36858057975769043, + 0.05837755650281906, + 1.1025807857513428, + 0.7949575185775757, + -3.2468795776367188, + -2.556741237640381, + -0.6579093337059021, + -0.8874800801277161, + 0.7922345399856567, + 0.4745541512966156, + 0.07610362768173218, + -2.041800022125244, + -2.2069954872131348 + ], + [ + -0.9264093637466431, + 1.9828436374664307, + 0.7907096147537231, + -1.260971188545227, + 1.7548933029174805, + -2.2685132026672363, + 1.1821955442428589, + -0.49968039989471436, + 0.10982713848352432, + 0.2384604662656784, + 0.2075445055961609, + -0.7872775793075562, + -2.076058864593506, + -1.1888585090637207, + 0.7447717785835266, + 0.2355569452047348, + -0.3163020610809326, + 0.3282558023929596, + -0.4075327515602112, + -3.2451019287109375, + -2.4086263179779053 + ], + [ + -0.7190575003623962, + 1.3702023029327393, + 0.8307373523712158, + -0.7533494234085083, + 0.574251651763916, + -2.3376822471618652, + 0.5512753129005432, + -0.04391670227050781, + 0.2242431342601776, + -0.46594029664993286, + 0.5667901039123535, + -0.12832416594028473, + -2.9870448112487793, + -1.2032369375228882, + 0.17313259840011597, + 0.5932936668395996, + 0.7428228259086609, + 0.6570835113525391, + -0.1891915649175644, + -1.3703107833862305, + -1.7630409002304077 + ], + [ + 0.799586832523346, + 1.0164330005645752, + -0.49519193172454834, + -0.6983597278594971, + 1.7632484436035156, + -0.9710203409194946, + 1.5416830778121948, + 0.9653489589691162, + 0.06498706340789795, + 0.2132291942834854, + -0.1650814414024353, + 0.06786654889583588, + -1.8782389163970947, + -2.3576126098632812, + 0.9410911798477173, + -0.576595664024353, + -0.8031080365180969, + 0.7120062112808228, + 0.6479252576828003, + -2.3928208351135254, + -1.4952012300491333 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.11725310236215591, + 0.3854118585586548, + 0.06966311484575272, + -0.16570186614990234, + 0.06442362070083618, + -0.43052077293395996, + 0.0, + 0.11858775466680527, + 0.0573427751660347, + -0.14938217401504517, + -0.01657569408416748, + 0.060478124767541885, + -1.6284780502319336, + 0.0, + 0.016584808006882668, + 0.18205130100250244, + -0.13064442574977875, + 0.15621143579483032, + 0.056775350123643875, + -1.2762084007263184, + 0.0 + ] + ], + "predicted_scores": [ + 7.788398742675781, + 6.110999584197998, + 5.820715427398682, + 4.917744159698486, + 4.872241497039795, + 4.84718656539917, + 4.464103698730469, + 3.651381731033325 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "256770874", + "selected_index": 0, + "state_hash": "3a7aa4ba27a5b34cdeb0f31ce9818588b1dd4ede96bf546f07302616c2e9b3a6", + "stored_base_utility": 0.8314920663833618, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20372186601161957, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20372186601161957 + }, + "base_progress": 0.20372186601161957, + "base_success": false, + "base_utility": 0.20372186601161957, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1758389174938202, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1758389174938202 + }, + { + "contact_quality": null, + "progress": 0.18668952584266663, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.18668952584266663 + }, + { + "contact_quality": null, + "progress": 0.2069244086742401, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2069244086742401 + }, + { + "contact_quality": null, + "progress": 0.19020694494247437, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19020694494247437 + }, + { + "contact_quality": null, + "progress": 0.22462211549282074, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22462211549282074 + }, + { + "contact_quality": null, + "progress": 0.22120237350463867, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22120237350463867 + }, + { + "contact_quality": null, + "progress": 0.5639582276344299, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.5639582276344299 + }, + { + "contact_quality": null, + "progress": 0.22967948019504547, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22967948019504547 + } + ], + "candidate_progress": [ + 0.1758389174938202, + 0.18668952584266663, + 0.2069244086742401, + 0.19020694494247437, + 0.22462211549282074, + 0.22120237350463867, + 0.5639582276344299, + 0.22967948019504547 + ], + "candidate_source_chart_ids": [ + "ms-5bddb39f792f20e5e6d4", + "ms-ecb37ebbab21a506d8ca", + "ms-95028134c6e3670b89d5", + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4", + "ms-95028134c6e3670b89d5", + "ms-95028134c6e3670b89d5" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank3" + ], + "candidates_evaluated": true, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "generated_tangents": [ + [ + -0.11020086705684662, + -0.013108022511005402, + -0.1264011561870575, + 0.35455426573753357, + 0.23730328679084778, + -0.48435285687446594, + 0.07835516333580017, + 0.052899062633514404, + 0.04665840417146683, + 0.06724787503480911, + 0.0012043118476867676, + 0.2893897294998169, + -1.6423239707946777, + -0.009604468941688538, + 0.1118192970752716, + 0.16256406903266907, + 0.041011497378349304, + -0.21209131181240082, + -0.019342273473739624, + -1.4293444156646729, + -0.02278698980808258 + ], + [ + -0.026120781898498535, + 0.018893331289291382, + -0.14871594309806824, + 0.261665016412735, + 0.1527482122182846, + -0.435959130525589, + -0.11187863349914551, + 0.06613901257514954, + 0.03334052115678787, + 0.012848034501075745, + 0.029305411502718925, + 0.21651701629161835, + -1.7776308059692383, + 0.0027957111597061157, + 0.08954580128192902, + 0.12566040456295013, + 0.06295119225978851, + -0.25235462188720703, + 0.006839275360107422, + -1.576141119003296, + 0.08545006811618805 + ], + [ + -0.0192459374666214, + 0.06497050076723099, + -0.22957196831703186, + 0.22761625051498413, + 0.20351259410381317, + -0.4425376355648041, + -0.052713632583618164, + 0.06681163609027863, + 0.008113421499729156, + -0.0003314986824989319, + -0.07100499421358109, + 0.22354894876480103, + -1.7586311101913452, + 0.06674952805042267, + 0.06904176622629166, + 0.09685906767845154, + 0.05805554986000061, + -0.18940134346485138, + 0.027307987213134766, + -1.6631107330322266, + 0.06823643296957016 + ], + [ + 0.023560263216495514, + 0.009691145271062851, + -0.14869794249534607, + 0.30883529782295227, + 0.17246560752391815, + -0.6573750972747803, + -0.04758906364440918, + 0.052341386675834656, + 0.046923957765102386, + 0.024762749671936035, + 0.006820760667324066, + 0.10632195323705673, + -1.573324203491211, + -0.037993401288986206, + 0.07132565230131149, + 0.11828601360321045, + -0.0035343170166015625, + -0.21954290568828583, + 0.066749706864357, + -1.4483566284179688, + 0.05538436025381088 + ], + [ + -0.0713127851486206, + -0.08605244755744934, + -0.4092229902744293, + 0.3931208550930023, + 0.19602271914482117, + -0.673324704170227, + 0.12838059663772583, + 0.15265193581581116, + -0.02485211379826069, + -0.03920639306306839, + -0.12418157607316971, + 0.06869977712631226, + -1.7395849227905273, + -0.03127090632915497, + -0.02653767541050911, + 0.16294054687023163, + 0.16098234057426453, + -0.2561621069908142, + -0.15497267246246338, + -1.359914779663086, + -0.105194091796875 + ], + [ + -0.08782057464122772, + -0.08472000062465668, + -0.31380751729011536, + 0.5326024293899536, + 0.19600187242031097, + -0.5924414396286011, + -0.07720458507537842, + 0.189269557595253, + 0.03365449607372284, + -0.01644543558359146, + -0.12385163456201553, + 0.04005500674247742, + -2.000369071960449, + -0.11754660308361053, + 0.07276436686515808, + 0.21560324728488922, + 0.12088972330093384, + -0.3859550356864929, + -0.07491222023963928, + -1.3207519054412842, + 0.11474438011646271 + ], + [ + -0.1525118201971054, + 0.03449248522520065, + -0.14602509140968323, + 0.22883886098861694, + 0.17169666290283203, + -0.24276667833328247, + -0.21707427501678467, + 0.19013364613056183, + -0.031076611950993538, + -0.0734015628695488, + -0.2084907591342926, + 0.11446955054998398, + -1.7619656324386597, + -0.02972029149532318, + 0.14165401458740234, + 0.11926732957363129, + 0.02618981897830963, + -0.23360036313533783, + -0.09116426110267639, + -1.7336742877960205, + 0.12123022228479385 + ], + [ + -0.09768658876419067, + 0.016946300864219666, + -0.1720522940158844, + 0.2824391722679138, + 0.2329549938440323, + -0.23956191539764404, + -0.1392601728439331, + 0.22499649226665497, + -0.019799647852778435, + -0.02801615744829178, + 0.004370331764221191, + 0.08938870579004288, + -1.7242966890335083, + 0.11016328632831573, + -0.05263916775584221, + 0.12067274749279022, + 0.06463602185249329, + -0.23037971556186676, + -0.08607134222984314, + -1.4778838157653809, + 0.0240279883146286 + ] + ], + "generated_utilities": [ + 0.1758389174938202, + 0.18668952584266663, + 0.2069244086742401, + 0.19020694494247437, + 0.22462211549282074, + 0.22120237350463867, + 0.5639582276344299, + 0.22967948019504547 + ], + "group_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8363267779350281, + 2.0, + 2.0, + 0.16206762194633484, + 0.5390529036521912, + 0.08151453733444214, + 0.5261932611465454, + 0.5265869498252869, + 0.5459305047988892, + 0.18360589444637299, + 0.16689825057983398, + 0.15426510572433472, + 0.0028923004865646362, + 0.15516109764575958, + 0.0506085604429245, + 0.19578850269317627 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.0005862116813659668, + 0.18783891201019287, + -0.02734696865081787, + 0.09201061725616455, + 0.010171383619308472, + -0.4856022596359253, + 0.0, + 0.10349468141794205, + 0.4567214846611023, + -0.1893925815820694, + 0.12298360466957092, + 0.3369159996509552, + -1.6010875701904297, + 0.0, + 0.304210901260376, + -0.3049007058143616, + 0.3435661494731903, + 0.3158445656299591, + 0.2931552231311798, + -1.2709691524505615, + 0.0 + ], + [ + -0.726529061794281, + -0.8121610879898071, + -1.0273469686508179, + 1.0920106172561646, + 0.3926190733909607, + -1.4856022596359253, + 1.8979665040969849, + -0.029249273240566254, + 0.008005961775779724, + 0.013417234644293785, + -0.10860864818096161, + 0.43491366505622864, + -2.6010875701904297, + -1.493168592453003, + -0.01047949492931366, + -0.0018908085767179728, + 0.019280990585684776, + 0.20726272463798523, + 0.45750224590301514, + -2.2709691524505615, + -1.4645930528640747 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7959330081939697, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.986337184906006, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.9291861057281494 + ], + [ + -0.9019005298614502, + -0.5901641845703125, + -0.920559287071228, + 0.991828978061676, + 0.2407551407814026, + -1.2655587196350098, + 1.7652356624603271, + 0.03909382224082947, + 0.04142814502120018, + 0.12125641852617264, + -0.2822783887386322, + 0.4936577081680298, + -2.5754144191741943, + -1.5686157941818237, + -0.22672900557518005, + -0.08431098610162735, + 0.09805622696876526, + 0.2479260265827179, + 0.4450404644012451, + -2.1972453594207764, + -1.4420135021209717 + ], + [ + -0.8822115063667297, + -0.9738405346870422, + -0.7449414730072021, + 0.8784142732620239, + 0.4583987295627594, + -1.3989503383636475, + 1.8331178426742554, + -0.006558071821928024, + 0.30383312702178955, + 0.2117801308631897, + -0.2538969814777374, + 0.13478082418441772, + -2.584805727005005, + -1.325882911682129, + 0.23593956232070923, + -0.2618964612483978, + -0.30780693888664246, + 0.24083535373210907, + 0.430472195148468, + -1.946203589439392, + -1.5077860355377197 + ], + [ + -0.39491432905197144, + -0.8895903825759888, + -0.9160035848617554, + 0.7220491170883179, + 0.36837494373321533, + -1.4429160356521606, + 1.4078493118286133, + 0.03969770669937134, + 0.007204015273600817, + 0.056278251111507416, + -0.4780465364456177, + 0.9121701717376709, + -2.512721538543701, + -1.6225675344467163, + -0.42074650526046753, + 0.15076839923858643, + -0.24544812738895416, + 0.08152849972248077, + 0.1987975835800171, + -2.712414264678955, + -1.9573136568069458 + ], + [ + -0.6879937648773193, + -0.8859440088272095, + -1.1152405738830566, + 0.8750674724578857, + -0.05955067276954651, + -1.5194538831710815, + 1.876329779624939, + -0.2068542242050171, + -0.5498799085617065, + 0.46419233083724976, + 0.42569708824157715, + -0.02127784490585327, + -2.5905418395996094, + -1.411050796508789, + -0.21327723562717438, + 0.20716798305511475, + 0.20993508398532867, + 0.33476364612579346, + -0.09872657060623169, + -2.5958731174468994, + -1.6374839544296265 + ], + [ + -0.5723236799240112, + -0.28514552116394043, + -0.6157487630844116, + 0.6151646375656128, + 0.32639649510383606, + -1.7217803001403809, + 2.5557403564453125, + 0.532052218914032, + 0.5977330207824707, + -0.5707970261573792, + -0.21707558631896973, + 0.8829224109649658, + -3.1287553310394287, + -1.0988328456878662, + 0.38229066133499146, + 0.0032835237216204405, + -0.3569248616695404, + 0.0031215548515319824, + 1.1060712337493896, + -2.3536932468414307, + -1.383341670036316 + ], + [ + -0.7901747822761536, + -0.8977189660072327, + -1.4548863172531128, + 0.5700533986091614, + -0.13018959760665894, + -0.8057670593261719, + 2.4019269943237305, + 0.5368256568908691, + 0.5227565765380859, + 0.28199371695518494, + -0.1637011021375656, + -0.2898564040660858, + -2.9142980575561523, + -1.8492696285247803, + 0.7177971601486206, + 0.2360217124223709, + -0.13862104713916779, + 0.9262527227401733, + 0.39495596289634705, + -2.470649480819702, + -1.3260751962661743 + ], + [ + -0.6053685545921326, + -0.3819047510623932, + -0.258634090423584, + 0.19338607788085938, + 0.19874510169029236, + -1.3407407999038696, + 2.5238022804260254, + -0.6922282576560974, + 0.8137098550796509, + 0.39421388506889343, + 0.09789760410785675, + 0.12572914361953735, + -2.518153667449951, + -1.6892600059509277, + 0.503250002861023, + 0.88390052318573, + -0.7520112991333008, + -0.3356115520000458, + 0.15291047096252441, + -2.5788943767547607, + -0.9403089284896851 + ], + [ + -0.464814156293869, + -0.6034223437309265, + -1.6505372524261475, + 0.6883972883224487, + -0.3758969306945801, + -1.785051941871643, + 2.605452060699463, + 0.550828218460083, + 0.4742209315299988, + -0.19333000481128693, + -1.002321481704712, + 0.48782211542129517, + -1.8015108108520508, + -0.8254865407943726, + -0.7333722710609436, + -0.4948355555534363, + 0.6016485095024109, + 0.5632485151290894, + -0.24367755651474, + -2.201673984527588, + -0.5806979537010193 + ], + [ + -1.4628195762634277, + -0.4433446228504181, + -1.9926375150680542, + 0.33977705240249634, + 0.03379333019256592, + -1.6042804718017578, + 2.221484422683716, + 0.7110810279846191, + -0.41500210762023926, + 0.08095695078372955, + -0.23683495819568634, + -0.43207624554634094, + -3.080759286880493, + -0.9071659445762634, + 0.47082334756851196, + -0.9117310047149658, + -0.06435621529817581, + -0.6813669204711914, + -0.2741369605064392, + -2.3763599395751953, + -1.000203013420105 + ], + [ + -1.382751226425171, + -1.7026821374893188, + -0.11846715211868286, + 1.5398613214492798, + -0.5885447859764099, + -1.8825289011001587, + 1.5456308126449585, + -0.320659339427948, + -0.964028000831604, + -0.1812019646167755, + 0.14226128160953522, + 0.4527861177921295, + -2.714468002319336, + -1.9451607465744019, + 0.195057213306427, + 0.16905252635478973, + -0.9247990250587463, + 0.9112424850463867, + 0.79449862241745, + -2.714893341064453, + -1.6473028659820557 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.02016383409500122, + 0.02509528398513794, + -0.02734696865081787, + 0.09201061725616455, + -0.04331648349761963, + -0.4856022596359253, + 0.0, + -0.022040890529751778, + -0.003832761198282242, + -0.01408458687365055, + 0.06165061891078949, + 0.02554956078529358, + -1.6010875701904297, + 0.0, + 0.009550465270876884, + 0.03566932678222656, + 0.013732164166867733, + 0.017749279737472534, + 0.0699148178100586, + -1.2709691524505615, + 0.0 + ], + [ + 0.023195505142211914, + 0.029890060424804688, + -0.02734696865081787, + 0.09201061725616455, + 0.06108534336090088, + -0.4856022596359253, + 0.0, + -0.034551952034235, + -0.15152660012245178, + -0.21811580657958984, + -0.063543401658535, + 0.23667243123054504, + -1.6010875701904297, + 0.0, + -0.1614403873682022, + -0.17222382128238678, + -0.09706380218267441, + 0.3660350441932678, + 0.13013160228729248, + -1.2709691524505615, + 0.0 + ] + ], + "predicted_scores": [ + 6.827688694000244, + 6.575668811798096, + 6.572710990905762, + 6.568657398223877, + 6.127616882324219, + 5.754226207733154, + 5.634440898895264, + 5.44002628326416 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "957101489", + "selected_index": 0, + "state_hash": "e8473553a2b17f5adf09e08e7271b5502d53a51225c17abac3d9f78906cf8c46", + "stored_base_utility": 0.8363267779350281, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8431589603424072, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8431589603424072 + }, + "base_progress": 0.8431589603424072, + "base_success": false, + "base_utility": 0.8431589603424072, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.21672730147838593, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21672730147838593 + }, + { + "contact_quality": null, + "progress": 0.1849215030670166, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1849215030670166 + }, + { + "contact_quality": null, + "progress": 0.21649955213069916, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21649955213069916 + }, + { + "contact_quality": null, + "progress": 0.1958213746547699, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1958213746547699 + }, + { + "contact_quality": null, + "progress": 0.22092504799365997, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.22092504799365997 + }, + { + "contact_quality": null, + "progress": 0.17803087830543518, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17803087830543518 + }, + { + "contact_quality": null, + "progress": 0.20323540270328522, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20323540270328522 + }, + { + "contact_quality": null, + "progress": 0.19053253531455994, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19053253531455994 + } + ], + "candidate_progress": [ + 0.21672730147838593, + 0.1849215030670166, + 0.21649955213069916, + 0.1958213746547699, + 0.22092504799365997, + 0.17803087830543518, + 0.20323540270328522, + 0.19053253531455994 + ], + "candidate_source_chart_ids": [ + "ms-626ac2ad9d0dd95f1a2e", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4", + "ms-ce9505086ea115f86649", + "ms-5bddb39f792f20e5e6d4", + "ms-9228f8295b9891cf011c", + "ms-9d12ad4ebdf37419994c", + "ms-9d12ad4ebdf37419994c" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank1" + ], + "candidates_evaluated": true, + "chart_id": "ms-9a14552b7db695dee92c", + "generated_tangents": [ + [ + 0.05048983544111252, + -0.06354224681854248, + -0.2508169710636139, + 0.3281157910823822, + 0.06176993250846863, + -1.0395859479904175, + -0.11872172355651855, + 0.04913552105426788, + 0.03442516177892685, + 0.06192578002810478, + 0.1901962161064148, + 0.15638986229896545, + -1.8924229145050049, + -0.1239984780550003, + -0.013814132660627365, + 0.08172887563705444, + 0.06453713774681091, + -0.039126038551330566, + 0.0215013325214386, + -1.6615486145019531, + -0.16162586212158203 + ], + [ + -0.061983928084373474, + -0.09459616243839264, + -0.2104627788066864, + 0.3599303662776947, + 0.05293440818786621, + -0.7814253568649292, + -0.02095434069633484, + 0.09287988394498825, + 0.02182001620531082, + 0.11593418568372726, + 0.1766892969608307, + 0.31205666065216064, + -1.836709976196289, + -0.14834998548030853, + -0.014053288847208023, + 0.13140583038330078, + 0.09235826134681702, + -0.08436663448810577, + -0.07792538404464722, + -1.6235504150390625, + -0.19722992181777954 + ], + [ + -0.06358054280281067, + -0.1707766354084015, + -0.4351976811885834, + 0.6192716360092163, + 0.17808198928833008, + -1.0362719297409058, + 0.02410733699798584, + 0.18320702016353607, + -0.057587020099163055, + -0.03287283331155777, + 0.014523003250360489, + 0.07170911133289337, + -2.2714781761169434, + -0.13795168697834015, + -0.00746951624751091, + 0.15431226789951324, + 0.24740000069141388, + -0.13378311693668365, + -0.14430412650108337, + -1.5733966827392578, + -0.14196494221687317 + ], + [ + -0.017947256565093994, + 0.0040266141295433044, + -0.13806882500648499, + 0.5576839447021484, + 0.055895641446113586, + -1.1047216653823853, + -0.23974275588989258, + 0.043097689747810364, + -0.030100451782345772, + 0.04504986107349396, + 0.06926173716783524, + 0.007569238543510437, + -2.192204236984253, + 0.08518990874290466, + 0.022515147924423218, + 0.07497668266296387, + 0.06646496057510376, + -0.03373345732688904, + -0.27405136823654175, + -1.905529260635376, + 0.02017819881439209 + ], + [ + -0.12511123716831207, + -0.14703741669654846, + -0.4999878704547882, + 0.4722348153591156, + 0.21037617325782776, + -1.1627404689788818, + 0.08314064145088196, + 0.15366435050964355, + -0.11278150230646133, + -0.03228030353784561, + -0.02198120951652527, + 0.04202157258987427, + -2.1234261989593506, + -0.05932125449180603, + -0.10859385132789612, + 0.11286456882953644, + 0.28593140840530396, + -0.034758344292640686, + -0.21397221088409424, + -1.6376261711120605, + -0.31162482500076294 + ], + [ + 0.09778532385826111, + -0.07935328036546707, + -0.1260325014591217, + 0.45217737555503845, + 0.0013529956340789795, + -1.891746163368225, + -0.10438776016235352, + 0.055621206760406494, + -0.03399578481912613, + 0.04440484941005707, + 0.11405788362026215, + 0.11137016862630844, + -1.7957154512405396, + -0.2559959292411804, + 0.01711001619696617, + -0.04273573309183121, + 0.1087232455611229, + -0.030084341764450073, + -0.1149483323097229, + -1.6477367877960205, + -0.058281928300857544 + ], + [ + 0.172541543841362, + -0.032881900668144226, + -0.22155597805976868, + 0.3285578191280365, + -0.09919175505638123, + -1.7492244243621826, + -0.034220099449157715, + 0.0644887387752533, + -0.04440489411354065, + 0.051439717411994934, + 0.07902070879936218, + 0.16825929284095764, + -1.7917665243148804, + -0.2121959775686264, + -0.049102213233709335, + -0.08361957967281342, + 0.12374039739370346, + 0.017558544874191284, + -0.11949673295021057, + -1.7161366939544678, + 0.07041347771883011 + ], + [ + 0.12216803431510925, + -0.0765799954533577, + -0.15727552771568298, + 0.3809538185596466, + 0.01112222671508789, + -1.882515788078308, + -0.05779761075973511, + 0.05188503861427307, + -0.041006267070770264, + 0.0496743842959404, + 0.17175006866455078, + 0.09753971546888351, + -1.8451467752456665, + -0.1975952535867691, + -0.0013269595801830292, + -0.07415686547756195, + 0.02080000936985016, + -0.04785656929016113, + -0.1824963390827179, + -1.6308767795562744, + 0.008575230836868286 + ] + ], + "generated_utilities": [ + 0.21672730147838593, + 0.1849215030670166, + 0.21649955213069916, + 0.1958213746547699, + 0.22092504799365997, + 0.17803087830543518, + 0.20323540270328522, + 0.19053253531455994 + ], + "group_id": "ms-9a14552b7db695dee92c", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.838127613067627, + 2.0, + 0.15671375393867493, + 0.17482513189315796, + 0.5684618353843689, + 0.19147612154483795, + 0.5643423795700073, + 0.1653006672859192, + 0.11670731008052826, + 0.14578887820243835, + 0.16058449447155, + 0.10923929512500763, + 0.13408902287483215, + 0.06845971941947937, + 0.02954423427581787, + 0.08425161242485046 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.22581616044044495, + -0.32600927352905273, + -0.054001130163669586, + 1.2553811073303223, + -0.1383839249610901, + -1.9753923416137695, + 0.0, + 0.09536236524581909, + 0.017635054886341095, + -0.06946105509996414, + 0.06630897521972656, + -0.10150641947984695, + -1.9112331867218018, + 0.0, + -0.23092225193977356, + 0.04748937487602234, + 0.3013715445995331, + -0.22072738409042358, + 0.0733548104763031, + -1.6514825820922852, + 0.0 + ], + [ + 0.22913068532943726, + -0.32600927352905273, + 0.24692901968955994, + 1.2553811073303223, + 0.3704858124256134, + -1.9753923416137695, + 0.0, + 0.036500319838523865, + 0.12820330262184143, + -0.09674224257469177, + 0.2668613791465759, + -0.14372897148132324, + -1.9112331867218018, + 0.0, + -0.27894535660743713, + -0.3322452902793884, + 0.16300754249095917, + -0.24123603105545044, + -0.2359180599451065, + -1.6514825820922852, + 0.0 + ], + [ + 0.0198141410946846, + -1.3260092735290527, + 0.011403528973460197, + 2.2553811073303223, + 0.5258303284645081, + -2.9753923416137695, + 1.8956809043884277, + -0.028990529477596283, + 0.007858656346797943, + 0.03128144145011902, + 0.5457771420478821, + -0.14793801307678223, + -2.9112331867218018, + -1.5743238925933838, + -0.011861175298690796, + -0.002163209253922105, + 0.015663353726267815, + 0.7587639689445496, + -0.07801014184951782, + -2.651482582092285, + -1.5696079730987549 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 3.7913618087768555, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1486477851867676, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.1392159461975098 + ], + [ + 0.13539990782737732, + -1.2862521409988403, + 0.28441768884658813, + 2.3658432960510254, + 0.4678441286087036, + -2.8791494369506836, + 1.8408029079437256, + -0.11893181502819061, + -0.1259741485118866, + -0.20943574607372284, + 0.4765416979789734, + -0.40862777829170227, + -2.8372273445129395, + -1.4697245359420776, + -0.08606886118650436, + -0.06554404646158218, + -0.27424973249435425, + 0.907739520072937, + -0.10130079835653305, + -2.6951611042022705, + -1.807652235031128 + ], + [ + -0.15379676222801208, + -1.7091000080108643, + 0.31153741478919983, + 2.2179949283599854, + 0.859725832939148, + -2.6038026809692383, + 1.98358952999115, + 0.22934654355049133, + -0.16202330589294434, + 0.02916553243994713, + 0.7702362537384033, + -0.0907563641667366, + -2.6495683193206787, + -1.394291877746582, + 0.2534489035606384, + -0.07077484577894211, + 0.10660970956087112, + 0.7866172194480896, + -0.41323983669281006, + -2.4847500324249268, + -1.5881152153015137 + ], + [ + 0.14051103591918945, + -0.8684865236282349, + 0.45214012265205383, + 2.499347686767578, + 0.310441792011261, + -2.875823497772217, + 1.4898443222045898, + 0.3396439850330353, + -0.033532168716192245, + 0.20189781486988068, + 0.3506202697753906, + 0.04831346869468689, + -2.4730942249298096, + -1.4369138479232788, + -0.4840375483036041, + -0.17814871668815613, + -0.1863224357366562, + 0.47034892439842224, + -0.5315045714378357, + -3.019744396209717, + -1.2571722269058228 + ], + [ + 0.5503659844398499, + -1.517655372619629, + -0.20272362232208252, + 2.5588791370391846, + 0.7160627841949463, + -2.551017999649048, + 2.423107385635376, + 0.18953359127044678, + -0.4268995523452759, + 0.014377228915691376, + 0.969605028629303, + -0.2440275251865387, + -2.385143756866455, + -1.4786417484283447, + 0.44389280676841736, + -0.11543146520853043, + -0.5315021872520447, + 1.2627880573272705, + 0.24797001481056213, + -3.17976713180542, + -1.8759949207305908 + ], + [ + -0.07551594823598862, + -1.6101253032684326, + 0.3926098048686981, + 1.726672649383545, + 1.0158472061157227, + -3.367068290710449, + 1.4499058723449707, + -0.5616514086723328, + 0.002434568479657173, + -0.5033466815948486, + -0.12249338626861572, + -0.07335222512483597, + -3.5959439277648926, + -1.0662813186645508, + -0.034436360001564026, + 0.512275218963623, + -0.04822078347206116, + 0.6117934584617615, + -0.745093822479248, + -2.2700355052948, + -2.090803623199463 + ], + [ + 0.6751457452774048, + -0.7857800126075745, + 0.11759950965642929, + 2.7748122215270996, + 1.3096001148223877, + -2.3352341651916504, + 1.4677345752716064, + 0.7219796180725098, + 0.21788907051086426, + -0.3245394825935364, + 0.08372032642364502, + -0.6393120288848877, + -2.8002729415893555, + -2.157836437225342, + 0.11339983344078064, + -0.2834126651287079, + -0.3824191987514496, + 1.130112648010254, + -0.8028184175491333, + -2.3846678733825684, + -1.8259286880493164 + ], + [ + -0.01608770713210106, + -2.1245148181915283, + -0.175740584731102, + 2.754368782043457, + 1.0032098293304443, + -3.7700157165527344, + 1.3539870977401733, + -0.7906507849693298, + -0.8311541080474854, + 0.8180835247039795, + 0.10450628399848938, + 0.01135176420211792, + -3.396463394165039, + -0.6868918538093567, + 0.22346650063991547, + 0.693323016166687, + -0.47104692459106445, + 0.8188157081604004, + -0.6246469020843506, + -2.4006571769714355, + -2.431541919708252 + ], + [ + -0.7931793928146362, + -1.7645766735076904, + 0.23125068843364716, + 2.816321849822998, + 1.1080517768859863, + -3.8820090293884277, + 1.8420056104660034, + 0.4984684884548187, + -0.9149360656738281, + -0.028868291527032852, + 0.14621934294700623, + 0.546608567237854, + -2.7883894443511963, + -1.3650472164154053, + -0.055590901523828506, + 0.41940492391586304, + -0.7235082983970642, + 1.1734092235565186, + -0.6126058101654053, + -3.217271566390991, + -2.512387275695801 + ], + [ + -0.5335642695426941, + -2.139784336090088, + 0.9540140628814697, + 1.7781559228897095, + 1.2117886543273926, + -3.71681809425354, + 2.461766242980957, + 0.4546531140804291, + 0.09786976873874664, + -0.8281612396240234, + -0.19971781969070435, + -1.1042157411575317, + -3.047807216644287, + -1.0697336196899414, + -0.22256404161453247, + -0.3800235986709595, + 0.3603542447090149, + 0.7543611526489258, + -0.23184219002723694, + -3.5156431198120117, + -2.1170644760131836 + ], + [ + 0.27338743209838867, + -0.9654052257537842, + -0.2700650095939636, + 2.6957814693450928, + 0.0643424391746521, + -3.094449281692505, + 0.8981494307518005, + 0.19603312015533447, + 0.5969407558441162, + -0.45200687646865845, + 1.466981291770935, + -0.45456382632255554, + -2.997983455657959, + -1.1903986930847168, + 0.5722758769989014, + -0.9932370781898499, + 0.28074830770492554, + 0.6244409084320068, + 0.6411240696907043, + -3.2179243564605713, + -1.905179738998413 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.025789255276322365, + -0.32600927352905273, + -0.02888326905667782, + 1.2553811073303223, + 0.013092517852783203, + -1.9753923416137695, + 0.0, + 0.04008308798074722, + -0.05201057717204094, + -0.008613351732492447, + -0.06225985288619995, + -0.0397564172744751, + -1.9112331867218018, + 0.0, + -0.02836347185075283, + -0.0055325529538095, + 0.053034864366054535, + 0.009740114212036133, + -0.006228044629096985, + -1.6514825820922852, + 0.0 + ] + ], + "predicted_scores": [ + 6.210387706756592, + 6.031088352203369, + 5.818679332733154, + 5.786194324493408, + 5.449621200561523, + 5.4391560554504395, + 5.10698938369751, + 5.087178707122803 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "77147264", + "selected_index": 0, + "state_hash": "49d38eabb7404e0028c75dd98e1239274b5dc2f04b9db8653001e44369ca8f3a", + "stored_base_utility": 0.838127613067627, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8326728343963623, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8326728343963623 + }, + "base_progress": 0.8326728343963623, + "base_success": false, + "base_utility": 0.8326728343963623, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.814136266708374, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.814136266708374 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.17079035937786102, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.17079035937786102 + }, + { + "contact_quality": null, + "progress": 0.8617495894432068, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8617495894432068 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 0.814136266708374, + 1.0, + 1.0, + 1.0, + 0.17079035937786102, + 0.8617495894432068, + 1.0, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-eb7189a4f2b0f0a5d4bd", + "ms-19a3669e096e98232ec8", + "ms-5dfbb16e173e1a3e2bd6", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-c80e71e55d51d8595499", + "ms-c80e71e55d51d8595499", + "ms-c80e71e55d51d8595499" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + true, + true, + false, + false, + true, + true + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank0" + ], + "candidates_evaluated": true, + "chart_id": "ms-9f5d57fe2153fd267774", + "generated_tangents": [ + [ + -0.036842867732048035, + 0.048847898840904236, + -0.04570475220680237, + 0.2915644943714142, + -0.08123436570167542, + -1.4965012073516846, + -0.01906418800354004, + 0.013378605246543884, + -0.02631569467484951, + -0.012372992932796478, + 0.34770116209983826, + 0.19430376589298248, + -1.3123862743377686, + -0.05775482952594757, + -0.12261678278446198, + 0.06757453083992004, + 0.0445905476808548, + -0.034035637974739075, + -0.2077113389968872, + -0.8378490805625916, + 0.12341295182704926 + ], + [ + 0.11348025500774384, + 0.08447839319705963, + -0.16422703862190247, + 0.258289098739624, + -0.018433362245559692, + -1.6425235271453857, + 0.009867280721664429, + -0.030288204550743103, + -0.07243704795837402, + 0.01182042807340622, + 0.3166850805282593, + 0.15702839195728302, + -1.2816481590270996, + -0.15220917761325836, + -0.11737127602100372, + 0.03544020280241966, + 0.12793458998203278, + 0.006233096122741699, + -0.31687653064727783, + -1.0269509553909302, + 0.160850390791893 + ], + [ + -0.20835839211940765, + -0.10486369580030441, + 0.057728081941604614, + -0.004304111003875732, + 0.23344916105270386, + -1.4573490619659424, + 0.11878395080566406, + 0.019034922122955322, + -0.12262396514415741, + 0.06636328995227814, + 0.4257264733314514, + 0.2640498876571655, + -1.5653103590011597, + 0.10354546457529068, + -0.05625971779227257, + 0.0666455328464508, + 0.2862924635410309, + 0.009381785988807678, + -0.2783547639846802, + -0.9582595229148865, + 0.04166419804096222 + ], + [ + 0.1694633960723877, + 0.08095662295818329, + 0.10999870300292969, + 0.3663580119609833, + -0.18532249331474304, + -1.7100986242294312, + -0.04058218002319336, + 0.01620161533355713, + -0.015664301812648773, + -0.06205829232931137, + 0.31903040409088135, + 0.17578113079071045, + -1.196927547454834, + 0.045018360018730164, + -0.10143482685089111, + 0.03413206711411476, + 0.06631229817867279, + -0.008357241749763489, + -0.1705082654953003, + -0.7727985382080078, + 0.22886992990970612 + ], + [ + 0.17452169954776764, + 0.0699475109577179, + -0.07850918173789978, + 0.16264608502388, + -0.011320486664772034, + -1.5561927556991577, + -0.006917595863342285, + 0.09690427035093307, + -0.03578643500804901, + 0.02951199561357498, + 0.33576828241348267, + 0.07957399636507034, + -1.2511686086654663, + -0.11705876886844635, + -0.10165093839168549, + 0.07081210613250732, + 0.15367388725280762, + 0.00503203272819519, + -0.2053247094154358, + -0.9252009391784668, + 0.16531416773796082 + ], + [ + 0.10311972349882126, + 0.07893295586109161, + -0.0973103940486908, + 0.20181289315223694, + -0.019721180200576782, + -1.6984442472457886, + -0.08015614748001099, + 0.05988536775112152, + -0.06321536004543304, + 0.014984451234340668, + 0.3754003643989563, + 0.11885345727205276, + -1.2185344696044922, + -0.0361989289522171, + -0.07925519347190857, + 0.09786573052406311, + 0.0974055826663971, + 0.04563935101032257, + -0.16137880086898804, + -1.018756628036499, + 0.13988687098026276 + ], + [ + 0.11710644513368607, + 0.0685444176197052, + -0.05941978096961975, + 0.323681503534317, + -0.002460569143295288, + -1.6858038902282715, + -0.21159547567367554, + 0.03656338155269623, + -0.048801787197589874, + 0.06270995736122131, + 0.3533012866973877, + 0.07704123854637146, + -1.2540138959884644, + -0.050995469093322754, + -0.09173603355884552, + 0.12975971400737762, + 0.07895532250404358, + 0.028982236981391907, + -0.11575859785079956, + -0.963138222694397, + 0.07556076347827911 + ], + [ + 0.14770382642745972, + -0.0066012367606163025, + 0.4265309274196625, + 0.13929837942123413, + 0.2602374851703644, + -1.796496868133545, + -0.7083551287651062, + 0.03829829394817352, + -0.15409168601036072, + 0.21598482131958008, + 0.1338273286819458, + -0.04723362624645233, + -1.3035222291946411, + -0.02220889925956726, + 0.02892022579908371, + 0.19045981764793396, + 0.1990739405155182, + 0.17386750876903534, + -0.42094820737838745, + -0.9603549838066101, + 0.04604174196720123 + ] + ], + "generated_utilities": [ + 0.814136266708374, + 2.0, + 2.0, + 2.0, + 0.17079035937786102, + 0.8617495894432068, + 2.0, + 2.0 + ], + "group_id": "ms-9f5d57fe2153fd267774", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8544998168945312, + 2.0, + 0.14427810907363892, + 0.1566883623600006, + 0.7847082018852234, + 0.6161996126174927, + 0.5900160074234009, + 0.12891459465026855, + 0.5811972618103027, + 0.12705880403518677, + 0.12002001702785492, + 0.07508957386016846, + 0.1335657238960266, + 2.0, + 0.5254277586936951, + 0.1602712869644165 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.07957560569047928, + 0.1778574287891388, + -0.06101015955209732, + 0.08770006895065308, + -0.13665570318698883, + -1.1242008209228516, + 0.0, + 0.20265339314937592, + 0.03117954358458519, + -0.08283126354217529, + 0.051615357398986816, + 0.006725620478391647, + -0.7304134368896484, + 0.0, + -0.03756266087293625, + 0.051428791135549545, + 0.13959813117980957, + 0.040047526359558105, + -0.04084143042564392, + -0.31500017642974854, + 0.0 + ], + [ + 0.31541678309440613, + -0.16606420278549194, + 0.09380461275577545, + 0.031209349632263184, + -0.10083025693893433, + -1.1242008209228516, + 0.0, + 0.09663133323192596, + -0.095388263463974, + 0.4012776017189026, + 0.051615357398986816, + 0.167547807097435, + -0.7304134368896484, + 0.0, + -0.6302167177200317, + 0.05524101108312607, + -0.14198890328407288, + 0.040047526359558105, + 0.5201720595359802, + -0.31500017642974854, + 0.0 + ], + [ + 0.004154659807682037, + -0.0034282731357961893, + 0.005892834160476923, + 1.0312093496322632, + -0.15828494727611542, + -2.1242008209228516, + -1.4059314727783203, + 0.002088826149702072, + 0.0016306412871927023, + 0.005124530289322138, + 1.0516153573989868, + -0.05936579033732414, + -1.7304134368896484, + -1.2081255912780762, + 0.010161396116018295, + 0.0017865521367639303, + -0.006222852971404791, + 1.040047526359558, + 0.07346899807453156, + -1.3150001764297485, + -0.9752891659736633 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.8118629455566406, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -2.4162511825561523, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1.9505783319473267 + ], + [ + 0.05903958901762962, + -0.05713164061307907, + 0.27611011266708374, + 0.9773679971694946, + -0.2122824341058731, + -2.198127031326294, + -1.1234419345855713, + 0.22609303891658783, + 0.14432841539382935, + 0.13455358147621155, + 0.8475942611694336, + 0.19007176160812378, + -1.4536203145980835, + -1.208316445350647, + -0.24895474314689636, + 0.08610261976718903, + 0.16216596961021423, + 1.2356702089309692, + 0.07108174264431, + -1.4601069688796997, + -1.0005295276641846 + ], + [ + -0.2862337827682495, + -0.34814661741256714, + -0.3317800462245941, + 1.383265495300293, + -0.3485583961009979, + -1.7799506187438965, + -1.6264256238937378, + -0.24356596171855927, + -0.18337340652942657, + 0.3351142108440399, + 1.36115562915802, + -0.4404119849205017, + -1.4858057498931885, + -0.9229021668434143, + 0.3078569769859314, + -0.18743832409381866, + -0.3414495885372162, + 1.0045359134674072, + -0.1438894122838974, + -1.5133963823318481, + -0.7331042885780334 + ], + [ + 0.14600041508674622, + -0.10555070638656616, + 0.06090353801846504, + 1.1715545654296875, + 0.14829085767269135, + -1.8377182483673096, + -1.2019046545028687, + -0.33082208037376404, + 0.27619102597236633, + 0.3936208188533783, + 1.4077775478363037, + -0.2165551781654358, + -2.094696283340454, + -0.7843748331069946, + -0.13239514827728271, + 0.208020880818367, + 0.17500364780426025, + 0.5608562231063843, + 0.5641188025474548, + -1.6192113161087036, + -1.1969176530838013 + ], + [ + -0.03659747168421745, + -0.2400680035352707, + 0.49718841910362244, + 1.0422266721725464, + 0.13088186085224152, + -2.068427324295044, + -1.1215412616729736, + 0.06697356700897217, + 0.19986751675605774, + -0.5827334523200989, + 0.9792195558547974, + -0.5810520648956299, + -1.7176907062530518, + -0.6760571002960205, + -0.08971267938613892, + 0.36716267466545105, + 0.4829793870449066, + 0.7045803070068359, + -0.3341261148452759, + -1.3750282526016235, + -1.086517095565796 + ], + [ + 0.09393255412578583, + 0.6277099251747131, + 0.653377115726471, + 0.5602235198020935, + -0.8483365178108215, + -1.7264833450317383, + -1.698084831237793, + -0.3303340971469879, + 0.43070027232170105, + -0.13159427046775818, + 1.0274523496627808, + 0.4760425388813019, + -2.1360630989074707, + -0.6506720185279846, + 0.3130929470062256, + 0.6801184415817261, + 0.6575731039047241, + 0.9053056240081787, + 0.18805859982967377, + -1.3225764036178589, + -1.0653810501098633 + ], + [ + 0.536294162273407, + -0.026106586679816246, + 0.11593607813119888, + 0.2775042653083801, + 0.1801595836877823, + -1.590940237045288, + -0.6846873164176941, + 0.5080465078353882, + -0.7678324580192566, + 0.24115782976150513, + 0.2675415277481079, + -0.3879689872264862, + -2.1869571208953857, + -0.7176487445831299, + -0.15474224090576172, + -0.5326727032661438, + -0.12406099587678909, + 0.411626398563385, + 0.3519209027290344, + -1.6924495697021484, + -1.4648362398147583 + ], + [ + 0.2688097357749939, + -0.7928301095962524, + -0.39608004689216614, + 1.3961963653564453, + -0.026196151971817017, + -2.263073682785034, + -2.044499635696411, + 0.055598195642232895, + -0.485431969165802, + -0.058985911309719086, + 1.781836986541748, + 0.7104123830795288, + -2.0781984329223633, + -0.6968748569488525, + -0.1437186300754547, + 0.8155919313430786, + 0.5188283920288086, + 0.23954623937606812, + -0.6596511602401733, + -1.3867920637130737, + -0.7448391318321228 + ], + [ + 0.11259422451257706, + 0.43877506256103516, + -0.585435152053833, + 0.12099790573120117, + 0.5070493817329407, + -2.101219654083252, + -1.3759371042251587, + -0.7516231536865234, + 0.7018498778343201, + 0.7899036407470703, + 0.9772143959999084, + -0.7032701373100281, + -1.4699928760528564, + -1.4471495151519775, + 0.4750489890575409, + 0.8803708553314209, + -0.25032687187194824, + 1.3429648876190186, + 0.7985087037086487, + -0.5786607265472412, + -1.563289761543274 + ], + [ + 0.4965847432613373, + -0.8452511429786682, + 0.21300016343593597, + 0.28797537088394165, + -1.0059970617294312, + -3.040114402770996, + -1.1050269603729248, + 0.7826927304267883, + 0.8645586371421814, + -0.9447845816612244, + 1.0121192932128906, + 0.5123239755630493, + -1.5470938682556152, + -0.6170141100883484, + 0.2538997530937195, + 0.912939727306366, + -0.8613666892051697, + 0.9517492055892944, + 0.7185115218162537, + -2.1534924507141113, + -0.6818757057189941 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.025899745523929596, + -0.03433045744895935, + -0.044045113027095795, + 0.031209349632263184, + -0.014532476663589478, + -1.1242008209228516, + 0.0, + 0.10768167674541473, + -0.020356399938464165, + 0.09216367453336716, + 0.051615357398986816, + -0.011370409280061722, + -0.7304134368896484, + 0.0, + 0.04558054730296135, + 0.038496289402246475, + 0.02504396252334118, + 0.040047526359558105, + 0.01217309758067131, + -0.31500017642974854, + 0.0 + ], + [ + -0.5198070406913757, + -0.8798871040344238, + 0.04596509411931038, + 1.55931556224823, + 0.31027060747146606, + -1.5283797979354858, + -1.6053413152694702, + -0.2754230797290802, + -0.012456716038286686, + 0.7998818159103394, + 1.5184144973754883, + 0.05059843137860298, + -1.85283362865448, + -0.9533926248550415, + 0.9978112578392029, + 0.8525571227073669, + -0.9611027836799622, + 0.32179969549179077, + -0.448661744594574, + -1.9452965259552002, + -0.7132223844528198 + ] + ], + "predicted_scores": [ + 5.260086536407471, + 5.183110237121582, + 5.1446919441223145, + 4.91237735748291, + 4.682591438293457, + 4.640739917755127, + 4.59874963760376, + 3.972043991088867 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1607773860", + "selected_index": 0, + "state_hash": "fdc7379ab4491c9f65860cadd3b2f3c9f2335f26aae91fb1c06d3a4edf858d6a", + "stored_base_utility": 0.8544998168945312, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8272150158882141, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8272150158882141 + }, + "base_progress": 0.8272150158882141, + "base_success": false, + "base_utility": 0.8272150158882141, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.15089993178844452, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15089993178844452 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.8353422284126282, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8353422284126282 + }, + { + "contact_quality": null, + "progress": 0.13203097879886627, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13203097879886627 + }, + { + "contact_quality": null, + "progress": 0.13766705989837646, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13766705989837646 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.19075119495391846, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19075119495391846 + } + ], + "candidate_progress": [ + 1.0, + 0.15089993178844452, + 1.0, + 0.8353422284126282, + 0.13203097879886627, + 0.13766705989837646, + 1.0, + 0.19075119495391846 + ], + "candidate_source_chart_ids": [ + "ms-19a3669e096e98232ec8", + "ms-19a3669e096e98232ec8", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-20f3cfcf183c69853b1e", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-f3ec6a4e6e3a778054d1", + "ms-0949097d19e597fc7f63", + "ms-f3ec6a4e6e3a778054d1" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + true, + false, + false, + false, + true, + false + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank2", + "ctt_transport_rank4" + ], + "candidates_evaluated": true, + "chart_id": "ms-b0fee26e1db967f90b81", + "generated_tangents": [ + [ + 0.08246950805187225, + 0.08750855922698975, + -0.0660984218120575, + -0.06540507078170776, + 0.1978612244129181, + -2.1107535362243652, + -0.23482418060302734, + -0.04672335088253021, + -0.0629846528172493, + 0.06864479184150696, + 0.22597938776016235, + 0.20545238256454468, + -1.647207260131836, + -0.30052053928375244, + 0.02503054589033127, + 0.005656566470861435, + 0.13866335153579712, + -0.0044862329959869385, + -0.06137773394584656, + -1.3930904865264893, + -0.24488231539726257 + ], + [ + 0.1859014928340912, + 0.07491914927959442, + 0.09269857406616211, + 0.003977775573730469, + 0.1321168690919876, + -2.0118014812469482, + -0.3092394471168518, + -0.023855969309806824, + -0.0012620426714420319, + 0.05011509358882904, + 0.2728317379951477, + 0.33122509717941284, + -1.6019604206085205, + -0.24240286648273468, + 0.014448247849941254, + -0.020438531413674355, + 0.09186503291130066, + 0.026910707354545593, + 0.07068610191345215, + -1.2765593528747559, + -0.21925383806228638 + ], + [ + -0.006058543920516968, + 0.06651297211647034, + 0.04328954219818115, + -0.06965845823287964, + 0.15326642990112305, + -1.8159501552581787, + -0.21902137994766235, + -0.007013410329818726, + -0.004383314400911331, + 0.07494658976793289, + 0.3194279372692108, + 0.28470128774642944, + -1.6724324226379395, + -0.16556812822818756, + -0.01442759856581688, + 0.02155320718884468, + 0.05166323482990265, + -0.026102766394615173, + -0.0037977099418640137, + -1.270317792892456, + -0.26662954688072205 + ], + [ + 0.0936608612537384, + 0.07955317199230194, + -0.11822423338890076, + -0.004888221621513367, + 0.23157638311386108, + -2.067514419555664, + -0.30729854106903076, + -0.0008642375469207764, + -0.05714868754148483, + 0.11828251928091049, + 0.244703471660614, + 0.11098344624042511, + -1.8146916627883911, + -0.19908224046230316, + 0.024550646543502808, + 0.004102073609828949, + 0.0627811849117279, + 0.007633984088897705, + -0.07723230123519897, + -1.5077738761901855, + -0.2014407217502594 + ], + [ + 0.14827285706996918, + 0.060327932238578796, + 0.03986567258834839, + -0.12716135382652283, + 0.23965947329998016, + -1.939994215965271, + -0.22944331169128418, + 0.10510952025651932, + -0.034818679094314575, + 0.09262728691101074, + 0.229600727558136, + 0.19778873026371002, + -1.5633316040039062, + -0.2708389163017273, + 0.017550159245729446, + 0.037912219762802124, + 0.19822438061237335, + -0.023131489753723145, + 0.037082210183143616, + -1.2809975147247314, + -0.22135639190673828 + ], + [ + -0.029288768768310547, + 0.03452843427658081, + 0.018285095691680908, + -0.0342509001493454, + 0.1954054832458496, + -1.9721804857254028, + -0.22642755508422852, + 0.1918209195137024, + -0.085311159491539, + 0.08068110048770905, + 0.0671565979719162, + 0.22854061424732208, + -1.9288711547851562, + -0.13851411640644073, + -0.023961801081895828, + 0.058769483119249344, + 0.22576399147510529, + -0.0660112053155899, + -0.08861735463142395, + -1.6937940120697021, + -0.22313708066940308 + ], + [ + -0.06338757276535034, + 0.062273234128952026, + 0.0781092643737793, + -0.04689791798591614, + 0.18885481357574463, + -2.0480566024780273, + -0.22808384895324707, + 0.23956647515296936, + -0.11650949716567993, + 0.10949895530939102, + 0.09395516663789749, + 0.1618867814540863, + -1.806831955909729, + 0.07582738995552063, + -0.018856864422559738, + 0.06067037954926491, + 0.20329849421977997, + 0.01653990149497986, + -0.04835501313209534, + -1.5202925205230713, + -0.15853941440582275 + ], + [ + 0.221816748380661, + -0.011786513030529022, + 0.03290960192680359, + -0.2630120515823364, + 0.23217853903770447, + -1.3343830108642578, + -0.6715918183326721, + -0.0034032315015792847, + -0.09411638230085373, + -0.048669032752513885, + 0.21319153904914856, + 0.1285199522972107, + -1.395736813545227, + -0.3864459991455078, + -0.0024505741894245148, + 0.1499291956424713, + 0.10702880471944809, + 0.00843365490436554, + -0.1614212691783905, + -1.4488356113433838, + -0.13157981634140015 + ] + ], + "generated_utilities": [ + 2.0, + 0.15089993178844452, + 2.0, + 0.8353422284126282, + 0.13203097879886627, + 0.13766705989837646, + 2.0, + 0.19075119495391846 + ], + "group_id": "ms-b0fee26e1db967f90b81", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8397466540336609, + 0.8594310283660889, + 0.14638642966747284, + 0.09418028593063354, + 0.7838175296783447, + 0.5982922315597534, + 0.5913184881210327, + 2.0, + 0.20996521413326263, + 0.17419946193695068, + 0.11757439374923706, + 0.1645697057247162, + 0.05191062390804291, + 0.08658662438392639, + 0.04989185929298401, + 0.05867189168930054 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.1510201096534729, + -0.21724483370780945, + 0.14240525662899017, + 0.003108598291873932, + -0.11609727144241333, + -1.6516358852386475, + 0.0, + -0.09976008534431458, + 0.061503566801548004, + 0.3070257902145386, + -0.08051937818527222, + 0.05735237896442413, + -1.6075372695922852, + 0.0, + -0.19649454951286316, + -0.15815469622612, + 0.2621642053127289, + -0.17578476667404175, + -0.034231506288051605, + -1.3808586597442627, + 0.0 + ], + [ + 0.014505341649055481, + 0.0403006337583065, + 0.5529788732528687, + 0.0714028924703598, + -0.41683781147003174, + -1.6516358852386475, + 0.0, + 0.41446545720100403, + -0.16278114914894104, + -0.4971221685409546, + -0.3640303611755371, + 0.02483367919921875, + -1.6075372695922852, + 0.0, + 0.5711643695831299, + -0.5244585275650024, + -0.5069849491119385, + 0.31709733605384827, + -0.4254770278930664, + -1.3808586597442627, + 0.0 + ], + [ + -0.044587142765522, + 0.03136333078145981, + 0.014485275372862816, + 0.04301099479198456, + 0.47328460216522217, + -2.6516358852386475, + -0.32303211092948914, + -0.01855158433318138, + -0.003608003491535783, + 0.029763570055365562, + 0.6359696388244629, + -0.1304762363433838, + -2.607537269592285, + -1.7957698106765747, + -0.009414590895175934, + -0.000939749414101243, + 0.018994441255927086, + 0.7486392855644226, + -0.06401278078556061, + -2.3808586597442627, + -1.722450852394104 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -0.6460642218589783, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.5915396213531494, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.444901704788208 + ], + [ + 0.18585285544395447, + 0.17557260394096375, + 0.08478302508592606, + -0.17644454538822174, + 0.7477058172225952, + -2.614673137664795, + -0.3018328547477722, + 0.2204062044620514, + -0.0497702918946743, + 0.1961878538131714, + 0.5304322242736816, + -0.15872131288051605, + -2.604066848754883, + -2.053436517715454, + 0.18444111943244934, + -0.0024187257513403893, + 0.0054171085357666016, + 1.0256288051605225, + 0.02724038064479828, + -2.426236391067505, + -1.6594898700714111 + ], + [ + -0.5335133075714111, + 0.12298225611448288, + -0.12197553366422653, + 0.0424814410507679, + 0.002416938543319702, + -2.8428826332092285, + -0.17827168107032776, + -0.3377644121646881, + 0.04258471727371216, + -0.15910103917121887, + 0.9169985055923462, + 0.003966793417930603, + -3.069180727005005, + -1.407257318496704, + -0.47542956471443176, + -0.2210284322500229, + -0.10021048784255981, + 0.7806655168533325, + 0.3228837251663208, + -2.058253049850464, + -1.556677222251892 + ], + [ + 0.20687156915664673, + -0.5511468052864075, + 0.3945963978767395, + -0.16104432940483093, + 0.4936041831970215, + -2.1271684169769287, + 0.02855244278907776, + 0.544975996017456, + -0.3336343765258789, + 0.029475202783942223, + 0.8120307326316833, + -0.3246060609817505, + -2.885145425796509, + -2.2030622959136963, + -0.28730231523513794, + 0.19937433302402496, + 0.19840951263904572, + 1.1192030906677246, + -0.3995746374130249, + -2.5594136714935303, + -1.1545796394348145 + ], + [ + -0.5497553944587708, + -0.39125651121139526, + 0.2725587487220764, + -0.27965104579925537, + 1.0584466457366943, + -2.1044363975524902, + -0.6898342370986938, + 0.36552220582962036, + -0.2289373278617859, + 0.24627351760864258, + 0.7968363165855408, + 0.14776766300201416, + -2.988917112350464, + -1.6111457347869873, + -0.5116392374038696, + 0.030796781182289124, + -0.08262427151203156, + 1.0816104412078857, + 0.336497962474823, + -1.684936285018921, + -2.290850877761841 + ], + [ + 0.18322327733039856, + 0.6938241720199585, + 0.5252001881599426, + 0.06770005077123642, + 1.120344877243042, + -1.8558356761932373, + 0.18930694460868835, + -0.7033196687698364, + 0.7861698865890503, + -0.0965423658490181, + 0.015094935894012451, + -0.5303559303283691, + -2.6036770343780518, + -2.589250087738037, + -0.468231737613678, + -0.5901691317558289, + -0.7109078764915466, + 0.986366331577301, + -0.7634903192520142, + -2.984590530395508, + -1.1651530265808105 + ], + [ + 0.7039692401885986, + -0.5346179008483887, + 0.1466517597436905, + -0.6284777522087097, + -0.07994896173477173, + -2.4640438556671143, + -1.102164626121521, + -0.26685386896133423, + -0.3165925443172455, + 0.8653677701950073, + 0.232782244682312, + -0.5054740905761719, + -3.400317668914795, + -1.4077497720718384, + -0.09987740218639374, + -0.27442091703414917, + 0.5428174138069153, + 1.5128605365753174, + 0.747156023979187, + -2.0367631912231445, + -2.464085578918457 + ], + [ + 0.4474923610687256, + -0.7824500203132629, + 0.5928975343704224, + 0.57558673620224, + -0.17630118131637573, + -2.176387310028076, + -0.3299017548561096, + -0.6172446608543396, + 0.5065999627113342, + 0.46063295006752014, + 0.5951727628707886, + -0.02366364747285843, + -3.254279136657715, + -0.8880517482757568, + 0.3714323043823242, + 0.12449074536561966, + 0.6677311658859253, + -0.20713549852371216, + 0.5062628984451294, + -3.2156763076782227, + -1.9652986526489258 + ], + [ + -0.437069296836853, + 0.615723192691803, + -0.7743021249771118, + 0.20228458940982819, + 0.4320293366909027, + -2.0403428077697754, + -0.32251936197280884, + -0.7280393838882446, + 0.10878247022628784, + 0.9395551681518555, + 0.6021312475204468, + -0.3748174011707306, + -2.0175063610076904, + -1.9166245460510254, + -0.10793138295412064, + 0.6512030959129333, + 0.3747653365135193, + 1.5698660612106323, + -0.7995363473892212, + -1.8378328084945679, + -1.0564453601837158 + ], + [ + -0.7214477062225342, + -0.5686705708503723, + 0.8819299936294556, + 0.7319017648696899, + 1.0695421695709229, + -2.0338172912597656, + 0.2973436415195465, + -0.19576850533485413, + 0.4506514370441437, + 0.9037411212921143, + 0.0996239185333252, + 0.5076434016227722, + -2.611851930618286, + -2.7078182697296143, + 0.022706281393766403, + -0.4812851846218109, + -0.40778082609176636, + 1.6509652137756348, + -0.5721896886825562, + -1.8604621887207031, + -1.2943172454833984 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.017527949064970016, + -0.13547566533088684, + -0.31396082043647766, + -0.06045208126306534, + 0.7718918323516846, + -2.996134042739868, + -0.0199548602104187, + 0.26194432377815247, + -0.303554505109787, + -0.09068091958761215, + 0.3007984459400177, + -0.39815279841423035, + -2.626121997833252, + -1.8776893615722656, + 0.25959014892578125, + 0.20619672536849976, + -0.25346171855926514, + 0.40209779143333435, + 0.1943899542093277, + -2.464694023132324, + -1.5338099002838135 + ] + ], + "predicted_scores": [ + 5.351569652557373, + 5.150033950805664, + 5.108182430267334, + 4.922300815582275, + 4.596928596496582, + 4.560333728790283, + 4.243762969970703, + 4.0818963050842285 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1358421325", + "selected_index": 0, + "state_hash": "b8e04d64f31138fff8f83f1bb017f9e8515565a2ada2ba5787ad5f06b796589c", + "stored_base_utility": 0.8397466540336609, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8307622671127319, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8307622671127319 + }, + "base_progress": 0.8307622671127319, + "base_success": false, + "base_utility": 0.8307622671127319, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.1599900722503662, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1599900722503662 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.821384847164154, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.821384847164154 + }, + { + "contact_quality": null, + "progress": 0.15789160132408142, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15789160132408142 + }, + { + "contact_quality": null, + "progress": 0.15891481935977936, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.15891481935977936 + }, + { + "contact_quality": null, + "progress": 0.16026467084884644, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.16026467084884644 + }, + { + "contact_quality": null, + "progress": 0.20434951782226562, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20434951782226562 + }, + { + "contact_quality": null, + "progress": 0.19699636101722717, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19699636101722717 + } + ], + "candidate_progress": [ + 0.1599900722503662, + 1.0, + 0.821384847164154, + 0.15789160132408142, + 0.15891481935977936, + 0.16026467084884644, + 0.20434951782226562, + 0.19699636101722717 + ], + "candidate_source_chart_ids": [ + "ms-01e6eaaaf40875e0ef5a", + "ms-5bddb39f792f20e5e6d4", + "ms-5a32c809ed679663dcb1", + "ms-01e6eaaaf40875e0ef5a", + "ms-852c671b0adeccfa4bf1", + "ms-5a32c809ed679663dcb1", + "ms-5bddb39f792f20e5e6d4", + "ms-5bddb39f792f20e5e6d4" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + true, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank3", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "generated_tangents": [ + [ + 0.16271671652793884, + 0.11835204064846039, + -0.012397438287734985, + -0.1449543535709381, + 0.6666361093521118, + -0.0610959529876709, + 0.05983695387840271, + -0.12314370274543762, + 0.05490633845329285, + 0.06102598458528519, + 0.011549975723028183, + 0.06348592042922974, + -1.7331287860870361, + -0.06759564578533173, + -0.01418175920844078, + 0.08610397577285767, + 0.02035468816757202, + -0.13417764008045197, + -0.08576458692550659, + -1.360227346420288, + 0.07081732898950577 + ], + [ + 0.05573435127735138, + 0.08733311295509338, + 0.0460299551486969, + -0.14449870586395264, + 0.6969571113586426, + 0.11252725124359131, + 0.14553365111351013, + -0.08895868062973022, + 0.08528045564889908, + 0.09448707103729248, + 0.019099678844213486, + 0.09892076998949051, + -1.745173692703247, + -0.02503219246864319, + 0.040615588426589966, + 0.15552745759487152, + 0.03870461881160736, + -0.1297275274991989, + -0.12249332666397095, + -1.3522653579711914, + -0.015355393290519714 + ], + [ + 0.05131131410598755, + 0.09055924415588379, + -0.004006803035736084, + -0.12880590558052063, + 0.5213919878005981, + 0.02825668454170227, + 0.0794881284236908, + 0.025297001004219055, + 0.07260261476039886, + 0.053302377462387085, + 0.018651578575372696, + 0.04935432970523834, + -1.71018648147583, + 0.11532102525234222, + 0.06491315364837646, + 0.03935879096388817, + 0.04305471479892731, + -0.029442936182022095, + -0.09051260352134705, + -1.34885835647583, + 0.13415014743804932 + ], + [ + 0.16039256751537323, + 0.10715499520301819, + -1.5407800674438477e-05, + -0.08977057039737701, + 0.6557028293609619, + -0.02549922466278076, + -0.05239427089691162, + -0.050411373376846313, + 0.03698960691690445, + 0.07752439379692078, + 0.015422597527503967, + -0.06018456816673279, + -1.77348792552948, + -0.09620897471904755, + -0.12304249405860901, + 0.08278532326221466, + -0.023905202746391296, + -0.11396853625774384, + 0.012908339500427246, + -1.378190279006958, + 0.15033148229122162 + ], + [ + 0.1748061180114746, + 0.12763012945652008, + -0.036204904317855835, + -0.22588416934013367, + 0.6186838150024414, + -0.1957300901412964, + -0.12827563285827637, + 0.015215441584587097, + -0.04022820293903351, + 0.07155010104179382, + 0.0013560131192207336, + 0.015266939997673035, + -1.793965458869934, + -0.012374207377433777, + -0.03334435448050499, + 0.11777634918689728, + 0.11445989459753036, + -0.07886438071727753, + -0.22546911239624023, + -1.4198358058929443, + 0.013972818851470947 + ], + [ + -0.04156471788883209, + 0.19573041796684265, + -0.17388007044792175, + -0.13879147171974182, + 0.6510802507400513, + 0.014952152967453003, + -0.06441938877105713, + 0.05719183385372162, + -0.057870566844940186, + 0.04107788950204849, + 0.045885320752859116, + -0.08660179376602173, + -1.7200990915298462, + -0.02635994553565979, + -0.08880971372127533, + 0.11992290616035461, + 0.2511446177959442, + -0.3118141293525696, + -0.258758008480072, + -1.4797537326812744, + 0.0482817143201828 + ], + [ + -0.0361819714307785, + -0.04947558790445328, + -0.37064018845558167, + -0.02005597949028015, + 0.7335584759712219, + -0.15239670872688293, + 0.5580345988273621, + -0.013057202100753784, + -0.06334485113620758, + -0.06316176801919937, + -0.050221458077430725, + -0.04865361750125885, + -1.955536961555481, + -0.15965412557125092, + -0.029360156506299973, + 0.17591671645641327, + 0.17993104457855225, + -0.18448616564273834, + -0.30352967977523804, + -1.5083882808685303, + -0.28336772322654724 + ], + [ + -0.061140745878219604, + -0.06449203193187714, + -0.28942009806632996, + 0.10007861256599426, + 0.7646355032920837, + 0.012566015124320984, + 0.4356676936149597, + 0.00030077993869781494, + 0.018040115013718605, + -0.05286494642496109, + -0.06451456248760223, + -0.1296241283416748, + -2.231977939605713, + -0.3048020601272583, + 0.020629890263080597, + 0.24416236579418182, + 0.22915402054786682, + -0.26925599575042725, + -0.2489616870880127, + -1.3599307537078857, + -0.06671296060085297 + ] + ], + "generated_utilities": [ + 0.1599900722503662, + 2.0, + 0.821384847164154, + 0.15789160132408142, + 0.15891481935977936, + 0.16026467084884644, + 0.20434951782226562, + 0.19699636101722717 + ], + "group_id": "ms-bc2d3b425d9163a62dbb", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8266608119010925, + 2.0, + 0.15905821323394775, + 0.13648143410682678, + 0.5621776580810547, + 0.18381908535957336, + 0.20885342359542847, + 0.18184903264045715, + 0.19253113865852356, + 0.5574395656585693, + 0.10362428426742554, + 0.0659959614276886, + 0.09997051954269409, + 0.2210000902414322, + 0.00010728836059570312, + 0.12723565101623535 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.059883326292037964, + -0.16710972785949707, + -0.04027438163757324, + -0.23492489755153656, + 0.34960758686065674, + 0.16551673412322998, + 0.0, + -0.27484941482543945, + 0.07304387539625168, + -0.38491085171699524, + 0.03056102991104126, + 0.16416949033737183, + -1.5270280838012695, + 0.0, + 0.01279987208545208, + -0.07740817964076996, + 0.07519636303186417, + 0.14440757036209106, + 0.0022157281637191772, + -1.3374013900756836, + 0.0 + ], + [ + -0.6122239828109741, + -0.0734642744064331, + -0.4064018726348877, + 0.11902926862239838, + 0.34960758686065674, + -0.07764682173728943, + 0.0, + 0.37407442927360535, + -0.08740249276161194, + -0.20959049463272095, + -0.37996697425842285, + -0.304497092962265, + -1.5270280838012695, + 0.0, + -0.9113190174102783, + 0.3964778184890747, + 0.4811243414878845, + 0.2385387420654297, + -0.11476142704486847, + -1.3374013900756836, + 0.0 + ], + [ + 0.3877759873867035, + 0.6924911141395569, + -1.0402743816375732, + 0.005685340613126755, + 1.3496075868606567, + -0.2760518193244934, + 1.2840001583099365, + -0.028063178062438965, + 0.0023769063409417868, + 0.03187502175569534, + 0.04654876887798309, + 0.3263346254825592, + -2.5270280838012695, + -1.7457892894744873, + -0.012772172689437866, + -0.008647367358207703, + 0.019657021388411522, + 0.3050904870033264, + 0.19037143886089325, + -2.3374013900756836, + -1.738369345664978 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.568000316619873, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.4915785789489746, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.476738691329956 + ], + [ + 0.1989155411720276, + 0.7169155478477478, + -0.9921301603317261, + 0.22762086987495422, + 1.516188144683838, + -0.23096826672554016, + 1.0414973497390747, + 0.0012522991746664047, + -0.0518387071788311, + -0.23540246486663818, + 0.08515512943267822, + 0.5060473084449768, + -2.5231378078460693, + -1.4593229293823242, + -0.09671962261199951, + 0.09134317189455032, + -0.07944301515817642, + 0.18403387069702148, + 0.2437569499015808, + -2.270580291748047, + -1.9640144109725952 + ], + [ + 0.256519615650177, + 0.7805030941963196, + -1.3583617210388184, + 0.3310847878456116, + 1.2735480070114136, + -0.4127388000488281, + 1.2665152549743652, + -0.19997301697731018, + -0.11129427701234818, + 0.1647017002105713, + 0.3767858147621155, + 0.4831879138946533, + -2.752143621444702, + -1.6790910959243774, + 0.2554209232330322, + 0.3251851797103882, + -0.12605378031730652, + 0.696884274482727, + 0.29535919427871704, + -2.726069688796997, + -1.704551100730896 + ], + [ + 0.49772778153419495, + 1.0026791095733643, + -0.5513590574264526, + 0.008379313163459301, + 1.7768750190734863, + -0.6102496981620789, + 1.7048192024230957, + -0.29564282298088074, + -0.20420490205287933, + -0.23934513330459595, + -0.34791648387908936, + 0.0690477192401886, + -2.602295398712158, + -1.8138929605484009, + -0.38315677642822266, + -0.16421082615852356, + 0.17502839863300323, + -0.13996875286102295, + -0.30929630994796753, + -2.544666290283203, + -1.2411268949508667 + ], + [ + 0.07701912522315979, + 0.9529868364334106, + -1.497549295425415, + -0.3016125559806824, + 1.8504154682159424, + -0.3136579394340515, + 1.6619524955749512, + -0.28860214352607727, + 0.20993559062480927, + 0.6072397232055664, + 0.04243222624063492, + -0.133962482213974, + -2.0948100090026855, + -2.1691184043884277, + 0.44940999150276184, + -0.4183289408683777, + 0.5662978291511536, + 0.5016863942146301, + 0.7054548263549805, + -2.282663345336914, + -1.8892805576324463 + ], + [ + -0.2981111705303192, + 0.038420140743255615, + -1.687281847000122, + -0.37934571504592896, + 1.1300733089447021, + -0.6208293437957764, + 1.8144867420196533, + 0.16701459884643555, + 0.5508663654327393, + 0.4797254800796509, + -0.09421360492706299, + -0.3499920070171356, + -2.5030596256256104, + -1.5871793031692505, + -0.33045580983161926, + -0.29071635007858276, + 0.5385350584983826, + 0.9102991223335266, + 0.3121117353439331, + -2.2699851989746094, + -2.081907272338867 + ], + [ + -0.28236815333366394, + 0.18060839176177979, + -0.32270634174346924, + 0.033799149096012115, + 1.128151774406433, + -0.7577929496765137, + 1.6937967538833618, + -0.813525378704071, + 0.48178502917289734, + -0.6699209809303284, + -0.03810972720384598, + 0.8273124694824219, + -2.2768704891204834, + -1.317199945449829, + 0.0046694036573171616, + 0.5900158286094666, + -0.12453534454107285, + 1.046316385269165, + 0.2770566940307617, + -2.0448076725006104, + -2.4180057048797607 + ], + [ + 0.26116815209388733, + 1.305420994758606, + -0.5357515811920166, + 0.6523149013519287, + 0.5250536203384399, + -0.6821175217628479, + 1.2918037176132202, + 0.3275931477546692, + -0.8845832943916321, + -0.6987056136131287, + 0.10023711621761322, + -0.2849085032939911, + -1.7303776741027832, + -1.0117536783218384, + -0.06847482174634933, + 0.6783800721168518, + -0.04633811116218567, + 0.0702061653137207, + -0.3159322142601013, + -3.004882335662842, + -2.0691566467285156 + ], + [ + 0.5418367385864258, + 1.1724635362625122, + -1.717900276184082, + 0.9217556118965149, + 2.316706657409668, + -0.2411489486694336, + 0.720655083656311, + -0.6330894231796265, + 0.03703225776553154, + 0.38938266038894653, + -0.8305367827415466, + -0.2152116596698761, + -2.614929437637329, + -2.045497179031372, + -0.9044661521911621, + 0.9391879439353943, + -0.2323634922504425, + 0.8392186164855957, + -0.6889055371284485, + -3.0831263065338135, + -1.0035808086395264 + ], + [ + 1.0110303163528442, + 1.0131109952926636, + -1.3200907707214355, + -0.433234840631485, + 1.9466626644134521, + 0.46447479724884033, + 1.528633952140808, + 0.40949776768684387, + 0.07322453707456589, + -0.12349443882703781, + 0.24514153599739075, + 0.005379408597946167, + -2.487516164779663, + -1.1890358924865723, + -0.026332437992095947, + -0.6102214455604553, + 0.6559669971466064, + 0.4371780753135681, + -0.6651540994644165, + -2.8161611557006836, + -1.4271454811096191 + ], + [ + 0.9640108346939087, + 1.0122193098068237, + -1.9809117317199707, + 0.40601760149002075, + 0.9296187162399292, + 0.601173460483551, + 2.229055643081665, + -0.4972975254058838, + 0.6548057794570923, + 0.6165774464607239, + -0.7309268116950989, + 0.697066068649292, + -3.0948128700256348, + -2.4828031063079834, + 0.16930773854255676, + -0.11851751059293747, + -0.8277174830436707, + 0.8252183794975281, + -0.4829612374305725, + -2.922985076904297, + -1.1981642246246338 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + 0.03792491555213928, + -0.040427446365356445, + -0.04027438163757324, + 0.02325565740466118, + 0.34960758686065674, + -0.05261008441448212, + 0.0, + 0.018791399896144867, + 0.05303370952606201, + 0.01128687709569931, + 0.01692008785903454, + 0.06139722466468811, + -1.5270280838012695, + 0.0, + -0.004229618236422539, + 0.017936373129487038, + 0.06901376694440842, + -0.04424762725830078, + -0.033029720187187195, + -1.3374013900756836, + 0.0 + ] + ], + "predicted_scores": [ + 7.299339294433594, + 7.067145824432373, + 7.013951301574707, + 6.319223880767822, + 5.728745937347412, + 4.369734287261963, + 4.050492286682129, + 3.6663787364959717 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1271718007", + "selected_index": 0, + "state_hash": "58b3d0c1cafc199ad58508912becbaef59038a655cef0caafd263ade5c9b53fa", + "stored_base_utility": 0.8266608119010925, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.20032641291618347, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20032641291618347 + }, + "base_progress": 0.20032641291618347, + "base_success": false, + "base_utility": 0.20032641291618347, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.19381189346313477, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.19381189346313477 + }, + { + "contact_quality": null, + "progress": 0.23963995277881622, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.23963995277881622 + }, + { + "contact_quality": null, + "progress": 0.21024136245250702, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21024136245250702 + }, + { + "contact_quality": null, + "progress": 0.13467177748680115, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.13467177748680115 + }, + { + "contact_quality": null, + "progress": 0.2017955482006073, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.2017955482006073 + }, + { + "contact_quality": null, + "progress": 0.20696409046649933, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.20696409046649933 + }, + { + "contact_quality": null, + "progress": 0.1936945617198944, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1936945617198944 + }, + { + "contact_quality": null, + "progress": 0.21807821094989777, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.21807821094989777 + } + ], + "candidate_progress": [ + 0.19381189346313477, + 0.23963995277881622, + 0.21024136245250702, + 0.13467177748680115, + 0.2017955482006073, + 0.20696409046649933, + 0.1936945617198944, + 0.21807821094989777 + ], + "candidate_source_chart_ids": [ + "ms-5ac1bf2d7d319955d45d", + "ms-95028134c6e3670b89d5", + "ms-b8492a3a812095651d75", + "ms-ee2fcbc94d41385713d0", + "ms-95028134c6e3670b89d5", + "ms-9ae5c6c4132218ecf49e", + "ms-016c5de39ed3c111889a", + "ms-c8706d0aa14a930236de" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank1", + "ctt_transport_rank6", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank6", + "ctt_transport_rank3", + "ctt_transport_rank4", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-d1960368ef76bbb10669", + "generated_tangents": [ + [ + 0.19640028476715088, + 0.2096945196390152, + 0.07385209202766418, + -0.256763756275177, + 0.7537413835525513, + 0.3147052526473999, + 0.5640119910240173, + -0.2542858123779297, + 0.06437408179044724, + 0.10259352624416351, + 0.2628327012062073, + 0.2347445785999298, + -1.871469497680664, + 0.571297287940979, + -0.01785074546933174, + 0.06142580881714821, + -0.3025362491607666, + 0.005423873662948608, + -0.4231855869293213, + -2.0950539112091064, + -0.12601861357688904 + ], + [ + 0.0627809390425682, + 0.1603173166513443, + -0.20445504784584045, + -0.22461488842964172, + 0.7926700115203857, + -0.1396835744380951, + 0.9558979272842407, + -0.03341652452945709, + -0.09778351336717606, + -0.06997280567884445, + 0.20971012115478516, + 0.08173273503780365, + -1.808558464050293, + 0.6446847915649414, + -0.10960032045841217, + 0.08301788568496704, + -0.08366130292415619, + 0.07870107889175415, + -0.4879615902900696, + -1.9494152069091797, + -0.4764796793460846 + ], + [ + 0.21831931173801422, + 0.11013790965080261, + 0.04802587628364563, + -0.05057603120803833, + 0.6370452046394348, + -0.25856971740722656, + 0.42446526885032654, + -0.18769535422325134, + -0.04787231981754303, + -0.048109270632267, + 0.1623188555240631, + 0.1620917171239853, + -1.741257905960083, + 0.6164520978927612, + -0.0207456536591053, + 0.08974412083625793, + -0.28634965419769287, + 0.023157447576522827, + -0.5793269276618958, + -1.7957277297973633, + -0.1764676868915558 + ], + [ + 0.3186380863189697, + 0.21159712970256805, + 0.00373879075050354, + -0.3227611184120178, + 0.6657334566116333, + 0.30726322531700134, + 0.5001685619354248, + -0.16107040643692017, + 0.03624623268842697, + 0.26211827993392944, + 0.3158920705318451, + 0.16389289498329163, + -1.5911295413970947, + 0.6357441544532776, + 0.020050078630447388, + 0.09652507305145264, + -0.1914803832769394, + 0.018958255648612976, + -0.2784596085548401, + -1.7731802463531494, + -0.1289038360118866 + ], + [ + 0.22214922308921814, + 0.22001208364963531, + -0.003042936325073242, + -0.19827944040298462, + 0.7902159094810486, + -0.2831520736217499, + 0.5995392799377441, + -0.1823556125164032, + -0.016991518437862396, + 0.09439627826213837, + 0.21597382426261902, + 0.2315910905599594, + -1.4815831184387207, + 0.5753792524337769, + 0.03296735882759094, + -0.0037064021453261375, + -0.23173846304416656, + 0.0810437873005867, + -0.42252302169799805, + -1.8860173225402832, + -0.34162893891334534 + ], + [ + 0.1852109283208847, + 0.20847393572330475, + -0.049645811319351196, + -0.20558671653270721, + 0.7042896747589111, + -0.2510814964771271, + 0.4876273572444916, + -0.12039870023727417, + -0.00768355093896389, + 0.11149121820926666, + 0.1688014715909958, + 0.15713095664978027, + -1.6489306688308716, + 0.6193579435348511, + 0.02712581679224968, + 0.024227909743785858, + -0.22401832044124603, + 0.08762368559837341, + -0.3901628255844116, + -1.922485113143921, + -0.23640349507331848 + ], + [ + 0.2828711271286011, + 0.2225273698568344, + -0.09926721453666687, + -0.31204161047935486, + 0.674272894859314, + -0.2707802653312683, + 0.26839739084243774, + -0.15598011016845703, + -0.02566390298306942, + 0.11355242878198624, + 0.2185007929801941, + 0.0926683098077774, + -1.4880703687667847, + 0.5231723189353943, + 0.03246282413601875, + 0.0991555005311966, + -0.09589745104312897, + -0.012649968266487122, + -0.3544027805328369, + -1.788517713546753, + -0.20161205530166626 + ], + [ + 0.15808746218681335, + 0.08072054386138916, + -0.12960389256477356, + -0.2147287279367447, + 0.63108229637146, + -0.02273125946521759, + 0.5305259823799133, + -0.21009698510169983, + 0.014720534905791283, + 0.16780930757522583, + 0.1487010419368744, + -0.1937374770641327, + -1.485355257987976, + 0.6777828335762024, + -0.031599801033735275, + 0.0647251307964325, + -0.3129230737686157, + 0.018576815724372864, + -0.6107969880104065, + -1.9060275554656982, + -0.17941057682037354 + ] + ], + "generated_utilities": [ + 0.19381189346313477, + 0.23963995277881622, + 0.21024136245250702, + 0.13467177748680115, + 0.2017955482006073, + 0.20696409046649933, + 0.1936945617198944, + 0.21807821094989777 + ], + "group_id": "ms-d1960368ef76bbb10669", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.842734158039093, + 2.0, + 0.1342049241065979, + 0.18085119128227234, + 0.5091985464096069, + 0.016112878918647766, + 0.5080842971801758, + 0.19944033026695251, + 0.5132148265838623, + 0.5265648365020752, + 0.11437654495239258, + 0.04028162360191345, + 0.04675377905368805, + 0.13603349030017853, + 0.12877747416496277, + 0.05102705955505371 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2770041823387146, + 0.08894228935241699, + 0.07033950090408325, + 0.12426833808422089, + 0.16520383954048157, + 0.023725181818008423, + 0.0, + -0.25569218397140503, + -0.057803913950920105, + 0.06703128665685654, + 0.1732010841369629, + 0.07705201953649521, + -1.0374658107757568, + 0.0, + 0.04586680978536606, + 0.028830496594309807, + -0.0374501496553421, + 0.10374182462692261, + -0.007100850343704224, + -1.7975506782531738, + 0.0 + ], + [ + -0.4268343448638916, + 0.13483470678329468, + -0.06075429916381836, + 0.19817759096622467, + 0.09922409057617188, + 0.6547578573226929, + 0.0, + 0.15026617050170898, + 0.002472013235092163, + -0.31349992752075195, + 0.5119487047195435, + 0.571077287197113, + -1.0374658107757568, + 0.0, + -0.5828977823257446, + -0.20166385173797607, + -0.7097158432006836, + 0.8187870979309082, + -0.0694284737110138, + -1.7975506782531738, + 0.0 + ], + [ + -0.23462048172950745, + 1.088942289352417, + -0.5585125088691711, + 0.10042184591293335, + 0.4514904320240021, + -0.12210102379322052, + 2.176525354385376, + -0.10348781198263168, + 0.19529865682125092, + 0.12132670730352402, + 0.7056414484977722, + 0.12544797360897064, + -2.037465810775757, + 0.6289246678352356, + -0.008351176977157593, + 0.0010435802396386862, + 0.017314812168478966, + 0.6334809064865112, + -0.3720659911632538, + -2.797550678253174, + -1.5395829677581787 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 4.353050708770752, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.2578493356704712, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.0791659355163574 + ], + [ + -0.09280626475811005, + 1.1662368774414062, + -0.8228501081466675, + 0.24497312307357788, + 0.5648186802864075, + -0.03522863984107971, + 2.259786367416382, + 0.14498579502105713, + 0.2030920535326004, + 0.39727938175201416, + 0.8023015856742859, + 0.3385764956474304, + -1.8738088607788086, + 0.7300182580947876, + 0.20887598395347595, + 0.23107871413230896, + 0.3028872311115265, + 0.7558343410491943, + -0.5975744724273682, + -3.0329887866973877, + -1.8127071857452393 + ], + [ + -0.13783538341522217, + 0.9487118721008301, + -0.3492928743362427, + -0.2956767678260803, + 0.2793389856815338, + -0.16702969372272491, + 2.558225631713867, + 0.17572781443595886, + 0.570004940032959, + 0.27895650267601013, + 0.6778483986854553, + -0.27156615257263184, + -1.8747575283050537, + 1.0240070819854736, + 0.27303546667099, + 0.33387133479118347, + 0.1442956030368805, + 0.24661725759506226, + -0.5561860799789429, + -2.9813332557678223, + -1.833754062652588 + ], + [ + -0.18553051352500916, + 1.0909820795059204, + -0.27226096391677856, + 0.1659364104270935, + 0.7108361721038818, + -0.27768880128860474, + 2.239938974380493, + 0.019483201205730438, + 0.21263465285301208, + -0.09839647263288498, + 0.9676799178123474, + -0.36985379457473755, + -2.133316993713379, + 0.3758644461631775, + -0.47750571370124817, + 0.17470555007457733, + -0.17035381495952606, + 0.2707958221435547, + -0.5672953128814697, + -2.394361972808838, + -1.668647289276123 + ], + [ + -0.3557406961917877, + 0.9564932584762573, + -1.0014276504516602, + -0.38222843408584595, + 0.41349664330482483, + -0.05637756735086441, + 1.6011608839035034, + -0.31295540928840637, + -0.15234358608722687, + 0.4648304879665375, + 0.2744198739528656, + -0.44805359840393066, + -2.1544065475463867, + 1.166503667831421, + 0.014861157163977623, + -0.23438170552253723, + 0.1278134286403656, + 0.5812121033668518, + -0.7571046352386475, + -2.5030438899993896, + -2.100955009460449 + ], + [ + -0.16500982642173767, + 0.7556273937225342, + -0.988593339920044, + 0.6213532090187073, + 0.15840065479278564, + 0.5245341062545776, + 2.452624797821045, + -0.5693200826644897, + -0.32241708040237427, + 0.49458175897598267, + 1.3341540098190308, + 0.059507817029953, + -2.304706335067749, + 0.7334492206573486, + -0.20702646672725677, + 0.32367274165153503, + -0.5396079421043396, + 0.9809319972991943, + -0.6172815561294556, + -3.197409152984619, + -0.9358728528022766 + ], + [ + -0.07666900753974915, + 1.8442692756652832, + -0.6603910326957703, + -0.40882760286331177, + 0.22092406451702118, + -0.23551997542381287, + 1.6105436086654663, + 0.09144749492406845, + 0.8141599893569946, + 0.8528898358345032, + 1.0663814544677734, + 0.3283998966217041, + -1.7124712467193604, + 1.2278027534484863, + 0.7039424180984497, + 0.3645748496055603, + -0.2968631386756897, + 0.4998055696487427, + -0.9362069368362427, + -3.461055278778076, + -0.8184676766395569 + ], + [ + -0.1520652174949646, + 1.405963659286499, + -0.7959750294685364, + -0.39229461550712585, + 0.46590372920036316, + -0.8267906904220581, + 2.952258586883545, + 0.5923218727111816, + -0.6996292471885681, + 0.8352785706520081, + -0.15220212936401367, + -0.1307106763124466, + -2.849807024002075, + 0.4983477294445038, + 0.5902999639511108, + -0.5390225648880005, + 0.13897128403186798, + 0.21545273065567017, + -1.1186240911483765, + -3.1762759685516357, + -1.0118725299835205 + ], + [ + -0.564094066619873, + 0.4751617908477783, + -1.3977731466293335, + 0.8189605474472046, + 0.17298781871795654, + 0.2476775199174881, + 2.8234164714813232, + -0.4518245458602905, + -0.7904537916183472, + -0.39086827635765076, + -0.05634337663650513, + -0.09099806845188141, + -1.9690911769866943, + 1.1869969367980957, + -0.5615661144256592, + 0.32865890860557556, + -0.8041903376579285, + 1.3324832916259766, + -0.6190662384033203, + -2.4903852939605713, + -2.3098156452178955 + ], + [ + -0.2328718602657318, + 0.9724385142326355, + -1.5535115003585815, + 0.7688977718353271, + 0.4409410357475281, + -0.8374075889587402, + 1.2719230651855469, + 0.653096616268158, + -0.2887840270996094, + 0.4892227351665497, + 0.9943453669548035, + 0.5566720962524414, + -2.496413469314575, + 0.6299278140068054, + -0.08689186722040176, + 0.024782167747616768, + 0.5747005939483643, + 0.023068726062774658, + -1.111369252204895, + -2.187150001525879, + -1.2047184705734253 + ], + [ + -1.1679872274398804, + 1.9997228384017944, + -0.45194220542907715, + -0.8861345648765564, + -0.5369991064071655, + 0.46434593200683594, + 2.175499439239502, + 0.7599186301231384, + 0.38914406299591064, + 0.6877046823501587, + -0.004208564758300781, + -0.1321776658296585, + -1.3474420309066772, + 0.8496907353401184, + 0.4218248426914215, + -0.9354817271232605, + 0.945868730545044, + 1.60379958152771, + -1.195396065711975, + -3.597724676132202, + -1.9041136503219604 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.04476936161518097, + 0.08894228935241699, + 0.06436485052108765, + -0.0775555968284607, + 0.04825553297996521, + -0.060782790184020996, + 0.0, + -0.07688627392053604, + -0.06308497488498688, + -0.03861912339925766, + 0.1235572099685669, + -0.0017572194337844849, + -1.0374658107757568, + 0.0, + 0.03638586029410362, + 0.0556478351354599, + 0.0387292355298996, + 0.044096171855926514, + 0.029627740383148193, + -1.7975506782531738, + 0.0 + ] + ], + "predicted_scores": [ + 4.789662837982178, + 4.267895698547363, + 4.223406791687012, + 3.989636182785034, + 3.9498660564422607, + 3.7960212230682373, + 3.673099994659424, + 2.7361390590667725 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "416183106", + "selected_index": 0, + "state_hash": "cd187560c1c39db9e6bf7d9a611a8b7108dc88766de58fd62b4317f0bbcc1b50", + "stored_base_utility": 0.842734158039093, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.8350887894630432, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.8350887894630432 + }, + "base_progress": 0.8350887894630432, + "base_success": false, + "base_utility": 0.8350887894630432, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.865778386592865, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.865778386592865 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + }, + { + "contact_quality": null, + "progress": 0.1467595398426056, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.1467595398426056 + }, + { + "contact_quality": null, + "progress": 1.0, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": true, + "utility_scalar": 2.0 + } + ], + "candidate_progress": [ + 1.0, + 0.865778386592865, + 1.0, + 1.0, + 1.0, + 1.0, + 0.1467595398426056, + 1.0 + ], + "candidate_source_chart_ids": [ + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-eb7189a4f2b0f0a5d4bd", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93", + "ms-5dfbb16e173e1a3e2bd6", + "ms-dc26a7037003b94b9f93", + "ms-dc26a7037003b94b9f93" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + true, + false, + true, + true, + true, + true, + false, + true + ], + "candidate_types": [ + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank0", + "ctt_transport_rank2", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank2", + "ctt_transport_rank2" + ], + "candidates_evaluated": true, + "chart_id": "ms-efdc07c985935d871adf", + "generated_tangents": [ + [ + -0.05860698223114014, + 0.04369671642780304, + -0.0003642439842224121, + 0.16397789120674133, + -0.010445520281791687, + -1.7564449310302734, + -0.18920999765396118, + 0.04669620096683502, + 0.004963694605976343, + 0.08446834981441498, + 0.2735006511211395, + 0.3326651453971863, + -1.5656996965408325, + -0.11894083023071289, + 0.04678551107645035, + 0.05955914035439491, + 0.03427165746688843, + -0.050642624497413635, + 0.012973636388778687, + -1.1163395643234253, + -0.12019971013069153 + ], + [ + 0.13518911600112915, + 0.0794111043214798, + -0.020963460206985474, + -0.00895296037197113, + 0.027858629822731018, + -1.8182843923568726, + -0.10103672742843628, + 0.19927749037742615, + -0.049562275409698486, + 0.0963386595249176, + 0.22088587284088135, + 0.27336210012435913, + -1.3436905145645142, + -0.1528788059949875, + 0.01853266730904579, + 0.046957455575466156, + 0.17621290683746338, + 0.0900363102555275, + 0.1809384971857071, + -1.142998456954956, + -0.09620139002799988 + ], + [ + 0.10362868010997772, + 0.038398973643779755, + -0.010404139757156372, + 0.01275297999382019, + 0.04852853715419769, + -1.8492344617843628, + -0.14889788627624512, + 0.1451866328716278, + -0.021449225023388863, + 0.09165805578231812, + 0.17682726681232452, + 0.2057131826877594, + -1.4815155267715454, + -0.2417202740907669, + 0.043272290378808975, + 0.06020967289805412, + 0.17743179202079773, + -0.04293885827064514, + 0.056448474526405334, + -1.2009186744689941, + -0.08430451154708862 + ], + [ + 0.11029098182916641, + 0.014361292123794556, + -0.06367388367652893, + 0.09728631377220154, + 0.09020566940307617, + -1.7351406812667847, + -0.16312217712402344, + 0.14239948987960815, + 0.003564263693988323, + 0.10082831233739853, + 0.22786083817481995, + 0.27086347341537476, + -1.3380085229873657, + -0.2649409770965576, + 0.054218970239162445, + 0.09831023216247559, + 0.20629757642745972, + -0.03452478349208832, + 0.0876840353012085, + -1.101557970046997, + -0.23052719235420227 + ], + [ + -0.01963682472705841, + 0.11445944011211395, + -0.02500179409980774, + -0.0197308212518692, + 0.08328165113925934, + -1.2299985885620117, + -0.5541284680366516, + 0.4182572364807129, + 0.037195004522800446, + 0.15475031733512878, + 0.3062090575695038, + 0.23689785599708557, + -2.347663640975952, + -0.1468450278043747, + -0.16971290111541748, + 0.0723450630903244, + 0.2985181212425232, + -0.12749190628528595, + 0.26839175820350647, + -0.9119347333908081, + -0.5532509088516235 + ], + [ + -0.1897812932729721, + -0.06167051941156387, + 0.16612344980239868, + -0.05314287543296814, + 0.20351320505142212, + -1.6314184665679932, + 0.006596177816390991, + 0.17027737200260162, + -0.11343333125114441, + 0.06522493064403534, + 0.27704447507858276, + 0.2916208505630493, + -1.7608485221862793, + 0.031241923570632935, + 0.06306252628564835, + 0.05768124386668205, + 0.33894219994544983, + -0.054138943552970886, + -0.17477935552597046, + -1.081310749053955, + 0.053815118968486786 + ], + [ + 0.030201315879821777, + 0.14871327579021454, + -0.10771158337593079, + -0.0532061904668808, + 0.16459324955940247, + -1.67531156539917, + -0.8611230254173279, + 0.3317149877548218, + -0.18816863000392914, + -0.21422088146209717, + 0.3490787148475647, + 0.6242982149124146, + -1.4872329235076904, + -0.04529920220375061, + -0.030703160911798477, + 0.06164492294192314, + 0.11357485502958298, + 0.11759739369153976, + 0.3729802072048187, + -1.235586166381836, + -0.6288784742355347 + ], + [ + -0.04409302771091461, + -0.029062695801258087, + -0.11780580878257751, + 0.06878486275672913, + 0.07232259213924408, + -1.6242724657058716, + -0.3504095673561096, + 0.18393154442310333, + -0.0379820317029953, + 0.28007423877716064, + 0.4622671902179718, + 0.3136138319969177, + -1.6597620248794556, + -0.20851437747478485, + 0.033345915377140045, + 0.11675897240638733, + 0.45757216215133667, + -0.15742127597332, + 0.10456898808479309, + -1.3649070262908936, + -0.32653769850730896 + ] + ], + "generated_utilities": [ + 2.0, + 0.865778386592865, + 2.0, + 2.0, + 2.0, + 2.0, + 0.1467595398426056, + 2.0 + ], + "group_id": "ms-efdc07c985935d871adf", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8197344541549683, + 2.0, + 0.8125, + 0.8125142455101013, + 0.7847581505775452, + 0.7794750332832336, + 0.5920823216438293, + 0.5697357058525085, + 2.0, + 0.5847549438476562, + 0.09384849667549133, + 0.07791829109191895, + 0.011395931243896484, + 0.09286168217658997, + 0.10894183814525604, + 0.0616525262594223 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + -0.2827143669128418, + -0.12206736207008362, + 0.1543778032064438, + 0.27268126606941223, + -0.17288367450237274, + -2.5338973999023438, + -1.7615662813186646, + -0.19161763787269592, + 0.1886080503463745, + 0.31686434149742126, + 0.5112922191619873, + 0.14046527445316315, + -2.17134952545166, + -1.8021173477172852, + 0.11115366220474243, + 0.25490811467170715, + -0.025668112561106682, + 0.4787765145301819, + 0.46609628200531006, + -1.711870551109314, + -1.5312864780426025 + ], + [ + 0.03046044334769249, + 0.042622197419404984, + -0.12103251367807388, + 0.7814357876777649, + 0.5093902349472046, + -2.648094654083252, + -1.5764049291610718, + -0.16382060945034027, + 0.15227201581001282, + -0.29746392369270325, + 0.394713819026947, + 0.044284895062446594, + -2.150869131088257, + -1.853789210319519, + 0.24474957585334778, + 0.16074210405349731, + -0.30960795283317566, + 0.986261248588562, + 0.4512072801589966, + -1.9782602787017822, + -1.814382553100586 + ], + [ + -0.47424501180648804, + 0.4193490147590637, + 0.31025299429893494, + 0.436926007270813, + 0.07080172002315521, + -2.2750933170318604, + -2.139326572418213, + 0.19072748720645905, + -0.05066118389368057, + -0.38944289088249207, + 0.11913672089576721, + -0.4728909432888031, + -2.630631446838379, + -1.9070930480957031, + 0.5720394849777222, + 0.1898895502090454, + -0.0872373953461647, + 0.3081148862838745, + 0.6700773239135742, + -1.8990029096603394, + -1.655537486076355 + ], + [ + -0.112115278840065, + -0.5390006899833679, + -0.27749133110046387, + -0.08230990171432495, + 0.18304404616355896, + -2.5003390312194824, + -1.177594542503357, + -0.37152859568595886, + 0.1827065348625183, + -0.1805598884820938, + 0.8884490728378296, + 0.3141748607158661, + -2.7738473415374756, + -1.8585447072982788, + -0.6611859798431396, + 0.5365979075431824, + 0.024639926850795746, + 1.0166820287704468, + 0.6261310577392578, + -1.3820469379425049, + -1.1859028339385986 + ], + [ + -0.013153300620615482, + 0.6059781908988953, + -0.7563084363937378, + 0.6436030864715576, + 0.4828413128852844, + -1.8755569458007812, + -1.896214246749878, + -0.445474773645401, + -0.6578570604324341, + -0.21610267460346222, + 1.3274874687194824, + -0.6251860857009888, + -2.005418300628662, + -1.463393211364746, + -0.40788042545318604, + -0.5932772159576416, + -0.6501284837722778, + 0.1491221785545349, + 0.8418278694152832, + -2.578186511993408, + -1.628711462020874 + ], + [ + -0.051258739084005356, + 0.6121965050697327, + -0.4303005337715149, + 0.378423810005188, + -0.4187525510787964, + -2.0054633617401123, + -1.5839252471923828, + 0.38306933641433716, + -0.8548648953437805, + 0.5034631490707397, + 0.3682137727737427, + 0.867935061454773, + -2.5900886058807373, + -1.8283436298370361, + 0.3810080289840698, + -0.6684208512306213, + 0.5362142324447632, + -0.17851078510284424, + 1.083348035812378, + -1.9181195497512817, + -2.306365728378296 + ], + [ + 0.5487861037254333, + 0.6947635412216187, + -0.3694673776626587, + 1.3571128845214844, + -0.20053730905056, + -2.9099156856536865, + -1.493881106376648, + 0.9321863055229187, + -0.355974406003952, + 0.6561991572380066, + -0.29231804609298706, + 0.4913230240345001, + -1.8409770727157593, + -2.0504229068756104, + 0.14528872072696686, + 0.483799010515213, + -0.32654285430908203, + 1.6123273372650146, + 0.03987070918083191, + -0.8449068665504456, + -2.3494603633880615 + ], + [ + -0.3259732127189636, + -0.5385614633560181, + 0.7304226160049438, + -0.12314724922180176, + 0.03945803642272949, + -1.5919837951660156, + -1.907224416732788, + 0.3930681645870209, + -0.8835994005203247, + -0.3352096974849701, + 0.17923185229301453, + 0.17222784459590912, + -2.512176752090454, + -1.859424352645874, + 0.536429762840271, + 0.11454571038484573, + 0.3300909101963043, + 0.5491204261779785, + -0.04762554168701172, + -1.1976318359375, + -0.5251960158348083 + ], + [ + 0.33097749948501587, + -0.04192434251308441, + -0.34520047903060913, + -0.04110339283943176, + -0.5535940527915955, + -2.2057979106903076, + -2.5934860706329346, + -0.9023091793060303, + 0.5658158659934998, + -0.6125177145004272, + 1.095580816268921, + -0.05609257519245148, + -1.606997013092041, + -2.0733447074890137, + -0.5881220102310181, + 0.8953573703765869, + 0.9258373975753784, + 0.27281180024147034, + 0.7141706943511963, + -0.9142226576805115, + -1.3141751289367676 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.05978316813707352, + 0.057473838329315186, + -0.09624473005533218, + -0.011721253395080566, + 0.028141841292381287, + -1.4311916828155518, + 0.0, + -0.024008113890886307, + 0.07049623876810074, + 0.03528204560279846, + -0.010002851486206055, + 0.010910652577877045, + -1.1452367305755615, + 0.0, + 0.020785929635167122, + -0.03374411538243294, + 0.11616832762956619, + 0.05772918462753296, + 0.01671847701072693, + -0.828272819519043, + 0.0 + ], + [ + -0.13237658143043518, + -0.3393171727657318, + 0.29588890075683594, + 0.8627858757972717, + -0.1666169911623001, + -2.520320177078247, + -1.6265922784805298, + -0.031582750380039215, + 0.11019003391265869, + -0.2699315845966339, + 0.9513242244720459, + 0.15523536503314972, + -1.7896702289581299, + -1.2776744365692139, + -0.3942914605140686, + -0.30590808391571045, + -0.3098926246166229, + 0.8258905410766602, + 0.29909753799438477, + -1.5711073875427246, + -1.5744072198867798 + ] + ], + "predicted_scores": [ + 4.8488054275512695, + 4.764893531799316, + 4.676024436950684, + 4.401856422424316, + 4.025679111480713, + 3.68040132522583, + 3.5672194957733154, + 2.0043821334838867 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "706479600", + "selected_index": 0, + "state_hash": "b482f902b03ca480898798a8b5f19263d7cbb18c066305330b0796b607427fa7", + "stored_base_utility": 0.8197344541549683, + "task_id": "StackCube-v1" + }, + { + "base_outcome": { + "contact_quality": null, + "progress": 0.03719887137413025, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03719887137413025 + }, + "base_progress": 0.03719887137413025, + "base_success": false, + "base_utility": 0.03719887137413025, + "candidate_outcomes": [ + { + "contact_quality": null, + "progress": 0.029491469264030457, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.029491469264030457 + }, + { + "contact_quality": null, + "progress": 0.0387403666973114, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0387403666973114 + }, + { + "contact_quality": null, + "progress": 0.04824826121330261, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04824826121330261 + }, + { + "contact_quality": null, + "progress": 0.0322902649641037, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.0322902649641037 + }, + { + "contact_quality": null, + "progress": 0.02138236165046692, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.02138236165046692 + }, + { + "contact_quality": null, + "progress": 0.021242722868919373, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.021242722868919373 + }, + { + "contact_quality": null, + "progress": 0.04859863221645355, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.04859863221645355 + }, + { + "contact_quality": null, + "progress": 0.03606592118740082, + "recovery": null, + "safety_violation": null, + "smoothness": null, + "stage_progress": null, + "success": false, + "utility_scalar": 0.03606592118740082 + } + ], + "candidate_progress": [ + 0.029491469264030457, + 0.0387403666973114, + 0.04824826121330261, + 0.0322902649641037, + 0.02138236165046692, + 0.021242722868919373, + 0.04859863221645355, + 0.03606592118740082 + ], + "candidate_source_chart_ids": [ + "ms-bb299baa34e49eafb9fc", + "ms-ee2fcbc94d41385713d0", + "ms-0b1dd0e538b8df15a6e4", + "ms-07b479748a91a9c92b20", + "ms-d4c5d7eea19d94314e2e", + "ms-0a4d4909900fb0bc4a9a", + "ms-2fef2f43af8009bb88f1", + "ms-5ac1bf2d7d319955d45d" + ], + "candidate_source_task_ids": [ + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1", + "StackCube-v1" + ], + "candidate_success": [ + false, + false, + false, + false, + false, + false, + false, + false + ], + "candidate_types": [ + "ctt_transport_rank4", + "ctt_transport_rank6", + "ctt_transport_rank7", + "ctt_transport_rank2", + "ctt_transport_rank1", + "ctt_transport_rank0", + "ctt_transport_rank3", + "ctt_transport_rank5" + ], + "candidates_evaluated": true, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "generated_tangents": [ + [ + 0.0818156823515892, + 0.7376402616500854, + 0.260955810546875, + -1.8874306678771973, + 0.11456064134836197, + 0.6268888711929321, + -0.3406338095664978, + -0.10714408755302429, + -0.3122781217098236, + 0.12723010778427124, + 1.0230602025985718, + 0.26463550329208374, + -1.9134429693222046, + -0.24195240437984467, + -0.12432089447975159, + -0.005063745658844709, + 0.22826595604419708, + -0.13979198038578033, + -0.04610705375671387, + -1.7523059844970703, + -0.07397112250328064 + ], + [ + -0.018952831625938416, + 0.783510684967041, + 0.12759295105934143, + -1.827862024307251, + 0.27751418948173523, + 0.7717159986495972, + -0.12066727876663208, + -0.17187073826789856, + -0.33216238021850586, + 0.12718318402767181, + 1.0422388315200806, + 0.3275884985923767, + -1.8554681539535522, + -0.3043630123138428, + -0.14465337991714478, + -0.01591307297348976, + 0.21231937408447266, + -0.17680726945400238, + -0.012129753828048706, + -1.9596967697143555, + -0.0648861825466156 + ], + [ + 0.025759495794773102, + 0.799794614315033, + 0.22568532824516296, + -1.8786691427230835, + 0.3682592511177063, + 0.43615004420280457, + -0.12424099445343018, + -0.19355234503746033, + -0.3693811893463135, + 0.037883155047893524, + 1.0797016620635986, + 0.2897043824195862, + -1.912314772605896, + -0.3106462359428406, + -0.10004118084907532, + -0.014992143958806992, + 0.15777814388275146, + -0.15011344850063324, + -0.1049012541770935, + -1.7705650329589844, + -0.1936914622783661 + ], + [ + 0.12796276807785034, + 0.7324540019035339, + 0.12481063604354858, + -1.946326494216919, + 0.21629221737384796, + 0.688870906829834, + -0.07897979021072388, + -0.19743359088897705, + -0.3120637536048889, + 0.1287928968667984, + 1.02448308467865, + 0.41274213790893555, + -1.6761292219161987, + -0.2994609475135803, + -0.15494367480278015, + 0.00253965612500906, + 0.21550241112709045, + -0.15498657524585724, + -0.052129149436950684, + -1.7956414222717285, + -0.056675300002098083 + ], + [ + 0.21264928579330444, + 0.6281499266624451, + 0.005421280860900879, + -2.297286033630371, + 0.22898004949092865, + 0.9802001714706421, + 0.04851001501083374, + -0.22916251420974731, + -0.2755090594291687, + 0.2792716920375824, + 1.0580480098724365, + 0.11975810676813126, + -1.6677312850952148, + -0.07127648591995239, + -0.15530571341514587, + 0.06525619328022003, + 0.15787893533706665, + -0.04296962916851044, + 0.005149960517883301, + -1.6606080532073975, + -0.1715010702610016 + ], + [ + 0.17323878407478333, + 0.6328699588775635, + 0.22087153792381287, + -2.0211126804351807, + 0.09133979678153992, + 0.2522924542427063, + -0.2608356475830078, + -0.02745489776134491, + -0.29795435070991516, + 0.19968056678771973, + 1.178425908088684, + 0.3355984687805176, + -1.8080487251281738, + -0.035159483551979065, + -0.2646137773990631, + 0.11011427640914917, + 0.23902000486850739, + 0.05524830520153046, + -0.16241887211799622, + -1.4475040435791016, + 0.16269637644290924 + ], + [ + 0.09056825935840607, + 0.7814904451370239, + 0.25533759593963623, + -1.9061659574508667, + 0.1476134955883026, + 0.7096577882766724, + -0.0997546911239624, + -0.24902793765068054, + -0.3023238182067871, + 0.14217494428157806, + 1.0836024284362793, + 0.33638328313827515, + -1.959166407585144, + -0.30432748794555664, + -0.14443287253379822, + 0.03149983659386635, + 0.20935511589050293, + -0.16482730209827423, + -0.08840945363044739, + -1.9092836380004883, + -0.09851974248886108 + ], + [ + -0.09150759875774384, + 0.8264052867889404, + 0.12514424324035645, + -1.967693567276001, + 0.16516178846359253, + 0.7895872592926025, + -0.1622459888458252, + -0.29067984223365784, + -0.3645361065864563, + 0.04662332683801651, + 1.0962855815887451, + 0.21468932926654816, + -2.3187038898468018, + -0.401155948638916, + -0.17081262171268463, + 0.04716471955180168, + 0.27044185996055603, + -0.22255687415599823, + -0.18857508897781372, + -2.237642765045166, + -0.1585465371608734 + ] + ], + "generated_utilities": [ + 0.029491469264030457, + 0.0387403666973114, + 0.04824826121330261, + 0.0322902649641037, + 0.02138236165046692, + 0.021242722868919373, + 0.04859863221645355, + 0.03606592118740082 + ], + "group_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_candidate_types": [ + "expert", + "near_miss", + "near_miss", + "wrong_direction", + "no_op", + "wrong_gripper", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative", + "random_negative" + ], + "hidden_chart_utilities": [ + 0.8558897972106934, + 2.0, + 0.04248148202896118, + 0.05394089221954346, + 0.057141318917274475, + 0.03495943546295166, + 0.08633947372436523, + 0.046462297439575195, + 0.043458908796310425, + 0.023027777671813965, + 0.061922311782836914, + 0.034134700894355774, + 0.010651648044586182, + 0.01158277690410614, + 0.02622869610786438, + 0.012083128094673157 + ], + "instruction": "Stack cube A on top of cube B.", + "negative_tangents": [ + [ + 0.4298704266548157, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + 0.09995961934328079, + 1.162198781967163, + 0.0, + -0.00648990273475647, + -0.3028862476348877, + -0.0050805360078811646, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + 0.23185759782791138, + -0.11529958993196487, + -0.058992013335227966, + -0.010845988988876343, + 0.06423944234848022, + -2.0730173587799072, + 0.0 + ], + [ + -0.48041507601737976, + 0.6002404689788818, + 0.5398462414741516, + -2.198392152786255, + 0.1505453884601593, + 1.162198781967163, + 0.0, + 0.32028457522392273, + -0.264798641204834, + -0.27968189120292664, + 1.143568515777588, + 0.623285710811615, + -1.6758570671081543, + 0.0, + 0.3762620687484741, + 0.20924779772758484, + 0.033039893954992294, + -0.22763806581497192, + 0.8496624827384949, + -2.0730173587799072, + 0.0 + ], + [ + -0.4385460615158081, + 1.6002404689788818, + 1.1335551738739014, + -3.198392152786255, + 0.11579504609107971, + 2.162198781967163, + -1.8792641162872314, + 0.44874686002731323, + -1.264798641204834, + 0.08758684247732162, + 2.143568515777588, + 1.074564814567566, + -2.6758570671081543, + 1.4318934679031372, + -0.01953941211104393, + 0.012724362313747406, + 0.02134235017001629, + 0.39346790313720703, + -0.15033751726150513, + -3.0730173587799072, + -1.573193907737732 + ], + [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.758528232574463, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 2.8637869358062744, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -3.146387815475464 + ], + [ + -0.4582313299179077, + 1.465004801750183, + 0.9327377080917358, + -2.900977373123169, + -0.12230202555656433, + 2.23574161529541, + -2.1357157230377197, + 0.42294827103614807, + -1.506612777709961, + 0.29363107681274414, + 2.034971237182617, + 1.023054599761963, + -2.559215784072876, + 1.1742838621139526, + 0.026679620146751404, + -0.0811251625418663, + 0.16039283573627472, + 0.3527735471725464, + 0.041033580899238586, + -2.8834660053253174, + -1.2895927429199219 + ], + [ + -0.8261920809745789, + 1.8162921667099, + 1.1455696821212769, + -3.5306012630462646, + -0.15104833245277405, + 2.447983741760254, + -2.128190755844116, + 0.19818204641342163, + -1.146593689918518, + 0.03477605804800987, + 2.4299705028533936, + 0.8797258734703064, + -2.331338882446289, + 1.5516352653503418, + -0.19862954318523407, + -0.2736409604549408, + -0.21770769357681274, + 0.47925806045532227, + -0.15124429762363434, + -3.0052826404571533, + -1.2108670473098755 + ], + [ + -0.2760573625564575, + 1.6254181861877441, + 0.8188510537147522, + -3.0690624713897705, + -0.22635167837142944, + 2.051481246948242, + -1.417292833328247, + 0.026889145374298096, + -1.3347617387771606, + -0.15878626704216003, + 1.9778507947921753, + 0.7640928030014038, + -3.1165366172790527, + 1.862673282623291, + -0.451316773891449, + -0.15708383917808533, + 0.26080578565597534, + 0.6509285569190979, + 0.23991608619689941, + -3.50675892829895, + -1.297291874885559 + ], + [ + -0.8321853280067444, + 1.9553383588790894, + 1.5495907068252563, + -3.790785312652588, + 0.3716217279434204, + 2.5994434356689453, + -2.1380205154418945, + -0.10642725229263306, + -0.7562419772148132, + -0.442053884267807, + 2.3244006633758545, + 1.161316990852356, + -2.699451208114624, + 1.6399294137954712, + 0.31003186106681824, + 0.5301566123962402, + -0.16996291279792786, + 0.9281869530677795, + 0.3683878183364868, + -2.650294780731201, + -1.1098875999450684 + ], + [ + -0.6997023820877075, + 1.4820023775100708, + 1.3613333702087402, + -3.8556365966796875, + -0.5036565065383911, + 1.4896399974822998, + -2.322740077972412, + 0.03597790002822876, + -1.1858596801757812, + -0.018024958670139313, + 2.0489397048950195, + 0.8914362192153931, + -2.3395535945892334, + 1.1835731267929077, + 0.5559698343276978, + 0.5892979502677917, + 0.4496755301952362, + 0.6100684404373169, + 0.22263532876968384, + -3.0098915100097656, + -1.455836296081543 + ], + [ + -0.594323456287384, + 1.846432089805603, + 0.8105549812316895, + -2.6211562156677246, + 0.2629232406616211, + 1.4025108814239502, + -2.3828163146972656, + 0.530456006526947, + -1.3874722719192505, + 0.49274298548698425, + 1.5545053482055664, + 0.4927254915237427, + -3.1490466594696045, + 1.5643255710601807, + 0.5331913232803345, + -0.53307044506073, + 0.7433652281761169, + 0.6546258926391602, + -0.74479740858078, + -3.721141815185547, + -0.9948303699493408 + ], + [ + -0.4409070312976837, + 2.4138050079345703, + 1.3223544359207153, + -3.3717033863067627, + -0.03033369779586792, + 2.350304365158081, + -2.112061023712158, + -0.22452247142791748, + -1.420605182647705, + -0.026168949902057648, + 1.4026775360107422, + 0.9532885551452637, + -2.6276464462280273, + 2.1231069564819336, + -0.49286600947380066, + -0.3450928032398224, + 0.5427718758583069, + -0.03872644901275635, + -0.6895387172698975, + -2.357243061065674, + -2.3816022872924805 + ], + [ + 0.5156354308128357, + 2.2980997562408447, + 2.0945849418640137, + -3.690303087234497, + 1.0696191787719727, + 2.266188144683838, + -1.0399069786071777, + 0.012727499008178711, + -1.2163764238357544, + 0.5646419525146484, + 1.1664416790008545, + 1.563025951385498, + -3.425304412841797, + 0.5126914978027344, + -0.18885818123817444, + -0.5277795791625977, + -0.1741226613521576, + 0.2763793468475342, + 0.8411282896995544, + -3.2598915100097656, + -2.2270073890686035 + ], + [ + -1.2896747589111328, + 2.3828718662261963, + 1.191500186920166, + -3.9747776985168457, + 0.7604506015777588, + 2.298020839691162, + -1.0116384029388428, + 1.177309513092041, + -1.6719520092010498, + -0.4176469147205353, + 2.709542989730835, + 0.26378321647644043, + -2.119570255279541, + 0.7668893933296204, + 0.6772738099098206, + -0.5338667035102844, + -0.7434203028678894, + 0.2167283445596695, + 0.03269544243812561, + -3.895913600921631, + -0.9124676585197449 + ], + [ + -1.323930263519287, + 2.338547468185425, + 1.4313098192214966, + -2.960175037384033, + 1.1079072952270508, + 1.5746114253997803, + -1.8946435451507568, + -0.4213535189628601, + -1.922096848487854, + 0.6645004153251648, + 2.1448850631713867, + 0.2334994077682495, + -3.4603066444396973, + 1.5447293519973755, + 0.25094738602638245, + 0.7163193821907043, + -0.10547596216201782, + -0.42469942569732666, + -0.49689921736717224, + -2.8712027072906494, + -1.6389087438583374 + ] + ], + "num_executed_including_base": 9, + "num_generated": 8, + "outcome_vector_schema": [ + "success", + "progress", + "contact_quality", + "safety_violation", + "stage_progress", + "smoothness", + "recovery", + "utility_scalar" + ], + "positive_tangents": [ + [ + -0.027082443237304688, + 0.6002404689788818, + 0.13355517387390137, + -2.198392152786255, + -0.03311736881732941, + 1.162198781967163, + 0.0, + 0.043807268142700195, + -0.264798641204834, + 0.0069312527775764465, + 1.143568515777588, + 0.07456481456756592, + -1.6758570671081543, + 0.0, + -0.01150114368647337, + -0.007074322551488876, + -0.018292216584086418, + -0.035625845193862915, + 0.05120308697223663, + -2.0730173587799072, + 0.0 + ] + ], + "predicted_scores": [ + 6.205906391143799, + 5.960236072540283, + 5.870675563812256, + 5.737954616546631, + 5.6567230224609375, + 5.656543254852295, + 5.328103542327881, + 4.695163726806641 + ], + "restore_error": 1.4901161193847656e-08, + "seed": "1919198188", + "selected_index": 0, + "state_hash": "6b9e4c034c70f8e93116064dbcb48aa5fea37958e04fa30734f597c38aefea8b", + "stored_base_utility": 0.8558897972106934, + "task_id": "StackCube-v1" + } + ], + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "source_index": "data/cil_charts_rgb_refs/train/index.json", + "source_split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_index": "data/cil_charts_rgb_refs/val/index.json", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381" +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..f60f7c80f0040b79fe24adb223927d4171811ff9 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics.json @@ -0,0 +1,3767 @@ +{ + "epsilon": 0.0, + "k": 8, + "mode": "measured", + "num_rows": 69, + "rows": [ + { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "chart_id": "ms-484ea653bbca2de3a4a2", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16621555087386797, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.725460946559906, + "proposal_oracle_utility_gain_over_base_at_8": -0.0015451908111572266, + "seed": "1231780997", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.725460946559906, + "selected_utility_gain_over_base_at_8": -0.0015451908111572266, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.274539053440094, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.274539053440094 + }, + { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.051884353160858154, + "chart_id": "ms-4b4cd12d70c0615e0930", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12800533274597592, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7146708369255066, + "proposal_oracle_utility_gain_over_base_at_8": 0.18629717826843262, + "seed": "1025740212", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6627864837646484, + "selected_utility_gain_over_base_at_8": 0.13441282510757446, + "selector_regret_at_8": 0.051884353160858154, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2853291630744934, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.3372135162353516 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2959344387054443, + "chart_id": "ms-6e268d1e2915740816a1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15809903475948942, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1475440973", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7040655612945557, + "selected_utility_gain_over_base_at_8": -1.2959344387054443, + "selector_regret_at_8": 1.2959344387054443, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.2959344387054443 + }, + { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.287484347820282, + "chart_id": "ms-8864796e31bbe93d7c5d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4327797462092032, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "seed": "551504635", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.712515652179718, + "selected_utility_gain_over_base_at_8": -0.011456906795501709, + "selector_regret_at_8": 1.287484347820282, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.287484347820282 + }, + { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.48447395861148834, + "chart_id": "ms-b04b489f4fbad93b65ca", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2534904144040469, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.6990174651145935, + "proposal_oracle_utility_gain_over_base_at_8": 0.27670493721961975, + "seed": "1113268940", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21454350650310516, + "selected_utility_gain_over_base_at_8": -0.2077690213918686, + "selector_regret_at_8": 0.48447395861148834, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.3009825348854065, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 1.7854564934968948 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c663f7e73e7e8376a05c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3242760158848682, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1528740417", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "LiftPegUpright-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "chart_id": "ms-2458fcfffb3f96c6ca1a", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "seed": "1273247072", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "task_id": "PickCube-v1", + "total_car_to_hidden_at_8": 1.922220028936863 + }, + { + "base_success": 0.0, + "base_utility": 0.050487834960222244, + "branch_car_at_8": 0.06330369412899017, + "chart_id": "ms-0345d8ac530f88cefd61", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1798098385334015, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09873073474489925, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.18499283492565155, + "proposal_oracle_utility_gain_over_base_at_8": 0.1345049999654293, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.12168914079666138, + "selected_utility_gain_over_base_at_8": 0.07120130583643913, + "selector_regret_at_8": 0.06330369412899017, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.05812069773674011 + }, + { + "base_success": 0.0, + "base_utility": 0.07743936777114868, + "branch_car_at_8": 0.006376962177455425, + "chart_id": "ms-0819eb89e627595e7e1b", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.14572584629058838, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1633033505229465, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.016623934730887413, + "proposal_oracle_utility_gain_over_base_at_8": -0.06081543304026127, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.010246972553431988, + "selected_utility_gain_over_base_at_8": -0.0671923952177167, + "selector_regret_at_8": 0.006376962177455425, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12910191155970097, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1354788737371564 + }, + { + "base_success": 0.0, + "base_utility": 0.19400137662887573, + "branch_car_at_8": 0.11550857126712799, + "chart_id": "ms-0dee962a5901fb8b5bbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22386163138867388, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.16342134773731232, + "proposal_oracle_utility_gain_over_base_at_8": -0.030580028891563416, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.047912776470184326, + "selected_utility_gain_over_base_at_8": -0.1460886001586914, + "selector_regret_at_8": 0.11550857126712799, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8365786522626877, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9520872235298157 + }, + { + "base_success": 0.0, + "base_utility": 0.04583283141255379, + "branch_car_at_8": 0.007768414914608002, + "chart_id": "ms-126eb8b36769f9ba4671", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1985912322998047, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.32227493498212384, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07069015502929688, + "proposal_oracle_utility_gain_over_base_at_8": 0.024857323616743088, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06292174011468887, + "selected_utility_gain_over_base_at_8": 0.017088908702135086, + "selector_regret_at_8": 0.007768414914608002, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1279010772705078, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13566949218511581 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-16661b2fd2c85f57a66c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22254540334575248, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1549324805", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00036396586801856756, + "chart_id": "ms-20e1efbb745fd02abc95", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.29573093343728263, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0012095769634470344, + "proposal_oracle_utility_gain_over_base_at_8": -1.998790423036553, + "seed": "1375492786", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0008456110954284668, + "selected_utility_gain_over_base_at_8": -1.9991543889045715, + "selector_regret_at_8": 0.00036396586801856756, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2422413924941793, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24260535836219788 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-2dc5138251afd26a040d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565621826", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.08252689987421036, + "branch_car_at_8": 0.004420019686222076, + "chart_id": "ms-387e6771a55be98f1a59", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20744429528713226, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.312878719204232, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.09707490354776382, + "proposal_oracle_utility_gain_over_base_at_8": 0.014548003673553467, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.09265488386154175, + "selected_utility_gain_over_base_at_8": 0.01012798398733139, + "selector_regret_at_8": 0.004420019686222076, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11036939173936844, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11478941142559052 + }, + { + "base_success": 0.0, + "base_utility": 0.03917040303349495, + "branch_car_at_8": 0.04887852817773819, + "chart_id": "ms-38f7298184a2c5005e94", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1767096221446991, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12442156901905659, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.11467790603637695, + "proposal_oracle_utility_gain_over_base_at_8": 0.075507503002882, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.06579937785863876, + "selected_utility_gain_over_base_at_8": 0.026628974825143814, + "selector_regret_at_8": 0.04887852817773819, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.062031716108322144, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.11091024428606033 + }, + { + "base_success": 0.0, + "base_utility": 0.009124159812927246, + "branch_car_at_8": 0.00928109884262085, + "chart_id": "ms-482c5c8d208b5907a2c2", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1834145486354828, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.18390629227998556, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.030993500724434853, + "proposal_oracle_utility_gain_over_base_at_8": 0.021869340911507607, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.021712401881814003, + "selected_utility_gain_over_base_at_8": 0.012588242068886757, + "selector_regret_at_8": 0.00928109884262085, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15242104791104794, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.16170214675366879 + }, + { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.0, + "chart_id": "ms-4da03300d92420e80ed0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33194730961537694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "seed": "1747178030", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8533687740564346, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0008774008601903915, + "chart_id": "ms-57be9ae181c077e7821f", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2838898926846715, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.010276774875819683, + "proposal_oracle_utility_gain_over_base_at_8": -1.9897232251241803, + "seed": "1549324805", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.009399374015629292, + "selected_utility_gain_over_base_at_8": -1.9906006259843707, + "selector_regret_at_8": 0.0008774008601903915, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9897232251241803, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9906006259843707 + }, + { + "base_success": 0.0, + "base_utility": 0.1544450968503952, + "branch_car_at_8": 1.990578512661159, + "chart_id": "ms-6388b5d2356c556c3e55", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.28218050665295125, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8455549031496048, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.009421487338840961, + "selected_utility_gain_over_base_at_8": -0.14502360951155424, + "selector_regret_at_8": 1.990578512661159, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.990578512661159 + }, + { + "base_success": 0.0, + "base_utility": 0.06173701956868172, + "branch_car_at_8": 0.039066337049007416, + "chart_id": "ms-638d2ba1917eff4d09e0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.11176212877035141, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.10082628405067513, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.14829117059707642, + "proposal_oracle_utility_gain_over_base_at_8": 0.0865541510283947, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.109224833548069, + "selected_utility_gain_over_base_at_8": 0.04748781397938728, + "selector_regret_at_8": 0.039066337049007416, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0025372952222824097 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.981676161289215, + "chart_id": "ms-652ad552ba98d2736620", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2367449700832367, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.08904583687943832, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "767747424", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.018323838710784912, + "selected_utility_gain_over_base_at_8": -1.981676161289215, + "selector_regret_at_8": 1.981676161289215, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.21842113137245178 + }, + { + "base_success": 0.0, + "base_utility": 0.03784912824630737, + "branch_car_at_8": 0.08733024075627327, + "chart_id": "ms-674fad0e7c042d23629e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.15573063492774963, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3185519049632928, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.12104523181915283, + "proposal_oracle_utility_gain_over_base_at_8": 0.08319610357284546, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.03371499106287956, + "selected_utility_gain_over_base_at_8": -0.004134137183427811, + "selector_regret_at_8": 0.08733024075627327, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0346854031085968, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.12201564386487007 + }, + { + "base_success": 0.0, + "base_utility": 0.12791956961154938, + "branch_car_at_8": 0.0684172473847866, + "chart_id": "ms-728b865677134d3788e3", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.22658847272396088, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.248063405084695, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.10312310606241226, + "proposal_oracle_utility_gain_over_base_at_8": -0.024796463549137115, + "seed": "750448296", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.034705858677625656, + "selected_utility_gain_over_base_at_8": -0.09321371093392372, + "selector_regret_at_8": 0.0684172473847866, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12346536666154861, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.19188261404633522 + }, + { + "base_success": 0.0, + "base_utility": 0.20529556274414062, + "branch_car_at_8": 0.05776144564151764, + "chart_id": "ms-7ea98b8f4d2220347ee0", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.27989473938941956, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10783128480272308, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.19780242443084717, + "proposal_oracle_utility_gain_over_base_at_8": -0.007493138313293457, + "seed": "565369392", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.14004097878932953, + "selected_utility_gain_over_base_at_8": -0.0652545839548111, + "selector_regret_at_8": 0.05776144564151764, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.08209231495857239, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.13985376060009003 + }, + { + "base_success": 0.0, + "base_utility": 0.18213583528995514, + "branch_car_at_8": 0.001691202400252223, + "chart_id": "ms-7f57045c114ddad153b5", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.227622389793396, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.43578303044475486, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.004366755485534668, + "proposal_oracle_utility_gain_over_base_at_8": -0.17776907980442047, + "seed": "278008524", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.002675553085282445, + "selected_utility_gain_over_base_at_8": -0.1794602822046727, + "selector_regret_at_8": 0.001691202400252223, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.22325563430786133, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.22494683670811355 + }, + { + "base_success": 0.0, + "base_utility": 0.0056487321853637695, + "branch_car_at_8": 0.0014750361442565918, + "chart_id": "ms-81f4e8858b2ba90aea2e", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1631626933813095, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.11431568558471643, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01331732701510191, + "proposal_oracle_utility_gain_over_base_at_8": 0.00766859482973814, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.011842290870845318, + "selected_utility_gain_over_base_at_8": 0.006193558685481548, + "selector_regret_at_8": 0.0014750361442565918, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1498453663662076, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1513204025104642 + }, + { + "base_success": 0.0, + "base_utility": 0.00698717450723052, + "branch_car_at_8": 0.04227004759013653, + "chart_id": "ms-8a72b73ac452bc190aeb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.10329639911651611, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1973708126521778, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07134183496236801, + "proposal_oracle_utility_gain_over_base_at_8": 0.06435466045513749, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029071787372231483, + "selected_utility_gain_over_base_at_8": 0.022084612865000963, + "selector_regret_at_8": 0.04227004759013653, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0319545641541481, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.07422461174428463 + }, + { + "base_success": 0.0, + "base_utility": 0.08285123109817505, + "branch_car_at_8": 0.04249020293354988, + "chart_id": "ms-a96bca1ef0b74f686018", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.1572066992521286, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.4035470095413562, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.07769393920898438, + "proposal_oracle_utility_gain_over_base_at_8": -0.005157291889190674, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.035203736275434494, + "selected_utility_gain_over_base_at_8": -0.047647494822740555, + "selector_regret_at_8": 0.04249020293354988, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.07951276004314423, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.12200296297669411 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-ac4fb33f3ab4db853995", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.32123621367047994, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "565369392", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-c31b98b3d3702c6f9c8c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "2054899126", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.037537217140197754, + "branch_car_at_8": 0.0038126707077026367, + "chart_id": "ms-c832439976a37f52d094", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2594742774963379, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.24086667034114959, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.029142936691641808, + "proposal_oracle_utility_gain_over_base_at_8": -0.008394280448555946, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02533026598393917, + "selected_utility_gain_over_base_at_8": -0.012206951156258583, + "selector_regret_at_8": 0.0038126707077026367, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.23033134080469608, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.23414401151239872 + }, + { + "base_success": 0.0, + "base_utility": 0.09138330072164536, + "branch_car_at_8": 0.001046299934387207, + "chart_id": "ms-c96b415f083f60857334", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.26426148414611816, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.17074054804518998, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01693703792989254, + "proposal_oracle_utility_gain_over_base_at_8": -0.07444626279175282, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.015890737995505333, + "selected_utility_gain_over_base_at_8": -0.07549256272614002, + "selector_regret_at_8": 0.001046299934387207, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.24732444621622562, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.24837074615061283 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-d73fd5fce55fb58ef8e9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15703220108978472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "534717141", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.14809735119342804, + "branch_car_at_8": 0.00038852286525070667, + "chart_id": "ms-da142025733b53fcf7eb", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.2643269896507263, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.18128422623207796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.0024978716392070055, + "proposal_oracle_utility_gain_over_base_at_8": -0.14559947955422103, + "seed": "141984615", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.002109348773956299, + "selected_utility_gain_over_base_at_8": -0.14598800241947174, + "selector_regret_at_8": 0.00038852286525070667, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2618291180115193, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.26221764087677 + }, + { + "base_success": 0.0, + "base_utility": 0.1362423449754715, + "branch_car_at_8": 0.030585486441850662, + "chart_id": "ms-e2bad6e99e8a281207cc", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20825156569480896, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.33910771964071507, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04993142560124397, + "proposal_oracle_utility_gain_over_base_at_8": -0.08631091937422752, + "seed": "1549324805", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.01934593915939331, + "selected_utility_gain_over_base_at_8": -0.11689640581607819, + "selector_regret_at_8": 0.030585486441850662, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.158320140093565, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.18890562653541565 + }, + { + "base_success": 0.0, + "base_utility": 0.09695577621459961, + "branch_car_at_8": 0.01554206945002079, + "chart_id": "ms-e30fd80adc089e9f697a", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18920521438121796, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.25602335450638297, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04263921454548836, + "proposal_oracle_utility_gain_over_base_at_8": -0.05431656166911125, + "seed": "585904130", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.027097145095467567, + "selected_utility_gain_over_base_at_8": -0.06985863111913204, + "selector_regret_at_8": 0.01554206945002079, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.1465659998357296, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1621080692857504 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-e9ed5d488afdfd6449d1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.10228227769065046, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.1320568472146988, + "proposal_oracle_utility_gain_over_base_at_8": -1.8679431527853012, + "seed": "626486813", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.1320568472146988, + "selected_utility_gain_over_base_at_8": -1.8679431527853012, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.8679431527853012, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.8679431527853012 + }, + { + "base_success": 0.0, + "base_utility": 0.09514755010604858, + "branch_car_at_8": 0.0030524134635925293, + "chart_id": "ms-ec013396e4aff6170ee1", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24502670764923096, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1325592590169343, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.06178152561187744, + "proposal_oracle_utility_gain_over_base_at_8": -0.03336602449417114, + "seed": "750448296", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05872911214828491, + "selected_utility_gain_over_base_at_8": -0.03641843795776367, + "selector_regret_at_8": 0.0030524134635925293, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.18324518203735352, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.18629759550094604 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.028240702114999294, + "chart_id": "ms-ee30ebaf98622759d888", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.6813618274184183, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.036433976143598557, + "proposal_oracle_utility_gain_over_base_at_8": -1.9635660238564014, + "seed": "636680137", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.008193274028599262, + "selected_utility_gain_over_base_at_8": -1.9918067259714007, + "selector_regret_at_8": 0.028240702114999294, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9635660238564014, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 1.9918067259714007 + }, + { + "base_success": 0.0, + "base_utility": 0.026694953441619873, + "branch_car_at_8": 0.023953834548592567, + "chart_id": "ms-f3f1d07cfd59b823da47", + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.17124073207378387, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3481076775975991, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.044700879603624344, + "proposal_oracle_utility_gain_over_base_at_8": 0.01800592616200447, + "seed": "2015956957", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.020747045055031776, + "selected_utility_gain_over_base_at_8": -0.005947908386588097, + "selector_regret_at_8": 0.023953834548592567, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.12653985247015953, + "task_id": "PullCube-v1", + "total_car_to_hidden_at_8": 0.1504936870187521 + }, + { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.954741045832634, + "chart_id": "ms-081952a63087fc8f83cd", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3183279052482729, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "seed": "2141802816", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04525895416736603, + "selected_utility_gain_over_base_at_8": -0.025548294186592102, + "selector_regret_at_8": 1.954741045832634, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.954741045832634 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8714426755905151, + "chart_id": "ms-121bc2e6676c1ac10911", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.21826309519081116, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "77147264", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.12855732440948486, + "selected_utility_gain_over_base_at_8": -1.8714426755905151, + "selector_regret_at_8": 1.8714426755905151, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.8714426755905151 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-22f153f0d4fa656018e6", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.1130050850209727, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1271718007", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.0, + "chart_id": "ms-27c9f70e97b8850cc3ab", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1643392210528543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.040287479758262634, + "proposal_oracle_utility_gain_over_base_at_8": 0.008260488510131836, + "seed": "1881223807", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.040287479758262634, + "selected_utility_gain_over_base_at_8": 0.008260488510131836, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9597125202417374, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9597125202417374 + }, + { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "chart_id": "ms-2867deaca7e98582620b", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33283923875771465, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "seed": "1189392917", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "chart_id": "ms-365d8c6de6de16f66744", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21318958083944717, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "seed": "1997090716", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.03280790150165558, + "branch_car_at_8": 0.00636233389377594, + "chart_id": "ms-3c326d05fec66e1d6bea", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.38232847915297685, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.02442134916782379, + "proposal_oracle_utility_gain_over_base_at_8": -0.008386552333831787, + "seed": "278008524", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.01805901527404785, + "selected_utility_gain_over_base_at_8": -0.014748886227607727, + "selector_regret_at_8": 0.00636233389377594, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9755786508321762, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9819409847259521 + }, + { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "chart_id": "ms-4dbdc6ed61dccebbfe92", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1880089150142482, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "seed": "794181626", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "chart_id": "ms-58950fed04d203b97ab9", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13380940221299786, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1113371261", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "chart_id": "ms-651bc0395d78d2819158", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06905173445346233, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "seed": "1483645336", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9587705433368683, + "chart_id": "ms-9fad82fd288e4296e354", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2664806376409974, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "seed": "1176943004", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.041229456663131714, + "selected_utility_gain_over_base_at_8": -0.11338427662849426, + "selector_regret_at_8": 1.9587705433368683, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9587705433368683 + }, + { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "chart_id": "ms-b8bfdd3e6c31bd98082d", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.14873333521535068, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "seed": "998761162", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9767832309007645, + "chart_id": "ms-fc8084d5cd7da937fc02", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.613109405537996, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1991414747", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.023216769099235535, + "selected_utility_gain_over_base_at_8": -1.9767832309007645, + "selector_regret_at_8": 1.9767832309007645, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "PushCube-v1", + "total_car_to_hidden_at_8": 1.9767832309007645 + }, + { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.048324257135391235, + "chart_id": "ms-3afd8959363e917591d4", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2643509727736545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21781498193740845, + "proposal_oracle_utility_gain_over_base_at_8": 0.015063345432281494, + "seed": "75070641", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1694907248020172, + "selected_utility_gain_over_base_at_8": -0.03326091170310974, + "selector_regret_at_8": 0.048324257135391235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7821850180625916, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8305092751979828 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8424437642097473, + "chart_id": "ms-3ff1aedb72f65a182b58", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22991638132379041, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1055567618", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.15755623579025269, + "selected_utility_gain_over_base_at_8": -1.8424437642097473, + "selector_regret_at_8": 1.8424437642097473, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8424437642097473 + }, + { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.0, + "chart_id": "ms-4048f8f6e047422e3c79", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06066548841424423, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8651304244995117, + "proposal_oracle_utility_gain_over_base_at_8": 0.6686463505029678, + "seed": "1055157354", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8651304244995117, + "selected_utility_gain_over_base_at_8": 0.6686463505029678, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1348695755004883, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.1348695755004883 + }, + { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.056304022669792175, + "chart_id": "ms-5b26c33feed6992386b1", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21028164676312622, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23198044300079346, + "proposal_oracle_utility_gain_over_base_at_8": 0.033537328243255615, + "seed": "1522272686", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.17567642033100128, + "selected_utility_gain_over_base_at_8": -0.02276669442653656, + "selector_regret_at_8": 0.056304022669792175, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7680195569992065, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8243235796689987 + }, + { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "chart_id": "ms-63ac245579b1188935ad", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.08971682945849302, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "seed": "211245842", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1611655950546265, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.868122011423111, + "chart_id": "ms-6c8d06bace98a28a8e4e", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.26263826459603723, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "seed": "1665739511", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13187798857688904, + "selected_utility_gain_over_base_at_8": -1.868122011423111, + "selector_regret_at_8": 1.868122011423111, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.868122011423111 + }, + { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07907943427562714, + "chart_id": "ms-72777024349c2aacd5e0", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5133629211325229, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.19852638244628906, + "proposal_oracle_utility_gain_over_base_at_8": -1.801473617553711, + "seed": "256770874", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11944694817066193, + "selected_utility_gain_over_base_at_8": -1.880553051829338, + "selector_regret_at_8": 0.07907943427562714, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.801473617553711, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.880553051829338 + }, + { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.38811931014060974, + "chart_id": "ms-7fc5696cb3dc7b9bb9f7", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5372965735602796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5639582276344299, + "proposal_oracle_utility_gain_over_base_at_8": 0.36023636162281036, + "seed": "957101489", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1758389174938202, + "selected_utility_gain_over_base_at_8": -0.027882948517799377, + "selector_regret_at_8": 0.38811931014060974, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.43604177236557, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8241610825061798 + }, + { + "base_success": 0.0, + "base_utility": 0.8431589603424072, + "branch_car_at_8": 0.004197746515274048, + "chart_id": "ms-9a14552b7db695dee92c", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.054992714566980724, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.22092504799365997, + "proposal_oracle_utility_gain_over_base_at_8": -0.6222339123487473, + "seed": "77147264", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21672730147838593, + "selected_utility_gain_over_base_at_8": -0.6264316588640213, + "selector_regret_at_8": 0.004197746515274048, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.77907495200634, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.783272698521614 + }, + { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.185863733291626, + "chart_id": "ms-9f5d57fe2153fd267774", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2143457671242473, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "seed": "1607773860", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814136266708374, + "selected_utility_gain_over_base_at_8": -0.01853656768798828, + "selector_regret_at_8": 1.185863733291626, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.185863733291626 + }, + { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.0, + "chart_id": "ms-b0fee26e1db967f90b81", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09625642405798478, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1727849841117859, + "seed": "1358421325", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1727849841117859, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.8307622671127319, + "branch_car_at_8": 1.8400099277496338, + "chart_id": "ms-bc2d3b425d9163a62dbb", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.30145083880557255, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.169237732887268, + "seed": "1271718007", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1599900722503662, + "selected_utility_gain_over_base_at_8": -0.6707721948623657, + "selector_regret_at_8": 1.8400099277496338, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8400099277496338 + }, + { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.04582805931568146, + "chart_id": "ms-d1960368ef76bbb10669", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21826032886833752, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23963995277881622, + "proposal_oracle_utility_gain_over_base_at_8": 0.03931353986263275, + "seed": "416183106", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.19381189346313477, + "selected_utility_gain_over_base_at_8": -0.006514519453048706, + "selector_regret_at_8": 0.04582805931568146, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7603600472211838, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.8061881065368652 + }, + { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.0, + "chart_id": "ms-efdc07c985935d871adf", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15321122852132601, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "seed": "706479600", + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1649112105369568, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 0.0 + }, + { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.019107162952423096, + "chart_id": "ms-fd2387aeed9bfc9a0d08", + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "mode": "measured", + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21294576266807125, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04859863221645355, + "proposal_oracle_utility_gain_over_base_at_8": 0.011399760842323303, + "seed": "1919198188", + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029491469264030457, + "selected_utility_gain_over_base_at_8": -0.0077074021100997925, + "selector_regret_at_8": 0.019107162952423096, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9514013677835464, + "task_id": "StackCube-v1", + "total_car_to_hidden_at_8": 1.9705085307359695 + } + ], + "summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.46099290780141844, + "low": 0.19680851063829788, + "mean": 0.3085106382978724, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 1.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3176470588235294, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.4057971014492754, + "low": 0.18840579710144928, + "mean": 0.2753623188405797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.0808477015729916, + "low": 0.6013251688004412, + "mean": 0.8163502178883101, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8163502178883101, + "per_seed": { + "1025740212": 0.528373658657074, + "1055157354": 0.19648407399654388, + "1055567618": 2.0, + "1113268940": 0.42231252789497375, + "1113371261": 2.0, + "1176943004": 0.15461373329162598, + "1189392917": 0.10841864347457886, + "1231780997": 0.7270061373710632, + "1271718007": 1.415381133556366, + "1273247072": 0.15342965722084045, + "1358421325": 0.8272150158882141, + "1375492786": 2.0, + "141984615": 0.09381213908394177, + "1475440973": 2.0, + "1483645336": 0.01641605794429779, + "1522272686": 0.19844311475753784, + "1528740417": 2.0, + "1549324805": 1.3787474483251572, + "1607773860": 0.8326728343963623, + "1665739511": 2.0, + "1747178030": 0.14663122594356537, + "1881223807": 0.0320269912481308, + "1919198188": 0.03719887137413025, + "1991414747": 2.0, + "1997090716": 0.13927479088306427, + "2015956957": 0.06711978986859321, + "2054899126": 2.0, + "211245842": 0.8388344049453735, + "2141802816": 0.07080724835395813, + "256770874": 2.0, + "278008524": 0.10747186839580536, + "416183106": 0.20032641291618347, + "534717141": 2.0, + "551504635": 0.7239725589752197, + "565369392": 0.7997656464576721, + "565621826": 2.0, + "585904130": 0.07529209802548091, + "626486813": 0.5054400166263804, + "636680137": 2.0, + "706479600": 0.8350887894630432, + "750448296": 0.11153355985879898, + "75070641": 0.20275163650512695, + "767747424": 0.5475241877138615, + "77147264": 1.4215794801712036, + "794181626": 0.18095605075359344, + "957101489": 0.20372186601161957, + "998761162": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9058472741949395, + "low": 0.4133020142714182, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.923898630168127, + "low": 0.5448560892550302, + "mean": 0.7180729379567007, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5746948152998502, + "low": 0.20552555694829355, + "mean": 0.38140176123681857, + "n": 47, + "num_samples": 200 + }, + "mean": 0.38140176123681857, + "per_seed": { + "1025740212": 0.051884353160858154, + "1055157354": 0.0, + "1055567618": 1.8424437642097473, + "1113268940": 0.48447395861148834, + "1113371261": 0.0, + "1176943004": 1.9587705433368683, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.9200049638748169, + "1273247072": 0.39307598024606705, + "1358421325": 0.0, + "1375492786": 0.00036396586801856756, + "141984615": 0.03539413997593025, + "1475440973": 1.2959344387054443, + "1483645336": 0.0, + "1522272686": 0.056304022669792175, + "1528740417": 0.0, + "1549324805": 0.010487629100680351, + "1607773860": 1.185863733291626, + "1665739511": 1.868122011423111, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.019107162952423096, + "1991414747": 1.9767832309007645, + "1997090716": 0.0, + "2015956957": 0.4314236156642437, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.954741045832634, + "256770874": 0.07907943427562714, + "278008524": 0.0040267681470140815, + "416183106": 0.04582805931568146, + "534717141": 0.0, + "551504635": 1.287484347820282, + "565369392": 0.057756672302881874, + "565621826": 0.0, + "585904130": 0.006800346697370212, + "626486813": 0.013256545644253492, + "636680137": 0.028240702114999294, + "706479600": 0.0, + "750448296": 0.03573483042418957, + "75070641": 0.048324257135391235, + "767747424": 0.5082327332347631, + "77147264": 0.9378202110528946, + "794181626": 0.0, + "957101489": 0.38811931014060974, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.53974167769536, + "low": 0.2651281852154152, + "mean": 0.4279891262761722, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4279891262761722, + "per_task": { + "LiftPegUpright-v1": 0.5199628497163454, + "PickCube-v1": 0.39307598024606705, + "PullCube-v1": 0.13753403203822123, + "PushCube-v1": 0.597546140734966, + "StackCube-v1": 0.4918266286452611 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.494531684580421, + "low": 0.2130161163808805, + "mean": 0.3381812960445962, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9148936170212765, + "low": 0.7432624113475178, + "mean": 0.8358156028368794, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8358156028368794, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.782608695652174, + "low": 0.5507246376811594, + "mean": 0.6666666666666666, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.850041654515774, + "low": 1.5397213380149701, + "mean": 1.7062028111638965, + "n": 47, + "num_samples": 200 + }, + "mean": 1.7062028111638965, + "per_seed": { + "1025740212": 2.0, + "1055157354": 2.0, + "1055567618": 2.0, + "1113268940": 2.0, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 2.0, + "1358421325": 2.0, + "1375492786": 0.24345096945762634, + "141984615": 0.20044784247875214, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 2.0, + "1528740417": 2.0, + "1549324805": 1.4027505218982697, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 2.0, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.5298813670873642, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 2.0, + "278008524": 1.113811194896698, + "416183106": 2.0, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 1.4266315797964733, + "565621826": 2.0, + "585904130": 0.23764699200789133, + "626486813": 0.6124684102833271, + "636680137": 2.0, + "706479600": 2.0, + "750448296": 0.23580759018659592, + "75070641": 2.0, + "767747424": 0.18863565661013126, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 2.0, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 200 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.6080652458080347, + "low": 1.1926357444861662, + "mean": 1.3999846732054932, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.676595744680851, + "low": 0.3964539007092199, + "mean": 0.5560283687943263, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5560283687943263, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.3333333333333333, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.8, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.75, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.75, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.823076923076923, + "low": 0.45843137254901956, + "mean": 0.6403318250377075, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6403318250377075, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 1.0, + "PullCube-v1": 0.35294117647058826, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.7333333333333333 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6231884057971014, + "low": 0.37681159420289856, + "mean": 0.5072463768115942, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.28064851230050547, + "low": 0.1975970437069943, + "mean": 0.237227826019069, + "n": 47, + "num_samples": 200 + }, + "mean": 0.237227826019069, + "per_seed": { + "1025740212": 0.12800533274597592, + "1055157354": 0.06066548841424423, + "1055567618": 0.22991638132379041, + "1113268940": 0.2534904144040469, + "1113371261": 0.13380940221299786, + "1176943004": 0.2664806376409974, + "1189392917": 0.33283923875771465, + "1231780997": 0.16621555087386797, + "1271718007": 0.20722796191327264, + "1273247072": 0.2134820953504291, + "1358421325": 0.09625642405798478, + "1375492786": 0.29573093343728263, + "141984615": 0.22785399017277783, + "1475440973": 0.15809903475948942, + "1483645336": 0.06905173445346233, + "1522272686": 0.21028164676312622, + "1528740417": 0.3242760158848682, + "1549324805": 0.2818476718903797, + "1607773860": 0.2143457671242473, + "1665739511": 0.26263826459603723, + "1747178030": 0.33194730961537694, + "1881223807": 0.1643392210528543, + "1919198188": 0.21294576266807125, + "1991414747": 0.613109405537996, + "1997090716": 0.21318958083944717, + "2015956957": 0.24731300175116924, + "2054899126": 0.0, + "211245842": 0.08971682945849302, + "2141802816": 0.3183279052482729, + "256770874": 0.5133629211325229, + "278008524": 0.40905575479886586, + "416183106": 0.21826032886833752, + "534717141": 0.15703220108978472, + "551504635": 0.4327797462092032, + "565369392": 0.2176430432872923, + "565621826": 0.0, + "585904130": 0.22254352429757418, + "626486813": 0.14946876705188256, + "636680137": 0.6813618274184183, + "706479600": 0.15321122852132601, + "750448296": 0.19031133205081463, + "75070641": 0.2643509727736545, + "767747424": 0.20625644377911734, + "77147264": 0.13662790487889595, + "794181626": 0.1880089150142482, + "957101489": 0.5372965735602796, + "998761162": 0.14873333521535068 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.24050993447352814, + "low": 0.2197552396470947, + "mean": 0.23152575538372436, + "n": 5, + "num_samples": 200 + }, + "mean": 0.23152575538372436, + "per_task": { + "LiftPegUpright-v1": 0.24381101581290862, + "PickCube-v1": 0.2134820953504291, + "PullCube-v1": 0.2291649560920931, + "PushCube-v1": 0.24319123348754632, + "StackCube-v1": 0.22797947617564457 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.26385682559725093, + "low": 0.20411366044304913, + "mean": 0.23259615761350452, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6372340425531915, + "low": 0.40212765957446805, + "mean": 0.5237588652482269, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5237588652482269, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 1.0, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6547511312217195, + "low": 0.18666666666666668, + "mean": 0.40962292609351436, + "n": 5, + "num_samples": 200 + }, + "mean": 0.40962292609351436, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 0.0, + "PullCube-v1": 0.23529411764705882, + "PushCube-v1": 0.8461538461538461, + "StackCube-v1": 0.4666666666666667 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5362318840579711, + "low": 0.3188405797101449, + "mean": 0.42028985507246375, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3446808510638298, + "low": 0.03900709219858156, + "mean": 0.21524822695035462, + "n": 47, + "num_samples": 200 + }, + "mean": 0.21524822695035462, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": -0.3333333333333333, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3646455505279035, + "low": 0.009803921568627449, + "mean": 0.18259426847662139, + "n": 5, + "num_samples": 200 + }, + "mean": 0.18259426847662139, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.058823529411764705, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2608695652173913, + "low": -0.014492753623188406, + "mean": 0.14492753623188406, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.3754943914410942, + "low": 0.9438222326516629, + "mean": 1.1692132722806012, + "n": 47, + "num_samples": 200 + }, + "mean": 1.1692132722806012, + "per_seed": { + "1025740212": 0.7146708369255066, + "1055157354": 0.8651304244995117, + "1055567618": 2.0, + "1113268940": 0.6990174651145935, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 0.725460946559906, + "1271718007": 2.0, + "1273247072": 0.4708559513092041, + "1358421325": 2.0, + "1375492786": 0.0012095769634470344, + "141984615": 0.08839488192461431, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 0.23198044300079346, + "1528740417": 2.0, + "1549324805": 0.6867360668256879, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 0.040287479758262634, + "1919198188": 0.04859863221645355, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.45940959043800833, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 0.19852638244628906, + "278008524": 0.01439405232667923, + "416183106": 0.23963995277881622, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 0.7870745907227198, + "565621826": 2.0, + "585904130": 0.029573063055674236, + "626486813": 0.06192737747915089, + "636680137": 0.036433976143598557, + "706479600": 2.0, + "750448296": 0.08245231583714485, + "75070641": 0.21781498193740845, + "767747424": 0.5790140572935343, + "77147264": 1.11046252399683, + "794181626": 2.0, + "957101489": 0.5639582276344299, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.3946353542561387, + "low": 0.6302031192446912, + "mean": 1.0310467479383005, + "n": 5, + "num_samples": 200 + }, + "mean": 1.0310467479383005, + "per_task": { + "LiftPegUpright-v1": 1.3565248747666676, + "PickCube-v1": 0.4708559513092041, + "PullCube-v1": 0.5247960129156982, + "PushCube-v1": 1.6972852945327759, + "StackCube-v1": 1.1057716061671574 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.1700718992818282, + "low": 0.7414891867108805, + "mean": 0.9435413414561797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5938770614889721, + "low": 0.05693793400762114, + "mean": 0.3528630543922914, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3528630543922914, + "per_seed": { + "1025740212": 0.18629717826843262, + "1055157354": 0.6686463505029678, + "1055567618": 0.0, + "1113268940": 0.27670493721961975, + "1113371261": 0.0, + "1176943004": 1.845386266708374, + "1189392917": 1.8915813565254211, + "1231780997": -0.0015451908111572266, + "1271718007": 0.584618866443634, + "1273247072": 0.31742629408836365, + "1358421325": 1.1727849841117859, + "1375492786": -1.998790423036553, + "141984615": -0.005417257159327467, + "1475440973": 0.0, + "1483645336": 1.9835839420557022, + "1522272686": 0.033537328243255615, + "1528740417": 0.0, + "1549324805": -0.6920113814994693, + "1607773860": 1.1673271656036377, + "1665739511": 0.0, + "1747178030": 1.8533687740564346, + "1881223807": 0.008260488510131836, + "1919198188": 0.011399760842323303, + "1991414747": 0.0, + "1997090716": 1.8607252091169357, + "2015956957": 0.3922898005694151, + "2054899126": 0.0, + "211245842": 1.1611655950546265, + "2141802816": 1.9291927516460419, + "256770874": -1.801473617553711, + "278008524": -0.09307781606912613, + "416183106": 0.03931353986263275, + "534717141": 0.0, + "551504635": 1.2760274410247803, + "565369392": -0.012691055734952291, + "565621826": 0.0, + "585904130": -0.04571903496980667, + "626486813": -0.4435126391472295, + "636680137": -1.9635660238564014, + "706479600": 1.1649112105369568, + "750448296": -0.02908124402165413, + "75070641": 0.015063345432281494, + "767747424": 0.03148986957967281, + "77147264": -0.3111169561743736, + "794181626": 1.8190439492464066, + "957101489": 0.36023636162281036, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7279047914040394, + "low": 0.04553533073756662, + "mean": 0.360669091189625, + "n": 5, + "num_samples": 200 + }, + "mean": 0.360669091189625, + "per_task": { + "LiftPegUpright-v1": 0.2895807276169459, + "PickCube-v1": 0.31742629408836365, + "PullCube-v1": -0.1264434560058255, + "PushCube-v1": 1.0201208339287684, + "StackCube-v1": 0.30266105631987256 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4210502535502926, + "low": -0.050174038460734635, + "mean": 0.225468403499479, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.43617021276595747, + "low": 0.19503546099290783, + "mean": 0.3226950354609929, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3226950354609929, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3917043740573153, + "low": 0.07529411764705882, + "mean": 0.21631975867269984, + "n": 5, + "num_samples": 200 + }, + "mean": 0.21631975867269984, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.2 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.36231884057971014, + "low": 0.14492753623188406, + "mean": 0.2463768115942029, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.16312056737588654, + "low": -0.16666666666666666, + "mean": 0.014184397163120569, + "n": 47, + "num_samples": 200 + }, + "mean": 0.014184397163120569, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": -1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": -0.3333333333333333, + "1607773860": 0.0, + "1665739511": -1.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": -0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.10512820512820516, + "low": -0.12352941176470586, + "mean": -0.010708898944193057, + "n": 5, + "num_samples": 200 + }, + "mean": -0.010708898944193057, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.11764705882352941, + "PushCube-v1": 0.23076923076923078, + "StackCube-v1": 0.0 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.08695652173913043, + "low": -0.17391304347826086, + "mean": -0.028985507246376812, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9823760857321785, + "low": 0.5794855141018167, + "mean": 0.7878115110437829, + "n": 47, + "num_samples": 200 + }, + "mean": 0.7878115110437829, + "per_seed": { + "1025740212": 0.6627864837646484, + "1055157354": 0.8651304244995117, + "1055567618": 0.15755623579025269, + "1113268940": 0.21454350650310516, + "1113371261": 2.0, + "1176943004": 0.041229456663131714, + "1189392917": 2.0, + "1231780997": 0.725460946559906, + "1271718007": 1.079995036125183, + "1273247072": 0.07777997106313705, + "1358421325": 2.0, + "1375492786": 0.0008456110954284668, + "141984615": 0.05300074194868406, + "1475440973": 0.7040655612945557, + "1483645336": 2.0, + "1522272686": 0.17567642033100128, + "1528740417": 2.0, + "1549324805": 0.6762484377250075, + "1607773860": 0.814136266708374, + "1665739511": 0.13187798857688904, + "1747178030": 2.0, + "1881223807": 0.040287479758262634, + "1919198188": 0.029491469264030457, + "1991414747": 0.023216769099235535, + "1997090716": 2.0, + "2015956957": 0.02798597477376461, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 0.04525895416736603, + "256770874": 0.11944694817066193, + "278008524": 0.010367284179665148, + "416183106": 0.19381189346313477, + "534717141": 2.0, + "551504635": 0.712515652179718, + "565369392": 0.729317918419838, + "565621826": 2.0, + "585904130": 0.022772716358304024, + "626486813": 0.0486708318348974, + "636680137": 0.008193274028599262, + "706479600": 2.0, + "750448296": 0.046717485412955284, + "75070641": 0.1694907248020172, + "767747424": 0.07078132405877113, + "77147264": 0.1726423129439354, + "794181626": 2.0, + "957101489": 0.1758389174938202, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8953473172508752, + "low": 0.2922459736466408, + "mean": 0.6030576216621284, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6030576216621284, + "per_task": { + "LiftPegUpright-v1": 0.8365620250503222, + "PickCube-v1": 0.07777997106313705, + "PullCube-v1": 0.38726198087747704, + "PushCube-v1": 1.0997391537978098, + "StackCube-v1": 0.6139449775218964 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.821739717160581, + "low": 0.4142426509491127, + "mean": 0.6053600454115835, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.22746946147382124, + "low": -0.3520016530264459, + "mean": -0.028538706844527254, + "n": 47, + "num_samples": 200 + }, + "mean": -0.028538706844527254, + "per_seed": { + "1025740212": 0.13441282510757446, + "1055157354": 0.6686463505029678, + "1055567618": -1.8424437642097473, + "1113268940": -0.2077690213918686, + "1113371261": 0.0, + "1176943004": -0.11338427662849426, + "1189392917": 1.8915813565254211, + "1231780997": -0.0015451908111572266, + "1271718007": -0.33538609743118286, + "1273247072": -0.0756496861577034, + "1358421325": 1.1727849841117859, + "1375492786": -1.9991543889045715, + "141984615": -0.04081139713525772, + "1475440973": -1.2959344387054443, + "1483645336": 1.9835839420557022, + "1522272686": -0.02276669442653656, + "1528740417": 0.0, + "1549324805": -0.7024990106001496, + "1607773860": -0.01853656768798828, + "1665739511": -1.868122011423111, + "1747178030": 1.8533687740564346, + "1881223807": 0.008260488510131836, + "1919198188": -0.0077074021100997925, + "1991414747": -1.9767832309007645, + "1997090716": 1.8607252091169357, + "2015956957": -0.039133815094828606, + "2054899126": 0.0, + "211245842": 1.1611655950546265, + "2141802816": -0.025548294186592102, + "256770874": -1.880553051829338, + "278008524": -0.09710458421614021, + "416183106": -0.006514519453048706, + "534717141": 0.0, + "551504635": -0.011456906795501709, + "565369392": -0.07044772803783417, + "565621826": 0.0, + "585904130": -0.05251938166717688, + "626486813": -0.456769184791483, + "636680137": -1.9918067259714007, + "706479600": 1.1649112105369568, + "750448296": -0.0648160744458437, + "75070641": -0.03326091170310974, + "767747424": -0.47674286365509033, + "77147264": -1.2489371672272682, + "794181626": 1.8190439492464066, + "957101489": -0.027882948517799377, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.16963527703132386, + "low": -0.24229603171138567, + "mean": -0.06732003508654716, + "n": 5, + "num_samples": 200 + }, + "mean": -0.06732003508654716, + "per_task": { + "LiftPegUpright-v1": -0.23038212209939957, + "PickCube-v1": -0.0756496861577034, + "PullCube-v1": -0.26397748804404675, + "PushCube-v1": 0.4225746931938025, + "StackCube-v1": -0.18916557232538858 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.11600385246245001, + "low": -0.37774778591077507, + "mean": -0.1127128925451172, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5746948152998502, + "low": 0.20552555694829355, + "mean": 0.38140176123681857, + "n": 47, + "num_samples": 200 + }, + "mean": 0.38140176123681857, + "per_seed": { + "1025740212": 0.051884353160858154, + "1055157354": 0.0, + "1055567618": 1.8424437642097473, + "1113268940": 0.48447395861148834, + "1113371261": 0.0, + "1176943004": 1.9587705433368683, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.9200049638748169, + "1273247072": 0.39307598024606705, + "1358421325": 0.0, + "1375492786": 0.00036396586801856756, + "141984615": 0.03539413997593025, + "1475440973": 1.2959344387054443, + "1483645336": 0.0, + "1522272686": 0.056304022669792175, + "1528740417": 0.0, + "1549324805": 0.010487629100680351, + "1607773860": 1.185863733291626, + "1665739511": 1.868122011423111, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.019107162952423096, + "1991414747": 1.9767832309007645, + "1997090716": 0.0, + "2015956957": 0.4314236156642437, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.954741045832634, + "256770874": 0.07907943427562714, + "278008524": 0.0040267681470140815, + "416183106": 0.04582805931568146, + "534717141": 0.0, + "551504635": 1.287484347820282, + "565369392": 0.057756672302881874, + "565621826": 0.0, + "585904130": 0.006800346697370212, + "626486813": 0.013256545644253492, + "636680137": 0.028240702114999294, + "706479600": 0.0, + "750448296": 0.03573483042418957, + "75070641": 0.048324257135391235, + "767747424": 0.5082327332347631, + "77147264": 0.9378202110528946, + "794181626": 0.0, + "957101489": 0.38811931014060974, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.53974167769536, + "low": 0.2651281852154152, + "mean": 0.4279891262761722, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4279891262761722, + "per_task": { + "LiftPegUpright-v1": 0.5199628497163454, + "PickCube-v1": 0.39307598024606705, + "PullCube-v1": 0.13753403203822123, + "PushCube-v1": 0.597546140734966, + "StackCube-v1": 0.4918266286452611 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.494531684580421, + "low": 0.2130161163808805, + "mean": 0.3381812960445962, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.324468085106383, + "low": 0.09574468085106383, + "mean": 0.20106382978723403, + "n": 47, + "num_samples": 200 + }, + "mean": 0.20106382978723403, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.29641025641025637, + "low": 0.08862745098039217, + "mean": 0.19330316742081446, + "n": 5, + "num_samples": 200 + }, + "mean": 0.19330316742081446, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.058823529411764705, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2608695652173913, + "low": 0.10144927536231885, + "mean": 0.17391304347826086, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.45390070921985815, + "low": 0.20567375886524822, + "mean": 0.3173758865248227, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3173758865248227, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.0, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7307692307692307, + "low": 0.2085972850678733, + "mean": 0.4609653092006033, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4609653092006033, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 1.0, + "PullCube-v1": 0.11764705882352941, + "PushCube-v1": 0.15384615384615385, + "StackCube-v1": 0.5333333333333333 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.3333333333333333, + "low": 0.15942028985507245, + "mean": 0.2608695652173913, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6496453900709219, + "low": 0.3801418439716312, + "mean": 0.5131205673758864, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5131205673758864, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.86, + "low": 0.38054298642533935, + "mean": 0.6483861236802413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6483861236802413, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.8 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5507246376811594, + "low": 0.3188405797101449, + "mean": 0.42028985507246375, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7874070694942542, + "low": 0.3649463011405979, + "mean": 0.5465996172535068, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5465996172535068, + "per_seed": { + "1025740212": 1.2853291630744934, + "1055157354": 1.1348695755004883, + "1055567618": 0.0, + "1113268940": 1.3009825348854065, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.274539053440094, + "1271718007": 0.0, + "1273247072": 1.529144048690796, + "1358421325": 0.0, + "1375492786": 0.2422413924941793, + "141984615": 0.11378062601822118, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.7680195569992065, + "1528740417": 0.0, + "1549324805": 0.7160144550725818, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 1.9597125202417374, + "1919198188": 1.9514013677835464, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.07047177664935589, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.801473617553711, + "278008524": 1.0994171425700188, + "416183106": 1.7603600472211838, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.6395569890737534, + "565621826": 0.0, + "585904130": 0.2080739289522171, + "626486813": 0.5505410328041762, + "636680137": 1.9635660238564014, + "706479600": 0.0, + "750448296": 0.15335527434945107, + "75070641": 1.7821850180625916, + "767747424": 0.05956761725246906, + "77147264": 0.88953747600317, + "794181626": 0.0, + "957101489": 1.43604177236557, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.158454726099758, + "low": 0.4225971201656149, + "mean": 0.7360350727635508, + "n": 5, + "num_samples": 200 + }, + "mean": 0.7360350727635508, + "per_task": { + "LiftPegUpright-v1": 0.6434751252333323, + "PickCube-v1": 1.529144048690796, + "PullCube-v1": 0.31061309059355957, + "PushCube-v1": 0.3027147054672241, + "StackCube-v1": 0.8942283938328425 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6270557242543504, + "low": 0.3067578205356266, + "mean": 0.48260227476577344, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.1480929385127618, + "low": 0.7059150926673007, + "mean": 0.9183913001201134, + "n": 47, + "num_samples": 200 + }, + "mean": 0.9183913001201134, + "per_seed": { + "1025740212": 1.3372135162353516, + "1055157354": 1.1348695755004883, + "1055567618": 1.8424437642097473, + "1113268940": 1.7854564934968948, + "1113371261": 0.0, + "1176943004": 1.9587705433368683, + "1189392917": 0.0, + "1231780997": 1.274539053440094, + "1271718007": 0.9200049638748169, + "1273247072": 1.922220028936863, + "1358421325": 0.0, + "1375492786": 0.24260535836219788, + "141984615": 0.14744710053006807, + "1475440973": 1.2959344387054443, + "1483645336": 0.0, + "1522272686": 1.8243235796689987, + "1528740417": 0.0, + "1549324805": 0.7265020841732621, + "1607773860": 1.185863733291626, + "1665739511": 1.868122011423111, + "1747178030": 0.0, + "1881223807": 1.9597125202417374, + "1919198188": 1.9705085307359695, + "1991414747": 1.9767832309007645, + "1997090716": 0.0, + "2015956957": 0.5018953923135996, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.954741045832634, + "256770874": 1.880553051829338, + "278008524": 1.1034439107170328, + "416183106": 1.8061881065368652, + "534717141": 0.0, + "551504635": 1.287484347820282, + "565369392": 0.6973136613766352, + "565621826": 0.0, + "585904130": 0.2148742756495873, + "626486813": 0.5637975784484297, + "636680137": 1.9918067259714007, + "706479600": 0.0, + "750448296": 0.18909010477364063, + "75070641": 1.8305092751979828, + "767747424": 0.11785433255136013, + "77147264": 1.8273576870560646, + "794181626": 0.0, + "957101489": 1.8241610825061798, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.5660717815924914, + "low": 0.7469345130387, + "mean": 1.153406745697748, + "n": 5, + "num_samples": 200 + }, + "mean": 1.153406745697748, + "per_task": { + "LiftPegUpright-v1": 1.1634379749496777, + "PickCube-v1": 1.922220028936863, + "PullCube-v1": 0.3950598559219061, + "PushCube-v1": 0.9002608462021902, + "StackCube-v1": 1.3860550224781036 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.9977632297837324, + "low": 0.6197721612187561, + "mean": 0.7946246277939096, + "n": 69, + "num_samples": 200 + }, + "n": 69 + } + }, + "thresholds": [ + 0.2, + 0.4 + ] +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics_by_seed.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..be8db8a041f14f9a31d6ab2d80f5b76541f1c105 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics_by_seed.json @@ -0,0 +1,1083 @@ +{ + "1025740212": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.051884353160858154, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12800533274597592, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7146708369255066, + "proposal_oracle_utility_gain_over_base_at_8": 0.18629717826843262, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6627864837646484, + "selected_utility_gain_over_base_at_8": 0.13441282510757446, + "selector_regret_at_8": 0.051884353160858154, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2853291630744934, + "total_car_to_hidden_at_8": 1.3372135162353516 + }, + "1055157354": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06066548841424423, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8651304244995117, + "proposal_oracle_utility_gain_over_base_at_8": 0.6686463505029678, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8651304244995117, + "selected_utility_gain_over_base_at_8": 0.6686463505029678, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1348695755004883, + "total_car_to_hidden_at_8": 1.1348695755004883 + }, + "1055567618": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8424437642097473, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22991638132379041, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.15755623579025269, + "selected_utility_gain_over_base_at_8": -1.8424437642097473, + "selector_regret_at_8": 1.8424437642097473, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8424437642097473 + }, + "1113268940": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.48447395861148834, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2534904144040469, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.6990174651145935, + "proposal_oracle_utility_gain_over_base_at_8": 0.27670493721961975, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21454350650310516, + "selected_utility_gain_over_base_at_8": -0.2077690213918686, + "selector_regret_at_8": 0.48447395861148834, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.3009825348854065, + "total_car_to_hidden_at_8": 1.7854564934968948 + }, + "1113371261": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13380940221299786, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9587705433368683, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2664806376409974, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.041229456663131714, + "selected_utility_gain_over_base_at_8": -0.11338427662849426, + "selector_regret_at_8": 1.9587705433368683, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9587705433368683 + }, + "1189392917": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33283923875771465, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16621555087386797, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.725460946559906, + "proposal_oracle_utility_gain_over_base_at_8": -0.0015451908111572266, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.725460946559906, + "selected_utility_gain_over_base_at_8": -0.0015451908111572266, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.274539053440094, + "total_car_to_hidden_at_8": 1.274539053440094 + }, + "1271718007": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9200049638748169, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.20722796191327264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.079995036125183, + "selected_utility_gain_over_base_at_8": -0.33538609743118286, + "selector_regret_at_8": 0.9200049638748169, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9200049638748169 + }, + "1273247072": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "total_car_to_hidden_at_8": 1.922220028936863 + }, + "1358421325": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09625642405798478, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1727849841117859, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1727849841117859, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1375492786": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00036396586801856756, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.29573093343728263, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0012095769634470344, + "proposal_oracle_utility_gain_over_base_at_8": -1.998790423036553, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0008456110954284668, + "selected_utility_gain_over_base_at_8": -1.9991543889045715, + "selector_regret_at_8": 0.00036396586801856756, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2422413924941793, + "total_car_to_hidden_at_8": 0.24260535836219788 + }, + "141984615": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03539413997593025, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.22785399017277783, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08839488192461431, + "proposal_oracle_utility_gain_over_base_at_8": -0.005417257159327467, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05300074194868406, + "selected_utility_gain_over_base_at_8": -0.04081139713525772, + "selector_regret_at_8": 0.03539413997593025, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11378062601822118, + "total_car_to_hidden_at_8": 0.14744710053006807 + }, + "1475440973": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2959344387054443, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15809903475948942, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7040655612945557, + "selected_utility_gain_over_base_at_8": -1.2959344387054443, + "selector_regret_at_8": 1.2959344387054443, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2959344387054443 + }, + "1483645336": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06905173445346233, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1522272686": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.056304022669792175, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21028164676312622, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23198044300079346, + "proposal_oracle_utility_gain_over_base_at_8": 0.033537328243255615, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.17567642033100128, + "selected_utility_gain_over_base_at_8": -0.02276669442653656, + "selector_regret_at_8": 0.056304022669792175, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7680195569992065, + "total_car_to_hidden_at_8": 1.8243235796689987 + }, + "1528740417": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3242760158848682, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1549324805": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.010487629100680351, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2818476718903797, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6867360668256879, + "proposal_oracle_utility_gain_over_base_at_8": -0.6920113814994693, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6762484377250075, + "selected_utility_gain_over_base_at_8": -0.7024990106001496, + "selector_regret_at_8": 0.010487629100680351, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7160144550725818, + "total_car_to_hidden_at_8": 0.7265020841732621 + }, + "1607773860": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.185863733291626, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2143457671242473, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814136266708374, + "selected_utility_gain_over_base_at_8": -0.01853656768798828, + "selector_regret_at_8": 1.185863733291626, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185863733291626 + }, + "1665739511": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.868122011423111, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.26263826459603723, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13187798857688904, + "selected_utility_gain_over_base_at_8": -1.868122011423111, + "selector_regret_at_8": 1.868122011423111, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.868122011423111 + }, + "1747178030": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33194730961537694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8533687740564346, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1881223807": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1643392210528543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.040287479758262634, + "proposal_oracle_utility_gain_over_base_at_8": 0.008260488510131836, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.040287479758262634, + "selected_utility_gain_over_base_at_8": 0.008260488510131836, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9597125202417374, + "total_car_to_hidden_at_8": 1.9597125202417374 + }, + "1919198188": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.019107162952423096, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21294576266807125, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04859863221645355, + "proposal_oracle_utility_gain_over_base_at_8": 0.011399760842323303, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029491469264030457, + "selected_utility_gain_over_base_at_8": -0.0077074021100997925, + "selector_regret_at_8": 0.019107162952423096, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9514013677835464, + "total_car_to_hidden_at_8": 1.9705085307359695 + }, + "1991414747": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9767832309007645, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.613109405537996, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.023216769099235535, + "selected_utility_gain_over_base_at_8": -1.9767832309007645, + "selector_regret_at_8": 1.9767832309007645, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9767832309007645 + }, + "1997090716": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21318958083944717, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2015956957": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.4314236156642437, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.24731300175116924, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.45940959043800833, + "proposal_oracle_utility_gain_over_base_at_8": 0.3922898005694151, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02798597477376461, + "selected_utility_gain_over_base_at_8": -0.039133815094828606, + "selector_regret_at_8": 0.4314236156642437, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.07047177664935589, + "total_car_to_hidden_at_8": 0.5018953923135996 + }, + "2054899126": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.08971682945849302, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1611655950546265, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2141802816": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.954741045832634, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3183279052482729, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04525895416736603, + "selected_utility_gain_over_base_at_8": -0.025548294186592102, + "selector_regret_at_8": 1.954741045832634, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.954741045832634 + }, + "256770874": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07907943427562714, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5133629211325229, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.19852638244628906, + "proposal_oracle_utility_gain_over_base_at_8": -1.801473617553711, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11944694817066193, + "selected_utility_gain_over_base_at_8": -1.880553051829338, + "selector_regret_at_8": 0.07907943427562714, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.801473617553711, + "total_car_to_hidden_at_8": 1.880553051829338 + }, + "278008524": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0040267681470140815, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.40905575479886586, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01439405232667923, + "proposal_oracle_utility_gain_over_base_at_8": -0.09307781606912613, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.010367284179665148, + "selected_utility_gain_over_base_at_8": -0.09710458421614021, + "selector_regret_at_8": 0.0040267681470140815, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0994171425700188, + "total_car_to_hidden_at_8": 1.1034439107170328 + }, + "416183106": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.04582805931568146, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21826032886833752, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23963995277881622, + "proposal_oracle_utility_gain_over_base_at_8": 0.03931353986263275, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.19381189346313477, + "selected_utility_gain_over_base_at_8": -0.006514519453048706, + "selector_regret_at_8": 0.04582805931568146, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7603600472211838, + "total_car_to_hidden_at_8": 1.8061881065368652 + }, + "534717141": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15703220108978472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.287484347820282, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4327797462092032, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.712515652179718, + "selected_utility_gain_over_base_at_8": -0.011456906795501709, + "selector_regret_at_8": 1.287484347820282, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.287484347820282 + }, + "565369392": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.057756672302881874, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2176430432872923, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7870745907227198, + "proposal_oracle_utility_gain_over_base_at_8": -0.012691055734952291, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.729317918419838, + "selected_utility_gain_over_base_at_8": -0.07044772803783417, + "selector_regret_at_8": 0.057756672302881874, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.6395569890737534, + "total_car_to_hidden_at_8": 0.6973136613766352 + }, + "565621826": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.006800346697370212, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22254352429757418, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.029573063055674236, + "proposal_oracle_utility_gain_over_base_at_8": -0.04571903496980667, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.022772716358304024, + "selected_utility_gain_over_base_at_8": -0.05251938166717688, + "selector_regret_at_8": 0.006800346697370212, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2080739289522171, + "total_car_to_hidden_at_8": 0.2148742756495873 + }, + "626486813": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.013256545644253492, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.14946876705188256, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06192737747915089, + "proposal_oracle_utility_gain_over_base_at_8": -0.4435126391472295, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.0486708318348974, + "selected_utility_gain_over_base_at_8": -0.456769184791483, + "selector_regret_at_8": 0.013256545644253492, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5505410328041762, + "total_car_to_hidden_at_8": 0.5637975784484297 + }, + "636680137": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.028240702114999294, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.6813618274184183, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.036433976143598557, + "proposal_oracle_utility_gain_over_base_at_8": -1.9635660238564014, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.008193274028599262, + "selected_utility_gain_over_base_at_8": -1.9918067259714007, + "selector_regret_at_8": 0.028240702114999294, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9635660238564014, + "total_car_to_hidden_at_8": 1.9918067259714007 + }, + "706479600": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15321122852132601, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1649112105369568, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "750448296": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.03573483042418957, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19031133205081463, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08245231583714485, + "proposal_oracle_utility_gain_over_base_at_8": -0.02908124402165413, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.046717485412955284, + "selected_utility_gain_over_base_at_8": -0.0648160744458437, + "selector_regret_at_8": 0.03573483042418957, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15335527434945107, + "total_car_to_hidden_at_8": 0.18909010477364063 + }, + "75070641": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.048324257135391235, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2643509727736545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21781498193740845, + "proposal_oracle_utility_gain_over_base_at_8": 0.015063345432281494, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1694907248020172, + "selected_utility_gain_over_base_at_8": -0.03326091170310974, + "selector_regret_at_8": 0.048324257135391235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7821850180625916, + "total_car_to_hidden_at_8": 1.8305092751979828 + }, + "767747424": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5082327332347631, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20625644377911734, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5790140572935343, + "proposal_oracle_utility_gain_over_base_at_8": 0.03148986957967281, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.07078132405877113, + "selected_utility_gain_over_base_at_8": -0.47674286365509033, + "selector_regret_at_8": 0.5082327332347631, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05956761725246906, + "total_car_to_hidden_at_8": 0.11785433255136013 + }, + "77147264": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.9378202110528946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13662790487889595, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.11046252399683, + "proposal_oracle_utility_gain_over_base_at_8": -0.3111169561743736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.5, + "selected_utility_at_8": 0.1726423129439354, + "selected_utility_gain_over_base_at_8": -1.2489371672272682, + "selector_regret_at_8": 0.9378202110528946, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.88953747600317, + "total_car_to_hidden_at_8": 1.8273576870560646 + }, + "794181626": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1880089150142482, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "957101489": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.38811931014060974, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5372965735602796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5639582276344299, + "proposal_oracle_utility_gain_over_base_at_8": 0.36023636162281036, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1758389174938202, + "selected_utility_gain_over_base_at_8": -0.027882948517799377, + "selector_regret_at_8": 0.38811931014060974, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.43604177236557, + "total_car_to_hidden_at_8": 1.8241610825061798 + }, + "998761162": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.14873333521535068, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics_by_task.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..f6aa04bc9a8cd61d3b1fd701266d1d3f6ed9d25c --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.5199628497163454, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.24381101581290862, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.16666666666666666, + "proposal_oracle_utility_at_8": 1.3565248747666676, + "proposal_oracle_utility_gain_over_base_at_8": 0.2895807276169459, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8365620250503222, + "selected_utility_gain_over_base_at_8": -0.23038212209939957, + "selector_regret_at_8": 0.5199628497163454, + "success_selector_gap_at_8": 0.3333333333333333, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.6434751252333323, + "total_car_to_hidden_at_8": 1.1634379749496777 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "total_car_to_hidden_at_8": 1.922220028936863 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.13753403203822123, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.35294117647058826, + "pairwise_causal_calibration_ece": 0.2291649560920931, + "proposal_oracle_success_at_8": 0.23529411764705882, + "proposal_oracle_success_gain_over_base_at_8": -0.058823529411764705, + "proposal_oracle_utility_at_8": 0.5247960129156982, + "proposal_oracle_utility_gain_over_base_at_8": -0.1264434560058255, + "selected_success_at_8": 0.17647058823529413, + "selected_success_gain_over_base_at_8": -0.11764705882352941, + "selected_utility_at_8": 0.38726198087747704, + "selected_utility_gain_over_base_at_8": -0.26397748804404675, + "selector_regret_at_8": 0.13753403203822123, + "success_selector_gap_at_8": 0.058823529411764705, + "success_support_gap_at_8": 0.11764705882352941, + "success_total_car_to_hidden_at_8": 0.14705882352941177, + "support_gap_at_8": 0.31061309059355957, + "total_car_to_hidden_at_8": 0.3950598559219061 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.597546140734966, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6153846153846154, + "pairwise_causal_calibration_ece": 0.24319123348754632, + "proposal_oracle_success_at_8": 0.8461538461538461, + "proposal_oracle_success_gain_over_base_at_8": 0.5384615384615384, + "proposal_oracle_utility_at_8": 1.6972852945327759, + "proposal_oracle_utility_gain_over_base_at_8": 1.0201208339287684, + "selected_success_at_8": 0.5384615384615384, + "selected_success_gain_over_base_at_8": 0.23076923076923078, + "selected_utility_at_8": 1.0997391537978098, + "selected_utility_gain_over_base_at_8": 0.4225746931938025, + "selector_regret_at_8": 0.597546140734966, + "success_selector_gap_at_8": 0.3076923076923077, + "success_support_gap_at_8": 0.15384615384615385, + "success_total_car_to_hidden_at_8": 0.46153846153846156, + "support_gap_at_8": 0.3027147054672241, + "total_car_to_hidden_at_8": 0.9002608462021902 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.4918266286452611, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.7333333333333333, + "pairwise_causal_calibration_ece": 0.22797947617564457, + "proposal_oracle_success_at_8": 0.4666666666666667, + "proposal_oracle_success_gain_over_base_at_8": 0.26666666666666666, + "proposal_oracle_utility_at_8": 1.1057716061671574, + "proposal_oracle_utility_gain_over_base_at_8": 0.30266105631987256, + "selected_success_at_8": 0.2, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6139449775218964, + "selected_utility_gain_over_base_at_8": -0.18916557232538858, + "selector_regret_at_8": 0.4918266286452611, + "success_selector_gap_at_8": 0.26666666666666666, + "success_support_gap_at_8": 0.5333333333333333, + "success_total_car_to_hidden_at_8": 0.8, + "support_gap_at_8": 0.8942283938328425, + "total_car_to_hidden_at_8": 1.3860550224781036 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/report.md b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/report.md new file mode 100644 index 0000000000000000000000000000000000000000..e9a71c9d3368ec022cfe935d9b674bf723f03e90 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/report.md @@ -0,0 +1,27 @@ +# Metric Evaluation (measured) + +K: `8` + +| Metric | N | Micro mean | 95% CI | Task macro | Seed macro | +| --- | ---: | ---: | ---: | ---: | ---: | +| base_success | 69 | 0.2754 | [0.1884, 0.4058] | 0.2270 | 0.3085 | +| base_utility | 69 | 0.7181 | [0.5449, 0.9239] | 0.6704 | 0.8164 | +| branch_car_at_8 | 69 | 0.3382 | [0.2130, 0.4945] | 0.4280 | 0.3814 | +| hidden_chart_oracle_success_at_8 | 69 | 0.6667 | [0.5507, 0.7826] | 0.8647 | 0.8358 | +| hidden_chart_oracle_utility_at_8 | 69 | 1.4000 | [1.1926, 1.6081] | 1.7565 | 1.7062 | +| outcome_ptr_at_8 | 69 | 0.5072 | [0.3768, 0.6232] | 0.6403 | 0.5560 | +| pairwise_causal_calibration_ece | 69 | 0.2326 | [0.2041, 0.2639] | 0.2315 | 0.2372 | +| proposal_oracle_success_at_8 | 69 | 0.4203 | [0.3188, 0.5362] | 0.4096 | 0.5238 | +| proposal_oracle_success_gain_over_base_at_8 | 69 | 0.1449 | [-0.0145, 0.2609] | 0.1826 | 0.2152 | +| proposal_oracle_utility_at_8 | 69 | 0.9435 | [0.7415, 1.1701] | 1.0310 | 1.1692 | +| proposal_oracle_utility_gain_over_base_at_8 | 69 | 0.2255 | [-0.0502, 0.4211] | 0.3607 | 0.3529 | +| selected_success_at_8 | 69 | 0.2464 | [0.1449, 0.3623] | 0.2163 | 0.3227 | +| selected_success_gain_over_base_at_8 | 69 | -0.0290 | [-0.1739, 0.0870] | -0.0107 | 0.0142 | +| selected_utility_at_8 | 69 | 0.6054 | [0.4142, 0.8217] | 0.6031 | 0.7878 | +| selected_utility_gain_over_base_at_8 | 69 | -0.1127 | [-0.3777, 0.1160] | -0.0673 | -0.0285 | +| selector_regret_at_8 | 69 | 0.3382 | [0.2130, 0.4945] | 0.4280 | 0.3814 | +| success_selector_gap_at_8 | 69 | 0.1739 | [0.1014, 0.2609] | 0.1933 | 0.2011 | +| success_support_gap_at_8 | 69 | 0.2609 | [0.1594, 0.3333] | 0.4610 | 0.3174 | +| success_total_car_to_hidden_at_8 | 69 | 0.4203 | [0.3188, 0.5507] | 0.6484 | 0.5131 | +| support_gap_at_8 | 69 | 0.4826 | [0.3068, 0.6271] | 0.7360 | 0.5466 | +| total_car_to_hidden_at_8 | 69 | 0.7946 | [0.6198, 0.9978] | 1.1534 | 0.9184 | diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/table.tex b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..21300725db63efe7c67a714392789bd66a841e3e --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/measured_metrics/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 69 & 0.2754 & 0.1884 & 0.4058 \\ +base\_utility & 69 & 0.7181 & 0.5449 & 0.9239 \\ +branch\_car\_at\_8 & 69 & 0.3382 & 0.2130 & 0.4945 \\ +hidden\_chart\_oracle\_success\_at\_8 & 69 & 0.6667 & 0.5507 & 0.7826 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 69 & 1.4000 & 1.1926 & 1.6081 \\ +outcome\_ptr\_at\_8 & 69 & 0.5072 & 0.3768 & 0.6232 \\ +pairwise\_causal\_calibration\_ece & 69 & 0.2326 & 0.2041 & 0.2639 \\ +proposal\_oracle\_success\_at\_8 & 69 & 0.4203 & 0.3188 & 0.5362 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 69 & 0.1449 & -0.0145 & 0.2609 \\ +proposal\_oracle\_utility\_at\_8 & 69 & 0.9435 & 0.7415 & 1.1701 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 69 & 0.2255 & -0.0502 & 0.4211 \\ +selected\_success\_at\_8 & 69 & 0.2464 & 0.1449 & 0.3623 \\ +selected\_success\_gain\_over\_base\_at\_8 & 69 & -0.0290 & -0.1739 & 0.0870 \\ +selected\_utility\_at\_8 & 69 & 0.6054 & 0.4142 & 0.8217 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 69 & -0.1127 & -0.3777 & 0.1160 \\ +selector\_regret\_at\_8 & 69 & 0.3382 & 0.2130 & 0.4945 \\ +success\_selector\_gap\_at\_8 & 69 & 0.1739 & 0.1014 & 0.2609 \\ +success\_support\_gap\_at\_8 & 69 & 0.2609 & 0.1594 & 0.3333 \\ +success\_total\_car\_to\_hidden\_at\_8 & 69 & 0.4203 & 0.3188 & 0.5507 \\ +support\_gap\_at\_8 & 69 & 0.4826 & 0.3068 & 0.6271 \\ +total\_car\_to\_hidden\_at\_8 & 69 & 0.7946 & 0.6198 & 0.9978 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics.json new file mode 100644 index 0000000000000000000000000000000000000000..d2a4c8871e194a3181d9f75a388125ad3aa19bee --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics.json @@ -0,0 +1,1918 @@ +{ + "checkpoint": "runs/ctt_residual_base_context_obj_seed2/model.pt", + "k": 8, + "measured_metric_summary": { + "base_success": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.46099290780141844, + "low": 0.19680851063829788, + "mean": 0.3085106382978724, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3085106382978724, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 1.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 0.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3176470588235294, + "low": 0.10153846153846154, + "mean": 0.2270286576168929, + "n": 5, + "num_samples": 200 + }, + "mean": 0.2270286576168929, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.29411764705882354, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.2 + } + }, + "metric": "base_success", + "micro": { + "confidence": 0.95, + "high": 0.4057971014492754, + "low": 0.18840579710144928, + "mean": 0.2753623188405797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "base_utility": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.0808477015729916, + "low": 0.6013251688004412, + "mean": 0.8163502178883101, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8163502178883101, + "per_seed": { + "1025740212": 0.528373658657074, + "1055157354": 0.19648407399654388, + "1055567618": 2.0, + "1113268940": 0.42231252789497375, + "1113371261": 2.0, + "1176943004": 0.15461373329162598, + "1189392917": 0.10841864347457886, + "1231780997": 0.7270061373710632, + "1271718007": 1.415381133556366, + "1273247072": 0.15342965722084045, + "1358421325": 0.8272150158882141, + "1375492786": 2.0, + "141984615": 0.09381213908394177, + "1475440973": 2.0, + "1483645336": 0.01641605794429779, + "1522272686": 0.19844311475753784, + "1528740417": 2.0, + "1549324805": 1.3787474483251572, + "1607773860": 0.8326728343963623, + "1665739511": 2.0, + "1747178030": 0.14663122594356537, + "1881223807": 0.0320269912481308, + "1919198188": 0.03719887137413025, + "1991414747": 2.0, + "1997090716": 0.13927479088306427, + "2015956957": 0.06711978986859321, + "2054899126": 2.0, + "211245842": 0.8388344049453735, + "2141802816": 0.07080724835395813, + "256770874": 2.0, + "278008524": 0.10747186839580536, + "416183106": 0.20032641291618347, + "534717141": 2.0, + "551504635": 0.7239725589752197, + "565369392": 0.7997656464576721, + "565621826": 2.0, + "585904130": 0.07529209802548091, + "626486813": 0.5054400166263804, + "636680137": 2.0, + "706479600": 0.8350887894630432, + "750448296": 0.11153355985879898, + "75070641": 0.20275163650512695, + "767747424": 0.5475241877138615, + "77147264": 1.4215794801712036, + "794181626": 0.18095605075359344, + "957101489": 0.20372186601161957, + "998761162": 0.06781657040119171 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.9058472741949395, + "low": 0.4133020142714182, + "mean": 0.6703776567486757, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6703776567486757, + "per_task": { + "LiftPegUpright-v1": 1.0669441471497219, + "PickCube-v1": 0.15342965722084045, + "PullCube-v1": 0.6512394689215237, + "PushCube-v1": 0.6771644606040075, + "StackCube-v1": 0.8031105498472849 + } + }, + "metric": "base_utility", + "micro": { + "confidence": 0.95, + "high": 0.923898630168127, + "low": 0.5448560892550302, + "mean": 0.7180729379567007, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "branch_car_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5746948152998502, + "low": 0.20552555694829355, + "mean": 0.38140176123681857, + "n": 47, + "num_samples": 200 + }, + "mean": 0.38140176123681857, + "per_seed": { + "1025740212": 0.051884353160858154, + "1055157354": 0.0, + "1055567618": 1.8424437642097473, + "1113268940": 0.48447395861148834, + "1113371261": 0.0, + "1176943004": 1.9587705433368683, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.9200049638748169, + "1273247072": 0.39307598024606705, + "1358421325": 0.0, + "1375492786": 0.00036396586801856756, + "141984615": 0.03539413997593025, + "1475440973": 1.2959344387054443, + "1483645336": 0.0, + "1522272686": 0.056304022669792175, + "1528740417": 0.0, + "1549324805": 0.010487629100680351, + "1607773860": 1.185863733291626, + "1665739511": 1.868122011423111, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.019107162952423096, + "1991414747": 1.9767832309007645, + "1997090716": 0.0, + "2015956957": 0.4314236156642437, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.954741045832634, + "256770874": 0.07907943427562714, + "278008524": 0.0040267681470140815, + "416183106": 0.04582805931568146, + "534717141": 0.0, + "551504635": 1.287484347820282, + "565369392": 0.057756672302881874, + "565621826": 0.0, + "585904130": 0.006800346697370212, + "626486813": 0.013256545644253492, + "636680137": 0.028240702114999294, + "706479600": 0.0, + "750448296": 0.03573483042418957, + "75070641": 0.048324257135391235, + "767747424": 0.5082327332347631, + "77147264": 0.9378202110528946, + "794181626": 0.0, + "957101489": 0.38811931014060974, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.53974167769536, + "low": 0.2651281852154152, + "mean": 0.4279891262761722, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4279891262761722, + "per_task": { + "LiftPegUpright-v1": 0.5199628497163454, + "PickCube-v1": 0.39307598024606705, + "PullCube-v1": 0.13753403203822123, + "PushCube-v1": 0.597546140734966, + "StackCube-v1": 0.4918266286452611 + } + }, + "metric": "branch_car_at_8", + "micro": { + "confidence": 0.95, + "high": 0.494531684580421, + "low": 0.2130161163808805, + "mean": 0.3381812960445962, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9148936170212765, + "low": 0.7432624113475178, + "mean": 0.8358156028368794, + "n": 47, + "num_samples": 200 + }, + "mean": 0.8358156028368794, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 1.0, + "1271718007": 1.0, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 1.0, + "1549324805": 0.6666666666666666, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.6666666666666666, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.0, + "low": 0.5941176470588235, + "mean": 0.8647058823529413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.8647058823529413, + "per_task": { + "LiftPegUpright-v1": 1.0, + "PickCube-v1": 1.0, + "PullCube-v1": 0.3235294117647059, + "PushCube-v1": 1.0, + "StackCube-v1": 1.0 + } + }, + "metric": "hidden_chart_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.782608695652174, + "low": 0.5507246376811594, + "mean": 0.6666666666666666, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "hidden_chart_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.850041654515774, + "low": 1.5397213380149701, + "mean": 1.7062028111638965, + "n": 47, + "num_samples": 200 + }, + "mean": 1.7062028111638965, + "per_seed": { + "1025740212": 2.0, + "1055157354": 2.0, + "1055567618": 2.0, + "1113268940": 2.0, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 2.0, + "1271718007": 2.0, + "1273247072": 2.0, + "1358421325": 2.0, + "1375492786": 0.24345096945762634, + "141984615": 0.20044784247875214, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 2.0, + "1528740417": 2.0, + "1549324805": 1.4027505218982697, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 2.0, + "1919198188": 2.0, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.5298813670873642, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 2.0, + "278008524": 1.113811194896698, + "416183106": 2.0, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 1.4266315797964733, + "565621826": 2.0, + "585904130": 0.23764699200789133, + "626486813": 0.6124684102833271, + "636680137": 2.0, + "706479600": 2.0, + "750448296": 0.23580759018659592, + "75070641": 2.0, + "767747424": 0.18863565661013126, + "77147264": 2.0, + "794181626": 2.0, + "957101489": 2.0, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 2.0, + "low": 1.2693931020796299, + "mean": 1.7564643673598765, + "n": 5, + "num_samples": 200 + }, + "mean": 1.7564643673598765, + "per_task": { + "LiftPegUpright-v1": 2.0, + "PickCube-v1": 2.0, + "PullCube-v1": 0.7823218367993832, + "PushCube-v1": 2.0, + "StackCube-v1": 2.0 + } + }, + "metric": "hidden_chart_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.6080652458080347, + "low": 1.1926357444861662, + "mean": 1.3999846732054932, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "outcome_ptr_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.676595744680851, + "low": 0.3964539007092199, + "mean": 0.5560283687943263, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5560283687943263, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.3333333333333333, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.8, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.75, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.75, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 1.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.823076923076923, + "low": 0.45843137254901956, + "mean": 0.6403318250377075, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6403318250377075, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 1.0, + "PullCube-v1": 0.35294117647058826, + "PushCube-v1": 0.6153846153846154, + "StackCube-v1": 0.7333333333333333 + } + }, + "metric": "outcome_ptr_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6231884057971014, + "low": 0.37681159420289856, + "mean": 0.5072463768115942, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "pairwise_causal_calibration_ece": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.28064851230050547, + "low": 0.1975970437069943, + "mean": 0.237227826019069, + "n": 47, + "num_samples": 200 + }, + "mean": 0.237227826019069, + "per_seed": { + "1025740212": 0.12800533274597592, + "1055157354": 0.06066548841424423, + "1055567618": 0.22991638132379041, + "1113268940": 0.2534904144040469, + "1113371261": 0.13380940221299786, + "1176943004": 0.2664806376409974, + "1189392917": 0.33283923875771465, + "1231780997": 0.16621555087386797, + "1271718007": 0.20722796191327264, + "1273247072": 0.2134820953504291, + "1358421325": 0.09625642405798478, + "1375492786": 0.29573093343728263, + "141984615": 0.22785399017277783, + "1475440973": 0.15809903475948942, + "1483645336": 0.06905173445346233, + "1522272686": 0.21028164676312622, + "1528740417": 0.3242760158848682, + "1549324805": 0.2818476718903797, + "1607773860": 0.2143457671242473, + "1665739511": 0.26263826459603723, + "1747178030": 0.33194730961537694, + "1881223807": 0.1643392210528543, + "1919198188": 0.21294576266807125, + "1991414747": 0.613109405537996, + "1997090716": 0.21318958083944717, + "2015956957": 0.24731300175116924, + "2054899126": 0.0, + "211245842": 0.08971682945849302, + "2141802816": 0.3183279052482729, + "256770874": 0.5133629211325229, + "278008524": 0.40905575479886586, + "416183106": 0.21826032886833752, + "534717141": 0.15703220108978472, + "551504635": 0.4327797462092032, + "565369392": 0.2176430432872923, + "565621826": 0.0, + "585904130": 0.22254352429757418, + "626486813": 0.14946876705188256, + "636680137": 0.6813618274184183, + "706479600": 0.15321122852132601, + "750448296": 0.19031133205081463, + "75070641": 0.2643509727736545, + "767747424": 0.20625644377911734, + "77147264": 0.13662790487889595, + "794181626": 0.1880089150142482, + "957101489": 0.5372965735602796, + "998761162": 0.14873333521535068 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.24050993447352814, + "low": 0.2197552396470947, + "mean": 0.23152575538372436, + "n": 5, + "num_samples": 200 + }, + "mean": 0.23152575538372436, + "per_task": { + "LiftPegUpright-v1": 0.24381101581290862, + "PickCube-v1": 0.2134820953504291, + "PullCube-v1": 0.2291649560920931, + "PushCube-v1": 0.24319123348754632, + "StackCube-v1": 0.22797947617564457 + } + }, + "metric": "pairwise_causal_calibration_ece", + "micro": { + "confidence": 0.95, + "high": 0.26385682559725093, + "low": 0.20411366044304913, + "mean": 0.23259615761350452, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6372340425531915, + "low": 0.40212765957446805, + "mean": 0.5237588652482269, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5237588652482269, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 1.0, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 1.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.6547511312217195, + "low": 0.18666666666666668, + "mean": 0.40962292609351436, + "n": 5, + "num_samples": 200 + }, + "mean": 0.40962292609351436, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 0.0, + "PullCube-v1": 0.23529411764705882, + "PushCube-v1": 0.8461538461538461, + "StackCube-v1": 0.4666666666666667 + } + }, + "metric": "proposal_oracle_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5362318840579711, + "low": 0.3188405797101449, + "mean": 0.42028985507246375, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.3446808510638298, + "low": 0.03900709219858156, + "mean": 0.21524822695035462, + "n": 47, + "num_samples": 200 + }, + "mean": 0.21524822695035462, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": -0.3333333333333333, + "1607773860": 1.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 1.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3646455505279035, + "low": 0.009803921568627449, + "mean": 0.18259426847662139, + "n": 5, + "num_samples": 200 + }, + "mean": 0.18259426847662139, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.058823529411764705, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "proposal_oracle_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2608695652173913, + "low": -0.014492753623188406, + "mean": 0.14492753623188406, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.3754943914410942, + "low": 0.9438222326516629, + "mean": 1.1692132722806012, + "n": 47, + "num_samples": 200 + }, + "mean": 1.1692132722806012, + "per_seed": { + "1025740212": 0.7146708369255066, + "1055157354": 0.8651304244995117, + "1055567618": 2.0, + "1113268940": 0.6990174651145935, + "1113371261": 2.0, + "1176943004": 2.0, + "1189392917": 2.0, + "1231780997": 0.725460946559906, + "1271718007": 2.0, + "1273247072": 0.4708559513092041, + "1358421325": 2.0, + "1375492786": 0.0012095769634470344, + "141984615": 0.08839488192461431, + "1475440973": 2.0, + "1483645336": 2.0, + "1522272686": 0.23198044300079346, + "1528740417": 2.0, + "1549324805": 0.6867360668256879, + "1607773860": 2.0, + "1665739511": 2.0, + "1747178030": 2.0, + "1881223807": 0.040287479758262634, + "1919198188": 0.04859863221645355, + "1991414747": 2.0, + "1997090716": 2.0, + "2015956957": 0.45940959043800833, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 2.0, + "256770874": 0.19852638244628906, + "278008524": 0.01439405232667923, + "416183106": 0.23963995277881622, + "534717141": 2.0, + "551504635": 2.0, + "565369392": 0.7870745907227198, + "565621826": 2.0, + "585904130": 0.029573063055674236, + "626486813": 0.06192737747915089, + "636680137": 0.036433976143598557, + "706479600": 2.0, + "750448296": 0.08245231583714485, + "75070641": 0.21781498193740845, + "767747424": 0.5790140572935343, + "77147264": 1.11046252399683, + "794181626": 2.0, + "957101489": 0.5639582276344299, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.3946353542561387, + "low": 0.6302031192446912, + "mean": 1.0310467479383005, + "n": 5, + "num_samples": 200 + }, + "mean": 1.0310467479383005, + "per_task": { + "LiftPegUpright-v1": 1.3565248747666676, + "PickCube-v1": 0.4708559513092041, + "PullCube-v1": 0.5247960129156982, + "PushCube-v1": 1.6972852945327759, + "StackCube-v1": 1.1057716061671574 + } + }, + "metric": "proposal_oracle_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 1.1700718992818282, + "low": 0.7414891867108805, + "mean": 0.9435413414561797, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "proposal_oracle_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5938770614889721, + "low": 0.05693793400762114, + "mean": 0.3528630543922914, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3528630543922914, + "per_seed": { + "1025740212": 0.18629717826843262, + "1055157354": 0.6686463505029678, + "1055567618": 0.0, + "1113268940": 0.27670493721961975, + "1113371261": 0.0, + "1176943004": 1.845386266708374, + "1189392917": 1.8915813565254211, + "1231780997": -0.0015451908111572266, + "1271718007": 0.584618866443634, + "1273247072": 0.31742629408836365, + "1358421325": 1.1727849841117859, + "1375492786": -1.998790423036553, + "141984615": -0.005417257159327467, + "1475440973": 0.0, + "1483645336": 1.9835839420557022, + "1522272686": 0.033537328243255615, + "1528740417": 0.0, + "1549324805": -0.6920113814994693, + "1607773860": 1.1673271656036377, + "1665739511": 0.0, + "1747178030": 1.8533687740564346, + "1881223807": 0.008260488510131836, + "1919198188": 0.011399760842323303, + "1991414747": 0.0, + "1997090716": 1.8607252091169357, + "2015956957": 0.3922898005694151, + "2054899126": 0.0, + "211245842": 1.1611655950546265, + "2141802816": 1.9291927516460419, + "256770874": -1.801473617553711, + "278008524": -0.09307781606912613, + "416183106": 0.03931353986263275, + "534717141": 0.0, + "551504635": 1.2760274410247803, + "565369392": -0.012691055734952291, + "565621826": 0.0, + "585904130": -0.04571903496980667, + "626486813": -0.4435126391472295, + "636680137": -1.9635660238564014, + "706479600": 1.1649112105369568, + "750448296": -0.02908124402165413, + "75070641": 0.015063345432281494, + "767747424": 0.03148986957967281, + "77147264": -0.3111169561743736, + "794181626": 1.8190439492464066, + "957101489": 0.36023636162281036, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7279047914040394, + "low": 0.04553533073756662, + "mean": 0.360669091189625, + "n": 5, + "num_samples": 200 + }, + "mean": 0.360669091189625, + "per_task": { + "LiftPegUpright-v1": 0.2895807276169459, + "PickCube-v1": 0.31742629408836365, + "PullCube-v1": -0.1264434560058255, + "PushCube-v1": 1.0201208339287684, + "StackCube-v1": 0.30266105631987256 + } + }, + "metric": "proposal_oracle_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.4210502535502926, + "low": -0.050174038460734635, + "mean": 0.225468403499479, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.43617021276595747, + "low": 0.19503546099290783, + "mean": 0.3226950354609929, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3226950354609929, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 0.0, + "1113268940": 0.0, + "1113371261": 1.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 1.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 0.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 1.0, + "211245842": 1.0, + "2141802816": 0.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 1.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 1.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.0, + "77147264": 0.0, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.3917043740573153, + "low": 0.07529411764705882, + "mean": 0.21631975867269984, + "n": 5, + "num_samples": 200 + }, + "mean": 0.21631975867269984, + "per_task": { + "LiftPegUpright-v1": 0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": 0.17647058823529413, + "PushCube-v1": 0.5384615384615384, + "StackCube-v1": 0.2 + } + }, + "metric": "selected_success_at_8", + "micro": { + "confidence": 0.95, + "high": 0.36231884057971014, + "low": 0.14492753623188406, + "mean": 0.2463768115942029, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_success_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.16312056737588654, + "low": -0.16666666666666666, + "mean": 0.014184397163120569, + "n": 47, + "num_samples": 200 + }, + "mean": 0.014184397163120569, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": -1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 1.0, + "1231780997": 0.0, + "1271718007": 0.0, + "1273247072": 0.0, + "1358421325": 1.0, + "1375492786": -1.0, + "141984615": 0.0, + "1475440973": -1.0, + "1483645336": 1.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": -0.3333333333333333, + "1607773860": 0.0, + "1665739511": -1.0, + "1747178030": 1.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": -1.0, + "1997090716": 1.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 1.0, + "2141802816": 0.0, + "256770874": -1.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": -0.25, + "636680137": -1.0, + "706479600": 1.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": -0.25, + "77147264": -0.5, + "794181626": 1.0, + "957101489": 0.0, + "998761162": 1.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.10512820512820516, + "low": -0.12352941176470586, + "mean": -0.010708898944193057, + "n": 5, + "num_samples": 200 + }, + "mean": -0.010708898944193057, + "per_task": { + "LiftPegUpright-v1": -0.16666666666666666, + "PickCube-v1": 0.0, + "PullCube-v1": -0.11764705882352941, + "PushCube-v1": 0.23076923076923078, + "StackCube-v1": 0.0 + } + }, + "metric": "selected_success_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.08695652173913043, + "low": -0.17391304347826086, + "mean": -0.028985507246376812, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.9823760857321785, + "low": 0.5794855141018167, + "mean": 0.7878115110437829, + "n": 47, + "num_samples": 200 + }, + "mean": 0.7878115110437829, + "per_seed": { + "1025740212": 0.6627864837646484, + "1055157354": 0.8651304244995117, + "1055567618": 0.15755623579025269, + "1113268940": 0.21454350650310516, + "1113371261": 2.0, + "1176943004": 0.041229456663131714, + "1189392917": 2.0, + "1231780997": 0.725460946559906, + "1271718007": 1.079995036125183, + "1273247072": 0.07777997106313705, + "1358421325": 2.0, + "1375492786": 0.0008456110954284668, + "141984615": 0.05300074194868406, + "1475440973": 0.7040655612945557, + "1483645336": 2.0, + "1522272686": 0.17567642033100128, + "1528740417": 2.0, + "1549324805": 0.6762484377250075, + "1607773860": 0.814136266708374, + "1665739511": 0.13187798857688904, + "1747178030": 2.0, + "1881223807": 0.040287479758262634, + "1919198188": 0.029491469264030457, + "1991414747": 0.023216769099235535, + "1997090716": 2.0, + "2015956957": 0.02798597477376461, + "2054899126": 2.0, + "211245842": 2.0, + "2141802816": 0.04525895416736603, + "256770874": 0.11944694817066193, + "278008524": 0.010367284179665148, + "416183106": 0.19381189346313477, + "534717141": 2.0, + "551504635": 0.712515652179718, + "565369392": 0.729317918419838, + "565621826": 2.0, + "585904130": 0.022772716358304024, + "626486813": 0.0486708318348974, + "636680137": 0.008193274028599262, + "706479600": 2.0, + "750448296": 0.046717485412955284, + "75070641": 0.1694907248020172, + "767747424": 0.07078132405877113, + "77147264": 0.1726423129439354, + "794181626": 2.0, + "957101489": 0.1758389174938202, + "998761162": 2.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.8953473172508752, + "low": 0.2922459736466408, + "mean": 0.6030576216621284, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6030576216621284, + "per_task": { + "LiftPegUpright-v1": 0.8365620250503222, + "PickCube-v1": 0.07777997106313705, + "PullCube-v1": 0.38726198087747704, + "PushCube-v1": 1.0997391537978098, + "StackCube-v1": 0.6139449775218964 + } + }, + "metric": "selected_utility_at_8", + "micro": { + "confidence": 0.95, + "high": 0.821739717160581, + "low": 0.4142426509491127, + "mean": 0.6053600454115835, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selected_utility_gain_over_base_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.22746946147382124, + "low": -0.3520016530264459, + "mean": -0.028538706844527254, + "n": 47, + "num_samples": 200 + }, + "mean": -0.028538706844527254, + "per_seed": { + "1025740212": 0.13441282510757446, + "1055157354": 0.6686463505029678, + "1055567618": -1.8424437642097473, + "1113268940": -0.2077690213918686, + "1113371261": 0.0, + "1176943004": -0.11338427662849426, + "1189392917": 1.8915813565254211, + "1231780997": -0.0015451908111572266, + "1271718007": -0.33538609743118286, + "1273247072": -0.0756496861577034, + "1358421325": 1.1727849841117859, + "1375492786": -1.9991543889045715, + "141984615": -0.04081139713525772, + "1475440973": -1.2959344387054443, + "1483645336": 1.9835839420557022, + "1522272686": -0.02276669442653656, + "1528740417": 0.0, + "1549324805": -0.7024990106001496, + "1607773860": -0.01853656768798828, + "1665739511": -1.868122011423111, + "1747178030": 1.8533687740564346, + "1881223807": 0.008260488510131836, + "1919198188": -0.0077074021100997925, + "1991414747": -1.9767832309007645, + "1997090716": 1.8607252091169357, + "2015956957": -0.039133815094828606, + "2054899126": 0.0, + "211245842": 1.1611655950546265, + "2141802816": -0.025548294186592102, + "256770874": -1.880553051829338, + "278008524": -0.09710458421614021, + "416183106": -0.006514519453048706, + "534717141": 0.0, + "551504635": -0.011456906795501709, + "565369392": -0.07044772803783417, + "565621826": 0.0, + "585904130": -0.05251938166717688, + "626486813": -0.456769184791483, + "636680137": -1.9918067259714007, + "706479600": 1.1649112105369568, + "750448296": -0.0648160744458437, + "75070641": -0.03326091170310974, + "767747424": -0.47674286365509033, + "77147264": -1.2489371672272682, + "794181626": 1.8190439492464066, + "957101489": -0.027882948517799377, + "998761162": 1.9321834295988083 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.16963527703132386, + "low": -0.24229603171138567, + "mean": -0.06732003508654716, + "n": 5, + "num_samples": 200 + }, + "mean": -0.06732003508654716, + "per_task": { + "LiftPegUpright-v1": -0.23038212209939957, + "PickCube-v1": -0.0756496861577034, + "PullCube-v1": -0.26397748804404675, + "PushCube-v1": 0.4225746931938025, + "StackCube-v1": -0.18916557232538858 + } + }, + "metric": "selected_utility_gain_over_base_at_8", + "micro": { + "confidence": 0.95, + "high": 0.11600385246245001, + "low": -0.37774778591077507, + "mean": -0.1127128925451172, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "selector_regret_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.5746948152998502, + "low": 0.20552555694829355, + "mean": 0.38140176123681857, + "n": 47, + "num_samples": 200 + }, + "mean": 0.38140176123681857, + "per_seed": { + "1025740212": 0.051884353160858154, + "1055157354": 0.0, + "1055567618": 1.8424437642097473, + "1113268940": 0.48447395861148834, + "1113371261": 0.0, + "1176943004": 1.9587705433368683, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.9200049638748169, + "1273247072": 0.39307598024606705, + "1358421325": 0.0, + "1375492786": 0.00036396586801856756, + "141984615": 0.03539413997593025, + "1475440973": 1.2959344387054443, + "1483645336": 0.0, + "1522272686": 0.056304022669792175, + "1528740417": 0.0, + "1549324805": 0.010487629100680351, + "1607773860": 1.185863733291626, + "1665739511": 1.868122011423111, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.019107162952423096, + "1991414747": 1.9767832309007645, + "1997090716": 0.0, + "2015956957": 0.4314236156642437, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.954741045832634, + "256770874": 0.07907943427562714, + "278008524": 0.0040267681470140815, + "416183106": 0.04582805931568146, + "534717141": 0.0, + "551504635": 1.287484347820282, + "565369392": 0.057756672302881874, + "565621826": 0.0, + "585904130": 0.006800346697370212, + "626486813": 0.013256545644253492, + "636680137": 0.028240702114999294, + "706479600": 0.0, + "750448296": 0.03573483042418957, + "75070641": 0.048324257135391235, + "767747424": 0.5082327332347631, + "77147264": 0.9378202110528946, + "794181626": 0.0, + "957101489": 0.38811931014060974, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.53974167769536, + "low": 0.2651281852154152, + "mean": 0.4279891262761722, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4279891262761722, + "per_task": { + "LiftPegUpright-v1": 0.5199628497163454, + "PickCube-v1": 0.39307598024606705, + "PullCube-v1": 0.13753403203822123, + "PushCube-v1": 0.597546140734966, + "StackCube-v1": 0.4918266286452611 + } + }, + "metric": "selector_regret_at_8", + "micro": { + "confidence": 0.95, + "high": 0.494531684580421, + "low": 0.2130161163808805, + "mean": 0.3381812960445962, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_selector_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.324468085106383, + "low": 0.09574468085106383, + "mean": 0.20106382978723403, + "n": 47, + "num_samples": 200 + }, + "mean": 0.20106382978723403, + "per_seed": { + "1025740212": 0.0, + "1055157354": 0.0, + "1055567618": 1.0, + "1113268940": 0.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 0.0, + "1271718007": 0.5, + "1273247072": 0.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 0.0, + "1528740417": 0.0, + "1549324805": 0.0, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 0.0, + "1919198188": 0.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.0, + "256770874": 0.0, + "278008524": 0.0, + "416183106": 0.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.0, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.0, + "636680137": 0.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 0.0, + "767747424": 0.25, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 0.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.29641025641025637, + "low": 0.08862745098039217, + "mean": 0.19330316742081446, + "n": 5, + "num_samples": 200 + }, + "mean": 0.19330316742081446, + "per_task": { + "LiftPegUpright-v1": 0.3333333333333333, + "PickCube-v1": 0.0, + "PullCube-v1": 0.058823529411764705, + "PushCube-v1": 0.3076923076923077, + "StackCube-v1": 0.26666666666666666 + } + }, + "metric": "success_selector_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.2608695652173913, + "low": 0.10144927536231885, + "mean": 0.17391304347826086, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.45390070921985815, + "low": 0.20567375886524822, + "mean": 0.3173758865248227, + "n": 47, + "num_samples": 200 + }, + "mean": 0.3173758865248227, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 0.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.0, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.0, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 0.5, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.7307692307692307, + "low": 0.2085972850678733, + "mean": 0.4609653092006033, + "n": 5, + "num_samples": 200 + }, + "mean": 0.4609653092006033, + "per_task": { + "LiftPegUpright-v1": 0.5, + "PickCube-v1": 1.0, + "PullCube-v1": 0.11764705882352941, + "PushCube-v1": 0.15384615384615385, + "StackCube-v1": 0.5333333333333333 + } + }, + "metric": "success_support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.3333333333333333, + "low": 0.15942028985507245, + "mean": 0.2608695652173913, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "success_total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.6496453900709219, + "low": 0.3801418439716312, + "mean": 0.5131205673758864, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5131205673758864, + "per_seed": { + "1025740212": 1.0, + "1055157354": 1.0, + "1055567618": 1.0, + "1113268940": 1.0, + "1113371261": 0.0, + "1176943004": 1.0, + "1189392917": 0.0, + "1231780997": 1.0, + "1271718007": 0.5, + "1273247072": 1.0, + "1358421325": 0.0, + "1375492786": 0.0, + "141984615": 0.0, + "1475440973": 1.0, + "1483645336": 0.0, + "1522272686": 1.0, + "1528740417": 0.0, + "1549324805": 0.3333333333333333, + "1607773860": 1.0, + "1665739511": 1.0, + "1747178030": 0.0, + "1881223807": 1.0, + "1919198188": 1.0, + "1991414747": 1.0, + "1997090716": 0.0, + "2015956957": 0.2, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.0, + "256770874": 1.0, + "278008524": 0.5, + "416183106": 1.0, + "534717141": 0.0, + "551504635": 1.0, + "565369392": 0.3333333333333333, + "565621826": 0.0, + "585904130": 0.0, + "626486813": 0.25, + "636680137": 1.0, + "706479600": 0.0, + "750448296": 0.0, + "75070641": 1.0, + "767747424": 0.0, + "77147264": 1.0, + "794181626": 0.0, + "957101489": 1.0, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 0.86, + "low": 0.38054298642533935, + "mean": 0.6483861236802413, + "n": 5, + "num_samples": 200 + }, + "mean": 0.6483861236802413, + "per_task": { + "LiftPegUpright-v1": 0.8333333333333334, + "PickCube-v1": 1.0, + "PullCube-v1": 0.14705882352941177, + "PushCube-v1": 0.46153846153846156, + "StackCube-v1": 0.8 + } + }, + "metric": "success_total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.5507246376811594, + "low": 0.3188405797101449, + "mean": 0.42028985507246375, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "support_gap_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 0.7874070694942542, + "low": 0.3649463011405979, + "mean": 0.5465996172535068, + "n": 47, + "num_samples": 200 + }, + "mean": 0.5465996172535068, + "per_seed": { + "1025740212": 1.2853291630744934, + "1055157354": 1.1348695755004883, + "1055567618": 0.0, + "1113268940": 1.3009825348854065, + "1113371261": 0.0, + "1176943004": 0.0, + "1189392917": 0.0, + "1231780997": 1.274539053440094, + "1271718007": 0.0, + "1273247072": 1.529144048690796, + "1358421325": 0.0, + "1375492786": 0.2422413924941793, + "141984615": 0.11378062601822118, + "1475440973": 0.0, + "1483645336": 0.0, + "1522272686": 1.7680195569992065, + "1528740417": 0.0, + "1549324805": 0.7160144550725818, + "1607773860": 0.0, + "1665739511": 0.0, + "1747178030": 0.0, + "1881223807": 1.9597125202417374, + "1919198188": 1.9514013677835464, + "1991414747": 0.0, + "1997090716": 0.0, + "2015956957": 0.07047177664935589, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 0.0, + "256770874": 1.801473617553711, + "278008524": 1.0994171425700188, + "416183106": 1.7603600472211838, + "534717141": 0.0, + "551504635": 0.0, + "565369392": 0.6395569890737534, + "565621826": 0.0, + "585904130": 0.2080739289522171, + "626486813": 0.5505410328041762, + "636680137": 1.9635660238564014, + "706479600": 0.0, + "750448296": 0.15335527434945107, + "75070641": 1.7821850180625916, + "767747424": 0.05956761725246906, + "77147264": 0.88953747600317, + "794181626": 0.0, + "957101489": 1.43604177236557, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.158454726099758, + "low": 0.4225971201656149, + "mean": 0.7360350727635508, + "n": 5, + "num_samples": 200 + }, + "mean": 0.7360350727635508, + "per_task": { + "LiftPegUpright-v1": 0.6434751252333323, + "PickCube-v1": 1.529144048690796, + "PullCube-v1": 0.31061309059355957, + "PushCube-v1": 0.3027147054672241, + "StackCube-v1": 0.8942283938328425 + } + }, + "metric": "support_gap_at_8", + "micro": { + "confidence": 0.95, + "high": 0.6270557242543504, + "low": 0.3067578205356266, + "mean": 0.48260227476577344, + "n": 69, + "num_samples": 200 + }, + "n": 69 + }, + "total_car_to_hidden_at_8": { + "macro_by_seed": { + "ci": { + "confidence": 0.95, + "high": 1.1480929385127618, + "low": 0.7059150926673007, + "mean": 0.9183913001201134, + "n": 47, + "num_samples": 200 + }, + "mean": 0.9183913001201134, + "per_seed": { + "1025740212": 1.3372135162353516, + "1055157354": 1.1348695755004883, + "1055567618": 1.8424437642097473, + "1113268940": 1.7854564934968948, + "1113371261": 0.0, + "1176943004": 1.9587705433368683, + "1189392917": 0.0, + "1231780997": 1.274539053440094, + "1271718007": 0.9200049638748169, + "1273247072": 1.922220028936863, + "1358421325": 0.0, + "1375492786": 0.24260535836219788, + "141984615": 0.14744710053006807, + "1475440973": 1.2959344387054443, + "1483645336": 0.0, + "1522272686": 1.8243235796689987, + "1528740417": 0.0, + "1549324805": 0.7265020841732621, + "1607773860": 1.185863733291626, + "1665739511": 1.868122011423111, + "1747178030": 0.0, + "1881223807": 1.9597125202417374, + "1919198188": 1.9705085307359695, + "1991414747": 1.9767832309007645, + "1997090716": 0.0, + "2015956957": 0.5018953923135996, + "2054899126": 0.0, + "211245842": 0.0, + "2141802816": 1.954741045832634, + "256770874": 1.880553051829338, + "278008524": 1.1034439107170328, + "416183106": 1.8061881065368652, + "534717141": 0.0, + "551504635": 1.287484347820282, + "565369392": 0.6973136613766352, + "565621826": 0.0, + "585904130": 0.2148742756495873, + "626486813": 0.5637975784484297, + "636680137": 1.9918067259714007, + "706479600": 0.0, + "750448296": 0.18909010477364063, + "75070641": 1.8305092751979828, + "767747424": 0.11785433255136013, + "77147264": 1.8273576870560646, + "794181626": 0.0, + "957101489": 1.8241610825061798, + "998761162": 0.0 + } + }, + "macro_by_task": { + "ci": { + "confidence": 0.95, + "high": 1.5660717815924914, + "low": 0.7469345130387, + "mean": 1.153406745697748, + "n": 5, + "num_samples": 200 + }, + "mean": 1.153406745697748, + "per_task": { + "LiftPegUpright-v1": 1.1634379749496777, + "PickCube-v1": 1.922220028936863, + "PullCube-v1": 0.3950598559219061, + "PushCube-v1": 0.9002608462021902, + "StackCube-v1": 1.3860550224781036 + } + }, + "metric": "total_car_to_hidden_at_8", + "micro": { + "confidence": 0.95, + "high": 0.9977632297837324, + "low": 0.6197721612187561, + "mean": 0.7946246277939096, + "n": 69, + "num_samples": 200 + }, + "n": 69 + } + }, + "num_rows": 69, + "report_type": "ctt_generated_measured_rollout", + "schema_version": 1, + "source_content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "source_split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "success_summary": { + "base_success_rate": 0.2753623188405797, + "base_utility_mean": 0.7180729379567007, + "hidden_chart_oracle_success_rate": 0.6666666666666666, + "hidden_chart_oracle_utility_mean": 1.3999846732054932, + "max_restore_error": 1.4901161193847656e-08, + "proposal_oracle_success_gain_over_base": 0.14492753623188406, + "proposal_oracle_success_rate": 0.42028985507246375, + "proposal_oracle_utility_mean": 0.9435413414561797, + "selected_success_gain_over_base": -0.028985507246376812, + "selected_success_rate": 0.2463768115942029, + "selected_utility_mean": 0.6053600454115835, + "success_selector_gap": 0.17391304347826086, + "success_support_gap": 0.2608695652173913 + }, + "target_content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "target_split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381" +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics_by_seed.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics_by_seed.json new file mode 100644 index 0000000000000000000000000000000000000000..be8db8a041f14f9a31d6ab2d80f5b76541f1c105 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics_by_seed.json @@ -0,0 +1,1083 @@ +{ + "1025740212": { + "base_success": 0.0, + "base_utility": 0.528373658657074, + "branch_car_at_8": 0.051884353160858154, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.12800533274597592, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7146708369255066, + "proposal_oracle_utility_gain_over_base_at_8": 0.18629717826843262, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6627864837646484, + "selected_utility_gain_over_base_at_8": 0.13441282510757446, + "selector_regret_at_8": 0.051884353160858154, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.2853291630744934, + "total_car_to_hidden_at_8": 1.3372135162353516 + }, + "1055157354": { + "base_success": 0.0, + "base_utility": 0.19648407399654388, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06066548841424423, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.8651304244995117, + "proposal_oracle_utility_gain_over_base_at_8": 0.6686463505029678, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.8651304244995117, + "selected_utility_gain_over_base_at_8": 0.6686463505029678, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.1348695755004883, + "total_car_to_hidden_at_8": 1.1348695755004883 + }, + "1055567618": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.8424437642097473, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22991638132379041, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.15755623579025269, + "selected_utility_gain_over_base_at_8": -1.8424437642097473, + "selector_regret_at_8": 1.8424437642097473, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.8424437642097473 + }, + "1113268940": { + "base_success": 0.0, + "base_utility": 0.42231252789497375, + "branch_car_at_8": 0.48447395861148834, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2534904144040469, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.6990174651145935, + "proposal_oracle_utility_gain_over_base_at_8": 0.27670493721961975, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.21454350650310516, + "selected_utility_gain_over_base_at_8": -0.2077690213918686, + "selector_regret_at_8": 0.48447395861148834, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.3009825348854065, + "total_car_to_hidden_at_8": 1.7854564934968948 + }, + "1113371261": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13380940221299786, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1176943004": { + "base_success": 0.0, + "base_utility": 0.15461373329162598, + "branch_car_at_8": 1.9587705433368683, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2664806376409974, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.845386266708374, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.041229456663131714, + "selected_utility_gain_over_base_at_8": -0.11338427662849426, + "selector_regret_at_8": 1.9587705433368683, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9587705433368683 + }, + "1189392917": { + "base_success": 0.0, + "base_utility": 0.10841864347457886, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33283923875771465, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8915813565254211, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8915813565254211, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1231780997": { + "base_success": 0.0, + "base_utility": 0.7270061373710632, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.16621555087386797, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.725460946559906, + "proposal_oracle_utility_gain_over_base_at_8": -0.0015451908111572266, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.725460946559906, + "selected_utility_gain_over_base_at_8": -0.0015451908111572266, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.274539053440094, + "total_car_to_hidden_at_8": 1.274539053440094 + }, + "1271718007": { + "base_success": 0.5, + "base_utility": 1.415381133556366, + "branch_car_at_8": 0.9200049638748169, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.20722796191327264, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.5, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.584618866443634, + "selected_success_at_8": 0.5, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 1.079995036125183, + "selected_utility_gain_over_base_at_8": -0.33538609743118286, + "selector_regret_at_8": 0.9200049638748169, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.9200049638748169 + }, + "1273247072": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "total_car_to_hidden_at_8": 1.922220028936863 + }, + "1358421325": { + "base_success": 0.0, + "base_utility": 0.8272150158882141, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.09625642405798478, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1727849841117859, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1727849841117859, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1375492786": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.00036396586801856756, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.24345096945762634, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.29573093343728263, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.0012095769634470344, + "proposal_oracle_utility_gain_over_base_at_8": -1.998790423036553, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.0008456110954284668, + "selected_utility_gain_over_base_at_8": -1.9991543889045715, + "selector_regret_at_8": 0.00036396586801856756, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2422413924941793, + "total_car_to_hidden_at_8": 0.24260535836219788 + }, + "141984615": { + "base_success": 0.0, + "base_utility": 0.09381213908394177, + "branch_car_at_8": 0.03539413997593025, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.20044784247875214, + "outcome_ptr_at_8": 0.3333333333333333, + "pairwise_causal_calibration_ece": 0.22785399017277783, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08839488192461431, + "proposal_oracle_utility_gain_over_base_at_8": -0.005417257159327467, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.05300074194868406, + "selected_utility_gain_over_base_at_8": -0.04081139713525772, + "selector_regret_at_8": 0.03539413997593025, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.11378062601822118, + "total_car_to_hidden_at_8": 0.14744710053006807 + }, + "1475440973": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.2959344387054443, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15809903475948942, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.7040655612945557, + "selected_utility_gain_over_base_at_8": -1.2959344387054443, + "selector_regret_at_8": 1.2959344387054443, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.2959344387054443 + }, + "1483645336": { + "base_success": 0.0, + "base_utility": 0.01641605794429779, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.06905173445346233, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9835839420557022, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9835839420557022, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1522272686": { + "base_success": 0.0, + "base_utility": 0.19844311475753784, + "branch_car_at_8": 0.056304022669792175, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21028164676312622, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23198044300079346, + "proposal_oracle_utility_gain_over_base_at_8": 0.033537328243255615, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.17567642033100128, + "selected_utility_gain_over_base_at_8": -0.02276669442653656, + "selector_regret_at_8": 0.056304022669792175, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7680195569992065, + "total_car_to_hidden_at_8": 1.8243235796689987 + }, + "1528740417": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.3242760158848682, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1549324805": { + "base_success": 0.6666666666666666, + "base_utility": 1.3787474483251572, + "branch_car_at_8": 0.010487629100680351, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4027505218982697, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2818476718903797, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": -0.3333333333333333, + "proposal_oracle_utility_at_8": 0.6867360668256879, + "proposal_oracle_utility_gain_over_base_at_8": -0.6920113814994693, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": -0.3333333333333333, + "selected_utility_at_8": 0.6762484377250075, + "selected_utility_gain_over_base_at_8": -0.7024990106001496, + "selector_regret_at_8": 0.010487629100680351, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.7160144550725818, + "total_car_to_hidden_at_8": 0.7265020841732621 + }, + "1607773860": { + "base_success": 0.0, + "base_utility": 0.8326728343963623, + "branch_car_at_8": 1.185863733291626, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2143457671242473, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1673271656036377, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.814136266708374, + "selected_utility_gain_over_base_at_8": -0.01853656768798828, + "selector_regret_at_8": 1.185863733291626, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.185863733291626 + }, + "1665739511": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.868122011423111, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.26263826459603723, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.13187798857688904, + "selected_utility_gain_over_base_at_8": -1.868122011423111, + "selector_regret_at_8": 1.868122011423111, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.868122011423111 + }, + "1747178030": { + "base_success": 0.0, + "base_utility": 0.14663122594356537, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.33194730961537694, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8533687740564346, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8533687740564346, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "1881223807": { + "base_success": 0.0, + "base_utility": 0.0320269912481308, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1643392210528543, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.040287479758262634, + "proposal_oracle_utility_gain_over_base_at_8": 0.008260488510131836, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.040287479758262634, + "selected_utility_gain_over_base_at_8": 0.008260488510131836, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9597125202417374, + "total_car_to_hidden_at_8": 1.9597125202417374 + }, + "1919198188": { + "base_success": 0.0, + "base_utility": 0.03719887137413025, + "branch_car_at_8": 0.019107162952423096, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21294576266807125, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.04859863221645355, + "proposal_oracle_utility_gain_over_base_at_8": 0.011399760842323303, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.029491469264030457, + "selected_utility_gain_over_base_at_8": -0.0077074021100997925, + "selector_regret_at_8": 0.019107162952423096, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9514013677835464, + "total_car_to_hidden_at_8": 1.9705085307359695 + }, + "1991414747": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 1.9767832309007645, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.613109405537996, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.023216769099235535, + "selected_utility_gain_over_base_at_8": -1.9767832309007645, + "selector_regret_at_8": 1.9767832309007645, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.9767832309007645 + }, + "1997090716": { + "base_success": 0.0, + "base_utility": 0.13927479088306427, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21318958083944717, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8607252091169357, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8607252091169357, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2015956957": { + "base_success": 0.0, + "base_utility": 0.06711978986859321, + "branch_car_at_8": 0.4314236156642437, + "hidden_chart_oracle_success_at_8": 0.2, + "hidden_chart_oracle_utility_at_8": 0.5298813670873642, + "outcome_ptr_at_8": 0.8, + "pairwise_causal_calibration_ece": 0.24731300175116924, + "proposal_oracle_success_at_8": 0.2, + "proposal_oracle_success_gain_over_base_at_8": 0.2, + "proposal_oracle_utility_at_8": 0.45940959043800833, + "proposal_oracle_utility_gain_over_base_at_8": 0.3922898005694151, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.02798597477376461, + "selected_utility_gain_over_base_at_8": -0.039133815094828606, + "selector_regret_at_8": 0.4314236156642437, + "success_selector_gap_at_8": 0.2, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.2, + "support_gap_at_8": 0.07047177664935589, + "total_car_to_hidden_at_8": 0.5018953923135996 + }, + "2054899126": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "211245842": { + "base_success": 0.0, + "base_utility": 0.8388344049453735, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.08971682945849302, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1611655950546265, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1611655950546265, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "2141802816": { + "base_success": 0.0, + "base_utility": 0.07080724835395813, + "branch_car_at_8": 1.954741045832634, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.3183279052482729, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9291927516460419, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.04525895416736603, + "selected_utility_gain_over_base_at_8": -0.025548294186592102, + "selector_regret_at_8": 1.954741045832634, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.954741045832634 + }, + "256770874": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.07907943427562714, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.5133629211325229, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.19852638244628906, + "proposal_oracle_utility_gain_over_base_at_8": -1.801473617553711, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.11944694817066193, + "selected_utility_gain_over_base_at_8": -1.880553051829338, + "selector_regret_at_8": 0.07907943427562714, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.801473617553711, + "total_car_to_hidden_at_8": 1.880553051829338 + }, + "278008524": { + "base_success": 0.0, + "base_utility": 0.10747186839580536, + "branch_car_at_8": 0.0040267681470140815, + "hidden_chart_oracle_success_at_8": 0.5, + "hidden_chart_oracle_utility_at_8": 1.113811194896698, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.40905575479886586, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.01439405232667923, + "proposal_oracle_utility_gain_over_base_at_8": -0.09307781606912613, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.010367284179665148, + "selected_utility_gain_over_base_at_8": -0.09710458421614021, + "selector_regret_at_8": 0.0040267681470140815, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.5, + "support_gap_at_8": 1.0994171425700188, + "total_car_to_hidden_at_8": 1.1034439107170328 + }, + "416183106": { + "base_success": 0.0, + "base_utility": 0.20032641291618347, + "branch_car_at_8": 0.04582805931568146, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.21826032886833752, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.23963995277881622, + "proposal_oracle_utility_gain_over_base_at_8": 0.03931353986263275, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.19381189346313477, + "selected_utility_gain_over_base_at_8": -0.006514519453048706, + "selector_regret_at_8": 0.04582805931568146, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7603600472211838, + "total_car_to_hidden_at_8": 1.8061881065368652 + }, + "534717141": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.15703220108978472, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "551504635": { + "base_success": 0.0, + "base_utility": 0.7239725589752197, + "branch_car_at_8": 1.287484347820282, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.4327797462092032, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.2760274410247803, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.712515652179718, + "selected_utility_gain_over_base_at_8": -0.011456906795501709, + "selector_regret_at_8": 1.287484347820282, + "success_selector_gap_at_8": 1.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 1.287484347820282 + }, + "565369392": { + "base_success": 0.3333333333333333, + "base_utility": 0.7997656464576721, + "branch_car_at_8": 0.057756672302881874, + "hidden_chart_oracle_success_at_8": 0.6666666666666666, + "hidden_chart_oracle_utility_at_8": 1.4266315797964733, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.2176430432872923, + "proposal_oracle_success_at_8": 0.3333333333333333, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.7870745907227198, + "proposal_oracle_utility_gain_over_base_at_8": -0.012691055734952291, + "selected_success_at_8": 0.3333333333333333, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.729317918419838, + "selected_utility_gain_over_base_at_8": -0.07044772803783417, + "selector_regret_at_8": 0.057756672302881874, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.3333333333333333, + "success_total_car_to_hidden_at_8": 0.3333333333333333, + "support_gap_at_8": 0.6395569890737534, + "total_car_to_hidden_at_8": 0.6973136613766352 + }, + "565621826": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.0, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 0.0, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 0.0, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "585904130": { + "base_success": 0.0, + "base_utility": 0.07529209802548091, + "branch_car_at_8": 0.006800346697370212, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23764699200789133, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.22254352429757418, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.029573063055674236, + "proposal_oracle_utility_gain_over_base_at_8": -0.04571903496980667, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.022772716358304024, + "selected_utility_gain_over_base_at_8": -0.05251938166717688, + "selector_regret_at_8": 0.006800346697370212, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.2080739289522171, + "total_car_to_hidden_at_8": 0.2148742756495873 + }, + "626486813": { + "base_success": 0.25, + "base_utility": 0.5054400166263804, + "branch_car_at_8": 0.013256545644253492, + "hidden_chart_oracle_success_at_8": 0.25, + "hidden_chart_oracle_utility_at_8": 0.6124684102833271, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.14946876705188256, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -0.25, + "proposal_oracle_utility_at_8": 0.06192737747915089, + "proposal_oracle_utility_gain_over_base_at_8": -0.4435126391472295, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.0486708318348974, + "selected_utility_gain_over_base_at_8": -0.456769184791483, + "selector_regret_at_8": 0.013256545644253492, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.25, + "success_total_car_to_hidden_at_8": 0.25, + "support_gap_at_8": 0.5505410328041762, + "total_car_to_hidden_at_8": 0.5637975784484297 + }, + "636680137": { + "base_success": 1.0, + "base_utility": 2.0, + "branch_car_at_8": 0.028240702114999294, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.6813618274184183, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": -1.0, + "proposal_oracle_utility_at_8": 0.036433976143598557, + "proposal_oracle_utility_gain_over_base_at_8": -1.9635660238564014, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -1.0, + "selected_utility_at_8": 0.008193274028599262, + "selected_utility_gain_over_base_at_8": -1.9918067259714007, + "selector_regret_at_8": 0.028240702114999294, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.9635660238564014, + "total_car_to_hidden_at_8": 1.9918067259714007 + }, + "706479600": { + "base_success": 0.0, + "base_utility": 0.8350887894630432, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.15321122852132601, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.1649112105369568, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.1649112105369568, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "750448296": { + "base_success": 0.0, + "base_utility": 0.11153355985879898, + "branch_car_at_8": 0.03573483042418957, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.23580759018659592, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.19031133205081463, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.08245231583714485, + "proposal_oracle_utility_gain_over_base_at_8": -0.02908124402165413, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.046717485412955284, + "selected_utility_gain_over_base_at_8": -0.0648160744458437, + "selector_regret_at_8": 0.03573483042418957, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.15335527434945107, + "total_car_to_hidden_at_8": 0.18909010477364063 + }, + "75070641": { + "base_success": 0.0, + "base_utility": 0.20275163650512695, + "branch_car_at_8": 0.048324257135391235, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2643509727736545, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.21781498193740845, + "proposal_oracle_utility_gain_over_base_at_8": 0.015063345432281494, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1694907248020172, + "selected_utility_gain_over_base_at_8": -0.03326091170310974, + "selector_regret_at_8": 0.048324257135391235, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.7821850180625916, + "total_car_to_hidden_at_8": 1.8305092751979828 + }, + "767747424": { + "base_success": 0.25, + "base_utility": 0.5475241877138615, + "branch_car_at_8": 0.5082327332347631, + "hidden_chart_oracle_success_at_8": 0.0, + "hidden_chart_oracle_utility_at_8": 0.18863565661013126, + "outcome_ptr_at_8": 0.75, + "pairwise_causal_calibration_ece": 0.20625644377911734, + "proposal_oracle_success_at_8": 0.25, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5790140572935343, + "proposal_oracle_utility_gain_over_base_at_8": 0.03148986957967281, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.25, + "selected_utility_at_8": 0.07078132405877113, + "selected_utility_gain_over_base_at_8": -0.47674286365509033, + "selector_regret_at_8": 0.5082327332347631, + "success_selector_gap_at_8": 0.25, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.05956761725246906, + "total_car_to_hidden_at_8": 0.11785433255136013 + }, + "77147264": { + "base_success": 0.5, + "base_utility": 1.4215794801712036, + "branch_car_at_8": 0.9378202110528946, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.0, + "pairwise_causal_calibration_ece": 0.13662790487889595, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 1.11046252399683, + "proposal_oracle_utility_gain_over_base_at_8": -0.3111169561743736, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": -0.5, + "selected_utility_at_8": 0.1726423129439354, + "selected_utility_gain_over_base_at_8": -1.2489371672272682, + "selector_regret_at_8": 0.9378202110528946, + "success_selector_gap_at_8": 0.5, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 0.88953747600317, + "total_car_to_hidden_at_8": 1.8273576870560646 + }, + "794181626": { + "base_success": 0.0, + "base_utility": 0.18095605075359344, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.1880089150142482, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.8190439492464066, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.8190439492464066, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + }, + "957101489": { + "base_success": 0.0, + "base_utility": 0.20372186601161957, + "branch_car_at_8": 0.38811931014060974, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.5372965735602796, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.5639582276344299, + "proposal_oracle_utility_gain_over_base_at_8": 0.36023636162281036, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.1758389174938202, + "selected_utility_gain_over_base_at_8": -0.027882948517799377, + "selector_regret_at_8": 0.38811931014060974, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.43604177236557, + "total_car_to_hidden_at_8": 1.8241610825061798 + }, + "998761162": { + "base_success": 0.0, + "base_utility": 0.06781657040119171, + "branch_car_at_8": 0.0, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.14873333521535068, + "proposal_oracle_success_at_8": 1.0, + "proposal_oracle_success_gain_over_base_at_8": 1.0, + "proposal_oracle_utility_at_8": 2.0, + "proposal_oracle_utility_gain_over_base_at_8": 1.9321834295988083, + "selected_success_at_8": 1.0, + "selected_success_gain_over_base_at_8": 1.0, + "selected_utility_at_8": 2.0, + "selected_utility_gain_over_base_at_8": 1.9321834295988083, + "selector_regret_at_8": 0.0, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 0.0, + "success_total_car_to_hidden_at_8": 0.0, + "support_gap_at_8": 0.0, + "total_car_to_hidden_at_8": 0.0 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics_by_task.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics_by_task.json new file mode 100644 index 0000000000000000000000000000000000000000..f6aa04bc9a8cd61d3b1fd701266d1d3f6ed9d25c --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/metrics_by_task.json @@ -0,0 +1,117 @@ +{ + "LiftPegUpright-v1": { + "base_success": 0.3333333333333333, + "base_utility": 1.0669441471497219, + "branch_car_at_8": 0.5199628497163454, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.5, + "pairwise_causal_calibration_ece": 0.24381101581290862, + "proposal_oracle_success_at_8": 0.5, + "proposal_oracle_success_gain_over_base_at_8": 0.16666666666666666, + "proposal_oracle_utility_at_8": 1.3565248747666676, + "proposal_oracle_utility_gain_over_base_at_8": 0.2895807276169459, + "selected_success_at_8": 0.16666666666666666, + "selected_success_gain_over_base_at_8": -0.16666666666666666, + "selected_utility_at_8": 0.8365620250503222, + "selected_utility_gain_over_base_at_8": -0.23038212209939957, + "selector_regret_at_8": 0.5199628497163454, + "success_selector_gap_at_8": 0.3333333333333333, + "success_support_gap_at_8": 0.5, + "success_total_car_to_hidden_at_8": 0.8333333333333334, + "support_gap_at_8": 0.6434751252333323, + "total_car_to_hidden_at_8": 1.1634379749496777 + }, + "PickCube-v1": { + "base_success": 0.0, + "base_utility": 0.15342965722084045, + "branch_car_at_8": 0.39307598024606705, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 1.0, + "pairwise_causal_calibration_ece": 0.2134820953504291, + "proposal_oracle_success_at_8": 0.0, + "proposal_oracle_success_gain_over_base_at_8": 0.0, + "proposal_oracle_utility_at_8": 0.4708559513092041, + "proposal_oracle_utility_gain_over_base_at_8": 0.31742629408836365, + "selected_success_at_8": 0.0, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.07777997106313705, + "selected_utility_gain_over_base_at_8": -0.0756496861577034, + "selector_regret_at_8": 0.39307598024606705, + "success_selector_gap_at_8": 0.0, + "success_support_gap_at_8": 1.0, + "success_total_car_to_hidden_at_8": 1.0, + "support_gap_at_8": 1.529144048690796, + "total_car_to_hidden_at_8": 1.922220028936863 + }, + "PullCube-v1": { + "base_success": 0.29411764705882354, + "base_utility": 0.6512394689215237, + "branch_car_at_8": 0.13753403203822123, + "hidden_chart_oracle_success_at_8": 0.3235294117647059, + "hidden_chart_oracle_utility_at_8": 0.7823218367993832, + "outcome_ptr_at_8": 0.35294117647058826, + "pairwise_causal_calibration_ece": 0.2291649560920931, + "proposal_oracle_success_at_8": 0.23529411764705882, + "proposal_oracle_success_gain_over_base_at_8": -0.058823529411764705, + "proposal_oracle_utility_at_8": 0.5247960129156982, + "proposal_oracle_utility_gain_over_base_at_8": -0.1264434560058255, + "selected_success_at_8": 0.17647058823529413, + "selected_success_gain_over_base_at_8": -0.11764705882352941, + "selected_utility_at_8": 0.38726198087747704, + "selected_utility_gain_over_base_at_8": -0.26397748804404675, + "selector_regret_at_8": 0.13753403203822123, + "success_selector_gap_at_8": 0.058823529411764705, + "success_support_gap_at_8": 0.11764705882352941, + "success_total_car_to_hidden_at_8": 0.14705882352941177, + "support_gap_at_8": 0.31061309059355957, + "total_car_to_hidden_at_8": 0.3950598559219061 + }, + "PushCube-v1": { + "base_success": 0.3076923076923077, + "base_utility": 0.6771644606040075, + "branch_car_at_8": 0.597546140734966, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.6153846153846154, + "pairwise_causal_calibration_ece": 0.24319123348754632, + "proposal_oracle_success_at_8": 0.8461538461538461, + "proposal_oracle_success_gain_over_base_at_8": 0.5384615384615384, + "proposal_oracle_utility_at_8": 1.6972852945327759, + "proposal_oracle_utility_gain_over_base_at_8": 1.0201208339287684, + "selected_success_at_8": 0.5384615384615384, + "selected_success_gain_over_base_at_8": 0.23076923076923078, + "selected_utility_at_8": 1.0997391537978098, + "selected_utility_gain_over_base_at_8": 0.4225746931938025, + "selector_regret_at_8": 0.597546140734966, + "success_selector_gap_at_8": 0.3076923076923077, + "success_support_gap_at_8": 0.15384615384615385, + "success_total_car_to_hidden_at_8": 0.46153846153846156, + "support_gap_at_8": 0.3027147054672241, + "total_car_to_hidden_at_8": 0.9002608462021902 + }, + "StackCube-v1": { + "base_success": 0.2, + "base_utility": 0.8031105498472849, + "branch_car_at_8": 0.4918266286452611, + "hidden_chart_oracle_success_at_8": 1.0, + "hidden_chart_oracle_utility_at_8": 2.0, + "outcome_ptr_at_8": 0.7333333333333333, + "pairwise_causal_calibration_ece": 0.22797947617564457, + "proposal_oracle_success_at_8": 0.4666666666666667, + "proposal_oracle_success_gain_over_base_at_8": 0.26666666666666666, + "proposal_oracle_utility_at_8": 1.1057716061671574, + "proposal_oracle_utility_gain_over_base_at_8": 0.30266105631987256, + "selected_success_at_8": 0.2, + "selected_success_gain_over_base_at_8": 0.0, + "selected_utility_at_8": 0.6139449775218964, + "selected_utility_gain_over_base_at_8": -0.18916557232538858, + "selector_regret_at_8": 0.4918266286452611, + "success_selector_gap_at_8": 0.26666666666666666, + "success_support_gap_at_8": 0.5333333333333333, + "success_total_car_to_hidden_at_8": 0.8, + "support_gap_at_8": 0.8942283938328425, + "total_car_to_hidden_at_8": 1.3860550224781036 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/report.md b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/report.md new file mode 100644 index 0000000000000000000000000000000000000000..a4e7dd1dc69c8031cc58af1ca95692d0cade08ce --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/report.md @@ -0,0 +1,13 @@ +# CTT Generated Measured Rollout + +Rows: `69` +K: `8` +Checkpoint: `runs/ctt_residual_base_context_obj_seed2/model.pt` + +| Metric | Mean | +| --- | ---: | +| OutcomePTR@K | 0.5072 | +| SelectorRegret@K | 0.3382 | +| SupportGap@K | 0.4564 | + +These are measured simulator rollouts of decoded CTT candidates, not PPTC proxies. diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/run.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/run.log new file mode 100644 index 0000000000000000000000000000000000000000..b467e3ff72579d608e7dee787e89f4d012cd270b --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/run.log @@ -0,0 +1,766 @@ +start +importing gymnasium/mani_skill +imported gymnasium/mani_skill +loaded checkpoint=runs/ctt_residual_base_context_obj_seed2/model.pt +loaded charts source=301 target=69 +generated proposals rows=69 total=552 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-484ea653bbca2de3a4a2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-4b4cd12d70c0615e0930 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-6e268d1e2915740816a1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-8864796e31bbe93d7c5d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-b04b489f4fbad93b65ca candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=LiftPegUpright-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=LiftPegUpright-v1 chart=ms-c663f7e73e7e8376a05c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=LiftPegUpright-v1 +env init sequential task=PickCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PickCube-v1 chart=ms-2458fcfffb3f96c6ca1a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PickCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0345d8ac530f88cefd61 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0819eb89e627595e7e1b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-0dee962a5901fb8b5bbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-126eb8b36769f9ba4671 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-16661b2fd2c85f57a66c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-20e1efbb745fd02abc95 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-2dc5138251afd26a040d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-387e6771a55be98f1a59 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-38f7298184a2c5005e94 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-482c5c8d208b5907a2c2 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-4da03300d92420e80ed0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-57be9ae181c077e7821f candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-6388b5d2356c556c3e55 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-638d2ba1917eff4d09e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-652ad552ba98d2736620 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-674fad0e7c042d23629e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-728b865677134d3788e3 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7ea98b8f4d2220347ee0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-7f57045c114ddad153b5 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-81f4e8858b2ba90aea2e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-8a72b73ac452bc190aeb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-a96bca1ef0b74f686018 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ac4fb33f3ab4db853995 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c31b98b3d3702c6f9c8c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c832439976a37f52d094 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-c96b415f083f60857334 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-d73fd5fce55fb58ef8e9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-da142025733b53fcf7eb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e2bad6e99e8a281207cc candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e30fd80adc089e9f697a candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-e9ed5d488afdfd6449d1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ec013396e4aff6170ee1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-ee30ebaf98622759d888 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PullCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PullCube-v1 chart=ms-f3f1d07cfd59b823da47 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PullCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-081952a63087fc8f83cd candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-121bc2e6676c1ac10911 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-22f153f0d4fa656018e6 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-27c9f70e97b8850cc3ab candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-2867deaca7e98582620b candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-365d8c6de6de16f66744 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-3c326d05fec66e1d6bea candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-4dbdc6ed61dccebbfe92 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-58950fed04d203b97ab9 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-651bc0395d78d2819158 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-9fad82fd288e4296e354 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-b8bfdd3e6c31bd98082d candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=PushCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=PushCube-v1 chart=ms-fc8084d5cd7da937fc02 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=PushCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3afd8959363e917591d4 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-3ff1aedb72f65a182b58 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-4048f8f6e047422e3c79 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-5b26c33feed6992386b1 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-63ac245579b1188935ad candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-6c8d06bace98a28a8e4e candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-72777024349c2aacd5e0 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-7fc5696cb3dc7b9bb9f7 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9a14552b7db695dee92c candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-9f5d57fe2153fd267774 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-b0fee26e1db967f90b81 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-bc2d3b425d9163a62dbb candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-d1960368ef76bbb10669 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-efdc07c985935d871adf candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +env init sequential task=StackCube-v1 batch=1 sim=physx_cpu render=cpu +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=0 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=1 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=2 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=3 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=4 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=5 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=6 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=7 shape=(1, 1, 16, 7) +execute sequential task=StackCube-v1 chart=ms-fd2387aeed9bfc9a0d08 candidate=8 shape=(1, 1, 16, 7) +batch done sequential task=StackCube-v1 +rollout complete rows=69 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_data_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_data_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..1ef49748c0932e9ccb1c83a065667e7a4032f7db --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_data_hash.txt @@ -0,0 +1 @@ +67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_index.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_index.json new file mode 100644 index 0000000000000000000000000000000000000000..21f76a4f847a39cd7631c9675f06c81edf567240 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_index.json @@ -0,0 +1,4179 @@ +{ + "audience": "train_retrieval", + "base_candidate_types": [ + "policy", + "policy_residual", + "anchor", + "base", + "expert" + ], + "candidate_type_counts": { + "expert": 2044, + "near_miss": 4088, + "no_op": 2044, + "random_negative": 20440, + "wrong_direction": 2044, + "wrong_gripper": 2044 + }, + "content_hash": "67f7b4a692f7c7e71da24378dc71e3399c5d35afc14c494869eba6087b765c42", + "dataset": "/scratch/knguy52/dovla/experiments/six_task_h16_collection", + "deployment_candidate_excludes_expert": true, + "deployment_clean": true, + "embedding_content_hash": "00859bfac7590abb24a3e7cc4c26fbb17122ba52d9f63369b8bb71a6ad1bc302", + "epsilon": 0.05, + "format": "cil_charts_npz", + "group_hashes": [ + "001fcaa1974d30d8f6edbd3e9117a10a084af107360ce1b7899dba37d093d073", + "00526cba701d9c5d7399d0436d2718fe144013e2e0277c3e00f247c3f2b026e2", + "007da68284552fa257c5929a32c040913c93b9f31569bc4a9e0e5f469972c3f5", + "0081224c84c2faa89c82c842265c41240c850ffc35d5400b57531935df3d704d", + "00bbd346a4f8658a96b2167b8ed3b4a4275cd535f5b46ff6a2ba6fe55ab5af22", + "00c41e13d6af229bd31262983b05d48413360e477204b62ba89278a110632a3c", + "00ccbf9e23e7dd8cc6ea5d91197bab1ee5d7e26f1c2e9dff0db7c48332e262d7", + "00df496b285db96e866529484da8bed3b079234ab8ed99faf38a9b105ae136d5", + "01335787d62e2c1bfa20e6a53a53239d19087e703ff07e2bb0666862536259b8", + "014ef3543741d318e6b6f9a3427e4423b1594cc26a73fbfa8266a1dadffc051b", + "015b93c295196814c0cf417d49ecf1d82d00e68458e42d2a0b3a1179d6d4c135", + "01a286ab7524d1d907099802c01da4995ebb8381d71bba0460100499c3c622f9", + "01e693c8677801418926281fd44938d17c1d275ed367253f31d1208676259a5a", + "023bdae7b5ddc517f49ec900a0e48dc698bfe1ec6f1a7cc7edfac21b46f37333", + "025c271db3f4ffb8e484ec4ac3f7244b0380bbf904037af516580e8d7ebf605a", + "0267094d9bc010468e6f684ab19365e65b31b38da78897277def7201c91bac27", + "0294a908f74ea0b1e54e4fb6d547fe5ba986ca4c6881572213732a0c867699ba", + "02fc97ab4aa793222f215b66d0ffc5c856706ec647c7482ee66344e066456af2", + "03020ce876d6f83841e56c51882dc8e6be8bf33ec9c9c836ab42ab8e4a586538", + "030b87c34937f69c95bb61bf59d9773b5a365a86fa73a60f619ccd0256d2968b", + "036bd56b347b3ea03e19a9a1e1aa865021d7b541fbdca003409a10e34912b3c7", + "0388ca1306e31d2ebd07b30a8d5366063523b4cb18d426c93351e7497cdf2721", + "03b05cc0e617e7ab7149752551af4303fa838be521fe4b22a15389558dfc0d60", + "0414958360a2c4e04be7534d9347e289e132993e773f7518bc8ee75a02ede570", + "041af35d269bba29a9d94019ad4c8e433bd9b11e3d2c12af4922829649df041e", + "042cc293ccfe917dac796cd6101f1d2f4fd0fc5c67a050c391399bb3c89b83e9", + "043d323f81c1208341028b4176ed8c4c56e69abb8c08aec31ddf093056d9db95", + "044f47ab5be1af5d16cf8dc96c3201970c551380fc2e97dcce7ae2f5265658b3", + "04529d1ba49a38cd27f3ffca329e63ab856f5c1e9cadcf7599c074ea64d26acb", + "045cb769e1cc19f043df64a32a05dc520442e9dd62fe0dcaac93c048c8a53589", + "04a1476f8f2cbb1493afb3e1849003222903e15a7cc7e519dfabb48bd1bfb57a", + "04c63f785efcf0d3591dd2051455fa4878b473bc2cbeee626ec7e549bbf46a14", + "04d6d326619cb05f115813e476605dca0485b6bcbb256765822fd1be38def668", + "04efaf4eb209e619534ffaa3bbb08fdc7a05759f77c96ede295435e899557617", + "04f4ff1016421b38f73be280be75ca4fea5c372f3a8d1cda4da185de229da025", + "04faed0a7dfd3ebc89a4bd2e8e075c9a81c861e396b860cf1172c5ff89245a11", + "05240853bb100723195b299a8e993517d7636f79d04d39f3d60ebe025863c321", + "054f169f2c20ec618c2805cc410f661edd54db72ff4e124ef2e8e695f5c41483", + "057423d91c0d82d8f62f114440e3d5800ff93a36d789dee2cb1558189230b3f5", + "058273a8b1145d6860b07cd133fbfb6439dadbc9337d32c27c9b6e353b9938c6", + "05b7a5da0c35c1c6c9dd3e767ba83f3a5d8c65cdc688f7c7583b086d2f5e578f", + "05c0e14adca2fb1afd50c9c82effa52f4a3c036d865a6ef07cf43ab6cb7cf9c1", + "05c0fa198643aa9b11a16b11f508c0f3e4876d76adb60edfd63e5b22da4f6096", + "05d812f28f8fa0fc86ae7e47fee0b87e82a8c147cd2c20a18efca3e4d1716ed0", + "05eff262f0c51434f1a2717cd888ac132d5418a211f5be302c76204c694c9c22", + "05f242687ca564fcce8802518ff330054ba4aa6fd52dc16b739af6e2cd73265c", + "060071dc4d719b0172224dadd17bbcad4074d9a025bd9753dc25bcaa82a1b731", + "0612f705ce60a6e5719459f4ed2ae70cc12c6f610553af267bba19f1079e159d", + "062f09156bfa5ecc2c90411a5c30cf7ab44cfdab8fdf6bba8dbf0d0264398669", + "0637e894caaf7866508328256b86a787acb4b59279b4bc4d77fa920669620b0d", + "06646b414506091dc9562eec9f7ef5a47fc11f0d46fefd5f17118e104334ad20", + "069838f6f7f1d064c02323989485d308d36f8650e6062785a461e2b6cff0b0b0", + "06b2ec0364fcc116b85d61aa2b8f007fc319e48756acfc228765b7aa0e79db87", + "06f3fd02d9a8cbf4fa564d32546c328fc318d993b9d2d1ef685324463c69ddbe", + "0713a51c5892c5558579ce4dffdc60b974477b43e53c3fe0ae4345b45549ec17", + "07202bb06fd690678da08362e77ce653818eba36201fadb49d49d1e56ec9c228", + "0749514f298a42cfe924e24c240e25f5ff48f47a898f8586b200cfa0701c7a37", + "0768ebbabf013c2a4b7f1aad3a2099c8c13cc1b0b6a083b446a17336f2d17d04", + "07721a0faa4a8ca8a5af0f07c002538292f7cf23086a796ac71f17d92c68ccd5", + "077f12bea73da5c959c3558c431a5137f203c4468023cd9bb80ed7aa72c5cb15", + "0781ab28ea0865deb62cdf28ff45c8db767f6acac236094d1ac9671a621c0b10", + "079f2e4745cdf1e6fe28c5fd9c44ff1154b2317c9670dce3df3e46c5956b6266", + "07d1cf8fa045404915eaa0bc582d7c1f1ea3e5417ebd7d9b076c0106cf937b49", + "07d78d133b041c1b387a66dde85c2e8d619868aadbe25df65adf9a537733fbe2", + "07d981b9235d6f62b04fe940d3b60888c7324180a1eaf5258b8b7d28dc9e09e1", + "082bc6e23d9f8848668b5e2bcf5d214b05d642fe9247181ac7efd87c1152c4dc", + "0844b2faa86b75616ad9e61099a8546dbe2ce19734eb93507a1c294eed6904a7", + "084e99784f5d3144472f2bb4c6ad302124d7f238dae7d6cb5c48e30e490e2b0c", + "085f9fcd72c44e1ec1f4ae40b1d183e6854e606726d8358ac37b9f360c0358c3", + "0884cb0cd7bfeda804f3351dfee94fedc6468594260bbe98c0e2414ec08781f2", + "088693fbb45b889e85ff958da47452272c49d1720c61f8a24e4ad652bc1fb716", + "089bd7ee69056cfb1908280fe2f2b1bac9fe708220e479097c6dd18fb259a24a", + "08bd69953da3b8045ffd02deb5f986a11f2d120614be22b53347d8514e5658fe", + "08ee326bc033bfde4df8541283f59dede6da6662c31cd3826c8938218f7a90a4", + "0905b1f6e979661d55c733e49ea636ad4829039c2f8327ff1f261409b6f9c8e9", + "090d54a2871e4dfca2aefefbc2e9663d0a7b710e7957d20c00c361dc43a1d2e1", + "09c524d9908967bff722776dacc5e2dc1841dd82e85f2eac79be2986be4727c7", + "09c52593507b34845144191b4a8edc5cd9b4808dcdc1593b396fe166b29ac84b", + "09da6fc458cc6cf1e4be0ceef803120255238b8962070c2cbffe715551717085", + "09e9d938e278c8cc5f8322c610fcea1283b7b77bafe842eb6449215d49394e02", + "0a9312ec31c472903cdb28026d86b90d551127135683773d122e17b6e0cfb857", + "0a9bc7df156ecfad97d28ee1227663785ce40cc737168364932f1a2e30cb90b1", + "0aa1a4571153dba5c1748f018cab757e1ece943ee4ea64a0f26bdae4f936d3e6", + "0aa4cb3d3b2e041fa4253c1cb23a914b8a5c644b56967d3741516fb17d3631f9", + "0ab83ad36b03e69d64d2ae4ef8c556097be124061af0360d32848dc41bdfb7ea", + "0abf91fd3685960f7b6e0812b4d8ce52b6a31224b4e739a506304cb46cd29fea", + "0ae69b83b02e7dbc4eb264be79de0da7185863d38bf366a9be1ae5d745b9a7fb", + "0af0a212eca005fb212d32eff4d5c2ce71ca2de8e9970ee416416ebe5d936f06", + "0af0b65f74f5a6edff26999f485e115fb04396168b31c5e77ef731b8b413e366", + "0b1ff010af5f1418d0f838f4f21a71c1025a797e50f0dfce6fdb578b4bdac377", + "0b41ad1882d3407ba74fd9527ddba492ebdb1356a231d41d3b97bea671cc902c", + "0b660aea784794a286ecd4ad09d7536cde2eb65a19280c8ba81d9112849a05af", + "0b9d988ac32a9d5138800792db6436da932ecaba29485cb95676d018b8c4cc66", + "0bb16a236a2dbbbc53df7f8f514f71076a1408dbe89e9b64f56ff6308149a5aa", + "0bbb7041a0e42c9d69fb059687967f4f5cf4dcec5a48a233d3a1c1b6ae117304", + "0bc9c1dc25638927adbf62edb1f44c41fc6468564c0c0bd07f151256e70edcdf", + "0be11c9bfdda8d9d43bf40a9567a2a7acc8b09343e6411ddb55a6b0097406f58", + "0be26fe12af26d5427c1b23a13e9a0ab66560bd0bab91a5023987f9a483a5abf", + "0bfafacef2b0488487659ba6e9f07942ddf7a9734f49746e14e68e8b903a5e57", + "0bfffb14c5e767e6b6a0d073a8c38b271abb71ec60be03aa3c6676de8620d1e1", + "0c0015ce0828d91018c21a25c3a988ae23e9a116b77ccfee298c2d770fcbc1e4", + "0c0136f75efec62c8234e61675eb5d7cc92a005887e1daf40961dda217effdf3", + "0c26b22b6f73a920acd2c3eb853dd13714c7d1a15cb7f01189612964543e2bc1", + "0c5ebfeb8ce3384fdf5c7466d9febb871c0a55b3dce9196adbf45beeb3c3a8dc", + "0c729263a7865648102e96eb98f764199f7e3f9be15d9852bb832d72aaed3cfb", + "0c8ad825fb1b90dad15342b1a38a73230557030fc6cb483d074998a391017af5", + "0c8edffaf99aaa4628cfc8af78d3b08d78e69e645baeb1ed83bfd23d9be43a5d", + "0caf685769ea343e0714af61762ed28dd8a3619c999cf2d503f590ec404494d5", + "0cb26bcd10a801c61924e70d10b27e1c10e58ec84a1f7fd1ea0fd4691d2d577f", + "0cb8a1e195b091af4c1ab6264bdadcf5dea204e12810c9f9674a810717bccb54", + "0cd782853dfd7e50a17e062e2466a0c2834bb6ae09ccca30173972814b444d92", + "0cdbd13e6ddc48ca28d0018a7b2a880b226489f679aee1a6ac782581c3894de8", + "0ce23e2fa8a749821680c196c1169b01e7e30af9b335e1cf59b8b62c58e04fa6", + "0d54c732a07f0fe8a6fc5458c2fec6b0e153a40bb33677bf852c815add211b7c", + "0d70dcc638707ec9f6834ffbcb4630e07abdccb780a1609d713a970f3f0ac1cb", + "0dc1128ba4c2992987f1080cfbf80add951244c5512109367ac2ff6dd8481f04", + "0dc6b7748f769c6f4fb5b5e760d69c37a5c91fa538ed1dd31a3d5ba4cbc5d607", + "0e292618af8adc8bb13df6fdc700564538050caaf00643122064b7a3376a1d0f", + "0e321b35d16bc03a5586ec91b97acf8c107f0b79c2825450f90f35f4fa3eaa0a", + "0e54f1bcdc7d87ecfc6e74d6de3e4c43d13474e1bec0b95138e7167710390990", + "0e846791b18e5014d92e8a138b09f6aa1edfa36549c1ef12a8ca4dc7e1a7e90f", + "0e95a43b5c9a6e453d36ad6f7cadaa8af2df3f11f0ca06fc4ebaa43fec26445e", + "0e9cfc6bb4e6178112967c91b2e35b0a3b38e389cc2aac0aab443dbd63f6b02f", + "0ed2c0dfa1163e363e7e1d89889e7f49e18528658208c0a068ca52e82d3159b5", + "0f1330ac1eb5b8485604be1a622a26c53c19cb305a61c61fc348020b7eeeec9b", + "0f207722dc163c7f09f47a4b114cde2e4d675c7399a3b2941f8c22954ad5df5d", + "0f298d9056aff724d852e7d30dbb93cff797cf382b8fefb1a23884389ba72ae2", + "0f4ded9465d1adb2948c4ebf574aeacac709555b4cba78e2f7ab0514f3d1dc60", + "0f6e062f0fbb5aad354b50ca9695824a01b4b1a2b6650fe534dcd517aae3a3d7", + "0f97e4c29d2ee4474f3e91f9728b59c2ff6ecc3ba0fe4709bea263555e4eecdd", + "0fc1fff21404a7c4c3bf43d0a946b349cddd81fd91394830c215687fd29635c4", + "0fc42e7c404cbbb29ebc6b271746a16e9226ff4270cd13ef194d66d9385f98e4", + "10bafd54576f39b903e32333fa9de033d34630504b3ccfe756933c7cd139fb22", + "10bc4cc7a43e48b9fd9fd2b4dae3488801ea0e043959d8838a5b6f2a485f0756", + "10cb9fea43a85b51aa089d8184d5e7ead3b732a471283229eb053f86a7b89c64", + "10d3721833fbb982ec10f2e41634a06e94f3e11be89b76c449dda794bf4afb03", + "11012a866289284e636f856a871945d4089e1218b25714ddb7e14e3cab1c2d05", + "113f5145a36a62fa140f4eda8be653dc498c33132e8440ba8bf91149e5bc1abb", + "1148a49a8e54f0192c8e4ffb72c997dd10918c28363d1dff62fb9176225d3e5e", + "115434f9bc4c1998ab24ed4071f8513198a71ea8711b65bd911d2283855eeef3", + "1169947cec8d4834b14ef01a1b21b27715efabed8c282612bad37c32b70a4cec", + "11757f25e1971b1949f99f53e1aec63fe120316d0d794ff04c5e3bf3e1e4469e", + "1177abdc80e9dd30dd06522f2a39b4b22ae70c1ca17a29414e0cd1c9015f2340", + "11bb2f5d4191b2c5785176893b48e085f0bb8944d6d257b64647ef8831eaf362", + "11cba72504d167af81d43fb4f4aa447fc4dc1cb8a873cd123fc80c00a4bf605a", + "12044591849b00fdcaee3693c012680c8dbe3ffe120fe6e771ec86954622880e", + "1227a4aa8cd9896b3e02709120920bc948bef60870f206080881533008607752", + "124103b3e01c126d1d7c42ab8999b7069a649f26afe76483bbe75e0516fc9092", + "1272ddae5bbaa8cee4a5917fa083c82b10d9280d0a6dc8f85968f6df3b822eb4", + "129467e910b6d1d31fdf7405d1aee0adacf2b45bcec803f8e1ae3926c3ecc432", + "12a1492fbc2c5986df9c059a33a7a0b9fc5445555bdf672b5fae0a0f34da8a86", + "12bfa8c65887fb9fb257aab37ef0aca1a2f50dda0836afe9a3845fa66295a1d7", + "12cfb3f21f142c20b8b75fe51a500470a46848c69674b4592ceaf5aad9d976e6", + "12d2cda4fddfc7694bbf706182fd2c9ab83964cef1cc1c7b2e3abe84dc6d910d", + "12e49f621d373e186d124b525688c4d1219e8a2b7dbed956835d6e64aefeb8e9", + "12e61e6a086c54726212dd8ae40498e7bf40a4e6fd9ab5b677c950d02eb7cd52", + "12ee15c35b3f1356c065b1161d2a63d7bf8bf083bf7485034dbdc0b40f23c525", + "1311e81322d103c2c43a03f83aca27a945245bf2a02de6eabc767baa61aa2442", + "1313d324d3e798a357662678e05bcb4c9a763b4c397bcf7816611f318b2ea4ae", + "135b237070f72bb9bc691fa076a6343c512900564e119ad27281243d2af0e42a", + "139e442c52e6ca3ad180fe14f1acb72016fc4c630405d7dcd2b8f848edd43a9e", + "13c92688765abb9394668fe90e9843b50fe58a60036450bfc308769592840eac", + "13c9558f5a69f6784d34d046906d2759d6c77af2104091f964afaf7bce5b13ba", + "13ccedcc5114c00535afd5d9e128b3f7011a2926486054ea868bd4a7b6e4b46f", + "13d043c134ac85a633c31260192367eabc7cf1767299124779b9f01a21ba59d5", + "13ee615d8033d8a0e64e589fad80b9efa6d9939c3a95a006300263b5fc7bf19a", + "14b218a5081e90244f5425b45e7d278a74ce673dd2e49343e0a82f4874fecd56", + "14c0c9817f4f22d5f2251efcc78b8ffd67926fba73fb45e17293b59cdb3dd535", + "1501199487ec32e664d5f69df0b7da5186e71defb0c64229e946c7c5f0a0cb2f", + "1507748f87930160bae3bacbc4b87aeafb13ff0584c0b3457b0b15e33af846a5", + "150ecef476986ef9da9b97b6f1ffa83c74fc8ea3d8d57213831b3566d328b61a", + "15128cce41835053eda7e79d8a527734f51a5f94f0e71bc4f7c30c82bbd7381d", + "1590a2e329844ef537803416fa313ec171f54da63051be0a66528df467f45636", + "15c10dd97eff18ff9c11efeaac85032c3db028f8a024a25b5732401570b5de2b", + "15ce55e979b5b57f83c2c79f86f4ac3685778340aeb72db247ade562c3dc9fa4", + "15d91dbb5bb6cbcd2971a2c9005a7d1591a02de0780530cdd09301babf1d4295", + "15db89c2bd9b6360edcfd4126de0d04ab42d10be7e7c9eaba23f1184493696b2", + "15eafc044f81072b73a7e4978efb8a5053684d910f074bb9de0900cee1bde5ec", + "164faca0fe5fbd3466a4751b72dd20c69f8349f0f6b35367b75dcd27d79fde6e", + "165aed8e4463db7858a18e57d902eb84591607731c33d0799fe2e1dd9943a16a", + "166ad8c3cb7f8dd902042e4720f289b39d64cc34e240f516c37a552b98ddc204", + "1680fbae27ac70b8a104e58a59fa2ba47adba09377dcac5e76e68153151a1a24", + "16d7a66ecb74238e0d120dac82df1f13ff15fcd44325e4de897be2612471637a", + "16daf6296ef9184c97ba0e67164997e16876f7a16f3feb490c704ec7bb942f47", + "17242b256da38439da805284116ccd22b6dc0b4f1162e966c58d2699b88ed142", + "1728fffcde610b0ea864ffe387cb258be47affaa5fa2444d7271475cced07a00", + "1730561bf6306e61c4609c774a86ca11b4e49a8d1b01c3425f94ee9ebf718042", + "17794ad07b32b794fba99b4aa198cab80c73a617c80388cb3938f1bc448c33b4", + "179a8d46e8da7103776cc9d41954954f9a30cd4455377cc6af1bf76a0704580c", + "17c358ed8931a3e41bc39832f8024ab4cf2ecebd3070060890e8381a61c7b344", + "17c67143df844a780a50ec041cfb21740ed75142c30dd9a778cf56d46c5e2f5e", + "17e01942d367267f255ecec7a0d0756a47a18c02e6a868ccf4709c4bb7e5d75a", + "17f64daf3c523045868995b9b5893aa69c6406419d51d0d2d5d6dccc95768fb1", + "1828cbf1e501bd910192ca5f0118ad162ca6e99b3d11a588966650e20e5194f0", + "182b16a6d5297ed49a61456d81e924731b627fa701a33bebf113b5d1dbac34a0", + "182f7365b61944dec5bb0919a2d802ef89258842fe129664444ed82a58cfcca8", + "18657e4bb7bcc144077e48065c56afbfaffd0001a13ffdeaf8f6bb2326d8c483", + "18d4f30d97e86421c7a2f9bd716a3944dc4745651cfa54bbbca295c0ed56a933", + "19b08945d0d2787fc572f103b8f4bec38912b933f1489b4c73c41ea3b26e141d", + "19de7f92ee293142b40d8214a494dfd201964ef788d8663ece4c2ab5d3c8612c", + "19e0efc3b92b86a743d8a25d8228e7a35b8767c51244a2940bf325a900680e95", + "19fd5cb3bbb4d83eb86d436bf9786e970c339b1d854e91ea35e9a144e4155203", + "1a12204deff18038d5b27ab69ab157c6feaedb6adfe5560c46e31174eeff0f13", + "1a169940b1033a1daefd265afd89c6491a7a05d9c479667ea58efe6168771027", + "1a644d661b36094b489ea6e2ed67bc9fed3a3f5adbc01583591fef5eb9704bb1", + "1b261a0ce7ec407cba1565a8e4c24f3219233746509f716cab8c4747ee3537c3", + "1b2f9895a0fffaf04836d9581bb8bb6c2529fedf2283da978f875c2860d61f2b", + "1b42c5fb7e97213a533064bfe6bdad601e007166165a8f29f63e7e7999362c42", + "1b47810cd051a297cd61db10f6fb32c434e35242afbb75ebb728a1a9fca6c7de", + "1b644336b4b6654215dbcd04f4fff2f80f003d27531a1a7d5ae344a96223ec0c", + "1b779aa158ed431977a27764618e2a4e373f3ce3c5e20f75c8938781881d0e68", + "1b7dcea7adab5f8b915a848e6415ef988dcb9c01cf49cf61c573955f0845a4a4", + "1b8a6a6fa1b179d5d5437229af8793f416cd6ed3492b1fa08b9883757ba91bde", + "1b945f3f2ba81f701ae6363cb35f8805bef6fd083cb7f96fc28b60bd59b2a42e", + "1bf373e4138ede234792ca2a31f7cdf7d8000b81213d183c592c57810e5f860a", + "1c7c2472654cefecab0ce73cfd71bdce6ba3723a2d1b0e60db8c98aade0214c0", + "1c8a863bb173fb7383178f4791d07dab6d7e61deafe0533713f1dbe165714347", + "1ca29fa47c93e63575ec16caceca3cccb1433ddf46316e5a0e516a370aba3c98", + "1cb925d6fff01c6d651d84259d1a8ac66ef3e62bbe6edbb2dfbf8a32ba6e254b", + "1ce64787a72f626f6a88392499a9255a86c5383a749e004594bb8c1bd28ab033", + "1d31a051b2ec1fc2654e62a20c125ab46a8257c375a84eed7028b7e4e52d89c0", + "1d520b036adf88c23e2c7b29f0332b8b2e0145a89df01b7704c4dad4a4b7a26e", + "1d5844a8d16fd769c49e5ec5a2dbda1f912441f777969d005b10b7a54ec6f222", + "1d61e39cdd0ec95fd2b45881a6ae2dcabf25bc7b199d93850c9cea4689561fe2", + "1d71ce873b57f889c07cbca6e061d4c9a7e9b09f7a5080257809e5ccce1ef3f1", + "1d7833e75164b04c32bacecf88e1adbce107ef962491f630fa7e7995e7dfd250", + "1d7cc58ad59e6b600b4bd9ad4df955100e8d941433168c6cc049e26e843bde9f", + "1dbf59ec48199ba97c18205303d128b76b9c663595181fd12274398a2ca084e3", + "1dc1fd9b8eaa6883f2faca7699c9f508d1f6a8a77686902ac177131cf2899099", + "1e069441905d0a241eb511339e9dbb1fa46aeddbb8297488c2f610ee338cce43", + "1e43c66eca65528152bfe4159ecdb59354ee36c34b54eb62e985b212b25acd13", + "1e50848ce7dc42b4b42718f33d0c415ab9a46775c69ca5a4b82276908bb943eb", + "1e76599c93125e60681aa1bd6ae72f50d0a47a9c49cde9c6756909e98464d2ce", + "1e79355b42972c791c37b68852c99930c49365068ee1a5b7da7692d2aad48a32", + "1ecd0b4f77b2aa35440911addeb9c567c92d0871fc1fa9100e21b2be14139d18", + "1ecd4ac9f9621863890d5f786017185d6fef36d9a90002909562bd6a95eca2aa", + "1f4c8d73bf3a9ac57f9e2b43d427baf173b127d556762c817091febec2e02b83", + "1fa770f0d73df883caa5caf32e572447960a6f4a39cb919b7e5ef0ba5d8d4264", + "1fa94044a3bd20aff19e7c24bf541dfb9f85059bda026eee808d37fcf149bf33", + "1fd4988ce77729f824275c64a1736a759fd53bf3a9802c5fb4320f94889edac9", + "1fff8991748bca60d71af9cc9b2b3827ab3180ae705ebd62243c072721f8f094", + "203223582c0e7fca69776a87bd032ce7acdc2a2294698ff3a98d3b434f009347", + "20c786ab915d0509301693df90eae1ebf07cd0291a9238b3c107325d77b631f6", + "2106a3207e94eb944903c4fba0c66e02ab7f70389b6fd2655e6a934365c53a28", + "211894e79d6b7bc3345ad4e7706cd6c2e18970084a6b2e493b65bc8d59a5c727", + "2158667fab766e543795e7d22e30a62e6b02552493d9ae5d40764e724deb48ac", + "215af1c5f11ad99c5e43328eb8f930dd88cef637a51840abef90e51f91cbc371", + "215f4b47964817c247e28ca440642fd20f53f922013de4b009d90cde4c9d82cd", + "218195c1668cc7deb5f978b763d886317fec3a8ba0cc600b515d689330f959bc", + "21852b6057ee5fa7766c8f5498de18cb6a9b2ff18129ef78c35ccb43cb3a41d5", + "2193db36c24d0b68f9a562323f3455e60bac32e17e92c25108afeba03c0b8d83", + "21ae1c762c3fff7bc2ff0b1334fa52ec4290d14699b4ad762802e086427e6ea9", + "21bb9b4fcfff8d963d835e84fd8b0a3f0e3bb00da0c31b3ab99f7eda3b9d1879", + "21df15768401d6a240f92e69c17482210d8decca91922111b82544359047545e", + "2208bc75cce9607aa11acf9f64f8a37a4d2f1f8f9d3dcd59328f72e9f2f573c0", + "22243656978707a34f369f92c925fbe6e807c42c9200857afa1712beabec982c", + "223348e6289ba2185e679f0cda88bb6a8d7a8f5f81325fb7199f27c88ddd5192", + "2244cf8a582b801857e5074afda78acceda9fa1730380526087424fef042006a", + "2266cacc0b7ff3f0808e75c671cbfcaae5ac8f347d0c0c809779a3d3dbfa5fe0", + "226c86fbe1e36d2904f345f76cfaba39a6d71be0bd735335cb0c503e3b5ca3e5", + "227faa5ea2706a8fb5dc63022bd1338300485f9df8f7fdbf1733b170fc67397a", + "228107ee33b2c2e9e1e2aa7156e4cad7d4313d56cbcc7245c6b6f0b5ab6ee0df", + "229f12e885bb5e4ac5d830c3548c117169f2ddf5f11b730ccd852fb076c32ad6", + "22a71995d6c30b1889e0ca873a9de38aa25197eaba39464831d78c8132c2aebd", + "22c123126020665ad6ff2e62a8ac0395674089bcfd7a2a5e95879a0f3f42ba66", + "22f8bee7eb3aa2f15cc3a50fde768180b27cff6da04fd3ac602d0588349fccb9", + "2354b73626bcb881817c40eb612de6c984ceea635873d56767fd7d27cf72f337", + "2366155c840f1847f61784e44b578172919a4fb17e789c5e9993adff27f20ac1", + "2393fc9fe882416a98301134c8f1bc40959cbb0df4a788c18d6ba97227dab57f", + "239edc4db69e5c6c2ec386800ed2a16c1d14f40c5296dc5b474fe02f303e2932", + "23a19cdcdb11c480c19b5e826ffa9ae9a41f75356503c4c189b952e7e7229c10", + "23e7778305fd2fcf3d8f178ff8b0542c668ef7e7e7ef8c1a1719742ed4205899", + "23f90b26c36b6a00f084465c27149865452885e7b6741d29261be0963c5addc7", + "2405dad17d1d80f91da8f5654687782f23179182fa8ea95d0e7942f024d88ff2", + "240e6c1a1da07b1182d1f9c34e2585ca561d75e0c7d9c467e9f668638e3d4769", + "243ba124b326bfb985fc9db69b613aa893389059ec2b6d5083705fbf4996f0aa", + "244a86be38dfd9d932f23b9d27a9416b40b633a754cc528b81f7987672c3ccb0", + "2470bfaedd598e75adcd25e04758049ee6770a8f94a68950d5f90509fadc948f", + "2483f6c4d3c44d993895b6443f371d662e7e0d212307b47f3ab729db8d59a20a", + "248838a33696f03df582c2fdf2f6079a214d6ff05333caa5f95f138dc9af2b42", + "24ae887a86eb0636d75da5749f2d58725e00b258a152666689851b7ada661760", + "24aff802e6017c90e6244f7ded77aed1fcbb545953cb200d04972648eced96ea", + "24bde8b59e05b32bb2b477d9106f41f2782c9842a470e70bf08bb527d24132e3", + "25029d6aa49a43a7e8643cdbe81f4689c354b4913f90703da4256b6d93cbe73d", + "2584bdcca36c2871dd912988d49fe1f921aa8ad95ef0c60409c774bec286b379", + "259297aebc704d7d5569506071b5d478f9e3b879717b63b09ab00e4608e514fa", + "25938b7d578e370d915f65c0643c37264dc75c7c2c5a4e86211fc222ee78798f", + "25aaa260d7f61482d0dece1a6dc0ebddda72f625bd38bba89694a83b834d7a71", + "25d303833c68d194606fe985b8b39f09cdc60f97efec4e116be8787bb23f98fb", + "2605fcc76d52f90b98dfa94952e2a390477323091d8d0651c1227a952c2c191f", + "26273a075d3f2fa5551130d04d65fa7cdbec6f2c4770ba144eced4ae719cbffb", + "267564c04ba8e005d6b23639c895d35cb0591e0291a9e263cf5cbde05cfef1bc", + "267b0a39e499f0278a58eca2a6205c39a1503c08694d52754c5d8a487ce64c1d", + "26e4de7e4ae0c7104c8c6aedae934d81d97dfc89169ebf415a3b0219eb8fa9cf", + "26e71a515501d52e4c0fae7edfadaab7638fd7152ac373d88be078abeb2d001e", + "26fa8b7a8f9dbb96c9ea366cbbece69d993514b5ad6be8d8522c7f65a4fc2b4c", + "26fe2653ac2789f9ff3ce5dcf164b2c7f228b648e61805e3855ae9c8bf59526f", + "27427c19368afa942253ff0a473755aacfe1f98982af2f540963275b55a49d1d", + "2771ffdf18dfdfd65d02bf1b8bb36aa56570b790a173759cba7442958b7c8e59", + "27794ef63ff9fb7240d2bda2254462fb9094bdd72226a306977f6b7d82821b01", + "27c5d39a379b325588db6be3aae94fb88d53d4cb5159be340b12466159c224ad", + "27cebfb6abc8671e279fff311fab47dda886f886c26118a8a17132cc55ffd2d0", + "280c1a759d77de7a2eb914d964e2bbd88d32640e4fc9885ea5ad6e97b7926ecb", + "283124debec6334f87ecad97e22eb29412f665347953741d545a1362d50cf700", + "284292effe9290e071b49536e7676e25b1f2b51403501d067e35c4cfd930cb7d", + "285dc9617d8a64ad63c678ff0ff60dfd6c2c395a2084e2dd1b423511b99f452f", + "289e975996c5a4d8054e12f8cd26151ffa576b2fdcbc04ce39d0b8e3853ae395", + "28ce9db3376e8b1aadaf10eacf458f510bc957fcdcfedfd029bad2cb1b07bc73", + "28dca848a9bb9643b1b2d9a4a8f2f7a044c5a1994acb722baf9f46915260dcbd", + "28dd980e1372e6396aed2934d45aeeef28430910a7ed953e7c0152bf548e9c91", + "2921ea55f352623ff28dd31ac8b6126a8df490e5433d55abfbc4c3bc45d04047", + "2924d65dfedd8cff2e2d9db5f0f5f65c545597e7aef61c5eee9215374b1c97cd", + "292f2b9e562927a46924f407c14ef193343db63063425a32603740d594b02d7e", + "293f27e5ab477371a20a6fe6c65b647c259a6a98c3b813fd186b7a356bb63c54", + "29462899b56ca8dc69f31a04da863dfb0aa826f16e099bfc2d5978c938316a6b", + "2969c04005b15ee1f9eda734cb9eeaa861702af06deef784dcb9b8fd06f97282", + "2974f03b745944abd7ccd676a1f3750e8f16ae68c9c074bdc65e10b5623990f7", + "29a08c002de030c471f68e1e749966abc348ebb2ceb422612ad8c9ff3fdff258", + "29b3e33c27623bb4d5e4c858238a1d9fd530fd43dd87173f95456244872a5ec2", + "29b6050cc305b3bb5e1a40aba11155d89c2967f36eef823d4bf93474dc862356", + "29c8e5d1c664155ba776984de03294cdd6d46e2a2481599278cb12b9c042b93f", + "2a08bcf84f783530d696add19ea25a3fcc1226584bd2dc1ea0bd38a805fed8be", + "2a550c026e9ff65d75443586c017ace88635598a8faf177cb70fdec7b4761cb1", + "2a621087236e8404b3c0e45a264692011e1e30e8be64b0555a45885758aab52f", + "2a74d3e3f8eaea1675636298b1992c4b487c22866d70d483ea9d44873dc8f085", + "2a7965e1a56d537f2a2fc4bc7c9a7c4ef69a5cad08ad8411b497598b73fd9715", + "2a84af45206416f82ec91b76b8453b31a3ad892082d4ebaf0ba02647725db5d3", + "2ab5a2e4a8ad61ab47ed5ff6fa9b024652368e9fc35e4dfce3a004ba7169f077", + "2b2568383c501e9b66e98687135285a10376864a191834e8192a47db47ada0d0", + "2b2a0e3b2e9a54e95583639e8663921d0c0dc95ad6e1eab5776c4c85f3d3c519", + "2b3ebd6abe46a3e2d257c221c2521e4f2fb92b20c830f3370929106fdf462276", + "2b4a13516d94a909b64523740d2a7b42d960130275ed885b8476e5fe1a053ea8", + "2b519f4c71b588b9255394601a55032d4137587d9b4bc84ad87311564d4573e6", + "2b5fe3cbaa0557413f47e9904a66c0bb44c8e85dfb076eb68dcdc7195c587738", + "2b694c3919c53658c7115697287ec18707f91c178535e1b93a839ea7236674f6", + "2b6fc7dd39006b19ab1ad37d025d46563503e4d3b475dfc3944e4f30cc0edb9c", + "2b72a5c938c4ead2c17c0a0fa02caf08ffba9cd0c24c9f3dcc5af30e259af482", + "2b72e11390e9cb9502e0c03612bf3462bf0762abbc3c5ae2adfc7cee8e60ce57", + "2b738c4c7c9fb5a740202f324f24154482ea4a7b6f828354fa6daf638cd02e60", + "2b7a77bebff0ce005faa414e6e225e537967e9aca9e489090bd0c7c109c0c497", + "2bd3904c8bb216a19bed7b0a6f91e6fd12654312bad26d9b7e117d21a837f039", + "2bfdfb35553a281bbb7b5dbffa56a58f3d0c6c59d30d0597d4504feba177384e", + "2c69a32e259e132f0c383c6cc4674ab062850bbe9240405338f47b5392091a66", + "2cbb005c3d38bcd89685e0838a6145254e733833d7df151efeb9fce97f445636", + "2cbf2e5dbd2bff53014a66a4cbd93f69f92f886c5b4ee02353e4ba9c94e3826f", + "2cf03c960330a31b32f11d7bb56831aaf8e883f1e62b05cb3f4453fd80e40fbf", + "2d1adbbeeaa8811ef41d9bbd778507b51163ef3021fa808f767943b465b8ce16", + "2d2323f574328014c75e32fea3155f729fde3861ecf369ab6f337d310c90c1b1", + "2d2d9b13157d8f72554f0f880795745f9b6ff6461dd073d584e756caddc4cdf2", + "2d3a44eee75068d618b9bdd36b116e8e40ccbad1c16156b8bd0b06369ae81ee9", + "2da33335bb58839ce28b97fdecbf093fa0213f9d9d79801e050957efd4d4b0ce", + "2db74393909f5acfc2d5e00ffa646cc924b6ea3dabfabba9329b260a560dc354", + "2dc05339260685d20603763b5ca9b7ce8dfbe566b95e3c0979f5898dc39922b4", + "2dc306af4f727cee39b8ec93cdeede9dd9b08460753a2365f828c08333c8639b", + "2dd3adb43e3b61853b8a30448e12a24e24618eb60be5ac6a6e6310cd608eec63", + "2dddc0fa022e4dcd9424b09b16f89f0f61255a55b0748ab2a22890e87878a0bd", + "2defa16f507f490fb59bb70e071d85414ef07276dfa21b5317f8d06c4e3b1a73", + "2df21916ed241f489d69e97d370e4d6d6941211f36a925a3b86d7ed1c0fc1cec", + "2e053f8b5edaeb383ad6e73c2388a18a8f9a6d6869b237fac4325d5a2245a564", + "2e07bf8f9debd974bcd7d7f5721df3d08df6e4a9a3c891ed518cf783df0c9e3a", + "2e2ddfb1b4872831c5c5ee20abea9046923a7fddcc7462c975f79d71af030ad5", + "2e3166ed803467db2f1e7955b14d163daa016118a0240832622a3a89e31cc290", + "2e34ef045128d6d9abf7e7bbb3b7d655a355c932450e8c216830216a55f45bf6", + "2e4abffd42d065405d5c50a7e89b60b4635129858fd422bd8a5bc56debfa3d1f", + "2e7dc207e0ec7256ce878bee56d6eb7e89851dfb15739e987b5154450714819d", + "2e8b45b76496d343b60a3084f4c983916d31c5b7eed40d45909452d83f749012", + "2e992008ae3b92ac0e6c56a979e6e97376f6ab01ba1657b26eddc2f1b774eb4a", + "2e9e3a0dc28d9cc455e426bedca616fdfab0fcb4b07db71afc3a3cc55de065b7", + "2eb409f84a30521f4d270324a38a35757203f0aa3cbe84a002bb76d075c4ee95", + "2ecc79683b3b1ee11cf498155da25ab786ee469d99f7bc0348d55409a9e59f67", + "2f04d8c5bb5359ce1f9de574cc90c3d40d4e3623368871c574c3dc5419ce88d2", + "2f0a059d5a5edfb859aec1f0e51a86a767e10f8debc5af8b1d705bac6c820659", + "2f20d184ed355324c87d927888e73b46809427944c6c76926b221a1ff10fc4aa", + "2f74cccff6a3b3582d3d5c8b80bb2a826f8e8338d1baea38010dcd660f90b724", + "2fd6faf46ffe17059af06ec0c9ba806ced076ca905ac45a24879deed1c45afd2", + "301f4176d152f5ca05d3a3b212924e746ad4e088b3b8f3cb9b468e8c75c4eadf", + "304b4b93752f54c39c79148e2f4fe81ea04595e2ca127ee9162dba7a49c5be9b", + "306722a747e9b40790bb9dfc3277629b283df3204757190dfa3c9a77bb82b6c3", + "3067e3bef680d92ca05a1b10a1efaf2313dc1a700c292d07ac763b0717807e72", + "306ae438448f671a8b163290a896bea20caba868a7989a4115c607936c5a23db", + "3085ad2c20aa1751700c2a52e56a31582262e6d33490a2d863b49ad42eedf490", + "308d8a8b4f266866938940a106851d72fe3e6c09d9a246b1d339a7eaed04a3ad", + "308fd860e7b20a5003f598b1560f3d3d9ca6c1ed6ca172b94d89d8c0a2a81f2f", + "309094bdd0171c5d9013e448ee826e8683edc9caa3b05cdb830474e451b2967b", + "309ea352f1fa69f0db7582c33dae60a6e94af9b27e5b520c5d904f944fa68126", + "30a504f78e25c60f73e949b01851fa600f1748f14eb937da25d60e34f8914834", + "30ccbbd11c3360af33dc437ae8648c6f844ac6c6ad3d9a4bc066f5be76a23793", + "30e05421984dc87342b7a0c5e0f5ada02174ed62e5ddd94515fdcb0562a23a7c", + "30f332c1a28575705598386d0a16c103751abdfede5f2248d79bf7fc1b502292", + "30fe071a03f9190a7b9bf43e6848989931924b3ed57feac707a887d398225ace", + "31321c89060394789fa478b8e5e55158d305a2ecdc1d3e1736ad1fc2da960561", + "3155118baf245eb3638c6b00a0dac1335b2908925cc9ea929080acc31b1d4692", + "3156405fdd7823ec3d73009465ded9eacca77ce01ad522b2686e3f0a53afd15d", + "31904cf5060e463ae0df2858d4ffe8c5e6f88f1b4fc97583a9942f4e39461bc7", + "31c34949f3dd67ded3e6234af496cdc55c1601ffb2e2f9566e0901cbc42104a4", + "31f19b0f89f217f06f53495c011828494e379517edb22220d1e08076081d2fd8", + "31fa2d2738687e84aeb791d1da281c35e009d609352b79e938b035c718a47530", + "32188bec2e0424b77ea0f3633c8be658cc35a739de5842dced47bb4cf29b2e85", + "3226aef36f5280e1a5f7aa6921eed13bc6f959316ac2e0b64006c092639f7580", + "3282223e993dd22d7998813c953ad2266e639d3d9f861168eb41a0e6accb55c8", + "32bd77fac7231b667834724f6fc99fb379885d7c0b217fbd5f741a8e40904b96", + "32d78508833e9de23b051b65fac7099f8a8582b13576e65ddd950cdba06bb3db", + "32eacb2a02b9c5c57ac5384a2c6670c7c1f2164a9895398fcc42d78f3ef0cacf", + "331915366ec39333ff7c232f1838bf30be30e90aebc2e357e9d36d7024e1131b", + "33452c95909e9c0cdfa89c3cc04a7b27102859779a85dd966559b993eb94a7b1", + "3351793a8c5db83f2f1877a1cc8bb57f7d76f3e614c55dbd58e8489491e24af5", + "337ea7d073d3cd9be0463afe0eb37c755c15240f54d10ed971cdfd4f09618d8b", + "337f9f4695b3e8a6d6ef006ebf5b1cde828e91ea4237ecc5992d6291292ee7cf", + "33ac16e402f3f2824708a8187c19404f952acb49f1ee8b4bbe5312ad585fb131", + "33ad56c3d64bd864c3a927b12fb95ac99c6dee0e6abc244fbf4b61149b16be74", + "33b0a6867562a918e8702903b476fa7100a1b356b6613d2e9c25035e31d0791c", + "33bd5cc6473cc48632de8dd7f0ae4c6d99d772cd07e29e5bf3b37f4f9e142d66", + "33c5f3455ea8abbe73312f89915385361981af36a329a1a357c3d7274c2a1d2f", + "33c7e4b1a1be597198141e9f4506cb9c9e0955b01e6fa2103b428cd8d187f5b4", + "33e3575cb0a63713d10b724814597d6160c2549a3ac1947125c10dfc02d66286", + "33f483a510152f19c6c4220525d2227535ac5ceb9fca7483f95ec74122dc0b73", + "33fc9890af9071f76d49957ca4596b2255dac4564c3238d02f369030ee0acde1", + "34169e0b266acc6516cf0abecb99fbf15d96010e9937ccd43726762a9716ac8b", + "3421114015b80b306d6ff05cda48ef6742e935d3e9f1fbd4a79c6eb378665f10", + "3432acb18a5419f76a291dc59126ca2a71d0ded8c0c5e6da08d58ac040d5a264", + "347682ce1cb3ed264f55a6f66f7908b2a5b4aced184463812ad49a7a785bdde7", + "34b271c48e0c0b2b2b29ac13d525b4a4fba844419fb6b88cd0c1b78cfca8c3a4", + "34ccab9180bf8852ae598aae302da522f7e2366d54fde419f9e990b4abcf1722", + "34ccb47ee7a64b14318d84d641a7046684f71ec4d01aadc54c3203a1085ac817", + "34f3ead3029e94a2a25c940365600dc03afde42ba6da925938431506431d3585", + "3543fb202207aa6628c47078d7f21f7590defca0b9eb7af4ba12ed5aa445c4e5", + "355cff0ae44e80d74b235b926e1e414860320ad4a101ec70fa99f9cc2481b626", + "35905154d89fd0df546a823190c2754e6d51f79cd30024c9c1acb9f2c7eda1a4", + "35a7d85bb77a51ffe29fa15e956a970b54810c1f618adbd8547a4efba9df46e5", + "35da11f528bc82d5f9ebf9f5bb9bc25c766cff532216af0646b1d8f15afd2baf", + "3619b67cc0937e9f89964936c4e2ef1897f48ad30b83e4224812c1222882511d", + "36253466aa8a53fc51bb8fc037d21924520d59aa36259be54b53f8d8c67015e5", + "36714da17a776533a53fdb0fac7fc0b051a1bd0bf85320ee3bbb09ff68da5b9a", + "36bfdfc702c50ddc4995779357499347962e08280e108d1f67fc6377ee56e225", + "36c654f26c8c4b5e4db51a5c1f2213b8206a5e4c88d84d5dcec4a7268b42265f", + "36d2173dcb129d98a1fbb9a25ecff8613706c046e4ccf8b49c07d0f8043f6d9a", + "374d2651979eeadfad2699f5a557d7db7174073b84eec55c9704fc171d768e12", + "376cff84d5eaabcd84de92e8037c98ff298ac1d372866331c49efed4d8ad0e15", + "37b247ed57b009b125d27738b5bec0372247fb76cb1cf225c16dff94019aa8fb", + "37c6a25a9fefc754d6d25c419a6485068c7a1dc1af5d07280145cbf7e849fa8b", + "380d3341ffe5b7eb9332109706e251b68211796ad2714d403a50e8809939f4b0", + "382e7932bbe71318dee082bd946788f610fa948c114d3a3bdd763553fb10aa59", + "3837c28ddc9543e2beff4e98d311d4a7ad2e675019dcfa0f38bc54dbdd00e1fe", + "384b8311b0fd3f43ae9e3c803c42ce0b5ac4f23f5dc1c1785def1cd01f34e99f", + "38680b605ac4b6e46bda8a39889e50dd8df12cc82c303e31928a16cf3e310cc5", + "387200a627634514bb8321770d460b668c9516375226ff1b129c54e00e743267", + "389b17e5ec1ac64f3b51977b3ebf9279409572fbbd1cc5779e5986f10ea0eaf6", + "389c9edea933534cebf8dccb4b7e03b96ebac870ab1c83bac3193688e5596152", + "38adf5a5620c39b5b3cd62b452de47b44487adbfcd855ff75b37eaa331277cb0", + "38be75538254cfe4d5e535b5bfeb08d320e36d2fc09e9d3fe0dbc369a03c6f2e", + "38c5e9c98332a91abd7c6654227b1eabf1717bc4c48eae5ba54d4250a1b8faae", + "38d86b76bfad976d136d0a0a6ebf1220b57bb58fc6d0c6bb1161474740643564", + "38f011dfb9503dbb7d7c5547e05c955e684a5de2a8a2431a40cde7c23e1b30a1", + "38f2f6a1c0398f9a45a2c8c4efe37de7ac56d3031102eede259aba10d6ae3fc1", + "392d57bd5fbc504434787a303f8e1c56064fd33fbb145766cf7e7f1a0088ba1b", + "393003d60574390abc414e5bad9f145b5a4382b3a34606d19e0f2612fce42834", + "396066036fb59e21a23d3dcd772be7f1834dc79369b98db5aab76fac639557ce", + "39907f7d1ac19d644f06417cd795e3ebcccc940383f1bd078e1b5c839f79057b", + "39943f932a5282c29a1719b4bd14cb195599345a3df6889c657702b127783e38", + "399da7528776a819912236825acf69f6cdf7b3ab904d49f16e970f0aed4053db", + "39aaf3de02157ee50731d29a7af009637f923a488b32a28213893cde9d796982", + "39cb6cbd40abd3a3b830ff12a87b423447419a54aea2df12ae424db1928a3aea", + "39d727d8064649b6d18281804dcfcc9d26bd27ff747b7a74252c7bfa62c6c01f", + "39dc7dc27988f1ec49f46b721feaabc2b63126a44c585f447c54abf33be9616d", + "39e5e558b78627797fa5fc55aab54c3353435867baecad912dc8c54062da8458", + "3a5343a13cc9fc4fe5e37203776a45663a5e988c50955ba5ee1ef78bc4f77623", + "3a573bba76d4f43d6e64238fa21e38aac63f8a2c511db774ae118d40c509ca29", + "3a8e9ea5bc01a4d394a2b7a91bf89b579395bd802683f24914a6c822969a7417", + "3a9ad0d2857e8b0dfac3bd237dfceb08c1db01b79a0169a868b8d588a589d35b", + "3ae9e28b8f68bb9676cac4f93d99c9704ef82ed42794bb2a9aca59f52833ede3", + "3ae9e7b2eb9044e1be1281a28177b8dc4ea1b4eca35f88a11157bc8f68e257f5", + "3aead832e9e872bc3a1d9324fd90910c8a904255c92159b2cb3202241d1c50b5", + "3b12a56b380ae79ca6c9a061b7e7902544a9cf51eea1b627206e44334fb3fc10", + "3b16dd05c50b6534be4523f8f34e0778455ad90ace4091604f18356a14b8edac", + "3b1bca7064e1eb773d2a2a55e2151392dbe3a9a44ae12a14017d258edb8842ed", + "3b29e3d5850f9099ae05d5171456f0a735ea8f07c137492557a93f425ef7e7c9", + "3b3b58ffa394f68cd450f57e00369649704aac9e4e954fe2d0c89f7ae75b634b", + "3bb376999dc724ca9695e469e5db3da657c83841343a498132d5849bf8e1db46", + "3bd236b7cd8f3bf3461f6e6eedfeb97ef0b0fcadfbc4a42164eea6546bfe0c72", + "3bea6795b923ffc0354dd3e3c427b242af16597e2c327e0bfd164026f1cc8c8a", + "3bec1a06ba82fe284d099ae5f195379ed8b870dc62a4ffbed1a67437805e3eaf", + "3c0d4da7c7013e037a799b975326ceb560a905a9381dce02c6540d46404e6dcc", + "3c3768c794292f1cb9a8e23c221abd3cfb5483be5893be5302dda43c86dd70f7", + "3c529806ad8b1c37819898031c2e8ac6a27d47b259acb7cf1691f51689c86651", + "3c611a1bddfd25ae3b7170d90da70818ca5e5327cefe4454b6628da9a08a6db4", + "3c7cc070b7fe2fada929c8cf26fb697e3c58062a6fe3ca348e1a1eda76b37625", + "3cb6873e95612d0fb931a822a8b55afed47585c707ae3f6136a682b244ea0f9a", + "3ccefeaa917898a3cd386dfe4d65199796e1cfc8bb4a73985af3f1e4ce3a26a1", + "3cedc03408a8b3da093700161a478a062762373e3a952e36a2e11992c6e089e0", + "3d2ff9c749ff902f5bfba3fd3f7a269b5aa37487001a7318e50f09fc043bb09f", + "3d3102629f40f0f6fc575a5a87ae7d73b895a8960e8746b056fba7f368a2595b", + "3d32a74525e557244497297607dd36cbbf4777e0778888152e1603d600b26564", + "3d60580449a07508fa4162d6c188e21c4dcea4014ae77b7b4908ff786d9456ab", + "3d74f4db788aa8bfd1a108c3cf1bed0958332e64df508f002310fdd13fb57e1b", + "3d74fce3a56854f176b3c1660203d4424bd3557c09ca2c224a3403946da2cc94", + "3dc736a1700bbadd6eb95f52a70b1d397f49aca10d5c94a3fe6f9ce946fedad3", + "3de589be6e9c38b0b10243bd7c2bc3055d643f29c3944339db80ab405b3a1c32", + "3de99f1d97b00c3baa52658450c06e846b07ea78cbe8a3338b7be49b03fbc357", + "3dfd24ce8083825ac0413bd174d48661f4e7a666faa6d6f54592d49fc244ce81", + "3e18849ab3807d887c0f4fa62a2bbf6e76a41dea1ac7950ab5a1ba4cf96af931", + "3e77980932066d7cbe8263a65005bcc889b865516454120c1f48f94c2fef9dbc", + "3e960f77007ccc87387c15f7aafe210ae38438f8f6c8ff64551033770046c8e1", + "3eda1adf6b8f49d33323ae8e6bc96a1cca229571f5927a804a6b878fef00e3c1", + "3f0c251e7b67fb6858967a27754f29ad110c89c0b158c4f2e5a96c05fd81b888", + "3f420e76b844336a91049960db17157980dfa4d5916d0b814d78cddbf57c4f86", + "3f5d876571875f965c438a8a58c96b7f32234911546f2c1590b83a5198acaa3d", + "3f6e99c36c36dd12dacfbe23c7ac87082c419842dc12bec4f74829f764a8e6f9", + "3f8f5d84da330a6c7016ad72d234c174aaf723b45cbe5d3a9145e675c2575568", + "3faf0136d7affd01be02bf5a9698317ee8a6ff6f6a0ba75779670d18c66a38b4", + "3fd34303b59e12e06bd772fc1b9792580da8a50e4d69dbf78702fe7bec9b8c0b", + "3fe4f6a0da2df15f272d64d4f9ea68425ad01fa3f0f6ee452ab9cffc98c34b18", + "400a9724d93c3ba1b67e2e8cec7225516e819676e4733d50ed33edafcab0a707", + "4012b33f2ecf8d1baf925426438baf8632a4fdb1f52dd767a572496a457e5ee3", + "4018bc333ed65ada31d42f42e5e51b29c6e927157e58cd60f037c1f26c49599b", + "404ef49e9e6d25710b2c12929318e60fd4601328688785d65844608044de48bf", + "40c9410e7b8ade1cb3a6fa71487e22f7892b5f85da1fbd14f433fabbc48cb583", + "40ebe8d29f30b9acd3993e9d72f36d2511b3a81c8d50a25b6304d3ebc4504eb9", + "40f4b6329e19c0ad22fb492fc79edcd242a2183f9fa3aff583dfcff16beaeceb", + "4119243479f2a04dfa7843c8a9357b64a7bbc4bb9378d364ba66938fa29993b3", + "41582150bb4e6f0252ef9eb9b8d9ccdc6d9111291411798bdca84198a1d29e79", + "41602d2b2ae7b9d18ce632b67e369cba36d272d2b0f412cd30a481cfa0cd49d1", + "417be49aa3741eb0120073ae71337f11595f05d93525afe5dcdcec42bcf24df5", + "41beec4ea2210c90956299942b14f90f8c6c450035b05f56a7b5a8fdd54db85a", + "41dfdd7f8a8debeeb472a7afc5a6a55d1139a39985f07eab8effaa20a412af3d", + "42090d888f6cf0a9bb0956166fbf65d0d16b4ee84bfb7975afa821ad8a10a869", + "421469c5402db04f1276760314af84edb5f2c5ca93e0d77a31f31cebacf0f96e", + "422872dac53957684f5fb702d6e11b39f4108487bb848c0d26b7b4376cef69a5", + "423003117279ba621fa3481afa37e09c1966b3fa7e49608f95ba53db066c4bdd", + "427ee0af0d810082509973505dda1b7b1befe244be465f40b3e642047132adf6", + "42a0a9a7c4bb7b6e66ec7f68c0905d2a4554f50cd078ea3465b2c3e4b6a7e085", + "430794193b81c68d824dc36a195b9570fab05a4a0b5b0be3c5ae5698cc82b6d4", + "434ba4341c13fc1fc6d71d9eee95ea2a39c06771630b215effea797ced42e9df", + "43536f1ba3748225251c8d65c5a5ba97747caf695d420f85e4b62457c8a9db9d", + "4356382bf08cc0c2d865bf6a0e018cbb2b90415c5dd37f5207c5a874e7e1b0c8", + "435933e580253d372d941f3a6b5be5352dbcf61dcd2355007b7f8369c5dc3728", + "43977f363939808102fc0d611a5e27cc924308b2a5a88939a46be95452a0f070", + "43e04de95dfed0fbb2c29a1c7485d3d6c66cc66ea7ad695e830915f1358c3a4b", + "43f0895baa6de45c86e086854700cc2ab2841f2afe5911496244f45ab3818109", + "440e2bcaa12caf38dd81cbd60439f72067e1767478c6b77e3613f76edb11e8c8", + "441a8a6e8fdc88b18e5a5174eab8a9f9b5110d15a3c5329ba01b9ed9da185aa7", + "441b9b6bb84b083a33a31a436fb9dca500ec5d7ca0051a1b7813eb2684be624f", + "4441befd5fd6304f1a65bf2406aa25ad86a0f30ca7bd4f5cd4deae4d384f1ca2", + "448596ed1a8fb48786ed62faa09ff8a9cc1ddb31179cfc0ba5eb60ee00400112", + "4494dfcb21b68387a1e0da5394813e325547a389598b224ce8b10b7bdb7ec945", + "44c16b61c03f9214c769001b2444ea84acbc91960c125fec67b7596862bb4bf3", + "44c9620d8eeeff6ccccd98d825af7354ed4eb4400131e03b5ea04409113b39e6", + "44da5db5b8e8740a38435d426e4eb9f855beeafa03098faaebc33ccea9d0b744", + "45244adc20436d2f5a427b80f4f1c15a73d8579ea5d2dbfb4bfc72e5f997d115", + "4557c4cd811e0166e218978bdb775fb6ea299c9603e3c20de6afb68e24706d9e", + "456b221337cbc952c55988375ec557286728cd771ef9408f1c764a4073a1e8f2", + "457bffb709cb5ccf6977d714e33b256783469ec035f37c071395e36e4cf4debc", + "45893369e05b3cf063d77f451ff8a6417a1f97bd6b4838c8dec22bef43bac523", + "45989e843b597dec4790098482c0063a6acd65d1c01bba9e8131bc813df32ad6", + "45dbc8cf1ef6e1c9ffc1c8b79f3e7513ac92f1e46154a0dbb7edf3d03ee898cb", + "46382d94897e05a802f9482b1e10fa0ade6053254dcabfc957022abbe14e500f", + "464151651c3adc93f0644b3f9b94b0e98dbb04136e92735dbbfe83b18658074d", + "4696efbc5ce832da96a9ef1c6f515d0e944264dd070cba698e9e89323ca114ae", + "46cd422c309d4f4394c38d7bcae4ce458613793042ad31626cca4a303cde1a51", + "46e704548e226c14dea88e9ff1acaf4de243139614902fdf3f5997d9c13fb34f", + "4783f6f77bb4629c69d6822d64cb2da730348eaa8a136a24194d24d53b83f559", + "47c6f0de97d395643f179a9f3247d2768707b5964c336eee0765202a603e393c", + "47ed9d7edc63708dae6e1f25927c5231c86b6ba6717983516b994ed544bb2e8a", + "47fefb31541d1c4cce72568f220febded0b5c4ec858cc6c1147050d8878b8083", + "4801fda1d5dd623aa273a8369de75adfae8657152a36650b6e8002da433117c6", + "48126690b86a0aecf87d8e7251173c0020ff593b458133ae8b560d6d9f5370da", + "4828a99d5491c1b8f564b661ac866eef5afea099c7f7ad2d67924e2c735a004e", + "484af435548059ee71cf025a6b8895e8576d14575677ef293037221f0624cc09", + "4856458e1a9f38fa310eba2ca517513c74fdbd8d62cf29196f5656460f42e268", + "4871fb0f89facb7ca44af55e031344953f8ef45c7b7585984f6fdf9e5946fe74", + "48a465de4fa18a9eb99967acba19b560edf8a061be1926210d5e96c837f18b11", + "48c30858be8459db0ca75bf71f5121e2f0b6ad18a328108363184efb2fecb713", + "48fa46445e65a643e29937c69e0538384ba7f1feef2088a73a6c39990ba26549", + "49253c445ce10685f3fef55e41d69dab439b9c9c07ad1d7fa96bfcfbb343e1e8", + "49371386ec950ac3831816f28cb358301a13abbdb3c56f5fb42003916cd59796", + "4938be3383313071b1269b37d6459e6a87242f58492b02dbc44f335d1a83c5ca", + "496d3f9c41547b2fa460c2054fb7c381747424a75117b3e471d5f5ced0067c3c", + "4979a34d6cf19c97162debc3a657c79cb50dabd39c747623d71f2c520c93f3f2", + "497cdb2183fd3f04cc5a0c73dee40295e9bf0a15a9d91234b7bf617280cba693", + "49817e31b53b5d8068fdda4e2e63369d63d548bed45065c03e04df508046367c", + "49b173f5c0aaed30183ede6f3fe27ac6e96db679cb07d35802d9576bcd672d88", + "49d89c167a538636e2cd7a049b633cc8c7f28d0e86860f2c30b3dbd3de4c8c92", + "49db0c2118ffe8b7f171b29e2e464253a1b10c551ae05ab473386a050c8cbcb6", + "49e4287b6ee26db52e0b2ba0229cb0cf295899071855dc3f665150df4a689ced", + "4a207c0a8a8e584fa8a3df1ad6d6a7572faec70672511c3f9fed0b5d608eed50", + "4a42d21f25dc546223c6113b33a861a36837ebe67506cbe9b0c1b75c8722d517", + "4a5e398567c199f6d294a82583b892fb3b0d3b5c40fded337ee09b634132d9ee", + "4a64db3dc71ce8b6eca7ff5091a0277e2b80ad862567edf4210832e7010a5521", + "4a77ffbfbd81da55bd8c5706cdb05b1dd3120466e1ac2face12b1206e0805a6f", + "4a7f6bac80765327bb0a049c8fc3cfce86f18cae7c8a1c6a57cd7dabec934d52", + "4ab9fc7611e72acf1d3a55fd6ebeb87ffd5b1efedee167da17118bb5705faa3e", + "4b09b1b44158cf0cc3de2907ced21de6de81b27ee094308de5e889de05fd66a7", + "4b0cc785d418234ca91e5b634d702589cd328168efca29e8cca374b3f0806b0a", + "4b0de9893a759494abf37b80d8c8b14487ba7c1f4c2933e3684cf0522d3b2e96", + "4b17ff76e6f9846ad966ec619bbfecec4c15f6155b12d82b97d286994927448f", + "4b5befafa28e84c8bb80aec49d60255781ed80956037db55bce16eaf8dcd7587", + "4b6543e98732849e1e2ba3b993f7da9340803e9e91a1f5f4c030ca972f720c1c", + "4b65ae471b05d20e97ee83c2c32fdc74e406e3019d0e9361b025dd1c2bff3584", + "4b82bb99d206dbe881b684309528b9a233f7c01aa2de42ada61be3497547eadd", + "4bb7a372a1e1de11644233db31b82c7d11e50c0628e7c8f58b2e21f988d4333d", + "4bc0b35b5c19fe1e718251ebaa8fcc726b3d0bcaf4dcf4552ddaf4bd822f97ac", + "4bd9eb8a050cdd74cb9e1348106cc1978b8607f0d34304f7bb856d86f99deae4", + "4befd84c3ba8fea7a5c64dd302955250b0d3ec2e3db7dbb7a1c2d2123865f273", + "4bf3d35df938a7a2900e0f5cfd4a174643b3752b74b69f762a579433444cb9d5", + "4c1549462902d97cee0fafa319ff25c0acc9a9d8b0f0c2fc046123b53e16f107", + "4c32c99d4808fc50a5c7d89b8a1fa32f8dad7f0ff8e06e748a1e4be8199b375e", + "4c33528daaa8c4791b76b49410b4081aa6bdaf0ed2db5bc7dfcd25f31ebc6a06", + "4c8d5cdb0ed963ff600d8701fc929d22a60812685d3b69868d4f42c9fea1bf6b", + "4cd852ab8d9d6c1307b5d037a8c2cbca1b9f0f651b9a4a8d75869ec89910a4e2", + "4cdaf3234c626e8930f9660042f269725faab25c1d6eac7714cf28ada00149ca", + "4d2c2a2036be8d1337f8f878f270dff29bd50ee2cabe82f4ebae082de71208a6", + "4d607073270ea17f497483ffd6baa03ae8fd6e770bf0e0094fc5f3ad9b1b38eb", + "4d747008088d0d38ae9c13e763ea7e35a3fab5859e0169b563aa077c1f5652a0", + "4da2ccad66b0320dfe85a3a91f9552177cac3735266fe014d2037a5a67257796", + "4da980d9ac346c38717f4a7e2361c05f0a5cfab7ede3d665368b67a7bddda4d5", + "4db5086479c285af9c641e7f93dfc4d3a7ecded77e9e29e377b7a654640e87c6", + "4dc96a99b46cd33924f46cb16d8a924dee4bcda345895a6c522eeead47208432", + "4dd7e0eec3c93e06a0151da75caa5a8dd09225d92f887f216671c589cc2f27c4", + "4e06b20920cb157f51231aa2aaa4b05d984392e7e0feac54112621758cabc3cf", + "4e129f2f9c65a1628d79caceade78b6ea056984fcc2e274f68ab01aebfe2140a", + "4e199e45884bb64c5eea327a917cf04e6de8e285df60592eb2b4afaa33729c43", + "4e26a5a289e257ad3df1d62ef154be1e724e71f19bbdc4d0ef54b0b5eb18aa8d", + "4e379fc475c786d5b1be61d19965483d11829f54164b0010d4283a33a8f60845", + "4e3ed35ca52c2e01a7cdd729e7068c1c03a5dc841390d426b04070b750adf764", + "4e4872b564cc0e5f969c3249956cf8aabcc79b0438edd7b02c65391e915fb6d1", + "4e7ae1e39495741171f3aaf74402143b8cf31d863924de41c80a69bb6186b89a", + "4eae55f012e712d2e10100b46eb90c597c8a0ce62bd167ee2f286e161434e0de", + "4eb8937b123b44f09c2574a572e384c7511a3ea950eac45d4aace87c2f44c391", + "4ed094cb2bb666fd3da8853a36964758d91dfe66d1bdd347e0f7c8b8c718ea0d", + "4ed6453dd0eadf7321ebfe6d030fefe0bb7daf1e52cf195ce43db0ff2fa5f4ab", + "4ed9df471fa4c9d9b7d6c29eaf127e1743ef80322b706f10008711fbdfad772d", + "4f186bd841dbf7ddaaca22aa1552a1e191844e3b80866158fcc37236159b7531", + "4f3d06d0b1335a13cb225acca00e85b351b3a5637c9f84e70bf434d52be4e93b", + "4f42fbaea85d8740f2a250daa88a9ad7dae53a07ab210746cc1526de742265a9", + "4f4bb1a8cf80e2c0847873006a5ddecc6871ee627588f1afb032a56779eb2cbf", + "4f865a399d2d50af188ffeb535c333d2017387ebf230f0b13e0e741dd35a350c", + "4fe42b5a3e9d4c0f6a2be6eacdd1a0cf45affa7cbb687c50b143da1179fe9c18", + "4feb47d1ff7a9cb4022ec5fc0914c6fbce0dd9f1a1a7af70a36c7c83949ddda3", + "500325e34d1afe2d2b79d83e7ebcc7e4b1f140c8113bc29fe712cf9298392906", + "504d7971b807bdc6ff779b6c274db38dafda22ac1a2b7d50becee1b3b6a2dad5", + "508a89c82aa71579d982948d197a6967e9bc64f5ec0b224467c4cfed96b88785", + "50cc531f76a921df9b6e2eef827c5921580930c58434a2778b01083e55967446", + "510d7c189bbf686a248aa8172d9553649b5d82cc389ec42c7f1c95bc3af9cb7a", + "511b16f4f474db69694757b57b5805f47decbecabef68ad4050812a972718198", + "5121cfd3d97b25de8e92fa91c182282af8d11d6a9035fd522bd2c9cfa1c3c075", + "5147c91d7c07fa96291abe56b71abe33fe5c8260d37c54b7f4c2008e802fd36f", + "5169e603e9ff88c8b5c21b97f23a0a4f96ddcb40ba25754c332c7cb0edb1640c", + "51ecd39f98794af8a8842cd2d4b41bc9cf9117825ff9915efd474464b2e5faa7", + "51f34f7fceaa4c434accbd8bd8596ac00f4a92642a437f46750ef45e718b69db", + "51fc880d84d3d8acb797768c4ebebbae41bdfc303777b4e52ec7ad95c08ed39c", + "522748f349598f8a5b5af35825819ab6ee2456f91aeccf1083dd7f6353480edd", + "5266828a56701e05b3c0ebfe65103f9d484b036c59b4e1eb0bda970bd5cf63bd", + "527e94eb60c278adb854c2849e5e1d508b6ef13f651ecd7aac0916528851f8be", + "52aa00e24b3fe442d1dd5d535a6b8e2287ee8a5e2bd85b334c377bd26b7bfca6", + "52b2e38e8e13617822a9291ee6c5365b5626ebe01d4272f39f75a7c9335c848e", + "53129650eca54ff4f0a67bb3cd93ed89c64d5dc87d945e9e8ac09ce72f8650ac", + "53214dd5fd650a6d93dc2d3ec89c65cc7d4503cdf665e7ae2b2bfb4895fbb112", + "5326f5cb0bfc01d08b7c17fea34cbdddb4d1a4fa5d907435c15a747b0750af20", + "532f79785cd9091700d6a0ec6a83d5d023e71e56ce704f91e3f5d92ec12a9af6", + "5335fd622f89353e5f12da72d7f570a680ea20fde15a4d030cc3e682917f3e08", + "535c9f3b9205d8d835cdf757b11cd0fe848c1d8bac1c660daa0e710abfac44c3", + "5368d271fa15e996184438f729026a741ceafa7b702e2f5319a02ed4a7537096", + "536ca7234857569d044bce93cd8058dab7eab3b19f4c4a56f2ac347a811fb435", + "53d325afbbaf0a085c8f4dab65d0ad4f6f3e48ad455df4b92efae664e3ed30cb", + "53f7214b828aa5afaf391ef2c4802d1f8d0c782a1d9bb71ff86c023134cda878", + "543c4fd0c392645155f627609c7d430a4e4d06e576863c514b20e1a564381e21", + "54a84ebc0507a9a9e843dc592c639f3b43c7d8e94c6019d71029c2d8e15dd930", + "54cc4fd24af0cacded407bd4f3bc683e25a9e969d89a010d36a0dbd586a2fc3a", + "54dc85bf612eafb02c47ebf4d67017c8de63bb7ced7344de32de836ea5551797", + "550f94d77469ebf1c5b125a72fed8df79c24a28f669fd339d74aa22ddac9b5a6", + "5513af533ca18fd5629a153368b994ac91b0449602a26297479018ab1f926071", + "552e2500e0c22bfc971e5fae6456053307211e2511a4b3f4edf67248e9e60c7b", + "55a8869fa9dc2191f7aad76c572cd5fbb72b826f2aa1566f0ede2fa89ec2a987", + "55fec48f1e3e713fd61d8afafc86d0bf0bb63b88212d7a3524d42201e4363d47", + "560da590c9a2d82030a253d044f3622ecb732576a98175b89f80efcf56d029ef", + "56330f94ec1211784277ffe3d0e6eb9d8570375c1502885ac9d0353bbe9b92ca", + "564b347329fe24e1e6550303be041361ae32520e753adcb0fa3b70d24f42953a", + "565a04ee80c7f76dd537e36950a9760676142d67f41fd998fc8295a59495246a", + "566a331cdaa86a9e637b843b124b48b2e492baf6e9a0856458f6192486912efc", + "56a30765776309f19a9e9cdd1f6d9456a23a1363636db79a0cf650bb7e31c547", + "56a94d11f9dd24ddc273a377cee421a7041081158351923c2bcdf1c180d3eae2", + "56afdf35be6e9e568b089419fdc113e430ed702e03f627833150a9b0f8b359fc", + "56b07e2d192eb70cc5a1dd8717ea1d104734dc3bf0247a85ec610a35ca8f5461", + "56bc76794589d321520fbcf73773be5152d5026d6117d231d5b49aac792836bb", + "56d9a85095e43f325d13a495567d69055af77d9eb0564a401e32cb3bc89336a5", + "56fd2f352e422edf3479dfdd0d0730885bcdc6e8140a93fb29efa33ce2e1e847", + "5750d0299820d644772be4cdb8b00a3dd45729a401c65134cb2a76cab7e4fb2e", + "578c2107d5f26305eed742fdc8f309810093277f54a5d831cb9d62431ff3761b", + "57aeaa2185be958180717afd2d67c85c071eac34214811e3478263c4d376c152", + "580347991ea4f30e4f531a31f6c561c15bde59a529de296194e4a35c2460692c", + "5810e162ef494be2cf77abb454b744a4f104dfc64a8bf30d81d8e065b3a0d7ce", + "58146c983a0dadd6f68056f5d0b5b285c1296cea6662566ad16257119665a177", + "581e0f0f3b40618fd8ef5e2f13555c891e6ad26e779700fbc9e7ac7a74370e85", + "585214c1720d13b39d0c41df6a0222e8eba17741364312ff3dc9aa1dab87743b", + "585e3e07fc2ef6b1dcbc173166d50ab12c9d23ab5b1ac9b6d2755fce6f4873b9", + "58773571cee830e082df51f368c9a63a7f20d290046b62633fdd995ede99418b", + "587db8c7d804f20dea3cd62a9c01057cb3b49601e4589440d2017bd02d638cc6", + "588a1e302469a7df1104d8bb094ca75222ee8da70b0d7f5466871c8247366fcf", + "5890e575bfe1cbd1718521af938e9e6042bf386075a3f2d18fa875cafb6bb992", + "5896dc528d2b7eb93675b4b1cc555efee4ff2eeccd4c3dfc8d36bceb18cf0e6e", + "58bafb2f5bb8a3b1eb354ecb07e3453d00501c6b5baed70aaab8391c3f400a2a", + "58dd3c4e88e304f5b06967264379c278b8fc6f530fe74a4d3aeeb00507badb80", + "58e1f86fa5ce3520e7d8f58889ee0d0c51e0157b76f12db19c7380abae59ccb0", + "58fd614ae697249681124a8c8fb27476cd7ac07bcb9bb5bec862323be835e426", + "59097dc61bdc1cb165e5fe9ef02fa705888f8737f2832e5e5355c1308efb9e82", + "591dee517c139be11a013a596aab5e3d4319d2d056f49a8cd9eee65859af7c6f", + "594c5ef422269e28c70f7684ae1395e7912c61532d21daa7e6c1372555afe0ae", + "596fbf2cced5eaa06817fe6cf08e2573bc73576833d15c973378433a3530b2c6", + "5973d3dab148e37f2b2ff8453d4c0db021866575b4278e4c53e4df8a700ab04a", + "59827f3a93b7f8823af0a16a43d17644aef2f7086b6138074a2df2a6ce5e6a48", + "59c53eacce4fab10ac3a923492063f77745b6b08badcc44256d82f74886e9e3d", + "59e170ca4b004d729ec102c5c05d2ddc53d12c725e025d112037eeb9dd788f7b", + "5a5d038adcb164227b02c45d95d545b73dc5aaf5c29f34d2dbae3866d12182ff", + "5a5faac5b9dd300ae710f7742a8f72e16d8ec858b153f397e45af70731cb6d43", + "5a65be1f3c040bd2c6e2fb583752e5dad7987e11e451d59e690f90ef14dee23c", + "5a665b818961b4ca6b32bf841165108602b3e27012b836b0f85d7561852d6d56", + "5a9e2a164309d31ea17ff845570e206871eb9f4a062edca50fa000b56818d599", + "5abe82d5f6eb9f97fc34764a7867c2f48f6ba3bef63e0c0db0283cb233d970db", + "5b077baa508db040e7bf3405264cf235c7d95ae59f33c54ce4fdd41a2297badc", + "5b0803ecdd5ae32f9940911e057431a19da47f172667396a50231caee56138dc", + "5b09088a32a4d31b4fa61cd7be14176699d042b759c867afe3664b6d4a3c8d54", + "5b29442d8070ea3f34d021c2229abbb1203ce47102ccc375e500ee28380a4999", + "5b48e419b56f1e0ac888f78d01256d7d9792181d244f7cf160bc6b33b86fa0ec", + "5b5d08a56b25f363a8b8eb54f1b66af6fd8d5e1d37ff19b56f08dde5868ba810", + "5b739a71f1b3b09bbf3d54543b662b0ea4eb22fe35809aa0ac14f2fdc343b6cd", + "5b8782b98afbfb414e51f003c063aca6dcc358ce27695ea52a0cb111b7ee3863", + "5b8ff653c10153620887a41370f6f1f7a862d06f9ff05002391c76ae0c7a7287", + "5b98921008beb016b9637d68f80eb8ad495391b3f2dbaa77ad64a705c315cbbf", + "5bc1788d1e6b4a285d30916d2f3846a50e57f7ca7a018eedebd0cfe875df9aa0", + "5c03f2b3a7c6f501261291db173abefc3d9d73a4094af49d78fcf6b76e505c5d", + "5c18ab98cb1bc02ac71d430fb9236c4b59f142d9d3ed4edac095b38ad1e6ce6f", + "5c24a8958cb0c33ff7617e4fd9770cb9e5b1f0d9932017bf0b7e9c7322516d53", + "5c25781bf806586340bf9b17c1f1dfc990e6dd21cadddd271448866f44d1c684", + "5c3b624fba16f6088eddcb0c9b9422eaa1aab3175b5e89732601d788440d1c16", + "5c4ed7d029b580c166984451d8c1fd42460bbba0a224c77b4e69f70cf1428a69", + "5c59eb61bb8b5953a21997aba7dd19476888f96424d4aa3e9f2eee253bfe9b25", + "5c7c610d7cc5465f9cbe4714c35ec928553493d0a16c9a63a0e57f8e4878fc6c", + "5c8b92dda4de3de4816c98e1d58444dc30f0e79493330a36d857be85bc4d3b79", + "5cb6260be3c98ae3ea566ba38130c44a50bf48304db36059d4c05f345458f0fb", + "5cd356062f8e1ada45e2ab1fed44d13c8b551331f1d485e4b6860466f78a8372", + "5cd993f6b096c7825ff9dac935488ce03f8dad2b060d67bf2505f22491292f5c", + "5cda3ced22b8cbf4998a18a9577a5ca62a34e37b38d118367d9d11d2d7d9b3b5", + "5d11328833f680ff28da140865138c3ba6383e972a7d95425a7f199a499017ac", + "5d13e695115cf2fd4c780da5bd73aa6774b7ceb334ebcd1bcede0e7716870565", + "5d1dc41467ed9f448b1dc703c862c1f6d51ed29037aef1e74964b8341f96034a", + "5d349e1d596ad3e2b74a6bbf355fdc4259476cd6977ed29600cf27ae50b32f2b", + "5d4597dfd3afff704c7d40fd84648673786f0d4a6776311ccac43be8832138f2", + "5d5c5ed5ef38e292d47d4e9f0849a6a550f03eaa18bf6f50d757cffe8d5a306e", + "5d8923c6ca6ca759ca9b471a2e08fc501c0ca65a8f38fc4e08fdbbf9b1c37702", + "5d90f1c2ea5e0973c16286af4fee863e69bd2c5dabf11e68efa4b30d15a38ba9", + "5daee84cb06225feda64408ceeac2f96556185059251680e18b092e894f6bf16", + "5dafd93880dfb3436f97a5eea1ecaa129a2b5b0a2765fd61b7ad752f4f45a8ef", + "5db84722f01e750446f95efb0f7b1e3e5be4e972df48183ef6449dbc33831e54", + "5dc3312ffcdd9b0392d350a016568b35465c467f4a8af3f8df8a01bd8d0b1a99", + "5dc49e102663cc053f4489ca3edcf207447454b65efbdeaa94f23a019e50b17d", + "5dc9d6774b9e06240c24fea6ac9ad4ab0a1c0fdae3b9236eb445bf7e05eb303c", + "5e58ca69ef681e85b55a5ee44016c5a277003ffb6fb80d76e1924d7911355b36", + "5eb24ff067b2d15897c338e09f972b4278a6a86b9167986594fcbfa1c4bc9c5a", + "5ebf7cc08cdfafc0c31b22d4f9c6765865ff9e63d72971016abc8c4dcec766e8", + "5ec3e6a888e606119f06059312103a6c70d1ddd2765dc399093457245bd07189", + "5eded2b78b90beb97074676bf47a1f3243b84ecd855a25aabc030a5a81102b55", + "5ee5a0f04e5de073e8b2e1ea283e8811305fb610276f297c8280b061019e3738", + "5eec32ec10e48bfc7f746c5560512e381d9c44eede3873a4d40f2854efd57476", + "5f9c8bf3d60bc1a4b796feff5e5340f352d4b70eeee37d3acd503fade03e1acf", + "5fde1897d8bbba084ff2a87e8fc4c103418060c56244f796f8b102d3ef6b70ae", + "5ff67526f77dbeabeb3eb9af58aae32ed2bd0af1c859deb6830c92b283cb720a", + "60072d766b1383035d005e1516a319b9c435c4e3ce59b69aeb5342f07dc380bd", + "604d392d36d6214c60fba0281725ef2125e573177d58d54d6a6f8183b73c3f66", + "60537dcabb3d7bdc1e312753525dd233fa8a7745c77ee26c77241a7886d0209c", + "60589d3cafbed181eecd4cc288e853fca6bcff092197b3b42995411ac6e3a2d2", + "6059c8c920478b870ae974d4422cba0ca49af2bdbda2fbbf10838cdcfa5c0387", + "60709160950e949aaf4b06a81bb046990a4a2699414d23ff7390998bf98867d7", + "60f873f907ee5d4f835f6bf079ea80950c32b0b885402692574ffa1bf340a0bd", + "612acbf37c5e1db127331fb6a7a9faa48b65756a36cccb51315bdd2663362227", + "6137250e31cccb6df9aef545f0af9e6f425d29f57df8c0bd59adde97f2a35066", + "6162e12b54d9c1dd7d764333543b0c1ffcf5660c585575595c04636beadcbd55", + "6171a6e20e268325d4a4a3fcf435febf5bd2572239d5e36dbc1ac4b091e86dc6", + "61ae93a705ef2553ecaf50ee273b55ca0e859f6be1b5501cc15ec1f78750f7f5", + "61bffd76ee2fc12fe529aed3fea7b656429f720be71e72d304b05ad787ffa1aa", + "61ecf85f0dcb3ce79c049cbf4246d31c3e9ee4fd97924b4d9786ae87ca6b6a6e", + "61f9efcef6f5be426991f087dd75332705677487880af8042569287517516522", + "620ade8ce0fc6e5781a57ab85ae405c9f62ea0360ec2a02d441ab4b6fb9f39d0", + "621d7c8e6248697e85db92cb715da91464290c722f306015957949842a1c820b", + "624d16569d89bc9da4b8f535ce8ed165b63ec7b7a290e116bdaaff37d0a09f2d", + "62a21d830ac26c364928f82f927d53233bfc87386ff2e97163fb981bdac82a0d", + "62db198b6f4c3f1a462a174b3d5d5a6078bc2dc15790360bdd7116e0f1e7f31c", + "62f9affb54a325d03ea5e8311b6fa62e04263251010f8f1e0545824b5250fc9e", + "62fbc7ac4cabfc232dfe0d8a78c55fc656073e2f6642396082245c9c98dd4998", + "63157ebe4e96307820a0d937d2a37bae3f9c14cca6cb651150156ec5c6d36448", + "6335cb3ff64c3cf8541dab770520a7cbcb6e3c8bec899c10393eefa9814f6ac6", + "6363f95dace9764ebb6dd5e4cd446dfe73be55be5f9e0b063ab5406af571075b", + "637340653f23ca873db36e16f9e98d6a1a1d7bd96baa07cf2afcad5123c182ee", + "637ccf26a58ec061ad2ee59d1f87c95f71f0bf8fa029b7ee9797b2a909c36ccf", + "63e61f2694503b1cd61ab7dc4c33759d884c3137d7e4f2a61709c562a01790cc", + "64642476b64edf3a64d69b2bf3231a5175d46c8a7db906ccb95271a787a799e1", + "646c682297f626e5d120f89acc519b581e8778b3b1dc6ffe16476a9835f3bcf0", + "6489ad49812acdb7da4187f73f716b78e1cd0b589e05fac85e5c75b1ec5cb1f1", + "6490ba8c17967e7a7203bce09753747dbdc0480c3f827fd4db746591b610a1d6", + "64d4e42816228f1b393c4571a5ab6e78acdc6e1a207df3bb942305c9de8d1c61", + "650b7cd2d1d95428a6e9ccf0dd9a40a72f1c636182446dce691c3169a007a87c", + "651182a33f678ada4b45eb1079ef7e8e105ae14b5bad8cb3952539de77cd7d05", + "6530adeb56d3e3fff647f3ee8d34065bed5ae533ea00772bce081e753c861291", + "655231a11c2e677666b6d7ba200414a8446e912ca60e5924b9ce1c6f8998f5d7", + "656f782ba203277a60d4a4c5a65a83feb7b139b96c5d5b0ecd132f21fb365f8a", + "65714992f01962d6e80f7afbd4c8e6332b419e436e0cd5d985958d03a754d787", + "6578e518dddb95d215dd3a7dbd1577616364d9359162715ab59552d3ec77d308", + "658b002d9661970ccaf80be9e81500d4a55990b813b813fa0846edece61b92b3", + "65a71290538408a4d8e1f694b3148e0f09b74528fe161cfe3f2778d148f23b26", + "65b2ebfcd8434bc57234573a03281c5ea42211945505d4bc7926620e7c037883", + "65f741be093e64aa33aea625d6cb2a0da9d0f66794d345d4d36a64c194f88a89", + "66106dad07085d8716cd3a57c38719985cfee9d7da7b9e75d42d653fde524a29", + "66278a5f3ea6f31ed36c22b8fffa8d927eda8d7f17f75bdcdcb3e476a900b092", + "666560495c0db35d8963004e3610db8147cba9b15b48e2b5298340a7a82411eb", + "669c686bd276c775b5e02b1268da9fb8d07c792351d0016576732ee0c9bbe7be", + "670869367ca4532f46d9214993c0f31081562026087e6d64941a9c4a7901875b", + "672bcec3d5308dc7eb9a921e5ef64fecce84cb150510afea304e61326daca37b", + "674b60cbcefefa4501172470e2073dbc4946967875c5451bbb39242b91c6ce02", + "675b54560835499f03023d08441c0ebd1e0d08f93c67df4d03476181c60717e2", + "675ee443873b67c1a0d4acf4fa7d50e28b2fb70531bf3af261d528478e59c0a2", + "677a7bf5d7402745cef77d25c358406a21a994b948b84b9627004b3eae0ff40a", + "67ae8e470a9c09f61434cfe21c78698058e3d595b0f68b7edb31c359c990f1c4", + "67b7367d7b14561a50011d56a056e3b8e4a46f32d6e44f48f335474600119352", + "68492785665af90395b524536599ba8101495f429eeed201c267d1a9486b8765", + "6873449ea7587bd681061ea00fb7a11653cc4d9611d6544a074bd95e1bd04434", + "68738cbc5f45b159196c3cb22db18e4c8daf229c4980efcb501427372dff8f66", + "6874e93c2da9e538c7b3817ab71c77b8523fc4ac9d4abc4ef62b46fca1afb8e5", + "688a3beeed114db0cc09136ff3e1f826a4c0009eeb0a5dba639d70c5b8479d10", + "68a2fe7ed3dbacfc51faf72da293adfba3d7aa87c52353002c82da7f71ad400b", + "68af249b1ba166e2336cd9e42a28de6d46fe383904832c4d6537e7352321d073", + "68b27de105169ec72e0d2b178ba1eb0c3fc0a98b4a7995b2b3e517c32a96c4bd", + "68db72e7e93af0434222b65801ab46dce3761afe35c0a144fc7954013456918d", + "68dfd709de674c13150cc384e52feef725a9bf0783546e621a96fbaa2bff3dfd", + "68e32d09a1873503fcd8c4473df014183b75fd59524c232428413f0e7c26069c", + "69247df2ed55b582d2560bfb3db2e758123d594cd88f2dba1e0fc4bfc33b1d29", + "69249f79f5aaaeac200aabe1636937412dfe678a99cd79bd6de302a2c889fb94", + "696e2f5c8d58f45c1879a88969d911893c391ec6cc625afbc119c5d31c1137ad", + "697e0f9df442d5d8ef90a2bdb81ddf81ecadbb8870c89f7021d46763fd8aef76", + "69a3819da4c4f0c147398fb5b7da7d804192caf89f06956ee2348b2ae023d499", + "69acfb9fbe3cd486ef3b93739328d90eb35b2431b15e512b081e8eae896d1e08", + "69ba2a070b08ab1d9e2b171e5ff96da782a6b2abe6d78847e82441fd8f6e48b3", + "69dad4799c84f69e502f17319717729db7b879694541dd198f51c79060f2b2f9", + "6a01220adc0cb957e87d4781fd8dd89b108d1e507b27296ad28773a0840567f9", + "6a0e6e4014dba325b39185dbf1a7a08e2cd10ea7c02bae337c795d68fa6ffafe", + "6a2a3bf1cda8da3cfd63ae6e9f6fbb55fd2ec6385749e7ca09f27edcede57584", + "6a2e45123f24f5b2a79c78a7eb96ab498b77a0247d3cb1d3a1086ba6c026e1ec", + "6a79de837ad6b4d5b78063a389d5a30efc754182c9a450f685bbab55fff0ebf5", + "6a7cea629eb5f5f3bc04f856a20ad4252f2ff11364e698f2bccc18891c9c0ff3", + "6aa8fa03e2650e4784898ef4e27a212426010d0cca9f9cea722b5a9d050197dc", + "6ac918e60536affa2c356c46bc2689ba51862f52e0c4da168f637655fd085de3", + "6acda15ef961735cc7d69ca607a4e11cb962c535ad6adab5bc2953ff51d6e9ef", + "6b166244dfacb745cd227432fd643b7259f9cddc517b05245f62ea70a95cf934", + "6b228ebec266d57dcaa49b58b1db767784f6486d45ee95dc131450a0479c9afe", + "6b27ba831dac670539efbe9131002e9213375a9a26af267c308e5f855c2486bb", + "6b4cf277783c75898a2559ae6e129f1a5af61eea9943b9d6a7d1e2248194141d", + "6b5c7b0afa8e4b320afd0b89e313c9b5fa62fa33bdeb7cdefc1673f9c4acfd4a", + "6b79946dea25c3685d5237e236c8931782449933fce6917c0c7204d6d7133fe6", + "6b937b0554365fa001d5016d0af872c629c510ab04a6e9e8580bccaae07047ce", + "6b9d661d4eb6199238cc5cf77affbd3616aec0b9a794418f256a14003b87c11a", + "6bcf7405aaf5b98665c658334cd647d60106a8b76717466a9c504d28cbc31c61", + "6bd312d9d400813835b84759667398a5c0b8fc2e05b5461da95fd5babc3ee785", + "6bdc16be5569ecceab477b89fe2586151da5c490333f5e0a1f596c4bf83f6b39", + "6c0b4e79c200c629e640557990dd39fd1b5fbf70f4ae153c137d6f35a563bca2", + "6c2f06a98f5f92eda14193ff996f38782ebae82f94c856ccfb148c4c5bf5f7a0", + "6c7fa58045d8afeadadb92380fa79952b72c66c3ca4a199ffd05b405009fe5e1", + "6c870298d4b3b9a594029a58de126c7d3912bd4c8aa8aa0956f96e0c94214237", + "6c93ceb690805df915a0d11307c3e898862ac2a8e2f03545d9f9f31333d97b9d", + "6c9d4b7e0d99d0168eda469e87ddcf684c3ff33d2a120784b3b6669089ff107e", + "6ccc1357aa0f377fa2486a55ec540541ef6f9143b4215c14f2533188603b0fd0", + "6cd5081e4647c16bfaa05928de0b79d48228f7163b2c097a78b43258081f0489", + "6cdd98c3d0ee8d4a939eee49d26bbc9744fc9e22e88563d636ad5b1d4572b211", + "6ce8a8829a4467d50859e5d48868ef249c31b4281ecface940b240804498a9cc", + "6cec9c740ba668e1bc5979a2f406be9d4326eaa440b49583caff5fff16a4a974", + "6d024da7f213afb6dddd051098fce2e2fc219009d7621833e4e0bfb3d459e93a", + "6d3a64c3c7d6bdcd11e121de5b26f8bb5b8ec0e16dd8e0a9614abb6279689583", + "6d3d79a21ed625cf8c2bd6fa5233f3b5eb112cea3b896a466208ff48db23837d", + "6d8429ee4d3409cf09dd4c1e1ee8d17ec622f3f65f2d04d2d5b46d3a1e19ce10", + "6d8b1d53d160411812b2e63ed22f7fab8a2865cc200ec6dd7a103a121405c5a9", + "6db6f17d30a1e7d929b1574447e0a58ae84fb52e785dc8ffeef72988e7b9918c", + "6de68bf5dbfc8d0a38db7653c7b28a6172424c2f8bb2e31a5318bd0636a34832", + "6e46fa4e2fa1ecc23219527e1ee220eb785973a0f8b404956297d5d6594a3847", + "6ec6d1488d2dade7f32eb4be6868b7ce41161dc4b88344dfb788047806eb3fcc", + "6f3203130b9b3bd29b81708bfc5fcaf3d48926c9fdf3e98eb330c02a99d4d702", + "6f6226cd7173c25f9283a26873c340a53ed94d4a734a6c38366fff4c82cbe894", + "6f8a46c31c1aceac05f0039bf6e9a36b209475ef18a584f1cb8d5e3bda48094d", + "6fb212de4b5bef6fc983bddb206fcb4978547681a9542de68b5abd7f51546fd7", + "6fd4ed214af38642cc6bb474abd126841f82ca4516af7d676fc525f3107eb4de", + "7005b0152b7e62e70a4ec3548246bc1b1d5729feb4958c2fd3688c137f69893a", + "70068c35f76f137bcbffaf69bfcf8103099b381ee31719b508515abb965fd586", + "701bec823b7e553bae03ac497ebec8a40592fea30cca8d91cf9efad75e6ad7ef", + "7064f9086f3605fd49bff512a1172dfbd0ceaf58a88410d163a8cd1a0ad34f94", + "7072448eee55c0872253c8a40a5bfdc2c36b8f7b1be17bebe825468540882db9", + "707966f3f0d9b4a67d3379ecf693b83b2a1e1c217358494af3504b95c137fe17", + "709835aa30c7d0a66f14b43475d8db201bf2bd39510d9ebdfad8848b84fc42cf", + "70ab03aa506b299df87e64cd28986c66d4a8dd1afa44066323e9cc1d9f89a6bb", + "70bcf0249de578d517dcd2d9ce720351b3aebc2b3408f92ffc31c3f264797cb1", + "70e8eb1a392cd7468a49fe73f3aaa99481ab7019d354ff555dbfa1901298e9d7", + "70fe15efe7ac3eafb95ce5f3f89daeaf76a65535fb98d50b60a029e4484de29f", + "71024b56d0f6a22d5fe9e7acd8ee1e7ec9a651721f305a2c7f3e4d5849781f5c", + "710f89d19eea86719bec11c771e48def42e5fae229de5f1f094639beb3c85efd", + "713f861df41f12f45b212350544849e022cd8665397f31300942f1ef32f1bc18", + "716c7dcc84d0813272c889fa45fdc626ef28b96df90f083f0f95e51b5909b7c3", + "7170b2876d55cff7bbbadde1cf60918fa82c96b47a419d0dddfe68656bb15f7e", + "71ccfa1efe119f4b3400c25f08c0cb253d6e84a3034595291688d0390362e7ef", + "71d64e8f87b796850cfde842ddfcc5397530526668b4758e95e3ff352ffb808a", + "7208d392e8466c55bd4db2b793267d4f537812981de57808548a24d575c00802", + "7241dc8b47fc985afcdf062800aaa0e4b3288f68ae23e6420d4174936ce1ffc5", + "725960adbfacfff83b82b7df54c856df7d95600c14505b43941eb4a624f2d833", + "7261079205d5dfa4e29d586c9b6d03ca1ee3e2225ef85043a4a4eaf8583df9cc", + "728722ded8b3e23f89f3fe1884bdbf69cb04c6f5b843bb307b4a62e2a2964762", + "729dfd823bb3d1363bbbe0f6beea353b4b9cdf5e47c3b6af17ffc74b854a2d88", + "72aabf4eaf51267a223ceecc011f0e24ba882cb2833bfb09f99d7000e4ce2935", + "72d78fd5e3c79370db8423179341cd9bba72ff6bd98a54b2a915a698196bef03", + "72e1b778da03bbf91d47c4c0fdb930d7ad38f76cf0490408e8b4df6eedac1ea6", + "730722ba5721418ddd857592947efd2f0e6f8166eaa20bf9cbab54a8f4b0a3cc", + "73546fed85b1e74894df9b8f32aa7bd56c478286997a8c92b08c67af0e5335f8", + "7355830e37a3f3593f8f485125738b6e7caaf9813042a1aa65a78148f98393e6", + "739d4dc3543c4be6ec96158cd1f81024d14489153b712afb0a7baa9a136af44c", + "73e3407bdb5f83b252fcf7c2ccd03c3a15690c91e77ab9a390e9d6e133770587", + "73f1a153831e1cab13cc9cb150325007b4b7ed7d147a5aef39665059d0d87327", + "73fb5ecd247e8ee18252483813c02d62106af8bc146cd4e3bd3a321f20cc866a", + "7461c878ab37114c8fa861bede453b9f180844dd0024a1031f8d0d5d4350be43", + "7485d46799828ad55580d2370456706f56e6b531b241d1c09d6e60fe61680738", + "7493553d2dbed8c03679691050369d2cbd955cfeb2e09469b115893f92f81f1e", + "74b8a267ac18fcae42bfc59b17bbfc54cb50efbc4983b4fc0de89fdbec45f26b", + "750b82983cca089581424966a9da92809d4739ff8a84fc81448b6c2144f0c151", + "750f7f9b8560ab8aca11e03d8825159f4a02c97263c4b164cc6b6cf8c627a367", + "7512074a62a9ea08ed848461d4b771a087bf6d0806d9d8e9de9f109de0b62713", + "754eeedfe7b3dcb1fb680ffd21ca550bea5bc1eed3c387b5889d69c739bf82dc", + "7550a0b2274ac6eadda85f132246d97cfdd1fbc17a8c60247c93e8a7d6d42352", + "7552868610e7a0492a47d7bb17f915b243f75b08fafc0ec2122db8ca25d02ba1", + "7574a2bed3690cf7b377627edac03b504ff654934a0e4df77254cab0678b3da3", + "758fbeca0e118842f28a52b4c62c1fbb9828f2e0a7c78c41db50bdc603bb82db", + "75a4c50f6b7b5f9416c27fabfc24bfce47505c2507c9e5c40689f1cf8edbe796", + "75b0312a7fe659ec2619fc0a00188c1c25648195d6e7d9a6f7064d3accd1b897", + "75b3017f16c079d88f1448c1f4b9665aeff8c770ee0174a59a6dac4af0945054", + "75c9c4a1f7af09bd21062dfb98f95b17143fb282d7ecc42f082167e36209e5b0", + "75dae78bf5c4fcc772bd1ae3d1e61f73ca0fd329342829d93701ecdb05cbe776", + "75f25b711155c22abd881f62cbdbe0bfe8236697ac644ba420b88a50799d6682", + "75f96d7235913b25c9433b1a0ec68104bc222192d7a8813010730f2a0659b3c6", + "7622d99087890010662d5c0ded17c8d5a71587f4e39fa1d389c45a24b04cbed9", + "762512c3c71425a88180aeb09010834265222f4af4988ca83e9d4b453058e003", + "763be34aaf034de4d21129b44404ebdc6f95c5e5aa07140a0215b3a831c0c975", + "7658c48e7502d6d1d0b8fbffe97f4df627e7974f75cdbb1b9ab6a31495a94ea2", + "765a03507d35544d425b3d04d0b5f8af819630c18ec83638bd7ca85fab92f351", + "765ea782be299b543c8481425b1da923aaa737acf2b736c47584089744724891", + "766672d016c0c6eb36997037b67867ab2c694726af1d88c375228194dfb90a87", + "767a47a19834085dd8e10c9d3d66ac5964793f8a9513a35b2dde4d1429df7a48", + "76a179bb27b3e317f8d4e66765cf407ff74fe9839e80bbd0d6de766e8bc90823", + "76d19a21140ea79bc1b758ce772ade48566e5a967ba98fc6319541f9e4cab3c5", + "7703f063aec2da8a8985f6e719c2a68d00f1ac73cb75800e295741fa4d40ea37", + "770e2608d147738abd66ba9367f4d31123a567da4e4237c08a58d1b357f6ba26", + "774be2ef21c84a0f0b67221f61609824f9e43ae746ceaa6ffdd72ddd8da88c49", + "774ec128960025ac079c5cc7860ba8866b328c0a910e4ee97d9cbc3948293a74", + "7771b23e1a872781d3c859508c8f81f5f0145aadcfd51d4937479e8e767e7326", + "777e1e2ad048ec24134ae9e9a2cc09bdb9e65db4a33aaf26a5571378fb576c86", + "779ed248c556b5bf3b3c773212e4390243e14167ee6adacbdc126e00057141df", + "77d7f1daec219ca893d2724e410ba5179573dffd22ac21b1ab2a67fc65159508", + "77f2ffd9c903d1cb59f3bb79ddb89b412ca1ee3529d555074ed4cd2e575d330f", + "77f65063c77913e76134b4c37ff835f373526c410e5b367c54fe4d8d613e473a", + "781ac47130a2dc741d2bec0037bbbf1faff9e21ed424934baacace67e827e599", + "7834f835213ed6839d9b2abd93cf8ac01ebe5bdca61c9855dcdb515765e61e2e", + "7841c519c3dac274e9c8c618fa6201314f47dd892aa879e0f41619ece8f58376", + "7847d17d7e5ba94d471bb710e54d9e653488c9753dd1d12661e44c04cb4793e7", + "78c796b942fe136b1056c697af7179df8ab8dd78e2c2bb385d9774710d9816e6", + "78c9fec65bf1ce348f0f3a33687b29f95d7056f34c580a0e4292702501694c01", + "78d4919ce6c9044423da92ac281f463a4de59ff03f9a54382fef3465b64f2aa1", + "78e0d7e8b4413c0759ce0b6c2bbb6a132b7ff8aed71e0ec46ca0d3669080c781", + "78f5bfe6f58ca23b2d85c54acea293e7e19ba60b63768934d1b65d1b44b31364", + "7910445c446ded88f3ae81db2004fe916506fadf83017f1d6c0481764fbc726d", + "791e0feef999d39967656eb542ccd5a5172cea4650d58f23bcb8fa8622551d0d", + "79493b75a4fdce4eaaf946864585719108489ee5f6dbb8440a70987d7ab3a14c", + "794f1d793b6807ddb2da2933851557a4ff3326c21d6a717897a475e7c8305e11", + "79665c355e938b7c9a5085d02e1f4454bbd1fa42d99ff76d56a0d37eede5505a", + "796cc42123f9a58eaa22cb3b3388e40160491dca862fb0a2aa17db03b45174be", + "79851d82ad4f83e06d475cd976b34d79a0884bf77624a6443337c585f5e7d88d", + "798ab8eda49c99f52524fab0e6c29e2fcea9c25bb7eb29e55d24f8bd928f5885", + "7997158c039d6cb8a0206f33a7fac43f12b6c05c50ab9523c5d319a07c9f47a4", + "79a8920832ba7837cd6d4b63760c7db8963bc9c4d5cc84b02d1656de6861cd11", + "79e246581d6965e0b6397f31780e76d0e681e72a94a031bd976890872e8c682f", + "79f7f98c4c91e9507352044f016d9373175a6410538d1d74765f5013de13efe9", + "79fe72616f57da92fbc0c265358909235cd86d5107d0e7d613adf8a67a868b91", + "7a0e515bd7dd1fc037d2793b5bb09d987967b38c7b0cdbaf484bdeebb6fffdc1", + "7a0e65a743023d8e6f5162be55768a3b850f64e76af441e3a5b99bf0528ec159", + "7a111a1887f6e3391d63b0141a7175207f02679e816cea6845b3486565b6cb17", + "7a1a59f3aa4d6387e5a5a2d61a4f4409428633616e29fa29d50ea0fc53bcd339", + "7a29256a151c5dfa124e5c339bf9e5bb74c2125c128b04d6fe46abf2a758f712", + "7a5b2cda69206c48b9e8d535cd1e96ea9cb9185fab4ce335a2956bcc05b0b115", + "7a70990914d64abb8b318d3bb2a06651e1832f21937c00651a6df6f061cd4842", + "7afef4f1d28f68d33ac3ffb659441e0102096da753d5c20969c536ea0dfed7e5", + "7b2cf8dd9d56b7ca7a7bfd1fa949e72969257010fc1d480bd54e043796eb182d", + "7b421ee486724cf4f7e562d916b247b7648a152d31428856c35c2c2db4813bd2", + "7b689b58e64f9ac6bee1eb14628884451cee03f41fdf2178916537c2b56133a0", + "7b6c5f005c475b264b2a7bc64a14240082a0ca96eaa41b4f9d59a0ed65e7d381", + "7b92e26c917b401a6baaa8cecdf723c0cd3d59666c778dca685bbc12ab9c6f5e", + "7bb5aab6218927cd29574273c5cdd63d39849a57c0ec9b59340302dac1ec5c5f", + "7bc0ebc0825f60d79a197e4d27689643d65798e598057d0ae33f72cd39a8d1ea", + "7be40af82f79492703783517086251c2be0511bcf1dd2b1659e4d627055acad2", + "7bf300a78229f582b67dce564df3158f02c03a7d17064a78d4b041ce41ffc3a4", + "7c13771c61816ecda59ab2026f2d2a85c882cfba9b967d5d41c31e338602eca1", + "7c249a595ca1f06717edfa3b35b2b487e2c2a47c29f75e4d52f4ac7625e24b41", + "7c5f4b2df36cdc780578afdd9ddf6d84a6cdf997977d0351ac0ff19bb403551c", + "7c671c670ce130f0d602bd6d0573ba85c6e754639d122ac4993bcea07d5e1b47", + "7c7f4330d058eff280e31afa750d2679d4e686f89f57a402ae5851cb33eaef95", + "7c809ee513fae51056cf753009b516a7d1c46af0c8b9d53761cde8467416e843", + "7c89620ab5745c5c096fed9a789fb4ef9a7038f3a03ac2a6aa033a960cea0759", + "7c90ac3e8b713181be6385e25e948a2bfe21ed28b0c882153787eb8be239c2d0", + "7c9aa9f7348291d8af6529b509632b76f898e46070c8b8cc5a74209f1f8e384c", + "7cf13a99136bc97bc57c8cbd7faf6c461f688084df1ec3a9d65f79c11c600491", + "7d4aeaf8a6c226ade1ca11ee3ae0217ebbe29c768e83b525129fd74895c0830a", + "7d653fa63d81af93ef7779035399a4e53411ec8a7f651256d7f721c1da327da5", + "7d6d2253f232f8dd1a23a77557f48814e176bb670dced172ff3ff7270115e2b4", + "7d83af9e9367d2ff00fe5848cfdb4ba392479eb3f1fe223f6828c6a735ce24e1", + "7d83e4f7e37ec9b9962eaf3c010021be13d664653d07999ca6c6a81e4fa3c427", + "7d9c7b07ccc1daa04385683f46ed47f8ff248fceaf24e67db501c065e8cb21b8", + "7da2911323411aaa29a9a089c7b549d57f54086f1831e8d11e614fb17e786326", + "7db20772098831748a7981a5ced53fd95f5e03e737349f7ac3900188cdd22f5a", + "7dd2da493ec5174b6763f9a0162334bf09572691c2d8908df6fa21d18cc67fa4", + "7de9b973552d85e3f6badbb2ee3721a640a18b97840ad0259039d6e6df85ff05", + "7dedf5136a89643bd0641830635e4f5a6ae5d68d3ecd6e4abc658a39da276f96", + "7df266244513d7042584a4a8e1b07a928fdcfd2eb2568be855f438b14502c66f", + "7e256ccc54272dfadec975e2192dcc8b899784a82c8eaafafc2320af6270edc1", + "7e4083c02abf3d1745e121c2404d91b27f2b0b972ee71c98dfc2fabc838eab8c", + "7e53515a2846c4ac3e3005a0d705ff1bc47fd08a4186c5bf3a5d4d342b7b76f3", + "7e648858b5dc20e69179df0316b44b36ba017448ff6bec6310da2a37e48093fd", + "7e6a0e320a76b2d7c4571302adb31af0cfc155f957166d39d44d4ad842d38c82", + "7e7078f9ba22a164fc61c762fb5db6d955ca1ca86a3b50b8961e9e5a3627e79e", + "7e80a3b28e467688135d12d509131fde13a4f17f81614a7b1f9d19379793e765", + "7ec7e941978523b6bb23d2fd3d55d6fe33879c6aebbd84fbb185117091c12b72", + "7ee3852676020e282bda9cef1c1045e25441647116ab309a214ec1f5f6f1610f", + "7eeaba0caf143763754abd5a641189db220b4f37abce91f1c66ea992dce96336", + "7eebdb6297c64378e8304231ece5291bd19ec690a37b2b58c90880cf89ad3bd8", + "7f175640c764b7df9f0cbc95bae421d5f9297fcf62160d65949b5b100557f9eb", + "7f792c5911a60663e0e9fa68a8f6be943a925cee621084cf3956ca175b27749e", + "7f823f60346810266ccd7ba91ea8984ca742aa3baf849349c8e0475b749dd68e", + "7f87c627f7f7412fee152b6c47e6eb8b6a13bd38fb9397d50a10f0be461e214f", + "7f8834cbc4e365c22d0e3918efe4d32714772b2c52fc2f836ba60f52662caeb8", + "7f98e2cfd1b02cbf7445f2f3c3fbe301e820dd82abfe4177d51c3285a507f3b2", + "7f9d4060748a069f6786af1e4860c6880d39e038135439b0b2458ad5d3c595a9", + "7f9df7d75ebce46939549aeb0324983263bb1d809264f4903d434452c7026514", + "7fb3d3b49f404b6314e29baa782768af91c6415ebd255ea3eded9dc3f584abad", + "7fbcaaf24c8235d541a5609f45fd7144db69a5defadf4bf6b518f9ef5f163e5e", + "7fcba24680c7ad95062aeb55090e10028546a5fd4f4a697d538593deeadabef7", + "7fd23e3b2cb3db7f81e7730037a1d3e53fc9774948a8b85613600590ac7d0f92", + "7fe0b89997665c24edc4173f17d379fb0241804761719324b77a3ad19cb134dd", + "7fec9ff3d09a0bcbe3db33b8066c3bacd1d09326c89e6e453e8c42ea49fda27e", + "7ff731d055995f2c7355f7a59174ff09d3ad70234c6ee73b5b29c2a0622d13df", + "8014b101a0e5de562df78d8629d92a6b367ecfce7c3f7eb968ca1178dcc94883", + "803fca0d27e485372ff299d0138f27c97487af7708745803e2eb557e40c71172", + "806f014e6df16a1654bf06eaf08ed3b60c299fc7b5d1c37d433b3a0ecaa81655", + "8096a2f2dcbbff46a37e430b9e9b6894d5f6b5c01c93a8187900dd7909ea0126", + "80b0d0b784604d32fdce1b467680d1630259424aaa829573fa50c6444cc0b1de", + "80f52225f3eeef206f805d640ed82231bf7c01b9f7cfee7ed524b1b299ce1197", + "80f9218c5e9118528da5dfd6780556e8443856b35e876c1685d2d25887b92367", + "810aa4c9218c0639ea1d9d7fd3811415d7444dc97f831a0dcaf75ef802242f74", + "812c2ac233a568a4da77e7b7487e4472579ac9bbd8ba095177caf7f1e2406b9a", + "814f2af21544b1b03d045f29be5b5d6defb02be970b8a4d85036944c3a408843", + "819ca88d90bf6c86b097fe09a8704530168f1b8931116e9e55affbaafd0ace33", + "81acd0d4f293b61cb77a0cfa090543beac2ab5792ca1e88447b16374fd1dc4cd", + "81d76fcd53d510136dca6c06bdd90619de32a48b1caa2da941bdcb661bf8af8b", + "81fe898b8769901447fb50c71d7d0627025ad9f7d533d59cc9f7d17eaaeefa60", + "82489daa4c05ab621452941f901dd0a684e640632f76836541fa3f45bdc27782", + "825ff15f0782c3ed5bdbb6412e3bb77aacf35c880840c8273142230c0df2c2c6", + "827fcbbece238ab1a8bfc63dc4ae4a456a40d215b9bb3f7c9383d49c1460d217", + "82826c41b67c800350648444e150c3a009db7301fb4c693f84f848977dd3e9fc", + "82d78f23b734d512517876ace50d7e7e19d9dcad4db80774ed3609ed8d3e49e6", + "83171042fc2b4dfc78224fb2b00fa8f1d00a038680000ea14462f14c604eba46", + "833133f91df4b418393445068bc2e9e93a065c463c8c8a25d3521e84f6414a93", + "834597c90083fb02ca00b90a38092f814fdb011897ed0cb4ee5c25bedb69d940", + "83aa7ee537812e09eb16fef58bf15fbc5efb929c298c5d5e1be5682bff8efd68", + "83b61aaa99d449b594ef00dfaa7d958c36d90dab01db14b2e04ee0db9c365445", + "83cbe1bf2b5a688a257ca8d424d4384d53a97da158822b9f086511e145839c23", + "83f7029e77724c8160a53346097015aeacaaaabb5e4e8e3930f21c82663783be", + "842402ca8d019c79cc2980fc26018ecdc2d715015f164225eb0112cf8b619587", + "84363142eb13b6d6a8c1ca2c282aeac21af2d69006b75733eea7b1ade7666d27", + "8463f76ac8b1f2ca1f2d0968a0dd799a8bb3d66f09eecc92341b45b3838840a5", + "8472c08437e9d065a8dfe76e302935c88138b65679974121599273edf2cd8d41", + "847cc4a4ff20a7cdb636264448eb18745c302f2ab0a10f1f61e1345cbe0bd5c2", + "847ef669e72c32f996d6a8245081aed7da92401654351337adeceac90899c4ab", + "848273eda2ed1190523b8ae401df6071e574af543b7d4237f4819ea8e795fb91", + "84bf01b039a60611ae8cb61ae6361a304f17ba7063425fa40f9836d7e76fd51c", + "84d4126224dd508c633f787902a15d111b6faca854ad91c470b008b84c466704", + "84d6d9ce5417edfffb97ce2f1947dd3b28381572a4bf02b041311b8a2ae6d3de", + "85026b06797489a856e2a5c1101cc0dcf0eaaa1b44ecc856f7df064abaed8e4e", + "8504c3f425c74ff723cb3a7f5c7e8df33f66e1eaa213cd2e71dca4f35491e68a", + "852359a7965660c3aa1296b56e4e98ecf4eb217a50e89ef12718539d31e0dfad", + "85374924744d63a016f40be7fbfbfcc00ef1baa374f4e9c31f35a039e4916d9b", + "856321fc3f1b3564e2631eb636aa5ece8bff0a96516788fb7ae44a68de39d799", + "856a22bd189c9505240da1131d3d09c44441396b58ee0031f0e69abecfbe5784", + "8571d2f43d6f338aa2691f71db7d7e10ead6c44764d7acfb62593f92afe32be0", + "85c19a7874cff0defb3daa6ed2411c6a300d40630127db54c7b1d72ab218ec8f", + "85d06bc43fa85e3a54b821385cf84b32ac25b95ae86a0d6448ec3a097436bae1", + "8602cd81454b46b26250823f90f178accf605c3cde02e5f940e4dd2ac634f131", + "863cb5c4d023cbdad947ac7cab27045fb066171a906913f4b8ff04ab9458217f", + "864c42307df55fcbe1f520b716e132e19f1ee4c0e4581b8fb51d1c8e0a2be428", + "86645cf08cbfdbba0c35ec11fccf41629fcc698746c542a776d96f1809aeafef", + "869892d5fe6eb6d2dbd806e2aa54b95f3e4087c76fed7f496ad8da9151917325", + "86c2413964f42e8db6d102682b3527c1d54fd1f36fcd167c75d1783ec9b2c682", + "86d5aa48fabe29b7086c8c48b33733fb5086507c03ce152f3dcff551fe403cfb", + "87251660f1c93538bd393e443bdea252c12178ad7b1fe3717c4cb456da1d5633", + "8726a1f04292bdbc4d137c58d3322127e3c73f977afe4f13da68584a409fab08", + "874aff72a30aa094c623b419ad6367bc9469095f544350d65af2c75f8fd78c60", + "8772efce2dcbd728dc3e6be593fcef89602e115e612aa50ede25ec3edab73f66", + "87a7443e2f33778a9f446b2e6bc81415c501381ad2f80caecd6cc5dee29a9e07", + "87aa334d871a911f774f07e7097a23960868f396ec4ef24f2d2a4d40a9d09150", + "87c60d890ad8c5c2a38be11978989d57c185a25969e84bcd8ba976a7f25070b4", + "87d1b82dd094c4acb9ad58a0ffbea7a5563a9936cd78f6e7fee778ebdcfcbe70", + "87dab3f34020532d6b44f52e4b7a035fd33bc5e4bab4c286faa547834a63b77d", + "87e20d968cd2fe30891243f403b3252a266e8db5fc90396caca386e302d0b8b0", + "87f4c982994ca15c1fc3d4d333ccf487b6548242feaab2b0dfdafda419bd0067", + "880779342113d7f59f2ac7f35283e3fa21c7f5635b19f6b2cfffd6bf6c9af087", + "881aa0facead55798fc83556e0e59977b06d1ec3cf3c254d86462c2dfce80736", + "884553201eb494d42a1993226344515c35c0d161c21d4a665a4239cfd45e005f", + "886ecdf27bbbdf53c2139328d18c174afcec3341a5aa4bd5ddc67e42a87c00c0", + "88a8b7550d896043b38333c24505d28737df4e73ec5fbc816795af28d9295048", + "88a9bdd66fc2344f8aff0b0d71c502b15dcc7bbe157324d6e40dd3058cc364e6", + "88b630562d5d85aa3ff2a6669b4bbfe5192067fc50d1cbcd58374f5d9d48c6f2", + "88ba13465bce1bb05049f99681cd787431e3a53903f0d3f51cfae12f95db0902", + "88c5f45e6084c9eb35b4377db489699d8f8fa81ced9a5c74341ae8f5217a0bdc", + "8918af679922da6f209ec9c23cf601bcf108176e52ff7fe2fcc20eb375fac7af", + "893c0f99b45c6e1ac223a4b470273b1a89431fb4ab140984aef93375e78e5e77", + "8981937b36eb34f0d585e60606ce56e8e67173cc3f04a22f46618d9f6e682e25", + "89875aece432ca420311ff4a23c57fd5b4c505e3c3b559165275845ae5e0e76b", + "89aae336d8ab2ca0ffe06a15efc2abde66c39f2d3f40fce5e85f111606b5e2a8", + "89b8caf01caefe1f5e873dee97904df01c28959a164af479fed1407c92add483", + "89c455c362afed63936ab400f5655351e685606805fec2d47dc36c40e15b971e", + "89c4b6d6bf0f071af66ee795ff42407c4fe150cb8e301997b721860501ab9a7a", + "89c84ad21dffffbd89fd8391e4a68536cd65177f9024c0fbb33ed0aa1988e9a9", + "89dc9547f563493b4c062ca9c1b56e277c0b973a53b4e11dec83468a4af1aab5", + "89e82b72fe11cbdc6c4685419428e65f1b96655d4eafa2343e891fd1f6429ea2", + "8a1f12fc3045a1ae803dd949931b6c477310b8636ed7c23b39af03b3dda64076", + "8a6baf027cbf0e268695248bd5dad15399913f37908c4a678622ca58b25ca568", + "8ae1576dcbfb6e37484071b60abefbee4a2d4183f8e8b21abcda98d615be1d51", + "8af1a4cecf6072ba4dfe2e726448b9d79c9350f80a171fc963e58b7fffcb0fd5", + "8af5596c49e5933241edd73e1c0349b84b28d9a9ef0bdb0b3f08dadcb3592d24", + "8afe556c423b4d558efaf132ca16b5fcd8ec07eab04614e72fcb67a700eedfe8", + "8b142c5aeb5e5c2ba168580db094b7b6da729837abe6199de738f104555e63df", + "8b181304e3a183cff73e26e12f68004c48cfbc7f66edb371622561c9ffb6d175", + "8b7f3e0da8bbb4bd381dd5faf22f3506c22b01f117a180f56f2060a15f7f3128", + "8ba5cc41b621f68b1da7a950361afac2ebd406bb12f8f21e1571be2680cea395", + "8bb3960b9aa49d3c7daf93c0b27a2e82677fe70954a5b1b8b40ff51e01f7b651", + "8bb4e371cb956240c12c6824fc1c42f113160973b79c22572e5140a5a29d4542", + "8bb8b135de8b7901b1b456ea52976b19d15090a7de2bc5afc5cf0e408102dbd6", + "8bbdea659991c41fc41e5d01605860cd53b22535b761954b847e34eb452c1bb0", + "8bcf7f0413754b850c9b24fa87068f48e73c559327425e1bfc82cae6fc4d6b89", + "8bd8787b5b3659b6315c160145c30b3aa5f799d10b61624b530ff90396a5b7ef", + "8c26ad6b9984963d835deae72d8b24856d38fef889bb05c10c85b3fd7a25cd6a", + "8c379436af1581c0c376678bb99b148d4756e39e9446cd804858c12009b2d373", + "8c4099078c2c0f6c69528c976e9308f5104108e2c6cfe95c3d2399672f5279b3", + "8c45266d075cdcfe5d59cd03fb88bc25a7a6ff511ef0b0cf9ae47fe5363b7d53", + "8c4cd3870bc9dc20ba127862a0ab3aac857ed1a4d0e2340dc7937eda0d115c21", + "8c4feea60b5188293e370c26dd06528cf6bdfb30518b3869007d3f80bdc3ef13", + "8c7148ff82aafe024eba8a2ebb64d6fd836fd75cbaac428abba1d92627fb07e0", + "8c7bdd8a6c39e9ac5222fb2b16861c1b24367708dc596f1bcc76c2c7e9c2b578", + "8d432dfd008cd8ced880729bfc45f3d548bc54922e5e6d4bacf680bcdc0a480f", + "8d5d02f66be9555b666419f16215b79e6929aa14a678c08d09d2480bc2a82b6d", + "8d717570e2a44975ba3d53e6575d4a295717dcf2564d14d62a2089a372a8fe34", + "8d72bf24d3835df975bd47aace23e91fe60bdb5cb0ba997ca78e52fbad343414", + "8d78008b2455a61d192635e692cf968000425c1b87e8ff46fd02b3816dcfae23", + "8d878504470b79be572a45051abba37a9ece730557e77120508eeb24db203851", + "8d96dec83d0411426e3245574e8c35f5b73c8df79c1390b684cae3a7950d76ae", + "8dc3568da85a0ea8b8b2d7c0bdf1cf458129cff09d04cbcb860575bd5d82ed95", + "8dc80b3bac6de50377d73b41145b48bb3ca6d23e99fc278cc28c1dba42bc1c5c", + "8ddd2ff6afd3e415f3a1316579c35833de765f667b9501e7e42da54f9eb4c7e5", + "8ecf4a9b224c936daa0053ab30a1858c67e74b17f2c82725389198ba26733706", + "8efc955a5a8e460ec1b481cef6a4a6f51552e6e1e49355f9feb912a0650757a4", + "8f030cf10304c8a821ff762ece320e3489870be3b40569b6663cbfafbe45d90d", + "8f03736ef3ed40b3dd7126f1c5b63059ec0a5b2ae71f67fb7d97de2fc16d2cf5", + "8f145b6dc0b3004631db47f710f7ab74a258adabbec957e85f67f15a99e5ef7e", + "8f4cb1bd07998f4bcc079c05285b81793fed07c094cb52e506b5fe2d654575d5", + "8f529f8b797d5cddfb9dbd137631316ec57cc23c6b3cd5249562e7fc8d8d80bd", + "8f91ee16ae905b7524533a6fcf40137005c080e4608becccd2c865d1e0933420", + "8f9d44973e1d625174356a6be2636e7f56339aa695ddc592dc65cd79f2355954", + "8fbead8e6f6dad8d2bd48456546b24c66bb9735ca70566daafb9118cdc7c83b5", + "8fdd16b6d541b7b81911d14d7f6430046b8069fda31d5c512e78f371f7df215f", + "8ff40cf299ee3337cd7fba77cb92fad05cbf3430ba4891c9545793a70b3a0384", + "9041d1ecb1c173f04808b57a9819a53b3156c1fe87df224597680c170297208a", + "9097a466fefe8548f577da17fd853422773485880e2d7c6085e8950694a316b1", + "90d310d3cc0848220d1011df2ecb0ef6045971926c923b6fa9ac2e8fbf8b2b50", + "90d7b9d3d86e904e19237a6f574643863bbc9146092f09a1b63fb18bcde427ff", + "90dbc96061612277e1e925b9beac596373152e74125ae7a9213bd1a1f8585e08", + "90ee3c873af25ebd77ed889cf4ef6410bb4b9eb00b32572f4651e93bd3b4743f", + "90fbcad54d4abeb3e68dbd7c4b923eabb630fc065a70246b8796627b3e3a1e96", + "911b8543dac7382e9e25b4dee5ce3774927cf78f57cb05eabdf33b3cdfa41e65", + "914f8bfa81a2c9fe0b168c645bbe45399099ace3ca439353229216f9a1b99587", + "91518c76c3bd6245cf1b051b958d8270e9d81f0373bfc56467dc896de7fa856d", + "91aeab64cf94aa7690f78bb5bb2edf91d64e5d8a037094b19a9c3e5feb183a60", + "91cabeda5e16e572b89e53f6ba3988fd8416fa7b03653f06969c126752dbed64", + "91d65e83e4fa2e40a29660e8697d3dedb1894b25601c3bb80df559389a9ca07c", + "927b077b9994f92cc17262f57066e2731c19fdf59f1ad23e9a187ad8a8e6a21e", + "92b4ffc72368d645dc6a5d181ec6cccaa6b2d5733d289a6fbd0d49da064ad778", + "92bacb8e4cb433020247552fb9c6cabd6455018488f8992ac6c202820aebe864", + "92c9c9caec17b174b853604267805f1caaaced45888eeb894cb5f07c3da60bf9", + "92d6eb7919388ebad69f0cdc179024407f2b0f10ade265a6bb82ceb3dd533319", + "92e08317b5264a5c75af6bd32f8f9d19d35dc2ae77a8fcebaee74b6349e2053d", + "9317c21bf5f13c7615b404074d4bdcd514250758aa7dec348b461e3ba7de2ea1", + "93717b7c08182de46766bedabbdd152710832ce74aa1e15d30d2d66748a74660", + "93d4e4a8d37483366afce2294a63b8e94031c2eb2595f06809f3fa154b8973bf", + "93da984a0282e86731dfcd961a8ce43b862a14c42d5b5d1e8874ffdd200d30d5", + "93f71412244e33e67e82e17efea5d58a9ecead2928ca36ac1e8cef2457f09296", + "93f7370ce73039faa1adee0d0cab9177772da8cb361d27ac1254cfdf74dfdd57", + "9433d8a34075bb592cf9c36bb1be21da08bdec9f9ef0718a0b61943b321b2098", + "945c821f819b42bed83f5450d180a86c23e6ce00695f7a18a105e2980b2ac2a2", + "948738b6923f5c6f6af94eaf47ada5f24436125e7b2d7532f2cb9b3f8a1ac64d", + "948b1fb3a88a247eacfa7863b50f8933ec9f89a59ce7cc3a17589ca825f5ad76", + "94a3567ffc300b6b0f7aa8e3de115ea0f5ada4d64e2184d2fc236c39056d500f", + "94ca3253fac86271dc0564b9de6cbeff335102bf817c744eb6d6434596879179", + "950c886241c044f7515eb00ccc25b1e9603899a3dfb4abf94727a1476bf37b05", + "9540e5430f9fac88e79b1e21bc8252affe0dc5893218218b2ff1839999223005", + "95473d6a2079786547cc70ab7f130c85e58839a865eebbcc11b0b75730e53b25", + "955f30c3eaf38525cd9d5503c4b1bea7b23c299cd6c8a6eaa6a6caa3748b24cb", + "95b77329d4972e58cce72f2ea588d614e3cf39cb424cd628e90e58fa2868a4f0", + "9620c1973d15d900e0cdfc235abab14254b5d29f240b8382ae9425fd47d18dcb", + "96290f319d1f2eb52a9f427d1df9bc5e5c149094606c64f51a172bb8111e1d96", + "963991dc96009d33a16333cd8dfd1995a7d274957508b25bb9f3c00683af0a85", + "9648696e71b0f84e436a6dd018c574b57d85ef948f6d5cc577a4964b9da64f68", + "9657dcc5c71ffa5ec11af1b35558bacbc681beefaf10373f74dfae9c13687f6f", + "96848b6b4616088854f302f4c39f7879232f208212d4816b6390ac11fb0eac1b", + "96af2b09a6a25be1f7f654be30ff50d473192cce28c69a5e89a1016ef5371093", + "96deb6e25f941a15a80a373e2ecb92809fa08772ea70db16466a16debe9842fd", + "96eb4c60b1ed6d1cdbb6d0a6e9df4e6d3072c037f5bf8fbb08d6f49fc0e96791", + "970c5fae48051e2e51273d468c630a6bb8abd4abf92cdc1e040ab2e5fe641145", + "97104c8814d8d54be64a2effe279e2de76ece7d9946d17f2bc721fc9ead045fd", + "97af82b0ce5128d252a339103965d1c99eca7a42ebbc19da35325e2cd716023e", + "97e800caad83fd0ea5bfc639fd4764cfe8a7a0fbb1268176e05015bdcff4ba19", + "9807f31242666c19189d7cd3d7007e6c617b49dabb548b216832e72d253341c9", + "980a7b92dfc3cb82bfde2b7660855cdff3a4960d863818224580dccbcb82a403", + "9811203c2c5bfc2d0a16b56c0aca2650534579d776a9cf0d2a949816af975180", + "981c07276056bc24a9bc32b733dedf96f088faf23bae6f053a5381b357f4a639", + "983888a1b90bcbf0f9188f2c89e0871aeef8a39bebfeac066c11418959c43089", + "98701e77e0b6c6887b96faf4f415863fe540bf5ea720db6c1df7da2bd60475f5", + "987defec81908223dae92da9b57746a6942845b340ba3f79cf8538dd83eda1fd", + "98889046817a4b80a4f39a5cb29ee9aacda0898af6ea7c02b82cbf7c1a75cf85", + "98b46bca82a90ea11dbd8c72a45f3697ba61059d391cb02b2cfbd98331ed4db1", + "98c28626cf398fb8e30aaf97dcc9b33e1eda33d6581088f76fdc2c5cbec9c375", + "98d35a6606cf800ca8622e8df0beb098445ed83ac11f1a14eebf21dffcf368d5", + "98ffdad2141e58ba3f1d928a7521c82d40aea01042acccab30f527e31d6d0c9f", + "996f5a33720e295a48e74c5316d54bbee57d280916362ca8f77e5d519db411fd", + "99984f7515c7d2a04b908a483c70a7cb0f2ca25605858592104a41e709665b30", + "99b4c019bfc6e68eb07326258763d2a30e257474a0da57b9791e6531c62e9e25", + "99bb9b16e809868a3c0e8195dfca2107f4bb888cf7a569b8e62a1dc8813d7323", + "99d4f33215908f8993f4002c40e20672c3831f527217db39469b57ac7bff8861", + "99e21fb94cc1441231170fed489593e95f264396012ce6d9f1887a4c693559b2", + "99e9ecd54eee0b8c28f53c9887b0f7f6e76a07fc2bc3defce7b09032c47fd533", + "9a528a5fe792a2afe2443a7f889a837a87c2f143271855c941c898e5bcb0b671", + "9a5f1226f926f45bcea2f48d1bbd21fb46b8607b63fdd464c5d429e41685a80e", + "9aed6498f33b8d9b4030e31861866f063c68284ff77e0d52c533eb19676a51e1", + "9aef46bf4b917c607a06ea8f84a33fc09b03e46d4161aafbccb836359b1cdf20", + "9b03128ac34991380aaf809fa4f6008fca9964dca37bb9a141af5be04b2dfd19", + "9b1c963738e61cb6be8b2320ec1f5d55b632d42de6852b77a1bdd26fa590f2ec", + "9b72c6281f55aa675ab785fbecd32f58bfed0a6b427b612b0872230734a2f6f4", + "9b93401d713707902cc42358969ae2e8a3f67a7abf682e6c9c662b6a4cd02c8e", + "9baaeff6a1950b08fe74b5224b5c12994587023cca5d9ccd79f2d5b9a9ed5961", + "9bd0c6c55c6ff96e5f018636211870a2aa45bd2f53041e5d6103e05afdbcadd7", + "9bda8160ad9e098a3db0011e26abcf4ae8137edffc8c7484befbf9c861f034ef", + "9c60ce7d1e20540bf17f23636fc186c7084acb8d696a47803d3a5667bfa20589", + "9c8ab72ebb2efef086b21855d5a4ae3f5292616e5a9c876f8bec8e28da0fddcf", + "9c91b3f38a7c6d1d242212504e68ba0a67a684a0d485f15dcd357bd19b411040", + "9ca581e36ee1174533857b9d926754b77009f9080246afaa63d9748d2d066095", + "9cab822d1fe9746b3beac1f5de7e24436882ed4a39ae207f3c9a51cf5b5097f0", + "9cf21be9a626e2ec27da045de4d7f4cce775ff1474fb7b4b1d361407f15bbc18", + "9cf6e80299c6ecc47c8d79c3c5f98265011381bcbf756a12c629ace25d71e6dd", + "9d20f9cd2e12a69400d1753f325f197a34cde2feaa0a6a41cbc8d114cb24d407", + "9d4d421526e0ce02fe621aefed63966502e7a6f1aefd76f27d7c5e7b57760626", + "9d5312a67b24be3aa7949b355c9627bb9982a017b95e263fb65e40ffabbbef35", + "9d5fc103de859786dedacc04713afa5f5ec908af5a8d4b6f654e4f35332f53fd", + "9d86dcec27b7472f83a0109dc136502848b7506e05e3d2b05a983f635bc617e7", + "9d91de5aac37435138b7a0fd208877e7f22c0f1ec931edd49824d81790758ccb", + "9e090e14941f35c1426fb43e188ac85fbf1d1a199e057b80de662be34937b5bf", + "9e0b3c8611c3983453fb5d666148abeb23e5f988cd09a0e5a34f61fb059d6898", + "9e0e34f721896560c995500d5251f2fac8c4562403cfb13ec4a2abf496d8e1a9", + "9e14ec38f8398efaf34b5b15bba9e47ab2019fad693dc51b513bbe274d9d160c", + "9e22c3ed476b2c040f1d9357094d17f92f041d539117569d75326ca8959e42dd", + "9e34c2de58f482b4d4e2a962ca938c8513d8555258088d2815b729ac114a5d10", + "9e397f028c0b7df416cfdbdab53e0b8a607d2348dd807343849ccd811e6dd429", + "9e513c1e436f575aaf5bd5c7df54304d8390ae88da87efdd3c9df0dfe3b395c9", + "9e81082b39b55ed18042c2c5371a652ed32337f6383d9e714cfeeadf99dd5851", + "9ea47633f98cf0baacd4966b055be5d3e71087b274b18821abfeefa6d304550a", + "9f06acce477a36f418f3cc826c27ccb555ee6a836ddcb8452cd76a766d8d4565", + "9f23be74bb52beb0da75166d46ee912a3ee0bf66954efee53a92ff558b0603ab", + "9f73734372aefdeadfafabfcff94725115e0cf4f9b15c18b6d671c93bc45d4b6", + "9f965d1380ea525701e2cc137a76dbf657d135908edb119b5ade89d517c642c1", + "9fbbc8610042a490eccef51f5d910410f875d56ea7ec41b455ce0d7f6fb76e1e", + "9fdb058c747884260689eae9bda96e5499101e7c85685c7441d62133692bf1af", + "9fea119b522607162bd0849202a5af02fbbcfcc9630713404eb24acf7fe44770", + "a038947bb5d061ac5c76c52746b1f3a3177cac57b6990c86694207fde7c6d52e", + "a0b1daea0fa36875362462eb688f7f3fdc0d602feb9df26f1ef223d31cc2da74", + "a0cc9e51bc6b5aac329313a6901cf0320d67cdc2ade26f4db2d279ec352b42f1", + "a0ea2d1b59920fdbf1565c05bf05ea3eb49b8a893dee51b93bf7d443d947b85e", + "a1240a45d1455232215af2dab413d40b707bb262e4183f3068350332abb33b22", + "a16a139ef9ca152e81fa85305fe2b5cb805153c6bdd2ea89b48a0f247ebbe12a", + "a178707a23e01e9e0d753982ef537e8ebd27f10881be88d117ebffd125c84112", + "a1828553aaf5d9ec4efdfa0a925fb03449f6db4b13c376691b34a6f4775b439e", + "a18640480013082dd7e7a22d73e66f35cc59f1fd1b93f5ed3de1cec43413bd5f", + "a1cb72da3133be47ced5d3fcab95755ef24d418f76f4fb0b4d47e6e8ff89a470", + "a1d812357c1be25f45e129a8b8635d6a2e8d8b2eeb0248984bbd64459918b28c", + "a21d104b5dd167736bce9902303d4ba1b1f02237c9b4a5f6eff0a7439042895e", + "a23825f174104e351285c36f589dc37770f5bc835d7d0e920e09573b704e62cc", + "a24336324a1e3800247283a001ffbf0c66ba08c1bfd2194aaa88e91da4c2a16a", + "a2456c53189fb0ad5fa63355cc5c3041e80729e753cad05922be0d65b48d2fe1", + "a25f880fefbc75dece8b8cfcab410d6383a6e87ba0d63b18cacf501ef8d0f4fb", + "a271158686e7f14f011abf7ccbfa93f0b0a444e3b356c8f623aa73fe8013d82c", + "a2a7b5c40a898db6db90eca16c9f41edde6feaadb445b78161abbd523d4ee84f", + "a2b0b4b8b334464890d9a5d9d6f39b05260fab4f43ba90faab4ee969594848a2", + "a2c6b407b6b42eaf49eb9077bff37c976cec72ddf64de83b95d34b75d5e09718", + "a2ec6a6102628055650d617239939fcbe2324cf862641123946607ee1eea542b", + "a335b1d36fab56c1b1ccfd69f872f9d3b702b312e2f8d64b3b63990e2a02b32b", + "a36b7eae29896149c7696386376724ce04884f2cf9ae685a42133a31bded0fd5", + "a38fe3c04414577e28a405c41a2ff11ccd80b808dfeef1bc20e36264da1368c7", + "a42c6de83dca6e8a8483d1725cd6a3f78811ae75be16ab8afa4d9a95965f30c6", + "a440cf910bbfe95dc9a81c122775b244009e5fbea638563e5760e4e482038dcc", + "a4776777ba6678085120b1ae5cbcf2d7739848f9ebfe6cb6d7b7bee59653964c", + "a49914e2c5326aff76aa60d7f5ee3c3640d622c9e8b4c68b37c3ccf680fbd7c7", + "a4bb7e0ab08ef465f0b663f20a013ec927c25d17ee3d4111eed4044b1d1c2980", + "a4c7c3bcb82cc25073907469db3e203766cb08f4cb9e2396165243e0708db374", + "a4c80a95b8f850c2dfbf410aa2f795d233320f9714d5237e5120063e6b5ee2c0", + "a4cffc4bf7e71400877b3811d34b1038e95f6053ee7be10b8990b485c7112140", + "a4ee41ba2c8e777a9555f962d285865998c7df97062acfdb9ce6caa79aa76bb1", + "a4ef3eea01ae764762e35970478dc48efcb323c6465751ac59d5101dea270504", + "a546c967ddb57c8307ccb939b6a2be51ec7639d4678662bb377661aaa3c80b38", + "a56d066f7b27d828d979ac0852165457388eb9fb7ccf509ce55be322a79336eb", + "a5def6fa74e09ff25a2be14c68cb7685ae5b85c27241afd2204da0810134086e", + "a5df13d7ca9fd83e65a0a5795994ed24336471126925fb292b760d5e8bc7ea98", + "a67c76e319bee136016fec9557c156c4c8715121226474fb21005392a92ad059", + "a682b5600e66458787495ec446edb276970f001498e62f2f7416d7445f85c8cf", + "a690a4c2865eaa72e8d5f2985a25fb8a77c688b977fe7f670d5ec3f1e41a9285", + "a69ca4c0ce5b896dcce10da086405f8d109c21fdc48bc2f174d6da28a516739e", + "a6ce6ff75d73dd15b9755db511202f12f64f6cc578ae887d28c1b5429c60df8d", + "a709d08dbcf24d3d86edf9333fb4b2f3767c49860619eb5844abebd7e8995130", + "a70dead3d5c69f30999bd47228b9a51d8b6a03009aa88510f1862af51e3d29f5", + "a71fff06653ad8cf199d4e6f37db51a88391f423f5ff498188361c1e7b6d34ab", + "a75e488dc3e774199bc944e1bb796138ca794094f25e6cd6fcc2697ff39752e8", + "a7acb1e2a2622b4f46a3c77fecb918f935fbee00f4698ddf64ca13299e0d779c", + "a847b4f5daa6a2730017bed869d6fbb7592a82e92d83e337a9f379a7bf071aea", + "a84ba5d88853d9f39f58293080d7675c8a20280d0950ed4b34f403b07fc94294", + "a864d70f399f5f45dec5d6850ee6cac2e405af6d81e08a0167230b795711345c", + "a8b1d5189fb760726fa336137e4278c0dbdd9c6b263f24be466d4d2e04af99a0", + "a8c190caef1cd42699635f6ab8c273c11a9c540afd96e14745854499dc90747e", + "a8cc6da58014750451c1aba1cc1b8ae1eb1f439ee203d1462972e4cd1238194d", + "a8ebc50ebd06ac561c84209d42e3395d57b7a927aed1790db3217dbb6d261ab6", + "a90aeee371c90fa42d40c1b41c630ec063f748040bc5006fd9c2b88372430d46", + "a90ee73b0f090b36a3b906fc3ba254d3d921f539e4fdf8178fa2764c3c515ff7", + "a951575db1c5632e10bbf2f3012328ad5e9db87c8c3d6642165a5e59c3790092", + "a953dae70fc62946d83ceeb303e8ae3838b0b0930ee3daec62514a2859a0f5d7", + "a953eb424669375abde80755e7260407805606f07e0a2475802425430ccfa20d", + "a981bdeb4c49cc8f338c986c9c80e5b0fb4179c88f512d472dbc4d32b6ddd837", + "a9baa63d7dab29028f641775aeabe1073f5bd65eafa0395caa9dc2b473acb87b", + "a9c76a3cd3ac63866e2fa0680a392bcc7fe67d652a27ab62c2f3fe50d07e90c3", + "a9d5e8a3bb7e612f9263323961dea5f8e066e61fea3ec027af77816c7545414c", + "aa055331e79144b7c663468f03e6fe569cd36baf77fa333e08bd6a7e3d4f242f", + "aa1d01d96bd3a988acec7c407d567aee6f731c70a4f449796468e2883a1a20c2", + "aa2c77c6f377cdfb433d54423d96748858541f03da3d8943586a1913416fc9f5", + "aa5b0cf7fa87a46134f260b548a74341fcecab639ad38e324f09dfa171c272e2", + "aa5c9f8b4c37a2dc2f29ee3e60e2b49487c0561c5a698c0fac5ef89eafc02d61", + "aa64551893a506054432e9e661b5efb425769dfbd9965568d21ad1eb10dff7d3", + "aa6ce1d0818f17de327f41b9a0c29e1cb143e49f5a039965bc42659ea7492197", + "aa97ca3097e3987f4519063fc8728c5b3b01365ccaf191553fcd10197670106d", + "aae9480c83f3cc084dca70e76f618f82b87bf2fb2e617143edf01665ef506093", + "ab62a849fcc9de65ce3e9f885d959a2b3454622150816ab94e10c0d8d5f0af14", + "aba3af84c0c932cb2ac57217c3c01c65cab49ec15f8773ffe5bbba1c7fdb4051", + "abdd082537018e878916cd25555617da6b91fa387c23f45b53fd80da215f15c7", + "abe660e96896609144d8f5994cae6cd441200a09e16a5a9080469f3023b91e86", + "abfc23b6077fd0a9b69141b3798a24ae1bf70e8a6e985057ad09c3c1c9e17246", + "ac2b9a3dcc45274eb9f2cd32498118f539bacfdf5705c3a9a60d68b8be68cebf", + "ac4a30459d90b1ab88ed2c6d96d1e6afa2aa1a59e708ff7350de36b260e4bd64", + "ac91bb18dc539462084798d093ebabc3c38ecdc88db10582b97bd1ec4063b1d0", + "aca5ebdeb25a6839fdf7d99c3c40094535dcc69fd098c51ea7e86f870a768388", + "ad1fea799f9b928ae94a7a9f820465a3d54a164276dd68f1fbc771e2458ea95d", + "ad4aa588253fd0b75effba32d3f3f62d2a0b20499b4b2ff20183c9e9dce8921c", + "ad4d7e703b7b024c40df24238902ab1f66c74d3982da912c354dbdb120b00a18", + "ad545207d62ac1a22cdee8252b85b32924c5e16627dd921a96a9c327c21ee4c9", + "ad9016fbde8ab97a81bc34a102503eb570e2d1cf3aa49de6d37cb6eb272261df", + "adb558ba9f031272bd864ed4c02cb66079249969b4a84714d8e904450bad301f", + "adf7df02e5ca3584edd4f5cdbcb27252efea4db7d58d67f833789d016a911c43", + "adf817abe34fd96eb5bb476d4ecd70d5838d3d1535b1a1fa7f89e48deba3f691", + "adfc6ca3cf146e6b70ec610f433048f8f2f61e4b0a9a61b00bcbd548aeac7c68", + "adfe8570209520592870637d0c6d89dfb53ddbf0d5162c38e7ffa79c154e6823", + "ae282a9feba3cb183bfc8ae5f3029dea7f85af44579e140d29111a5ba56a8a8f", + "ae4a88c7e31198956db2ffbb52fd170f8391acb7af4b75f0f7e17c52ed28be38", + "ae5753a98c376c9e7d29d50ddc0e0a6c9eaa15a6fa983e15fd5bf6a70dc1cbb2", + "ae61bafda0632145d2c840cbf5b9ee880f5d515b1dfdf84255024d3aa9ebb866", + "aeb84e1cc03533df9a567948e3096a4651622ddb8c65d273ac6ed9cacb76b27e", + "aec8370a06a5d1b6bc64be94fcffea3bbcb31bd7c3f87072f6eabe7c2f74353f", + "aeda0d224f68d7d2271fb38d4cdea0bab20c6d678bac9e7415be86e1428a8f2f", + "af0eab4a7ff3a9f565c5d441ee48c6728a8d2572bb47ba29bf054ad05c830fa2", + "af3d43fd362bb702008257f845c0fbd7fb0b247038bf11c8577d02b448ab9644", + "af40469b4be5f0f237a06204c6410d570c9b1713dd294125dca3fd71e5e67947", + "af93c5d79d26229544af7194357c4a6874a024c4f513e73ffb9b6fff55d8f800", + "afb9f2bf7e8bd33d310cdc0db5a756e3a4783e435816d67a201b8ed3f797b1ef", + "afc38038f2d082d7566ca44715c132c0d531f317a1a3d54a88178e187124b18b", + "b02963656a19f9f7dbd1c3065b907b214dac7cec8ae7b0ca8bc882a31112e43d", + "b02f7b28daa1a2f7aab7632e16b8cac778bc70c3e8a1b92d2d9bbf3721e3a4ca", + "b03c695f732f3d794a58bd6ccb30774a7e9f77e6335038a1b81997cfd39be8ea", + "b0641b7d679fce235e06223f2fc03382a110fda99b2624b408fdf9edd18c9a02", + "b067b6ae3257a455bfa37d6317ea11da0285343065480c68b88fdbf1f564c456", + "b088b8fb0d23433e44bfb8495d0bcdf14a90a8688a40a33d77ffa1434e4bd021", + "b08f6cb49d8f2c0cf7fc4529717ef050a3d5ac48257e43bcbb21ffdf9ef56721", + "b09d4968b098e1be873a624d4735318198e28acc81516d23ff2a718d5e05da0e", + "b0c9a587407aff5c3b96b28cb92227db0849d7b1fb4dd54ed01e9e194748fda6", + "b0d272d65e86b83427688ceb352a31af5bcabcac115cbeda72ce0cd8df37c3c1", + "b0f9d0a383327c41c70b0d99cc1e17d9e4b0f1a1766c4b1c410a847baec0eaab", + "b1188a1e16f91b3e25c4437f118bede028ee084b2d52ef9cc226ac04a33076f2", + "b148ce2a75b2aafba33d67b9414f1a8b5a355f84f16d4c0b6438da02ff71d9dc", + "b14c6e982b47a9f7cbc83e2178db724a8dc1a43b96d54f4d28ccc5f2a03de985", + "b164746f375cbce5e956785bb3571c970f97f0e3ea06b58f63e6f031ba2b1d58", + "b16a445cba1d8945f315d29452fc1bb6b36006f64c9dd3a71818a89102c48dd7", + "b16c7b13be12df7a0a17561e577053f14140827fbe7f4e220b1d3db6df56a2f0", + "b17984a1dea67fbaa09b778d2318f797f0b0096ae071c7380d13b2913befd546", + "b186db261318ce9d0eec255ee4aa33caad5d23c3fbfcb7b0cc74a01c7ccfec43", + "b1940f4e9449beaa5e5553b1396d1bf7ae7cc18512cb8927bd44996302e44010", + "b19d1adb2abaf3a1939ab6fdd40020c0fe19361f943dfba64dee935859ee905f", + "b1a5bac13634f77e22d4f4d18e98ea341b72743117f0720cb05cfeb06188a979", + "b1d03318fe70818bd4a120e56e58c65af108ca8089ca38b33aab4d2f49666b22", + "b1e5ef44a5efd3a383ae8ccbbae9e741e221ea1fbcc5d05bd23ed1f2fc3e5fa3", + "b1e84bde7f97e4c7525cdf0971c26f570dfa7821392c3414da9bf6d49989be9a", + "b1ff72519a4f541cf01e4ae32345cede5aaf9d36c67d965e8981c26a139c0460", + "b260bbe4a2dff2585030152513fbd0eecfa57c138379e0087ea6f906c03201a8", + "b286fa0eb10cb2daf7436686cd9582632439c280b04b6b7539e1cf79a9f11f93", + "b2ce9e4d70b36c0cd47252751ed47b8c7765839750ab046b270f863a82f19fa5", + "b2efd303d551237f7c1248e6619bbf925479058a16a716f102d67b94e494131b", + "b31cae085781532bef6eef09b4bfa31dacf2bc02dc20fa2d8d922a7e9f985f43", + "b322dac84aec62b55e85728718e2a0d9ddab14bfebcbde23f2eb6b949dd4759e", + "b341b490f5f8c125d57f9d9633f8af89f022d09083e79b096a593a6b23336d13", + "b34596b6121b6f5302625bfc13325f97a351ee3824e7b08171554f043e7213e0", + "b37d030e04976ad9717464b22b3854a50114d068ed51509060bc73376a484116", + "b37d200ac7c0c826ddce7614f9224f2365e8ab7732483874f4ca181394c2a5e5", + "b390d84397dacfca64a5a0e664c32603b4064baa103beea8c699d807793ee9f7", + "b3a19406b80a96b58ffb2186de2d9ecbcb3ea04f0079be0b038fc3676d2f5763", + "b3c4733af59558e62a4d1f9f5ac676e225e448d7cf70c381d9a3927d5f49c450", + "b4090022ae5f781ec0846b887e8576989db3ba34f51aa66d8b619eb4ff73f665", + "b464834e6ccec4397ee4c4f488c26779ddf6a067b7258b25fcd31bf1d61c77f4", + "b46d29e256ffda81cd2812cbeb60119d4c3521ed84384a74970b74876f81bc7b", + "b46df5a380426793e33ecd6a5cf0ef9cac8fab77a13a0d9a1dbf80ad98ff4f2e", + "b4d3a904bae484b4816a592f36be4cfc175b6af30db3d5fdbc11d3692a42ceef", + "b4e84424287c53a682e4e2c85b57e730c130cbdcaeee4c5f08e481b0c9767469", + "b5731d35376fc114009c9ecc6feb8061e4326f8127de14c332ae7fd0f1c8e8f9", + "b5740ca18f0bc9a3f6fa9ca025e89a4f4ae9c571ff67d75890cb3d6414463e69", + "b57d18041db98a5c8a9f58f983b9b0d8771bbfad555450164542942faa0da287", + "b590eea2c394badae79c7964c2752de3e07cd42725af5153ad6e0f4f0f81560d", + "b5af2eac889304f3f2106cc0cd4e4269657876909cf0b1a6a5e1422caa69c026", + "b5bac383570e16b15b2e55caec37e1761c6ec382a8b744174d29687b4508a94c", + "b5c30c652903550a6533882aa07c3c99df4e03ad64056ba2df1245cd8d2556f2", + "b60f4d4880ebd0615405671fadd4969804cd9ce9c249ff4af97d235871d45324", + "b6266da8eded1619177f4f8b08f28b6dc348b362741465078fe9e3f05d516da9", + "b647017f986b484639ada0c1521f4cbf3d09bb6d852f17be1892fb2e22bad569", + "b657cd7a60e964d2f7b07dc6eaf7e1a4e91f6fb08c90180d3dfa1792260fffc1", + "b67925a01180149949620a0dddcc956c5af9cc1579fb8026c717ac2c2d6f81d8", + "b68f5c89a885438b835515c63cfc0cf094b9b480cbf4067fabc35fc73e54ae70", + "b69f06e4f6f273d7a8a6150e56c64e6ea920641fb286e654a6d548a0ec5e3ce7", + "b6afb900709a13953b9833e507e5b0c53761bf68ba514642a45fcf3fe7dacea2", + "b6b8789681c6078777268a6ff1098d67a2176a210b052275e8bdfd378f0b7be7", + "b6bb805f019769e7c22c3099958cc7079d8261a3ceb6760f2307b824170a06b0", + "b70a4c5b5ad5ef61ce3dfe870e9a3f04dce28b863c72efd2302aaf9d0284dabd", + "b712f04ad27bfc82a95a0a9539c5cf93b2ac2b0f938ef593d5301645a3dd8232", + "b7358990749f89a239d0c35e4454c1f2256f2696f43c0b9a6dfca0a01e8c234d", + "b73736da3712e2ac62d7296642cc9b095a26ff278454b9c800c8e88ac19b0c68", + "b75225cc5620623a5f0aeb1ae56a99714cea6aac6ad220343d4ac6fe3681e22b", + "b75f3b57a8761038dc34e8c06242932cd7a3a42a4684ad04c3d0aac1c8e01f35", + "b79f8493c9f22103dc29fb01c9606f726684a35e81826d67290a251a494b2d6d", + "b7d02801cf1eaa121d3e3e958f782edd778ca42dd1acb0d63030f50b6ecae0f4", + "b8089da48eb95d0fa7aa7178c148f878c61fc5c425e163276050d66769b39570", + "b848f9c1cacdfec2ca9ec77091fa926373901a81e89893ffd44b8e34e813c6ac", + "b8a81aac98d4107d71a94ed1e7dc03918b99566b6fef297c43daa713c1e0aec9", + "b8ac6150c2c004604580621d18eb3af49871dc2626c1d951334225bc65d1c438", + "b8b5a6b49726a5e7522829273581631f66139462cee12dbd41ca7b93d8012a3f", + "b8b736ff78f5dd8bdae19eefa05ac7204f4f07187d163341e9316f2caa7921aa", + "b8c3bf17a916c8272eec662a19fcd3ba15885deb7013e3165443147be8624afa", + "b8d7cedc5d15d681707f2f38e63c1b38836456b98577792d229443c26cffebe8", + "b8df782b5cc955e789adf0cc2dba357393d352da099418073e9a9c255f0f9797", + "b8fd7e25bc04fd03c01b2c376dcd64328a543ebcd9d53564ac8113be74b1aac4", + "b928ed0ade3ad7ae4d974356691db8dd8a7fa18ed135a9aef8d746184aeda3a9", + "b93fba82189b4ca8c32a2557f46a130f50b57171e1951004e5378a2d8fad05fd", + "b97305ccba8b3ff42187b7db6c2859328d1799cd55cd581fd3cda2ab303cfd93", + "b9a967c30fe2122901176bf823703e243ef604bc34f75753fe0d8161c46578ef", + "b9acce70b7e12e8b34844749eea5419deb55261a739d1519432c2cfb66dc567d", + "b9c540741090fe4388183904f3f19738a7b727103ccb50bdcaa87abe0a812dec", + "b9dc3d032fec920e4b54a5f3e44c3d2ecf6c8c09745da579b3fe1a22eefa9bbe", + "b9ef19b21ea3253daab567caafe43363eafd77cf8df382c906231547b7d1eade", + "b9f2cddfcd21ccd881f6178c8388173f4997370ec9a2a5b3a3f10309af4cea8f", + "ba3e5e077781c9e12718fd602c426becd35555b333346d5c8b8d0e4b8d59fb75", + "ba3ea31cba469ba9522e17bd900bea020ef196b15bb4f6082ba322d71c7cd2aa", + "bab819a3345cc38eb9d2f4b3d594f7040b75114388442241d3dcc83ce9289ff7", + "bacfed8aa647656b317f21fae6db862e24fc59481a66338f040977139dc3d059", + "bad11986a7cf2e25876f9ef3449c835f2a40ce7ea2f8bbe7da9bdd9beda4a3d8", + "badae713e39d2bb3121a95ff0de7e6fd013bdb9dc86119b49f7edf04222f175a", + "bae48bf75fa4d9c777766969cea7efa83e9175b9735a27008fe457060dac51ab", + "bb17ee93889768038584160ae37403acb9401651630ab3a405abd65214f8e1dd", + "bb1a561eeef5a67d08fa6850904216b1d7593e365271505e36cee5c56e6e886b", + "bb53e6ce9e0343c8e4a1a689015ffcbb21f3db895b2ad5ae51472dfae8f98759", + "bb811974b2088e8fdc2606f0c8c076e55e48972c5213cf3234945c63bb9aa8d3", + "bb82c9ba73073bc7bf00d3315367a6e01d136bcd9a9f0677b9ee642fd2a59227", + "bbb54ad3f3e4d91ef75bc242b35a9a28625963d91f7c123d99d53c2ddd58ba60", + "bbe0f13c903510cd9b65c6bcbe40c141847bd3289e8d27a17c494d98104f1ad6", + "bbea53187321d8577bf4de25cbaedc339cd30cab7900e2d0f9dfceff92e30cd9", + "bc491ccaea000e7928611c69f1b3e7ab5f24f03ada9c29fcbb69f7fc05aec117", + "bc65a7f288310c5ea632964514744d5054e03a6862fb478ff41f6ebfb5e4d749", + "bc6adec90a6dd989815d2ebd2d3d122e5472c349eaab762ddfac58eaf23a9f57", + "bc8cb19dfa4512030c9cfd5171d50b53c2524193b48d715df8bb2f349d24a0be", + "bcb180173212c62ba2392972845d7d66e70d6efbf4d99f19703185b405d8d6b3", + "bcd8b31f715818910c72709e4e53ec79e715058b3b8b20b88d421485b142705a", + "bd2121f0f8916cb944dc0b9cce499d8c8201bb9ed855484806832f0e3774cb14", + "bd55dfb70701260beac95f7ae6c29b0cc675b33a56815d4368ba5a5a9b2fb960", + "bd68f00870fa2e09f15aaa7d6f7f5e3c0e0c8e9df9fd13caeb5ba5ec721129c3", + "bd6bf91e43218326db970a15eb8c67d056a452ea307dfa1826cb6633acf7da0c", + "bd6e313ca24faa2d2310f0608762f6c5fac8ed5041e62e6e8b5c7c80edfddeef", + "bd932b7f64314837ebadb99027ac33363e1ee0729674016f754c0ccad8e8f101", + "bdc0a195aca3787839f81cfb8e1b323702fa9650364a46c5fcd573a2c49f60c7", + "be0b46e7fc8a373d20a3fcfbafb196c04fad15e6cb4292c552dc243ae1323f09", + "be4265c2119ae94dd3dd6269a67b63bac95797b3cba30b31349625b1607919a6", + "be543b6873587292c6e6a41172e329c02e95e4a64e504f19ea40c4cb61acaeaf", + "be6c45e7f69bc3a50da61647200e2ac1ae02bbb1a8ced24f59103c3044bf24a3", + "be7d2336c310249ab90834b1f81317af638e6dadbaf11073b38092b368151dbe", + "bea496e9c1a81e100601aac156c4c56a930441a8a23838954bbb3323a21047fd", + "beacab588382cbada4b9001299ebb9fc2af37622555922c641cccb25aec00506", + "bed8a7542d9679d11feae206a51d62c627da9221a91b6835876f1600176d7dc3", + "bf0cd750da6ba9752ef29d82afc70701c2f466f4f7762efb7c587c973e66d8ea", + "bfb2e72e247a7f021efea6c2ec93634f2e549a87d06a6d518c04b2b8ced37d7b", + "c00ac129f5040ff62e1ed55dc195170aed57ceecf978448290f4c98b6f2b7cda", + "c00e8564f343e9d6d7bbf1516dbba91b91abf3c89dd7b0637417f310e9e51245", + "c01a0385e94fc2fcccf46f244ca591e67737a37030caf872033f60323dd42c4a", + "c02899f51d77c003d5e83858d1c9b0f4f01b04de3f7642167221b4f13a0b2982", + "c094c64f759b68c09c223f21b1cfd965b766db99af89b76efe2170546a2ecf6a", + "c0c3bc99b53fd5e09e26d8814ddf1ba8920932745eef252fc09e240d1c0e0d3e", + "c0dc7ebe001f9d7d104c791c9de10c31255c5a850e14fc31dedffa78e6541e83", + "c0fc70c7452a8d99076484534e581dba98bd1e867317810feb0a103d2e5c37b1", + "c10514926b6b320fbab45c471f278074787252ffec0aee89aa55d78ee98abc39", + "c116eb782fbedff0cc22f419af954749c075c7af985c9eff7d0c165d76144ba8", + "c16d20b8b99da3786748ec64c23f1d4514e948d12aa8f5bc5187a38454282226", + "c16e7d672eaf324069aa23d5d6a1b5f138a706dcdb9d093c6ef42bd08d2b890f", + "c1791b84f1d8438c1ca830aa9d7176dd9115d3ceb6fd23aa7b204fcb05165ce3", + "c1a4323c6bd7607433d6ddf4c76e4502e8b7a6b3b72fa385c906cdef3699cc93", + "c1ad7d9b7be6927a9c31647b1f69ebb2da21c369c6a817099e6976b981c10deb", + "c1c21f7564d5ef802a6cb2a5fc333b10067d5814640df797701eb8fe4b114e7b", + "c1c2b04639ef27293b03468107a7b2017f793a7c5ce17232db08ad396c23fcdd", + "c1c84ed5e9f2228c43bd94eb64102cd7d02ed700cf4f712be746eec31807506c", + "c1d0a5bb9e5792eb0903e793a7e924bafd953d7e53982e0f31e650162658cfbd", + "c1e5ec301d21dd885bc19330ff1cc10a1e028bc22cc866015a3e550ae1762066", + "c1ed275f0d09590a03749fbf16e0767add13f879a89ca1c55950f460feeaf205", + "c215aada505a3b2c249a1aa1e3e6b5ae1817942bc390c1e2366096e8e438f996", + "c221e076bc9d99365c242a0b180ee5895b305510d85fe5b1e1e4fca94ecef40a", + "c2456af35e9845a2a973de93fb3bbf8031c8f2e22dba10d9496fa138065731ca", + "c2461e62faef7d19e61d6a84bddd561ee9746eb30d0b28779b0f24b0ca84dba0", + "c25b6a2ca778cb99b613fe51208ffc8ef01a48f9516645701c248300d6ceceab", + "c273d409d18c86f08b74523ff515897dfce4c5bf272ef773512fa8328eb3e808", + "c27aaa5652f29d382f2e42082cb18850ddbe91d2c142d8cf62d16fa36e6dc829", + "c299e5c291e69611dad567130400548bbca22948ac9a2be5d86cf0be1ed59c8c", + "c2bb18b10dc4435b7225f86e6c9ddeb04669f2b2a062cb6d28fcb9fe61e7d29f", + "c2c199ea4cefaead9e23e15c44d8f94855398419bc748403163652b18e0f14cc", + "c2da835ea1010e60a621f955df541bb0d11ad051b342041497657f9d3a0f44bd", + "c2f750075e3b223f783bba59ca6cb7537feff873a77554742e170652df62a787", + "c2fa1db7cc0f17f1b740a1d3da1b0c2ce7e1cfb7abc7f92eb342cbdc0336983a", + "c308697a46022bb28dd37829dd583edbc414f1646d6ba88d8e788d74fe75e3f4", + "c33f6e1d953c2e2459176f207c5837fb768cc7eddbeadf4619aa2777546de62e", + "c35b5d06952680ddbc500436b29ba9576bc8f11bcca1f5977d6714edc5510ceb", + "c35b835d08530179fdaab703ae0f98b31eb83ff1b1a9707c341029c0516d955d", + "c37bc89dca3af800f44aadf1d78e4f7e3a3e89af77ea633d31651e1a95eda347", + "c3af48670ea6f3200c2ff85ace910f488e272e926055891d86b862e0b353bfe3", + "c3c41e5552308fa4970fbda82afa59b66f489c27a7db4d5d94a60d0db8c8039c", + "c3f8ff3e207767558db61456c7f127af9d79c2d0eb50a8270bce5ea45db2c543", + "c40d78ed2ee68ff20485aef5e34579f5f4054a6fcf5d9b305e7291b701bad3e3", + "c44cb714ef7ca78ac4a00b50760b3abec03706a7545d5031a3a5ab948f7e3830", + "c4736ad4146e4335178db8a9741c652d5d05833fd419a4d44301e034289f86d8", + "c52d67b0644f775244057009d1d807cf959238e8b094a325b89130b66fb13549", + "c5935c567c338bdd710e01da38b97eb94bf8a373c524ffa90f06e141ca21e0bb", + "c5bd03645c8290866a1d39f3b34ff2bb3e804bf5a0d19d1c38b6f3327ead98aa", + "c5c447a38bbbb89458230f11f0b3732d0fc3dfa2e7dc93aa8177bd348c3d8b13", + "c5d45b34ecba22e99f79cd23ae74c5a3af84174d5379bcda1564d7fa6fec5495", + "c5d4d80494eec80c6ed832e4cd5132266d42a0076ac5053fafe48ddc173abaf0", + "c5f95ddaaf9299738b4281a94832b4558835d454e6f79f15b9b41df9e4f5d760", + "c5fe2f8f0295e4674f673532c94ee14bd9cb9a81e0bc9cde32814bb3576865fb", + "c611664dfe03fc5ee5c7fbe647d0d25e57afaa76bfcf90ec9e82f3331b41dcb5", + "c61963f555a95cfc758ce53612e8f0e4cd2841cbb614aff60605dc6ed944617b", + "c620c3c9379df967de6fd8f4c84db0679e9d2dad81cbfb3e2d77d5ce454aa3a2", + "c638f92723f379d1dc48135cdd1a27c1edfddbc74c7ffaab10cd4879227f346b", + "c63f67dffc9e3aa8d60e13154e76d0db63e3fa987ef4c910f9912ca63a0f6f04", + "c64430ee7118581e5bde54d8e9856c15054d73a04f4899f64ab8b59a320e9365", + "c646b12f66f51279cd7c736b218ad1f413585472da68bb60bb92369f7cc66a03", + "c660922239aca421a5cba165941bf8cf071cd9b118cdeec290fe7f5095a38663", + "c67c22645c881371accb498fdc5160ca3f3a69cebea7d152970ae608de1f480a", + "c6c3f5e34e82d2e364d88b418291b2cc75a632eab80e23a502411656fe5e2af8", + "c6d7bbf143e0a344523a1d435ff82552e0ea7c64c6eccff80f9bbe7d56107045", + "c6e87c32c76b5a2e213fde365de66d9f0fc396743bd3d759388be531c3299d42", + "c6ea608314b0fb56168c2fcf3fe90f112a36997db1f7f52b767255a85dd2de9d", + "c6f1e298f5c323a4991b06bf766f36b810701cb0522045b27f73aa8a2d8081d5", + "c71638e6645944d5989bff886d6604db71549d3e91a8138277d2f9d85c9bfd26", + "c729c560656c7822b69406e8dbfddec6ac00bab0a37680eeca66c468bf163a39", + "c753105e888b26ab8eb96284b0c6ae789b38ec094f3a428b913dab070eb0451e", + "c76ecb80f592ab61cf9933db4a4ead1e171011c713a0d4b7de404783b40f05bc", + "c774bd3b040c5bed6d13003b638a72d4e9a2ba11dcabb1ab4a80a641d5d7d37e", + "c7835c562b64fa359cc458c940feb654af46e8b10e2e84fea0e25f077b9f1e41", + "c7848b1fee9b082918121acfb53913b4a8ab7103e792b429e76c9657150e070a", + "c78bf4ee459dd9e2a4acd4a83fb6178ae4f3ec991c0571dbaf28b6cd57407261", + "c79a23775dd5de96daa4f02e7aa240eb03a1da8f63b61e81bdcb891e29416905", + "c7dec169ce85e8407d5039e463a0fe4a5686cbd6f9167dda51caac44cabbf1a1", + "c7f25797d0ee04f41ae508ba50790f03ae751892f9e24bef0a3e447fa25822d5", + "c8bb562f8f805383903a5574b9f84445414378205aab1d3d22b5559a7aeef6cf", + "c8d9cd0644ce09573c71564657ee8e0a78a66162fc7d207a8567710c8f7d7fda", + "c8e463592713e1c7727569c6e667d8918152fc4cc28c15fa75ce9b41239475bb", + "c909c842b7d32ea12f44d92ac041cbdc2c64248e9522ae465168aaff9e899909", + "c91c361212e050943a81e7bafb3d4f2b3a17c63f4373f240593f76233b48e5db", + "c92bfca22b81581e51629b216c9389ede9575af63c0cb28097310c7035952bd4", + "c94ee22263a5ae8a3bab787be129a555d8c2b662564b238c6d11cf1d88fae542", + "c9785645bf1b8f42290c3c10e0036dea320a18d1bbd8df0d8b57f974565bc139", + "c9d61d4310b1e84c3208b667dba9ae044e0acfc1fe96b96d8e2a11fd1f4d9322", + "c9ebe28a3b00f9cecc4ec7d71ff046f07fb16d4377fe33f68711a4145243b523", + "c9f778d144174b667b4657826699683e72d081916ab3d4443ea5cf1de63b5ebe", + "c9fb44774055fb5761f738df4c8bbda5c59121dc85ed27811ee16a7b17b8a234", + "ca02950b9ed42a6cd522865e22c5d5da36b6f4cd1206ce8f36fec5c2a9fb7100", + "ca0ecdc8c99fe496f1a0728042f34bb8477aac8d6f6eb988220200fa60ba3d8a", + "ca894012f7454a07dc52cb1a1f0f3ff3df6ac6c9066f13c331ed5c1d73eb0f3b", + "ca8b4797bb2a647ab2051a4a299d4e1ffbed14988e084b66330bac78a06aca3c", + "caea21412875078c8e0d019066c50138b67a46835a5cc2c25945dae6a099059a", + "caeb0797f58b1448e6f64d2636587d63efbb6e27607c26a6fa3b660296288487", + "cafff4262ddd03cd199ebce7acff792b8e0f906c2f90c0f7b556f8faaff4228b", + "cb1e6c8197a549f105abbef457a379bd4d71aa5c35d63d780733cf1f77e3e5f1", + "cb308fddcab3858dfba8e2d1705b9ca9aea23cdaf9dcbfddf4b77d6db5db9040", + "cb3d92ef090fd329dceea69ac6d48aa42aa88149701769b1a4625ba80cbceefa", + "cba2b38f8e1d3963539ceaa0303e8669a015599a0e01062fbe4e82e8f359c667", + "cba925939b83f0d297ea329100f6dd9ec95d783625be4ba508391646ad22388e", + "cbc4cbe38d4e3f010efbfc9d4a6d021783b5f8d66a9edebb9e6afc046ef0ee4f", + "cbd9adb58482cd85fc5c54d0a13ebb0e0be278c3edcfe7a42b8b246cc8b46e27", + "cbdfe9dad04b6df20ab06dc513414ca71c92b6b01d7e74b79a13ab76ae590474", + "cbe87cae84854bc4177b85a47b953f39523e61561c2488db4b4d64374cebf4f1", + "cbf3c36ef8b5885ced1792a28616aaabddeaaf368413906e9f7ff6ac103c9ad0", + "cc1299ddca29d3f47b825ac93c64b361242fa25803f328fca13441684a4c7a63", + "cc1b30d08252f2224567abf1482c833f629a31afede28893f45868a3122d025c", + "cc8b9c23a4766d262667973e449c8e1fb6f5ea9182359fe848983de8e69420f5", + "ccae31e6c91561c34ae01567e9f442e6ff4ae66e8b2ff665c8fb8199b0222253", + "ccc80694b03de71ba7eb7837ce36fd2c8ff6cc6a7e121f1a0010e21394ec7d28", + "ccc9952cb786577ce9a48ec50a19331a7e3b5aeaed521227fe6b36ca1650e8a4", + "cccab9452697a5822f7228d9ab3198a4e3f2b73810aa78705005edf91c8b68ff", + "ccccbbabccb218f9dd35e3245a04c9f58c86e3d11edef57aa70d17ed0a47ca5a", + "cce04483aafe15033058fc57a6adae63474c8886b7fbd33c2a214c02b64b66c5", + "ccf474053b489d3012904f3d2c634e04ed67f79fb7f258d12174ceffe4d581ee", + "cd03e2df6e42a8883dfbe44e1c0efc58bf63aa6f6d32d47fedd8bba1e23a973e", + "cd1847ee00a06fcac86360000d06315bdcb62abf7dac02e82799277175c37a03", + "cd1c9e3e32e853d95f9f87241c0907ac60f999a01c96947e80829a4cb472a2f8", + "cd1de18adf0a6ef2f23a850f3b25be3407533a2b7cce889d7ebc5ef5c137a3df", + "cd456c7ce5577a4d33433324f28e893d9257b69bbcb020525de57e17f7ca2379", + "cd6590458cb77b5c7aa0bc28bb00582e6f5b6693f59af5d38df77b4db73b7aa3", + "cd90dd5b59b902f716500ba7c07bdcc5f9d0265757f2d1367a47af2c143adb32", + "cdf4874cc5a4273cab51aa533b5fb08d7eb14cab1179572e46c68715013abf23", + "cdfc4db314c0d71b8350c0022310e352a396e01c94d446739e8fc33f6603a4c7", + "ce80b2a6b2332251556de83f456f5be22ddf5c98131073cfb4db9469f6c06eb0", + "ce8347d62ffbfe3ef6cc58ae9ad45b4a527df7b40b899e0d7e09e5cdfb0d254f", + "ceca7e2e8199bad3a54ecbfb618ba5292f78d1f3731bd38a7f8f9999ff8bb941", + "cee38d8376a08f3ca2660337ae409abce9f7f89102199b7b55be928111c3e34a", + "cef24ac7588db91521430146ed719c060006d2d6ed9315fb63fb24975dc094e4", + "cf66b4aae4a945bbca0bab0f912f0a584ac11475c3caa7c65afc461052f2d415", + "cf6b951dda214e04ac328b5e572449c55180815430fcb0c181cf61da6e998975", + "cf844beaf2e6c685aaeeffb989285275203a16da7b6c92a4478a82de33bb6982", + "cfb6ab02ca975d55b0a6c1878cc4a3f22ee15f7f9ce6642e1490eaac8fb8da77", + "cfc5f785a585166a29ef45f8f3887658907612b4704cad2606fdd86580d9306e", + "cfdd01086c7ef1d3ec816295c00994d3133a0c9fc0f097c579a38f67ec965436", + "cfddea7938adca95988dc306017212e307a7d9d29300a49e14781cfe582b3cb0", + "d01fe0cc347b04c146e981c9f3c86510cb9b4ac16c75439afcafa961284857eb", + "d070e8c8a5b0b356d8fbc0d5e50dfbc8cba88939108ff3bb17c8cee06ef92dba", + "d073ee715c3c64a67b1dc52f02c0645f2e16dc4b8934d870046d611511240593", + "d0a2f11cdc874d7797789d80bb38d398145d118b2f161f08d6f171b571673b78", + "d0b0069247d86164502ffe2c678701ba41e1ab19cfdaa75461bc56b6b257a399", + "d0b354e0d29108ea1816cefb8c4aef012ef0c74a5dcea0fb7683de54eaad24a6", + "d0db6137e2be2f61916b26e69df32c0a1525d91867ec2a6baccd1b05dbf733b7", + "d0f579e298d6c00b884b463527e40a31c703d780772924526fc4601c3ef12a72", + "d12dd9f2c736de3dbf0ab05b0c4715204d65d682112f01d6f0a881719086827e", + "d164afffcdfbad0c8b8829a9dfeb78cc90d6a4245e4744a2f4f9462af84ab382", + "d192fa159a7984c4b8d688747b547f3c310c85249445eda2a9bac352ade8b47a", + "d1a0d5cce1543295d9ac16a559390ca2ef9625fbec0a148a88662b3bdbc07566", + "d1a395ca68b59f7acdb1ae4b11b826a5ce25f79c9184abf7dbdac2d87d2a53de", + "d1ab005534a41df9b1eb7c7af7275121d5cac159e60361088178022ec008693b", + "d1b5feafb908fc6394d1e094bef4e084d8003e475c7314265c8e0fa1bfc63484", + "d1c84b48656978636a6b54ab2d8c5a289df23fe8ba0cd8d8f4e1dfc7873112c2", + "d1f34f69500b5ea583cfcdb117ef5cda1352b29fff00113740607e752e8d522f", + "d219e8bb14cc94caa1833f01f066fa4bd963a3624286e9fca56ed00866356b39", + "d23cd9664d80088a3a9a96950b1d0859a1e198edbc456bbeded1a83db91be9b6", + "d240fafaf43c653236bbf6eebbf58cb3b479c5f0bb18c1552eb55949f8abc091", + "d24203d0614be5b8da34b8412d279a17f0441bd4381ace8178a1c044142b48b0", + "d25a40be63ec05dd9772fbfbc282715edcdea3bac2a6050a637367a6be37f0d6", + "d26c279d4047b77c1a4a9a95383b7487e2e22bc12aee632ae36466fb6c02a0c0", + "d28ac0816f3c9f0bf18e3d08d90ebd4a752239f4f23d06bff4f47cc0c8423ce4", + "d2cc4fbd2e43a98674c6c4e68b970c0ddfc139b3d0db1f70db40e8210e8402ad", + "d2d2ec5406caaa6d416c4acbf9dba6eaa4c259b8a97dd1203fc90bd92cff0bf7", + "d2eddfd7392636028a12000fe3b0d06120fbc56d0b5bb92f34a7a52ec0feaa82", + "d33d3d89031950904ea761667310076f6b95a4cc5ca930b7b31b91131f7121a1", + "d3638aaebe3f77b329913705ada57e8ecbfe424f447bb858b96a4aea55f7cd29", + "d37f403d2e1a6ad27ffa735476b4bc9cc58d1031e7a6ee8c42ae69932c9c56d6", + "d3da0f35aa5303b3ababfc1859247dd0328001748311a88cd77376fadaea8a70", + "d41d6abf63e47839422bfe192daeb310402091b9e7e024b25ec534131766d17e", + "d41f562e5632185f7696701cf7294f8deb89416449143d857d247de6eabd1101", + "d436ffd00330337a12dc5021ca62dd16010db019b10d98dbc3d88d878eab6fb4", + "d43d1f6f23de682b557ae10b04fb03b9578a38684ebb2c75ad53680a39bc6b42", + "d4887e482e42cb9ce2d5a0350d934526145985b1e312e711792f24c38a951ed9", + "d4af0e0f77184ac84939d282b93b1018614e0fb91a40d6a0aaea1109f37d8301", + "d4c681632a45ce471ed9a1798c14e9f6bc335eb80162e5f1371611462d639d14", + "d4f514dc89a34da7c37a834886b1bd0144061a816f77fe6cef2bfd066955d183", + "d52bed0de5961ce7653d028389a47d304bbd5ec2fb492688d88e01c20c9e9751", + "d53949e80f745b4d64e9553b036572189079413f781fcd75d9c956d7a09aaa53", + "d55253c49dd42e9a2dec3363f86c4c8c1184894b8f07fe730a51c6660b1edbb9", + "d55b36cf5c50bc67d8bb6b130ab1cf86a08e9317ea19ea7ea9a3df4645d6646d", + "d5737cab82e5a56e1ace27cd0dfecf5e81a85a5b80e246956891a90b2ae92e5f", + "d581b656c72970c3bde09f7167783111a25b8fbc77dbad63a9a5042e6e1c02d3", + "d586cc1e5ffecd95fdc88554b909d8221f17d16a4726358d2bc327a2d254b8df", + "d5c3519f7ae4e585e1c3cd23634b0d912b7ef200686b7ab800b31be4b3e272fb", + "d5ccdc32319e9be322f989a297b46a9a0b100979c009b6ba6deb1bec2f3365f7", + "d5ee28369b14bfb5029dccae76c41cc3188b947e4f7c045d5945febeec161b14", + "d60003a162be418f023b13145c3b1429932f426df639d42f7eda9c12b25f09bb", + "d61fe033fb6964bad0a578ee4c4536c6634c299ac42a84d963a3743a618558ac", + "d62433ee8b0120631f7f4af20bdee7f25e9625184b009a790d3cbd85e25cd0d3", + "d624e46ef452e34d883a49229c0136009bea51e568c553540b303a7d2c94cf82", + "d639559a862fb3e47517bc78973650a49c35c11e75d16771260d838e65ce4854", + "d6a34c88ccf7607bfea26640f4fedb3197571406ef9225dbebb65252b12dc1c4", + "d6ad863a975be3ebe83c4ad6f1ce0c26beff8b91b3fe36887fd13ea576a6e568", + "d6ef592700ea7d33fbd82279ac402a40bf172cbe8b4903576bba1299252fb198", + "d6f318978802318d0e5ac779d37df16f5a0d831be6d654aa57f1e88c33c4417e", + "d71f2212c6c3b2416020d45b3e01d88c807edcf43f6b83e95b4cea0de13befef", + "d728f827d33c65af123c6671f18997b4e8f26289ed3f940faa60b4eabac80ea0", + "d74727b5d722908bbbb3ecbc7cc5b5d356a3c09635fdb78e3fb23a8d9a090622", + "d76911df66f2f4125ce433021ad75a37b8f11dc1e98310a3a5eb00bd2585c467", + "d7691a888b5923650a28fe3758cff2345bb1ae8d80ee8f44bc43de702259da8c", + "d783b3620e8f7e8de20c14c7c66a742cde0bde86556180ee97689ab3626c19de", + "d79918e84e2867f0b93095aa13d9df173a1306205e2c62a26c95d5e75af9a4f8", + "d8051dc72cec1c8f55d0163c4f6606d459ff34a268c1c639323160148604f86a", + "d8117061146d01c9377d0a4142990c2a3eb36399b490ad643275ebfac9e53933", + "d86a6cb10f44ae323616539741800437205a848404368791f18793c8697c27c0", + "d88c67aaeecd2123733c147b75b18a7e23d67ac8abaf7cebc51f004b001cbf94", + "d9064a1bb6a2b88b73173f4e5258feae5b2d2022e91639f3ddeee2c6adc6597f", + "d90c3aae92540da66419e78719f2313129851b552e2f3d21effabdbd0b311e5a", + "d93cae2b91c10e8b3f169f19ce95c444cc9c8b2b46c47e4556d2830ace7381c1", + "d93e2156fbfa90b936ae9334eb4dfda6fbd1b597cab22cb2c90be37a77932af2", + "d9406b1fc98b36f27aaae17128b74ec926f8ad01aa3dac817410f4cff97acba3", + "d956c4f0e28406e4c7b6bef2d310b7df8fa0cbd440eedb9c06f3ece7c5e4ee42", + "d957c2a65445dae0b2e0f0bc24514a284d28a55cf39ee1cee5a8d3e688b925bf", + "d9931641acc262693d83fef4e7c13905f928c7442a51ad8fdf6e1d6f74646982", + "d9a77f4a690ed3e06d58633a85543fd0c80cc41136e7f258c47f1552fb075343", + "d9f3a7eb5261064dfb2740a6bb3e8dfe8cabce064c91099800bf73db39f00fe9", + "da09f8de280cc5727564880afd5046632dc2d4650e1576b3569c862785995645", + "da11ecebd84b29778efb2c763b45ba7e11e55c26ea665e136dc6148fc81385a7", + "da151d715afef7c4222446b54520107722f3c1294f727eec51cee471cd982e01", + "dab4e81d25783c9bd0f74aee16ee181acf0af223526b364ec8945928d7c673e9", + "daf8e49ac3283bbbed06fe7c1cb3f514bbcccf5e32dd189c4cab86c74f013636", + "db1e47ccc5090a6d2281a7bb9d33077e0659d7bb7c697b21a9e649b6a3e3745e", + "db290975bec0205404d595bae0d1f9f9ee1fd7bc26f9175121ad300bcb31bd59", + "db2e86a479922463332e6278dbf55e367557fcab64dda06d3cc7de5c22906df6", + "db4b658fe727a99f255ea3957d2d3529947b6caac5ec48fe25c1708db1fb984a", + "db5d60dbab7a234266c0956d863a5f5c5a100ae4067a4d46e833133108272f8c", + "db91903e6de419db6c737b68d92f7f85721221ebe4c27813f1cd08cef5ecce5b", + "dbb5bbcf389e31bf48d994286a8395a089e7a6de641dc06567efc06a8125c08b", + "dbc1a9a5ff51b0d3d41c72676706adef865ab3e45d26ff2a05a993487a3a8029", + "dbcf50070e709dee8be0cef48068aa573818c8de9d5b7c440bd01c1719eef6a6", + "dbd36f5d6918f0b3f987d0d18bde8833f41024e3019ed93025db2b78afaf7f49", + "dbed23923526ee3ad9c1ba501c4c7150bfad61ab420e5e711e2b4d4e76192d25", + "dbf0ff60e08b430fcdaa06cedddef05cdb2fd205061012a0ae4113d9e22c7c18", + "dc1e5c4c9a22b0020c807905cfd648879658bf757e481c49d677f7ab315ccb3a", + "dc234840669526b8354dedff55c633a89fae838cdce0ae3c777b20bd0ff8a5b0", + "dc48000c38c5cdb704442f68e2b1faee6e061401cb9df35454212f46001bc316", + "dc6c45d3fb6024fbec0d0c553ceb9182658161d39902d52c837631db86229ae4", + "dc700ef65fb407d53f0af7dc467f57549ed1c4b6ccf8f743f380ada78301915c", + "dc78b2e1c869e9e4cef3a00177f73ecaf699a511a4bfb691b0be761977d35758", + "dca104eb57c18b8fbeb17dd08fd35b458a0cd4c870e1202653e4452fc46ccd00", + "dcc4579b164b16e503e32fc81f19470ff3982c05d4f61a41a063d1c25adf8e71", + "dcd3bae5bf55c08d3de86830ddb7571d60fcd64bf47134a6cd8fa7d15d10fc4e", + "dd29a8e572daee4a427ebdfcfd9398d6ff0384d4d2fce00c2eeb5055d110b081", + "dd8cb0abc314ebe78432a219bbfb7ada743bb081249088447dbcc4e4cc70694d", + "dd938aa5437ea65e7b2e1a8ab074abc3fc74de10e486f9256553eee63262b54b", + "dd95113e1f59e4c6968eb8bffa51d996a80a07bceeafedf95a1c8c58b302715d", + "ddd7e5361fe63c2df295a8cc68e8dd7678892867ff1cd46dd837175f30fe4175", + "ddd82ffd7398afcd1b42a4859b65af105c55dfbe21564d87addedf163028a75f", + "dde71388ecf1192c2b1cf1b0bb1341e5800ccf19e54417b3d6a3a791e60e97f9", + "ddeaa5c8e0724599daca62a8952f9f3dbdcabb48576783d4a5921d67bf6290a4", + "de1f40bae42ab53dc5a1a9154c96b94109cb63db2939d268d8c27be0f50c72aa", + "de2f03f07e8f5c21b5d86803f1c512267e8cdfc5580d1f7767383bc7dfc14501", + "de484ca0333fb6a1426bc6e8423a3ecf0e8cc6176a92c3d55155189b9e5bd488", + "de6da3d412c132de0603ee71fbf57148663c9652ca08988d366a2bd38032211e", + "de79493ba0e7a9bee56efd8a6291d858b5a59c282990fd67bcd24b8bb3e8529a", + "de7d61196c0267bbc0f2e2350157db8607f5ca2005af62e78b6c492a53e627d1", + "de8b3255316fb1ba2eab755aa341fb1645b172862215080cdbe07d595f6e1a67", + "dec04ecea85213d947af9b3d88ed75d20fef74ebdb0a5b319477e7f40a2b019c", + "dec68eb79dc49e5ebef9247cdb8ca00020536d911cdb3d504a44903e0f13813a", + "dee4c4ad7cce6ba20f17b691f3829bab814297ad16f25a705c017156787705c0", + "df72209a247f2662c87a21897664b840207d054f5a609083c4e4bd58c913dede", + "df97baa5c6e3317a4b7199a23efdf9739b610abcf987760302ca7749e6bee48e", + "dfa0964ccd4294a4786d98030d7f786a51de585ab9fcc66e9c5af18d7d61fa22", + "dfad806f4668af0fdeb83f2124e6049759bfc6a1ba71467f70a3e241d50db23a", + "dff55f5278fbef8d13bf51671d73b47fe02ae344308f737e751d224de5ae0c9d", + "e00b6adec3f29f19699b8ba00228d2c78f2d72bb996d0c4ca64e232b18bc36ef", + "e0348e9295bee7097b7872c7f50ca11c038e81dc92e7f76edf179e8ce64728bd", + "e09ce2a126107b366b8f8967eb0e8acd23e46f2100aae5d70dd3320ba973d21f", + "e0c8e621f083e33474b8b839fa19f52a709ad11cef6da76b1000469c3a4d84e4", + "e0cc2ed2a8b47079fd14310f898e4a16dbc27fddb31b8855b89f3d9ecc5cb42f", + "e0ee1ac0a2e7f6e414418734e3f8accbc6f9f09832a454b5ead8ab7eff5bd3a4", + "e11e96d715757aae37c340799218673e837289d9b80fae6534ec3f91dcb01699", + "e13160d8f2421179342e179055d54da9de069a1ccd13c16dc04928481d8d024a", + "e13c156e3c6b33114561f9a03066a2d1f474760c197ab3e22dda33996abc6225", + "e1490b6ff8322a5aa02caa25ee79e0e8b6dbd41368f32b055a83b38b3f20b072", + "e155b99821af5d977974bf58e1807c82d232156f53041f13339bcac76da04221", + "e15d71bda0132aa953c779dc21d8eaec2160b70dfe978c4a64c59cfaec344854", + "e1833ccd30ff1b270164c4ca86a26f362a7b4d378abe951db23af2437292a7a5", + "e1a0656684ec308cae1e1c9973c7db7ac78b8891d318efa3854fd0ccd6b55223", + "e1e126ccf4e5b5d3f928fb86c7c249c071384cfb6d6c49f59748fe062b89270f", + "e21ce0dc2ee47f468c8e7b8a13c61223d7379531ba486d121feec867fd8e6e7f", + "e22bc8f83c92edc88ee5e4dae3d64c162633b5e1e73d21dbcc85adfaf90b809b", + "e22d4e35d578d3bc10e8c0bb073103c053b3ae33c0f42e5f40ee6f843acff811", + "e2306bf5516952d77c9a0b4a41ab36ceabd3bd43413b14a2dceddb4c0342d351", + "e27557f997a9172b8decf36332ec7af376eb8cfbcce1577279b2a79db50b74ac", + "e2b6036fc7df03dd49b347b097e61a41f1608ff15d2268190f460d5d4d17eaa7", + "e2cca3b592eb4f08c768a27df03111493aa29246e2f896a08bbd1316dcc9c8b5", + "e2d431394164bd0c41c89f9b8adf6088435011a64d396a874e55abdd7269e4bd", + "e2e4de26dd09f5f661a27182517d173c2b5784b62b363fb94489ff795bbce5af", + "e2e71862d3b63a9440de0b65100f8f3a5f868f4685a2a8f514805033a88c9ef6", + "e3161696c0ff6c0c240ab5c8a39baf2cee8329d591aa4219abb2e6a8f339c0af", + "e31ec9fbd684cebfe22f4fd79d494c1940868bbb1b706892044aea02326b6703", + "e331d60266d5fb71a2cf5b4e342c841d9e08c449c00bbf5f21b41061d5e7916f", + "e3d29931e874f9fb5e99c63c1da162bd8be6723ba3b15e2a3d115353a095984b", + "e3ed4db7194f77d1944709b3080b212e4d1148881725a441b6a89a66747d152f", + "e401536f4e5869d0aafdcee910b572aed4bc9214aeb90a0b3ac026b6f2a36b15", + "e42ef36206320b49147f2d4545be3c521ffca988fdb68b93de1e150d74f45d80", + "e437847d2cd237d56bb0f4e94d08b96d54a346efbf5e00bd4403001bca402b3e", + "e49a197326a94c87b190f6dd10eeea7819b2bc890e12060ed234afdc5c549f21", + "e4b7acbe1a91e932a5ff62c651e8bbce6f578675db5c9f073bc0f4f595a4ba7e", + "e4c8ffb54e9731092c01d80eb64f15c244287cd8ce2755042e43812f9ae308dc", + "e4eb7f4d1ddbcc4ffb54b2bb8cf2f0a3d25bb5bc69d96ce8d49ca9aaa1346e6e", + "e4fbe2e7051ce629dbf11bec69934aa615a86ce36f050d38530722b082d96f11", + "e4fc8dec6fb6f8ce4464735036575318c30469a8841f3421203662d728c7863d", + "e5560868dd89e19db66b810054778ecfdfb79dbee842c17cd59d5f9259c8e745", + "e57a10a2ec65c306f28602a78f921856b4da3dde464448a376161c9207ad5642", + "e59cfb000b5e37220df6031d4b75f9caa5064d8e250d4c71555ccb917f9f4f53", + "e59db623511c79fa44e3331011355c450de3267fdee43f73a6e1c3be6d535c58", + "e5c3d31414b03e9ebffec47d7e6539abacca63883994e7797d0449e759904e8d", + "e62dc0358bd61478c2d074d0fd346a78581416f6da7f5badf1b14638501df1b3", + "e65253c864af09523087a95d6e93bb100f4d23a89d0cf9e7e149ca7d75796ebb", + "e663a99cbff1eedab7d4eefdf715b20b94288ea6aeea064d3fa5652427a79223", + "e6f99b65c2153acbba6bfceaedb81e6022862e7ca651aab3f2a6dfadd0b50d1e", + "e705fd902403d9cc9857a3cb79418b2309a2a73617d28b1ebfd9afb8affd6903", + "e720b0907b7c559e2ad11daf498bce7e81b90e04f6b7daeb902fcb445315791d", + "e7304b63fa61c5bf44279793451cce774c87dea397e7ebcb2db6415df9b98818", + "e74b7a55a37a76b850f7df1fdfbd4007b58480460e036b16cd2aba06a8170201", + "e74d07886141e7b7f17aacef7846cb3851eca347bbeb9bf9e12115209ea7a89d", + "e77a1c97ce579a6463489cdeb9d291a1deef4981ed63886f079a0cad1b71b24c", + "e7cc860c5ee1d94b6ddb825db86998db190ac87293ce08f59ce1f18720460690", + "e7efe271fb083d4aa855fc07a557b0ded89c5af326a13937164f32d4cd5ef697", + "e7feeb46423d16a543c678b1d9e622de494d4ec17a1d51340eac3b6ad0f8af20", + "e80068e7a72ce50a0bd03b687439a393a446208f30c8e9a042f4987eb1995c8c", + "e81a7edda92f777c9606500d272f71dcaad22047e5043f9760d290a7b9f5c210", + "e8314d44fdc7f77c3837fb3f8a00072e6957542d09afab98e04c0078a54f0de2", + "e86b8058966437966248ca46b07426dfb0a3f053336043edb21475056a20fde7", + "e88c64b97523bd0fcd353a58a965d2d49cb5d8d4ff848a2984bfe33ce0ae52aa", + "e893a900a6014aae7750c958e2ec9b4cff8b442c2d7330cad9ca387b4541587a", + "e8a7b252b4bd1f8edb0ab2702106d8186b92de10c4669bd7c2edc7a6e162c5ef", + "e8e85111e0a93a445476d3004291721ec6f144962efea2a6c2ba30b1aba3569f", + "e93bfc5bc342a1682e229484069d4cde0420f1092212a91dd88b20618aef5f3e", + "e9748c12d086fff748c634103440f42822b4d30d330de751fb7ebd16fbb65fa5", + "e97a1a46b82cf13f9847dfbc47322620100f5113275f800ed2440577f989165f", + "e9a667208b6e17c44ae2a228803f68bc443fd9890863921ba73faa43e0531ed3", + "e9c70d4e8fb45ba63db4e9e130bfe555bebb46a30dac84bc65efcccd5bcf74fc", + "e9d4b30aace05a47643d8a3306b7607b465c9af32f343e6c27e15afbe2ed9ac8", + "ea7322d351765ee91211e726f2a2e71b148614ff858476f2d314faff22537095", + "eaa227c5a58d2552cdc53b0ea798ef846e669c58bf6e1236a9f3ac8df3d5bad7", + "eaa547e3196dbfc5bf312fd6ea2d6860d9ae18fce3872e91d94a259e4146bc55", + "ead919499710acc08c0afeceef3fbf1e194220008ff82f204e997b21f0c4d1ae", + "eaff850268079777284976516c8b4e741a24febc416e9616a9b0ce64d0a39bc2", + "eb078772e8c9314f42cf175c06ec13c2a59fa5156144670cd43b0cac5ff3a7d1", + "eb17893ad911455c681dd6323aa29fba6306391cc375a157a2febb36171a11f9", + "eb5f7c73c14782faf0db5f533399f067787dc7c0042fd8b4dc6117e4b7dd4488", + "eba58ab888533f8846fdc9933d6d36e7325f463e132129ed5e040d70375a2393", + "ebba08751abc0f9c4f5074fae111f889f1cbc096694fdcca5011e5d753d5c630", + "ebed887eb371af718ef561d2d7488b49d834caadd3af8c5a0544b68efb7b7526", + "ebfb11f55ace4943dcd0b4c8c78d8e3316e1d4d22c73aae77880a891d2273e12", + "ec0c3c07db268251f8cdc4756dd27ebdd9a37526cbb44d0cf5a9c1f2ed52ed7c", + "ec4d703860817d225317b3ca6fe99f767d67bf59c14201c1109b89fbc2703fbe", + "ec4de8153a1f4115e7e6eb09ca88b9a38349ac2993c137e62bbf7d638e1e5415", + "ec54410110150cd6c1ebd1d382d132d0627034a939e7b6ef93c2170db5a5ec62", + "ec59c3922f577bac3bb7132377d65a763a014b6cac1269973bef2f8541a63075", + "ec9081187fd745bfa1c3c7f9d05a46149def1200bbc2772183179385df2525fb", + "ec9da9a275700cd08b7f6cc22ad718bf73621761924b7366ddbcbf6a783c85a5", + "ecbb1070a795551da662c74f7809c8ca608ac26e70e1ddca1c73c026143465d7", + "ecbb17cf37f94c9c7b3f89231851aafcbe0925e7e4d7eb96d5ff54427e014aa5", + "ed0b2d5e9d81368ac6029b17c0b2126f43dd2eec9201c76bef086c3a4f0cfb17", + "ed1e78181721b3f5aa283d9ebf81462ef18abe21419874ced31934518045a5f0", + "ed545e76d727586c418f0744d339402358a3aa5540da6a397c659d86a0bc38b5", + "ed616c04d4a39d94cc2066556580c2263bd90b6fe017b8fe86b8ca13f019ae4e", + "ed7f80b722d4c5d6cdee6e8c5c621f956149df086e3723cd4064ba0d5be6838f", + "ed82fe48d398b7d2cf5fd18e30e272c93df532b8c2c1467ebf9ca356c33f909a", + "edc1ef0e343b63c4b1fc0ac740deb67667e878d75a060dd098ed0e06050f649c", + "edc5ed01c736565a07399504b2f911ad7a8b02670da0dc1eeb2ca5a780cf8d09", + "ee3750871ccfbf83bb92d4c931e88d78a6e592eb39409134c1a18c8461722172", + "ee9448da8478284de63644ea593656cd7c9fd513177eb3e58283ba84a8c1079b", + "eec253d9e8deb02ffa7e8a78467d48c67b5e593e36f46b2cb74fd3804823d5bd", + "eed982158217d01a1d5b5c628a6aee7f4aa9758aa44e35ae6b48ae4b0b36bc23", + "eee65bd6251f9ce7acacc87488126d67cca954487a386830d5e4f3304bfd5b70", + "ef0211b2a54bc7071af61659e4a0487f611c1ffb44d722cf8b6a7f47a266559d", + "ef0aeed7749cc3cdf075c187cf83633e54450383a329c6e2c0dd9d9c893d7271", + "ef4c5f83618c0f55d9ec021f80e2c8feed4a3044076ab4825cdec43b091da1d1", + "ef588ad2795460519841155c165a062f282045f733d1ed6d5cb1d548a306fbf1", + "ef8a439b7f1d5e93bc57ef1607c0aeb59f791fe38a8a8280daad941542de8cf8", + "efad264184bd7aa09ef2eb1b312f42ac1954241db19463bb7ce5f07e3b9c7419", + "efd01beab68ff2d4fcb1e4f0286207fcbfb74427295d34575d8ceb391c9c5806", + "efdf1bc52ff67b6eca8933310fe35f6e2d914fe396c9ea7b65cc8eada41c3fa6", + "f021fbaabe1c089409a58f516da03fd2e19c31b074e05be889fca1151e50fe49", + "f037cd9b21c0dbb33b365f4bd160389d80fa5373a748b3dbf9be255f0fe64338", + "f03eaf76ad7edd7350374cba202dc95389556f56bf4eb3c02a32c61551fb8921", + "f04338b6f05255d1a4efc150f3a1a2d5ba1533e874fba4247596b3442faba88f", + "f057e1afc93790016ca44dc2cffdbf67c7d98134488954fe16fca88479c0f422", + "f0611536592f851e763da6c07e4f7b6fe767250d886b1bc1f5ac527edf740d29", + "f06b9f2044daa583f152be142881eb31a0b7caf424e1e2596359a7d2eb00524b", + "f071dd23de9824926063c391a86f4129f95dbaf9034032c70f807bfee403f15a", + "f0750574f2b36357f51ca81b9248c6b2d7154de18878f944a4a438a7d26e26ce", + "f082fca71ee6941d5c60d24a7c7358b281461c8c0a2e2c55e4373f220c7a25b3", + "f0a1e26d253c2e1370acaf11fcd05ada4585b63dc572ea53813e7f0870337be9", + "f0c004c62fcb2c23aece72277c417f7d33656d9d9e0e05187494ef6f29056018", + "f0cc7ffae79b48422c342ed7ee7ab2f9c117c7d9b92102ea48b05a11e47402e7", + "f0cd108d4d1b11a4bed5645b1b295f9ff0699a6c07434ee79fb7764684c53617", + "f0d7c76e820802f368a015e8589209c0b1ff010af501316b0e4a82e7590cf821", + "f124e0aab0ff513e104eb3ba2ff2c974505d10ffd7bfb3087781fca6d1f2c852", + "f12c64024ab41f66b085ec4528a3d78c82c6dd83c61812ecf475a1e0f12f9004", + "f13265389b401334fde5646bc13b7e7c4f8a05c630878615034f77541b1396f3", + "f14f547e4ae201c702c05d04b1024a226baab1f7f20e74559c94c8b5800e0359", + "f167fa0d6df35de3a5ba0b255a2ab45436efeb3b143a7cdaf0e8a6a68ec77f6b", + "f19f0f821f7cddeb39e9fce9ac5bc78d6a24e64275b28b2290779ea1bf99fd19", + "f1b00f410ccaef0d7ae613b337569f38d35d6520b51cfbe13970ba7263367342", + "f1c038bbd103b1de60b0ffa944d940168af7f5cb28ad2f619f58a53c5b63eacc", + "f23baa5741bccea2fb140328e4e00ab0f5cb1e1e424701bcd5a4c4be33f6e098", + "f23e67c6b6e6c08a3e110eaa2331c1068330ed9eea335a54606d1d2e156ed26f", + "f25aed755465da7d5c3b4e3c95a25600cc4bc635e9640e7ef993d1c38040b344", + "f2669e86800b3022fd2a15123b693b94b45af55681b255b85a85dbada47af1da", + "f2a404f19cfe1b22fe2af0d23235a5f7146d784a3b1a0692d6bb8bdb3044b552", + "f2b1b88aa102690a9c394f9214a04cfddc9c153861459fb689f46bcc63f8f4e2", + "f2b5f26abaeded0cdbbbd330a97f60a2cb3af54e0e4014f0cb1c0251a2989be3", + "f2f5a1978daeeaf25c8ffa085e09762e27630f6001497d9ad2be7923124031a3", + "f3aae264c5324b53d7c9dfa3c93a217cc9acb8b59786172b2436713fb7bf04db", + "f3acee3ba45918475e3cb3f57aa38f918024778585bb85d7ad778e5e8ff12046", + "f3b1ea2e2365db574a8289dfbed5fb65424c09de311a0297f2229dcc65e52243", + "f46ad557fe453b456d94d7591a9482f084e3c502e334a445ca0cfab1d465c943", + "f46bed6638a9631ec01f7450a8edffdfd1cd597efa1e3c18a7b971ec9b56d5c4", + "f4826ae6b3ed964c98c870304a5985692a6401ad5d2bd799fbf648f3e455a44b", + "f48b5f5117b250997fe6c64af116867753b003ba870eb9e70cc511a7d6cc8830", + "f4b1501caa6b19f21b015ab1cfb44af7a77d9cf11436e9b8e74ed73802fd4274", + "f4d2e4d7dd527388994c2677651db31a41e812bde44eef9c6683a97ba479129c", + "f4ee92a108e08b0836423cb6772b501eb452a4ae5400192dc88bce99518f91c7", + "f4f047a3a53e130dee3f20a27f71637e3b13c6437223c8a641b6b4a9f6df27ef", + "f4f81e3dda331425e5f951652c77ee7deda8d3fb5cbc5a2bc1e9ea53db2b9320", + "f51a262120089fda895a9f80697ca8b686e8c057f19ee736daf479b7826271d5", + "f55b877df863c6b4a5a3e233892e11c00368a8a49ffd2dfd7a1feada89702cd1", + "f573ff54d3b07aaf198fef31cf632c82a409ed59eba85845ff96b0367d964b1e", + "f5750ff4545c2651debea6d7c4464bb4d92721eae8fbecea7c011af94b084fb5", + "f58cceafea23ddd305ba490e7f8cdd8ae1df5b02f66e07bd698b2d1353ec578d", + "f5939607dc859b81f8b9eeda2fbb4bf58e9ac3f3f259f2141284fa44fe350cc7", + "f5deaac65043d57651d7e4530e62761b8014a54771fb2fed110eb0b30e86f3d3", + "f6041f907c89e38602fc4f723a7ba880ba6ba6293494427f433675fc139c5740", + "f63d5f0e4107f691719d2c7e1b0aa0fa40a7fd868acce8040031abdc046fe06d", + "f63f72c986c0997616af03a1c7a6a2a01ce412229e7cd8f2592de4b60789a350", + "f65732cbd65b8f3f8b4b3257ae9be69f496d828b288a643c54070577e23694c8", + "f65b8d423e8c5a5869491f91860120e2ad932f87aa646d51ccc5c97e478e8b74", + "f67204bc7998220c42535161b3da945d9522fb4f66d804b7bda97391d18ed41e", + "f67a153cc00c83debd8241b7b91eb4ebc6efda81127a3143614b4ec202362dbe", + "f67c79723180993631bcd816141ea11c780d74ce1f167dfd20d64eee34020ee3", + "f68e1fb5c0a10c18c1b436659a676060f2ff347549074643cc376516187d5e18", + "f6b57a0cddb03aed35a73ccd10e928ba35eec1f593614a9621e383a15a4bbc59", + "f6fe4d7f7d890ab3e96cd6deecace45ccf2b7703dfe4d7cbd19fc38bf96a4fd4", + "f72103979549dc42fd342428606c57e0056e1ad7aa6367c9ad53b6eae3ba84b1", + "f73acce0d0caaee1fbb643dfb565a921a94617a3349d93a8469b9760fb7d2dd2", + "f77aeecf0782ee72b26dbd0bc2afdd9474c7a16608b383e35e8e7247ca1719ca", + "f7869b4843638e7f3dfae398dda8e0a7560cd9c5aee80270a5267ae8866cf734", + "f7897905419d70b23db05fc0a1001d0b1a3b0af0f832117479edb4e0143bf2fd", + "f78b1994b24b4fdae392f7b46573346d468ed74676c110875db6860536411972", + "f792dd2b474132905f755c2f31fdc6d04573311b6838f293d104ecbba7ef9bf1", + "f7a7d978a948f36568666db2f266897d1d6db97c881afa967813a1d90c861ed0", + "f7ac2ef367d2595d4316d92f2f5a1e0b3e3ef0c52b13bc6903ab626e0768daef", + "f7c9b0a8be9f9fccfb26c2cd67badd6c4baae6fa4720cb9b9ce32718d0d42ce0", + "f7d15ee450b14cfad3a63d8a40a182d6cf2d729e5427d1bd43b1d514fdb2ac18", + "f844dac31b49b7e2be2afc4507c6835c6275992bed428d5c2811406eb0636ffd", + "f8856132a7ee942f9935d819d8bd1ac2e8db73d3ad1e3504231ec44eac8b4b2a", + "f88d7b87fbb90b5eed6107d4ca238347cbf7bd9960d11e6dbd6202336ba47a48", + "f8bff3207a9c4d7acba4b30d1fe9159776e6dc1cfb0a684cc73660264474adcd", + "f8df2bc65afbba35fc3a1f266114766feec630ea2a193325a0610fdde899e85b", + "f8e8e4500b2f9425a8c19279d62346ba5015130418ab023656960a73308e39c8", + "f8f3be4f9c461671164909f3a6a4360a0a18664a5783a29b2f869345d28689ba", + "f9044f70f67651a5074910319f059bd29ba17d5aaa643cd022af04092ec5b8fb", + "f9373e58869e091a50e8fca190323e20ad36dad195a8bf0aafb84d405ffb41e6", + "f9426611ad2ed31b5fab523266149e5a7076e35454e8b4b21b69761164b56c46", + "f952fb8637baad559eb2a8d8541a65dbdd0453b111a4f459b15c12e741274545", + "f963862beab529f79ec7450689199006366b9f9eb602dcf88829b30549b0dff9", + "f9b818c580b690669d02210dfef753cf6d8cdeee736dd43edf3e3873be4f3593", + "f9bf5accfa2f7af0cd3c16bc75bdbecfad15455e969c43a2386ea5e0dc164b76", + "f9fd406f361fbc59236ff4e5db9fb03bf34b0d52a314e50b9a06463060c39b9a", + "fa03e828df8c0572b8a53b3ede30b1093736785e63a68fbebcab58688c8d25fd", + "fa03f44e759f0f2b45d4ae60b6bdf1f883c7b28c8e584f4c9dbdfd0da8acf021", + "fa2d7063043b250978556d2efd5c69280ac146530785e68eded6c8c4a2272f2a", + "fa3375c4d494fbde579c72af92d9b836e215537068065aa024adda70b3e2fb87", + "fa44d75d52a573658e4fb168bff6591313105d2b215ec101d87c8003e5ae1ee3", + "fa601c703bdf0f05901fda4215366afbd6b8da68a13a1acb1ae5a7d0ca551dad", + "fa86d37d13aa868ffe88887c502f54675183fb72a6bf0f0eda73353cc93a6f1c", + "fad28332415da4ed66e0a02710033fae4e7974bd895649061f6fe1447aa48eb7", + "faeecb415d0620338cffc4971adac667836da581e69d97a32e8cff554889bb23", + "fb0c91e2f07bb12e03c91114d97b28f384dacc22f55c9c2534fbb5341594ad79", + "fb1c4b9dce86b632b78ffea1501e361fcff7a4d914c99eeb5646a1483c17a75a", + "fb1ca8b33a54a6968735c70926614093f25f1485371b0b5c94a50866f1561c02", + "fb4006f12e6539562628afbdffc1114c6109fcef4021e01a31272d54f7973643", + "fb61451de49728bbd653e2fb66435c9d7551b5716e4d795fb8ce5de400c31628", + "fb63b0298dea67ca91f29157c90022fdd2f6b692c615cd54213b93b435c04a68", + "fb87ea8200382d0ee7b1a4ddf3ca0d7068d3f3967fbc1cdab44b22301b06a094", + "fbaa467be2afcb06c9ee22987e7ea16fa3e79c192296b6c4129f07c8f60714be", + "fbb08add215e5c4520b18356ee5e2cdcfb7397916a93d3df4d9e0dd0232af675", + "fbb5155bf20f9145c10c42fe56c7cf1d1c6b7a8d5f7c748184a431c101113f0b", + "fbcbd4a4a21904d22d4d54896b1429e206a079dce30c8eca0933af05232a14a4", + "fbfd191d086e9d8c6ca4a41d248079bd285124897f997b424966b625f3a67a1a", + "fc31a936db1402433f3384dfdab6f361b46d3556e4c7d95f52bb844018d491b1", + "fc4ddb1b80c7669514ad1ff5d62406a0fe7e8688782d5189a850dcff081340d0", + "fc6c4e67ebdb78dcb98619014a4419b074e164bd5e992991575dfd2a66291e4b", + "fc72d6536dfdeb226151f1905bd74a43e8f04d2e7c4b12c36f6c3433c50d0ce7", + "fc7dd06302e49c42e86e4e8a4e9e093e918964e38a9e35ce206109c4b92b5391", + "fc8a7c5bbd30a4fe7b08b44f2057a67dfcf5588a1ffc249d405f6bf9756da974", + "fcc5e54f7c00f05f6f78b3cf6c1daeafcd911a02240265d44cbc2b853d6a5ab3", + "fcc6d55972e8e70cf5ea47dab7fc1b7efa3f4dc786c5a33ba46e4ae892947a72", + "fce6f3c6682f3b46233f31f7424fe98ca8855762fc84632c4019464a3c06caa5", + "fd057104993516baf2087ccaf9a5f36fb95373e33a94a82810c7a7ad048306f9", + "fd18b02e97f3f2538c2c21b3d618c0f0d47678fdd8494ad14a0fcc5ad1bd95f5", + "fd2c9a9f34990dbe30ab307f0950e8a255c1d575db5e24ca54618ecd92be3263", + "fd2debd3f338a433e73fa8321ffd1581b173f088dea2590a4850810f55616bf5", + "fd44d05b0be2cf03d5487ef0eacdbb3b76e5c00fd547db7623d7d26cda7fcd3f", + "fd4bc5136625be7774541ac903c7c4f174833235a42a9853adfd8371097f6fc1", + "fd4ee2313d81fb52f0930d955c668ed97fe3b85bf1a1c92c6d28c7253c84fab7", + "fd68aad1a332f4bca84e4a4c75d469604e8f51a2bb6e02f734349ba870efac86", + "fda15d0579879a85bf0495c5862956f75fb23e793ba45977c389721b079c6c77", + "fda896bad99a7e6f63b12edb5586f732b73ba6187afddea5b9f2c1b9deec86cb", + "fdc4083896df70525d6cb8838de7de6e510385561045aae9b6a11d9eb7174957", + "fddf26cb81896c40e3033f1d43c911b706fb58b681757d1c607e5cd8ba997df5", + "fde50489ec700e54054b2c5a63ee1b596e1dd9d25e1178b9d97bda5cadfb99c0", + "fe0d45c84237f46dbcd90a2d90b7d7103a5b1ae79d9641dfaa7e7b5d7c155382", + "fe2db61e327fde7d349bff423eedb5664b2d8d268a1050d50d700e44a4a4ec84", + "fe45796d92ed2ffa9edadee0a8ed0baf06a01ea06f7e2b3b280a3660399707a0", + "fe4834ee67e3dc86f371db6861ab5514c8451744449c22c80ddfe0855b2441a4", + "fe5b72d2a8a29732c5307f43f7212a492ff5720c9eec8e98b31c01b57293e447", + "fe7e015b2a149f7e3addde43798f0860be1f92ef8e0beb0032e9282e858f1a2d", + "fe7e12db4ca7b495ab3de2b8e812ea23219baf38291e7a5836944eee383c1362", + "fe860d68dbfdccebe089b378ebc4e3beaea70d1a8b24d2334c47799a5edff5a7", + "fea5a8461835b1382a1110a77224df1ce8a067bed704f1a6076904647f068b1d", + "fef252c6261533f1b9784314052f8e1e48ac6bf10dfc302c24b718ea18b32274", + "ff22d9c91fd4d74876b2b41c3a653963d348c8f2040039d5bd0af665a5b59af5", + "ff3168092a52292e5e43e7db6331bc539b4b37f1df12d19b1c616ff3c2dc0ea0", + "ff3542e42ba107e85406f63a9d670e4d53f34d98f13b7e53baa3de8f44b05879", + "ff3f03ddfca7aae413d0716ae39a988d7964e752108c348a4a4423616d85f81a", + "ff55ea16c630a7b728e6616bb5427410845da8046007564c6258d7d02b986859", + "ff6427309ced42e9287b4540496c6ede902122ee4ca6763c4267c75792f11ef5", + "ff65174fde7ad22bd4838e4c6a501c904ea8406aaa9a233f1f15b266abb8e1a1", + "ff7e5550a6afa9804c92139375dbd8bdb6930b456f9266869cd42c7384cbc6d2", + "ff88a6da61a4a0fc99ea252508a180d48600f13212246fe266b5a02b64ac14d3", + "ffc2588c1509c663c1406c1494a663b87978bc0c3620feedae2857f5c22ca4d7" + ], + "include_outcomes": true, + "label_counts": { + "negative": 24912, + "neutral": 6636, + "positive": 1156 + }, + "leakage_contract": { + "deployment_must_not_read_outcomes": false, + "nontrain_outcomes_are_evaluator_only": true, + "train_split_only_for_retrieval": true + }, + "max_flat_action_dim": 112, + "num_groups_exported": 2044, + "num_groups_scanned": 2873, + "num_groups_skipped_by_split": 829, + "num_rows": 32704, + "object_embedding_content_hash": "fe97ac428d6b0ed4da7060eb6f697c1392b4a8a09988a10c0f633b1e457bd568", + "object_embedding_manifest": { + "dataset": "embeddings", + "dim": 64, + "extractor": "rgb_object_layout_v1", + "num_embeddings": 2044, + "path": "object_embeddings_rgb_layout.npz", + "reads_outcomes": false + }, + "observation_embedding_manifest": { + "dataset": "embeddings", + "dim": 32, + "extractor": "rgb_jpeg_stats_v1", + "num_embeddings": 2044, + "path": "observation_embeddings_rgb_stats.npz", + "reads_outcomes": false + }, + "retrieval_index_allowed": true, + "schema_version": 1, + "shard_content_hashes": { + "charts_00000.npz": "43e04927549fe3aadafe63067bb82c16c1167c54d0c8ebd47d4114a814f338f1" + }, + "shards": [ + { + "num_rows": 32704, + "path": "charts_00000.npz", + "sha256": "6f3211ef91cac305045b540e71fa40156ee697abff9db3c8f994149a2fc1b663" + } + ], + "skip_counts": {}, + "split": "train", + "split_fractions": { + "test": 0.15, + "train": 0.7, + "val": 0.15 + }, + "split_hash": "867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372", + "split_policy": "stable-hash", + "split_seed": 0, + "state_hashes": [ + "000b25ed5631161d0436064f18bf2927ebfd97feec3b027cb7a3fc7f802faf32", + "005c6ef90461f4461053d2f248598e4dc200fdb34d95079ba9b6ee9cdc796282", + "006bca42e3c79ccd3a2cfe015bf7c713f69f779e9f9b6e05e6d60e05adc1ef2d", + "00889249a42766e52451f1930d859b4e9c6a18eb2d2a23e4a466065e1ec6cd8f", + "0091b5c64f92215ad5e12cb639483bcb463e589fb1442461b3b7f2e1f99bd18f", + "00a20929cfedf805498d877fc9a08bf4e18093022ea54d5e205ecd22b0aff85f", + "00bf989afb844173147aa217cda23c2d540be3549b8d54cee51524018f0886d4", + "00c9e85ebf618703709c8e915feb24d5f9fc6ef4b1ba77a7fb4bbd07f931b588", + "00f7d5c91383370fa6a6cf7e0da5227d76d4148a8d74c28136bb2e2c32e9d5b7", + "010ae081b71f7c0e58868fcb7737f45102882974c383ff4d8c1b02ce1d895661", + "0117fdd7ec26170b0745e65b08ea88008b09508b62e77d62a69473f535c25715", + "012a634e5771527516fd013df470d5b27c8a795d224f217fa9655c378cbdc6ae", + "0147c0991e5aec13d38c9a12625b9824d5e7a497734845173f31878b6daf3369", + "01642e4ace471a0ee60cd4a110c921359d11750c72c4a4a17274dc285f8de946", + "016859c711e2d334a59ec1acb53b154655e7875d1c095191ac681a0d6f2d459b", + "01a740cfda480c25d931fd14305d6b661829ab5ebeb80ab3efb4bc1b7d400958", + "01cabb034d28efce276c7c37f7c4d1b3fa2c9d00b140b109d619358e1a99ac3d", + "01cc39c8b7c4511f037106757c86d8a9147e5306a4d30e1bae3b41bde0455895", + "01d46a34d74e86012c60ccbcd0cf56dbba216277d41a455b8f04dd1367dbf373", + "01d93a2edcd9393b7d5f2543c37638ee18a7f731976a7f58e144de69747df1fd", + "01f9e4c4ee47d37a28ff9581e3c433583b0b46f8edea284260559029e7c04bf4", + "020d8719f66fdc0b0c47e07ef57b3ac2a06173c2c743a699b972fe3996758e9a", + "02251ce01393ea7b7a290137bfe09e465ff090cee97d821901fb100038b3fb16", + "0251eba2dbfe3f1be9cbbb27321cef00827b19394ac802bde83bcce72d7be6df", + "02752960d18784921c0f4b6022bb7a034bce292b03909ebbd0353c12d48ab314", + "02860b143f2e95695756174fceb8ddb331e7510aa0454f124250edb7863fd5c5", + "02af5467f98ae64c370ed93e1b882b5259adf7e1a41a00e36d0b0a64bb6ef756", + "02df6abd276ead475e81d9a264ff6c4187b0505a8c8d36e0300ee9fde4d4c731", + "031cb9954afaa630fd3a550eb9641912cb237d06db40d1acd81a7f6cc7ebdc31", + "032090600396eef932cda799b7e4ca43ec002031b155583d41ddf5f884b0e066", + "03422e256c0166427e92805dd77424b2025b7be0dfa3d2cacdd17e844c91a80a", + "035290871ce5028fce982cb3000b928e26a7daddacfa891677740136e5f415c6", + "0393682c39f9ad90feafe5ccadfe0e5cd11db16f15286b404bc4ff620d0a9164", + "04016ddff5f00fa3ef76c2246256ff48d5c7bf896cb3563f358e0ca9b5cad5d4", + "04042fe5cd41177bde33eb3dfc7c022d68426c8bce2b386b6c3d744b3b6c8ba9", + "0413d5ba033ddb1d8be7f3fd905b3a3fea63495b9e2bc6636af73a0cb799b926", + "04463ccaaca5d28ca99f63f4f53e070efa4ad2299568bd63a9105702336b5ab6", + "04553237c84ff09a6543e606ab9df00f3fdd3359b381e9cfcecb9f0cf8f66c58", + "04736afd75d9b70defe4c540c6b91495c1d483916acab8b9cb88954faaf5d7fb", + "04891991eaac4f5fe2628582b59594f435c2f81d561f2c42ec1dd98ea57408b9", + "04980e08e9f08128a95235e4570f5c3b26e9900930f8e1901b1cc805055d835b", + "04e020a7f9b5a6b03c4d0cf02f1a5226b30f77918560b372a4a86342886926b3", + "04f06ba196bb7367e0483cf316dadeaec8bee82115844252a7139a70a7c202a3", + "050edccb9a8c424bb0da9dbe686f19e78196f374d0a0a6d8f47b92cb2f8766fe", + "05165e6acce39716961beb3bb0ee00c1158ebf8fa3658dc70048d10cd99093cf", + "0518bcd40edaebdb67c915e851d4938473f26ce6dca50a21e69f83fc32288c89", + "054a1bb649b430a86f7802a9457cc6a83d15cd5b8ab6bf8adf410843ce7aa13b", + "05550c96fe8ba235ebbc76dd6570b63df19358382aafb3937726a87d268713b7", + "0583a22e745ea93e01a9d2f9b53642289d9a05b222a73c0fecf8e710fa1b59e7", + "05b5cf5f903d65c126181a2bf230af182068934b33d8184849b6b8981f68ebdb", + "05c6d0e8769e7b9de479e7e02edf2eeda58b5ebddebf2e1f15c962e257d49e40", + "0602af5175c035dfbe398a4014adbeedfe194d75ed41a0b7eb8a79b48ff6bccf", + "060d8bf103ac5fde82e3703ca8e2578b4f8c7bb7c5cb738f5d1209b919c61dba", + "06257bd3fa90f0afcecc239680999406ab79980eb303d45a2d1fafea670e4a38", + "069f55a960c8724588c1ffd62afe2fd0bd83efd2e34cf6a5168032643e6352f2", + "06eb82c273f65f490d85d60a39406f6cf2318d14ee9596ecb1cf7fcc3ac9f8da", + "0743154e958abc52c160fce6065de25d968ab60c7c7a536d400d2295bfc27255", + "079b15a218441cffd25d9f3aa501b9143a3292bb7005f9eae05704e55f3b38ec", + "07de76536f57dde25b6235024f5a41e215d6a13d9f823e2ef50e4e84c87fb739", + "07f15fafa5887d8328b676cfb87c6f8f7dcae9cb187b8b6969e2f3a4ef37e8d0", + "083bb24051300e75dbd52e0202170794bc727994f278922c2d158af777a3f2cd", + "086bbd948427f350f1d964b288a729ae80da38291e409fb7e0f0c688d99562d8", + "08713adb8a3215938b380fbc0c63f235e063847962fe02dca3f6d20113fea1cb", + "088e20d6cc294aded49228f0ac9a7191e7cc9fb848bc19a6ec9d03fb7258748b", + "0890c32a99d90761a2f31733eedbd91b1980913ef735752d889c0710fe572764", + "08b122b9a1c8a40b5965898017846a28d6377690db7128b7b31417f45d1e407b", + "08b58c89333a6a06c7714c86704b3f06bd2ad23a18360ca5e8ac7a2bde0649ce", + "08c6791c2cfa2927363fe34b6af41498531aa6816ad3edd85c3f9835a1ac1014", + "08db846838425971e9024677f1cc8e070c502bcbeabaf05f6f3631dbed7a3ed6", + "090f2f8b58a9dc0ae63008a8fac34a18b2aa93522542bb472a1202fa81e69465", + "091953502d8a89e9db47809dba001d5756c622e3aba8318722f8da0054bf92f5", + "092e168f994875a90490b287f3aeb7ad7ec27428bfd07d0e1ec1645dd2e01af8", + "093675a243ffec08a1c179a6757e916d5cd8dd9cc46921469e62bbf3bc12fba1", + "0947a97e05cd96908d88dd2db1de6be0175c37c2a635e5506ab0ef60723e9244", + "09a45c5c82ae74f3fa1dbfdd04e9c9cbb68ed2af9f96dbf6cbe731b22c5f00c6", + "09f192983656d6b34d005d8824dd443c5aa2a86212e1ef363d3ca398a61127bc", + "0a2105ebcf3307835abd4a7f5a7ea78b1b8061a118837a41879beec8ac9c4333", + "0a39ffbc1c4821c186d48f34e505d1dd889f98280e19b2967c7296fdbd4686d9", + "0a7a2bb9c6b1a50d1a9522163191dde71a243bd834f093cefb2ff1481dc637bc", + "0a7da33713c8db7bd3bc44ae27bb34e21ee7301152b2bb9d76626ec934e57f5a", + "0a994de647582a2ed0d6ce05c5ee895783f3d0a242425bd51bce5a205f5e0c58", + "0aacc576c7717acf5a36d239d777b8220079ff790301e35589802e502b65cc6e", + "0ab3580b189b821099bf7214360927a818a8e415f10d4ee7006a7d6ed518446f", + "0aec7bce1ce582391a1644d3411acad983fd582a91af97f199631e487d483d9a", + "0afa74ff678a87361267b42d66b59fab3466e8dbf8b95a5f891c25ee4a0a6ee5", + "0b034ef0b1596741e3285665c5bab78be118eeca05fac3b0685c0e405e60b01a", + "0b856ee1f7054bfe985a8a4929618c2f8c241b97d548c3ba2edd0ce3c8636aba", + "0b8bf874669ae24a086caf5364a73cc92e86e1baa7da60ddd21cc9dc58cffe54", + "0bedc58061fd753c26bcf63e803d240be5e7f5a044b0628b5d5e456b35345fad", + "0bee130933c311df93b7e72ecb227b6894a91763ffc616339264184411326529", + "0c0ab38d6a39a42e4cbcecdc53231d83e1b1e645f5b24b1a6fdc8ed2213d108b", + "0c3ffc0a5ed70915980abb6a49e16622554ee49a5730d559dddfab9115b6ab85", + "0c7a98ae4b33c672b70b431ab840bf764156ad0a3143c8df83ae1af6ffea6551", + "0c95367a82ef1977f91b7bdece3b7d452ac5947ac67445d20680d4324e6d70e3", + "0cb834d2ef4c99da95a871846baf987a939a014b4cca2f156974445a21c89196", + "0cc76f5c6a58bb1c614293fa3e9294f64fd03eb893cbd05576a9e93ee282ae61", + "0cdbd10f984a65865896ad1dc53188dacf141bf7ef001a0762752abc4207d819", + "0cf838444d8848a8b327d97bc22b1e3ecf4c14bffb554ebe9dbcccf35078d516", + "0d2cf77cb168280d8d9d3b9ca4d212baf17223c4da3f1f0dbedaf7d95bbfb42b", + "0d31cf5b5687ceab9361200aff34004ff93663803b4644b27d1d959f457d6090", + "0d77208f3083852c88c631f5a4102ad7684253ee7a8f2ecb5105725de98a9f22", + "0d84d509f5df6bc0a68efbeece7432b092fa13b52609b417755a8db67632970d", + "0d95a334465d12b48b4882fcb17e7bcf3c9c1e4dc69e753f044d91f7525654e0", + "0dc074e26e6b013f64ab01a46d120a9f9ddcffb756692e9dd7105039c88af908", + "0de8d03b14fedb9561b9015c19f8ecaf855b222a2423b0fc9a502374ce4a6c2a", + "0e0cd8063d131f6694b924dbd28dc10dfaf98b2f7df5ff6bb011ab411cc528ef", + "0e33d93a2b330cc4879f36904e581a044e9934fe4c2f28734201782301bb9a40", + "0e55cd1f05f691f5f7316ced762325ce7b77786edb3e9e65486d53c6e34d123a", + "0ea0a077924dd4cecbe0a3cf6e56b97055a69138eabc98f73d668cdc55a8d342", + "0f5b30575f5d8d9fbb772dde91794d51ae97b2f4a84b8986885d9ce9982bd455", + "0f60d64a0247253787e778e9e05d247a67c0c1e6ad497d8d596c13a8515015f1", + "0f76c338852c84ba09db812d81c87051f18e3a4175f866f1748dcbf8051d58b3", + "0f87489e07b3fd86800dc412d9dba47d153dca7d2a2bf1a2bd499bdf2c6ad911", + "0f9b2fb89c06a7b3f998a4e878ac7359a60ddb43ce1d9d46c5298eb2ce78ae10", + "0fd61f330351d6e64c0b9d1e6e17e73760f6e839bce6dfd46aa8958b0a00e0d7", + "101dfa78c8e779ffd065d635677a806d1220e5d870d64ac8bd95376840c9e7c0", + "101f400f3a1a701669887d3522d66c35e1dd66e64a55542c144f74dd4c7228a1", + "101f8a873e2ff5f5f2422c858ee11adcadb2eb241c8e329c5ca8d6b156feca65", + "102bd69c2a0b6d60b3e093cef71d13b91187ab631524b3fde873a75a795f82a1", + "1030bb1d5219a2c2d7b4560f0b8fed5a0b7645b178bf2a25224f3e8a1620a5e4", + "1047bd5df24c19ad178a6ba3b0e1108087b995bcf277520efa1ba27511aca823", + "1057678ed417e6ca1ca396a9143da3da3c7a44d7824c00178707a929a980993b", + "105fb9809fe0bc910f4fa8fb60b5a9bca2a517ed9781d2c99525cef6bd7a7c33", + "1076382ed2d4fc79f71805538631478e0cbb956e3884829ea3582c5ee8511b31", + "109679dd2218dfbb038bcb415aa96ae23b53c35a1a9cf156656633fd33b12051", + "109be5cf5ff9399f14ee012b0e509b867c597216a88f62b615f0fa4f79cb313d", + "10e050af1016ad6490f688299853f4c6a26b49fc52e36d8e84bb16b1d6242f63", + "1106d1fdacb747420db88c70e23d0522e849e8ec815670c0ff7c826e4a75d53f", + "111c5243a9942e1524b878a9931a8bab82b9f22dcb1a804a118b7d5745e1c557", + "1132f5177070b804b78222ed58670c88c119a10914c7cd820a71b18061fccc4c", + "113618a22be4bee42ee2bb850f8ccc18363063f6b3cdc8a30a8ca58b9b39fd3e", + "1142609193b022cf38eec3ec0870d6c51b2c09eaaf6138a8d55deb11a6c5f22d", + "11450674cb59ca3534c525d225161770cb93022c728f4835a7f52574e959525d", + "114707d425d986244f3e56284f7bd171563cf48974a42fd7a9280980c67a3f95", + "11773ee6fd7210cf7c79880cb39fa7cf7ab2829df19080e82cad70c285ae4268", + "11b4798cc26f0599eecdc28b4d71317e23c4acf33bb37093c50a39a188631945", + "11b56f2db4d76ba9adf943bc17191c2cb6e5a85d85a1208593cb7c1be386ed44", + "1209427eab050c56945e46d87c5c97c8037c55ddae6e4f0aa97a12c04b38226d", + "1233ac6385a62a731f93e267599b5c5169646ecc236c05196bb6256a4601e223", + "12a42cd1df46b7dcdbe7e2b9a63a24bf047a07cb453c4064a2f92be8e58e7a95", + "12b0cf2d082125418884cb235be8f0cf114690e28e8e68ff8de681b5a0a801a0", + "13149b1fd7056e439febeb35a6867dcbb149e9081d37fa40e83b1f2918ea5e6a", + "131b8be76b5625f24c2b90ef450e09681b8105c48bb24612d260f7abe159b85d", + "133e83462cc2534113fba38b5377d44f087fd0aa62fb17119a2d775c4d36ae13", + "135c96d100c2bbb210c05b0fd0ef4847d85bce2f7db80ff53a6db6a34388ecb8", + "13600885db220c1356adecf1f18bea3b6548495d5a2dd92685a3405e6dc32601", + "139065da5be130bbc91483d942c99736367a5b8e88fed0352cea387a7a8dd805", + "13ea6dc51f604aa5829da023023854a2495048e1f0760c9b3693d1b6d6cbc295", + "140b360113e9fb4ba876e619c0f710422a72e52423a3a5c80a7ed758f1681024", + "1443c885727a11619a79c4d8beb93b4f70e2eb0d682dfd1cf245d1e66fa67beb", + "145fb05e3a2ae5ce368a34f6fcd96b7ef31dbf82c770652fdb4a93e1309f5cad", + "146051089e993bfdffdd37eaaa68c9bede30409ab45c27b17c19c9bdfdfcc447", + "1480b5f9e56ed421dfa2682a888b5340df9e3417b2bf8c26eb628970dec5a891", + "14a571926fb18de1ad06566225635e5562408c6b1e2f78c46b959033de658e70", + "14b5b48bc2a11d15c666669958c19a316df7da7ae93585e4fc29782003f22093", + "14eb14bc234531012f5a646c2481f6f876e53d1549adc80ef13030c6ca20ba62", + "150e06f72127d24afa4e0520537a06b3c6948119d290c73c8bce75462e52b706", + "151401539a88f6fd7ef8230cac5454f2ba76c8e3413bbb677dafdf138979eadf", + "152b07d8b0784d39ddbc1d8033cb660d667cda442b82103cf2cee4bc344b62e1", + "1589857b2270441a120ba8e1402c97cb04438de24c09d023d2255f6916f012e7", + "15c85017940438224999372bf58525a0aa05fae056b9b62153189e1b19774ea7", + "15d9ef11aa04e54d29e0e84e71fe641dc54224f835c497ea74c18a6c2f710e53", + "1627d9af029612789104cebb56c6f6ca76795364d6ea2d2bb6eb4911ae636d1d", + "1652f6d3ba2e0a8818b2449312190f6e266553fe636d1f9ab834dd7984a01386", + "167ef625245b0fba78daf7d0bbbc5e6fe0aefaa9dbe1e57dac61fa631b475138", + "16946e5f2cb7e92c9b2e2be8f3e6e03e7c4a7b884986bd8db1b427600a1e6591", + "16a262c3f0d6c6f5df9156d5b413d2ac9c076056df7e3b059c0bacfd32a02c74", + "16ed36710a71acf4e83a0c735591a8d8c90197e92a38efcc8f5ecf64a1868ec0", + "16ef1975b3bdb478bc236271efdc9cffab09408902a5f97ce16f5d71136edb63", + "1705a03558836eb72c3179c8c3f78886201b56975c7b1ee7ef61a9cadeb4a57b", + "1711dc5e3e5570e0e338a8747c3c62bcafd93fb89e20f4596ccdf0da8c1a31ce", + "17218e8f2c3254089caec4db721323a772301788f1ae0e1d540fb82101af6777", + "1729956f439d4baafa124ad35906905bce52a38cee105d0384caf2bf4b1b8c16", + "1748ee905ebf6e5544ca8a20a2dc34e69272edfde8c325294b5a6e7feaed3445", + "174d8e6fd793f2b7032fbf0d980e5f4d8bd766c85f9873782481450c8baf2078", + "178a0f21298df6295ec0adc89eed35c2c7b0010375ade0b3ea639cb06e4c7c34", + "179c1cca46f3cb63e3f75c021f41dee206bc6a554eed8b17a49522eeb93db65d", + "179db1683a5d69775c9698822868220d9908f07495335ad2473ab81fd62943b6", + "17b9ff51808a80a062bbbf618fd89a0d21a707ddfeadc23243cb82a6aa2f9241", + "17bdbd2ced1b33326fd60ffe6f5af3a474d9323baf21761a637897088221a48f", + "17caa6df22d23dbeeba5faf91e1d00809f676ec09c65139d493616baa2d814c7", + "17cb801862c13d2966d77f9841a7a3e020ec4eaf3b76f14adb6aa1816047b308", + "17e6e2f1435e3d01d6654206b99090f419461dbfb37993d5a7fc128169279cfa", + "180c43a6c0de8a43420887064895b3aa42bc535816446275fee6a6a6af523c30", + "184ce76e1968d8a3369ccd694d266187a1189d85515eef1759b0c1c6f784d838", + "1859ef068a39acc92b320dadbb60f49dcec9bb2799dbc8c6f25f54151240f0ff", + "18ab98247de6b45481ea7b0934e111b3fb070f46173f5f586bb9d45f4cd3c700", + "18c879052385c74614d1a44ec0ca146ff4413a852dc5ed816189f45047c61ce5", + "18df2522938daae00fc6bf415f08a5503d5d4b600b060b5e514628b10a8363dc", + "18e88569169f27453c3a17a88edbe98bddf953e9022234f2e5a0bcba4849d111", + "18fff60f2e7239cd15e46ca71f35847df2b47d7e499474cdd1d3512894750bbd", + "191f38fba735695d000d80a4521af73f630fea434a5676d87d1b27c544bd10cc", + "193ddaf83e3bfa7913045010dc99c78a7900f686f81bb1a64c72c8402131b494", + "194d8b0abc3141bb77465d1a491f4c23166105fef6a0ec0b46dbab0fa0b27a97", + "197409c1266c07fe3c0307fa9dbc526ccee630d158390c766a97a8b3b521d16a", + "19abf2d41c0502ea57f2d87647da2fd9096befc28cb60052e5c1f9a03815c5fc", + "19b6783b167f943186601698d9096375a33996a58185b4fd8f6a1ba54222b021", + "19de583fa8e8a4aa2ba3b2362b6f4a4d5139dd4c45718c1b6a13f34cf5614017", + "19e44efb9c3709c75976aa81dae9336fabd9f3a16efd14724881cb3730964579", + "19f460b02fce9592cf785408a8e17e51bf35598a9f123262c56d9cbf144a1aa0", + "19fa18cc19f1a47e94f37519208bb6dcefc4b91981884f101182e1025c76433f", + "1a296224e9477d2b02c8f3f9abdb975a926f5b16f09a262c00a1db29c7e29127", + "1a5a2327dbf13f4d8062a3a676c7c3be2acc4469f771f49f385b6f608303f885", + "1a5ca891c3e8863c73d87d409285913f3ff31d7d816ff7d769b18424406d0b72", + "1a6fe7be0ae3c5a1eddbdea9cdb59ffec5cbe937bad1daea304c4a4c33ffdfc8", + "1a770631048156b7cb85f7f1c3604ba29f06fb0194113c0cdebb9741cb21cf0d", + "1a9e514e8ebae8faf4aa9b2561e82c4da462ac342967fd23eeb62d69c788c37f", + "1a9fad98a2bc4349a33356f65471b768c20daab285e0ec8926cc2cbd64519950", + "1adf55444a134beb42228bb9fdcea16e6ed4ea5514f059ad5f241bec3c07e58a", + "1b44657071336edcebbd0778f6a189a4c5d407483596e41b82cf2caa223f7184", + "1b4d8ab0d77e3a7b204f36bd251a684a81f307e3e5c353299c03e46527bcbe57", + "1b692f8a75a468d9d73d47e69bed36a362534ac7439e4b7ffb46a28ca6ae4300", + "1b938aacb3f82d2eba748f4d3ce039643df540aa52b8a226b1c47559dfd95300", + "1ba01ade66bbd22f55c0b6c4fa5d5ad80ebf7c31f10ff208dea06bfa5acb145f", + "1baf7fdda1317cc8e1aa739628f9f87d48b7e0fdf26e88fcad20f570602d58d6", + "1bd4b7607ffc276d7dea3a901f5c49dd25aa8450579c89f9983848e7d9b68b8b", + "1c260a74d7a7e4a86538fd40f497ddaa09cf0bdb87d3cc4a2f491d1b76cad2e5", + "1c2f381834ffddbaa77f1c0985c687d61ef5d1d87723ba0026aac8c53f6c61a0", + "1c4b6cbb1a2a0dd5e7445c1927f918866bc9ea76cea0d3c46c79421831ed4b09", + "1cb958ce0de35812bd01a62f72785aeaf1d2a0b942e3ea7be0b3893287d0bcfd", + "1ce18ff05a935eb586210f789946ff86af6512f5626fd0f6f3d90aa823c59b26", + "1cf5eb085a340488bf274e08c6abbafb32b94ff4456da6286a373409848f1373", + "1d53c386b644f1386934ba661826894ad14674b148d6199883619500cbfeac72", + "1d693019454f375a0ee1f16e9a0cc473cbf941f780b4eab01c210c6ab7d1ddc3", + "1d78802c9564ceb0481995708138e6af5daf67039ec5fffaa7b6e62c2a09b7af", + "1d8b9fdf05159ad613bdf0714f03e8b12dc33b3f78389bb315d13c336f8497e6", + "1d9d0ad3ae1bd0b3c2584d50c9f535b95f6f981c0c306a903ab3a14622fa4e83", + "1da17175ee9ae5d0b54fbb2a6afe84ab8f750228fec9acbe63af0fdc34e86a27", + "1dddb86c43db3223ceeb167535187e1dc4f77f558311137ab9598e173f42fcf4", + "1e37d61f66212b48b8f89e47c6a666b81600b191dbf1b93cd77301e19b59bc1d", + "1e60e9b00d53547071ebd296021a97f135965fc4d14108f39b2422776efadafa", + "1e898b25e792d44792dae03c892d9509da8138095c2ee7ff05f747df251ef63d", + "1e9370a197078bb8a937eac632798a8fc73f985b22a08e3e4f4418a7d252a53f", + "1e9a4d1e09c1d1eac8f0ddfc3732c0a6625aed1cfb4792cc9f0c35b226bea6d3", + "1ecff6afd06d6caa8afee5203626181994051d7a71897efddc9be4c42af91989", + "1ed9801a7d77ece07e35f4a3dcb5ae794b3fcb8d8b30472ba56281fb661950d7", + "1f1974c9c42bcb251ec0a6fd11a37ba070cba50c94b94d4d8173f17c7b3a8669", + "1f2197f0f714f26bbceeadb235929047fa69008d79e878abd900f1599e4eaaaa", + "1f5676917e8606c554ef1e53c14771da295818cbcc218eb002cdd7f1ed92cac8", + "1f93422b066c8f014f661708d7b347338edd9f76fe8a7e7b2318c15128416d7d", + "2006d578e24eebea02053b839228dc47b6caeacf097a56d9d747b1819df472ae", + "20403264c1f7c368497fb1b777303827fa9983e06820d7281639af0bf3f4e96e", + "20646ba67096da4fb1ab927a11101408c675c2eb98f47805aaa769a8b3db67f9", + "20bb27f8873c4d93c946b6891f3e513ec84ca2b6f09a73049952615b0248e599", + "210a7a5467b03c8bc0c2cf65ac6d5db5d76a98b285bc0a1e9c9c117b1768021d", + "212234d9d3b54e23a153e6222de1af78eabd8b93df9f6436078ee4b2a3223f68", + "2126cfceae5b8e513d078433eef8fe23109167b12bbff8d1c360ccee15bd0e61", + "2149243c6ba95fedbe578ddf8ac61ff2325b3ebc36705d3e278142f3806a062f", + "2151099bc42497224ddfdf04bd012b865f8565cf17b37ec2ccd7a4a73403b1fc", + "2163bf387c2bf5b804e987bf19844e67db7a90704c25bcad773aef04d4d859a5", + "21c4f19658aba33d1fb5c99d6e0d6c0c139f5218e44d78a77ffa5284896141b9", + "2207b41b5d1d5d2916d616e2af7d8aefecafb390937d39fa6e2a540a3bd10e4d", + "220831fdb2b9d540590561a6fa4d2b97ce6d6482f2ce58f567f9089ea8387831", + "22280f10fa1c1437d2664ada33ffba110c92a720bd7d5d657803207dd18535c7", + "22a36a44c5a57339ad24d7023c86fa5588a6300e8637b51351dd7ac95da29064", + "22f355bcedcb413a3a1dc60fe601ecca2584fcb94b8f2b56491c6361d23791f2", + "22fe153e2c7dc2a356f3551ddab83540615177d8066b248bf72168113be476cc", + "2314f1f7d20b72aa7d1d275a77ff5e154b3ca6e5beecce0cff9cba24fb95f873", + "23523c218d0de27396382b1ed9debbe4d895135feff96a92af88cafe23eee549", + "235c8c8b46ccec4d668a3c9c9acf7e1691674e6d76bd31cdc5b8919d9c62d4bb", + "23750121de6757ae56f097ccf73f62973ab859693341fc53986837e6cdef1a50", + "2376eb1a205defae31db9d21736c3a9981730d3e8de1c237e084b331f56f0226", + "23c8cc053a3e3937b369c9a82b97dbab7c1b3418079d83bb808c8ee6a8cb977f", + "24037ecc9e35cbc55aaec3c4beac71aa1b73ed68c84ba0882c32d53a156835d3", + "24063a84bea566065e3eb027a021d7321f6073ab346602abf8bbd590a787e463", + "24418099cd07ceb506972b9e4e1787eea959ca7655ea273a189bdd687bb39f54", + "2442a7acfa983a1ea38aa0f9d72e26e644620ea256c96537a9ea9146d9453022", + "245709bde24a4091af4962982e892c9e60ecbeebb4467db6ce16bdaadcfb3d9d", + "2459c1f49eef7a04b4e30c35ab426b182f335c3502bff32a32b15b42f73ad032", + "24894d48044a516cc510afc78da561f4baffc729562753188a040e7378af3d26", + "248a9d49901fe119fb30582e47717afe7ae45690000a45fcc8ee35518753d89f", + "249ecfe12202bdf2985659df13fd59ccb8802ae2dc6188499f1386d7362c0cab", + "24a0451be2da0b8e65249852523eedd2ec51ed37745039663151334f2023c326", + "24c84ffd078a3121a4145a43c56e4f1a2bf0af2f3cfdb189f98c838731066f04", + "25063678270b59e0dca5a6da6eba3292f23b73d217c9cac51605b15560e77120", + "2518b9c1010b8d9b2d7d1cd22c1483a017a24ce6480e7d11b44e414ef44b0765", + "25348dd365bc036dac3ca8c5d136c534990cb2faa13ed65b9beaab47203d8084", + "254464dd9570b6db1fd50422d4c43c65e00e79e73b7f67805753d87663fa78c1", + "254ab0473645a7d33ea1c7177f62059d16b3aed7d778d712fa260af6b41fe5ed", + "2562e4795a7465ef02f84e915ca1b58067581b7131b29252f9dfdea193c17fbc", + "25b468e54479d43367920591e3ed6aeebd9abf6f10e1eea7317a9385008d1170", + "25c0469c9130572e4564ba51afc94384fe426a6bb26817c98e57d5bb2226ffda", + "261a4e8be9e066d28fe884861cda7b2f9157f907f5fbc1e367352c42f3878753", + "26212fe5d8453ae051b0ac57fe878c9227f645900a62cf21bcbfdf6a0fc34c7b", + "2656e30609ebc0b05b195b9c0e09ddca9d3dcf58b9c2407eb1baa979794c909e", + "266db8f2f7eb2cb3582b4069583e6e96c68f8b4c2111cac4214cd4572c1bf692", + "2679142ca668cd2ed57827268096b51b059693540395cb5346e1c31aaa903641", + "26b4a45547ce31c9abe7b9e18b8dde2b06083198e899247c49f3da64f310c18e", + "26b9354f82f2e5c5d5264dd5e3d91b597a90830ed119946f4e086ecbb69e4c1c", + "26d40782ae484f44ff1d3a6c9a760c457b48f9cc56194998c017ca391b7d597a", + "26db33a4e5b448543a2a7852a03a797297f34982049f1bd0dc7ebf3807f9f981", + "26e92dd8552c48b677811eca3aa2246840723432f17775fafe537fdd1fb61302", + "26f397e39bd00131ad71a7aad609c68e156ba48c68b264166234d946ab4419d2", + "27066882d04770d7e9e64e700a19a6f9370b9a34ad3802ca1f9bdf19c37770bf", + "270b00f071e7989acc5fce6eb464501e6b57d9c336002964e9eb65617383d72f", + "273a74693860dfe7fcfb511a287decae91dd4c8a72b71af522017505694a6794", + "27407481c77c04104695dc23de6a69977aef5457e2ad988c1644523701f13d7e", + "2740a0e42da39c2fa8c568cbe32098b1c3d225756665665556f3f7f6a43a0495", + "27858d3f5cb16b93823d8914d2f8c3372e7c686c9e54b1ffd81c5b934ad1e5d8", + "27a7460340351ea1d4644e1db88e17f3eb72666eb67d741f28010308ab3a7a41", + "27b0bb9643a61edfdc28887735cbc0983b19ca2d152c614359ca281fa5d5d57a", + "27b242c5b4cf58073dc81620e1d34d802768844a3a6b1f6175330bcd5dfd74fd", + "27bcda081c48a50079e57660ce90ed1035693e751717c2460abc3724fce99a8a", + "27e7476fd5be3d2d690ae3d918652b34fb880dcbdf5868fdc723b92538497555", + "280189bb83c0b59ea0a19cc763241246d9c875cc395a2a44b8981b58913503ab", + "28124203f106909f1df4f1eb9f24106733b41600a23bc9ee10f3500647c78219", + "2828d558dee61df8b392e18e255063d3cc328b0bbf2bee4245c4378cf3452423", + "2855125525b095570c5c67a3e29341c7acc8d003430f0eaa7c9ffd3b62a4b82d", + "285ebf730d8e2e3c9de075c89cd76833a4604d7bf38e55e1f232514825053166", + "286613917a09de3d887c18c08de2c8318e746292f5960f14629b6837fcea83e4", + "287ab60d93d0337301bcbfe0c86095fc2efa96ec49f298a840203dfa6a6811cb", + "287eeaf0dc4a3c7d7ea6d5a3dc340ad31e89a936ea03e1418e7d334500e9bac8", + "2885cd50b962d924a12f2dce415a974cef12dd736a2d3d6808497099ad4242fb", + "288a9d61d75d3a9ac82165408ab2f83d773bc6976d0a596343e0115f37baef8f", + "288cdd186e53aa4e0eeda60339143adc8cf1b20a7568a5275edaeb2ac5d44d30", + "2892fb34d6fdf1048f8348e29c1e2c3eaff561a648a9016ecac7390198e0a8df", + "28a2333a43a7143735131c539910d2b9a76961757224329d64832322fe34de18", + "28b23e9ea4c919838a761bb97144a6472f73bd6f7713209efcee7f30f2735306", + "28b9b8824154fa65a6140ea02b8b19d23f258f3981b58eeb1157aa0157d12a55", + "28efb908bbddbda776159898adba8ac6c589dc6b3d37829b590c807ed2eac206", + "292c695b9d20625d27207381f8a3e70898aee3771622f38b71e090a86afc0a58", + "294a4a2c1a832c0388fe80f5beecfaf4ca644cd360c07309573cc1577bcd972e", + "294e7b801a7eb160b3a4e5a424080598ba1f1b3f73400f5138a60c3ef0bccffc", + "29744e52041dfdb69de822316a881dc2e91ac164a8483c53c3f8cea36de54686", + "29b0d55d68bd9dfa81634efce57079b22ef6fc8199b081b7c1381731de5c48a6", + "29c4bc894cc82a5ab9c0b81405b35d21f9f9cff75b7f5f33413a1fe01a7a7943", + "2a2c87339df141c33a723fbfdd435a731a0d6852dd65ca14e95e6abf6d92e75f", + "2a4492b8f60864d826a76eed372bf588abf7407050f4fcdbb4b3ccfb73559183", + "2a5f58badc336d55b3dc5815488b62b0667ec6fb54d6bae91243f46c9062b24e", + "2a943b3c5c254092702cda0c505dc040cbabdc7e04bd7658b35481f0c5f699bc", + "2aaaf70ea612431d001d3466f0b41627abfe991e7ecf768ebe8734785812e243", + "2aceba1f4c30f8131dc9ffac25a01aebf23b15c34b19f5fa8926aa4e63a7cd8f", + "2ade1798df72258a18c346a6051b1ee1779e06d9dad7c27137403468fc630bac", + "2aea7980b8e26c8d434644d2b1a4c65ccffc39ba85b00eb8de41675c42756667", + "2b0881118dff8870ad09e53fcd20f0e4663e45c8ca5f38df1c4bf17361a2a13e", + "2b08e0a8c9d17fd5c672d90351625b6dd1ab26e2a77f68ae04ef065c3a3b8758", + "2b360994ca527b6f16c2aecba4370306e7afbd6c7d20995e38f96bba71da6be0", + "2b38841187c474862deb3f9e0eb5065ada3046739c7ca2b8f62f0c85a62fc8da", + "2b42c4ee1c3d0d4d83dab44e08dd6ede637b0ddd7c16b8190b5b08988715b45e", + "2b4f44a83e139d6234b2609433bf8436c4460bcfe236addcfbe63595c8ec994f", + "2b621cec625ced5ee26b5a268b34a664e21be930eaacf81e052434a037cdeac3", + "2b7cc30d5340a2b66dcf09fdd5c6e5ca85bd388bfbd3345630990eed07578f80", + "2b9cf96df99a58f4ebce65d21e097d819c00d44ad2fd27a98d3308aa88e6b3d1", + "2ba4807fa727358531b1957300c9abdfb1ba797fabb7a7dec22a4b2dbb8c8171", + "2bc9be0fd0d56c399fd79f919ba90ce52faaab5bc69a62519ebd214e17489472", + "2bcc96d40c2a72958ad61b4a1aa2123c8d356bee6aa1ab4a661135a5bc90569b", + "2c1654b1ef1308a13bfe2ff712e531a32297032794072730a00ece77f458fe61", + "2c27838dad793f315a486df8361278039fbb7199008c026422040c445f2b9f86", + "2c387f23eb35c3d4a52f59dc80cd87a9fbd1ddf6314944a60e7fba59d0765ed3", + "2c43cc1bbee25412c64d941f66df407bfbf6b4fa6dec9128336bb1f301c6d7a2", + "2c4432fe90d8817433832a4800d0bf40c932a4692404b1fa73eab43bfade88a7", + "2c70cf437ff6f42edeb8f76651a33fbf7230d6f0b672cdd08bca19916c1e30e5", + "2c7e589a02095c96e26ff938206dd582151dabadacfc5a8721ef8cfa86527b94", + "2c956b88706537a241524c1abc8ab57a320c8711f8503cb8aaa4bb17f3200cba", + "2c9add30016eb8b582370afb233e756a2c0e71441505cd96142f27e36e88c1d9", + "2ca3401452d5d5866ec65739ab2665c8b5f0be1c614dd7059de089fec3a711e6", + "2cb99fea0cad751b8a4a1b941ed87172d52d55fb43fb83638e31b9efbf4233fb", + "2cc0152d812d1e4a0d511ff3c3a9916e776aa614a366f1f73d3106e6897bc52e", + "2d1bb608b747735c0b5a69c4e51fd79ed6b5cb826d2a61cfa642cd13f8989b41", + "2d2dadaec7a8758c8dc8309497a471fa2e019e60f9d6cd3aa19b4ec0a51d2b99", + "2d4cd5e46d3aacbf9b82a6edbe2411a74c4a93212246c1bb2372372e4bf3274d", + "2d683c2c3b49680b6de9b50e2276b7bdcc137261f37cabba58a7748d299a5494", + "2d7a92c2c3fa714388b8ac5806366ceb8121ad8f5d2f8b9984b2fd1ec6b06e37", + "2d89f2626ffa73c6be327d17d2a6db0779eae4beca522ce7f24665f70ccad02d", + "2dbf667179319df46d094d753e44d5dfaad646777e08b29b4c51d58c6447496a", + "2dd0949bc9ed9322b494caa15ea82863a6a116f20e0e6651c1d5e6b6eb43f52c", + "2de26fa321cc0f2e464c49e500b33d39797c3b6be17a60668063ba26dfa76e42", + "2ded610c1de810d2c462f51208ee2d8f471d9a3995a6bc6932c5fac629560bdf", + "2e119b1f0a1c7fb0f1b2c3f5ec1db7e89f7657c009a48b309538b396e36b2ad4", + "2e28d2c5a315902609c458005876a85506dafbdf01751fdcaab44a68d0a3bfa2", + "2e6dd85f441bd65e0fc4220329cafb0408430e5d8de2ab07a5990095ff43e78e", + "2ea77ca16c79e921ab5851baac1f0fb33529d7c76a7d2526fe3e403761fde1bf", + "2ecfa522280f5eac8bdf12c4ddf4c4a33c5b60f36f306a8e2561bd7c05d3f148", + "2eecb84249abc69da89a65bc6cafef171db054f0399e30b7cd2778512941fae8", + "2f53841fd202a63ce5944e746c15fa26e17302e631b041f1b64022545e851d77", + "2ff450770fb2fcaa0867b408f1a5e66c2b65770311e4c4ca0eaf4e68c6e04a3f", + "30511fbb595277baec4c896231d4e9295ae744a425e66949fc1c9f745e951a7f", + "30653eee6ae7bbc39c131176c37c2b9fd5ec2c50008eb84f5ae21627c1fc12c5", + "306a71235602fe4889afe2a1efc2fec8907396abc8edb9b814c710abda504325", + "30aae25405ca1260a5434470f4cf866a9039e557139dc0260f75f35dcba741bf", + "30d6674f194cb2b76d9e31310c49eed068a97c0f27598c46d930a2f6fa59ea10", + "30e628d6fa4953917a5c581772f8ddee9fbc06c63a4d2956bed54b6ea7323c73", + "310e11e670166746bec5b4b32994cc60083db42de75f51d351e07fcdc7d3de46", + "31121904f1bcb9dca43ffd7e0ba17e8b4553fefb8385245b890a54e572f8ed32", + "312ce46a276e976e2336873ecf88d8e6ce8da47e9d1c280fe7934b1cbce52834", + "3133263be7c1529f939a42c98f9f5cf54339ab8cc8ad88ecf24b653c2ab2e526", + "31619efc33f3642822abcc8df062b55ed10485df2a6b91649e86018ab2f2c9f8", + "316d5b71d81c1d3b43e88a63aa20bfca9a2ae50618a4dbd9c3338f5eacbbbe25", + "31c8c284b98bdd6e6b6e76e48688b98041b67fde8ed080bb821b723063f24866", + "321231bad3b78e54ae2e36b015490b90756bb489d54c3f4a474665fa9db95b23", + "32136646f2c328cdea70a654cc9aabf770a36f339acbeae5f8948e40c0e2d3da", + "3229ed80b09d389cd3ab365025abc3e5a0541efccf5213b41200a8dc12d24ffb", + "3253500b87ad1014ba69e19395cd95459110bc8b1dc37e4a906778e19d71d04d", + "3257111be2381ffac0f0e668d5a3ba794eb8c4c2dd03e57bf13bc6d4471b88b7", + "326c201a620cc66c4a4d90735a7009ad601df58ca76002c49a0cb39831f37d71", + "32a616312a1e0a3a8b2b427320a861a432451b405d7ba02a7158d76b055f39a9", + "32b93c82ffe40795f963cc8a2f0b18d9ceae277f114b027f79e4d36fe9b085ab", + "32f8abe41933ba53748abf536e0e0ccf4cc4e79bd65f4c33f1df78f52b45d970", + "32ff9812853c478ec4ba6d031cd30a38f47f295d2103f6bdd1e28450958fc14a", + "3338ae2f7ba490655d53c884258f45d848437fd173c216a38837ca2cc09f8e5a", + "333988cd2d484d4f92a17e109724085c52ad8c70ce1759c53553210e50d0f0c9", + "3355c5148bb66e9b9638e671d428c271469a8ca2de794275c94af852e8209424", + "337b9dab25d59bbfeac2ddf9ff3b194f0ed6c876b99b071cb42fb2574a58a0d3", + "33b73f56f58baa737f8591faf7618590c2ecd86bcba29259e69585da9dbc521a", + "33bcda29c685106c1c0f44fdb5946a975f15b59275e8ae3d7c0ad7f8d404987c", + "342b09937c0ecbdf5b01afe4d178961d86950c31daa9d90fbb5f9bfebb5d8a3b", + "34373b6e9907806a7985134899c6979611d5c9e7e176d41b97e5880ecbed2fa4", + "344a203506196c9215b5e9f57aacda02e90cadd3a9430befb19349d13afd9178", + "3472ab26eca549d514b26dde208c03dcfe5a7a764be1c5514b9bf3aecc6a66f1", + "3480bb22f27a0d18b2df530a5dce8893367e0f3855fe65da9a1be6638f97e27c", + "34937198be44c29e1ed3159e7ec17dacb6df05fc6cc9c0b1663beef0e95462f1", + "349c29d22cba205fe77921efacec0ac6ef8b71ea131f273e91dcc5d042771f75", + "34b6365aa65685adfb11f115ccaea4ad1bebf026fd9e7b0318313a7d75a27c18", + "34c35b63588dfdcb6be0ce3eb3398ccb276afe2f2ea8a6ac3a30921c858d9b53", + "34d179a8f3ec05d380cf2667f5bc24b504fe9a125ec62f5759bf76225d5e50c3", + "34e69d9035c99f06a5aa13492a356a90693976f53d37de5868016b5519877481", + "34f74afd2a016440c205ac27a027ca4ca31ae30eb360a5241462ff6cfde32dd0", + "3540be5c5eb3cd046b586a603ddd6b954bfae55c076a97a22f77e4dd40ac9493", + "354ff8174aa039d4a22a628025b8a9a6ec415f7a6a6cf5874a722c1f36b7da40", + "360027a341c08574c0477f28fdfc05e786916e797b7559af6348a68273739d25", + "362be82bc53aeba93a56f0c4cd5f2a371fcac8f45d5e7240b2866865e8a03804", + "3690398863a71cbf8c4fca9137a29a0e6e15b71470a2f0fa1cdf06a5d0986bd0", + "371b1838c0b76c4959eb92dbb7a645db0e915ffa25c3906b2959d5cc627c3fde", + "37337b4f60f4e996080e8d99ab02621bc21eab68b26607ed1e8ff6a7797b0ea2", + "373f72cd6fdabdb9446a1a73ffcec5022c36c185aa4036ad5429b02f7f6ed666", + "37522879f3d15105cda2d3ab6b48fed2900198e74e844df6fe086b385d0a9122", + "3790c32a444c5c6f3edb16980563d016559ab40aa9a48700e292b413223ea56e", + "379eec9aab24cee73bdca8e950ba019a1f57af3f88bf83f14540496a223e8420", + "37ecce1d750a93d7ff5e6ce79171495cc95a596a892a565921fba2898fdd868b", + "3827b9b917f9e9c1e0c92f935c31ae04e70a8ecc430e1a17df84688c499264cc", + "382ed00337d1740c1abcba4e6b069e5c19fae28ca9f551087ad3d8223b10e821", + "384398950404d022e3d937bfbeea616ed984f4800284a2548af95bd875e718e0", + "3859548b787c8da0ffb479bccf43817cec3aa1e6ae9c065952140c89add2b017", + "385bde3762f16b48d40ffca0f7e13cb61a3685cac72681c1ea6d46b8565fcbe7", + "386620005ea0a9aeca9e2115e73f6a315f651d1e6a33c570438104467964ff23", + "387f75f652efc68830d25b8fa3e05b3e67b26413579c7ca31092a97da03ebe90", + "389fe21426476539bc015e368b4b1db1d44dd895995e1aa3a9f082fdac2b995b", + "38ae7ecb7169e273a8ff2ace53eec67dd77941dfab4ca3610f84cf489ac06c1e", + "38f13a2e254e38adef39a763b20eb8dd3ff8fdc529bae0429891336ee448c286", + "39000ae05180499f9212cda76d3251fc437ba01b55d5cc9387822f0673bfd14b", + "390f26525ac002e5a9812985150e0b5a53bd4bc13d39d33dff76c58c5943ed1c", + "3922d60da591994cba312d2f2a8fc8d8117c09b20b198274129a070f113cd5c6", + "392d57bad566f771464837d462a4f940f6bb8b307ab19c5e6cd0b93b94c90d10", + "393395504d82649ea066d408f61d423ade8334b7d7bc95f2949b1a91a3c21898", + "393400bd5410c8a61b80eb939fd77272f5baad15e273c7401838185b5fb8d548", + "3937272e73fb81158ba96b14584d01423a8d789a1c439c3ac19085358e4d605f", + "3940d2f105eaadd7273423da9474f33a55787dc9d4e3c4c334ec2fe28f4a568c", + "39516f23294ca9fbc6114cb20c160c6b7f0c205d63889193cb85f834e1bb329f", + "39570eea5b1d6207bbfc9b1792bf673b1c6aaf0e8934cf232a771fd4eb0b42d5", + "395a62303d9fbcbf1a19506d29fd1a95337a3e998115b8f45602dd04f19f27d1", + "39764a681f8e76c170113a5ffb08d4678cefde18fbe999acd5134bc1333c4a86", + "39b3d97686cdbb4c091fdd041c7ce589bab27b9a794f798052cb0cd13fe8873e", + "39bc2f74fa976e69d6dd2a8c599e695dd3ce6bd2a28fb764ace7acc236da158f", + "3a0a35b88e4e046bb4809979d6c0638f0e673a23342bd0f845602f9d68d83446", + "3a8799a5aeb66e6b62a8f9d1971a49d558032ef93454ef38af18d6ce59d58c64", + "3a9d4a48fd9eff24b30053f0563e2cd0a8fb066a1f20f131840a902aa8d29440", + "3abcf8119ef5d704352ac5c3393f8905d420e5cb6e269df9355c046d33513644", + "3ad8e387b6b41cbd255f061e746d929211b17dfa1dc4c2bb3f988a731a6a1467", + "3adc8c039c613b98ff8c20e0670f7a758ed2e9e39feb135d9f5365a08ad47615", + "3aef058e6d5fda406e7e9d808a96e70f18ba8e13162edb2908661f7cf00ce7f9", + "3b072f9113509c9d2cbbf1c8844cdef49e2a497e972c6f6671e89870fdcf7012", + "3b2fd03b31505badf3cc0c0ae855f48bd2783c9893c35e9ccc88f55110271fae", + "3b76d52cda8f218ad86f297985c5938c24a4b6b632854f3011494b7cb744268a", + "3b9e0b44fe5d97d8001018fd4570ff07c06d80ba961d940a98c4472ed3ae83f8", + "3ba52aff6dc34cf2624a72e08e61cd95f3de7295b25b0cd669b267c9bab081b9", + "3ba52cf7bc2fb85c021648f84774c9837dc9612bd7a619c517c1c66e4ef44875", + "3bb56255d36bc07beb145539c98a47d0d3aa1ba617524b5948e3b2c22ad1645c", + "3bb92d37f525b8e3c1af2c617cc6ce9c7780bb4041eadded976f54ceed6cd655", + "3bc4dbf30cb70d551f8355bcca87cecc0d215b5744e0b50c9b0c783ced0a21da", + "3bc883d49816b88701ebaaebcba750fba11e2680392e23ee31f486c0c5862b68", + "3be2384dee2306d6a9284b55449b778ce79b220658a9b1bd21dd9118ab14ee21", + "3bf059c7df21bfa5f8b729db597f2ccfa845b0b3b1e732fc1e10f4425bcd3782", + "3c780b30a7f1aa34eef3a076aeb0d0c9e0b6233d652829a80d6d09464475a2ac", + "3c7aeffc56ec06c0d657ea706dbf66038fb53b3ab45ce57abd52c188a56adc31", + "3c9c25ac5614ff2f35703f3c8b16d214c19cfaa98be57f16c62419a05248f1ec", + "3cae26f451b61e3273296c01d01d60a12c16719d2ba21ec35e56f42cb37196b9", + "3cb0441e03264e1f033bcbc9a5bbb0d4410cde112281ac6549a918354325e2fb", + "3cfd4dace103cd378e4de22dadcac1e856aa3297ba31185bf3324ad0de33c9ea", + "3d178d2959ab3643134b553b2bf9240b0faadd3169023160903ed01a928021fd", + "3d2baf4da5720c05a85d5458cab3caf75b88dbc812e96696877dfea74760ecee", + "3d5b80db97a27e956b860cef66224a4e13774242ec0cbd808cf605e22dff6ed4", + "3d6e35262a8b1c59a282867c635be36e6d3a19a8d926f243f8811ec445b36782", + "3dbd487c90e62aeb025e3e17bf1dada719a29d9d02eb890b08f8d335ce9bfa82", + "3df597c5e716d2d5c9df79f49d6d278fa9cb40e3c3dc9b62d0d570a8f6e844ea", + "3e8457255e410c29054eb03ecbe8b264f1b3c5f74cb676fae8febbe706f1ac01", + "3e9459ca0a452caa4ffe6c08f088f780b56014a8fceb4475633d52b79ec71102", + "3e9cc0fb1dd20c269828064dca21f3594ddb7a2b2de3befbe6a5aa7d122210f1", + "3ea36a3f2725b618cef04969dd397b1631dda55ae703a926f78e6ce73bacd4ce", + "3ec85c9d41a20e1fbd27a3d3a341d28dee5281763720a6e6198b3bee1f336d37", + "3ee3d21edd0432f918f2b2f2da8d028916a311c6d1db676955e5b985ead589b4", + "3ee6a26718544fff1c7ecf9b145d04616e2405948378f9ec342251f56d9c233e", + "3f006610eb2c7a2e531da7e7e997d21202e3510dc0ac56a508ea567c3b7f863f", + "3f39376f90046200bff93b447e0e6a0dceac5d3c783811289e0b98cac2c429ee", + "3f836c4a17f0b9d0af547785829a9330ea4702b83d86098ff7dd612c3fc4f642", + "401436e97b2f59480c5c391ab5bcaac864876386e2bf617497cf181fe45690fa", + "4021fc9ac7e29c383c2a74fcecbb0bbffc07306dd5ad62445cdc087135e45ad2", + "405ce23ce6f135491c13e994d6051c77b74bd5133bf7b92fdd05083a78a306d2", + "40a61127f13657828dfbb6be83c8d0ddc2ab8b6e6ea8b0d23ecceaaed43f9250", + "40bc5caf332cfa6dff82a65aba8c88b49d8d9258240b7dce0a0a208ea44e2397", + "40fd462b084290e729d6e367d8e05017492546885d3dadadacc0e1c5d045131a", + "412893375eef842497f0b60c9a438f557df0632085d4bc926c66b080c40673e4", + "41b4b99877e41a0d93cb88c9bc81db9ef7d14c83c6b9357b01090b103131406b", + "41cf7462c3ae41845da13e06bc518236acd8fd18f38cd7e1b279c38678392caf", + "420435e1d25b95bf6ab1c9a55c923e409d5da847168091b3760226e243ba2c14", + "420eeea9117f59559bd541412bcb29cf8554b2ebb1d4b8638371e8d73b15ff2b", + "42218ce98ad3d8d50bc9cb306062f0baa439662373548602415acc7e59d6e9fd", + "42558ca06ed02105c1166604d705f4f767a1e36c7a78618437fc36f0fc7ba0ac", + "4255e9bfe477e94d2520a86c29f52e6041a4b05683c78288689f0d3254db0d02", + "42649b4f1d4cfb99397aaa33778874f4012567a6f2dd991393b094ae0596857d", + "4265b41d4e437989877b223b245e1063734921b5400fd04ba45ac53c5ffe90c6", + "4267a36f64ce7838ad9e10bfdba933172a098d263bb63c86ad57267027942326", + "426b2a61d885d5483224791a3c349ddbaee80a35e9c40f80bfa17fa1f7ddf578", + "42777dcca97c89d7e5dab61e5d9d50f5296e32579e2cb8063baf1f68b8921d90", + "428f274d9a39516b33a8ce19bbf358eea69640c134c81028375e2d49aaa891a9", + "42922521409bdb49b86404d39136e93ea75eb0580955c13fe8cd2cba85756686", + "42d0312b0a32fbd94ca1b9eaa4b8e3129fa96947832f97a6a44d15675e697585", + "42f5ca82146746b778bfe5ec97acedd150aea7e3294c9898623fb86a53b70300", + "430d0e82480ebb89b687f5648879a0c26589aa9c9f887d3327b279c330b2b591", + "43295a9e46b5e2ec8e671c5a130fdd7f942b7efb434d66f7b7835620e2a859a5", + "4349b4d1462a48bca23750e495a84a3b950f705dfcb3d11c0cffe385eaa19ee3", + "436a928e4fe1f3e129a24d447dd62821a8b2a5c9055189f0dea60828b7f361a3", + "438e8de1b84bb27d2c051c461ba76ebc5de760814e0988577684197b0005b3f5", + "43bb7b981ac7954e20a941c170ae59219c91c77a2966d776942db5e4028f24ee", + "43cc161ca50b1b7d86a7e025187296c87c6d289f8e9e7bb86f8b6d1edaf7f026", + "441586d4cbf2db66b066c6a372442092654c3d87158fd9e46e6091fb5feb634c", + "4424a4bc86c0f46c8093e3bed4442f576579feacfc43f2366f36e4fe6edec869", + "44650da1115b47ef24e431eaaebc63ce3482b9888382be4309e9e08ae417f4e6", + "447de9ca3a4c845fd9fff0d19c05e240558368ea14d8a481281c42c1b3f12984", + "4498af1a55b9b415d066ded464919d2a55edb6969b8cc00092adbc52edba84e4", + "44cb0c73cd673eb2469fbeb0f283e7eeea30e78fd242c7fee547887a1acc4f92", + "44cc9e16828fa11b5d0a4541d3a29383c8f97c4eaad6f8b8cd7ad9b804f77951", + "44d383d0c482930c758e9abf75ef1b4145b7b91a7f3ba4a8e41563c30cc60009", + "44d43be322fe6f463b76c7e48e789ff01e07d3712606fd6155204d9e96fa7f53", + "44e9bcea8279515cc4de38c7c1b47cf7db5b1d5fe7b3373f99f31a334341f2b3", + "45092edc1b76ae550294388a7008a4673b601988e5d4dd47e0d5e42fb3e1ef01", + "45607e5b779450299b4a4e3d2bec1d1bfaa5de408b81221fbf109d782b8dcc06", + "4591e8241665cbe9eb8e1e475f97037849633d40bc1e3dbec237d87d292913fa", + "45c3b0d13cc47a16729ce2bdc9c4463a2b73f299ec743fb2c3cd93dfdab96286", + "45d6fe5461af10acda5c405e9d1a098a709cdbd59c89c6da0b41ab7b9565c360", + "465d1b63766e37d26df675c644396e902d587b2ed23cb7654dcb05fa05543133", + "46844362e00bd617fccf5dec58b106fc8b40c69054f5ea96fd3f08a6f7106d10", + "468917e21826e30fbf98f2b87d396d0472fe5b98c216eb12819fc588e2288f32", + "46affc777d8595127e09bdbabb975693f690868625ca609955ac8bf61669c912", + "46c916e7635bbf037b90447dde28310890f8b755c054927867966849863981c8", + "46d8a3840800d6114f9e23ff4c6920318183c011fae7580a86042feb881ebfb9", + "46dba5eba3ae9cab894e7240ac8ea3ee6475750d6c22bc1ab75749ad11c90516", + "477dea60e11636b609f77bf206708ea7dbf025d7729d90f692a3e732b08b21fc", + "478481dae1cc617790b53d41d5783e8000b51a433ce3bd0979e2fd5512f0324e", + "47966500228dc154f1e5184b4eb5c1d7079d2cf0ab6d6873751954f85ac2fdf7", + "47d6af26ce42dec6de81734d68bf4d2e12153415a0048b9645858e67a46e57d0", + "47de3552f64938c412f636cf0c628e63813bc8464b3d15d6b39b25bc509f718b", + "47e7e6462074a1a7dbe81ecfe3c03772620597901fc0063e09b85f360053f888", + "47edf18219d7e7f42ce9b3fe479cfcadf04f52ae02a7cd02586b965f5a9163d3", + "47fadbe243f65705a8d6f991f805a499136f721c1165f8e6f93f186f0a0007c7", + "48130f01e7241c2a0cc995b602940e1a52ee6d6e4bbd1f64580f9ed0817685e6", + "482d446437c7d574e8059cb27f6f258acf28bb2570996ad698ace431b7bb501f", + "4844fa3bfab190bfb8f62b622ca7923aa2473fcb339f49964d2043046948edf5", + "48873a484e5925c760df63afa2c3c1a66198254db1ca5717cdd081b5c79c0516", + "488c31db771d6fb4a209ccc928baea074f090e7fb56ff863b2e569d9ed1bc372", + "48a37c04e31111c927f2a066db22f491afe23df1eb94d9d9288c999f471aa982", + "48e31de5c3115f218c5eae6d25a2d4cd097cc97480aa61a93b876eeae03a1871", + "491364a39d23fd232b902a1bad2e3b9aa7884909a16969eaecaca24c4d125257", + "4938fbb14d7f0a430b584a9a5d28a53667e267d6669b4c6419dd6ec9bb6b77f4", + "4983a313916042ed6a0c6c2bd9d48cb8a2904585e99db84a6a7d9bbbb105f99f", + "499aa49d7834b33ab4bd44e7f51e7fca36bb850bfc76b3b5b8542db1df09e628", + "4a2726cdf7b53daa50ce0d8f82988880cb1ab67aac8e3eb096e386b06ee377eb", + "4a2f05baddba7f86c20f9bd95ec3a75ba9e604c10ddcb3c0b46f0926feba4bc5", + "4a459a84cefab2ae16c9dd41d5a5e38abbdddf9e88763fbea8a358e8f0a45450", + "4a4a6d458d69c78b42eecc5d11a1565e210edac4cc7754485a4f7dc2ec353a10", + "4a6ef7f11a9b59fff7083b81620a26bbf8bd849ccea64eac497b37a58ae69826", + "4a7ffb8d73d336b41de75ae007d05cb60d78fa29f979886dc1f4758cf66a14ff", + "4a871532448ee3c6d421c8a462db8e257039a1dd1547315b004671a4a3b79eec", + "4a8761e3fbb633cf41fee33db760dcf10f1c1c281fe188bdce901883371631ef", + "4ac00b172d90a8f879b320d58f5ce6f177b4f0f1f2fdc4896d8513121e1bc444", + "4ac2003f5ff908998db16efd8639d53cb3d5d26d28fa7221ed0f8163f6c953d9", + "4b01d0ffe4419a55d185fc6a50f1b80d629d87d5d44caa68a4f002cad0331eb7", + "4b10518d510ef77e3c7337bf0761660350d2350b22cea604bf6da5cfc569ecf7", + "4b20bff40caf59eaf8e5b8468dbb031f2239f249f8eeb09eea3a2c5aab036f8f", + "4b3210a8d9fa69888af9d8c56c0def3bb79e5e05f0a8be419e0bf58797864eda", + "4b5b4b5c0af753edf62720b09c8461fbf6917106b875aa95e5429c8b66e08c76", + "4b6a1d6c205a38fe3b7c533c282dd7b29ac4b5ff6864409396020a569999b374", + "4b7f80cc29961955ea40c7d3567b35a02164cd1533d958d883be82c4ba37b31d", + "4b8a1def4d36441651de8f179889ad3bd052bb0becb9605941116db4018c95f8", + "4bbf36b7c2c14caba6070bbae9458af2c813b599c8f6f3c6e2cf9705b21a99fd", + "4bc646e0147b8c2f1d49768a352f413b830a2332df659f02e944e942d47a4786", + "4bd9d910a8041060f92d96bf5eb568ce0140fe65e0015b82973300cf0e5b0688", + "4be8d922a5de73ec325c9248fc1416187e5c829799430a4dd333e4a7553150ea", + "4c1afc535fe4498f8e6ac334816ed0b9bbbabf835f590d477284915804ec6b02", + "4c54b6a006e39b19e32a114c684e4687e5bdb997e3c1d32299df1a6d821e51e4", + "4c6311c9c35a3420cdf7020ede4be8fd5f7b42235f8f7691677857ecd3987a48", + "4cfd7bdd44bea3204d1c2a9061be31965670616e9683866ae44294f5376a6f40", + "4d04a412a956ca16c52fc241daa05d7607215f93580007a531b0d254fd9f52d7", + "4d052ee0eb223728de2c5566ab2bc2532d03ad930b5f2cc92f0a40c1e75dc478", + "4d6bfda867e611caabef7d76612f0b5aaa6418d8225f4e6d42fa953a950d17e4", + "4de79945a00baf1aa259d5875bfcc7727bf66a46dc8fa9bc2231d72b821d48bd", + "4de923ffbb46a3f8e6ac63c98a68521fd21c1ab0f77649c95a30bacabead4e5f", + "4dee6a3056ebdf8fadc76c651ada138068fced8ecb685dead613ff9f1ce21912", + "4e1e2a65b95720debed07bccc2ab2fb4530f46961a49a58b3d06121dba5ac69d", + "4e39cb23a75019301edf20fc52aafa4cd6af401443571086d35b2bd0578b742f", + "4e5a8e5c4554d8fa8a888d419e03dedf35272f60c09c06494bdd6025f1aaf47a", + "4e5e55b9e0c37032de19375ef4f580467a7d16c24c117a936a64b1acedfbe1f9", + "4e96aab0b69cd5652b2226c538170f17d2c99dd6fbe6558086d131f1ab426fba", + "4ec8791a63530bde19bd54031d055d324e66610201287ff0a15c2b1a2fda1534", + "4ed68cecaf66306974ee0a1c02ac6d5522fd388a6ebf63f1260ed6bd778e6ccb", + "4ee615c9b08f46dde44466f10ad5597cb8da99157dcf86430e3409d29ee2e6e7", + "4f2bfd951868f8f07f40b14079b16caf8ad4ff9cb426f1f444e36016d29c6d3d", + "4f44da2e6a0ee3611a2f59dcbd74e92f9003581abbe0ffdf0eaa4441c485f5cc", + "4f5acdad32a85f1f0913249599e5c961a1bceeca6258089074bd6c79525915ff", + "4f7ba8c30d185c5a7dfa3e424b3c9e8f2a2bffb63293f27438fe741526458cb2", + "4fbfc62a8da5a39510020c9ce45669045efe132b191c2e611b5db80f6ff1911e", + "503aa9adefdf6b5f8bc9fabe338b915ccb08a00434799b4b2a9887bd2475433a", + "507197a8f0ca6fbc5ea59f39fb0f1f93f463b87ccef674d37f216214cf048ec6", + "508d052b57de3a3b4e3c114fd09f3c0e1390e61331e018ee6636a39d34c9ea42", + "508ee9e55ec34d55b104c6fc261d9eb746e4e70512ad86cc0b4309420b47ab66", + "508f1063fa711383380384576cf4b882743780c40f5b068d19ca20f21a78498f", + "50dd754d35675a097c9ac8e548f50f62d860771bebba8ca0c32e8db8da81bc38", + "5106b9ea61a0c1ad92726963b125faff828e1c21cd1872565f50b1cd3aec3886", + "51164d3f4bf9e63d7d12f2557f54de4f38cfd15f84d3c9ac6aba4e7b4f7cf00e", + "511f1c32d909dba225053747092b92f90309b832ca3b77266befcaaa1127124b", + "5125d348c3b07f6c3178ec0d4f202d1651e134a8c5c54b12c2a51f703328f066", + "513fcc34df2dbdeeff9e169cbb3da37e3f749e7fc1015c015a9cd546b0044572", + "5145f5c1bbdd99afaa2fd7f44f5e9e1f65aa81e02190f562903a48943cbf81d6", + "5177307c8c4ff23aecd65de6866fb2638d3d7ec38857cfa69a6b1735d3ae4ceb", + "519a433335a476427b4e44e79c849f394218055b0b3d55bb5d827d57404235f0", + "51ebce2f08cf13324964a1a686b9d497861cc807cf4f352c649a44836c3e94a5", + "520c9cec4ebc993350ca357fcfb6a8c3f6b94ad865e20f46ddcb517c414bf9dd", + "5220fecc9698d8c201b67957faed62daa1a60bacef7cf5654cddf7e2b7ccc75c", + "52259a41e170c1c126230665c1e0e2b9fcd62ebc41309bf079f2d8372c1e6d95", + "522e80a35e589f1d0ad9f4f7b6b6cd0b718e84a27b5001690f1e62f9b932360b", + "5238327c3ec5fa63a5a4cb5dbadda9d28c0486b41c53f91661a2130bd715baf8", + "5244e6ef0b5ed9f3207115544961ddc1ebc1911fd6964bacb01d8bf76c00d659", + "524c7192963314b4237bce27dd2894d568e7fc3372c9b7d76e772e53bed3f6b2", + "5255f663fa401436723b5d5f7894ed1097b9ff37dfaf3d2b746e8f5ba8467982", + "52764263ad81241602654110f7beef81da96da4ec7b8c60799edbd5857587e37", + "52a60872f2deacda62eae94debfa7f6940e2d58712570cab942848334ec35422", + "52f3bca068bbdcd19eaff06f35b06f7e535d88eec057180ef79c1d408b3b6630", + "52fa559775b9d6bc1a7a600084774c5960ed4a3bb7fc494ebf021e5cd1cbfdfb", + "52fad5a8f3e0c1a5efe527864537eb8c3ee9ba1d18e21bd0553925d1a4dac3cb", + "52ff1f7716b9f49065fbbdcc8ef2ce9ddb14b91daa85fdb2729237d0e59f6cc5", + "5305de227271bd209a44b2db1b715e7d261f4c402556d75a55e0f1b527f60567", + "530b04bc8910bae2df31a59dc8ac9b7ef4f4dc854b82f552a23044fb168e24c0", + "5330727dddfd3cfbd27b98ab33714e4c1acd7aefc2386acb5fd788a7657ce576", + "533ba29ec6475d2a3d97751dcf7cd534b0ac20aa01c93b7bb6fbf637e84f4acc", + "535739cdc04feef32fb71d37d31105fda85e8465082bf605b5441435aada8542", + "5387d01cc44757a81ae1ccf1111f3a77fe8c210cdf6ed1a991192a80f4cdb0d0", + "53b018be0e50f8ad61778e2330ed74ec2ffc8bec9b5d06a8cba967046f7fe86b", + "53f26ef79ca484c2bff9c31ed2c0b2d6ac2fc5a437d9b29fb1c5ab9dca002e8e", + "542faf6ec88e75b833dc99ce6d5fd4d30eb3373afc85f9f535fcabc38d0ccf1c", + "5468da1bfe41c940866fee9927d6945071614195f680f2b64b83c55bba3192f9", + "5494e181e6577586ec41dd3e5abc372a33e765cb03daf56d3b20db748dc79b80", + "549cc016d0d8b032362d4d1649408bfffa31446c2590b2e12804ebbdc47fb2b6", + "54bf2c3e0ed21b66938eb7623c57deac2f09ee563dea39c3e14d86a9217d2654", + "54d8b11fa7465c7082792100e111f8e33492e254f3892a41af6f78fc123d2454", + "54e5ea5fe2853c502e96c870d0111a15365a5f6bce1f3193a7b9c348645481c0", + "54e9dd90dc1debc55834f9788aaded500adec9d7619cc66817eb3b8eda688b94", + "55043e55212e6205cba548efc9313fdbb51cfa8cca57640fbff9dc8aab528853", + "556445fd46eda083b9309061f7e2fa74f036037f632b9b970eeb6722d23303ff", + "5573a9b44b03589da0cabf94c4a8e8937abc3d157356d864293a721fffd6b64c", + "55bf14f4a1a1eb840ef9a1784a65ba2d04e37319e45a2acf8349860cbef6a232", + "55d1286538ca7f5a4fd5356054a3b8f1dc9526876d81e36a0bdf741c802f1650", + "55e77f7587c4d4a56c639ef0c76bf425495c87ba7fe9cc3ca0c1cf436418d585", + "55fa2fdacb594e28831ed04b9b0fa4a047abd8e550f7d5eb724679036e528d53", + "5613fb98363341a6104fac31488fe8fcf907baf29f54278bbd9df7342c0f8685", + "566626d5c211e1d702e139dc08be33384e6d823d5cec75a6cb7b3310c6998a0b", + "56c1062c0c0ed6a0ba04664b78a6355cd2fcbcdac3aa93bc06819def28aa37f9", + "56e70514de14b66605e2b2e23053ed771e60f1e0de950cc1fd0c510a9a5301c2", + "5706b028c6b3503ed4ec4b5f490a5ccd746f48f3a8520d31d1d83cca2b8594c4", + "5709be9646cf910153f3f75a496597712cb7411e2f9447c7fef7fd492b32286e", + "571925f0b66c738b52105644b24558c21771db1f4432842dcb87a1ab468a74a9", + "571d8a20fccfe7b2de32adddf80c3e3f437341d84ae5fea77e5c55d8d9201dfc", + "5725a7914b447c22de18d20518bfd925addee19a17d889f3625836ebcadc972e", + "5740788c353fc6c81713fd4349e0db8b199ae0e1e8e4d5adc84dac9087e53e47", + "5743c10d43e919f0b522f535e9682357ae759de4f2044f5913aa11483dff0881", + "5743f3b3ae3820ce68338256058963f97e55e5242640d1dc30f974297d9391a3", + "57441d4c49333dd837ecafa0038bebcee2d990af8ecb55d4ff3b34ee9e0fe376", + "5792777bcf215d5f29e1271849674a435ed2a491f59bc498a24f761716bc833c", + "57bf1e0123dac8c3ff51ddd5c0e66620bcc27ae225e8c35927529b2fde62383f", + "581c0ff964ecde7b34cfd519ffec1d7846dacb2d96dcaab573231b5e824f341f", + "582bf2770b2a7a438b46966f406f82fa7f23dd8ad0f4c516837964af31808117", + "583ff7d92c90bc9811ad96d829f25bd68db1d0f5baefc02207c3c55179ffbd1e", + "58587d94c44401d22c7570b425d78588aaa439b5ba8a02532cc32fafd64b790b", + "5861d48f3a22385c21966caf1cbff9e534b2bc6d9b24b04a1c2bc92a175101cc", + "586df6b16d0371f939f01dd0fca066135a548ec526170a270879a775c4d6e8a7", + "58953bd77119c5a75dc8db52eae11aaf841ce9f6acf9c724aac634e0916c98d9", + "58af007dbfbb09cda0c25a22146bde68c6101750a61cc2bb16773de4fdd0288e", + "58c399854348d4943780f0593aa64371481e164045bc1e02af6cd5435b68a781", + "58dbdaab4d64ff0407fed71b3944f8a45d07694f2a39bb72cad56d94606c61e6", + "58e1c39129cfeeb249160e0272d7a56800521b08204d103bdce6b8867569c0ae", + "591de6a6a561511570150c61e0dc026f7babbe07e022131910a92f7523704d93", + "593124d7a97e611c01bcd92a663ff46fe9cf2d26cb22366fe35be7232ea1908c", + "5951167c8577ebc5e40b7323822b0b24a5da3268fbed0121456adcd5546090b7", + "59958e12c7427964002f36c93678bd6093ddc70364b2a735f592ac7b53d303e4", + "59a558b41fee06ccd7a1c840b603c99b1de92ec6ea6deed3152c01066a533212", + "59c8aee345f6da52cfba936abbdd140b1f318a80b616fa7778d12bab59819e3a", + "59f012671896e2a38807e6186a07c1111855296e31327c5fdd2a1dba5b785f3f", + "59f0eebad0170449246bb9826bc605a71fb122133ad7d4e7d99c69603f8629e2", + "5a07dac7bd8ad5b87afa1bc38b0276e6480da48f5e6d17b7825c8098bf88c1ec", + "5a0ba0d9ac4d2cd1f66309b325bff32bbe54816c4dcfc42064244df39d65a2a1", + "5a28a5206f5de7aa53c740e1b5b5b6cbbed409faebf4eb9f803d7c63a5198e79", + "5a3bfba80d88b4a3978f734acd65bebd13219893a477685dd8a71d513d02f424", + "5a7cf8dda296956ca9d978266b4cba91b8c4923c9b21ea98e740a35cb1901b50", + "5a82c9e9b43a93b1d64933552ec8836d741ca1406b29f8ff0c121ab3447c795a", + "5aa9412ceba584bc77c6bbabf4fd18ed16253403928e49524ed59535670e5e1c", + "5aaf3e59356111f7d87a5be93f33806c428219dfa4ab46dc5790f8503e31c581", + "5abfec56f23ffc8b44eee5e710ce21cef1efa1b0987dec1ecf800891da7b1001", + "5ac17fd232e43b44cbc0abeb3705d2538a647767056a2973c6dc82f24ed377fc", + "5af847ef35663184ba6a26712cb6a7344a96f2fcf273fa3a292b4bb46701bd02", + "5b2ff51ee5fc993256b0d9a247a5fef9874fab06a2fb561da7741d48c98c4700", + "5b4f43cff4bb797638bfd67b131632468a2c82a8f46e8028c4b2f70e5c374e5f", + "5b5cee99d8eef512becf6b4c6f2adc28a7674b2a5b2b034f6299ef87e135cc1a", + "5b81733f6c2d4875de6d68334be907addfee1ee916eb178a17b4a3532e57ce79", + "5bb74799899c9c13b3de7b14145ee9a36e517b3255b98fc68ed551e3f168eaca", + "5bf76a6863797b85f12c1e9657dbb98cbfeda76a85e5649366bb51c2e25ae0ea", + "5bfd333e5ee50ebc3ef7343214119e6485fcfe62177243c9ed44ca94ab862a42", + "5c0660c0a889c03272b8cb05ddefc3bb0f3208eceb119af380c167b29c1152a9", + "5c12ed807c668a70cf1e43c34b6976fa59d4a75b94cf5f46e6a66553dc551001", + "5c3800b2100255c46b68a1c6b3949bf252472c389be1176689bbe9072892db81", + "5c7875a957f4a01842286acce36973d6f1b43f0b7be8cf309ce3d3fc1b8a576a", + "5c81f028f79f4c7834047b6f0e6a9099cddec1698b588f49fb095cf79ce66096", + "5c89cbf0a0308716fa318e4b90d30aa624a3420ecbcdbab00885ded6dec88f6e", + "5c95d9702aa1067cb6ed8b2ca35dced65dd135880926fe90f6723d32342ddbfc", + "5cb3005a1e6be72d09b1fae86de8b44c7aef9f7b3fbcaf7be1314471d533b087", + "5cc8076479ea7c2283aa6ca570696a1ff9830912ffa176e470a8360a0b94b1e8", + "5cdfefce4177c0db9df5fe05c4d9838a299082541dc99c4a2b2ca62eb09973d0", + "5d197d6f8601540b20d9c8650ac6c7db604ea308f7ebaa6455f4f4f22bb9095a", + "5d5711143100eee894c191c7308a6310af56b08efd5d96764e4633c39d472f9c", + "5d77ac5abf8d323e7a23e3e0fb820cbd184050de53e1f6592102f3ce1888782f", + "5d8d05965144cc8ed052b1ebea298aeaa8acc790f297bf6b360d8799ffefe1a5", + "5da330a2d7539cb8079584a52f6d704f1dbed9de147c04d8001ff5e11b0536ea", + "5daa16db0f4094f7ea1436e6b4bc0ff8eb07efc22c3b63675509a52a235c51a9", + "5db712157f9dc54dead72fde0ed3dea5df4cdc2f9572b76c471c5c22b9232bef", + "5dd622ce281dfa0b24d79e8399b00c68bc086ea44ce4326920fb148b0e6f4d78", + "5df83d897e2924cb350b4a3305db4d44e1d066c990aed69b29e43e0f6698f533", + "5e1afb6bc141daf303dbab37f66ee10ebbe8b31ab3c5656946238e2eafd6340d", + "5e24f0ff25337bcf32fac07d4030e897af5dabefcfbeb5f1cf2e5403ba82b9c2", + "5e3432bde574450d89842f92daf06262e8cc64d281cc107ce3f103dd32cc0eb9", + "5e4b5c42a003ceaea591f2895a3e0f3102897dd8b1ecfe91a8bd5b5f562e96f2", + "5e589de637f00be39babfdfb626967ed63a1cd173634ed66cd240dfd436de90f", + "5e685214a2659cfab9e0ef45c26ecf4f5698b852eac4238043d62e60fa539519", + "5e6eb52c3e99332abab43be7928ca3633f011753547397f5a314b3e166035a48", + "5ea7348a56ef47dcb20a2c685063b693a67edab31a66863d7db871acf8dc477a", + "5eb9a73a975f07b91f8b3e72ba5c8456bc036afa6e319978295073db5afc4a6f", + "5eb9c1eaefa61748f03a84a8985aa3574ff723f4ded3b96cc6fd312de9f478a7", + "5ef8af7db8c8467bc40fa36bd555fafe62a1ab4e9946b732bd14b8b7785fda5d", + "5f250cb3bcadecde0bd95597f5d4fba1b26740d4d34d012595438f01455e9e59", + "5f2f285781ede7bd7228d4bc4cd74e2e679af4ab6f79188c80ea37525786638c", + "5f3490442150aef643b464dc288c4a064717afa7e6fe82a0db06d12bb4376fc5", + "5f3ade9c6ab3e134e04cd4a11338b71299eb710e0b2aebeb923e7dbe832020e3", + "5fe9694c951479112476e396fb24b6701c41954c967143688ecf85829914c8c9", + "5ffe2d07115394912d1f6396ab7085e45e9aa33eb85d986ecc24d1b720cdaa1e", + "600952a30695601677ef7a4b490fbcca600890c72bb1fcba30e6608f52a3b048", + "601938fde46de7f2ea79924da1579b5c5776505e76b41d86957a714552446738", + "6050db010826953bf2c017835d673e3201d38873160548b2efd17911823f3310", + "607db073c05e934b6c553b86c9b277d3ff247c7d7a6b8cfb70158246d56eb0be", + "607e3ebbfd8fa181ff70f397c7b22d0ad49597cb5cc85ead3b09821969d1e139", + "609f5fcd87f8b7224c173b7705321238e7406b3a54298777d7fec85ae6eb2ada", + "60e15063301376c03a8acf67435bef4242d6ae4b1836722c25a338b8c1bf4024", + "618d56cecf230518ff456c9888a33aaa90bf6b63edbca25f220851a7d2047900", + "6192d9e645ce38581a2ceafa43b9f5a92a38ea33439bd627797e16f54d0d2792", + "61a51010912747df01da5cd08f06e107a55b3e04a0ca7f99169432a3c2ff7872", + "61cd4801f08d19367b08eefc34e54ca6118b7c9e45e4e40bfd123134f8dc6425", + "61ce3e015ff0053f90457530820a7a8d46fe13e094e36b34c455a9c5ba1259b3", + "61f7a10997a6de054614fb67050e4ba86dafac0e7e82cac440206469de7de82b", + "62188821e56e67a9095da01261d799a8f4303257116295a6d51560bf7f96d272", + "621d8c4f8a40c4cc92b77b97df8bcc633317af6589211de5485863c573897f32", + "625750077e5ef43a7436d8a4e067f2bc09fc9c037e0a37b9c6116697c9af3c53", + "62702c6a819aeb5926918fc55115d3ad2a7667b0465dde4335e9f15cc5e91e33", + "62767847449284fce1f687e9c8015a80ae44b4e5b3d741177f2d5323e438f32b", + "62b1c59113149af010c492e62e5ccb43ec4adc764b9536cd0852fe9bcf2c897f", + "62dc61fd40b554612b1ba15c8cfc78a3a034eab498edbb67758f37664179fc37", + "62df329d815d6957eb334e427fe52d6ce5d18863f4e2e812c982004f24bd1d3b", + "631fc475f1c091e56e359a9aa4471130ab9a5895582c35da8b378c747967ba26", + "633a4fd35f046f06b4a4dc0e5a90b1e477763a2124535e3e425c78e25527bd4a", + "633bf032214ab02575b9908ade4b36342e8801f3eb6dda731f807b6aa89547c8", + "6341a2b834df92c235865fc9924d178335a4744ac558f4a2f96bf9b25441bd0d", + "63606c5d2195afd67a85725058a3dabb0855cc4b56ed7d66048433b523200118", + "6376c53f85f5ef69d5b59e136d2ad58d7b045fe50d0696662cc988f8bf145673", + "637b7b47f8f66722655cd95433ff31ae247dc7285c3348a7abf5dda931b84f1d", + "638daf7eb92d77f62d09ad78bb2c3f915e9809db388c9262deffd339d42f8b10", + "63a27ddfb2656a6745bb08f49faec53d3e559269161b2ee647134804ec18305a", + "63a986aaa6d340794b5a30bfeef97f8fb7b8fb91e641a447f129bf1dcd635f55", + "63e0e5c2a713106bccfd18bd22f26b4403229956df862889834518e440e6416d", + "63ef9a580cb94305c24269d0542a6ec21cd0f1e2c91e57f55ffbac99bc7163a6", + "6423ea5b8363fe772ca1e62ed51591ef0e2c7cad699b917253275cf8a71321db", + "64365e6dd5353b5442d4700932937991b7775997dd01d8fe6453abc7cd9a3e0b", + "643db93d793e3e730baa3209418e81e3f268dbb1e7a481ddcf59300a95b2d8d7", + "647a28aad4b02ff76bf2259ce3e1c17ddc7b444c3ac52705e681c64cb175291d", + "647f41c5b6dfa99ea202bde2c72d94b72b96e1e16cb297d7e00557a37c227154", + "64a7a3087e7a42aa66ea37486727c3874a65ac8611e49acf44d804965ea352b4", + "64d351d40a34f7884df2f840b7fa21727f8fed6df9e05e5b1fa6bdddbec49ebf", + "64d5c7ace5d90a493707fde2ba69c96975472d619423de17d258a14a73d6c04c", + "64d807ab260eb9d668450a8f1935359de7f4df518fbd924d220a81c6329301f6", + "64f76ea78bd84b0f135d3ede3c9285c77bb30e844a15dcc5eaf6301ba843cff7", + "6513fa0432f4384c96452465cc6c3482864a8e8fd9e7805b5a6a162e049ae062", + "653a93f89a8da3751e456c49059224b46698b894b64350b7e3c63f44658e0d8f", + "65647f03bd50562243692b63cc1fdfc4a754a814a21bf410cf4e0c48540d95c0", + "659c2ec601cdf969de0359d6b82058e778b563e99581d841af4c429ddaa1edb4", + "65a83ee09df22d92c5b76d1ed8e2e1f8d63cc02c4abe190763340d3d8313eeab", + "65ade03bf0d5ee974f9c8bb9d4c4a00a66b48e1bbfc7386d34c7a4170d16c49e", + "65cdab08c1e4a155c6f8e5617b93d8648b3b6a5912d4b790d02857d3507be9d6", + "65e5649aea03c20cd090502514db112dc347491b82ab8eca8b6f394a5c1b51fc", + "65ef0031ba9a2d6b0ec1fe5f29e47b3450c348c8d790e5e4d4900697c304f440", + "65f80b4528d187f9bcb15ee1794c0f43689c5d0b08bc2a75eaf2a31c52bd341c", + "65f9260cf8b1be768366e28a641a648f4769425ce0b37774259be274d7a64038", + "6605f42fbdae3b25ab1530fdb2e5d131f1b1ad59ed024da45c2da15fc8fce44e", + "6620b6219ad7c9bb82b4fc14b69fdbe3e0bb3f0a843350dea575f2a68352f8bb", + "662575859843a090be178ef11c3ff949b26975089fe6a0dc0efe041da3363c95", + "66477eb8101c0eae9ca8d6e00fc5af48b1ebe634380b7a368bc9887036d9edc9", + "6656cf871958da5c32aeceafe2572dd475124aaaf701cdc309f577d418044b5e", + "667f496aaae9c55dc80a3241ed1680bdd57dfda585610cc577356107c47a67e9", + "6681e3c1384378849a7a9295a3c4a4753b4b65445095201402636da3729f2322", + "66f368c851503c59694f9ee32a9f32927b8fe9fa1e80dbbb701f87bcc048e57b", + "66fcec4db0d060ad0ae50ac16bcf92257fb5cebf0a8cacaf0097b4a3cf590463", + "67252f580d6b891d16bfaf4180aed1b01ebb7ab5d6ffe0ff6dbcc3f315c1cad1", + "675c6a52ee8c1097c94aabe8f89da07c84bfb1729b4fc83d082035b4098cbf39", + "67aa1cb881d30869a3568440aebfd1ff0c8e52743ffdf4deb1f648dfa9ec27da", + "67aedb4ea431c92855d75899e571ffc8a0566485bce9ba162205149c951b08fb", + "680b5d1f7823434a26a54fa967b9243361a3bc8505f6d3014aad56e8255fee8e", + "683d4185c296a570ac27cc4108e8c1b7a14bd682b0b04879cddc4786448ff178", + "68516ac9f1a2c06694609dbe17e755798c4972d137779cbbed90bda9ab0dbe94", + "6873b65c87de9d43a3be6c7f6c3dad94b85dd07d3063f127fb8ff9736ed74855", + "687dd21c9c339af790cc61e3520fcb729bdd986e40acb3cbac5b21882b0a7d7a", + "689c596a1a402a711028033b1e00b31e7a30d93605654124841b30283edbfde6", + "68d2c928a05ee93db92e1161dd420921dfc98e3bd055cea5383ed8c450cb9a10", + "68f0dda42d4f45fe70c134d627e1ffa69479dd40159fd5d8164b14649e82b2c4", + "68f2c4b0e3f60bf7f398c9b6aa28bacf6d2f5e7234b3f0857e3d06920c214c04", + "691a1ea063bea59e2ea787fd37e06c333998f93705c02f8c88badb80f0e08abf", + "69296a5c433d9a8b385e82a2c65bee4a2de9a80b4f77441559a9ea83f3f6f1a8", + "6974bf3ae3d564d5bcc545a797bc0b2e11f69be43b840119bcf58374566c858b", + "6974f12d69e3af683d1ab7a7d20a865486783d23b4e1c282b154cb6faaebac71", + "698c4b79a847e21ce2dfceb683173bb2517d401de64c50844cd4ff796bbd98cc", + "698d53ba96b04fa3bbc0ff889ce2ec3398df89fe1d09022d4c1870a6360c9074", + "699c232eb369996e6e1955d8755aa4df666ea7adbd9171521c1106deaf28a2b9", + "69d39be858ab5a0d1731a1a0de016d5ef8d400c011d243b6a5182f1adf354ee8", + "69e7227a17f559f45e0e53f484042ed1b12edea43a4f1c9115d0134f7dda7465", + "6a3a2abc2148efeb99cf92d43659fe4437ad3f381fc5cec5cf18bc297c6fec96", + "6a690eb87882ab82e8202172c9cd7f6e3c9fb672069eb0ab8dac0e7dc7d05de0", + "6a75bcba06bd24c12169079e32622867190ffaf07100424ed00e1858c69822db", + "6a91e9529558fbf167a7cf0893a07bd8e16faed710a86e06af068fa15bb67873", + "6aa592fba84736f361ddcf66707fe15babd6f50b5f1fd49c21c272ecfd7f205d", + "6b4719b2a5a8835e0740c73c2bbf6b94213866794b174ecfece0f737f1cd001d", + "6b51b99bbc348bee5c4e4604da974fc566d70867a4d0e011d55a252e06cb0db2", + "6b7822407423f5266f91eba2227e33c75eb6cb85dc67efc76b07fc06ddfad88d", + "6b9663fe2428b5109a5721f7e8866e9e69d8a62b1f270019ecc36a347603e821", + "6b9c5aefef9b58f1e8e192fff34cf257b08e466aeaf65a2853c9a0d88a19502f", + "6bcf9d9355d514c522b60268466703f3338beddfec4380bd24d7c022b8ec87c7", + "6bd01a73d20d03563f91657146f0a6e11c082e8da4d74ab6f76bd146ad9610e2", + "6be5b5b85ec4edb66ee377e4682a06b3882e7b7c6ea4b8b7f7d3fc74337e2dbe", + "6bfccfa455cc73d12acfec2372639c0a80e7e809e43f3e9aa536677a4497fb60", + "6c06419d83f639d799195f586480be6d1f65ad715cb88b9f2700086f6b70e594", + "6c2f8a4d7b49e4bfd26d82b1779e7fd4b6ce7f78de2e47b7005a63132718bc98", + "6c66f8b2ea659391d439527556839fdf14b43d7104cfaceaa2c50e54b7bbf966", + "6c7909f2a90be8e7963c586b202e174f445d8d3d7dec3b6e864e136502484008", + "6c7b914983c75cd033d8e9266cc5df962bf7329113b58dbd8b29830e93ad4e7c", + "6c8284565dfd154013854f88ef41a73491447a92b69ee0313f8266d2007737f9", + "6c86c884635b1713342b8a2104ab5ffcf17c5f5e388a2f1396f36280c8135b66", + "6caeaa904146cf8358cb8ccf09c6a44c64e1942acd24fba6432ebfe674209788", + "6ce3537f885d05ca6480438c7ce3e87d1f7d2521eb0f31f5c08e47cf301be28a", + "6d36f8a775d3eaa6a2c0b0ad390e01f35bb6b8ea75409807ec9b9a3c93be592c", + "6d4fe2a0608ef4ef22a00655ca12540eb6636334c5202261a66dbb6d10a58ea4", + "6d896096e52d1cccf599dc7fa94fcde94fff526220ca8c4ffc626c42495df1a2", + "6d96d09c7e5a6a9112a35e96e7b57c65de7883a1f51f9a5db07146ce857e07cd", + "6db2b190ba0fd5fc752e7285ae272f7a391803e52660e928273987805c9ea2ba", + "6dd7c52cad4d13afdcda3ca78630bf63ac86ee745524929fa91c1cc9ccb544cd", + "6df1495b93ab4d7cb31c85976c88767a2906f21144a7a8b856ed57c5d584700d", + "6dfeb5a052c7866d1798360e88d5714fab6693772b66e0417d45c74b5763c33d", + "6e21c0b9482031bc9f387ddd33a53ab464c4a00c1eae99f69ace97054c81abae", + "6e428659f64733b176f25f790389d3f4347ce2862ab67d3ae1fd4ca9a42cd330", + "6e586147e1f0d42aeeee54bdb5236d097e305ab7aae76514ee6f4acd0bceed63", + "6e6347ede261653f4adc6b01200384e013ba4103de13e31f7cb82cb17eb3e109", + "6e63daf41296594baabea559f145029b9fc5364946a47ad7a4899ae6f6c8f9f7", + "6e90c6bd75b3a980e10388fdefdce25fc783e51729cf6f00c8f652b0f4355365", + "6ea1ebde2882b97883e62dfd80d3918f6c8de7406f0de4ce9619c25b2438755a", + "6eaf4819e64e8f094c8f1964c77bb2aeb952e2691b520f2ec2fa7e11f52fdcb3", + "6ef70efbf5db1a44d48e3738de90d485aea621d09b3efc0a541198a5d5b20bc0", + "6ef9b67e2d1d4d36228c7b5218e96042ae7ad65312c2022c52b4b14c67686487", + "6f25c4816728e39049eb9e3a648e2ad713e25e33507c05f83c84126d5507052f", + "6f3ab3548e1e543697f0eb90ee19d20ee9fb9bbd8b940606a4b0fd8ee6595c0f", + "6f3b4625f41a1f8b908a4f6c900231e22e702083803b05f1de408a1b58cc987d", + "6f3e158f32511ccc6690cd472aa9f02321cf7d353e167241a7c6ff92134c5994", + "6f4218c3d3934cb40b30b14df5a72de35d8dbd1146159083cdabe234471186bc", + "6f79af20b759c197b9bd3b1b63a504fabf62e15f570d70c5a8afc28aee5559aa", + "6f89168841ee0bd7f09a457fddb74638c7bde20dbcc26695b6a702ce2126e883", + "6fa265f656f2ee4d7a2b8c55b477ded670ecf48496fb9c4afd98bc0d32d39ebb", + "6fa315dac5b32cf7abf9acfe27c648859b2ccaa3b6e161adeff396ade3723978", + "6fbb766aa08fb955aa7577e9a7f2ef8d9fafa9dd0fc8d3521ebf5f460301cfda", + "6fc5414ef35c32d41c91fbd549f5bdbc0d8b0db216280c83544ab0b8738b6f00", + "6fcd3d40f63cbd59389972fb750ee27e1adcdef6a5274943948f40f56116fb14", + "6fe35d8f5fa325e35f4cde71cda545655b1c653f1772bfd8ce8fcdc280162ada", + "6ff0db043be5147c89ea98552ecdb3d43eb872a907cc4797c01bb226d980f5b2", + "701eca67391efee74e0293d67cfd39e615baa695474d81970834e9e62c7d3cf9", + "70333759946dbc2f084534f94f76d6604dc5dc3ebde87e79bf4bbdc12a7dfe9a", + "703768057979fb6f20c571ccc72439a6595a20f77c4b01d4b2d1bed855e1cd42", + "703b9ba161ba6e11b97f64d87fcdb54c9f53272699dbefe94990d83d2271590f", + "703f7f554bbbfe686479fb152fef9e4e0f8310a827e86b8515adf24b07a620ed", + "707e3480e3adf85aac5025238159eb94804c76fd671e28e23176d75048a511b3", + "709f5003d5a255e258b1cb7a6b18c421a2e1826b448c72ed8847eeac5497ef3e", + "70b3a8211a06b1d0caa06b96b49926e57109a461918cc02186fe43840ed9cc3e", + "71088e9aae4a8185d5d27950295b872a98ea093428f97864c6ae60a04f332a98", + "714c7d5079aa57ea41eaf89e238ecf1f20918147887e41bcb8d6b2f6dd05d8f7", + "71673963d8dad3c8041c1c46fef6c23da0c49ea1df404712d1d2cfec197b0574", + "717caea1d37df06e06b5cb5b9cbee7081e65491abdddc40c5f073f499eb97c89", + "71826c795a682894dabef3ff7ce3208029ce95212b8b8149010dd01522a124a3", + "71853c6e61e8a444c03052320348c810c0073bfb0d31ebddcfb9c4e9c007c02c", + "718c33f21779928837d659c7614a844276b6e76a5fcc9b62a092a25fa2c29184", + "7192b4a737df2076f40c10e6893c2ffa904b42694f1938960401a11cf8738d68", + "719712ae6554824342c77f11e57208d5fa78d40528df8a18e61d48b54430ed47", + "71be758ab30f48eba0225f88b5e4677fd61309234c44078db44801c879ba3074", + "71d9b63e6b32b3a79ceab2553a5d4b8d0d060c02034ce041a0d86081287d851e", + "721f9f634bf6ebd5c7ebed59495abec3cd11c9b72631477995ecba11e5dfd8c2", + "7233fc7cf525a3643d7abe89bfc1f9b2cd0a11f9936a54d7797967f646c07116", + "72462d383c86b76753eb6817608dd89ec609c406cbae7ed4e717b595909de462", + "725ff03054935ab9923ad90ecaca69238e39046d3fbd14000e0b698401754d8f", + "72695a3c8e402351d00ad77e266b6ba8a282aca36289e88e36c61709877919fc", + "72e78cfdeeffe0bd2597a166d4da21981b87977df5f8c29d250cb10e51cb6581", + "72f4d67ba20569c6b7596ec75c0e033971e148698de7f4bebcdae8fa8554db17", + "72fa79b3fa115e9812888bf78b82495c5f4cc6a234ce5adbc3eb386984c3184c", + "72fc5100b1654ec387007afe56d8bf5eca6072396db1f7d944936eea41695f21", + "73024882c66bcc50e1eb8fa5534a49ae5dfef16fa8869944b961cfab6364c0af", + "730b69a152e1b93d05dc8d4dd430cf48831c4edf53558e148161613762d5e6c4", + "73298093213dcc7424d426cb063933c34faad99e3ecc54f6a1f9da12e0757b16", + "73466bffa17e0f61131673c325dde5e475e2534dced253fa6c2c041060fe8d16", + "7359f0b367c18c3514982a300af59a3715a3cfbbf68ffe9619532be27eddc830", + "7364b82fd2ea76068008093d85b78ebb04a31672269cf9b25c4fd1d1a4c6190f", + "739ed04dab3d45687f8a69040dd7d3d9193876342abc769ea1f83d4cf177934a", + "73a4760cfd30333ca79502c85b198c3776db9749fa4c4b38edca375db671140f", + "73ab62d6f93a31d65bf4b47d0752b58e6bca8ee092e30a0b75ed176165770b60", + "73e19802a19244d6ba391c14e7e910267e01e9d9a1823acff209361dba4f418e", + "742111f5094433a5f24f30d74366d717136d0c7f310c03cff6e4c7e4d2f16972", + "743c7e0bcaeefc11b5879252821ec7ba7f7e617480ba2989bfda8db33d43b8a9", + "74684ee19ac68b9d5652c36f29d803ac7fb131c41bba960ba12cbfb38f420374", + "749d83f2bb09989fe35c1b6eabec5bc4c5730e2dc87ca1f2cf173ea5fea2034a", + "74a3878c4403073afe2c4a2779a4460b958fb4687381f2f40792a88a808cff1f", + "74b8ff4e4480e118d6a2f4918ed91f2bc0585219cdbbf6062ce8fb1996f647d1", + "74ba9ba206af46de2e61639e2df0374a8248861b38702c513176fff43a393c41", + "74d5ad78c8c335d3eb3e4b4be59efe582982aa2f607c15de98d25794595b3139", + "75049ffb0430bb1c3b413f523d00860d9ebe79608c5b96638715b44457d8580f", + "7521f48d4093fd7a9f6f645199fcf1745851467131ac37581781ef7d2b225df6", + "7540f3ca514bcf1ec786b4b7d11ba9f68eee79fdf891ed8c4f711c40e5ba8b25", + "7579cec0a34fdeefe775b1c85d11e936c39b6429e30189253c52d8a9230aff9f", + "75926f87d7473582ce80f378332a95c7c20b6a27ae07575eaafbd3203b23322a", + "75b7d995c7b34a8667be75ed3691324e218b44f94f772c4b2c3765401f01d106", + "75e79e1f3af52e503798047cccc630e2d91cf5525851c0884290c03dfde1aa80", + "760dbbc813ccd9ae23ce4af764de33059e888bf038d7a65e5d6816892f9cc103", + "764089293fede6ed51e409246c5cb35248fd9f2538c7ef10c37b36cda222fcad", + "767475c5471db706e5873690e0ce6b40160e0cfffae596f2e0609f971f41f849", + "769b6e89a2fbebc067461c2c54e8f79f242c729bd7206183ba670dc65b275f82", + "76b0bc810534bba4a859942e88a604edb3d394b8421d516298302ff23557409d", + "76e965b06bd3d10aa463f44812e8ea04eb8e67c95f382595a70be22c42dbdd4b", + "76fd68580002a202777d8bee3d257cc28cffeee5de1799bec497fac4427e0c04", + "77250960a3a38b0e46d4366dd3387196ab80ef8e5c4e224587156f973a55e094", + "773fc1d055664538e2caff1408b4786eceb0348027158c0ce749ee031220ac93", + "776acd210e8fe45c59dcc77d5c4c9249019230746124fd78e363d5842cce43d4", + "7794ba7ff588975da27457e5de81fd2021a7d5f8657c33672e36b38cd7bd13a0", + "779804956eb5a224028889be5dffaaea3d890ee8403e346b8a68051f8c122260", + "77cffae22072a98ce40bd10afed90a8a4ef52833566e3b14c4fe5c3603f0555d", + "78196725b339be6d51593df871fa995b35e14ed2dd70cac976afa3cb7330c05e", + "78435438eadc53e4b65b80e6f52e7d3b8f989b81bad7b49fb4b06e6fed2ce3db", + "7858336ed2c00e5618549fbbd572a335c911c09a4976bef5f8ac1f08a5933920", + "785bf60336e1b28f1892dcbfba28bfa6898b93802fe99093ce45e8fc308c7916", + "78622374cece0a3dd71545fd5579fa5ba742fb0e275d452adedbe3bb821a5edb", + "78698ce70fc4b006e2d8f6374ba1cfcf74c0a9c4b29b075c957d92f942cd73d0", + "786a54e1510ecf3e0c15b22d8b1cf54d50d94297cb6bae026b522534374c78af", + "78a73a8ad0f98bc4a4d8638ad881b815a3e32bf2a2949f180602e7875134aa1c", + "78a8a9c1a3b7908c79c7f78d726116cc70a98e29b738ac171e11b2fb25e1151c", + "78afb450b01a4b9fceeeb74d3b40965be61cba7e43cd68c7dbd7028e89268bb3", + "78ee6e15729bc5127e4e4fa362dc21a2482f39a6fd3cecbb9de2fff3ad884382", + "78f740cf7bf686664e07a6dfd745854f5b16eb9d211acefbbccf1f903b3c7fcc", + "79393d7ca4a928730d3ba7c7b33d3ac19a72629becdbfd5266baa2058dc936ec", + "794bb3b63d8687eb22490f6255abac5d7e1a4eb7f1852766700090be9c4ed871", + "7970e3c540590f3d367a29d9b5fb6c0fd23c809dfc652e359680d9e03bd1c86b", + "797f3cafd57e7c848b365be2a79eabc1cadf42857f5a2abc1eb2850b082a1f42", + "798f00477a084973c5c1f4a93cde9839dce6ca25634a4bb83e78e5e3ce9e703a", + "79a445fe8117b072375d8cb67d2ce686e79cca3d029ce8a703255513edd715c7", + "79b2565cd36a5aa7332f4cd279edc77d9daf30771d5d7b961967603af3d770df", + "79b9c50866c5fa2e695540ce7a267c0889d0f5f3f30abfb820d44a407009c1fe", + "79d537183284063b3c119f5bf642b045592f50620b0feb4e444beec840a003e9", + "79f36210ee715b0f9867bf90584758664f37847083647139d14935ead9805adf", + "7a1e81189da79ee94e93d47aebd5bcd6fd85b716169e9561f213678523a4a17c", + "7a51f2d1ce1a72560afb60675ff504b47fa01bb657a0dccf975a371525239d89", + "7a88199756be7bd36308fdb0db2319a89bad7be1a4336c7eb15f92eadede9be0", + "7a9442e42d1113539ae1654ed1ecbe54c67bf72e55e467c57688c0f4e4d31e43", + "7aa0646914320a193ed7cb20bbbe65d900bc659a2a581f5a44055c60197ed130", + "7aa4be1cc1fb76f5ee2d307f24110c84d565971c030a848da542be8edf937c07", + "7aa850a2ca05f925ad97095285f06535102b93bf507e85dbbd53f1ca3d000952", + "7acaf7a8d5f3879c46553fec4f100fb2a1dc9f8ed0c0d8a883489427ef2d28ea", + "7aefadb4ddffb69dd946a207c21f412da4554397a8b06865e239c602d2f41a55", + "7af5155fdd59640ac809c59916c3620598a785fae44a3e702aae488c119c5198", + "7b16ac75723b6f9894eb0599c963725531588337e41ad546eb8395327d5e0697", + "7b21c9201cb94c0510eadce9d4cd9a069c097f584a50c41e7968ce3fabf34d6e", + "7b85738369d19b76ee12a33138b6fd71f12b61c541322bd99b6e8c76f28d8f84", + "7b9501a5c4baca2b8e964d4fd1e9d6ac724d7a0a1c0160cfbc089920f900fcf1", + "7bb52dacdb2687faf5c22439851c2c834dd54c485d7543d0aed875ee4a1a394c", + "7bbfca7a917b961e83f03bf91996b05f5668bc93facc9cb94ed1fef301a115b6", + "7bc8cd8c3a7c5107c3c2bcbea68a070cf1ac231a74057404dee584fdb70c0a03", + "7bdb0e3427bab33b42fe14d28182c3273838199c7b439da9e1bf2fbbe30ce74a", + "7c01f13b6ffeaa36f3d6754644c6b5f5f62ba7d3adff55ea78ee05281a7151b1", + "7c13410bc3b2cafdecde4586c74704a26f911d86a07f147af9af91f24fac60fd", + "7c15efa53621d248accaee695bbc7adce989c445911c33070ed5377af5a738df", + "7c276fdf802f10307b08c02b3597e7ddca1ff05d81e58426241b978b1295bd4e", + "7ca8ff06f6624f3dd7194560b088908c42e11b69a2b015b252e84e391519fcdf", + "7ccc8fa8248956bd43d77506a4c23697e30fa61f2f5602063e885c278657066a", + "7cd938fabf4f322c80e458c82e4e29ba87807a0d77af5714dbf740c14d4096d2", + "7ceb188274648cdcac071879cdf812555a194a99b60c088d389b7db82149584e", + "7d1cb0401330cbba5c232aab8f71fb185f52d939d5a474c481db009167520432", + "7d26d34a80b40971e91ed54ebe1c94d874fa06f7ca0cc1710112faf2d02410eb", + "7d28016ba3f18a075f25cd72c23c8aa3248df56228fb1a52a723d9a24e726368", + "7d4978222a424efebe70733bbf7ebf56ab35a773b50ccb82a690b07d65576549", + "7d4da588cc4a42eb2dbf5920937d1af624734e7b94c9e9fd35e000de9b19958f", + "7dce8a0adb7f8202cfa2bb850fd97ff64521cadc0ca4a4d564ef4b6abe260b69", + "7e14c223156ea6e79def806060fd0c9888732cc9959313d42a56e6a384e1828d", + "7e23d0dd4cd6bd622866dec150a1d38b77187d2895e0dda5229305902ab23be7", + "7e7c44b1e6c90932b9d8b3496d700cea042dc79e088873bd16584bfb0065d21c", + "7e9b8e888284ef44482249468bcf1d3c05e248d8f83365adfa032d7155b7c39c", + "7eaf4730cb6b13e2ac7db67cf61b6a29f5715cbd294685f743fe5b1545a2e173", + "7ec7ac968403f21ebc8bac18aeae134a206b12c7524c4057e2b47a42fecef24c", + "7efe2a5615bc2ac81c2a8c56a439dafbb7e04bc721a6857129b0dbb7156a49a8", + "7f02c41794b097eede7d73b9d380ee74f7df101cb0fcb2c5ed18268fa95dcf87", + "7f4eeac34b3358cd586794fe9fb121217a86464deb4b1493bfd413c57ae77af2", + "7f6c61ff00f23fd6ae6b96d61a303f6e44363ff4d1679b13b9cef244e814cc84", + "7fd7e3e665311b4248ea1af06a835d00dae5a427922abe1f898fe66a958e5382", + "7fdaf4bdab2643b67df88ccbdd1cf0d5878361070c6374a12939e7cf93bc733b", + "80313a3c133d5918239dbaf1248d1b2abb392f39b294c81ea689af6ee0707fd2", + "8031bebb0d9126c9be5daa6beeca271b8f7879957fe35e0e47c17060eb9b7811", + "8074859bcf62c319d896a8f29d98216f8991ab9bdf49f879ab36d33dd155d9d2", + "80ad51c06c23fceb63beb5462345dcb9a2d1b67054015609ca1189c2308e8f8a", + "80ae9c3044117ffd27c3f375f0e29c9e75d2a3edba632de71adfb25ce66b0771", + "80e4cae673b060511ceade38c0a68359b9e878300c9d5d0259b6552deaa5a1d6", + "811b18091fbf1a6e5d1452fbbdbc2e9ecab73191974e0214e38bf70cbd0010fc", + "8124f366da84fb287245aca463572ef89c274a6360cc0e9bdee7d477ef041215", + "81518eb16b5d44c22df9334eb65fac0dcc85031b0c946172485819306c58aa8e", + "818dbba72cafd429131e635b3ef6f3dcd5f3d40c359d6f615235f11eb769bd27", + "820c4004f3ca9ed9556f3e68fdcdd8ab6e394dba0f50e532d682344629184ee0", + "827972c6d14855d08db1c34677e97fd79a18c0bd9fb2b9e9e45f6a0a05a2952c", + "82d226725b39f5e86eb5bbfdca67c52fa239ee867160c45a87c90194f9c3e509", + "82f4be5f4a60f1413f24d360f20b179fded451bb7f109f1f6b7a15a7b5393f80", + "83134fa9b0a26a50c2fcdf580f8ab7160a8404bdfb887cf727d7588c080b4ce7", + "835422aa07b758d79e0f7f35f6e5db7bf413b872aa8a71f2abaa82a96bc8c799", + "836ad77eb86cc6612922f0fbd4a848187aa5c64219811459d4bbebfe01ecc345", + "837b550ef7d135d2c0dcd5984d065228dd5442d3351d9cca94321162d8b14af1", + "838f6d15062b30326ad7199db9d071f7507bccf1fceb33f3e8a204d285382faa", + "83d185f09165dc5d972254bea55079367c28a0fb300ebbf3fe92bd74fe2d93c4", + "844a4ccb1843663785075b9e9f858bfd930d305b1a5c84753cf9c0a2a0295da7", + "845d12f4cfa04b863ab3301a97d26363b521787bc09ed660cb283520b35e9b9e", + "848898dbe0c01cc84e63e5f2095e35255b43bc886f0e27eaa81e8b8260a297ed", + "84a94da8a7ac516f98b806874e3ffaf05d4dbc506126b3dbfe613fdead407985", + "84bd9538257b7814607343164e7ed9b5e4342b8b4f7cd3545f72fc1ddd06896a", + "84cf3d3a21806b2e0a25f2137a8df5a2838f1b0b747657386e8e4018a9f7009b", + "84e808828a69df041c29bac0e30230b711a94e906ac9820f06655213f786b6e7", + "84f7e204b791c99dd098aadcde86e776a5ceabcfd3061aa4c654a7635de22f9c", + "856769850310c7f661ab3a987a49aac3f19b580db8f96ab79daafa718e247f79", + "8570530e03ba5bd517b643180a182c36fe9c233ee4f5cdcfda9adf6544d97e5b", + "8574f58478a831ff1c012fef160d321b0e4a0c4e0aa1bfd4f02ce1a1fe3c9b43", + "85a6f82c5d501f4f85312ee5c78761f3711c14fe917f5141fea49f97adb7aa27", + "85beba3967198e7e0def9d27d36611e722d7660db0c44da4dc75932d501c438f", + "8631f5eb9a74ac31db1a05ed767754d8ec31f089a80f38116a71014bda213882", + "863bb5e180baa89cd2770b88f31edf3988b20dbbac613a409311b9465e593c5c", + "864a2260f794a96c454c1e311409f8b7eb71278f8cef9f6f1b610de80c6f347b", + "865f0cf27ae5e60a2195c621c0ac35b3bcc9324ec41b8986fbc714ba480f9ea5", + "8699aad8ce4823f923740806ea40269c4ef238e140c811865958e98d665cb08f", + "86d85370fe7dd74a8023657798682dda13199fab06e13bd306d4a19dd8193a0d", + "871cb4c9c8f611b49d7743fd17eb92d14640bbdee058490611bca68dda174f3d", + "872c8acb3d12e47c416d8c271212920cc4654d7339111e4d89065ae2f6b15285", + "873f4d11489712ec090a4451aa926f41fd3800f396ab4613c7055f9710a41565", + "874f46d9052509cd07edba8888b99776024653a9dba3aa2ac48a63f232f7eff2", + "87afdca89f997e5f096f51341578d2ab8b8c5f1f091e2011d0df87b28570cbd2", + "87d36a47ee715c077744344ee6e94618940f64b34d8f232fa15248e96ec679e6", + "87dc2cc17041f5f8857c27415fac432301a74c3f7cc6e3ab1e6b10a0f124ad34", + "87fe37b344acf82bdc4a7d88fea87ed6570374d3511bdab793b90e0d2e1f2a0d", + "883ace1beaaea3781f22637789a1a7fa17bfb1ce7542e941d81bb468a7d4c6e5", + "8846f36ac79147eb10014d75f6fd6585d01f3490d72496a0bdd74fc5b1e7f181", + "884db55185577f00049d89caa0ffd4f57d3fec4fde57b2baec680c6e4ff4836e", + "885febe6854c1e5acb517005f8a6da8f35ed7099c45193832ca4fa5e3ce91848", + "8882fa13167d65377a7ba90a7a4dcedcfac1e80b54c84a5b946296074674ba31", + "88bc8e9c23670d5bd20a398979d863b137b267aa1fd7d8c84678d3f72a163be4", + "88cc71bd445254f00b73120853a0b61cfd04c7a6f912d657198109ef8deb7020", + "88da3f334bdf2598c22e1c4cfc7e4f27813fab04b71a1c94fbd394963104f5ac", + "88ef35c4215e6661b6b13b887413f6cfb5fe007fbe0eb7c31ab8f8667d49f939", + "88f24ed049e9cfdc5e6e52664281db32ee6d605f8880ff9795f339c8b8cb6ca7", + "88fc7fb8d44b1eb7909f8ee97234e419796e3b2262957bf06855040d53b30e20", + "8906b268b99b4ac3c7eade7be1ff2265be2c7fd00c92ab2c1ef2a3a4fd4d2cdc", + "8909455ec2ed4207fb2252286af435945ef7f4441b14dcdb2233b2c7e473c62c", + "892da162016430553dc0ebf569bcde69a9d968cfb7fd52167ed6e243a99af167", + "8981887e94552109b3cd72aff3dfa3c9d76eaaaabebf0254c890e8a58ddcab8c", + "89a3a2ac4e3d523b93cf14975ce15e5fd39483055aea8eb9754967463ef4bb22", + "89d52a436b5a0a0f5caf3ec0dae813d38b2ec6f90f9b6ad984155b2ef628451a", + "89fffa58748f397976e7995342db915a9ef79590640577796c7d99ee674e0eaf", + "8a12f9756e56fecb2d2654523cc2b63ea0219362fcd3ab082bb95812968ec31b", + "8a258b19bfa17e45608be2a3076e02f86b2bebced13facefc1f28a5c503e3958", + "8a38325262b8f4fab2f9586fcb48caa450546bbf8881b2512fc7ae56f0463327", + "8aa00ccd9ec106af3f8d6269121ff1dbd381673b740a26106da23ba717c5267c", + "8b09e0958549cdad4bbcb5072d88bb28b2d16887b3580879796753f2ca73dbfa", + "8b1b7db85dfb029764987b5d104bb3bd23bce31c4c15285536218025bcde4bc3", + "8b2e208352c7bf5089566c2285fb25c19a7e4c68d62a1a45ba3eb0642f5a44a5", + "8b3e1ea8341e7a38998e66b52af92276c2ab7d089fe710bbebe40c37fdf9c210", + "8b3e415c88f3f697e9264f31cec88cb364776b3b2174b6ec9d2c9ae8ab41bb2a", + "8b455bed9daf8f050877d9ad9a1d0008e1c5b5b47af5f2608ff6d63d7f2778f4", + "8b4dcf783f047800dfce66ac900496e13d11389aa43f11d0ba0aa17a53d35900", + "8b4e9df3726ccc0a874a7657bea1c03e50a1e63855a8fa4aec0ca050d16c5cf8", + "8b91b7882239b55dccd3f2a3b699b870aa6e3cf9820fb32f7f64fe0975716466", + "8bc15315ec593223c023b73d8e47bc4fa17ec93d3eb420598028076918e9e59a", + "8bd6765985b355a8426754aae723dea54f2e1ac544ff0a1a3442941e6febaac8", + "8be8a2bdaa02f79e7cdb4c7c3906826fa6bf90a20bcf9285cad4b3b105aeb2ba", + "8c02e41c60007b69416eb7689404b46186087ccfe0f5989f20784c2c196b7b1b", + "8c8d4af696736585ce185cc8af41d0b6109a2ad22d4bfa782ea7ef9a3834e55f", + "8c8e11c3bd2c532170df6ca8a165d44f1e707806299fe6ffcab13a6f2e122521", + "8cb3a08d1ccf19c8725c6a98ed7b3386e5799fab11ace8108c2883579f682714", + "8d177f26ef02fd353303673b91e46d8390241c1c279dbdb22c991b3c4e9f51e7", + "8d28413eeca9c997dac5d6ec5ab6ab12517e44b8c4a51ca3ba7c859491df3952", + "8d9ef2d42ea02122ee5863b202c05351599d62885d2b98f96bb737e4ee6ebc33", + "8dbb7ce9aa0f47e558051a724cd235cdeda295cef4ae6af454076079bdff514a", + "8dcfd3fb6ca84c1f95efc8f897513707d70ebce78c2518aa8f65e814e71f3545", + "8df113b565bd3cc6cec8965e630711bb168b1cc181f53bce4c6b7ce61ba56aac", + "8e22151cba0ac0a1e46d5ddce156fd63cca4eae19b77238cca06c8c8ce821eb8", + "8e2eb253feab70a0380720d136c671a52ebd609d5c1272ccdc6e3ad461435156", + "8e4f60fbc9c964d0ebbd10f8b9265be69a3a3edc2119ea12426ef43119d4ad5c", + "8e5ceea749d717624820f70c22b33842bca091f4ae50be442a7ec0f09c83303a", + "8e6d9e83a4e8ecd5052dfac7c65bafb828ac22f1788885c36a58ec7aca9d27d9", + "8e9c2d035ed8eff3f545b75e289b7e582c7968100ea73e6fa33b0be1d5bb50b8", + "8ead263dc6ed57e40bc9a47ef5e9f5c3e646b986f2ea12f7e6892ead5de2ab8c", + "8f170427867d4318616b12da898037de0f181a5431d3acded4af696cf1326370", + "8f4222b557e6df356279166322bbd1705e8a56c35891f90c1530cf4bb2eb54a0", + "8f44f3b78ef1575f9b970421c721350dc71b5db2af4dec9abaab2ca7578652e0", + "8ff84b817ee44a102d307bb8083ced2d3cdedf4cf0079e6c459c7b677de3a329", + "901a8a97dbb978e7d127091a6fba78c0400b75a12bbf3d417b28fb0af80b6782", + "905bcd129bbf0751e06533ac956befd3daaaa9ee2e6645cc6fa3ecf71fbb9a89", + "90695b391187c27e2685fb7e84f2652a507fb2a02449a190d3267bf7cb3f5b45", + "906b270dd3e6ce8f729128dd2cb3031beb593dc7429e4dd36b13f401cdf792b5", + "908adfeb33ccf16b1cbda5b092c58b268e734268ff6c60cfc466cc856adf2baa", + "90a9b33fee83fcf0446eb2bd3672676120fa0145c84b4a808785d5cf67fb307d", + "90b17242f7b1bf044dcb0c8b348a6b20c4f43458b8a4263562653701f136c4ef", + "90cde577fd11b1b1735407d95de549f9c0a53404b7f2d12fcd870cb927e9f059", + "90d2284908e605264442186b3da69f670bf446ef3616ef6e730a093e090e2b3a", + "90ef7fa7bf2f7675c592189ef0b9aee190f6b142d6df567bc67852a3fe46d9af", + "91436958d8b110e882c5c4a525733c39f6c910e196f4deb9f42603dd7c4603ad", + "91629fdb7a0c28df18abce3897b3b96b2554f0a3b724004d415f8cd7a3a47e92", + "917dd26021510d66f517c0d401ecb5d7db01a25a1195fc4ecb9a7a363814268b", + "919a549f4845967ac2f847626e2b00056eecf902c91d9a4be8378a41640fbf34", + "91bd8da2cffc4096f89a36c40aa3d00916cdcc73f8a681efa0222941be36543e", + "91ffe0cf3888e203f9d71c38b09f1fa3d34ba073342f26cc49bdd72e92ab0206", + "920c0664db9a83dcfaf66ae2e13aef9ade367d3221902ff79efbec849060b80a", + "922aa008e201c6d77a55d168be1f336b56911fc60a133a2ed900f3cf13d73431", + "92563ce948515f1da8aceedf199f6ed7dd85ecc6de1f26c98b99655deafe03cc", + "92697ea9bfbcfec599c6d210f730325aaf05166191250d1f162d162258961af0", + "926aa9c880645612620c26ddcd7a444ffabb7b7edf31b3bc2d22b50819698712", + "9291a90d8529ecbfa4905aa368ef4026cb7a430e3e6714b9b8f334dbe5cb96d2", + "929700cddcbccce723799f822366c39243dbc4b470d2164354f1168a9b7c97a3", + "92bf7e3f8a5589cf3497c2a9dc4b945e169b87e44a1c01240873eb41bb77ee8d", + "92c5af8113a4129e4e6b5e6a9e16a45471fdde0ec55ab2cf1f64aa1fba547d5d", + "92c8bc3fc53f871322fa08f65bd33d640fdc555892939d08208ae59cbc888a18", + "92e75868a799ff640fc178d95d09d447e721af90c051b2ce8d970f9037c04670", + "931e6ab771d72e7d47b63fc46fccaea8b91fcc80aa97a456bee36974cac43006", + "932f61eac4752e0bd55dae596f5fb82c6a5655beed2cc0f209ff157dbe89e180", + "9363a4d2b67faf2bb77955660010ced2a615c568d3761c57a3bf2ff6404ec4a6", + "94a44eb8c02107750fa3c557560e6bec0aef76e28851d236b2d6cf85565a07f5", + "94d38a120b004f5a6b66229add7e74e72dbd074772ba15d1742d4a647d2713c8", + "951e91b14a12ad616ca8991f1132fa4f763d1858822d0157ebffc429b49cae4c", + "952cdfdd6d836ab796722cb133ce2ee16b14348de138f70a79f1c6af33ad9c38", + "95312bd603102ebca1809ff72dbbf8e073586e5c0a4c68cacc6199d1e1224e6d", + "95406e941d31e39578bd8abbdb76f1cb2b3eb7bcc360261d849b45b641718be8", + "9568476e7af8a3f709e95241c456daeb7c9b80accea85bb5f7f71ad6b4387f33", + "956ff6130d82abbacde17e05a5bf2923b8a4a993b65c424d4f9c95a30427714c", + "9593d227ea9654f8734f7c51145c6d3eee4179603c050ccd5642cb50d2f790d1", + "95a05b8ab460184690fc31a8a13698b286f0e92bdcb33fb5cf6af66a29b3ab47", + "95a5f3efbe93d44838d1f96f3621c477d24e786415f4c81c9879dd403e4b4278", + "95c254bc14997a2e0c244f9213f7a2230d543948e2c41e38937cb69235ce7eb8", + "95c4df01b8d01f9506b8814f2631c677d1604853eb37cc30abd19bd33be912c5", + "95ca30ba70b76786b6fbc3b5aad9ca496be3168819291372dae2eb2b6e70e234", + "965257f692454bf6a792cebadc934b730ecfa2a179d3fe46df2232fb32d57d91", + "96d82f098fbb8db5b98ec2e8f8ec63a3641b4c660c41aaaab61cb09ac4ae15ba", + "96f19408441d972335469bf0b4364ad2cd39f3e435ba1b0e02972576769b7970", + "96fa526e45620e358f25a3d8752067f54c93554cca27cf49eb56c7913c4e8f7b", + "97320818d86e303e534a01a6dcf3774bfb866340af2a33897cdeb7ddd3f8de06", + "9747edba14159f5fb25629223ff12e9bc0d216302826344d592c5105e428919c", + "975bf5d76668602e6ba980cee7f109c9434a3a37c1b8b35d93fb5e8e7a608a18", + "977685b7cffb73f1ef26fa6a4f2b7fb5bd358a4f8f839052a2c89dbbb8a09670", + "97abf8d14bff7ab218a3cfe00a321ac0e109c828bf806129aa3b2c33ee6fae09", + "98002f7653792d2b2c41892baeed0c7871b7856cb26b78caf2195b83d11d135d", + "98302df238b9b63340092368d30b1e5290ffdca36d5a5b3d24a79f63f7dcce21", + "9834e883bee15cbce032514b9ab9335f702664589143d62a1fc71a032af3d901", + "98722a51369c5ca8848ee4144a01909eaaafb0722e5b8c04ce0c87d9aa096dab", + "9875cd5eccda046323b2e00db35a097674bff9ec0541d5b3eed73f5f8e39caf1", + "987eb90a6d108de4e939e23419ffe5a45a8f0caba5219dc6d14f049b0393ce76", + "987f394e337fb104b903dda46a5301aaff287730007f46ef25afadae5b7dbc78", + "98b3c272f364d556fa622ce5c522e9fc0f060dc4aef276bb44dd1d46d5e9db10", + "98d225867299292f12703bffec008805ea807aee642be225324659d309d6b226", + "98d302541fa8f33ca7ca098bd107aea4ed97518af5859ebb10c8a6ea039bae2f", + "98e07240600a5cea93c946519efac38d57c0b3858ae6a47638bea60c21879471", + "98e654baa488ccf9b9007a817167c6973be3bce500d7f03d1fdd73298cae5e1f", + "99428849c4d6e31bebde27e9099ab532683054ce8c153a9d10a7b65e56f932bf", + "9945f0c167dffd618b6f84eaf7a8709781a93caa6e0ba0a6c2393c54e5b7f3b7", + "9958c8980b2b1b12f3715a6f85c7aca1486f43cc2ce26172440b894f9fcbac94", + "9982a84e51afd9d4e847ca4dec6ad6d8fe88c8849074f780cc82a921f363473c", + "999257b0754b17852fb928aaf87ed897706aaa3588e087cfe4f1eea6960450aa", + "99b3b41376a8c953826ee94c16fdb7a8043cacc75e44148bc4d48cad1c6758b2", + "99c3c79104bcaaffc5c38c1c85b4b96bc953dbaba19994863b50bab0a91c4dc2", + "99e82a068e68ff5b9d9b6de848bb2669cee8923cdf9256a0794faac08625f004", + "99f302b695c73ae896aa6a57bb5c781cd38940e32e98672cd1c6387ca00a73ea", + "9a42dbb9f9aa960515c14ac7992fa86c323fd97df6deb622daf0e340b30e4f93", + "9aabcb855d31dc16f8bed9fc53fd312eaa100f5b252bbc59eacfa0743e7f12c7", + "9ad7db7364cd7d7d8697de9aea722bfec1955477728fef1bd63114bdb4e42224", + "9adcd005b974106a7f456d51d6b603fbd89c076c1d74fb6965712e9c35e073b6", + "9ae8a425fd5b7d3f65a6939b75afa5f607f45380d5fd4a54146672e940a05b1e", + "9b4eb0e364d3349b3d3f10bbe399fd8ce1368111f97d521c7636a349572fc063", + "9b572af176a0ce08ca4766f2fcfae2a2a6f44ce8138c941be9aef566beb694ad", + "9b804167e135aac643a0d5144bc61a7c188d47eb0a6e17f0ccfb5cb08183022e", + "9b8670857d068299982e3e12da1e040cbc4089da59f6652f7b8ae34584c276c0", + "9b9cb54e75f99488e11db80a28db4063de7fb107041291028130bb6637a82dac", + "9b9ddee0c23e3340dad578eaf73967b63092832f46e26829f7d05b493ff48473", + "9c0f001259d0ecae9b91379fc580c00e436b58d9715f51e6e39ddf6459606997", + "9c22c606bc416d61f094cc2dc17efd03777200b2037a27f5d485c97884b231e3", + "9c304d4792c3378127ba41e6a5ce0730ceb9098676fe4bbd4dcc5c55be621260", + "9c4b420bda3ad85c9610c1ddb0930fdae31c0509c31a65d65f48ac32d72eb2a4", + "9c4ed10cefd936aa1c795de9324409d690e05d54899d1a9e1bfc432bc45e9b31", + "9c5428b74d0734c1798659a5c2a60cdd104a631bc5c2cd482108ede09800926f", + "9c58dd0f52e8ade322cd6875f3d98cff28ff26657432258b9248a9b60400f9cb", + "9c6eb67983bac3503d1fa4f3e747f02c7a25d3de09fbd646cfff2d010810f726", + "9c97222a640a39803482f4e08a69e01a0450a3be06fe4fc9641cf0131b0e86c6", + "9ca67f9e97473ad9f858f9fd62d6c2f9c7709049980088db569a4728f7623aca", + "9ce73e107b7cba29d5ad4ac9b3a920127a0d4b2f7c81ad090d2152bd4f1bcd48", + "9d14045957382f9a922548f9dbe983f021f3d102ac0b4d2c2658dc4700d3b952", + "9d1592120d27fe3d51e75420ae1269de3b705afc3dbb5b89cce8de4ccb52b16e", + "9d3c066c345a544216a3e13515026d70b30b2bf65442999ada7ee95942fda828", + "9d80dd48429037b3827ba214b30643adf392eec87a0c3ba9d2167e12557a58a6", + "9d80ee70666e9432e5b6b7c04bf60f050ef6ef8ddc40d02486ab6eadc0cc1191", + "9dbd67c8b94fe422989baf694e2aa381769117e3664a310407b683265fa032ad", + "9df49ef86b0558c59f03245b2744e5ec1ed8f24add5adda97f61534de4fe4c5f", + "9e2772494f3e3a4d63bea115187ea31220252ed837a201930c9015ad51d4c0b7", + "9ec4ac3f75b6b18803e497d20e16b30f6b5a98d905e8afb47e895ed64aead6cc", + "9edd451a1be159f01f6432f0ad82ef62570ee660cdced89909882eb0d78c5a90", + "9eee7d2578410d51fa92385eff3d2357afc8f8e200163f99102ed21acfe7839d", + "9ef26a47cf3da4e812aabe3aa2cd7e1e6b73276f12f82643281967d5dc7bbd45", + "9efbab6a3cec41e6dc9c76eed55e3b427f7b4926803c4373ff88ebac5b4a1d87", + "9efd4b51ad36d81e214f27d70fb6e36ecc4c3566a1b659f78d6259742fcd9bb4", + "9f1009087c7b37ae089117668f728aaa5f3ecb22baa8c604cbb56eda5096b1f0", + "9f438393cc1d673b507b5163f7f99fa21fc9af792a0578aa7c117d8ef2d9614d", + "9f4608d4eab9ae10ac70d874f81e00f0c06ca749edd729478bc739c0a8fc1976", + "9f537e23b1e63ebdaeba8c83d6a910e290c782791d0652b8bed9fadc802ae38b", + "9f720ffc8c64a4886030d06ee66ddba984162e4e90567fdcd1e371052c71ce7d", + "9f83ec16c6dfdc98808e6539b3af01ce77f36771064307578895db61f041d296", + "9fa6a3c0a80ac376f44c6818436d1ba91c105d80f58bd984524b3801c57d4033", + "9ff10d17184c153e9069f5b824aec53bd5f435325952615bf79a6e0a430dd425", + "a03e7717c89a3aaa577413130e453605801d85ca172be688e483096d54d40777", + "a05f659e06fb5ca28fc989c75b2092aa41efc477340b3143c980c624439bd579", + "a070b405d69bb48b537162f88d038381a8b8253fdf90a8f080ec52c83da09f8c", + "a078011b24a347ae2c5be1d725a3d4c6ffe933a58d37fe7c5e5085315a9793ed", + "a079e506e625b7b14bf23dfc1608663d4d16ea2a8e7aa8b9e50662ba862a93f1", + "a0a32a724d874bd58f98b5f7fb50b1d157a6ebc89f6efede46e9142b9fd5d16f", + "a0b6c02e901895db03c5ea136c311331a2982addeabca9117e552b623ea8646a", + "a0bf65f9ebed8f1cd09c82107f1c208997cf8b2a17c9e2b6b588e0d3f1b3f609", + "a0c09e25ef17e1692554bdd8e22ba9a13b8919a359ae77ca15e82a818f9cc228", + "a0df1828ea87abce9638cfc85257ec619ed709cbe2192fc222708c13c256e38e", + "a0ea59161b4c689a3ecaf735f546c10d007594c544fc010f836cb7d01ea5d197", + "a122c64266b92c9dee0fa5ad5d81772d0180f88416ca5ce41e7fddbfa8ca4cd6", + "a123e568191dbccc159dbf0e2610a7dd4d4cd1d901d5fd255d84b36e93171dd3", + "a1279e59cdc47708df9e5014384f1f7824738edf0986e296526bc7763f6dface", + "a12b0d2bd7e72eb43a9219cbb0b6c5c593d94ddbcd01820f06a7a12d7a30c575", + "a14884b94402c0efadd3fc5e9dff09bfed8621c78ac78a3e50340e6e06b2a36f", + "a1854a9e9f442c7bd612070ce1950c4ed7094fee3b31beeac51ea5f545fd0c57", + "a18bd1bd8c4c4b16779ad1695fb61ebaf9d343672af48bee35b47fe1414968a3", + "a1a5267ab45977532a38fbff6777b9544779982f6f4e83c549de66b63a4ba068", + "a1ac8f2d5688e6cd4c46638539fd8e8fc274b8077ef84fd47e3cb3e148864cfe", + "a1b96a861d0b7be41db93fdf76b9ca4b90e24e451e2895b9d90cbdec14cfca30", + "a1bad290aaa74df21c1058d796b601398ebb6a0223dc2ab35831eb8d222fb5ae", + "a1dbe74c9cf79fc19bee00cae7b280cbeca083839382f8b8ffd335846c4f1ef1", + "a1e8da8c76d26d280439cb1705749ba35d2ed1cad5321c1556c45d65e5e0e26f", + "a23a151a9824adf8b8e54ea8f026762ceb74574168c081a84ee58f86948adada", + "a251b51e2297b9639a578d470f6813079c90edb65aec82753a9a1ac005dec569", + "a267aff604611134b035bdd7e8d744e6a463af69a6b4f9083a31b23d584e7946", + "a2ab534a12a3231a13d0028ae76a9672c44afad2b9b1e01ee05c49d375ae442c", + "a2b9e6fc6089a192910ffa79508000f4f459f5533cdbf6dcf7a23ceb3a745c6c", + "a2bf76422d88c525965efcecea741bd1f94a6c86c8608a6843ae2601f56aaa5d", + "a2f189b2f7404dd679f481e41b8d61fecf5da0d92409edef587cd6463f037895", + "a341833440fd48aea26699fdc1624a52eb3db99cd103932100d6a55bb6b0ee78", + "a366b08e727752b0261a79be508a76bf912873a09755df37d9d550f16436e359", + "a37b642ebec2bc4f64baece4a7c71adcd4d4cfbf1a2220e14058b3edc38fa6dd", + "a3803b601e55ef8bf2797ab8276df011c7eeef666383810ff7ade10167c04019", + "a3b25f99314074318a875968b4626d99a5b42dcf7b8c402be4c2698bead84d08", + "a3cc1097f0e358e7aaa41eccc5cc0001745cbbac0ca383a6d0185cc111c9d6ed", + "a3ccd3ee0afb191d916a5727318209623918e7cd970493710189c20786956af1", + "a3d40ca71837fabb3666dcd6bb36d375d4b6ff612a79e53c51ea31b9d7ec3dee", + "a3dc406694c180921d703ab6353d32c97a8e7222eea4e4d4c163f6779b381129", + "a40af89757b4e1f7e4c4fadcded4a6cc724a18d86d2c57296c12458f1d762eea", + "a4354136ce6b985a5d3d980778b5fddaa5ba5f445b7ce6270a9c5f85963a1a7e", + "a43cab807744dcf2bbd2d088e8cd43af87096233291eae48711afa8040b8249d", + "a49d7dc6f8218a62df895bbdb45910dc182f25ac2b7344427198f0218d0438e8", + "a4b8b9783603e6964a3fe485be7d515cdd6f9cd92baab45ae4563ce8a445995a", + "a4fca0e1b8fa35e1450019470bbacdef5dc45aac4feed5e002547d9d6f59476b", + "a5351a3819981a736641c3c50a4d6ca3faabe5ae8e32edc28bc1cef141c40d6b", + "a5617e451323f4ddb657cc1ac0c9b2b08ba01d0c9ee6b5bb90adef7d4c333568", + "a5776d9e11b439905583492e705b91a1b42b34abc508d79ce0e0722f6ca6acef", + "a583abb75ff1fa4bb7d52db7b9109127fe822d281aa9003149444f944021bafe", + "a5b422afc53594c54b4c4b16be01e923e9c79f8b593736f49ac225c951cd8c14", + "a5dfd9da32550e164a6c63eb08c1f6a2547987d2b33969ff7b0215f8ee622dc6", + "a5e995f954f1646970f15c779b6fe6be94c0ee52144ff5a2e9d72eff33ffeb2a", + "a621ad9f22c1fec373c9baffcff0c941a8d7f792f0a18586dc5125a627878116", + "a6683640657870fe56c4af8333bb373efe582acd241443c5de17ba3453619bb6", + "a6791df4f079d4614a3a1f216f1d86eaa5f3bdc98bd273f56b69b0592b41dd26", + "a6eddb8ff9858cd674acb7cc225eca94121cf50c2a9de3ef2ac05ee0adf5b0f0", + "a6f6c8f1dd7d6a28fe80567947e1c77305fdfb7190e3f94f79b2cee4b8449d3e", + "a702a62a0ce17078c88d1416a10b81c9aec85b78d2a706cbf77272e2efc1a4f7", + "a708819bb0384b19c9ebcba75702ae96f641ee3cf79fd712077e727912c0457a", + "a724449ef878facb7ee0b008e220ea864963f0e7e39cdd31a43864ef9921025c", + "a74224ae36d9293a32a0505281ad4f1733bbe621873bb0c984f8954d316daab3", + "a743bd88fce22643ef830c23b68506dd375abac6465ad60b4cb8351040e62454", + "a750263eddf945283109f003cd8046cdc153df71899d09b30556ca8fff014fa1", + "a772cf9ca1bf588c36712a36ddad337b29af5ab0401c3264dae1a0bfd0a8bf0f", + "a7d7fdd7df2d66a787622eb9a27212a1dd3f1f8a07c7e77c3be9fc959d771e69", + "a7eb99092abaf7f8b9d0670b5906ddf805a1773f1b4f62f6dc9fb5538132a693", + "a7f0ca54f2700303978ec97d9f365ad1f917ce94f606d2595f0a539985bd6e1b", + "a841679522059f62ce5a950cad182a3ef1ff961be3332d2f38192bf05624b9ab", + "a86b3699691acf7e426f5c0ce4272cfac4d634a3a106902dbf1938e2435dbc2d", + "a89081c801b7308c86904b0f441fea806e7acceed9c420f90078295498919d44", + "a890bfd6614b9ff75e4acf074b26d9f7875caf6d281bbceda0e42ecf78a7455d", + "a89c9af7c6d1d0818e2fdaaf3dd920dd20d4afb730e80b8d2640bdf1fba75ea4", + "a8ce83c637cca3cd88eda23cec113036b92b12ea61ee6bec7f5a00ccb2f946ec", + "a8f018689fafbdf00bab10aff5d77035210177e8a86a94d7ac6f631271851cb3", + "a90d7bdac4e8db9bd85268ea2b0601d32b28138dbb1af43ea9ed8475656fc803", + "a914075c60372981b697d3f25a5d0303feb5707b71cbdb81aa59e20afa38a69c", + "a933b42767b675f23c55a3bd0888051a90a6db5c12e528a1109f45c86d265b9c", + "a93d49bda71d2aadc8ab5ebc10c9136812a71ddd0e0e3a257930a06d9ad120a1", + "a950359abacff930799b98035d2dee1a0c84943487c21065b25c417a480728e3", + "a956b733a72c588a886dd80f07b9a50a7c50c71eb090eedb9813bca88fcd414c", + "a9d932a62fa73f6d6a96d146fe28c041b9452dccd5a6d0131f996ebeab7ab8f0", + "a9e1f90410ff4001727d240e543b1da201dca7540cdbd5720fbabc52d55555c5", + "a9f0c0de38ec436a4af0aeb96a04249d84df15ccc080c0225546d5006fccf5ac", + "a9f7ffb91f44bd27ad011c0721a6697ddf77dcd7e05a2473c2b2ae4b45d6ed67", + "aa49fbe535345917bb469c8f07a4bb19b263320a1b878247504ace3b668429ff", + "aa634009f9ffa363413b6b8490516ec76b3e37545a792226271e5dae9333653b", + "aa66f54b57d2ce29fd90750a393ddad20927e731f383c2bde590741963435ceb", + "aa685e32c88aac051f95e9663d74d92536871bb1570ca2f880e3649d72dcb4df", + "aa6cbf5104804f5cb7e86fd753e4e5369ca2299c248fa5386b2c6441ecf79107", + "aa89b61df7e81d444be2c2c9b7d86a55f27d70841a4009d08bf2f6f8809b7426", + "aa8f9285a8e65fce85ef3357eaa84a104d40ef1658045a2b95c913c664d90598", + "aabd8f2fd6a25e53262d8c8cf76418c49fbe2de5ec7f69b813a9013e64b2a608", + "aac35f875f1806cd3f3547362515bcda9d6151c2231d41c0f8bb62ffaaa40e8f", + "aaee3bf077a76386aa71f808b32df60a5b57cf0ad84413c35999abfdaed06797", + "ab30982785dbf8b2be3b1781cc58d1e85fcd0e061889e46d23b7abe964202dde", + "ab8664776033525115237381c15dfc31c621d66c93c0d28c58952644d0bfca09", + "abb567bcb73b46d0668540edfb8b70a816a17fc2d26e8cce5978631a3c0a4b71", + "ac2020054cd25ce5be47042209efe44a780470610210e47a07bae9b1d75c6d51", + "ac59fde9d43a11dd89fedeb2d49a2ad8e722b2be68b9fd5c5ac9520fb0cadcb0", + "ac6dabee5b5707147eb79e3bb6b790df357dc26f77e37e5b6af43845234e82a1", + "ac7a9078774e79a56683c947338c4740b4d0218ce1296b660c91fd8fae06a0d4", + "ac908e26fed79816dea69ab9577341edd806b565fcd971f8a86fff8e3db4b932", + "aca04b419e208d72cb2769900f22462f9df8b0502bcefec22556bdedef44a110", + "acc4ea38d065d2f3336ed088cc991b07e581d37456d4370fcd40d81f6f36b7e6", + "acc5c7ac1c8bc43afd9fc0d870d8a65432604c53294a77ee31eb6f60637d130c", + "acec02d51cbb9f79b85881f375b122f7fd1bd7483964f86729ad4aa8cbd922ef", + "acf9d606aa42cdce69f9bf465a11ddefbfb50d2b8800efae9644276aea08b2a4", + "ad0bf4a8f205d5339e85a4212cef2b20b7ced6280c2e9f5f45ccb004b6af322a", + "ad511b083a403a2b6d901119d09bc310afbe220f66b4cbc997ce6e1b43923b62", + "ad7bc98c757fe2eef34f27ca38ebf39cc7560a6475ea5fb65b53a6ac62361296", + "adc0bc79f7cdebfda01aefe2fa44fb0406fca002c0afe4f286022b704074625b", + "ae16ab0f91f2f878129865aca27304a11882aaf09f7a1f6c9c886216601fdec6", + "ae21a5986a959789da9807444a242b62794e9551de2b3e33d472a60a891e9181", + "ae39a4ac22a55fcf1a8a78ebdc58fb8d1fe5f7a6b4a51f0ea36e65cea0c9c868", + "ae39ab3cbe928abdccc9f6eec045dd500e343938bad19e3ef945b06d5fa16e60", + "ae454bc89861d4a5c6244640aa99db82c11f4196f6386e5a66739bc9a458e534", + "ae5bfe44bf7768585ce234c04d1d0494bddf10ccd26964ba59f0dd47f0c2b268", + "ae8e2940d3dc04d7a6a8a6aa07aea6b1c7e943d11ade25e6dd5e489dbe0ee8d3", + "ae8fdce92af832a981dea810c831d8c267879922314f42c4e9f6275776c154d7", + "ae936a3462d7e22809bd94ae5931ae70430f00031ae97dd2bd0e7ff78343e728", + "ae9ec3f3a37a86ce3e8f4605da3f65632e534875fb5bd4ade1db25c1beb1e1fa", + "aeff9c08f60704690cebdcb3f6ec1b9095c9e7d75504a384fbc0bbc11c9d5341", + "af1db02874a387fbc5f94838d3a0b80d3a940e21e95730f6b7d4cd6bcb6dc022", + "af40ef6e683b838b9d629021d698846993a024b916555f7af5ceebcc0dcb73d0", + "af5d9abbeb59f0510b4d1f5d5b4bebfafabafbc03e030c467c68fc1026a60db1", + "af6ac547c28b21459f56a8506afb797e43f8186c9c08519b19983ba48a8e48e3", + "af94252639de288cd696f79c779ec18bdd6f087b485d465a9a2d53be647e4bb3", + "af9874ea784ef6a5422de0f475430314d5c5c80f2d627c9c4d5b101f2dbd5c95", + "afb8867def2977d96ab00821322d5f342c123bee85e33c9530001026a22b1fc2", + "afcee626e7e0ddf98780a26e9ce0627de9b45db75e8bf1d50f86844c0c8e9018", + "afd476b9cf674144af007dc146cf82e7dbc8fe486b80888bfc9d82f70588a23b", + "afe06523c00a6e5b945484aebfad43ef059ad8d648cb1bcc680ca26c034a6f80", + "b02acd3d0ec221ad7b8893f16b79fcd8bbd831f74409baaa2184b623a4c63829", + "b03763967523aa5f431d80ce741e8418d8764ba8316afaa6bd804d51f9247553", + "b078f6c1ab0b1f8866eb77d939d2d70f9ec748e9a45cc09fee00a1b9f8cc7898", + "b07c85be90171841365bd23640932d9a41511e389f7ffb9adfcc5ab66d89bf57", + "b0ac14cdfcd1f096709f5e82ca65c67c2e0a27f2ba6d688a954c356501cb7d20", + "b0b04d5848dc77d7e5908eb8c0a6df6887c743bfe1078b13b14571ca3269ccfb", + "b0b999be0229df105d8b877277e75f63a392daf57752c18b3fc14a6c1ff674be", + "b0c43e875e30694fc93cfee195abe7069714ae7655347bc779694bc60694b637", + "b0d3870ec2a569f0f320dd68d4c728fa7c82c1423ad214286e3e5f9de19086cb", + "b0e33fe309f08f0aa5750e1184da66b702ddef2b9ecbbcf2e3fa3d35b465a72e", + "b10d1de22a03af66a12caeede346d3b183929a809d5beda7f13a42e69eb190ec", + "b11f3bcb872439562b3233ed7bc52f2786a0b449a41d6701b74666208f092e5a", + "b12ad0ccf78825d7347c3a7bf4aa5d5ca460dec3ee1584db25f09518fe20baec", + "b156c635011a93a648f076dd68f26a0c9aee6e5e674ee4a207653d65f451338c", + "b1628145ee10ce0474906869897ccda3c70dcc5334d33223d950aeb2ff9f0f57", + "b18d03055f8f39c8a459af611514a947e6b08e51d17cec2d74f2159cb22c50ec", + "b1f012120595870ca5d56c46ee5882430dd91ac6be210da64182088b15236bfa", + "b2366b6bca134f7a8b7347dac3bb0b100db6d3cf1de9ea0eeb417e1a324b9538", + "b24726902e3146d4ab97bf67178ed77840fe5d9ff6942d105a48c79f6deaa0c2", + "b288350c5709578836c834d4955a2fb3faacf583df46a9edfc32c44c116a8ba9", + "b29ee483be358080d8f6db6047e42600fd7481adc4b86dbee6facc4199b39ea6", + "b3088bbd2f0cb560ca5edd8269eefde6e52e98807dcd2daaaa04ff531f49a983", + "b30afabdba5dc3dd57cfc03f8c8142972748ebe769127fb481171ec84d5e6782", + "b30f9b407fb8ab04e4108164a282def889f0c6a659a6925051dd82835f739858", + "b35a6351bae968e768fa415340252ca2358af96c3181ec6a2b414656302165f1", + "b3a4be33968e42a4b8851312c7d9c996e078c7ddb25aa484a34e559c65d17fb0", + "b3dd9456f5fd8e73e0a9c7b0ef5bbade876a54e0b1f91868efbefaed8e6b62c0", + "b3fcd7c5df285b09027b8ad2356c12e05736373385da2ba556796de7e43b3094", + "b417c0e7dac8894fe8b58e4a9b2ae13525ef8065d68181dba35457570e7e19b8", + "b44002ae51ac10d1d0851b77a8c5e5109e7369fae9d73b459e7b3aa3f0f6cbdf", + "b45ee5251159a5e01dc599e6e1a30a304859484cbd066d9ecb6d8cc0e714ec27", + "b4782cfa6af28c51d1a53555e241f4f68589f6e89c84b7246261344db3bfae0f", + "b47f6e0b37f9793776fa983dba2dd994a42c32346b4b1f4c9278ffa6ce3529fa", + "b48314748c45c8dff55a2316ed448bd72f20d231778e80368376bbe483a2b1b8", + "b4a51cfb2f14fac383167432544f2b82e7e120014acf948aaadd33ae1ed331ca", + "b4ad08fb9090c4ab98399dbb2e98a5cf0b334bd88580b38250cf13c9d386b4fb", + "b4b38ab0899c091d3ddfc7a88603261cc3446a70c5bc09d3ebf651f3f0d6832c", + "b4ccdc6ff79881b8e5ce94394ba1e8000be7611793c98624c09b7737325617e2", + "b4e174aaa52c030cf6b034355335fdc549f44681908979ff6500ea6dde0cb67a", + "b513c611ee93c3033d681ac34dc5334fcae74e64dcdd1cdd16ca84269a4cdb6d", + "b5326045d5af7216b123b3f45417dccbf6e9cd74ce36508cb1801793f1ec789c", + "b58a15be3759b35178f2a230d09976799f0bfd2bd4b47f730ffc5133c8ab0d37", + "b59a22535f3efb4fbb1a9e3cb7a414a382ee70835a3a34bfe3f7304879a0076c", + "b5b5a06c87e9b6185f039dc51b297bcb413f09f577fd54dabb2522256e0d95cd", + "b5bbbafcf9f6b175e3dff8d0aaca59059cf2299247cd7d825304a26a13e25b13", + "b5c1a87db5d6e07258cff72927b1820ab5e83b10c0029456d43476447dfa473f", + "b5c242e5c5c2dd850f6448fcf141f42aee95261f7cc5ec287991dbc845ee0cec", + "b5c7c8e294414ef68009c0ae31352d550b47a4ef468a29e034a81acff91f6c40", + "b5d318dd57a9c80b3e8c3fb510c8109ad38bee6a3fe0e4a495619be1758227da", + "b5de3dfc01a42bd025e966c95808dde6a80c364b73cbf8f0d42f1f5a4e5eb023", + "b5ebd74e821ac2afbf6df49f7d9d4071750dfa86cfb91dba56c403af731dccd9", + "b5f2d01087a437e4ad8e7ed4a37db82ed01e39b8ce4fc4e5213cecd480371678", + "b66b4005cbfc9b8c8548e0348990af5da72fdb3bfe9fe1c0e01852ae0ee02eb8", + "b684f330e18b73f11336a3ff76bab18dbdbbdcbbb4e9401707118fb8d319ccba", + "b6b886286302a6c5faba0e67d898b1b416706ef980e779930e8673699c7811ce", + "b6d1a49654eb4109c9a1426b33c4b3eee473f744c7dbb6f0fd0bc859d4d07ff7", + "b6d8eddbc4f2611621906fd72f117fb090909ae263c6583654380efda492570d", + "b75ec2550dcf9a890403ef27db99f24849cc1f172d08d68e57e5aa2639b6db3e", + "b7fe798ebcf5157ee4dd11701414ff41e15d63f74ca55895e53db3a3e6b7d68c", + "b82d43dc022095090b121186bf6029625117111d18c836eb0b4d463a2a00d70b", + "b84de002e997f9f9e0df1349dcd3238bd91f30dd163180a17dd7cddaa2df57ed", + "b922630a7c7fd034f00cf9d2f940f32862470390df5e6b97858a32e3b3411290", + "b98ea5c34a50ce0e0dd1cdfd6075f7b1832f0d5d1252b7459feaf0a9ddc56dea", + "b998048a5255c631eb43e8cc0c8b892e29b7b9de408e4d99fb3447862c734e7a", + "ba0e2abe57d4e8976f7b6dee8c399cc76d672974a63968bb22e74fa3f3c07c42", + "ba17117fe128ec4754d6f6d360227770b44ae0d3c0ccd6cd676c15e61a959b6e", + "ba51369f3f4aa59d6bf7f75cb9359705bc8da0df0c6d68c8608f4286d76d333d", + "ba73c3a2d34ebf5fb4faf99ec8b0d20104f805520fcc4242316692e239584e31", + "bb0abbf276e17a783bcb113938c42f3c3c47a263a3bdf13e3efe2881ed0cebd9", + "bb14ff83d06e928142c58f803627b6b6067f7d7010725f231d3bc3991537325a", + "bb679d31a017ae04a18eaebff3f6fb8f429b92f73b9da58c65f8f662af751fda", + "bbb6568d2f337cb7194aa2b2aec0fb6a188c46d06a714fd27f0eb79f524c8315", + "bbd3c5a145a404a237f0d5d6900cc4b3c1767160954322a286491c874a50da5d", + "bbeecbe54c37f58d527baf903067f9dad4708779c45cd500d58c01f62b7f3b32", + "bbfcf175ab1c5bc358e67c71a7e7b68a7ff599eec1846eadc93113b45f15f036", + "bc2bd8f50cf3c34d2cb52e4acb44cbccfcf12768a96e14c39f8a117dc8d1f120", + "bc547c6f717ee21173258f4c941a3a3c4acd7c8578ebf8d35ef8b98dff04ddfd", + "bc5f61fb37b1c01c737ab760249b7f12da334e3adaa471ecf20116794b71c1e4", + "bc64c4c70b7cd5ff2acec0c8eb0b7de2b95c0f95df1107612c6c86aacd750b75", + "bc6eaf5d6e55fd6755fd47c42ba065cfd837ee952d1baba7caebd5b18b5a04b4", + "bcb3fc371ffa17c1c073cc2accd53923efeb08a01d24ed7d806532ed460eeb2f", + "bcb499ca031752491bf542f8ec41e8be53b4ade5d1955f0e25156181d3bdc13d", + "bcd9ad96c094d2b37d619be8f3fafa21e1d7c541e355a5dd8aba1427e2c5da0d", + "bcdd90d4e12d43067f3bed7dc8774068e4f42335771892bdb17e8a69afcee82e", + "bce86da49ca3b3e13bce9c8a69617ebb5e761edbaed7ac9cb914e3ea681d14d6", + "bcf87d42810cf1d06f5b7e1c63f715ac6407cb2fa729488030612a8c5be72cf7", + "bd23a935c7370573992718d3062157b935bdba9e90d5ce38a2f039792ac7706e", + "bd36a8c360935cbe0fbdf5b58237bb8eec29a2620100a7ea3363f1b977e044c9", + "bd4742ece96e7508c7fe3019146b9e886d39ffa7c03cbc486edd29e807e97ac3", + "bda5ef41f329a3daca0dcc206bf1fa8e743c267226364a383065e9b453d7a5a8", + "bdb0721795731acec21268778b65231fab27a672cf0c89d144ce4de017dd3da9", + "be02f86b8ffa1a1090f50d83e4592e652fab03b76eac2de1d3bd1461630612b3", + "be267196ec200de80b7d4ffa4fbee39772cf74936647dd847851289092f42f62", + "be6cc9db2102eeccb24879afe2971e6fe5aaeda2ef1042fbeeeba79a1a1c5635", + "be77bcf22d386e03355561d906a4d01f536230205251150ad805e06b9e4bcced", + "be9678da0d814cb4b23b38dcc097182a377fd1c709439cae4d9d8549ede3e1ef", + "bebe2a929aaac911b350d2d921386029b225bf1e3d3d08da214a9a5dc3bb62a0", + "bed10168e4ddf079981c1b85076f28a3ce061584175feec439e0f737b950865b", + "befdb6fb0fb17e561e51bee3afca24285c8055bdcd5ed793ac12cc49939185e6", + "bf120a9a0f7df85c1ba2da3658c8978748e3c3b1bebaa82fa8d8439ecb4da32e", + "bf12504a200f2bc6c6502eb732bbf3eea946be65c0837729ef477e1ede70ce20", + "bf21233f56d276bfa502ad634f5e8c7f7ea04421db5d894da274b2652d65eec4", + "bf3eb231d9cb0dca6d3f37766e4f5ce99a62ac72526d638dd7f28a8923834afa", + "bf7e9e164281f29ef0eb79a1e142065d86ce32e9c927c9d94c079ffa98174eb3", + "bf90af2b98795bd827adf4c4b2d0b7a583b37f29240a8f5ba4edcf20c5a9f3c1", + "bf97fb94799f55df9377e1a428aea35a9aafafaa53a5b8d65d902542d8c6fd23", + "bfd67afb7e0d59083ae71a16f8540a54890e5fb890350363b0fe33e344e8879e", + "bff20b1fcfb36e27b079aae822ec04b4b25a2a5809a1be3387eb49542ebf8735", + "bff67f7214a8e1eb41e7bd601e76092330f41d8ac24b5824972c1a4d6c050be3", + "c024a64ca09b458d19a05c552070c1342763c6d68337bca48f0097432fc3595f", + "c034c03b47b63c202ec02eed6420878b1c4bb0e2ac260d91df82c54f1ca190de", + "c04ed340bae75e79cdd78942e9335de2837e00b273efa911e62f7e53eab5ef25", + "c05e4185c5bc92b4a260dfc81e9faee47539d118f13474933e5ccfcf0be3f740", + "c06524578ce761107a87557d97f85990be11337eaf5e3ca90fdc4b260baa75ff", + "c06549de3238c0c6e7b2f07f003b2f0ffb4c9c27edff4bfe101a5795e0bc6f5f", + "c072202f9b3b28a346cde65ce651963430c942590739690c3084bca8119015f3", + "c07233637a353018d3f7bea24ae21554a01430c39ee202d62a458c13b199c5af", + "c072353b9e9217c2abaed2c476d9a771b759c59578cceb93d264740aec6a4384", + "c07921530aa4faf4287314001c48ba740df2388ca8ec94dfbfb74a3eb50d58bc", + "c08c7e43682e8e4c87d8be637f9d82d4b82429e4aa372e6a2e30d373e93b4d32", + "c0a1f61328f349ecf9f32d154d912097d236ca5200c509253c56dbb2ae7fa413", + "c0b5635ded2ca1c72e4ae38072bf0bb69a5cbbd13eb59acd42599e3e50aa4066", + "c0c4e1754896994da712d47d8507b7f875e99a4d5f65910d989c73aa800d109a", + "c0d55523dbf8dc16c804f877dc889ddebcd2a5359ee7148c21d78935c2fac7f9", + "c0e43d2d0c4f96d5a4d1d3ab1f1469c31d8efd1c05132a75461cc6e8c826f0b9", + "c0ed6130b0bdd0359e918cb5d3b2f6423a42fb6efc4af3d556d1cf4e984c0cec", + "c0efbdbb716f87ad84e409e078f1919460f3de01f2b33e53ce0ffae28b8941c7", + "c0f652e25c47ebc5fc2e18598b76e83c656643bdb75602a92289fb01fa0066af", + "c102e2f2303f130e73f2b2dd75b6603fdaaba996a1ca4ba043e9c136f7c15420", + "c1253ef0ffbce0d7f76f114b7184a12dc245ef7cdf7eb3ea2d51332101a4ea08", + "c16f94d277ce05d085f72444894a40c6ec0479c0a49398e2da51073effdf2fbb", + "c16f9bcd2c4d8f79169ad9955e892aad835f6ced928d1579087faf42955bdc19", + "c180609592cc3870f3dc650c79360da83f5102115b4df9aadd7023e0b7691554", + "c18677c93d36734e09db302b680fad865fdad625dd42ac98a56afe0bb23c7b79", + "c1b11ab212a92c06204de8eb8e33db59c06fe2cc9350b692f91238af02b77270", + "c20c660029c5bb5c25ab60ffa6cbd63eb50e1ea43dac36926bd32c87e0d191c9", + "c21d65e6dabab9487a553c295e2149e194dd0b17a36b209dda8c6acbc49b691c", + "c21fcd5859d67db5a7fb511fe21e68f37cf6d0ae1a7eb3695e3e46e74d1caf67", + "c2288e2a9b6d9e23c664e5a238815c204fc92379ca4b1de8e369d491015f3fa6", + "c24a3fcab5943b4336cb25e2c4496b62055967aa48d8876577b9c7bd67cf7c80", + "c2a1d0fefbd6321505cb03d69d49cdef50ab1cb792fec9cf4a23a2dd0ff2cb62", + "c2cfee3f872b61bf99bf2bd71381f5c7cd279e249bf804b7b8e2861ccafc7776", + "c31106c0aafcb04e0f97c35994ecb7e89348b7948861d5261adfa6579c5f4f1f", + "c33032b249679e0f5113a3448b152fc582acce16b6c145398ea5da4b13027e68", + "c34b9d242a85f0dd41f34a88dafc8faf1b3ac03488731cd6131f4083819dfbb8", + "c354687dcc12700beaa4c12307ed810a27f3ce079cda07e476c5f80acdf9d8be", + "c3612869eed6d45631154807b57745bca7ce167ebf45787d9bd5bcf8b234fc74", + "c383a9a9768dc0b25b1259b736264f763ef1b1c651cdf24a475bd267c3b951d6", + "c399984d6b0c9d1999e2805b521bfe0a27488982249c821889da7727d760f1ac", + "c3b28e516f3b74a15b151adcf82fd9aed69d90a91aea048fb03ebf3ccccd7de3", + "c3d2ab670f1ccd44a266fff4bde9c07cd7c7911142e4a8abdd2e23a07d822b3f", + "c3da57acf36595c800970f7880c8852e7c04d7daad8a0f27105cefb8068eb5e2", + "c41af0d05d256ace803806241254b43a9ce52c0fa512ac9a1a5027c6a26a6bb2", + "c450137ddf1ac2940dcec86a56e99ff42ee74a420d86eb28cdebbf2a25960012", + "c4536a2b42d40e58686bed1dc5c422da58fe3cc6dc8e7e9e6f9ab1e9120019b2", + "c48a0a0c5647f069fb67e33e9475256d4f7edffebed6d1472825ee72753aba88", + "c4aaaf90719f53cd17a3187436bb383c505c1ecdbe6907c43f5513461b3acae4", + "c4cab42c78c297b8fcc09523dd25ae6a8a8086e5b4263ac26e18f1a333ea1787", + "c4d45104076f6520102d542848213e0889b2d563c9777c13fd4389c7929f0a70", + "c4da74fd5cb8e52860b27008effe3da41e6360d9934a66a52020437cb0c0a153", + "c4f814cc6ed768d0e56a13d19679631906b07418c3e033ea24f143d827a40193", + "c529c725b71da442936bb8ab052712caa8812c1f3176170f460642223274a589", + "c542024eaaaa752bfc798493ee4945c876146979711fb9cc49c9b71d84a508ce", + "c552a359e5e1a4bd1ff4d6a5313e086d4fc9382444d7bbdc6372d25052301ce1", + "c55839974c8b040c7bbf1adf0077afb3d0e699044c74be927217f2c445f7091e", + "c55c2a1121a5d3fe7144f9a737652ea0e0ccac440b34ca5ae6f739322c14971a", + "c5616f4c2c8e787dc1a217841c54667eccaff4d25a6f203811b31b8171a283bf", + "c59ee61269705faaf361a3fa88eea141e108dfe28a427c6e488b13ae88de7165", + "c5a5a6d1f10a0a1d8d7eb22f171e311c031144da9c8e1aaff5225089cec00163", + "c5a6a1c441fb287fc9d2416bddfa134666d87296bc9079fdb40ef3dd88f834af", + "c5b2951e583e3f5d7bab20aa1f620dc8e62071cb0d9fcf36011cee5c79f24f0a", + "c5c0af94a126c1b7d8be21886530273bf5db83f4ecbe624034dd36af5038a16e", + "c5de598be6e1b4db13e787c719b2852ab2988caa4d1909389f5c60a3b4839232", + "c5e4cdc0aa0d84e16a372597c6261b520cad72f635271c0942fcfac0a3aa6849", + "c65f6259756a3f7330763fc5b04a03c6a7bfa4d7adba7b24cb52bf25f8ccf582", + "c667a9f939ab7cbc24f16808a496913e96c956f1a30dcaf190ddebbd2e2c37c3", + "c685c7deda7bee11ae26a52bd9c2e0bc69725db3da8c11ca31de05126b59444c", + "c6a6f1bcce0906960d0db892e8a31402332a77c0a34f30af2d1a1fe4577f1ca5", + "c6e52e8fc9cc1326987043c2e5cd275d9739742e759dc500b48dc1e1afb9990e", + "c6ff80d4c29f48b524d61b8d7637ee883a639940f373acaca91985a8fd2760fb", + "c723c0f98c601d0f562a1069910741b66a215bf266cf88a2bf46d39e54c8337c", + "c73658bc27f27a413c5b379e75960bb121ee0ffb6e20209bb8ba3822d0d7b73c", + "c76e6da2382e79e7b091a92fe16a4b1a4329533e5ccf782dfcac91a092585722", + "c79078deea0cabe9b6cda04bbd16a271bf55231129a32d3923b5d61f0e164493", + "c7b43b191278a9fbe9cca46d7250b30521923e3c969a1c377a266770d2994618", + "c7ce0c2ae2170931ae88ed3aa666e043a9571ac1c725ad769b055a7249bc1fae", + "c7ecfe011cdb48ac32b4a4ffc380ffcd844620329f25b3eb275dec9548fdb13f", + "c850c40819cf6be6039abc709e34a90a41d0299aacd5432bde96a32b187c7939", + "c8d5f6a03cf5578ff3c27f20418a5fd7e2f75936fdf843e7c6dde8119b2202b1", + "c8d631c5dd8e4329af5d97d73eb7d0bd2a0c7f18ce291731c4ae9b5540a300ba", + "c92d8d90bb4f444c193024326c23bbf6e700afa1564e9212e1c39529ed08c2b6", + "c932e5515c72dc46b1695007001ce1032a77d3a8b1b313c2c892ef4b2c9d7d4b", + "c98f48a896e4f421facf2c022912144fffe504e3db694226092561f802f22830", + "c995710e45ca54270ba28255ecf88460be706305607354f0ea067c3843aa43ae", + "c9ab4df781c5b5c2ce6e3521532cdf914185234fe0f0a4d8efe2f882a53ed368", + "c9b3e6759a8e4bd06b705333df6b1e39d0f285fe2f389f5aff84c1d543ba0512", + "c9bc72a44e5f32adc8d7f3408a983a192e4fe97bfa408126b3074c0b759499fd", + "c9cba066b29ed7b13272cd5ed7ba87332d37b1ca861a14523b68537a4bcfcf91", + "ca0dfd82bd822da0e32efd6402e176a98e724020726654e2317af944b1ae8471", + "ca79360ba5ff1285e65649a6d3a83a99010576e08d618eceb90ad103aaf03490", + "ca88c8f7dc5e0b1dc4b8f5000cab018d8a2c7b1c1f160311bafcb5e388a9452f", + "caa3bfb83ed58e30949a863b79a2af63f9e16ee16b7d7c311d14362e907fbe9f", + "cacc91e38cde889f3a3ed7ed8b914d6ffbb1231c0a4ed13e2bcff10fc3464b43", + "cafd41841a6971c11257fb7e6189c7785498e20635721574650fc381c539681e", + "cafd9a430506f2cbc372666b6543a890bc6934870dd4b5ee9040a47032390b94", + "cb0517ee94da12b8eee80c94ff7346510d18d7e285f951a87e78762cc47d9e58", + "cb072729d8819401aae11fca56357df42a34fb91fd5b902cd006d7fc36af2f24", + "cb10006e3b9263921ecadde5362e86bdf3a8146c22db36e458f8f4bcc6e526f4", + "cb1312d8be0cbdb89d8cb7c9e4971edd45995484af46b1efd4fe7f1825e4031a", + "cb15b5de90f1f391f397471faf423fd6e70259c74cc4209a792986e4e9d57bbf", + "cb1da3e4cfac0866b4f42bb6104d1ca463379e9a1b15b2ad7a1669d70f457ca7", + "cb21511ac20b769340b179d9e5b6f1683b3a13fb44093f68ca225be514b20b83", + "cb8904eaf404eca0238f1f2c1532feb5746bd94ed9d7577383ad11a8f4e62d77", + "cbddf4858f884f84126efe1c43b65fb1269d916d0f94ee73cc4b3b5dc2949e7e", + "cbf5706c38795e3efdadf678690e074e3f2512ff9868d8665655811d1053a5dd", + "cc170a33d8290a975c2c965f0893eb728ac611d55b3b06b87be576b85bb57877", + "cc25f4b88d039fa902ba63f48ac72b27a1c82c7a1552040ceeab9dc788052d3d", + "cc30c9c182b14d50bca7900e9af56708d83adfe54179fbcae40b2db471db46e2", + "cc4f1dea9120114b4b3bfabbc5d96c080ce5ec16d50501bd4543a2817e6b834b", + "cc68e4b42102346f64fcfc28a5407d758d87cad2b058b6b500587e60b1d54188", + "cc88dd027865c2036edfe3895b017b69977f1cbf6a90cb1ef8596f5e0d886f78", + "ccacaec81b3af38e6b70ce193c830d1081aa02a06cf99dbd5c13e035afc0a526", + "ccd0ebc51296dfcf4e63cf0560dda61e542e6aa4183958b1595f30a01057778e", + "ccf399adea2bbc343bd217129eb44c3eb6516916108c3b4e8d126179356fb07f", + "cd371cb03771295a33e5ad744fd245c02d34de6b06609f3923a6c95554b66775", + "cd380281327a8079b9eec2f202ec9f51f4008f9ce6d096f523dc9901015bd901", + "cd3a0a64425073cb18a40113b21de741a01166932eac1b1b8bd358dc85832af5", + "cd456c636492018ca447218c02d8fa52ef443b088cc6a57a3aaa83aebd71c370", + "cd4b0b156adb451ce45ef6f93d44353b81d1ae76ee0d8eaacab68832c26ba075", + "cda8fef741223993341e4ed5d553000635a1bbb0c66a2b0a5723542077dcdfc9", + "cdac002519a0b137114e4761cfa75f82e189ce7ed319655f15c37ef04e4896d5", + "cdc0e13996861748790ed8d99cf83a43a8bf8c1d2063dc89ab890325a5eefc15", + "cdcfcbeb98cd99aa567ab0c7bff236b3572cc9a2500507b189f6f28de7425860", + "cdd0ae1676d3f8a072fc5218723f6e2e515a8258b82f51bacdc2a3f78c9b2348", + "cde33a13b4c099644749d2facbf2d4341a9e9ee10354095f774310cdcdefabb1", + "cdf8d5aa0a6cc5b51a2944f3c570de8794e9e945317c3e1a5aecd87cee420299", + "ce005b7688f239258d6e6714763b3a37aabfbc9ec585ac5fb624e228c2a102b3", + "ce0823ac2f688885cf9e1a6a3f45844332ea6a428d685c1cb041feeec764ca32", + "ce7855544d63ef0544126ec17880120aeb4cd2b834c5bec78f33e409a1b117c3", + "ce796c3e11929e06bda4cf091d8108ddbbdb1f16b95cbb86dbbe3950b36dc8b8", + "cef2f0750274e9d6fb19a80350a41255bf66e58e100a4bfb1fff6833308eb661", + "cf0491fdfbf94a1fdabf6bc700e5bae8a8aa4ede01b3174cd79f8c8fdefa8fb6", + "cf1206186d32ff4c5fcde159ed0322a4a16ee0be90a79192f80830cab670ca56", + "cf1421b73484081f4683ac57aba50831a901313c0e4010c12485e5b4d2d54445", + "cf1b6ca2a327b7a0553e51527285f213016fc8fcdb2841895a5350141066bbd7", + "cf596932c32af9f99e3a9d9c4a434c18d49b0698a62ecc760ceda02606a8e77d", + "cf6e1977d9fc2dc361ce318e15d9e0200f80299349b480efbc05bbbd8fc53c85", + "cf82c6faacff4d071925e08e101145a1d140a21a95c246e6a0c221837cbdf464", + "cfb37033c2cc4ff84000d6587e152ab3bc37bf44d06a3b8d8371e3ec36b0b7d1", + "cfb3713401986e4be228d6139d98c1b4b5059d8eaea055cc4817c965d0827347", + "cfc5981766f3757e4f015d9cb9c905266e1e5a068d91cc23c9fb54d0f4626177", + "cfccd1f89b646ed289d68e41563e44cefc6d903526af3b51e003dd0e6ea2eb4b", + "cfe2005d6603a710aad7afa82c13c9b27c382f62cd602d371e2505dde16fa1aa", + "cffb93ee92c901b673855d2fc800de2344a16785b8e750abffa64b12b4643e81", + "cffd8cb1754770684de7bdb190219a8c81d077ab861f6f1a2838d5b8caa24af1", + "d016b007e6a831837ffe35c8ab482f5c1220e3a96fdb81c493576dc94498bb0c", + "d03d5a0fd96b5ac4ed689dd49336670aacb7b3d0f727347766b71119d421b34d", + "d04f3c5d101b4e1c5c63a1f09bbe3af3cb7a1e3acbef5b6ad46ba8af4059b743", + "d07053f7c2fe9a0bc31c4bf4223236226952198b195504b958c6f67bc3e24a0b", + "d07e1e4b8e401e7a91201526e70e8f8b9c894db55cd961c81fe855d3053bc5b8", + "d09c34f46a32f80b3780e18e54292a344cb74eed88945a3da07a69d7d3dca009", + "d0b47688297a825cc4a6d72cfb45e24d17a5da9d8f3d43cba2af8f13dbf3621c", + "d0dd6364b3886aa40068a8537aa6719ecb9d3a7e185ca5338b3bd4eae43aac31", + "d0f14e441af05c93061d2b8b45ec72b5191e1c998991ab103ee48902537dafef", + "d1667d0fdb73ec24000d778bb95c383e345dea0315c43ae462e510731b674b18", + "d1695227ba09c29e9e60c5d5305e2b1812134ef5bb04656b7d54846e970c51b4", + "d1831fdbd3721ac4d291560f54ff189aaf13adb759d344b8360495a5436f49f6", + "d1ca52ea943499cb88d83ed1a1d4696727e30fcf9e1c1a888f1f8661de1e5b64", + "d1e292907c03ea6dc5f98dea5f5e8d8d514fbf4cdafcd28ab4a8755c637cfe34", + "d21907d45a7fe36bb446dea4c790889f472b6de896800ed0d68c0d73ec23edff", + "d22d1baeb2fd07a953539f8ffe95fa404fe31cba8058f2234ec48007e0ac9cb9", + "d22f899fae63fc221efb42c1b8c03c110d1d9b365c3f17267261e5c445454f4e", + "d2309680eac9074c3d554b75c8477b7466b29c0ca6140b4bb61dcca09c877a6b", + "d254fb11bcd49612eb4d2189ed13085213ca8f113c9dab52eb93979adb7dcffd", + "d2771a1ea6f906469c07a1e8fef04e6e643d707d7125a38478f997778c1cc90d", + "d28acf5fbcf2510e976669e6ab05fb86db09a1dfd1eaed44fb96c5363dfe8733", + "d2addf95aa41f5e13a6ad590b2713f0ea7670624ceb3df45f5e6f0847fe73863", + "d2d9d70590b8374c968d8d982113565843967802241d93306f007f9b79c1a99b", + "d302274ec5e8a3e1aeaec35db97550937668e42bdbf98e53948dd4b834171dea", + "d3138688b8d2099d4601de9ffed504cedb79976e47aad6522e2d4ff6f13d5994", + "d3169979e682be12dd5305e08dbc513429285768617d1b7a98efa20286501d58", + "d31b09e18c9a5c15705119822e09ed574aa647bc38e21d57355968a65fbc8a05", + "d322f0d4bf8d5e22bc3835516eac087d05a568838effc1b703114422b44d5cb4", + "d34180ef4ca91e938c1245aa42202fd82b50fb53a8a2c42e4fc8eb478c24b02f", + "d3524da45fa4191acb576840d36223f45e1bd7e25e406b6150005897291bf4e0", + "d365b706c8193737aa79d0ce71d6d7489ea46780b17df89ad02a66c13d9c1eb8", + "d371916d7c47430779a55070128d0311441547566cd9f18afdfea10e69191ba9", + "d37493a4cf9bae160adeda969501f311b2c616a56327b492b027721b48801afc", + "d382ea48424cfc817aaff993d824152802559459d71465f25a13e90436043cf8", + "d388e45a663cbf86578fe0cb8d19c0166b4b11e0d1a3b48103390561e7c0acad", + "d3d1dedb04ba701168bcf97bfff1d5589fafcda8ea1153f361a0bc94e3689996", + "d3e4dfff94666f5375e8da4d0a1c4167909eecfdef85ee5f7f85900e08797095", + "d3fd3eb99a4b1e0cd42b805bee67fa353a11bd39c51935e07233d751e01800fd", + "d4177cb874b7a7307336b8774dbec5db48ba5bb281861f49f45ba104c609c56d", + "d42724b6e4f8209ea7139fb53c761b47e923fc360ab6c2c93727d2880c4d2ec4", + "d4445cb8041006a06f30940f452fb25ab5718c99054d263aec03e37ffa554a0b", + "d45f77a1c72f28f05ad7a0d26f74ab8e263d381cfc8e330920e7ff92de4e7e78", + "d461cf05c001b85297342bec480204f28d5573fb3c2c7a0f7588bb8e9aad4dee", + "d487e8e2502e45a634f6d83a3d784e66e203476dbdac51742edae02202a73ccf", + "d4911ee2e307625b1af25cd1ce9e9e42e8f1013a53e9c0e6c5ed1aa6c762879f", + "d498da45a43990513ae88b937d74bfa2a2eaa5d6c20d4fbd8aebd2a4a1a77feb", + "d4cb4c626dc946ed93e1e15283bc68a6b313b2449ec60051b1d25589277b8bc7", + "d4ec74e5fac26adc6ac7afb3b1fd3ed6ab7a2288d882660a95d0d949844e3eac", + "d51c08ca2193e5449fe9d63168b50130e9700ef88c377c6b79f8eff85ce9bc18", + "d530bb300a81c9bcf846ef44ba7d1bf309be17f5647b366e0b321990d6566e8d", + "d532cd7d6f2171be22137ed4b733dd0c79efcd8f560663a4e6530051b69dca1f", + "d54c515b8daaac7db5022b06c73f56eb455768823e71a8c7105c70702c9f0def", + "d55a55aef96c54b8234b8c52f62e44e93728af52d16af7818b952416007eee1d", + "d56f0479216784c4f31c98511368936d9ec46f53ef36c30779033bf833716bc7", + "d5866744aa5485039257eec813b3c798ae708920b48c50c55f3ac91a7d8b313d", + "d5963fa825935ade10ebeb3a5ab89f7c29b5a89859b6e4d9a05b99f3fe76a6e6", + "d5c42ad099d946dd538d27751c569e77104260c88208ebb17303e037b33c8de2", + "d5c64604cb597fa9233354ad4c6184f50e075e3f608fc4e066e39ce41485133f", + "d5cfbba01fec87614efa807630f78b2d40aa00c3f54ef03d5b3d8313eb964564", + "d5e76dc3ce2665b4abb00a96203625c578cfece64c99dff710b28b6ffa184fe4", + "d5fceb5d50d5e30dee026f099317ac53d4a1641564148f8994f97a7fd3794a1e", + "d604760d8544d6548d357c8fb7f7b56b51452ba1d9d21e3374f8135977800802", + "d61e5c37f01be9e876f3202371ec9eda45c423b52980e16b0f15058ec2bf9e8b", + "d6629213250691aec35c30304183005f2177b5b7930ae4a0264ac795944d6779", + "d66418fe345340cba51a03ec40d1dbfa622275dc1fb4fb4839071b8bad8531ac", + "d6937eadac2b6c3d0375e46e5492df69273ab4945242e12f5628df05457158a0", + "d6db4c02d1721ac2784f2c309b97d099d69a2c73b80f2b53af0e478f6ead5828", + "d6deced34e316b96c5ad0ee3041f14f090a8115b76ff4765e1ef0b1d1c5936c1", + "d70eee734906d8157ca2a4adc428a0279e887ea9df38850873a991d40b363b08", + "d74097f6558874f34dc121b870a042ae2798ef27e12dc8a8ce49fa834817d81e", + "d75e0996adcd6c44d978b2b56b517d28d52b277a00ea557b949d71eb6731c1f9", + "d7812ecddd19c0888a7ddc1b19504b9d13ce3b9f08594e8af96f539716bb12c8", + "d786281abd5086e355ca25841d15069947ee83ff1febee9f91a3a6b985e3529d", + "d7aa736b01c8f04fdf206f169aeaa16b5f11775467f659105030ea3a635f66e3", + "d7c5ef08eb5c2c55507d0554d06da42f076ba5dd8c4cd7ebe95eb9cdd72ecff7", + "d7c808f88a1ee2dd7e0508779b7296f465c97a790a1525c38f27d2956730c8f5", + "d7ceec1687b45dda147c887a9a1747e8b29eed1ab2736335e4f941c40bc36f50", + "d7f0bfacc9d7918ef2c50b51f2760b9399f1a087a00a26d12551f63f6c4f3bcf", + "d8041796f4cbba48d6f3e468f182eea71c719e1b4515082b170adb67fa5640c2", + "d827cdcb1f4a28085009d79166e0766be4ceadbec5654943377567edd6d1a9ab", + "d88522c4f93e5c1e6670d82ea79ca300ffbc0f894ffc9430224000ef782f8fa1", + "d8a1213d2b8a725dad7dd1f6c94e46c5402c708b811d1964ff09ed5640b8e29f", + "d8b9c5e74d8d3e00148b98b8ba7b62cf980c8d0d59a576bc372d6c95ffa25b5a", + "d922ae2fa9fce5b9a8bca376cb078bab270b7d38baf9d2f30e3f9f5740432dc0", + "d98c7ec8189d9bdbd3096d624214e8490e275e1e14130031e146652aeb1de3af", + "d9a8b49b226594f5688d633f92b6dadbdfe670ae73c2ed8b8c6f8f11a74a50b8", + "d9cf4ab91413fcca5d5ebc76de93248aa95deb050b70ba1ef5329cfd9813c918", + "da06ca9f1bd4c1116e446737bc45bc24139610e4d86cc12c49912575f0ad3702", + "da0b0dc816f0a9044f295db801b007b657e1bc9119c0d2973d864f992479dbf0", + "da1c6e9277c0a19b8158e84dd9580a3164df4c2766a9866dbf11776c1196b07b", + "da2aa0974c90a4b92ac0847661095b9ac66d42c72563d0764baa12494a398183", + "da372dcf7884fd6dd5878f012626f38cd2c0a0c00ede4fe1e9b5d1c627f025d7", + "da40f051387657cb3841435ef1eb8de999d0b481a498c56ce69e405499948e52", + "da459a3bd11b2eb7dc03c679865c777fa390bd595961a61937a5994056aaddcf", + "da46003d68d66486d0dfad911aaa7dc805d34d411b78d471ada75cfc3ac403aa", + "da78a8e6bed8c5080da3ee344b34a450c3dc3c8b397ae24a8f6f1073e4000786", + "da97b31bae61c6b47bcbd6df3e428cb672e7ce25e615dba3d3e786113d88b8b5", + "daa90216b4b743a3cf2c9ad5b266a3c73c54207705060a7a9012f0e30317e131", + "daf3c47d33f1186ec51d15b5c6543abf982cc1557a3a6e1744e911a07e164b0d", + "db0e91755ff469e050b088764aff3c0ebbc90339907aa7bb9a5bcfce8c41b185", + "db1ac06f5945913376b6e0738efbb5dece43cad713af671cb01ead48ff4320e1", + "db2b2e354461be6b9fd1d9f65f83f608c82f37dc34b4123fc46a7170b0aa320e", + "db4d2acf1804f056caa6338a0c1121ebe7e872eb623820d2faf28f33033e589b", + "db8af8c419ca5c9e68fefafcc963f225091d9a50843113db6693d9e4bb4e0d2f", + "db9e83e985cacab6d1fdbca4aeae0719315ed7812438c1614b407f86dee3e6c1", + "dba3f95e581a67af2cc56bcd034a08321d9429a6a4a6558e0dc45e8d6e49c4c5", + "dbf89cef1ba7324ab6b407e149ac280eecac0c8b5820bc3d7448391ca59aaf70", + "dc06ce6ab156ba6a1b0200c9b2b3da835da67acd9a2a0d949d2d99d4b4bacf22", + "dc5f6880b7ab6efc2987eb81ea52bd648018d5f2dfaa1793f37640af324f35fe", + "dc9f64fe826159e80dc4a3b3e431c2ebfb89813de20dd3728fef9075dddb7180", + "dcbf7df88c53f2241c6d065d9706914e797856c427a9a7748f1a63714e035b57", + "dcc17cdd0fe848571eb98a651b77b0fca689d676576e396cef0e1e6213309cc6", + "dcd9c20f60c91d021109231b65061a5110cef9adfabc1381bf85aa06f3e4cb7b", + "dce77d78d4180ba19dd613af265e892ce7f35409a0e0d06331d101ab388d5d67", + "dcf195c7d41770f7c6c6a8f9869603f5d971e42f6322417ea4f804c457e858c5", + "dd428ea6b43db62554dc7f571fc9c9f4d735af775c0220921bb872acf95b9a21", + "dd5fb7c5f41c05f16a82beafa5bbf35b793c75a6cbf7c55f9bb6e1d78d5f13c4", + "dd61cece503090b07d75c50d1586fe3b5e7d7244216f79adb88ae8f631c0b5c0", + "dd814e024eef95fbbd404fe77becdd1b85b1cd84a1e6669e45198a45a7581e73", + "ddd1bb658f9157c87bd8b8e37c73e56b5fd70d5776d39e89af71a6b5fa75a5f9", + "dde0cb6bb333a5d275d2a42260faffe1f3c35a21cd71f9e6ce2dc65c5c364dd3", + "de2b5031bb88c517a74c9c7ce89d5663604cad24204589e5f04990665617bb19", + "de3c9542ce9158d9b846bca4c0e919fdaaeedcf781e0f3a532fbbdb0ef4c5225", + "de3d5a79d11855a00ba7a1fcf7b77a1f53fc6e6e64832cd9eafa469d0488c4c0", + "de4e63972c76033624e557a1ebe022edb003eae1549e6c00e897c55e547c34de", + "de5e4e21a02c52d22cbf382a6a968e4a68f0be6d26a80e997135747fbbac0e3b", + "dea0a83b2388d405870ab8d20e6701d12f7770a0ef891d35e8a6c60a24f549db", + "deb7bdd29c4ebab19db23e28bdd715a7e08efc63f1bd6c99937993271d26bec2", + "deea6967477e2d07a943fa1612a56e40712723c4cac464676ba95d03745c5712", + "df22cfdbe476a57a3a5b02c2ce8942fbbdfa2bef3af34e3ef92dd087d7848fb9", + "df32dadc5ea31fbf05bfc17765453c4b752ce748ea5c11f3eaf461cb4c4cf86e", + "df36ee603a248f177b0f7faf15812cc77475edc5e0eb4b3c2b90630d54211fe8", + "df3fdb2f1e2b9712dba895c5dba6d01ecfc53b1dfdd4a63a1db80df373135f3b", + "df762849fc9890946bcc0b5797d4d924e430849dd3c2859dff2c4ba7f009172d", + "df79c31a3880ff4606e683dea00a9f2d9c667074a3e15c7cbf315e51a8d47919", + "df9e0382e541f41c49bbf53abb33be62a5d85dd0901b3364dda0956337788d3a", + "e01fb81986221bb00578cc81023e8ae9d432c72ff0177fdde4057d96bcceb463", + "e04134fc94dbef26800f264b5643d7502849a1f4b1fc6af5ddc23a1a8f7af260", + "e04cd88d4542f2cfd01ba7a165a772991dd8df58dc9876b6a6633cbf5204dd32", + "e053ce301796e47ad5b2eed409cbe269442cbb8169c7c3135f9905161c621b07", + "e0756dca1abef5b526f8ae94b26d938328865bb3d80245bf87f8b4fe291612cf", + "e07f02c55b571705efbd7fd1acc36170409c3da0abbc1324f2f87179e4100de5", + "e0b00d3bedb2f4d54817338493a959e7725e0dadf958f41949ba385709a3b716", + "e0fb21f8466f8b028c0a8ec4ce48a6991ce3f73b80c1a300fabf49ea1f11b5e2", + "e0fd0bab853b2375c1458b581b9c4eb10a892740899b2730f77d5cf452315f50", + "e1116deb7a312c0773f7504fa68643e7642d7e07edc6f76ecf0e7f2317e4eaae", + "e116e702614e7e8dffd26029d472791b380e98b24796b9b0729c1431cb11073b", + "e13b045d58d8eb0173956697314814b0818b090fe63a02b4d383f050bf947302", + "e149e8d8cc8840f38238c52e2e4e095923ba0c1ba192fbd4115df6c612e69238", + "e15cf52cd356a8e4b4c394beab2276b76fddd40d600f4979bf671b8a8ae8d908", + "e165291d79e8169ad9ece304325eab1ff728bf614fd944a40d06ca1213341f43", + "e17bd7410c31d2059a4638b5d8798f1863143f7a099d1b60e03308a8c0065fda", + "e1e001d88d85d034c5aee96eb55f261c5318176bfb9acfdbeaa8c16c5302c6b9", + "e1e228481bbac4845368d96e9979b2e28a4d9c96046d94d4eb75ee8add763555", + "e1eda798f2f368b8857b491ca6731a72e832ea80fd29033e752239e12597c627", + "e1edaacf2d378c94eb4fcfdf03a1f080e3a7f2f5498718719b203f7e3f5b1abb", + "e1ef250815b1426c96147c4072ab677ef792e1109910427d9cced32834e50a4a", + "e20c328c0bad99a61bd6801575f9d5f9e8e1f31f4d6567cb9ab4ed4cff110771", + "e21d30dd0a17f4e3a48c59368fcc31959cdd1081319e280121522a0cffeb886f", + "e231c038fed93023890dd0e30ba99e3d0bd4f43d22f14c82756c138840a31774", + "e2422f3fc47eed98843f471f11ee45a4d5017d6ba68f5977ab3108eb2cda62a0", + "e249962c602b987e4520b848a2433337a2615e537a00dca6df1ba86146614d09", + "e299be1a347f7ee34b86aa0acd5d1251813ceda172c868b32c74c927067b830c", + "e2a8a71bf005884953ba2c79222dce333654a560e088e9cf38dac85680b1f735", + "e2b5b8cd854bdecc517cbd0a199160bebb1e56907278546f752c1d5a0e93a57c", + "e2b8f68c5ba84769f8c704bf4157ccd3e99b01b121dd3a4e8cf3a6792b8d6d39", + "e2c05ca53adfa0f9b3e5c432b2d6aa14c7825cf879d6befc9b384a02a1128624", + "e2cce80dae5af088e4d5a2f938be265a32d40a5aa29e216adc94c6af85769342", + "e2d6766f2fc6adec1f494b3c97ada10bc6b3dd89fffaa6b9c7774c0140e702e8", + "e305c6624bc5303875d34fbb385598ae3418b4f5036fe9763d6c5ba5b3261fa2", + "e328ec3ce32b204b4b8d6bf0f63bb2937b8070ab8158d792b746a4cf5bef0112", + "e3290eebc099a02b3bda9c30ab81e7a91f11b4635d3d68ba488795fceb7292ee", + "e354482685966bcfaa5ccd17d666dec791abb29017bed1cff35c1f3b48427196", + "e371f2ecf7e74a3b1a21a0d2dbd76ff061ede68511f0e504ca9f3b0562f666ff", + "e38810c68a7a67fdc05d198102f932ec2302ed507f9268ba85bbf84940be2bb5", + "e395caeba922f8b90effac45dc8e59be8078bce39141d8faf98ac3b4a2be832d", + "e3ad990bf0964b8d707c91fa546956dae31678ed77f2bdb827b9cc7659ce9718", + "e3bc95a739f1260d432bccbf2bb6e8e588e957d1128bca21b8175ad944922069", + "e3ca4bb6e4d1ee23cc72e89ad926308195c8cb17ab4fa4e240ccba8b99bde1e0", + "e3e53335a806f53b0df5e5632c0a5ce39d062736641aa05d81be66bece91cdee", + "e406fde4c19b80fa9ab6dd1a9dc4482b579dda8e387196c8218a5e2d36cf767f", + "e45054295a867fd9f601fb78e919ee099ae8bd87be5bd91f9055d9b5c95ba1c7", + "e4833559ec5fce3dee483ba72c851d1ead07e7a693163e5ec4d14afd7d3e4ada", + "e49d798fd16d767825c4ad445843bc037f30c731303cf07626052611103e69dc", + "e4afd66f886e9c1db0d2fc427c08f765d3f5ea8a57a76c1f3f675dcd1693b797", + "e4b88f8bc22ac0755f21e7e57a5daa1242c722dd27e5d88218a71fcd377044ea", + "e4bc88414306bf55b62cb428f40f2eb464a9b27b3d65dbe712f85ead7604bfcb", + "e4cbf1b81022ebb6a3f4ca228bd6e1d533e051087cb3be4dae37a6b335b64b27", + "e4e28808ae8dfd46216a2fb9252d8f9afafa858d11cf775076b653c004cb32db", + "e4e2bb9879bfece542a719a3d80a69b09404c4de19ea69c7c27e2d2f73c6ce92", + "e4f68bc2988442906fb3146a0c313b75bf7d98c78b03fbba3449eba222b4e2ff", + "e4fbaaedb205db953e8d24af46d3937040d565da17dd3c1a6358598ec4065910", + "e5154a7d17607655f0bdfe3160a78ccf8e0ba51ea39acae0376350886f8e1845", + "e55299d0bc052cb970f3e49f110d9d50c45c8c5aac7b7ded84020228bf1ede03", + "e556d85a21c5f358d7d57d3c4126a0476dd687aed40bfc633440a5f9114c0a62", + "e5e8e61a6acf4409b3b546bc34c4c5c3df6f8230d7e4ccfc33a2679da4ba8940", + "e608c7c92c378f3a308d3f279f0eaa97c8fb7e7d8dd97ea445b1c690362a3a38", + "e60b254147e2585fbbf72d47ea2954b9a673d8717c7d3e910bc513246e5089c0", + "e61fac6a1d5695f0b9010ab423fa50e035f5a29cc33ad776963371df7620b0b9", + "e6a517e9dfa6af4da7ac8a6d5ec19af46219981a80f523afa4e93b8a807c69e6", + "e6c72ed82e109f3797ee8861b619ab6088818c184d9ad5e96af3cb2381b7e95e", + "e6d6496eb3909c504ba87b96fb20e29c2a2cb1d6d15222570dacbdb4d6e0b239", + "e6daf10fc8aacdfb63cebf93b94bec2053e8fea782c7e9e02034387db2cab032", + "e6f45d4600f715028e9eefe9ca9ebeb9e34923defa01abcb5ebe3a997d8f077a", + "e758b129c07dbf4795ff3c35ddfb08b94af172a15cd90c21805ddd3269795d7c", + "e75ca95dce785b8977f2b24392174c21ce5b772035b87b24fd8f7df2894e8b04", + "e7669041b406fe2fb9605a2dea02b57ed708712921edaf3049e30c15c0e49870", + "e769fd9bd29296b03b3bfa821ce720a82c7fad4f6c3545defa203fb70f3f5890", + "e778fc043354b52215651eee4314565d3b8a52b3060f76014c43e768a0723ad5", + "e77fa12e38c90a278fb7af1465517449accfa94a16aed341a3c9f7c8a19d2acf", + "e7b9e659a56a75a9cc66abaf2878026f9ec9ea7b6753b744fadc05e8b4fe2e49", + "e807a1dae7d3b43b44646da39c2f93eac68f75c5d408bb4c7df269b2c31bfa6b", + "e809ef944112bcb401a996b7c48e9c60d7c951635b878b53a9a4e8e7198d5771", + "e8210d2b3a53f28d90cd4c0d5d563c4974afb154e58a950de60a6ce1e94b98a0", + "e83f1bc3ef5cdb9898f71848e2455310e173d9e625f6f62622ee9ca1f067d6cb", + "e8a3e432bb9a9a7a777dda70ead4e914e3852169e92cc1213435d2800a4b0d15", + "e8ab701f9c746b2050930d13cd14c1a4da09470e84a4af00182b240e0190f2c2", + "e8ddd4a423f53d9e73e89994910905d4177eba6ba48a22b6e94fb46c2eedbdc4", + "e8e065640856dc967ec7df4f09627d3df869baf4768b8b9f52d5455aa3824b98", + "e8e555f7ad29c9b6cf48afe7e613a9e7326b0ed49d8c4fa98574dce237cc8bd6", + "e8f7b249bfc5fbf13cf71c70f52c4cfe7ba312f9b2eda18ed1b85f95947dcf0b", + "e931fc764175ccbc37d92970ca353c3ef516278cde0fd8302140552eab5eee68", + "e935cfce5873192c80a1489d0905c0ec0a2c93bf66fddb4525e4ba1a261bad2e", + "e94950b99f9e3a0dccef7b2aad372368c1d0aa60ed9d46f44440f91152bfa9d8", + "e9c3fced5b9ee5c1b0564c4b0e97bdf8036c915928d6d71306c6a8153a3dbf52", + "e9f20a5843a9b56c9f60c76fe1a4a20502d3ac52f6a79e6bb0fddfe57c734f4b", + "e9f7e71b5ac971ee7edd8954535c7bb37a9b4423bf2b4c334a4d837a719d6940", + "ea5df06840f024216c7060de9e0e27e831a22648e6cc40f3cb3c39f9ca632790", + "ea677fca7f169e12f3c208c112812333ff2c7a322418d181c9a173155006659c", + "ea73e0364115fda77575b697cebee340ad79f5d70bb0c3e1a84fcf9366041a13", + "ea8c900045b92ce2c1fcfbde002b9cd3fba951e0930ffa5506af460aa540aad3", + "ea9fb8c996a5e0d3355b766f1236bcd7a1fa9071da553f229d72821e270d5241", + "eaa4e3b998e89bd22997baed140e1377accf6b6bae9a9d1e86054d583885d929", + "eada10e544c804df305e002ae188e023f385ec4ec1aad6532c21a721981d28c3", + "eadf398b6f49fcd70b7893a706e3ca3cec3931e2f79f1bf0478725d835df8094", + "eb2184aec299ddbafd802d22f3c13fd5f78cc0b39aa17d20217480ea1b539150", + "eb21f51a46305dbcfafcae086fbd51bbd1bba56b0c8cea89d67ea33f61e8e540", + "eb49069e0d0803d055f3d2498d02c39396653693a7bad1d32dc458e3118a2218", + "eb715472371cabefba3465b6b77fbff6aa0fc2b4f00c8fe7e55da7cdb1e57c16", + "eb829ab015a5470d658bfc7cf8d8a4f9ed7b0649fc04477878c2c5bb086ac465", + "eb919932d5bcb710486119df62f73e823b2662eb21355d6a3229976093c579e3", + "ebf1af91b52cd8ca4efe45784949e3cc2e8f246d7f4360202ae6f3749a725c3b", + "ebfc8c6d358667cd32e57e33ffbf9aa00db477c85e0e78f25cd2889ffe8a0fb2", + "ec128b250d230602ce125413a544982ece6e917ebe41620a35b3c2da7e64751f", + "ec248ceff6e47b10f0d67f19204dde463c9a1408cc04ba680abb10619a4edd29", + "ec34aae96b40eb7e90848804ed9ceccdb00855bf550a25ea89d1e9387475d256", + "ec3ff85db7bc96ce057455dac92b0d1e8b1a53813902595a4ecfb7eb248527e4", + "eca600fa3e3f7716c80a84047650bb56523b5eef746421cfd879f7de4e393677", + "ecd38846f9a886c1c8ced9a9bccb141c92951cf2c1a0762d75b46a4e27ac1979", + "ecfeccdc4944cd60927341295292880891b0047b9767a3067c38a6a88fc892e4", + "ed3411e5dbd97935027bab83e0de5f1844b0203fdb471007197c8fd12abcbb75", + "eda0be2d9847f3bed6e1f0897845672968add857c9059f415357dbd6a513948c", + "eda11580ce1dcbae0093fc69bea154e1b5e2c429c40a29a07cb6fa005413095f", + "edcf7090faa657a30800ba1effb6f5336ce6c6da7d67f34fdd7bdef9ae1799cb", + "edd3144bce78365312149662b177843b7be7ebc94852e60db48883321cd1c28d", + "edd8157eaf2fabdfa0228ccf1bbdf8da438b5c85aad7e5e2b16415e03225fc9c", + "ede58af4da98df112c7e8d3e3bf12d1dc95cd6a2c2133c5eeb854eeb92549e48", + "ede62eb7c52421c015c5c55624a6f4eeb50adf742ced0cb0e8103793174b4336", + "edea1d001bd33c4521b4976f8c90c447ceb5f39377994d58cbe80b20760855a1", + "edeefba63f3b8bdd4193ebd0f4bde202512ec7186db9680f83ea3503d2bf2221", + "edf05e34b8627e131cc9b0011ee2814775b3ef1f15e7df2a994799b77349e766", + "edf49d0ccc646cd8e65b71c90acbe0577f1e70a44488387d10d5af0ff0ec4ac1", + "edfe570ddea7c44dcf50e5d209f1d0858ac2f6d1f5f4314a47b9291c1e7761eb", + "ee27d30b6be2213a9955755bf53c689e22a1d36d6d937c6e5ce30f033baf1ff9", + "ee2eba2147ebe6e0be37840a004b67948c629fdba8fdfa4717838f8f14fe6655", + "ee58e5f3069e7a722ea63d9f4adc2a70363f44578c99b653e9242f51966d3557", + "ee67b0345ba3a7220dc499ea6cd43247969026767eca25819111f4437406e5f5", + "ee81040064f573d55bc892a32c0a6a9633ecb54f5e92dbf3e5250168a9ff7f8e", + "ee9ebd625b437d3e95188418939dc28c7bf4ba90a218a9ad07adda258499360d", + "eeabdf9f8b0db1bceb0bae0890b5fdfb9326eedbc40dcb85c7f26c6e14f3d37c", + "eed3e97400c4186ba00d90ef1ef61a2add556118dc8721a9d0cf4fd1677821ab", + "eee22c4fc66a21e9250f8e48c48b4b20312e15ad4dc92eda8fd1a8e84ec3b28d", + "eef0e5551b49791fa467c9827c9b6e3bf48fb58d15608a863f3b54531d6e7c2d", + "ef014ca51bfe5dce1a5cadcb691a09e845e1f189e197edcb1b057577e7db2b09", + "ef21ffe1a20987ae754772d981827616c2430fb2ee65fc446dc12b890ac44691", + "ef5bc23227ee57f961995d4db74011fa4e4f67326381286a17660c10f2bdaf2a", + "ef5e8925c07e682028717a42aaf87e36955876bcf1b2cf90c11c9f31047dc205", + "ef650facfe06596e2790017a7de7284ae386555ce70df6ac6792510b0c7782d0", + "ef651f38f75bb4454622f634118195c8fa2f380a02c85e542d0d5e8d40a312d3", + "ef98d4518adc923a900d7f16d0eee156539b83cf1eed9b777c661648296c8ac5", + "efa37ee575b79f5cfe55a72d9df54837d8ad4596af2cd6c9be85fcc147f607ee", + "efc630fb564e71a9a422d6fa7b95119128cb8e883fc59f15dbecfe36e9ecaa44", + "efdc2d00f2661a9477844c248a90bf38774d9ac165b55caa12814e13ffda9f6b", + "efe84d821bfa612112cbb36d005691aaf4429ed7d43863abd6028e2628aed9e8", + "efed2f5dff554dffd1f0016ade25144b7bfc1b05491f22dfd977473d08d90243", + "f002dc29bf45877bccf6944d2b2f85ca3bb5e62003414b4dd8d0da1a8f0239f0", + "f039cb6e4f3b43eb5fbd395eb0922df5deff0be36ebe5339171e5fda1a175bc0", + "f064a324d8450b5bf59a974ff9ca07478be793974aadacac4ba283d522ce1031", + "f0653049808872abb6486f7a0237281fea4cdbf633c2b9b7377b97c036c5b636", + "f0878d7d54e80a2f4cb870d77044819820cec0639b0c3ed55d68060e02aa2534", + "f0c51e190f9e1d34a145c340fcd2f3702770119d1504da2e90ee8cf61ebc98ce", + "f0dbdfdffab87118e1549236292638f7846f0346115debc135c9f7ab76481624", + "f0f6100f7f043d80e2d8ce1d779db6b497b549634415cf66ddb07b5cfc32a9ba", + "f0f910c3e7bdc180f5d4bdcf9969badcd6f9ce1ad0021aaac14c0e6268b93593", + "f113925bba3c99674527e3a1674af50ccc53272dfa59bb7e3ac95713e0263cc9", + "f11a0117829cfa3cba8005d6f7853eb1b972bd0c946a92175a4e94725e03721e", + "f11fa2ba5323f3df29ad79ad699aa5f4a6de4b5a44843d15393588dec3aecd42", + "f12815e47d204a27bd5b1f5f7d006bbb61f7bc8ebc81a5a238e3ca323bf50fe9", + "f140b3f6e23c614a37a424499d2195d9390aebede5e7d7249528f8c97ac4f8d4", + "f141613c3556bc991572da8b83f7e173bc25c96339e883c0419ef6909a1b4c9a", + "f176f0da7b059ac9e4eaf82d45200a273674b1a3c7762d988c8b45f72a9f5c9f", + "f18f388ab015171d6aab7c42b2d0b7460f2051e823cc7209f11a2a158eb9e69f", + "f1e8698545df73640275657660f6d64d39e679ad11a8e51db2beff9efd7d575d", + "f201a6a126befab7f6ecc164e97bd97e7fb96d41c449d5afed4e70bc38dca2ef", + "f219e244f6d65c5d2d342f118c7f3b86f8d17e9fefb358dc2bac5517a93f4403", + "f232edac8808e595c7660ed3940acc5757889c36fc23cbaf94c0256c0ca0e4de", + "f235b5c48056676e8b72a6ab386dffb67c99cd2e777a366a7bf2271009947a0f", + "f275219cb79a11c529c59fff67cbe509a3b4b494baa631231dc2ebdc434cd4e7", + "f297043452426d4badda87fe216cfaf3218081193df564a8e509b1e88436183a", + "f2eb09aa74a3916a82b93cbb45e64b6987b4b0db2bb41508039da083373192a4", + "f3202ff50ba57b89a223afda873dc154246a315d02cabd86d92e79aac87b43ee", + "f36947f97e0c076049cf7bbfc4b1f9e29a44ccd257ba819c8a4390ccd8c104f4", + "f36fa28a0d82ff6459afb0cc4c214a3f00a2f2c05ce33a7bac83708b92e8f790", + "f370cfc1ec34d5682a22d438f262a6466af86ebb6075f28a95852a8a9e1368fe", + "f3bf79df83a9498bef664e804ba1ac9abb0540d1aae420d363aca076e898e2c5", + "f3c7e653d4b7b48da3d6827590d2571ec55a0e2fc93089bbf0729b5118932cf5", + "f3e69d248ea724fffe205c18afedb24684d9e9732f7123bb52cb080c0813276a", + "f424781bd9b11fb67fcdee09cb5e383ad7eb8e73c9549f7cb2da216a5e597c38", + "f451ebfabdb362efd297cbce185efb7a1fbcbcb0ee813e81eb06cdc8ab9241e9", + "f4607d690baf013d7a802204f90eff95d514e819f345a9ca28a1ce98f372d49b", + "f46093cd26aaa07510e0b7bb6ea87df4989f67d720ede606e1ab1efe80d63d76", + "f4af0630de6af54e875ca3b5d161441a92c31a3d2755c4679296ec56ed1a4208", + "f4d75bd491dfcf2f2794dd3ade4da77c7f9d4ae201ba391c79c3885bd2a3a032", + "f4d97df423579ef674f017107340bb087391536562fd7949514ec8a0d6f3457e", + "f510d227beb77e05cc85eeb4cbaf7a037ab4696ddd28337cb3d066a514e20344", + "f5811d175868d36ea16bbf4edfaa39742b67911bc911f6407c853d4a60c2004b", + "f5a72bbabf09277fb14b7d4c26a8ad14fb18e35b770b84c63adff036b1130211", + "f5a9086e74abd4849d1138c889e6363e1ad753a23eea3b336c9b5098dcbbbbf0", + "f607d317a2120ccef46ceca3f4b01107e96dfafbf7e59b9eaac61518a9384894", + "f648007ae7f6484e0bbbb8540f09dd3799d97892127e3719cf24202000f58eed", + "f657f2650b495c498d64853a352dbddc540ef5fbf88c00612eb41fbff9b443c0", + "f687ac299deaec19007e4a2e760d551d30b34298db875526723ccc4fb29c4543", + "f6919a9c489ddc3392482baf57b77b9d6dcf57701f91d52f39622b411357b59c", + "f700762310adf887ff5ccb9d8345a9b8c70fb579f323066c2a784120612038a3", + "f729f479bac37a44de358818074337e1df23ac866b68c37ce94bad896108f756", + "f74f1b83529ecd86b42df89fb3db019bd37dbdfc0cebc44c1a8295617836aacb", + "f7601fe05dbc775a3669d8cb7056b902646da54214fb50515b5e46c2409e2c6e", + "f7691fdfdb3293b4544fc7a352f0f0bb36f9dc5f9f7be92351ec0713b440fba3", + "f7702d266606741d021f0a71da9cf4742074f901e150b84f94d792303db58e6f", + "f785b6dd3ff60c5fe59dd1c191c094844203febfd64e047f37c5d77173f3f66d", + "f7ee42dc5ffe55dbbe926bfb15e73fc7856a879741e98cc2dde8d0d93d5c3952", + "f7f4125d2f2282786b18a12631ecfdc43b84d6831f40e667c2e29307177a24df", + "f80da66a27af19b2789125c6b3f6d7c1761be79cf443343933c1e9dcf40fc996", + "f80fca2b74648131a8a080a1e8ab8a6f7d611bcbfb2c96206e11f4c6ccedcb35", + "f82c04cf08a5d7ffbc2052d4f95bed1564bfb469e14b3d1abebd310baa7a318f", + "f847a0c8222272968b74e8818e2021323d0ceb7e2e2a59f40e6ce7259b249359", + "f85696718a674949094404ab65331bcf680c5db6db8bbce2d0ca66cb795c2937", + "f87e78a9e384348c22fd07fef02a503472337a3cbed6fe3b0592793fda93d178", + "f881585742358979d9c7078d9838a7cbab4cf29046bd22840a529cf9a7687a10", + "f896d7cc6906c2c999a594df63f5db5abca03fd217c6d22b4c083c8f1a847dd8", + "f8bd539030c47ef4ff3a14d114494cdb38bab3c1e3bdc3265b293fa311d6370d", + "f8d90aaeeabd0645344faedbe4a1f9b051f2bb897fc5fb001e02a6de84f45a3e", + "f9323ea2196bfa6deea220e7416a5fffa89909695221079f04b0259bea505bca", + "f95f0181a9ef304fcf345420a6136c5dd128dedf7d29ea48d056f24cdb990061", + "f987345695896ce10e780add318656a40f3036d2dfa95daf70776c2d89909d70", + "f9c47c28d51431a91e70491c1ca8cab97e592279a94f9607368228549ad57936", + "fa07c7f20c27c4bf9145d52efbe65dbec7fe7e8e0a74543437b056b2fa40b98e", + "fa1b71dfc12a4f385623fe52b9f2454dd162217414070904c282e74116447700", + "fa31e969c67e0e4953325d3a91af4672b1f3d68ee721c1f819f040838966c625", + "fa39b77c7e175c3622da909e0c48dc141bc2421906f477bcdf99ba3ca389d679", + "fa47b56854afae0bb9ae5678b80137f64bbfa32e3a177583c97dde0caae50788", + "fa8721a6a99ad2ee7c2e31f26f5a28a03aa2e6cedb9f099c81b1947da8c079de", + "fab9244a98d17007ca287be76d375653d42376c8a40c9837d697b5a9d54442e5", + "fae85d58a94ee8ac87350af04247eb429dc13eba424d783babdb3b40962959fb", + "faf20ae2f6ea21bc12fcba0fed95fc2d2c66fa525eccb574b82d2e775d33e72c", + "fb0104a9ad8c9302afa9844b799df77114a1f599ae73fb70ca182c18657746d6", + "fb06e5d2ecd8c36cadbee98f013d650a6c07bb2040955789da0d930f95d97e87", + "fb6c478eee898b673291ae1ccaec9d0c1e3118308994e6e9ed615548ea682490", + "fb7a06547c76adb0113a91ae5e50d2de4241269eb736261a6943d7cf576ec5ef", + "fb8086cbe71654a8393479e76bc7865b3813caf6dfeb169c9b65234afdf316ea", + "fb8db1967a1bbbdcde29dbce41454d1cb692aa293ac147ebbc0ce88f94357194", + "fbae6636795b7b141ee220b0c648196fbf00d711eb5b0730b7981fcc79d98e0c", + "fbddd6b3a7a38eb467e4dcd74d53870097e4244fe2869fecfca9e44780db9ab1", + "fbe4c736b1b9641c14842d2f9c2d6a4a10e6838033b53bf5bbabfab58c8932d1", + "fbe89b5c637a113631eb2ebfc096d50b3df29a2732f57a74aa8b237b547a04f6", + "fc288e5b6716568bfc9909a0e3645678a20e33954d8cf3222dbbee1033550e88", + "fc32e8d507aa88822efa6300d76556838e30a182d9b8084a59c05ad83a83a15f", + "fc437094c0ca48af5d90e3736f836efc88679b90be80bfbcf782d4459d7b089d", + "fc501b523438dab044b23b812eda04e81412320fabdf2c7dbdc8830909c2d660", + "fc66d4b1c09bd0e559c579f228d3c38a078e45e9d1305d4cfff09a47329a0bbb", + "fc86885e0b1a0de5f04adcfaf51758add43ef7ff78fe43cc43a50cf664176586", + "fccd668b17136cdbb99e09013df6c6dc0a5a91f1eb1bf22f989a21cefa20ae82", + "fcd1b5d1a225fba957d02e346ad9878be14a6204b601bc7f2d3e9a6b06216294", + "fce089f68c0ae00d5da1e754906c73f1c3dda9140a37617dd6eb83c4b2c2ba48", + "fce84ffad30fafc2e31b69dcd740f4d6d675a37c7639473101c780d5061b1a03", + "fd17ec469ece96894cbad6f7c7e391c4a4c110283bcce36d923414a824f835ac", + "fd386d3ba151f051475e843a9e93e392fc7cd8f3193b21c2a78870ef190a8629", + "fd56a8f2722ba03958b443a558c4e13465f2984cd852cdcf87dab011f33b24a8", + "fdaa56189cd67b46adeae1059d510dbfb12edbcdc518bfa2e400140fe3b09bcf", + "fe19349e611215277bbee9b125024dec258002ca7dfcd3a7d6e2b5935dcfed50", + "fe196574c75b0c689806a6ec1f4ea29cb90df6513c167c4c8f168a3cc4af9676", + "fe1c600d5df592d4c32d16ff5e494006f5c98b172fc2b4f6e1697a713ff561ee", + "fe2b7d4e8a2279775c1366b1447b7c2ca7b9c78f80250778eeee02afe18fb26a", + "fe35b9a5ab367c6ce9b369d900c2e296de0d65106a374f90da0fb7339e6a1f8e", + "fe605b386e9ad2f29da2d05a1ecbeeddca2f4e96e997dc7f936f2da629381fc3", + "fe79db37640f9939ca2efc575b9ea6f0ada86bcfc2fa03c6d33decbd4adea4c6", + "fe8a0a9d5dd6d92788187afa795eaac740e028d09f80a76682e4f0c33a5f136b", + "fe94f8c3fc0116c93a617302afdda03a1964b143983d60e257509a690fedca37", + "febc1c3d860db8a9fbe1fee5d8c1f502d19797294cac207b19f790661312b229", + "fee93db0be8b667fcad98db04b454a25912e56cf54f3fb8637fdbc0eabdcf2a6", + "feeb03e7148982cbe914b41c73f9e6f399ccdfccdac69269fb3a92d214b253c2", + "fef7239ff6ce720d7a669ff227efa115fdeec7720a1dd0c35e5b1fdc58e9bd6a", + "ff07c472654ffc93b1e5cc2d73137b9c00da43f91f81c163d8f1ce6e3549ce46", + "ff1b6d0ed5dc56d68521316910c4d5f1e9e4c149199a4fc4ea61f27bfc7292d0", + "ff25cc50d0ec9a6667cb3562ea5a4db8311bafb7b6da50189ef2b88e8f792eed", + "ff3206f9c2acccdd3717c968f3a263cab72d9bd0ae23a9d3457d0129e01f6af9", + "ff6826aba61a52110343bc8400371d982003dfba256edac36ebd3559a2b924b0", + "ff79f3a026549cc2971adfd50a9b5308ea8e872e0cd3b1f88f9923d18d9ef983", + "ff7b6aaafe05fb6bfae3c4b5d4d2085bc210bf47117d02702d8b3aeabb31f0aa", + "ff9132bfcc7b97ca2285f63b6b98c9e5ca470341af195fa9ab9b33e5597cfd77", + "ffb1eb364e359c1ea440fd1aaae3ace5524bcb3e61da03b813c249d7f77de6f7", + "ffd5b2e7c3e69af9156ecbf8478ef0d2911bb1ceb72ff6c5dc2207e71ce09a82", + "ffde31f75b901ba39b1a965ac5540655b4a238fbff179c2e51a2ead30aada872", + "fff33bb40f0d5856f295bd86e969edec5147c0aac5672ff4f959b4dfad835dfd" + ], + "task_counts": { + "LiftPegUpright-v1": 5664, + "PickCube-v1": 11264, + "PullCube-v1": 4240, + "PushCube-v1": 5872, + "StackCube-v1": 5664 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_split_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..26988e766c6751325c7f72d33b199bd133350cde --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/source_split_hash.txt @@ -0,0 +1 @@ +867a94a1c67686c082765671a7bf8d74a9d633085e1b2619c83197c60bf58372 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/split_hash.txt b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/split_hash.txt new file mode 100644 index 0000000000000000000000000000000000000000..8533e80f235606cef4ba0119a3ac3806d63a0983 --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/split_hash.txt @@ -0,0 +1 @@ +9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381 diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/table.tex b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/table.tex new file mode 100644 index 0000000000000000000000000000000000000000..21300725db63efe7c67a714392789bd66a841e3e --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/table.tex @@ -0,0 +1,28 @@ +% Auto-generated by scripts/eval_metrics.py +\begin{tabular}{lrrrr} +\toprule +Metric & N & Micro mean & CI low & CI high \\ +\midrule +base\_success & 69 & 0.2754 & 0.1884 & 0.4058 \\ +base\_utility & 69 & 0.7181 & 0.5449 & 0.9239 \\ +branch\_car\_at\_8 & 69 & 0.3382 & 0.2130 & 0.4945 \\ +hidden\_chart\_oracle\_success\_at\_8 & 69 & 0.6667 & 0.5507 & 0.7826 \\ +hidden\_chart\_oracle\_utility\_at\_8 & 69 & 1.4000 & 1.1926 & 1.6081 \\ +outcome\_ptr\_at\_8 & 69 & 0.5072 & 0.3768 & 0.6232 \\ +pairwise\_causal\_calibration\_ece & 69 & 0.2326 & 0.2041 & 0.2639 \\ +proposal\_oracle\_success\_at\_8 & 69 & 0.4203 & 0.3188 & 0.5362 \\ +proposal\_oracle\_success\_gain\_over\_base\_at\_8 & 69 & 0.1449 & -0.0145 & 0.2609 \\ +proposal\_oracle\_utility\_at\_8 & 69 & 0.9435 & 0.7415 & 1.1701 \\ +proposal\_oracle\_utility\_gain\_over\_base\_at\_8 & 69 & 0.2255 & -0.0502 & 0.4211 \\ +selected\_success\_at\_8 & 69 & 0.2464 & 0.1449 & 0.3623 \\ +selected\_success\_gain\_over\_base\_at\_8 & 69 & -0.0290 & -0.1739 & 0.0870 \\ +selected\_utility\_at\_8 & 69 & 0.6054 & 0.4142 & 0.8217 \\ +selected\_utility\_gain\_over\_base\_at\_8 & 69 & -0.1127 & -0.3777 & 0.1160 \\ +selector\_regret\_at\_8 & 69 & 0.3382 & 0.2130 & 0.4945 \\ +success\_selector\_gap\_at\_8 & 69 & 0.1739 & 0.1014 & 0.2609 \\ +success\_support\_gap\_at\_8 & 69 & 0.2609 & 0.1594 & 0.3333 \\ +success\_total\_car\_to\_hidden\_at\_8 & 69 & 0.4203 & 0.3188 & 0.5507 \\ +support\_gap\_at\_8 & 69 & 0.4826 & 0.3068 & 0.6271 \\ +total\_car\_to\_hidden\_at\_8 & 69 & 0.7946 & 0.6198 & 0.9978 \\ +\bottomrule +\end{tabular} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/target_index.json b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/target_index.json new file mode 100644 index 0000000000000000000000000000000000000000..bf98fd3b10c4105d20658db53fde359eba827cfc --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/target_index.json @@ -0,0 +1,929 @@ +{ + "audience": "evaluator_only", + "base_candidate_types": [ + "policy", + "policy_residual", + "anchor", + "base", + "expert" + ], + "candidate_type_counts": { + "expert": 419, + "near_miss": 838, + "no_op": 419, + "random_negative": 4190, + "wrong_direction": 419, + "wrong_gripper": 419 + }, + "content_hash": "1aacf9de98648fa36bef14e137eb43c7c6973170804650814dae41d09842c708", + "dataset": "/scratch/knguy52/dovla/experiments/six_task_h16_collection", + "deployment_candidate_excludes_expert": true, + "deployment_clean": false, + "embedding_content_hash": "e4374b56912fdb6814eb0320518733d1602a90e0dd2f9268f65c8a148751f963", + "epsilon": 0.05, + "format": "cil_charts_npz", + "group_hashes": [ + "008d30588ad015db473bad0a71c13039cb1d7725fd6345f85f61b4f135a5914f", + "010f7c7c4e75115a818b44fcca59b1b036ccd9e3bef5c336a53008f8a7b0c90c", + "01349bd14e96a28b27485fa7b4152088fee51e14509318f34e89cccdcd369c86", + "026a5501ef47551172cabd8769a56ef391320107d1c27757256c6804a0bc52d2", + "029379cca8b2cc44b70649722a5e1a91cb6e609ca2ca13f242e01ba12bfca76d", + "029627d900d662439e83a0985be181083aee1388afd0887f6e0fbe802d8d33f0", + "047b5299a16717c2aea07ceeb278229f0b4993fd8fac84c7c7added14a47ee77", + "04d0793c098cc4e382d401ded0ea3d56c9677d2ad834b5901da53ccda9e7d606", + "05989b6549b3ff8e8e6c76d5ac972447f33600ec31b946a59777fd915ee136f6", + "059f2165f0f92f81ec8c873a978423ce310464a3cc27b1b08eb88fa7d5a5b3ed", + "05ec227fb1c3e112625c754c648e95e326449767ace3e1602902db3ad71e378e", + "05f2d0050690209c8abcbbf38499e0e195f5fe9078d5964a2fa4d351dc340a4f", + "072314f777216049af237bb018df2e562ad21accca2928fc4a1728c4c0a70dca", + "0746e4881ae94e6843c1d7e85f889ac5196b2a58ca2f1cf5e2c731735ccd70f8", + "086e7180f0fc674f6b011339a3ab06feb8a4d914744385734c4af6938d63ff60", + "08bce62400dec82cb7f3de6cfdd7e831fc3733bb7d920e8a13144702403468d2", + "0a68ad44f481250311d0f54d46938b1fd557893af5d9efca367ded1c61e3c2e8", + "0b50e4bf134f6df63b79deb0d5c1e8428d36b52ac3b97260c5d2d3f3286c025d", + "0b5309f59e3b116aaafffd79932bf558afe148cfa503ee100797c472adbb66dc", + "0b677f82a30861323128c400c43186f364d115c3646f77e0b5b6fbd9aba93af8", + "0b7b725b4ba7064a7598c6b283e2bea2f81f95b08e5390df39bdca14cf174397", + "0be3d37ad2ea7b9c4768dc6be2b505a8569caafe7e3c850bfc5f466cb00ac715", + "0cd039b28bc0a289515b0552105237c97f010ea8f52f1d79d15ac23b2c5f51b5", + "0d02743f28fb0fc84f8aa9b502d34a15c82714c2afb9c8ea8a0e2731a134cba8", + "0d5ad7cb6f339dd2b38a88d486f273f01a85efad9cf16205236872fce6299e6e", + "0f0804f0ceaa4f105cabcbe15bcf37c28c8b2deb76adbc77d10f106b31d1a19d", + "0f64d8c709a8f757bc9a972ac48f24f5f9f618bc9dc10c00bdced3ea5995cffe", + "0ff07d6f8629a231fac814ae3f8a9d18d9f2d5a5d3144b16a20ae7d1aa202cf4", + "115cc5a336dcfd42aaf2e70e91399dcaea0c67511c2d2776f0583d533f3ab1a2", + "1214e59fe6c7a0a2c552d75d76556b94ed956f9483f43c23c9c918ef74bed5c4", + "13bdb78c493f25aaa60ae00be517b8d81a019edd3fed9db1d110730c682cf077", + "152c1154221df1c61379262f4431935a6643a8b056939e6067a24277dbd95144", + "1736a3e41589f068c5583aa47361c5a65a38c756d7c83dd2b291e533f8ccf29b", + "185899d72412376f441583214f89c8b4f5919141ecc7d26040d421ce38aa827a", + "1859b39dcf1792ba35aae7f6525f1c5bebabaa6cf96764b6d91efd065b7c5b72", + "19164061e9741ae8144c089d22914c6687147fb1eb9769ac6138ef3ff3aa3bd6", + "1985928f34b5eb98e58fc7b2e0ba76f219326d56aae7e44d5e05b7c0e37a6a47", + "1a6b5ec8a4bba5d6cc47c74786b709c3f7fcc745cecf70a73dab8b5e1196081a", + "1af3466db9afc8d2081db54c173835b8ca6499309fe4f3f3ab21c9e73e73c360", + "1bd01f63cd4031a48713788ec00bd2f31261029118e648e055cf33c285d3ed15", + "1c59c34bb7ec1e57a6c4cb6b99b155a268a5905621800c9d03e32626d2aaedc3", + "1c84f9a3ba87babf7017b4310f0995d45d5e2a6ec3856e30afa5792152a078df", + "1e0dd390e3e7b388f54330562460b0c813e82ad862d26c66b42c6c3d2e46e2bb", + "1f7225e5a9e00d9ab87b51faeb0874623ac983f56dcdf4a855d0da29f861cbda", + "1fa325165da53abe38062a07aba9d719b26a7e890f8663f320ba2eae96072d14", + "21707fe0167d5227f7f416747410e40ae3f78c6c2b825361ff3f3091d56f42f2", + "21b0c38799dbd91817654ead09968002f251beb305f2a0dd488e70665cb691e0", + "222b477e56cdc6dd529442698720ed23b2fb6089d0a934ed3cb97104669ad1be", + "227321345173cebc9198ba533e27dc7f68bfd22fe075783a60ec7c32a37ed68d", + "240fa7babba75db5977aa68dce1f0d77015a455839915d71ebc76b252acf7399", + "24bf7cb1b06503ef7d2d0d33b11d92f096a2d9f043ec3454dd24190c2f447410", + "26ac77dfd2a9ee1f8346cc862a4c76e04a552d4bf09a95dba0fd72e86c22d544", + "26faf0b54827c513fd7411133f3d655c69be1762be7262b7c95a59a53c8e5376", + "28ffdf18cab5b40f176891bfad5f86b67c78fe0a164a773f0ea4c9e4eb3e2e89", + "2974927c1ec6cdb00ca053ab7064ccfc1528d35084141b0b55baf42bd1679e94", + "29c243497d2fe43220d59c5689f78e8fc25be720880c7c96fe3bb83cee571730", + "2c519a92b9c61870a25bd465e387ddd56cf785dac01d3d799f1336071a688cf6", + "2cc11b7818e043aaa695b6cb39e23ca678666972630ac06f4f9f233d1f7f0808", + "2d3b45195ad9548f6d927e45edd80c0472fe6b99444d7bae2078279d94f38dcf", + "2dd4376fd28fc3d5be4efdb5e4d0788a73c5e2e017433df7afcdf4e295073e71", + "2eda514c8c3130822ecfb887b782233433591d277b4d5544f35869951575a3d5", + "2f62b63771280281f0bd7b76aeaf94b18feb3d9e77d5f4c3e6c941bd1b24f682", + "2f6c7e63a96faafb2769f3bad09af0dfb334b7572bed6de64bf9e9fe48a46bb2", + "2fc4bcded9e3b5488db04aa8d76d91801cd086ca4f9b1297717a52bc367e9b16", + "2fd10e31d0173bf7895979bebccc703d39b3bac009853985801006d7d6e2005d", + "300fe4259c0457ea26c5c5ddcc0ddd771457c628008b83e328aad35d009a7988", + "31a5cc9dd9197d83dbbd3d3ca4142fbf28b2fc8201bb741270773cb1894cfc7b", + "320f6a196e59b2b4edd608ebf7ee79f17c8ad5466630843e775035c4846b90c7", + "3228ee331db1a462dda0f3f48e0e61c214e7605e1108cce578ea5f2fd595f1d8", + "333b07c20959eea559a91f453fed7eb811df3829c14be6ce7a0f2cf0d2b43920", + "33555b978758146c6ef5db44724a5c7d0c6bd220b6f59563564431729312103e", + "33e88827861239c6dd5fc396fc3565daba3dba151859cb16e84bb12f27cdecf7", + "3404ffd881c45722234ea44705c34a255fd4c7bcb8bf6bac83c63d67405866bd", + "3419d427c11b72a992314179016bb0a9148e4e87233c09d9a2670db8fcca245f", + "34fe8be6a6881aede4d578671114f9ecebff8022ff512590dcb98d7eb56136a7", + "362adb9e2f87f785eea8ff421cc49108735a1156b118db43f7dfb3640de23566", + "363bb1c2cc2d0815667febd486acfcfbd569db438fa75e5e535e1533c735964f", + "36b9c0d8e8dbf42d0ecf7b80842f7ece7e817229b8d9d81b9abaf789f2c84bad", + "38d12b23358dd923d0dea27b84fe3be71eb3da4d43c60a7437fd0d79429cc151", + "391a987cfdc35a7003584b41e687fcf13b3ea16a0bdea3579e38b9b68283dbff", + "3975173592d19589ee2483b3a9c729a1b09e357dcb9b99289e91e364a55d34a4", + "39e42a7c1b6df133a041cae19e34e0f2b67091d7aaa9fdc857e8141d1a02e0b2", + "39ea289587d7c4a9fd58e23fb6d50bf46589a1a9d30d37f385ad74547d61262a", + "39f53df98841de6d36ebe7d912d78c71b7a4880cc7bc08db0e52dab1678006fe", + "3c6ca95fd406cdcabd3371a318249e933f024d2d96c627aa2735a8cf7fae7804", + "3c90d5d7da13b39be123adb5d07405cd5957e4bfb7c514020dcd16c358ab80c3", + "3cad974195456ee5a2db5145a90ca0f2558e212a4c893a0616803f4b58326e77", + "3d546da4e682f1e5e2e378874bb802a821fcf1a2cd8f3ad02fb4c2c2539ed504", + "3d7dd030d76915bc67fd843d4a0f7f16e93fe56d1ed2dfceb77e3c1e846aa2a4", + "3ea9916cefe474f99143e9d69e0363de2625d9ce34d093043f1731a1a36ed598", + "4085a47a2f6b662206c0c7fe411d7b713e52f429bdefb47b0aee4f7f7eb40065", + "40fd8700ab3b156b4b0f82ea40b48dd00341dac9dea27e31a89bb53587a81fc2", + "4177ad99c83f31a03223c2c57ad0633ee8d930604b5da6143288bb9d95a073c5", + "4199a7885060607ed8db3b1218328efd16d4d2ccf80bc1a81d0bc590ebed3232", + "426e498c237a053ceccae427cb0541c4b237d508b4464f772fa89f78625b4120", + "427b4bd0585c94622bc045b12e05ac66ac352697c428ca2962489b2e099e099d", + "434cf031465e32cc99d0be0df1be0e209bf206de47292393be87ae541ffaecc0", + "438a37d6b4bd0e2fd1adfd430acdf90e336993bd959f79d5bf71fb37a825150d", + "43ceee1b0eca37c6537d26a22ae452abb5881e096dcc2413a056be46ffceddf4", + "43d3ff2bed6137b1430b5b5a3b10e54c39ecddcccbbd9a41e9a16ec6604be4cc", + "44632cc411cf104f311f9118f831b8b096458e112ba5644c00e7c4c230cfcca6", + "44bb557346fda02484ec041ca21fb14dfb3de5c9721d5d129ef0e9ba51181358", + "450ebccf76e662aaf43e9bde1076c30ef05eb7a7ca03cc5e40d242d4088010ab", + "459c03ca4b12a9927ec189194a6dc2732521761d69a41490909503ed15d0be51", + "45d4b64e2aecad1a79fdcc592484933cf1a922e6ef3db9e9aa626ffe9e4607fc", + "46c00e829542ab0f4441761c7a1c22ac053712d38c1f1cd57bc6bed307e62feb", + "47416d5350e161048d0d62abae130ee6e13e620c1453601bb68f12566d3a8aa8", + "4884708d82754bea1caf4baf09354f1013b3932daeecfa07713ee2ebc17ad8b5", + "48e3602d53ab14882fb5a3fc964efb696a97b153329b4be01765b62164b31cc5", + "48fdfb473480e244ab50a0aac797379a1d447f455569fcb32d54fb3eea07a388", + "49c4c3c498d85c4b459939a083610984c215e633e361f0ccd2d6e11fa3fbce39", + "4aa7582e97431f30c9db15f9ed5bb7568ca31cbc5861c2c64abc20827ae986fc", + "4b63bed5359a285c67e9ff7d84280f7b0a40c641104fa0a1a50af5ce0ca419d7", + "4bf3a488c5402697c040893291a578005cb34907d802b68844fc5774842462ab", + "4c6d7902a1e22b2bf6f7f4167a860be5ed39be0aed7c909f23dc649a728b3cea", + "4d0c7ab2ff619a787e1e947cfcf1c100528c47c16c90aebe19e3281d5747a55e", + "4dba02fb52c247ab7473ed24861f38cfb281456a57a480e4d64457180ebd2f70", + "4f81b09cefc1d9a5f91ae4cc1d5b2788c7c0e0b384e17512e82d4bf023c24999", + "506d3395d1567f02c0c44436838ea9889d35976ecbcb7ea30f9a50adca34ae3a", + "50ab237de0f3ad4f4d676158d6f474657d272f8ae0dba7ca4950f917c0233b85", + "50b8547d0dfb7ea95945ad82c54b630cc3ab54e0ab01033a9f19166fa41f58d9", + "50dd63169fcb6480776acb6cf689077b9fe565bd6acffec4e86bbe8b12411424", + "5196ffdffe946a62f4500745d1912f3fdb667cf7cd0cafa5ae7a035b896db048", + "51d45b6d1badda9da12208d1b9a1ff775403485fda5a8652a0e8d7d499f4939b", + "51f1b9dd8bd0de616f527f8848f95818854511884d434a2b394115e1bcc7a070", + "52479f35626dbd059b00c6c86e23379ab399d11ce8042b4399b3bc9c099fcc2b", + "52fee2dc2bd87e3c1a6c0620c5c627b2cc30490936440fc6122e0432b983f5c4", + "53d530ffa92afed3ce280ed6285c4888ba3bb59ff1cbd07fac523a5318f00ec3", + "5480d25e04a50a517a951793e91afcdbc3ea583236d08af15ca500be17218b43", + "54b739816771589b437a20ea18ad6c027b5bc534aa88a3be253cce7a37acd3f1", + "56137914bc27ccc2862b7a1a1452b01a33e6fa0d263c6afa6ab0607d40a8f099", + "5741844eeabbf8c6c21ca90af591fd3c50da313baa957a4053557992b8cc4866", + "5747a957d9972c2d47c89fd759afc4a8531a338971ce57954a785f61761f57c7", + "57acbd0e674472d2028036494f2e7e3ef2d052f4edac55fed900c1dc89f1347d", + "5897e01933db895c1bcadd473c11310268cbf5caf38c818f81d4bc96e93a4f10", + "5910b96219ff6c27133fcd192fdbc1c14b2930f1eba4593f03a31db8b7e3291f", + "5987281f5dfa9bd2089a2ca5da16d8ab94bbc2b908f1d04bf6548d9d79d6f0b3", + "59d30ed358c5b684b4eb3b6a32c5c62986761e80d85b519e1a48f6b639ccbbce", + "59d9238f47488e2bdd936a0d1c63ea4e763e511fb9b5c903862e239c35475ed4", + "5a678af69532034429641cbeabdcdcd84eade06b83dac385dd6d8d7ea36195b9", + "5bf1b42a54729b348c61228315ecb19a7b2ceede4d0e7b8dc9afa8891bc81d58", + "5d0f3f38df807dd181beb9335c96aec831ddabc4d9a634aa7c54334ab16c1d75", + "5d4e9ddcebff2ec81105e8ae79f57de3c984200f9994e88c466a0aa07c8e1320", + "5d515949f2580c7204d7e0ea416bb67a32d528e4ff3041f37bd94c08b56c5357", + "5e418b7e9d172a9813cabe7b9bf34c168db366015d182770fae70f0ac41422c0", + "5e78b744add77bebb64bcf18bf09a83d5597e058afec7410d5da6eb169100c30", + "5eac061f4ceb6a0fc89ae4a1cee37ca22bf27174233561d82042b577f736242b", + "5ffa8307514fd765cae152d50388c2481438d15691116a76f78c1fcef9d2ba57", + "600b24bbb9011d209e230efc0651ecbfdf9192844de83e795d5993ee034c8f06", + "602956f2c532b51b46cdd0c64db43929f489e5804e760c69806aed0e26b3514f", + "60c343e6f949a3098f6f33740971af76511890cd47b68c88f7ce1894d6d46b0e", + "61497e92cfa672eff759375d67829981565e5be07bb87bda0caf2525f38171c4", + "6214f701d647152e08612cd72bd1e15b64329676d053324533b8c928d6e6c9d4", + "62f0b0da66a12c0b2bb3b7f1324906246d8aa8c22dcf738728d169df29b85cbb", + "6349e990f1235293d2dbad51467abd02bf7dab9fe63a77e1f5e661bb5e1a1672", + "63b494c4eaafcc2da75d4d1235d198b3c0b0c06f6bf762a6a5fc7baeb1a9e385", + "64010a7e425884a51a7a5195beffa3367cb63dc05d7d9af85e6089f959e7e589", + "64bd5fbc7e70465034494d5dd5147ece3426224ab802b7051041606c9af60ea3", + "6518e953250447419c08c59d2ee155fd52ff507b28eeb3a83433bd89432a37ea", + "651c0e51e5e9e00565b3824a7f42082fd052a2df3b8df4e1caee0cd8e50e90ce", + "65afb3b6725976a6ee42025db81e84c935fdc10361f657ebb33d9cfdc442d502", + "662a3ab296db484aed6b385a16f29bce8467cf9e45bd85b7d4603ef7aa75a911", + "665000f91e6250f222e1d20d8821ac20395c23127d25b29b2a18d824efe5d95c", + "665b59a56dfe89b21ce7f3e1078d4070fc5d6c6aa9c32fc101782b87c94da10e", + "66db6cd423a630f043ba70653d9cb2d3912d39d74ba20aba42f4ed49be24515c", + "6707e8af129b5c01f0227ea9337cc805a44e08594231bff59fced200904b68ca", + "6762618ec7b24c3b7302187e22d53c4c8fc36c5479d927e7b22f82df0096ab81", + "68aa926879c32f5120a16375df788268f6d466b722c653eae0cb489b64c2715e", + "68c7948e1397aae7ada0e711538dbde0b894aeb6c65a2a28b789072222c7d04b", + "6978f93cc00cdf51bfb5d79aef73aa1e6d9cefa78fa0452fbd6c7d536692e095", + "6a4016ddc183de9be99d891de4af7b52506be085bdbd7b623afe6ae67d380743", + "6aa36251dd6981548c9da087d57cdd769ef61ed00e6b91d9ef9ad66fd8042151", + "6b317d5548424aa5a1e695f983622fffea91e6839ad1998a13000c2510316cd6", + "6bed9461397e0217ff6532c954a833dc28fece0cb46e647554a189e66524cb97", + "6c583b7ca612a308e259dfa946fa8cf45fc8b8ba31262221f833ff3f44783095", + "6ef52217b53a0484ef0bc7e133620ce5236c4268111f9f9e6562732ddb3c1bf4", + "6f019701fa2e5327cdf14bbfefca64df0f5aeecb6928fb52e5a01acae8633fe1", + "6f25c0d6d8191dec23c8d22c518212f1d2507ad4e3d4e4ac34869a5e3d988cf5", + "6fb7eb6b8b430c8ef441afd0ace15a4c42dfd23b3ac134c399abfa06390058ea", + "7258d7dd2d99620e2f9563f387392f056353d9cd0feaa9065c2a8be194a95f91", + "730b0ee7d1a69be505cfa2d8f82ef12619218566820fcbf2c2049d781cedf6ad", + "7345a52d3dfba26a5528debf07f380ed05b6e5a1dd313e7e108226d74edb2724", + "75038a8ab21048d173a77df2a2d9410cf256f6a587ef333f82d545a2b0da5495", + "772ecbeaf77547ccf7b918354bc234fffe749626c7bc4b6644544711300e3795", + "77a5e2b24fd4ffc2325efccbe53d2d0af89831ac3081665825a5d19b0f99bf58", + "77b31ca7d2196bab4fb31060a8c79365e306514ebcdaadc089a12ad03b7ab734", + "77da10edb61dd80d381746bfb7929ee08eaf96f64b65f535695e5f076fce2861", + "792e6b2c4a8865ef31fdb6e497bf4c8299a0cefa2fe47777c8b20dc4be86fae9", + "7a19124e06706c046acb04bc05c52c019ab35a5c0bc2d9d48e752d4ae0942009", + "7b750fab861ef3d12d2b387ed46455aa8b65ad72ade5ca7d253bfc8d8408c160", + "7bd6f159ff7920fed389c26281943a38f556af30c29d4aa402d5d1ad808a39d4", + "7d913b1ad623536ac5812120629dd8cb0455511983d5c0d1d26277b9aa0b5105", + "7e010a7eadabd6135dcbc0a81d71a0c4a40d9ec4d8e4a41b9bc887d64fbf8c8a", + "7e030ac9c9322183567b1955bd4924366422f8ba5246e22a232e5932702f2069", + "7f64fa79eb60a5f4b50f1b33da9b9b3d99737cef21da808c825c33092615cbc8", + "7f9ff6f99c5de72a9908c3fadfde6298fadfdb9eb36c32219e4a7fa10892f642", + "7fdaabf2a4be809033161efb0bbc2828cbefe75c439216450b11d705a706a1e6", + "80bda934d0265c977e22f84b9a47bc985de3740895abeee0e54cad76fd791093", + "8196baac48edec4fd98a232dd60880cfc89bfb76b772d9ce7c857a4051429a44", + "81afc453735d484ff81d6fcb46b6738fc9de062504b255a5d366d15833b2b93c", + "82ba96b0db417c596fbe2c6f05edd0aeb2e82e0903016792d0bbcfe6551bd17e", + "840446c401008c1f4f1b1f82237092cbc1a020e55cc02ce94792c3fd4c4f4ee0", + "8488122d93a4ca2429fe5228db45ba7185a2290931adc5c1c99e8cbab534ac39", + "860f89295df9fcc82bbdc1b569404fbc284cd0db6eb4aedac0d72ceae19e27aa", + "865a93b7d07abbf7233f13edf40ac7ac9c8013b44bb5b1d8f56f6628c9db00e6", + "86d9accc836046ab5fe18c08db4a4d0d5d65eef66c487b81d39befa68a892c69", + "8744f7b069ab27db815e0d8ddafb255992a61c37084b201454809d668506fd34", + "8755482d237160251dbe35fe0020ba85c14e541289f1dec4f6494305fbd8df89", + "87a181cd2f9e280ca7cdc7f0fde3e07112a363891302d8682860c081c35921b4", + "87f6acac31dff38cd80e58f8371fff6142712d897de20092d418c4a430a80633", + "88a012cdc526b418b4aa88816d251083f4853e870482f8e924e8b6a8af8ba731", + "893c75484cb41d8965549dea2b8a2bc089cb463a1e6f8469546dc92c8705de39", + "89d95882eaacf6f7cb28c46b23ba22c3ca11986ad0365890f461a800c7d087d4", + "8a41d9b49b6fb8253d7df35219e7867110656805953c6d665f128bc74765ef89", + "8a6dc00f6e7e5cd26234fa4eeb4d15a8547fa593afc36997459216137d1300b7", + "8aab254e85254d5cf1f1103bed51c9cdf8d6e287dda1b267b7d2adf0d513eb2e", + "8b7b0c0c36a755ab9fdc1c810ccf6abe54e24c1fd381c8da44fe1fefd0d05d76", + "8d11dd1ec76d295ab531cef7f05a62c5e138f2084880886a78c6855f907a19f4", + "8db7ec8b19bdce5f9886dc235e95546452a592b45a0e0377fae405b80564eef6", + "8dd973f2ade114309f025349bd4852d7fb3fed114871ab48e19583c48260626b", + "8e2579fa847f97c9029eda36fde5da04309c4e42188364edd4ade1cafdbaa213", + "8e94f3617db242fa9ca8ec1d20a20134c46fb1301f1c585863055e5e1ccfc87f", + "8ef7802d4d36504e28d179e70878185878721e7aa7b791aeb9f4510ae926e73f", + "90287d800986804804b7ef7bcdeaa94888d09b6329918ddc0136dbd31c7ac8c0", + "913df90c98f124b2ac5f5565e5e86e1043f79f34f1c6410e7843886c4bfd138a", + "9143d67f035c7c08bf56e3bb65f4d597492147e3bd06e575d05b9617fd5a2e6b", + "914a0ddd825254d368ed88a4e4c0e1492b62b685b54d43a74eb809c21c0b057d", + "9155e1827763664471baf371a8b288f53632986039fe13fe8bc519af27b7448e", + "9180decef7665bdc3dfc930e38fdaa742261568d8a323e5ebcbb5dda83ab7435", + "92c3f4c9a5c73d021484d9c22b11dc3f0a5938d92e8b4a3ac29a540daf62cdae", + "93308310bdac1dc182ac074cf0f452d5857412a3dc381d6adab000413862c294", + "93fe1b775c9d5a005b4c24ed4cdcbeace7c443fd96acde340cf37531e0d958c8", + "94adae68e8f80732228ddd487310402d7af461be7ebb5f483fcc14fad5c84368", + "954022aeea5a16063fab63a3400d1e5ae2e4c1cf3af56c94d437dcb613d6eb9e", + "95848d52f9d2b349485284334c4efe48bc0c9d0e4f3993dc3fa0b93917c0c794", + "970ae009f20045d56e65f7c177ebc783dc82bf777b5dec048d9ffb62c2e7073d", + "975adc9af6b12272dc8484adb4210f1137470d6c77b2982cf4809f2cff9b712c", + "980d048a07e2ed128f0cfc5768babeb12f7b8c1102dd264d40304becd711ada4", + "982679e3e46a917a3cff3891b491fba40130ce6ec338858450dd68b6dd6ee16a", + "98d2821446cb8cf7f5342382fc409bf4e1531234751095148028095c4d68c0b9", + "997b672b7ce7473d2751343cde3d2aaeaf840cb344d368ebbadd3d2205a3f079", + "9a0729ed13e288c93f267bb3557f4698a73ed5aa31a55a8e84b558b94825f262", + "9ab25a8cc539aeaf87f23200ed1c3f9667d44963b7f6da1e903f16736bc509e3", + "9ada1587db68d41c577e46d5046862a766502fb2dafa18a139e5f797b035c295", + "9b6021119181becda810b14ab492087af39bbb14c7b7a73d51f6e9f274aa83eb", + "9bc128b3db4a990e7db2c97b5c7761eb5452183da0e8b815fe35c85de239251e", + "9c72c21529d08c5069a521af5f213b99dc9741ab154c07c3bd0b4937801e6cfb", + "9d09df5c56e375bc9454202fa81f76636c4357c73e2f5000368c4300c30c9d8a", + "9e25bb1895a025f06794aba14ab0cdbbc2d4c900b747d42d48957a823e0bab5d", + "9e71e5498441b4b84fda9d4a381c841cdf9dda4efa3e0911ec8866e7fb9a9d6b", + "9ed736060546283eaf7a9e6d7f2be31ab3cfcce0d01a7602dc9bf3311dc84567", + "9f30c7f9e60ba8535bf2d90500801f28f4177b66ea3a2366eb50ef7ff7f9b9db", + "9f3b2d807346a6cb4c758c408fef30a978ae8b2b82ea419f13d447daad058f5e", + "9f9a2aec986ef7f8a6fdf55160dcae5c969717a28c50a5bd8a3b46bc0ad415fd", + "a00ac37fd929bfad607c2c6799f93375b96a8f3011cf2f49ff9f47201ca01987", + "a12758b2b9df7c5e40f379dbc2c5374e2fa0508e5f21c7b5cc38dda34dcc7102", + "a20b63cd6f96f12ec818d2ee4b0e6f2017b73aab05981d5a46d2c466e7d82432", + "a2727388face2b4a7efc7b6c58838a198a9e1f420e0b8852b925386e77a83f70", + "a2cd9da5ec1623c7aa739bda183c8101f171d8713dcd39db1ac5871ff56b4017", + "a5119d9b97da13e0e5999ef6c91227ce8a3b17de9bb77acf4abec2e00259b822", + "a544ddf0f57c78a44f54dc946cb6885693ed7372a89f15bd914569f1d9849ee3", + "a54c68ec203d07c2a967d02825037471086ba238263be088d59d9c7f00612ed6", + "a5ba2e61cefc84d8908c6b2bb830d91790177360ad02b3a744c640ff5b45d6df", + "a72c67526742bf14cf400ad3515ed8b87c2fc7bb4f95070a0faea29c8ea984d9", + "a775848264bf7b4a3777a556d11f38e2994d92714c0c7734b5a4d46162c84226", + "a83bf42daf636e7a77f77955757dcca4224440cff075bf7eb0370795f9d8663c", + "a93f33032986ce24e99eb38f10fbd9a62d75462d5adeca5b4e236cef0836bd90", + "ab78cf2c57454d46dbd7fea49f3293f01e7c6dc9ae10e2eb2b4fb20a3648ecdd", + "ab8b5ff6eae18b911f6e8dc11c2a34f65b1c9b6d116c91a00b8a8792096688ed", + "ab9e518e9c2bd1aa4fa1e395553620371f0c79804c89f5daeb6ce18d5b66af23", + "ac5ecc054864495a34f0b9a16a5473b786d7497474773dcb84947ed204009aad", + "acca44ec9bfa0a02833526adbaf86c7041a0ac281b28cb7ee91f18fd09b078f9", + "ad8f425b29b9b12876e4fc5fd18c3f867046ae41c81ec55cf0b6d208e85dd742", + "ae3fc0f559f37778cfdf27f640bd35a16aa6b76c4ffecf8d42baf37e4a5a7905", + "aefc7849c632d4044a926f20f1d94b82af1b7ed6dc0167d2cc249b218a343a93", + "af28d652f179b9718c6a9a9e74f9e8e7f12c437a5ad54a75d80a82c23dc8e644", + "af82611bf1b54c6df4bc9285d7264be4ed86af1711891cbdf484def25a7d98ef", + "b0ea0f0cf25530508c4a9f67de55afe8038355b7e0b860af7ba9f04aa31a3e06", + "b1bc36c06168dda8d850b543cbd2d870694b4869230cd985b20dc8b6ca23c1d1", + "b20791ea51ec80c181aa6a54ebc7d1383974764c55ee7dd3cb3cca64948266b9", + "b34c178c6a7c97b598af258fa1a791d9c0c61e8432517e3d91953c0cde131d7c", + "b40f7a9d22cda22a063bc22dd3b4db5c3598a98609594494193ed48dc8192cb0", + "b461e54387e0a47ece2b22c3dcc65391d86517ead678ec33a5d901fbbadc0926", + "b4ccc21e5ea8003a945bf634d6b800fc44eee0aad713f12e57e1984c525a65f9", + "b714c7af15a1b9b8b710eb5c7611595076321e58f55c377e69c9121093522bcc", + "b8004aa52a284d2556c496f43d30dc6fb1cdd091695e1e50c1a90f0d65f92b09", + "b81b7e5043ba8065513ae1e47dd72ba75abd98d33e82a7ca7e78e051cbe43cf2", + "b8876098ea9a407eb70ae6ec08951a4f55579c2e0c08716a7c6cc43156beba4d", + "b9285dbe600a0d883bb1a8106ab84908e1daec78d16497c41f6d64424cb98313", + "b97e8fd40a8dbf94cb8e2011dac90a9480e3f112d431fc70ab75ac357644f8e2", + "b9b5303b9443950e596764e3ab43ba388dc88bd9e9fa4db7feef060ce896c62f", + "b9c2abe9f6ecb2b7b77d64e7191d9ed09a36554e9421caf1b2e65c5463365230", + "b9fefd5cdff2e6c28088cc0a53e346a2c097998df8b6fc297222153e1daf65df", + "bb2bbefd2a016c0ad8e9e3e8858cd69d918f0eca2138b4f2289c7afd8ac1f4d6", + "bb30b5791e4307912aa9adbb690455c076e533c6285272da8d12e78f51db283c", + "bbf53aba9adeea04e3f958a95feecb13ac2d29b45a69478899314a996c78e544", + "bc6192bc3b2826b4e1b0b4d086536b6565c66d2c8bd66378de084b53625943dc", + "bce154fde165db7793c519eb7cd7b398ff52e9cd42578142e5c8ee8fb0656547", + "bd1dc7793ae70f7d999d0e13a56b492cda6f3ab0de6301699294ad15b001ac7b", + "beb32919cb900c53a3953442a5d071436449a4b9ec5b2bce6056f3d0a1f87714", + "bf6f5ac70503dd191546b89ffc869bc3e28efae51bf05ec330535f6e5286e301", + "bfc5863c1a54513bc8256de6b55beb115b2fa384a58f23530fcf523dcbb52e1d", + "c0b1165b54a3e58bae71e154a65337aed1918c1d065439d9a3f9d65f7d1324a3", + "c10afbedc26936296433deb12073cbd34317fe0ab7899c916c8cfe4001642ac8", + "c14e6d6cb9f0b03afcc8cc196c3a29e3e9e2306becbe004eb2313e306cd0a74c", + "c17f9ccd9fd2ea3d7d9230a42289690c9decb88e0a880a42037a79e94b100e02", + "c309e10ae79aa8821732639fddbc5af8d7480dcab2d0316dd331ad859a03eb98", + "c390b96eed7402670db4562842a2aaa9581848dee1538a66faef88134835c045", + "c39690ec883cace98ebcccef51fe48fc75b323082a2c0ad951b40531383a9717", + "c403e6f15abb1ca9d51a883ecc88cfcee37d64058e19559d87cc8e9659369ddc", + "c628c9f4b7dd0a2931ccd0a68cdb70335c49d125adc341c75ced526e0b5d7439", + "c66503618c5c0194e190365bfdd6f18fc85f6e8b14f248c0724575689225af85", + "c669ba0197f300ec35c9cda7a239e8a69e73488459a625ab9476b09624287715", + "c9ab26239629a397da5a94679f176211063e4fabfd3c41a2481405a5f70b2e71", + "cae0fe58a9afff0750883faf68e9d4ed511d4adf9f799828e6c7c331d0b2733d", + "cb0f5ddc19e8db59d7affe16f76ef2585cc9a4c992ec4aa144bd25fbaae42eb0", + "cb42cf2a13ecee4f5c3585d182f1a5e0172acde5cdb5dca20df348c61d5707e5", + "cb7117229c57ca9d821d8ec24d4a922d8348a5fe3325939bf84dfb18f11eb5a9", + "cbde24769f686efa9b3c354173679e351c5ea0d76fadb3e69593a7ead04a1ee7", + "cc70ac956671e95234876cc83614c2f88db9110ecce9381fd0ee64803b299429", + "cd36b417bafdabd7721c88c17d2c29c33e9a11cc0594f52a54950906d4a44fbb", + "cd47f108a726fc44a0eaaf3d4bde604dbdf8c41861f2c9a64ef11c7a36c8d6a5", + "cd4b3470071fd954fce44c84264d4bc9e61790b37d2b5e9374efef01316e6dfc", + "cda1f2569cdb637d0550db64ff59e80ba7330792ac95484c37cb0e3d113eccf8", + "cdf79c0142aaa8c5353ce2d676a01a9865903051dc987c8b73cdd73c76ec9c23", + "ce43ed1b892a3a7b12e0bcadb397a918718cd979e9e0795395c986f56bc2a750", + "ce7c329ebd2e1b1d9bbc17a9b9b74bcd914df56e03745413b6dce38eb4772bcf", + "cf2039c1d46672c9f9a5515c3d60575e6be00fcb93c2cbc1477459ff50955ef5", + "cfc4f4161ce90cb06c135ded875919f4675c107e786fb6adaa9070e9fe56d34b", + "d0ac396c83185c55d031ae1fbfa33bbb017d14a316cabb55503b6fe772fdee36", + "d11329eb550769a78aa2d9f8be77708dd9ed7527d706dad9fa2ed230fa38fe5a", + "d140b4148ccabf6f483642b7ce61825956b6e0a84ca2bd32a08a98f84a200a0a", + "d152d1675da0f33cf86b1664536034d4c9f4ddfc901e51c018b87280284756df", + "d2d52cebb51643d7f8ad3b2e52987b7c7f68e66e172418fb42636767f927ffb8", + "d42152e189ac72bdfc7e818137ba41cb5265a008aa3cdcacfdc560f5af5341e0", + "d49d739f974de58a57bc4f74207ae2bc8d8e66361adeaf55e961a27aa1eaae65", + "d563f93f87d17243f00fe477d8e78157ae7edf218e5c88e941cde003bea36a6f", + "d5a907cbc19ff057e4dafd6cbd075d983f7e71085148ddff5920871bc2167575", + "d5e7f6b0de3d88f2eef5cb8276e5f45caa1fb6e902025627b04b490b5d98ade1", + "d65210aa0d78bb32b64e066dcb64b6f8f71f71779a5d499354ddfeab22f1332d", + "d6f1ed8d805236ccf9f98fd65500bb4aa9ad8c41f8a071ddc29ca8e752929ae7", + "d7ad91e423e48f64697d40ac5d829b3386ed420ea0fd9841676704f0a062dbb0", + "d87661fbf955f5f80d07effe80080355aaeddb42c6cb0409ebf62ecb5393b674", + "d90b629f237efb49c3ab6d0aff93c838b4db299e6d7feee7636508d12b5c5fdf", + "d911da1cc1aa1a02712d5b1a762cf0b5d5ab389492c1a4b0559299f05493d3ec", + "d99afab46c17b26a07d8f71fedfadaca28767a8f350a2d9589736ce7da6cc3bc", + "d9df49c2c7600b22a4c93bb4b1f9dc9b1157f0117b215b7d1c00e26770ed5582", + "dab08cc34ed2fcdd470d03536ca642ff4624b381eaf9d536195bbe17ebe9c17b", + "db064204080f8555787bb6221183aa561add66a3383b4ba439f45c6ee5a2d96b", + "dccbc6279f3222dbde7759e12a7dda491c55472378491e24482e4ba527959a06", + "dd5269687a6c1f75d0234677f460ecef52a58ff78ab2f9ca6822a1316b36c5c4", + "dd87ae7fff40771e69e0a6180d1305fc6d1d0ce432e9aced610fb3f49194e7c6", + "def0de3ee663dc1c9d1b8df1f0127da4720d8dc16adffce81c060c330dda769b", + "df1f96b7a4242177ecb40fbebf6ad7091106e10dcb7b8be68cdf62ae9532d7ad", + "dfaa421efe0204df2a51dd0f34cc72c3310aab26426b107e3973b954cebb730c", + "dffea2882ee92b26abce07a5376e46308bc5e7a44749f72858eadc0aaf862fa3", + "e0077b61aca56e8aa2a018e68311f0e6aefb5d030952537cb7eef012f41369a1", + "e031d45f43b36e3034b2966c9f75218e710ec72dcc2a5e970c1d4244e4913e31", + "e0de0c7672d9da979ef118f4983eb9ad6ef0402fc8e6f5935c75326e26231f68", + "e1b16c59c51132e39f175f624b134609dc6280d818f8c47d9b6eea77e7b193bf", + "e2679dd5d22d82f7095519aa46615e702dcdf209163df9cdb734d295f70ed8cf", + "e2fa296a21818b123e3d7f367826c5e997b1371c858beb73610b110ca8ce4839", + "e50781f70c95585a91804012aa888669166226a18d9fd8d104cf901f565bb067", + "e579dd010621e89c430a10b2996eb3b216029e121a77237418f7280ad0f5d088", + "e61eb554944b8e39f620309e492c5ac7361b46cd9706619aa516e1c87ab2f11e", + "e6ad5e3ada47a44524ddd841faee42a564c8b8fff2eb255362771dd9f3cd5a55", + "e76923fb355d375ab50ff13eeb1ffa782eaae18ccabfa627716742e9c1390653", + "e7afca6e54dddeefdb0e7f1b02cc0cf2abf00561ecc8086fb882ed8ced4928f9", + "e84d29a0132a44780d83de149ff7581d54b4bc7acba7241efd3758dc613e3cc4", + "e906c0d38c20ca1f640313df9905bb9a39b48d7fe15c57a2ab75f1565a191fe5", + "e9173b067eb363df353546f8567b60b5122ced586ee8c343ccbb3bda6ab73592", + "e9795c286533eb785b99b4ec8b8d78f7a1eb8bb725e0d7916cafbeee1ca8003a", + "ea4b1dc02232d972aefbc800b4156284095112baf01168056ddc24b8a247f512", + "ea7664f177fe6c9fea15443fbf74a78eaed5618ae0f37bf189a0b726d17a5d4f", + "eb3537c28c26f4f61dee000695de989b41525c029c7d92eaff101a6c58fbbfdb", + "ebba25c528191213226e54e837bb727c0dd9fb48ecfcbfce91412f7fddee51a5", + "ebd7b1abe0d22ca80aa8f56e1d62c8a38ff410749cc212d3da44eb580195e0f9", + "ed4e3c8d856aaa25e7adb8ef0ab63276a38f2de8d211ddff05b794283c206470", + "ed82a193afed8645087d6ca4289eb20157008afd83272002a5669d25cfc51400", + "ee15e6d1acea63bf964c93eee42be5dd3029058f8770dd9caf4d5337a2c01bd9", + "ee5d303b876c4dac62384687b039f9658fff97c84fd87b08ff59b99a50a2c6cf", + "ee623702315a339ba49bed2fe17471ae117b38e2e55d2469e7cdd4c8ab88565a", + "eeea07e1f63be94a7ee57285c6c517fd3ffefdb1f48e02d1318b7d0d204db5f2", + "ef727e08aa8670e19394b9f923ac237625629c01368ef4a89e9cb565aad5d2ff", + "efc515168087302956545ad66a1b2923d107e4142ad40dc14c43ebfbc769e247", + "f08cf650a03124cb4ff3d44fd4a70abe0342eb048dca6644323b335770a9a083", + "f1176ced474ef2f0ade0eea28d45b69b1c172cb97124465766a696984397688a", + "f2143c8de49b21841348de9bd75cac7ffb13b71d118249abb1917698450f638b", + "f28ef034d38ffdec076a212d91fa453834eeab284db013d48370f45b23ac1e1e", + "f2f034448bd3ed4bf77478619b5c901fcb70f6a8f3ecab9dbbaa143a6953d9d0", + "f4d897d2739e313ae9a7f4fab2d18e96b751d08263efa31b92c65c39b569a099", + "f5636406724c3560b13585d91d6a1b58716ad37a776edc88af285d7b05f70d76", + "f56756762723fb4248c7c9cbc4288058fd1bf9a5b92dddc39c7454bdac885124", + "f582d63ab10b23f5177f0615badaa3359f0a0f7a56fb4165437c0ccb3bcceccb", + "f661f3a1841fceb3f780a7b40257af20ed3e704c463eeecf783b2c749d7a916a", + "f6685d129fc3f57ba35b71ce97af43735c428fee02c6c50fca3b8dfcce32da38", + "f72fe1265f2add1e7058bae000d66be9f4ec18762d6413e0bc58548f043a59ab", + "f731dbd87c85ffe281c1d8740aa242fa9a7b1d7210e0b1ca69592d11f6f11f8e", + "f7d9e96925c19e11d9a4329b0ff009a383d758857a44287115687ddc7d098d93", + "f7f300fb09d9d92cf906384f012ebe09835fb0d03b902e2a230c001b302b9274", + "f8fac33ac4e76d8146004ac03838c9e4977f31db2e3226693233e411ae4c5cdc", + "f919f6e81d3e9733aa6e62846129ac10370ca19d3e72ff8594876e9e77334efa", + "fa49239daee14b0d44eed2285a78ae6d7ea97346a5f71a9439f394afac870c63", + "fb3c26f9e5d25c1414b4eab1076ffe73023df0950ccb8798d2b2359e7060ff69", + "fb7d518eeb067ac77240f19ac6cece874b2af904d69198f83fab2f6f7de203f9", + "fb9f9805796702942ff8ce23f4b5f2e2abcecba678a82eaa95eb2f35d3d37cc6", + "fc3e326d60df78352c6da66ac491963b7c457797ebcd1e96c2436594fe59a6c7", + "fc6bb05f32006e045e36a3300044ffd7205edd7a2ace65ddb7fc77df71abd6f6", + "fc96267aab2baa5d3ae73bb0adc76accb567533bcbf6a69ddf358b7ece8e51de", + "fd03ea8635c19ccd8b6496bccce844e72697d6a638a7fe75234441e839b7615c", + "fdbb4ed5157a5396e2fb421e74d4d4cf4bf3131cca117026ea1f51d77a857994", + "fe4209d418bd8745b7e03e0450b65aa434d749e4f0b0f3793a1849ebbe1f93cb", + "fe54a9ab1415ed784e1c9d3aaa035570a6cf495a610171edc863f5743524e7f6", + "fe578fe58231dea95dce9a9201b55208564b4a59479d910d2144111d16e809eb", + "fe62a651f7a3b22cfc8af16729766ce8b0dba209d70e68d12dea3c7ff7c5d3c9", + "fe895f4166f609e50dcbadd349f7534f86356f5dc12f47d6511059ddd8fcb4d1", + "feaa87eec12904d90fed4e3ae7cd9189e9adf521af0c660ccfc811e39398c52a", + "fee98f221151bb980cdb862306da346bf3b0d9cec0b9cbdd9c2ba13ef635201a", + "ff1366cc917bc6dcdc630665d80888e067a67153e0291eb502ce1c061598ceb9" + ], + "include_outcomes": true, + "label_counts": { + "negative": 5134, + "neutral": 1325, + "positive": 245 + }, + "leakage_contract": { + "deployment_must_not_read_outcomes": true, + "nontrain_outcomes_are_evaluator_only": true, + "train_split_only_for_retrieval": true + }, + "max_flat_action_dim": 112, + "num_groups_exported": 419, + "num_groups_scanned": 2873, + "num_groups_skipped_by_split": 2454, + "num_rows": 6704, + "object_embedding_content_hash": "411ee7d4c96458879f14fc9460c0e10d5309d83aacd5552c6bcbe089e81838bc", + "object_embedding_manifest": { + "dataset": "embeddings", + "dim": 64, + "extractor": "rgb_object_layout_v1", + "num_embeddings": 419, + "path": "object_embeddings_rgb_layout.npz", + "reads_outcomes": false + }, + "observation_embedding_manifest": { + "dataset": "embeddings", + "dim": 32, + "extractor": "rgb_jpeg_stats_v1", + "num_embeddings": 419, + "path": "observation_embeddings_rgb_stats.npz", + "reads_outcomes": false + }, + "retrieval_index_allowed": false, + "schema_version": 1, + "shard_content_hashes": { + "charts_00000.npz": "6f6ea9dd2f6809cbed119598842a1dd6baaf5db099587a2748a00264aafae4e8" + }, + "shards": [ + { + "num_rows": 6704, + "path": "charts_00000.npz", + "sha256": "538fac2468d1bde33b9e76f5233e331040a8c76ff5fd224727bd8140686f4071" + } + ], + "skip_counts": {}, + "split": "val", + "split_fractions": { + "test": 0.15, + "train": 0.7, + "val": 0.15 + }, + "split_hash": "9d1dc1c10868f76fa080895df0f2730c173f12b6db14932599d55c2b60859381", + "split_policy": "stable-hash", + "split_seed": 0, + "state_hashes": [ + "00febbd1a7ad126d88b65aac55a35a9c4c63bc11e0f6e45344ddd4969242b950", + "0161713e96d5c5e1a45422cbbd0367210454a00f7d6e9840771090f4849fcb5b", + "0271bb52312dbcc4b57060d87dd5b3a514bb13bb1e20b235ad8204d61ad50a24", + "0544a922f6c9c662f2c9238466411bd1942870f7f182a508fd124141ef777c01", + "054fa57f5f9a67e3839a4c9674f9146dda13d27aa43c99ab5f87dedb1c0c2712", + "069933a0983331f68db2455b6a2b4fa216a97e119e1fb5b42a65adc252aca505", + "06a95069882817d6afbcaae8ec6bc12ed361c0a9446c56639880a8b9223c0c74", + "06d2eae8a8bb4c4503348959d65191f1f652edf6267e750861de79a6e0f1b1be", + "08353e08c94f126c924237211ac6e55a78461a8ff88eb8a1dbddbf844b6f2ec0", + "0905ba70e353c374ec760960c11b7a7914e3a0f401bf94a7772225dc39a94cc7", + "0a48350b21b9cbc27742e61775d9a3cd611daa5905f96a2cc3c59625d4b081ed", + "0b7be6179dacba118f3fdbe0e890ffc2d4ae1d15a87a215f41574bda0b24907a", + "0c4be6df74c82811154373ce6212cb0d7766cfe37020a987e1c1a553fc82aba4", + "0c5278be095a7a37efb891610364a48e4a4d84d044f63fab14e15e9351d2a9d8", + "0cab5c0c4fd5dd2183a8bc5891151fb5e5697bb605a5f7aa3c85f87f0503a804", + "0cf52a33f7d5f568029fdd8a8a3b634cdc7443ec30e9c127a73f25543ccd2c30", + "0d7cc8eda8b418bc8993e1fdf3fd64353a35fafd2501660fe5d87c6f7ef3d692", + "0d873ad4e7f5c34b9002a03754159bc9f2641e80bb46292920d135468141f582", + "0d8db9d71f59cad390736c77c29a6ee9f9f24de0c5250e1b8115164f4bc33f66", + "0dc29c4c945813a6553cd08d2dddb3982abec790f14fca7f70dfab255302960e", + "0dcfcf107cd9dfedc537d2da4969654c2c4fc3d04b1176f8e1426633edc8fae3", + "0dff2cb77a11d1888d2a48a76c8da04fe4e0daa54b5194bdb5ea100e2eb48ab2", + "0e23f36f5e66b1e4a3a7c074a6757653fecb8ebf1beebcd1e78033f5c2656a35", + "0e3910690cb8e214f63534f41f2acc8afec60992e30a2ed4d2fb61fcd93604a2", + "0edee0a0681248fffae8e5677c3658aa6a8b59d9eb3e33d90ea3027923b85216", + "0f0d99b44408bc284f59809b1266764117c2426549ba6b7e22b897e4c24cdfd1", + "0f19062a067ca2c041986c3de45c4e8d4e0f32838adcf5b52c29dac097394b47", + "1023ebe09fd0537cc30774636aa5c98411ae6ed1c2dc259a2624bbd063eaf23e", + "1113eef231093ad70c840dc75559d4ca74bd3d8dc4aaefa1c0b04354fe6776af", + "11f96b224bfcd397ee4f98e388350e3b65418b00a9ab4d2746128b42b12b43d5", + "158c57e861f5dbf4fafa3370948c03172a7354fd34511a04af24d744cca2d11e", + "15a9a5593a8b8ab430be4f23e1279949867e92ff352f3e2033c69495d6f636f1", + "16a1c922cc898f22123974a540e044a4ab649945807f078770480f7297e8e90c", + "16ddcd38b720cafaa913dab508ceefddc7620486bb42e9e3b0a9b81e7155543f", + "16e7bf765913827062088fe3aab96fcdad44b4615afb788a2a07e434d272b389", + "171fce66778284ce4d146eac1ca6327cab37d27bbc4be27a2a9fff473038245b", + "17547f1d97d7d63690d4f5ae56430531c2bcb8c2289abdea53e0151d4bd9b79d", + "18d69dcc611026a63a4602aa482606f68eb3ce416d10ba04f9fadd6b3622a991", + "1921b5d9fe877551d561618aa96a78ddc9480216ba450d537d5e4b27206c4999", + "19cb279a5068d0754add0249c6c2577e07f80aa33c56746853c796f5418fc7d8", + "1a43f6367e6d71af41f099f5216e60ec305d43486fe887af0c2f19849f406b75", + "1a5d3ae36c55dc60075de4acb795d4258bd60953aaca4786039280f763703323", + "1ad053ad6219459720e464e61ba1165cbbe5210a308e3f80962cc0e4376e38fd", + "1c665a2c6b863f8415c790f358f9138f56958247ed05638712d6357c208708da", + "1da1e71bbc20c3fee4c7494196a98750f0081b55357d61ee7cdc53945bb7965e", + "1de189fddeff2424f521f5026da163632e4417fac1f9336255f0e62614ee5bb2", + "1e6a33aa31538f769c69f26afd8c3fb7c5f0290023735688823d00e5e34f22db", + "1ed8594ffd73dc1116a58995bf378e36dbf065e9af2b003e643eb9933e7422dc", + "1f7544e2474c3c34d37aa4fd972618a6f29b1e84a054ebe0dffc58c8a2e7a71b", + "1faf9206f8f302ab744b3d89effef3a2a055b0e600c874ec05045043e1dfbfa7", + "20191c4fcda51e9e83b7e86251c8a03607d12daf307afbc00d51d85e4e7def84", + "2053f765d67e1e99fa2f2d67b7c8dbdc0edda26a58e6602fd99f9153ec1e826c", + "207798bd670f5ca13a6fe29e77f180432de0fe8e09512311636a9c49b6d27523", + "20cc15006255a0f0abb3e7f4e59313cbaee4bb4c70b8fda8bf24a8429d7bfeff", + "213e926b1edf4eb61706e283cd09447dbb4e7e529420a94918b6dcc6593f0b29", + "2144d1c886bb5f05a02f9f782ab4ef7451d251b76cf8774d23862aaedf517c32", + "2171371427e17c1995dad1d3526082f7b27faa7ee8f8d115d3e1f5a29fe1bba6", + "21736847d7537ba2fbee1d21f9e7c6f55dd5c9df2239977d889770ddf12484b2", + "225fc26bfe12be03306bfd0e7e321879c3e0515185c5941d4162e432fe6ec2a3", + "22af0dbc179d12978f8a50de641f892441c0fa6ff71eca94e9a7d4fb6d805446", + "232da4dad7749216ed41234b7490922221df235896962a2155fcb14e975705d9", + "23466412690e3c55214d93f7681b36635a8e1359f0ede7506e7c8801e12a3fb7", + "23b377bbdaf00ea187f5864295d60d63f998e9923abd69ee7308b5224d0e37f7", + "23c0c827d7f0d4742515d5227c112062f3eea5b8a374fae7b9f04a34b8a1ddf8", + "25438255a311c51c782bd1ca4591dbe9229bc3be0160d5ef35b016acb0850b61", + "256f6415a7e248845e28ac5e7d542da9867ae0f5a8765a3eae7809291bec1e3a", + "279cef5e2ebb26ddcc87c670075598b35cbc4366b39e9fd22c16533fd9f6f79a", + "28a15d04c873a079c1d06c470d2d77b8e96e025c30b859f176dba529fdcc514e", + "28e0ea4b46d79646462b01eaa2c675b5872098bba0bf3d66a75d1c1a750261ce", + "290401966fd54d78073a1984e84efd046b150ad5b91413e6f5bbcfd35a27c19a", + "294c656a8b36af0bf9249985a8fdac65c4530e7c81d78a0b226f721ad6f9d2fe", + "2a018927dc86ad3fbecca897ae6ec63485d334434829d3670fe94776f8444bea", + "2aba3b30872a625774dc16ad32767e78fcc9e0a1e8e800f976967682c2fabf89", + "2b68052385906b5436a650c00b9c3ffe0253943c3a86a40187fb964d4fb2870c", + "2b7e6e58aaa56a83b488a80cf01f40bfd6fbd5941855bb29c2ce21221ef78374", + "2bc8e606bc7f0a2875ad9a588f3e7a0aa32df858fe5542ff8f7c102cc9c9f14a", + "2d0fab9e61b78cd62d1ca7d76326481635d04d61812c74ab9cbcc2b0e0a3ab1a", + "2d666e1b77c3039e2574d2c71967860a070f2166462f049a3245013d8bb9ec71", + "2d71b5cd66d133bb09dab6f0f35dd703104df4b77255a9ee1a8315d1ca6f59d7", + "2e247e7348b1c46bfa9b0068a40df1f45a4ab4d9c7ac366e14d0a9f99e7d63ab", + "2e7618499962df0d0c566b843a23f7f4fb31301bc9cbab802d427206f496cce8", + "2e89f06988ec2b92a1c8b18b2b3697ee0404bde77298adca7ce981dc1b1a7417", + "2edbff51b1020474d5e181f95bfd43c6fbfed555f6b5d5687917c0817d4ba0e5", + "2f08ab47ee8e608031042183ba73f76856f60fb0262a8adabbdd29700e91da9d", + "30928509ef611054e0fec9b07ce84529aa3afb560aab8a0afad741c493c3efc8", + "312adebbcbcb2b79ef60eb52a89d93e0dbced13464e3bf66ecfbec402be9c7af", + "32bb5ed54dbb72237062117185d38803a2821325ae9186773df5dbd2ee73155e", + "3301879024b9c79869a02253edec7b100d7677fcb2023acd6e9ae7c9cf46308f", + "333fc8cfc5f76fd98905846b73548b0023cac4f5f10cf49d3ea5efd8e0c15fff", + "3376c21147b72dc0474a3899c9eb74d175f9b78d228913dd7c3a27bc18f6af3b", + "34a2ee666ee8be188582c8f7c239b6ff96df48562fe53f34fbc9006c61b87b23", + "35f48e0a0b32a3966c94cbbb83fac82d9d6c89b6ae9194c446e085b265ef05b3", + "36178548038fbcdee704bc77af6a77590b37e6a3915480091ac1d77d9fe92a59", + "3622d0ee6c6d00a746bda15fb39d7044c31bd30e627a4a7ba5737477cc2c788f", + "3655a761324c8dc8844e8ba8b9a4772e2453ea55f7c1520925ef154dfd43df4a", + "36c71852489c6888128d3ff5c2e62d599759c400acb473bcc0a06b2f9acf066b", + "36f646f44b39a2775ef7f08cdd6256eb68b045ff9d2ef373e8826165c2fbe98d", + "36fa82aa1f5bee9ced8fb9c2bde5246529f5735b6aef2abfa3398721a222afcd", + "3718724484b8054be4ba240b10122a71068067812d05e7b9963932051445168b", + "371f27874971dd4d10050be4bbbc1e38e9afe1d8bdf5de5bbc6b3b480334e7ce", + "37487957af8670032f1ebd2698dffcfc99b5e2931094bbce10a1f5a41d17e5b6", + "376260250744115f58dd61526c1195fa32865622d2bf0daab219ef3491f00a8a", + "377d211e4bed8b5fe1de5384be5ba8ac62161b150784f0e7d2b6dc3568e4d508", + "37fdb61efd2db8d7ee146b537977800e6c42f93a35f1b452a8a76ae50aa2e1a5", + "3802165b910c46febd084599f3ad3bb68e7f94946aa316b3be4077dc64d6892c", + "380464d744f9af313aefefee8de18cddcbd32aa3e09aaf4ae13b60c0cdf2c535", + "385b3e8838fe0639b71bdb89d173cde82ce32e8218e8a6900f7d544855d3402c", + "3864afe56d299c4e93a7b4b270e10fb22a03024afefbbc4676456fb2bf09ee2a", + "386a215a2dedbc0166ce9e574929a13416acf7edf161c0c281a7d3752c1664f9", + "399ab40ccb7266269c21475263d0e7264b2b983c333db7aa09ca0c3ff532101e", + "39e3ae6e36f8d2befbcd1d844052de9ec49b87d041c1152f23ce51b5fe4a7365", + "39fc414f3d1216defb30c0764ca78e922de5fae956c8f6fcc0c6cd3ff3c03c5d", + "3a3c4ce053b1d397bbba192d6cfcf15b8c08f6e43f2286971a9321c4a1b9f783", + "3add4fdf06b87cfe5ea9fbb1f1a6fdb2b2f53181cb4f4819abaa511222c728a4", + "3b1c9478132c0351de3a4efd39668453b42aa1eb060e7f6a1def9379b74c4012", + "3b76ae93b0a9d50a284a8556fe33d6bd5255ea98d1c84822d166a4e5a2380a90", + "3c8e3b6a6980d85319a45619c74f75689a10d57455f60ad363b0db859fe33cb4", + "3e44df029d4120641875fb004a2a937c49fe4779a7ce5b1c15df1c7c1e3393a0", + "3e7dbcf1d6545bf86b069dd68aa26c7b2d4deaf965dca2f37a2c7b327ec007a8", + "3f27b12be9a2fd631200a9fbd83b29deecde1e760ec54035723fe0793de1e445", + "4136b9be1dde7532b54d1ce1f96598e153c15c3a76ec32b95813cc38b070b954", + "41498d0adc8e023aca47897192364c2ce5745b089f33013b8b9deb69e24b942a", + "4167b16627d6e167a2dbd1be8484490ec0c6da9de79b106634ad88b9b65e93b9", + "4171cf812123e2b6f87fb20b97a1992b79181a242809805cb68c7ec70913dd6e", + "41919df4cca85f0b531324cfffd6584be4830bc8bd9a988fdf7881d30a8dde56", + "4236092dcd909a41dc844bb5007d916aaf2142ba0c7332fbf96947b5b8ee61b1", + "426d33af3a135919ef5d2b06dabf99d1a1c15ed3b1bb2b9e4c2978a5a107b0cf", + "4372ac740d1727b2385f90171db7c748a03a6fa3849940c98f4b9754a7b29d43", + "442548c3a3f961d677d15b34974fe1729a5c922853972634cd4ef9a66a130b3e", + "446f95aee72b7659374dd227a9ffe3150e198e66df6434c3b4d8c9e563c35ffd", + "449d06ce2a53a15c8070a0555ccfe750c1f25fed1317c69aa040aa181a24353b", + "454c2f13e99ab69415a54b5e6b0d4d5f934892e1b5083e568be81894e1ca2a9b", + "464f92220100eb1bbd1b22e9d03626667f349d7ea6f74c23f1bcef7fc88d5ba0", + "47a9c4f96c6872687900bf48c444e916d883ae0147879a5136565d9f781aaf1e", + "4843e199e6ede40f1655b18465b9caba2d2d1afb569e977520703101f30088d7", + "49417479651902941b3e22068b8f19676e4f484938e9068df2d418739e62cc18", + "4a6cab077475699ab9b8df7a8e6207d267d0d2358da13cf6493010703c398c6c", + "4aa88d7007545b593c65410ddba0b240f986229a5b3eaf101d52da62d1d145f1", + "4ab7c6c04df7d5502cd7a523e550cff7e0e5cd589b392491a7331b95ee9da323", + "4b81a042ffa2df32f8f83b5604e4a00759eda66d89a404a2558eafba6d4fee6c", + "4bf18fcd7c9afb43f6ae51b4dc792f1c53af66240e23b0fccb23b7c3d8aebc1c", + "4c03b46384e7d52ff10e415873ecfa8c5c15e6a35dcad5767afed802c3d1200c", + "4c9e5cab65487fb267c99ba75713cfb3a2a030f49e6cfcfb4f49ba84f398714b", + "4d0131e19ef58b57be98408e41c9a58aa60c41ab4dd830f587cf209bd3972720", + "4d25e2d4328ed58c8f62e38711768a25b57fdc929210950785171937151217ba", + "4e5b04961304eba5fdde7a3060e1a8a98a4a72f1f260a71090f499a79769aa8b", + "4fbaa3eadac6794630ca6e6754e7f252387e419761876d548179007e6e70bdb5", + "5079ae092765e2fb5b517d123bd809195e2b2d26b55db1509aeaeefaac437113", + "507eb82aa5d9d9c694ec304b6dfc7e339efd292c0f36d8ad69de93022801fd97", + "50f8c0bafafcc8b5c0908d4a4ce21c87bf5b8dc7524cf3513aec538e4efde930", + "52bc6084f0ae006eb0c52083a34981493ce7918c8d02d4b5467b92a16cad52a9", + "52cf2ba1f84ccfbf1812ba17714ffb951d5a5503465df2fb7e14fd81930c82b9", + "53e0f8565233b4daba347cd65931b0d03d3be14b00b81f53eb19e8732f7fad75", + "54276b11b5dc801246bf871ed20acea86c35bc3e48d65668292c54894cda707d", + "5573e534e62463125b164ba4d53d7ff101f2040bc3e43ea9f8e66e8a78a66432", + "55c17cc62f79dcc544b49163b3e763a294a0b661482408b35ec48b2afc4eba4e", + "5797212ef801c231eeb5e6157caeb769c9f1c954c945c6e41f6b1de27fe6ea5a", + "58157eae91b262404e5660758e41c875ce3d2823b2e4e5fc30e8a3e1384561ae", + "5842b9664210226005965140e501df8cfd1193dc1def74032e3fb54e6b9217ef", + "590d891c164938e5e47d3dd069e1c81dced1f8a6e52a3020aa35ddce011f6dfb", + "599decd9be71c7b255562b1d9623c90819211305e579d8ad4a3af74c5ce8c4d6", + "59f04a0707f4139930d13192eb42ef9cf0a5193fa98f46e64ec927d3bd34f662", + "5a53a1e8f981c816e78f88933c5d2cf7aba4d9a9a28e953315d817abf3e937cf", + "5a8eec221fecdba8dfeba990ba2ef824a40da43c4167f70dac07ffd3335e8fbf", + "5b231a214ee5db29e6857bfe09c6a36d9dbd7dd2b85aa25528d47902099bd8c1", + "5c670f5657e3227b5ec3bc5199ed81e653deb046b4f6fb539c40281da5887418", + "5d8536a1f3f9d4054766e67598a6212a995f62d36f57fd8241d4432b800d429b", + "5d95de9ba01f727e8fad9c3476b8010bc5753d93b9f775ec94e307aeaab0488d", + "5dd35ce96ea763c051ec4c81442ea8b847f0d357a99fd385ca5b09e9cce36a22", + "5e1aee289d761d9dc1711bc6b3c97c0eeddc16b630c815839a156e42e7f39195", + "5e4eff8ba21f97afac6f8d2f6467b6e285e21dba4b21c6bc0f58d2f086edc2ab", + "60a111176ae35f70f9585afa440fe7f3d18977376c267480fa3094c3393adf62", + "60c84890cf7803f6ae595c3566b6aa4176405f2a8ba83c0f6cd6e000247ca271", + "60f630ba7d00adbbbe73b85429e665bd3c041464321cb346d27774365cab4682", + "61aa27300c1a31db1051074f03440d13ac491b0a5577f0b3de363e5a3b189bd9", + "6206882c83f707aa1f3259cd8cbc1f890f45bc4a85124cc602dc3318eeb6d3d8", + "635c92d494d7796b0c2f484147287f53a89a765e464d054cafb688b9d96655cc", + "6361a57a00da67d6102113b3f3ba0bbb7634c08cbd2096db4289f234380f6778", + "657d6b2557b1397c99b24a9e5777a3e28bb62b4a942ccf9da164c08e6a655e0d", + "65b892cbc5957bc2cc57375a544114a6475b8d9c274b5204d6e0d9c1b9cae15e", + "65de868c437f8dcddad0e37f6533940fde34b39148b9f63104f41d8145610798", + "65fc6b04f85a036c86076adfc04b741aedfe77a79a90db37b4023174ed4604da", + "6728ff8f1d45daa81b955edc59625766ed7b9e5b26f65edeb3dcd646675cdda1", + "69a10964b21da0fecd5c463c6816f544cb1efb299cfcd7a50a4b3a8ae59a9e0d", + "69b1edf50126e222eedc15ef65e9aef983fb9d340ccb13cb2740f509cb7019a9", + "69b58c84600892b4fa2cef6512cc93f04e14415058a2e2275bfdbb6ffece1495", + "6b71c1274c1625f9d19ef6d1f7044365d9dbff6bb252a6445648ff98c073baed", + "6ba83bb33eb0852b8a304b335ab26c5716a00bafeb0b1619146263c25ee0966c", + "6d4eb328e7293fa796088ce8a44db34d6e177e36d4bbb103f369859a08842c5b", + "6d9ecd81dd1ed4c5e6fc7bcec4e40d9097f9f818422e122794aaddfe260780eb", + "6dab1cd77a4ef3a9aee45a8537f58ab30d455d55700428a5ffab4022c2ee9d7d", + "6e29a3f512c9a33928ddfaeb68dae4c9ff946678744704d45ee053726b09e5e2", + "6e2d61579abd8f5acab73369a191a425e369300e3de717d6b5e8b1c79cfb2fae", + "6e6db234d66d1aab39325523447331e08d1255a7d81c67c10348b3abb03f24b3", + "6eea19a0e64ae3f4ec5dc0e5178341635ff858ec9c2f935976c054dafcdc05e4", + "6f76d0be63f1094f195d03f468ab82e36ae3f5f846d96d70812668b5d8e2f7db", + "6fcf3251ef3b9a95d9d2917e3604c3e2c1c05d2f4004fbe0686920abf3574575", + "71415a4b8c2ece02ddf3e9f1abe357c30c11029627c5510ec937582f36eb2939", + "71de47b052ee81350c5ad7224bc7dcbb5be7c4fd375f0821a479c31d82c9bdbc", + "727ca5e770a26fb5fa53155067cd05cc54bec3392d6a1856765f966ca660f9a4", + "72a6b851451517c6bdda780093ffb0e1f713db1580fb466d1dd390c2603f1d23", + "73e64ca24a747c2550b8a82dd413d5fe30e26f03fa95771d660c4b21561b7877", + "7498d1733ee3bfc7e7cdb11d6ad5a65369420da5ed5db9c4d752ae710f7e6e00", + "74e57ea458f6a044861dd23b053a31372df73e0b47e139b1c30d126edf775ee3", + "761134143d39da9361ac778266206266d961c8a76d60b063ff66903a0c51bb87", + "77e531ba2e0fdfb9d13b55ac7c023bec7acdbc08a87322fbec534cdc5020410a", + "77fe4e7237e2a146bc81019079776768d43ca9d55e19baf25f6db7ead3cb3fa7", + "7a56f9770759b68564c8bdbb38baa964b4acce93a3687488b18bf4a82561be32", + "7c419f28cac41a890964171a7ede35e4df22f5166d04b34ba2f1f5c530e895cb", + "7cb35abb83b346ffac1f0a58212082119a81f64f5cceedbd7b8af7810f2ebc07", + "7cfc2744e8e2c90dec0a973312a07f72b33ccff72595d6531f66c31ed099c6e1", + "7eb1f194992237cecaa642db7c5140b22a58ccf39c3aea4a21afb463b58b728f", + "7f60c45a4a714bf1066d53bf7d2b72abac5194fa5979d3aa68b93b64c9facecc", + "8243a06a68ff61d79c805f2c45bcd25fea70bea8fad91df198b432ab470ce9f8", + "8350ee6948c386f427a68e9cce71b828dda126548c1704c5aca0d040de3e1ec5", + "839775ebcdb6e011391fc3c3a66abd24907fcccd0d2c939b5cb390a65fb76f07", + "83d122ef72fc3ac252eefea7659e06a26f887d24374c04a5d750299f76aaed7c", + "843281e77ecc5a5210dbcd0b0b98edf7b95537fb6cdacbfb74fea2f8b11d8a51", + "85044cb38abf6ea983bce7170966ab3062fcdf01b28e25048481770a3418b82d", + "852cb696bf9bcb0a2c8c25bb5d111528a92006f3345eb8a47b3eef0efa343d72", + "85654a7b2d601c501d06ccec3bdcba969240c6862c59a34fa4ee9e3216a6f978", + "85797e1a9396533183d3551f5f62e960350b2cc8692881077242461722204027", + "86f1519103b9098184c2719a2553fcb4f24abf9635e7db4f4e09808b8af84f2c", + "86fde29349d8159afd645bd5b7c5556bf74a7563f5d27f5252cce6523681f6c8", + "87100034cff240e23384afe44efb0b09e90603f382896bd58c186faf0fe4aeb5", + "8751179f0094283c5cf1ddede063340f3a84784459a23264adc86421baf930a9", + "87d27f35acbb6f4734a61bf3a13edbe579fbb5d4ccf246ab2378e24b7558253b", + "88b7ccfb5fa2d3d4bec816f96e0a2dc5a15ce4ea602df297725a58dcb0806732", + "896df52c27a5c01769e1e6e9fbfbb795f43e7de78ac34f34a40bb37c147de197", + "896e19fbf4b3372093f28200f82ff64bedf1d38b1f7fe349d998496342162082", + "89e41d06c9b85a5e85183168ae099df4f324852eed8ec1314e347767563fea96", + "8a69885fdc86627e9cf7ec2880a8c8348b0cd02df31962c0c4d2c389a1a066df", + "8a8bdd653602f2f158019a5f439e3a899cf0a85921aa14cca082d6e603560909", + "8ae4bf13a85c96359e899a85b465784a6bdc84f8f36315f706c34d955beabd29", + "8b6ae722b7977d64014fa0af8f179df23ff208fd2bfae650d6c2277dde6979a3", + "8bc2f96c2f5cfec2923d02f631b6adb72273f0596f19d1de7b39d20acf0b6b12", + "8bc5b83371b1c90418b00374bfd06f8db036885a988bd6833ab0c06fb1ea98df", + "8bca1c6617cbbfd5ce3cfa106987cf26130f8539c26044e1a22a2f51fb1aca1f", + "8c05750c1ca939baf32d25d1f2d92a61800b49a76426ec28c1559d59606a61eb", + "8cb8ad276815c315802c4d97c5820cee5b1077c684875ebebd549432c2c4a812", + "8d091dc0975a50221f3b19a0569128ca17b1a52d01fe047e380b0aaf0e63b52d", + "8d314a3c1bf7b6c300dfecf79320d58bf6d2c23fb174f92d5c64d5e8542a96d7", + "8fd961b8dda83744a40dc807e6d7d2a3ea9d5d1bd799c1e1d5b2fefa666d6ec4", + "8fe2c0bd9c12919205e8c8b29ea61bd96d89616e9b7498b2ee8013446d5dd10f", + "902b91e3a5080e9f94da80889ceaa51c5f31d964846008622b6ddbeb07e28392", + "90e57c604e72df20a9694ebcd97e3fd9f00c5e5d8ec239e84a7b8a0c431e66f8", + "92928d4b346d02eac80d60c98e94f9f1b113f7d01134383db932754c4c5bd3d1", + "9393b510d59ccd16774e17789b3a01f90a64554dffdb9eac197d050c0bf5d09a", + "954890c63eef6f575f68e4f37adcaf21f4a2e343bb291d0b5585d0ea6d46a6b3", + "954c283eebad88edcd070cdb36ab55e457424f3fcb9210f59386cd3e4ee4e74f", + "956fbfd3ce94d4619ffdd9194deb85dc6d82e89fa199cc978e0cfca1d3e8ad6b", + "9588b1651ce73d10cae6b77ac6b991868c806a63979dc3b4be971f686e9723b8", + "95af0aec0162909a0addf5f0002455abb2bd61577e28562a5ed00c2260cb8fbe", + "964083a52374c2723dcb95bfed5636587a298497bf58e1183b60af055c46f397", + "96909c33a461e46a0d938d888e4371d1295f04535166bc16c9dfbb35c9d552da", + "9bd4c58f5a4d057257cae54e018f4eb88f9816f3cf7f88949345453a70512ff9", + "9c674f43e5ec1a37fb7eeb8264a5b4d785f17f716f1897c41616ed37333ba70a", + "9c6faadf417db194a1a2e68cfc6a4792339ca759edfe44c747fb4c516d7a66c2", + "9ca9acceac4275ff088b9bcd89d78019a7f71f284365c0681882194a8af11390", + "9ceaf7a3ba12f73c742a6f86a81c708c03240515b9cc5b89cae86ec4601a744e", + "9d8ede4ea688ed3e8334867c8f34e34e5c070ac993ae9bcf04489e7a25aec07c", + "9dee4ee61296b82b98e8972d47e802ee8ac8a5008b5e03cd5b130d0edfa15c77", + "9ef46448570d1e27b98284d64a08cec2d25a8118fad8017d44a99f4f6c061c47", + "9f1bd954340e3d26d52465ccbcd4b72baa30a3ce520e2a6e4358b5617b176963", + "9f585ccfb9a92e299ff0f76a9a777b4c5eeec4189f4fa9ab50c1cf3d74c21e05", + "a17f4334476d179113e2db7a6dbff71ecae86b10dfb2ad7fe2d8cc9135b4f2a5", + "a3170ffc83f7e0cc197350a8d74d85500d474673998ef1c62f1ddc3f364eb53e", + "a31eca43deeec05441c35933fc20b48f45b191042da018d7f6d97232c1ed9ce3", + "a322d6d1f0b71546ad4e9f746bc3fbf1742df0ecc431dca06584e44f29328e43", + "a35f0920ead6068375bb19cceff473cc971e9b578a0bc963fe7e617e2fdea9d3", + "a43f30a7fdf9c7a1561c6eb9aaca66890cdec58295265ab8d305ff076e36e0a8", + "a467e1194c7dc803eec26bc9535be71b58444bb5748d179e2a4e5e0662d3e0de", + "a49644cf53180202b00609f3cddc005a01e2086bb63369c711ff0950c8f8c214", + "a4b7c79b9897333ce6ceb761dc6854a40225ee9211a54101ba64dcbc92e0a6fe", + "a52945e1d0442f75f9a004ecb76d0e2b17e6137b124e73953d23c99aefa7749b", + "a52d52c4694251b5798e3478578d4d39fe1e964a70548f6388cd3c5e5cdb0fa7", + "a53fbb7223f7fdf1f7b641c51f0f8005ec0f32cfce74d9276001d8d8ea1151dd", + "a59577e4bbf52734f1931725d54881da6d66231bc9fe4c238d840b8c5444fa4b", + "a59ba23421ebc70cc23935fec64df1f948ef7e38c9ae6ae5d51491aea195046b", + "a939547a17e2245da625a539382110eaadcb4aa5b5e0821c15d6ae3a017f55e4", + "aa1da523736c9af8a5c7af126aef2900c7e06c0ba501113ecaed22f8982f7cda", + "aac283530a6c5a5e222a951421b18af4933a1f893ddc009d2e500340dd5d5abc", + "ab4a571a4eae6c30751bdee0aed400dc75363f94dc0f4b7782cb4556e83c23f3", + "aca0c3bd9fdcf931da31abcc3e4ec560120c83ab86deee1e627b5fe8e06b21b6", + "aef1e30af94b53299662a893e7d3a08099e1a9e2431e39096bd55d3eefc3953c", + "af28f08a766c0dfa0650194c27fbfb2b9b057e676e959dcde8c0dc04ba0372a6", + "af680a60c99d6ed6f75252add7208bbac1f27c4263be29008866b314963f9933", + "af8587819a0c45ae5a25878cc8bedc571a5a14d719ff7ad25bd3b088af294b3b", + "afb57d77ba6a109f6435059dac6617930c19567b3a9a5c690932e04ef7b7d45a", + "aff6e17d88bd7ef94395eaa52ed7f8bfa1a84a322d28c1535cdbaeff3fe137c8", + "b015523bce4598e2b2d077d075a5eed6fae4235b2851862ec3edf2a2a454b89b", + "b018d81f318adafd8aeaf9997b18e1e23da168b5fc28e640cf50d570dcc9e315", + "b10f856b83f430458887304c2f3609fc5bf70132d9546fbbf763183d9c8ec5b6", + "b12d970acf0fb01d206f2547177e9c4008abb0d81d20a8df2faf566176cb042f", + "b1daedac8c8e33c5708b2cb66bd406e57633cf05b6bbd95c11f9e752b35da4b5", + "b246439be3988b28f0fee9316443e3a92a37bf97ce2be5c748f04ddf800418ff", + "b3ae5672ebbcf36d12467c9fefcd760018e4b5733a34c41bce09e760bfaa9715", + "b50ce258f9c2c3a1b1284b7a573a2fa4a11ebb787929d39011a47002463ea7df", + "b50fca44807c252ef01dd95698262033154f8707905f779c0a20abc6439ca0d9", + "b5a7e7146a98035523ef6490183cdde177444103df52e1217bf8e72ada4d6328", + "b6324c2d34799565940dcccdd23e1360e974a46b44c209a6a7a9fdabf790d044", + "b66654f95bae5b30f1a326c90dc930b954368834385eaa21725571afe94b354d", + "b677f0aee329cf1b061cc0e2601a9c33218c5c10c0cace5820236313a0825d3e", + "b679de5b8df5d6b74ce1c8331418e05575117dfb8903a5381b6e92d9eb0b6a4a", + "b68d92aa8fccb4f75dc0ad09e9b6d567f7d40d421ae5321b36bb0a48236e598e", + "b72fa8f582c7fcc689962e8ffd5f8fe3f837aba4cd11ce7687532498c11fede9", + "b7761c8e81ab553d5b43ef3e570e6639d8d4fd3d3b217198f7bcc86b0078db4d", + "b7f2c74cc6970aa943740ba5c7fa947e4dc991233fd35e51412ced790d20298d", + "b8c14ae75cbc0d0dcb2d256815e6e4fab110ea29a3582201c0df5cebe47544d5", + "b8c87db8cf709cb8f4bbd8e684279f796def8980d809fd174a79690113de7207", + "bac19e11a6db7315a718dcf8236ca104336de292ca5553afe82f9418239b4271", + "bae7c6cd0a3c6e5f44834d64879b260dece1a4232f3fcb1f01756b03391c8831", + "bb5d7e079e9ab21431f60dd6c4495a3984001039620203c0962965e5f42de079", + "bbd5c9fa1463d0d789c66b421b274e424b1289de879bc2095a39924521dcdf06", + "bc107f5bfc824a5872fb29eb153fe68133664f84a0bda0c1f040629cc5722e55", + "bc28efd0ad227d98655ebf16a86d39e0eeca17d2f7e4a1bb14562b9772777332", + "bd570017cc62932d88f56095bb4fc99aa0d3f718317c089808b4649309f57322", + "bed8189cf521818dded6793b749c12a91df30fa033b1d0d628b74b078aa3d17f", + "bef655041d427966b37d503ec3ee6a608a397e34e50b3faf19c7c8010d32e9e9", + "c0b19497f20c96ff5a998cdfd8fe901b2636548e67f7521e17d8c1c3a344b119", + "c1fe0dcfc510b321f89d86e9e28d959be4b9f7191bbdfe75d94aa00ba907fecf", + "c29fa01c7d7c2300dc9f248122d101f2ae7096dbdff573efbb51b271db18a237", + "c2c4d1a16f5c13da6ead578d5ea39e79319c2fbe4b950d9a9eef447f0d885985", + "c3207049b4fe55b2130b223f9d327b21f11dc51c5ca20eca2a8d75c13f6000e5", + "c4c4cd450d60c5748c8fc36d10c75f182cfe952711617dacb76a0b0e7391a35b", + "c53776402fe96a74a9aa02b2ae732f279a3ed258aebd35e7f7347da28491856d", + "c5608d269923a2a3ee69d360c52f71f1eea591a603304c908ddb465f737c0782", + "c56cf2d189daaefc764ff1bbcf667d4625be73dbcdfe9c42e0f503d59b5ccd23", + "c60a678a37f48d744f7ba785c4adc7de65317982199801ebd603090315c85ca6", + "c6244a7b5686ad0e553cd0fb349481c14e4390ec7a65d414da7dead452a535d2", + "c6dba1af7f4d12fe2720b67f0ba6ce4984cf9d47c6d288f2467681d02f5af6d4", + "c7418675ff86e0833d60ea80a8c8a0f9b99d239421ca68164c6824a99a276f54", + "c77af3500dc3ee06cb5dd5ef1e7cffd70a3c7bca08b9c00196884cc3dae8db90", + "c9feb7939ea3febccdec91c170527060563ee0dbf0dac5b75092ef5c5bcc9e6f", + "ca67eca42b424c6374979d63df4eeae00381446c7c7eea0934a3a21b9352cd9c", + "ca83a160910b077d3ca4b6f0185b4a99549d8287d187a4c204ffcb6867514887", + "cbddadeb108dc36a2cf5e686e2cb3a143946afe47ceb15dcc185af4b272e3777", + "cc47d275990836af665da5f4dcb0c118d3d4897d376ebeac53ba6f207dbb8a64", + "cd015dbf90e521f2443eb714d30785c1d21240e44dc10e1f8464ff146866f6d1", + "cd7d0bcd5fbfa262877644a5d88d8e2b6016e6cb9a880761c1cc1823c4e6c621", + "cdd8100bc160c8c866913e8a033cce5f402efb1fc1538247d4cbcda3e1216659", + "cdef8f05ddad0db32cc966def25db6898a05c42bd15391b77c619eaf5ab2a904", + "cf4dd171afff5c96a512d37c25ea9e292f1cca25a2d267b2aef735d51605de17", + "cf99daf73630fb1d3448b7bbf0b4ba0558c886dd8ae1fd77a38a0b610fa1ea3a", + "d0a9e8772c316ac795de39d315b5ce38b72472b4df16d6ea6ae473c94c853a0b", + "d0c4b2351028946098f41c3fe28981d2aee8e3f5caf4ba4fcb12462d0c2fa12c", + "d0d11e3470945260027f6fefa6a4d0a3cbd29d2ecb9b9420a14ff6fa952c957d", + "d0f6f9c2fb93b869433b409b0093f0e15e226e6d750632a90a0e6e970a3464b6", + "d1ac7a2471d394057a588ec47c74a0b78e7cc2f495bb1585b184eb72dfd4ccce", + "d2615c76b309b1437f0d1f0294b8e88fc5fe3e9259a41a9ae4cdf137ef73609f", + "d3723031d231e59cd40ba8632e01ec7604f5197a48a45898cd0e314541331c87", + "d440f8f0db5b04f2ada14de102a3b3a7e3e774ab487a6bde0275a4344d62dd16", + "d584d77a56c70ad17c27a192efd074d4f5f74452f2c62ebae6b627d80ae063f6", + "d5b1bf93014f200928b481640c3ae57597cf73a0f1668a77e07ddd8683a81508", + "d652ef33b9ffeeff77ae17aeeea979ab8c1d403431bdf245867abf4566b54689", + "d66bff5b50f3ca994a03faa77246bf9e3a8193a4903fec25d5f56f7fee31611a", + "d66e373e83c0d5bf391ef83eb47b2ea4971409df16bf2d2d3f90e4819c641cc6", + "d723c5e330147c034be5d384e3c539f323c50b01326b901b3f78ec8a29d13b41", + "d842c9b772dbd601bfad1c3d830df5aacbb685507247666a80d8ef7ec0ab4db2", + "d92d42c9d643429671fe1f7384014ca15ae89ce49be0c5d524d2172938585986", + "d9efd2aa3c0a57f3bb93916653cfee963e4b1141917d703e61509d8d6dab7c78", + "da5e94dc767915969e1c8188d29035197a22ba0b9d8be1cebc04e5a8203646a0", + "da96925490c07aa86ee90d9b7c31bd59fdc56d27c891cb6728059322add4260b", + "db258468bee6ca4d5ca59334c4e45ca958aa6df89ceadebead6c04ec77feca35", + "dd2f29960e09d45097d7c7324130b96c27ee8483c4dca1b0045dd58a158b4838", + "dd794a7a3fcf1fe4a062e9664df3eecc6f45f0f15bb1672fe552f117acc1f646", + "dd97f6d3742257341118baa40d95ac216a80d20d39d47ec9c17e37234151d783", + "ddcb6f80de7e1ee656dabcf2566d3a78fb97806bbb6077a66d0b1e9d8a3e251c", + "de0beb18c2c3a2c895ca5f7a01665041577ef8373f8037591a1b1bd9b906e13a", + "dedc28fe961870e02089cb419fe3be918a27dd6568af8724f7e6879f749cd029", + "df67a5c802fc30ab72aeababbb0e93dd8e5b42cd9646ebc54ff9f47240c6f361", + "dfaa69ca87d3288942dcaf896c9ba8b8a6b0ba61006631efac7e521e8b543c9f", + "e128aa827a20a9ca1a057cf1991d49872999d78244f9f9d126399e8aa70b4841", + "e170828ddffedb42c27cf62dc347fca5ebcde643e3b4585691189c321cfda9c4", + "e1bb15a4b9aa87ea3327bde96322b226d9e407fee96824e7281a4142d9c32592", + "e1c5715b0a722ae7aa244fc75ea30dcfa239f6af640e6ba5d4937d22c85cb982", + "e24cb331e947114a105313fec3abee7bba02e65792962b11f789ff65354bca55", + "e34faf75bfbb6ec5d6fea460d34a300f33d61d5544249ce0adb97e3c9a8bf469", + "e417c01c804a4b846fb4ab0bc5f5e635822dd469814b6f454344a371a0c54846", + "e4a3df05b8fa77155ebe3f0e59b48ed9727f70e8624623cb69596dd0ccb2acff", + "e55e578cf3630b4265a38473546efd00faeaccae3b69cafbd46c38b43810159b", + "e58a39b6c5e754b1bfd881b5e6b3072eb678e7909a173dbe589b1f1046cc3c61", + "e614a0919bca602cb5416a1b69b025e7679766f28a630f8fe36a044c9b6c19a2", + "e730a01ae3f75be9423d32d2fc782f0250689840a9f35f552f9234956b91b651", + "e7cbfefa2b400f6c2a4088c06e774c341f1fec283d7e6de39d6aa38f8fb16eff", + "e83f6805a42db291bb19df80219ebf6257f4ca9965f7c96acf9c84c19845466f", + "e8b62f0abf8484fd663882a6423adc003b27b151b2e081544cdb328480b0efec", + "e968c0a922d5fbb0063602dfac63455d45924717d817299133a56a73fa9c85d9", + "eb68932633b2c16e1761b398a52354406182ae85cbc65ae7e27dbcfdb667d6af", + "eb961efc66c3dda1fceeffc0f5dcb270b6f354484fe5ff983e7252533d2db5b6", + "ee27e25f902e654569ef7a8ded0e028eea1f7b48cce472d02537f9c58b90f31b", + "ee9c47b6572c11e77808aa5278255d5baaa53327776a2eb1f3c02ac21706667e", + "eec2faad73968e9d8df0cc9171b24b238365533e490c3c1da68f2a01b3a58dac", + "f01042ebfb2d85430a7264450902c9d7f8f208343cbfa398de5efe062158cb10", + "f0262b9ee515bf10133044931f0078e678627bee08a7021dae6532b1d10652eb", + "f07a5ec90031146f8a1b92c877a9ef88d90b67725b687daa47850afcb8a4f5bf", + "f137e08187e31a78bf93686f27df66e24ba62fb370115cf099ab3452b3e84934", + "f18c1737e19dbb670db3262115d9d640533cf10ba73f8196ddb5a7d582744feb", + "f194576d56014f50ba3f94c92baf120c3dbcebe176b661154484c2a92b7b40c5", + "f3088bbb09667126643e781a17542b06cf97833d29b414ff29cdfc0c249b0025", + "f36b21cc9f9651ab59717c67501c154929fdb69d2124fcf87beb5cb4eddc385b", + "f37badc006085509e7f393cc62d362f41a6300fe2a54c46c6e0763b4afbfe7f7", + "f3bc1c4b3902b647d6660ae83b6a272dbf80b85cbcb7b57d74afbe94d6d754b1", + "f54e7d6b2f7b5aaa89a466187ae089784a605b946d50d221e21185b14ad2e5a0", + "f5579f35655c4c99b49931e63782e64494470ac21c1693fecacad3d3d54291bb", + "f70bac72a9d4055c60922c63e2f2cb3b77aa90250cc071a96f67d3ab658a4a12", + "f7899c890521825ae7f31b8c107f2ac9cf8e50097d4c06b92e226a772ba7b925", + "f82052e29e6ca52c7c633334e5c939674ca671cf5a592db59f882e88c5bbfdac", + "f937b82c14d253a7eecc1fc8effc545517af158c38d181727dcb8205e42a27bf", + "f9576ef7044564daed5e3bd01e196df3a69109244971a106c44d8108f286f08d", + "fa65af636e0484758f08f422fe84c5ccc340fd7f33a96fe5bd63e2a9c110fe6c", + "fc969058a53f8827440a2d1750091bc41aaded65e487bc033c81f2140b997321", + "fd18f20b9ea23bcb6fc5091c74383ba65a8b58c19f12da01c74b82ce5f821843", + "fd5336c29326b4bcacb4c55cedbe14a1d5e61ef269102110f643f665d1f8f03e", + "fdb70a2a5773acfdd00811abbadb690df8422aaaed7bd93893f576e2f5704bac", + "fe2303f90cb10eb75f13d551f88d2103bfb4f0c83682f807942f4a768b6852d1", + "fe6e280d14d7dd0fe23a487c684330c060896eee257e80121bc188a4f6110c09", + "ff5cbd0b1fdca636b746e9d39e0c2e984969c249bd093bcdd886290622f87775", + "ff6bbc3315c35a873ae49a7aea0911da2039eb0bdfd754beec5b119d5ca46b29" + ], + "task_counts": { + "LiftPegUpright-v1": 1056, + "PickCube-v1": 2384, + "PullCube-v1": 928, + "PushCube-v1": 1008, + "StackCube-v1": 1328 + } +} diff --git a/runs/ctt_residual_base_context_obj_rollout_val69_seed2/train.log b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/train.log new file mode 100644 index 0000000000000000000000000000000000000000..f3a4fcd4ad27e61b5e9307c6dd2aae8913a4f1aa --- /dev/null +++ b/runs/ctt_residual_base_context_obj_rollout_val69_seed2/train.log @@ -0,0 +1 @@ +CTT checkpoint trained separately; rollout used checkpoint=runs/ctt_residual_base_context_obj_seed2/model.pt diff --git a/runs/summary_ctt.csv b/runs/summary_ctt.csv index 0c5b6ec129209ea9583b787f112bb96c45fccba9..1acb0aab9fe6898630c5663dce43e2526fbf7d7b 100644 --- a/runs/summary_ctt.csv +++ b/runs/summary_ctt.csv @@ -1,5 +1,6 @@ method,run_path,clean,status,gate,base_success,success,proposal_oracle,hidden_oracle,support_gap,selector_gap,success_support_gap,success_selector_gap,outcome_ptr,pptc_0p20,pptc_0p40,negative_near_0p20,calibration_ece,seeds,ci V0 residual transport diagnostic,runs/reproduce_v0,yes,measured diagnostic baseline,legacy diagnostic,0.2974,0.3890,0.4435,n/a,0.1264,0.0545,0.1264,0.0545,n/a,0.2366,0.5269,0.0533,n/a,3,see run +CTT residual validation measured aggregate,runs/ctt_base_context_obj_val_rollout_comparison,yes,measured rollout K=8,"fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3",0.2754,0.2029,0.3816,0.6667,0.5369,0.3501,0.2947,0.1787,0.5024,n/a,n/a,n/a,0.2110,3,see measured_metrics CTT residual base-context-observation,runs/ctt_base_context_obs_nonlinear_dominance_basic_positive_train_to_test,yes,train feature=base_context_obs,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,120,see run CTT residual base-context-observation,runs/ctt_base_context_obs_nonlinear_dominance_context_success_train_to_test,yes,train feature=base_context_obs,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,120,see run CTT residual base-context-observation,runs/ctt_base_context_obs_nonlinear_dominance_context_tangent_positive_train_to_test,yes,train feature=base_context_obs,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,120,see run @@ -15,6 +16,9 @@ CTT gated residual,runs/ctt_gated_residual_full_seed2_val_proxy,yes,proxy,not me CTT gated residual,runs/ctt_gated_residual_smoke,yes,train,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,unknown,see run CTT gated residual,runs/ctt_gated_residual_smoke_proxy,yes,proxy,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,0.5000,0.7500,0.0573,n/a,unknown,see run CTT gated residual,runs/ctt_gated_residual_val_proxy,yes,proxy,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,0.1739,0.5362,0.0281,n/a,47,see run +CTT residual validation measured,runs/ctt_residual_base_context_obj_rollout_val69_seed0,yes,measured rollout K=8,"fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3",0.2754,0.1739,0.3478,0.6667,0.6088,0.3487,0.3333,0.1739,0.4638,n/a,n/a,n/a,0.2266,47,see measured_metrics +CTT residual validation measured,runs/ctt_residual_base_context_obj_rollout_val69_seed1,yes,measured rollout K=8,"fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3",0.2754,0.1884,0.3768,0.6667,0.5194,0.3633,0.2899,0.1884,0.5362,n/a,n/a,n/a,0.1740,47,see measured_metrics +CTT residual validation measured,runs/ctt_residual_base_context_obj_rollout_val69_seed2,yes,measured rollout K=8,"fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3",0.2754,0.2464,0.4203,0.6667,0.4826,0.3382,0.2609,0.1739,0.5072,n/a,n/a,n/a,0.2326,47,see measured_metrics CTT residual base-context,runs/ctt_residual_base_context_obj_seed0,yes,train feature=base_context_obj,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,unknown,see run CTT residual base-context,runs/ctt_residual_base_context_obj_seed0_val_proxy,yes,proxy feature=base_context_obj,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,0.2609,0.6667,0.0351,n/a,47,see run CTT residual base-context,runs/ctt_residual_base_context_obj_seed1,yes,train feature=base_context_obj,not measured,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,unknown,see run diff --git a/runs/summary_ctt.md b/runs/summary_ctt.md index 1118f9069b76a7b28b46f8394500369b56dc6e55..4bb0fd9c5b48fac97017df4f142edf04c4d8a967 100644 --- a/runs/summary_ctt.md +++ b/runs/summary_ctt.md @@ -3,6 +3,7 @@ | Method | Status | Base | Selected | Proposal oracle | Hidden oracle | Utility support gap | Utility selector gap | Success support gap | Success selector gap | OutcomePTR | PPTC@0.20 | PPTC@0.40 | NegativeNear@0.20 | Calibration ECE | Run | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- | | V0 residual transport diagnostic | measured diagnostic baseline; gate: legacy diagnostic | 0.2974 | 0.3890 | 0.4435 | n/a | 0.1264 | 0.0545 | 0.1264 | 0.0545 | n/a | 0.2366 | 0.5269 | 0.0533 | n/a | `runs/reproduce_v0` | +| CTT residual validation measured aggregate | measured rollout K=8; gate: fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3 | 0.2754 | 0.2029 | 0.3816 | 0.6667 | 0.5369 | 0.3501 | 0.2947 | 0.1787 | 0.5024 | n/a | n/a | n/a | 0.2110 | `runs/ctt_base_context_obj_val_rollout_comparison` | | CTT residual base-context-observation | train feature=base_context_obs; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | `runs/ctt_base_context_obs_nonlinear_dominance_basic_positive_train_to_test` | | CTT residual base-context-observation | train feature=base_context_obs; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | `runs/ctt_base_context_obs_nonlinear_dominance_context_success_train_to_test` | | CTT residual base-context-observation | train feature=base_context_obs; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | `runs/ctt_base_context_obs_nonlinear_dominance_context_tangent_positive_train_to_test` | @@ -18,6 +19,9 @@ | CTT gated residual | train; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | `runs/ctt_gated_residual_smoke` | | CTT gated residual | proxy; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.5000 | 0.7500 | 0.0573 | n/a | `runs/ctt_gated_residual_smoke_proxy` | | CTT gated residual | proxy; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.1739 | 0.5362 | 0.0281 | n/a | `runs/ctt_gated_residual_val_proxy` | +| CTT residual validation measured | measured rollout K=8; gate: fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3 | 0.2754 | 0.1739 | 0.3478 | 0.6667 | 0.6088 | 0.3487 | 0.3333 | 0.1739 | 0.4638 | n/a | n/a | n/a | 0.2266 | `runs/ctt_residual_base_context_obj_rollout_val69_seed0` | +| CTT residual validation measured | measured rollout K=8; gate: fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3 | 0.2754 | 0.1884 | 0.3768 | 0.6667 | 0.5194 | 0.3633 | 0.2899 | 0.1884 | 0.5362 | n/a | n/a | n/a | 0.1740 | `runs/ctt_residual_base_context_obj_rollout_val69_seed1` | +| CTT residual validation measured | measured rollout K=8; gate: fail:selected<47.45,oracle<50,support_gap>7,selector_gap>3 | 0.2754 | 0.2464 | 0.4203 | 0.6667 | 0.4826 | 0.3382 | 0.2609 | 0.1739 | 0.5072 | n/a | n/a | n/a | 0.2326 | `runs/ctt_residual_base_context_obj_rollout_val69_seed2` | | CTT residual base-context | train feature=base_context_obj; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | `runs/ctt_residual_base_context_obj_seed0` | | CTT residual base-context | proxy feature=base_context_obj; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.2609 | 0.6667 | 0.0351 | n/a | `runs/ctt_residual_base_context_obj_seed0_val_proxy` | | CTT residual base-context | train feature=base_context_obj; gate: not measured | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | `runs/ctt_residual_base_context_obj_seed1` |